feat: Renovate updates a component and its dashboard in one PR #57

Merged

ivuorinen merged 3 commits from feat/renovate-owns-the-dashboard-pins into main 2026-09-26 22:03:08 +00:00

Conversation 0 Commits 3 Files changed 4 +197 -26

ivuorinen commented 2026-09-26 20:40:38 +00:00

Owner

Copy link

Closes the gap that makes #47 fail on every rebase, and removes the last
"a human does the other half" note from renovate.json.

The problem

A dashboard pin is a tag inside a URL plus a sha256 of the exact bytes:

dashboard redis \
  https://raw.githubusercontent.com/oliver006/redis_exporter/v1.91.1/contrib/grafana_prometheus_redis_dashboard.json \
  8b4b727e4fd984cc2ca969e52ca48e5333c1882559982691c6b267c3c7a2c8cc

Renovate bumps the image in redis-sample/deployment.yaml, the pin above stays
at v1.91.1, and tests/version-sync.bats fails — correctly. The standing
answer was a prBodyNote asking someone to move the pin, recompute the sha and
regenerate. So the PR could never be green on its own.

If a test can prove two pins disagree, the relationship is machine-readable and
the fix can be automated too.

Three pieces

REFRESH_CHECKSUMS=1 makes the generator record a mismatch and rewrite the
pin in itself rather than exiting 1. Replacement is by the literal old hash —
unique in the file — so it works both for inline pins and for
LOKI_CHART_SHA256, referenced from five calls. Written via a temp file, not
sed -i, whose syntax differs between GNU and BSD.

Without the variable the integrity check is unchanged. A mismatch still
fails loudly with both hashes. Only an explicit opt-in rewrites a pin.

Two custom managers read the generator's pins, which nothing read before —
the kubernetes manager sees only image tags, the helm manager only Applications.
They now resolve all eight:

datasource	pins found
github-tags	argoproj/argo-cd, traefik/traefik, prometheus-community/postgres_exporter, oliver006/redis_exporter, henrywhitaker3/adguard-exporter, TwiN/gatus, grafana/alloy
helm	loki

depName deliberately matches the image or chart depName, so the component and
its dashboard group into one pull request.

postUpgradeTasks then refreshes the checksum and regenerates the
ConfigMaps into that same branch. It is allowlisted in
gitops/apps/renovate/configmap.yaml — the only place such a command can be
permitted, since a repository cannot grant itself the right to run commands. The
regex is anchored and names one script.

Verification

Corrupted a pinned checksum to 64 zeroes:

1. normal run → exit 1, naming expected and actual
2. REFRESH_CHECKSUMS=1 → repairs it to the real hash
3. normal run → exit 0, clean tree

renovate-config-validator passes, all 39 bats tests and 24 hooks pass, and the
manager regexes were run against the real script.

What is not yet proven: postUpgradeTasks executing inside the Renovate
container. It needs curl jq sha256sum tar, which the generator checks for on
startup — so a missing tool fails loudly rather than silently skipping. The
first grouped dashboard PR after this merges is the real test; #47 should be it.

Note on #47

Its dashboard JSON is byte-identical between v1.91.1 and v1.92.0, so that
particular bump only moves the tag and the # Source: comment. The machinery
matters for the next one that isn't.

Closes the gap that makes #47 fail on every rebase, and removes the last "a human does the other half" note from `renovate.json`. ## The problem A dashboard pin is a tag inside a URL plus a sha256 of the exact bytes: ``` dashboard redis \ https://raw.githubusercontent.com/oliver006/redis_exporter/v1.91.1/contrib/grafana_prometheus_redis_dashboard.json \ 8b4b727e4fd984cc2ca969e52ca48e5333c1882559982691c6b267c3c7a2c8cc ``` Renovate bumps the image in `redis-sample/deployment.yaml`, the pin above stays at `v1.91.1`, and `tests/version-sync.bats` fails — correctly. The standing answer was a `prBodyNote` asking someone to move the pin, recompute the sha and regenerate. So the PR could never be green on its own. If a test can prove two pins disagree, the relationship is machine-readable and the fix can be automated too. ## Three pieces **`REFRESH_CHECKSUMS=1`** makes the generator record a mismatch and rewrite the pin in itself rather than exiting 1. Replacement is by the literal old hash — unique in the file — so it works both for inline pins and for `LOKI_CHART_SHA256`, referenced from five calls. Written via a temp file, not `sed -i`, whose syntax differs between GNU and BSD. **Without the variable the integrity check is unchanged.** A mismatch still fails loudly with both hashes. Only an explicit opt-in rewrites a pin. **Two custom managers** read the generator's pins, which nothing read before — the kubernetes manager sees only image tags, the helm manager only Applications. They now resolve all eight: | datasource | pins found | |---|---| | `github-tags` | `argoproj/argo-cd`, `traefik/traefik`, `prometheus-community/postgres_exporter`, `oliver006/redis_exporter`, `henrywhitaker3/adguard-exporter`, `TwiN/gatus`, `grafana/alloy` | | `helm` | `loki` | `depName` deliberately matches the image or chart depName, so the component and its dashboard **group into one pull request**. **`postUpgradeTasks`** then refreshes the checksum and regenerates the ConfigMaps into that same branch. It is allowlisted in `gitops/apps/renovate/configmap.yaml` — the only place such a command can be permitted, since a repository cannot grant itself the right to run commands. The regex is anchored and names one script. ## Verification Corrupted a pinned checksum to 64 zeroes: 1. normal run → **exit 1**, naming expected and actual 2. `REFRESH_CHECKSUMS=1` → repairs it to the real hash 3. normal run → exit 0, clean tree `renovate-config-validator` passes, all 39 bats tests and 24 hooks pass, and the manager regexes were run against the real script. **What is not yet proven:** `postUpgradeTasks` executing inside the Renovate container. It needs `curl jq sha256sum tar`, which the generator checks for on startup — so a missing tool fails loudly rather than silently skipping. The first grouped dashboard PR after this merges is the real test; #47 should be it. ## Note on #47 Its dashboard JSON is byte-identical between `v1.91.1` and `v1.92.0`, so that particular bump only moves the tag and the `# Source:` comment. The machinery matters for the next one that isn't.

ivuorinen added 3 commits 2026-09-26 20:40:39 +00:00

fix: CronJobRepeatedlyFailing now clears when the CronJob recovers ... cef0dcbd12

Counting failures over 24h is a statement about history, so the rule
stayed true for a whole day after the cause was fixed. Renovate was
working again within the hour and the alert would have kept firing until
the next afternoon. An alert that does not clear on recovery trains you
to ignore it, and the next real occurrence looks exactly like the stale
one.

It now also requires that the most recent scheduled run did not succeed.
That comparison is period-independent on purpose: on a healthy CronJob
both timestamps come from the same run, so the gap is that run's own
duration. Measured across all six CronJobs here, healthy ones sit between
-0.4 and -1574 minutes whatever their schedule, while the broken hourly
one sat at +291 — so the 30 minute threshold is headroom for a run still
in flight rather than a tuned number.

docs: tell RenovateNotSucceeding how to find the failing repository ... dae1ff00b3

This instance updates eight repositories and Renovate exits non-zero if
any of them logged an error, so the alert usually means some other
repository is broken while turingpi-cluster itself finished cleanly.
Three separate causes did that in two days: an EHOSTUNREACH to
git.esle.fi, an unresolvable preset in laituri.me, and a 404 on a
hand-pinned action in fiximus-maximus.

The per-repo result exists only in the run's log, never in a metric, so
the query that extracts it now lives in the alert rather than in whoever
last diagnosed it. Includes the detail that `kubectl logs job/<name>`
returns nothing here and the pod has to be named.

feat: Renovate updates a component and its dashboard in one PR ... b6874749d7

A dashboard pin is a tag inside a URL plus a sha256 of the exact bytes.
Renovate can rewrite the tag but not recompute the digest, so the two
halves moved separately: the component bumped, the dashboard pin sat
still, and tests/version-sync.bats failed the pull request. The standing
answer was a prBodyNote asking a person to do the other half — which is
why #47 failed every rebase.

A check that can prove two pins disagree is a check whose fix can be
automated. Three pieces:

  - REFRESH_CHECKSUMS=1 makes the generator record a mismatch and rewrite
    the pin in itself instead of exiting 1. Replacement is by the literal
    old hash, which is unique in the file, so it works whether the pin is
    inline beside its URL or held in LOKI_CHART_SHA256 and referenced from
    five calls. Without the variable set the integrity check is unchanged:
    a mismatch still fails, loudly, with both hashes.

  - Two custom managers read the generator's pins, which nothing read
    before: the kubernetes manager sees only image tags and the helm
    manager only Applications. depName deliberately matches the image or
    chart, so the component and its dashboard group into one PR.

  - postUpgradeTasks refreshes the checksum and regenerates the
    ConfigMaps into that same branch, allowlisted in the self-hosted
    config, which is the only place such a command can be permitted.

Verified by corrupting a pinned checksum: the normal run fails exit 1
naming both hashes, REFRESH_CHECKSUMS=1 repairs it to the real value, and
a plain run then regenerates with a clean tree.

ivuorinen merged commit 9b0ee1a856 into main 2026-09-26 22:03:08 +00:00

ivuorinen deleted branch feat/renovate-owns-the-dashboard-pins 2026-09-26 22:03:09 +00:00

ivuorinen referenced this pull request from a commit 2026-09-26 22:03:09 +00:00

feat: Renovate updates a component and its dashboard in one PR (#57)

ivuorinen referenced this pull request from a commit 2026-09-27 02:10:08 +00:00

fix: one post-upgrade command, and a dashboard that matches its pin

ivuorinen referenced this pull request 2026-09-27 02:12:35 +00:00

fix: one post-upgrade command, and a dashboard that matches its pin #63

ivuorinen referenced this pull request from a commit 2026-09-27 02:17:05 +00:00

fix: one post-upgrade command, and a dashboard that matches its pin (#63)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

ivuorinen/turingpi-cluster!57

No description provided.