fix: allowedCommands, and validate the things that validate #61

Merged

ivuorinen merged 3 commits from fix/validate-the-validators into main 2026-09-27 00:18:14 +00:00

Conversation 0 Commits 3 Files changed 8 +511 -20

ivuorinen commented 2026-09-26 22:25:28 +00:00

Owner

Copy link

Fixes the failure in #59 and closes the gap that let it ship.

The bug

Post-upgrade command '...' has not been added to the allowed list in allowedCommands

I wrote allowedPostUpgradeCommands. Renovate's docs are explicit: "This
configuration option was formerly known as allowedPostUpgradeCommands."

Why nothing caught it

The renovate-config-validator hook is scoped ^renovate.json$ and passes
--no-global, so it checks this repo's config against the repository
schema. The self-hosted config is a different schema and was never checked
at all. The old name is still valid repository config — so it validated clean,
was ignored at run time, and surfaced hours later in a PR.

scripts/ci/check-renovate-global-config.sh validates it as global config.
Verified against three states:

input	result
current config	passes
allowedPostUpgradeCommands	fails, printing the rename as a diff
invented key	fails — Invalid configuration option

The Renovate version is read from the CronJob image tag, not pinned again — a
second copy would sit in scripts/ci/, which no custom manager targets, so it
would age silently while looking maintained.

Auditing every other validator

One more gap, latent: kube-linter, kubeconform, promtool and helm-render
matched \.yaml$ while no-secret-manifests matched \.ya?ml$. A .yml
manifest would have been checked for Secrets and skipped schema validation and
security linting. The repo already uses .yml elsewhere.

scripts/grafana-dashboards/renovate.json looked like a gap but is a deliberate
exclusion — a Grafana dashboard that happens to share the name. It's recorded in
expected_gap() with its reason.

tests/hook-coverage.bats

States what each validator is responsible for and fails when its pattern stops
reaching all of it. This is the fourth time this class of bug has cost time
here: the bats hook that never fired on its own subjects, markdownlint "passing"
an untracked file, the Renovate global config, and now .yaml vs .ya?ml.

Verified by reverting kubeconform to .yaml$ with a .yml manifest present —
it names the file — and by breaking the bats pattern, which names renovate.json
and README.md.

v8r

Validates the repo's own config files against their published schemas:
.pre-commit-config.yaml, mise.toml, .markdownlint.yml, .yamllint.yml,
.v8rrc.yml, renovate.json. check-yaml/check-json/taplo parse these;
none looks at whether the keys inside exist or hold the right type.

It ships no .pre-commit-hooks.yaml, so it's a local hook with the binary pinned
via mise's npm: backend, as markdownlint-cli already is.

Confirmed it catches real errors:

.pre-commit-config.yaml#/repos           must be array
renovate.json#/automergeType             must be string
mise.toml#/tools/helm                    must be string

Deliberately excluded, with reasons in .v8rrc.yml: Kubernetes manifests
(kubeconform validates them against real API schemas including CRDs, which
SchemaStore cannot) and .forgejo/workflows/ (v8r resolves them to GitHub's
workflow schema, which is not the CI system running here; actionlint covers
them). renovate.json matched multiple SchemaStore entries, so .v8rrc.yml
names the schema explicitly.

One trade-off: v8r fetches schemas over the network and caches them, so a first
run offline fails rather than passing quietly.

Verification

All hooks pass, 48 bats tests pass.

Fixes the failure in #59 and closes the gap that let it ship. ## The bug ``` Post-upgrade command '...' has not been added to the allowed list in allowedCommands ``` I wrote `allowedPostUpgradeCommands`. Renovate's docs are explicit: *"This configuration option was formerly known as `allowedPostUpgradeCommands`."* ## Why nothing caught it The `renovate-config-validator` hook is scoped `^renovate.json$` and passes `--no-global`, so it checks this repo's config against the **repository** schema. The self-hosted config is a **different schema** and was never checked at all. The old name is still valid repository config — so it validated clean, was ignored at run time, and surfaced hours later in a PR. `scripts/ci/check-renovate-global-config.sh` validates it as global config. Verified against three states: | input | result | |---|---| | current config | passes | | `allowedPostUpgradeCommands` | **fails**, printing the rename as a diff | | invented key | **fails** — `Invalid configuration option` | The Renovate version is read from the CronJob image tag, not pinned again — a second copy would sit in `scripts/ci/`, which no custom manager targets, so it would age silently while looking maintained. ## Auditing every other validator One more gap, latent: `kube-linter`, `kubeconform`, `promtool` and `helm-render` matched `\.yaml$` while `no-secret-manifests` matched `\.ya?ml$`. A `.yml` manifest would have been checked for Secrets and **skipped schema validation and security linting**. The repo already uses `.yml` elsewhere. `scripts/grafana-dashboards/renovate.json` looked like a gap but is a deliberate exclusion — a Grafana dashboard that happens to share the name. It's recorded in `expected_gap()` with its reason. ## tests/hook-coverage.bats States what each validator is responsible for and fails when its pattern stops reaching all of it. This is the **fourth** time this class of bug has cost time here: the bats hook that never fired on its own subjects, markdownlint "passing" an untracked file, the Renovate global config, and now `.yaml` vs `.ya?ml`. Verified by reverting `kubeconform` to `.yaml$` with a `.yml` manifest present — it names the file — and by breaking the bats pattern, which names `renovate.json` and `README.md`. ## v8r Validates the repo's own config files against their published schemas: `.pre-commit-config.yaml`, `mise.toml`, `.markdownlint.yml`, `.yamllint.yml`, `.v8rrc.yml`, `renovate.json`. `check-yaml`/`check-json`/`taplo` parse these; none looks at whether the keys inside exist or hold the right type. It ships no `.pre-commit-hooks.yaml`, so it's a local hook with the binary pinned via mise's `npm:` backend, as `markdownlint-cli` already is. Confirmed it catches real errors: ``` .pre-commit-config.yaml#/repos must be array renovate.json#/automergeType must be string mise.toml#/tools/helm must be string ``` **Deliberately excluded**, with reasons in `.v8rrc.yml`: Kubernetes manifests (kubeconform validates them against real API schemas including CRDs, which SchemaStore cannot) and `.forgejo/workflows/` (v8r resolves them to GitHub's workflow schema, which is not the CI system running here; actionlint covers them). `renovate.json` matched multiple SchemaStore entries, so `.v8rrc.yml` names the schema explicitly. One trade-off: v8r fetches schemas over the network and caches them, so a first run offline fails rather than passing quietly. ## Verification All hooks pass, 48 bats tests pass.

ivuorinen added 4 commits 2026-09-26 22:25:28 +00:00

fix: CronJobRepeatedlyFailing now clears when the CronJob recovers ... cef0dcbd12

Counting failures over 24h is a statement about history, so the rule
stayed true for a whole day after the cause was fixed. Renovate was
working again within the hour and the alert would have kept firing until
the next afternoon. An alert that does not clear on recovery trains you
to ignore it, and the next real occurrence looks exactly like the stale
one.

It now also requires that the most recent scheduled run did not succeed.
That comparison is period-independent on purpose: on a healthy CronJob
both timestamps come from the same run, so the gap is that run's own
duration. Measured across all six CronJobs here, healthy ones sit between
-0.4 and -1574 minutes whatever their schedule, while the broken hourly
one sat at +291 — so the 30 minute threshold is headroom for a run still
in flight rather than a tuned number.

docs: tell RenovateNotSucceeding how to find the failing repository ... dae1ff00b3

This instance updates eight repositories and Renovate exits non-zero if
any of them logged an error, so the alert usually means some other
repository is broken while turingpi-cluster itself finished cleanly.
Three separate causes did that in two days: an EHOSTUNREACH to
git.esle.fi, an unresolvable preset in laituri.me, and a 404 on a
hand-pinned action in fiximus-maximus.

The per-repo result exists only in the run's log, never in a metric, so
the query that extracts it now lives in the alert rather than in whoever
last diagnosed it. Includes the detail that `kubectl logs job/<name>`
returns nothing here and the pod has to be named.

feat: Renovate updates a component and its dashboard in one PR ... b6874749d7

A dashboard pin is a tag inside a URL plus a sha256 of the exact bytes.
Renovate can rewrite the tag but not recompute the digest, so the two
halves moved separately: the component bumped, the dashboard pin sat
still, and tests/version-sync.bats failed the pull request. The standing
answer was a prBodyNote asking a person to do the other half — which is
why #47 failed every rebase.

A check that can prove two pins disagree is a check whose fix can be
automated. Three pieces:

  - REFRESH_CHECKSUMS=1 makes the generator record a mismatch and rewrite
    the pin in itself instead of exiting 1. Replacement is by the literal
    old hash, which is unique in the file, so it works whether the pin is
    inline beside its URL or held in LOKI_CHART_SHA256 and referenced from
    five calls. Without the variable set the integrity check is unchanged:
    a mismatch still fails, loudly, with both hashes.

  - Two custom managers read the generator's pins, which nothing read
    before: the kubernetes manager sees only image tags and the helm
    manager only Applications. depName deliberately matches the image or
    chart, so the component and its dashboard group into one PR.

  - postUpgradeTasks refreshes the checksum and regenerates the
    ConfigMaps into that same branch, allowlisted in the self-hosted
    config, which is the only place such a command can be permitted.

Verified by corrupting a pinned checksum: the normal run fails exit 1
naming both hashes, REFRESH_CHECKSUMS=1 repairs it to the real value, and
a plain run then regenerates with a clean tree.

fix: allowedCommands, and validate the things that validate ... d17346ce98

Renovate refused the post-upgrade task in #59 with "has not been added to
the allowed list in allowedCommands". The option was written here under
its former name, allowedPostUpgradeCommands.

That name is still valid *repository* config, and the
renovate-config-validator hook is scoped to ^renovate.json$ with
--no-global, so nothing ever checked the self-hosted config: it validated
clean, was ignored at run time, and the failure only appeared hours later
inside a pull request. scripts/ci/check-renovate-global-config.sh now
validates it as global config, which rejects the old name by rename and
any unknown key outright. The Renovate version it validates against is
read from the CronJob rather than pinned again.

Auditing every other hook's files pattern found one more, latent:
kube-linter, kubeconform, promtool and helm-render matched \.yaml$ while
no-secret-manifests matched \.ya?ml$. A .yml manifest would have got the
secret check and skipped schema validation and security linting. The repo
uses .yml elsewhere, so this was waiting to happen.

tests/hook-coverage.bats states what each validator is responsible for and
fails when its pattern stops reaching all of it — the fourth time this
class of bug has cost time here, and the first time it is checked.

v8r validates the repository's own config files against their published
schemas: .pre-commit-config.yaml, mise.toml, .markdownlint.yml,
.yamllint.yml, .v8rrc.yml and renovate.json. check-yaml, check-json and
taplo parse those; none looks at whether the keys inside exist. Kubernetes
manifests and Forgejo workflows are deliberately excluded, with reasons in
.v8rrc.yml.

ivuorinen force-pushed fix/validate-the-validators from d17346ce98 to 29a001927c 2026-09-26 22:26:58 +00:00 Compare

ivuorinen added 2 commits 2026-09-27 00:12:13 +00:00

ci: install every pinned tool instead of keeping a second list ... 73c5ba1c5c

Each job named the subset of mise.toml it needed, which made the two files
a pair that had to agree with nothing checking that they did. Adding v8r
broke it in the obvious way: pinned in mise.toml, invoked by a hook,
absent from the workflow, green locally and "v8r: not found" on the
runner.

Both jobs now run a bare `mise install`. The guard written for this an
hour earlier is deleted with it — a test policing duplication that can be
removed is complexity defending complexity, and hook-coverage.bats now
says so where it lists the instances of this bug.

The cost is install time and a few more release lookups. mise runs
unauthenticated here, so a rate limit would appear as a 403 and the fix is
the PAT described in the workflow header, not a return to a second list.

docs: add CLAUDE.md ... c31231e531

Written for the next agent rather than as a tour of the tree. Covers what
takes several files to work out: that merging is what deploys and a direct
cluster write is reverted while looking like it worked; the four
categories and what applies each; why the CRD installer has the shape it
does, including that ArgoCD excludes hook resources from its diff so an
all-hook app never syncs and a value inside a hook never re-triggers one;
which version pairs must agree and which test holds each; and that a green
check here has repeatedly meant a check that examined nothing.

Every command in it was run before it was written down.

ivuorinen merged commit 7a4d07b4a6 into main 2026-09-27 00:18:14 +00:00

ivuorinen deleted branch fix/validate-the-validators 2026-09-27 00:18:14 +00:00

ivuorinen referenced this pull request from a commit 2026-09-27 00:18:16 +00:00

fix: allowedCommands, and validate the things that validate (#61)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

ivuorinen/turingpi-cluster!61

No description provided.