Skip to content

feat(ops): audit what is DEPLOYED, not what is committed - #439

Merged
catomean merged 1 commit into
mainfrom
feat/runtime-conformance-audit
Aug 29, 2026
Merged

feat(ops): audit what is DEPLOYED, not what is committed#439
catomean merged 1 commit into
mainfrom
feat/runtime-conformance-audit

Conversation

@catomean

Copy link
Copy Markdown
Collaborator

bitbaum/fleet already has hosted-supabase-audit.sh, whose stated job is "does any repo still point at a Supabase we retired?". It ran on 2026-08-28 and reported success while printcraft returned 500 in production, pointing at a hosted project whose DNS no longer resolved.

It missed because it runs git grep over repo checkouts, and the reference lived in /opt/printcraft/shared/.envnot in git, deliberately, because the box is the environment SSOT. The gate was scoped to the wrong substrate: it checked the artifact while the configuration lived in the running system. A check that looks in the wrong place is worse than no check, because it answers.

The rule: source audits run in CI against git; runtime audits run on the box against what is actually deployed. The fleet had a good collection of the first kind and none of the second.

Five checks, each a shape that has already bitten:

check the incident behind it
env names a retired managed host printcraft, 500 in production
configured host no longer resolves same, invisible while pages returned 200
process holds keys its .env cannot provide 25 vitareba cron jobs died at the next restart
declared node engine the box cannot satisfy openclaw ran on 20 declaring >=22.19.0
app unit not active

Found on its first run, before this was committed:

revamp-info  declares node >=22, box runs 20.20.2 — and it IS deployed
revampit     points at a dead trycloudflare.com quick-tunnel host
aoz-demo     unit inactive

Alerts per finding, never one aggregate — host-check spent six weeks unable to fire because one permanently-failed unit pinned a single boolean at bad. "Could not look" gets its own key so it can never be mistaken for clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UvjGNAS9CMfEGNW26tUR4P

bitbaum/fleet already has hosted-supabase-audit.sh, whose stated job is "does
any repo still point at a Supabase we retired?". It ran on 2026-08-28 and
reported SUCCESS while printcraft returned 500 in production, pointing at a
hosted project whose DNS no longer resolved.

It missed because it runs `git grep` over repo checkouts, and the reference
lived in /opt/printcraft/shared/.env — not in git, deliberately, because the
box is the environment SSOT. The gate was scoped to the wrong substrate: it
checked the artifact while the configuration lived in the running system. A
check that looks in the wrong place is worse than no check, because it answers.

The rule this implements: SOURCE audits run in CI against git, RUNTIME audits
run on the box against what is actually deployed. The fleet had a good
collection of the first kind and none of the second.

Five checks, each a shape that has already bitten:
  - an env still naming a retired managed host (supabase.co, neon, planetscale)
  - a configured host that no longer resolves
  - a process running with keys its .env can no longer provide — config broken
    on disk for hours while the process holds the old values, which is exactly
    how 25 vitareba cron jobs died at the next restart
  - a declared node engine the box cannot satisfy
  - an app unit that is not active

Found on its first run, before it was committed:
  revamp-info  declares node >=22, box runs 20.20.2 — and it IS deployed
  revampit     points at a dead trycloudflare.com quick-tunnel host
  aoz-demo     unit inactive

Alerts per finding, never one aggregate: host-check spent six weeks unable to
fire because one permanently-failed unit pinned a single boolean at `bad`.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UvjGNAS9CMfEGNW26tUR4P
@catomean
catomean merged commit 63c9f2e into main Aug 29, 2026
1 check passed
@catomean
catomean deleted the feat/runtime-conformance-audit branch August 29, 2026 07:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant