From 70a671bffa00b21d94de185d595206503d6f7170 Mon Sep 17 00:00:00 2001 From: William Zujkowski Date: Wed, 19 Aug 2026 02:11:08 -0400 Subject: [PATCH] docs: add blog-artifact-check to the pre-publish gate Auditing ninety agent-assisted posts found the highest-yield defect was never a wrong sentence. It was an artifact the post pointed at: a config file the tool silently ignores, a flag that does not exist, a gist created in a 21-second batch weeks after the post that presents it as working notes. None of the existing Layer-1 audits look at those. blog-factcheck verifies claims against cited sources; nothing verified the config against the tool that has to accept it. The gap is structural rather than an oversight. Machine-generated config is plausible by construction, and every format forgives it: tomllib.loads('[scanning]\nworkers = 4\n') # parses clean yaml.safe_load('expiration: 2025-12-31') # parses clean No parser errors, no test covers it, and CI cannot see a key the tool ignores. It needs the upstream schema fetched and compared key by key, which is a Layer-1 judgement call, not a grep. Calibrated on real findings: a grype 'expiration' absent from IgnoreRule so the suppression never lapsed; an entire osv-scanner.toml of invented keys; Trivy --policy, whose real counterpart has inverted semantics. And the strongest signal of all, a measurement attached to an invented key, which cannot have been taken. --- AGENTS.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 66712213..14bcf5a6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -168,6 +168,7 @@ The Layer-1 skills are author-local tooling in `~/.claude/skills/blog-*/` │ blog-nda-check → contextual NDA-compliance │ │ blog-argument-shape → thesis + evidence + falsifiability │ │ blog-visuals → zine doodle, diagram a11y, tokens │ +│ blog-artifact-check → gist provenance, keys vs upstream │ │ │ │ blog-deep-review (separate, deliberate, expensive): │ │ 4 parallel adversarial reviewers — reason-to-exist, │ @@ -219,6 +220,7 @@ The Layer-1 skills are author-local tooling in `~/.claude/skills/blog-*/` | Topic overlap with prior posts | Layer 1 (`blog-overlap`) | Author decides refinement vs new arg | | Thesis / evidence / falsifiability | Layer 1 (`blog-argument-shape`) | Author-time editorial judgment | | Visual coverage, diagram a11y, design tokens | Layer 1 (`blog-visuals`) | The prose audits all pass on a wall of text | +| Linked artifacts: gist provenance, config keys, CLI flags | Layer 1 (`blog-artifact-check`) | Machine-generated config is *plausible* and parses clean, so no CI check can see an ignored key. Needs the tool's upstream schema fetched and compared key by key | | Unanswered objections, uncited prior art, self-flattering arithmetic, claims that outlived their evidence | Layer 1 (`blog-deep-review`) | Needs adversarial judgment and a web search; too expensive per-commit. **Reviewers are confidently wrong — recompute every challenged number from raw inputs before accepting a correction.** | | Build correctness | Layer 2 (pre-commit) | Fast, blocks bad commits | | Astro design tokens | Layer 3 (`audits.yml`) | Per-commit, blocks broken design |