fix(examples): replay-kit honesty — real CLI invocations + real outputs + fail-closed testCmds

[matt82198]

Fixes three verified P1/P2 defects in the merged first-wave replay kit (#682). Evidence: conductor3/audits/2026-08-03-refine-r1/breakit.md items c1, c2, c4.

c1 — the documented validation command does not exist (P1)

examples/first-wave-baseline/README.md:187 and docs/FIRST-WAVE.md:161 both told adopters to run:

python tools/wave_templates.py validate <path>

That subcommand accepts only --template {saas,data,library,all} and no file argument:

$ python tools/wave_templates.py validate examples/first-wave-baseline/wave-manifest.json
wave_templates.py: error: unrecognized arguments: examples/first-wave-baseline/wave-manifest.json
exit=2

The walkthrough's own gate step failed as documented. The real standalone-manifest validator is tools/wave_manifest_lint.py (also exposed as aesop wave manifest-lint <path> — byte-identical output). Both docs now use it, explain why the old form exits 2, and retain the --template all form only where preset validation is genuinely the intent.

Two honest notes now in the docs rather than glossed: the lint emits WARN: testcmd_validity: No testCmd specified because that check reads a top-level testCmd while the wave engine reads per-item testCmd — a linter quirk, not a manifest defect — and consequently --strict exits 1 on this kit.

Also corrected: aesop wave <manifest> (README "Next Steps") is not a runner. aesop wave is a namespace with verbs preflight, manifest-lint, template, scorecard, resume; passing a path exits 2 with Unknown verb. Replaced with the working python driver/wave_loop.py --manifest <path> --one-turn.

c2 — fabricated "Expected Output" (P1)

The README's block claimed a checkmark, Items: 5, and per-slug file counts. validate_manifest returns None and the documented snippet prints exactly one line:

Manifest is valid

Every output block in both docs is now captured from a real run or explicitly labelled an estimate. Specifically:

• Duration tables previously advertised "honest expected duration" while reporting numbers nobody measured. They now say they are planning estimates.
• The jq inspect step's pasted output was not even jq's default format, and jq was never listed as a prerequisite. Replaced with a python one-liner whose real output is pasted.
• The "Expected green indicators" and merge-transcript blocks were invented. Replaced with a real captured run (below) and a note that merge output depends on your remote/CI.
• Noted that the ✓ in --template all output renders as a backslash-u escape on non-UTF-8 Windows consoles — a console artifact, exit code 0 either way.

c4 — fail-open testCmds (P2)

Four of five gates passed before any work was done, i.e. they would bless an empty PR. Measured on a fixture repo in the pre-work state:

item	old gate	pre-work result
enable-skipped-test	grep -q 'it.skip' ... && echo 'Still skipped' || ...	exit 0 — echoes "Still skipped" and passes
fix-doc-links	test -f docs/ARCHITECTURE.md && test -f docs/SETUP.md	exit 0 — only checks the files it is editing exist
add-eslint-config	... && grep -q 'lint' package.json	exit 0 — lint matches eslint in devDependencies
simplify-util-functions	npm test -- helpers.test.js | grep -q 'passing'	exit 0 — suite was green before the refactor (and the pipe swallows npm's exit code; passing is mocha-specific)

All four rewritten to fail pre-work: assert no .skip marker remains and the suite passes; resolve every relative markdown link against the filesystem and list unresolved targets; require a real lint script and actually run npm run lint; require the Refactor goal: marker the prompt already demanded plus a green suite. Item prompts updated so each gate is a stated contract rather than a surprise.

The claim is executed, not asserted

Writing "these gates are fail-closed" in prose would repeat the exact failure mode being fixed. examples/first-wave-baseline/verify-testcmds.sh builds a fixture repo in the pre-work state, runs each testCmd verbatim out of wave-manifest.json, applies the five fixes, and reruns them:

########## PRE-WORK: nothing implemented -- every gate must FAIL ##########
  readme-typo-fix            exit=1   OK   (fails as required)
  enable-skipped-test        exit=1   OK   (fails as required)
  add-eslint-config          exit=1   OK   (fails as required)
  fix-doc-links              exit=1   OK   (fails as required)
  simplify-util-functions    exit=1   OK   (fails as required)

########## POST-WORK: all 5 items implemented -- every gate must PASS ##########
  readme-typo-fix            exit=0   OK   (passes as required)
  enable-skipped-test        exit=0   OK   (passes as required)
  add-eslint-config          exit=0   OK   (passes as required)
  fix-doc-links              exit=0   OK   (passes as required)
  simplify-util-functions    exit=0   OK   (passes as required)

RESULT: all 5 testCmds are fail-closed.

Negative control: restoring the old fix-doc-links gate makes the script report FAIL-OPEN (passed before the work was done) and exit non-zero, so the verifier detects regressions rather than always printing green.

Limitation stated, not papered over

simplify-util-functions proves the file was touched and the tests are green. It cannot judge whether the refactor improved anything — that is not mechanizable and is now documented as a human review call instead of implied by a passing command.

Also

Added [ISOLATION: sibling worktree] to all 5 prompts, clearing the linter's prompt_sanity warnings (WARN: 5 prompt warning(s) → PASS: All prompts valid).

Scope

Touched only examples/first-wave-baseline/** and docs/FIRST-WAVE.md. tools/wave_templates.py deliberately not modified — the docs were corrected to match the real CLI, per the task constraint. Root README.md untouched. No CLAUDE.md changes were required (sync gate clean).

Verification

All run on this branch: wave_manifest_lint exit 0 · validate_manifest exit 0 · verify-testcmds.sh exit 0 · secret_scan --staged exit 0 · claudemd_lint · claudemd_sync_gate · watcher_linter · portability_check · subprocess_guard · agent_prompt_hygiene · verify_test_coverage · verify_test_suite_count · wave_templates validate --template all · verify_gates_wired · ci_gate_runability · CI-parity shell syntax check — all exit 0.

🤖 Generated with Claude Code

[matt82198]

Evicted from the merge queue: batch #736 (integrate/q-1785730980) red with every member individually green

[matt82198]

Evicted from the merge queue: batch #766 (integrate/q-1785803317) red with every member individually green