From 51f1469d1db9b5c2dba3f060953565e26984a4f6 Mon Sep 17 00:00:00 2001 From: mzored Date: Sat, 29 Aug 2026 05:42:26 +0400 Subject: [PATCH 1/2] chore: dogfood records how the cross-host review is run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The codex invocation was re-derived from scratch every time: the stdin redirect it hangs without, the model flag a ChatGPT account refuses, the timeout binary this machine does not have, and where the verdict starts in the log. It also records the two things that make the review worth running — the qualifying bar from AGENTS.md in the prompt, and checking git log -S before blaming a sentence for a run that predates it. Co-Authored-By: Claude Opus 5 (1M context) --- .claude/skills/dogfood/SKILL.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/.claude/skills/dogfood/SKILL.md b/.claude/skills/dogfood/SKILL.md index c37fa64..ad3f16d 100644 --- a/.claude/skills/dogfood/SKILL.md +++ b/.claude/skills/dogfood/SKILL.md @@ -89,6 +89,27 @@ in something the kernel reserves for them. One session can prove wording is broken. It cannot prove that agents in general need a new procedure. Resist adding steps, gates, or ceremony on that basis, here or in the package. +## Review it on the other host + +A change to the shipped instructions gets a review from Codex before it is finished. The mechanics are +settled; do not re-derive them each time. + + codex exec --sandbox read-only -c model_reasoning_effort=high "$(cat prompt.md)" out.log 2>&1 + +The `"` tells you which release a sentence entered, which is how you find out +that the run you are blaming ran on text that did not exist yet. Record what was confirmed and what was +refused, with the reason, in the release notes. + ## Report Say what you found, what the evidence supports, and what stays uncertain. Where there was a fix to design, From 01dac7179faee08d76d967126dec34b011e67286 Mon Sep 17 00:00:00 2001 From: mzored Date: Sun, 30 Aug 2026 04:14:30 +0400 Subject: [PATCH 2/2] chore: make dogfood receipt analysis causal --- .claude/skills/dogfood/SKILL.md | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/.claude/skills/dogfood/SKILL.md b/.claude/skills/dogfood/SKILL.md index ad3f16d..ebefa82 100644 --- a/.claude/skills/dogfood/SKILL.md +++ b/.claude/skills/dogfood/SKILL.md @@ -36,9 +36,13 @@ Aim at whatever the question is about. Usually some of: - **Which package version ran**, and what its text said at the time. Compare against that version from git history, never against the current tree. -- **What reached the agent's context.** This one inverts conclusions when you get it wrong: a file path - appearing in a command is not proof the file's text reached the agent. Searching a file puts matching lines - in context, not the rule. Look for the wording itself. +- **What reached the agent's context.** This one inverts conclusions when you get it wrong, in both + directions. A file path appearing in a command is not proof the file's text reached the agent, and searching + a file puts matching lines in context rather than the rule. The other direction is easier to miss: a path + pattern under-counts, because `cd .../references && cat tracked-work.md` never contains the string + `references/tracked-work.md`. Search for the file's own opening sentence instead. A path-based scan once + reported zero loads where there were several, and the wrong number was already in front of the owner before + it was caught. - **What the owner actually asked**, in their own words. Owner input arrives through more than one channel, so do not read only the obvious one or you will miss turns, including the moment permission widened. - **What it cost.** Elapsed time, tokens, and how much of both went to subagents. Transcripts carry per-message @@ -59,6 +63,28 @@ honest default, not a failure. Count honestly and in whole sessions, and pool only sessions where the same text governed. Two deviations in one session are one observation. A finding the run noticed and silently dropped leaves no trace, so anything that looks like conformance is an upper bound. +## Reproduce before naming a cause + +A pattern across installed sessions is an observation, not a cause. Before saying which sentence produced it, +run the failing case with everything held fixed but the package, and run it on the unchanged package too. It +may not reproduce: a field failure seen in eighteen long sessions loaded fine in every isolated run of the same +package, which located the cause somewhere in what those sessions carry and not in the wording that was about +to be changed. That is a result worth having and it is cheaper than shipping the wrong fix. + +Two mechanics that are easy to get wrong and quietly invalidate the run: + +- **Prove the candidate is the package under test, from the transcript.** Do not take the model's own + inventory of what it loaded: asked, it went and read the disk and named the installed plugin path, which + says nothing about what was in its context. The base directory the skill itself reports is the evidence, + and it must point at the candidate rather than the host's plugin cache. +- **Isolate the other host before asking it to review.** Pointing only its own home at a scratch directory is + not enough; it also reads a host-agnostic user skill directory, so it will load the maintainer's personal + skills and the installed package it is supposed to be judging. Point the operating system home there as + well, copy in only the credentials, and check the session header before trusting the output. + +A cross-host review round converges when it is told what earlier rounds settled and what was refused, and told +not to raise those again. Without that it re-proposes them, and the rounds do not end. + ## Design the fix Only when the evidence names a defect in the package's own wording. A verdict of `UNVERIFIED`, a run that