From 6c785284ff78e4628721a4a68fc05ac00a3738fc Mon Sep 17 00:00:00 2001 From: mzored Date: Sat, 29 Aug 2026 05:42:26 +0400 Subject: [PATCH 1/4] chore: dogfood records how the cross-host review is run MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The codex invocation was re-derived from scratch every time: the stdin redirect it hangs without, the model flag a ChatGPT account refuses, the timeout binary this machine does not have, and where the verdict starts in the log. It also records the two things that make the review worth running — the qualifying bar from AGENTS.md in the prompt, and checking git log -S before blaming a sentence for a run that predates it. Co-Authored-By: Claude Opus 5 (1M context) --- .claude/skills/dogfood/SKILL.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/.claude/skills/dogfood/SKILL.md b/.claude/skills/dogfood/SKILL.md index c37fa64..ad3f16d 100644 --- a/.claude/skills/dogfood/SKILL.md +++ b/.claude/skills/dogfood/SKILL.md @@ -89,6 +89,27 @@ in something the kernel reserves for them. One session can prove wording is broken. It cannot prove that agents in general need a new procedure. Resist adding steps, gates, or ceremony on that basis, here or in the package. +## Review it on the other host + +A change to the shipped instructions gets a review from Codex before it is finished. The mechanics are +settled; do not re-derive them each time. + + codex exec --sandbox read-only -c model_reasoning_effort=high "$(cat prompt.md)" out.log 2>&1 + +The `"` tells you which release a sentence entered, which is how you find out +that the run you are blaming ran on text that did not exist yet. Record what was confirmed and what was +refused, with the reason, in the release notes. + ## Report Say what you found, what the evidence supports, and what stays uncertain. Where there was a fix to design, From 094e4a926270014e7fbe98d1bedbf3dce2a76a3e Mon Sep 17 00:00:00 2001 From: mzored Date: Sun, 30 Aug 2026 04:14:30 +0400 Subject: [PATCH 2/4] chore: make dogfood receipt analysis causal --- .claude/skills/dogfood/SKILL.md | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/.claude/skills/dogfood/SKILL.md b/.claude/skills/dogfood/SKILL.md index ad3f16d..ebefa82 100644 --- a/.claude/skills/dogfood/SKILL.md +++ b/.claude/skills/dogfood/SKILL.md @@ -36,9 +36,13 @@ Aim at whatever the question is about. Usually some of: - **Which package version ran**, and what its text said at the time. Compare against that version from git history, never against the current tree. -- **What reached the agent's context.** This one inverts conclusions when you get it wrong: a file path - appearing in a command is not proof the file's text reached the agent. Searching a file puts matching lines - in context, not the rule. Look for the wording itself. +- **What reached the agent's context.** This one inverts conclusions when you get it wrong, in both + directions. A file path appearing in a command is not proof the file's text reached the agent, and searching + a file puts matching lines in context rather than the rule. The other direction is easier to miss: a path + pattern under-counts, because `cd .../references && cat tracked-work.md` never contains the string + `references/tracked-work.md`. Search for the file's own opening sentence instead. A path-based scan once + reported zero loads where there were several, and the wrong number was already in front of the owner before + it was caught. - **What the owner actually asked**, in their own words. Owner input arrives through more than one channel, so do not read only the obvious one or you will miss turns, including the moment permission widened. - **What it cost.** Elapsed time, tokens, and how much of both went to subagents. Transcripts carry per-message @@ -59,6 +63,28 @@ honest default, not a failure. Count honestly and in whole sessions, and pool only sessions where the same text governed. Two deviations in one session are one observation. A finding the run noticed and silently dropped leaves no trace, so anything that looks like conformance is an upper bound. +## Reproduce before naming a cause + +A pattern across installed sessions is an observation, not a cause. Before saying which sentence produced it, +run the failing case with everything held fixed but the package, and run it on the unchanged package too. It +may not reproduce: a field failure seen in eighteen long sessions loaded fine in every isolated run of the same +package, which located the cause somewhere in what those sessions carry and not in the wording that was about +to be changed. That is a result worth having and it is cheaper than shipping the wrong fix. + +Two mechanics that are easy to get wrong and quietly invalidate the run: + +- **Prove the candidate is the package under test, from the transcript.** Do not take the model's own + inventory of what it loaded: asked, it went and read the disk and named the installed plugin path, which + says nothing about what was in its context. The base directory the skill itself reports is the evidence, + and it must point at the candidate rather than the host's plugin cache. +- **Isolate the other host before asking it to review.** Pointing only its own home at a scratch directory is + not enough; it also reads a host-agnostic user skill directory, so it will load the maintainer's personal + skills and the installed package it is supposed to be judging. Point the operating system home there as + well, copy in only the credentials, and check the session header before trusting the output. + +A cross-host review round converges when it is told what earlier rounds settled and what was refused, and told +not to raise those again. Without that it re-proposes them, and the rounds do not end. + ## Design the fix Only when the evidence names a defect in the package's own wording. A verdict of `UNVERIFIED`, a run that From 3d1504b0ae431e685ab26d5b377eb42ace80ccb7 Mon Sep 17 00:00:00 2001 From: mzored Date: Sun, 30 Aug 2026 18:29:34 +0400 Subject: [PATCH 3/4] fix: scope SkipHow host integration to repository --- .agents/skills/dogfood/SKILL.md | 84 +++++++++++++++++++++++++++++++++ .claude/settings.json | 5 ++ .codex/config.toml | 2 + scripts/check_hosts.py | 8 ++-- tests/test_hosts.py | 1 + 5 files changed, 96 insertions(+), 4 deletions(-) create mode 100644 .agents/skills/dogfood/SKILL.md create mode 100644 .claude/settings.json create mode 100644 .codex/config.toml diff --git a/.agents/skills/dogfood/SKILL.md b/.agents/skills/dogfood/SKILL.md new file mode 100644 index 0000000..b3fb329 --- /dev/null +++ b/.agents/skills/dogfood/SKILL.md @@ -0,0 +1,84 @@ +--- +name: dogfood +description: Inspect the owner's Codex or Claude Code transcripts to explain SkipHow behavior, cost, or version effects and decide whether the evidence supports changing the package. For this repository only. +--- + +# Dogfood + +Use real sessions to separate what SkipHow said from what the run did. Explain the observed behavior without +turning one transcript into a general theory of agents. + +This is a contributor skill and does not ship. Do not invoke the `skiphow` skill while examining this +repository. The package is the subject of the investigation, not its authority. + +## Locate the evidence + +Search the owner's local transcripts for a distinctive excerpt from the session. Transcripts are private +JSONL files, usually one per root or subagent session. Host formats change, so inspect the records you find +instead of relying on a fixed parser. Prefer direct search and disposable extraction over a maintained tool +for these private, unstable formats. + +Claude Code keeps sessions under the projects directory in its configuration home, with subagents in separate +files. For Codex, active rollouts are under `$CODEX_HOME/sessions/YYYY/MM/DD/`, or +`~/.codex/sessions/` when `CODEX_HOME` is unset. Archived rollouts are under the sibling +`archived_sessions/` directory. App task tools can identify the thread ID; the corresponding rollout +filename ends in that ID. + +A Codex text search may match subagent rollouts because they inherit parent history. Use +`session_meta.payload.id`, `thread_source`, and +`source.subagent.thread_spawn.parent_thread_id` to distinguish the root and follow its descendants when +their work matters to the question. + +Keep transcript contents out of delegate briefs and external output. Copy private material into a durable file +only when it is necessary and has been checked. + +## Reconstruct the run + +Read only as broadly as the question needs, but preserve the distinctions that determine the answer: + +- Identify the package version that ran and compare against that version from git history, not the current + tree. +- Establish what instruction text reached the agent. A path in a command or search result does not prove that + the file's rules entered context; look for the wording itself. +- Recover the owner's actual requests across every input channel in the transcript, including later turns + that changed scope or permission. +- Compare the actions and tool results with what the run reported. +- For cost or execution-health questions, use transcript timestamps and usage records and include relevant + subagents rather than estimating from the visible conversation. + +In current Codex rollouts, `session_meta` identifies the thread and lineage, `turn_context` records model +and execution settings, `response_item` holds messages and tool traffic, `event_msg` carries turn timing +and token counts, and `compacted` records context replacement. These are landmarks, not a stable schema. +App task summaries can locate a run, but the raw rollout is the evidence for exact context, actions, timing, +tokens, and subagent work. + +## Judge the evidence + +Distinguish three explanations: + +- The package text was missing, ambiguous, contradictory, or unreachable. One session can demonstrate such a + defect because the governing text and its context are inspectable. +- Plain wording reached context and the run departed from it. The transcript demonstrates that incident, not + a general failure rate or a need for more procedure. +- The expectation does not match the contract. SkipHow may deliberately leave the choice to agent judgment, + or the project may narrow the package's default. + +Use `UNVERIFIED` when the transcript does not distinguish them. Count observations by whole session and pool +only sessions governed by the same package text. Several deviations within one session remain one +observation. Apparent conformance is an upper bound because a transcript cannot expose a finding the run never +reported or acted on. + +## Change only what the evidence reaches + +Design a package change only when the evidence identifies a defect in its wording or placement. A cost +measurement, an isolated departure from plain text, or an unresolved cause ends in a report rather than a +manufactured fix. + +Check [`docs/decisions.md`](../../../docs/decisions.md) before reopening settled reasoning. Read +[`docs/prior-art.md`](../../../docs/prior-art.md) when a comparable mechanism would materially inform the +fix, not as a standing research step. Prefer the smallest correction that removes the defect. Consider whether +the text will reach the requests it governs and what instruction cost it adds to future runs. One session may +prove that wording is broken; it does not prove that agents generally need a new step, gate, or workflow. + +Report the reconstructed facts, the conclusion they support, and what remains uncertain. If the owner asked +for a change, implement the justified correction under the repository's contributor rules. diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 0000000..4f21818 --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,5 @@ +{ + "enabledPlugins": { + "skiphow@skiphow": false + } +} diff --git a/.codex/config.toml b/.codex/config.toml new file mode 100644 index 0000000..cd30cc3 --- /dev/null +++ b/.codex/config.toml @@ -0,0 +1,2 @@ +[plugins."skiphow@skiphow"] +enabled = false diff --git a/scripts/check_hosts.py b/scripts/check_hosts.py index 6cdcf8a..52b0e26 100755 --- a/scripts/check_hosts.py +++ b/scripts/check_hosts.py @@ -129,12 +129,12 @@ def _marketplace_manifest(host: str) -> tuple[str, Path]: def _plain_marketplace(destination: Path, host: str) -> Path: - _marketplace_manifest(host) - metadata = ".agents" if host == "codex" else ".claude-plugin" - _payload(ROOT / metadata) + manifest, candidate_manifest = _marketplace_manifest(host) _payload(PLUGIN_ROOT) destination.mkdir(parents=True, exist_ok=False) - shutil.copytree(ROOT / metadata, destination / metadata, symlinks=True) + destination_manifest = destination / manifest + destination_manifest.parent.mkdir(parents=True, exist_ok=False) + shutil.copy2(candidate_manifest, destination_manifest) shutil.copytree(PLUGIN_ROOT, destination / "plugins/skiphow", symlinks=True) _payload(destination) return destination diff --git a/tests/test_hosts.py b/tests/test_hosts.py index 97b7477..5481154 100644 --- a/tests/test_hosts.py +++ b/tests/test_hosts.py @@ -89,6 +89,7 @@ def test_plain_marketplace_matches_exact_candidate_and_rejects_repositories( tmp_path: Path, ) -> None: source = hosts._plain_marketplace(tmp_path / "plain", "codex") + assert not (source / ".agents/skills").exists() assert hosts.verify_plain_marketplace_source(str(source), "codex")[0] (source / ".git").mkdir() passed, output = hosts.verify_plain_marketplace_source(str(source), "codex") From d820fb68e65518b8f859a3d73113f331589cd3cb Mon Sep 17 00:00:00 2001 From: mzored Date: Wed, 2 Sep 2026 21:37:42 +0400 Subject: [PATCH 4/4] fix: give the contributor skill one home and keep it out of the payload Two open pull requests and one uncommitted working tree were three attempts at the same thing, pulling in different directions. This lands the coherent result. The Codex copy of the dogfood skill was a second identical file. Two identical files are the defect 2.16.0 spent itself removing everywhere else, and they drift at the first edit to either. .agents/skills/dogfood is now a symlink to the Claude copy, with a test asserting the pairing, so there is one text and one place to edit it. That symlink is only safe because the marketplace source no longer copies the whole .agents tree. It now copies the manifest alone, so a contributor-only skill under .agents never reaches the Codex payload. Without that, every isolated install test fails on the symlink, which is why the working tree carrying this test was never committed. The skill text is the longer version, which records how to read what reached an agent's context, how to reproduce before naming a cause, and how to run the review on the other host. The competing revision removed that material. This session needed all three and had to rediscover two of them, so the evidence favors keeping them. Co-Authored-By: Claude Opus 5 (1M context) --- .agents/skills/dogfood | 1 + .agents/skills/dogfood/SKILL.md | 84 --------------------------------- tests/test_package.py | 11 +++++ 3 files changed, 12 insertions(+), 84 deletions(-) create mode 120000 .agents/skills/dogfood delete mode 100644 .agents/skills/dogfood/SKILL.md diff --git a/.agents/skills/dogfood b/.agents/skills/dogfood new file mode 120000 index 0000000..c705a2f --- /dev/null +++ b/.agents/skills/dogfood @@ -0,0 +1 @@ +../../.claude/skills/dogfood \ No newline at end of file diff --git a/.agents/skills/dogfood/SKILL.md b/.agents/skills/dogfood/SKILL.md deleted file mode 100644 index b3fb329..0000000 --- a/.agents/skills/dogfood/SKILL.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -name: dogfood -description: Inspect the owner's Codex or Claude Code transcripts to explain SkipHow behavior, cost, or version effects and decide whether the evidence supports changing the package. For this repository only. ---- - -# Dogfood - -Use real sessions to separate what SkipHow said from what the run did. Explain the observed behavior without -turning one transcript into a general theory of agents. - -This is a contributor skill and does not ship. Do not invoke the `skiphow` skill while examining this -repository. The package is the subject of the investigation, not its authority. - -## Locate the evidence - -Search the owner's local transcripts for a distinctive excerpt from the session. Transcripts are private -JSONL files, usually one per root or subagent session. Host formats change, so inspect the records you find -instead of relying on a fixed parser. Prefer direct search and disposable extraction over a maintained tool -for these private, unstable formats. - -Claude Code keeps sessions under the projects directory in its configuration home, with subagents in separate -files. For Codex, active rollouts are under `$CODEX_HOME/sessions/YYYY/MM/DD/`, or -`~/.codex/sessions/` when `CODEX_HOME` is unset. Archived rollouts are under the sibling -`archived_sessions/` directory. App task tools can identify the thread ID; the corresponding rollout -filename ends in that ID. - -A Codex text search may match subagent rollouts because they inherit parent history. Use -`session_meta.payload.id`, `thread_source`, and -`source.subagent.thread_spawn.parent_thread_id` to distinguish the root and follow its descendants when -their work matters to the question. - -Keep transcript contents out of delegate briefs and external output. Copy private material into a durable file -only when it is necessary and has been checked. - -## Reconstruct the run - -Read only as broadly as the question needs, but preserve the distinctions that determine the answer: - -- Identify the package version that ran and compare against that version from git history, not the current - tree. -- Establish what instruction text reached the agent. A path in a command or search result does not prove that - the file's rules entered context; look for the wording itself. -- Recover the owner's actual requests across every input channel in the transcript, including later turns - that changed scope or permission. -- Compare the actions and tool results with what the run reported. -- For cost or execution-health questions, use transcript timestamps and usage records and include relevant - subagents rather than estimating from the visible conversation. - -In current Codex rollouts, `session_meta` identifies the thread and lineage, `turn_context` records model -and execution settings, `response_item` holds messages and tool traffic, `event_msg` carries turn timing -and token counts, and `compacted` records context replacement. These are landmarks, not a stable schema. -App task summaries can locate a run, but the raw rollout is the evidence for exact context, actions, timing, -tokens, and subagent work. - -## Judge the evidence - -Distinguish three explanations: - -- The package text was missing, ambiguous, contradictory, or unreachable. One session can demonstrate such a - defect because the governing text and its context are inspectable. -- Plain wording reached context and the run departed from it. The transcript demonstrates that incident, not - a general failure rate or a need for more procedure. -- The expectation does not match the contract. SkipHow may deliberately leave the choice to agent judgment, - or the project may narrow the package's default. - -Use `UNVERIFIED` when the transcript does not distinguish them. Count observations by whole session and pool -only sessions governed by the same package text. Several deviations within one session remain one -observation. Apparent conformance is an upper bound because a transcript cannot expose a finding the run never -reported or acted on. - -## Change only what the evidence reaches - -Design a package change only when the evidence identifies a defect in its wording or placement. A cost -measurement, an isolated departure from plain text, or an unresolved cause ends in a report rather than a -manufactured fix. - -Check [`docs/decisions.md`](../../../docs/decisions.md) before reopening settled reasoning. Read -[`docs/prior-art.md`](../../../docs/prior-art.md) when a comparable mechanism would materially inform the -fix, not as a standing research step. Prefer the smallest correction that removes the defect. Consider whether -the text will reach the requests it governs and what instruction cost it adds to future runs. One session may -prove that wording is broken; it does not prove that agents generally need a new step, gate, or workflow. - -Report the reconstructed facts, the conclusion they support, and what remains uncertain. If the owner asked -for a change, implement the justified correction under the repository's contributor rules. diff --git a/tests/test_package.py b/tests/test_package.py index 753b8d0..98155bc 100644 --- a/tests/test_package.py +++ b/tests/test_package.py @@ -76,6 +76,17 @@ def test_marketplaces_publish_only_the_plugin_directory() -> None: assert claude["plugins"][0]["source"] == "./plugins/skiphow" +def test_dogfood_skill_is_repo_scoped_and_shared_by_codex_and_claude() -> None: + claude = ROOT / ".claude/skills/dogfood" + codex = ROOT / ".agents/skills/dogfood" + + assert claude.is_dir() and not claude.is_symlink() + assert codex.is_symlink() + assert codex.readlink() == Path("../../.claude/skills/dogfood") + assert codex.resolve(strict=True) == claude.resolve(strict=True) + assert frontmatter(codex / "SKILL.md")["name"] == "dogfood" + + def test_release_metadata_uses_one_version() -> None: release = read("VERSION").strip() codex = json_object("plugins/skiphow/.codex-plugin/plugin.json")