[bench] docs: add CHANGELOG.md; docstrings describe the code as it is (5.3)

CHANGELOG.md

@@ -0,0 +1,158 @@
+# Changelog
+
+All notable changes to Bench. The format follows Keep a Changelog, and the
+project follows semantic versioning. Dates are the merge dates of the release
+tags. Since the runner began recording verdicts, partway through 1.0.0, every
+change has landed through Bench's own governance pipeline and carries a
+receipt in the operational ledger; the earliest 1.0.0 work predates that.
+
+## [2.1.0] - unreleased
+
+The theme is making the claim survive a skeptic: cheaper verdicts, a real
+installer, honest numbers, and tests for the properties that used to live in
+comments.
+
+### Added
+
+- `pip install` gives a `bench` command; `bench install --project PATH` writes
+  the hook settings with an absolute path, appends `/.bench/` to the project's
+  gitignore, and installs the ledger commit guard as the project's pre-commit
+  hook when it has none, otherwise reporting the existing hook and how to
+  call the guard from it; `bench uninstall` reverses exactly what install
+  wrote, from a receipt it keeps at `.bench/install.json`.
+- Per-stage wall time recorded beside token counts in every entry; the
+  dashboard shows median and p90 latency per week and the README quotes the
+  ledger's own median tokens and seconds per edit.
+- Prompt caching of the constitution and repository context on the anthropic
+  provider; the claude_code provider lifts the constitution into its system
+  prompt file so the CLI reads it back from cache. Cache reads are priced at
+  the cached rate in `cli stats` and the viewer.
+- A recorded model override per stage: `BENCH_CHALLENGER_MODEL`,
+  `BENCH_DEFENDER_MODEL`, and `BENCH_ORACLE_MODEL`, with the model actually
+  used written into each stage's record and filterable on the dashboard.
+- A derived tip cache beside `entries/` so an append reads and validates only
+  the entries it links to. Never authoritative: the auditor ignores it, and a
+  stale, missing, or doubtful cache falls back to the full scan and rebuilds.
+- An append lock so two sessions on one machine cannot fork the chain, held by
+  retirement across its whole mutation window. Appends wait as long as the
+  holder lives; retirement's own acquisition is bounded and refuses loudly.
+- A ruff and strict mypy gate in CI over every source package and the hook.
+- Tests that run the hook as a real subprocess and pin its host contract:
+  exact allow and deny JSON on stdout, exit code zero always.
+- Tests that a prompt-injected diff cannot move the verdict: offline placement
+  proofs for every stage and provider, and an opt-in live run that expects a
+  VETO.
+- A per-project constitution layer: a governed project's `bench.json` may add
+  `P-` constraints and raise core severities, never remove or redefine a core
+  constraint; `cli constitution` prints the merged result and its sources.
+- Chain retirement under C-008's single bounded exception, with archive
+  verification before removal, an anchor entry opening the successor, and
+  `cli audit-retirement`.
+- Published-copy sanitation records and their audit, for the case where a
+  chain's content must be removed from copies that retirement cannot reach.
+- A dashboard in the viewer: weekly veto and pipeline-error rates, verdicts by
+  C-007 scope, constraint citations, token cost, and seconds per stage, all
+  computed by the same helpers `cli stats` uses.
+- A migration command for clones that predate the private ledger location.
+
+### Changed
+
+- The operational ledger lives at `<project>/.bench/` for every governed
+  project, Bench included, and is never a tracked artifact: an entry records
+  the change it governs, up to the diff budget and with binary and
+  out-of-project content reduced to metadata, so a committed ledger publishes
+  source. The legacy array segment is frozen and new entries are written one
+  per file, each named by its hash, so no entry is ever rewritten; an append
+  names every current tip, so a chain that comes to have two tips is
+  reconciled by the next governed edit.
+- Constitution v7 splits each constraint into the rule the models read and
+  commentary they do not; prompts carry the rule only.
+- Cosmetic drift in judge output (an empty remediation on PASS, a blank
+  advisory, a severity written as WARNING, a finding index written as a
+  string) is repaired and recorded on the entry instead of failing closed. A
+  missing or unknown verdict, or a citation outside the constitution, still
+  fails closed.
+- The subprocess bypass is bound to a per-call nonce the provider records on
+  disk; a bare `BENCH_SUBPROCESS=1` or a guessed token is governed normally.
+- Every subprocess call carries a timeout, enforced by a test that scans the
+  source tree.
+- The CLI parses arguments with argparse; `--help` lists every command and its
+  flags, and a bad flag is a usage error rather than silence.
+- Diffs are bounded by a character budget as well as a line budget, so a few
+  very long lines cannot carry an unbounded payload to the models.
+- The Oracle's per-stage timeout in this repo's settings is 300 seconds, so a
+  slow judge on a constitutionally heavy diff is not mistaken for a strict one.
+- Path normalization has one implementation in `utils.diff`, one deliberate
+  fallback copy in the hook pinned equal to it by a test, and the dead
+  path-traversal placeholder is gone.
+- The viewer file is created owner-only from its first byte on Windows as well
+  as POSIX.
+- README statements that had drifted from the code were corrected and pinned
+  by tests.
+- Docstrings and comments describe what the code does now. Release history
+  lives in this file; the one date that remains in source names the frozen
+  anchor entry whose shape the retirement auditor must keep reading.
+
+## [2.0.0] - 2026-07-24
+
+### Added
+
+- Global governance: the hook can be registered in the user's settings and
+  govern every project on the machine, with verdicts routed to the ledger of
+  the project being governed.
+- `SECURITY.md` and a documented governance boundary: files written through
+  Bash or MCP tools never reach the hook and carry no verdict.
+
+### Changed
+
+- Governance fails closed. A pipeline error, a judge that cannot be reached, or
+  a response that cannot be validated denies the change instead of allowing
+  it, and the ledger entry records the error as a pipeline error rather than
+  a ruling.
+- Stage models moved to Sonnet 5 for the Challenger and Defender and Opus 4.8
+  for the Oracle, with `utils/api.py` the single source of truth for model ids.
+- C-004 type safety tightened to the precision the constitution states.
+
+## [1.1.0] - 2026-07-12
+
+### Added
+
+- The claude_code provider: stages run through the local `claude` CLI in
+  headless mode on the user's subscription, with the stage system prompt kept
+  at system priority over the untrusted diff and a reentrancy guard for the
+  judge subprocess.
+- A test suite and CI workflow across supported Python versions.
+- Centralized governance logging.
+- `utils/stats.py` as the single source of truth for ledger statistics, used
+  by `cli stats` and the viewer alike.
+
+### Changed
+
+- The hook emits the documented `BENCH VETO [C-XXX]` reason format and surfaces
+  Oracle advisories on PASS.
+- Paths resolve against the repository root derived from the source file
+  location, not the working directory, so in-repo edits from a subdirectory
+  are governed as in-repo.
+- The openai SDK is a soft dependency, installed only for the OpenRouter
+  provider; the anthropic SDK has a pinned lower bound.
+
+### Fixed
+
+- Response parsing is guarded, silent catches log, and verification is
+  anchored to the ledger meta pin.
+- The viewer's temporary file is no longer removed before the browser loads it.
+- Hardening against the findings of a security audit of the pipeline.
+
+## [1.0.0] - 2026-04-21
+
+### Added
+
+- The governance pipeline: a PreToolUse hook intercepts Write, Edit, and
+  MultiEdit, and a Challenger, a Defender, and an Oracle rule on the change
+  before it lands. The Oracle's PASS or VETO is binding.
+- The constitution, `bench.json`, with constraints C-001 through C-008 and a
+  version field.
+- The hash-chained, append-only ledger: every verdict is an entry whose hash
+  covers its fields and links to the previous entry.
+- The CLI: `verify`, `ledger`, `stats`, `constitution`, and `viewer`, a
+  self-contained HTML browser for the ledger.

README.md

@@ -357,13 +357,13 @@ The corollary is worth stating: keeping a ledger out of git also removes git as
 
 ### Ledger Format
 
-The ledger is stored in two segments, and the split is what lets two branches record verdicts independently.
+The ledger is stored in two segments: a frozen legacy array, and one file per new entry.
 
 `bench-ledger.json` is the **frozen** legacy segment: a hash-chained JSON array that every reader still reads and nothing writes. `ledger-meta.json` is frozen with it, as a permanent pin on that segment's tip hash and entry count. Every new entry is written to its own file at `<ledger_dir>/entries/<entry_hash>.json`.
 
 That shape exists because the original single array was rewritten in full on every append. Two branches appending produced divergent chains with no valid resolution: interleaving breaks the hash links, and rebasing rewrites hashes, which C-008 forbids. The array was frozen rather than migrated for the same reason — C-008 permits no moving or rewriting of an existing entry, and a file that is never written can never conflict.
 
-`previous_hash` therefore holds either a string (legacy entries, one parent) or a sorted list of parent hashes. An append names **every current tip**, so after a `git merge` leaves two heads, the next governed edit names both and the fork reconciles itself. There is no merge command: the reconciliation is an ordinary governed, hash-linked entry.
+`previous_hash` therefore holds either a string (legacy entries, one parent) or a sorted list of parent hashes. An append names **every current tip**, so a chain that comes to have two heads, however it came to have them, is reconciled by the next governed edit, which names both. The ledger is untracked, so a `git merge` never touches it; a fork arises only when entry files reach one directory from more than one source. There is no merge command: the reconciliation is an ordinary governed, hash-linked entry.
 
 Beside `entries/` sits `tip-cache.json`, a derived record of the current tips. It lets an append read and validate only the entries it will link to instead of re-reading the whole chain, so what an append reads does not grow with the chain's length; only a directory listing still does, and that is two orders of magnitude cheaper than reading the chain. It is never authoritative: the cache is keyed to a digest of the entries listing, so any file added or removed outside the writer invalidates it, a missing or doubtful cache falls back to the full scan and is rebuilt from it, and `python -m cli verify` does not read it at all. Deleting it is always safe.
 

cli/commands.py

@@ -817,8 +817,8 @@ def cmd_migrate_ledger() -> int:
     """Populate this clone's private chain from the pre-migration location.
 
     Only needed once, and only by a clone that existed before the ledger
-    became private. Checking out that switch makes git delete the formerly
-    tracked chain under ``ledger/``, and nothing can repopulate ``.bench/``
+    became private. Checking out that switch makes git delete the chain such
+    a clone tracks under ``ledger/``, and nothing can repopulate ``.bench/``
     from git because it is ignored by design. Without this the clone would
     resolve to an empty chain and silently open a fresh GENESIS.
     """

hooks/pre-tool-use.py

@@ -43,10 +43,10 @@
     traceback.print_exc(file=sys.stderr)
     run_governance_pipeline = None  # type: ignore[assignment]
 
-# utils.diff (ledger entry 8bc4a3671a13, verdict PASS) provides the
-# hardened extractor: binary detection, truncation with governance-critical
-# line preservation, and change_type labeling. Imported in its own try
-# so the fallback path below can engage independently of pipeline import.
+# utils.diff provides the hardened extractor: binary detection, truncation
+# with governance-critical line preservation, and change_type labeling.
+# Imported in its own try so the fallback path below can engage
+# independently of pipeline import.
 try:
     from utils.diff import build_diff_info as _build_diff_info_hardened
 except Exception as _diff_e:  # pragma: no cover: degrades to the inline fallback below
@@ -91,20 +91,16 @@ def extract_diff_info(tool_name: str, tool_input: dict[str, Any]) -> dict[str, A
     governance on them so the pipeline can decide what to do, but we
     don't fabricate fields.
 
-    Global governance behavior (differs from prior fallback): when a
-    file path resolves outside the Bench repo root (_REPO_ROOT), the
-    fallback no longer returns the sentinel ``[PATH_TRAVERSAL_BLOCKED]``.
-    Instead it delegates to ``_fallback_normalize_to_cwd`` (defined above
-    in this module) which normalizes the path relative to CWD (the
-    governed project's root). This enables governance of edits in
-    external projects without over-blocking. A ``_path_normalized_external``
-    flag is set on the returned dict so pipeline stages (Challenger,
-    Defender, Oracle) can see that the path originated outside the
-    Bench repo and was normalized via CWD heuristics.
-
-    C-005 test coverage: both CWD-normalization branches (cross-drive
-    ValueError and escapes-repo-root) are covered by
-    TestFallbackExternalNormalization in tests/test_hook.py.
+    A file path that resolves outside the Bench repo root (``_REPO_ROOT``)
+    is normalized by ``_fallback_normalize_path`` relative to the working
+    directory, which Claude Code sets to the governed project's root, and
+    the returned dict carries ``_path_normalized_external`` so the pipeline
+    stages can see that the path came from outside the repo.
+
+    Test coverage, both in tests/test_hook.py: TestFallbackExternalNormalization
+    covers the cross-drive and escapes-repo-root branches, and
+    FallbackAgreesWithPrimaryTests pins the fallback equal to
+    ``utils.diff._normalize_path`` on the same inputs.
     """
     if _build_diff_info_hardened is not None:
         return _build_diff_info_hardened(tool_name, tool_input)

ledger/chain.py

@@ -11,14 +11,12 @@
 beside it as a permanent pin on that segment's tip and count. New entries are
 written one per file to ``entries/<entry_hash>.json``.
 
-That split is what lets two branches record verdicts independently. A single
-array rewritten in full on every append gave two branches divergent chains that
-could not be merged — interleaving breaks the links, rebasing rewrites hashes —
-so the array was frozen rather than migrated, because C-008 forbids moving or
-rewriting an existing entry and a file that is never written cannot conflict.
-``previous_hash`` accordingly holds a string (legacy, one parent) or a sorted
-list, and an append names every current tip, so a fork left by a merge is
-reconciled by the next governed edit.
+A file that is never written cannot conflict, and an entry file named by
+its own hash is never rewritten, which is what C-008 forbids; a chain
+assembled from more than one source combines as a union of files.
+``previous_hash`` holds a string (legacy, one parent) or a sorted list, and
+an append names every current tip, so a chain that comes to have two tips,
+however it came to have them, is reconciled by the next governed edit.
 
 Writes are atomic via ``os.replace`` on a same-directory temp file, so a crash
 mid-write cannot leave a half-written file on disk.
@@ -92,10 +90,10 @@
 class LedgerReadError(RuntimeError):
     """Raised when existing ledger content cannot be read or would be lost.
 
-    Fail-closed on the write path (C-001, C-008). A corrupt ledger previously
-    read as an empty chain, which made the next append restart from GENESIS and
+    Fail-closed on the write path (C-001, C-008). A corrupt ledger must not
+    read as an empty chain: the next append would restart from GENESIS and
     overwrite the damaged file. Refusing to proceed preserves the evidence
-    instead of destroying it; the caller in ``pipeline.runner`` already logs and
+    instead of destroying it; the caller in ``pipeline.runner`` logs and
     returns the verdict without a receipt rather than blocking the developer.
     """
 
@@ -134,14 +132,6 @@ def resolve_ledger_path() -> str:
     therefore never a tracked artifact of the repository it governs, and
     dogfooding does not earn an exception to that.
 
-    Bench's existing chain was relocated to that path before this branch
-    removed the special case: the legacy segment, ``ledger-meta.json``, and
-    every entry file were copied unchanged, and ``verify_chain`` reports VALID
-    at the new location with genesis hash 4e98fb41 unchanged. No entry was
-    modified, reordered, or removed, so continuity is preserved and no
-    retirement was triggered, which matters because a storage-location change
-    is not a permitted C-008 retirement trigger.
-
     Claude Code invokes hooks with the governed project as the working
     directory. That is the same assumption ``utils.diff`` already relies on
     when normalizing paths that fall outside the Bench repo.
@@ -486,9 +476,9 @@ def _entry_file_names(entries_dir: Path) -> list[str]:
     a tip the cache never links to; ``_FOLD_CASE`` carries pathlib's rule.
     Names are not checked for being regular files, which would cost a stat
     per name: ``glob`` does not check either, so a stray directory with the
-    suffix is treated exactly as before. Its appearance changes the listing,
-    the scan refuses it as unreadable, and the auditor reports it. It is
-    never a tip, so the cache never links to it.
+    suffix is handled by the scan, not the cache. Its appearance changes the
+    listing, the scan refuses it as unreadable, and the auditor reports it.
+    It is never a tip, so the cache never links to it.
     """
     if not entries_dir.is_dir():
         return []

ledger/migrate.py

@@ -1,9 +1,9 @@
 """One-time upgrade path for clones created before the ledger went private.
 
-Bench used to keep its own chain as tracked files under ``ledger/``. That
-exemption is gone: every governed project, Bench included, now keeps its
-chain in a gitignored ``.bench/``. The commit that made the switch untracks
-the old paths, so pulling it makes git delete ``ledger/bench-ledger.json``,
+Such a clone holds Bench's own chain as tracked files under ``ledger/``,
+while every governed project, Bench included, keeps its chain in a
+gitignored ``.bench/``. The commit that made the switch untracks the old
+paths, so pulling it makes git delete ``ledger/bench-ledger.json``,
 ``ledger/ledger-meta.json``, and ``ledger/entries/`` from the working tree.
 
 Nothing in git can repopulate ``.bench/``, because ``.bench/`` is ignored by