test(bench): first honest correctness numbers, the tool-result cap measured, and the gate confirmed live

tools/harness-bench/FINDINGS.md

@@ -1043,3 +1043,115 @@ element in the shared chat surface, which is its own change.
 Deliberately **not** addressed by pinning a specific retired id anywhere in the
 harness — that is a hardcoded denylist in test clothing, and it rots the moment a
 vendor retires something else.
+
+## 10. The first honest correctness numbers — and the ceiling raise bought none of them
+
+Three arms on `claude-haiku-4-5`, 2026-09-20, against a stage built from the tree
+that carries the `stage_rowset_write` gate. Stage discipline first, as §5 demands:
+
+```
+staged src mtime   Sep 20 16:10:44
+newest src commit  Sep 20 15:55:07      ← stage is NEWER than the tree
+```
+
+### 10.1 Completion is not correctness, measured
+
+§6.1 predicted that "the first honest correctness number this produces will look
+worse than `4/4`". It does:
+
+```
+recursion set, prompt set v2           completed    correct
+  limit=25                                3/4          2/4
+  limit=500                               4/4          2/4
+heavy set, limit=500 (six tasks)          5/6          3/6
+```
+
+**Raising the ceiling bought +1 completion and +0 correct answers.** At limit=25
+`t3-todo-driven` was stopped by the step ceiling — §1's finding reproduces on a
+different model and a rewritten prompt, so it is robust. At limit=500 the same
+task _completed_ and its final answer was a serialized todo-list update rather
+than the ordering it was asked for. The published `3/4 → 4/4` was a true
+statement about termination and said nothing about whether the work got done.
+
+Two `completed` heavy tasks were also simply wrong: `h2-crossref` returned
+`TOTAL=33 TOP=omar` and `h5-longchain` `ADA=15 LIN=5 OMAR=13`. A completion count
+reads both as wins, at full price.
+
+### 10.2 `t4-long-chain` has never measured a chain
+
+`t4` made **1 model call and 0 tool calls** in both arms here — and did the same
+in both v1 arms on a different model. It fails the same way every time: it
+answers about the _previous_ task's content (`Beta 9, Alpha 4, Gamma 6` is `t3`'s
+data) or handles only the first number (`No. A prime number must have exactly
+two distinct positive divisors…`). The prompt asks the model to "work through the
+numbers one at a time", but nothing in the task requires a tool, so the run ends
+at the first assistant message. A one-call task cannot be a long chain. `t4`
+should be rebuilt on tool calls the way `h5-longchain` is, or deleted; as written
+it contributes a confident row to every table and measures nothing.
+
+### 10.3 The tool-result cap, measured for the first time — it fires
+
+`h6-bigread`, rebased onto `/memories/` so it needs no folder grant, ran for the
+first time in any arm. Reproduced twice (pinned alone, then inside the full arm):
+
+```
+results OFFLOADED before the model saw them : 1      ← the pre-model cap FIRED
+largest object in the store                 : 63,793 bytes   (the designed fixture size)
+largest result that got THROUGH the cap     : 2,665 tokens
+```
+
+`outcome_ok` was `Y` both times and is deliberately not the evidence: H6's agent
+authors its own fixture and can answer from memory of the seed. The `offl` column
+is the claim. Of the five claims §5 listed as unreachable, four are now measured;
+MCP namespacing still needs two hand-connected servers.
+
+### 10.4 The `stage_rowset_write` gate, confirmed on a live request
+
+```
+                      tools resident    tools segment
+before (§8's arms)          20             7,910 tok
+after  (this arm)           19             7,010 tok      −900, exactly as predicted
+```
+
+`stage_rowset_write` is absent from the request; `publish_artifact` and
+`revise_artifact` are both still present. This is the ledger's own per-call
+record of a real request, not an offline composition — i.e. the saving is at the
+seam the model reads, which is the distinction that killed title-slimming (§6).
+
+### 10.5 A correction to §7: the process boundary is not the driver either
+
+§7 found a store's first run cold 67.7% of the time regardless of elapsed time,
+and named its own blind spot: "first in store" conflated a process start with a
+prefix change, on a corpus of heterogeneous journey configs. These arms are the
+homogeneous case that corpus lacked — three fresh stores, identical config,
+minutes apart — and **all three opened warm**:
+
+```
+first model call of a FRESH store      input     cached
+  recursion arm 25                     12,973     7,621
+  recursion arm 500                    12,973     7,621
+  heavy arm 500                        13,183     7,621
+```
+
+So a process boundary does not by itself cost a cold prompt; a **prefix change**
+does. §7's 67.7% was measuring how often two journeys happened to share a config.
+Note what stayed cold: the cached 7,621 is almost exactly the tools segment
+(7,010), so the provider cache carried the tools block across stores and nothing
+after it. **Hypothesis, not yet verified:** the system block diverges per store
+within its first few hundred tokens (a path, a date, an id), which would make it
+structurally uncacheable across sessions that share everything else. n=3, one
+afternoon, one model. Diffing two stores' assembled system prompts would settle
+it, and the system-block attribution work is what makes that diff readable.
+
+For the shipping product — one store per user — what matters is prefix stability
+_across that user's sessions_, and the second call of every store here was ~99%
+cached (12,963 of 13,047). §7's practical conclusion stands: the opening run of a
+session pays for whatever part of the prefix is not byte-stable.
+
+### What these arms cost
+
+Three arms plus a one-task validation — 15 runs, all priced: **$0.0644** on
+`claude-haiku-4-5`, summed from `cost_micro_usd` in the stores rather than estimated.
+The v1 arms (`runs/arm-25.json`, `runs/arm-500.json`) are left untouched as the
+record §1–§6 cite; the v2 arms are `runs/arm-25-v2.json` and `runs/arm-500-v2.json`.
+Per §6.1, no v2 number is spliced into §6's trajectory.

tools/harness-bench/runs/arm-25-v2.json

@@ -0,0 +1,165 @@
+{
+  "recursion_limit": "25",
+  "user_data_subdir": "journey-bench-recursion-25-1789901236640977000",
+  "tasks": [
+    {
+      "task": "t1-trivial",
+      "claim": "the cheapest possible run terminates and answers",
+      "run_id": "0ea392b968e2456a8a657c3fc4802b97",
+      "status": "completed",
+      "safe_error": null,
+      "outcome_ok": true,
+      "expected": "(?i)\\A\\W*ready\\W*\\Z",
+      "answer_head": "ready.",
+      "seconds": 2.5,
+      "llm_calls": null,
+      "tool_calls": 0,
+      "input_tokens": 12973,
+      "output_tokens": 48,
+      "events": 21,
+      "cached_input_tokens": 7621,
+      "total_tokens": 13021,
+      "duration_ms": 1374,
+      "tool_invocations": 0,
+      "tool_rounds": 0,
+      "orphaned_rounds": 0,
+      "reconciled_rounds": [],
+      "peak_parallel": 0,
+      "delegated_rounds": 0,
+      "subagents": [],
+      "namespaced_tools": 0,
+      "tool_name_collisions": [],
+      "tool_failures": [],
+      "model_calls": 1,
+      "peak_result_tokens": null,
+      "peak_result_bytes": null,
+      "offloaded_results": 0,
+      "peak_stub_tokens": null,
+      "budget_notes": 0,
+      "super_steps_estimate": 6,
+      "terminal_code": "run_completed",
+      "answer_tail": "ready."
+    },
+    {
+      "task": "t2-three-steps",
+      "claim": "a three-step chain carries a value forward without losing it",
+      "run_id": "7512cbf59c2d4b15b278b623c13ae373",
+      "status": "completed",
+      "safe_error": null,
+      "outcome_ok": true,
+      "expected": "(?i)T2\\s*=\\s*30\\b",
+      "answer_head": "Step 1: 7 + 5 = 12\n\nStep 2: 12 \u00d7 3 = 36\n\nStep 3: 36 - 6 = 30\n\nT2=30",
+      "seconds": 3.0,
+      "llm_calls": null,
+      "tool_calls": 0,
+      "input_tokens": 13047,
+      "output_tokens": 169,
+      "events": 38,
+      "cached_input_tokens": 12963,
+      "total_tokens": 13216,
+      "duration_ms": 1555,
+      "tool_invocations": 0,
+      "tool_rounds": 0,
+      "orphaned_rounds": 0,
+      "reconciled_rounds": [],
+      "peak_parallel": 0,
+      "delegated_rounds": 0,
+      "subagents": [],
+      "namespaced_tools": 0,
+      "tool_name_collisions": [],
+      "tool_failures": [],
+      "model_calls": 1,
+      "peak_result_tokens": null,
+      "peak_result_bytes": null,
+      "offloaded_results": 0,
+      "peak_stub_tokens": null,
+      "budget_notes": 0,
+      "super_steps_estimate": 6,
+      "terminal_code": "run_completed",
+      "answer_tail": "T2=30"
+    },
+    {
+      "task": "t3-todo-driven",
+      "claim": "a checklist-driven task reaches the right final ordering",
+      "run_id": "b97bbdf8334e4a23bf356085b5740a0f",
+      "status": "failed",
+      "safe_error": null,
+      "outcome_ok": false,
+      "expected": "(?i)T3\\s*=\\s*beta\\s*,\\s*gamma\\s*,\\s*alpha\\b(?!\\s*,\\s*\\w)",
+      "answer_head": "",
+      "seconds": 9.1,
+      "llm_calls": null,
+      "tool_calls": 4,
+      "input_tokens": 14272,
+      "output_tokens": 200,
+      "events": 296,
+      "cached_input_tokens": 13979,
+      "total_tokens": 14472,
+      "duration_ms": 7592,
+      "tool_invocations": 4,
+      "tool_rounds": 3,
+      "orphaned_rounds": 0,
+      "reconciled_rounds": [
+        "write_todos"
+      ],
+      "peak_parallel": 1,
+      "delegated_rounds": 0,
+      "subagents": [],
+      "namespaced_tools": 0,
+      "tool_name_collisions": [],
+      "tool_failures": [
+        "write_todos:tool_run_failed"
+      ],
+      "model_calls": 4,
+      "peak_result_tokens": 91,
+      "peak_result_bytes": 341,
+      "offloaded_results": 0,
+      "peak_stub_tokens": null,
+      "budget_notes": 0,
+      "super_steps_estimate": 22,
+      "terminal_code": "recursion_limit_exceeded",
+      "answer_tail": ""
+    },
+    {
+      "task": "t4-long-chain",
+      "claim": "the longest chain in the set finishes the work it was given",
+      "run_id": "53000a8cffa9447e87962dca639be5a1",
+      "status": "completed",
+      "safe_error": null,
+      "outcome_ok": false,
+      "expected": "(?i)T4\\s*=\\s*2\\s*,\\s*3\\s*,\\s*5\\s*,\\s*7\\s*,\\s*11\\b(?!\\s*,\\s*\\w)",
+      "answer_head": "**Checklist for sorting task:**\n- \u2610 Sort alpha 4, beta 9, gamma 6 by value (largest first)\n- \u2610 Display the sorted rows\n- \u2610 Create a final table\n- \u2610 Output T3= line\n\n**Step 1: Sort by value (largest fi",
+      "seconds": 11.7,
+      "llm_calls": null,
+      "tool_calls": 0,
+      "input_tokens": 13891,
+      "output_tokens": 1022,
+      "events": 214,
+      "cached_input_tokens": 0,
+      "total_tokens": 14913,
+      "duration_ms": 10554,
+      "tool_invocations": 0,
+      "tool_rounds": 0,
+      "orphaned_rounds": 0,
+      "reconciled_rounds": [],
+      "peak_parallel": 0,
+      "delegated_rounds": 0,
+      "subagents": [],
+      "namespaced_tools": 0,
+      "tool_name_collisions": [],
+      "tool_failures": [],
+      "model_calls": 1,
+      "peak_result_tokens": null,
+      "peak_result_bytes": null,
+      "offloaded_results": 0,
+      "peak_stub_tokens": null,
+      "budget_notes": 0,
+      "super_steps_estimate": 6,
+      "terminal_code": "run_completed",
+      "answer_tail": "Beta 9, Alpha 4, Gamma 6"
+    }
+  ],
+  "session_dir": "/Users/parthpahwa/Library/Application Support/0xCopilot/journey-bench-recursion-25-1789901236640977000",
+  "memory_files": {},
+  "largest_offloaded_blob": null
+}