[howlcipher]

This is the spec-only freeze for held-out v3. Lain Iwakura (bot-assurance-0001) authored it; I'm committing it verbatim.

• 35 tasks, x01–x35: 17 records/JSON, and 3 each for text, table, numbers, lists, strings and logic.
• TestHeldoutV3SpecsFrozen pins the sha256 hashes.
• No solutions are included. Implementers must not read the specs before the records slice is done.
• x23's trailing comma is intentional.

[howlcipher]

Verdict: merge-ready (COMMENT-only; APPROVE is blocked for this account). Reviewed head 4be16d5d33a0936ec3370554e1865bee97b8f3b0 against base d887b96b, using one fresh checkout.

1. Spec-only

• The diff is 4 files, all under experiments/token-efficiency-heldout-v3/: the freeze test, README, build_specs.py and manifest.json. It adds no solutions, references or results.
• The manifest schema matches heldout-v0/v1/v2 exactly: top level {version, tasks}, task keys {id, category, description, inputs, output, kind, expected}, and kinds limited to text/json.
• Rerunning python3 build_specs.py gives a byte-identical manifest.json (cmp passes, and the tree is clean afterwards).

2. Pins

• I recomputed the sha256 of README 309081e4…, build_specs.py 199d297e… and manifest.json 85bcde28…. All three match the test map, and those are all three spec files.
• TestHeldoutV3SpecsFrozen passes.
• There are 35 tasks, IDs x01–x35 with no gaps. 17 are records, and text, table, numbers, lists, strings and logic have 3 each.
• x23: the trailing comma is in the expected output, not the input. It is the last row 9,kai,, because id 9 has no dept and the dept field is left empty. That matches the spec ("dept is empty when the id has no match"), and the oracle is unambiguous.
• Oracle ambiguity: none blocking.
  • x01 says "compact JSON", but the oracle uses default json.dumps separators. All six fixture paths resolve to scalars or missing, so the output can't differ.
  • x08 is the only text-compared JSON output, and its oracle is exactly sort_keys plus (",",":") on ASCII keys.
  • Every other JSON task uses decoded-value comparison, so key order and whitespace don't matter.
  • Number formatting is all integers (x14 is string-formatted cents).

3. Overlap (checked against heldout-v0/v1/v2 manifests, 60 token-efficiency-v0 Python solutions and 61 falsifiable-v0 task dirs; 223 items)

• No ID collisions, no matching normalized descriptions and no reused fixture contents.
• Top fuzzy pairs, all with similar wording but a different operation:
  • x13/h17: max ms per level from JSONL vs count per LEVEL from a pipe log. Score 0.73.
  • x12/v26: composite-key map vs sum by cust. Score 0.69.
  • x34/w10: day-of-year to MM-DD vs seconds to H:MM:SS. Score 0.68.
  • x32/v22: common prefix vs palindrome. Score 0.67.
  • x16/h30: entries array vs inverting an object. Score 0.65.
  • x29/h20: keep-last dedupe vs first-seen dedupe of tags. This is the closest in meaning, but it differs in both input shape and which occurrence it keeps. Acceptable.
• No join, transpose, interval, topo, anagram or forward-fill tasks appear in the prior sets or harness.

4. CI, gofmt and tests

• CI build: success.
• gofmt -l . is empty.
• go test ./... exits 0 (75 ok packages). The apps/* flake didn't reproduce this run, so there was nothing to rerun.

5. Hard nos: intact. No Go or prod code changed, so there is no -compile-bc/HFIR flip, TestProdFlipCriteriaLock is untouched, nothing new references tip-lock 4d74dbcf, and there's no live LLM or H1 PASS/KILL claim.

Nits (non-blocking, don't edit the frozen files; note them in a follow-up)

• The README says "Input file names … are new", but five reuse earlier names: x01 in/config.json, x12 in/people.json, x14 in/prices.json, x25 in/nums.txt and x32 in/words.txt. The contents are all new.
• x01's "compact JSON" wording doesn't match the oracle's default separators. It's harmless here, but worth tightening in v4.