Control-flow + core built-ins (pure sugar); heldout-v2 post-control-flow results set

[howlcipher]

Control-flow and core built-ins that the held-out v2 baseline (#146) showed were missing, plus a new post-control-flow results set for held-out v2. The frozen specs and baseline results are untouched and keep their pins.

Language (pure sugar: no new opcode, capability or effect)

• (% a b) and (// a b), floored like Python. Exact within |a| < 2^53. Division by zero gives the existing RUNTIME_ERROR.
• n-ary if, for elif chains: (if c1 a c2 b ... else). cond already existed; its else clause was wrongly rejected under --profile governed, and that is fixed: the governed scan now reads clauses structurally and still scans their bodies.
• (break) in a for/while body. It is rewritten to a loop flag plus guarded later statements. A break outside a loop or in value position is a compile error.
• (range [a] b), and negative literals -1 / -2.5 (lexer).
• max_by / min_by / sort_by (stable, keys computed once). Empty max_by/min_by gives a VALUE_ERROR.
• pad_left / pad_right / str_repeat / rtrim / ltrim.
• (to_base n b) for bases 2..36; any other base gives a VALUE_ERROR.
• let destructuring: (let ((a b) lst) ...).
• map_keys is unchanged. Keys are strings on every backend, including Go and JS, so sorting numeric-looking keys numerically would change existing output everywhere. (sort_by k (map_keys d) (to_int k)) covers that use instead.
• Bug fix (VM): the signed int64 multiply overflow check multiplied the operands as uint64, so any product with one negative operand, such as (* 3 -2), reported "integer overflow" on every path. Real overflow is still caught, and there is a test for it.

How these forms are checked:

• Every head has an arity entry; the drift and fuzz tests pass.
• controlflow_cli_test.go checks the same output on the interpreter, governed -run-bc and HFIR bytecode, with no capabilities required.
• It checks that value errors behave the same on all three paths and that a stray break is rejected.
• It checks that Go and JS reject every form with UNSUPPORTED_ON_TARGET before writing any output. Negative literals are the exception: they are plain literals and build on Go/JS.
• It checks that governed builds fail closed when an exec is nested inside any new form or a cond clause.

held-out v2: postcf/ (separate set)

• 25 of 37 HowlFrame files were rewritten. The other 12 are byte-identical copies of the baseline.
• Python/JS/Go use the baseline references, the same verifier and the same tokenizer rules. 148/148 pass.
• run_postcf.sh reproduces the results byte for byte, and they are pinned by TestHeldoutV2PostCFResultsFrozen.

cl100k	median before → after	total before → after	W/T/L before → after
vs Python	1.306 → 1.154	1.324 → 1.167	2/0/35 → 5/0/32
vs JS	1.136 → 1.028	1.167 → 1.028	7/2/28 → 16/1/20
vs Go	0.590 → 0.519	0.586 → 0.516	37/0/0 → 37/0/0

On o200k: vs Python 1.306 → 1.164 median and 1.321 → 1.165 total (5/0/32); vs JS 1.083 → 0.968 (20/0/17); vs Go 0.582 → 0.520.

HowlFrame still loses to Python. The per-task and per-category tables are in REPORT.md. Two caveats:

• w03 and w25 were also restructured with pre-existing filter. About 21 of w03's 52-token gain comes from that, not this slice, per Motoko's heldout-v2: token baseline on 576c87c1 (before control-flow) #146 golf.
• The records tasks do not move at all.

#146 nits

• REPORT now says o200k is not neutral for JS: 1.136 → 1.083, and wins go from 7 to 10.
• The workarounds list now includes the governed ban on unknown heads like count, and the let limits.

Checks

• gofmt is clean, and go build/vet/test ./... exits 0. The Parallel-run flake: apps/task_api and apps/status_api fail under full go test ./... #147 flake did not show up in this run.
• The run.sh scripts for v0, heldout-v0, heldout-v1 and heldout-v2 all exit 0, with the committed results byte-identical after rerun.

Pre-existing on main (not changed here)

• parse_json is not supported under -run.
• HFIR bytecode run of the baseline w22 gives CAPABILITY_DENIED on WRITE_FILE.

Hard nos: no production -compile-bc/HFIR flip, TestProdFlipCriteriaLock is untouched, #90 stays Partial, and no live LLM was used.

[howlcipher]

Review of e9f9f15 (base f731538): merge-ready, with one should-fix to file as a follow-up issue

I tested this in a single fresh checkout. Each item below lists what I ran and the real exit code.

1. Multiply overflow fix: correct

I ran each case on all 3 paths (-run, governed build+-run-bc, -compile-hfir-bc+-run-bc), and the paths agree every time.

• These now succeed with the right result: MaxInt64*-1, MinInt64*1, 2^62*-2 = MinInt64, -2^62*2 = MinInt64, 3037000499*-3037000499, 3*-2, -3*-2, and 0*MinInt64 / MaxInt64*0 = 0.
• These still give integer overflow: MaxInt64*±2, MinInt64*±2, MinInt64*-1, -1*MinInt64, 2^62*2, (-2^62-1)*2, 3037000500^2, and 0-MinInt64.
• mulOverflowsInt64 is correct: it special-cases zero and the -1/MinInt64 pair, then checks c/b != a.

% and // behave like Python for small operands. (% -7 3)=2, (% 7 -3)=-2, (% -7 -3)=-1, (// -7 2)=-4, (// 7 -2)=-4, (// -7 -2)=3. Division by zero (including (% 0 0)) gives division by zero on all paths, and (% "a" 2) fails at compile time.

Should-fix (follow-up issue, not blocking): above 2^53, results can be silently wrong because the desugar goes through float /. The docs state the limit, but there is no runtime guard:

• (// 123456789012345678 1000) returns 123456789012344 instead of …345.
• (to_base 123456789012345678 10) returns …668.
• (% MaxInt64 10) gives a false integer overflow.
• (// MaxInt64 1) and (// MinInt64 -1) give CONVERSION_ERROR. That one fails closed, which is fine.

Two possible fixes: real integer % and // lowering, or a VALUE_ERROR when |a| ≥ 2^53. The second fits the fail-closed approach better than a silently wrong answer. None of the benchmark tasks get anywhere near 2^53.

2. Governed cond/else: only the else head is newly allowed, and the scan still fails closed

The new scanNode branch skips only the literal else symbol and scans every clause's test and body. Each of these gives PROFILE_FORBIDDEN_CONSTRUCT (exit 1) on both build and check:

• a banned count in the else body
• exec in the second statement of else
• a lone (cond (else (exec …)))
• duplicate else clauses
• else placed before a test clause
• exec as a clause test
• ((else) 1), where else is used as a test and is still rejected as a construct

The same holds for exec/count nested in n-ary if, break loops, range, pad_left, to_base, destructuring and max_by. Effects are not hidden either: read_file inside an else or a max_by key shows up in inspect --caps as filesystem, and the run gives CAPABILITY_DENIED without a grant. The scan runs on the source tree before desugar, so desugared forms can't get around it.

3. Arity, Go/JS, parity

• Every new head gives ARITY_ERROR with line and column when called with the wrong number of arguments, including (break 1). I saw no panics. A non-symbol binder in max_by gives a clear compile error.
• TestControlflowRejectedOnGoJS covers all 15 forms on Go and JS: UNSUPPORTED_ON_TARGET, no Built, no output file.
• The drift, fuzz, coverage, arity and controlflow tests all pass with -count=1 (exit 0).
• Edge cases agree on all 3 paths:
  • empty range (0, 5 5, 3 -3, -2); range has no step argument
  • range -3 2
  • pad_left shorter than the string returns it unchanged; a multi-char pad gives xyxya
  • str_repeat with a negative count
  • to_base 0 with bases 2, 16 and 36 gives 0; to_base -255 16 gives -ff; bases 0 and -2 give VALUE_ERROR
  • rtrim "", ltrim " "
  • break through unless, cond, a let body and the then-branch of if (the statement after it is guarded correctly), and while true with break

Nit: a destructuring length mismatch is silent. ((a b) (list 1)) binds b to nil, and extra elements are ignored. That is consistent with how list_get misses behave, but it's worth documenting or making it a VALUE_ERROR.

4. Results: reproduced exactly

• git diff f731538e..head is empty for tasks/, results/, solutions/ and gaps.json, so the frozen specs, the #146 baseline and the Python/JS/Go solutions are byte-unchanged. The freeze tests pass.
• run_postcf.sh (tiktoken 0.8.0) exits 0 and leaves git status clean. The sha256 of all 5 postcf/results files matches the pins in TestHeldoutV2PostCFResultsFrozen. verification.json has 148/148 pass, and the coverage test checks the hashes.
• My own counting script, independent of count_tokens.py, gives:
  • cl100k vs Python: median 1.306→1.154, total 1.324→1.167, W/T/L 5/0/32
  • cl100k vs JS: 1.136→1.028 median, 1.167→1.028 total, 16/1/20
  • cl100k vs Go: 0.590→0.519
  • o200k vs Python: 1.306→1.164 / 1.321→1.165
  • o200k vs JS: 0.968
  • All of this matches the PR. HowlFrame's total went from 4084 to 3599 cl100k tokens (−485).
• Of the 37 HowlFrame files, 12 are byte-identical to the baseline and 25 were rewritten.

Attribution (cl100k). I rewrote the four suspect tasks using only features already on f731538e and checked them with the base binary: 37/37 HowlFrame pass, verifier exit 0.

task	base	best on main	postcf	from main features	from new features
w03	104	83	52	21	31 (rtrim)
w25	128	122	66	6	56 (range+%)
w34	227	221	208	6 (n-ary and, already on main)	13 (negative literals, destructuring)
w37	199	196	184	3 (n-ary and)	12 (destructuring, elif)

• About 36 of the 485 saved tokens (~7%) came from features already on main. About 449 (~93%) is a real gain from the new features.
• w11 is mixed (it also leans on the existing count_by/map_keys), so ~90% is a safer lower bound.
• Adjusting the baseline for those 36 tokens gives about 1.312 instead of 1.324 total vs Python. The gain from this slice is then roughly 1.31→1.17.
• The biggest single-task gains are to_base (w31, −63), range+% (w25), pad_right (w08, −41) and sort_by+// (w11, −33).
• The PR's own caveat (~21 tokens on w03) matches what I found.

5. CI and tests

• CI build is green on this head.
• gofmt -l . is empty and go vet ./... exits 0.
• go test ./... exits 0: 74 packages ok, no FAIL. The apps/* flake didn't show up, so there was nothing to rerun.

6. Hard nos

The diff doesn't touch TestProdFlipCriteriaLock, any -compile-bc/HFIR default, 4d74dbcf, or any live-LLM or H1 verdict code. #90 stays Partial.

Verdict: merge-ready. Please file a follow-up for the silent wrong results above 2^53 (guard them or use real integer division) and for the destructuring mismatch nit.