Script-size optimization, Phases 0-2: cost model, EC constant pool, liveness scheduler

[icellan]

Phases 0–2 of the script-size optimization brief: baseline instrumentation, an exact
script-byte cost model, and two prototype optimizations behind opt-in flags. No
modular-domain analysis, no witness hints, no scalar-multiplication algorithms yet.

Default output does not move. All 72 fixtures still reproduce their checked-in
expected-script.hex byte-for-byte, script-size-check reports 72/72 ok, and the Go and
Rust cross-compiler golden tests still pass. Both new flags are off unless asked for.

Results

conformance/tests/p256-wallet 958,792 → 304,463 bytes (−68.2 %). That fixture is the
brief's "959,592 B reference implementation" — the number came from Rúnar itself.

variant	corpus bytes	vs baseline	changed	grown
current (shipping)	13,526,563	—	—	—
--stack-scheduler=liveness	13,526,482	−0.0 %	34	0
--ec-constant-pool	6,285,154	−53.5 %	9	0
both	6,285,073	−53.5 %	43	0

p384-wallet −76.4 %, ec-primitives −67.4 %, arithmetic 28 → 18 B (−35.7 %).

What the baseline found

58 % of every byte this compiler has ever emitted is a constant push, and 56 % of the whole
corpus is nine numbers pushed over and over — each curve's field prime, re-pushed by
fieldMod at every modular reduction. 20,025 pushes of a 34-byte literal in p256-wallet
alone, 71 % of the fixture. Pooling removed 99.7 % of that.

Two assumptions in the plan died on measurement, and are recorded as dead rather than quietly
dropped:

• Eager dead-slot retirement. 657 of 387,749 OP_PICK/OP_ROLL sites in the corpus are
  deeper than 16; typical depths are 2–5, and depths 0–2 are single-byte opcodes anyway. Every
  drop would cost 1–3 bytes to save approximately nothing.
• That a generic scheduler could reach P-256. The crypto emitters build their own stack
  layout through ECTracker and never pass through 05-stack-lower.ts — 13.4 MB of the
  13.5 MB corpus is out of a generic scheduler's reach. This is the single most important
  architectural fact the slice established.

Also confirmed: brief Phase 3 (fix-point peephole) and Phase 15 (OP_PUSH_TX binding) already
ship.

The scheduler miscompiled, and what caught it

The first working scheduler produced a script for if-without-else-multi-temp that ran to
completion, left a truthy top-of-stack, and accepted a witness the shipping compiler
rejects. Byte counts, the goldens for every other fixture, and 4,099 compiler unit tests all
passed while that was true.

conformance/witnesses/ replayed through runDifferentialExecution caught it — deployed
script versus the ANF interpreter, on witnesses the repo had already committed to, 2 of 86
cases. Cause: restoring spilled values immediately before an if leaves the parent stack in a
shape lowerIf's arm reconciliation was not written for. Fixed by a precondition (refuse to
spill in a scope with control flow ahead) rather than by trying to make the two agree.

Worth flagging for reviewers: my first bisect acquitted spilling, falsely. With commutative
reordering disabled the method-level cost guard simply preferred the baseline schedule, so no
spilling happened at all and the failure vanished for the wrong reason.

Verification

• cost-model.test.ts — estimateScriptBytes(ops) === emitMethod(...).scriptHex.length / 2
  for every method of every fixture, before and after peephole
• golden-invariance.test.ts — all 72 goldens reproduced on the default path
• ec-constant-pool-equivalence.test.ts — OpenSSL P-256/P-384 signatures plus every SEC1
  rejection case (r=0, s=0, r=n, s=n, all-zero sig, wrong message, wrong parity, truncated,
  oversized), on the real @bsv/sdk engine, 44 assertions
• liveness-scheduler-equivalence.test.ts — the conformance/witnesses/ corpus replayed
  under every variant, 86 cases
• 4,099 compiler + 1,380 testing + 67 conformance-runner + 125 CLI tests pass

Recommendation

The EC constant pool is worth a 7-tier port. Largest single byte win available,
curve-parameterized rather than curve-specific, and proved equivalent against an independent
oracle. Landing it means porting to compilers/{go,rust,python,ruby,zig,java}, regenerating
9 goldens, re-stamping conformance/script-size-baseline.json (the −67 % shrink trips its
50 % guard by design), and adding provenance entries. Not done in this PR — that is why
the flag ships off.

The liveness scheduler should stay experimental. It is correct and never grows a fixture,
but outside small arithmetic contracts it buys 0.1 %. Not worth a 7-tier port; worth keeping
as a gated mode so the next scheduling idea has somewhere to land.

~30 kB is not reachable without the algebra, but the next step is now measured rather than
projected. Patching the reduction emitters behind a throwaway switch, re-measuring, and
discarding the patch gives:

fixture	shipping	+ pool (this PR)	+ reduction sinking
p256-wallet	958,792	304,463	179,796 (−81.2 %)
p384-wallet	1,963,300	463,435	272,584 (−86.1 %)
ec-primitives	1,332,782	433,880	258,160 (−80.6 %)

The two are complementary, not independent: without pooling the cheap fieldSub form pushes
the prime twice and p256-wallet gets larger (958,792 → 999,371).

The more useful result is what that analysis actually has to prove. The short-reduction
variant passes 256 EC oracle assertions — OpenSSL signatures on both curves,
ec-on-curve-canonicity, ec-degenerate-add, ec-mul-scalars, p256-p384-scalars,
p256-p384-ecdsa-verify — and is still unsound:

• multiply / add / mulconst need only dividend ≥ 0, which unsigned coordinate decoding
  already gives — ~70 % of reductions, trivial analysis;
• subtract needs subtrahend < p, which decoding 32 unsigned bytes does not imply.

ecAdd((0, 1), (2^256 − 1, 1))
  shipping : …fffffffdfffff85f
  sinking  : …0001000003d0        0x1000003d0 = 2^32 + 977 = 2^256 − p

So brief Phase 4/5 is a sign lattice plus a < p bit that only subtrahends carry —
materially smaller than a full modular-domain lattice, and the difference between passing 256
oracle assertions and being correct. Details in §3.7–3.9 of the results doc.

One consequence worth flagging: at 179,796 bytes the split is 69.6 % stack-shuffle / 26.7 %
arithmetic, and OP_PICK (×36,683) is the largest opcode. Once a reduction costs 3 bytes,
ECTracker's own operand shuffling is the bottleneck — so a typed field-element IR under the
crypto emitters moves ahead of Straus/comb in the ordering.

Commits

1. feat(compiler) — cost model + size instrumentation (read-only, no byte moves)
2. feat(codegen) — the two experimental backends, plumbing, and the benchmark runner
3. docs(experiments) — the three reports

Reproducing

pnpm --filter runar-conformance run script-metrics                                   # where the bytes are
pnpm --filter runar-conformance run script-metrics -- --fixture p256-wallet --detail
pnpm --filter runar-conformance run script-metrics -- --compare current,liveness,ec-pool,both

[icellan]

Superseded by #160, which consolidates this branch with the other two into a single PR. No commits are lost — #160 contains all seven, and the combined measurement is in its description.