Cut the dev gate's memory and disk cost, and a third of its wall clock

[kridaydave]

Cuts the developer gate's memory and disk cost, and about a third of its wall clock. Two commits, both test-side or measurement-side; no production behaviour changes.

What it changes

1. Fixture tarballs stop staging a full copy of themselves. tests/support/tarball.rs built every fixture tar in a Vec, handed the whole thing to the gzip encoder, and compressed at level 6. The bomb fixture is 200 MiB of zeroes and the entry-count fixture is 100,001 headers, so in an opt-level = 0 build the staging and the level-6 deflate were most of the test. It now streams straight into the encoder at Compression::fast(). Nothing downstream can tell: the only compression ratio any test asserts is that the bomb still arrives under 1 MiB, which holds at every level.

2. The packument size estimate stops building a String to measure it. packument_size_estimate called serde_json::to_string(sig) on every signature block purely to read .len(). It now passes a counting io::Write sink to serde_json::to_writer: same CompactFormatter, same byte stream, same count, no transient string.

3. The 256 MiB memo fixture is now 96 MiB. Same 64 versions, signature blocks of 768 KiB instead of 2 MiB. Still 1.5x the 64 MiB per-document ceiling that the test exists to prove refuses.

Numbers

All on the dev box (Fedora, rustc 1.98.1, 4 cores, load ~1.0). "Paired" means both sides measured back to back on the same box, alternating, so drift hits both.

Warm gate, same box, back to back

tree	scripts/verify.sh	lib tests
perf/overlap-and-window (before this branch)	77.27 s	26.27 s
this branch	42.69 s	12.49 s

git checkout 17e9bc2 -- src/registry/npm.rs tests/support/tarball.rs
/usr/bin/time -f "%e s" ./scripts/verify.sh      # -> GATE-BASELINE-TREE: 77.27 s
git checkout HEAD -- src/registry/npm.rs tests/support/tarball.rs
/usr/bin/time -f "%e s" ./scripts/verify.sh      # -> GATE-AFTER: 42.69 s

1.81x on the warm gate.

The memo fixture alone, paired A/B, three rounds:

OLD 256 MiB   11.85 / 11.88 / 12.02 s   277 MB peak RSS
NEW  96 MiB    4.52 /  4.46 /  4.45 s   112 MB peak RSS

Scenario-suite memory, paired A/B, three rounds

round	before wall	after wall	before RSS	after RSS
1	15.03 s	17.02 s	333792 KB	30188 KB
2	14.14 s	13.93 s	333820 KB	30432 KB
3	19.80 s	12.37 s	334020 KB	31732 KB

Peak RSS 326 MB -> 30 MB, in all three rounds. Wall clock here is inside the noise on a loaded box and no ratio is claimed for it.

Disk

No code needed for this one. The growth was stale artifacts: 237 incremental session dirs and three copies each of the four largest test binaries, from branch switches.

$ du -sb target                                        # 11.19 GB
$ cargo clean -p blueline
    Removed 28457 files, 11.6GiB total
$ du -sb target                                        # 2.09 GB, 2.91 GB after the gate rebuilds

11.19 GB -> 2.91 GB, 3.8x, and dependencies are untouched.

Honest note on the target

The brief asked for 2-3x. On disk it is 3.8x. On gate wall clock it is 1.81x warm, which misses. Two things did not deliver:

• PERF.md's diagnosis of the 20 s test was wrong in a way that mattered. It describes the to_string call as materialising "~256 MiB of transient String". It never held more than about 4 MiB, because it serialised one version at a time and dropped it. The counting writer is still the right code, but it was worth ~7 MB of RSS and a few percent of wall, not half the test. The real cost was the 256 MiB fixture plus its escaping at opt-level = 0, which is what commit 3 attacks.
• PERF.md section 7 item 3 points at the incremental caches. They are real (4.85 GB across 237 session dirs) but the larger term was 3.2 GB of duplicated test binaries in deps. Same one command clears both.

Two known costs remain, both measured and both left alone:

• Eight mock-server tests in src/registry/npm.rs each burn a fixed 600 ms accept window, plus a 10 s idle poll in a_packument_is_fetched_once_per_package: ~6.8 s of serial sleep, about 1.7 s of lib wall after threading. The fix is a server stopped by the test rather than by a clock. It is mechanical, touches 9 tests with per-test request-count expectations, and wants to be its own commit.
• Warm incremental builds are unchanged: touch src/diff.rs && cargo build --locked is 8.05 s, same as before this branch.

What still guards the packument memo ceilings

The 256 MiB was four times the 64 MiB ceiling. Cutting it to 1.5x is safe because the ceiling was never pinned by this fixture's size, and that was checked rather than assumed:

• the_packument_memo_ceilings_are_the_documented_sizes pins both constants as values (64 * 1024 * 1024, 67_108_864, 256 * 1024 * 1024, 268_435_456), so a mutant that changes the constant dies there regardless of the fixture.
• the_per_document_ceiling_is_strict pins the one-byte comparison (> vs >=) by handing the ceilings into insert_within and using a 1057-byte document. Its own comment says the real ceiling "cannot express this without a 64 MiB fixture" -- which is why that helper takes the ceilings as parameters.
• the_whole_memo_ceiling_is_strict_and_adds_rather_than_multiplies covers the total ceiling, including sum-versus-product.
• the_packument_estimate_is_exact still computes its expected signature length with to_string and compares, so the counting writer keeps an independent oracle that did not change.

The fixture's remaining job is the wiring: a large many-version packument going through the production insert with the real constant and being refused. It still does that. What it no longer buys is a 4x margin, which only mattered against a mutant that raised the constant and the assertion pinning it in the same build.

Every assertion is preserved. The floor moved from a loose > 200 MiB to >= SIGNATURE_FIXTURE_BYTES, which is tighter rather than looser: an estimate that dropped the signature term would land near 64 KB and fail either way.

Checks

./scripts/verify.sh green on this branch: PASS Format, Clippy, Tests, References, 839 lib tests and 22 scenario tests, 0 failures. Rebased onto main at afd724c; both commits apply cleanly on their own without the rest of perf/overlap-and-window. No Cargo.lock change, no new dependency, no unsafe.

The red "Supply-chain audit" check is not from this diff

It fails on rustls 0.23.45, pinned in Cargo.lock:

RUSTSEC-2026-0346  Debug impls for client session values expose TLS secrets
RUSTSEC-2026-0347  Buffered application data sent to outer server after ECH rejection
RUSTSEC-2026-0348  Brotli certificate decompression can allocate up to 1 GiB before authentication
RUSTSEC-2026-0349  Unauthenticated ServerHello can downgrade later connections

All four are dated 2026-10-14 and are patched in rustls >= 0.23.46.

This diff changes no dependency: git diff main..HEAD --name-only is
src/registry/npm.rs and tests/support/tarball.rs only, and Cargo.lock is
byte-identical to main's. The audit job reads only that lockfile. main's last
three CI runs are green because they finished before the advisory database
updated (last-updated 2026-10-14T15:17:45+02:00); a run of main started now
would fail the same way.

The fix is a lockfile bump to rustls 0.23.46, which is a dependency change and
so needs its own decision and its own PR — deliberately not smuggled in here.