ssa,cl,build: release per-compile memory in in-process drivers

[cpunion]

Problem

The golden test harness (cltest, llgen) compiles ~85 packages inside one process, and each compile's memory lived until process exit: cl.test RSS climbed monotonically from 140MB to over 2GB during the Testrt+Testdata suites. At suite level (sandboxed 16GB cgroup, go test -coverprofile ./...), peak memory was 9.9GB post-#2024 (5.0GB before it) — this co-residency pressure is what killed 7GB macOS runners (jetsam SIGKILL) and starved ubuntu runner heartbeats (exit 143), see #2045.

Two retainers, found via heap profile at the end of the suites (1.1GB retained):

1. Caller-tracking memoization pins every compile's front end (introduced with the uniform runtime-caller tracking in runtime,cl,build: Go-style caller info — funcinfo tables, link-phase pcln, FP unwinder, gc-verified conformance (stage-5 backbone) #2024): runtimeCallerBaseCache / runtimeCallerExtendedCache are global sync.Maps keyed by *ssa.Package with map[*ssa.Function]bool values — both keys and values pin the whole go/types + go/ssa graph of every package ever compiled in the process. The profile showed 216MB in go/types.(*Checker).recordTypeAndValue alone, plus x/tools/go/ssa and go/parser data.
2. No LLVM disposal existed anywhere: each compile's llvm.Context/TargetMachine/TargetData (C++-side memory, invisible to Go heap profiles) accumulated for the life of the process.

Fix

• Add Program.Dispose() (context + target machine + target data, guarded, idempotent). internal/build.Do defers it for all modes except ModeGen (whose callers read LPkg.String() after Do returns — llgen.GenFrom disposes there instead); the cabi tests (ModeGen loops over arch×mode×file) dispose per iteration.
• Replace the global caller-tracking sync.Maps with an exported cl.CallerTracking, created once per compilation by build.Do's context and passed to NewPackageExWithEmbed for every package of that compilation — the same driver-owned, compilation-scoped flow as cl.Patches. cl keeps no package-level data state, so nothing can outlive a compile and pin its front-end graphs. NewPackage/NewPackageEx keep their signatures and compile as one-shot compilations; ct == nil does the same. (Design discussed in proposal: scope compile-level memoization to the compilation instead of package globals #2049, closed as implemented here; concurrent in-process Do remains gated only by the pre-existing cl config-flag globals, parked in the Proposal: runtime funcinfo metadata and fast FuncForPC lookup #2004 ledger.)
• Single-shot llgo CLI behavior is unchanged (memory was reclaimed at process exit anyway).

Evidence

Per-process trajectory (darwin/arm64, cold llgo cache, RSS sampled every 5s, Testrt+Testdata in one cl.test process):

variant	first	last	peak
before	140MB	2051MB	2051MB
Dispose only	139MB	1896MB	1896MB
Dispose + compilation-scoped tracking (this PR)	129MB	562MB	562MB (-73%)

The trajectory now plateaus instead of climbing monotonically.

Suite level (16GB-cgroup sandbox, linux/arm64, go test -coverprofile -covermode=atomic ./... — same protocol as the 9.9GB post-#2024 number in the #2045 investigation):

process	before	after
build.test peak RSS	3153MB	1451MB
ssa.test peak RSS	2889MB	619MB
cabi.test peak RSS	2291MB	400MB
co-resident top-process RSS at worst sample	6211MB	2907MB (−53%)

Peak anonymous memory of the whole suite cgroup (the metric that triggers OOM/jetsam) is 2.9GB with this PR — below pre-#2024 levels. The memory.current ~10GB readings are page cache from llgo-cache/object/coverage writes (reclaimable, not kill-relevant), so the earlier 5.0GB/9.9GB cgroup peaks over-stated the resident footprint; the real regression was the per-process accumulation fixed here. On 7GB macOS runners, post-#2024 co-residency sat at the jetsam threshold; with this PR the same co-residency is ~1GB.

Validation

• go test ./cl ./ssa ./internal/build ./internal/cabi all pass locally on the final version.
• Runtime unchanged: cabi 74s warm-cache after vs 108s before; disposal cost is negligible.
• New tests: Program.Dispose nil-safety/idempotence; NewPackageExWithEmbed with a shared non-nil CallerTracking (public API); existing caller_frame tests exercise the shared-instance cross-package memoization and the nil one-shot path.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 92.30769% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/llgen/llgenf.go	0.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

[cpunion]

Suite-level re-measurement done (16GB-cgroup sandbox, linux/arm64, go test -coverprofile -covermode=atomic ./..., cold llgo cache, 5s sampling) — same protocol as the 9.9GB post-#2024 number in #2045's investigation.

Peak RSS per test process, before → after this PR:

process	before	after
build.test	3153MB	1451MB
ssa.test	2889MB	619MB
cabi.test	2291MB	400MB
cl.test (darwin/arm64, Testrt+Testdata)	2051MB	537MB
co-resident top-process RSS at worst sample	6211MB	2907MB (−53%)

Peak anonymous memory of the whole suite cgroup (the metric that actually triggers OOM/jetsam) is 2.9GB with this PR. The memory.current peak still reads ~10GB, but that is page cache from llgo-cache/object/coverage writes — reclaimable under pressure, not kill-relevant. This means the earlier 5.0GB (pre-#2024) and 9.9GB (post) cgroup peaks both over-stated the resident footprint; the real regression was the per-process accumulation fixed here.

Relevance to the runner deaths: on 7GB macOS runners, post-#2024 co-residency (cl.test + cabi.test together >4GB anon, plus toolchain processes) sat right at the jetsam threshold; with this PR the same co-residency is ~1GB. Runtime is unchanged (cabi 74s warm-cache after vs 108s before; disposal cost is negligible).

Remaining sandbox test failures are the known environment noise (root container makes chmod-000 files readable; linux/arm64 golden gaps) — identical failure set before and after.