Skip to content

Revert "cltest: share one go-build cache across captured runs"#2045

Closed
cpunion wants to merge 1 commit into
xgo-dev:mainfrom
cpunion:revert-gocache
Closed

Revert "cltest: share one go-build cache across captured runs"#2045
cpunion wants to merge 1 commit into
xgo-dev:mainfrom
cpunion:revert-gocache

Conversation

@cpunion

@cpunion cpunion commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Reverts the #2035 cache sharing. The causal matrix from this week's investigation:

combination ubuntu coverage job
heavy suite (stage-5 branches, weeks) + fresh per-test GOCACHE green
light suite (pre-backbone main) + shared cache green
heavy suite + shared cache, one VM runner killed ~8 min into the heavy phase (exit 143), reproduced on main and every stage-5 branch
heavy suite + shared cache, one suite per VM (#2044) green (4/4 ubuntu, 2/2 mac)

Same-spec local runs (16GB cgroup) rule out memory (9.9GB peak) and disk (~4GB); the remaining mechanism is platform-level: the per-test fresh cache forced go-loader re-typechecking between llgo compile bursts — a natural stagger. Sharing removed it, the four heavy packages' clang/lld bursts aligned into sustained saturation, and the runner agent starves until GitHub reclaims the VM ("runner has received a shutdown signal").

This revert restores the stagger and unblocks main and the seven stage-5 leaf PRs immediately, at the cost of the measured speedup (mac -30%, ubuntu -43%). #2044 (one heavy suite per VM) is the structural fix that makes the sharing safe again — sharing can be re-landed after it.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@cpunion

cpunion commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator Author

Closing: the revert does not cure the failure family — its own CI shows the mac coverage job dying anyway (internal/cabi SIGKILLed by the kernel at peak co-residency of the four heavy test binaries on a 7GB runner), i.e. the cache sharing only changed the failure mode, not the cause. Investigation continues into a suspected compiler-side memory growth (post-backbone same-suite peak doubled 5.0GB -> 9.9GB in a controlled 16GB sandbox); the fix will come as a new PR, with #2044's one-suite-per-VM split as the structural companion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant