Skip to content

AU-01: read the corpus one module at a time (bdd-verification OOM) - #536

Merged
afflom merged 2 commits into
release/v0.4from
fix/v04-au01-corpus-memory
Nov 15, 2026
Merged

afflom merged 2 commits into
release/v0.4from
fix/v04-au01-corpus-memory

Conversation

@afflom

@afflom afflom commented Nov 14, 2026

Copy link
Copy Markdown
Member

Requested by Alex · project thread

Root cause

The just vv (bdd-verification) leg sometimes ran out of memory. It runs the 91 cases of the verification part in one test process, on libtest's default four threads, on a 16 GB runner. The leg sets no verify profile, so each case verifies at width 1. In the failure on #405 (run 43582153525), memory climbed to 13.6 GB with swap nearly full while AU-01, AU-08, AU-09 and NE-01 were all running. The runner then thrashed for five minutes and the job was lost.

I measured peak RSS locally, running each case alone at --test-threads=1:

Case Peak RSS
AU-01 11,467 MiB (all in the test process, no child processes)
VR-11 1,564 MiB (Lean child process 1,253 MiB)
AU-09 340 MiB

Verifier telemetry from a passing run puts the largest Lean process at about 1.8 GB. AU-01 is the outlier. corpus() walked the repository and collected every semantic module as a parsed Json value before checking any of them. The corpus includes modules of tens of megabytes, and holding all of them at once in a debug build takes about 11 GiB. Whichever case happened to be running next to AU-01's peak was the one that failed: NE-01 on #405, and AU-08, OR-01 or VR-11 on v0.4 pull requests.

Fix

corpus() is now a lazy iterator over the same walk. AU-01 checks the same modules, in the same order, with the same assertions, including the anti-vacuity counts. Each module is dropped once it has been checked, so AU-01's peak is now roughly one module rather than the whole corpus. No test is skipped or weakened.

  • The leg's parallelism, ci.yml, .github/vv-rows.json and .github/bdd-parts.json are unchanged, so SPEC §9.4 and RP-14/RP-18 are unaffected.
  • cargo check -p repo-conformance --tests --all-features passes locally.
  • The leg's ci-summary "Largest process" column will show the new peak in this run's evidence.

Overlap with #526

#526 edits only authoring_api (AU-08), at lines 1288 and after in the same file. This PR touches only corpus() and its doc comment (lines 74–106), so the two do not conflict. AU-08 is left alone here.


Generated by Claude Code

The case collected every semantic module of the repository as a value before
checking any. The corpus includes modules of tens of megabytes, so AU-01 alone
peaked at 11.2 GiB resident in the debug test process. The bdd-verification leg
runs it on four test threads beside AU-08, AU-09 and the Lean processes of
NE-01 and the VR cases on a 16 GB runner, and the leg intermittently exhausted
memory and swap and lost the runner, failing whichever case was running then
(NE-01 on #405, AU-08, OR-01, VR-11 on v0.4 pull requests).

corpus() is now a lazy iterator over the walk: the same modules, in the same
order, under the same assertions, each dropped once checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3M8ULnzEQwLLngyBBX

@afflom afflom left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bounded source review finds no blocking defect in the AU-01 corpus change. The same repository walk, traversal order, exclusions, skipped-error handling, assertions and anti-vacuity counts remain. Returning a lazy iterator removes retention of every parsed module at once; each owned module is confined to its AU-01 iteration. #526's AU-08 changes are disjoint.

This establishes the lifetime/retention improvement in source. The reported RSS values and OOM attribution remain owner measurements/interpretation here; no independent peak or speedup is claimed. No new measurement or repeated check is requested.

Existing owner retains complete exact-head gate43625607403 and actual merged-tree acceptance; reproducibility alone is insufficient. Receipt pr536-au01-lazy-corpus-source-review/receipt.json. No tests/builds/probes/producers or stopped#431/oldTC03 validation were performed.

afflom commented Nov 14, 2026

Copy link
Copy Markdown
Member Author

just vv (bdd-verification) passed on this head: largest process 2,463 MiB (was 5,972 MiB on a recent passing run, 11.2 GiB peak for AU-01 locally), lowest free memory 11,263 MiB, no swap used.


Generated by Claude Code

@afflom
afflom merged commit f76a972 into release/v0.4 Nov 15, 2026
68 checks passed
@afflom
afflom deleted the fix/v04-au01-corpus-memory branch November 15, 2026 04:00

afflom commented Nov 15, 2026

Copy link
Copy Markdown
Member Author

Post-landing audit: actual f76a972 has parents4727a691 + reviewed43546c60, but its tree differs from that final proposal. Prior AU-01 source credit is retained; actual composition needs its own applicable evidence.

Proposal run43649543322 completed64 successful jobs/725 successful steps and3 skipped Intel obligations. Exact proposal reproducibility passed. Actual run43674846380 is pending with zero jobs; actual reproducibility43674846356 passed all seven steps. This is not full acceptance under the current full-premerge instruction, and a successful aggregate does not turn skipped Intel work into executed checks. The existing external-waiver conflict remains unresolved.

Existing owner retains actual-tree qualification and policy disposition; preserve successful receipts at their exact bindings. No cross-run aggregation, retrospective approval, producer repeat, retry or CI control was performed. #431/oldTC03 STOP and frozen #316 evidence remain unchanged.

humuhumu33 added a commit that referenced this pull request Nov 15, 2026
…#535, #536) into the arrays branch

The generated outputs that both sides rewrote take release/v0.4's copy, or
are removed where the line no longer has them; regenerate.yml rewrites the
whole set from this tree.

Lean moves to 4.32.2 with #518, so the array authority follows the other Lean
rows: LEAN-ARRAY-4-32-1 becomes LEAN-ARRAY-4-32-2, cited at f3b06c705e6c.
The three files its statement rests on (src/include/lean/lean.h,
src/Init/Prelude.lean, src/Init/Data/Array/Set.lean) are byte-identical at
f054605aea4b and f3b06c705e6c, so the statement carries over unchanged.

runtime-1.2/library.toml keeps the line's toolchain and the branch's Arrays
module. LANGUAGE_1_2_SEMANTICS_ID is the merged tree's, as RP-10 recomputes it.
CHANGELOG and VERIFICATION keep both sides' entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
afflom pushed a commit that referenced this pull request Nov 15, 2026
Brings #536 (the AU-01 corpus memory fix) and the line's other landings;
the aarch64 cargo-test leg lost its runner twice on the older base.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3MAztfvN671cYrybev
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants