Skip to content

Cut the full gate's time without weakening it (§9.4) - #526

Merged
afflom merged 12 commits into
release/v0.4from
claude/project-thread-tolt0k
Nov 15, 2026
Merged

afflom merged 12 commits into
release/v0.4from
claude/project-thread-tolt0k

Conversation

@afflom

@afflom afflom commented Nov 13, 2026

Copy link
Copy Markdown
Member

Requested by Alex · project thread

Baseline (run 43433085919, last green full gate): 205 min wall, 1738 runner-min, 70 jobs. Every case keeps its assertions; nothing is cached across runs.

Bootstrap verified once per run. examples re-verified bootstrap (53 min) that bootstrap export had just verified through the same engine.verify at the same profile. export-bootstrap now runs the recipe's before-steps on a clean copy and holds the export's verified directory to the recipe's after-checks (no telemetry, records equal bootstrap/expected/verify); the leg runs without:uor-atlas,bootstrap. RP-14 pins both assertions. Expected: examples leg 110 → ~57 min.

Interpreter stops deep-copying (calculus/interp.rs). Values, environments and list tails are shared through Rc; lookups, steps and weights are charged as before. BS-15's evaluator phase ~800 s → 239 s locally; every bootstrap/stdlib/preservation differential uses it.

Quadratic Rust lowering fixed (calculus/realization.rs). kind() serialized a whole node to JSON to read its tag; now reads the tag only (kind_is_the_json_tag checks every kind). BS-07's 143 harnesses render in 21 s (was >25 min); this bounded the slowest Intel/Windows/arm64 lanes (BS-05/06/07).

AU-08 takes the committed sources on one thread per CPU (1200 s → 575 s locally).

Parts. New stdlib (SL + EX-09) and language-models (LM) parts out of rest; MD-13 joins production, which already builds its models fixture. Host lanes unchanged in effect (new parts sit in moves, which ran them as rest).

Neither language/ nor schemas/ moves; check-freshness passes.

Checked locally: cargo fmt --check, clippy -D warnings (workspace), check-freshness, check-test-inventory (545 tests, 41 parts), validate-model, validate-spec-links, RP-13/14/18, .github/scripts unittests, lexlean lib + integration tests, CF/GN/TC differentials, BS-15/BS-06/BS-07 through their rustc comparisons, AU-08. Intel lanes get rebalanced from this run's timings in a follow-up.


Generated by Claude Code

The `bootstrap export` job's export already verifies the bootstrap project
whole through the same `verify`, selection and profile that the `examples`
recipe runs, and the `examples` leg then verified it again (about 53 of its
110 minutes in run 43433085919). `export-bootstrap` now is that project's
part of the recipe: it formats, locks, checks and builds a clean copy as the
recipe does, exports, and holds the export's verified directory to the
recipe's checks (no telemetry, normalized records equal to
bootstrap/expected/verify), printing the recipe's lines, which the job
asserts. The examples leg runs `without:uor-atlas,bootstrap` and fails if it
verifies bootstrap. `without:` takes a comma-separated list.

SPEC §9.4, RP-14 (pins and plants) and CHANGELOG follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3QVAsEzzaQPMzJnNz1
`realization::kind` serialized a whole calculus node to JSON to read its
`kind` field. The Rust lowering asks for the kind of every node it lowers
(`value_at`, `term_at`), so lowering a value or a term was quadratic in its
depth: rendering the rustc-differential harnesses of BS-07 spent over 25
minutes of a debug build on it, and the bootstrap cases' harnesses dominate
the supported-host lanes (BS-05/06/07 took 57/51/73 minutes on Intel).

`kind` now runs a serializer that stops at the tag, the first field an
internally tagged enum writes, so it costs the tag alone. Locally, BS-07's
143 harnesses render in 21 s instead of more than 25 minutes, and its 1524
rustc runs still agree with the oracle. `kind_is_the_json_tag` holds the
answer equal to the JSON form's field on a sample of every kind of every
syntax class.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3QVAsEzzaQPMzJnNz1
AU-08 runs the authoring API's round trips on every committed source, one
after another: 55 minutes of the Intel `moves` lane, 26 on Linux arm64 and
19 on macOS arm64, most of it after the lane's other cases had finished. The
sources are independent, so they are now taken from one list by one thread
per processor, each on the stack every authoring case reads on
(`lexlean::READER_STACK`); the per-kind counts are summed across them and
held to the same bounds. Locally the case went from 1200 s to 575 s on a
loaded 4-core host. Every source still gets every assertion; a failing one
is raised after the others finish.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3QVAsEzzaQPMzJnNz1
The reference interpreter shares its values (Rc), extends a scope by one
shared binding instead of copying it, and takes a list's tail without
copying it. Lookup depth, steps and weights are charged as before, so every
outcome is unchanged; BS-15's evaluator phase measured ~3x faster locally.

bdd-parts: SL-* and EX-09 become the `stdlib` part (the stdlib and helpdesk
exports, whose waiters held three of rest's four threads), LM-* the
`language-models` part, and MD-13 joins `production`, whose verified_models
fixture it shares (it verified `models` a second time in rest). Rows added to
vv-rows.json; both parts run in the Intel/Windows `moves` lane, which already
held them as part of rest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3QVAsEzzaQPMzJnNz1

@afflom afflom left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two independent bounded source reviews found no blocking defect in the submitted efficiency changes.

Bootstrap still reaches the unchanged engine.verify(Entrypoints) through production build, with the same width/memory/hints. Export runs the recipe's before-steps and checks the actual retained verified directory for telemetry and complete normalized expected/verify equality before artifact upload. The examples row then excludes bootstrap; ordinary local All behavior remains. This removes a demonstrated duplicate verification without substituting package equality or a log marker for verification. Same-run custody and fail-closed consumers remain.

New stdlib/language-models parts retain one vv row and moves-host coverage; dynamic rest exclusion and inventory comparison preserve whole-suite coverage. MD13 shares its existing verified-model fixture in production. AU08 distributes each existing path once, retains assertions/exclusions/anti-vacuity counts and propagates worker panics.

Interpreter Rc sharing is immutable; visible list tails, binding/parameter order, fuel, steps and weights remain consistent in the inspected paths. Tag-only serialization is used by the closed internally tagged calculus types and retains their tag names; no arbitrary-Serialize equivalence is claimed. No blocking semantic defect was demonstrated. Sharing can retain list prefixes, so this review makes no measured-memory claim.

The timing reductions and local test results remain owner evidence; neither benchmarks nor checks were repeated. This is source credit, not final acceptance. Finish the existing exact-head full CI43515398695/repro43515398878, review the complete resulting inventory and actual merged-tree equality before delivery. Preserve valid existing products; no extra identity writer or universal gate is requested.

@afflom afflom mentioned this pull request Nov 13, 2026
Each reader thread holds up to one module, about 1 GiB at peak, so the
case is bounded near 4 GiB beside the other cases of its leg on the
16 GB runners, whatever the host's CPU count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3QVAsEzzaQPMzJnNz1

@afflom afflom left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bounded delta review: c0f5f9b directly follows reviewed556c955 and changes only AU08's reader concurrency cap. The same path queue is drained exactly once, with unchanged assertions, counters, worker joins/panic propagation and anti-vacuity thresholds. No blocking source defect found; prior source credit and inherited findings remain.

Source establishes at most four readers, not a 4 GiB RSS bound. The per-reader/whole-case memory figures remain owner estimates; no memory measurement or completed check was repeated. No producer inputs or committed products change.

Exact-head CI43627806271 was canceled with no jobs, while repro43627806184 passed. Prior556 CI43597143352 also ended canceled, including one failed Intel build/non-Lean job and four canceled jobs; its successes cannot supply current full acceptance. Existing implementing owner should disposition the canceled final-head gate and provide complete acceptance plus actual merged-tree equality before delivery. No retry, competing patch or CI control was initiated. Receipt pr526-c0f5-reader-cap-source-delta/receipt.json.

afflom commented Nov 14, 2026

Copy link
Copy Markdown
Member Author

I cancelled runs 43627806271 and 43597143352 on purpose: this lands after #536, which fixes the AU-01 OOM in bdd-verification. Once #536 merges, I'll merge release/v0.4 in and run one complete gate on that head. I merge only when every check on it is green.


Generated by Claude Code

A newer push cancels the pull request's older run, and the Intel and
Windows aggregates, guarded by always(), still ran and failed their
'every lane succeeded' step, so every superseded head showed a red host
context. They now run on !cancelled(): still after a failed or cancelled
lane of a run that goes on, never in a run that was itself cancelled, which
leaves no evidence. RP-14's pins and plant follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013Asa3QVAsEzzaQPMzJnNz1

@afflom afflom left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bounded integration review at81249b2: AU-01's lazy corpus matches the previously reviewed #536 implementation, while AU-08's capped reader implementation and its obligations remain byte-identical to the prior reviewed version. The inspected efficiency/custody/partition source blobs retain their earlier credit; no new blocker was demonstrated in this integration delta.

The aggregate cancellation guards retain joins when prerequisites fail while allowing canceled workflows to stop. That source behavior does not convert a canceled or skipped job into acceptance. Four reader threads remain a concurrency cap, not a demonstrated memory bound.

The existing integrated gate43675919328 is still pending. Complete final-head acceptance, same-run obligations, exact reproducibility and actual merged-tree equality remain required; previous-head green or canceled runs are not combined with this run. No new producer, repeated local test or competing qualification was started. Stopped imported history received no runtime adoption.

@afflom
afflom merged commit a367f4d into release/v0.4 Nov 15, 2026
70 checks passed
@afflom
afflom deleted the claude/project-thread-tolt0k branch November 15, 2026 09:32

afflom commented Nov 15, 2026

Copy link
Copy Markdown
Member Author

Post-landing acceptance audit: final81249b2f proposal CI43675919328 completed66 successful jobs/750 successful steps plus3 skipped Intel obligations. Exact proposal repro passed. Actual merge a367f4d has tree280266ee, unequal to reviewed proposal treef7a0c687; actual CI43699719484 is canceled with zero jobs, while actual repro passed.

The bounded source improvements and completed successful work retain their credit, but this is not full premerge or actual-tree acceptance under the standing instruction. No skipped/canceled aggregation or retrospective approval. Existing owner retains actual-tree qualification and relevant source disposition; no repeated producer or gate requested. Complete inventory receipt cbf6af24c464f1d8df9902664a5331bc4f6f9b7e81246895425f7def029f9241 also records #529 separately. All stopped-runtime and frozen evidence boundaries remain unchanged.

@afflom afflom mentioned this pull request Nov 15, 2026
humuhumu33 added a commit that referenced this pull request Nov 15, 2026
…#535, #536) into the arrays branch

The generated outputs that both sides rewrote take release/v0.4's copy, or
are removed where the line no longer has them; regenerate.yml rewrites the
whole set from this tree.

Lean moves to 4.32.2 with #518, so the array authority follows the other Lean
rows: LEAN-ARRAY-4-32-1 becomes LEAN-ARRAY-4-32-2, cited at f3b06c705e6c.
The three files its statement rests on (src/include/lean/lean.h,
src/Init/Prelude.lean, src/Init/Data/Array/Set.lean) are byte-identical at
f054605aea4b and f3b06c705e6c, so the statement carries over unchanged.

runtime-1.2/library.toml keeps the line's toolchain and the branch's Arrays
module. LANGUAGE_1_2_SEMANTICS_ID is the merged tree's, as RP-10 recomputes it.
CHANGELOG and VERIFICATION keep both sides' entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants