Skip to content

perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614

Open

Description

@zackees

Context

Focused next work order extracted from #585: reduce the CPU cost of reusing large pages while preserving the memory savings already shipped. Deliver one measured optimization, with a complete memory and latency ledger.

#584's memory savings originally cost roughly 11–15% CPU on large-class and ephemeral workloads. #609 recovered modest CPU gains; #611 then narrowed retired-page searches and merged as 236533318c4e22d7104fcc671ecf70f41e136a67. These historical costs are motivation, not a measurement of today's remaining debt. Both follow-ups claimed no new RSS saving. #610 is complete; do not repeat its search-bounds optimization.

Observed main at filing: 227eef06ff03f92966f38508441d6c2f60a75135. Freeze the exact current merged main SHA at campaign start, after verifying it includes #611, and use that same baseline throughout the candidate comparison.

Benchmark prerequisite: restore and verify benchmark-stats

Execution split: #615 owns benchmark restoration and its separate PR. This issue owns the CPU/memory optimization PR, after #615 is verified. The detailed prerequisite below remains the required handoff contract.

Also covers the user-reported failed build step, run 39577325702 on source ee3623f6b7a485754a0a0aa3462d8cba43390e8b (2026-10-21).

The fatal error in ci/build_benchmark_allocators.py was ArchiveError: tcmalloc generated module lock mismatch: expected 340a645e18fb1cdaed3dfd465312a3447de84ab31f78bd7db1863e2ee80f79f6, got c0b842b0e615e31c165a5d7f005f97fe6c0dcbed05576ebf17967d0702232544. Measurement never started. The earlier soldr cook warning about no root Cargo.toml was nonfatal and is not the demonstrated cause.

PR #608 subsequently merged the matching repair: a checksummed TCMalloc dependency-lock patch, exact Bazel 9.1.0 selection, and query/build with --lockfile_mode=error. It also records successful full-mode verification on its own source. Therefore first verify that current main contains this fix and that the existing benchmark-stats workflow succeeds on the current source; do not duplicate the patch or treat this historical run as proof that main is still broken. Fix any remaining reproducible blocker in the existing builder/workflow. Inspect ci/build_benchmark_allocators.py, rust/benchmark-suite/allocators/allocator-lock.json and its patches, .github/workflows/benchmark-stats.yml, and the allocator-builder tests.

  • Retain the original failing evidence and verify fix(bench): snapshot TCMalloc Bazel dependency lock #608's regression tests cover lock corruption/tool-version drift (RED → GREEN for any newly discovered defect). Preserve checksums, pinned dependencies and fail-closed validation; do not accept an arbitrary new hash or skip TCMalloc.
  • Record a successful full-mode benchmark-stats run on the exact repaired/current main source, covering native allocator build, complete measurements, raw-data validation and site rendering/validation. Verify normal scheduled publication is restored, with fresh benchmark data and a recorded run/source SHA. A reduced smoke run or a green build alone is insufficient.
  • Complete this prerequisite before relying on the benchmark pipeline for this CPU campaign. Keep benchmark repair commits distinct from allocator changes. If fix(bench): snapshot TCMalloc Bazel dependency lock #608 fully resolves it, record verification and reuse that fix without unnecessary code changes.

Proposal

  1. Attribute before editing. Profile large-class/8, large-class-ephemeral/8, and the chart-build short-generation ephemeral workload on the frozen baseline. Separate retired-page selection, retirement/publication, geometry/page-map work, free-list formation/page initialization, and arena/OS work. Record event counts and sampled CPU attribution; distinguish instrumented work counts from timing evidence. Use ci/attribution_probe.py and the existing perf-ab/event/USDT tooling; acceptance timing follows the CI measurement process in docs/dev-loop.md.
  2. Choose one demonstrated source of avoidable CPU work. Inspect mi_page_repurpose_retired, mi_page_set_geometry, and _mi_page_init in src/page.c, src/page-retired-search.c, and their page-map/retirement callees. These are investigation points, not a preselected culprit. Preserve resident reuse, selection semantics, page geometry, aging, retention and purge policy for this work order. Prefer removing redundant bookkeeping or initialization after proving its invariants.
  3. Implement and validate one optimization. Add a focused RED → GREEN regression/work-count test for the specific redundancy or invariant before implementation. Measure the candidate against the frozen baseline with unchanged traces, seeds, touches and operation counts. Preserve actual executables, raw results and rejected revisions.

Acceptance criteria

  • Publish baseline/source SHAs, attribution, commands, and the selected mechanism. Evidence identifies avoidable work remaining after perf: narrow retired-page searches using existing owner bounds (#610) #611.
  • Focused test is RED on the baseline and GREEN on the candidate; ownership, concurrent retirement/scavenging, page-map validity, and initialization/failure paths affected by the change remain correct.
  • Declare target/control cells, sample caps, noise methodology, service bounds and material-risk bounds before measurement. Cover the existing 12 default perf-ab rows plus the sparse 64-slot diagnostic used by perf: narrow retired-page searches using existing owner bounds (#610) #611, including small/Larson controls, profiler-on, random/bursty and ephemeral variants.
  • Report per-cell throughput, CPU/op, minor faults, peak RSS, after-drain RSS and sampled p99, with 95% intervals and original PASS/INCONCLUSIVE/FAIL labels. Show at least one statistically supported target CPU/throughput gain; no arbitrary minimum percentage is required.
  • Preserve memory savings and release bounds. Claim zero memory credit unless a new reduction is actually established. Apply AGENTS.md rule 12: no confirmed per-cell regression beyond its allowance, no p99 credit, and no gains averaged across cells/hosts. Resolve uncertainty under the documented acceptance policy; prior owner acceptance of perf: reduce large-page bookkeeping and complete measurement gates (#585) #609/perf: narrow retired-page searches using existing owner bounds (#610) #611 is not a waiver for this candidate. Do not rerun unchanged failures to obtain green.
  • Pass applicable full C/platform, native MSVC, win-GNU, Rust/layout, sanitizer, memory and default-fast-path identity gates. Preserve C-core/Rust commit separation and local-gate stamping. Put the final ledger and disposition in the PR and this issue.

Decisions

  • Priority P2; benchmark restoration/verification first, followed by one optimization PR, branch perf/<issue-number>-large-reuse-cpu.
  • This issue pays down CPU while protecting memory. Further persistent-buffer RSS reduction is separate research: perf: next steps after #575 — pay down #584 CPU, large-path fork/upstream regression, sparse-large gap #585's corrected persistent/8 result was 1.224× TCMalloc, but changing geometry or purge policy would broaden this task.
  • Do not retry retired-body discard, rejected aging/budget settings, or perf: reduce retired-page best-fit search overhead #610's rejected inline prototypes. Any later release policy requires reuse-distance evidence first.
  • If attribution finds no justified optimization, record that evidence here and stop under rule 7; do not invent a policy change or claim completion of the optimization.
  • No new GitHub Actions workflow files or required C dependencies.

Related issues

Activity

  1. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    Baseline investigation started

    Frozen source for the initial attribution: 227eef06ff03f92966f38508441d6c2f60a75135, after #611. Benchmark prerequisite is split into #615; full current-main run 39815683542 is still queued, so publication is not claimed restored and no candidate acceptance campaign has started.

    Local sampled profiles of the unchanged baseline identify _mi_page_unpurge_all as one potential avoidable cost (about 0.66% self samples on large-class/8 and 1.39% on plain ephemeral in these diagnostic replays). Kernel symbols are restricted; these samples are attribution, not portable timing estimates or evidence of a gain. Disassembly shows the no-work case still enters the full function prologue, saving five callee-saved registers plus the frame pointer and reserving 72 stack bytes before testing the purge state.

    A separate instrumented replay counted calls and calls with either a nonempty unformed purged range or nonzero formed-purge bitmap:

    Original workload Calls Calls with purge work
    large-class/8 1,078,045 0
    large-class-ephemeral/8 1,073,032 0
    short-generation chart ephemeral/8 265,819 0

    The probe used atomic counters and is deliberately unsuitable for performance comparisons. Its patch, reports, exact baseline executables, compiler/host identity and raw perf data are retained locally for the eventual durable evidence archive; the probe has been removed from the source worktree.

    Next hypothesis: a small public entry with an exact no-work check and a separate slow body can avoid entering that large body for clean pages, without changing any purge/retirement policy or metadata. Prove the no-work dispatch and dirty formed/unformed cases before implementation; evaluate performance only after the benchmark prerequisite. This is not yet an accepted optimization.

  2. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    Candidate and predeclared campaign

    Draft PR #616 contains a small clean-page unpurge entry and the unchanged restore body as an explicitly noinline slow function. C-core and Rust regeneration are separate commits; current stamped head is 287cac2f664a. Local gate 45/45 PASS; sole source reviewer CLEAN. The diagnostic work-count regression is RED on the instrumented original implementation and GREEN on the candidate. C Release purge/event/large-span tests and C++ Debug/full event/large-span tests pass. The optional C++ purge-hole target fails on unchanged-source goto-across-initialization errors; its C test passes. This is not a performance acceptance claim.

    Before any measurement, declare the following for baseline 227eef06ff03f92966f38508441d6c2f60a75135 and this candidate:

    • Wait for fix(bench): restore and verify full benchmark publication after TCMalloc lock failure #615's full benchmark/publication prerequisite; do not dispatch acceptance measurements before it is verified.
    • Stage 1: exactly large-class/8, large-class-ephemeral/8, and large-class-ephemeral/8 short generations (chart build). Each gets one original cohort: 60 initial actual samples/arm, adaptive cap 120, null arm and sanity gate enabled, unchanged original traces/touches/seeds/operation counts.
    • If stage 1 has no confirmed FAIL, incomplete/sanity failure or service-bound violation, and at least one supported target CPU/op or throughput gain, measure all remaining original cells exactly once: complete the existing 12 default rows plus sparse-large-buffers 64-slot budget/8. Do not repeat stage 1 or replace failures.
    • Record all six metrics with original intervals/verdicts, exact source/build identities and actual executables. Zero memory credit is claimed. Preserve rejected revisions and incomplete data. No host pooling or new numerical tolerance.
    • Existing rule12 controls acceptance: no confirmed scenario FAIL, no latency/RSS/service-bound spike, supported target gains, passing CPU/throughput controls. Uncertainty outside the documented acceptance policy needs attribution or explicit owner disposition; acceptance of older PRs is not inherited.

    The full platform checks are running on #616. The benchmark baseline run has progressed past native library build into the full workload suite; restoration is not yet claimed.

  3. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    Benchmark prerequisite #615 is now verified through publication: full run https://github.com/zackees/mimalloc-pprof/actions/runs/39815683542 passed, and independent local branch/Pages audits match the sealed artifact. Benchmark delivery remains a separate PR.

    All nine original replacement platform workflows for CPU candidate 287cac2f664a46703ebb6522eef8c8dbeb984cbb passed. Archived API records show its tree is identical to CI's test merge c9eab6aebbc68a4763b6063f1ebbf6060b6fe39e (a910b85e21192b946f9d7acda1096c60d437c582). Reviewed attribution and complete platform logs are retained in local evidence commits, not yet pushed; the measured candidate is unchanged.

    Started the predeclared original stage-one cohorts, each 60 initial samples/arm, adaptive cap 120, null arm and sanity gate on, no variants and unchanged workload parameters:

    Fixed baseline remains 227eef06ff03f92966f38508441d6c2f60a75135. No performance result or acceptance is claimed yet. The remaining ten rows are conditional on the previously declared stage-one criteria; no unchanged reruns or cross-host pooling.

  4. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    Stage-one result: candidate not accepted

    Exact baseline 227eef06ff03f92966f38508441d6c2f60a75135, candidate 287cac2f664a46703ebb6522eef8c8dbeb984cbb. All three original cohorts completed with 120 actual samples per arm (base/head/null), unchanged traces and sanity gates. Original executables/hash manifests, raw samples, workflow logs and independently recomputed ledgers are retained. No hosts are pooled.

    No statistically supported target CPU or throughput gain. Across 18 metrics: 10 PASS, 8 INCONCLUSIVE, 0 FAIL; zero memory credit in every row, all release bounds held. The stage-two condition is not met. I will not expand this candidate to the remaining rows or rerun it unchanged. It is rejected for delivery despite successful correctness/platform gates.

    The measured chart binaries confirm the intended prologue removal, but structural work reduction is not a measured benefit. CPU changes in both ephemeral targets lean worse, and the plain-ephemeral RSS interval is unresolved; neither is silently relabelled as equivalence. Further implementation requires fresh attribution within the original CPU-only scope, not weaker thresholds.

    large-class/8

    Original run: https://github.com/zackees/mimalloc-pprof/actions/runs/39823333694

    base 227eef06ff03 head 287cac2f664a on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

    Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

    workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
    large-class/8 0% INCONCLUSIVE +0.1% [-0.6, +0.6] / 0.7% PASS -0.0% [-0.4, +0.3] / 0.3% INCONCLUSIVE -0.6% [-2.0, +1.4] / 2.2% PASS +0.1% [-1.0, +0.6] / 1.1% PASS -0.0% [-0.2, +0.0] / 0.1% PASS -0.4% [-1.8, +0.1] / 1.8% PASS

    large-class-ephemeral/8

    Original run: https://github.com/zackees/mimalloc-pprof/actions/runs/39823337377

    base 227eef06ff03 head 287cac2f664a on AMD EPYC 7763 64-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

    Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

    workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
    large-class-ephemeral/8 0% INCONCLUSIVE -0.4% [-0.7, -0.1] / 0.5% INCONCLUSIVE +0.3% [+0.2, +0.4] / 0.3% INCONCLUSIVE +0.0% [+0.0, +0.8] / 0.8% INCONCLUSIVE -0.0% [-1.0, +0.6] / 1.0% PASS +0.0% [-0.0, +2.6] / 0.1% INCONCLUSIVE +0.0% [-9.4, +0.8] / 9.8% PASS

    large-class-ephemeral/8 short generations (chart build)

    Original run: https://github.com/zackees/mimalloc-pprof/actions/runs/39823341873

    base 227eef06ff03 head 287cac2f664a on AMD EPYC 9V74 80-Core Processor, 60 reps (up to 120); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

    Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

    workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
    large-class-ephemeral/8 short generations (chart build) 0% INCONCLUSIVE -0.3% [-0.9, +0.4] / 0.8% INCONCLUSIVE +0.4% [+0.1, +0.9] / 0.4% INCONCLUSIVE -0.4% [-0.8, +0.0] / 0.7% PASS +1.4% [-0.7, +2.1] / 1.4% INCONCLUSIVE +0.0% [-0.0, +0.0] / 2.6% PASS +0.0% [-0.0, +0.0] / 2.6% PASS
  5. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    V2: call-site guard, predeclared campaign

    V1 remains rejected for no supported target CPU/throughput gain; all original evidence is retained in PR #616. Its wrapper has been replaced with a guard at retirement publication, so clean pages never call the restore function. Original restore behavior remains for dirty pages and other callers. Optimized assembly confirms the clean path has no restore call; caller register-save prologue remains.

    Candidate 59d369609ab47358df0f883b6419252f1ffbf03a, frozen baseline 227eef06ff03f92966f38508441d6c2f60a75135. Main changed only benchmark evidence since the baseline. Separate C and regenerated Rust commits; local gate 45/45, source review clean. Focused clean-retirement test failed before implementation and passes after it. Real-discard tests independently cover formed-only and tail-only restoration and reuse; C event/purge/large 3/3, C++ event/large 2/2. Performance benefit is unproven. Prior platform passes apply only to V1; fresh V2 checks are required.

    Declared before dispatch: initial 60 samples per arm, adaptive cap120, null arm and sanity gate enabled, zero memory credit, no environment/define variants, unchanged traces/seeds/touches/operation counts. All six metrics with original 95% intervals/verdicts, same-run host comparison, existing 1300ms release bound. No cross-host pooling, unchanged reruns, threshold changes or new uncertainty waiver.

    Stage one: large-class/8, large-class-ephemeral/8, and short-generation chart ephemeral/8, one original cohort each. Proceed to the remaining ten rows (all12 defaults plus sparse64-slot diagnostic total13) only with supported target CPU/throughput gain and no confirmed FAIL, incomplete data, sanity or service failure. Retain every outcome and all exact binaries/raw samples. Missing/uncertain acceptance remains unresolved; not a pass.

    Benchmark prerequisite #615 is complete through merged PR #617 and full published run39815683542. PR #616 remains draft pending the full acceptance evidence.

  6. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    V2 platform failure and test correction

    Original ASan run 39831434073, owner-gate Debug lane, failed test-event-counters: _mi_page_free_collect asserted _mi_gate_held_theap. The new test called this internal function without acquiring the owner gate. This is a test defect and a real failed platform gate; it is not marked passing or retried unchanged.

    The failure reproduces locally. The correction holds the owner gate across internal page access/sweeping, including both clean and dirty tests, and releases it before deleting the heap. The focused test passes with Debug/full + owner gate + actual ASan enabled (__asan_init verified), and with the gate disabled. An initial local setup lacked the ASan header and compiled sanitizer support out; its RED/GREEN results establish gate coverage only. The later sanitized configuration explicitly enables it.

    Only the test changes. Allocator sources, Rust amalgamation, build flags and performance harness remain the measured V2 implementation, so the three original live measurement cohorts are retained without duplicates. Original platform failure logs are preserved. Full platform validation of the corrected tests is still required before merge, and performance acceptance remains pending. PR #616 stays draft.

  7. 33 remaining items

  8. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    V6 rejected: no supported CPU/throughput gain

    Tracks #614. Benchmark prerequisite #615 was delivered separately in merged PR #617. The CPU improvement objective remains incomplete; this draft is not ready to merge.

    Measured candidate a40d8d6e717df31859103a034bf349d32e7760fc; fixed baseline 227eef06ff03f92966f38508441d6c2f60a75135.

    V6 removed a redundant final free-list link store. Its focused RED/GREEN test and all ten platform workflows passed, but the three original target cohorts produced 14 PASS, 4 INCONCLUSIVE, zero FAIL and no statistically supported target CPU/throughput gain. All release bounds held; zero memory credit. No stage two, unchanged rerun, threshold change or waiver. The candidate was removed in C commit d9dc4e1a and separately regenerated Rust commit 31c92eea.

    The plain ephemeral cohort retained inconclusive CPU, faults, p99 and drain RSS results, including the broad drain interval; these are not equivalence claims. A green workflow is not performance acceptance.

    Evidence under .github/assets/perf-614/: v6-structural.zip, v6-measurements.zip, v6-platform.zip, their external digests/member manifests, exact source snapshot, actual measurement executables, raw samples, run metadata/logs, source manifest and ledger-v6.md. Native MSVC, Windows GNU, native Mac x64/ARM64, Rust, default fast-path identity, memory and sanitizer coverage passed. The minimal-only memory job is skipped in the full lane; full-lane memory coverage is retained.

    V1/V2/V4 also lacked supported CPU/throughput gains; V3 had confirmed regressions; V5 was rejected before measurement for a reentrant collection risk. All rejected production changes are removed; evidence remains preserved.

    V6 original stage-one ledger

    Baseline 227eef06ff03f92966f38508441d6c2f60a75135; candidate a40d8d6e717df31859103a034bf349d32e7760fc.

    Rejected: no statistically supported target CPU/throughput gain. 14 PASS, 4 INCONCLUSIVE, zero FAIL. No stage two or unchanged rerun.

    Original per-host paired cohorts only; zero memory credit. All ten platform workflows passed. Platform success does not establish performance acceptance.

    large-class/8

    Run 39855326807; AMD EPYC 7763 64-Core Processor; samples per arm: {'base': 80, 'head': 80, 'null': 80}.

    Metric Change % [95% CI] Allowance % Verdict
    ops/s +0.8402 [-0.0645, +1.4413] 0.6799 PASS
    cpu s +0.0272 [-0.2649, +0.3017] 0.4542 PASS
    minor faults -0.4357 [-2.6087, +2.2026] 2.6432 PASS
    p99 operation ns -0.5771 [-1.1513, +0.5482] 1.6103 PASS
    peak RSS MiB +0.0131 [-0.0578, +0.0684] 0.0893 PASS
    RSS 0.5 s after drain MiB -0.1289 [-1.9111, +1.5691] 2.0848 PASS

    Release p95: 1040.0 ms; bound: 1300 ms; held: True.

    large-class-ephemeral/8

    Run 39855330463; AMD EPYC 7763 64-Core Processor; samples per arm: {'base': 120, 'head': 120, 'null': 120}.

    Metric Change % [95% CI] Allowance % Verdict
    ops/s -0.1958 [-0.4755, +0.1010] 0.6521 PASS
    cpu s +0.1477 [+0.0069, +0.3486] 0.2342 INCONCLUSIVE
    minor faults +0.3929 [-0.3802, +0.7905] 0.3968 INCONCLUSIVE
    p99 operation ns +0.5028 [+0.0000, +1.0616] 1.0037 INCONCLUSIVE
    peak RSS MiB +0.0230 [-0.0210, +0.0839] 0.1020 PASS
    RSS 0.5 s after drain MiB +0.0000 [-0.7731, +10.9190] 0.8235 INCONCLUSIVE

    Release p95: 1050.0 ms; bound: 1300 ms; held: True.

    large-class-ephemeral/8 short generations (chart build) short generations (chart build)

    Run 39855334782; AMD EPYC 7763 64-Core Processor; samples per arm: {'base': 110, 'head': 110, 'null': 110}.

    Metric Change % [95% CI] Allowance % Verdict
    ops/s -0.1823 [-0.5360, +0.2481] 0.5711 PASS
    cpu s +0.1091 [-0.1198, +0.2890] 0.2946 PASS
    minor faults +0.0000 [-0.3861, +0.7605] 1.1407 PASS
    p99 operation ns -0.0517 [-0.9932, +0.5873] 1.0965 PASS
    peak RSS MiB -2.5124 [-2.6753, +0.0000] 2.5489 PASS
    RSS 0.5 s after drain MiB -2.5124 [-2.6753, +0.0000] 2.5489 PASS

    Release p95: 1010.0 ms; bound: 1300 ms; held: True.

  9. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    V7 preflight update (not measured): specialization of the selected retired large singleton removal has a focused RED/GREEN work-count test and clean implementation review. The C and secure C++ large-span tests pass, as does the actual ASan + owner-gate large-span test.

    Additional validation exposed two results that are not waived:

    1. Secure C++ Debug (MI_SECURE=5, MI_DEBUG=3) cannot compile unchanged test/test-purge-holes.c: gotos to done at line 1939 cross initialized locals in test_owner_sweep_pacing. Reproduced with identical compiler flags against frozen baseline 227eef06ff03f92966f38508441d6c2f60a75135; the test source is byte-identical. Exact command and diagnostics are retained locally.
    2. The candidate ASan + owner-gate purge suite passed 34 cases but failed new-thread-does-not-sweep-at-once at line 2313: 6 discards observed, expected 0. The reuse test passed. A matching frozen-baseline build is in progress to attribute this failure; no unchanged candidate rerun is being used to erase it.

    Under rule 7, optimization implementation/measurement is stopped pending validation attribution and disposition. No performance workflow has been dispatched and no gain is claimed. Actual executables, logs and source evidence are retained.

  10. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    Follow-up: the matching frozen-baseline ASan + owner-gate purge run passed. Thus the candidate pacing failure remains unresolved; it is not classified as preexisting or waived. Both actual executables and logs are retained. The secure C++ compile failure is separately reproduced on the frozen baseline. V7 remains uncommitted and no performance measurements are dispatched.

  11. zackees commented on Oct 22, 2026

    @zackees
    OwnerAuthor

    V7 validation attribution

    Candidate purge test: 34 pass, 1 pacing failure (6 discards vs 0). Baseline matched build passes. No unchanged candidate rerun.

    Additional source/log evidence: MI_DIAGNOSTICS=1 in actual candidate compiler flags. Event counter implementation is unsampled monotonic atomic counts; print omits zero values. No test event resets and no production reset callers. Both final candidate and baseline event lines omit large_repurpose, meaning zero successful repurposes in those processes. MI_EVENT_LARGE_REPURPOSE occurs before the specialized removal call. Thus candidate specialization was not executed by this failing suite; this does not prove a test-only bug or waive the pacing assertion. Generic removal source is unchanged except extraction and a macro resolving to the exact original setter in production.

    The test computes global discard-count change around thread C. Other threads including the scavenger remain enabled; it does not attribute the discards to C. This is a potential attribution defect, not yet proven root cause. A diagnostic should identify the discarding thread and reason before changing the assertion or allocator.

    Secure C++ compile error is independently reproduced against frozen baseline identical purge test source and flags. A narrow compile repair would move initialized locals before reachable goto edges without changing checks.

    Rule7 stop on optimization work and measurement remains in effect. Asked owner whether to expand work order to diagnose/fix validation blockers. No new approval inferred from elapsed time.

  12. zackees commented on Oct 23, 2026

    @zackees
    OwnerAuthor

    Published the current V7 implementation, separately regenerated Rust source, and hashed validation evidence to draft PR #616 at 6155a79f6701. The local pre-push gate passed 45/45. New issue #618 tracks the secure C++ compile error and unresolved purge-pacing failure, with exact reproduction evidence and RED/GREEN acceptance criteria. V7 remains unmeasured and unaccepted; this is the requested draft handoff, not completion of #614.

  13. zackees commented on Oct 23, 2026

    @zackees
    OwnerAuthor

    Validation blockers (#618) resolved on #616 head f043323 — see #618 for attribution + RED→GREEN. Both were pre-existing on baseline 227eef0, not V7. CPU measurement may resume; no gain claimed yet.

  14. zackees commented on Oct 23, 2026

    @zackees
    OwnerAuthor

    perf-ab status for V7 (head f043323 vs frozen base 227eef0):

    • Run 39896572843 aborted at the first-rep sanity gate on larson/8 after-drain RSS (base 48.3 → head 208.5 MiB). In the same repetition the null arm (a second base build) read 208.0, so the gate tripped on base noise. Filed perf-ab sanity gate ignores the null arm: base-noise drain spikes trip it #620; reran with --no-sanity-gate. The per-row ledger still judges drain RSS against the null-arm noise floor.
    • Run 39897074404 hit the 90-min job limit at rep 25 (all 12 rows had 20–25 reps). The checkpoint is complete: false, so per protocol no verdict is drawn from it. It is kept as evidence.
    • Now running the same parameters as two exact-name groups of 6 rows each (dev-loop.md's remedy for long adaptive groups). The ledger will follow.
  15. zackees commented on Oct 23, 2026

    @zackees
    OwnerAuthor

    V7 verdict: REJECTED (rule 12)

    Complete perf-ab runs comparing head f0433230 with frozen base 227eef06, 15 reps (up to 25), with a null arm: group 1 39905247635 on AMD EPYC 7763, group 2 39909998440 on Xeon 8370C. Rows were judged only within their own VM; nothing was averaged across hosts.

    • Confirmed FAIL: large-class-ephemeral/8 CPU/op +0.7% [+0.4, +1.0] against a 0.3% allowance, with zero memory credit.
    • Target cells lean toward regression: large-class/8 CPU +0.8% [+0.3, +1.2]; random-large/8 ops/s −1.9% [−2.5, −1.0] and CPU +0.8% [+0.5, +1.5].
    • No cell shows a statistically supported CPU or throughput gain. Per perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614, no rerun was made to turn the FAIL green.

    Earlier attempts, kept as evidence: 39896572843 (sanity gate tripped on base noise, #620) and 39897074404 (90-min timeout, incomplete; diagnosed and fixed in #621).

    Disposition: V7 joins V1–V6 as rejected. The #618 validation fixes do not depend on V7 and continue separately in #622. Closing this draft.

    Full ledgers

    perf-ab run 39905247635: https://github.com/zackees/mimalloc-pprof/actions/runs/39905247635
    base 227eef06ff03 head f04332301b27 on AMD EPYC 7763 64-Core Processor, 15 reps (up to 25); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

    Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

    workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
    large-class/8 0% INCONCLUSIVE -1.3% [-2.0, +1.0] / 1.2% INCONCLUSIVE +0.8% [+0.3, +1.2] / 0.6% INCONCLUSIVE -5.5% [-34.2, -0.9] / 32.2% PASS -1.2% [-3.2, +0.5] / 3.3% PASS +0.0% [-2.5, +2.7] / 2.7% PASS +1.7% [-0.8, +3.2] / 2.3% INCONCLUSIVE
    large-class-ephemeral/8 0% FAIL -0.1% [-0.9, +0.6] / 0.6% INCONCLUSIVE +0.7% [+0.4, +1.0] / 0.3% FAIL +0.4% [-0.4, +1.6] / 1.1% INCONCLUSIVE +0.0% [-2.1, +0.6] / 1.1% PASS -0.0% [-2.6, +2.5] / 2.7% PASS -0.7% [-11.6, +14.1] / 14.1% INCONCLUSIVE
    random-large/8 0% INCONCLUSIVE -1.9% [-2.5, -1.0] / 1.7% INCONCLUSIVE +0.8% [+0.5, +1.5] / 0.8% INCONCLUSIVE -0.7% [-1.8, +0.7] / 1.8% PASS -0.4% [-1.5, +2.0] / 3.5% PASS -1.1% [-2.4, +2.2] / 3.3% PASS -1.1% [-2.6, +2.4] / 3.3% PASS
    random-large-bursty/8 0% INCONCLUSIVE +0.0% [-0.2, +0.2] / 0.3% PASS +0.5% [+0.1, +1.0] / 0.7% INCONCLUSIVE +17.9% [-42.2, +71.2] / 44.7% INCONCLUSIVE -1.0% [-1.4, +0.3] / 1.0% PASS +0.0% [-4.9, +3.7] / 5.0% PASS -5.2% [-9.0, -2.3] / 8.5% PASS
    random-large/1 0% PASS -0.5% [-0.9, +0.2] / 1.5% PASS +0.5% [-1.8, +2.0] / 3.5% PASS +0.0% [-0.7, +0.7] / 0.7% PASS -0.3% [-0.9, +0.9] / 1.2% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [+0.0, +0.0] / 0.0% PASS
    small/8 (control) 0% INCONCLUSIVE -1.6% [-3.5, +0.7] / 4.5% PASS +2.5% [-2.7, +4.5] / 3.9% INCONCLUSIVE -0.6% [-1.2, +0.6] / 1.2% PASS +0.0% [+0.0, +0.0] / 0.8% PASS +0.0% [-0.0, +0.0] / 0.0% PASS +0.0% [-0.0, +0.0] / 0.0% PASS

    perf-ab run 39909998440: https://github.com/zackees/mimalloc-pprof/actions/runs/39909998440
    base 227eef06ff03 head f04332301b27 on Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz, 15 reps (up to 25); 4 logical CPUs, 8-worker cells have 2.00 workers per logical CPU (above 1 means oversubscription); arm order: rotating-reversed-blocks-v1

    Rule-12 ledger (CLAUDE.md; ci/perf_ab_ledger.py): saved% of the reducible peak-RSS gap, then per metric the paired change [95% sign-test interval] against its allowance (saved%/3 for CPU, throughput and faults; noise for memory and for cells that saved none). INCONCLUSIVE is not a pass.

    workload saved% verdict ops/s cpu s minor faults p99 operation ns peak RSS MiB RSS 0.5 s after drain MiB
    larson/1 0% INCONCLUSIVE -3.4% [-4.6, -1.2] / 4.2% INCONCLUSIVE +3.5% [+1.1, +5.8] / 4.9% INCONCLUSIVE +0.0% [-0.9, +1.0] / 0.5% INCONCLUSIVE +0.1% [-1.4, +2.8] / 1.8% INCONCLUSIVE -1.3% [-1.4, +0.0] / 1.3% PASS +0.0% [+0.0, +0.0] / 0.0% PASS
    larson/8 0% PASS -1.3% [-2.9, +3.4] / 3.3% PASS +1.4% [-1.8, +3.9] / 4.6% PASS -15.4% [-29.5, +40.1] / 74.9% PASS -2.7% [-6.0, -0.7] / 3.2% PASS +0.7% [-1.3, +4.6] / 4.7% PASS +0.7% [-1.4, +2.1] / 2.4% PASS
    larson/8 (chart build) 0% PASS -0.3% [-2.6, +3.2] / 3.5% PASS +0.4% [-4.1, +2.2] / 5.0% PASS +14.6% [-10.5, +64.5] / 79.0% PASS +0.0% [-2.0, +2.1] / 4.7% PASS -1.5% [-6.6, +4.9] / 5.2% PASS +1.0% [-1.0, +340.9] / 342.2% PASS
    large-class/8 (profiler on) 0% PASS +0.2% [-0.6, +2.7] / 1.4% PASS -0.5% [-2.4, +0.7] / 1.1% PASS +0.0% [-6.8, +1.0] / 1.7% PASS -1.0% [-4.4, +1.4] / 4.3% PASS -0.7% [-1.5, +1.3] / 2.0% PASS -0.3% [-2.0, +1.4] / 2.0% PASS
    large-class-ephemeral/8 (chart build) 0% INCONCLUSIVE +0.3% [-0.6, +0.8] / 0.7% PASS -0.3% [-0.9, +0.5] / 0.5% INCONCLUSIVE +0.0% [-1.1, +1.1] / 1.5% PASS -0.2% [-1.8, +1.8] / 1.8% INCONCLUSIVE +0.0% [-2.3, +2.6] / 2.6% INCONCLUSIVE -9.8% [-11.2, -0.0] / 11.2% PASS
    large-class-ephemeral/8 short generations (chart build) 0% INCONCLUSIVE +0.9% [-0.7, +2.2] / 1.2% PASS -0.9% [-1.3, +0.3] / 1.2% PASS -0.4% [-1.1, +0.8] / 2.2% PASS +0.2% [-1.1, +2.7] / 2.1% INCONCLUSIVE +0.0% [-2.7, +2.5] / 2.7% PASS +0.0% [-2.7, +2.5] / 2.7% PASS
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions