fix: resolve purge-test C++ compilation and pacing validation blockers #618
Description
Context
Blocker extracted from #614 and draft PR #616. The CPU candidate V7 is published for review, not accepted or measured. Two validation defects need resolution before that work resumes.
1. Existing secure C++ compilation failure
test/test-purge-holes.c:test_owner_sweep_pacing uses early goto done paths across initialized scalar locals (stamped_at, before_in, after_in, elapsed, before_out, after_out). This is valid C but rejected when CMake intentionally sets this .c source to LANGUAGE CXX under MI_USE_CXX. The test calls internal allocator functions with C++ linkage in that configuration, so simply forcing this test to C is not a demonstrated solution.
Reproduced with MI_SECURE=5, MI_DEBUG=3, C++17 against byte-identical test source on frozen baseline 227eef06ff03f92966f38508441d6c2f60a75135. Diagnostics report jumps to done at line 1939 crossing initializations. A likely minimal repair is an inner scope for measurement locals, with cleanup outside; preserve the timing of clock/counter reads and all checks.
2. Unresolved purge-pacing assertion
Actual ASan + owner-gate Debug candidate run: 34 cases passed; new-thread-does-not-sweep-at-once failed at line 2313 with 6 discards, expected 0. The matching baseline build passed with 0 discards. Do not call this a baseline failure or an accepted flaky test.
The test measures a process-wide discard counter around thread C. Background scavenger work remains possible, so the delta is not necessarily attributable to C. This is a hypothesis needing thread/reason attribution. Diagnostics were enabled and unsampled; both runs recorded zero successful large repurposes, proving V7's specialized removal did not execute in the failing suite. That narrows the investigation without waiving the assertion or proving a test-only bug.
Evidence / reproduction
Draft PR #616 contains the evidence archive, digest and handoff. Archive SHA256: c1a9c4ff57f1694031ed38aeee4b1e1dc9f9f05802fe925b76afe642d06465e1 (46 members; 4,864,167 bytes).
The archive preserves actual candidate and baseline executables, CMake caches/flags, commands, failed and passing logs, current source patch/new headers, and per-member hashes. Relevant members include v7-baseline-cxx-command.json, v7-baseline-cxx-error.log, v7-cxx-build.log, v7-asan-tests.log, and baseline/Testing/Temporary/LastTest.log. Original local paths in command JSON must be adapted to the checkout location.
C candidate commit 2ec4d32e; generated Rust C 28ef6ad7; fixed baseline above. The focused large-span test passes in C, secure C++ and actual ASan + owner-gate builds. No V7 performance run has been dispatched.
Proposal
- Fix C++ scope compatibility without moving measurement calls earlier, removing const unnecessarily, changing cleanup, or weakening checks.
- Attribute each discard observed by the pacing scenario to its thread and trigger. Establish a focused, reproducible RED signal for the real defect.
- Fix the test if accounting is polluted, or allocator logic if pacing is genuinely wrong. Preserve separate coverage for background release and owner-side pacing; do not disable required behavior merely to make the test pass.
- Record the disposition on perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614 and perf: large-page reuse CPU investigation and draft V7 (#614) #616 before resuming CPU measurement.
Acceptance criteria
- Original C++ compiler repro is RED; repaired test compiles and runs in C and secure C++ with the same checks and read timing.
- Pacing failure has concrete thread/trigger attribution and focused RED → GREEN evidence; an unchanged passing rerun is not a fix.
- Original candidate failure and matching baseline pass remain in the record; no threshold relaxation, swallowed failure or waived assertion.
- Validate affected ASan/owner-gate, ordinary C/C++, native MSVC and Windows GNU configurations, plus relevant scavenger/release tests.
- Preserve C/Rust source separation and regenerate Rust if allocator C changes; pass the required local gate before push.
- Link the fix/evidence and unblock decision on perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614 and draft PR perf: large-page reuse CPU investigation and draft V7 (#614) #616. This issue does not claim completion of the CPU improvement.
Decisions
- Priority P2; this is a validation blocker, not a demonstrated V7 memory-safety defect.
- Keep this issue focused on the two purge-test blockers; CPU acceptance remains perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614.
- No new workflow files, unrelated policy changes, or relaxation of perf acceptance.
- The current publication is a handoff, not authorization to merge the CPU candidate.
Open questions
Which thread and purge trigger produced the six observed discards? Does the deterministic fix belong in test attribution or allocator pacing?
Related
- perf: reduce large-page reuse CPU after #611 while preserving RSS and p99 #614 / draft PR perf: large-page reuse CPU investigation and draft V7 (#614) #616: blocked CPU optimization and complete evidence.
- feat(benchmarks): short-lived-thread large-block workload on the README scaling charts — show the deficit, then verify #477 #478: original short-lived-thread pacing scenario referenced by the test.
Activity
fix: resolve purge-test C++ compile and pacing blockers (#618)
build: regenerate vendor C for #618 fixes
zackees commentedon Oct 23, 2026
#618 disposition — fixed on #616 head
f0433230(Cbd05be78, Rust vendor75e4afc1)1. Secure C++ compile. RED: frozen baseline
227eef06withMI_USE_CXX=ON,MI_SECURE=5 MI_DEBUG=3, C++17 fails with jump to label 'done' crosses initialization (line 1939). Fix: the measurement section oftest_owner_sweep_pacingsits in an inner scope, so the earlygoto doneskips it whole. Clock and counter reads keep their order, every check is unchanged, and cleanup is unchanged. GREEN: compiles.1b. Allocator defect found once the file compiled as C++ (also on baseline). The secure-C++ suite segfaulted every time (3/3 runs on baseline and candidate) in
idle-thread-releases-large-page. Backtrace: scavenger →_mi_pages_release_retired→mi_page_discard_slack→_mi_os_discard→ eager-zeromemseton the MI_SECURE≥5 trailing guard page (PROT_NONE). The slack range ran to the end of the slices and included the guard page. Fix (src/page-holes.c): subtract_mi_os_secure_guard_page_size()fromhi, matchingpage_noguard_sizeinarena.c. GREEN: secure C++ purge-holes 32/32, 3/3 runs.2. Pacing assertion: attributed to the background scavenger, not to thread C.
MIMALLOC_SCAVENGER=0--new-thread-pacing, own process, scavenger off)The test reads a process-wide counter, so the scavenger's sweep of thread A's abandoned page during C's life was charged to C. The defect predates V7: the baseline fails more often. V7's path was not exercised (0 repurposes). #478 is about the owner thread's first tick, so the case now runs in its own process with no scanner (
test-new-thread-pacing,MIMALLOC_SCAVENGER=0;MIMALLOC_PURGE_HOLES=1). That mirrorstest-reserved-page-reuse. The background-release cases stay in the default invocation with the scavenger on. No threshold changed. The original candidate failure and the baseline pass from the evidence archive stay on record. The baseline passing once was luck under low load.Also fixed: PR CI
build (release)and the other Release rows failed with-Werror=unused-parameteronpageinmi_page_queue_remove_retired, because the asserts compile out. AddedMI_UNUSED(page).Local validation: owner-gate Debug (
MI_DEBUG_FULL+MI_DIAGNOSTICS+MI_PPROF; no ASan headers on this host, CI's asan row covers ASan) 86/86 passed. ReleaseMI_PPROF70/70 passed. Secure C++: purge-holes passes. Local gate 45/45.Not fixed by this change:
test-fork-locks*fails to link on the baseline too. Filed as test-fork-locks fails to link under MI_USE_CXX (C++ linkage of _mi_test_* hooks) #619.test-no-process-detach-skippedfailure under heavy concurrent host load: 30/30 passes on both baseline and fix when rerun. Untouched by this diff; recorded, not waived.Native MSVC and win-gnu run in PR CI.
Unblock decision: #618's validation blockers are resolved. The V7 CPU measurement under #614 may resume. No CPU, throughput or RSS gain is claimed. #616 stays a draft until #614's perf-ab campaign and platform gates pass.
fix: register test-new-thread-pacing only in static builds (#618)
zackees commentedon Oct 23, 2026
Fixes split out of rejected V7 #616 and onto main in #622 (closes this issue on merge). #616's verdict: V7 rejected (confirmed CPU FAIL on large-class-ephemeral/8).
Merge pull request #622 from zackees/fix/618-purge-test-blockers
fix: purge-holes secure guard page, C++ test scope, and scavenger-pol…
build: regenerate vendor C for zackees#618 fixes