mi_heap_delete: detach the theaps before the pages are moved and free them after

[robobun]

Stacked on #24 (its two commits come first, merge it first). This PR is the last two commits.

Problem

• mi_heap_delete frees the theaps of the heap before it walks the pages (heap.c:236), and the detach marks a theap with theap->tld = NULL (theap.c:672). A thread that used the heap before still gets that theap from the heap's thread local while it frees blocks during the delete (_mi_page_associated_theap_peek, from free.c:369 and arena.c:1269), and then reads theap->tld, or a freed theap.
• A new test-heap-mt variant (pages abandoned by an exited allocator thread, freers that allocated from the heap once) segfaults in its first iteration on Do not dereference page->heap on the cross-thread free path while mi_heap_delete can free the heap #24: mi_theap_matches_thread (internal.h:714) <- _mi_page_associated_theap_peek <- _mi_arenas_page_try_reabandon_to_mapped <- mi_free. Found by the self-review of Do not dereference page->heap on the cross-thread free path while mi_heap_delete can free the heap #24.

Fix

• The detach clears theap->heap instead (under the tld lock, after the unlink), as _mi_tld_detach_theaps does on thread exit. That is the field the peek compares, so a free that runs after the detach gets NULL and leaves the page abandoned.
• mi_heap_delete and mi_heap_destroy detach (and merge the statistics, as before), then walk the pages, and free the theap structs last. A free that got the theap just before the detach still owns its page and the walk waits for it, so the struct and its tld stay valid for as long as that free can use them.
• _mi_heap_theap_peek returns NULL for a detached theap instead of asserting. The statistics of a deleted or destroyed heap do not change.
• Verified: the variant crashes 3 of 3 runs on Do not dereference page->heap on the cross-thread free path while mi_heap_delete can free the heap #24 and passes 3 of 3 here, plus 18 loaded runs, TSAN and ASAN. Full ctest in Debug and Release. The second commit fixes an arming race in test-fork-user-heap seen during that (notes).

Background

• A theap is a thread's view of a heap (its page queues). Each heap has a thread local slot, heap->theap, through which a thread finds its theap for the heap. A cross-thread free uses it to reclaim an abandoned page or to put a page back on the abandoned map.
• A theap is on two lists: the thread's (tld->theaps) and the heap's (heap->theaps). The detach takes it off the thread's list under the thread's lock. The delete frees the heap's list.
• theap->heap == NULL already means "not usable" here (mi_theap_is_initialized).

Notes

Fail before, on the #24 tree with only the test change (Debug, MI_DEBUG_FULL): 3 of 3 runs segfault right after the variant starts, with reclaim on free disabled as in the committed test:

#0 mi_theap_matches_thread                 internal.h:714   (theap->tld is NULL)
#1 _mi_page_associated_theap_peek          prim-tls.h:422
#2 _mi_arenas_page_try_reabandon_to_mapped arena.c:1276
#3 mi_abandoned_page_try_reabandon_to_mapped free.c:312
#4 mi_abandoned_page_unown_from_free       free.c:332
#5 mi_free_try_collect_mt                  free.c:436

With reclaim on free at its default the same run crashes through mi_abandoned_page_try_reclaim (free.c:369) instead. The committed test disables reclaim on free (and restores it afterwards) on purpose: a freer that reclaims a page holds it like an allocating thread, and the delete moves a held page without waiting for its thread, so a holder must not free into its pages during the delete. That is the existing contract (issue 3 in the notes of #24) and this PR does not change it. theap->tld is NULL rather than freed in the crash because the thread's cached theap slot holds a reference (_mi_theap_cached_set); a thread whose cache moved on to another heap reads a freed theap instead.

Pass after: ./mimalloc-test-heap-mt 500, 3 of 3 runs, 4 variants. Loaded: 6 copies pinned to 2 CPUs, 500 iterations, 3 rounds, 0 crashes in 18 runs. TSAN (./mimalloc-test-heap-mt 30): 2 usable runs, 0 reports. ASAN (MI_TRACK_ASAN=ON): test-heap-mt 100 and test-heap-delete-race clean. test-heap-delete-race (the delete against a thread that is still exiting, deliberately outside the contract) passes in Debug, Release, TSAN (no crash, same kind of stats reports as on the unfixed tree, 34 vs 43) and ASAN. The new variant also exercises the second site of #24 with every page abandoned, in Debug and Release.

ctest Debug: 20 of 21 pass (test-purge-holes unformed-tail, same as on the unfixed tree, see #24). Release: 20 of 20. During one Debug run under ctest -j4, test-fork-user-heap hit the 1500 s timeout in case_b before it forked: the stall hook is armed after pthread_create, and the thread had already exited (the log stops after the case_b banner, the alarm only covers the child). It does not involve the code changed here (case_a had just run 200 deletes in forked children). The second commit arms the hook before the thread exists. 18 further runs of the test, 12 of them with 2 CPUs for 12 copies, passed with and without that commit, so it is a rare race of the test.

Ordering argument for the free-after-walk: a free can only obtain a theap of the heap through heap->theap while it owns a page whose page->heap is still this heap. The walk moves every such page, and for an abandoned page it waits until the owner releases it (mi_heap_visit_page_at). When the walk returns, no free holds a page of the heap, so nothing can still be using the theaps, and nothing can obtain them anymore because the detach happened before the walk. The exception is an OS allocated page that a freer owns: the walk skips it (issue 2 in the notes of #24), so that freer can still use the theap after the walk. It also uses the freed heap itself on that path, so OS pages are not made worse or better by this PR, they need the design of their own noted in #24. The exiting-thread protocol is unchanged: _mi_heap_detach_theaps still takes the thread's lock, and mi_thread_theaps_done walks its list under that lock, so a theap is either still in that list with its heap set, or already unlinked.

Two things the new window (detached, not yet freed) does not cover, on purpose. A freer that obtained the theap just before the detach and then bumps a counter on it (pages_reabandon_full, pages_abandoned) does so after the statistics were merged, so those few increments are not carried into the heap. Before this PR that freer crashed. Merging a second time after the walk would make them exact at the cost of a second pass over the 4 KiB stats struct on every destroy (bun destroys a heap per transpile), so I left it. A thread that allocates from the heap while another thread deletes it is outside the contract: _mi_heap_theap then still hands out the detached theap through the heap's slot, as it handed out the freed one before.

The destroy path, which bun runs on every arena reset, does the same work as before in a different order, plus one uncontended acquire of heap->theaps_lock. The destroying thread's cached theap survives mi_heap_free_theaps on the unfixed tree as well (the cache holds a reference) and is evicted by the next allocation from another heap, which mi_heap_new always is. I checked that with a destroy-then-new probe on both trees: the new heap got the old address 1000 of 1000 times and allocations from it went to a fresh theap on both.

src/static.c compiles as C++ with clang 21 without warnings in the debug and release configurations bun uses.

[robobun]

#26 is now stacked on this PR and adds only a regression test (test/test-heap-aba.c) for the theap cache ABA that the theap->heap = NULL store in _mi_heap_detach_theaps closes: thread A caches its theap for H, thread B destroys H and creates H2 at the same address, A's mi_heap_malloc(H2) must not be served by the detached theap. On this tree it passes; with the store replaced by the old theap->tld = NULL line it fails (127 blocks that mi_heap_of attributes to no heap in Release, a segfault in Debug). The same store was the whole fix in the first version of #26; that version is withdrawn in favour of this PR.

[Jarred-Sumner]

Folded into #27, which combines #22–#26 and replaces the per-reader fixes with one delete/destroy teardown protocol (pin-before-claim). The commit from this PR is carried there as-is.