Snapshot support on top of the upstream sync - #16
Open
Jarred-Sumner wants to merge 6 commits into
Open
Conversation
…live samples; size the location table to the sample count The thread that held the profiler lock in the parent (or in the process that built a snapshot) does not exist in the child, so the lock is re-initialized there. mi_prof_visit_live() lets an embedder walk the live sampled allocations, and mi_prof_lock_is_free() lets it assert the profiler is quiescent before freezing memory. The location table used while aggregating grows with the sample count instead of being fixed at 4096 entries.
An embedder can freeze the current heap into a snapshot and map it into later processes. This adds what the allocator needs for that: pages captured in a snapshot get the thread id MI_THREADID_FROZEN, so frees of their blocks are dropped on the (already slow) cross-thread path and collection, purging and the idle hole sweep leave them alone; mi_theap_freeze() marks the frozen theaps; mi_arenas_seal_existing() stops new allocation from landing in frozen arenas; mi_theap_adopt_current_thread() lets the resuming thread take over the frozen main-thread state (otherwise every page it allocates carries a thread id nobody has and all of its frees go cross-thread); mi_heap_snapshot_hints_enabled() exports the deterministic-placement decision, made once, so the embedder's own reservations follow the same rule. Placement is deterministic when the host executable says it can carry a snapshot (MI_HEAP_SNAPSHOT_HOST_FN) or MIMALLOC_DETERMINISTIC_HINT is set; a capable process that is not taking a snapshot keeps its early allocations above MI_HEAP_SNAPSHOT_RESTORER_FLOOR so they never sit where the snapshot will be mapped. The hole-sweep bookkeeping moves from __thread variables onto the tld: on macOS the first touch of a thread-local from inside the allocator makes dyld allocate the TLV block with malloc, which re-enters the allocator before the variable exists. Nothing here runs in a process that neither takes nor maps a snapshot, apart from the frozen-id compare on the cross-thread free path.
The frozen thread-id sentinel keeps its value under the widened flag field and stays distinct from the new detached sentinel (abandoned is now `<= mapped`, detached is an equality, frozen is neither); the frozen check precedes the padding check on the cross-thread free path; the theap and detached-tld initializers carry the snapshot fields; thread adoption is unchanged, since upstream's own thread init re-stamps a tld the same way.
…fitting the buffer (getenv now reports an over-long value as EAGAIN rather than truncating; the marker's value is a path)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The snapshot support from #13, re-applied on top of #15 (the upstream sync). Supersedes #13 once #15 lands; Bun's pin moves to this branch's merge.
<= mapped, detached an equality — frozen is neither, audited); the frozen check precedes upstream's new padding check on the cross-thread free path; the initializers carry the snapshot fieldsgetenvnow reports an over-long value as an error rather than truncating; the check tests presence now (this was latent in #13 as well)