Repository navigation
perf: optimize snapshot replay lookups - #82
Conversation
codeasier
left a comment
There was a problem hiding this comment.
Cross-model review of 717e3ca...068cb1c (2 commits, 4 files). Two reviews independently agree on Finding 1 (medium); Finding 2 is a lower-severity ordering-invariant concern. The indexed block/segment deletion (detach_block/remove_segment) and the keyed-bisect insertion are otherwise sound. CI is green (provenance, lint/format, Python 3.10-3.13). The review sandbox could only run read-only git commands, so the 567-passed suite was inspected statically, not executed. Test gaps noted inline.
| for idx in range(start_idx - 1, -1, -1): | ||
| segment = segments[idx] | ||
| segment_end = segment.address + segment.total_size | ||
| if segment_end < new_start: |
There was a problem hiding this comment.
Finding 1 (medium): a valid left-adjacent segment can be skipped when cross-stream segments overlap.
The backward scan breaks as soon as any segment ends before new_start. Segment ends are only monotonic in (address, stream) order when same-stream ranges never overlap, but cross-stream segments may overlap by design (see snapshot_lookup._scan_overlapping_segments_for_stream, and this PR's own test_alloc_or_map_segment_merge_finds_neighbors_across_streams).
Repro: stream-0 [0x1000, 0x1100), stream-1 [0x1000, 0x1080) (same start, shorter size), then a new stream-0 segment at [0x1100, ...). bisect_left(0x1100) = 2, the loop visits the stream-1 segment first, its end 0x1080 < 0x1100 triggers the break, and the valid stream-0 neighbor at 0x1000 is never inspected. The replaced full scan skipped non-matching streams and found it, so replay state now diverges (fragmented instead of merged); a later free/unmap resolving into that range can fail the exact-size check in free_segment or the corrected_right_idx guard in merge_mapped_segment. Note the outcome is also ordering-dependent on which stream sorts first.
Suggested fix: only break on the same-stream monotonicity that actually holds (e.g. break when segment.stream == stream and segment_end < new_start), or track a running max end while scanning down. Add a regression test with a shorter, same-address, different-stream left neighbor (the current multi-stream test uses equal sizes, so it never hits the early break).
| new_end = new_start + new_segment.total_size | ||
| stream = new_segment.stream | ||
|
|
||
| start_idx = bisect.bisect_left(segments, new_start, key=_segment_address_key) |
There was a problem hiding this comment.
Finding 2 (low): this bisect-by-address assumes snapshot.segments stays address-sorted, which a left shrink can violate.
_shrink_segment(snapshot, ..., "left") in snapshot_mutator.py mutates segment.address = new_start in place without re-sorting the list. With overlapping cross-stream segments the list can become address-unsorted, and this new binary search then skips neighbors that the replaced linear scan still found. This is a latent dependency the PR adds rather than a new defect (_shrink_segment is unchanged), but the docstring's claim that "segment starts are sorted" is not enforced anywhere.
Suggested fix: re-insert/re-sort after a left shrink, or make the neighbor lookup ordering-insensitive (e.g. an endpoint index). A shrink-then-map regression test would pin the behavior.
Summary
The implementation uses the standard library and existing lookup indices to keep the dependency and behavior surface small.
Verification
PYTHONPATH=src python -m pytest: 567 passed, 1 xfailedruff check .black --checkon changed filesSnapshot provenance decision: updated
Closes #74