Repository navigation
Conversation
Clearing a range on a session whose storage sits on a dependency truncated the dependency to the start of the range, so every block after the range silently resolved to the session's own (empty) storage and was lost. Copy the blocks after the cleared range into the session's own storage before truncating the dependency. Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
Reproduces holepunchto#775: on a named session, clear(1, 2) also dropped every block after the range (has(s.length - 1) was false, get() never resolved). Asserts the cleared range is cleared, the surrounding blocks survive, and the default session is untouched. Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
clear(start, end) treats end as exclusive (clear(4) clears only block 4), so the blocks after the range start at end, not end + 1. Copy from end so the block right after the range is kept too. Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
The range end is exclusive, so clear(1, 2) clears only block 1: block 2 must survive with its value intact. Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
[Medium impact] The change affects clearing ranges on sessions that use dependent storage. The fix appears safe to merge, with non-blocking improvements for memory use and the new test's failure behavior.
|
| const rx = this.storage.read() | ||
| const blockPromises = [] | ||
| for (let i = end; i < flushedLength; i++) blockPromises.push(rx.getBlock(i)) | ||
| rx.tryFlush() | ||
|
|
||
| const blocks = await Promise.all(blockPromises) | ||
| for (let i = 0; i < blocks.length; i++) { | ||
| if (blocks[i] !== null) tx.putBlock(end + i, blocks[i]) |
There was a problem hiding this comment.
Clearing a small range near the start of a large named session now reads every later index and holds all returned blocks in memory at once. Sparse sessions also create a read and promise for every missing index.
Use createBlockStream() and bounded batches to stage the copies before applying the clear and shortening the dependency.
Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/session-state.js
Line: 629-636
Comment:
**Small clears use huge memory**
Clearing a small range near the start of a large named session now reads every later index and holds all returned blocks in memory at once. Sparse sessions also create a read and promise for every missing index.
Use `createBlockStream()` and bounded batches to stage the copies before applying the clear and shortening the dependency.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| t.alike(await s.get(2), b4a.from('3'), 'block at the end boundary still resolves') | ||
| t.alike(await s.get(4), b4a.from('5'), 'last block still resolves') |
There was a problem hiding this comment.
The new test uses waiting get() calls to check local blocks. If the regression returns, a missing block makes get() wait for replication with no request timeout, and this test has no peers. Pass { wait: false } to these reads so missing blocks produce assertion failures instead of unfinished checks.
| t.alike(await s.get(2), b4a.from('3'), 'block at the end boundary still resolves') | |
| t.alike(await s.get(4), b4a.from('5'), 'last block still resolves') | |
| t.alike(await s.get(2, { wait: false }), b4a.from('3'), 'block at the end boundary still resolves') | |
| t.alike(await s.get(4, { wait: false }), b4a.from('5'), 'last block still resolves') |
Prompt To Fix With AI
This is a comment left during a code review.
Path: test/clear.js
Line: 306-307
Comment:
**Missing blocks stall the test**
The new test uses waiting `get()` calls to check local blocks. If the regression returns, a missing block makes `get()` wait for replication with no request timeout, and this test has no peers. Pass `{ wait: false }` to these reads so missing blocks produce assertion failures instead of unfinished checks.
```suggestion
t.alike(await s.get(2, { wait: false }), b4a.from('3'), 'block at the end boundary still resolves')
t.alike(await s.get(4, { wait: false }), b4a.from('5'), 'last block still resolves')
```
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.|
Closing as duplicate of #775, which already fixes this. My apologies - I mistook that open PR for an issue when scanning the tracker. |
Fixes #775.
Problem
On a session whose storage sits on a dependency (for example a named session),
clear(start, end)truncated the dependency to the start of the cleared range. From that point every block after the range resolved against the session's own storage - which never held those blocks - so they were silently lost:Change
Implements the approach outlined in the issue: before truncating the dependency,
SessionState.clearnow copies the blocks after the cleared range (endup to the session's flushed length - the range end is exclusive) from the dependency's storage into the session's own write batch, so they stay available once the dependency no longer covers them. Blocks that are absent (sparse ranges) are skipped. Default sessions are untouched - they have no dependency to truncate.Test
New test in
test/clear.jsreproducing the issue's example: afterclear(1, 2)on a named session, the blocks inside the range are gone, the blocks before and after it (includingget(4)) still resolve, and the default session is unaffected. The test fails on currentmainexactly as the issue describes (blocks after the range reporthas() === false, andget()of the last block never resolves).Verification:
test/clear.js13/13,test/basic.js658/658,test/batch.jsandtest/atomic.jsall pass;prettier --checkandlunteclean.Co-authored-by: generaltony007 144252170+GeneralTony007@users.noreply.github.com