Skip to content

Fix clear() on sessions with dependencies losing later blocks - #944

Closed
hakarrr wants to merge 4 commits into
holepunchto:mainfrom
hakarrr:fix-775-clear-with-dependencies
Closed

hakarrr wants to merge 4 commits into
holepunchto:mainfrom
hakarrr:fix-775-clear-with-dependencies

Conversation

@hakarrr

@hakarrr hakarrr commented Nov 8, 2026 •

Copy link
Copy Markdown

Fixes #775.

Problem

On a session whose storage sits on a dependency (for example a named session), clear(start, end) truncated the dependency to the start of the cleared range. From that point every block after the range resolved against the session's own storage - which never held those blocks - so they were silently lost:

await core.append(['1', '2', '3', '4', '5'])
const s = core.session({ name: 'batch' })
await s.ready()
await s.clear(1, 2)
console.log(await s.has(s.length - 1)) // false, expected true
console.log(await s.get(4)) // never resolves

Change

Implements the approach outlined in the issue: before truncating the dependency, SessionState.clear now copies the blocks after the cleared range (end up to the session's flushed length - the range end is exclusive) from the dependency's storage into the session's own write batch, so they stay available once the dependency no longer covers them. Blocks that are absent (sparse ranges) are skipped. Default sessions are untouched - they have no dependency to truncate.

Test

New test in test/clear.js reproducing the issue's example: after clear(1, 2) on a named session, the blocks inside the range are gone, the blocks before and after it (including get(4)) still resolve, and the default session is unaffected. The test fails on current main exactly as the issue describes (blocks after the range report has() === false, and get() of the last block never resolves).

Verification: test/clear.js 13/13, test/basic.js 658/658, test/batch.js and test/atomic.js all pass; prettier --check and lunte clean.

Co-authored-by: generaltony007 144252170+GeneralTony007@users.noreply.github.com

hakarrr and others added 2 commits November 9, 2026 01:55
Clearing a range on a session whose storage sits on a dependency truncated the dependency to the start of the range, so every block after the range silently resolved to the session's own (empty) storage and was lost. Copy the blocks after the cleared range into the session's own storage before truncating the dependency.

Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
Reproduces holepunchto#775: on a named session, clear(1, 2) also dropped every block after the range (has(s.length - 1) was false, get() never resolved). Asserts the cleared range is cleared, the surrounding blocks survive, and the default session is untouched.

Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
@hakarrr
hakarrr requested a review from a team November 8, 2026 20:26
hakarrr and others added 2 commits November 9, 2026 01:57
clear(start, end) treats end as exclusive (clear(4) clears only block 4), so the blocks after the range start at end, not end + 1. Copy from end so the block right after the range is kept too.

Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
The range end is exclusive, so clear(1, 2) clears only block 1: block 2 must survive with its value intact.

Co-authored-by: generaltony007 <144252170+GeneralTony007@users.noreply.github.com>
@greptile-apps

greptile-apps Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 4/5 Tier: apex

[Medium impact] The change affects clearing ranges on sessions that use dependent storage.

The fix appears safe to merge, with non-blocking improvements for memory use and the new test's failure behavior.

Fix All in Claude CodeFindings

  1. P2 Small clears use huge memory ▶
  2. P2 Missing blocks stall the test ▶
Fix with agent prompt
### Issue 1
lib/session-state.js:629-636
Clearing a small range near the start of a large named session now reads every later index and holds all returned blocks in memory at once. Sparse sessions also create a read and promise for every missing index.

Use `createBlockStream()` and bounded batches to stage the copies before applying the clear and shortening the dependency.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 2
test/clear.js:306-307
The new test uses waiting `get()` calls to check local blocks. If the regression returns, a missing block makes `get()` wait for replication with no request timeout, and this test has no peers. Pass `{ wait: false }` to these reads so missing blocks produce assertion failures instead of unfinished checks.

```suggestion
  t.alike(await s.get(2, { wait: false }), b4a.from('3'), 'block at the end boundary still resolves')
  t.alike(await s.get(4, { wait: false }), b4a.from('5'), 'last block still resolves')
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

SessionState.clear() now copies inherited blocks after the cleared range into the session's own storage before shortening its dependency.

  • Named sessions keep later blocks when a range is cleared.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["Queue removal of the cleared blocks"] --> B{"Non-default session with an inherited tail?"}
  B -->|Yes| C["Read and queue copies of later blocks"]
  B -->|No| D["Flush writes"]
  C --> D
  D --> E["Shorten the dependency when present"]
Loading

Reviews (1) · Last reviewed commit: "Assert the end-boundary block is kept" · Reviewed by Greptile

Comment thread lib/session-state.js
Comment on lines +629 to +636
const rx = this.storage.read()
const blockPromises = []
for (let i = end; i < flushedLength; i++) blockPromises.push(rx.getBlock(i))
rx.tryFlush()

const blocks = await Promise.all(blockPromises)
for (let i = 0; i < blocks.length; i++) {
if (blocks[i] !== null) tx.putBlock(end + i, blocks[i])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Small clears use huge memory

Clearing a small range near the start of a large named session now reads every later index and holds all returned blocks in memory at once. Sparse sessions also create a read and promise for every missing index.

Use createBlockStream() and bounded batches to stage the copies before applying the clear and shortening the dependency.

Prompt To Fix With AI
This is a comment left during a code review.
Path: lib/session-state.js
Line: 629-636

Comment:
**Small clears use huge memory**

Clearing a small range near the start of a large named session now reads every later index and holds all returned blocks in memory at once. Sparse sessions also create a read and promise for every missing index.

Use `createBlockStream()` and bounded batches to stage the copies before applying the clear and shortening the dependency.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code

Comment thread test/clear.js
Comment on lines +306 to +307
t.alike(await s.get(2), b4a.from('3'), 'block at the end boundary still resolves')
t.alike(await s.get(4), b4a.from('5'), 'last block still resolves')

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Missing blocks stall the test

The new test uses waiting get() calls to check local blocks. If the regression returns, a missing block makes get() wait for replication with no request timeout, and this test has no peers. Pass { wait: false } to these reads so missing blocks produce assertion failures instead of unfinished checks.

Suggested change
t.alike(await s.get(2), b4a.from('3'), 'block at the end boundary still resolves')
t.alike(await s.get(4), b4a.from('5'), 'last block still resolves')
t.alike(await s.get(2, { wait: false }), b4a.from('3'), 'block at the end boundary still resolves')
t.alike(await s.get(4, { wait: false }), b4a.from('5'), 'last block still resolves')
Prompt To Fix With AI
This is a comment left during a code review.
Path: test/clear.js
Line: 306-307

Comment:
**Missing blocks stall the test**

The new test uses waiting `get()` calls to check local blocks. If the regression returns, a missing block makes `get()` wait for replication with no request timeout, and this test has no peers. Pass `{ wait: false }` to these reads so missing blocks produce assertion failures instead of unfinished checks.

```suggestion
  t.alike(await s.get(2, { wait: false }), b4a.from('3'), 'block at the end boundary still resolves')
  t.alike(await s.get(4, { wait: false }), b4a.from('5'), 'last block still resolves')
```

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Fix in Claude Code

@hakarrr

hakarrr commented Nov 8, 2026

Copy link
Copy Markdown
Author

Closing as duplicate of #775, which already fixes this. My apologies - I mistook that open PR for an issue when scanning the tracker.

@hakarrr hakarrr closed this Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant