Repository navigation
Count the blocks actually cleared when clear() is called with diff #945
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -616,6 +616,28 @@ class SessionState { | |||||||||
| } | ||||||||||
| } | ||||||||||
|
|
||||||||||
| if (cleared !== null) { | ||||||||||
| // Count the blocks that actually had data, so the caller can tell how | ||||||||||
| // much was cleared. | ||||||||||
| const clearEnd = Math.min(end, this.length) | ||||||||||
|
|
||||||||||
| if (this.isDefault()) { | ||||||||||
| for (let i = start; i < clearEnd; i++) { | ||||||||||
| if (this.core.bitfield.get(i)) cleared.blocks++ | ||||||||||
| } | ||||||||||
|
Comment on lines
+625
to
+627
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
On a normal core, this synchronous loop checks every index, including every missing block in a sparse range. Large clears therefore block other JavaScript work for an avoidable per-block scan. Use
Suggested change
Knowledge Base Used: Sparse bitfields and availability state Prompt To Fix With AIThis is a comment left during a code review.
Path: lib/session-state.js
Line: 625-627
Comment:
**Sparse clears scan every index**
On a normal core, this synchronous loop checks every index, including every missing block in a sparse range. Large clears therefore block other JavaScript work for an avoidable per-block scan. Use `core.bitfield.countSet(start, clearEnd - start)` instead; it counts by page and already backs `count()`.
```suggestion
cleared.blocks += this.core.bitfield.countSet(start, clearEnd - start)
```
**Knowledge Base Used:** [Sparse bitfields and availability state](https://app.greptile.com/-/switch-tenant?tenant=tetherio&next=%2Fcustom-context%2Fknowledge-base%2Fholepunchto%2Fhypercore%2F-%2Fdocs%2Fsparse-bitfields.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly. |
||||||||||
| } else { | ||||||||||
| const rx = this.storage.read() | ||||||||||
| const hasPromises = [] | ||||||||||
| for (let i = start; i < clearEnd; i++) hasPromises.push(rx.getBlock(i)) | ||||||||||
| rx.tryFlush() | ||||||||||
|
|
||||||||||
| const had = await Promise.all(hasPromises) | ||||||||||
|
Comment on lines
+630
to
+634
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A named-session clear with Knowledge Base Used: Core lifecycle and mutation API Prompt To Fix With AIThis is a comment left during a code review.
Path: lib/session-state.js
Line: 630-634
Comment:
**Clear loads all blocks**
A named-session clear with `diff` now reads every block and retains all the results in `Promise.all()` before deleting anything. Peak memory grows with all the block bytes being cleared, plus one promise per index. Use bounded batches or `createBlockStream()`, as `count()` already does, so clearing a large range does not require loading the whole range into memory.
**Knowledge Base Used:** [Core lifecycle and mutation API](https://app.greptile.com/-/switch-tenant?tenant=tetherio&next=%2Fcustom-context%2Fknowledge-base%2Fholepunchto%2Fhypercore%2F-%2Fdocs%2Fcore-lifecycle.md)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly. |
||||||||||
| for (const block of had) { | ||||||||||
| if (block !== null) cleared.blocks++ | ||||||||||
| } | ||||||||||
| } | ||||||||||
| } | ||||||||||
|
|
||||||||||
| if (end - start === 1) tx.deleteBlock(start) | ||||||||||
| else tx.deleteBlockRange(start, end) | ||||||||||
|
|
||||||||||
|
|
||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -181,13 +181,11 @@ test('clear blocks with diff option', async function (t) { | |
| const cleared = await core.clear(1337) | ||
| t.is(cleared, null) | ||
|
|
||
| // todo: reenable bytes use api | ||
| const cleared2 = await core.clear(0, { diff: true }) | ||
| t.is(cleared2.blocks, 1, 'the stored block was cleared') | ||
|
|
||
| // const cleared2 = await core.clear(0, { diff: true }) | ||
| // t.ok(cleared2.blocks > 0) | ||
|
|
||
| // const cleared3 = await core.clear(0, { diff: true }) | ||
| // t.is(cleared3.blocks, 0) | ||
| const cleared3 = await core.clear(0, { diff: true }) | ||
| t.is(cleared3.blocks, 0, 'the block was already cleared') | ||
|
Comment on lines
+184
to
+188
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The restored test checks only one stored block and its repeated clear. It does not check counting across holes or limiting the count at Prompt To Fix With AIThis is a comment left during a code review.
Path: test/clear.js
Line: 184-188
Comment:
**Range counts lack tests**
The restored test checks only one stored block and its repeated clear. It does not check counting across holes or limiting the count at `length`. Add a default-session test that clears a partly stored multi-block range with an end beyond `length`, and an out-of-range call with `diff: true`. The existing `clear(1337)` call checks only the no-diff return.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time! |
||
|
|
||
| await core.close() | ||
| }) | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isDefault()also includes atomic sessions, butcore.bitfielddoes not include their staged appends. After creatingcore.session({ atom })and appending a block without flushing the atom,clear(0, { diff: true })clears that block but still reports{ blocks: 0 }. Read the atomic session’s own stored blocks instead of the parent’s bitfield.Knowledge Base Used: Sessions, snapshots, and views
Prompt To Fix With AI