Skip to content

Waits for resume in compaction and stops bad keys leaking io - #115

Merged
kasperisager merged 6 commits into
holepunchto:mainfrom
tur-nr:feature/usage-api
Nov 3, 2026
Merged

kasperisager merged 6 commits into
holepunchto:mainfrom
tur-nr:feature/usage-api

Conversation

@tur-nr

@tur-nr tur-nr commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Problem

approximateSize and compactRange encode their keys after io.inc() but outside their try. A key that fails to encode leaves io held, so the next suspend() never drains.

compact and compactRange also don't wait for resume the way flush() and the stats calls do. Called while suspended, they fail with EINVAL.

Proposal

  • Pulls the repeated resume check into _runWithIO(fn), which takes io, waits for resume and releases io in a finally around the whole call
  • flush, approximateSize, currentWalFile, getProperty, compact and compactRange go through it
  • batch._flush() keeps its own check, because a failed resume there already releases io through _abort() and _onfinished(), and the wrapper would release it twice
  • No signature changes

Tests

A key that fails to encode makes approximateSize and compactRange reject, and a suspend() and resume() afterwards still complete. Before the fix, both hung on the suspend.

Suspend, then call compact and compactRange: neither settles until resume(), then both resolve. Closing while they wait rejects both with "RocksDB session is closed", same as a waiting flush(). Before the fix, both failed straight away with EINVAL.

Notes

Compaction issued during suspend now waits for resume instead of failing straight away. Worth a line in the release notes, like #113.

This PR started out adding getUsage and getDiskUsage. They've moved to their own rocksdb-usage module, built on getProperty and currentWalFile, so only the fix is left here.

The compactRange test uses a string key because compactRange reads an object end as options, so a Buffer end is taken as opts. That's left for a separate fix.

@tur-nr
tur-nr force-pushed the feature/usage-api branch from bcf6030 to 6a42e4a Compare October 1, 2026 09:54
@tur-nr tur-nr changed the title Adds getUsage and getDiskUsage Waits for resume in compaction and stops bad keys leaking io Oct 1, 2026
@kasperisager

Copy link
Copy Markdown
Contributor

@tur-nr Is this something that should land?

@tur-nr

tur-nr commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

@kasperisager it is, however I need to clean it up as I believe there are some unnecessary code. Will do this now.

@tur-nr
tur-nr force-pushed the feature/usage-api branch from 6a42e4a to 9ec4788 Compare November 3, 2026 03:47
@tur-nr
tur-nr marked this pull request as ready for review November 3, 2026 03:49
@tur-nr
tur-nr requested a review from a team November 3, 2026 03:49
@greptile-apps

greptile-apps Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Moves repeated resume checks and io counting into _runWithIO.

  • Database calls wait for resume and release I/O after errors.

No actionable issues were found. Tests were not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Operation starts] --> B[Increment io]
  B --> C{Waiting for resume?}
  C -- No --> G[Run operation]
  C -- Yes --> D[Release io while waiting]
  D --> E{Resume or close?}
  E -- Resume --> F[Retake io]
  F --> G
  E -- Close --> H[Retake io and reject]
  G --> I[Release io in finally]
  H --> I
Loading

Reviews (1) · Last reviewed commit: "Hold io for flushWAL through _runWithIO" · Reviewed by Greptile

@kasperisager
kasperisager merged commit f67c04d into holepunchto:main Nov 3, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants