Skip to content

fix: a JS-parsed ingest takes its turn in the table write queue (SLEEPER-PROD-8469) - #1

Closed
claude[bot] wants to merge 2 commits into
mainfrom
claude/nested-txn-8469
Closed

claude[bot] wants to merge 2 commits into
mainfrom
claude/nested-txn-8469

Conversation

@claude

@claude claude Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Requested via Slack thread

Before: Partitions that can't use the native shredder (player-season and player-games in sleeperbot's player_stats store) were written by a synchronous table.overwrite(), which skips the table's write queue. If another partition's native shred was still running, it held SAVEPOINT nitro_shred on the same writer connection; nitro doesn't queue executeAsync, so nothing stopped the sync write. Its batch's BEGIN EXCLUSIVE then failed with "cannot start a transaction within a transaction" (SLEEPER-PROD-8469). The batch's ROLLBACK also aborted the running shred, which then failed with "no such savepoint: nitro_shred" (SLEEPER-PROD-846A). The partition write was dropped, and after 4 such contentions the store went fatal: it reopened and refetched, and in some cases moved to its in-memory database. All 5 executeBatch events in 8469 have a matching 846A event from the same user in the same second.

After: The JS-parse path, along with ingestRaw's fallback after a failed shred, goes through shred(..., { inJs: true }). That path waits its turn in the table's serialized() write queue like every other async write, so it never starts a batch while a native shred is still running.

How: RowTable.shred takes an optional { inJs } (src/table/types.ts). In src/table/sqlite.ts, shredOrParse skips the native shredder when inJs is set but still runs inside withDeferredIndexes(serialized(...)). In src/define_partitions.ts, ingestRaw's inJs now awaits that call instead of the sync overwrite. Two define_partitions tests are updated, and a row_table regression test checks that a JS-parsed shred waits for a native shred that hasn't finished. tsc is clean; yarn test: 721 passed; yarn test:prod: 698 passed.

Not covered:

  • lib/ is rebuilt in this PR, but shipping still needs a new gitpkg tag and a bump of the pin in sleeperbot's clients/app-shared/package.json.
  • Bugs in the nitro fork: cpp/sqliteExecuteBatch.cpp:63-64 sends ROLLBACK even when its own BEGIN failed, so it rolls back the other caller's transaction; and executeAsync is not queued by DatabaseQueue, so startOperationSync can't see it.
  • The "database table is locked" executeAsync events grouped under 8469. The likely sources are the fire-and-forget ETag write and index DDL outside the queue; not confirmed.
  • The onContended message ("usually means its dedicated reader never opened") is misleading here. The reader is enabled, and the collision is between two writes.
  • Not tested on a device.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WS7KpoVVBG1REsUE1GkM2e

… its batch never starts inside a running native shred's savepoint (SLEEPER-PROD-8469)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WS7KpoVVBG1REsUE1GkM2e
@claude
claude Bot requested a review from sleeper-luke October 6, 2026 04:05
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WS7KpoVVBG1REsUE1GkM2e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants