Repository navigation
[PR] fix: same-tick ingest dedup — skip the redundant writer-off build after ingestRowsNow (#6782) - #6786
[PR] fix: same-tick ingest dedup — skip the redundant writer-off build after ingestRowsNow (#6782)#6786ranxianglei wants to merge 1 commit into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.10098-pr.6786.5e47413This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6786.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 查重: 不重复 — screened the 100 most recent open PRs: #6786 is the only open PR referencing #6782; no parallel fix for this thread. PR state (verified live, 2026-11-15 ~02:20Z): open, not draft, mergeable (no conflicts, 72 commits behind master Code review (full diff read against the exact PR base The fix implements direction 1 from #6782 (same-tick dedup, minimal change) exactly as proposed:
PR claims verified against the code:
Path audit (which callers can/can't hit the dedup):
Tests (local, targeted — full suite is CI's job): checked out the PR head files onto the exact PR base and ran:
CI (verified live via mirror cache, ~02:20Z): 30 checks — 0 failed, 14 still pending (guard lanes queued, test matrix running), e2e lanes skipped per their gates, One minor finding (non-blocking, for your call): Verdict: mergeable pending the full CI matrix going green. The fix is correct, minimal, well-tested, and matches the direction proposed in #6782. Suggested path: rebase onto master (72 behind), merge when the matrix is green; the memory-hygiene finding can be folded in as a follow-up commit or tracked separately — your call. 结构化八问(供人工 reviewer):
收工判定: 可合并(等全量 CI 矩阵转绿 + rebase;内存卫生发现由 owner 另行决定)。 |
35a6af1 to
74c6aa5
Compare
The mechanism this PR implemented (skip the redundant writer-off guardedBuild after ingestRowsNow already covered the tick's state) landed on master via #6791 (the #6780 fix) with a stricter gate (durabilityArms: epoch+1, tail ref, no decompositionLog, no contentStoreDirty, no pendingLedgerAlign, metadata bytes unchanged, single-shot arm). The conflict rebase resolved src/persist.ts to the master version; the PR's weaker ingestedRevision/ingestedTailRef gate would have skipped builds master's gate deliberately runs, so it was dropped. Only the behavioral regression tests remain — they assert shadow.stats.ingests and pass against the master implementation.
74c6aa5 to
1d763bd
Compare
(ai: opencode/vllm-qwen/qwen3.8-27b) Merge conflict resolved and pushed. The PR was dirty because it and master's #6791 (the #6780 fix, merged today 12:48 +0800 — 9 minutes after this PR's commit was created) independently implemented the same same-tick ingest-dedup mechanism in the same region of Key judgment in the resolution: master's #6780 mechanism ( Verified locally: 七问:
|
(ai: opencode/vllm-qwen/qwen3.8-27b) Stale blocker re-trigger — nothing to do this run. Verified live just now: the PR head is 七问:
|
What
When
ingestRowsNowalready ingested the current content in the same tick (the prepare phase, before the forward), themarkDirty-triggered writer-off coalesced build was running a second full delta-ingest transaction (3-table full read + tail_json rewrite + tx commit) for an empty delta — the watermark was already advanced by the first ingest.This adds a same-tick dedup signal:
ingestRowsNowrecords the revision andlastMessagesreference it ingested; the writer-off build skips the redundantguardedBuildwhen the proof revision is exactly +1 (only the unconditionalmarkDirtybumped) and the tail reference is unchanged.Why
#6782: every save's ingest transaction unconditionally full-reads the session's 3 ledger tables (refs/blocks/ccr) for final-state reconciliation — O(session size) fixed cost. 0.1.214 amplified this:
ingestRowsNow(PR #6326, fixes #3397) + the writer-off microtask path (commit 5a80e2b) made the heavy transaction run TWICE per round pre-forward, the second being an empty delta that pays the full fixed cost for nothing. In the #6780 A/B scenario, ~half the ~3.2s pre-forward cost was this no-op.How
ingestRowsNownow callsnoteIngested(session)after a successfulshadow.record(both branches), recordingsession.revisionEpochandsession.lastMessagesreference.scheduleSaverun() computesskipRedundantBuild: true wheningestedRevisionis defined,proofRevision === ingestedRevision + 1, andingestedTailRef === session.lastMessages. When true, it skipsguardedBuild(the rows are already durable) but still runs the full settle protocol (noteDurable,lastSaveLanded,runIdleSpillIfIdle,drainFlushWaiters).ingestRowsNow;noteOwnCommitwas already recorded by it.noteMetaFpis a no-op in writer-off anyway.Regression test
tests/issue6782-same-tick-ingest-dedup.test.ts— 3 tests:ingestRowsNow+markDirty→ the writer-off build skips the redundantguardedBuild(shadow.stats.ingestsstays at 1, not 2).ingestRowsNowand the microtask (a newlastMessagesreference) → the build is NOT skipped (ingestsis 2).markDirty) → the build is NOT skipped (ingestsis 2).Verification
npm run typecheckclean.tests/issue6782-same-tick-ingest-dedup.test.ts(3/3),tests/issue3397-durability-before-render.test.ts(2/2),tests/storage-save-path.test.ts(3/3),tests/storage-full-flip.test.ts(10/10).Closes #6782