Repository navigation
[PR] fix: watermark-HIT window save must not retire the verified prefix (#7346) - #7357
Conversation
… the verified prefix On a fastpath HIT the server pins only this turn's ~2-row window (snapshotMessages(session, wpHit.windowCores)). snapshotMessages stamps the pin raw (lastMessagesFolded=false), so the save presents the window as the complete raw log and the #3285 authoritative-set sweep retires every ledger row the window omits — a 100k-session prefix drained to the ~2-row window each turn, with the prefix cache lost on every full-list turn. Mark the pin non-raw on HIT turns (lastMessagesFolded=true) so persist.ts stops passing the raw tail and the sweep never fires. The window is still ingested through the bounded envelope view; a genuine truncation (raw tail) still sweeps. Regression: tests/issue7346-pf-window-ledger.test.ts pins both directions — a non-raw window save keeps the verified prefix, a raw window save still sweeps it. Fixes #7346
[bot] 🏷 Verified PR #7357 (the #7346 fix) — clean, correct, CI green. What I verified
Two minor points (not blockers)
8问
判定:PR 干净、修复正确且最小、回归测试钉住了关键行为、CI 全绿。 |
[bot] 🏷 Follow-up: filed drive-by issue #7374 (labels: scope/ci, p3) for the test-gap noted in point 1 above — the #7346 regression test simulates the fix's downstream effect (rawTail=undefined ⇒ no sweep) but does not exercise the |
…the verified prefix The sibling test (issue7346-pf-window-ledger.test.ts) pins the #3285 sweep/no-sweep contract by calling shadow.record directly with the two rawTail shapes, but never drives prepareOpenai — deleting the fix line 'if (wpHit !== undefined) session.lastMessagesFolded = true' would leave the suite green. This test drives a real watermark-HIT turn through the full proxy in shadow mode (sidecar -> full resend -> HIT) and asserts the verified prefix survives in the ledger after the HIT turn. Verified: passes with the fix line, fails without it (got 2 live rows = the swept window). Closes #7374
…rmSync (win22 EBUSY) The fullpath regression opened a shadow-enabled SessionStore but only called cancelAll() at teardown, which cancels pending writes WITHOUT closing the unified sidecar's SQLite handle. Windows cannot delete a directory while index.db is open, so the temp-dir rmSync failed with EBUSY on the windows-latest shard (job 136191107534). closeQuietly() (the documented short-lived-store close) releases the handle before the cleanup unlink. Idempotent, never throws; test logic unchanged.
Fixed the red Diagnosis — the shard's only failure was this PR's own new regression test Fix — added Verified — targeted local run of both #7346 tests passes (Linux; the EBUSY itself is Windows-only). Pushed as the final head 七问:
判定: 修复已作为最终 head 推送,全量 CI 运行中,先前红的 windows 分片已转为运行态;本轮无需进一步动作。 |
…assertion is not masked by win22 EBUSY The prior teardown (closeQuietly after closeServers, inside the try block) only runs when the test PASSES. When the assertion fails (the fix line is removed — the whole point of this regression test), the throw skips closeQuietly, the shadow still holds the index.db lock, and the rmSync in the finally block fails with EBUSY on Windows, masking the real assertion error. Move closeQuietly into the finally block (guaranteed to run on both the passing and failing paths) and use rmBestEffort (bounded retries, swallow persistent EBUSY — the #3397/#4986/#6780 pattern) so the assertion error propagates cleanly and the temp dir is best-effort cleaned up.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.11138-pr.7357.18ca6cfThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr7357.tgz
npm install -g packageInstall instructions are refreshed on each push. |
ranxianglei
left a comment
There was a problem hiding this comment.
Approved — leg-1 fix is correct and well-tested. One scope flag on the issue (not a code defect).
The change (src/server/prepare-openai.ts:672): after snapshotMessages pins the HIT window, if (wpHit !== undefined) session.lastMessagesFolded = true; marks the pin non-raw before ingestRowsNow, so the #3285 authoritative-set sweep sees a pruned view instead of a "complete raw log" and stops retiring the verified prefix. Miss turns are untouched — a raw full-log save still sweeps, so genuine truncation is still handled.
Tests are solid: the sibling test pins the sweep/no-sweep contract at the storage layer (rawTail omitted vs = window), and the fullpath test drives a real watermark-HIT turn through prepareOpenai, so it fails if the fix line is removed.
Scope flag for #7346: the issue names TWO legs; this PR fixes leg 1 (raw-tail). Leg 2 — the false DIGEST DRIFT on plain/empty-envelope sessions via hydrateSlimCcrStore — is not here (that function doesn't exist in master yet), and the issue says leg 2 forces the sweep even with leg 1 fixed. I can see leg 2 is exactly what big PR #6785 lands: it touches src/storage/slim.ts, src/persist.ts, src/storage/shadow.ts and adds tests/issue5874-ccr-slim.test.ts (the very test #7346 names for leg 2). So: merge #7357 now (leg 1 is a real, correct, tested fix) but keep #7346 open until #6785 also lands.
Housekeeping: 45 behind master (update); build-artifact is the stale pre-token-fix 404, re-running.
What
On a watermark-HIT turn the server pins only this turn's ~2-row window (
snapshotMessages(session, wpHit.windowCores)).snapshotMessagesstamps the pin raw (lastMessagesFolded=false), so the save presents the window as the complete raw log and the #3285 authoritative-set sweep retires every ledger row the window omits — a 100k-session prefix drained to the ~2-row window each turn, with the upstream prefix cache lost on every full-list turn.Fix
Mark the pin non-raw on HIT turns (
lastMessagesFolded=true) insrc/server/prepare-openai.ts(the only watermark-HIT pin path — the watermark fastpath is openai-wire only, KDD #15).persist.tsthen stops passing the raw tail, sohasRawTail=false→completeLog=false→ the #3285 sweep never fires. The window is still ingested through the bounded envelope view; a genuine truncation (raw tail) still sweeps.Leg 2 (not reproducible in master)
The issue's second leg names
hydrateSlimCcrStore/slim.ts/ theincremental list too short to overlap the ledgerlog line — none exist in current master. The described desync (empty CCR envelope → null hydration → persistent DIGEST DRIFT →completeLogsweep on every save) is preempted by theliveCcroverride atsrc/storage/shadow.ts:504: both digest legs flow through the same override, so an empty-but-defined contentStore is consistent on both legs and no drift latches. Reported as not-reproducible; no change made for leg 2.Regression
tests/issue7346-pf-window-ledger.test.tspins both directions (the sweep only runs on the full path, so the watermark is cleared to force it):Behavior change
On a HIT turn the handoff-reply stash rows are now skipped (
src/handoff-response.ts:149guards onlastMessagesFolded !== true) — consistent with the existing folded semantics (a bounded folded re-render is a pruned view, not the raw tail). The reply is still persisted viametadata.handoffResponseand re-sent by the client. No change on non-HIT turns.Closes #7346