Repository navigation
fix: 重启交接后 unified 存储跨进程污染固化(DIGEST DRIFT 永不愈合)— #3285 - #3288
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3943-pr.3288.3abfa33This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3288.tgz
npm install -g packageInstall instructions are refreshed on each push. |
… full path sweeps foreign message rows (#3285) Restart handoff with two processes sharing the state dir let a previous writer's index.db rows and last_ingested_seq watermark survive into the new process: the new writer inherited foreign delta trust (empty unverified deltas) and its full re-ingest kept foreign orphan rows via INSERT OR IGNORE + identity remap, so every sampled digest check flagged DIGEST DRIFT forever. The b9e82f6 heal (clear watermark on drift) was insufficient because the subsequent full re-ingest still kept orphans. Fix: resetAllIngestWatermarks() at writer init (once per boot; readers/export untouched), and the ingestSession full path sweeps message rows absent from the plan when the plan is a complete raw-tail log (completeLog gate — envelope-only plans are pruned folded renders and must never trigger sweeps). Regression test: tests/storage-ingest-watermark.test.ts restart-handoff case (fails without the fix).
…3285) The e2e incremental-lane case covers the user-visible 409 loop; this unit case pins the mechanism itself — a session carrying any "inc:" ext key survives a completeLog ingest untouched, while a marker-less control session still gets its foreign rows swept.
d62bf55 to
d6e703a
Compare
(bot) 🏷 (ai: vllm-qwen/qwen3.8-27b) 冲突已解决并 force-push:新头 d6e703a,mergeable=true(现 blocked 仅等 CI)。 冲突面只有一处,且是「双方各修了一遍同一处」的良性重复:
结构化八问
本地定向验证(§7.2,非全量):typecheck clean; |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.484-master.d0ebe24 # this exact build
npm install -g billion-context@master # always the newest merged build
|
What & why
[storage] unified shadow DIGEST DRIFT ... migration parity brokenfired on boot and never healed across 8h of steady traffic after a host restart (#3285, v0.1.197 field case). Root cause: the unified store's per-session ingest watermark (sessions.last_ingested_seq) carries no writer identity, and the full-ingest path only ever INSERTs (INSERT OR IGNORE) — so under a restart-handoff overlap (old instance still draining while the new one boots; both share the machine-global state dir per the #2671 contract) the new process inherits a foreign watermark's trust AND foreign message rows that no later save removes. Every sampled replay digest keeps flagging drift forever; the #3166 heal (watermark-clear on drift) is insufficient because the subsequent full re-ingest still keeps the orphans.Two changes, both in the unified store:
resetAllIngestWatermarks()runs once at writer init (UnifiedShadow.maybeOpen). Each session's first save after boot takes the exact full path, re-establishing trust from this process's own view. Pure readers (export) never call it.completeLog(plan carried a raw-tail log): rows whose identity (msgId + content hash) is absent from the plan are released (CAS refcount) and deleted. Envelope-only plans are pruned renders (Identity pipeline: key sessions by (agent, conversation id) in the unified store — decouple fold ledgers from protocol/host origin (#2938 / #2847 / #2671 defect-1) #2960⑤ folded view) — sweeping on those would delete raw rows interned beyond the render window, so they stay insertion-only.tests/incremental-lane.test.ts"divergent resync survives a cold restart"). Lane-owned sessions (anyinc:*session-ext key) therefore opt out of the sweep.Behavior changes (disclosed)
messagesrow set changes once (foreign orphans removed) on the first complete-log full ingest — this is the repair; clean sessions see no difference (their plan covers every stored identity).In-pass fix (unrelated, clearly labeled, commit 3)
tests/issue3148-upstream-circuit.test.tsfailed on every shard of master itself (verified against the base commit's own CI): the #3148 "exact clamp event" expectations (cap 1360) predate the #3071 4096 capability floor, which is intentional fail-closed behavior (clampOutputBudget, bounded bywindow - input) with its own pinned test suite. Updated the two stale constants + the explaining comment to 4096. No source change; the recorded event and the wire decision remain exactly equal (the test's actual invariant). Dup-screened: no open issue/PR addresses this red.Tests
verifyDrift: 0), steady-state delta resumes.Related: #3285 (de-anchor facet there is host-side — DSH Desktop rebuilds/resends reconstructed history across its update+restart; the DIGEST DRIFT facet is what this PR fixes). Not closing #3285.