Repository navigation
[PR] fix: incremental lane — warn when two hosts share one conversation; prune dead-branch cache marks on resync - #4882
ranxianglei wants to merge 11 commits into
Conversation
…une dead-branch cache marks on resync #4868, two P3 improvements in the incremental lane (both verified against the code paths; no correctness impact either way): 1. Dual hosts sharing one conversationId mutually kill each other with zero visibility: each resync invalidates the other's acks, both burn 3 consecutive 409s into the sticky REJECTION_CIRCUIT=3 full-resend fallback until process restart. Every envelope now carries an optional per-process instanceId fingerprint (randomUUID, one value per host process, stamped in buildBody so all four native adapters get it for free); the server records the last successful resync source per conversation and warns once per 5min cooldown when two DISTINCT sources replace the same canonical within 60s. Missing or malformed fingerprints degrade to anonymous (no false alarm); old servers ignore the field, old clients omit it. - src/agent/incremental-client.ts: INSTANCE_ID + __incrementalInstanceIdForTests, buildBody stamps it - src/incremental-lane.ts: noteResyncSource (bounded map, budget pattern like droppedParamWarned), called on successful resync turns; reset seam clears it 2. inc:cache-marks grew unbounded across branches: resync replaced the canonical but entry.marks kept every historical id forever (map + ext key rewritten whole each turn, O(turns across branches)). applyTurn now prunes marks to the live canonical set on every resync before merging the turn's own marks. Ids are unique per session, so a surviving mark can only attach to a live message; under pluginLaneServerHistory the marks were already orphaned at assembly (derived-id keys vs pinned ids) — the prune makes storage match assembly reality. Behavior changes (additive): new optional envelope field instanceId; new warn log line for suspected dual writers; marks lifecycle OLD = all historical marks kept forever → NEW = pruned to the live set on each resync. Both rollout features (pluginIncrementalTurns, pluginLaneServerHistory) default to 0% — opt-in installs only. Tests: dual-writer alarm positive + same-source negative + missing/malformed fingerprint negative; marks pruning (dead branch pruned, live/new survive); client envelope stamp stability. PLUGIN.md documents the new field.
The #4491 host message-id field is accepted by the lane (payload.msgIds, src/incremental-lane.ts) but was never added to the documented envelope shape — readers could not discover it. One-line addition to the field list while this PR edits that very sentence for instanceId.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9411-pr.4882.1159c97This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4882.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) Re-check on the "behind" wake: no new sync needed — origin/master (23158db) is already an ancestor of the PR head (556b776), so the branch is fully up to date; all CI checks on that head remain green (26 success + 16 legitimately skipped, zero failures). |
(ai: vllm-qwen/qwen3.8-27b) Re-synced after master moved (#4854/#4817): merged origin/master cleanly (no conflicts), pushed new head a59ff13 — local typecheck + targeted incremental-lane tests green before push, and now all CI checks on that head are green (11/11 core test legs + every guard job); branch is up to date with master again. |
(ai: vllm-qwen/qwen3.8-27b) Re-synced after master moved again (#4856): merged origin/master cleanly (no conflicts), pushed new head 23cc847 — local typecheck + targeted incremental-lane tests green before push, and now all CI checks on that head are green (11/11 core test legs + every guard job); branch is up to date with master again. |
# Conflicts: # src/incremental-lane.ts # tests/incremental-lane.test.ts
…23 (#5522) The WC-023 ledger commit (f862e4a) accidentally deleted three permanent WIRE_RULES entries (WC-010 cache-breakpoint cap, WC-013 reasoning.summary, WC-014 Jinja head-system) and overwrote WC-011's #1733 provenance with unrelated #4882 text - violating the ledger-grows-only rule (#1304). Their enforcement clauses and gate tests were never removed, so this is a faithful byte-for-byte restore of the known-good records from ca532d4. Verified locally: tests/wire-contract.test.ts 23/23 green (ledger self-check now sees all 25 rules) and tsc --noEmit clean.
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved and pushed: Q1 — What was actually wrong? Not ordinary drift: master's history was rewritten after Oct 31. The two old-lineage merges this branch absorbed on Oct 31 ( Q2 — How was it fixed? MERGE (consistent with all six prior sync rounds on this PR), enforced by a deterministic invariant: final tree = live master + exactly the PR's own diff. All 97 conflicts taken on the master side; two sweeps of stray auto-merged files that silently retained old-lineage content (9 + 14, spanning CHANGELOG.md, src/web/, src/util.ts, docs/architecture/webui, reference/architecture.md, …) reset to master versions; then the PR's original patch (extracted verbatim from its two commits Q3 — Semantic conflict resolutions (unions, not pick-a-side):
Q4 — Verification: typecheck clean; targeted suites Q5 — Push discipline: fetched before pushing; remote PR head unchanged ( Q6 — CI state: cannot assert yet — the mirror check-cache has not picked up Q7 — Decisions needed / behavior flags: none needed — every resolution was mechanical or a strict union. Behavior flags: (a) the prune line now sets |
[ework] CI fix verified green ✓ |
1 similar comment
[ework] CI fix verified green ✓ |
(ai: vllm-qwen/qwen3.8-27b) Re-synced against the moved master and pushed: Q1 — What was wrong this run? Master advanced again after last night's sync: Q2 — How was it fixed? Merged live master into the branch (7th sync round, same pattern as all prior rounds), enforcing the same deterministic invariant: final tree = live master + exactly the PR's own diff. The CHANGELOG conflict resolved to the master side (a strict superset — it contains everything the branch side had plus the new entries). Verified afterwards: Q3 — Semantic resolutions: none beyond the CHANGELOG superset. One interaction checked explicitly: master's new #3397 durability-before-render hunk in Q4 — Verification: Q5 — Push discipline: fetched before pushing; remote PR head unchanged ( Q6 — CI state: the mirror check-cache has not picked up Q7 — Decisions needed / behavior flags: none. This sync introduced no behavior change of its own (the behavioral deltas in the tree are master's own merged PRs, not this PR's). Both rollout features remain 0% opt-in; no config surface; no kernel changes from this PR. |
[ework] CI fix verified green ✓ |
[bot] 🏷 Status confirmation after process restart (work was already complete and pushed before the restart; the earlier 七问 summary above covers the full run):
|
Closes #4868
Two P3 improvements to the incremental lane, both verified against the code paths before implementing (the issue's analysis was accurate on both counts).
1. Dual hosts sharing one conversationId now get a clear warn
Before: two incremental clients on one conversation each resync and flip each other's canonical; every resync invalidates the other side's acks, so both burn 3 consecutive 409s into the sticky
REJECTION_CIRCUIT=3full-resend fallback until process restart — with zero log evidence that this was dual-writer contention rather than ordinary drift.Fix: every envelope carries an optional per-process
instanceIdfingerprint (randomUUID(), one value per host process lifetime, stamped once inbuildBodyso all four native adapters — pi/omp/dsh/opencode-native — get it without per-site edits). The server records the last successful resync source per conversation (noteResyncSource, bounded map with the same budget pattern asdroppedParamWarned) and warns once per 5-minute cooldown when two DISTINCT sources replace the same canonical within 60s:Missing or malformed fingerprints degrade to anonymous (that side simply cannot be fingerprinted — no false alarm); older servers ignore the unknown field, older clients omit it.
Rejected alternatives: socket/pid fingerprinting (sockets rotate across reconnects; pid is never transmitted), header stamping (would need edits at all four adapter dispatch sites instead of one).
2.
inc:cache-marksno longer accumulates dead branchesBefore: resync replaced the canonical but
entry.markskept every historical id forever — dead-branch marks lingered in the map and were rewritten into the ext key whole every turn, O(turns across branches).Fix:
applyTurnprunesentry.marksto the live canonical set on every resync (before merging the turn's own marks). Ids are unique per session, so a surviving mark can only attach to a live message — nothing can be mis-attributed. UnderpluginLaneServerHistory, marks were already orphaned at assembly time (keyed by derived content-hash ids while canonical ids are pinned host ids); the prune makes storage match assembly reality there too.Behavior changes (all additive, disclosed per policy)
instanceId(never forwarded upstream, never persisted)Both rollout features stay at their current defaults (
pluginIncrementalTurns0%,pluginLaneServerHistory0%) — opt-in installs only; no config key, env var or CLI flag added. The model wire is byte-identical in both compression modes (the fingerprint dies at the lane boundary; the prune only removes marks whose ids are absent from the served history).Tests (targeted suites green locally; full-suite verdict = CI on the pushed head)
tests/incremental-lane.test.ts: dual-writer alarm positive (A seed → B stale-delta 409 → B resync ⇒ exactly one warn naming both sources); same-source resync ⇒ no second alarm; missing/malformed fingerprint ⇒ zero alarms (gate discipline, both directions); marks pruning — dead-branch mark pruned on resync, live/new marks survive (asserted against the persistedinc:cache-marksext row).tests/incremental-client-recovery.test.ts: envelopeinstanceIdpresent and stable across plans (seed → ack → delta).msgIds?field entirely — added while editing that very sentence forinstanceId.No kernel/ changes, no config surface, no persistence format change (ext value shape unchanged — fewer entries after a resync).