Repository navigation
[PR] integrate: #5901 (snapshot plane) + #6564 (msgid delivery) — pre-merge validation - #6680
ranxianglei wants to merge 71 commits into
Conversation
…6539) Complements the client-side envelope wiring (#6532 shipped dsh openai+ anthropic). This is the 'record' half of the msgid lifecycle made whole across every wire: - prepare-google / prepare-responses: read + strip the biliMsgIds envelope before it can reach the upstream, then pin behind the same rollout gate as the openai/anthropic twins. A length/shape mismatch is rejected (bail) at the pin — never leaked, never mis-pinned. - handle.ts: a non-body id source (x-bili-msg-ids header / loopback harvest) now stamps the parsed body on EVERY wire, not just openai/anthropic, so the four pin-consumers all see it uniformly. - rollout: pluginLaneServerHistory 0% -> 100% (GA, owner ruling 2026-11-13). No client calls the co-gated /__bili/plugin/render endpoint yet, so flipping only activates pinning. Roll back with BILI_ROLLOUT_PLUGIN_LANE_SERVER_HISTORY=0. Three tests that pinned the old default-off contract now force the off state explicitly (env=0) instead of deleting the var.
Principle 1 of the four-principles proposal (discussion #6262): identity, not bytes, decides which prefix a fork inherits. The plugin-protocol channel (dsh/omp/pi declared forks) gains an optional msgid track: - a host-pinned parent exposes the host's stable message ids AS its snapshot rawIds, so the child may present the host id of each matched prefix message (prefixMsgIds) plus the hash-run length (driftFrom); - the server accepts a byte-drifted prefix only when every presented id equals the parent's rawId at its slot; the receipt is marked status "drifted" (or "expanded+drifted") with the verified driftFrom echoed; - a gate failure never fails the fork: the byte-verified strict fork proceeds exactly as before (no 409); malformed shapes are 400; parents without host-pinned ids make the track a no-op (byte path unchanged). Client: lanes inject their existing parity-pinned collectors (dsh hostMsgIdsOf; pi/omp laneMsgIdsForWire on the anthropic wire with the same content-pinnable gate the incremental pin uses); fork-adopt performs only the core-alignment length lock — no new validator copies, msgid-rule-parity matrix untouched. opencode-v2 is not wired in this PR (byte track, follow-up). Tests: tests/issue6262-fork-msgid-track.test.ts (8) — client prefix extension across a drift cell, the byte-fallback arms, a full-chain proxy round trip, and the server gate (drifted receipt / strict fallback / 400 arms). Docs: msgid-infrastructure.md consumer 6, plugin-protocol.md section 8, architecture.md.
Carrier ① (the envelope body field) had zero client wiring: the server consumes biliMsgIds on both wires (prepare-openai/anthropic, gated on pluginMode + the pluginLaneServerHistory rollout), but no client ever stamped it. dsh's stamping lived only on the anthropic-wire incremental lane (#4491), so every openai-wire full-resend turn — dsh's main path — arrived id-less and the store fell back to content-derived digests, which in turn leaves the fork msgid identity track (#6293) with nothing to match. maybeStampEnvelopeMsgIds now stamps full-resend model bodies (both wires) after the incremental lane had its chance. Resolution never re-implements the host's wire transforms: the pi-ai lockstep walkers (laneMsgIdsForWire / laneMsgIdsForOpenAiWire — the SAME functions the pi/omp lanes pin with) prove wire-element ⇔ host-message pairing by order-preserving fingerprints, bail all-or-nothing; the kernel's own converters then prove core-alignment via the #3642 wireIndex back-pointer (every non-leading wire element must own exactly one core — parallel tool calls / reasoning+text / text+image parts split one host message into N cores, and N cores cannot share one host id). System-role entries are excluded (they ride the hoisted/folded leading system, never 1:1 to a non-leading wire element). Any bail leaves the body byte-identical → content-derived ids, exactly today's behavior; with the rollout off the server strips the field, so default installs are untouched.
#6525 B7) dsh's flat host messages differ from the pi-ai shape the shared walker's host-side fingerprint assumes in two places (role tool vs toolResult, block tool-call vs toolCall), so every tool-bearing turn bailed the all-or-nothing envelope and the turn-1 host-pinned rows churned back to content digests on the next full-resend. Normalize dsh's shape for fingerprinting only, in the dsh resolver: the envelope ids still come from mm.id and the wire-side pairing is untouched, so no mis-pin is possible. Found via a real-dsh e2e dump (after turn 1: live rows 11/11 content digests). Now: live rows 11/11 host-pinned UUIDs. Two regression tests pin the accept (a dsh tool turn core-aligns and stamps) and the retained bail (parallel tool_calls still refuse, one host id cannot cover N cores).
…6525/#6539) The server pins host ids against the POST-HOIST core count (prepare-openai validates ids.length against openaiToCore(parsed).length, which hoists the contiguous leading system/developer run out of the fold space). opencode V2's x-bili-msg-ids header was stamped per WIRE message, INCLUDING the leading system, so its count always exceeded the core count on the openai wire and the pin was rejected (a safe content-derived degrade — but opencode V2 never actually pinned on the openai wire). Core-align the V2 extractor: skip the contiguous leading system/developer run before collecting ids so the header count equals the post-hoist core count. No-op on the anthropic wire (system is a separate field, never in messages); mid-conversation system messages keep their ids; a leading-system-only turn degrades (no empty header). Whole-turn poison contract unchanged.
…#6539 review) Dual-agent review of the #6539 integration found the egress gap: a stamped full-resend body rode biliMsgIds/biliLedgerSeqs out through the verbatim forward branches that bypass the four fully-processed prepares — strict-schema upstreams 400 'Extra inputs are not permitted', and #661 body fingerprinting breaks. - server.ts: protocol-neutral scrubMsgIdFields (byte-identical pass-through when the fields are absent, so #661's contract is untouched in the normal case) + prepareCountTokens egress hygiene. - handle.ts: scrub applied on every verbatim branch (unattributed passthrough, bypass, v1internal background, commandcode unconvertible, no-messages fallback, side-request body, auto-mode forge, final fallback). The #1884 signed-passthrough escape hatch is deliberately NOT scrubbed — the body-covering sdk-hmac-sha256 signature would break; byte-untouched wins. - dsh-native.ts: resolveEnvelopeMsgIds now calls the shared coreAlignEnvelopeIds instead of the cherry-pick'd inline copy (review M1); the fork-adopt msgids use the SAME core-aligned resolver (m1: the old per-wire-message hostMsgIdsOf could never pass the adopter's msgIds.length===core.length lock); explicit incremental-lane guard (m2); ENVELOPE_MSG_IDS_FIELD hoisted to module level with a parity unit test (n2). - opencode-v2-msgids.ts: comment corrected to the actual responsesToCore sysDevHead / anthropic separate-field rules (m3). - non-ASCII note text removed (n1). New: tests/issue6539-verbatim-scrub.test.ts (7). Regression: msgid/envelope/fork unit 70/70 + 28/28 + 22/22; wire e2e google+responses 17/17; plugin-fork 52/57 (5 pre-existing cold-restore fails).
) opencode 1.x's ONLY host-id seam is experimental.chat.messages.transform (harvested per-session by harvestMsgIdsTransform); the wire body only exists at fetch time. So the alignment runs at the shared fetch chokepoint via a new state.beforeSend — proving the harvested turn POSITIONALLY against the ACTUAL wire (alignedMsgIdsForWire), never a verbatim copy of the stored list: a same-session A-transform/B-transform/A-fetch interleave degrades to content-derived instead of mis-pinning. - anthropic-wire only; a turn pins only when every message lowers 1:1 to one core (pure-text user/assistant) — tool/reasoning parts or any count/role/ text drift drop the whole list. The server re-proves count parity + canonical shape and strips the field before any upstream forward (#2821). - Also feeds the shared watermark slot (pfCaptureIds) with the SAME proven list — this replaces the DEAD call that sat in harvestMsgIdsTransform (it fed host objects {info,parts} to a wire-shape proof, so it could only ever clear the slot; V1 watermark was count-only with no tail-ids). - Guards: never restamp an incremental-lane (diverted) turn, never clobber an already-present x-bili-msg-ids (a V2 request stamps its own), budgets match the server parser (4096 ids / 16KiB). New: tests/opencode-v1-envelope.test.ts (9).
…6539) Boots a real startServer + fake upstream (no agent) and drives the exact envelope the dsh/pi/omp/opencode clients stamp: A. normal plugin lane — biliMsgIds pins every host id into the store (the plugin snapshot's rawIds come back as the host ids, not content digests) and the field never reaches the upstream. B. verbatim (bypass) lane — the same stamped body is scrubbed before the forward (the F1 review fix), so biliMsgIds never reaches the upstream. 2/2 pass; typecheck clean.
The dsh/pi/omp full-resend envelope (biliMsgIds) only covered the openai and anthropic wires; a /responses request (body.input) safely degraded to content-derived ids. This adds the responses dialect so all three host agents carry host msgids on every wire the proxy can pin. - pi-msgids.ts: laneMsgIdsForResponsesWire + respWireFingerprint/respUserParts (lockstep pairing via the shared makeEntryTake/partsLowerThrough; conservative bail on mid-history system, opaque items, reasoning, and any item the kernel would split into N cores). coreAlignEnvelopeIds now accepts "responses" and proves 1:1 against the kernel's responsesToCore wireIndex — the perEl check is the final no-mis-pin safety net. - dsh-native.ts / pi.ts: resolvers accept the responses wire (wire = parsed.input). - tests: 11 new cases (1:1 tool round, lead-system hoist, multi-part/mid-system/ opaque/reasoning bails, dup-id bail, resolver pairing with and without lead). Regressions green: pi-omp 34/34, dsh 15/15, opencode-v2 10/10, fork-track 8/8, msgid full suite, wire e2e google 4/4 + responses 13/13.
… write) Implements the snapshot-table compression foundation from discussion snap_rows / snap_log / snap_ckpt, schema v12), the single-transaction commit primitive with S4 checksum verification, and the P1 shadow write that derives the plane from final DB state after each ingest. P0 (zero behavior change, flag default OFF): - schema v12: the four plane tables + idx_l1_ledger, created by migrateSchema (from < 12) and fresh installs; purely additive, rollback = DROP the four tables + user_version rewind (T1 pins the byte-identical legacy dump in both directions). - src/storage/snap.ts: the commit primitive (applySnapCommit / applySnapInTransaction) — events apply in order, one snap_log entry per event with the post-apply S4 aggregate, seq strictly monotonic past max(log max, ckpt_seq), optional fold into snap_ckpt (three- piece meta: kstate verbatim + coverage mapping + S4 aggregate, log truncated at the fold). Read-side rebuild (readSnapView) recomputes the aggregate and kills the plane for the sid on any mismatch (R1, sticky). L1 identity ledger (snapL1Record): issue/arm/echo/hit/wait, hard-refuse on a raw-id → ledger conflict (R3). GC releases rows only after their cover vanishes (invariant iv). - two-level flags: storage.snapBase.enabled (master) ⊃ storage.snapBase.shadowWrite (lane), config-file only (#2030), resolved in knobs (fileSnapBase) and re-asserted at the store. - /__bili/status gains a snaplane field (flag state + counters + dead latches); sessionStorageStats gains the plane dimensions. P1 (shadow write, read side still never serves): - after each committed ingest, one store transaction records the L1 decision and applies the derived events: steady-state delta appends take a fast path (no full scan); everything else is a final-state diff over live rows + block coverage (cast/update/ delete/compact, pos-ordered, deterministic). A round that cannot be addressed produces no events (R2); a crashed gap self-heals on the next save because the derivation is a final-state diff. - fold at 128 log entries or on any compact; the folded plane is checked against the legacy view (A/B oracle) — a mismatch kills the plane for the sid. Covered positions are legitimately inactive in the plane and are accounted for in the ckpt coverage map. - deleteSession cascades the four tables (no orphans, T6). Tests: tests/issue5874-snapbase-p0.test.ts (T1/T1b/T2/T2b/T2c/T2d/ T6 + flag nesting) and tests/issue5874-snaplane-p1.test.ts (T3 B1 trap nail / T4 A/B incl. crash gap / T7 coverage map / T9 bounded growth). Schema-head assertions in the existing migration suites bump 11 → 12. Timing dependency (#5855): the unmerged fastpath re-land also numbers its two table groups v12/v13; whichever lands second renumbers its block mechanically (independent CREATE IF NOT EXISTS shapes). mysql backend: the plane is sqlite-only for now (the async backend opens without the flags). Refs discussion #5874.
…n open sqlite files (#5874)
…5874) snapGcCoveredRows removed rows outside the commit primitive without updating the anchor they aggregate over: after an ordinary host-side rewind (raw-tail historian sweep retires rows, the diff flips them active=0 with cover_ref='deleted'), the immediate release left the last log sumAfter covering rows that no longer exist, so the next readSnapView tripped R1 on a healthy plane and the sticky dead latch kept the session's plane closed until process restart. Removal now re-anchors in the same transaction (a pure-reanchoring fold at the current log tail), preserving the intended release timing. Also adds the missing P1 fast-path coverage: every prior P1 test saved through planIngest(full), so snapFastAppendEvents never ran in CI. The new suite drives the real watermark/delta machinery (UnifiedShadow.record): steady appends ride the fast path ('append' kinds), and a rewound tail voids the delta, converges byte-for-byte through the diff, and keeps the plane alive across the release round (regression pin for the R1 latch).
The derivation reads final DB state, so how the legacy rows arrived (complete log vs delta) is irrelevant to it; the param was threaded but never read.
Temporary soak change (revert before merge per owner instruction): when the config file is silent, the snapshot-table plane now defaults to enabled + shadowWrite (was off). Every hermetic test and the local real-traffic gate therefore runs the shadow write path on all sessions; the config file can still pin it off explicitly. Spots to revert (all marked 'SOAK' in comments): - src/storage/store.ts: field default + open() fallback - src/server/admin.ts: snapplane status default - tests/issue5874-snapbase-p0.test.ts: default-expectation asserts - tests/issue5874-snaplane-p1.test.ts: crash-gap store pins OFF
Per-table storage relationship map for the snapshot-table foundation (#5874, this PR's schema v12): every table card shows what it stores, when it is written, who reads it, and which feature it serves — ingest, shadow-write, fold, egress (legacy today / plane in P2), hydration, deleteSession cascade, GC, and the /__bili/status surface. Registered in docs index.
…ries The design doc pair (proposals/snapshot-compression-foundation) moves from the proposal branch into the implementing PR, per the docs-centralization standard: - bilingual README.md / README.zh-CN.md + 4 arch diagram pairs (svg+png) + the arch-migration generator script - status line updated: P0+P1 implemented in #5901 (schema v12, shadow-write soak, zero egress change); P2 egress cutover still pending owner ruling - §5 version-stacking corrected to reality (landed v12 on v11 master while #5855 is still open; later-landing side renumbers mechanically) - §9 keys finalized (storage.snapBase.enabled / .shadowWrite, config-file only) - §12/§13 notes: P0+P1 shipped as one PR; #5855 prerequisite not waited on - cross-links to the per-table map docs/storage-relations.{en,zh}.svg - registered in docs/index.md + docs/index.zh-CN.md
…AGE_SCHEMA_VERSION (#5874)
…th (#5874) docs-sync red on fb17874: the committed generated rows claimed default 'on (soak; off in release)' while the canonical seeds (website/config-reference/process.yaml) say default false — the generated artifact had drifted from its source. Rerun tools/gen-config-docs.mjs generate and commit its output (2 rows per file, en+zh). Note for reviewers: the #5901 soak (dab9292) makes the code default ON when the config file is silent until the owner-instructed pre-merge revert; this doc describes the seed state, which equals the post-revert code.
…hema versions Field incident (2026-11-08, shared index.db): the unmerged #5855 re-land consumed schema v12+v13 (pf_* planes) into a database this branch's v12 block had never seen. migrateSchema's early return (from >= STORAGE_SCHEMA_VERSION) skipped the DDL, so the first save died with 'no such table: l1_world' and every session's plane was marked dead (R2, by design — but the root cause was the version gate). The DDL is pure CREATE ... IF NOT EXISTS (no backfill, no rewrite), so it is safe at any from-version: ensure the four tables by TABLE-EXISTENCE PROBE before the early return (the discipline v2/v5/v6/v8 already use), instead of gating on user_version < 12. Both merge orderings now coexist; the final master version number stays a mechanical merge-time hygiene step. user_version is never downgraded (the early return leaves it alone). T1c pins the exact field state: v13 database with sibling pf_* tables and no snap tables → open → snap tables created, version stays 13, sibling rows untouched, legacy dump byte-identical, plane functional. 14/14 suite green, 35 storage regressions green, tsc clean.
…CI)
readPkgVersion is polled every 500ms by waitFor in the registry/advisory e2e suites; a poll landing inside the updater's remove-and-reinstall window hits a transiently absent package.json and the raw readFile throws ENOENT out of the predicate, killing the subtest at ~1.6s despite a 60s budget (ubuntu red job 128697545236; windows green on identical code = race). All four copies of the helper now return null on ENOENT ('not yet', not an error); one-shot assert sites still fail loudly on absence.
…-for-byte guard (#5874 P2) The outbound compressed context is now assembled from the verified snapshot plane (mapEgress) instead of the in-memory history, on all four protocol lanes, guarded by a tripwire: the plane view is built first (anchor summaries at each covered region + surviving rows + the client tail, the same seam walk as the inbound view), the lane's exact post-processing is reapplied, and ONLY a normalized exact match switches the outbound bytes. Any mismatch keeps the legacy bytes; after three mismatches the session is sticky-excluded for the process lifetime. Every degrade reason surfaces on /__bili/status under snaplane.egress. - new src/server/snap-egress.ts; wired into prepare-{openai,anthropic, google,responses}.ts right after the legacy processedMessages - store accessors: UnifiedStore/UnifiedShadow/SessionStore.snapEgressView - config: storage.snapBase.egress / egressCohorts (soak default ON at 100%, release cut flips to opt-in) - admin status wire: snaplane.egress {cutover,fallback,planeMiss, excludedSids,degradeReasons} - tests: 16 new cases (view assembly incl. reply-lag/rewind/remint/gap seams, tripwire normalization, anchor rendering parity, gates + sticky exclusion); docs en/zh updated (configuration, proposal record, index)
…stem role (type check over tests)
…ed prefix rows The client harness wraps every resent message in a per-RESEND envelope tag (<acp tokens=.. time=..>mNNNNN</acp>) whose time attribute changes each turn, so the snapshot plane can never store it byte-for-byte. That made the tripwire fire on every folded dsh turn (shape@0:text) even though the plane content was identical, and sticky-excluded those lanes from the cutover. The egress view now pairs each surviving plane prefix row with the identity- matched message from THIS resend (id first, then envelope-stripped signature) and adopts its exact tagged text, but only when the pair is content-equal modulo the envelope — otherwise the plane text stands and the byte-strict tripwire judges it. Envelope changes between turns still seam-match because both bySig indexing and the seam walk compare envelope-stripped signatures.
…, lock sync, internal types - website/config-reference/process.yaml: seed storage.snapBase.egress / storage.snapBase.egressCohorts and regenerate docs/configuration*.md (config-surface-guard, docs-sync) - reference/architecture.md: register src/server/snap-egress.ts (architecture-docs) - package-lock.json: sync the kernel workspace to 0.0.189 (lock-sync) - snap-egress: drop export on the file-local SnapEgressDeps / SnapEgressStatusWire interfaces (unused-exports)
…-refs pass over the plane view The <acp tokens type time>mNNNNN</acp> tags are NOT client resend data — they are system metadata rendered by the kernel's render-refs node (the LAST pipeline node) over turn.messages. The earlier per-resend envelope adoption was therefore the wrong model. The plane view now reaches byte parity by re-running the kernel's own renderer (renderVisibleRefs with the deterministic default counter) over the assembled view, using the same post-turn state the lane already holds: refs from state.messageRefs, immutable first-sight times hydrated from state.tsStamps, strategy from the lane's renderTags knob, timestamp mode from resolveMessageTimestamps (env > file > default ON). Render throws degrade honestly (render-error, legacy bytes kept). Lanes pass renderTags; T18a/b rewritten to assert kernel-render parity and first-sight time stability; T18c retitled to the content-edit divergence case. 19/19 + regression suites green.
…-body M2 retries no longer wipe the plane, and the first chained round seeds its frontier at the stored-world boundary (gated on a trusted, never-refused chain) - Residual 1a: an empty delta (frontier already covers the world — the host retried the same body after an upstream failure) is now classified as a NOOP: the chain id is kept, no developer message is injected, and the empty input rides the provider's own cache. Before: id stripped + dev injected = a blind response the host would chain from forever. - Residual 1b: stage 0 recognizes a full replay (every incoming id already in the stored canon, carrier seqs at/below the plane frontier) as echo+replay — zero side effects. Before: an M2 retry ([m5] vs head m1) read as headMoved -> diverge -> snapStage0Reset: a harmless retry wiped the plane. - Residual 2: the first CHAINED round of a keepResponseId chain carried no committed frontier (round 1 had no id), so it re-seeded the whole world the provider's chain already holds from this proxy's own round-1 full view (2x the world's tokens, one time). Stage 0 now proves the safe case (probe + nothing gone + new elements -> the chain provably holds the stored world) and seeds the frontier at the stored-world boundary; everything else keeps the plain re-seed. - B4 hardening: a REFUSED selection round now sets a persisted responsesChainBroken marker (committed with the session). While set, the frontier seed is blocked and the round re-seeds the full view — the in-request flush wrote the refused turn's rows before the forward, so the plane holds more than the provider's chain and trusting storedFrontierPos would lose the refused content. The flag clears on a full-view/noop acceptance; the pending delta now carries its kind (full/noop/partial) so the relay can maintain it. - Tests: p3 42/42 (new: residual-2 unit shape pin, residual 1a+2 e2e through the proxy, A1/A1d legs now use isolated DBs — the two legs are ALTERNATIVE client behaviors on the same turn and sharing a file let leg M1's canon contaminate leg M2's verdict). - Regression: p0 10, p1 4, p2 17, fastpath 2, snapshot-render 13+6, snapshot-shadow 5, storage-engine-proofs 12, opencode-rolling 10, responses-previous-response-id 7 — all green.
…a delta frontier only for an id this proxy minted (#6181) previous_response_id proves nothing about which world the provider's chain holds. A host forking/rewinding to an older response id (or carrying an id another proxy minted) can pass the strict-extension proof yet lose the gap between the old chain's world and the stored world when we seed at the stored boundary. - session.lastMintedResponse {id, frontierPos}: the latest response id THIS proxy emitted for the session, paired with the plane frontier the chain provably held — recorded in the relay from the id observed on the outbound stream (SSE response.created/completed scan on the exact bytes written to the client, json.id for non-streaming), accepted forwards only (a refused round mints nothing the client chains from). - store facade snapPlaneMaxPos (store/shadow/persist) — the lane records the post-flush plane max pos as prepared.mintFrontierPos. - seed gate: the first-chained-round frontier seed now also requires lastMintedResponse.id === previous_response_id AND lastMintedResponse.frontierPos === storedFrontierPos. - persisted-frontier trust: a chained round whose id this proxy did not mint last drops the persisted frontier and re-seeds the full view (id kept, dev injected — the pre-seed status quo, at-least-once, never a loss); the persisted state is cleared so the next round re-establishes verifiability from a fresh seed. - B3 latent bug: the pure-chain-continuation route reuses the compact-passthrough base, which hard-codes resetAfterSuccess — scheduling a native-compaction rebase (resetSessionCompression) for the next turn although nothing compacted. Dropped for the B3 route. - p3: per-round upstream ids in the e2e harness; the Residual 1a+2 e2e chains from the ids the responses actually carried and gains a round-4 fork case (old id + full current world + new element → full re-seed, id kept). 41/41.
…ate) The Residual-2 hardening commit added a raw mintSseTail.slice(-32768) which the text-safe family gate (tests/text-safe.test.ts) rejects — raw negative string slices reintroduce the #816/#1615 lone-surrogate family. Replace with safeSuffix(mintSseTail, 32768) (same last-32768-code-units semantics, surrogate-pair-safe). safeSuffix was already imported.
… rule) + mint tap spin Two regressions surfaced when the branch was re-based onto the #5901 head (f000197, which carries #6326 durability-before-render): 1. B4 e2e: #6326's ingestRowsNow persists the ledger rows BEFORE render as an UNTAGGED save; its final-state diff then deleted/re-homed the pinned plane rows (ids the tagged plane cannot address — R2 doctrine) and the next tagged save marked the plane sticky-dead. snapShadowSync now guards untagged saves against a sid that holds active plane rows with a PREFIX RULE: the save is allowed only when its ledger-id view prefix-identically extends the stage-0 l1 envelope (that save is the one that materializes the rows stage-0 declared — ids agree, idempotent); a stale view (re-derived kernel ids, pre-turn transcript, compaction) is skipped for the plane, the disk record still lands, and the plane converges at the next tagged save or stage-0 resync. The guard runs BEFORE the transaction (after it the destructive diff is already committed). 2. Residual-2 e2e: the mint-id scan (the record that lets the next chained round seed the delta frontier at the stored-world boundary) existed only in the compress-loop (injected) branch; plugin and non-injected responses turns never entered it, so the first chained round re-seeded the whole world. The scan is factored into MintIdTap and rides a tee'd twin of the plugin-lane stream and of the plain (non-rewrite) proxy stream — both branches where the client-facing bytes ARE the upstream bytes. The tap fix also repairs a latent infinite-spin in the old inline scan: the event-boundary recompute ran only at the loop bottom, which the non-marker skipped — a non-marker event preceding a marker event pinned the boundary on a shrinking tail and spun the loop forever (event loop blocked; no timers fired). The recompute is now unconditional on every pass. p3 42/42 (incl. B4 frontier-commit-on-accept + Residual 1a+2); p0 10/10, p1 4/4, fastpath 2/2, p2 17/17, responses-previous-response-id 7/7, snapshot-render 13/13, render-chain 6/6, shadow 5/5, storage-engine-proofs 12/12 (kernel rebuild for #5901's compress.ts), opencode-rolling 10/10.
…wline in the #5901 merge The 941447a merge joined the egress-cohort desc_ja value with the next '- key: sessions' entry onto one line, which tools/gen-config-docs.mjs rejects (quoted value not valid JSON) — docs-sync + docs-coverage red. Split the line back; generator checks pass (coverage ok: 204 env + 63 keys; i18n ok: 254 keys).
… v11 test title, comment indent - pr-body-6299.md: #6299's PR-body draft (already merged) was committed at the repo root — authoring artifact, not repo content (violates the docs layout: root keeps README*/AGENTS/CHANGELOG only) - tests/storage-ingest-watermark.test.ts:594: title said 'migrates to v11' while the body has asserted v12 since this PR bumped STORAGE_SCHEMA_VERSION - src/server/prepare-openai.ts:204: comment block started at column 0
…6480's pure-sqlite default flipped the absent-selection JSON writer off, and these suites assert the legacy JSON persistence surface (session files / cold-disk-restore content-store reads). First exposure: the ci:full gate ran them on the #6480-merged tree; reproduced on bare master, fixed by the explicit dual-write pin. #6480
… the CI storage pin in notice-registry (master #6480 regressions) Two master-side regressions first exposed by this branch's full test shards: 1. fork-ccr-adoption reads the companion session ENVELOPE off disk, but the #6480 pure-sqlite default stops writing JSON envelopes. Pin dual-write in the test (same one-liner as the native e2e suites). 2. The notice-registry suite is hermetic by design (absent config file, fresh XDG) but the CI test baseline pins BILI_STORAGE_UNIFIED=legacy (preload-storage-baseline.mjs, #4011). An explicit selection is a user decision that legitimately hides the fresh-install default-flip notice, so the two 'fresh install sees the default flip' assertions failed on every shard. Lift the pin for this file only. Master merged #6480 without a main-CI run (the push event was dropped), so neither regression had been seen in CI before this branch.
… pin (Windows EBUSY/EPERM) persist-metadata-fold-gate.test.ts to keep the legacy envelopes their assertions read. unified=full opens index.db, and Windows cannot unlink a file whose SQLite handle is still open — both suites rm'd their temp dirs without closing the stores, so the windows-latest shards failed deterministically (EBUSY unlink index.db / EPERM rmSync tmpdir) while every ubuntu shard passed. Close the stores (closeQuietly) before rmrf. Same master-side regression class as the notice/pin fixes in 7b6eb98: master merged #6480 without a main-CI run, so none of these had been seen.
…t-off revert, test coverage, docs - B0: on non-cohort chained bypass rounds the in-request flush was the only consumer guarded by the one-shot skipPlaneFlush; the relay success path re-arms the debounced save and that save rode turn.messages' DELTA view as a tagged world — snapViewEvents (final-state diff) deleted every plane row the delta lacks. The lane now clears session.lastPlaneMessages on bypass rounds, so later saves derive untagged from the DB final state (append-only rows — correct). - Default off (discussion #5874 section 9): the 2026-11-08 100% soak default is reverted in all four spots — store.ts field + open() fallback, admin.ts status payload, snap-egress.ts gate (egress now opt-in, v?.egress === true) — and the p0 flag test now pins default-off. e2e suites arm the flags explicitly via a dedicated config file. - Test gaps from the review: T2 covers the in-transaction rollback leg (valid append + R2-unknown update in one batch => nothing lands); T9 raises N 40->160 so SNAP_FOLD_DEPTH (128) actually fires and asserts a fold happened (ckpt_seq >= 1) — at 40 saves the bound was vacuous; the byte-identity oracle's LEGACY_TABLES now includes kv_store (was 11 of 12); p2 G2-G5 set egress: true explicitly (they had relied on the soak default). - Docs: process.yaml egress default false + 3-lang regeneration; proposal READMEs (EN/zh) status line, DDL block (actual v12 columns incl. envelope / r3_latched_at), probe-first migration note, stage-5 row, section 9 key list, section 10 metrics all updated to the direct-authority + default-off state; config.ts schema comment; stale comments fixed (session.ts lastPlaneMessages, handle.ts B3 string-input admission, snap.ts l1 upsert, p0 T6).
The #5901 rebase auto-merge kept BOTH ../persist.js imports: master's standalone 'import { getStore }' (added for the #3397 ingestRowsNow durability-before-render call) and this branch's 'import { SessionStore, getStore }' (added for the in-request SessionStore.flushOrSnap plane path). Both are needed symbols (getStore at the ingestRowsNow + budget calls, SessionStore at the flushOrSkip call), so the superset import stays and the redundant standalone getStore line is dropped. Fixes TS2300 duplicate identifier surfaced by the post-rebase typecheck.
…seeds (docs-sync guard) The #5901 rebase resolution of website/config-reference/process.yaml dropped the desc_ja lines for storage.snapBase.enabled and storage.snapBase.shadowWrite. Master's gen-config-docs.mjs now requires desc_ja on every seed entry, so the docs-sync CI job fails at the 'generate' step (exit 1) before the drift check. Restored both lines verbatim from the pre-rebase PR head; generate now exits 0 with no drift and docs-coverage stays green.
…— pre-merge gate:local + full-CI validation # Conflicts: # src/server/prepare-responses.ts
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9939-pr.6680.1f2cfd0This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr6680.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 查重 (dedup): Not a duplicate. This is the unique pre-merge integration vehicle combining the two open PRs #5901 (snapshot plane) + #6564 (msgid delivery). No other PR/issue does this combined validation. 标签: What I did — independent validation of the combined tree at head
gate:local / real-agent note: the body's "gate:local in progress (Lanes A/B/U)" is the operator's local last-mile real-model gate (needs real 8问 (结构化结论):
判定: 合并树在我能本地验证的范围内全绿(编译 + 241/241 定向测试 + CI gate 已过、0 失败),合并解析正确且与顺序无关。等剩余 9 个 CI guard/docs 车道转绿后,owner 可按任意顺序合并 #5901 和 #6564。我会确认最终 CI 状态。 |
🔶 需要决策 背景: 合并验证树(#5901 快照平面 + #6564 msgid 递送,head 待决问题: #6564 的 选项(各带代价):
推荐: A + 测试微调。252/270(标记丢失)是真回归(缓存标记应保留,与身份方案无关);287(ref 漂移)像测试需按 KDD #13 更新。 不决定的默认行为: #6564 CI 红不可合并(owner-only merge);验证载体使命(证明两 PR 一起绿)未达成,需先处置 #6564 回归再重跑。 CI verdict (full detail): The 11 test shards finished: 3 failed (ubuntu 22/2, ubuntu 24/2, windows 22/1), all on the same 3 tests in
Root cause (verified locally): #6564 flips
NOT an integration conflict — it's a #6564 regression from the flag GA. #5901 neither causes nor fixes it. 6 extra Windows failures = flakiness (not a regression): windows 22/1 also showed 6 failures in 8问 (结构化结论):
判定: 合并树未全绿(3 红,根因 = #6564 flag GA)。验证载体使命未达成。需 owner 决策(修 bug / 更新测试 / 回滚 GA)后重跑。不合并、不关闭(owner-only)。 |
… the msgid GA (#6539) The pluginLaneServerHistory GA (this PR, 0% -> 100%) turned on the canonical-id/seq stamps on the incremental lane's re-entry body for every plugin session. When that body carries the client's rolling cache_control marks (opencode's cacheControlRetouch channel), the stamps make the kernel's first-sight pass re-render the marked content into <acp>mNNNNN</acp> markers, which drops the wire-level cache_control breakpoints and drifts the ACP refs. Gate the stamps on the mark channel: coreToAnthropic splices entry.marks into the assembled items, so a non-empty marks map is the precise signal that this turn's body carries marks. Mark turns now re-enter exactly as they do with the rollout off (no stamps) — the pre-GA behavior that always passed — while non-mark turns keep pinning. The lane still owns the canonical ids. Tests: opencode-rolling-cache-marks 12/12 (was 9/12); msgid/envelope/fork 148/148; incremental-lane suites green; dsh real-machine envelope e2e ALL PASS.
integrate: #5901 (snapshot plane) + #6564 (msgid delivery) on master
DRAFT — validation vehicle, not a merge target. This branch merges the two
open PRs (#5901 + #6564) onto the latest master to run the full remote CI +
local
gate:localagainst the combined tree, BEFORE either is merged. Itproves the two land together cleanly and green; it is not itself what gets
merged (merge = owner, per repo rule).
What this is
origin/master@ 9495c76.#5901(2026-11-07_snapshot-table-foundation, head eb0670d) = the snapshot-tablecompression foundation (P0 schema v12 + four tables / P1 shadow write / P2 mapEgress
direct authority + P2-final six-stage chain, all flags default OFF).
#6564(2026-11-14_msgid-proposal-6539, head 52e54c7) = msgid delivery acrosssend/record/use for dsh/pi/omp/opencode × all protocol wires (proposal 提案:msgid 全生命周期「发 + 记 + 用」补齐(车道无关 × 全协议) #6539).
src/server/prepare-responses.ts—[PR] msgid Delivery: Proposal #6539 Full Implementation (Publish·Record·Use, dsh/pi/omp/opencode × Four Protocols) Consolidat | msgid 递送:提案 #6539 完整落地(发·记·用,dsh/pi/omp/opencode × 四协议)合成一个 PR #6564's inline responses-wire pin was subsumed by [PR] storage: snapshot table compression foundation P0+P1 (schema v12 4-table + shadow write, all off by default, zero behavi | storage: 快照表压缩底座 P0+P1(schema v12 四表 + 影子写,默认全关、零行为变化) #5901's
runStage0(samevalidateMsgIds+applyMsgIdPinunder the samepluginLaneServerHistorygate);the auto-merge had also left a duplicate import + duplicate
const incomingMsgIds,both removed. Resolution keeps the seqs-aware read and routes
incomingMsgIdsthrough
runStage0. No double-pin; pinnedmsgsflow to the kernel/store viarenderMsgs = s0.kernelInput ?? maybeSnapshotView(...).Local validation on this branch
tsc --noEmitclean.snapshot-render + responses/google e2e + hermetic msgid-envelope e2e: 111/111.
msgid track fires (
drifted).gate:local: in progress (Lanes A/B/U).Why a draft, why not merge this
Two open PRs each need their own owner merge (repo rule: no non-human merges).
This branch only de-risks the combined rollout: if both land green together
here, the owner can merge them in order (#6564 → #5901, or #5901 → #6564 — this
branch proves order-independence of the tree).