Skip to content

[PR] Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table - #6155

Open
ranxianglei wants to merge 10 commits into
masterfrom
2026-11-11_snapshot-foundation
Open

ranxianglei wants to merge 10 commits into
masterfrom
2026-11-11_snapshot-foundation

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What this is

This lands the snapshot-table compression foundation — the formal, complete
version of the incremental-compression bridge that today only works as a
stopgap. In one sentence: the compression history (the "fold ledger") now
lives in server-side tables, and the outgoing request prefix is built from
those tables instead of being re-derived from the bytes the client happens to
re-send.
That makes the prefix byte-stable across client byte churn, which is
the whole point of prompt-cache-friendly compression.

This implements the full proposal in #5874
— all six stages, not a partial P1/P2 slice:

raw bytes → protocol convert → rewrite (incl. compression) → ingest into the
snapshot tables (LSM) → outbound rewrite → out

Why

The current compression path re-derives what to send from the client's resent
history every turn. If the client serializes a message slightly differently
(one whitespace, a reordered field, a re-encoded image marker), the derived
prefix changes, the prompt cache misses, and the fold we already did is
wasted. The foundation pins the ledger server-side: each logical message gets a
stable position, the table holds its canonical form, and the prefix is rendered
from the table. The client converges to the canonical form in one turn on any
mismatch — it self-heals and never bricks.

How it works (the six stages)

  1. Identity — the host's per-element message-id envelope (biliMsgIds,
    the Plugin lanes: pin identity to client msgids, render outbound from the server-owned history | 插件车道服务端正史:消息身份钉在客户端 id 上,出站前缀由正史渲染 #4491 channel) is matched against the standing "world" canon: echo /
    probe / scatter / issue / diverge / resync.
  2. Convert — protocol → common core (the public kernel codec, unchanged).
  3. Rewrite — the kernel fold is unchanged (zero kernel edits). A fold
    produces one compact event; the plugin endpoint commits it in-request and
    the next model request reconciles the tail.
  4. Ingest — a single-transaction commit primitive (new storage schema v12:
    l1_world, snap_rows, snap_log, snap_ckpt). Rows are append-only
    (a position is never reused); pre-fold bytes ride the content store; the
    checkpoint fold is the bounded-lifetime boundary (no global TTL).
  5. Egress — active rows in host order ++ this turn's new cores, then the
    same downstream the legacy path uses (nudge / carrier / codec). Byte parity
    with the legacy path is pinned by an A/B corpus test.
  6. Out — a low-ratio "tripwire" samples and byte-compares against the
    legacy view; a mismatch yanks that session back to legacy and raises an
    alert.

Safety (it cannot brick a client)

Every failure mode is a per-session, fail-safe red gate that degrades to the
existing legacy path — never a crash, never a half-state:

  • R1 checksum/aggregate mismatch → that session's plane is permanently
    marked dead and it runs legacy.
  • R2 the egress join chain can't be reconstructed → no events this turn,
    the whole turn runs legacy (next turn re-derives).
  • R3 an enrolled session arrives without a valid envelope → hard 400,
    zero side effects, and a breaker latches it to legacy after repeated
    faults.
  • R4 a structural marker is lost (e.g. a restart mid-commit) → the session
    is conservatively held to legacy until it re-baselines.

Rollout (off by default)

Two-level nested rollout using the existing base mechanism:
snapshotFoundation (master, 0%) contains snapshotLane (100% within
the master). So flipping the master gate is the only knob that turns the
feature on. The cohort is a config list (snapshot.cohortHosts, default
["pi"]). The foundation is a full-resend msgid-envelope host feature: a
host only participates if it sends the per-element id envelope. Hosts that
don't (for example the dsh CLI) are simply not enrolled and keep running the
unchanged legacy path — so enabling the feature can never break them.

Verification

  • 64 hermetic tests, all green:
    • A/B byte-parity corpus through the real pi plugin lane (the prefix is
      byte-identical to the legacy path, modulo the timestamp attribute).
    • Logic: world match, reconcile, hydrate, lane decision.
    • Storage: v12 upgrade pin, DROP-rollback drill, commit atomicity, both
      backends.
    • Decompress: a dead block's content is served from the covered rows; after
      garbage collection it returns an explicit audit receipt (never a silent
      empty body).
    • Growth: rows / log / checkpoint stay bounded by the checkpoint cadence
      (the 0.1.200 → 0.1.201 upgrade (#3658 identity change) mass-unfolds persisted sessions incl. Anthropic lane; downgrade re-churns #3752 "unbounded growth" red line).
  • typecheck clean; all CI guards pass (unused-exports, test-import-graph,
    lifecycle-ledger, config-surface, lane-guard, version-guard).
  • Local release gate (gate:local) runs the foundation at 100% (master +
    lane on) — see scripts/release-gate-local.mjs.
  • No kernel changes (acp-kernel is untouched).

Scope note

This PR targets the pi plugin lane (the host that sends the msgid envelope
and full-resends). Other lanes (anthropic wire, dsh native, non-plugin) keep
their existing behavior — the foundation yields to them rather than double-
handling, per the proposal's "each flow has one home" routing. Extending the
cohort to additional msgid-envelope hosts is a config change once their
envelope is confirmed, not a code change.

@ranxianglei ranxianglei added the ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR label Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-11_snapshot-foundation (555ea1b)

Latest build — install this exact version

npm install -g billion-context@0.0.10558-pr.6155.f12338b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
555ea1b billion-context@0.0.10558-pr.6155.f12338b build
5850ee0 billion-context@0.0.10484-pr.6155.d1a3684 build
48979d0 billion-context@0.0.10188-pr.6155.71d4da5 build
66a342e billion-context@0.0.10154-pr.6155.3111b4d build
6bdbf56 billion-context@0.0.9536-pr.6155.838bc51 build
ee69344 billion-context@0.0.9299-pr.6155.e4627ec build
14e5afb billion-context@0.0.9039-pr.6155.31cd182 build
37b0801 billion-context@0.0.9026-pr.6155.1a5a712 build
e17f18c billion-context@0.0.9000-pr.6155.fb99325 build
05806d9 billion-context@0.0.8999-pr.6155.f2485f5 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6155.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table [PR] Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table Nov 10, 2026
ranxianglei pushed a commit that referenced this pull request Nov 10, 2026
…#6153)

The test file and its header cited #6155 — a number that on GitHub
belongs to an unrelated thread (snapshot-table compression foundation).
Cross-references in PR artifacts must use GitHub numbers; the source bug
report for this fix is GitHub issue #6153. Pure rename + header fix;
no behavior change.
@ranxianglei ranxianglei added 马上合并 enhancement New feature or request labels Nov 11, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

Verification conclusions (#5874)

CI (full matrix, head 14e5afb1): GREEN — all 35 jobs of the CI run pass,
including every test shard (ubuntu 22/24 + windows 22/24), all guard/static
jobs (architecture-docs, docs-sync, docs-coverage, config-surface-guard,
unused-exports, test-import-graph, lifecycle-ledger, lane-guard,
version-guard, lock-sync, mysql-contract against a real MySQL 8.0, …):
https://github.com/ranxianglei/billion-context/actions/runs/42941675192

E2E lanes triggered by the ci:full label: real-codex, fake-codex,
codex-persona, pi-codex-ws, dsh-plugin-lane, native-dsh, native-opencode,
local-registry, theorem — all pass. One lane, e2e-native-pi, failed
twice on the same two tests (#2791 compressCalls 1 vs 2). That failure is
pre-existing, not a regression from this PR: the identical two tests fail
identically on a run of the base commit 58a87962 without any of this
branch's code (run 42887264198, job 130078824385). A rerun of the failed job
is in flight.

Feature-on evidence (the 100%-on requirement)

The foundation is verified fully enabled in two places:

  1. Hermetic A/B inside the CI matrix (tests/snapshot-foundation-ab.test.ts):
    two real proxy instances, one with the foundation master+lane rollout
    forced on (BILI_ROLLOUT_SNAPSHOT_FOUNDATION=1 + BILI_ROLLOUT_SNAPSHOT_LANE=1),
    one forced off — same client, same scripted 6-turn corpus including a real
    fold through the plugin endpoint. Every upstream byte is identical
    (modulo the time attr), the decision tokens read
    arm, hit, hit, hit, hit, hit, the R3 hard-reject path returns 400 with
    zero side effects, and /__bili/status shows the session green with a
    committed checkpoint. It runs under the exact CI preload chain
    (including the BILI_STORAGE_UNIFIED=legacy baseline, which the suite
    pins to full itself — [PR] fix: make full-mode storage tests deterministic (#4004) #4011 lesson).
  2. Local real-model gate (gate:local, config biliConfigFor() with
    rollout.features 100%-on + snapshot.{enabled,cohortHosts:["pi"]}):
    Lane B (model switch via relay, same lineage across the switch and a
    proxy restart, both markers recalled, cache 92.67%) passes. Lane A
    currently cannot complete on this machine's second sglang instance
    (8299): that build answers non-stream summarization calls with ~10k
    reasoning tokens and zero visible text (finish_reason=length, text_chars=0), so the preflight fold never produces a summary, and the
    instance emits no cache report at all. That is model-service behavior,
    not product behavior — in the same run the foundation lane itself is
    inert by design (the gate's dsh client is not a cohort host; the log
    shows zero foundation activity and zero errors from the new paths), which
    is exactly the property being proven: 100%-on does not break a real
    client that isn't enrolled. The canonical gate endpoint is 8199 per
    ~/system/release-gate-local.md; the Lane-A verdict on 8199 is pending
    GPU scheduling (8199 is currently degraded by the co-resident models).

Local test evidence

  • Foundation suite (64 tests: storage T1/T2/T6, decision-layer logic incl.
    the B1-trap ×3-restart oracle, pinned-head/re-pin/fold-only-scope,
    T4/T5 byte parity, T8 decompress green-lane + post-GC honest exit, T9
    growth bound): 64/64 pass under the CI preload chain.
  • npx tsc --noEmit clean; npm run build clean; all guard scripts pass.
  • The 5874 surface is isolated by design: cohort ["pi"] + mandatory
    per-element msgid envelope; every other client/protocol keeps running the
    unchanged legacy path (which is what the A/B phase-A and the whole rest of
    the 1190-file suite exercise).

@ranxianglei

Copy link
Copy Markdown
Owner Author

Final status: all gates green

Local real-model gate (gate:local, 100%-on config) — now PASSES:

-- Lane A: fold + post-fold continuity (window shrunk 262144 → 16384 mid-session)
  ✅ turn 1 completes          ✅ no fold while corpus is fresh (protected)
  ✅ S40 marker answered        ✅ turn 2 completes after window shrink
  ✅ fold fired mid-session (shrink 98%)   ✅ nudge injected (hard)
  ✅ preflight engaged          ✅ S07 marker recalled from FOLDED context
  ✅ cache recovered after fold (hits 0,64%)
-- Lane B: same-session switch qwen3.8-27b → glm-5.2 (different upstream)
  ✅ switch turn completes      ✅ same lineage across switch + proxy restart (forked + adopted)
  ✅ cross-upstream baseline demoted  ✅ both markers recalled after switch
  ✅ request routed to glm-5.2  ✅ new-upstream cache warms up (hits 92,79,99%)
== gate summary ==
A: 9/9 green   B: 6/6 green   elapsed: 230s

The earlier Lane-A failure on this machine was the co-resident sglang build answering
summarization calls with zero visible text; with the healthy instance (cache reports
enabled) the fold, nudge, preflight, post-fold recall and cache recovery all hold.
The foundation lane stays inert for the non-cohort dsh client in both lanes — the
property under test (100%-on does not break a real client that isn't enrolled).

The one remaining red, e2e-native-pi, is a pre-existing break at this PR's base
commit — not a regression from this change.
The decisive evidence: this PR is based
on 58a87962, and the pure-master scheduled runs of that exact commit
(failures of the real pi package-native lane job at 2026-11-10 19:38 and
2026-11-11 00:35, headBranch master, no PR code present) failed the identical two
#2791 tests with the identical compressCalls 1 vs 2 numbers. Newer master commits
pass. So the lane was already red at the commit this branch stands on; the two tests
exercise pi's cross-protocol fold reuse, a path this PR does not touch when the
foundation is off (the e2e env runs the default 0% rollout).

Requirement scorecard

  1. Complete six-stage pipeline, final version — implemented end to end
    (raw bytes → convert → rewrite incl. compression → snapshot-table ingest →
    outbound rewrite → out), plus the P3 consumer switches.
  2. No reference to existing implementations — the decision layer
    (src/snapshot/, 8 modules) is written from the proposal and kernel public
    surface only; the A/B test is an independent oracle.
  3. CI full matrix at the feature's own 100%-on verification, passing — all 35
    CI jobs green (run 42941675192); the hermetic A/B runs the foundation forced-on
    byte-for-byte against legacy; every ci:full e2e lane passes except the
    pre-existing base-commit flake above.
  4. Local gate, same 100%-on conditions, passing — Lane A 9/9, Lane B 6/6.
  5. Verification conclusions in the PR — this thread.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Dup screening: same topic family, not a duplicate — there is one related parallel implementation PR (#5901; see the decision request I'm posting right after this). No other same-signature rows among open/closed mirror or GitHub entries.

Independent live verification at head 14e5afb1 (I had reviewed the previous head e17f18c, which was red — both PR-specific defects are now fixed):

  • CI: mirror cache shows 30 checks, the only failure = real pi package-native lane (e2e-native-pi); all test shards, mysql-contract against real MySQL 8, and every guard job green.
  • Your pre-existing-red claim — verified: on pure master at this PR's exact base sha 58a87962, the scheduled e2e-native-pi runs failed at 2026-11-10 19:38Z and 2026-11-11 00:35Z — both timestamps match your report. The adjacent e2e-native-dsh lane flapped fail→pass on the same sha, which points at environment flakiness rather than code. One honest caveat: no completed e2e-native-pi run exists yet on the newer master sha (33104b1f) as of my check, so "newer master passes" remains your evidence, not mine.
  • Local re-run at 14e5afb1 (clean env): snapshot-foundation-{decompress,storage,logic,growth} + issue4666-migration-dedup = 67 tests, 64 pass, 0 fail, 3 skipped.
  • The two defects I diagnosed at e17f18c are fixed exactly as diagnosed, in 37b0801/14e5afb: (1) T8a/T8b depended on ambient BILI_STORAGE_UNIFIED — without the knob, SessionStore({enabled:true}) opens no unified sidecar, so foundationStore() was null on clean-env CI; the module-top "full" pin is the correct hermetic fix; (2) stale v11 pins in tests/storage-null-identity.test.ts:48/:55 → now 12. I also reviewed the two mysql fixes: the frontier query now takes GREATEST(log.max_seq, ckpt.max_ckpt_seq) — correct, since after a fold the log-only MAX understates invariant ii — and the DDL comment re-wrap fixes MySQL's parse error.
  • Minor observation (report-only): the new config keys are disclosed under "## Rollout (off by default)" rather than a literal "config surface" heading; the substance (what/why/default/migration=none) is present.
  • Labels applied to this PR (judged on its own diff): enhancement + p3 + scope/storage — off-by-default foundation, zero behavior change in default config.

八问(结构化评审)

  1. 是什么问题:这不是缺陷修复,是 提案:快照表式压缩底座(六段固定链,单一存储真源) #5874 提案的完整实现——折叠账本(fold ledger)改由服务端表持有、出站前缀从表渲染而非从客户端重发字节再推导,使前缀在客户端字节抖动下保持字节稳定(prompt-cache 友好)。
  2. 什么原因导致的:不涉及(功能实现)。要解决的机制问题:现有压缩路径每轮从客户端重发历史重新推导前缀,客户端序列化微差(空白/字段序/图片重编码)即导致前缀漂移 → 缓存 miss → 已做的折叠作废。
  3. 影响范围与严重程度:默认配置零行为变化——rollout snapshotFoundation 主门 0%;即使开启也仅限发送 msgid 信封且命中 cohort(默认 ["pi"])的宿主;dsh CLI、anthropic wire、非插件车道结构性不参与。合并时对现网用户无直接影响;灰度区内是全新代码路径。
  4. 怎么修复的(实现策略):六阶段管线 raw→convert→rewrite→ingest(schema v12 四表 l1_world/snap_rows/snap_log/snap_ckpt,单事务提交原语)→egress(走与 legacy 相同的下游)→out(tripwire 采样字节比对);kernel 零改动;所有失败模式 R1–R4 均为按会话降级回 legacy 的红门,不崩溃、不留半态。
  5. 是否有回归问题:我审过的既有行为清单——①存储迁移 v11→v12:纯增量四张新表,旧构建打开 v12 文件忽略多余表(src/storage/schema.ts:98 声明),8 个既有存储测试的 version-pin 已全部更新(含此前漏掉的 storage-null-identity,新 head 已修,本地复跑通过);②mysql 后端:frontier GREATEST 修正 + DDL 注释解析修复,mysql-contract 对真 MySQL 8 通过;③decompress receipt:T8c 钉死「无 substrate 时 legacy receipt 不变」;④wire/消息形状:A/B 语料测试钉死开/关两臂上游逐字节一致(除 time attr),nudge 阈值/输出格式未动;⑤kernel:零编辑,边界保持。残余红 e2e-native-pi 已独立验证为基线 commit 上的继承性失败(master 两次定时跑同挂),非本 PR 引入。如实说明覆盖范围:深读了全部新增测试层、两个 fix commit 与 storage/mysql 路径;未对 49 文件 / 5700 行做逐行全审。
  6. 是否破坏缓存:否——该 PR 的目的就是让前缀更稳;A/B 证明两臂字节一致,本地 gate 报告 fold 后缓存恢复(hits 64%)与跨模型切换预热(92/79/99%);未触碰任何失效策略。
  7. 是否改了设计方向:是(有意)——出站前缀的事实源从「客户端重发历史的再推导」转为「服务端快照表渲染」,即 提案:快照表式压缩底座(六段固定链,单一存储真源) #5874 提案方向的落地(KDD fix: cache health — remove condense, fix isTitleGen, preserve cache_control #13 D 车道快照渲染的基础件);数据流新增 ingest 表→egress 渲染一段,legacy 路径完整保留为降级目标。
  8. 是否引入了新的配置:是——rollout 新键 snapshotFoundation(主门,默认 0%,内含 snapshotLane 100%-within)+ snapshot.cohortHosts(列表,默认 ["pi"]);另有测试/灰度 env BILI_ROLLOUT_SNAPSHOT_FOUNDATION/LANE。迁移方式:无需迁移(默认关;schema v12 增量建表随下次打开库自动升级)。文档已同步 website/config-reference/{rollout,snapshot}.yaml。

Verdict: mergeable — code and behavior verified green at 14e5afb1; remaining gates are (a) the #5901 supersede call below and (b) the inherited e2e-native-pi red on master, which belongs in #4599, not this PR.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

🔶 需要决策

背景一句话:#5874 快照表压缩底座设计现在有两个实现在途 PR,都挂着「马上合并」,但两者从同一 base 58a87962 分叉、在 18 个核心文件上重叠(含 src/storage/schema.ts——两份 v12 迁移不能共存),只能合一个。

待决问题:主线保留哪个?

选项各带代价:

你的推荐之外的默认行为(不决定时):什么都不动——两个 PR 保持 open,合并队列里继续压着两个互斥条目;我不会自行关闭任何一个(关闭权在你)。

(已在本线程打上 needs-decision 标签。)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Merge conflict resolved — merged current master into the PR branch and pushed.

What I did this run:

  • Confirmed the live state: PR [PR] Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table #6155 (head branch 2026-11-11_snapshot-foundation) was merge_state: dirty against base master @ 33104b1f95. The 5 new master commits are dominated by Remove the compressAtFoldPoint feature — it reorders resent history and caused an authorship-misattribution incident in a live session #6132, which removes the experimental compressAtFoldPoint fold-point-relocation feature across kernel / host / web / docs — and the PR branched from before that removal.
  • Merged master (33104b1f95) into the PR branch (a merge, regular push — not a rebase). Git's 3-way merge auto-resolved with zero textual conflicts: the PR's snapshot-foundation additions and master's fold-point removals sit in non-overlapping regions of the 7 shared files (src/config.ts, src/server/prepare-openai.ts, src/stream.ts, src/web/client.ts, src/web/i18n.ts, docs/configuration.md, docs/configuration.zh-CN.md).
  • Verified the auto-merge is semantically sound, not just textually clean:
    • No orphaned references to the removed feature in host src/ — the only remaining compressAtFoldPoint strings are master's own intentional INERT_REMOVED_COMPRESS_KEYS deprecation list (src/config.ts:2814-2818) and the docs/lifecycle.md:239 ledger entry.
    • The snapshot-foundation feature is fully intact (src/snapshot-plane.ts, src/snapshot/*, rollout gates all present).
    • npm run typecheck (builds kernel + pi-subagents, then tsc --noEmit over src + tests) → exit 0.
  • Pushed the merge commit 0aade4d81b.
  • Git-level proof the conflict is gone: git merge-tree of base 33104b1f95 into new head 0aade4d81b → exit 0 (clean); and 33104b1f95 is a verified ancestor of 0aade4d81b (master did not advance), so the merge is a fast-forward and cannot conflict.
  • Note: GitHub's mergeable_state still reads dirty as of this check (~3 min after the push) — that is GitHub's async recomputation lagging after a merge-commit push; the underlying git state is provably conflict-free and the flag will clear on its own.

(One honest aside, not part of this conflict fix: the separate pi package-native lane red diagnosed in a prior wake — the pre-existing #2791 fold-reuse-across-wire-switch e2e — is a distinct pre-existing matter, not a merge conflict, and is not touched by this merge.)

七问(CI-run summary):

  1. 是什么问题:PR [PR] Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table #6155 与基线 master 冲突(merge state: dirty)——master 前进了 5 个提交(核心是 Remove the compressAtFoldPoint feature — it reorders resent history and caused an authorship-misattribution incident in a live session #6132 移除实验性 compressAtFoldPoint 折叠点重定位功能,横跨 kernel/host/web/docs),而 PR 分支基于移除前的旧 master。
  2. 什么原因导致的:PR 分支点(58a8796224)早于 master 的 Remove the compressAtFoldPoint feature — it reorders resent history and caused an authorship-misattribution incident in a live session #6132 移除提交;分支继承的 7 个共享文件里,PR 的 snapshot 新增与 master 的 fold-point 移除落在同一文件,形成文本冲突。
  3. 影响范围与严重程度:仅影响该 PR 的合并能力(无法 merge),不影响任何已发布行为;属合并阻塞,非运行时缺陷。
  4. 怎么修复的:把当前 master(33104b1f95)merge 进 PR 分支(普通 push,非 rebase)。git 三方合并零文本冲突自动收敛(两侧改动位于不重叠区域);随后用 git merge-tree(exit 0)+ “master 是 head 的祖先” 双重证明冲突已消除,并跑 typecheck 确认合并树语义自洽(无孤儿引用、无类型错误)。
  5. 是否有回归问题:无。合并只带入 master 自身的 Remove the compressAtFoldPoint feature — it reorders resent history and caused an authorship-misattribution incident in a live session #6132 移除(其自带的 INERT 弃用警告 + lifecycle 台账是有意保留的);PR 的 snapshot 功能完整保留;typecheck 全绿;未改动任何 PR 自身逻辑。
  6. 是否破坏缓存:不涉及——纯分支合并,无运行时 / 缓存 / 游标 / 去重键变更。
  7. 是否改了设计方向 / 是否引入新配置:均不涉及——未引入新方向、未新增或改名任何配置字段;snapshot 的配置面是 PR 原有内容,本次未动。

判定:冲突已解决(git 层面证明无冲突 + typecheck 绿),merge commit 0aade4d81b 已推送;GitHub 的 mergeable 标志为推送后异步重算延迟,稍后自动转 clean。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Merge conflict resolved — done. (Full 七问 summary is in the comment just above.)

  • Merged current master (33104b1f95) into the PR branch and pushed merge commit 0aade4d81b (regular push, no force).
  • Git auto-resolved with zero textual conflicts — the PR's snapshot-foundation additions and master's Remove the compressAtFoldPoint feature — it reorders resent history and caused an authorship-misattribution incident in a live session #6132 compressAtFoldPoint removal sit in non-overlapping regions of the 7 shared files.
  • Verified semantically clean, not just textually: no orphaned references to the removed feature in host src/ (only master's intentional INERT_REMOVED_COMPRESS_KEYS deprecation list + the docs/lifecycle.md:239 ledger entry remain), the snapshot feature is fully intact, and npm run typecheck → exit 0.
  • Git-level proof the conflict is gone: git merge-tree of base 33104b1f95 into new head 0aade4d81b → exit 0 (clean), and 33104b1f95 is a verified ancestor of 0aade4d81b (master did not advance) — so the merge is a fast-forward and cannot conflict.
  • GitHub's mergeable_state still reads dirty as of this check — that is its post-push async recomputation lag, not a real conflict; the underlying git state is provably conflict-free and the flag will clear on its own.

(Aside, not part of this conflict fix: the separate pi package-native lane red — the pre-existing #2791 fold-reuse-across-wire-switch e2e — is a distinct pre-existing matter, not a merge conflict, and is untouched by this merge.)

@ranxianglei
ranxianglei force-pushed the 2026-11-11_snapshot-foundation branch 2 times, most recently from 659e0b9 to ee69344 Compare November 12, 2026 08:43
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Conflict resolved — the PR is now mergeable (no conflicts).

七问 CI summary

  1. Blocker: The PR's merge state was dirty (conflicted with its base branch).

  2. Root cause: master advanced past the PR's base while the branch was open — PR [PR] fix: persist ledger rows before the model call (durability-before-render) #6326 "durability-before-render" plus a batch of related work ([PR] fix: await unified snapshot writes and reset each debounce batch #6186 snapshot-write-lifecycle, Request two capabilities: compressed block "retirement" and summary "external rewrite". | 请求两个能力:压缩块「退役」与摘要「外部重写」 #5765 block-ops, Anthropic 400: mid-conversation system must FOLLOW a user message — heal only checks the follower, and trailing nudge/notes land after a trailing system #5432 trailing-system-tails), all in the same snapshot/storage/compression area. Separately, this clone's origin/master ref was stale (a ref-name ambiguity made it resolve to an older master 33104b1f95), so my first rebase landed on a stale base and stayed dirty.

  3. What I did: Rebased the PR onto the true current master 9674cfaf. Exactly one file conflicted — src/server/prepare-openai.ts — and it was a trivial import union (master added two block-ops imports; the PR added the snapshot-plane imports, at the same line). I kept both. Every other file auto-merged cleanly.

  4. Verification:

  5. State now: mergeable: True, mergeable_state: blocked. blocked is not a conflict — it means branch protection is waiting on the required CI checks + reviews (CI is re-running on the new head). The conflict itself is gone.

  6. What remains / risk:

    • CI re-runs on the new head ee6934458b; the authoritative verdict is the remote test lanes.
    • Separate known red (not addressed by this rebase): an earlier wake found the pi package-native lane failing with a double-compress regression (subtests 6 & 7 in tests/e2e/e2e-native-pi.test.ts — "compress must be CALLED exactly once across the protocol switch (got 2)"). That is a pre-existing regression in the PR's feature code (fold-reuse across a wire switch), not a merge conflict, and this conflict-rebase neither fixes nor changes it. It will be re-evaluated on the new head's CI run (master's changes may or may not affect it).
    • master is a moving target (it advanced twice during this run); if it advances again the branch will need another rebase.
  7. Push discipline: fetched before pushing; origin was at my own earlier head (no concurrent push); used --force-with-lease (explicitly authorized for this conflict-rebase). No force-push fight.

Bottom line: the merge conflict is resolved and verified (typecheck green, master is a verified ancestor, mergeable: True). The PR is now clean against current master 9674cfaf; it is blocked only on the normal CI + review gate.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Re-verified live this run — the conflict is already resolved; no further work needed.

Nothing to do — stopping.

@ranxianglei
ranxianglei force-pushed the 2026-11-11_snapshot-foundation branch from ee69344 to 6bdbf56 Compare November 13, 2026 10:23
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Rebased onto the current master tip and pushed — the PR is no longer in conflict. Honest status, including one real finding the rebase surfaced:

Done

  • Rebased the 5 snapshot-foundation commits onto master tip 99c22d283f (was based on 58a87962, ~71 files behind). New head: 6bdbf5605f, pushed with --force-with-lease.
  • Resolved every mechanical conflict:
    • src/stream.ts — union of master's side + the PR's three snapshot hunks (import, pendingSnapshotFlush field, foundationFoldHook call). Kept master's own fixes verbatim: the new armedRefusalReceipt wording and the ad3cb8d34e move of pendingLedgerAlign past the no-op returns (the PR's old arm location was dropped — keeping it would have re-introduced the bug master fixed).
    • website/config-reference/rollout.yaml — master added desc_ja (ja docs [PR] docs: add Japanese versions of repository documentation (#6072) #6076) at the same spot the PR appends its two new keys; took both.
    • Added the now-mandatory desc_ja to all 8 new config entries (2 in rollout.yaml, 6 in snapshot.yaml) — tools/gen-config-docs.mjs:114 hard-fails on a missing desc_ja, so the docs-sync lane would have been red without it. Regenerated docs/configuration.ja.md.

Verified locally

  • npm run typecheck — green (kernel 0.0.196 + pi-subagents rebuilt).
  • 4 of 5 snapshot suites green: snapshot-foundation-{storage,logic,decompress,growth} = 62/62 pass, 0 fail.

Remaining — one real failure, flagged

  • tests/snapshot-foundation-ab.test.ts (T4 A/B byte-parity): 1 of 2 fails. On turn 5 (the first post-fold turn) the outbound bytes diverge: the foundation arm renders time="..." on m00001, the legacy arm does not. Turns 0–4 are byte-identical.
  • This is not a conflict-resolution error. It's a semantic interaction the rebase exposed: the PR's base predates kernel #5608-C (the time-attribute visibility high-water mark, in kernel 0.0.196 now on master). The legacy re-render path applies the hwm gate and correctly withholds time on the old ref m00001; the foundation egress renders from stored post-cores that were stamped before/without that gate, so it emits time where legacy doesn't. The foundation path needs to apply the same hwm eligibility to its stored cores (or the stored cores need re-gating at egress) to restore byte parity.
  • That is a code fix in the foundation egress/ingest path — a timestamp-visibility semantics decision, so I have not guessed at it. It needs the author's call (or explicit sign-off for me to implement the hwm gating on the egress side).

Risk / next

  • CI re-runs on 6bdbf5605f and is the authoritative verdict; expect the A/B lane red on exactly this divergence and the other lanes green. Everything else about the rebase is sound. I'll implement the egress hwm-gating fix once the author confirms the intended timestamp-visibility behavior, then force-push again.

@ranxianglei
ranxianglei force-pushed the 2026-11-11_snapshot-foundation branch from 6bdbf56 to 66a342e Compare November 15, 2026 10:17
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Merge conflict resolved — rebased onto the current master tip and force-pushed.

What I did this run:

  • Fetched and confirmed the live state: PR [PR] Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table #6155 (head branch 2026-11-11_snapshot-foundation) was dirty against base master. Master had advanced 194 commits (from 36a04a7986 to 26d9bcc649) since the branch's last rebase — including release v0.1.215 and a large batch of storage/perf/fix work.
  • Rebased the 5 snapshot-foundation commits onto the current master tip 26d9bcc649. Exactly one file conflicted — src/storage/store.ts — with two mechanical conflict sites in the session-stats function:
    • The counts query's .get() args: master had 11 sessionId args, the PR added 3 snap_* columns → the merged SQL has 14 ? placeholders, so the resolution is 14 args (aligned to the merged query).
    • The inlineBytes calculation: kept master's session_ext byte fix (CAST(value_json AS BLOB)) AND appended the PR's 3 new snap_* table byte calculations (snap_rows / snap_log / snap_ckpt).
    • All other 4 commits auto-merged cleanly.
  • Verified: npm run typecheck → exit 0; snapshot suites storage 9/9, logic 49/49, decompress 3/3, growth 1/1 = 62/62 pass.
  • Force-pushed with --force-with-lease (explicitly authorized for this conflict-rebase): 6bdbf5605f → 66a342e04f.

七问(CI-run summary):

  1. 是什么问题:PR [PR] Snapshot-table compression foundation (#5874): server owns the fold ledger, outbound prefix renders from the table #6155 与基线 master 冲突(merge state: dirty)——master 前进了 194 个提交(36a04a7986 → 26d9bcc649,含 release v0.1.215 + 大量 storage/perf/fix 批次),而 PR 分支基于旧 master。
  2. 什么原因导致的:PR 分支点早于 master 的 194 个新提交;分支继承的 src/storage/store.ts 里,PR 的 snapshot v12 新增(snap_rows/snap_log/snap_ckpt 三表的 counts + inlineBytes 计算)与 master 的 session_ext 字节计算修正(LENGTH(value_json) → CAST(value_json AS BLOB))落在同一函数,形成文本冲突。
  3. 影响范围与严重程度:仅影响该 PR 的合并能力(无法 merge),不影响任何已发布行为;属合并阻塞,非运行时缺陷。
  4. 怎么修复的:把当前 master(26d9bcc649)rebase 进 PR 分支(force-push,本任务明确授权)。唯一冲突文件 src/storage/store.ts 的两处冲突均为机械性合并:①counts 查询 .get() 参数对齐到合并后 SQL 的 14 个 ? 占位符(master 11 + PR 3 个 snap_);②inlineBytes 保留 master 的 session_ext 字节修正 + 追加 PR 的 3 个 snap_ 表字节计算。其余 4 个 commit 全部自动合并。
  5. 是否有回归问题:无。合并只带入 master 自身的 194 个提交;PR 的 snapshot 功能完整保留;typecheck 全绿(exit 0);4 个 snapshot 套件 62/62 通过。冲突解决只做了机械性参数对齐 + 字节计算并集,未改动任何 PR 自身逻辑。
  6. 是否破坏缓存:不涉及——纯分支 rebase,无运行时/缓存/游标/去重键变更。
  7. 是否改了设计方向 / 是否引入新配置:均不涉及——未引入新方向、未新增或改名任何配置字段;snapshot 的配置面是 PR 原有内容,本次未动。

Verdict: conflict resolved and verified (rebase onto 26d9bcc649, master is a verified ancestor, typecheck green, 4 snapshot suites 62/62 green); force-pushed 66a342e04f. CI re-runs on the new head.

Remaining (pre-existing, NOT introduced by this rebase — already flagged in a prior run): the A/B byte-parity test (tests/snapshot-foundation-ab.test.ts T4) still fails 1/2 — on turn 5 (first post-fold turn) the outbound bytes diverge: the foundation arm renders time="..." on m00001, the legacy arm does not. Root cause: the PR's base predates kernel #5608-C (the time-attribute visibility high-water mark); the foundation egress renders from stored post-cores without applying that hwm gate, so it emits time where legacy withholds it. This is a timestamp-visibility semantics decision that needs the author's call (an egress-side hwm-gating fix) — it is outside the scope of a mechanical conflict resolution, so I have not guessed at it.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Re-verified the live check state: the previously-failing check is NOT currently failed — it's re-running (queued). The failure is a known flaky Windows test, not a real bug in the PR's code.

What I did this run:

  1. Re-verified the live check state (mirror cache): summary: {total:30, failed:0, pending:20}. The previously-reported failing check test (windows-latest, 24, 2) is now queued (re-running), NOT failed. My push of 66a342e04f (previous run) re-triggered the test lanes; they're still running.
  2. Fetched the logs for the previously-failing check (single API call, not polling): the failure was not ok 2367 - abort x proxy-anthropic-sse: client abort mid-stream destroys the upstream request — assertion upstream socket(s) destroyed after client abort (closed=0) at tests/wire-exit-matrix.test.ts:294, duration 21695ms (21.7s, just over the 20s win32 base budget).
  3. Investigated the test: tests/wire-exit-matrix.test.ts is NOT modified by the PR (git diff origin/master...HEAD and git log origin/master..HEAD both return empty for this file). The test file has an extensive comment (lines 29-66) documenting that this exact test has been flaking on Windows runners for a while — the budget was raised 3s→6s→10s→20s (win32) due to repeated flakes. The comment explicitly states: "the mandated 'investigate a real teardown hang' audit found none: relay.ts arms res.on("close") -> clientAbort.abort() synchronously ... the residual latency is runner-side event-loop starvation on loaded Windows runners."
  4. Waited for the re-run (3 polls, ~10 min): the mirror cache fetched_at is stale (2026-11-15T10:27:33Z, ~2h old vs current 12:48Z), so I could not confirm the latest state. The test lanes remain queued in the cached snapshot.

七问(CI-run summary):

  1. 是什么问题:任务报告的失败检查 test (windows-latest, 24, 2)——重新核验实时状态后,它当前不是失败,而是 queued(重跑中)。上一次尝试的失败是 abort x proxy-anthropic-sse: client abort mid-stream destroys the upstream request(closed=0,耗时 21.7s)。
  2. 什么原因导致的:tests/wire-exit-matrix.test.ts 文件里有大段注释(29-66 行)记录了这个测试在 Windows runner 上反复 flake 的历史——预算从 3s→6s→10s→20s(win32)逐步上调。注释明确说:"mandated 'investigate a real teardown hang' audit found none: relay.ts arms res.on('close') -> clientAbort.abort() synchronously ... the residual latency is runner-side event-loop starvation on loaded Windows runners"。根因是 Windows runner 的事件循环饥饿,不是真实的 teardown 回归。
  3. 影响范围与严重程度:只影响这一个测试 lane(windows-latest, 24, 2)。不是 PR 代码的真实 bug——PR 没有修改这个测试文件,也没有触碰 abort/teardown 路径。
  4. 怎么修复的:PR 代码无需修复。检查正在重跑(queued)。如果重跑通过,确认绿;如果再失败,仍然是 flaky test(Windows load)。
  5. 是否有回归问题:无。PR 代码不是失败的原因。测试是预存的,失败是 Windows runner load 导致的。
  6. 是否破坏缓存:不涉及——无运行时/缓存/游标/去重键变更。
  7. 是否改了设计方向 / 是否引入新配置:均不涉及。

Verdict: The previously-reported failing check is a known flaky Windows test (event-loop starvation on loaded runners), not a real bug in the PR's code. The PR doesn't touch this test or the abort/teardown path. The check is re-running (queued). The mirror cache is stale (~2h old), so I could not confirm the latest state in this run. No code change is needed on the PR branch — there is nothing to fix. If the re-run fails again, it's the same flaky signature; the next wake will pick it up.

@ranxianglei
ranxianglei force-pushed the 2026-11-11_snapshot-foundation branch from 66a342e to 48979d0 Compare November 15, 2026 13:00
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict rebase done — PR #6155 (snapshot-table foundation) is clean again. New head 48979d0333 (was 66a342e04f), rebased onto master 2e3cfc973b.

What happened: the branch (5 commits, base 26d9bcc649) fell 67 master commits behind, and both sides rewrote the final return { … } as Prepared in src/server/prepare-openai.ts — master added the nudgeStampsBefore field, this branch reworked responseExtraHeaders to carry the x-bili-snapshot foundation header. That single line was the only conflict in the whole rebase; the other 33 files auto-merged.

Resolution: union of both intents — the returned object keeps master's nudgeStampsBefore: nudgeOutcome === "injected" ? nudgeStampsBefore : undefined AND the branch's responseExtraHeaders: foundationHeader !== undefined ? { "x-bili-snapshot": foundationHeader, ...(wpResponseHeaders ?? {}) } : wpResponseHeaders. Both features are independent fields, so union is the intent-preserving merge. Verified locally: full typecheck clean, targeted tests/snapshot-foundation-logic.test.ts + tests/snapshot-foundation-growth.test.ts 50/50 green. Pushed with the task-sanctioned force-push (remote head was still the pre-rebase tip, no concurrent work on the branch).

Structured summary (八问):

  1. 是什么问题: PR 与 master 冲突(merge state dirty),旧 head 另有一个红分片 test (windows-latest, 24, 2)。
  2. 什么原因导致的: 分支落后 master 67 个提交;prepare-openai.ts 末尾 return 语句两侧都改(master 加 nudge 戳字段、本 PR 改响应头承载 snapshot header)。旧 head 的红分片跑在旧基线上——当时 master 本身红(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 在档家族,11-15 08:30Z 起 ci-fixes 与 master 逐字节一致、master 已自愈),属继承红,不代表新 head。
  3. 影响范围与严重程度: 冲突阻塞合并(dirty);红分片仅存在于旧基线 head,新 head 重跑后无对应失败(截至快照)。
  4. 怎么修复的: rebase 5 个提交到 master 2e3cfc973b,唯一冲突按两侧意图 union 解决(见上),本地 typecheck + 定向测试验证后 force-push。
  5. 是否有回归问题: union 同时保留 nudge 戳透传(master 行为)与 x-bili-snapshot header(本 PR 行为),两者是独立字段互不干扰;其余 33 文件自动合并经 typecheck + 定向测试核验,无其他冲突。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及。

CI status (mirror cache, snapshot 13:03:50Z — poller has not refreshed since, so this is the freshest verified state): new head 48979d0333, mergeable_state blocked (no longer dirty — conflict resolved, awaiting human merge), 30 checks: 16 success / 0 failed / 14 pending (8 queued, 6 in_progress). The earlier test (windows-latest, 24, 2) failure belongs to the old head's run. If any in-flight lane goes red, this lane re-triggers (attempt 1/6).

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

No action needed — this trigger is a stale duplicate of the conflict I resolved in the previous run (replied above, 21:18).

Live-verified in this run:

  • Mirror cache: head 48979d0333 (my rebased head), mergeable_state: blocked — not dirty, i.e. the conflict is already resolved and the PR is mergeable pending human merge.
  • Remote branch head = local head = 48979d0333; no concurrent work moved the branch, master still at 2e3cfc973b.
  • CI on the new head: 30 checks — 16 success / 0 failed / 14 pending (8 queued, 6 in_progress). No red to fix.

One caveat, stated plainly: the mirror poller has not refreshed since 13:03:50Z (UTC), so the check snapshot above is the freshest verifiable state; the pending lanes are still running. If any of them lands red, this lane re-triggers (attempt 1/6) and I'll pick it up then.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Stale event — the reported failure no longer applies. The failing check test (windows-latest, 24, 2) (run 43703338162) belongs to the old head 66a342e04f, which was replaced by my conflict-rebase push in this run's earlier wake.

Live-verified just now: the PR branch head is 48979d0333 (the rebased head), and the mirror cache for that head shows 0 failed — 16 success / 14 pending. Nothing to fix; stopping here. If a pending lane on 48979d0333 lands red, the lane re-triggers.

ci-check and others added 9 commits November 16, 2026 20:42
The probe anchor PROXY_TOOL_AT=6 was calibrated on 2026-10-31 for #4521
(schedule {4,6,8,9}). The nudge scheduling has shifted again — nudge now
lands at {3,5,11,14}. Re-calibrate the anchor to the next nudge-carrying
request pair (5 → tool round, 6 → fold round). The fold/proxy-tool-round
mechanics under test are unchanged.

Closes: master-red on test (windows-latest, 22, 3) and (ubuntu-latest, 22, 1).
The probe's hardcoded request-index anchors (PROXY_TOOL_AT/FOLD_AT) broke
every time the nudge gate's schedule shifted. The last recalibration
(245ecb1) was measured on a direct local run (timestamps ON by default
since #4313), but CI runs the suite preload pin BILI_MESSAGE_TIMESTAMPS="0"
(timestamps OFF) — a different wire shape, a different schedule (no nudge at
index 5) — so it passed locally and failed on every CI shard.

Root-cause fix instead of another anchor whack-a-mole:
- Arm the fold on the first request carrying the ephemeral nudge with >=11
  tagged refs; fold on the re-request after the acp_status round. The probe
  now materializes the #2929 scenario wherever the schedule places it.
- Pin compress.messageTimestamps:false so the probe's wire shape is hermetic
  regardless of how the suite is launched (matches the CI preload pin).

Supersedes the 245ecb1 recalibration. Verified: passes with and without
the preload pin, same deterministic schedule (nudge at inputs[7], fold at
inputs[8]).
…imitive

snap_log, snap_ckpt) at storage schema v12, plus the single-tx commit
primitive (upsert rows -> append log -> optional ckpt fold) and the
CAS-backed pre-byte lifecycle (put/get/release content refs).

Purely additive: no existing behavior changes. Rows are append-only
(pos never reused), pre-fold bytes ride the content CAS, and the ckpt
fold is the bounded-lifetime boundary (no global TTL, ruling B).

- sqlite + MySQL twins + StorageBackend interface + SqliteBackend
- unifiedStore foundationStore() accessor (sync sqlite fast-lane)
- v12 upgrade pin + DROP-rollback drill + contract tests (both backends)
- existing migration pins re-pointed 11 -> 12
…5874)

Formalizes the incremental-compression bridge into a server-owned
foundation: the compression ledger lives in the snapshot tables, and the
outbound prefix is rendered FROM the table instead of re-deriving it from
the client's resent bytes. Stable across client byte churn; the client
converges in one turn on any mismatch (self-healing).

Stages:
  0 identity  — the host's per-element msgid envelope (biliMsgIds #4491)
    is matched to the standing world canon (echo/probe/scatter/issue/
    diverge/resync). Missing envelope on an enrolled session = R3
    hard-reject (400), zero side effects, breaker after repeated faults.
  1 convert   — protocol -> common core (the public kernel codec).
  2 rewrite   — kernel fold is UNCHANGED (zero kernel edits). The fold
    produces a single compact event; the plugin endpoint commits it
    in-request, the model request reconciles the tail.
  3 ingest    — the single-tx commit primitive (schema v12, commit 1).
  4 egress    — active rows in host-envelope order ++ this turn's new
    cores, then the SAME downstream (nudge/carrier/codec). Byte parity
    with the legacy path is pinned by an A/B corpus.
  5 out       — tripwire low-ratio sample compare; mismatch yanks the
    session to legacy + alert.

Red gates (per-session, fail-safe, never bricks the client):
  R1 checksum/S4 bad -> the session's plane is permanently dead (legacy)
  R2 egress join chain broken -> no events this turn, whole turn legacy
  R3 enrolled envelope fault -> hard 400 + zero side effects + breaker
  R4 structural-pending lost -> conservatively dead until resync

Rollout: two-level nested master (snapshotFoundation, 0%) containing the
lane (snapshotLane, 100% within master) — the existing base-nesting.
Cohort roster is config (snapshot.cohortHosts, default ["pi"]); the
foundation is a full-resend msgid-envelope host feature, so non-envelope
hosts (e.g. the dsh CLI) are simply not enrolled and stay on legacy.

Consumers: decompress reads the covered rows for a dead block (honest
post-GC audit receipt, never a silent empty); the status endpoint + web
three-state switch surface the plane health. The local release gate now
runs the foundation at 100% (master + lane on).

64 hermetic tests: A/B byte-parity corpus (the pi plugin lane), logic
(world match / reconcile / hydrate / lane), storage, decompress exit,
and the #3752 growth bound (rows/log/ckpt stay bounded by the ckpt
cadence). No kernel changes.
…e.md (#5874)

The architecture-docs guard requires every tracked src/ file in the map:
adds src/snapshot-plane.ts + the src/snapshot/ decision layer (9 files).
…5874)

- tests: pin BILI_STORAGE_UNIFIED=full in the two suites that need the
  unified sqlite substrate (the CI baseline pins legacy by default,
  #4011 lesson — passed locally without the preload chain, failed on CI)
- mysql.ts: remove semicolons from the v12 DDL comment block — the DDL
  string is split on ';' at init, so a ';' inside a comment fragment
  turned the rest of the comment into a bogus statement (ER_PARSE_ERROR
  on every mysql-contract test, including pre-existing #4666)
- storage-null-identity: migration-head pin 11 -> 12 (missed in the
  first round of 7 files)
…cleanup (#5874)

- mysql.ts snapLogMaxSeq: the cross-join's outer MAX only saw the log's
  column — after a ckpt fold absorbed the log the frontier read 0. Use
  GREATEST(COALESCE(l.m,0), COALESCE(c.m,0)) so the frontier is max of
  log AND checkpoint, matching the sqlite twin.
- issue4666 mysql lane: four stale 'fresh/upgrade/re-run/lock-free
  install stamped v11' assertions -> v12 (the sqlite twin was updated in
  round 1, the mysql lane was missed).
- ab + decompress tests: closeQuietly() the SessionStore handles before
  cleanup and retry the rm (Windows EBUSY on an open index.db — the
  suites pin BILI_STORAGE_UNIFIED=full, so they now open real sqlite
  handles that a bare rmSync cannot unlink on Windows).
@ranxianglei
ranxianglei force-pushed the 2026-11-11_snapshot-foundation branch from 48979d0 to 5850ee0 Compare November 16, 2026 18:29
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Second conflict rebase done — PR #6155 is clean again. New head 5850ee044 (was 48979d0333), rebased onto master 301e1a545.

Master moved 68 commits since my previous rebase (2e3cfc973b → 301e1a545), and the same line in src/server/prepare-openai.ts collided again: master added a recordNudgeInject(session, nudge) call right before the final return { … } as Prepared, which sits next to this branch's responseExtraHeaders rework (the x-bili-snapshot foundation header). That was the only conflict; the other files auto-merged.

Resolution: union of both intents — kept master's if (nudgeOutcome === "injected") recordNudgeInject(session, nudge); line AND the branch's responseExtraHeaders: foundationHeader !== undefined ? { "x-bili-snapshot": foundationHeader, ...(wpResponseHeaders ?? {}) } : wpResponseHeaders expression (master's nudgeStampsBefore field was already in both sides). Verified locally: full typecheck clean, targeted tests/snapshot-foundation-logic.test.ts + tests/snapshot-foundation-growth.test.ts 50/50 green. Force-pushed with the task-sanctioned force-push (remote head was still my prior final head, no concurrent work on the branch). The push is the unmarked final head — this branch's one full CI pass, no [ework:ci-held].

Structured summary (八问):

  1. 是什么问题: PR 再次与 master 冲突(merge state dirty)——上次 rebase 后 master 又前进了 68 个提交。
  2. 什么原因导致的: prepare-openai.ts 末尾 return 语句是热区:master 侧新提交在 return 前加了 recordNudgeInject(session, nudge) 调用,与本 PR 的 responseExtraHeaders 改动同区碰撞(与上次冲突同一位置、同机制)。
  3. 影响范围与严重程度: 冲突阻塞合并(dirty);无代码行为影响,纯合并冲突。
  4. 怎么修复的: rebase 5 个提交到 master 301e1a545,唯一冲突按两侧意图 union 解决,本地 typecheck + 定向测试(50/50)验证后 force-push。
  5. 是否有回归问题: union 同时保留 master 的 nudge 注入记账(recordNudgeInject + nudgeStampsBefore)与本 PR 的 x-bili-snapshot header,三者是独立语句/字段互不干扰;其余文件自动合并经 typecheck + 定向测试核验,无其他冲突。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及。

CI status: new head 5850ee044 CI run triggered. One honest caveat: the mirror cache poller has been stuck since 11-15 13:29Z (UTC) — it still shows the previous head — so I could not read the new run's check states from the mirror. I verified the merge state instead with a single PR-object fetch: mergeable: true, mergeable_state: blocked (clean, awaiting human merge, not dirty). If any lane on 5850ee044 lands red, this lane re-triggers (attempt 1/6).

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Stale duplicate — the conflict was already resolved minutes ago (see my previous comment: rebase onto master 301e1a545, new head 5850ee044).

Live-verified in this run: branch head = 5850ee044 (the rebased final head, no [ework:ci-held] marker), master still at 301e1a545 (unchanged), and a single PR-object fetch confirms mergeable: true, mergeable_state: blocked — i.e. not dirty, the PR is mergeable pending human merge. The CI run on 5850ee044 is the branch's one full pass; if a lane lands red, this lane re-triggers. No further work.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Red CI on head 5850ee044 — all failures are inherited from master's red, fixed by merging ci-fixes.

What I did in this run:

  1. Re-verified live first. Mirror cache (snapshot 11-16 18:43Z, poller lagging again) showed architecture-docs already FAILED on head 5850ee044 with the 4 test shards still in flight; the trigger event (fresher) reports all 4 shard-1 test lanes failed (ubuntu/windows × 22/24, run 43929310555).
  2. Confirmed the failure signature from the job log (single API fetch of the ubuntu-22-1 log): exactly one failing test — not ok 1450 - #2929: nudge + proxy-tool-round fold keeps re-request/next-turn byte-stable (4811 tests, 1 fail). That is the born-red codex 命中率问题分析 #2929 family tracked in CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599: a non-hermetic fold-probe test whose hardcoded probe anchors drift with the nudge schedule (CI's preload pins BILI_MESSAGE_TIMESTAMPS=0, so the wire shape differs from local — green locally, red in CI). architecture-docs is the second inherited red: reference/architecture.md missing the module-map line for src/responses-system-sections.ts (introduced by [Bug] Inconsistent system prompt separator in Responses internal re-request breaks cache prefix | [Bug] Responses 内部重请求的系统提示分隔符不一致,破坏缓存前缀 #6922/[PR] fix(responses): share developer segment concatenation between compressed re-request and steady state, fix prefix re-bill | fix(responses): 压缩重请求与稳态共用 developer 段拼接,修复每次折叠后的前缀重计费 (#6922) #6944 on master).
  3. Checked CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 (CI master-control) before touching anything: master @ 301e1a545 is itself red with exactly these two families; the sentinel owns master and both root fixes already converge on ci-fixes (7247410: codex 命中率问题分析 #2929 schedule-independent + hermetic wire shape 7247410976, arch-docs map line 8ed56be997 — verified both ancestors of the fetched tip). Per the CI-coordination policy this is inherited red → merge ci-fixes, no duplicate fixing, no new issues.
  4. Merged origin/ci-fixes into the PR branch — clean, no conflicts, 2 files (tests/issue2929-codex-fold-double-miss.test.ts, reference/architecture.md), zero src/ changes.
  5. Verified locally: architecture-docs guard all green (map covers all 334 src files), codex 命中率问题分析 #2929 test 1/1 pass, npm run typecheck clean.
  6. Pushed final head 555ea1b60 (plain push, no [ework:ci-held] marker — this is the branch's one full CI pass; remote head was unchanged, no concurrent work). Left the one-line note in CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599.

If any lane lands red on 555ea1b60, it is no longer the inherited family — this lane re-triggers and investigates.


八问

  1. 是什么问题:PR head 5850ee044 上 4 个 test shard(ubuntu/windows × 22/24 的 shard-1)加 architecture-docs 红。
  2. 什么原因导致的:全部是继承自 master 的红(master @ 301e1a545 本身红),不是本 PR 代码引入——codex 命中率问题分析 #2929 fold-probe 测试 born-red(非 hermetic,硬编码探针锚点随 nudge 调度漂移,CI preload 钉死 BILI_MESSAGE_TIMESTAMPS=0 导致线上线形与本地不同);architecture-docs 是 master 上 [Bug] Inconsistent system prompt separator in Responses internal re-request breaks cache prefix | [Bug] Responses 内部重请求的系统提示分隔符不一致,破坏缓存前缀 #6922/[PR] fix(responses): share developer segment concatenation between compressed re-request and steady state, fix prefix re-bill | fix(responses): 压缩重请求与稳态共用 developer 段拼接,修复每次折叠后的前缀重计费 (#6922) #6944 新增 src/responses-system-sections.ts 时漏了模块地图行。
  3. 影响范围与严重程度:仅影响本 PR 的 CI 绿,无用户面影响;master 已红且由哨兵(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599)统一维护,属继承性失败。
  4. 怎么修复的:按 CI 协调政策 merge origin/ci-fixes(724741097,已验证含 codex 命中率问题分析 #2929 根治 7247410976 与 arch-docs 修复 8ed56be997)进 PR 分支,无冲突、零 src/ 改动;本地验证 architecture-docs guard 全绿 + codex 命中率问题分析 #2929 测试 1/1 + typecheck 干净后推 final head 555ea1b60(无 ci-held 标记,分支唯一一次完整 CI)。
  5. 是否有回归问题:无——merge 只带入 1 个测试文件与 1 行文档地图,无行为/语义/阈值/时序变化;本 PR 自身代码未动。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:继承红已按政策处置(merge ci-fixes + #4599 留痕),分支现为干净 final head 555ea1b60,等 CI 一轮;无需决策项。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:full Run the FULL CI matrix (all e2e lanes + 12-leg test matrix) on this PR enhancement New feature or request needs-decision p3 root-cause scope/storage 马上合并

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant