Skip to content

Incremental turns: POST /__bili/plugin/turn — native clients send only the delta (#3248) - #3253

Merged
ranxianglei merged 7 commits into
masterfrom
2026-10-15_incremental-lane
Oct 15, 2026
Merged

ranxianglei merged 7 commits into
masterfrom
2026-10-15_incremental-lane

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #3248 (phase 2: the incremental lane proper; phase 1's hash fast-path turned out to be subsumed — see notes).

What

Native-mode hosts that keep their own full local history (dsh, pi, future omp lanes) stop resending it:

Contract

  • 200 = upstream body streamed through + x-bili-applied-seq (canonical length after the delta) — echo as next turn's sinceSeq.
  • 409 {error: "seq-mismatch"|"turn-in-flight", serverSeq} — client recovers by realigning sinceSeq or resync: true full seed. Strict: sinceSeq must equal the server's canonical length; 0 against a populated canonical is a mismatch, never a silent replace.
  • 503 — lane runs legacy-only; the canonical log lives in the unified messages table (storage.unified shadow|full).
  • Sticky system (omit = keep last; session_ext inc:system), anthropic request params via allowlist, plugin capability headers forwarded, /__bili/ loopback gate via the shared adminGateDenial.

Why it is this shape

Tests

tests/incremental-lane.test.ts (8): seed→delta assembly asserted at the upstream; 409 + resync recovery; cold rebuild from the unified table after pool+cache reset; session-interop (same session id as normal traffic); sticky system; turn-in-flight; request-param allowlist (incl. request.messages smuggling rejected); legacy 503. Regressions: plugin-fork + memory-diagnostics + storage-unified (64), golden-storage-render + issue833 + wire-contract (10), tsc clean, unused-export gate clean.

Prepared by an agent under the local git identity (ranxianglei), human merge required.

Native-mode clients stop resending full history: turn 1 seeds the
canonical (unified messages table), later turns send only the delta
(including the previous assistant reply, client-owned). The lane
assembles the full anthropic body and re-enters the proxy pipeline
in-process (module-local forward, no loopback hop) — the session is the
client's own conversation id, so incremental and full-resend traffic
share one session, one fold ledger, one canonical, and may interleave
freely (mode switch needs no ceremony).

- Strict ack contract: x-bili-applied-seq header; 409 seq-mismatch /
  turn-in-flight force a client resync (sinceSeq realign or resync:true
  full seed); crash window = resync, the client stays the source of
  truth until it prunes (threads-lane model).
- Canonical = kernel BiliMessage core form — exactly the rows the
  unified messages table already holds (rawTail ingest #3007 + watermark
  #3166 feed it from normal traffic too). Cold rebuild after restart /
  LRU eviction replays the table; malformed rows are a 409, never a
  silent truncation.
- Sticky system prompt (session_ext inc:system), cache-control marks
  persisted (inc:cache-marks), anthropic request params via allowlist,
  plugin capability headers forwarded, /__bili/ loopback gate via the
  shared adminGateDenial.
- Requires storage.unified shadow|full (503 legacy-only). v1 keeps the
  O(full) pipeline walk over the assembled prefix (zero-hit, cheap); the
  server-authored prefix fast path is tracked on #3248.

Tests: tests/incremental-lane.test.ts (seed/delta/assembly, 409 +
resync recovery, cold rebuild from the table, session interop, sticky
system, turn-in-flight, request-param allowlist, legacy 503).
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-15_incremental-lane (cc0ff60)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3761-pr.3253.584c1fc

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3253.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到。这是 #3248 phase 2(增量 turn lane)的交付 PR,分支 2026-10-15_incremental-lane 已取到本地。

查重:不重复——open/closed 里检索 "incremental"、"/__bili/plugin/turn"、"sinceSeq",只有这一条增量 lane 的在途工作(#3248 主线),无并行重复 PR/分支。

正在按 PR review 规范走完整审查(原 issue 方向 → 回归清点 → 代码 → 测试有效性 → diff 整洁度),稍后按结构化八问给结论。CI 状态一并核对中。

- PLUGIN.md: manifest.capabilities.turn does not exist in the plugin
  manifest (capabilities object has externalSummary + fork only) — state
  that detection is by endpoint + 503 instead of claiming a capability.
- PLUGIN.md v1 caveats: document the append-only resync behavior — a
  diverged-canonical resync appends rows beside the stale ones, so a cold
  rebuild answers 409 until one more resync (durable replacement tracked
  on #3248).
- CHANGELOG.md: 'interoping' -> 'interleaving'.

Review fix-ups for PR #3253 (no code changes).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论(结构化八问)

1. 是什么问题:功能 PR——#3248 的 phase 2。native 客户端(dsh/pi 等自持全量历史的宿主)不再每轮重发整段历史,只发增量+游标;服务端在 unified messages 表里维护 canonical 全量,拼出完整 anthropic body 后进程内重入管线,上游照旧只收折叠快照。

2. 什么原因导致的:不适用(需求实现)。动机:#3248 把 #2926 threads-lane 的"客户端发增量、服务端持权威历史"模式泛化到 native 车道。

3. 影响范围与严重程度:纯新增面——新端点 POST /__bili/plugin/turn(loopback-only,与全部 /__bili/ 同门)+ 响应头 x-bili-applied-seq。依赖既有 storage.unified(未开则 503),当前无任何宿主接入(dsh 侧接入是后续项),合并后对存量用户行为零变化。严重度:无(新增能力,非修复)。

4. 怎么修复的(设计要点):canonical = 管线本来就在写的 kernel BiliMessage core 行(rawTail ingest #3007 + watermark #3166),无第二套存储,全量重发的会话切过来直接可用;会话身份故意不命名空间化——合成请求携带客户端自己的 x-bili-plugin-conversation(管线里最强信号),增量/全量共享同一 session/fold ledger/canonical,双向切换零仪式;严格 ack 契约(sinceSeq 必须等于 canonical 长度,0 对非空 = mismatch 绝不静默覆盖;409 → resync);持久化搭现有 save path(响应后 best-effort flush,崩溃窗口 → 409 → resync);v1 保留 O(full) walk。

5. 是否有回归问题:审过触碰路径的既有行为,逐项——

  • handle.ts 拦截:仅 POST + URL 精确等于 /__bili/plugin/turn(query 允许),其余 /__bili/ 路径不受影响;adminGateDenial 与原 handleAdminRoute 门禁三检(tunnel 头/loopback/origin)逐条核对等价,错误文案一致;
  • 会话身份:合成请求走同一套 session-id 解析(含 dsh persona fingerprint、side-request verbatim 排除);interop 测试证实 session id = raw conversation id、stats 正常计数;
  • persist/storage:unifiedRawMessages 是只读新增(shadow→store.getMessages 透传),写路径与 append-only 语义未动;
  • relay/writeHead:核实成功路径必先显式 writeHead(relay.ts:1025)再流式写 body,错误分支同样显式——monkey-patch 注入 ack 头在所有响应路径可靠触发;
  • 无阈值/默认值/时序/输出格式漂移,fail-fast 未变吞错。

两个边界发现(均不拦合并):

6. 是否破坏缓存:否。组装 body 走标准管线(system 字节稳定、acp_summary 锚位不变);cacheControls 从存储行经 inc:cache-marks ext 原样带回;同会话增量/全量混跑时前缀缓存看到的消息字节与之前一致(canonical 即事实源)。

7. 是否改了设计方向:否。纯增量式 lane,沿用 #2926 既定模式,架构/边界/数据流无转向。一个观察供 owner 决定:PR 正文 Closes #3248,但 phase 3 项(server-authored fast path、R1 message-id 收敛、SSE 聚合、dsh 接入)正追踪在 #3248 上——合入会把它关掉、这些项失去家;建议合入时另立 phase 3 tracking issue(或 owner 自行处置)。

8. 是否引入了新的配置:无新 env/config/CLI key。端点可用性复用既有 storage.unified(关则 503)。新 wire 面:POST /__bili/plugin/turn + x-bili-applied-seq,PLUGIN.md(en+zh)+ CHANGELOG 已记载。

验证证据

  • PR head 本地构建通过(npm ci + npm run build);定向测试 tests/incremental-lane.test.ts 8/8 通过;回归 storage-unified + plugin-fork + wire-contract 82 tests / 0 fail。全量套件以 CI 为准。
  • 测试有效性:断言有牙——直接断言上游收到的组装 body(角色序列/参数/偷渡拒绝)、ack 头值、409 serverSeq、冷重建后行为,不是恒真断言;smuggling(request.messages)与 legacy 503 均有钉住用例。已知覆盖缺口:divergent resync(见上,已文档化+立项)。
  • diff 整洁度:8 文件 +648/-1,每行都服务于本 PR 目的,无空行/重排噪音。
  • 小修已直接推上 PR 分支(commit 7bf7995b6,标注独立):① PLUGIN.md 声称的 manifest.capabilities.turn 在 manifest 里不存在(capabilities 对象只有 externalSummary+fork)→ 改为"未进 manifest,按端点 503 探测";② CHANGELOG "interoping" → "interleaving";③ v1 caveats 补 divergent resync 行为说明。新 head 的 CI 已全部排队,前 head 10/12 check suites 绿(查时 2 个还在跑)。

判定

可合并——待 7bf7995b6 的 CI 转绿 + owner 批准。两个边界发现均不阻:持久化缺口已文档化并在 #3248 立项,理论边缘留接入清单。

awork and others added 4 commits October 15, 2026 14:53
Windows CI (test shards, node 22+24) failed teardown with
'EBUSY: resource busy or locked, unlink ...sessions\index.db': the
harness closed the proxy and upstream but never the store's SQLite
handle, and Windows cannot delete a directory while index.db is open.
closeQuietly() is the documented seam for exactly this (persist.ts);
the storage-unified suite already uses it. POSIX masked it because
unlink of an open file succeeds there.
…ds slice past the rejected branch

External review of #3253 found the persistence gap: a resync replaces the
canonical in memory, but the unified messages table is append-only — the
stale branch's rows stayed, so a cold rebuild served the longer row union
and answered 409 until the client resynced once more (one extra round per
restart).

Fix: handleIncrementalTurn records the table row count at resync time
(session_ext key 'inc:base-seq') AFTER flushing the outgoing branch's
pending save, and readCanonicalFromStore slices rows past that epoch —
the rejected branch keeps its append-only audit copy without polluting the
served canonical. Legacy-only storage degrades to the previous full-union
read (warn once).

Test: divergent resync survives a cold restart without a 409 loop —
seed an old branch, resync to an edited one, restart, next delta serves
the edited branch (3 messages, no old-branch leakage).
@ranxianglei

Copy link
Copy Markdown
Owner Author

Review finding ① (divergent-resync persistence gap) is now fixed in-PR (2ec6b40): a resync: true records a base-seq epoch (session_ext inc:base-seq) after flushing the outgoing branch's pending save; cold rebuilds slice the unified rows past that epoch, so one resync heals — restarts included — while the rejected branch keeps its append-only audit copy. Legacy-only storage degrades to the previous full-union read (warn once). New test: divergent resync survives a cold restart without a 409 loop (9/9 green in tests/incremental-lane.test.ts). Finding ② (sticky-system × persona fork theoretical edge) stays on the host-integration checklist — dsh's stable system text does not trigger it.

The concurrent-turn test settled the first request with a 40ms sleep
against a 150ms upstream delay. Under load (observed locally on the
review box: 1 failure in 4 runs, then clean), the first request can
reach inFlight after the second is already issued, flipping the
expected 'turn-in-flight' into 'seq-mismatch'. Widen both margins
(120ms settle / 800ms delay) so the overlap no longer depends on
scheduler luck.
@ranxianglei
ranxianglei merged commit 360abaa into master Oct 15, 2026
41 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.443-master.360abaa (rolling master tag).

npm install -g billion-context@0.0.443-master.360abaa  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@github-actions github-actions Bot mentioned this pull request Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant