Repository navigation
Incremental turns phase 2: marker divert on ordinary model requests + native client helpers (#3248) - #3273
Conversation
Native extensions send incremental turns on their ordinary model requests:
stamp x-bili-plugin-turn: 1 alongside the usual x-bili-plugin header and post
the incremental payload — the body shape (conversationId/sinceSeq/messages,
no top-level model) is the final word, so a marker on a non-incremental body
replays through the normal pipeline byte-semantically unchanged, and hosts
keep their provider URLs untouched. The lane forwards to the ORIGINAL path.
- rollout feature pluginIncrementalTurns (default 0% = byte-identical
installs; force via config rollout.features.pluginIncrementalTurns or
BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1), still requires a live unified
sidecar; disabled lanes answer 503 on shaped turns only
- GET /__bili/plugin/turn capability probe: {ok, protocol:1, feature,
enabled, storage}
- seeds imply resync:true (first incremental turn never 409s; interop
full-resend traffic replaced client-authoritatively); a system-only
change rides an empty delta (canonical unchanged, sticky system updated)
- src/agent/incremental-client.ts: shared client state machine (probe TTL
cache, wire-hash prefix diffing, sticky system, dead flag, ack bookkeeping)
- reference wirings: pi (before_provider_headers marker + payload
replacement + after_provider_response ack) and dsh-native (fetch patch
beforeSend/headersFor/onResponse; opencode/omp-native follow the same
recipe per the in-code notes)
Tests: probe rollout states, marker divert e2e (seed+delta assembled),
non-incremental ride, disabled 503 + plain ride, helper state machine
(seed/delta/edit-resync/sticky-system-empty-delta/dead), body-shape unit.
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 按失败签名/主题检索 open+closed 的 issue 与 PR( 处理姿态:这是 [PR] 工单——拉分支核对代码、确认 CI 状态、按八问体例报结论,不重做。正在审:单 commit |
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.3777-pr.3273.7a73e6bThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3273.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…NAL_ONLY exports - scripts/check-unused-exports.mjs: git ls-files --cached --others --exclude-standard — a brand-new module is exactly where INTERNAL_ONLY leaks happen, and a tracked-only scan passed locally right before CI failed on the same tree (the fresh src/agent/incremental-client.ts was untracked at the local gate run on #3273). - src/agent/incremental-client.ts: IncrementalPlan / PlanArgs keep their leading export off (module-internal shapes; callers infer).
… instead of replaying an empty body (#3273 review) A >cap (or stream-erroring) body on a marker'd request used to be swallowed: readBody rejects AND destroys the socket, the catch left an empty buffer behind, and the replay path re-sent that empty buffer into the normal pipeline — an upstream-facing 400/reset where the identical unmarked request gets bili's own 413 JSON. Mirror the pipeline's own body-read failure classification (413 on BodyTooLargeError, 400 on any other stream error) and stop.
…ne (#3273 review) The phase-2 addition (applyTurn accepts an empty delta when a system update is present) was only exercised at the client-helper level. Add the end-to-end pin: shaped turn with messages=[] + new system forwards the assembled request with the updated sticky system, canonical length unchanged; an empty delta WITHOUT a system stays a 400.
…sions (#3273) withFeatures merges fixtures into the live ROLLOUT_FEATURES registry and restores only what it added itself; the exact-list assertion predates any shipped entry (#3062 shipped the registry empty). The pluginIncrementalTurns entry added by this branch made the assertion fail deterministically on every shard-3 CI lane. Assert the sorted union of shipped + fixture keys and keep the per-feature enabled/source checks sharp.
(ai: vllm-qwen/qwen3.8-27b) Review 完成。我没有重做工作,而是拉分支逐文件核对了全部 10 个文件的 diff、跑了定向验证、复现并修掉了 CI 红。结论:可合并——新 head 结构化八问
待拍板:409 恢复语义(docs 与实现不一致)
CI 轨迹(备查)
判定:可合并(默认关、零影响、CI 全绿、review 发现均已闭环)。409 语义项不阻塞合并,拍板后我按选项落地。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.448-master.0e9360a # this exact build
npm install -g billion-context@master # always the newest merged build
|
master moved past the PR base (ranxianglei#3273 incremental lane, ranxianglei#3199 unified web lists, codexForge kind=prompt, network.allowPrivateHttpProviders docs). Conflicts were all additive: - src/server.ts: keep master's codexForge kind union (+"prompt") AND the PR's autoModeForge Prepared field. - src/server/handle.ts: combined import lines (knobs + auto-mode-forge + v1internal); PR interception point verified to still run BEFORE the ranxianglei#388 side-request split. - CONFIGURATION.md / CONFIGURATION.zh-CN.md: keep both new env-table rows (BILI_ALLOW_PRIVATE_HTTP_PROVIDERS, BILI_AUTO_MODE_FORGE), alphabetical. Verified: tsc clean, 56/56 (auto-mode-forge + e2e-anthropic + web-i18n + text-safe), test-suite-lock gate, import-graph gate, build OK.
Phase 2 of #3248: a native extension sends incremental turns without changing its provider URL or calling a dedicated endpoint — it stamps
x-bili-plugin-turn: 1on its ORDINARY model request and posts the incremental payload there.Why this shape
conversationId+sinceSeq+messagesand NO top-levelmodel= incremental; anything else (a normal anthropic body) rides the normal pipeline unchanged even with the marker — a host may stamp the marker whenever the feature is enabled and convert only the turns it wants, and a raced-off probe can never break a request.src/agent/incremental-client.ts(probe TTL cache, wire-hash prefix diffing, sticky system, per-conversation dead flag, ack bookkeeping). Reference wirings: pi (extension events) and dsh-native (fetch patch); the others follow the same in-code recipe.Rollout (requirement: default = zero impact)
pluginIncrementalTurns, default 0% — every install stays byte-identical until enabled. Force on withrollout.features.pluginIncrementalTurns: true(or a percent) orBILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1.storage: "legacy-only"on the probe and answer 503 on shaped turns only — plain traffic is untouched.GET /__bili/plugin/turn→{ ok, protocol: 1, feature, enabled, storage }(loopback-gated).Semantics carried over from phase 1 (+ one addition)
resync: true— the first incremental turn never 409s, whether the canonical is empty (fresh) or already live (interop with full-resend traffic: the client's view is authoritative).skip).Server changes
incrementalTurnBodyShape()strict shape check; marker divert inhandle()buffers the body once, diverts shaped turns to the phase-1 lane (forwarding to the ORIGINAL path), and replays non-shaped ones throughhandleRequestwith the marker stripped (same admission, no loop).applyTurnaccepts an empty delta when a system update is present.Tests
tests/incremental-lane.test.ts15/15: probe rollout states (default off / env force on), marker divert e2e (seed + delta assembled upstream,x-bili-applied-seq), non-incremental ride (model rides, no divert artifacts, text preserved through the normal pipeline), disabled rollout (503 shaped / 200 plain), helper state machine (seed-implies-resync, delta slicing, edit → resync, sticky system incl. empty-delta system update, dead flag), body-shape unit (accepts incremental, rejects anthropic/garbage/bad seq).Regressions: dsh-fork-adopt, issue833 plugin lane, #1542, plugin-fork, side-lane-invariants, #2601 all green; tsc clean; unused-export gate clean (4 waived).
dsh-native.test.tshas the known local 15s-timeout env family (reproduces on clean master; CI green).Fixes the native-access half of #3248 (the dsh client adoption and the server-authored prefix fast path remain on the issue).