Skip to content

fix(agent): proxy-death fallback no longer forwards incremental payloads upstream (#3304) - #3529

Merged
ranxianglei merged 5 commits into
masterfrom
2026-10-17_incremental-direct-send-guard
Oct 18, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-10-17_incremental-direct-send-guard

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Closes #3304

What was broken

The incremental lane replaces the outgoing request body with a lane-only delta envelope ({conversationId, sinceSeq, path, request, messages} + x-bili-plugin-turn marker) BEFORE the fetch layer runs. When the bili proxy dies inside the 30s probe window and native-intercept gives up on respawn, its fall-back direct send forwarded THAT mutated payload to the real upstream → hard 400 (the envelope has no top-level model). Pre-incremental behavior was "proxy down ⇒ degraded direct send of the full body still works"; after the lane landed that guarantee silently broke for every host whose adapter swaps before the intercept sees the request.

Affected instances (one mechanism, four lanes): dsh-native and pi / pi-native (already on master via #3273 phase 2), opencode and omp (#3292). claude/codex CLI never enter the lane.

Fix (A+B hybrid, as proposed in the issue)

Every fall-back direct send in native-intercept.ts now goes through an optional state.prepareDirectSend(url, input, init) hook with three outcomes:

  • as-is — no lane touched this request (hook absent or no turn id) → byte-identical to today's behavior;
  • restored — the lane hands back the stashed pre-swap body; the direct send carries the ORIGINAL full payload (availability preserved);
  • unrestorable — the body WAS mutated but the original is gone (stale/gone turn id, hook crash) → the direct send is refused locally with a TypeError telling the host to retry the turn (safety: mutated bytes never reach the upstream).

Correlation:

All six real-upstream fall-back sites are guarded (routed gate-fail, routed give-up, signed passthrough opt-in, raw gate-direct, bootstrap-fail, rewrite give-up); non-model URLs, compaction diverts, self-URL sends and retry-to-recovered-proxy paths are untouched by construction.

Behavior changes (disclosure)

  • Non-incremental traffic: zero change (hook absent ⇒ as-is, byte-identical dispatch).
  • Incremental traffic in the "proxy died within the probe window AND respawn found nothing" window: OLD = mutated delta forwarded upstream → 400 this turn; NEW = restored full-body direct send succeeds (availability), or a local TypeError when the original bytes are unrecoverable (host retry re-runs the normal decision). No data loss either way — the host keeps full history and the server canonical state never saw the delta.
  • onGiveUp still fires exactly where it fired before; the refusal path additionally reports onDispatch(url, "refused") for observability.

Config surface

None. No new env var, config key, CLI flag, or default value. No new dependency.

Kernel boundary

Not touched. All changes are host-side (src/agent/*).

Tests

New tests/issue3304-incremental-direct-send.test.ts (23 tests): the strict hook's five states; envelope sniffing accept/reject matrix; withRestoredBody semantics (string/Request inputs, init precedence); integration through installNativeFetchIntercept with a dead-proxy stub — rewrite give-up restores the original bytes, routed give-up restores through makeTarget(routedTarget), missing stash refuses WITHOUT any upstream call, throwing hook fails safe, bootstrap-fail restores a swapped Request-object input (opencode shape), and no-hook falls back byte-identically to pre-fix; plus per-adapter assertions that dsh/omp/opencode/pi each stash the EXACT pre-swap bytes under their correlation token. Adjacent suites green locally (native-intercept, native-intercept-selfheal, opencode-incremental, omp-native-incremental); full suite delegates to CI.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-17_incremental-direct-send-guard (d08d5c6)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4309-pr.3529.9c727d5

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3529.tgz
npm install -g package

Install instructions are refreshed on each push.

Catch 2026-10-17_incremental-direct-send-guard up to current master (13 commits,
incl. the #3394/#3395 live-state-slot refactor that rewrote installNativeFetchIntercept).
Single conflict in src/agent/native-intercept.ts: kept the #3304 withRestoredBody helper
and adopted #3394's installNativeFetchIntercept(initialState) signature + doc-comment.
The #3304 guard (prepareForDirectSend/directSendOrRefuse) sits inside the per-request
fetch-chain callback after the 'const state = g[ACTIVE_STATE]' binding, so it reads the
active state under host hot-reimport - semantically compatible with #3394.
Verified locally: tsc --noEmit clean; 107 targeted tests green
(issue3304-incremental-direct-send + native-intercept + native-intercept-selfheal).
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 PR #3529 本身就是 #3304 的修复 PR —— #3304 是该缺陷唯一的源 issue(open,带「发版/下次发版」标签),描述的正是「增量 lane 把 delta envelope 转给上游导致 400」,本 PR 的 fix + test 与之逐一对应。近期未发现针对同一失败签名的其它 open/closed issue 或 PR。

独立审完(未照抄 issue 结论,diff 逐段重读 + 本地复现验证)。先说结论:技术方向对、测试扎实、自身 head 上 CI 曾全绿;唯一阻塞是与 master 刚合入的 #3394 的冲突——我已替你 sync 到当前 master 并本地验证通过,现可直接等 fresh CI 绿后人工合并。

结构化八问

  1. 是什么问题:bili 代理在增量压缩轮次中途挂掉、且 respawn 探不到新代理时,native-intercept 的「降级直连上游」兜底会把 lane 已经替换成增量 delta envelope(没有顶层 model 字段)的请求体原样发给真实模型 API → 硬 400,这一轮直接失败。它悄悄打破了增量 lane 落地前「代理挂了还能全量直连」的既有保证。

  2. 什么原因导致的:根因是增量 lane 在 fetch 层之前就改写了请求体(换成 {conversationId,sinceSeq,path,request,messages} + x-bili-plugin-turn),而两条 give-up 兜底路径把被改写的字节直接转发给了一个解不了它的 provider。修的是「改写字节到达未准备好的发送方」这个源头,不是在传输途中过滤/伪装症状(疏而非堵,litmus 通过)。

  3. 影响范围与严重程度:只影响适配器在 intercept 之前就 swap 请求体的四个实例(dsh / omp / opencode / pi),且恰好落在「代理死在 30s 探测窗口内 + respawn 什么都没拉起」这个窄窗口。增量功能默认关闭(rollout 0%)→ 当前线上零暴露;rollout 后最坏是单轮失败(下一轮自愈),无数据丢失(host 保留完整历史,服务端 canonical state 从未见过这个 delta)。属「特定条件下的单轮降级」,不是全局不可用。

  4. 怎么修复的:A+B 混合。每个真实上游兜底直连都先过一个可选 prepareDirectSend hook:① 拿得到 lane 暂存的 swap 前原始体 → 用原始完整体直连(恢复可用性);② 拿不到(turn id 失效 / hook 抛错)→ 本地拒绝并抛 TypeError 提示 host 重试本轮(安全兜底,绝不把 lane-only 字节发给上游)。非增量 lane 没有 hook → 走 as-is,与今天逐字节一致。

  5. 是否有回归问题:无。 我把 installNativeFetchIntercept 里所有 send() 站点逐一清点:as-is 路径逐字节不变(有测试钉住);只有那个 bug 窗口的行为改变(正是本 PR 目的);无 config / persistence / wire-shape / kernel 漂移;恢复发送携带的是合法完整原始体,x-bili-turn-id header 对上游无害(provider 忽略未知头)。另见下方「我处理的阻塞项」:本 PR 与 master 上的 [BUG] DSH 插件热替换后 fetch 拦截仍绑在旧模块实例上 → 模型请求全部直连、压缩静默失效直到重启 #3394 在同一函数冲突,已 sync 并验证。

  6. 是否破坏缓存:不涉及。不碰缓存 / 重试游标 / 去重键;canonical session state 从未写入过该 delta,host 侧完整历史保留。

  7. 是否改了设计方向:不改。只在既有 native-intercept 上补一个防御性的 prepareDirectSend hook(未来新增 lane 也可复用),数据流不变。

  8. 是否引入新配置:无。不加任何 env / config / flag / 默认值 / 依赖。

我处理的阻塞项(已 push)

PR 原本基于旧 base,和 master 刚合入的 #3394/#3395(installNativeFetchIntercept 参数改名 state→initialState + 引入每请求 ACTIVE_STATE live-state-slot)在 src/agent/native-intercept.ts 单点冲突。我已把分支 sync 到当前 master(merge commit 2a97c8a46,快进推送、无 force),冲突解法 = 保留 #3304 的 withRestoredBody + 采用 #3394 的新签名/注释。语义安全已核实:#3304 的 guard(prepareForDirectSend/directSendOrRefuse)位于每请求 fetch-chain 回调内、const state = g[ACTIVE_STATE] 绑定之后,所以 host 热重导入下读的是 active state,与 #3394 天然兼容。

  • 本地验证(sync 后):tsc --noEmit 干净;定向测试 107 条全绿(issue3304-incremental-direct-send + native-intercept + native-intercept-selfheal)。
  • 现状:PR 现 mergeable=true(冲突消除)、已追平 master;fresh CI 正在跑(21 个 check 排队中)。

测试有效性

真断言、有牙:校验真实转发字节 —— 恢复时 calls[1].body === 原始体、拒绝时零上游调用、bootstrap-fail 的 Request-object 恢复对应 opencode 形态、无 hook 时逐字节等同旧行为;4 个 adapter 的 stash 精确性 opencode/omp 走本地 probe server 实测,dsh/pi 经共享 hook + 代码核对。

一处非阻塞小建议:pi 的 piIncrementalOriginals map 只有 TTL 清理,没有像 sibling originalBodies 那样的 256 上限。因 TTL 有界、单进程、并发会话少,实际风险可忽略 —— 建议后续对齐一个 evict-oldest 上限即可,不作为合并门槛。

判定

可合并(待 fresh CI 绿;人工合并)。不建议 auto-merge —— 按 §7.4 属 must-stay-human:动了 fallback 行为(产品决策)+ 新增 hook 机制 + load-bearing wire-adjacent 路径。CI 我盯着,绿了就能合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

跟进(接上一条八问审核):已把本分支 sync 到当前 master,解决了与刚合入的 #3394/#3395(live-state-slot 重构)在 src/agent/native-intercept.ts 的单点冲突 —— 保留 #3304 的 withRestoredBody + 采用 #3394 的新签名;guard 位于每请求回调内、ACTIVE_STATE 绑定之后,热重导入下读 active state,语义兼容。

本地验证通过:tsc --noEmit 干净 + 定向测试 107 条全绿(issue3304 + native-intercept + selfheal)。快进推送到分支(merge commit 2a97c8a46,无 force)。

现状:PR mergeable=true、已追平 master;fresh CI 零失败 —— 全部 fast gate + build + docs-sync 绿,仅剩 6 个全量测试分片(ubuntu/windows × node 22/24 × 3)在收尾。

判定不变:full CI 绿后人工合并即可(不 auto-merge —— 动了 fallback 行为 + 新增 hook + load-bearing wire 路径,按 §7.4 属人工合并)。

ework-agent added 2 commits October 18, 2026 13:52
Only conflict: src/agent/native-intercept.ts — both sides appended a new
export after replaceRequestPayload. Kept BOTH (union of intent):
- withRestoredBody (#3304, this PR) — full-body restore for fall-back sends
- bindLiveStateSlot (#3394/#3558, master) — live-state slot rebind on install

Evidence: typecheck + tsup build green; targeted suites green on the merged
tree — issue3304-incremental-direct-send, native-intercept (covers
bindLiveStateSlot/ACTIVE_STATE + hot-replace), native-intercept-selfheal,
opencode-incremental, omp-native-incremental, incremental-lane,
incremental-client-recovery (130 pass / 0 fail). Full-suite verdict per §7.2:
remote CI on this pushed head.
@ranxianglei
ranxianglei merged commit 2524598 into master Oct 18, 2026
41 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.566-master.2524598 (rolling master tag).

npm install -g billion-context@0.0.566-master.2524598  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant