Repository navigation
[PR] fix(omp): arm the direct-send window guard so a proxy outage cannot egress an oversized history (#5032) - #5077
ranxianglei wants to merge 10 commits into
Conversation
…not egress an oversized history (#5025) Without a live proxy, directSendOrRefuse measured the raw body against a window it read from the stamped x-bili-plugin-context-window header or from state.declaredWindowForDirect. On the pi lane BOTH were empty on exactly that path: before_provider_headers returns at `proxyBase === undefined` before it stamps the header, and the lane never armed the slot (#4894 armed dsh only). The check then failed open by construction, so a long session's unfurled history (multi-million tokens) went straight to the provider, whose "input exceeds the context window" 400 repeated on every retry and made the session unrecoverable for as long as the proxy stayed down. Record the per-request ctx.model.contextWindow at the TOP of the headers hook (before the readiness return) and arm it as declaredWindowForDirect in pi-native.ts. An unknown window records undefined — the guard keeps failing open instead of guessing — and a healthy proxy is unchanged, since the stamped header still outranks the reader. Verification: tests/issue5025-pi-direct-window.test.ts (5 cases: recording before the early return, unknown-window reset, the armed reader, local oversize refusal with no upstream egress, and no false refusal when the body fits); typecheck; issue4894/pi-native/issue3433/issue2072/fix-1897 suites; unused-export and test-import-graph gates.
…nnot egress an oversized history (#5032)
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.10905-pr.5077.c566fa1This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5077.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。本 PR 即 #5032 清单里「omp 半边」的修复本体;open+closed 按失败签名(declaredWindowForDirect / oversize-refusal-fallback / direct window guard)重查无其他条目或在途 PR 覆盖 omp 车道。同家族条目关系:#5027 = pi 半边(本 PR 叠在其 head 之上,需先合),#5037 = opencode 半边(文件不相交,无依赖)。 标签: bug, omp, scope/wiring, p1 —— 缺陷类;缺的是 native-intercept 共享护栏的接线(协议接线层);严重度与 #5032/#5025 同级——代理宕机期间超窗 omp 会话每次重试重发整个未折叠历史 → 上游 400 死循环,会话完全不可用 = 用户不能用,p1。 开始审核:方向核对(原 issue #5032)→ 回归清点(触碰路径既有行为)→ 代码与测试有效性 → CI 状态(读镜像缓存)。结论按八问体例回复,稍候。 |
(ai: vllm-qwen/qwen3.8-27b) 审核结论:可合并——唯一前置条件是先合 #5027(stack base)。 查重:不重复(omp 半边,与 #5027/pi、#5037/opencode、#4894→#4904/dsh 各占一车道)。 按八问逐项:
测试有效性:5 个用例全 hermetic、走真入口( CI:#5077 head |
# Conflicts: # src/agent/pi.ts
… inline default flip arms an origin waiter at import time that resolves the inline pseudo-origin, so awaitNativeProxyOrigin() no longer stays undefined and the direct-send early return this suite pins is never reached (manifest fetch lands in the hermetic sink); set BILI_PI_INLINE=0 before a dynamic pi-native import (same pin family as e2e-native-pi #4599, same import idiom as issue1958-routed-warn.test.ts)
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE (posted as a comment-style review — GitHub rejects event=APPROVE from this account because it authored the PR; the approval judgment stands as written below) — the omp half of #5032 is correct, complete, and well-pinned. Verified independently against master @ 58a8796 (= PR base; the branch contains the current master tip, so the effective merged state is exactly this diff).
Direction (issue #5032): the failure is real and the diagnosis is right. On a degraded direct send, directSendOrRefuse resolves its window as declaredWindowFromHeaders(...) ?? state.declaredWindowForDirect?.() (src/agent/native-intercept.ts:1804); omp never fires before_provider_headers (confirmed by four pre-existing code comments: src/agent/omp-native.ts:14, src/agent/pi.ts:1351, :1377, :1589), stamps no header, and had never armed the reader — so the guard silently failed open in exactly the outage state it exists for. Supplying the missing window input at the arm (rather than filtering payloads in transit) is the right layer.
Correctness:
- The recording seam is omp's only per-request event (
before_provider_request), recorded synchronously as its first statement — before any await — so it lands even when every readiness await below degrades to wire mode. Gated onagent === "omp"; the onlycreateBiliPluginconsumers are pi (default) and omp (grep-verified across src/), so no cross-lane writer race in one process. - Window source =
ctx.model.contextWindow, byte-for-byte the valuereportOmpRuntimeInfoalready ships server-side assource: "client-config"(src/agent/pi.ts:1398–1401) — the guard measures what the proxy enforces. Unknown/absent ⇒ cleared ⇒ fail-open; never guessed. - The arm sits with the other module-level arms in src/agent/omp-native.ts (same object, same process lifetime as
prepareDirectSend/beforeSend); header precedence over the reader is untouched; all degraded call sites funnel through the one chokepoint. - Stacked #5027 content (pi lane) sanity-checked: identical pattern, its own suite green; the "merge #5027 first" note stands (those symbols are not yet on master).
Regression audit (prior behavior on touched paths):
before_provider_headers: two synchronous statements added at the top of the existing try — no awaits inserted, no early return moved, rest byte-identical.before_provider_request: zero change for the pi lane (gated); omp gains one non-throwing sync read at the top.- Degraded omp sends WITHOUT a declared window: reader returns undefined ⇒ measurement skipped entirely (no clone, no buffer) ⇒ byte-identical to pre-PR.
- Fitting bodies: doom check passes ⇒ direct send proceeds unchanged (pinned by test 5).
- Wire/proxy-mode paths, config surface, persistence,
kernel/: untouched. - One intended feed: the resolved window also reaches
noteResolvedContextWindow(native-intercept.ts:1805, #5996 empty-slot doom check) — a designed consumer, same value class.
No unintended regression found.
Behavior change (disclosed in PR body; wording is accurate): OLD — once omp's proxy died, any-size history egressed uncompressed → upstream "input exceeds context window" on every retry, unrecoverable while down. NEW — when the client declared a window, bodies whose core text provably exceeds it are refused locally (HTTP 422, x-bili-gate: oversize-refusal-fallback, retryable wording), zero egress; everything else unchanged.
Tests: five cases pin BOTH directions per #3617 (record-before-await / unknown-clears / arm-in-place / oversized → local 422 + empty egress sink / fitting → 200 + egress intact). Assertions have teeth: disarming the reader flips case 4 to 200+egress; a false refusal flips case 5. Hermetic like the sibling suites (XDG_STATE_HOME redirect, env save/restore, fetch stub restored in finally, _resetForTest). The new file needs no registration — CI shards by the readdirSync(testsDir) glob. Ran locally on the PR head: 15/15 (issue5032-omp + issue5025-pi + issue4894 shared-guard) and 161/161 across adjacent suites touching these modules (omp-native, pi-native, native-intercept, issue5032-opencode, omp-native-identity, issue1958-routed-warn). Remote CI on head 5912250: all check-runs completed green (37 success + 13 skipped, 0 failures) — verified with a single check-runs GET because the mirror snapshot was ~17h stale with pendings still listed.
Note: the opencode half already landed on master (6393e0b, tests/issue5032-opencode-direct-window.test.ts) — no overlap. No small findings requiring fixes.
八问(中文):
- 是什么问题:omp 车道在代理挂掉降级直发时,把超大历史不加压缩直接发给上游;上游必然 400,且每次重试重发同样内容——代理恢复前该轮永远卡死(#5025 家族在 omp 上的残留)。
- 什么原因导致的:共享超限守卫的窗口来源有两个——请求头 x-bili-plugin-context-window 与车道自装的 declaredWindowForDirect 读取器;omp 从不触发 before_provider_headers、从不打该头,而本 PR 之前也没装读取器,两个来源在降级路径上恒空 ⇒ 守卫整体失效(fail-open)。
- 影响范围与严重程度:仅 omp 原生车道 + 代理故障窗口期;期间历史超出模型窗口时完全不可用(整轮卡在上游 400 循环)——特定条件下的可用性缺陷;窗口内请求与无声明窗口的场景不受影响。
- 怎么修复的:在 omp 唯一逐请求事件(before_provider_request)最前面同步记录客户端声明的 ctx.model.contextWindow(与 reportOmpRuntimeInfo 上报的 client-config 同值),并在 omp-native 模块级把该读取器装进共享守卫——治本(补上守卫缺失的输入),不在传输层过滤。
- 是否有回归问题:逐项审过既有行为——headers 处理器仅在原 try 顶部加两条同步语句;request 处理器对 pi 车道零改动(有 agent 门);无声明窗口的降级发送仍完全跳过测量(不克隆不缓冲,字节级不变);窗口内请求直发字节级不变;wire/proxy 模式、配置面、持久化、kernel 均未触碰。未发现无意回归。
- 是否破坏缓存:不涉及——不改任何缓存/重试/游标/去重键;唯一新增数据流是解析出的窗口喂给 #5996 的空槽 doom 检查(native-intercept.ts:1805),本就是设计好的消费方且值同源。
- 是否改了设计方向:未转向——沿用 #4894/#5027 既定的「车道供窗、共享守卫测量」架构,只是补齐 omp 这一格;opencode 半区已在 master(6393e0be85),无重叠。
- 是否引入了新的配置:无新配置/env/默认值;行为变更已在 PR 正文披露(旧:任意大小放行→上游 400 循环;新:已声明窗口且核心文本可证超窗时本地 422 x-bili-gate: oversize-refusal-fallback、零出站,其余不变)。
判定:可合并(已提交 APPROVE review)。 合并顺序提示:本 PR 堆叠在 #5027 之上(其符号随本 diff 带入),请按 PR 自身说明先合 #5027。
[bot] Review done: PR #5077 verdict APPROVE — posted as PR review id 5862121009 (comment-style because GitHub rejects |
[ework] CI fix verified green ✓ |
# Conflicts: # src/agent/omp-native.ts # tests/issue5025-pi-direct-window.test.ts
(ai: vllm-qwen/qwen3.8-27b) Conflict resolved and re-merged — PR is mergeable again, CI running on the new head. What I did this run:
八问
收工判定: 冲突已解决并推送( |
(ai: vllm-qwen/qwen3.8-27b) Red CI fixed — inherited master-level flake, not a regression from this PR. The only red check was This PR only arms the omp direct-send window guard ( Action taken (per owner policy 2026-10-24 CI coordination — inherited red converges on
Note: the mirror check-cache was stale (poller stuck, 八问(中文):
结论:红 CI 已修(继承性 flaky 测试,merge |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE (comment-style review again — GitHub rejects event=APPROVE from the author's own account; the approval judgment stands as written). Re-review of the updated head e2d66be; round 1 (review id 5862121009) covered 5912250.
What changed since the last review: two merge commits only (origin/master + origin/ci-fixes into the PR branch — the standard CI-failure coordination flow); no new content commits by the author.
- The PR's own content is byte-identical to what round 1 reviewed: the 11-line
if (agent === "omp")window-recording block at the top ofbefore_provider_request(src/agent/pi.ts:1553), the module-level armstate.declaredWindowForDirect = piDeclaredWindowForDirect+ test hook (src/agent/omp-native.ts:222, :467), and the 172-line two-direction test file (tests/issue5032-omp-direct-window.test.ts). - The stacked #5027 (pi lane) has since merged to master (
piDeclaredWindowForDirectnow at src/agent/pi.ts:880 on master) — the effective diff shrank accordingly and the merge deduplicated cleanly. The 3-dot diff against the current master tip (e0d63bf, which the branch contains → zero drift) carries only the omp hunks; no conflict-resolution damage. - The ci-fixes merge rides three flaky-test fixes (all byte-identical to origin/ci-fixes — the sanctioned sentinel flow): tests/issue4612-gc-online-guard.test.ts (21k autocommit inserts collapsed into one transaction — fsync perf, #4599, no assertion change), tests/storage-durability.test.ts (drops the #4609 verdict cache after corrupting the db so quick_check re-runs from fresh evidence, no assertion change), tests/wire-exit-matrix.test.ts (
abortAfterFirstBytenow waits up to 5 s for the proxy to actually forward before destroying the socket, with a done-guard against double-resolve — teardown assertions unchanged). All three are setup/timing fixes, not assertion weakening.
Verification on the new head: locally 15/15 core (issue5032-omp + issue5025-pi + issue4894 shared guard) and 183/183 across the three ci-fixes files plus the adjacent suites (omp-native, pi-native, native-intercept, issue5032-opencode, omp-native-identity, issue1958-routed-warn). Remote CI on e2d66be: all check-runs completed green (37 success + 13 skipped, 0 failures) — verified with a single check-runs GET; the mirror snapshot was captured mid-run with 11 pendings.
No kernel/ involvement, no new findings, no small fixes needed. The behavior-change disclosure in the PR body (fail-open egress → local 422 x-bili-gate: oversize-refusal-fallback when the client declared a window and the core text provably exceeds it) remains accurate. The round-1 merge-order note ("merge #5027 first") is now moot — #5027 is on master, so this PR can merge directly.
八问(中文):
- 是什么问题:omp 车道在代理挂掉降级直发时,把超大历史不加压缩直接发给上游,上游必然 400 且重试重发同样内容,代理恢复前该轮永远卡死(#5025 家族残留)。
- 什么原因导致的:共享超限守卫的两个窗口来源(请求头 + 车道自装读取器)在 omp 降级路径上恒空——omp 从不触发 before_provider_headers,本 PR 之前也没装读取器 ⇒ 守卫整体失效(fail-open)。
- 影响范围与严重程度:仅 omp 原生车道 + 代理故障窗口期;历史超窗时完全不可用(400 死循环)——特定条件下的可用性缺陷;窗口内请求与无声明窗口场景不受影响。
- 怎么修复的:在 omp 唯一逐请求事件最前面同步记录客户端声明的 contextWindow(与 reportOmpRuntimeInfo 上报的 client-config 同值),模块级装进共享守卫——补上守卫缺失的输入(治本),不在传输层过滤。
- 是否有回归问题:逐项审过——pi 车道零改动(agent 门);无声明窗口的降级发送仍完全跳过测量(不克隆不缓冲,字节级不变);窗口内请求直发不变;wire/proxy 模式、配置面、持久化、kernel 未触碰;本轮经 ci-fixes 带入的三个测试文件改动均为 flaky 修复(事务化插入 / 清判定缓存 / 等转发再断连),断言未削弱。未发现无意回归。
- 是否破坏缓存:不涉及——不改任何缓存/重试/游标/去重键;解析出的窗口喂给 #5996 空槽 doom 检查是设计好的消费方且值同源。
- 是否改了设计方向:未转向——沿用 #4894/#5027「车道供窗、共享守卫测量」架构,补齐 omp 一格;opencode 半区与 pi 半区均已在 master,无重叠。
- 是否引入了新的配置:无新配置/env/默认值;行为变更披露保持准确(旧:任意大小放行→上游 400 循环;新:已声明窗口且核心文本可证超窗时本地 422、零出站,其余不变)。
判定:可合并(APPROVE)。 本轮 head 已包含当前 master tip(e0d63bf8c8),无堆叠依赖残留,可直接合并。
[bot] Re-review done on updated head e2d66be: verdict APPROVE (PR review id 5914800978, comment-style because GitHub rejects self-approval). Since round 1 the branch only gained two merge commits (origin/master + origin/ci-fixes, standard CI-coordination flow): the PR's own omp-guard content is byte-identical to what was already approved in round 1, the stacked #5027 has since merged to master so the effective diff now carries only the omp hunks (3-dot diff vs current master tip e0d63bf, which the branch contains — zero drift), and the ci-fixes merge rides three flaky-test fixes that are byte-identical to origin/ci-fixes with no assertion weakening. Verified on the new head: 198 targeted tests green locally (15 core + 183 ci-fixes/adjacent), remote CI all-green (37 success + 13 skipped, 0 failures). No kernel involvement, no findings, no open ends — the round-1 "merge #5027 first" note is moot, PR can merge directly. |
Resolve conflict in src/agent/omp-native.ts: keep master's createOmpDirectSendGuard() (side-lane strict guard + prompt_cache_key drop) AND the PR's #5032 window arm (state.declaredWindowForDirect = piDeclaredWindowForDirect). Union of the ./pi.js import.
(ai: vllm-qwen/qwen3.8-27b) Resolved the merge conflict on this PR branch and pushed the merged head. What I did this run
八问 (structured review)
判定: 冲突已解决并推送(head |
(ai: vllm-qwen/qwen3.8-27b) Re-verified live before acting: the conflict is already resolved. The PR branch head is No further work needed this run. |
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Round 3 — head 1eaf49a195 (merge of master a446a3c1c8 into the PR branch).
What changed since round 2: a single merge commit. The effective 3-dot diff against current master shrank from 6 files to exactly the 3 PR files — the three ci-fixes flaky-test fixes landed on master and dropped out of the diff. The PR now contains the current master tip (zero drift, mergeable).
Re-verified on the new head:
src/agent/pi.ts(+11): the record block is byte-identical and still the FIRST statement ofbefore_provider_request(before any await) — now at pi.ts:1705,if (agent === "omp")at :1712.src/agent/omp-native.ts(+11): arm + test hook unchanged; the arm sits right after master'sstate.prepareDirectSend = createOmpDirectSendGuard();line.tests/issue5032-omp-direct-window.test.ts(+172): unchanged from rounds 1–2.
New master interaction checked (642124b, "drop the stamped prompt_cache_key on a direct send"): master swapped omp's prepareDirectSend from createPrepareDirectSend() to createOmpDirectSendGuard() (pi.ts:972 — wraps the shared prepare, strips the stamped prompt_cache_key from the outgoing text body). Composition with this PR's arm is safe by ordering: directSendOrRefuse (native-intercept.ts:1729) invokes state.prepareDirectSend FIRST (hook at :1667, unrestorable check at :1736), then resolves the window at :1805 (declaredWindowFromHeaders(...) ?? state.declaredWindowForDirect?.()), then the doom check. So the pck wrapper runs inside prepare, before measurement — the doom meter sees the actual egress bytes; the PR's arm is consumed downstream and is orthogonal; the wrapper only shrinks body text (and passes unrestorable through untouched), so it cannot turn a refusal into an egress. The wrapper is independently pinned by its own suite (tests/omp-direct-send-stamped-pck.test.ts), and the PR's arm is pinned by cases 3–5 of the PR suite. No finding.
Tests: local targeted 19/19 core (issue5032-omp 5 + issue5025-pi 5 + issue4894 5 + omp-direct-send-stamped-pck 4) + 164/164 adjacent (omp-native, pi-native, native-intercept, issue5032-opencode, omp-native-identity, issue1958-routed-warn) = 183 green on the new head. CI on 1eaf49a195: 37 success + 13 skipped, 0 failures.
Behavior change (disclosed in the PR body, still accurate): OLD — with the proxy down, an oversized omp history egressed direct to the upstream and looped on upstream 400s (the window reader was always empty ⇒ measurement skipped ⇒ fail-open). NEW — when the client has declared a window and the body is provably oversized, the send is refused locally with 422 + x-bili-gate: oversize-refusal-fallback (retryable) and nothing egresses. Fitting bodies and the no-window path are byte-identical to before.
No kernel/ touched, no kernel-change declaration, no config surface. No findings this round — nothing to fix, no decisions needed.
八问(中文):
- 是什么问题:omp 车道的直发窗口守卫没被装弹——omp 从不触发
before_provider_headers,窗口读取器永远为空,代理宕机时超大历史直接发往上游,陷入上游 400 循环。 - 什么原因导致的:共享守卫的窗口来源只有两条路——请求头或
state.declaredWindowForDirect;omp 没有盖头缝(无before_provider_headers),而 state 字段无人写入,守卫只能整轮跳过测量、fail-open。 - 影响范围与严重程度:仅 omp 车道 + 代理宕机的降级直发路径。修复前:超大历史直发上游(上游 400 循环,用户表现为请求失败);修复后:本地 422 拒绝、客户端可回退。适配体与无窗口路径不变。
- 怎么修复的:在 omp 唯一的每请求缝
before_provider_request里,在任何 await 之前同步记录ctx.model.contextWindow到模块级变量,并把读取器写入 omp-native state 的declaredWindowForDirect,复用共享守卫既有的消费点(native-intercept.ts:1805)——不新增任何测量/拒绝逻辑。 - 是否有回归问题:审过的既有行为——pi 车道(
agent !== "omp"短路,不变)、opencode 车道(独立,已在 master)、适配体直发路径(字节不变,case5 钉住)、无窗口路径(fail-open,case2 钉住清除)、unrestorable 拒绝臂(#3304,独立分支)、新合入 master 的 642124b pck 剥离(已验证组合顺序 prepare→测量→拒绝,互不干扰)。无回归。 - 是否破坏缓存:不涉及——未触碰缓存/重试/增量游标;pck 剥离是 master 自身行为(642124b004),非本 PR。
- 是否改了设计方向:不涉及——复用既有共享守卫消费点,无新架构。
- 是否引入了新的配置:不涉及——无新 env/config/CLI。
Verdict: APPROVE (comment-style — GitHub rejects event=APPROVE from the author's own account, so this review is posted as a comment carrying the approval verdict.)
[bot] PR #5077 round-3 review done (head 1eaf49a, master merge only — effective diff back to the 3 PR files): re-verified the arm + new-master interaction (642124b pck-strip composes safely, prepare→measure→refuse ordering), local 183/183 targeted, CI 37 success + 13 skipped / 0 failed. No findings. Review posted (id 5966664461, APPROVE verdict, comment-style). |
[ework] CI fix verified green ✓ |
Fixes the omp half of #5032 (the opencode half is external contribution #5037).
What & why
Same failure as #5025 on the pi lane: with no live proxy, the native intercept degrades to direct send, and the shared oversize arm in
directSendOrRefuse(src/agent/native-intercept.ts) had no window to measure against on omp — the lane never armeddeclaredWindowForDirect. Result: multi-MB raw history egresses uncompressed and loops on upstream 400s until the proxy recovers.The window source is the client-declared value
ctx.model.contextWindowfrom thebefore_provider_requestevent — the same numberreportOmpRuntimeInfo(src/agent/pi.ts, #1531/#955) already ships server-side assource: "client-config", i.e. exactly what the proxy enforces. No new config surface, no registry lookup (which would measure a different value than the proxy enforces).Changes
src/agent/pi.ts: recordctx.model.contextWindowat the top of thebefore_provider_requesthandler for the omp agent, before any readiness await; an unknown/absent window clears the record (fail-open — never guess). Reuses [PR] fix: arm the pi lane's direct-send window guard so a proxy outage cannot egress an oversized history #5027'spiDeclaredWindowForDirectreader.src/agent/omp-native.ts: armstate.declaredWindowForDirect = piDeclaredWindowForDirectalongside the existingprepareDirectSendarm (+_declaredWindowHookForTesttest hook).tests/issue5032-omp-direct-window.test.ts: 5 hermetic cases pinning record-before-await, unknown-window-clears, arm-in-place, oversized body refused locally with zero egress, and fitting body sent direct unchanged (both directions per Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617).Behavior change (flagged)
x-bili-gate: oversize-refusal-fallbackbefore anything egresses. Bodies that fit are byte-for-byte unchanged; clients with no declared window still fail open.Merge order
This branch reuses symbols introduced by #5027 (
piDeclaredWindow,piDeclaredWindowForDirect); merge #5027 first. The head is stacked directly on #5027's head, so this diff view includes #5027's files until then. It does NOT depend on #5037 (disjoint files).Verification (head 8e669d7, stacked on #5027 head 8ffff3a)
tests/issue5032-omp-direct-window.test.ts+tests/issue5025-pi-direct-window.test.ts— 10/10 passtests/issue4894-direct-window-guard.test.ts,tests/pi-native.test.ts,tests/issue3433-pi-incremental-guard.test.ts,tests/issue2072-native-caller-stamping.test.ts,tests/fix-1897-leaked-tools-side.test.ts— 38/38 passnpm run typecheckclean