Repository navigation
[PR] fix(#4452): surface body-rewriting-gateway overflow starvation in relay log (observability stopgap) - #4455
ranxianglei wants to merge 1 commit into
Conversation
…#4452) An intermediate gateway that rewrites every reject into one generic body (Cherry Studio et al.) strips the token counts / window from each overflow 400, so inspectContextOverflow returns isOverflow:false and bili's whole overflow self-heal silently never engages — the route starves of calibration and re-sends the same oversized payload forever. Decision-free observability slice of #4452 (behavioral recovery awaits owner sign-off on thresholds): - add isMarkerlessOverflowReject() (util.ts): a 4xx overflow-shaped reject whose body matches no CONTEXT_OVERFLOW_PATTERNS (excludes 5xx, which already self-heal via the local estimate, and any body that carries a marker). - the relay logs a loud warn naming the starvation + the last meter reading (settled input vs effective window) instead of only the opaque `<- upstream 400` line. Purely additive logging — no behavior change when isOverflow is true, no new config surface.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.5351-pr.4455.0a9e787This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4455.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。本条是我自己为 #4452 落的「可观测性止血」PR(#4455,分支 按「自己的 PR → 核对代码 + 确认 CI + 报结论」处理,未重做。核对结论(结构化八问):
一个已知边界(不拦合并,供你定夺,随行为切片一起细化):谓词是「400/413 且无标记」的宽口径,所以非 overflow 的 400(典型如 strict-reasoning-echo 首次 400,body 带 reasoning_content 但无 overflow 标记)也会触发这条 warn,属误报噪声。方向上这是刻意的——宁可多报也不漏掉被剥标记的真 overflow;若要收窄(如排除含 reasoning_content/text 的 body)属于 #4452 的决策面,留到行为切片在同一分支一起定。 测试: CI:pushed head 正在跑(当前 success 25 / skipped 13 / in-progress+queued 12,暂无红)。全量回归以远程 CI 判定:代码核对通过、范围干净(仅 3 文件、纯增量日志)、可独立合入作止血;等 CI 转绿即可由你合并。 |
(ai: vllm-qwen/qwen3.8-27b) CI 转绿:pushed head( 结论不变:代码核对通过、范围干净(仅 3 文件、纯增量日志、零行为变更、零新配置)、 |
What & why
An intermediate gateway that rewrites every reject into one generic body (Cherry Studio et al.) strips the token counts / window from each overflow
400, soinspectContextOverflowreturnsisOverflow:falseand bili's whole overflow self-heal silently never engages — the route starves of calibration and re-sends the same oversized payload forever (#4452).This PR lands the decision-free observability slice of #4452 so that silent starvation becomes diagnosable now, independent of the behavioral decision still pending with the owner.
Changes
src/util.ts: addisMarkerlessOverflowReject(status, bodyText)— a 4xx overflow-shaped reject whose body matches noCONTEXT_OVERFLOW_PATTERNS. Deliberately excludes 5xx (already self-heal via the local estimate, arm emergency shrink on upstream failure to break relay-5xx deadlock #604) and any body that carries a marker (self-heal engages normally then).src/server/relay.ts: when such a marker-less 400/413 arrives, log a loud warn naming the starvation plus the last meter reading (settled input vs effective window) instead of only the opaque<- upstream 400line.tests/issue4452-markerless-reject.test.ts: pins both directions (Test Blind Spot Retrospective: Reinforcing Negative Assertion Discipline, Tightening Fake Upstream Strictness, Thin Inst | 测试盲区复盘:补反向断言纪律、假上游严格化、薄安装冒烟、形态矩阵与生命周期覆盖 #3617) — generic body -> flagged; marker-carrying (OpenAI + CJK) -> not flagged; 5xx / 2xx / other-4xx -> not flagged.Purely additive logging. No behavior change when
isOverflowis true; no new config surface.Scope / what is NOT here
The behavioral recovery (meter-corroborated arm/refold, or route degradation after N consecutive marker-less rejects) is a preflight/metering behavior change awaiting owner sign-off on thresholds (#4452 decision request). That work will land on this same branch / PR (one PR per issue). This PR is intentionally mergeable standalone as the stopgap.
Refs #4452 (not closing — behavioral fix pending).