Skip to content

fix(#1812): fold integrity — net-shrink gate, post-rebuild meter anchor, summary budget, image-arb compression (A/B/D/E, review pending) - #1818

Merged
ranxianglei merged 5 commits into
masterfrom
2026-10-01_vllm-overflow-recognition
Oct 6, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-10-01_vllm-overflow-recognition

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Fold-integrity bundle from #1812 — four defects, all ework-agent originals, directional review pending (per owner). The other two legs of #1812 landed in their own PRs: dialect recognition/honest-failure → #1838, dsh window handshake → #1836.

Defect Origin Mechanism Commit
A #1819 (split from #1812, merged back) fold accepted even when summary+retained ≥ folded span → net growth reported as "N tokens saved" 6ee3c8b4
B #1820 (split from #1812, merged back) post-rebuild usage meter anchored to stale inflated reference (observed 358% vs true) → false EMERGENCY nudge right after rebuild d5572e45
D #1775 (independent report) summarizeRange prompts carry no output budget; over-cap summaries discarded whole after halving retries (halving can't fix tool-dense spans) c1803306
E #1800 (independent report) #496 image arbitration short-circuit permanently disables auto-compression for image sessions while still forwarding 97e72ea4

Semantics kept from the original stack assembly:

  • D's truncate-to-cap rescue output additionally passes A's net-shrink gate (truncation produces it, monotonicity admits it);
  • E's outbound = rebuilt preserved alongside B's post-rebuild meter anchor (setPostRebuildAnchor).

Local: typecheck clean; per-defect suites (issue1819/issue1820/summary-length-budget/issue1800/preflight*) + e2e-overflow-selfheal/e2e-grow-compress all green on this head. These fixes only pin post-fix behavior — red/green verification against the pre-fix defect is part of the pending review.

Behavior changes (disclosed, old → new)

  • A: non-shrinking folds previously accepted + misreported. Now rejected (context flat) and the receipt reports net growth honestly.
  • B: post-rebuild metering from stale-inflated reference (~3.4×) → rebuilt payload size; false EMERGENCY T1 right after rebuild no longer fires.
  • D: prompts carry an explicit per-chunk character budget; an over-cap candidate still shorter than the folded content is truncated at a line boundary and folded (instead of discarded).
  • E: arbitration short-circuits only when nothing is compressible; otherwise folds the compressible text first, then forwards for upstream billing arbitration.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-01_vllm-overflow-recognition (99e2150)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2382-pr.1818.db13b28

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1818.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 PR #1818 review — 可合并(mergeable)。逐项核验如下(审核姿态:先审原 issue,再回归清点,再代码/测试/diff)。

验证过的证据(不是只读 diff)

  • 130k上下文窗口被占满触发无效压缩越压越大 #1812 日志取证(5.9MB / 30637 行):vLLM 原文 prompt (N tokens) + max tokens (M) exceeds the context (131072); requests are never truncated 出现 15 次(重试循环坐实);首次 400 前一行的 nudge meter 正是 usage=46% (69472/150000) —— 按错误声明窗口计量的 46%;全日志 7 条 "armed emergency shrink" 全部来自 5xx/网络失败路径,没有一条来自 vLLM overflow(识别缺失坐实);"refolded and re-sent" 出现 0 次。
  • 代码级判据实验:用日志里的逐字 body 分别跑 master 与 PR head 的 inspectContextOverflow —— master:isOverflow:false(逐字 400 透传,无 arm 无 refold);PR head:isOverflow:true, window:131072, promptTokens:69886(两个括号数字各归其位,没把 prompt 大小误当窗口)。
  • 旧 clamp 数学复算:refold 时 armed baseline=131072(source=usage)喂进 max(baseline, est)、对照声明窗口 200000 → cap≈62374 < 65536,重试 75k+62374≈137k > 131072 仍溢出;新算法 min(声明, stated)=131072 + fresh estimate + stated-prompt floor → 重试必然落回窗口内。缺陷描述与代码一致。
  • Rebase 到当前 master(bdc170b,落后 183 提交):干净应用,无文本冲突;master 侧对 inspectContextOverflow/overflowRefold/clampOutgoingOutput 区域零触碰(git log -S 为空)。rebased head 本地全绿:typecheck ✅ · 3480 tests / 0 fail / 19 skip ✅ · build ✅(测试数从作者的 3236 涨到 3480 是 master 新增用例,非本 PR)。
  • CI:pushed head 全绿(test ubuntu 22/24 + windows 22/24、e2e(codex 真实链路)、registry e2e、build-artifact 等 13 项),GitHub mergeable: true。作者沙箱里 codex e2e warmup 失败确属环境因素——CI 上通过。
  • 交付物合规:两个相邻发现已按政策拆成 preflight accepts folds that EXPAND context when a weak summarizer regurgitates (越压越大) #1819(越压越大)、post-rebuild usage estimate inflates ~3.4x after a big fold, firing an EMERGENCY T1 nudge into an over-budget context #1820(重建后 usage 虚高 ~3.4× 触发 EMERGENCY T1),均带合规 来源: #1812 … 标记、一 issue 一根因,且日志中确有对应现象(shrink 347% ×3、EMERGENCY T1 行在案)。

结构化七问

  1. 是什么问题:vLLM 方言的上下文溢出 400 不被识别 → 无 arm、无 使用pi 压缩后就提示prompt is too long #1195 救援,客户端把 400 当致命错误陷入重试死循环;同时 nudge meter 按错误的声明窗口显示"才用了 46%"。叠加一个潜伏缺陷:即便识别了,refold 重建的输出预算也对着声明窗口+被污染的 baseline 计算,重试大概率仍溢出。
  2. 什么原因导致的:两层独立根因。(a) CONTEXT_OVERFLOW_PATTERNS 只有 /exceeds the context window/i,而 vLLM 措辞是 exceeds the context (N)(无 "window" 一词)→ 正则失配;(b) context window problem continues #987 arm 把 lastInputTokens 置为 stated 窗口后,使用pi 压缩后就提示prompt is too long #1195 refold 的 clamp 仍用 max(baseline, est) 对照声明窗口——baseline 恰好等于真实窗口上限,算出的 cap 对真实窗口毫无意义。修的都是决策点本身,不是症状遮蔽(判据:vLLM 若再换措辞,扩展点就是既有的 pattern 表,机制不变)。
  3. 怎么修复的:(a) inspector 增加 vLLM 方言模式 + 从 exceeds the context (W) 提取 stated 窗口 + 从 prompt (N tokens) 提取上游 tokenizer 的精确 prompt 计数(仅内存传递,不落盘);(b) refold 重建的 clamp 改为对照 min(declared, stated)、输入用全新估计并以下游自报 prompt 数为下界(防 CJK chars/4 低估);普通非 refold clamp 路径字节级不变;confirmedContextLimits 不学习(context window problem continues #987 策略保持)。
  4. 是否有回归问题:审过以下既有行为——① 正常路径输出 clamp:表达式与日志格式逐字不变 ✅;② 其余溢出方言(Anthropic/llama.cpp/Gemini/SGLang/OpenAI):新模式为 OR 追加、parse 分支插在方言专属分支之后,既有方言测试全绿 ✅;③ arm/持久化/不学习语义:代码核对 + 测试钉死(confirmedContextLimits === undefined)✅;④ side-request guard 读取的 overflowArmTokens 字段未动 ✅;⑤ 两种压缩模式(plugin/proxy):diff 不含模式分支代码,共享 prepare* 路径两侧同效 ✅;⑥ wire 形状:只改 bili 本就写入的 max_tokens 系字段的取值,工具参数逐字节不变 ✅。一处需知会的语义扩展(非本 PR 引入):vLLM 400 现在会进入 使用pi 压缩后就提示prompt is too long #1195 refold 通道,若 refold 重试也失败,客户端看到的是重试自身的裁决而非原始 400——这是 使用pi 压缩后就提示prompt is too long #1195 既有设计("answer with the retry's own verdict"),一致延伸;PR 描述里"the original 400 still passes through verbatim"一句对此情形表述略不准(仅 retry 无可读 body 时才回退原始 400),建议改一句描述,代码无需动。
  5. 是否破坏缓存:否。正常请求体字节级不变;refold 重发的是首次 forward 已被拒的请求,不存在可失效的上游前缀缓存条目;会话级 arm 字段为既有行为。
  6. 是否改了设计方向:否。在既有 context window problem continues #987/使用pi 压缩后就提示prompt is too long #1195 救援机制内补一个方言入口 + 修正该机制内的一个潜伏 clamp 缺陷,无架构/边界/数据流转向。
  7. 是否引入了新的配置:无新增 env/config/CLI flag/默认值(PR 声称与 diff 相符,已核对)。

行为变更披露(有意,已按政策在 PR 正文列出旧→新):① vLLM 方言 400:恒透传 → 识别+arm(stated 窗口)+同请求一次 refold 重发,失败则回退透传,客户端可见契约只会变好;② refold 期间输出预算:对照声明窗口+armed baseline → 对照 stated 窗口+fresh estimate(仅 refold 路径)。附带效应:nudge meter 在 arm 后按 stated 窗口计量,从虚假的 ~46% 回到接近真实的占比——这正是修复目的之一。

Auto-merge gate(§7.4)八项全过;合并动作本身留给人工。判定:可合并。

@ranxianglei
ranxianglei force-pushed the 2026-10-01_vllm-overflow-recognition branch from 62cf0d6 to 5f444f5 Compare October 1, 2026 05:29
@ranxianglei ranxianglei changed the title fix(#1812): recognize vLLM overflow errors; clamp refold output budget to the stated window fix(#1812): vLLM overflow recognition, reject expanding folds, anchor post-rebuild meter Oct 1, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-01_vllm-overflow-recognition branch from 71520a8 to 576cb68 Compare October 1, 2026 07:26
@ranxianglei
ranxianglei force-pushed the 2026-10-01_vllm-overflow-recognition branch from 576cb68 to 97e72ea Compare October 1, 2026 07:44
@ranxianglei ranxianglei changed the title fix(#1812): vLLM overflow recognition, reject expanding folds, anchor post-rebuild meter fix(#1812): fold integrity — net-shrink gate, post-rebuild meter anchor, summary budget, image-arb compression (A/B/D/E, review pending) Oct 1, 2026
antclt pushed a commit to antclt/billion-context that referenced this pull request Oct 1, 2026
…r strips x-bili-plugin-context-window for the process lifetime

Root cause of the ranxianglei#1812 mis-sizing handshake gap: dsh's catalog always
knows a window (user-set models[].contextWindow or the built-in default),
and headersFor stamps x-bili-plugin-context-window from the
ModelInfoCache — but refreshModelInfo cached {provider, model} (no
window) whenever resolveModelInfo failed once, and the early-return only
compared provider+model, so a single boot-time race (catalog still
loading) latched the process headerless. Every request of the session
then sized against bili's unconfigured 200k fallback.

Observed live on the same dsh install against the same model: two proxy
runs sent x-bili-plugin-model without any context-window header (proxy
warned ranxianglei#1569 and only survived via a lucky registry-peek match), a third
run sent the header on every request — intermittency = the boot race.

Fix: a matching cache entry is final only when it carries a window; a
failure-shaped entry retries after a 30s cooldown (test hook:
BILI_MODEL_INFO_RETRY_MS). Successful resolves with a window clear the
retry state; selection-change semantics from ranxianglei#956 are untouched.

Test: new dsh-native unit test — resolve fails twice, model id still
stamped, window absent; after the cooldown the next stamp carries
x-bili-plugin-context-window: 262144 (attempts >= 3).

Part of ranxianglei#1812 (handshake root cause; the error-text extraction rework
stays in ranxianglei#1818).
@ranxianglei
ranxianglei force-pushed the 2026-10-01_vllm-overflow-recognition branch from 97e72ea to 437f63e Compare October 6, 2026 14:45
@ranxianglei

Copy link
Copy Markdown
Owner Author

已 rebase 到当前 master(437f63e),冲突已解,CI 应绿。 语义合并说明:

  1. post-rebuild usage estimate inflates ~3.4x after a big fold, firing an EMERGENCY T1 nudge into an over-budget context #1820(post-rebuild 锚点)适配 master 的 DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933/preflight: one failed summarization call fails the whole turn with a non-retryable 502 and blocks the session #1987 形态:
  2. 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800(图片单独超窗的文本折叠)rebase 后为空提交,已丢弃:master 的 Bill images and text separately — the image estimate's ±15× error bar poisons every window decision (dual-channel accounting) #1843 双通道方案 + 演进版 tests/issue1800-image-arbitration-compress.test.ts 已完整吸收其全部语义(冲突解决全取 master 侧即为等价结果)。
  3. 测试适配 master API:applyRanges 现返回 ProxyToolResult([decompress] toFile 将虚拟 URI 当磁盘路径,导出写入失败仍返回成功 #2204),preflight accepts folds that EXPAND context when a weak summarizer regurgitates (越压越大) #1819 测试改为取 .text;address() 类型收紧,post-rebuild usage estimate inflates ~3.4x after a big fold, firing an EMERGENCY T1 nudge into an over-budget context #1820 测试加 cast。

本地验证:tsc 干净;npm run build 通过;PR 自带 3 套件(#1800/#1819/#1820)13 用例全绿;受影响区域回归(cache-ledger、#1933/#1987/#2117/#2122 校准、preflight-compress×3、#1993/#2189、#857)全绿。

方向不变:fold integrity A/B/D/E。review 后即可合并。

ework-agent added 3 commits October 6, 2026 23:00
summarizeRange prompt now carries an explicit per-chunk character budget
(maxSummaryLength spread across chunks minus join gaps); an assembled
summary over the cap but still shorter than the folded content is
truncated at a line boundary to the cap and folded instead of being
discarded whole after halving retries.
…w estimate

On bytes-billing upstreams (unknown hosts) the conservative base64/4 image
estimate can push the payload estimate over the context window while the TEXT
portion still fits. The #496 arbitration branch returned `prepared`
unconditionally in that case, skipping preflightCompress entirely and
recording no compression state — so auto-compression stayed permanently
disabled for image sessions (issue #1800: preflightCompress ran 0 times across
the whole session while every request was forwarded).

The branch now short-circuits ONLY when there is genuinely nothing to fold
(compressibleRanges empty). Otherwise it remembers the arbitration, runs
preflightCompress to fold the compressible text portion, and — if images alone
still keep the residual estimate over the window with no upstream overflow
evidence — forwards the folded payload for the upstream to arbitrate billing.
The image rides along byte-exact either way. No config/env surface added.

Adds tests/issue1800-image-arbitration-compress.test.ts pinning the regression
(verified: fails without the fix with 0 summarization calls, passes with it).
Rebase adaptation only — no behavior change:
- issue1819 suite: applyRanges now returns ProxyToolResult (#1875); read .text
- issue1820 suite: narrow server.address() like every other e2e harness
@github-actions

github-actions Bot commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.86-master.2ab0903 (rolling master tag).

npm install -g billion-context@0.0.86-master.2ab0903  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant