Repository navigation
[PR] fix: count masked mid-history sys/dev items in responses estimates (#3713) - #3717
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4384-pr.3717.56c117cThis exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3717.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 全仓 open+closed 检索(按失败签名: k̂ 校准 / responses sys/dev 估计):本 PR 是 #3713 唯一在途修复。同家族但机制不同:#2407/#2418(k̂ 分母漏 thinking 质量,已合)、#3645/#3650(校准后 deficit 与 foldable mass 尺度不一致,仍 open,不同机制)、#2398(Responses tools 预算延后,已关)。 标签: bug, codex, dsh, scope/kernel, p2 —— 缺陷修复;现场证据点名 DSH native plugin / Codex wire;范围属上下文管理/校准核心;能用但近窗口轮次被误杀,定 p2。 审核结论(独立审读,非复述 PR 描述)① 方向(原 issue 关): 成立。我独立核对了代码路径:#1638 的掩码循环( ② 回归审查(逐项清点触碰路径的既有行为):
③ 代码: helper 放 budget.ts 合理(两侧共用正是去重);命名达意( ④ 测试有效性: 两个 pin 都走真实 proxy 的 plugin mode(
⑤ diff 整洁度: 3 个文件(+307/−16),每一行都与本 PR 目的相关;无空白重排、无 lockfile/生成物 churn;单线性 commit,base 为最新 master(0 behind)。干净。 ⑤b 代码整洁硬标准: 全项过(命名/死代码/重复/错误处理/魔法值/注释/风格/依赖/职责/测试同标准)。 结构化八问
验证与判定
|
(ai: vllm-qwen/qwen3.8-27b) CI 状态更正(接替上一条的"慢速分片"说法):
给 owner 的选项(CI 控制动作我不擅自执行):
审核判定不变:代码审核结论(建议可合)基于其余 5 个绿色 test shard + 全部门禁(static-gates/kernel-guard/config-surface-guard/docs-sync/lock-sync 等)+ 我对 diff 与测试的独立审读;这个挂起的 shard 属于 CI 基础设施噪音类(§7.2),不改变对代码的判断。 |
…3713) On the Responses lane, plugin/codex requests mask every non-leading role system/developer item in place (__bili_inplace_sysdev, #1638) while projecting to core, so those items enter neither projection.msgs nor projection.systemParts. prepare-responses built systemPromptTokens, localInputEstimate and lastLocalTextEstimate from exactly those two projections, so mid-history sys/dev mass was invisible to them while estimateWireOverhead (the preflight base multiplied by k-hat) DID bill it: every usage report settled an inflated sample and k-hat drifted toward the accumulation ratio (2.85x in the field), until raw x k-hat crossed the context limit and preflight rejected near-window turns with a non-retryable 502. Count the masked items back into all three estimates with the same extraction + chars/4 method estimateWireOverhead applies to the wire (shared helper responsesSysDevItemText), so the k-hat denominator and the preflight base measure the same set of wire items.
a09abbd to
585bbb5
Compare
(ai: vllm-qwen/qwen3.8-27b) CI is fully green on the rebased head ( Notes on the one transient red along the way:
Ready for human merge. |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.640-master.0e9ea92 # this exact build
npm install -g billion-context@master # always the newest merged build
|
What & Why — #3713
On the Responses wire, plugin/codex requests mask every non-leading
role: system|developeritem in place (__bili_inplace_sysdev, #1638) while projecting to core, so those items enter neitherprojection.msgsnorprojection.systemParts. prepare-responses built three estimates from exactly those two projections:session.stats.lastLocalTextEstimate— the k̂ denominator (the DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933/[follow-up #2369] calibration k̂ denominator omits projected thinking mass — extended-thinking routes over-scaled under the two-way clamp #2407 learning pair),session.stats.localInputEstimate,session.metadata.systemPromptTokens.Meanwhile
estimateWireOverhead— the preflight base multiplied by k̂ — counts ALL sys/dev items ininput[]. Every usage report therefore settled an inflated sample (billed ÷ under-counted denominator), and k̂ drifted toward the accumulation ratio as mid-history developer notifications grew (field: 2.85× on DSH native plugin / Codex wire).raw × k̂then crossed the context limit and preflight rejected near-window turns with a non-retryable local 502.Fix
Count the masked items back into all three estimates with the SAME extraction + chars/4 method
estimateWireOverheadapplies to the wire, extracted into a shared helperresponsesSysDevItemTextin src/server/budget.ts so both sides provably measure the same set (estimateWireOverhead's responses branch now maps through it — pure extraction refactor, byte-identical, pinned by tests/wire-overhead.test.ts).Behavior change (old → new)
lastLocalTextEstimate,localInputEstimateandsystemPromptTokensnow INCLUDE the mid-history sys/dev token mass. Old: omitted (incident turn: ~110k recorded vs ~347k actually billed). New: included. Why: the items are provider-billed and counted by the preflight base; omitting them from the denominator is what inflated k̂.Tests
delta=36, expected ≈ 6559.No config surface change. No kernel change. No version bump.
Fixes #3713