Skip to content

[PR] fix: stop stale session input baselines outliving their measurement context (#2683) - #2725

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-11_cross-model-baseline
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-11_cross-model-baseline

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2683

What

Two producers of ONE mechanism family — session input baselines that outlive the context that measured them — each verified against the issue's live logs and reproduced with e2e fixtures:

  1. The failure arm adopts a bound, not a reading. armFailureShrink (no-usage relay 5xx / network failure) persisted the char-count UPPER BOUND of the outbound payload (estimateCoreMessagesUpper class, 1.7–4× real) as session.stats.lastInputTokens. The fix: arm emergency shrink on outbound payload size, not raw body (#1493) #1498-F1 bound floor was justified by the kernel's tool-result truncation loop consuming the arm as remaining depth (arm emergency shrink on upstream failure to break relay-5xx deadlock #604 rescue) — but since the emergency shrink / preflight 的本地估算用错了视图:超长会话(客户端历史 127 万,压缩后 outbound 仅 ~16 万)在上游失败无 usage report 时,bili 按原始历史估算 #1492/任何一轮没有拿到 usage 报告的请求,都会把上下文分母放大 12–20×(nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟) #1839 reader quarantine NO processTurn feed passes an estimate-grade arm into any band (preflight runs noEmergencyTruncate; every decision floor gates on source), so the bound's only live effect was phantom mass: inflated preflight entry budgets, responses-lane output headroom, displays — surviving model/window changes until the next usage report. (Issue's live case: 441,193 armed against a 202,885-token payload.)
    → The arm now adopts the same billing-caliber reading the preflight trigger uses (k̂-scaled chars/4 + wire overhead + image reserve), and its log names the provenance: (k̂-calibrated outbound reading) / (raw outbound reading) (issue proposal feat: OpenAI Responses API support (/v1/responses) #4).

  2. Trusted baselines survive model switches. A usage report measured under model X kept deciding requests for model Y — only ORIGIN changes demoted it (DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 F2). After a mid-session switch the foreign bill pinned the preflight loop's decisionTokens above the fold target forever (the floor survives folding — folding shrinks the payload, not the bill) → folds until summary-budget exhaustion although the payload fit. (Issue's live case: 566,322 billed under a 1M-window model judged against a 400k window → two ~3-minute emergency folds at 62% of new-window occupancy.)
    → A real model change (both sides known; unknown→known never counts) now retires the trusted-grade baseline at the per-request model-stamp site — before prepare sizes the nudge and before the trigger runs — logged with both models and the retired value. The next usage report re-anchors. Precedents: [Bug] 上下文展示估算 206K,usage 返回后为 98K:希望校准估算并区分安全上界 #2117 B already gates k̂ by model ("a factor learned on another model acts as absent"); DSH 分叉会话后触发大量 preflight overflow compress #2383 names foreign-model baselines untrusted for regime selection.

Behavior changes (old → new)

  • Armed value on a no-usage failure: char-count upper bound → billing-caliber estimate. It may now land BELOW a future bill on dense JSON/CJK routes where chars/4 undercounts (fork 会话 lastInputTokens=0 时 preflight 退化为字符估算,大载荷被低估后不压缩直接裸转上游 #553) — acceptable because estimate-grade values are quarantined from every decision floor and k̂ closes most of the gap where route+model provenance matches.
  • Usage/overflow-arm baseline + usage anchor across a model switch: trusted verbatim → retired until the next usage report. Forensic state preserved: cumulative inputTokens meter, last-usage timestamp (contextTokensAt), calibration ring, cache-ledger history. Same-model window moves do NOT retire (the bill stays valid → genuine overflow still folds); estimate-grade arms survive (byte-scale property of the payload).

Config surface

None — no new env/config/CLI keys.

Both compression modes

Retirement sits at the request-handling chokepoint before prepare/trigger, so it applies identically in plugin mode and proxy mode (the mode split happens downstream). The arm-caliber change affects both modes equally (both consume outboundPayloadBreakdown). No wire-shape change: message rebuild, system handling, tool injection, acp_summary, preflight protocol all untouched — only session-state bookkeeping values.

Tests

  • New tests/issue2683-cross-model-baseline.test.ts: G1 cross-model retirement e2e (566K billed under claude-a → claude-b turn makes ZERO summarization calls, no phantom (566000/…) meter, re-anchored to the real 6K bill); G2 no-usage network failure arms inside [chars/8, char-count) with the provenance clause, retry forwards without folding; G3 control — a SAME-MODEL over-window bill STILL FOLDS via its trusted floor (floor semantics preserved for genuine overflow).
  • Re-scaled #1498-F2 assertion in tests/issue1493-arm-outbound-payload.test.ts (raw-view measurement pinned at chars/4 scale; char-count adoption now explicitly forbidden).
  • Updated pins in tests/context-provenance-outbound.test.ts (cross-model authority loss vs same-model retention; fixture growth sized so the billing-caliber arm still outgrows the fixture bill and the replace-stale-usage mechanic keeps firing).
  • Targeted batch green on the rebased head: typecheck + 58 tests across 11 suites covering every touched consumer path (issue1493, issue1839, issue2313, issue2484, issue1595, issue1569, issue2033, silent-backend-nudge, issue857, context-provenance-outbound, issue2683). Full suite to CI.

Known side effect (disclosed): a host that mixes multiple models under one UN-FORKED session id will see freshly established baselines wiped at each switch — self-healing (the next usage report re-anchors within one round trip); persona-style hosts are forked per KDD#9 and unaffected.

@github-actions

github-actions Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-11_cross-model-baseline (5a8b5cb)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.3351-pr.2725.2b406e0

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2725.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到 PR #2725(修 #2683:跨模型陈旧基线)。开始按规范审查:① 原 issue 方向核对 → ② 回归审查 → ③ 代码审查 → ④ 测试有效性 → ⑤ diff 整洁度。结论稍后按结构化八问给出。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review #2725 — fix: stop stale session input baselines outliving their measurement context (#2683)

结论:可合(人工合并——触及 meter 决策路径与默认值语义,按 §7.4 属 human-only 层;不满足 auto-merge)。

结构化八问

  1. 是什么问题:两类「陈旧测量上下文」——①模型切换后,旧模型计费下的 input baseline(如 566,322 tokens @1m 窗口)继续按新窗口(400k)度量 → 幻影 142% meter,对已只占新窗口 62% 的 payload 反复触发紧急折叠;②网络失败无 usage 时,failure-shrink 把字符数上限估计(441,193)写进 armed 线,而真实计费约 202,885 → 同类误折叠。
  2. 什么原因导致的:input baseline(usage-grade / overflow-arm)是会话级状态,但其测量上下文(模型 tokenizer + 上下文窗口)会在会话中途变化,状态没有生命周期管理——陈旧读数在新上下文里继续参与决策;failure-shrink 取的是保守上界而非计费口径读数。根因在测量与上下文的生命周期错配,不是折叠逻辑本身。
  3. 影响范围与严重程度:切模型的会话(尤其大窗→小窗)+ 请求失败未返回 usage 的会话。症状是每几分钟一次重复紧急折叠(上下文被反复压缩、token 浪费、体验劣化);无数据丢失、不致不可用。中严重度。
  4. 怎么修复的:双管齐下,都在源头/决策点修——①retireCrossModelBaselines()(src/session.ts:575)在 handleRequest 入口检测模型 stamp 变化(handle.ts:1798-1803,lastModel 写入之前),清除 usage-grade/overflow-arm baseline,下一次真实 usage 报告自然重新锚定(G1 实测重锚到 6k);②failure-shrink 的 arm 从字符上界改为计费口径读数(chars/4 × k̂ 校准 + overhead + image,与 preflight 触发表达式同形,src/preflight.ts:1226),日志加 provenance 措辞。不是下游过滤症状,是撤掉生产者。
  5. 是否有回归问题:逐项审过既有行为,未发现回归:
    • 同模型路径不变:G3 对照测试钉住(source 保持 usage、真实 overflow 仍走 trusted floor 折叠、无误报切换日志);
    • side lane(title-gen/classify)dsh 等辅助人格在 handle.ts:1637 早退,永远碰不到 lastModel 写入 → 不会翻转模型键,KDD#9 persona-fork 兼容(fork 会话是独立 Session 对象,各自持有 baseline);
    • overflowArmTokens 的消费者(「BUG」sideRequestGuard 会永久拦截进程内第三方插件的 LLM 调用,导致与MemOS冲突 #1110 侧请求 413 守卫)清除后回落到声明窗口——对新模型是正确语义;
    • k̂ 因子仍受 [Bug] 上下文展示估算 206K,usage 返回后为 98K:希望校准估算并区分安全上界 #2117-B 模型门控(util.ts:107-112),origin 失配降级为 raw,永不为零、永不毒化读数(util.ts:86-95 已核);
    • plugin/proxy 两模式都经同一 handleRequest 入口,对称生效;持久化往返安全(undefined 字段以 legacy-unknown 语义存活);
    • 显示面:statusInputBaseline→0、displayContextBest→null 直到重锚——诚实的「未知」,仅 1-2 轮展示缺口,不进任何决策路径。
  6. 是否破坏缓存:不。retire 只清本地 stats 字段,不改消息字节、不动 system prompt,prefix cache 不受影响;k̂ 校准环保留(retire 明确不清它),重锚后校准立即复用。
  7. 是否改了设计方向:没有转向——完全落在原 issue 建议内(fix: keep system prompt byte-stable for prefix cache #1 用校准估计不用上界、fix: address all code review findings (condense, routing, session-id, dead code) #2 模型/窗口变化退役 baseline、feat: OpenAI Responses API support (/v1/responses) #4 provenance 日志),无新配置面。
  8. 是否引入新配置:无(env/config/CLI 一律未动)。

行为变更披露(旧→新 + 原因)

  • failure-shrink arm:字符数上界(Math.max(textEstimate, estimateCoreMessagesUpper)+overhead+image)→ 计费口径读数(applyEstimateCalibration(textEstimate+overhead, k̂,…)+image)。典型场景显著变小(441k→~203k 量级),消除误折叠;k̂ 不可用时降级 raw chars/4,仍远小于上界。
  • 模型切换:usage-grade baseline 跨模型存活(幻影 meter)→ 立即清除,直到下一次 usage 报告。这是本 PR 的核心行为变更,正是修复本体。
  • 日志格式:failure-shrink 行新增 (k̂-calibrated outbound reading) / (raw outbound reading) 后缀;新增 warn 行 model switched X -> Y — retired cross-model input baseline (N tokens)。
  • 已知取舍(PR 已披露,我认可):不按模型分叉会话的宿主若在同一会话内交替多模型,每次切换都会清一次 baseline——下次 usage 报告自愈,可接受。

测试有效性(④):真 e2e(startServer+fetch,非 mock 判定)。已做牙齿验证:恢复 base 版 src、保留 PR 测试 → G1/G2/#1498-F2 全部失败,G3 两侧都过 → 测试确实复现原 bug,非恒真断言。#1498-F2 从 >= TEXT_CHARS 重标度到 [TEXT_CHARS/8, TEXT_CHARS) 是对原回归(raw view 被度量而非 overhead-only)在新标度下的忠实迁移,显式禁止退回字符计数;context-provenance fixture 7500→60_000 chars 的理由成立(计费口径 arm ~15k 仍需压过 fixture 的 10k bill,replace-stale-usage 机制继续被驱动)。

验证证据:PR head 58/58 定向测试绿(typecheck 干净);rebase 模拟——cherry-pick 到最新 master(65ea85b4f)零冲突(唯一重叠文件 handle.ts 的 master 侧漂移只是 #2709 passthrough 分支,不同分支位置),rebased head 上 58/58 + typecheck 依然全绿。全量回归留给 CI(本地测试纪律)。

diff 整洁度(⑤):8 个文件全部与本 PR 目的直接相关,无跑题文件、无空白噪声。

收工判定:代码、测试、既有行为审计均通过;可合,等 CI 绿灯后由人工合并。

…ontext (#2683)

Two producers of one mechanism family - session input baselines that outlive the context that measured them:

1. armFailureShrink adopted the char-count UPPER BOUND of the outbound
   payload as session.stats.lastInputTokens (a 1.7-4x over-reading). The
   #1498-F1 bound floor predates the #1492/#1839 reader quarantine: no
   processTurn feed passes an estimate-grade arm into any kernel band
   anymore, so the bound's only live effect was phantom mass - inflated
   preflight budgets / output headroom / displays surviving a model or
   window change until the next usage report (live case: 441,193 armed
   against a 202,885-token payload; after a 1M->400k model switch the
   meter read 142% and two ~3-minute emergency folds ran on a payload
   already at 62% of the new window). The arm now adopts the billing-
   caliber reading (k-scaled chars/4 + wire overhead + image reserve),
   and its log names the provenance.

2. A usage-grade baseline measured under model X kept deciding requests
   for model Y (only ORIGIN changes demoted it, #1933 F2): after a
   mid-session model switch the foreign bill pinned the preflight loop's
   decisionTokens above the fold target forever - folding until budget
   exhaustion although the payload fit. A real model change (both sides
   known; unknown->known never counts) now retires the trusted-grade
   baseline at the per-request model-stamp site, before prepare sizes
   the nudge and before the trigger runs; the next usage report
   re-anchors. Estimate-grade values survive the switch (byte-scale
   properties of the payload); same-model window moves keep the bill
   valid, so genuine overflow still folds.

Behavior changes (old -> new):
- armed value on a no-usage failure: char-count upper bound -> billing-
  caliber estimate (it may now land BELOW a future bill on dense
  JSON/CJK routes where chars/4 undercounts, #553; acceptable because
  estimate-grade values are quarantined from every decision floor and k
  closes most of the gap where route+model provenance matches)
- usage/overflow-arm baseline + usage anchor across a model switch:
  trusted verbatim -> retired until the next usage report (forensic
  state preserved: cumulative inputTokens meter, last-usage timestamp,
  calibration ring, cache-ledger history)

Tests: tests/issue2683-cross-model-baseline.test.ts (new: G1 cross-model
retirement e2e - zero summary calls + re-anchor; G2 arm scale band
[chars/8, char-count) + provenance clause; G3 control - same-model
over-window bill still folds via its floor), re-scaled #1498-F2
assertion in tests/issue1493-arm-outbound-payload.test.ts, and updated
pins in tests/context-provenance-outbound.test.ts.
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):目标 issue #2683 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

判断:关闭有误 —— #2683 的修复不在 master 上。 已按你的留言 reopen 并续上(见文末 #4583)。

证据(2026-10-24 对 master d95799d86 逐项查证):

  1. retireCrossModelBaselines:master src/ 全库不存在;src/server/handle.ts:2321 只有裸 stamp lastModel = reqModelId,无任何退役逻辑。
  2. preflight baselineFloor 的降级仍是 ORIGIN-only(DSH: 切换模型后,preflight 在窗口远未满时提前触发压缩 #1933 F2),MODEL 维度未处理——该处注释本身就记录着 "after a mid-session model switch the stale cross-route baseline kept arming preflight"(origin 修了,model 没修)。
  3. Producer (a) 原样在:outboundPayloadBreakdown.armEstimate 仍是 Math.max(textEstimate, estimateCoreMessagesUpper(msgs)) + overhead + image——char-count 上界 arm 未变。
  4. 清理时关联的 Inbound markers: one declarative strip engine for all four wires (#2789, #2798, #2811 plan B) #2815 是另一机制家族(strip/reconcile/identity:Google (Gemini native) lane never strips ACP panels/status markers from incoming history — vestigial import in prepare-google #2798/acp_status visibility markers: stripMarkerLines removes the head line but leaves the multi-line report body in incoming history (#1029 follow-up) #2789/公告:release v0.1.193 修复dsh 切模型导致上下文暴涨问题,修复fork问题,修复claude codex dsh等自动压缩导致会话破坏问题 . 另外修复几十个bug. 我本地场景有限,未必都测全,麻烦大家小场景先测试下是否有问题.任何问题及时反馈. #2821),diff 不触碰 lastInputTokens / baselineFloor / armFailureShrink 的任何生命周期。

即 #2683 的两个 producer 至今都在 master 上活动,以 completed 关闭 = 修复被丢弃。

已做:

八问(要点,完整版见首轮 review):

  1. 是什么问题:会话输入基线活过了测量它的上下文——跨模型账单当幽灵 meter + 失败 arm 采用上界,触发假折叠。
  2. 什么原因:两个 producer——arm 取 char 上界而非计费口径读数;usage-grade 基线只在 origin 变化时降级、model 变化不退役。
  3. 影响范围:命中「无 usage 失败」或「会话中途换模型」的会话;性能下降 + 假折叠,无数据丢失。
  4. 怎么修复:打 producer——billing-caliber arm 读数 + model-stamp 咽喉点 eager retire。
  5. 回归:已审;G3 钉住同模型真超窗仍折叠(floor/fail-fast 语义不变),rebase 中 [Bug] Jimp is active, but image accumulation still triggers Responses WS 1009; duplicate request size not reduced (v0.1. | [Bug] Jimp 已生效,图片累积仍触发 Responses WS 1009;重复请求未缩小(v0.1.200) #3701 分支行为原样保留。
  6. 缓存:不涉及。
  7. 设计方向:不变。
  8. 新配置:无。

行为变更提醒(old→new,详见 PR 正文):arm 值从 char-count 上界 → 计费口径估算;真实模型切换时 trusted 基线立即退役、下一次 usage report 重新锚定。

若你当时关闭的本意是「暂不修」,请再把 #4583 关掉即可——决定权在你。

@ranxianglei ranxianglei changed the title fix: stop stale session input baselines outliving their measurement context (#2683) [PR] fix: stop stale session input baselines outliving their measurement context (#2683) Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant