Skip to content

fix: count projected thinking mass in the host estimator so k̂ calibration stays caliber-honest (#2407) - #2418

Merged
ranxianglei merged 2 commits into
masterfrom
2026-10-08_calib-thinking-denominator
Oct 8, 2026
Merged

ranxianglei merged 2 commits into
masterfrom
2026-10-08_calib-thinking-denominator

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

Follow-up to #2369, fixes #2407.

estimateCoreMessages (src/preflight.ts) counted only m.text. Signature-only thinking blocks (#1320) bill upstream but project as empty text; the residual is attributed to CoreMessage.thinkingTokens — so the k̂ calibration denominator (the prepare sites) divided a bill that includes restored thinking tokens by an estimate that excludes them. Thinking-heavy routes learned an inflated factor (composition-stable, passes all evidence gates, bounded only by the 4× clamp) and over-triggered compression.

Change

  1. src/preflight.ts — estimateCoreMessages now goes through the kernel's countMessageTokens(m, defaultCountTokens) (text + thinkingTokens), the same caliber the kernel's own meters (recommend/receipts/status) already use. Every consumer — k̂ denominator at the prepare sites, preflight fit/round budgets (preflight.ts:1088/:1195), fit gates (server.ts:4697/:4735/:4753), output clamp (budget.ts:48), image-cost textSide (loop/core.ts:399, server.ts:5104) — now carries billed-caliber mass. This also closes a latent inconsistency: the kernel counted thinking mass in its recommendations while the host fit gate ignored it.
  2. src/acp-status.ts — the est-view total line follows, so BILLED INPUT · est-view · ratio stays honest on thinking routes (same caliber as the kernel-rendered per-message breakdown).
  3. src/session.ts — doc comment on lastLocalTextEstimate updated.

Why this is safe

Tests

New tests/issue2407-thinking-caliber.test.ts (4 pins):

  • estimateCoreMessages counts thinkingTokens (exactly the countMessageTokens caliber);
  • post-projection invariant: estimateCoreMessages(msgs) + overhead == measured (the residual is absorbed exactly);
  • k̂ stays caliber-honest on a thinking route: est 20,000 (text 10K + thinking 8K + overhead 2K) vs billed 21,000 → k̂ ≈ 1.05, not the pre-fix 1.75;
  • heavy-thinking route: pre-fix would learn 3.5; post-fix learns ≈0.95 (the text caliber was accurate all along).

Local: typecheck clean, all estimation-relevant suites green. Full-suite local run's only failures are the environmental attach-mode/installer class that clean master also fails locally (CI passes them) — CI is the gate.

Not config / not kernel

No config surface change, no kernel/ change — pure host-side estimator caliber.

…ation stays caliber-honest (#2407)

estimateCoreMessages counted only m.text, so the k̂ calibration denominator
(prepare sites) divided a bill that includes restored thinking tokens by an
estimate that excludes them — thinking-heavy routes learned an inflated
factor (up to the 4× clamp) and over-triggered compression. Route the
estimator through the kernel's countMessageTokens (text + thinkingTokens),
the same caliber the kernel's own meters use; acp_status's est-view total
follows so the BILLED/est ratio stays honest on thinking routes.

Fixes #2407
@github-actions

github-actions Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-08_calib-thinking-denominator (426a4fd)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2616-pr.2418.7befb92

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2418.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到。这是已完成工作的 PR(#2418,branch 2026-10-08_calib-thinking-denominator,base = #2369 合入后的 master)。我不重做实现,进入审查流程:拉分支核对 diff、逐点验证「为什么安全」的三条主张(非 thinking lane 字节级不变 / 无双重计数 / 既有 pin 不动)、本地定向测试 + 等 CI。结论按结构化八问回复。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 审查完成。结论:可合(CI 27/27 全绿、mergeStateStatus clean;本地定向验证全绿;回归 pin 已验证在 unpatched master 上确实失败)。合并留给人工——此 PR 跨 3 个模块触碰热文件(preflight/server/budget 消费面),按 §7.4 不属于 auto-merge 档。

审查证据

结构化结论(八问)

1. 是什么问题:k̂ 校准分子是上游实际计费输入(含 restored thinking),分母 lastLocalTextEstimate 及其同源估计器只数可见文本——signature-only thinking block(#1320)被计费但本地投影为空文本,其质量落在 thinkingTokens 字段上而估计器从不读它,thinking-heavy 路由学到膨胀的 k̂ 并过度触发压缩。

2. 什么原因导致的:分子/分母口径不对称(根因层),不是采样逻辑错。膨胀是 composition-stable 的(thinking share 稳定时比值恒定),因此穿过全部证据门(band [0.2,5]、x2 一致性、≥2 样本、route+model keying)被发布——这正是 #2407 的核心判断,本 PR 的 pin 3/4 数值(1.05 vs 1.75、≈0.95 vs 3.5)与此吻合。

3. 影响范围与严重程度:仅 extended-thinking(Claude signature block)且 thinking share 稳定的路由;受 4× clamp 与 usage baseline floor 双限界。Medium——是 #1933 损伤类(过度触发/fold churn)在 thinking 路由上的可达通道;非 thinking 会话字节级不变。

4. 怎么修复的:在源头统一口径,而不是逐调用点打补丁——host 估计器改走 kernel 自己的 countMessageTokens,与 kernel 所有计量点(recommend/receipts/status/breakdown)共用同一函数。所有消费方自动继承计费口径,顺带闭合了一个潜在不一致:kernel 的建议计量数了 thinking、host 的 fit gate 却没数。比 #2407 早期尝试(helper 并行版,未合入)更优:消灭了第二套估计器,不留口径漂移空间。

5. 是否有回归问题:逐点审过既有行为,结论如下——①非 thinking 会话:thinkingTokenValue 守卫使缺字段/坏值恒为 +0,13 处消费点全部字节级不变(134/134 定向套件背书);②thinking 路由的 fit gate/preflight trigger(:4697/:4735/:1088):估计变大→判定更保守,这是有意变更且方向正确(修前可能把真实超窗载荷判成 fits 盲发,现在本地就能看见;#1839 forward-once 仲裁路径保留兜底);③output clamp(budget.ts:48):稳态由 max(usage baseline, est) 主导、baseline 本就含 thinking,实践值基本不动;baseline lag 时 est 侧更诚实,只会更保守不会更松;④image-cost 学习(textSide):修前 thinking mass 会被误计入图像成本样本,污染 #1843 L1 的 learnedImageReserve——本 PR 顺带修掉了这个 #1320×#1843 交叉污染;⑤acp-status est-view total:纯展示,与 kernel 渲染的 per-message breakdown(本来就走 countMessageTokens)恢复一致;⑥fail-fast/吞错语义、阈值、时序均无漂移。残留观察(不拦):(a) 高增长轮次(新增可见内容 > 上一轮 thinking mass)时 prev-bill 锚定的投影 gap≤0、该轮不投影,样本仍有残余偏差——但此时分母的大文本项本身就在,比值接近 1,且属 prepare-time 估计的固有约束(当轮账单要等响应才知道),非本 PR 缺陷;(b) estimateCoreMessagesUpper 保持 text-only 是合理的:投影依赖 usage report,而 upper 变体生效的场景(localInputEstimate fallback)恰在无 usage report 时,+0 无碍。

6. 是否破坏缓存:不涉及。prefix cache/重试/去重键均未触碰;校准 ring 是会话内内存态,键(origin+model)不变,仅 thinking 路由的值语义变化,window=3 在升级后 ≤3 个 turn 内自愈,无需迁移。

7. 是否改了设计方向:没有转向,是回正:k̂ 回到 #1933 F1 原始意图(纯 tokenizer 偏差因子),host 侧估计器与 kernel 计量收敛到单一口径源,消除双轨。

8. 是否引入了新的配置:无。无新 env/config/默认值,无 kernel 改动(countMessageTokens 是既有导出 API)。

⚠️ 行为变更披露(必须报告项)

estimateCoreMessages 及全部下游消费点的计量口径:text(+sys/tools+images)→ 同左 + host-projected thinking mass。效果:extended-thinking 路由发布的 k̂ 从膨胀值(双向 clamp 下最高 4×)回落至口径一致时的 ≈1;fit/clamp/image-cost 各决策点在 thinking 路由上按真实计费质量判定(更保守、更诚实);非 thinking 会话完全不变。原因:分母若不含已被计费的质量,比值就把「载荷组成」混进了「tokenizer 偏差」,把 #2366 的双向 clamp 变成 thinking 路由专属的提前触发通道。

测试有效性

4 个 pin 中 pin 1/2 是真回归 pin(已在 unpatched master 实测失败、修复后通过);pin 3/4 是后果 characterization(钉 sampler 数学,防校准门将来被改松时静默放行膨胀样本)——组合覆盖成立,断言有牙,无恒真项。

判定:可合。 唯一待办是人工 merge(https://github.com/ranxianglei/billion-context/pull/2418);合入后 #2407 可关。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到系统提醒。审查结论已在本 issue 4 楼完整发布(comment id 25616):PR #2418 可合——CI 27/27 全绿、本地定向验证 134/134、回归 pin 1/2 已实测在 unpatched master 上失败(有牙)、13 处消费点口径逐点核对无回归;含结构化八问与行为变更披露(估计器口径 text-only → text + projected thinking mass)。本条仅为提醒确认,无新增内容。唯一待办:人工 merge(https://github.com/ranxianglei/billion-context/pull/2418),合入后 #2407 可关。

prepare-google.ts was the only one of the four protocol lanes never
writing stats.lastLocalTextEstimate, so Gemini-native routes could never
learn the calibration factor — settleUsageReport found no pending pair
and every downstream gate stayed raw-estimate caliber. Mirror the
anthropic/openai/responses write (same estimateCoreMessages caliber,
title-gen side requests skipped like the #532 rows above).
@ranxianglei

Copy link
Copy Markdown
Owner Author

Addendum (小问题直接修): Google 车道补上 k̂ 分母。 排查 #2407 时发现 prepare-google.ts 是四条协议车道里唯一从未写入 stats.lastLocalTextEstimate 的(settleUsageReport 永远等不到 pending pair → Gemini-native 路由永远学不到校准系数,所有下游门一直跑 raw 估算口径)。已在同分支补齐:镜像 anthropic/openai/responses 三家的 prepare-time 写入,同一 estimateCoreMessages 口径,title-gen 旁路请求同 #532 一样跳过。

新测试 tests/issue2407-google-khat-lane.test.ts:真实 google wire 过代理打假 Gemini 上游(usageMetadata 报 3× chars/4),三轮后断言 calibratedEstimate 发布且 ≈3。A/B 验证:还原 prepare-google.ts 后测试精确失败于 calibratedEstimate must publish on the google lane (got undefined)。邻接 41/41 绿(typecheck 干净)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到 addendum。这是对 PR #2418 分支的增量提交(Google 车道补写 lastLocalTextEstimate,同机制族:k̂ 校准管道)。进入增量审查:拉新 commit、核对「唯一从未写入」的主张与三家镜像写法是否逐字同口径(title-gen 旁路跳过门)、本地跑新测试 + 邻接套件 + A/B 有牙验证、等 CI。结论按八问回复。

@ranxianglei
ranxianglei merged commit c296fb4 into master Oct 8, 2026
28 checks passed
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.143-master.c296fb4 (rolling master tag).

npm install -g billion-context@0.0.143-master.c296fb4  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[follow-up #2369] calibration k̂ denominator omits projected thinking mass — extended-thinking routes over-scaled under the two-way clamp

1 participant