Repository navigation
[PR] fix: clamp learned image cost to the payload's own prior (#4153) - #4156
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4952-pr.4156.e149339This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr4156.tgz
npm install -g packageInstall instructions are refreshed on each push. |
settleImageLearning attributes the entire usage residual (billed input minus the text-side estimate) to images, so a route whose bill outruns bili's text estimate by reasoning/system/tool mass or relay accounting converges its learned per-image cost to a nonsense fixed point (448k observed on a 2k-prior 1080p screenshot). With imageTokenCap defaulting to 0 nothing clamped it, so a session only ~69K over the window fail-fasted on a phantom 855K deficit. Three guards all missed it: the backstop sat at 1e6, the cap clamp branch was dead at cap=0, and the TTL refreshed on every settle. - Consume side: the learned reserve for a payload may never exceed LEARNED_VARIANCE_BAND x that payload's own prior (nImages-independent), so an already-poisoned persisted value is neutralized immediately no matter how it got there. This is the load-bearing cure. - Backstop reduced 1e6 -> 100k for the no-prior path only. - Write side resets on fingerprint change so a reconfigured route stops EMA-ing off a poisoned base (previously setting a cap changed only read behavior and never diluted the stored value). Downward corrections (learned < prior, the #1843 rescue for over-counting bytes priors) are untouched: the ceiling bounds only the high side. Fixes #4153
zjbc2000
left a comment
There was a problem hiding this comment.
审过 PR #4156 的实现,逐行确认三点,都不需要返工。
一、承重墙确认
if (typeof prior === "number" && prior > 0) {
per = Math.min(per, (LEARNED_VARIANCE_BAND * prior) / nImages);
}
return per * nImages;per ≤ 15 × prior / nImages ⟹ per × nImages ≤ 15 × prior。total 上界与 nImages 无关这一点实现对了——这正是 consume ceiling 比 settle backstop 强的地方,你把两边的关系处理得比我建议的更完整(我原本只说「backstop 不能当解药」,你的做法是保留它的作用 + 另加真正的解药)。
prior 从 imageReserveFor 单点穿入,else 分支保留 LEARNED_PER_IMAGE_MAX 兜无 prior 路径。调用链没有第二处消费 learnedImageCosts,所以没有绕过点。
二、细节一(fp 重置)—— 是我的替代方案不成立,你选得对
我原话给的是「或 settle 侧把 prior 上界作用到 prev.cost」。审下来这个方案做不到:noteForwardedImageFacts 记录的是 {nImages, textSide, host, fp},settleImageLearning 拿不到 prior,要穿就得改两个发送点的签名。一行重置不仅更小,而且语义更准——它实现的是「重配即视学习库失效」这条显式规则,而不是靠一个数值上界去猜旧值该不该信。 撤回我的替代方案。
三、测试覆盖超出我的要求
nImages-independence 用 n ∈ [1,4,40] 钉住;settle-many 专门验证「40 图 → per 71,500 过 backstop,但消费侧仍钳回」;reset 验证 seen 归一后 EMA 从干净基底继续。这三个正好覆盖了我提的两个细节,没有漏。
rescue 和 band 两条也说明你没有把上界做成单向压制——learned < prior 与带内上修都保留。
一条观测建议(不阻塞合并)
审 estimateWireOverhead 时确认 textSide 不含 base64 字节:
return defaultCountTokens(sysText)
+ defaultCountTokens(JSON.stringify(parsed.tools ?? []))
+ (protocol === "responses" ? countLoadedToolTokens(parsed) : 0);所以 billedTotal − textSide 在构造上就只剩「图片 + 其它非文本质量」,残差归因的思路本身成立,缺的只是上界——你的修法对准了真问题。
但这也意味着 reasoning tokens 与 relay 记账会照样落进图片分母。15× 的带能兜住本 issue 的 211× 偏差,但若某路由的 reasoning 占比本身就超过 15×prior,仍会贴到带边缘。
建议(可选):在 settleImageLearning 里加一条 debug 级日志,把 observed / per / appliedClamp 三者一起打出。理由不是防御性,是可诊断性:这次我能把根因钉死,全靠 learnedImageCosts 恰好持久化在 metadata_json 里,可以直接读出 448,724.59893398033 并与 context − local-est 除尽对账。如果这条学习值只存在内存里,本 issue 根本无从下手——同理,下次若出现新形态的偏差,一行日志能让下一个人少花两小时。
不加也完全可以合。CI 我这侧看到 26 pass / 0 fail / 3 running(ubuntu 24 与 windows 22×2 仍在跑),kernel-guard / config-surface-guard / docs-* / lock-sync 已绿。批准合并。
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 按失败签名检索 open+closed issue/PR( 标签: 判定为 Review 进度: PR 分支落后 master 64 个提交,已本地 rebase 到当前 master(干净、零冲突);rebase 后 head 上定向测试 36/36 绿(#4153 新套件 + #1843 rescue + e2e-image-billing + #2078 outbound-estimate + 相邻 image 估算面)、 |
d5cdbc7 to
64fcd2b
Compare
(ai: vllm-qwen/qwen3.8-27b) Review 完成(独立复审,未复读 issue 侧结论):判定 = 可合入(§7.4 第 8 条涉 fallback/默认值变更 → 人工合并)。分支已 rebase 到当前 master(干净、零冲突)并推送,新 head 结构化八问:
测试有效性(质量关):真测试、断言有牙——事故 fixture 精确钉死(448,724.59893398033 / seen=791 → 钳回 127,500)、nImages 无关性循环(n=1/4/40 总额恒 15×prior_total)、backstop admit/reject 双向、rescue 保留、cap∩band 紧者胜双向、fp reset + EMA 连续性。任一层回归(删钳/抬 backstop/删 reset)对应测试即红,不是恒真断言。 代码质量:命名达意( 验证证据链:rebase 干净 → 本地定向 36/36 + tsc + build → push → 远程 CI 在推送 head 上 30/30 绿(全量裁决以 CI 为准,本地未跑全量)。 结论:可合入(人工合并即可;合并后 #4153 经 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.807-master.bd9f5fe # this exact build
npm install -g billion-context@master # always the newest merged build
|
Fixes #4153
What was wrong
settleImageLearningattributes the entire usage residual (upstream billed input minus bili's text-side estimate of the same forwarded payload) to images. On a route whose bill systematically outruns the text estimate — reasoning/system/tool mass, relay accounting, a larger client preamble — the learned per-image cost converges via EMA to a nonsense fixed point. Observed: 448,724 tok/image on a 1920×1080 frame whose pixels prior is 2,125. WithimageTokenCapdefaulting to0, nothing clamped the value, so a session only ~69K over its window fail-fasted on a phantom 855K deficit.The three existing guards all missed it:
LEARNED_PER_IMAGE_MAXsat at1e6— three orders of magnitude above any plausible single-image bill.cap > 0 ? Math.min(cost, cap)branch was dead at the defaultcap = 0.Fix
LEARNED_VARIANCE_BAND ×that payload's own prior (learnedImageReserveinsrc/cache-ledger.ts). The band (15×) is anchored to the Bill images and text separately — the image estimate's ±15× error bar poisons every window decision (dual-channel accounting) #1843 documented encoder-variance range. It is nImages-independent in total and neutralizes an already-poisoned persisted value immediately — regardless of how it accumulated or whether a cap is configured.LEARNED_PER_IMAGE_MAXreduced1e6 → 100kfor the no-prior path only (still admits every realistic single-image bill, including large byte-billing payloads).settleImageLearningresets instead of EMA-ing off a stale base when the route fingerprint (billing/cap) changed, so setting a cap to mitigate now actually clears the poisoned value rather than only changing read behavior.Downward corrections (
learned < prior— the #1843 rescue for over-counting bytes priors) are untouched: the ceiling bounds only the high side.Behavior change (old → new)
imageTokenCap=0LEARNED_PER_IMAGE_MAX)Incident numbers: pre-fix context 1,852,920 (deficit 855,406 → fail-fast); post-fix the per-image cost clamps to 31,875, the reserve drops to 127,500, and context falls to the usage baseline 1,069,263 (deficit 69,263 → compressible).
Config surface
No change.
imageTokenCapkeeps its name, type, and default (0); its existing per-route semantics are unchanged. The new bound is derived from the payload's own prior, not from a new knob.Tests
tests/issue4153-learned-image-cost-ceiling.test.tspins the incident end-to-end (the exact 448,724.59893398033 / seen=791 fixture), the nImages-independence of the total cap, the reduced backstop, rescue preservation, cap∩band composition, the settle admit-vs-reject semantics, and the write-side fingerprint reset. Targeted suites green locally:#4153+#1843(14 tests),e2e-image-billing+#2078 outbound-estimate(8 tests);tsc --noEmitclean. Full-suite verdict deferred to CI.Single-module scope (
cache-ledger+ its one call site inserver.ts); nokernel/change.