Repository navigation
feat: dual-channel image/text accounting — pixel prior by default, learned per-route cost (#1843) - #1857
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.1917-pr.1857.0301619This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr1857.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Review 结论:可合 —— 但审中发现并直接修掉了一个真实的回归(CI 全矩阵红的原因)。本 PR 是前一会话(#1843 线程)的工作,我按「识别自己的 PR」流程做了重新核验 + 修 CI,没有重做实现。逐项如下: 1. 是什么问题:旧图片计费默认对未知 host 用 bytes(base64 长度 ÷ 4),高估可达 ~15×(#1800:6 张截图估算 278,161 vs 实际计费 18,870),把真实用量在窗内的图片重会话永久挡死在 preflight。本 PR:默认改像素先验(pixels for every host,bytes 显式 opt-in)+ 窗口门拆 text/image 双通道 + 新增 2. 什么原因导致的:根因在计费先验而非窗口逻辑 —— bytes 估算只对字节计费 relay 正确,却成了通用默认;且图片 token 质量全部计入与文本同一个数,文本明明装得下也触发压缩/fail-fast。方案方向(换先验 + 拆通道 + 实测校准)打到了根因。 3. 怎么修复的:L2+L3(cf4f222)默认像素先验 + 双通道门;L1(8e80b21)发送 chokepoint 捕获 {nImages, textSide, host, fp},settleUsageReport 单漏斗结算。我对 PR body 每条声明做了 diff 级核验:count/cost walker 逐协议对齐;loop/core 捕获用的是活体 coreMessages(原地 mutate,非陈旧快照);direct-forward 路径的配对风险只会保守退化(stale facts → observed 低估 → 样本丢弃,不会高估);生产调用方 sideRequestGuard 已同步更新签名。 4. 是否有回归问题:
5. 是否破坏缓存:prefix cache/重试/增量游标不受影响。新状态 learnedImageCosts 存 session.metadata(随 #626 持久化),只读参与门控,不改 wire 字节;fp(billing mode:cap)+ 24h TTL 防止跨制度使用陈旧估计。一个设计注记:fp 不含 env BILI_IMAGE_TOKEN_CAP —— 无害(学习值基于计费事实,effective cap 在读侧 min() 应用)。 6. 是否改了设计方向:无架构转向。计费先验选择从 host 白名单改为「默认 pixels + 显式 opt-in」,窗口判断拆双通道,均在既有 preflight/preflight-compress 子系统边界内;L1 复用现有 usage-report 漏斗与会话持久化,无新数据流。config surface 合规: 7. 是否引入了新的配置:有 —— 收工判定:分支已追加两个 commit( |
[bot] 🏷 CI 更新:新 head |
f8fe760 to
0f0d57f
Compare
…s into text/image channels (#1843 L2+L3) L2 (de-poisoning): resolveImageBilling auto now resolves to pixels for EVERY host. The implicit base64/4 default on unknown upstreams carried a ±15x error bar into every window decision (#1800 incident: 6 screenshots estimated at 278,161 vs 18,870 billed). The pixel prior (~1k-3k/screenshot) is the right order of magnitude for all known vision encoders; bytes billing becomes an explicit opt-in for byte-counting relays. L3 (config cap): new imageTokenCap field (global + per-provider route, added to the named-provider routing fields), parsed leniently like imageBilling. Precedence: BILI_IMAGE_TOKEN_CAP env (live) > route > global. Threaded through every image-cost call site (preflight breakdown, clamp ctxs, side-request guard, audit stamp, image-compression pass stats). Dual-channel gates: preflight trigger and fold-loop decisions now compare the TEXT channel against target - imageReserve instead of adding the reserve to every total. Algebraically identical firing set (max(B,T+R)>=C <=> max(max(0,B-R),T)>=max(0,C-R)); an image-estimate error can no longer arm or stop folding. Kernel-facing quantities keep the total view (conservative, byte-identical kernel behavior). Image arbitration is re-shaped to the same channel form as the existing post-fold check: textChannel < limit < textChannel + reserve — this deliberately tightens the initial check's old overhead-blind textEstimate<limit test so a true text overflow fails fast instead of forwarding. Incidental fix (same mechanism family): the google clampOutgoingOutput call site passed no billing mode (implicit bytes default); it now resolves billing + cap like every other protocol path. Tests: issue767 unit assertions updated to the new auto resolution; two e2e controls that relied on the old default pin explicit bytes routes; new e2e pins the de-poisoning (default route forwards an image-heavy payload the bytes estimate would brick); fix-488 A2 rewritten with real PNG fixtures (the flat 16k fallback would otherwise disable the clamp scenario); issue857 and issue1800 pin explicit bytes (their over-window premise is the opt-in class). New units: parseImageTokenCap leniency + config-cap precedence.
The pixel prior is the right order of magnitude but still a prior: a
llama.cpp vision encoder bills ~4096/image, GLM-V something else again,
and the window gates keep running on the prior until evidence says
otherwise. L1 closes that gap at the single usage funnel.
Mechanism (all three response shapes settle through settleUsageReport):
- capture at every SEND chokepoint — the direct-forward send site
(streaming and non-streaming alike; streamed turns settle in plugin.ts
and never revisit the request body) and the compress-loop fetch —
recording {nImages, textSide, host, fp} per session via a WeakMap with
the same last-write-wins pairing guarantee as noteForwardedBody;
- settleUsageReport folds the captured facts into the billed total:
per-image cost = (billed - textSide) / nImages, EMA alpha 0.5, stored
per host under session.metadata.learnedImageCosts (persisted like #626
capability memory), invalidated by TTL (24h) or fp change
(billing mode/cap reconfigured for that route); out-of-band samples
(negative observed, >1M/image) are quarantined, never learned;
- every window gate's image reserve now prefers fresh learned evidence:
reserve = learnedPerImage x nImages(current payload), falling back to
the prior estimate when no matching entry exists. The trigger floor
keeps the total view (usage-grade baselines include billed images);
the text channel compares against limit - reserve as before.
textSide mirrors outboundPayloadBreakdown (estimateCoreMessages +
wire overhead) so observed image mass = billed total - textSide holds
by construction. Audit stamps and the shrink-decision pass keep the
pure prior (diagnostics / physical-size decisions, not window gates).
Tests: unit coverage for the count walkers (all four wire shapes), the
raw probe gate, host keying, EMA/TTL/fp/cap mechanics and rejection
bands; e2e rescue scenario where a bytes-prior session is arbitrated
forward on first sight of screenshots, learns ~1.9k/image from its own
usage report, and then survives a stamped stale-high baseline that the
300k prior would fail-fast.
sideRequestGuard gained configuredCap at position 5 in cf4f222, shifting headroomCap/armedLimit for every positional caller. The production call site (src/server.ts) was updated but the 11 test call sites in tests/side-request-isolation.test.ts were not: the armed-window case ('armed smaller → blocks') lost its armedLimit (landed in headroomCap) and failed on all CI matrices, 'armed larger is ignored' silently became a no-op, and the image-boundary case now caps image cost at 1 token (masked by the earlier abort). Insert explicit undefined at position 5 to restore intended semantics at every site.
parseImageTokenCap drops non-positive/non-numeric values silently (same lenient pattern as parseImageBilling); the new docs claimed 'ignored with a warning'. Correct the wording in en+zh.
0f0d57f to
73b761b
Compare
follow-up) The two surface tests (acp_status section + /acp panel line) exercise the real running version — package.json read at runtime — but their fixtures hardcoded 0.1.179/0.1.180/0.1.181. v0.1.180 shipped between the green PR run and the #1870 merge, so spanNotes (> running) dropped the 0.1.180 line and master's CI went red on f7cc681 (and every PR rebased onto it, e.g. #1857). Fix: derive SURFACE_ENTRIES from VERSION via semver.inc (NEXT/NEXT2), keeping the explicit-version unit tests hardcoded (they pin spanNotes logic, not this repo's version). Verified rot-proof by bumping package.json to a future version locally — suite stays green across the simulated release.
…version (ranxianglei#1870 follow-up) The two surface tests (acp_status section + /acp panel line) exercise the real running version — package.json read at runtime — but their fixtures hardcoded 0.1.179/0.1.180/0.1.181. v0.1.180 shipped between the green PR run and the ranxianglei#1870 merge, so spanNotes (> running) dropped the 0.1.180 line and master's CI went red on f7cc681 (and every PR rebased onto it, e.g. ranxianglei#1857). Fix: derive SURFACE_ENTRIES from VERSION via semver.inc (NEXT/NEXT2), keeping the explicit-version unit tests hardcoded (they pin spanNotes logic, not this repo's version). Verified rot-proof by bumping package.json to a future version locally — suite stays green across the simulated release.
… pins the ranxianglei#1857 verification) tests/e2e/e2e-image-billing.test.ts synthesizes a corpus of structurally real images in pure Node (valid PNG/JPEG/WebP/GIF containers, real dims, 4.7MB 4032x3024 photo, EXIF orientation, truncated + garbage blobs) and drives it through the real billing pipeline against a mock upstream billing the Qwen2-VL formula. Pins the three ranxianglei#1857 failure classes: parser truth, the >=700x bytes-prior poison ratio, and the learned-cost rescue (12.6MB arbitrated forward -> usage teaches per-image cost -> stale-high usage baseline no longer bricks the session -> baseline regresses to real billing) plus the default auto->pixels route riding a malformed blob. Runs ungated (~1s, zero network, zero tokens): npm run test:e2e:image. CI: .github/workflows/ci-image.yml auto-runs on PRs touching image/window -gate surfaces (src/image-*.ts, preflight, server, cache-ledger, kernel pin).
… lost in rebase The rebase onto dfca5a9 resolved package.json from the PR side for three lines that must come from master: version 0.1.180 (content commits never touch version; merging 0.1.179 would downgrade master), acp-kernel 0.0.100 (exact pin follows master), and the test:e2e:image script (#1843/#1857). Lockfile re-synced from master plus the ws dev-deps.
… lost in rebase The rebase onto dfca5a9 resolved package.json from the PR side for three lines that must come from master: version 0.1.180 (content commits never touch version; merging 0.1.179 would downgrade master), acp-kernel 0.0.100 (exact pin follows master), and the test:e2e:image script (#1843/#1857). Lockfile re-synced from master plus the ws dev-deps.
… lost in rebase The rebase onto dfca5a9 resolved package.json from the PR side for three lines that must come from master: version 0.1.180 (content commits never touch version; merging 0.1.179 would downgrade master), acp-kernel 0.0.100 (exact pin follows master), and the test:e2e:image script (#1843/#1857). Lockfile re-synced from master plus the ws dev-deps.
Closes #1843
The billing half of #1800: every window gate consumed ONE number mixing a
text estimate (±~20%) with an image estimate whose error bar was ±1500%
on non-OpenAI upstreams (
ceil(base64/4)). #1800's incident: 6 screenshotsestimated at 278,161 tokens vs 18,870 by pixels (15×); real usage 169,805
was under the 257,952 window all along.
What changed (two commits, per the issue rollout)
L2+L3 — de-poisoning, no new state (cf4f222)
autobilling now resolves to the pixel prior for every host;bytesbecomes an explicit opt-in only (the 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 poison class). Thefirst-party host classification table is gone — it only ever decided
between two estimates where one was now the universal default.
imageTokenCap(global + per-route): per-image cap,positive integer; precedence
BILI_IMAGE_TOKEN_CAPenv > route > global.The env var keeps working exactly as before.
{textTokens, imageReserve}instead of one blended total. Text-sidedecisions (preflight trigger, fold target, stop checks, fit, relax,
exhaustion) compare the text channel against
limit − imageReserve;kernel-facing quantities keep the total view, so kernel behavior is
byte-identical. The fold loop never sees image mass — an image-number
error can no longer hijack text-side behavior. Algebraic equivalence of
the trigger rewrite is proven in the preflight.ts comment block.
arbitration fires when text fits but
text + reserveexceeds the window;fail-fast keeps its no-evidence hatch for over-prior payloads.
L1 — learning layer (8e80b21)
settle funnel (
settleUsageReport, all three response shapes):capture
{nImages, textSide, host, fp}at every send chokepoint(direct forward — streaming AND non-streaming — plus the compress-loop
fetch), settle pairs them with the billed total:
perImage = (billed − textSide) / nImages, EMA α=0.5, stored per hostin
session.metadata.learnedImageCosts(persisted like Codex(ChatGPT 登录)preflight 摘要被 400「Stream must be set to true」拒绝,长会话返回 502 #626 capabilitymemory), invalidated by 24h TTL or fp change (billing mode/cap
reconfigured for that route). Out-of-band samples (negative observed,
(
learnedPerImage × nImages(current payload)), falling back to theprior when no matching entry exists. Converges within a few requests
regardless of the upstream's vision encoder (llama.cpp ~4096/img, GLM-V,
…). Audit stamps and the shrink-decision pass keep the pure prior.
Config surface (per policy)
New field, additive, nothing renamed or re-semantics'd:
imageTokenCapimageBilling: non-positive / non-integer values ignored with warning, never throw. Precedence:BILI_IMAGE_TOKEN_CAPenv > route > global.Why the existing surface couldn't express it:
BILI_IMAGE_TOKEN_CAPis env-only today — process-global, so it cannot vary per route, which is
exactly where billing modes already live (#767). Considered alternatives:
keeping it env-only (rejected: route-level caps are the natural unit —
different relays bill differently on one machine); folding the cap into
the
compresshierarchy (rejected: it bounds image BILLING, notcompression behavior). Migration: none needed — field is optional, old
configs unchanged. Documented in CONFIGURATION.md en + zh-CN.
Behavior changes (old → new)
bytes→pixels. Old: unknown upstreams chargedceil(base64/4)(±1500%;the 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 15× incident). New: pixel-tile estimate (~3,145/screenshot
typical, bounded [765, 2805] for parseable images, 16,384 flat fallback
for unparsable containers) — the right order of magnitude for every
vision encoder, overestimating OpenAI/Anthropic first-party by design
(safe direction). Explicit
bytesstill available per-route/global forbyte-exact OpenAI-family billing. Tests relying on the old default were
updated to pin
bytesexplicitly where the scenario needs theoverestimate class (issue857, issue1800) or switched to real PNGs
(fix-488 A2 clamp scenario).
used overhead-blind
textEstimate < limit; new usestextChannel < limit(includes wire overhead) — matching the shape ofthe EXISTING post-fold check. In the narrow band
[limit−oh, limit), apayload that previously arbitrated-forward now fail-fasts: it IS a true
text overflow (overhead included), so refusing is the correct call.
Outside that band the firing set is algebraically identical.
clampOutgoingOutputsitepassed NO billing argument (implicit
bytesdefault even when theroute said otherwise) → now resolved billing + cap like every other
protocol site.
session.metadata.learnedImageCosts—persisted JSON, keyed by hostname,
{cost, seen, ts, fp}; entries dieafter 24h without refresh or on billing/cap change for that route. No
persistence-format version bump (metadata is user-editable free-form,
same channel as Codex(ChatGPT 登录)preflight 摘要被 400「Stream must be set to true」拒绝,长会话返回 502 #626).
usage-grade baselines by subtracting the image reserve before comparing
(usage totals include billed images; the text channel must not double-
count them). The total-view floor used by kernel-facing quantities is
unchanged.
Verification
issue857, issue1800), preflight*/compress-loop*/image-compress* globs,
usage/cache-ledger consumers (cache-ledger, issue1547, host-usage-postfold,
usage-observability/totals, issue1595, issue793, issue589), neighbors
(session-gc, named-provider-bind, responses-tool-images, fix-781,
fix-1205-file-parts, issue1493-arm-outbound-payload, strip-images×2).
the count walkers (all four wire shapes), raw probe gate, host keying,
EMA/TTL/fp/cap mechanics, rejection bands; e2e rescue — a bytes-prior
session is arbitrated forward on first sight of screenshots, learns
~1.9k/image from its own usage report, then survives a stamped stale-high
baseline (usage-grounded, at the window) that the 300k prior would
fail-fast.