Skip to content

feat: dual-channel image/text accounting — pixel prior by default, learned per-route cost (#1843) - #1857

Merged
ranxianglei merged 5 commits into
masterfrom
2026-10-01_dual-channel-image-billing
Oct 2, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-10-01_dual-channel-image-billing

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #1843

The billing half of #1800: every window gate consumed ONE number mixing a
text estimate (±~20%) with an image estimate whose error bar was ±1500%
on non-OpenAI upstreams (ceil(base64/4)). #1800's incident: 6 screenshots
estimated at 278,161 tokens vs 18,870 by pixels (15×); real usage 169,805
was under the 257,952 window all along.

What changed (two commits, per the issue rollout)

L2+L3 — de-poisoning, no new state (cf4f222)

  • auto billing now resolves to the pixel prior for every host;
    bytes becomes an explicit opt-in only (the 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 poison class). The
    first-party host classification table is gone — it only ever decided
    between two estimates where one was now the universal default.
  • New config field imageTokenCap (global + per-route): per-image cap,
    positive integer; precedence BILI_IMAGE_TOKEN_CAP env > route > global.
    The env var keeps working exactly as before.
  • Window gates split into two channels: every decision point produces
    {textTokens, imageReserve} instead of one blended total. Text-side
    decisions (preflight trigger, fold target, stop checks, fit, relax,
    exhaustion) compare the text channel against limit − imageReserve;
    kernel-facing quantities keep the total view, so kernel behavior is
    byte-identical. The fold loop never sees image mass — an image-number
    error can no longer hijack text-side behavior. Algebraic equivalence of
    the trigger rewrite is proven in the preflight.ts comment block.
  • Image-side arbitration (fix(#1800): fold compressible text when images alone exceed the window estimate #1801 path) and fail-fast run on the same split:
    arbitration fires when text fits but text + reserve exceeds the window;
    fail-fast keeps its no-evidence hatch for over-prior payloads.

L1 — learning layer (8e80b21)

  • Per-route learned image cost from upstream usage reports at the single
    settle funnel (settleUsageReport, all three response shapes):
    capture {nImages, textSide, host, fp} at every send chokepoint
    (direct forward — streaming AND non-streaming — plus the compress-loop
    fetch), settle pairs them with the billed total:
    perImage = (billed − textSide) / nImages, EMA α=0.5, stored per host
    in session.metadata.learnedImageCosts (persisted like Codex(ChatGPT 登录)preflight 摘要被 400「Stream must be set to true」拒绝,长会话返回 502 #626 capability
    memory), invalidated by 24h TTL or fp change (billing mode/cap
    reconfigured for that route). Out-of-band samples (negative observed,

    1M/image) are quarantined, never learned.

  • Every window gate's image reserve prefers fresh learned evidence
    (learnedPerImage × nImages(current payload)), falling back to the
    prior when no matching entry exists. Converges within a few requests
    regardless of the upstream's vision encoder (llama.cpp ~4096/img, GLM-V,
    …). Audit stamps and the shrink-decision pass keep the pure prior.

Config surface (per policy)

New field, additive, nothing renamed or re-semantics'd:

Field Scope Type Default Notes
imageTokenCap global + named-provider routes positive integer unset (= no cap) parsed leniently like imageBilling: non-positive / non-integer values ignored with warning, never throw. Precedence: BILI_IMAGE_TOKEN_CAP env > route > global.

Why the existing surface couldn't express it: BILI_IMAGE_TOKEN_CAP
is env-only today — process-global, so it cannot vary per route, which is
exactly where billing modes already live (#767). Considered alternatives:
keeping it env-only (rejected: route-level caps are the natural unit —
different relays bill differently on one machine); folding the cap into
the compress hierarchy (rejected: it bounds image BILLING, not
compression behavior). Migration: none needed — field is optional, old
configs unchanged. Documented in CONFIGURATION.md en + zh-CN.

Behavior changes (old → new)

  1. Default image billing for hosts without an explicit mode: bytes →
    pixels.
    Old: unknown upstreams charged ceil(base64/4) (±1500%;
    the 疑似#767后续:图片计价高估会使preflight自动压缩永久失效,同时请求仍被转发 #1800 15× incident). New: pixel-tile estimate (~3,145/screenshot
    typical, bounded [765, 2805] for parseable images, 16,384 flat fallback
    for unparsable containers) — the right order of magnitude for every
    vision encoder, overestimating OpenAI/Anthropic first-party by design
    (safe direction). Explicit bytes still available per-route/global for
    byte-exact OpenAI-family billing. Tests relying on the old default were
    updated to pin bytes explicitly where the scenario needs the
    overestimate class (issue857, issue1800) or switched to real PNGs
    (fix-488 A2 clamp scenario).
  2. Arbitration band tightened at the initial check. Old initial check
    used overhead-blind textEstimate < limit; new uses
    textChannel < limit (includes wire overhead) — matching the shape of
    the EXISTING post-fold check. In the narrow band [limit−oh, limit), a
    payload that previously arbitrated-forward now fail-fasts: it IS a true
    text overflow (overhead included), so refusing is the correct call.
    Outside that band the firing set is algebraically identical.
  3. Incidental same-family fix: the Google clampOutgoingOutput site
    passed NO billing argument (implicit bytes default even when the
    route said otherwise) → now resolved billing + cap like every other
    protocol site.
  4. New session state (L1): session.metadata.learnedImageCosts —
    persisted JSON, keyed by hostname, {cost, seen, ts, fp}; entries die
    after 24h without refresh or on billing/cap change for that route. No
    persistence-format version bump (metadata is user-editable free-form,
    same channel as Codex(ChatGPT 登录)preflight 摘要被 400「Stream must be set to true」拒绝,长会话返回 502 #626).
  5. Usage-grade baseline projection: the preflight text channel projects
    usage-grade baselines by subtracting the image reserve before comparing
    (usage totals include billed images; the text channel must not double-
    count them). The total-view floor used by kernel-facing quantities is
    unchanged.

Verification

  • typecheck + build clean.
  • Targeted sweep 319/319 green: image-billing suites (issue767, fix-488,
    issue857, issue1800), preflight*/compress-loop*/image-compress* globs,
    usage/cache-ledger consumers (cache-ledger, issue1547, host-usage-postfold,
    usage-observability/totals, issue1595, issue793, issue589), neighbors
    (session-gc, named-provider-bind, responses-tool-images, fix-781,
    fix-1205-file-parts, issue1493-arm-outbound-payload, strip-images×2).
  • New suite tests/issue1843-learned-image-cost.test.ts: unit coverage of
    the count walkers (all four wire shapes), raw probe gate, host keying,
    EMA/TTL/fp/cap mechanics, rejection bands; e2e rescue — a bytes-prior
    session is arbitrated forward on first sight of screenshots, learns
    ~1.9k/image from its own usage report, then survives a stamped stale-high
    baseline (usage-grounded, at the window) that the 300k prior would
    fail-fast.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-01_dual-channel-image-billing (73c4958)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1917-pr.1857.0301619

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1857.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到。这是上一个 daemon 会话为 #1843 落地的 PR(两 commit:L2+L3 cf4f222d + L1 8e80b21d)。按规范我不重做,改为独立复核:拉分支核对代码与声明是否一致、跑定向验证、确认 CI 热文件触发(ci-e2e),最后给结构化七问结论。初步已核对:分支 head 8e80b21d、base master 3b2fa2cc、15 files +795/−145、state=open。开始逐条验证。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Review 结论:可合 —— 但审中发现并直接修掉了一个真实的回归(CI 全矩阵红的原因)。本 PR 是前一会话(#1843 线程)的工作,我按「识别自己的 PR」流程做了重新核验 + 修 CI,没有重做实现。逐项如下:

1. 是什么问题:旧图片计费默认对未知 host 用 bytes(base64 长度 ÷ 4),高估可达 ~15×(#1800:6 张截图估算 278,161 vs 实际计费 18,870),把真实用量在窗内的图片重会话永久挡死在 preflight。本 PR:默认改像素先验(pixels for every host,bytes 显式 opt-in)+ 窗口门拆 text/image 双通道 + 新增 imageTokenCap + L1 从 usage report 学习 per-route 实际单图成本(EMA α=0.5、24h TTL、billing fingerprint 失效)。

2. 什么原因导致的:根因在计费先验而非窗口逻辑 —— bytes 估算只对字节计费 relay 正确,却成了通用默认;且图片 token 质量全部计入与文本同一个数,文本明明装得下也触发压缩/fail-fast。方案方向(换先验 + 拆通道 + 实测校准)打到了根因。

3. 怎么修复的:L2+L3(cf4f222)默认像素先验 + 双通道门;L1(8e80b21)发送 chokepoint 捕获 {nImages, textSide, host, fp},settleUsageReport 单漏斗结算。我对 PR body 每条声明做了 diff 级核验:count/cost walker 逐协议对齐;loop/core 捕获用的是活体 coreMessages(原地 mutate,非陈旧快照);direct-forward 路径的配对风险只会保守退化(stale facts → observed 低估 → 样本丢弃,不会高估);生产调用方 sideRequestGuard 已同步更新签名。

4. 是否有回归问题:

  • ✅ 可合并性:干净 worktree git merge origin/master 零冲突;真实 diff(相对 merge-base 4724903)恰为声明的 15 个文件,无夹带。
  • ✅ 代数等价:无图时(R=0)decisionTokens/textTarget 逐字退化为旧表达式;kernel-facing 量保持 total view → kernel 行为 byte-identical(两种 baseline 制度 + relax 路径都验证过)。
  • ⚠️ 两处有意收紧(PR body 已披露,复核成立):①仲裁带 [limit−oh, limit) 由 arbitrate-forward 变 fail-fast(真·文本溢出,方向正确);②google clampOutgoingOutput 原先未传 billing(隐式 bytes),现统一 resolve billing+cap(修不一致,非回归)。
  • ❌→✅ 审中发现一个真回归(即 CI 红):cf4f222d 给 sideRequestGuard 位置参数插了 configuredCap(第 5 位),生产调用点更新了,tests/side-request-isolation.test.ts 的 11 个调用点漏了 —— armed-window case("armed smaller → blocks")丢失 armedLimit 在 4 个 CI 矩阵全挂;"armed larger is ignored" 静默变 no-op(为错而绿);image boundary case 的图片成本被 cap 到 1 token(被前面的 abort 掩盖)。属「首遍不完整」类:作者的定向 sweep 没覆盖这个文件。已直接修(owner policy:不管谁写的,发现就修):commit 8ecb24d7 在全部 11 处显式插入 undefined,调用行与我本地跑绿的版本逐字节一致。
  • ✅ 本地证据(merged tree + 修复):全量 3544 tests / 3525 pass / 0 fail / 19 skip;分支 tip 上目标文件 18/18 绿。

5. 是否破坏缓存:prefix cache/重试/增量游标不受影响。新状态 learnedImageCosts 存 session.metadata(随 #626 持久化),只读参与门控,不改 wire 字节;fp(billing mode:cap)+ 24h TTL 防止跨制度使用陈旧估计。一个设计注记:fp 不含 env BILI_IMAGE_TOKEN_CAP —— 无害(学习值基于计费事实,effective cap 在读侧 min() 应用)。

6. 是否改了设计方向:无架构转向。计费先验选择从 host 白名单改为「默认 pixels + 显式 opt-in」,窗口判断拆双通道,均在既有 preflight/preflight-compress 子系统边界内;L1 复用现有 usage-report 漏斗与会话持久化,无新数据流。config surface 合规:imageTokenCap(global + per-route + env,优先级 env > route > global,宽松解析)en+zh CONFIGURATION.md 齐全,alias routing-fields 清单同步更新。

7. 是否引入了新的配置:有 —— imageTokenCap(正整数,unset = 无上限):global + providers.<url>.imageTokenCap + BILI_IMAGE_TOKEN_CAP(live-read)。迁移:现有用户零迁移(unset = 旧 nocap 语义);依赖旧 auto→bytes 行为的需显式设 "bytes"(文档已说明原因)。顺手小修一处文档失准(f8fe760):原文写非法值「ignored with a warning」,代码实为静默丢弃(与 imageBilling 宽松解析先例一致)——已把措辞对齐;若希望改为响亮告警可另立 follow-up(0 是合法的「无上限」表达,加告警需区分对待)。

收工判定:分支已追加两个 commit(8ecb24d7 test 修复 + f8fe7607 docs 对齐,均零行为变更),新 head f8fe7607 CI 已触发(e2e/version-guard 已过,4 个 test 矩阵运行中)。本地证据齐备(merge 干净 + 全量绿 + 代数等价核验),待新 head CI 转绿即可合并。合并按钮留给你:#1857

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

CI 更新:新 head f8fe7607 全部 13 个 check-run 绿(含之前全矩阵红的 4 个 test job:ubuntu 22/24 + windows 22/24,以及 e2e / hermetic registry / lane 系列)。「待 CI 转绿即可合并」的条件已满足 —— 可以合并了:https://github.com/ranxianglei/billion-context/pull/1857(合并按钮留给你)。

ework-agent added 4 commits October 2, 2026 20:27
…s into text/image channels (#1843 L2+L3)

L2 (de-poisoning): resolveImageBilling auto now resolves to pixels for EVERY
host. The implicit base64/4 default on unknown upstreams carried a ±15x error
bar into every window decision (#1800 incident: 6 screenshots estimated at
278,161 vs 18,870 billed). The pixel prior (~1k-3k/screenshot) is the right
order of magnitude for all known vision encoders; bytes billing becomes an
explicit opt-in for byte-counting relays.

L3 (config cap): new imageTokenCap field (global + per-provider route, added
to the named-provider routing fields), parsed leniently like imageBilling.
Precedence: BILI_IMAGE_TOKEN_CAP env (live) > route > global. Threaded through
every image-cost call site (preflight breakdown, clamp ctxs, side-request
guard, audit stamp, image-compression pass stats).

Dual-channel gates: preflight trigger and fold-loop decisions now compare the
TEXT channel against target - imageReserve instead of adding the reserve to
every total. Algebraically identical firing set (max(B,T+R)>=C <=>
max(max(0,B-R),T)>=max(0,C-R)); an image-estimate error can no longer arm or
stop folding. Kernel-facing quantities keep the total view (conservative,
byte-identical kernel behavior). Image arbitration is re-shaped to the same
channel form as the existing post-fold check: textChannel < limit <
textChannel + reserve — this deliberately tightens the initial check's
old overhead-blind textEstimate<limit test so a true text overflow fails fast
instead of forwarding.

Incidental fix (same mechanism family): the google clampOutgoingOutput call
site passed no billing mode (implicit bytes default); it now resolves billing
+ cap like every other protocol path.

Tests: issue767 unit assertions updated to the new auto resolution; two e2e
controls that relied on the old default pin explicit bytes routes; new e2e
pins the de-poisoning (default route forwards an image-heavy payload the
bytes estimate would brick); fix-488 A2 rewritten with real PNG fixtures
(the flat 16k fallback would otherwise disable the clamp scenario); issue857
and issue1800 pin explicit bytes (their over-window premise is the opt-in
class). New units: parseImageTokenCap leniency + config-cap precedence.
The pixel prior is the right order of magnitude but still a prior: a
llama.cpp vision encoder bills ~4096/image, GLM-V something else again,
and the window gates keep running on the prior until evidence says
otherwise. L1 closes that gap at the single usage funnel.

Mechanism (all three response shapes settle through settleUsageReport):

- capture at every SEND chokepoint — the direct-forward send site
  (streaming and non-streaming alike; streamed turns settle in plugin.ts
  and never revisit the request body) and the compress-loop fetch —
  recording {nImages, textSide, host, fp} per session via a WeakMap with
  the same last-write-wins pairing guarantee as noteForwardedBody;
- settleUsageReport folds the captured facts into the billed total:
  per-image cost = (billed - textSide) / nImages, EMA alpha 0.5, stored
  per host under session.metadata.learnedImageCosts (persisted like #626
  capability memory), invalidated by TTL (24h) or fp change
  (billing mode/cap reconfigured for that route); out-of-band samples
  (negative observed, >1M/image) are quarantined, never learned;
- every window gate's image reserve now prefers fresh learned evidence:
  reserve = learnedPerImage x nImages(current payload), falling back to
  the prior estimate when no matching entry exists. The trigger floor
  keeps the total view (usage-grade baselines include billed images);
  the text channel compares against limit - reserve as before.

textSide mirrors outboundPayloadBreakdown (estimateCoreMessages +
wire overhead) so observed image mass = billed total - textSide holds
by construction. Audit stamps and the shrink-decision pass keep the
pure prior (diagnostics / physical-size decisions, not window gates).

Tests: unit coverage for the count walkers (all four wire shapes), the
raw probe gate, host keying, EMA/TTL/fp/cap mechanics and rejection
bands; e2e rescue scenario where a bytes-prior session is arbitrated
forward on first sight of screenshots, learns ~1.9k/image from its own
usage report, and then survives a stamped stale-high baseline that the
300k prior would fail-fast.
sideRequestGuard gained configuredCap at position 5 in cf4f222, shifting
headroomCap/armedLimit for every positional caller. The production call
site (src/server.ts) was updated but the 11 test call sites in
tests/side-request-isolation.test.ts were not: the armed-window case
('armed smaller → blocks') lost its armedLimit (landed in headroomCap)
and failed on all CI matrices, 'armed larger is ignored' silently became
a no-op, and the image-boundary case now caps image cost at 1 token
(masked by the earlier abort). Insert explicit undefined at position 5
to restore intended semantics at every site.
parseImageTokenCap drops non-positive/non-numeric values silently (same
lenient pattern as parseImageBilling); the new docs claimed 'ignored
with a warning'. Correct the wording in en+zh.
 follow-up)

The two surface tests (acp_status section + /acp panel line) exercise the
real running version — package.json read at runtime — but their fixtures
hardcoded 0.1.179/0.1.180/0.1.181. v0.1.180 shipped between the green PR
run and the #1870 merge, so spanNotes (> running) dropped the 0.1.180
line and master's CI went red on f7cc681 (and every PR rebased onto it,
e.g. #1857).

Fix: derive SURFACE_ENTRIES from VERSION via semver.inc (NEXT/NEXT2),
keeping the explicit-version unit tests hardcoded (they pin spanNotes
logic, not this repo's version). Verified rot-proof by bumping
package.json to a future version locally — suite stays green across the
simulated release.
@ranxianglei
ranxianglei merged commit 8ae9225 into master Oct 2, 2026
14 checks passed
ranxianglei added a commit that referenced this pull request Oct 2, 2026
ci: hermetic real-image billing e2e lane (#1843 follow-up, pins the #1857 verification)
antclt pushed a commit to antclt/billion-context that referenced this pull request Oct 2, 2026
…version (ranxianglei#1870 follow-up)

The two surface tests (acp_status section + /acp panel line) exercise the
real running version — package.json read at runtime — but their fixtures
hardcoded 0.1.179/0.1.180/0.1.181. v0.1.180 shipped between the green PR
run and the ranxianglei#1870 merge, so spanNotes (> running) dropped the 0.1.180
line and master's CI went red on f7cc681 (and every PR rebased onto it,
e.g. ranxianglei#1857).

Fix: derive SURFACE_ENTRIES from VERSION via semver.inc (NEXT/NEXT2),
keeping the explicit-version unit tests hardcoded (they pin spanNotes
logic, not this repo's version). Verified rot-proof by bumping
package.json to a future version locally — suite stays green across the
simulated release.
antclt pushed a commit to antclt/billion-context that referenced this pull request Oct 2, 2026
… pins the ranxianglei#1857 verification)

tests/e2e/e2e-image-billing.test.ts synthesizes a corpus of structurally
real images in pure Node (valid PNG/JPEG/WebP/GIF containers, real dims,
4.7MB 4032x3024 photo, EXIF orientation, truncated + garbage blobs) and
drives it through the real billing pipeline against a mock upstream billing
the Qwen2-VL formula. Pins the three ranxianglei#1857 failure classes: parser truth,
the >=700x bytes-prior poison ratio, and the learned-cost rescue (12.6MB
arbitrated forward -> usage teaches per-image cost -> stale-high usage
baseline no longer bricks the session -> baseline regresses to real
billing) plus the default auto->pixels route riding a malformed blob.

Runs ungated (~1s, zero network, zero tokens): npm run test:e2e:image.
CI: .github/workflows/ci-image.yml auto-runs on PRs touching image/window
-gate surfaces (src/image-*.ts, preflight, server, cache-ledger, kernel pin).
ranxianglei pushed a commit that referenced this pull request Oct 3, 2026
… lost in rebase

The rebase onto dfca5a9 resolved package.json from the PR side for three
lines that must come from master: version 0.1.180 (content commits never
touch version; merging 0.1.179 would downgrade master), acp-kernel 0.0.100
(exact pin follows master), and the test:e2e:image script (#1843/#1857).
Lockfile re-synced from master plus the ws dev-deps.
@github-actions github-actions Bot mentioned this pull request Oct 3, 2026
ranxianglei pushed a commit that referenced this pull request Oct 3, 2026
… lost in rebase

The rebase onto dfca5a9 resolved package.json from the PR side for three
lines that must come from master: version 0.1.180 (content commits never
touch version; merging 0.1.179 would downgrade master), acp-kernel 0.0.100
(exact pin follows master), and the test:e2e:image script (#1843/#1857).
Lockfile re-synced from master plus the ws dev-deps.
ranxianglei pushed a commit that referenced this pull request Oct 3, 2026
… lost in rebase

The rebase onto dfca5a9 resolved package.json from the PR side for three
lines that must come from master: version 0.1.180 (content commits never
touch version; merging 0.1.179 would downgrade master), acp-kernel 0.0.100
(exact pin follows master), and the test:e2e:image script (#1843/#1857).
Lockfile re-synced from master plus the ws dev-deps.
@ranxianglei
ranxianglei deleted the 2026-10-01_dual-channel-image-billing branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bill images and text separately — the image estimate's ±15× error bar poisons every window decision (dual-channel accounting)

1 participant