Repository navigation
fix: multimodal token accounting + store:false in Responses preflight summary (#488) - #489
Conversation
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-489Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr489.tgz
npm install -g packageThis comment is automatically updated on each push. |
[bot] 🏷 Review of PR #488 in progress. Per AGENTS.md triage rules I'm doing: (1) verify both root causes actually exist on master with code evidence, (2) layer analysis (root cause vs symptom), (3) evaluate whether the fix hits the core, including both compression modes (§6). I'll run typecheck/test/build myself and post a full verdict here. |
… Responses preflight summary (#488)
acd9dce to
f7deccc
Compare
|
[bot] 🏷 Review complete — verdict: fix is correct; branch was stale, now rebased + extended. What the review found
Verification (on the rebased branch)
The open question from the description stands: default billing = b64 length/4 (matches the byte-counting relay in the report); pixel-tile upstreams should set |
[bot] 🏷 Independent review of PR #488 (branch Root-cause verification (independent of the PR's own claims)A. Image bytes invisible to size decisions — CONFIRMED, and the layering is right. On master, B. Missing Fix-design audit
Verification run (my sandbox, on the PR head)
Findings from the reviewMinor (report-only, please close before merge):
Major problem found during analysis → filed separately: the two acp-kernel codec gaps the PR works around are silent data-loss bugs in the kernel itself — Open question (default billing model)Recommendation stands: keep chars/4 as the default. The #488 evidence (7 screenshots ≈ 1.4M tokens; pixel-tile billing would cap at ~11K) proves byte-counting relays exist in the wild, and undercounting there reproduces exactly the permanent 400 loop this PR fixes. Overcounting on tile-billing upstreams (official Anthropic/OpenAI) only triggers compression earlier — the safe direction. The Ready to merge once findings 1–2 land (or accepted as an immediate follow-up — your call). Per repo rules I won't merge it myself. |
[bot] 🏷 Re-review of Rebase validated
My checks at
Clamp extension — soundness review
Your out-of-scope flag (dev prompt ~1.9K) — verified accurate
Carried over from round 1 (still holds at
Still open before merge (minor)
One pre-merge recommendation: the PR touches Merge stays human-only per repo rules. If items 1–2 land (or you explicitly defer them), I consider this reviewed and ready. |
|
第三方独立 review(在 f7deccc 上全部本机复跑):
|
[bot] 🏷 收到第三方 review 汇总,逐点确认: 1. 全量 codex e2e 通过 — 补上了我这边缺的最后一道门槛。 这正是我在两轮 verdict 里标记的 pre-merge 建议(我的沙箱只跑通了 2. #496 的定性成立,我修正第一轮的一个措辞。 我当时说"高估只是提前触发压缩,方向安全"——这个论断只在 fit 门表现为"提前压缩"时成立;我低估了极端情形: 3. forward-once-then-learn 方案的两个实现观察(供 #496 实现参考,不阻塞):
4. 合并前剩余项(与你的清单一致,补一条 rebase 注意):
结论不变:代码本身 ready;文档两项 + rebase 验证完成后即可由 owner 合并(按仓库惯例合并人工执行)。 |
& ranxianglei#496 changelog Carrying PR ranxianglei#489's doc gaps noted in ranxianglei#496 (items 1 & 2): add BILI_IMAGE_TOKEN_CAP to the CONFIGURATION env tables (EN+ZH) and add [Unreleased] changelog entries for the ranxianglei#488 image-count fix and the ranxianglei#496 forward-once fix. Drop these hunks if already added to ranxianglei#489.
What
Issue #488: codex session with 7 screenshots → payload ~1.49M tokens vs 1.05M window → every request 502-stuck. Two root causes, both verified in code:
A. Image bytes were invisible to every payload-size decision. The wire codecs move images out of
CoreMessage.textinto sidecars (imageBase64/rawResponsesItem/…), soestimateCoreMessages()(text-only) undercounts image-bearing payloads up to 15× — yet the images ARE forwarded verbatim. Consequences: the preflight fit gate (result.payloadEstimate < limit) declared "fits" for an over-window multimodal payload and forwarded it anyway (upstream 400 → retry → identical → permanent loop); self-heal window learning recorded text-only sizes.B. Preflight summary request omitted
store:falsefor the Responses protocol. Codex relays reject it withHTTP 400 {"detail":"Store must be set to false"}, killing the rescue exactly when it is needed. Only the synthesized summary call was affected: the main forward path passes the client body through (codex sends its ownstore:false) and compress-loop re-requests spread{...requestBody}.Changes
src/image-tokens.ts(new): host-side image share of a RAW request body. Known base64 payloads chargedceil(b64len/4)— the same chars/4 ruledefaultCountTokensuses, matching how byte-counting relays actually bill (多模态会话上下文控制失效 + preflight 救援 502(codex responses store:false) #488 evidence: 7 screenshots ≈ 1.4M tokens rules out pixel-tile billing, which would cap at ~11K). Remote URLs we cannot size get a flatREMOTE_IMAGE_TOKENS = 4096.BILI_IMAGE_TOKEN_CAPenv clamps the per-image cost for pixel-tile upstreams that bill by tiles (official Anthropic/OpenAI), where the default intentionally overestimates to trigger compression earlier rather than undercount.src/preflight.ts:summaryPayloadresponses branch now sendsstore: false; new optionalPreflightDeps.imageFlooradded to every size decision inside the preflight loop (initial estimate, per-round floor-max, per-round result) so folds can never declare success while image bytes alone exceed the window.src/server.ts: preflight trigger + fail-fast detail + self-heal window learning all add the image share of the actual forwarded body; when images alone exceed the window the fail-fast error says so explicitly ("Images alone account for ~N tokens … compression cannot remove them").tests/fix-488-multimodal.test.ts: unit tests for the estimator (all three protocols, data URLs, remote URLs, multi-image sums, raw-body parse gate, cap clamp) + two e2e regressions through the real proxy: (B) preflight summary requests carrystore:falseand the rescued session forwards 200; (A) 7 × 15K-token images vs a 10K window → withheld 502preflight_compress_failedwith the actionable image note, nothing forwarded upstream.Scope notes
store:falsefix applies to the proxy-synthesized summary call in either mode; image accounting is mode-independent (it measures the forwarded body).allImagePartsonly captures data: URLs).Pre-flight (on rebased branch f7deccc)
npm run typecheck✅npm test984/986 — the 2 fails (plugin-agent.test.ts) fail identically on pristine master in this sandbox; environment-dependent, unrelated to this changenpm run build✅Open question for @Rika-xie before merge: the default billing model (b64 length / 4) matches the byte-counting relay in the report; if your upstream bills by pixel tiles, set
BILI_IMAGE_TOKEN_CAPaccordingly.