Skip to content

[PR] fix #4488: harvest usage from an OpenAI trailing frame whose choices stay non-empty with an empty delta - #4494

Closed
ranxianglei wants to merge 1 commit into
masterfrom
2026-10-24_4488-openai-usage-trailing-frame
Closed

ranxianglei wants to merge 1 commit into
masterfrom
2026-10-24_4488-openai-usage-trailing-frame

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Root cause

src/loop/adapter-openai.ts parseStream harvests usage from exactly two frame shapes: a frame with no choices at all (choices:[], the #593 fix) and a frame carrying finish_reason. An include_usage relay that delivers usage on a trailing frame whose choices array stays non-empty — one choice with an empty delta and no finish_reason (OpenAI's own include_usage final-chunk shape, or a gateway variant of it) — matches neither, so its usage was silently dropped.

The drop was internal-only: that same frame still fell through to the empty-delta meta forward (adapter-openai.ts L672-673), so the client kept receiving the raw usage frame. What broke was bili's own ledger — no kind:"usage" event was ever emitted, so recordUsage -> settleUsageReport never ran for the real sample and nudge stayed pinned on the last stale anchor (growth=0) until the context grew past the upstream's limit (502). This is why [acp-usage] never appeared across the reporter's 154-round freeze window.

Fix

One guarded branch in parseStream: when a frame carries usage but has no finish_reason and an empty delta object, yield a kind:"usage" event (with openaiCachedTokens normalization). The guards keep every content / finish / tool-call frame out.

Deliberately no continue: the frame falls through to the empty-delta meta forward unchanged, so the client-visible stream stays byte-identical to pre-fix and only the internal ledger gains the sample. (A continue here would sever that forward and change what the client sees — a regression test pins this.)

Because core.ts consumes kind:"usage" last-wins into one value that feeds both settleUsageReport (internal ledger) and emitCompletion (client completion), harvesting on the trailing frame corrects both with no double-count.

Both compression modes (AGENTS.md §6)

  • proxy mode: harvested usage flows parseStream -> core.ts last-wins -> settleUsageReport + emitCompletion. Fixed. No wire-shape change (the forwarded bytes are unchanged).
  • plugin mode: the ACP agent owns compression; bili still runs the same loop/parseStream path for accounting. The new event is internal-only; the client-visible stream is unchanged. No tool/nudge injection affected.

Verification

Behavior change disclosure

None to the client-visible stream. By design (the no-continue fall-through) the forwarded bytes are byte-identical to pre-fix; the only observable change is that the internal ledger now receives the real usage sample it was previously missing — which is the defect being fixed. No threshold / default / timing / output-format drift.

Relation to #4383 / #4413 / #4443

Independent defects, one symptom (the freeze). Those are the read-side fix (a stale small anchor degraded to the local estimate as a safety net); this is the parser-side fix (make the real sample land every round). Complementary — both wanted. Not a duplicate of #593 either, which fixed the different choices:[] shape.

Closes #4488

@ranxianglei ranxianglei changed the title fix #4488: harvest usage from an OpenAI trailing frame whose choices stay non-empty with an empty delta [PR] fix #4488: harvest usage from an OpenAI trailing frame whose choices stay non-empty with an empty delta Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-24_4488-openai-usage-trailing-frame (908e59c)

Latest build — install this exact version

npm install -g billion-context@0.0.5408-pr.4494.4b4ec42

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
908e59c billion-context@0.0.5408-pr.4494.4b4ec42 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4494.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 检索 open+closed PR 无其他 parser 侧 usage-harvest 修复在途;与已合入的 #593/#589(修的是 choices:[] 形状)机制不同;与 #4383/#4413/#4443(读侧 stale-anchor 降级安全网)同症状家族(nudge 冻结)但机制不同——本 PR 让真实样本每轮落地,那三个让陈旧锚点降级,互补不重叠。

标签: bug / p1 / scope/wiring(对齐原 issue #4488;p1 因受影响会话内部记账完全失明直至 502;未打客户端标签——修复是通用 OpenAI relay 解析路径,不绑定特定客户端)。

结构化八问

  1. 是什么问题:部分 OpenAI 兼容 relay 把真实 usage 放在最后一帧,该帧 choices 仍非空(一个 choice、空 delta、无 finish_reason)——OpenAI 自己 include_usage 的收尾形状。bili 只认「无 choices」和「带 finish_reason」两种形状,这帧的 usage 被静默丢弃,台账收不到真实样本,nudge 钉死 growth=0,上下文涨过上游上限即 502(报告者 154 轮冻结窗口的直接原因)。
  2. 什么原因导致的:parser 的 usage 采集覆盖漏了第三种帧形状。该帧本身仍经空-delta meta-forward 转发给了客户端(所以客户端一直有数据),但内部从不发 kind:"usage" 事件 → recordUsage → settleUsageReport 从未执行。根因在 parser 采集面,不在下游消费面。
  3. 影响范围与严重程度:走「usage 只出现在非空 choices 尾帧」这种形状的 OpenAI 兼容 relay 的所有会话。受影响会话内部记账完全失明(无 [acp-usage] 日志、无 settleUsageReport),nudge 冻结至 502 = 特定条件下不可用 → p1。客户端可见流不受影响(原始帧一直在转发)。
  4. 怎么修复的:parseStream 加一个带守卫的新分支(帧带 usage + 无 finish_reason + delta 对象为空 → 发 kind:"usage",cached-tokens 走同一 openaiCachedTokens 归一化)。刻意不加 continue:帧继续按原路落到空-delta meta-forward,转发字节不变,只有内部台账多一个真实样本。
  5. 是否有回归问题:触碰路径的既有行为逐项清点——两个既有采集位(!choice 分支、finishReason 分支)各有 pinning 测试且未动(新测试 4/5);新分支与旧两形状互斥(要求 choice 存在 + finish_reason 缺失),每帧至多一个 usage 事件,core.ts last-wins 收敛为单值、重试时 settleAttemptUsage 是替换不是累加 → 无重复计数;负向控制(无 usage 的空-delta 帧不伪造事件)通过;no-continue fall-through 有回归测试钉住(客户端流字节不变);tool-call 轮 suppressCompletion 路径不受影响。一处 PR 描述低估了:bili 自己合成的 completion chunk(文本轮的 finish 帧是 bili 生成的,不是转发的)在修复前 usage 字段为零填充 {prompt_tokens:0, completion_tokens:0, total_tokens:0}(core.ts 每轮 usage={} 初始值 + adapter emitCompletion 对 truthy 空对象做 ?? 0),修复后携带真实数值。这是预期内的改善方向(客户端自己的 mapUsage 不再读到零),但「client-visible stream byte-identical」严格说只对转发的上游字节成立,对 bili 自合成 completion 不成立。建议在 PR 描述补一句,非合并阻塞。
  6. 是否破坏缓存:不涉及。前缀缓存/cache-ref 策略未触碰;采集到的 cachedTokens 只喂本地命中率统计。
  7. 是否改了设计方向:没有。纯 parser 采集面补洞,plugin/proxy 双模式判定逻辑未动(两种模式下该事件都是 internal-only,不影响工具/nudge 注入)。
  8. 是否引入新配置:没有。

独立验证(非仅信 PR 描述)

判定

代码质量达标,§7.4 auto-merge 条件除「full-suite green」外全部满足——而该条被 #4498 红风暴卡住(根因 bisect 到 f013414/#4390,修复 PR #4502 在途),与本 PR 无关。建议:#4498 修复、master 转绿后 rerun 本 PR CI 即可合并,无需改代码;可选在 PR 描述补一句第 5 问提到的合成 completion 行为变化(旧值→新值:零填充→真实 usage)。合并操作留给你(agent 不合 PR)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: zhipuai-lb/glm-5.3)

已并入合并列车 #4551(master 合并,merge commit 保留原作者署名)。本 PR 关单收尾;回滚点为 #4551 单提交 revert。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p1 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bili drops an OpenAI-compatible stream's usage when it arrives on a non-empty choices frame with an empty delta

1 participant