Repository navigation
修复:流式 message_start 的 input_tokens 恒为 0(issue #1) - #3
Open
xrj-dsh-agent wants to merge 1 commit into
Open
xrj-dsh-agent wants to merge 1 commit into
xrj-dsh-agent wants to merge 1 commit into
Conversation
xrj-dsh-agent
force-pushed
the
agent/fix-stream-input-tokens-25f0bbeabb87
branch
2 times, most recently
from
November 3, 2026 12:21
7ec74a9 to
7cae120
Compare
xrj-dsh-agent
force-pushed
the
agent/fix-stream-input-tokens-25f0bbeabb87
branch
from
November 3, 2026 14:48
7cae120 to
7dde8d2
Compare
- 新增 core/protocol/estimate:按 CJK 约 1.5 字符/token、其余约 4 字符/token 估算请求体输入量;内联图片按一张图固定值,避免 base64 长度把估算放大几十倍 - AnthropicStream 的 message_start 带估算值兜底(首帧已带上游 usage 时用真值), 收尾 message_delta 仍报上游计费的真值;上游整轮没报输入时收尾沿用估算值, 避免客户端按字段合并时把首帧锚点覆盖回 0 - message_start 的 usage 始终带 cache_read_input_tokens(缺键与 0 在部分客户端里 表现不同,上游 issue aimod-cc#424 的同一形态) - /v1/messages/count_tokens 与流式首帧共用同一估算器,上下文预算与消息 usage 同源 - 非流式聚合路径不变:Collector 仍只取上游真实 usage - 新增集成测试 server/tests/anthropic_message_start.rs(9 条用例):首帧估算、 首帧带 usage、上游无 usage、非流式不变、估算器口径
xrj-dsh-agent
force-pushed
the
agent/fix-stream-input-tokens-25f0bbeabb87
branch
from
November 3, 2026 15:16
7dde8d2 to
f9956c9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is by an automated agent, not a human.
修了什么
流式
/v1/messages的message_start以前把usage.input_tokens恒发 0,客户端(Claude Code / Claude Desktop / Hermes)拿不到输入锚点,/context退回本地估算、把每轮输入逐轮累加(issue #1:2.6M / 1M)。本次让首帧带上请求体估算值,收尾的message_delta仍报上游计费的真值。为什么首帧拿不到精确值
先查了本仓库流式管线的 token 来源:
requests.prompt_tokens只来自上游 usage(core::upstream::usage::report_usage),而真实 usage 只出现在最后一帧:stream_options.include_usage的收尾帧(空 choices);message_start也只有input_tokens: 0占位,真值在收尾前的message_delta(见anthropic_outbound::ChatFromAnthropicStream::absorb_usage)。网关没有上游分词器,六家上游也都不提供 tokenizer 接口(
/v1/messages/count_tokens一直是本地估算)。所以首帧只能给量级正确的估算,真实值由message_delta校正 —— 这也是上游 issue 建议的修法。改动
core/protocol/estimate:CJK ≈ 1.5 字符/token、其余 ≈ 4 字符/token;内联 base64 图片按一张图固定值(否则一张截图会按字符数算成几万 token)。AnthropicStream::new接收估算值:message_start用估算(首帧已带上游 usage 时用真值);message_delta用上游真值,上游整轮没报输入时沿用估算 —— 客户端会把message_delta的 usage 按字段合并进最终消息,收尾再发 0 等于把锚点覆盖回去。message_start的 usage 始终带cache_read_input_tokens(缺键与 0 在部分客户端里表现不同 —— 上游 issue [Bug] /v1/messages 流式 message_start 缺 cache_read_input_tokens,message_delta 只回 output_tokens(Hermes 缓存率恒 0) aimod-cc/Agent2API#424 的同一形态问题)。/v1/messages/count_tokens与流式首帧共用同一估算器:同一个请求在 context 预算与消息 usage 上读数一致。AnthropicCollector仍只用上游真实 usage)。SSE 帧对照(同一段上游 Chat SSE)
请求 fixture:Claude Code 风格(system + tools + 工具结果),估算值 896;上游 Chat SSE 在收尾帧报
prompt_tokens: 1412。改前(本 PR 改动前的代码;3239d12f 与当前 main 在这几处一致):
改后(本 PR):
验证
f9956c9a):artifactspassed、后端测试(agent2api-server)passed(cargo test -p agent2api-server,306 条 lib 用例 + 9 条新增集成用例)。output_tokens为 0、cache_read_input_tokens有键;首帧已带上游 usage 时用真值(400 + 缓存 100,Anthropic 口径input_tokens不含缓存);上游整轮没报 usage 时收尾沿用估算、不回落 0;非流式聚合仍报上游真值(行为未变);估算器 5 条口径用例(ASCII ≈ 4 字符/token、CJK ≈ 1.5、20 万字符 base64 图片不被放大、空请求下限 1、随内容增长)。curl -N请求/v1/messages(stream: true),首帧message_start的usage.input_tokens不再是 0,末帧message_delta是上游读数。已知限制
prompt_tokens逐 token 对齐。response.created的 usage 口径另论),避免扩大改动面。server-tests闸;本 PR 涉及的 6 个文件在 main 上自 3239d12 起没有变动,改动内容与原始基线一致。关联 issue #1。
🤖 本条回复由 AI Agent 生成