Repository navigation
fix: stop tag-echo filter from rewriting tool-call arguments - #1056
Conversation
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-1056Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr1056.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 已拉取分支核对代码并复跑全部验证,结论:可以合并(wire 形状变更,按仓库规则由人工合并)。 分支状态
代码核对(逐载体)
本地复跑(PR head): CI:head 上 8 项 check 全绿(build-artifact、test ubuntu/windows × node 22/24、version-guard、两个 codex 零 token e2e)。全量模型 e2e 本环境上游不可达、按设计跳过,已由 mock A/B 覆盖本改动的 wire 行为。 一个排序提示(非阻塞):#1054/#1053 也改 按 §7.4,此改动属 wire/message 形状变更(两种压缩模式都受影响),归入必须人工合并的类别。 中文摘要:核对了 PR 全部 diff(三种协议 + 压缩循环适配器的参数透传、死代码清理、回归测试),本地复跑 typecheck/test/build 全绿且与声明一致,CI 8 项 check 全绿,diff 干净无版本改动,可以人工合并。 |
Tool-call arguments carry user intent that hosts execute/persist (write/edit tool content, bash commands); a shape-based render-tag false positive silently corrupted them before the host ever saw them, and an unclosed tag could break host JSON.parse. Arguments now pass through verbatim on every carrier (openai chat streaming + non-streaming, anthropic input_json_delta + tool_use input, responses function_call_arguments deltas/done/completed, compress-loop adapters); the echo filter keeps its job where it belongs — model prose.
28eeac6 to
65cefd0
Compare
ranxianglei
left a comment
There was a problem hiding this comment.
独立审查(第二双眼睛)——结论:可合并 ✅
逐 diff 核对了全部 6 个文件,方向与实现均正确:
覆盖完整性(四条参数载体全部恢复逐字透传):
pipePluginChatWithStripOpenAI 流式:toolArgStreamFor/tcall 流整体删除,tool_calls[].function.arguments原样转发- Anthropic 流式:
input_json_delta不再映射partial_json,syntheticTail不再发input_json_delta - Responses 流式:
function_call_arguments.delta/.done直通,stripResponsesText不再碰arguments/delta字符串 - 压缩循环适配器 ×3:缓冲重建的
tool_call.arguments不再stripAcpTags,OpenAI taint-replay 整块删除
正文剥离保留无误:content/reasoning_content/reasoning/thinking/text_delta/reasoning_summary/output_text/marker line 全部原样;Google 路径只剥 text parts。
死代码清理干净:stripAcpTagsDeep/findAcpLikeTagSnippetsDeep/stripToolCallArgs 全仓无悬挂引用;argStreams/flushArgTails 仍服务 reasoning_summary 过滤,非死代码。
测试:5 用例覆盖三协议流式+非流式+unclosed-tag 场景,断言参数与上游逐字节一致且可解析、同一流里正文标签仍被剥。
验证(本机重跑):rebase 到当前 master 7520bdb (v0.1.130) 无冲突 → typecheck 0 错 / 2081 pass / 0 fail / build 成功;分支已 force-push 更新(28eeac6 → 65cefd0)。
已知代价(PR 描述已声明,同意取舍):#933 的「回显 ref 经参数进 TUI」回归为观感噪音,非数据损坏;根治在注入侧 renderTags 默认策略,属产品决定。
按 §7.4 本 PR 动 wire/message-shape,须人工合并。
…S.md (#1039) - tag-echo-filter.ts header: canonical invariant block — arguments are user intent, byte-exact, never routed through any filter from this file; echoed tags in args are cosmetic noise fixed on the injection side, not here. - adapters (openai/anthropic/responses) + plugin.ts openai delta path: one-line pointers at every verbatim tool_call emission site. - AGENTS.md §7.3 Wire fidelity: new 'Tool-call arguments are user intent (#1039)' guardrail so future 'fix TUI tag noise' patches don't re-add argument stripping.
|
补充(owner 要求):把「参数逐字透传」固化为原则,防止未来为修 TUI 观感 bug 再把过滤器挂回参数流。commit
门禁重跑:typecheck 0 错 / 2081 pass / 0 fail / build 成功。 |
Every fix must remove the mechanism that produces the defect, never hide its trace by rewriting payload in transit. Litmus: would the fix hold if the same root cause surfaced in a different shape? Canonical failure: #933's TUI tag noise was masked by stripping tool args mid-stream, which became #1039's silent data corruption. Out-of-scope root causes get a documented boundary note, not compensation in the wrong layer.
|
补充 2:按 owner 要求,AGENTS.md §7.3 顶部新增第一性原则(commit
以 #933→#1039 为反例教材收进去;与既有 "Symptom ≠ mechanism"(归因纪律)、Wire fidelity 里的 tool-args-verbatim 不变量互相引用,三层闭环:归因→修法→不变量。 |
…config wipe, session cap, drain leak) Address five verified defects from the #1064 bug collection: - prepareResponses: a drop-only echoed bili compaction item (a marker blob with no extractable summary) set echoReplaced=true, suppressing the forge-time summary fallback and silently losing the captured summaries. echoReplaced is now set only on a real replacement, so a drop-only echo still re-injects the forge-time summaries. - Image-token INPUT estimates omitted the billing arg (defaulting to "bytes") while the output-clamp sites passed it. The anthropic/openai/responses estimate sites now pass imageBillingFor() so pixel-tile billing hosts (openai.com / api.anthropic.com) are not inflated ~35x relative to the clamp. - plugin-install codexInstall ended the [mcp_servers.bili] block at indexOf("\n["), which misses an indented next table header and lets the block run to EOF, deleting everything after it on refresh. Use the same table-header detection as codexRemove. - getSession disk-reload path bypassed the MAX_SESSIONS cap guard (it returns before the new-session check), so evict-then-revisit grew the resident pool past the cap. Enforce the cap on the reload path too. - awaitDrain registered three one-shot listeners but removed only the one that fired, leaking two per backpressure wait (MaxListenersExceededWarning). A single shared settle now removes all three; forward()'s main compress-loop pipe uses awaitDrain instead of a bare Promise.race. Not included because already fixed upstream after the audit base moved: the codex compaction_trigger double-forward (fa18d6d, #1068) and tool-call argument tag rewriting (65cefd0, #1056). Remaining #1064 items needing an owner decision are documented in the issue thread.
…config wipe, session cap, drain leak) Address five verified defects from the #1064 bug collection: - prepareResponses: a drop-only echoed bili compaction item (a marker blob with no extractable summary) set echoReplaced=true, suppressing the forge-time summary fallback and silently losing the captured summaries. echoReplaced is now set only on a real replacement, so a drop-only echo still re-injects the forge-time summaries. - Image-token INPUT estimates omitted the billing arg (defaulting to "bytes") while the output-clamp sites passed it. The anthropic/openai/responses estimate sites now pass imageBillingFor() so pixel-tile billing hosts (openai.com / api.anthropic.com) are not inflated ~35x relative to the clamp. - plugin-install codexInstall ended the [mcp_servers.bili] block at indexOf("\n["), which misses an indented next table header and lets the block run to EOF, deleting everything after it on refresh. Use the same table-header detection as codexRemove. - getSession disk-reload path bypassed the MAX_SESSIONS cap guard (it returns before the new-session check), so evict-then-revisit grew the resident pool past the cap. Enforce the cap on the reload path too. - awaitDrain registered three one-shot listeners but removed only the one that fired, leaking two per backpressure wait (MaxListenersExceededWarning). A single shared settle now removes all three; forward()'s main compress-loop pipe uses awaitDrain instead of a bare Promise.race. Not included because already fixed upstream after the audit base moved: the codex compaction_trigger double-forward (fa18d6d, #1068) and tool-call argument tag rewriting (65cefd0, #1056). Remaining #1064 items needing an owner decision are documented in the issue thread.
…config wipe, session cap, drain leak) Address five verified defects from the #1064 bug collection: - prepareResponses: a drop-only echoed bili compaction item (a marker blob with no extractable summary) set echoReplaced=true, suppressing the forge-time summary fallback and silently losing the captured summaries. echoReplaced is now set only on a real replacement, so a drop-only echo still re-injects the forge-time summaries. - Image-token INPUT estimates omitted the billing arg (defaulting to "bytes") while the output-clamp sites passed it. The anthropic/openai/responses estimate sites now pass imageBillingFor() so pixel-tile billing hosts (openai.com / api.anthropic.com) are not inflated ~35x relative to the clamp. - plugin-install codexInstall ended the [mcp_servers.bili] block at indexOf("\n["), which misses an indented next table header and lets the block run to EOF, deleting everything after it on refresh. Use the same table-header detection as codexRemove. - getSession disk-reload path bypassed the MAX_SESSIONS cap guard (it returns before the new-session check), so evict-then-revisit grew the resident pool past the cap. Enforce the cap on the reload path too. - awaitDrain registered three one-shot listeners but removed only the one that fired, leaking two per backpressure wait (MaxListenersExceededWarning). A single shared settle now removes all three; forward()'s main compress-loop pipe uses awaitDrain instead of a bare Promise.race. Not included because already fixed upstream after the audit base moved: the codex compaction_trigger double-forward (fa18d6d, #1068) and tool-call argument tag rewriting (65cefd0, #1056). Remaining #1064 items needing an owner decision are documented in the issue thread.
Fixes #1039
What changed
The tag-echo filter added for model render-tag echo (#361/#294/#933/#935) was also hung on tool-call argument streams. Arguments carry user intent that hosts execute/persist (bash commands,
write/editfile content); a shape-based false positive silently corrupted them before the host ever saw them, and an unclosed tag could swallow past JSON structure and break hostJSON.parse. This removes the argument side of that surface everywhere; prose fields keep being stripped.Carriers made verbatim:
choices[].delta.tool_calls[].function.arguments(chat pipe) and non-streamingmessage.tool_calls[].function.argumentsinput_json_delta.partial_json(chat pipe) andtool_use.input(non-streaming)response.function_call_arguments.deltadeltas,.donepayloads,output_item.done/response.completeditem argumentstool_call.argumentsno longer pass throughstripAcpTags; the fix: strip model-echoed render tags from tool-call arguments on every wire path (rebase of #934, Fixes #933) #935 cross-fragment taint-replay in the openai adapter is removed with itstripAcpTagsDeep,findAcpLikeTagSnippetsDeep,stripToolCallArgs) and the per-stream arg state machines (toolArgStreamForet al.)Prose stripping is untouched:
content/reasoning_content/reasoning, anthropictext/thinking, responsesoutput_text/summary_text, marker lines — all still filtered (covered by existing suites + the new tests assert both sides in one stream).Evidence
tests/fix-1039-toolargs-verbatim.test.ts(replaces the fix: strip model-echoed render tags from tool-call arguments on every wire path (rebase of #934, Fixes #933) #935 arg-stripping suite, which asserted the now-removed behavior): 5 tests covering all three wires streamed + non-streamed, asserting assembled arguments are byte-identical to upstream including the tag, hostJSON.parsesucceeds, and prose in the same stream is still stripped. Includes the unclosed-tag-in-arguments case from the issue.PARSE: ok). Pre-fix the same cells showed silent truncation (107→57/95 bytes) and, at some fragment sizes, duplicated output.Pre-flight (rebased head
28eeac6over master615a496/ v0.1.129)npm run typecheck— cleannpm test— 2076 pass / 0 fail (2 skipped = e2e gate)npm run build— successE2E_CHECK=1): codex CLI present, dist fresh, but the default local upstream is unreachable from this environment, so the full run is skipped here — the mock A/B above covers this change's wire behavior directly.中文摘要:把 tag-echo 过滤器从工具调用参数流上摘掉(#935 引入的参数改写面全部移除,三种协议 + 压缩循环适配器),参数现在逐字节透传到宿主;模型正文的回显标签剥离保持不变。新增 5 条回归测试(参数含标签时与上游逐字一致、宿主可解析、同流正文仍被剥),并用固定 SSE mock 做了经 bili vs 直连的 A/B:所有分片粒度下输出与直连逐字节一致。rebase 到 v0.1.129 master 后 typecheck/test/build 全绿。