Repository navigation
[PR] fix: execute proxy-owned tools on the non-stream Anthropic path (#5957) - #5966
Conversation
A stream:false Anthropic response that calls a proxy tool (compress, acp_status, search_context, decompress) was rewritten once: each tool_use became its result text and stop_reason was forced to end_turn, so a compress round ended the client turn with the fold receipt and the model never saw the result. Port the streaming decision (core.ts reRequest gate) to this lane: proxy-only rounds execute through the same executors, append the original assistant turn plus a tool_result user turn to the forwarded body, and re-request; mixed or tool-free rounds fall through to the untouched one-shot rewriter. Bounded by MAX_LOOP_ROUNDS with degradation to the pre-fix shape at the limit.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.8687-pr.5966.ef5a901This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5966.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 同家族但机制域不同——本 PR 是 Anthropic wire( 开始审核:分支相对当前 master 新鲜(1 commit ahead / 0 behind,base = master tip |
…eam Anthropic loop
(ai: vllm-qwen/qwen3.8-27b) Review of PR #5966 — verdict: mergeable, human tier (client-visible message-shape change on the Anthropic non-stream lane → auto-merge gate §7.4 "must stay human"). Dedup conclusion was posted in floor 2: not a duplicate; the OpenAI-lane analogue is open PR #4037 (issue #4019), different wire protocol → separate thread justified. What I verified independently (head
|
|
Field check of #5966 (head Setup
Results (6 runs each)
Problems found: none in this PR. The runs had no thinking blocks in the replayed assistant turn, so the live check does not cover the signed-thinking replay. Your test covers it. One note for others who reproduce this with Headroom: if the Headroom process has no network and no tiktoken cache, it skips compression and adds no CCR markers. Then it keeps the streaming path and the non-stream lane never runs. Set Thanks for the port. From our side it can merge. |
(ai: vllm-qwen/qwen3.8-27b) Status update (the full review with the 8-question checklist is in my previous comment):
|
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1112-master.ba0ae45 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Problem
A
stream:falseAnthropic response that calls a proxy tool (compress,acp_status,search_context,decompress) was rewritten once byrewriteJsonSteps: each proxytool_usebecame its result text andstop_reasonwas forced toend_turn. A compress round therefore ended the client turn with the fold receipt as the turn's only output — the model never saw the fold result, and any follow-up in that turn was lost. The streaming lane has no such gap: its loop re-requests until the model yields text or a real client tool call (core.ts:1531).This is the Anthropic-lane analogue of #4019 / #4037 (OpenAI chat-completions).
Fix
New
src/compress-loop-anthropic.ts(compressLoopAnthropicJson), wired into the relay's non-stream section forprepared.protocol === "anthropic":tool_useblocks from the current upstream response (sameisProxyToolForclassification as every other lane).applyConfiguredCompressionfor compress,executeAnthropicProxyToolotherwise; raw result text, matching the stream lane's carrier), append the original assistant content plus atool_resultuser turn to the forwarded body, and re-request. Retrieval injections ride the re-request after their ack (Feature: content-addressed message store — ID-referenced tool results + acp_retrieve (built-in CCR) #1097 parity).MAX_LOOP_ROUNDS; at the limit the last response falls through to the one-shot rewriter (bounded degradation to the pre-fix shape — never an undeclared proxy tool_use handed to the client).The continuation always ends on the tool_result USER turn (never an assistant prefill, #5605), and the replayed assistant turn carries its content verbatim — signed thinking blocks included (KDD #10). Like #4037 the re-request resends the UNFOLDED body; the fold materializes on the next client request (prefix-cache friendly).
Behavior change (disclosed)
Non-stream Anthropic proxy-only rounds now make one extra upstream request per proxy-call round and deliver the model's real post-compress answer instead of ending the client turn with the receipt +
end_turn. Client-visible turn output changes from the receipt text to the final answer.tests/ccr-v2.test.ts("e2e CCR v2 non-stream") previously pinned the buggy shape and now pins the fixed one.Tests
tests/issue5957-nonstream-anthropic-proxy-loop.test.ts(8 tests): execution + re-request continuation shape for acp_status / search_context / decompress / compress; negative gates (real client tool_use, plain text, mixed rounds → zero fetches, byte-identical passthrough); two proxy calls in one round → single re-request with both tool_results in order; signed thinking block rides the replayed assistant turn verbatim (KDD release v0.1.3 #10); rejected-compress receipt rides the tool_result while final text still reaches the client.tests/ccr-v2.test.tsnon-stream e2e: turn 1 now asserts the client gets the model's answer (no tool_use block), the fold receipt rides the re-request's tool_result, unfolded history is resent on the re-request, the folded view materializes on turn 2 outbound, and the range restore rides the turn-2 re-request.Note: open PR #4037 touches the same relay.ts non-stream region (OpenAI lane) — no textual conflict at this base, but review them together if they merge close in time.
Closes #5957