Repository navigation
[PR] fix(anthropic): never send an outbound body that ends on an assistant turn (#5604, preflight-fold tail) - #5978
Conversation
… turn (ranxianglei#5604) A preflight fold of the tail (soft protection relaxed to 0, or a tail tool pair in neverPreserveRecentTools) can leave the rebuilt Anthropic messages ending on the surviving assistant turn. Anthropic and Bedrock reject that as assistant prefill ("This model does not support assistant message prefill", "final assistant content cannot end with trailing whitespace"). prepareAnthropic now closes such a tail with one thin ephemeral user nudge, inserted before the trailing system run and after all cache-stable content, via the kernel's insertBeforeTrailingSystemRun. A client-authored assistant tail (deliberate prefill) passes through unchanged.
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。与 #5605 同症状家族(#5604)但机制不同:#5605 修 failed-compress 的 loop re-request(第一机制),本 PR 修 preflight-fold 的出站尾巴(第二机制)——两者都是 #5891 讨论里点名的 #5604 子缺陷,互补不重叠。检索了 open issue/PR(assistant-tail / prefill / 5604 关键词),没有其他条目动这个 egress-guard 机制。 标签: bug / claude-code / scope/wiring / p1 —— anthropic-wire 出站层的缺陷(prepare-anthropic.ts);Claude Code 是复现载体;p1 因为一旦触发会话直接死掉(fatal 400 循环),虽然 #4898 之后触发概率已很低。 Review verdict: mergeable. Verified live just now: PR open, base = current master What I independently verified (beyond reading the description):
One coordination item (non-blocking): #5441 merge order. #5441 adds Minor observation (accepted, no action needed): unit test 6 pins the client-prefill pass-through using the same array for both arguments; the distinct-array variant exercises the identical code path (two independent Behavior change disclosure (old → new): previously an outbound body ending on an assistant turn — produced by bili's own fold — was forwarded as-is → upstream HTTP 400 ("This model does not support assistant message prefill") → client retries the same body until it gives up → session death. Now: one ephemeral user nudge ( 结构化八问(review 结论)
判定:可合并——建议 owner 在方便时 merge(合并顺序注意上面 #5441 那条)。 |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.1103-master.c4bae93 # this exact build
npm install -g billion-context@master # always the newest merged build
|
…anxianglei#5970 retained folds, ranxianglei#5978 anthropic egress tail, ranxianglei#5981 truncated compress receipt Advances the base from 33d4951 to 7d18b7a (15 commits, 35 files). Four files overlap with this lane: src/plugin.ts, src/server/relay.ts and the two generated docs. All four auto-merged; upstream added `noteTruncatedUpstream` calls inside the plugin pipelines it already touches, while the lane's strip gates live elsewhere in the same files. Notable upstream commits adopted: - 0f93793 feat(pi): pi.commands — configurable slash-command registration surface - 4f80702 fix: recover retained compression blocks against fork replay - 8c98898 fix(anthropic): never send an outbound body that ends on an assistant turn - adc863a fix: tell the model to re-issue a compress whose args were cut in transit - 38e6b07 fix: bound the lost-in-transit re-issue advice with the session breaker - d1b2d8f fix: 会话历史按真实维度拦死 — count gate + 32MiB ingest cost band - fa447f9 fix: exempt image-bearing bodies from the ingest cost band - 33bec44 fix(launcher): force-takeover the wedged same-lane proxy holding the canonical port - 2826c51 fix(launcher): field-tune wedged-takeover timings + quiet win32 ESRCH alarm
Summary
A preflight fold can leave the outbound Anthropic
messagesarray ending on an assistant turn. Anthropic and Bedrock reject that body as assistant prefill:This model does not support assistant message prefill. The conversation must end with a user message.final assistant content cannot end with trailing whitespace(same shape, when the surviving assistant text ends in whitespace)This PR is the smaller, targeted variant named in the #5891 / #5605 discussion: one guard local to
prepareAnthropic. It is the second mechanism of the #5604 symptom family (the first one is the failed-compress re-request that #5605 fixes). It closes nothing by itself. It does not touch #5605.Root cause
tool_result(for example a largeReadresult).relaxedConfig,src/preflight.ts:1554, used atsrc/preflight.ts:2318,:2359and:3125). The same happens without the relax when the tail tool is inneverPreserveRecentTools.tool_use/tool_resultpair. The text part of the last assistant message survives, so the rebuilt array ends on that assistant message.prepareAnthropic(src/server/prepare-anthropic.ts) forwards the rebuilt array as is. The upstream rejects it, and the client retries the same body until it gives up.Line numbers are on master
848f59b99.The change
src/server/prepare-anthropic.ts, after the rebuild and beforededupeAdjacentOutputConfig:ensureUserTail(outbound, clientMessages).assistant, and the last non-system client message is notassistant.[bili] Earlier context was compressed above. Continue the task., withinsertBeforeTrailingSystemRun. The nudge goes before the trailing system run and after all cache-stable content, so no cached prefix moves.Notes on the shape:
insertBeforeTrailingSystemRunis already exported by the kernel on master (kernel/src/wire/anthropic.ts:492, viaacp-kernel/wire). The host never called it. This PR imports the kernel copy, so it applies to master alone and does not depend on [PR] fix: insert Anthropic ephemeral tails before trailing client systems (#5432) #5441. If [PR] fix: insert Anthropic ephemeral tails before trailing client systems (#5432) #5441 merges first, its../util.jsimport of the same name collides (TS duplicate identifier). Fix: drop one of the two imports. The kernel doc comment says the host copy is interim, so the kernel import is the long-term one.tests/cache-proof.test.tsandtests/cache-friendly-plugin.test.tsfail (6 tests). Their clients send their own assistant-tail bodies, and the nudge rewrites those bodies. So a client-authored assistant tail (deliberate prefill) passes through unchanged. The guard fires only when bili itself turned a user tail into an assistant tail. If you prefer the strict egress invariant, change one line and declare the nudge as a carrier inNOTE_MARKERSofcache-proof.test.ts.ensureUserTaillater.Tests
New
tests/issue5604-anthropic-egress-user-tail.test.ts:Readpair over a 24K window. Preflight folds the trailing pair. The test checks that the forwarded body ends on a user turn and that the last message is the nudge. Without the guard it fails: the forwarded body ends on the surviving assistant text, which is the exact upstream-rejected shape.Results on this branch (
848f59b99+ this commit):npm run typecheck: pass.npm test): 10384 pass, 7 fail, 64 skipped. The same 7 tests fail on plain master in the same sandbox (dumps-dir,identity-harvest,issue5627-storage-failure-forensics,launcher,mitm; they depend on the home directory and network). None of them touches this code.Bedrock measurements
Chain: client -> bili -> LiteLLM -> Bedrock (eu-central-1). Builds: "before" = master
848f59b99+ #4898 + #5441 + #5605 + #5966. "after" = "before" + this PR. #5966 is in both builds so that agentic sessions continue after a model fold.Direct reproducer (the e2e test shape against real Bedrock): a Claude Code-like body with two small
Readpairs and a 240K-char trailingReadresult, 55K-token window, fresh session per request, 3 requests per model.does not support assistant message prefilldoes not support assistant message prefillAgentic runs (an agentic Java coding task, Claude Code -> Headroom -> bili -> LiteLLM -> Bedrock, Opus, 55K window, 3 runs per build): 0 upstream 400s in both builds (98 requests before, 90 after). No outbound body ended on an assistant turn, and the guard never fired. Every run finished with a non-empty diff. Earlier, the same task on master + #4898 + #5441 + #5605 + a bug-4 loop hit this error in 1 of 3 runs (4 of 21 requests, and the session failed). So the residual is rare after #4898, but it is real and fatal when it happens.
Haiku regression cells (Claude Code -> bili -> LiteLLM -> Bedrock, 4 sessions each, before / after):
small0/84 and 0/108 upstream 400s,g2b0/29 and 0/33, Headroom in frontg2b0/27 and 0/30, replay 0/35 and 0/32. In all cells, no outbound body ends on an assistant turn, and the guard never fires on bodies that already end on a user turn. Cache hit and saved tokens stay in the same range (for examplesmall0.764 and 0.771).Spend for all measurements: about USD 20.6.
Links
insertBeforeTrailingSystemRuncopy (see the import note above).thinking ... cannot be modified400 (0.1.212, Claude Code) #5891: the discussion that named the targetedprepareAnthropicguard.This PR closes no issue by itself. It is the targeted variant. The owner can still choose the shared egress guard (option (b) in #5605).