Skip to content

[Bug][anthropic] fold summary rendered as user right after a mid-conversation system message → 400 "role 'system' must precede an 'assistant' message" (0.1.204, since #3992) #4292

Description

@Maziar123

Environment

  • billion-context 0.1.204 (npm latest), plugin lane dist/agent/omp-native.js in omp 18.12.6, Linux.
  • Upstream: Anthropic Messages API, claude-opus-5-5. The client sends beta mid-conversation-system-2026-04-07.
  • Code references below are master 94f75ccea (2026-10-22).

Symptom

After a fold, every following request of the session fails with:

400 invalid_request_error: messages.N: role 'system' must precede an 'assistant' message or end the array; the directive-only form (content: [] with output_config) is accepted at any position
  • Three sessions hit it on 2026-10-22: a main session, a subagent, and a large long-running session.
  • N moves with the fold boundary (messages.1, .2, .13, .110). It points at the fold boundary, not at a fixed message.
  • It is permanent. Each later request carries the same folded history, so the session returns 400 until its fold state is dropped. bili.log logs [invariant-judge/main] I2 violation: carrier-drift as a warning before the request is sent anyway.
  • The client's own request (as logged before the proxy) satisfies the rule. The violating sequence exists only in the body bili rebuilds.

Root cause

Since #3991 / PR #3992 (released in 0.1.204), the Anthropic wire keeps plain-text mid-conversation role:"system" messages verbatim (kernel/src/wire/anthropic.ts:384-387, rawAnthropicMessage → rebuildSystemMessage). The fold summary, however, is still rendered as a user message anchored at the first folded message:

  • kernel/src/prune.ts:182 collectSummaryAnchors → :232 insertAt: earliest ?? fallbackIndex, the earliest of the block's direct/effective message ids.
  • kernel/src/prune.ts:314 pairSafeAnchorIndex and the 192版本缓存命中率低codex #2733 first-user clamp adjust that index for tool pairs and the cache pin only. Nothing checks whether messages[insertAt - 1] is a mid-conversation system message.

A fold whose range begins right after a system message therefore produces:

client:  … system | assistant | user | assistant …   ← valid
bili:    … system | user(summary) | assistant …       ← 400

Under the beta, a mid-conversation system message that is not directive-only must be followed by an assistant message or end the array. In agent sessions such messages are common: background-job notices, steering messages delivered to subagents, <system-reminder> blocks, and the tool_addition carrier.

Why the self-heal never fires on the Anthropic lane

  • src/server/relay.ts:374-375 arms compatProtocol only for openai / responses. The learn-on-failure retry at src/server/relay.ts:844-848 is gated on compatProtocol, so an Anthropic request never enters it.
  • Even if it did, the markers in src/compat-roles.ts:114 detectSystemPlacementError (multiple/second system, "index N", "beginning/start/first") do not match must precede an 'assistant' message or end the array. hasOffHeadSystem also only inspects the openai messages and responses input shapes.

Second trigger: no compress call needed

After a proxy restart, a session that is cold-loaded with no blocks gets bili's preflight overflow fold (bili counts claude-opus-5-5 as a 200 000-token window). On one session that fold started right after the session's tool_addition system message and gave the same 400 (messages.2, three times), although the model never called compress. A client-side rule such as "don't start a compress range right after a system message" is therefore not a sufficient workaround.

Evidence

A/B against api.anthropic.com, same model, small fixed history:

Messages Result
system → assistant 200
system → user 400 (wording above)
system, system → assistant 200
trailing system at the end of the array 200

Replaying a failing rebuilt body with each change applied separately:

Variant Result
bili's order (system → user(summary) → assistant) 400
summary inserted before the system message 200
that system message sent as user 200
the client's original body (no fold) 200

Proposed fix

Make the fix at the wire level, so that every producer of a summary is covered (model compress, preflight fold, any future anchor). As the last step of the Anthropic conversion, move each mid-conversation system message that is not directive-only forward to the next assistant message, or to the end of the array when no assistant message follows:

function isAnthropicDirectiveOnlySystem(m: AnthropicWireMessage): boolean {
  return Array.isArray(m.content) && m.content.length === 0 &&
    (m as { output_config?: unknown }).output_config != null;
}

export function hoistAnthropicMidSystems(msgs: AnthropicWireMessage[]): AnthropicWireMessage[] {
  let pending: AnthropicWireMessage[] | null = null;
  let out: AnthropicWireMessage[] | null = null;
  for (let i = 0; i < msgs.length; i++) {
    const m = msgs[i]!;
    if (m.role === "system" && !isAnthropicDirectiveOnlySystem(m)) {
      out ??= msgs.slice(0, i);
      (pending ??= []).push(m);
      continue;
    }
    if (pending !== null && m.role === "assistant") {
      out!.push(...pending);
      pending = null;
    }
    out?.push(m);
  }
  if (out === null) return msgs; // no mid-conversation system message: same array, byte-identical
  if (pending !== null) out.push(...pending);
  return out;
}

Call sites:

  1. kernel/src/wire/anthropic.ts:436: in coreToAnthropic, return out; → return hoistAnthropicMidSystems(out);
  2. src/server/prepare-anthropic.ts:519: messages: rebuiltMessages → messages: hoistAnthropicMidSystems(rebuiltMessages). Change only this Anthropic rebuild (the one carrying system: systemOut), not the look-alike rebuilds of the other protocols.

Properties:

  • It is a stable partition: the relative order of system messages and of all other messages is kept.
  • Directive-only system messages (content: [] + output_config) stay where they are.
  • An array with no mid-conversation system message is returned by identity, so the common path costs nothing.

We run exactly this as a local patch on the 0.1.204 dist/index.js and dist/agent/pi-native.js. Since then there has been no recurrence across several folds in several Opus sessions.

Complementary changes (optional):

  • Kernel: when the anchor lands right after a verbatim mid-conversation system message, insert the summary before that message. This is variant 2 of the replay table; it also passed.
  • Relay: arm the compat retry for anthropic, and add a placement marker for must precede an 'assistant' message.
  • Consider making the I2 carrier-drift judge block the send instead of only warning, since here it predicted the 400.
  • Regression test idea, next to kernel/tests/anthropic-system-message-passthrough.test.ts: a fold whose first folded message follows a mid-conversation system message must still emit that system message directly before an assistant message, or at the end.

Related (not duplicates)

Activity

  1. ranxianglei commented on Oct 22, 2026

    @ranxianglei
    Owner

    ework

  2. darkness-A commented on Oct 23, 2026

    @darkness-A
    Contributor

    Independent confirmation: Claude Code CLI 2.1.308 / bili 0.1.204

    Adding an independent reproduction from the Claude Code CLI proxy/launcher lane, rather than the native plugin lane in the original report. This matches the system/summary boundary failure in this issue, so I am adding evidence here instead of opening a duplicate.

    Environment and scope

    • billion-context@0.1.204, Claude-specific proxy (claude lane).
    • Claude Code CLI 2.1.308, model claude-sonnet-5-5, Ubuntu 26.04.1 LTS.
    • The CLI was used through a separate host integration, but its Claude Code channel bypasses that host's independent bili bridge; the affected request went through the Claude-specific proxy.
    • The running dist/index.js SHA-256 matches a separately downloaded, exact official npm 0.1.204 package:
      104502cb11fa98e205f61ba9cc0dde8d3d03b57d8330bfcbafe84f8d130ed2d9.
    • This report concerns that exact published version. It does not claim to test current master, a newer release, or a proposed fix. No production files/configuration/ledger were changed or proxy restarted during diagnosis.

    What happened (2026-10-23, UTC)

    1. 04:52:51.707–04:52:51.714: a compress call completed successfully, creating a block with a 15,805-character summary and approximately 49,448 reported tokens saved. bili refreshed its post-compress fold and issued the internal re-request.

    2. 04:52:52.604: that re-request returned HTTP 400:

      messages.1: role 'system' must precede an 'assistant' message or end the array; the directive-only form (content: [] with output_config) is accepted at any position
      
    3. The user selected Continue. The first ordinary request still failed at 04:55:17.713. Its recorded client shape contained 50 messages and the same mid-conversation-system format.

    4. A subsequent CLI request at 04:55:17.723 changed shape: 50 → 45 messages; the mid-conversation system messages were no longer present, and these three beta flags disappeared:

      • per-turn-control-2026-07-01
      • mid-conversation-tool-changes-2026-07-01
      • inline-tools-2026-09-15
    5. 04:55:21.567: that later request received successful usage (input=65790, cached=35145, as reported by bili).

    Thus, in this client the session appeared to recover after Continue, but the evidence shows a changed request format consistent with CLI compatibility fallback, not a successful identical retry and not bili repairing the fold. These observations do not prove that removing those flags is sufficient in every case or is an appropriate fix. Compression itself had succeeded; this was not a loss of compression blocks.

    Independently reproduced mechanism

    Using the actual published bundle's anthropicToCore, prune, coreToAnthropic, and createInitialState functions:

    Input wire:    user → system(text + effort) → assistant → user
    After prune:   user → raw system → summary(kernel system) → user
    Output wire:   user → system(text + effort) → user(summary + later content)
                                                       ^ invalid successor
    

    The original system survives through rawAnthropicMessage / rebuildSystemMessage; the covered assistant is removed, and the newly rendered kernel summary has no raw system payload, so the encoder's role === "assistant" ? "assistant" : "user" path makes it a wire user. No check restores the content-system boundary in this path. In the private persisted block, the immediately following real assistant's reasoning/tool-call IDs were also among the covered IDs. No private IDs or message bodies are included here.

    The placement rule is documented in Anthropic's mid-conversation system message documentation, under How it works: content-bearing systems require the appropriate predecessor and an assistant successor (or end of array); consecutive systems are judged together. The empty-content effort-only directive is an exception. Text plus effort is not that exception.

    Fully synthetic offline reproducer

    The script below extracts the pure functions and their declaration closure from the pinned bundle; it does not import/start the proxy entrypoint. All message text and the compression block are synthetic. This avoids sharing private transcripts, prompts, tools, session IDs, credentials, or databases. The script verifies the exact bundle hash before running.

    Prerequisites: Node.js >=20. In a disposable directory, install:

    npm init -y
    npm install --ignore-scripts --save-exact billion-context@0.1.204 typescript@5.9.3
    # Save the script below as public-repro.mjs in that directory.
    node public-repro.mjs ./node_modules/billion-context/dist/index.js

    Results actually obtained against the separately downloaded official bundle:

    Synthetic case Input successor rule Output successor rule
    Text + effort system; following assistant folded Valid Invalid: system at index 1 followed by user
    Same system; no fold Valid Valid
    Empty content + effort only; following assistant folded Valid Valid under directive-only exception
    Plain-text system; following assistant folded Valid Invalid: system at index 1 followed by user

    4/4 assertions passed. These are assertions reproducing the bug and controls, not a claim that the behavior is fixed. liveNetworkUsed=false, productionModified=false. The small validator checks only the successor constraint in the observed error, not the complete Anthropic protocol. This is not an exact replay of historical outbound bytes and does not make a fresh upstream request to validate the 400.

    Copyable public-repro.mjs (all inputs synthetic) ```js // Offline reproduction for billion-context#4292. All message text is synthetic. // Run from a directory containing typescript@5.9.3: // node public-repro.mjs ./node_modules/billion-context/dist/index.js import fs from 'node:fs'; import assert from 'node:assert/strict'; import { createHash } from 'node:crypto'; import { createRequire } from 'node:module'; const require = createRequire(import.meta.url); const ts = require(require.resolve('typescript', { paths: [process.cwd()] })); const source = process.argv[2]; if (!source) throw new Error('Pass the official 0.1.204 dist/index.js path.'); const text = fs.readFileSync(source, 'utf8'); const sourceBundleSHA256 = createHash('sha256').update(text).digest('hex'); assert.equal(sourceBundleSHA256, '104502cb11fa98e205f61ba9cc0dde8d3d03b57d8330bfcbafe84f8d130ed2d9');

    // Extract pure function declarations and their local declaration closure.
    // Do NOT import the proxy entrypoint or execute its startup code.
    const ast = ts.createSourceFile(source, text, ts.ScriptTarget.Latest, true, ts.ScriptKind.JS);
    const defs = new Map();
    for (const node of ast.statements) {
    if (ts.isFunctionDeclaration(node) && node.name) defs.set(node.name.text, node);
    if (ts.isVariableStatement(node)) {
    for (const d of node.declarationList.declarations) {
    if (ts.isIdentifier(d.name)) defs.set(d.name.text, d);
    }
    }
    }
    const selected = new Map();
    function add(name) {
    if (selected.has(name) || !defs.has(name)) return;
    const node = defs.get(name);
    selected.set(name, node);
    function walk(n) {
    if (ts.isIdentifier(n)) {
    const parent = n.parent;
    if (!(ts.isPropertyAccessExpression(parent) && parent.name === n) &&
    !(ts.isPropertyAssignment(parent) && parent.name === n)) add(n.text);
    }
    ts.forEachChild(n, walk);
    }
    walk(node);
    }
    const names = ['prune', 'anthropicToCore', 'coreToAnthropic', 'createInitialState'];
    names.forEach(add);
    for (const name of names) assert.ok(selected.has(name), Missing ${name});
    const ordered = [...selected.values()].sort((a, b) => a.pos - b.pos);
    const code = ordered.map(n => ts.isVariableDeclaration(n) ? var ${n.getText(ast)}; : n.getText(ast)).join('\n');
    const moduleText = import { createHash as createHash5 } from 'node:crypto';\n${code}\nexport { ${names.join(', ')} };;
    const { prune, anthropicToCore, coreToAnthropic, createInitialState } =
    await import('data:text/javascript;base64,' + Buffer.from(moduleText).toString('base64'));

    // Checks ONLY the successor constraint named in the observed error.
    // Not a complete Anthropic validator; consecutive systems are one group.
    function invalidContentSystem(messages) {
    const errors = [];
    for (let i = 0; i < messages.length; i++) {
    if (messages[i].role !== 'system') continue;
    let last = i;
    while (messages[last + 1]?.role === 'system') last++;
    const hasContent = messages.slice(i, last + 1).some(m => m.content.length > 0);
    if (hasContent && last + 1 < messages.length && messages[last + 1].role !== 'assistant') {
    errors.push({ index: i, nextRole: messages[last + 1].role });
    }
    i = last;
    }
    return errors;
    }
    function run(name, system, foldAssistant = true) {
    const body = { messages: [
    { role: 'user', content: [{ type: 'text', text: 'Synthetic opening request.' }] },
    structuredClone(system),
    { role: 'assistant', content: [{ type: 'text', text: 'Synthetic historical response.' }] },
    { role: 'user', content: [{ type: 'text', text: 'Synthetic later request.' }] },
    ] };
    const { msgs, cacheControls } = anthropicToCore(body);
    const before = coreToAnthropic(msgs, cacheControls);
    assert.deepEqual(invalidContentSystem(before), []);
    const state = createInitialState();
    if (foldAssistant) {
    const ids = msgs.filter(m => m.role === 'assistant').map(m => m.id);
    assert.ok(ids.length > 0);
    state.blocks.push({ blockId: 'b_test', active: true, tier: 1,
    summary: 'Synthetic summary.', directMessageIds: ids, effectiveMessageIds: ids,
    directBlockIds: [], createdAt: 1 });
    }
    const folded = prune(msgs, state);
    const after = coreToAnthropic(folded, cacheControls);
    assert.deepEqual(after.find(m => m.role === 'system')?.output_config, system.output_config);
    return { name, beforeRoles: before.map(m => m.role),
    afterCoreRoles: folded.map(m => ({ role: m.role, summary: m.id.startsWith('acp_summary_'),
    rawSystem: !!m.rawAnthropicMessage })),
    afterRoles: after.map(m => m.role), invalid: invalidContentSystem(after) };
    }
    const textAndEffort = { role: 'system',
    content: [{ type: 'text', text: 'Synthetic operator instruction.' }],
    output_config: { effort: 'high' } };
    const reports = [
    run('text+effort / fold assistant', textAndEffort),
    run('text+effort / no fold', textAndEffort, false),
    run('directive-only / fold assistant', { role: 'system', content: [], output_config: { effort: 'high' } }),
    run('plain-text / fold assistant', { role: 'system', content: [{ type: 'text', text: 'Synthetic operator instruction.' }] }),
    ];
    assert.deepEqual(reports[0].invalid, [{ index: 1, nextRole: 'user' }]);
    assert.deepEqual(reports[1].invalid, []);
    assert.deepEqual(reports[2].invalid, []);
    assert.deepEqual(reports[3].invalid, [{ index: 1, nextRole: 'user' }]);
    console.log(JSON.stringify({ sourceBundleSHA256, testsPassed: reports.length,
    liveNetworkUsed: false, productionModified: false, reports,
    limitation: 'Synthetic inputs and fold block, actual pinned bundle functions. Not a replay of historical outbound bytes or a live API validation.' }, null, 2));

    
    </details>
    
    ### Regression / fix considerations
    
    - Please cover both the ordinary outgoing-request path and the compress-loop's internal re-request. This incident first failed on the internal re-request and failed again on the subsequent ordinary request.
    - Include content-only systems, text plus effort, empty effort-only directives, consecutive system groups, tool changes, and multiple folds. Validate both predecessor and successor placement as well as tool-use/result pairing.
    - The actual pinned bundle preserves raw mid-conversation systems, while its packaged source map does not describe the same system-message implementation. For this reproduction I inspected/extracted the actual bundle, not that map.
    - The wire-level boundary is an appropriate place to validate the final result, but I have **not** tested the proposed hoisting patch in this issue. Any repair should preserve instruction scope/timing, output_config, tool additions/removals, and pairing semantics; simply deleting systems, converting all of them to user, inserting a fake assistant, or moving all of them to the top-level system is not a verified semantics-preserving fix.
    
    No private transcript, original system prompt, request/session identifier, credentials, or database is included. The public repro above uses only synthetic fixtures; private historical evidence is summarized rather than uploaded.
    
  3. ranxianglei commented on Oct 23, 2026

    @ranxianglei
    Owner

    (ai: zhipuai-lb/glm-5.3)

    修复 PR:#4396(kernel: fold 孤儿 system 致 Anthropic 会话永久 400 — 出站折叠签名守卫,Fixes #4292)

    一句话:压缩折叠会把合法化客户端 system 消息的 assistant 覆盖掉,而摘要锚点必然插在被覆盖区的最早位置,于是出站变成 [.., system, user(锚点), ..] —— Anthropic 位置规则直接 400,且每次重建都产出同样形状,会话永久不可恢复。

    修复放在 kernel 发射层(coreToAnthropic,插件/代理两种模式共用):识别「sidecar system 紧邻摘要锚点」这一折叠孤儿签名,仅对这条 run 重配音为 user(内容逐字节不变)。健康流量零影响——守卫不触发时出站逐字节不变;Pi 的 tool_addition system 往返测试不改一行通过。

    kernel 0.0.159 → 0.0.160;kernel 全套 1459/1459、typecheck、wire-contract 22/22 全绿。kernel 变更走人审合并。

  4. added a commit that references this issue on Oct 23, 2026
  5. ranxianglei commented on Oct 23, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    对 #4396 做了独立复核(不是复述修复描述,是给这个 kernel 改动加第二双眼睛)。结论:方向对、层次对、无回归;顺手补了一个精度缺口,已推新 commit 87bb2c492。kernel 变更走人审合并,我不代合。

    结构化八问

    1. 是什么问题:客户端开了 mid-conversation-system beta,一次压缩折叠之后整个会话永久 400(role 'system' must precede an 'assistant' message or end the array)。报错下标 N 跟着折叠边界走,不丢折叠状态就一直坏。
    2. 什么原因导致:[Bug][pi] anthropic wire: mid-conversation role:"system" messages are re-emitted as user — tool_addition blocks dropped on 0.1.202, hard 400 on master #3991(v0.1.204)让会话中部的客户端 system 消息带 sidecar 原位 verbatim 重发;但折叠会把"合法化"这条 system 的那些 assistant 覆盖掉,而 prune 把摘要锚点插在最早被覆盖位(kernel/src/prune.ts insertAt)——于是孤儿 system 后面紧跟锚点,锚点按既有映射发成 user,出站变成 [.., system, user(锚点), ..],撞上 Anthropic 位置规则。每次重建都产出同样形状,所以是永久。
    3. 影响范围与严重程度:p1。任何用 mid-conversation-system beta 的 Anthropic 客户端(omp / Claude Code 等),只要折叠边界落在一条中部 system 之后就中招;两个触发口都覆盖(模型 compress + 重启后的 preflight overflow fold)。命中即该会话不可用,直到丢折叠状态。
    4. 怎么修复的:放在 kernel 发射层 coreToAnthropic(插件/代理两种模式共用),识别"无 sidecar 的 core system(= 摘要锚点)紧邻在 sidecar system 之后"这一折叠孤儿签名,只把这一 run 重配音为 user(内容逐字节不变、换新对象、sidecar 永不被 mutate)。按签名触发而非按位置合法性触发——按位置触发会破坏无损往返不变量、把 Pi 的 tool_addition system 错误重配音(那正是 anthropic wire silently drops unknown top-level blocks (incl. PDF document) when rebuilding history #3410 时代被上游拒绝的形状)、还会对健康流量动字节(伤缓存前缀)。
    5. 是否有回归:逐项过了——健康流量守卫不触发 ⇒ 出站逐字节不变;Pi 尾部 tool_addition/effort run 往返逐字节不改一行通过(anthropic wire silently drops unknown top-level blocks (incl. PDF document) when rebuilding history #3410 未回归);客户端自己发的位置非法形状保持 verbatim(kernel 是传输层不是校验器);sidecar 对象不被改写。无回归。
    6. 是否破坏缓存:不。守卫不触发时出站逐字节不变;只有本来就在 400 的孤儿 run 才改变形状(那些会话此前已不可用)。
    7. 是否改了设计方向:无架构转向,改动落在既有 kernel wire codec 内,两种压缩模式的设计都不动。
    8. 是否引入新配置:无新增 env/config/默认值。

    我顺手修的精度缺口(commit 87bb2c492):原实现的重配音循环把所有尾部 system 都转成 user,包括 directive-only(content: [] + output_config)。这类是 Anthropic 明确"任意位置都接受"的形态,把它转成 user 会造出一条空内容 user 消息、自己反而 400。改成跳过它们(保留 system)——正好对齐 issue 原提案里的 isAnthropicDirectiveOnlySystem。触发概率低(纯 effort 标记通常在数组尾,不会恰好落在中部锚点前),但这是 p1 可用性修复里一条真实的"自引入 400"路径,几行就能收,按"小问题当场修"处理,并补了回归测试钉住。

    本地定向验证(全量交给 CI):kernel 目标套件 12/12(含新增 directive-only 用例)、bili 集成 5/5、npm run typecheck 干净。base = 当前 master tip(未落后,无需 rebase)。我复核时 CI 还在排队,现在会在新 head 87bb2c492 上重跑——请等 CI 全绿再合。

    残留(作者已如实披露,可接受):#4311 锚点跳过场景不构成漏洞(幸存的 assistant tool_use 会重新合法化其前的 system);未来若出现"非锚点邻接"的其它孤儿化机制需要新签名——目前不存在这种机制。

    判定:代码层面 LGTM;kern

  6. ranxianglei commented on Oct 23, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    上一条结尾被截断,这里补完整判定:代码层面 LGTM —— kernel-change 声明齐全、kernel 版本 0.0.159→0.0.160、双层测试本地通过(kernel 目标套件 12/12、bili 集成 5/5、typecheck 干净)。这是 kernel 变更,走人审合并,我不代合;请等 CI 在新 head 87bb2c492 全绿后,由你合并 #4396 即可。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions