Skip to content

[PR] kernel: heal system positions on the final Anthropic wire array (#4570) - #4715

Merged
ranxianglei merged 23 commits into
ranxianglei:masterfrom
Mako-L:2026-10-25_system-position-heal
Nov 9, 2026
Merged

ranxianglei merged 23 commits into
ranxianglei:masterfrom
Mako-L:2026-10-25_system-position-heal

Conversation

@Mako-L

@Mako-L Mako-L commented Oct 25, 2026

Copy link
Copy Markdown

Fixes #4570

kernel-change: healIllegalSystemPositions only enforced half of Anthropic's mid-conversation system rule and ran before the host appended messages; the heal needs the predecessor half and a public entry point for the final wire array.

Errors seen in production (Claude Code → bili → Anthropic)

messages.35: role 'system' must precede an 'assistant' message or end the array; the directive-only form (content: [] with output_config) is accepted at any position
messages.8: role 'system' must follow a 'user' message or an 'assistant' message ending in a server tool result; the directive-only form (content: [] with output_config) is accepted at any position

The second came from a compress-loop re-request ([acp-loop] upstream error 400). No open issue covered it.

Root cause

  1. "must precede" ([Bug][anthropic] 0.1.206: system 400 still fires on nudge turns: the heal keeps a trailing system, then prepare-anthropic appends user notes after it #4570). healIllegalSystemPositions (kernel/src/wire/anthropic.ts, called inside coreToAnthropic) keeps a trailing system because it is legal at the end. Then the host appends user messages after it:
    • src/server/prepare-anthropic.ts: host notes (sysNotes), nudge directives, and the retrieval note go onto rebuiltMessages after coreToAnthropic, and rebuilt is built from that array unhealed. The decision side request (sideBody) appends buildDecisionPrompt the same way.
    • src/loop/adapter-anthropic.ts buildRequest: compress-loop notes are appended after coreToAnthropic too.
  2. "must follow" (new shape). The heal only checked the successor. A fold can remove the user turn before a never-fold sidecar system while kept/protected assistant turns stay, so the system ends up right after an assistant (or at index 0). Anthropic rejects that.

Rule (checked against the live API)

A content-bearing role: "system" message must:

  • follow a user message, or an assistant message whose last block is a server tool result (*_tool_result, not tool_result). Index 0 is illegal. A directive-only system does not count as a predecessor.
  • precede an assistant message or end the array.

Consecutive systems are judged together. The directive-only form (content: [] with output_config) is legal anywhere.

A/B results: [U,S,A,U] 200, [U,A,U,S] 200, [U,S,S,A,U] 200, [U,Sdir,S,A,U] 200, [U,A,Sdir,A,U] 200. [U,A,S,A,U], [U,A,S], [U,A,Sdir,S,A,U], [U,A(tool_use),S,U(tool_result)] 400 "must follow". [S,A,U] 400 at messages.0 ("use the top-level 'system' parameter"). [U,S,U], [U,S,S,U] 400 "must precede". Re-voicing a system that sits between tool_use and its tool_result as a user message is itself a 400 (tool_use ids without tool_result immediately after), so that case must move, not demote.

Fix

  • kernel/src/wire/anthropic.ts: healIllegalSystemPositions enforces both halves. A system with an illegal predecessor moves, whole and in run order, to just after the next user turn (this keeps tool_use/tool_result adjacent and keeps tool deltas as system). If no user turn follows, it is re-voiced to user, as the [Bug][anthropic] 0.1.205: 400 "role 'system' must precede an 'assistant' message" still occurs with #4396 present (mid-conversation system + tool_addition, cf. #4400) #4457 heal already does. The successor half is unchanged (re-voice, strip tool_addition/tool_removal, drop if empty). New export healAnthropicSystemPositions(messages) returns a healed copy.
  • src/server/prepare-anthropic.ts: heal rebuiltMessages right before rebuilt, and heal the sideBody messages.
  • src/loop/adapter-anthropic.ts: heal the compress-loop request when notes are appended.
  • tests/wire-contract-fakes.ts / tests/wire-contract.test.ts: new ledger entry WC-023 (both halves), enforced in the Anthropic validation-parity fake, with a gate test and a ledger self-check probe.
  • kernel/package.json: 0.0.163 → 0.0.164.

No user-visible content is dropped. Legal layouts stay byte-identical.

Tests

  • kernel/tests/issue4570-system-position-final-wire.test.ts (7): the [Bug][anthropic] 0.1.206: system 400 still fires on nudge turns: the heal keeps a trailing system, then prepare-anthropic appends user notes after it #4570 note-after-trailing-system shape, a trailing tool-delta run before a note, a sidecar between two assistants, a system between tool_use and tool_result, a run after an assistant, trailing-after-assistant and index-0 systems, and a legal-layouts-verbatim negative. Against master source, 6 of 7 fail (the negative passes).
  • tests/issue4570-anthropic-final-wire-system-position.test.ts (3): compress-loop buildRequest with notes after a trailing system, a post-fold sidecar after an assistant, and a no-notes negative, all checked with the WC-023 validator. Against master source, 2 of 3 fail.
  • Two existing fixtures put a system at index 0, which the live API rejects. They now start with a user turn: kernel/tests/issue4292-orphan-system-guard.test.ts and kernel/tests/anthropic-system-message-passthrough.test.ts.

Local: npm run typecheck clean. npm --prefix kernel test 1480/1480. Targeted root suites (wire-contract, anthropic lanes, #4292/#4159 system tests) pass. The full-suite verdict is the CI test lane.

Mako-L and others added 2 commits October 25, 2026 15:40
…ianglei#4570)

Anthropic's mid-conversation system rule has two halves: a content-bearing
system must follow a user turn (or an assistant ending in a server tool
result) and must precede an assistant turn or end the array.

- healIllegalSystemPositions now enforces the predecessor half too: a system
  after an assistant (or at index 0) moves past the next user turn, keeping
  tool deltas, or is re-voiced to user when no user turn follows.
- New healAnthropicSystemPositions re-heals the array actually sent after
  prepare-anthropic appends host notes, nudge directives and the retrieval
  note, on the decision side request, and on compress-loop re-requests.
- WC-023 records the rule in the wire-contract ledger.
The kernel version bump in the parent commit left both package-lock.json
files at 0.0.163, so the lock-sync CI job failed and every test lane was
skipped. Regenerated with npm install --package-lock-only exactly as the
lock-sync job does; only the acp-kernel version lines change.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重 (dedup): 不重复。同家族已关闭的 #4292/#4396/#4457/#4465 修的是这一机制的 fold-summary 半边(内核内 heal 的时机与降级);本 PR 是另一半——在最终出站数组上强制完整的位置规则,包含此前从未检查过的 "must follow" 半边。按失败签名(两条不同的 400 报文)检索 open+closed issues/PRs:无在途重复。

标签: 已打 bug claude-code omp scope/kernel p1(保留 pr)。理由:触发即该会话每个请求硬 400、会话不可用,按仓库 p1 口径(会话永久不可用)定级;两条生产报错分别来自 Claude Code 与 omp 车道;修复主体在 kernel wire 层。

Review(PR #4715,head f88c1f26e,含我补的 lockfile 提交)

① 方向(原 issue)。#4570 的根因分析成立,且修复打在决策点上而不是症状上:heal 此前 (a) 只查了 Anthropic 规则的 successor 半边,(b) 在 host 追加 notes/nudge/retrieval-note 之前运行,所以真正发出去的数组仍可能非法。修法 = 对最终数组强制两半规则:kernel 导出 healAnthropicSystemPositions,三个 post-coreToAnthropic 追加点重跑它(src/server/prepare-anthropic.ts:518 decision sideBody、:577 稳态路径 rebuilt 之前、src/loop/adapter-anthropic.ts:308 compress-loop)。我把规则编码与描述里的 live A/B 证据逐条对照过(连续 system run 按最近非 system 邻居判、directive-only 豁免、index 0 拒绝、server-tool-result 前驱合法)——一致。

② 回归审查(diff 触碰路径的既有行为清点):

③ 代码审查。手工追踪了 healIllegalSystemPositions 的 move/demote/drop 全部分支,包括多 system run:moved 集合保证多个 system 跳过同一个 user 时保持 run 顺序([A,S1,S2,U,X] → [A,U,S1,S2,X]);终止性有保证(每个元素有限次处理,move 只前进、demote/drop 即消耗)。四个 host 发送路径全部审计过有无「追加后未 heal」缺口:prepare-anthropic(稳态 + decision 侧体)、adapter-anthropic(compress loop)、threads lane / incremental lane / count_tokens / plugin-render(均无 post-append,内核内 heal 足够)。无新依赖、无新配置面、无新 env var。一处非阻塞设计观察:当 predecessor 非法但紧邻后继是 assistant 时代码选择原地降级而不是后移到下一个 user 之后——降级保住了槽位但剥掉了有功能的 tool-delta 块(WC-022),后移则能让它们从稍晚处继续生效。两者都产出合法线且文本保留,#4570 第二条生产报错的形状走的正是降级路径;值得后续想一想,kernel 域本来也是人工把关。

④ 测试有效性。断言有牙:violations() oracle 独立编码了两半规则;正向行覆盖 nudge-turn 形状、post-fold 形状、run、leading/trailing;负向行钉死 6 种合法布局逐字不变——over-heal 回归会红在这里,退回 successor-only 会红在 must-follow 行。host 级测试走真实 adapter buildRequest + WC-023 validator,且带门禁负向行(无 notes 时合法尾部 system 仍是 system)。

⑤ diff 整洁度 + 顺手修掉的小问题。diff 干净,除了一处遗漏:父提交 bump 了 kernel/package.json 到 0.0.164,但两份 lockfile 留在 0.0.163 → CI lock-sync 失败、所有 test lane 被 skip。已在 PR 分支直接修掉:提交 f88c1f26e "chore: sync acp-kernel 0.0.164 into root and kernel lockfiles"(按 lock-sync job 同款命令再生成,diff 只有 3 行版本号),已推到 Mako-L:2026-10-25_system-position-heal(42c3d0e86..f88c1f2),CI 会在新 head 上重跑。PR base 落后 master 8 个提交,与 PR 触碰文件零重叠,预期无冲突。

本地验证(定向,按本地测试纪律):typecheck 干净;kernel 定向套件(issue4570、issue4292-orphan-system-guard、anthropic-system-message-passthrough、issue3410-wire-passthrough、wire-bili-message-roundtrip、issue3939、issue4457-sidecar-system-folds)全绿;root 定向(wire-contract + issue4570 host 测试)全绿。全量裁决以 f88c1f2 上的远程 CI 为准。


中文八问

  1. 是什么问题:Anthropic 拒绝两种位置的 mid-conversation system 消息;bili 的 [Bug][anthropic] 0.1.205: 400 "role 'system' must precede an 'assistant' message" still occurs with #4396 present (mid-conversation system + tool_addition, cf. #4400) #4457 heal 只查了「后面必须跟 assistant」半边,且在 bili 自己追加 user 备注之前运行,所以真正发出去的数组仍可能非法——整个请求 400,会话报废。
  2. 什么原因导致的:同一机制两个缺口——(a) heal 时机错:coreToAnthropic 内 heal 之后,host 又追加 sysNotes/nudge/retrieval note 再发送;(b) 规则不完整:从未检查「前面必须是 user 或以 server tool result 结尾的 assistant」半边(含 index 0)。
  3. 影响范围与严重程度:Anthropic 线路上重建历史中出现非法位置 system 的会话(Claude Code / omp / pi 车道都可能)。触发条件:nudge/retrieval/decision 侧调用轮次紧跟尾部 system,或长会话 fold 把 sidecar system 前的 user 轮折掉。触发即该会话每个请求硬 400,需重启/新会话恢复——按仓库口径 p1(会话不可用)。报告者 v0.1.206 一天内单会话命中两次。
  4. 怎么修复的:把完整位置规则作为 backstop 施加在最终出站数组上——kernel 导出 healAnthropicSystemPositions,三个 post-coreToAnthropic 追加点重跑;前驱非法的 system 优先整体搬到下一个 user 之后(保住 tool delta 与 tool_use/tool_result 相邻性),否则原地降级为 user(剥掉 WC-022 禁止的块,空则丢弃);directive-only 豁免;WC-023 进 ledger 并由 fake upstream 强制。
  5. 是否有回归问题:未发现。合法布局字节不变(测试钉死);既有降级语义不变;thinking 签名不受影响(只动 system 消息);passthrough 轮次仅在原本必 400 的数组上改变行为(有意变更,与 [Bug][anthropic] fold summary rendered as user right after a mid-conversation system message → 400 "role 'system' must precede an 'assistant' message" (0.1.204, since #3992) #4292/[Bug][anthropic] 0.1.205: 400 "role 'system' must precede an 'assistant' message" still occurs with #4396 present (mid-conversation system + tool_addition, cf. #4400) #4457 backstop 一致,已披露)。
  6. 是否破坏缓存:仅受影响会话在 heal 点之后的字节变化(替代方案是 400,不可避免);heal 确定性执行,客户端每轮重发自己的字节、每轮得到相同 heal 结果,其后前缀稳定;未触碰 cache key/重试/增量游标机制。
  7. 是否改了设计方向:没有——沿用 [Bug][anthropic] 0.1.205: 400 "role 'system' must precede an 'assistant' message" still occurs with #4396 present (mid-conversation system + tool_addition, cf. #4400) #4457 的 backstop 架构,扩展为完整规则并移到正确的调用点。一个非阻塞取舍已标注(后继是 assistant 时降级 vs 后移),可作 follow-up 讨论。
  8. 是否引入新配置:无。

判定:方向正确、实现与测试达标、无回归发现;等 f88c1f26e 的 CI 全绿即可合并。kernel 改动按 Kernel Boundary 归人工合并,我不代合。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

CI 跟进(新 head 状态):

@Maziar123

Copy link
Copy Markdown
Contributor

I reported #4570 and tested this PR against our failing shapes through the real request path (startServer → handle.ts → prepare-anthropic.ts → coreToAnthropic), asserting on the body a fake upstream received. Trees: this PR at 35c3fc64 and master at bcbd1a3d.

Shapes:

  • A, "must precede" (nudge turn): the history ends with a mid-conversation sidecar system, which is legal at the array end after coreToAnthropic; the host then appends its own user note after it. (The test arms the note through stableSystemAnchor, the same append site as the nudge arm, because it fires deterministically.)
  • B, "must follow" (fold orphan): a sidecar system left between two assistants after a fold covered the user turn before it.

The folded view is injected at the request boundary; the fold itself is not driven.

Shape master bcbd1a3d PR 35c3fc64
A 400: messages.3: role 'system' must precede an 'assistant' message or end the array legal
B 400: messages.2: role 'system' must follow a 'user' message or an 'assistant' message ending in a server tool result legal
legal layouts, directive-only system unchanged unchanged, same order

Your own tests on the same trees: tests/issue4570-anthropic-final-wire-system-position.test.ts 3/3 on the PR and 1/3 on master; kernel/tests/issue4570-system-position-final-wire.test.ts 7/7 on the PR (it does not load on master: no healAnthropicSystemPositions export). On the PR head, tests/wire-contract*.test.ts 28/28 and the kernel anthropic-system-message-passthrough, issue4292, issue4457, openai-system-hoist suites 34/34.

One behaviour worth a line in the description: a sidecar system carrying tool_addition/tool_removal is demoted to user and those blocks are dropped (the WC-022 trade-off your kernel test pins). Example: […, user, system{tool delta}, user(note)] reaches upstream as […, user, user(text), user(note)], and a trailing […, assistant(text), system{tool delta}] with no notes is demoted the same way (master leaves it and gets the 400). The array is legal either way, but the tool delta no longer takes effect on that turn. The local workaround we run keeps such a message as system and moves it past the following user messages (to just before the next assistant, or to the array end), which keeps the delta.

From our side this fixes #4570; we will drop our local patch once a release with it is out.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Thanks for the A/B through the real request path — asserting on the body the fake upstream actually receives is exactly the evidence bar this fix needed, and it lines up with my review read (both shapes 400 on master, legal on the PR, legal layouts byte-identical).

The demote-vs-move behaviour you flagged is intentional, and the two halves deliberately take different paths:

  • Illegal-successor cells (your Shape A — host-appended user note after a trailing sidecar system) demote IN PLACE. Two constraints force it: WC-022 forbids tool_addition/tool_removal blocks inside user messages, so any demotion must strip them anyway; and moving would shift every byte from the moved slot onward (prefix-cache churn beyond a single message flip) while also changing which assistant turn the delta applies to. In-place re-voicing touches exactly one message's bytes, cannot create a new position violation, and — decisively — the alternative for these arrays on master was a hard 400 of the whole request, so "legal wire + one lost in-history marker" beats "no wire at all". Cost, as you put it: the tool delta no longer takes effect on that turn.
  • Illegal-predecessor cells (your Shape B — fold orphan between two assistants) MOVE past the next user turn, run order preserved, deltas intact, tool_use→tool_result adjacency kept. Demotion is unsafe there by construction: a user between a tool_use and its tool_result is itself a 400 (live-verified in the A/B table in the description). Your trailing example […, assistant(text), system{tool delta}] with no notes is the predecessor-illegal case with no following user turn to move past — that's the documented fallback to demote, same strip.

Your local workaround (keep as system, move past the following users to just before the next assistant / array end) is a valid alternative default — it keeps the delta effective at the cost of moving more bytes. Flipping the default from demote-to-move is a behaviour decision (delta effectiveness vs byte-minimality / cache stability), not a defect in this fix, so I'd rather track it as an owner decision in its own issue than quietly change semantics inside this PR — say the word and I'll file it with your examples as the spec.

Paste-ready line for the description, since you asked for one:

Behaviour note: a sidecar system whose successor is illegal (e.g. a host-appended user note after a trailing system) is re-voiced to user in place; per WC-022 its tool_addition/tool_removal blocks are dropped with it, so the tool delta no longer takes effect on that turn (pinned in kernel/tests/issue4570-system-position-final-wire.test.ts). Illegal-predecessor systems instead move past the next user turn with their deltas intact, falling back to demote only when no user turn follows.

Status after your master sync: head is now 35c3fc64; base..head diff is exactly the intended 12 files (+413/-41), kernel correctly bumped to 0.0.168 (master had advanced to 0.0.167), both lockfiles in sync (the earlier lock-sync miss is gone in the new history). CI on 35c3fc64 is fully green — all 30 checks passed, 0 failed. The earlier red was inherited master noise (the storage-lane families tracked in #4599); the ci-fixes sentinel has since landed in master as well.

Verdict: review stands as posted — direction correct, no regressions found, tests have teeth. Kernel territory stays human-merge per the Kernel Boundary; ready whenever you are.


中文小结

  • 现状:作者已把 master 同步进分支,新 head 35c3fc64(kernel 0.0.168、两份 lockfile 已同步),diff 恰好是预期的 12 个文件;CI 全绿(30/30,0 failed)。此前的红是继承自 master 的存储车道红家族(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 跟踪),与 PR 代码无关。
  • 行为取舍确认:你标记的「demote 丢 tool delta」是有意设计——successor 非法的 system 原地降级为 user(WC-022 禁止 user 消息携带 tool-delta 块,降级必须剥掉,且原地翻转字节扰动最小、不会制造新的位置违规);predecessor 非法的(你的 B 形状)走移动路径、delta 完整保留,仅在后面没有 user 轮可越过时才回退到降级。你们的 workaround(移动并保留 delta)是合理的一种替代默认语义,但改默认值属于行为决策,建议另立 issue 由 owner 拍板,不在本 PR 里静默改语义——需要的话我来立项,用你的两个例子当规格。
  • 判定:可合;kernel 域按 Kernel Boundary 归人工合并,我不代合。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

这是一条重复/过期的告警,无需新动作。

它指向的是 run 40571488486 @ head 0934aa9c9 上的 lock-sync 红——那个 head 已被取代。当前 GitHub 上的 PR head 是 32d8d315b,即上一轮已推送的一行锁文件修复(kernel/package-lock.json 里 packages[""].version 0.0.168→0.0.169)。

刚在当前 head 32d8d315b 上按 CI 原样复核:根目录与 kernel/ 两处 npm install --package-lock-only + git diff --exit-code 均零漂移通过,工作树干净,没有可推的东西。该 head 的 CI 目前 0 失败(test / static-gates 分片仍在跑)。

结论:lock-sync 上一轮已闭环,这条是同一旧事件的重复转发——不重开、不重复修。完整八问见上一条 bot 回复。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) Merge-guard sync done: merged origin/master (9b6ec23) into this branch with zero conflicts, bumped acp-kernel 0.0.168 → 0.0.169 per the kernel version rule, pushed head 32d8d315b — all 30 checks green (one ubuntu-24 shard initially red on the claude-native hook e2e spawn case; reproduced locally green in <1s, single rerun of that job passed → flake, not regression).

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) Merge-guard round 4: nothing to sync — origin/master (9563c1b) is already fully contained in this branch from the previous push (verified 0 commits behind), so no new commit was made; state after last push (head f6f671f7b, kernel 0.0.170): all 14 CI workflows completed success — main CI (all test shards + gates incl. kernel-guard/lock-sync) plus every e2e lane.

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Merge-guard round 5 done: merged master b08fbf361 into Mako-L:2026-10-25_system-position-heal (no conflicts) — head is now b3f3b054d, no longer behind; GitHub reports mergeable: true, state blocked only for required reviews/checks re-running on the new head.

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Merge-guard round 6 done: synced master into the branch (it advanced twice mid-round, so both 58e61a3d5 and the current tip 09a5b5b77 are merged in), folded in a concurrent stale partial-sync commit from a parallel process, and bumped kernel 0.0.170 → 0.0.171 (this PR touches kernel/). No conflicts. Head is now ccbf6f308 — no longer behind; mergeable: true, state blocked only for required reviews/checks re-running on the new head.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

merge-guard 第 6 轮同步完成:master 本轮推进了两次,已把 58e61a3d5 与当前 tip 09a5b5b77 都合入分支;同时折叠进一个并行进程推的过期半成品同步 commit(pr4715-rebase);因本 PR 动了 kernel/,按 kernel-change-vs-base 规则把 kernel 版本 0.0.170 → 0.0.171。无冲突。分支头现为 ccbf6f308,behind 已解除——GitHub 报 mergeable=true、state=blocked(仅剩新 head 上的必备评审/检查在跑)。结果评论已贴在本 PR。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

本轮 CI 红(head ccbf6f308e,run 40910163003,3 个 test 分片)判定为继承性 master 级 flake,非本 PR 引入;已触发失败 job 重跑(HTTP 201),本 PR 侧无需改任何代码。

证据链

结构化八问

  1. 是什么问题:CI 三个 test 分片都红在同一条 subtest——「冷重启后未保留的父会话应立即服务其完整 raw tail」([Bug][fork] Cold restore cannot use an intact unified raw tail for a non-retained parent (#4266 item 4) #5140 回归钉),拿到 409 而非期望的 200。
  2. 什么原因导致的:不是本 PR 的代码缺陷。根因在 master 侧 [PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144(cold raw-tail fork)引入的冷恢复路径存在非确定性竞态——同一 master sha 既有全 fail 也有全绿 run,本地稳定复现全绿,只在 CI runner 负载下偶发 409;本 PR 只是经 round-6b sync 把这段 master 代码带了进来。
  3. 影响范围与严重程度:仅 CI/验证面,生产零影响;严重度=偶发假红,阻塞本 PR 的绿判定但不反映任何真实缺陷。
  4. 怎么修复的:本 PR 侧无可修代码(ci-fixes 亦无 delta 可合)。按仓库对已确认 flake 的标准处置,对 run 40910163003 触发 rerun-failed-jobs(HTTP 201 已接受),重跑大概率转绿;根治([PR] fix: recover non-retained parent fork snapshot from intact unified raw tail after cold restart (#5140) #5144 冷恢复竞态)归 master/ci-fixes 侧,已由哨兵接管(floor 487)。
  5. 是否有回归问题:审过——本 PR diff 不碰 fork/snapshot/persist 路径,不引入新回归;本次红本身是被继承的 flake,非本 PR 破坏任何既有行为。
  6. 是否破坏缓存:不涉及(纯测试面 flake,无缓存/重试/去重键语义变化)。
  7. 是否改了设计方向:无。
  8. 是否引入了新的配置:无。

状态:rerun 已触发、进行中;若重跑再次命中同一签名,即为 floor 486 所述「待 #5144 侧处置」的同一 master flake,届时交由哨兵/admin,不在本 PR 重复排查。收工判定:本 PR 无自身缺陷,等重跑转绿即可合并(合并不由我执行)。

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Merge-guard round 7 done: synced master dcf760753 (#5211 revert) into the branch — clean merge, no conflicts. Kernel stays at 0.0.171 (master is still 0.0.170, so no re-bump needed this round). Head is now abdd01aa5, no longer behind; mergeable: true, state blocked only for required reviews/checks re-running on the new head.

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Merge-guard round 8: no new sync needed. Master is still at dcf760753 (unchanged since round 7) and head abdd01aa5 already contains it (0 commits behind locally). Verified against the live GitHub API: mergeable: true, state blocked (not behind) — the behind flag was a stale signal. Nothing pushed this round.

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Merge-guard round 9: same state as round 8 — master is still dcf760753 (no movement since round 7), head abdd01aa5 already contains it (0 commits behind), and the live API reports mergeable: true, state blocked (not behind). This is the second consecutive behind firing with zero upstream change, so it appears to be a stale/false-positive guard signal rather than real drift. Nothing pushed this round.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

Merge-guard round 10 done — this time the behind signal was real: master advanced 14 commits since round 7 (dcf760753 → 44067a5bc). Synced and pushed; head is now ee02243cd, no longer behind.

What came in (all 14 commits = the #5188 fork/snapshot sidecar-recovery line):

Sync details:

  • Clean merge, zero conflicts (incoming touches persist/sidecar/fork-test paths; this PR touches kernel wire heal + prepare/adapter + wire-contract tests — disjoint).
  • No kernel re-bump needed: incoming diff touches nothing under kernel/; master stays at kernel 0.0.170, branch keeps 0.0.171 (> base, rule satisfied).
  • One self-inflicted hiccup, disclosed for the record: my first push went to the base repo by mistake and created a stray same-named branch there; it contained nothing beyond what the PR branch carries and was deleted immediately. The push that matters landed on the fork: Mako-L:2026-10-25_system-position-heal abdd01aa5..ee02243cd.

Live API after push: head ee02243cd, mergeable: true, state blocked — not behind anymore; blocked only for required reviews/checks re-running on the new head. Since this sync also brings in the #5188 fix for the exact flake signature that red'd the previous CI run, the rerun should come back clean.

Nothing else pending on the guard side; next firing will re-verify drift. Kernel territory stays human-merge per the Kernel Boundary.

…i#5438 two-sided heal; keep final-array backstop; kernel 0.0.181)

Conflict resolution: master shipped the sibling fix for this family (ranxianglei#5438 via PR ranxianglei#5522: two-sided heal inside coreToAnthropic, demote-only) plus the insertBeforeTrailingSystemRun helper (host wiring pending in ranxianglei#5441). Take master's approved heal semantics wholesale; keep ranxianglei#4715's residual contribution: the healAnthropicSystemPositions standalone export + the three host final-array call sites + WC-023 pins (ledger entry kept from master's side, one entry per id). Branch move-past-next-user design superseded by the owner-approved ranxianglei#5438 approach. Kernel bumped 0.0.180 -> 0.0.181 (kernel-change-vs-base rule).
@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

@Mako-L — the branch conflicted with master because master absorbed a sibling fix for this same family while this PR was in review: #5438 (owner-approved kernel authorization) shipped via PR #5522 a two-sided heal inside coreToAnthropic (demote-only semantics, follow-side legality per WC-022), plus the insertBeforeTrailingSystemRun helper whose host wiring is still open in #5441.

I resolved the conflict against current master (new head 15adc9b44, pushed to your fork):

Local verification on the merged tree: typecheck clean, kernel targeted 46/46, root targeted 26/26. CI is running on the new head.

One residual gap worth an owner decision (not fixed here): under demote-only semantics an orphaned tool-delta sidecar loses its tool_addition/tool_removal blocks when re-voiced to user (WC-022 forbids them outside system); your original move design would have preserved them. Flagged in the issue thread.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
(ai: vllm-qwen/qwen3.8-27b)

Merge-guard outcome: no action needed — the conflict was already resolved by @Mako-L before this round ran, and I verified that resolution independently rather than re-doing it. State after their push: head 15adc9b44, 0 commits behind current master (23a826d37), kernel correctly at 0.0.181 vs master 0.0.180 (net kernel change, rule satisfied), diff vs master converged to 8 files (+283/-8) with this PR's residual intact (healAnthropicSystemPositions export + both host final-array backstop call sites in prepare-anthropic.ts / adapter-anthropic.ts). Live API: mergeable: true, state blocked — not in conflict anymore, blocked only for required reviews/checks re-running on the new head. The demote-vs-move delta-loss gap they flagged is tracked as an owner decision (#5031 family), not something the guard should guess at. Nothing pending on the guard side.

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

Findings-only review of the final head 15adc9b44 (net diff vs master 23a826d37: 8 files, +283/−8). This is the first review pass on the post-conflict-resolution tree — the earlier in-thread reviews covered superseded heads that carried the move-based engine. After #5438 landed on master via #5522, the branch converged to: master's approved two-sided heal engine inside coreToAnthropic + this PR's residual = the healAnthropicSystemPositions final-array backstop (kernel export + three host call sites) + tests adapted to demote semantics. I reviewed the net diff independently.

① Direction (original issue #4570) — correct, and unchanged by the convergence. Both production 400s stem from the position rule being enforced before host appends (notes / nudges / retrieval note / decision prompt) and before fold-orphaned sidecars land on loop re-requests. Re-running the same predicate on the final array fixes it at the send point; the single source of truth stays in the kernel (the #5438 engine, already owner-approved).

② Regression audit (existing behaviour on touched paths)

  • Legal layouts: byte-identical — the heal is a no-op when both halves hold; pinned by the "legal layouts round-trip verbatim" negative gate (six layouts incl. server-tool-result predecessor, directive-only exemption, trailing run at array end) and the host no-notes negative. ✅
  • Directive-only systems: never healed, never legitimize neighbours (master contract preserved). ✅
  • Index-0 systems: untouched (deliberate #5438 contract, pinned by test). ✅
  • Thinking/signature blocks (KDD #10): the heal touches system messages only; warnAnthropicThinkingPairs matches by stable tool_use id over assistant messages, so index shifts/drops from the heal cannot misfire it. ✅
  • Prefix cache: the heal fires only on layouts that previously 400'd the whole request → zero churn on healthy sessions; deterministic healing keeps client-resent bytes stable across turns. ✅
  • Call-site coverage: all three post-conversion append sites are healed (prepare-anthropic.ts final rebuiltMessages + nudge-decision sideBody; adapter-anthropic.ts loop notes). Every other coreToAnthropic site audited: src/loop/core.ts:740 (thread body — sends the converted array verbatim), src/server.ts:3052 (count_tokens — verbatim), src/plugin-render.ts:156 (diagnostic render, not model egress), src/incremental-lane.ts:356/362 (fragment splice into client-resent bodies — separate mechanism/lane, see observation 2 below). No unhealed append remains on the full-resend path. ✅
  • Behaviour change disclosure (intended — the fix itself): illegal mid-conversation system layouts — old value: sent as-is → upstream 400 → session death; new value: healed in place (demote to user, tool_addition/tool_removal stripped per WC-022, empty entries dropped). Previously-fatal arrays only; nothing that worked changes.

③ Code — the kernel addition is a 12-line additive wrapper (copy + run the existing engine; fresh objects only, input never mutated); host changes are three one-line wraps. No new dependencies, no config surface, no env vars. Kernel Boundary discipline on the final diff: kernel-change: declaration present, version 0.0.180 → 0.0.181 (CI-enforced), kernel: commit prefix. ✅

④ Tests — real assertions with teeth: both production shapes reproduced through the real entry points (adapter buildRequest checked against the WC-023 ledger validator; kernel direct calls with an independent oracle mirror); both-direction gates present (an over-heal would red on the verbatim-legal rows; dropping the follow half would red on the must-follow rows). Two observations, neither blocking:

  1. The kernel-test oracle encodes the positional halves only, not the tool_use↔tool_result adjacency constraint — so the "a system after an assistant is re-voiced in place" row pins [A(tool_use), U(demoted), U(tool_result)], a shape the PR description itself live-verified as a 400. That shape cannot be produced by bili's own pipeline (the turn-integrity carve forbids folds splitting a tool pair; sidecars land at turn boundaries), and raw client input carrying it already 400'd on master — no practical exposure, but the oracle's boundary is worth knowing. It feeds the #5031-family discussion rather than this PR.
  2. The dsh incremental lane splices coreToAnthropic fragments into client-resent bodies; a fold-orphaned sidecar could in principle create an illegal position at a splice boundary that the final-array backstop does not see (different mechanism/lane — #4570's production hits were full-resend shapes). Awareness note only, no action requested here.

⑤ Diff cleanliness — clean: 8 files, every line serves the PR purpose (wrapper + 3 call sites + lockfile syncs + 2 test files). One process item: the PR body is stale in four places after adopting #5438 — it describes the dropped move engine, claims WC-023 as new (it shipped with #5438/#5522), cites kernel 0.0.163 → 0.0.164 instead of 0.0.180 → 0.0.181, and lists fixture edits that are not in the net diff. Recommend updating the body before merge so history matches code — author-owned, non-blocking.

结构化八问

  1. 是什么问题:Anthropic 在 mid-conversation system 消息位置非法时拒绝整个请求;bili 的 heal 跑在 coreToAnthropic 内部,即 bili 自己追加 notes/nudges/retrieval note 之前、以及 fold 孤儿 sidecar 在 loop 重请求上显形之前——nudge 轮与 compress-loop 重请求持续发出非法数组,整个会话 400。
  2. 什么原因导致的:位置规则在转换时刻执行、而非发送时刻执行;转换后追加(host notes/nudge/检索注记/decision prompt)与 fold 孤儿重新制造了内层 heal 已清掉的位置违规。
  3. 影响范围与严重程度:p1 可用性——anthropic 线路上重建历史含非法位置 system 且该轮有追加注记、或 fold 把 sidecar 前的 user 轮折掉的会话(Claude Code / omp / pi 车道;报告者 v0.1.206 单会话一天命中两次)。触发即该请求硬 400,需本地补丁/新会话恢复。
  4. 怎么修复的:owner 已批准的 #5438 两半位置 heal(现已在 coreToAnthropic 内)作为独立 pass 在最终出站数组上重跑——kernel 导出 healAnthropicSystemPositions,三个 post-conversion 追加点各调用一次。
  5. 是否有回归问题:未发现。合法布局字节不变(负向测试钉死);directive-only 与 index-0 契约保留;thinking 哨兵不受影响(按 id 匹配);无缓存影响;仅此前必 400 的数组行为改变(即修复本身)。已知已跟踪残留:demote 会丢孤儿 sidecar 的 tool delta(#5031 家族,owner 决策中)。
  6. 是否破坏缓存:否——heal 只在请求此前必死的布局上触发;健康请求字节稳定,确定性 heal 保证客户端逐轮重发得到相同结果。
  7. 是否改了设计方向:没有——规则单一事实源留在 kernel,host 只是更晚调用它。
  8. 是否引入新配置:无。

判定:最终 head 无阻塞性代码发现,CI 于 15adc9b44 全绿(镜像缓存核对:全部 executed check success,skip 均为设计内门控 e2e 车道)。因 PR 触碰 kernel/(Kernel Boundary:人工评审档,禁止 auto-merge),agent 权限下不能 approve——授权请求见下方 🔶 评论;建议合并前更新 PR 正文以匹配收敛后的设计。

@ranxianglei

Copy link
Copy Markdown
Owner

🔶 需要决策

PR #4715 (head 15adc9b44) touches kernel/ and carries a kernel-change: declaration — per the Kernel Boundary (AGENTS.md §Kernel Boundary, rules 1 & 3) all kernel changes sit at the human-only review tier; agents may not approve or self-declare. Requesting owner authorization.

Gated diff (net vs master 23a826d37):

Host-side call sites (not gated; reviewed clean): src/server/prepare-anthropic.ts (final rebuiltMessages + nudge-decision sideBody), src/loop/adapter-anthropic.ts (compress-loop notes).

Context: the #5438 heal engine itself was already owner-approved (shipped via #5522); this PR's residual kernel change is only the additive wrapper plus the version bump. Agent review conclusion: mechanism correct, call-site coverage complete, tests real, no blocking findings (full analysis in the review above). The residual demote-vs-move delta-loss semantics are tracked separately as the #5031-family owner decision.

Options: A — authorize the kernel-change declaration; the PR then awaits a human merge. B — decline; no host-only alternative exists (the heal predicate lives in kernel; duplicating it host-side would fork the single source of truth).

Recommendation: A.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b) Review lane 收工:最终 head 15adc9b44 的 findings-only review 已发布(无阻塞性代码发现,CI 全绿;因触碰 kernel/,按 Kernel Boundary agent 不能 approve),🔶 需要决策 授权请求已发(即本条上游回显)+ needs-decision 标签已打——等 owner 拍板 kernel 授权后由人工合并。建议合并前作者把 PR 正文更新到收敛后的 demote 设计(move 引擎/WC-023-new/版本号/fixture 列表四处过期)。

@Maziar123

Copy link
Copy Markdown
Contributor

I prepared an additive merge candidate for this PR against current master.

Candidate commit: Maziar123@3740f16

It resolves the conflicts in src/loop/adapter-anthropic.ts and src/server/prepare-anthropic.ts while preserving this PR's #4570 final-wire-array healing and current master's #5604 user-tail and #5688 dedupe work. Targeted verification passed: 17/17 root tests and 7/7 kernel tests, including the reported trailing-system retry case; npm run typecheck is clean. The candidate also brings the kernel package version current so the kernel guard accepts the source change.

I cannot push directly to this PR's branch, so the candidate is on my fork: https://github.com/Maziar123/billion-context/tree/local-repair/pr-4715-20261109. Please cherry-pick or use it as a reference. The remaining demote-versus-move choice for tool deltas is still the open #5031 decision.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved and pushed — I adopted your merge candidate verbatim, after verifying it independently instead of trusting it blindly.

What I did this run:

  1. Confirmed your candidate is exactly the merge this PR needed. 3740f16f's parents are 15adc9b44 (the PR branch head, untouched by anyone else — re-fetched right before push) and 406193b20 (current origin/master tip, all 452 commits advanced since round 11). So pushing it is a plain fast-forward of the PR branch — no history rewrite, no force-push.
  2. Reviewed the conflict resolutions. Master's [Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604 user-tail (ensureUserTail) and [Bug] fold reconstruction can emit two adjacent output_config effort directives → deterministic 400 on Anthropic lane (pi thinkingLevel max + OpenRouter) #5688 dedupe (dedupedMessages) rewrote exactly the regions this PR touches. Your resolution keeps the [Bug][anthropic] 0.1.206: system 400 still fires on nudge turns: the heal keeps a trailing system, then prepare-anthropic appends user notes after it #4570 backstop as the last transform on the steady-state path (prepare-anthropic.ts: appends → ensureUserTail → dedupeAdjacentOutputConfig → healAnthropicSystemPositions → rebuilt), heals the nudge-decision sideBody at its own build site, and heals the compress-loop notes append in adapter-anthropic.ts. I checked the kernel engine behind the wrapper: healIllegalSystemPositions (kernel authorization request: heal must enforce the FOLLOW side of Anthropic's mid-conversation system position rule (#5432) #5438) is demote-only (in-place re-voice or drop, no reordering) and the wrapper copies before mutating — so the new comment "never reorders, so [Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604's trailing-user tail stays last" is factually correct against the current engine.
  3. Verified locally (targeted, per local-testing discipline): lock-sync gate exact commands — root and kernel/ both zero drift; npm run typecheck clean; kernel/tests/issue4570-system-position-final-wire.test.ts 7/7; root issue4570 + wire-contract (WC-023 gates) 26/26; master's own lanes over the touched regions — issue5604, issue5688 ×2, issue4292, anthropic-lane-nudge 22/22. Total 55/55.
  4. Version discipline: master had advanced acp-kernel to 0.0.189; the branch keeps a net kernel change vs base, so the candidate correctly sits at 0.0.190 with both lockfiles regenerated (that's why lock-sync passes). bili package.json untouched (0.1.213).
  5. Pushed: Mako-L:2026-10-25_system-position-heal 15adc9b44..3740f16f (fast-forward). Net PR diff vs master is the same 8-file residual (+292/-10); the PR's two test files are byte-identical to the previous head, and kernel/src/wire/anthropic.ts is also byte-identical to the previous head — master never touched it in these 452 commits.
  6. CI state, honestly stated: the mirror cache still serves the OLD head (15adc9b44, fetched 2026-11-04): 0 failed / 11 success / 19 skipped — the skips were the dirty-merge-state skips, i.e. there was no code red, only the conflict blocker. After my push the mirror has not refreshed yet, so I can neither confirm nor deny the new head's check state from here; per policy I'm not gh-polling. origin/ci-fixes carries nothing beyond master (fully absorbed), so no sentinel merge was needed. Expect the full matrix to re-run on 3740f16f.

Still open, unchanged by this run:


中文结构化小结(八问)

  1. 是什么问题:PR 分支落后 master 452 个提交且与 master 冲突(dirty)——master 的 [Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604 user-tail 与 [Bug] fold reconstruction can emit two adjacent output_config effort directives → deterministic 400 on Anthropic lane (pi thinkingLevel max + OpenRouter) #5688 dedupe 恰好重写了本 PR 触碰的两个 host 文件区域,合并被卡、CI 全部 skip。
  2. 什么原因导致的:不是代码缺陷,是时间差——PR 在评审期间 master 持续前进(尤其 [Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604/[Bug] fold reconstruction can emit two adjacent output_config effort directives → deterministic 400 on Anthropic lane (pi thinkingLevel max + OpenRouter) #5688 两条 anthropic 出站改造),与本 PR 的最终数组 heal 调用点在同一批行上撞车。
  3. 影响范围与严重程度:仅阻塞本 PR 合并与 CI 重跑;不影响任何运行时行为或用户面。
  4. 怎么修复的:直接采用 @Maziar123 提供的合并候选提交(其两个父提交恰为「旧分支头 + 当前 master 头」,即本次该做的 merge,fast-forward 推送、不改写历史)。冲突解法保留本 PR 的 [Bug][anthropic] 0.1.206: system 400 still fires on nudge turns: the heal keeps a trailing system, then prepare-anthropic appends user notes after it #4570 backstop 为稳态路径的最后一个变换(追加 → [Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604 尾卫 → [Bug] fold reconstruction can emit two adjacent output_config effort directives → deterministic 400 on Anthropic lane (pi thinkingLevel max + OpenRouter) #5688 去重 → heal → 发送),sideBody 与 compress-loop 两个追加点各自就地 heal;kernel 版本按纪律 0.0.189→0.0.190,锁文件同步再生。
  5. 是否有回归问题:未发现。本地定向验证 55/55 全绿(kernel 7 + root 26 + master 侧 [Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604/[Bug] fold reconstruction can emit two adjacent output_config effort directives → deterministic 400 on Anthropic lane (pi thinkingLevel max + OpenRouter) #5688/[Bug][anthropic] fold summary rendered as user right after a mid-conversation system message → 400 "role 'system' must precede an 'assistant' message" (0.1.204, since #3992) #4292/nudge 车道 22);lock-sync 双端零漂移;typecheck 干净;合法布局字节不变由既有测试钉死;heal 只动非法 system、不重排、不改输入对象,[Bug] failed-compress recovery receipt (#5220/#5231) leaves the re-request ending on an assistant turn → upstream 400 (trailing whitespace / prefill not supported), subagent dies #5604 尾卫建立的尾部顺序不受影响。全量裁决以新 head 的远程 CI 为准。
  6. 是否破坏缓存:不涉及——纯合并同步,未改任何缓存/重试/游标语义;heal 对合法数组是零分配的 no-op。
  7. 是否改了设计方向:没有——维持 round 11 收敛后的设计(kernel authorization request: heal must enforce the FOLLOW side of Anthropic's mid-conversation system position rule (#5432) #5438 引擎 + 最终数组 backstop),只是把调用点适配到 master 的新变量流。
  8. 是否引入新配置:无。

收工判定:冲突已闭环并推送(3740f16f),本地证据齐;等镜像刷新后的新 head CI 结果。kernel 授权(floor 73 的 🔶)仍待 owner 拍板,kernel 域人工合并不变。

@ranxianglei
ranxianglei merged commit 4446301 into ranxianglei:master Nov 9, 2026
56 checks passed
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1134-master.4446301 (rolling master tag).

npm install -g billion-context@0.0.1134-master.4446301  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][anthropic] 0.1.206: system 400 still fires on nudge turns: the heal keeps a trailing system, then prepare-anthropic appends user notes after it

3 participants