Skip to content

[PR] fix(affinity): preserve parallel tool rounds across wire formats - #6660

Merged
ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-14_parallel-tool-affinity
Nov 14, 2026
Merged

ranxianglei merged 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-14_parallel-tool-affinity

Conversation

@Afloat16

@Afloat16 Afloat16 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #6659

Problem

Parallel tool histories do not have the same affinity identity across wires. Responses splits adjacent function_call items into separate turns when no assistant commentary precedes them. Gemini's user-parts loop overwrites earlier functionResponse bodies. A stateless client switching protocols then mints another anonymous session and loses access to the prior session's fold state.

Change

Create a pending assistant run for Responses calls regardless of commentary, and flush it at the existing tool-result boundary. Collect every Gemini result body and emit a tool turn for each in order, retaining wrapper normalization and any trailing user text. Both fixes make one parallel round project identically across OpenAI, Anthropic, Responses and Gemini; no kernel or provider-payload code changes.

Verification

Base 1ff89e878522f3e4d4147067973e2dc26e648923; Node 24.22.1 on Linux.

  • Unchanged source: six new focused regressions fail, seven controls pass. The strengthened real loopback HTTP proxy test also fails by observing two anonymous sessions on the Responses switch; its Anthropic control passes.
  • Fixed source: 84/84 tests across affinity-neutral, affinity-crosswire-fold, prefix-affinity, affinity-persist, prefix-affinity-orphan, prefix-affinity-simhash and fork-adoption.
  • The proxy test carries a real parallel tool round through OpenAI → Responses → Gemini, preserving both results, one session, existing fold blocks, the summary carrier and absence of unfolded covered text, without a new fold request.
  • Focused strict TypeScript check of the changed source and both changed test files passes. The normal npm run build and git diff --check pass.
  • Full-project typecheck and live-provider/Codex E2E were not run for this branch. Full regression remains the hosted CI verdict.

Compatibility

Ordinary single-call projections and OpenAI/Anthropic projections retain their existing behavior, covered by the controls. Active anonymous sessions containing the affected multi-call/multi-result shapes can miss once across this upgrade because their former projection omitted or split content; the client's retained transcript allows a fresh fold. Explicitly identified clients do not derive their identity from these anonymous hashes. This is the same projection-migration limitation documented in #2839.

This follows #2839's single-call implementation and does not duplicate the separate snapshot, SimHash, or core-ID reconciliation changes.

Hosted CI follow-up

CI run 43566811984 finished with 25 successful jobs, two failed Windows test jobs and one skipped job. All Linux test shards and both Windows/Ubuntu static gates passed, including the full typecheck and build. The overall CI result remains red.

The two failed jobs report only the existing SQLite teardown-lock failures:

  • Windows 22 shard 3: three persist-metadata-fold-gate cases fail while removing open index.db files (EBUSY); the runner's automatic retry repeats them.
  • Windows 22 shard 5: ten persist-zstd cases fail on the same open-file cleanup; the automatic retry repeats them.

The affected fixtures still remove their temporary directories without closing all SessionStore instances. The exact suites and cleanup cause are already covered by upstream issue #6547 and open fix PR #6561. This PR does not duplicate that work. No failure in these logs belongs to this PR's new regression cases.

@ranxianglei ranxianglei changed the title fix(affinity): preserve parallel tool rounds across wire formats [PR] fix(affinity): preserve parallel tool rounds across wire formats Nov 14, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review of PR #6660 — reviewed independently: read the diff, traced all four wire projections, ran the red-green cycle locally, and read the CI job logs directly.

查重: 不重复 — this PR is the fix for #6659 itself; #2839 is its predecessor (single-call projection), not a parallel fix. No other open PR touches src/affinity-neutral.ts.

① Direction. The bug is real and the fix hits the root cause. I reproduced both defects on the base source locally:

  • Responses: two adjacent function_call items with no assistant text between them produce two separate anonymous assistant turns — each hashes to a different identity, so the cross-wire chain forks.
  • Gemini: the user-parts loop kept a single toolBody variable, so with N functionResponse parts only the last body survived.

The fix is at the projection layer (the producer of the identity chain), not a symptom mask: Responses now merges adjacent calls into one pending assistant run (flushed at the existing function_call_output boundary), and Gemini collects every result body and emits one tool turn per body. After the fix, one parallel round projects identically on all four wires (verified: [asst(c1,c2), tool(r1), tool(r2)] on OpenAI / Anthropic / Responses / Gemini).

② Regression review. Existing behaviors preserved on the touched paths:

  • OpenAI and Anthropic projections: untouched (no diff on those branches).
  • Gemini single functionResponse (with or without trailing text): byte-identical output; wrapper normalization unchanged (verified across all 5 wrapper shapes: plain string, single-string object, multi-key object, array, null).
  • Responses single call (with or without preceding commentary, with or without trailing call): same turn sequence and same per-call fragment hashes.
  • MAX_TURN_FRAGS=64 prefix cap still applies to merged runs.

Intentional changes (all disclosed in the PR's Compatibility section):

  • Responses: N adjacent calls with no commentary: N turns → 1 merged turn.
  • Gemini: N result bodies in one user message: 1 turn (last only) → N turns.
  • Responses: call followed by an assistant message (unusual shape): 2 turns → 1 merged turn, aligning with how OpenAI renders the same shape.

Consequence: active anonymous sessions whose history contains the affected multi-call/multi-result shapes miss once across this upgrade and mint a fresh chain — the same projection-migration class #2839 documented (I checked #2839's upgrade note; the claim is accurate). Single-call projections and identified clients are unaffected.

③ Code / ⑤ diff / ⑤b cleanliness. 22-line change in one source file; minimal, correctly named, no dead code, style-consistent, no new deps, no kernel/provider-payload changes. Clean diff: 3 files, 115+/16-, every line purpose-related, single linear commit.

④ Tests. Real teeth, verified red-green:

  • 8 new unit tests with deepEqual on exact canonical output. On the base source: 6 fail (responses×2, google×2, Gemini body, Responses rounds), 2 anthropic controls pass. On the fixed source: all pass.
  • The Gemini body test pins all 5 wrapper shapes + trailing text; the Responses rounds test pins the flush boundary (2 calls + 2 outputs + trailing call → 4 turns).
  • The strengthened proxy e2e runs a real parallel round OpenAI → Responses → Gemini through a loopback HTTP proxy, asserting one session, both results present, fold carrier, no unfolded covered text, no new fold request. On the base source it fails with exactly two pfa-* sessions (the fork), as the PR claims.
  • 84/84 across the 7-file affinity area on the fixed source (verified locally).

CI. 28/30 green. The 2 red shards are 100% inherited master-level failures, not PR-caused (verified in the job logs):

Verdict: mergeable — code is correct, tests are real, no regressions; the only blocker is the inherited CI red on the fork-hosted head.


八问审核结论

  1. 是什么问题: 并行工具轮(一轮里多个 function_call / 多个 functionResponse)在 Responses 和 Gemini 两条线上投影出的身份链不一致——Responses 把相邻调用拆成多个匿名 assistant turn,Gemini 只保留最后一个结果体——导致无状态客户端切换协议时匹配不到原会话,铸新匿名会话,原会话的 fold 状态被孤儿化。
  2. 什么原因导致的: 中性投影(src/affinity-neutral.ts 的 neutralTurns())只按单调用形状实现:Responses 分支仅在已有 pending assistant run 时才合并(无 commentary 时每个调用自成一轮);Gemini user-parts 循环用单个 let toolBody 变量,后到的 functionResponse 覆盖前一个。
  3. 影响范围与严重程度: 仅影响历史中含多调用/多结果形状的匿名会话(显式识别的客户端不受影响);跨线切换(openai→responses/google)会 fork 会话、孤儿化 fold 状态、触发重新 fold。p2——能用但有明显问题(身份分裂 + 额外 fold),非完全不可用。
  4. 怎么修复的: 在投影层(身份链的生产者)修,不在消费端遮:Responses 把相邻调用恒定合并进一个 pending assistant run,在既有 function_call_output 边界冲刷;Gemini 收集全部 functionResponse 体,按序每个发一个 tool turn。修后同一并行轮在四条线上投影一致。
  5. 是否有回归问题: 无。OpenAI/Anthropic 分支未动;Gemini 单结果形状字节级一致(含 5 种 wrapper 归一);Responses 单调用形状 turn 序列与逐调用 fragment hash 不变;MAX_TURN_FRAGS=64 前缀上限仍生效。有意变更(PR 已披露):Responses N 个相邻调用 N turn→1 turn;Gemini N 个结果体 1 turn(仅末个)→N turn;call 后跟 assistant 消息的异常形状 2 turn→1 turn(与 OpenAI 渲染对齐)。后果:受影响形状的活跃匿名会话跨升级 miss 一次、铸新链——与 fix(affinity): project responses/google wire renders into the neutral identity domain (#2791) #2839 同类的投影迁移限制(已核对 fix(affinity): project responses/google wire renders into the neutral identity domain (#2791) #2839 升级说明,属实)。
  6. 是否破坏缓存: prompt cache 机制本身不受影响;匿名会话 hash 是身份键,有意投影变更使受影响匿名会话重 fold 一次(新链)——已披露,与 fix(affinity): project responses/google wire renders into the neutral identity domain (#2791) #2839 先例一致。
  7. 是否改了设计方向: 无。不动 kernel、不动 provider payload、不动配置面、不动持久化格式;纯投影层内修复。
  8. 是否引入了新的配置: 无。

结论: 可合入(代码正确、测试真实、无回归)。唯一阻塞是继承的 master 级 CI 红(Windows EBUSY 家族,ci-fixes 已修,#6645 待合);fork 分支本账号无推送权限,需作者自行 merge ci-fixes,或等 #6645 合入后 rebase。

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Duplicate relationship note (from the #6675 review thread): PR #6675 (opened 2026-11-14 08:14 UTC, ~1h42m after this PR) fixes the same issue #6659 with the same mechanism — line-by-line diff of both branches shows the src/affinity-neutral.ts code changes are semantically identical (Responses function_call items always merge into the one pending assistant run; Gemini emits every functionResponse body in order).

Per the first-come-first-served policy this PR (#6660, the earlier one) is the one to keep; #6675 is the later duplicate. The full independent review of the shared fix (red→green verification, regression sweep, 8-question checklist) is posted on #6675. Outstanding items on this PR if it is kept: rebase onto current master, labels, and the two red Windows test jobs (claimed pre-existing SQLite teardown-lock failures, not yet verified against master).

Closing/labeling decisions are owner-only — flagged there for the owner.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Noted the duplicate relationship — agree: #6660 (this PR, opened earlier) is the one to keep, #6675 is the later duplicate. I've now verified the two outstanding items that were still open on this thread, live:

1. The two red Windows jobs — verified pre-existing, and now moot after rebase.

2. Rebase onto current master — needed, clean, and verified.

  • Master has moved 17 commits past this PR's base (1ff89e8 → 9495c76).
  • None of this PR's three files (src/affinity-neutral.ts, tests/affinity-neutral.test.ts, tests/affinity-crosswire-fold.test.ts) drifted on master since the base → the rebase is conflict-free.
  • I merged the PR head onto current master in a throwaway worktree and ran the affected suites: 32/32 pass (affinity-neutral, affinity-crosswire-fold, persist-zstd). The rebased tree is green.
  • I cannot push to the Afloat16 fork (permission denied for this account), so the author needs to rebase 2026-11-14_parallel-tool-affinity onto master — after which CI should be fully green.

Labels are owner-only per the note — leaving those to you.

Verdict (unchanged): mergeable. The fix is correct, the tests are real, no regressions; the only remaining step is the author's rebase onto current master, which also resolves the two inherited red jobs.

@ranxianglei
ranxianglei merged commit d9f5f7b into ranxianglei:master Nov 14, 2026
46 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Parallel tool rounds fork anonymous sessions when switching to Responses or Gemini

2 participants