Skip to content

[PR] fix: Antigravity Gemini-4 1M context window + v1internal continuation-leg replay - #6969

Open
ranxianglei wants to merge 8 commits into
masterfrom
2026-11-17_gemini4-window-continuation-replay
Open

ranxianglei wants to merge 8 commits into
masterfrom
2026-11-17_gemini4-window-continuation-replay

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Closes #6963

What

Two interacting bugs broke Antigravity (CloudCode) long-output turns with HTTP 400:

  1. Missing 1M context window — gemini-4-* and the role-suffixed CloudCode ids (gemini-pro-orchestrator(-argon), gemini-pro-agent, gemini-flash-worker-agy, gemini-pro-latest, gemini-flash-latest, gemini-flash-lite-latest) matched no CONTEXT_LIMIT_TABLE entry → anonymous 200K default → preflight 502 at ~227K input and clampOutgoingOutput clamping 262144→4096 at ~168K (the issue's output budget clamped 262144 -> 4096 … window=200000 log line).
  2. continuationToken payload mutation — the v1internal continuation leg re-ran the full stateful pipeline (nudge / status line / ACP tag re-render / output clamp), mutating the outbound body vs the minting leg (issue logs: leg1 764.9KiB → leg2 779.9KiB). Upstream binds the token to the exact bytes it saw on that leg and 400s any deviation ("does not match the continuation token").

How

  • 1A: add gemini-4* and gemini-(pro|flash)-(orchestrator|worker|agent|latest|lite)* to the table at 1_048_576 (before the gemini-3 row). Table philosophy: a too-large guess self-heals on the first upstream overflow ([Question] deepseek-flash 被静态兜底表解析为 128k,真实窗口 1M —— 是有意设置,还是兜底表过期? #852); a too-small one strands the session (bili出现连续压缩情况 #282).
  • 1B: every v1 egress records its inner on session.transient.v1ContinuationReplay (in-memory only, latest-wins, countTokens egress excluded — it never mints continuation tokens). A continuation leg (token found at envelope meta / inner root / generationConfig, camelCase or snake_case) replays the cached inner byte-identical, splicing back only the client's fresh maxOutputTokens (accumulated across legs — upstream tells the client to pass a higher value) and the token itself; a stale token from a previous leg is stripped from every location first. The pipeline is skipped entirely; the relay re-wraps with the incoming leg's own envelope meta (fresh requestId). Cache miss (restart / orphan token) falls through to the full pipeline with a warn log — pre-fix behavior.

Behavior changes

  • New: continuation legs on v1internal (Antigravity) sessions replay the minting leg's egress bytes instead of re-running the pipeline. Previously the leg was re-pipelined and upstream 400'd (the bug). Non-v1internal traffic is untouched (the gate requires v1Meta !== undefined); non-continuation v1internal legs are untouched (no token → no replay).
  • New: session.transient.v1ContinuationReplay — in-memory only, excluded from the persistence projection, ~1MB bounded by the session's own egress size.
  • Changed: lookupContextLimit returns 1_048_576 for the Antigravity Gemini line (was: 200K anonymous default). This moves the preflight wall and the output clamp for those models from ~200K to ~1M — intended; per-model route config still overrides.

Tests

  • tests/issue6963-v1internal-continuation.test.ts (new): hermetic splice-helper tests + 5 e2e lanes through the real proxy (byte-identical replay, orphan-token fallback, countTokens no-clobber, inner-root/meta token locations, enveloped response). Test A fails on the pre-fix code with 5948 !== 8192 (the clamp mutating the leg) — verified by reverting the handle.ts hunk and re-running.
  • tests/routing.test.ts: new row pinning the Antigravity line at 1M + controls for the pre-existing Gemini entries.
  • Targeted suites green locally: issue3129-v1internal, issue3280-v1internal-loop-rounds, v1internal-multiline-sse, adapter-google, google-*, preflight-max-output-tokens, preflight-window-clamp, routing (124 tests). Full suite = CI.

Accepted boundary

A continuation leg whose model output emits a compress tool call would enter the compress loop with empty processedMessages (the replay Prepared ships minimal message arrays — both empty, the same shape the countTokens lane's fallback path ships through the same relay). Pathological: the model is mid-prose on a continuation leg, and pre-fix that scenario 400'd before any response was read.

ci-check and others added 5 commits November 16, 2026 20:42
The probe anchor PROXY_TOOL_AT=6 was calibrated on 2026-10-31 for #4521
(schedule {4,6,8,9}). The nudge scheduling has shifted again — nudge now
lands at {3,5,11,14}. Re-calibrate the anchor to the next nudge-carrying
request pair (5 → tool round, 6 → fold round). The fold/proxy-tool-round
mechanics under test are unchanged.

Closes: master-red on test (windows-latest, 22, 3) and (ubuntu-latest, 22, 1).
…1M (#6963)

gemini-4-{carbon,argon,hydrogen}-{low,medium,high,xhigh} and the
role-suffixed CloudCode ids (gemini-pro-orchestrator(-argon),
gemini-pro-agent, gemini-flash-worker-agy, gemini-pro-latest,
gemini-flash-latest, gemini-flash-lite-latest) matched no
CONTEXT_LIMIT_TABLE entry and fell to the anonymous 200K default:
preflight 502 at ~227K input and clampOutgoingOutput clamping
262144 -> 4096 at ~168K. A too-large guess self-heals on the first
upstream overflow (#852); a too-small one strands the session (#282).
… legs (#6963)

A v1internal continuation leg (continuationToken present) re-ran the
full stateful pipeline — nudge / status line / ACP tag re-render /
output clamp — mutating the outbound body vs the minting leg. Upstream
binds the token to the exact bytes it saw on that leg and 400s any
deviation ('does not match the continuation token').

Every v1 egress now records its inner on the session (in-memory
transient, latest-wins, countTokens egress excluded — it never mints
continuation tokens). A continuation leg replays the cached inner
byte-identical, splicing back only the client's fresh maxOutputTokens
(accumulated across legs — upstream tells the client to pass a higher
value) and the token itself, at whichever of meta / inner root /
generationConfig (camelCase or snake_case) it arrived in; a stale token
from a previous leg is stripped from every location first. The pipeline
is skipped entirely; the relay re-wraps with the incoming leg's own
envelope meta. Cache miss (restart / orphan token) falls through to the
full pipeline — pre-fix behavior, with a warn log.
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-17_gemini4-window-continuation-replay (bcd7251)

Latest build — install this exact version

npm install -g billion-context@0.0.10482-pr.6969.ea6ff1e

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
bcd7251 billion-context@0.0.10482-pr.6969.ea6ff1e build
114ea7a billion-context@0.0.10467-pr.6969.f3f1b96 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6969.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: Antigravity Gemini-4 1M context window + v1internal continuation-leg replay [PR] fix: Antigravity Gemini-4 1M context window + v1internal continuation-leg replay Nov 16, 2026
ework-agent and others added 3 commits November 17, 2026 01:36
The probe's hardcoded request-index anchors (PROXY_TOOL_AT/FOLD_AT) broke
every time the nudge gate's schedule shifted. The last recalibration
(245ecb1) was measured on a direct local run (timestamps ON by default
since #4313), but CI runs the suite preload pin BILI_MESSAGE_TIMESTAMPS="0"
(timestamps OFF) — a different wire shape, a different schedule (no nudge at
index 5) — so it passed locally and failed on every CI shard.

Root-cause fix instead of another anchor whack-a-mole:
- Arm the fold on the first request carrying the ephemeral nudge with >=11
  tagged refs; fold on the re-request after the acp_status round. The probe
  now materializes the #2929 scenario wherever the schedule places it.
- Pin compress.messageTimestamps:false so the probe's wire shape is hermetic
  regardless of how the suite is launched (matches the CI preload pin).

Supersedes the 245ecb1 recalibration. Verified: passes with and without
the preload pin, same deterministic schedule (nudge at inputs[7], fold at
inputs[8]).
The type is only referenced inside src/v1internal.ts itself; the gate
flags it INTERNAL_ONLY. No dts surface is emitted (tsup config has no
dts), so the non-exported type in the exported signatures is safe.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: opencode/qwen3.8-27b)

查重: 不重复 — scanned open [PR] rows: no other open PR fixes #6963. Nearest neighbor #6967 (401 re-request on v1internal) is a different issue and a different mechanism (re-request auth, not continuation-token byte binding).

Review of PR #6969 — head bcd7251a (8 commits, 9 files +632/-29), Closes #6963

Direction (① original issue): matches. Issue #6963 = two interacting bugs: (1A) the Antigravity Gemini line (gemini-4-*, role-suffixed CloudCode ids) matches no CONTEXT_LIMIT_TABLE row → anonymous 200K default → preflight 502 at ~227K input + output clamp 262144→4096 at ~168K; (1B) the continuation leg re-ran the full stateful pipeline (nudge / status line / ACP tag re-render / clamp), mutating the outbound body vs the minting leg (764.9KiB → 779.9KiB) — upstream binds the continuationToken to the exact bytes of that leg and 400s any deviation. The PR fixes both at the root: table rows at 1_048_576 (too-large self-heals on first overflow #852; too-small strands #282), and byte-identical replay of the cached egress inner at the pipeline entry (src/server/handle.ts:3030-3075) — a decision-point fix, not a payload filter in transit. The splice touches only the token (stripped from every location, re-inserted at the client's location) and the client's fresh maxOutputTokens.

Regression (②), verified against pre-change code, not just the diff:

  • Non-v1internal traffic untouched (gate requires v1Meta !== undefined); non-continuation v1internal legs untouched (no token → no replay); countTokens egress excluded from the cache (it never mints tokens); cache miss (restart / orphan token) → full pipeline + warn = pre-fix behavior.
  • session.transient is excluded from the persistence projection; ~1MB bounded by the session's own egress size (per-session cap 4 GiB, src/knobs.ts:515-535).
  • Every relay consumer of the replay Prepared's empty message arrays audited individually: fuse (count gate 0 > max false, measuredOutbound prices the actual body string, byte floor 0), createHandoffReply (session+stream only), all prepared.* accesses guarded, response-side v1UnwrapStream intact, replay runs under withSessionLock like the normal path.
  • Seam forensics: noteInboundMessages(session, []) → firstDivergentInboundMsg returns null for an empty prev → NO false [Enhancement][cache-ledger] attribute client-side mid-history insertions to a client bucket — seam forensics currently blame bili for host churn (found in #2997) #3023 client attribution; worst case is one advisory seam-suspect event if a continuation-leg sample shows an unexplained miss (token bytes at the tail look like a break) — forensics only, non-blocking.
  • Minor, non-blocking: INCOMING dump skipped on replay legs (small observability gap in the acp-cache diff pairing); noteForwardedImageFacts textSide imprecise for replay legs with images (learning stat only); model field ordering v1Model ?? googleModel vs the normal path's googleModel ?? v1Model — same resolution on the v1internal lane (the model rides the envelope meta).
  • Egress recording covers all v1 egress sites (initial send, retry rungs via finalizeEgress, loop re-sends via wireTransform); runEgress user transforms run before the record point, so the cache holds exactly the bytes upstream saw.

Code (③): clean. Splice helper is pure, narrow, well-named; the gate is four explicit conditions; cached-body parse failure degrades to the full pipeline with a warn (no silent swallow). No dead code, no magic numbers, style matches the repo.

Tests (④): hermetic helper tests pin precedence (meta > inner > generationConfig), both spellings, non-string/empty rejection, and splice semantics (token + budget only; contents/tools/sessionId preserved; no cross-location leak). 5 e2e lanes through the real proxy: byte-identical replay, orphan-token fallback, countTokens no-clobber, inner-root/meta token locations, enveloped SSE response. I reproduced the pre-fix failure myself: checking out the merge-base handle.ts makes Test A fail with AssertionError: expected 8192, actual 5948 — exactly the clamp mutation the issue reports. One weakness (minor): Test A asserts structural deepEqual, not byte equality — byte-identity rests on V8 JSON.stringify insertion-order stability (holds in practice; the mechanism is pinned structurally).

Diff cleanliness (⑤): the PR-specific diff is tight — all 9 files relate to the fix. The three-dot diff also carries the #2929 test recalibration, which is the sanctioned origin/ci-fixes merge per the CI-coordination policy (2026-10-24), not bundled content.

Base freshness: the PR was 53 commits behind master; the author merged master + ci-fixes into the branch (1a4450b, bcd7251). PR hunks do not overlap master churn in the same files. Verified on the current head: npm run typecheck clean; targeted suites green — 37 PR tests (issue6963 + routing) + 83 related (issue3129/3280-v1internal, v1internal-multiline-sse, adapter-google, google-*, preflight-max-output-tokens, preflight-window-clamp, issue2929).

Labels: bug, p2, scope/wiring, antigravity — present and appropriate (p2: narrow client line, Antigravity long-output turns 400; not a data-loss/cache-leak class, so not p1 per the 2026-10-19 revision).

CI: in progress on current head bcd7251a (mirror cache 2026-11-16T18:11Z: 0 failed / 11 pending / 6 success / 13 skipped). Cannot claim green yet — mergeable state is blocked on pending checks, not a conflict.

One PR-body inaccuracy (minor, no code change needed): the "Accepted boundary" section says a model-emitted compress tool call on a continuation leg "would enter the compress loop with empty processedMessages" — actually the useRewriter gate (src/server/relay.ts:2515) is compressInjected && processedMessages.length > 0, and the replay Prepared ships compressInjected: false, so the replay leg never enters the compress loop; such a tool call would relay verbatim to the client. The conclusion (pathological, accepted) is unchanged.


8问(结构化结论)

  1. 是什么问题:Antigravity (CloudCode) 长输出轮次被两个叠加 bug 打成 HTTP 400 —— Gemini-4/角色后缀模型线落到匿名 200K 窗口(~227K 输入触发 preflight 502、~168K 触发输出预算被钳到 4096),且 continuation leg 重跑有状态管线导致出站字节与铸造 token 的那一轮不一致,上游按字节绑定 token 直接 400。
  2. 什么原因导致的:(1A) CONTEXT_LIMIT_TABLE 没有 gemini-4* / CloudCode 角色后缀 id 的行 → 落匿名 200K 默认;(1B) continuation leg 走完整有状态管线(nudge/status line/ACP tag 重渲染/输出钳制),而 token 绑定铸造轮的精确字节,任何状态化重渲染必然漂移。
  3. 影响范围与严重程度:只影响 Antigravity (CloudCode) v1internal 车道;触发条件 = 长输出轮次(输入 >~168K 钳预算、>~227K 撞 preflight 墙)+ 多腿 continuation。该场景下轮次完全不可用(400/502),但面窄(单一客户端线)、非数据丢失/缓存泄露类 → p2。
  4. 怎么修复的:(1A) 表内按 1_048_576 补行(偏大猜值首次溢出自愈 [Question] deepseek-flash 被静态兜底表解析为 128k,真实窗口 1M —— 是有意设置,还是兜底表过期? #852,偏小猜值卡死会话 bili出现连续压缩情况 #282);(1B) 每个 v1 出站把 inner 缓存在 session.transient.v1ContinuationReplay(仅内存、latest-wins、countTokens 出站除外);continuation leg(token 在 envelope meta / inner 根 / generationConfig 任一处、camelCase 或 snake_case)整轮回放缓存 inner 字节,只回拼客户端新的 maxOutputTokens 和 token 本身,管线整体跳过;缓存未命中(重启/孤儿 token)降级回修复前的完整管线 + warn。
  5. 是否有回归问题:未发现。非 v1internal 流量不碰(gate 要求 v1Meta !== undefined);非 continuation 的 v1internal 腿不碰(无 token 不回放);countTokens 出站不缓存;缓存未命中 = 修复前行为;session.transient 不进持久化投影;空消息数组的 relay 各消费方逐一核过(fuse/handoff/prepared.*/响应解包/锁);seam 取证不会产生误判客户端归因(最坏一条 advisory 事件)。有意行为变更(PR 已披露):Antigravity 线的 preflight 墙与输出钳制从 ~200K 移到 ~1M;per-model 路由配置仍可覆盖。
  6. 是否破坏缓存:否。前缀缓存面未变;回放腿发的是与铸造轮相同的字节(这正是维持上游 continuation 绑定所需的);唯一新增状态是每会话 ~1MB 的内存 latest-wins 缓冲,不落盘。
  7. 是否改了设计方向:无架构转向 —— 在既有管线入口(handle.ts)加一个窄 gate + v1internal.ts 纯函数 splice + 一个 transient 会话字段。
  8. 是否引入了新的配置:无。没有新 env / config key / CLI flag。

判定:APPROVE —— 无阻断项;CI 在 bcd7251a 上跑完转绿即可合并。本地证据:typecheck 干净、120 个定向测试在当前 head 全绿、修复前复现确认(5948 !== 8192,与 issue 日志一致)。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[v1internal] Missing 1M context window for Antigravity/Gemini-4 models + continuationToken payload mutation causes HTTP 400

1 participant