Skip to content

[PR] fix(omp): drop the stamped prompt_cache_key on a direct send (#6802) - #6803

Closed
Maziar123 wants to merge 1 commit into
ranxianglei:masterfrom
Maziar123:fix/omp-direct-send-stamped-pck
Closed

Maziar123 wants to merge 1 commit into
ranxianglei:masterfrom
Maziar123:fix/omp-direct-send-stamped-pck

Conversation

@Maziar123

Copy link
Copy Markdown
Contributor

Fixes #6802

Problem

While a proxy is claimed (BILLION_CONTEXT_PROXY), stampPromptCacheKey adds the session id as a top-level prompt_cache_key, and the proxy strips it before forwarding. When one request falls back to a direct send (proxy not ready, or respawn failed), the claim is still set, so the body is stamped but no proxy strips it. The omp lane's prepareDirectSend (createPrepareDirectSend) only restores or refuses incremental bodies, so the field reaches the upstream and a strict upstream answers 400 prompt_cache_key: Extra inputs are not permitted.

Fix

  • stampPromptCacheKey records each value it stamps (bounded set, oldest out first).
  • New createOmpDirectSendGuard() in src/agent/pi.ts: runs the shared incremental guard first (restore / refuse unchanged), then drops a top-level prompt_cache_key only when its value is one this process stamped. A key the host set itself is kept. The body is returned as restored, which directSendOrRefuse already sends through withRestoredBody.
  • src/agent/omp-native.ts uses this guard instead of the bare shared guard.

The stamp itself, the proxy path and other lanes are unchanged.

Tests run

  • New tests/omp-direct-send-stamped-pck.test.ts (4 tests): stamped key dropped; a host's own key kept; body without the key untouched; non-JSON body untouched. Against the old guard (createPrepareDirectSend) the first test fails and the other three pass; with this change all 4 pass.
  • All 43 test files that touch the stamp, the omp lane or the direct-send path, with proxy variables unset in the shell → 1047 tests, 1045 pass, 1 skipped, 1 fail. The failure (runLaunch aider: nothing declared → default hosts whitelisted for cert-MITM (#1048)) fails the same way on the base in my environment.
  • npm run typecheck → 0 errors. npm run check:unused-exports → clean. npm run check:test-import-graph → ok.

Note: four tests in native-intercept.test.ts / issue3304-incremental-direct-send.test.ts fail on both base and branch when BILLION_CONTEXT_PROXY / PI_PROXY_ANTHROPIC are set in the shell running the tests, and pass when they are unset.

Not run

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 This is the only open PR fixing #6802. Related prior work covers different mechanisms — #2583/#2589 (Cowork-transport stamp leak, fixed by withholding the stamp), #3556/#3736 (openai-wire probe-and-remember pck), #5890 (per-process pck probe) — none covers a per-request direct-send fallback while the proxy claim is still set, which is exactly what #6802 reports. The issue author already did this due diligence.

Review — verified on the branch, looks good. I checked out pr-6803 (based on current master e7a42bab, 0 behind) and confirmed:

  • Bug is real. The old guard (createPrepareDirectSend) returns as-is for a stamped body with no turn-id header, so the stamped prompt_cache_key rides to the upstream → the [Bug][omp native] 0.1.214: a degraded direct send keeps the stamped prompt_cache_key, strict upstreams answer 400 #6802 400. I reproduced this against the old guard directly.
  • Fix is correctly scoped. Only omp has stampsBodyPromptCacheKey: true (src/host-profiles.ts:94); dsh/zcode/pi/opencode are all false. So only the omp lane can leak the stamped pck, and this is the only lane changed.
  • Fix is correct. createOmpDirectSendGuard runs the shared incremental guard first (restore/refuse unchanged), then drops a top-level prompt_cache_key only when its value is one this process stamped (bounded Set, oldest-first eviction). A host-set key is kept. The cleaned body returns as restored, which directSendOrRefuse already sends via withRestoredBody and measures for the doom/byte-cap checks — so what is measured is what goes out.
  • Tests are real. All 4 pass on the branch (ran them). Test 1 (stamped key dropped) fails against the old guard — a genuine regression test. The other 3 (host key kept, no-key untouched, non-JSON untouched) pin the conservative "only drop what we stamped" behavior.
  • Clean. npm run typecheck exit 0, check:unused-exports clean, the now-unused createPrepareDirectSend import is removed from omp-native.ts (verified no other refs), no as any/@ts-ignore/console.error, comments explain the why. The stamp/proxy paths and all other lanes are byte-unchanged.

Non-blocking observations (flagging, not gating):

  • In the no-stamped-pck case the body is read twice (once in the guard to check for the key, again in directSendOrRefuse for the doom/byte-cap measure). Same pattern as the existing createPiDirectSendGuard, direct-send fallbacks are rare, and it's cheap for string bodies.
  • The 1024-entry cap means a process that stamps >1024 sessions evicts the oldest sid; a direct-send fallback for an evicted session would leak the pck again. Extremely unlikely (1024+ live sessions in one omp process). Acceptable.

CI: all checks queued as of my check — the full-suite verdict is CI's.

Behavior change (flagged per policy): on the omp direct-send fallback the outgoing body previously carried the stamped prompt_cache_key (→ 400 on strict upstreams); now it is dropped. Old → new: pck present (bug) → pck dropped (fixed). This is the intended correction — the field is bili-internal and the proxy already strips it on the normal path.


八问(供 reviewer):

  1. 是什么问题:omp native 车道在代理降级(未就绪/respawn 失败)走直发时,请求体里插件盖的 prompt_cache_key(= 会话 id)没被剥掉,严格上游(Anthropic)直接 400。
  2. 什么原因导致的:插件在 proxy claim 生效时把会话 id 盖成顶层 prompt_cache_key,正常路径由代理转发前剥除;但直发回退路径用的共享 guard(createPrepareDirectSend)只处理增量 body(还原/拒绝),不碰这个字段,盖上去的字段于是漏到上游。
  3. 影响范围与严重程度:仅 omp native 车道、且代理恰好降级(claim 仍在)的窗口内触发;触发时该回合 400 失败。p2(能用但有明显问题,需代理健康/重试绕行),非 p1(正常路径不受影响)。
  4. 怎么修复的:给 omp 车道单独的直发 guard(createOmpDirectSendGuard)——先跑共享增量 guard(还原/拒之不变),再仅当顶层 prompt_cache_key 的值是本进程盖过(有界 Set 记录)时剥除;宿主自设的 key 保留。
  5. 是否有回归问题:审过既有行为——盖戳路径、代理路径、其他车道(dsh/pi/opencode/zcode)均未改动;共享 guard 的还原/拒绝语义不变(先跑、结果被尊重)。唯一行为变化即上面 flag 的直发 body 少一个字段(本修复目标)。无其他回归。
  6. 是否破坏缓存:不破坏。代理路径(唯一涉及 bili 缓存的路径)不变;直发打到真上游、不经 bili 缓存;且只剥本进程盖的 key,宿主自设 key 保留。
  7. 是否改了设计方向:没有。在既有直发 guard 架构内加一个车道专属 guard(沿用 createPiDirectSendGuard 的既有模式),无架构转向。
  8. 是否引入了新的配置:无新 env/config/默认值。

判定:代码正确、范围精准、测试有牙、diff 干净;CI 绿后可合。属 wire/message-shape 变更(降级路径出站 body 少一个字段),按 §7.4 留人工 review 档,不建议 auto-merge。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 全绿。 全部 check 跑完:37 success + 13 skipped(gated e2e 车道,本 PR 不触及其路径,预期跳过),0 failed,无遗留。11 条 test 矩阵腿(ubuntu/windows × node 22/24 × shard)全绿,guard/lane/kernel/version/config-surface 等门禁全绿。

判定不变:代码正确、范围精准、测试有牙、diff 干净,CI 已绿 → 可合。属 wire/message-shape 变更(降级路径出站 body 少一个字段),按 §7.4 留人工 review 档,合并由人工执行。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI run 1/6 — merge-conflict resolution, done.

Rebased the PR head onto current master and force-pushed (conflict rebase, as this lane is authorized to do):

  • Old head 642124b004 (single commit on top of e7a42babec, 2026-11-15 master) → new head 480bba8d42 on top of current master 301e1a5459.
  • Exactly one conflict, in src/agent/pi.ts — a pure adjacency conflict: master's nudge-debug module wiring (nudgeDebugWired / nudgeNotifiedTurns / nudgeFullPending, from [PR] feat: pi lane — show the injected nudge text verbatim in the terminal when debug is on #6151) and the PR's new createOmpDirectSendGuard() both landed at the same spot after createPiDirectSendGuard(). Resolution = keep both blocks (union of intent; no semantic interaction — one is module-level state, the other a standalone exported function). The PR's other two files (src/agent/omp-native.ts, tests/omp-direct-send-stamped-pck.test.ts) auto-merged clean, and the rebased commit keeps the original authorship and message.
  • Final diff is byte-identical in scope to the original PR: 3 files, +128/−5. No other changes were made to the branch.

Local verification on the rebased head (targeted only — full-suite verdict is the fresh CI pass this push triggers):

  • tsc --noEmit — clean.
  • tests/omp-direct-send-stamped-pck.test.ts (the PR's regression test) — 4/4 pass.
  • Adjacent suites around the conflict zone and the guard's dependencies — 34/34 pass (omp-native, omp-native-incremental, nudge-debug, incremental-client-recovery).
  • Mirror cache for the old head showed all test lanes green, so the conflict was the only blocker; the new head carries no [ework:ci-held] marker, so this push is the branch's one full CI pass.

Live state after push (verified via GitHub API this run, not from memory): mergeable: true, mergeable_state: blocked — the conflict is gone; blocked now means the usual branch-protection gate (maintainer approval for an external-fork PR), not a merge conflict.


七问

  1. 是什么问题:PR [PR] fix(omp): drop the stamped prompt_cache_key on a direct send (#6802) #6803 与 master 产生合并冲突(merge state dirty),无法进入合并流程。
  2. 什么原因导致的:PR 基于 2026-11-15 的 master(e7a42babec)开发;之后 master 合入了 [PR] feat: pi lane — show the injected nudge text verbatim in the terminal when debug is on #6151(nudge-debug 接线)等变更,在 src/agent/pi.ts 中 createPiDirectSendGuard() 之后的同一位置新增了模块级变量,与 PR 新增的 createOmpDirectSendGuard() 函数发生相邻文本冲突。
  3. 影响范围与严重程度:仅阻塞本 PR 的合并,不影响任何已发布行为;冲突为纯位置冲突,无语义矛盾。
  4. 怎么修复的:rebase 到当前 master,冲突按「双方都保留」的并集解决(master 的 nudge-debug 状态变量在前,PR 的 omp guard 函数在后),force-push 更新 PR 分支头。
  5. 是否有回归问题:无。rebase 后本地验证:typecheck 干净;PR 回归测试 4/4 通过;冲突区与 guard 依赖的相邻套件(omp-native / omp-native-incremental / nudge-debug / incremental-client-recovery)34/34 通过;最终 diff 与 PR 原意图逐文件一致(3 文件 +128/−5),未引入任何额外改动。
  6. 是否破坏缓存:不涉及——本次只解决合并冲突,未改动任何代码语义,缓存/重试/去重逻辑零触碰。
  7. 是否改了设计方向:不涉及——未改架构、边界或数据流;最终 head 未带 [ework:ci-held] 标记,将触发本分支唯一一次完整 CI。

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI run (re-trigger) — no action needed; the blocker is already resolved.

This wake reported merge state dirty, but that is a stale reading: it was computed from the pre-rebase state. Verified live against the GitHub API in THIS run:

  • PR head: 480bba8d42 — exactly the rebased head pushed in the previous run (05:53 UTC).
  • PR base: 301e1a5459 — identical to the current master head, so the branch is at the tip of master; no new commits have landed on either side since the rebase.
  • Live merge state: mergeable: true, mergeable_state: blocked — the conflict is gone; blocked is the normal branch-protection gate (maintainer approval for an external-fork PR), not a merge conflict.

Nothing was changed in this run — no fetch-integrate, no commits, no pushes (there was nothing to integrate: the remote PR head was already the rebased head I verified). The full CI pass triggered by the previous run's final-head push is the one that will give the green/red verdict; I have not seen its result yet and will not claim it.


七问

  1. 是什么问题:系统再次报告 PR [PR] fix(omp): drop the stamped prompt_cache_key on a direct send (#6802) #6803 与 master 冲突(merge state dirty)。
  2. 什么原因导致的:触发信息是上一轮 rebase 推送落地前的陈旧快照(触发时间 05:54:48,推送完成于 05:53),并非新冲突。
  3. 影响范围与严重程度:无实际影响——本次运行经 GitHub API 实时核验:PR head 480bba8d42 即上一轮 rebase 后的头,base 与当前 master 头 301e1a5459 完全一致,mergeable=true。
  4. 怎么修复的:本轮无需修复——冲突已在上一轮解决(rebase + 保留双方代码的并集解法 + force-push),本轮只做实时状态核验,未做任何改动、未推送。
  5. 是否有回归问题:不涉及——本轮零代码改动;既有行为未触碰。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及;上一轮最终 head 未带 [ework:ci-held] 标记,该次推送触发的完整 CI 结果尚未产出,待 CI 出结果后再汇报,不预先断言绿/红。

While a proxy is claimed (BILLION_CONTEXT_PROXY), the omp plugin stamps
the session id as a top-level prompt_cache_key. The proxy strips it
before it forwards. But when one request falls back to a direct send
(the proxy is not ready, or its respawn failed), the claim is still
set, so the body was already stamped and nothing strips the field.
A strict upstream then rejects the request:

  400 prompt_cache_key: Extra inputs are not permitted

Record the values the plugin stamps, and give the omp lane its own
direct-send guard: it runs the shared incremental guard first, then
removes a top-level prompt_cache_key only when its value is one this
process stamped. A key the host set itself is left alone.
@ranxianglei
ranxianglei force-pushed the fix/omp-direct-send-stamped-pck branch from 480bba8 to 921649e Compare November 17, 2026 17:42
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI run 2 — new conflict resolved + red-CI diagnosis, done.

This wake was NOT stale — master had moved ~20h (301e1a5459 → a259e0b8fa), so the PR was genuinely dirty again. Two things were done this run:

1. Conflict rebase (again, same spot, same pattern). One conflict in src/agent/pi.ts: master's nudge-debug follow-up (#6151 strict-fidelity work, 9a5b3eaa48/8b5f9fa8cf) added nudgeNotifyText() at the exact location the PR's createOmpDirectSendGuard() occupies. Resolution = union of both, same as last run (master's function first, the PR's guard after). The other two PR files auto-merged clean. Rebased head: 480bba8d42 → 921649e5f9 on top of current master a259e0b8fa; diff scope unchanged (3 files, +128/−5), original authorship and message preserved.

2. The architecture-docs red on the previous head — diagnosed, already fixed by master. Job log (fetched once for log detail): FAIL: reference/architecture.md missing: src/responses-system-sections.ts. This is a master-side doc-drift window, not a defect in this PR: #6922 merged src/responses-system-sections.ts without its module-map line, and the follow-up 8ed56be997 docs(architecture): add src/responses-system-sections.ts to the module map (#6922) landed on master AFTER this PR's base. The rebase picks that fix up — node tools/check-architecture-docs.mjs now passes locally on the rebased head ("map covers all 325 src files, no ghosts"). No doc changes were needed in this PR branch.

Local verification on the rebased head (targeted only — the push triggers the full CI pass):

  • tsc --noEmit — clean.
  • tests/omp-direct-send-stamped-pck.test.ts + adjacent suites (omp-native, omp-native-incremental, nudge-debug, incremental-client-recovery) — 39/39 pass.
  • tools/check-architecture-docs.mjs — green.
  • Final head carries no [ework:ci-held] marker → this push is the branch's one full CI pass.

Live state after push (verified via GitHub API this run): head 921649e5f9, mergeable: true, mergeable_state: blocked — conflict gone; blocked is the maintainer-approval gate for the external fork, not a merge conflict. The new CI run's verdict is pending; I will not claim green/red until it lands.


七问

  1. 是什么问题:PR [PR] fix(omp): drop the stamped prompt_cache_key on a direct send (#6802) #6803 再次与 master 冲突(master 20 小时内前进了 20+ 个提交),且上一 head 的 CI 有一项 architecture-docs 失败。
  2. 什么原因导致的:①冲突——master 的 nudge-debug 严格保真后续提交在 src/agent/pi.ts 中 createOmpDirectSendGuard() 的同一相邻位置新增了 nudgeNotifyText(),纯位置冲突,无语义矛盾;②CI 红——[Bug] Inconsistent system prompt separator in Responses internal re-request breaks cache prefix | [Bug] Responses 内部重请求的系统提示分隔符不一致,破坏缓存前缀 #6922 合入 src/responses-system-sections.ts 时未在 reference/architecture.md 补模块映射行,该文档缺口在本 PR base 之后才由 8ed56be997 修上,属于 master 侧的临时文档漂移窗口,非本 PR 缺陷。
  3. 影响范围与严重程度:仅阻塞本 PR 合并,不影响任何已发布行为;CI 红项在 rebase 后自动消失(继承 master 的修复)。
  4. 怎么修复的:rebase 到当前 master a259e0b8fa,冲突按双方保留的并集解决;architecture-docs 无需在 PR 分支改动——rebase 已继承 master 的文档修复,本地 tools/check-architecture-docs.mjs 验证通过;force-push 更新分支头。
  5. 是否有回归问题:无。本地验证:typecheck 干净;PR 回归测试 + 相邻套件 39/39 通过;architecture-docs 检查绿;最终 diff 与 PR 原意图一致(3 文件 +128/−5),未引入额外改动。
  6. 是否破坏缓存:不涉及——只解决合并冲突,零代码语义改动,缓存/重试/去重逻辑未触碰。
  7. 是否改了设计方向:不涉及;最终 head 未带 [ework:ci-held] 标记,触发的完整 CI 结果尚未产出,待出结果后再汇报,不预先断言绿/红。

ranxianglei added a commit that referenced this pull request Nov 18, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

Your fix is merged — it landed via the wave-11 batch merge (#6827 → master a446a3c1c), which carried your commit verbatim (642124b00).

GitHub did not auto-flip this PR to Merged because the head of the branch moved after the batch snapshot (a rebase onto newer master changed the commit object ids), so the PR's current tip is not literally an ancestor of master. A git range-diff between your snapshot and the current head shows all 11 hunks unchanged, 0 changed — the content on master is byte-identical to yours, and your original commit 642124b00 is in master's history.

Closing as merged-via-batch. Thanks for the clean single-purpose fix.

@Maziar123

Copy link
Copy Markdown
Contributor Author

Thanks — confirmed: master 04ecbb42dd has the fix exactly once (642124b00 via #6827).

Side note: because the merged commit is a rebased copy, refs/pull/6803/head (921649e5) is not an ancestor of master. Merging that head onto 04ecbb42dd applies cleanly, but it adds createOmpDirectSendGuard a second time in src/agent/pi.ts (master has the function at a different spot), and the ESM build then fails with "Multiple exports with the same name". Nothing to change in master; the old head just should not be merged again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][omp native] 0.1.214: a degraded direct send keeps the stamped prompt_cache_key, strict upstreams answer 400

2 participants