Skip to content

fix(loop): unify Responses JSON loop trigger extraction & proxy dispatch (#1439) - #1448

Merged
ranxianglei merged 3 commits into
masterfrom
2026-09-27_unify-responses-json-loop
Sep 27, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-09-27_unify-responses-json-loop

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Unifies the non-streaming Responses JSON loop (src/compress-loop-responses.ts) with the unified src/loop/ implementation, closing the semantic drift reported in #1439. The JSON path is the only surviving "keep originals as baseline" leftover from the unified-loop migration (the OpenAI/Anthropic legacy loops are already gone); it had drifted on two axes:

  1. Text triggers — the streaming adapter (adapter-responses.ts) recognizes 4 text triggers (compress / acp_status / search_context / decompress); the JSON loop recognized only compress. Same protocol, only the stream flag differs, yet the other three triggers were dropped in non-streaming responses.
  2. Proxy dispatch — the JSON loop carried its own private executeProxyTool copy missing absorb, rules, and image_full, and used the old parseCompressInput(args) signature without callId.

Approach (Option B — surgical dedup)

Instead of routing the JSON path through the full runCompressLoop generator (the issue's suggested Option A), I removed the duplicated logic so both paths share one source of truth:

  • Shared trigger extraction — added extractResponsesTextTriggers() in src/compress-tool.ts; both adapter-responses.ts (streaming) and compress-loop-responses.ts (JSON) now call it. Any tool added later lands in both paths automatically.
  • Canonical dispatch — deleted the JSON loop's private executeProxyTool; it now delegates to the canonical executeProxyTool in loop/core.ts (now exported). Closes the absorb/rules/image_full gap and adopts the callId observability param — no per-path drift going forward.
  • buildVisibilityMarker moved into loop/core.ts; the misnamed 23-line src/compress-loop.ts (its last remaining export) is deleted and all importers repointed.

Why not Option A: the JSON orchestrator's final wire shape (one JSON body) genuinely differs from the streaming SSE generator, and folding it into runCompressLoop would be a larger wire-shape refactor (§7.4 human-review territory) with higher risk for no behavioral gain. Option B removes the actual defect — unsynced private copies, the stated root cause — at low risk while keeping each transport's thin orchestration intact. Full file deletion can follow as a separate refactor if desired.

Behavior changes (old → new)

Intentional changes to the non-streaming text-protocol path only (streaming path is byte-identical — it already used the shared extraction/dispatch):

Area Old New Why
acp_status / search_context / decompress text triggers in non-streaming text responses ignored by the extractor; the raw <acp_status>… tag was left in output.text and leaked to the client (underscore-namespaced tags are deliberately not stripped by the echo filter) recognized + executed like the streaming path; result surfaced as a read-only marker, tag stripped from the reply stream/non-stream parity (AGENTS §6). The issue called these "silently dropped", but the actual old behavior was worse — they leaked raw
absorb / rules / image_full proxy calls in the JSON path [Unknown proxy tool: …] dispatched correctly via canonical executeProxyTool closes the dispatch gap; the text-mode system prompt already advertises absorb, so the model could previously be taught to call a tool the JSON path didn't understand
parseCompressInput in the JSON path (args) (args, callId) observability parity with streaming

Untouched: config schema, persistence format/version, message-ref/id logic, src/update.ts, and the acp-kernel pin.

Regression test

New tests/fix-1439-responses-json-triggers.test.ts (3 tests) asserts the JSON path now (a) extracts all four text triggers identically to the shared extractor, and (b) executes acp_status / search_context text triggers without an upstream re-fetch, surfacing the expected marker while stripping the raw tag. Teeth verified: the test fails on clean master (pre-fix) and passes on this branch.

Pre-flight (rebased head, base = master @088bb348)

  • npm run typecheck → clean
  • npm run build → success
  • npm test → 2455 tests, 2449 pass, 2 skip, 4 fail — all 4 are pre-existing environmental failures that fail identically on clean master (live-ps host-pid walk, two claude hook-e2e cases, and the #393 claude-opus-5-thinking window case); none touch this diff's code paths. My 3 new tests are within the passing set.

Per AGENTS §7.4 this touches wire/message-shape reasoning across both compression modes, so it is flagged for human review rather than auto-merge. The gated E2E codex suite (tests/e2e/, ACP_TEST_E2E=1) exercises exactly this Codex code_mode JSON text-protocol path end-to-end if you want an extra real-client signal before merging.

Closes #1439

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-27_unify-responses-json-loop (a13c275)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-1448

Each push to this PR publishes a new version under the pr-1448 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1448.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,开始按 PR Review 规范审查本 PR(原 issue #1439 → 回归审查 → 代码/测试/diff)。先说结论框架:这是 wire/message-shape 相关改动(非流式文本协议路径),按 §7.4 属人工 review 范围。审查中,稍后给出完整结论(含既有行为清单与中文摘要)。

@ranxianglei
ranxianglei force-pushed the 2026-09-27_unify-responses-json-loop branch from f431db9 to 9a66f03 Compare September 26, 2026 17:58
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of PR #1448 (fixes #1439) — verdict: mergeable once the remaining CI checks go green

① Original-issue direction. #1439 is real and the fix hits the root cause. Evidence: I ran the new behavioral tests against an origin/master worktree — both FAIL there with exactly the "trigger executed and marker surfaced" assertion (on master the acp_status/search_context text triggers are silently dropped, never executed). The defect was drifted private copies; Option B removes the copies instead of masking the symptom — right layer. Choosing B over the issue's suggested Option A is sound: the JSON orchestrator's final wire shape (one JSON body) genuinely differs from the streaming SSE generator, so folding it into runCompressLoop would be a larger wire-shape refactor for zero behavioral gain.

② Regression audit (pre-existing behaviors on touched paths):

  • Streaming Responses trigger extraction: logic moved verbatim into extractResponsesTextTriggers (same trigger table, same indexOf loop, same call_text_… stamp format); the adapter now just delegates → streaming behavior unchanged.
  • buildVisibilityMarker: byte-identical move src/compress-loop.ts → src/loop/core.ts (diffed old file vs new function line-by-line); all importers repointed (3 adapters, 2 tests, stream.ts comment); grep confirms zero dangling references to the deleted file.
  • JSON dispatch table: canonical executeProxyTool covers all five kernel ACP names (compress/decompress/search_context/acp_status/acp_cache — same underlying handlers as the deleted private copy) plus absorb/rules/retrieve/image_full branches → strict superset, nothing dropped. Added callId param to parseCompressInput is observability logging only.
  • Call classification (PROXY_TOOL_NAMES → proxyCalls vs realCalls): unchanged by this PR.
  • Untouched: config schema, persistence format/version, message-ref/id logic, src/update.ts, acp-kernel pin, nudge cadence. Plugin mode unaffected (this path is proxy-mode response processing; in plugin mode the carrier is the agent's own tool call).

③ Code. Clean. One precision note: the PR body's "absorb/rules(image_full) … old [Unknown proxy tool: …] → new dispatched correctly" is slightly overstated — those names are NOT in the kernel ACP_TOOL_NAMES, so wire function_call items for them never reached the JSON dispatcher (classified as realCalls both before and after); the real deltas are text-trigger parity + single-source dispatch. No user-visible difference either way. Related residual from #1439's own analysis: in responses text mode the absorb system prompt is still injected (src/server.ts:3521) although only the readonly toolset is advertised — left as-is here; owner's call whether to follow up separately.

④ Test validity. The 3 new tests have teeth — A/B verified: fail on master (no marker, trigger not executed), pass on the branch (📊/🔍 marker present, fetchCalls === 0 i.e. no upstream re-fetch, raw tag stripped not leaked, prose between triggers preserved). Not tautologies. Acceptable gaps: no direct decompress-trigger-execution test in the JSON path (extractor-level coverage exists in test 1); no absorb/rules/image_full dispatch test (parity is now structural — literally the same function as streaming).

⑤ Diff cleanliness. 12 files, every line serves the dedup; the two one-line test import repoints are necessary fallout; no whitespace churn, no unrelated files.

Behavior change (intentional, disclosed): non-streaming Responses text protocol only — acp_status/search_context/decompress text triggers: old = silently dropped (stripped as noise, never executed, result invisible) → new = recognized + executed like the streaming path, result surfaced as a visibility marker, raw tag stripped. Reason: parity fix for the #1439 drift. Streaming paths stay byte-identical.

Rebase: the PR was 11 releases behind master (base 088bb348 ≈ v0.1.147 era; master is now 825f1529 = v0.1.158). Rebased cleanly (zero conflicts) and force-pushed the feature branch: f431db95 → 9a66f033. Pre-flight on the rebased head: typecheck ✅, build ✅ (dist-import-annotations: OK), full suite 2854 tests / 0 fail / 12 skip ✅. CI on 9a66f033: the e2e job (real-codex lane — exercises exactly this code_mode JSON text path) is already green, version-guard green, unit/build jobs in flight.


中文摘要:把 Responses 非流式 JSON 压缩循环里与统一循环漂移的私有实现(文本触发器识别 + 代理工具分发)收敛为单一来源,修掉 Codex 文本模式下 acp_status/search_context/decompress 等只读工具标签被静默丢弃、从不执行的缺陷;回归测试已验证在 master 上失败、修复后通过,已 rebase 到最新 master(force-push 特性分支,未动 master),CI 的 e2e codex 通道已通过——剩余 CI 检查转绿后即可合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI 全绿确认 — rebased head 9a66f033(base = 当前 master 825f1529)上全部 13 项检查通过,包括 e2e(真实 codex 走 code_mode JSON 文本协议通道)、hermetic registry e2e、四平台 unit test、build-artifact。PR 现在处于可合并状态(mergeable/clean)。

Review 结论维持上一条不变:修复打到了根因(删除漂移的私有副本而非掩盖症状),既有行为逐项审计无回归,行为变更(非流式文本协议的三个只读触发器从静默丢弃变为执行+标记+剥离标签)已按旧值→新值披露,回归测试有牙(master 上失败/分支上通过已 A/B 验证)。

合并操作留给人类:#1448

中文摘要:CI 13/13 全绿,PR #1448 可合并——合并动作按规范由人工执行。

ework-agent and others added 3 commits September 27, 2026 09:47
…triggers, canonical proxy dispatch, buildVisibilityMarker moved to core (#1439)
…ster (#1439 review)

Rebase conflict resolution: master's #1419 threaded a 5th ctx arg into
executeSearchContextTarget inside the local dispatch this PR deletes — kept
the deletion (the shared src/loop/core.ts dispatch already carries the new
arg on master). New pin: a no-trigger response returns the SAME object
reference — the JSON loop only re-serializes when a trigger fires.
@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review (rebased onto master → a13c275) — ✅ approve

What it does (verified): one shared extractResponsesTextTriggers (src/compress-tool.ts:263) for SSE + JSON paths — verbatim move of the adapter's implementation, so streaming is provably drift-free; the non-streaming JSON path previously recognized only compress (#1439: acp_status/search_context/decompress text tags leaked raw to the client in non-streaming mode). Dispatch unified into executeProxyTool (src/loop/core.ts:185) — superset of the old JSON private copy (absorb/rule/imageFull branches + callId into kernel diagnostics). buildVisibilityMarker moved verbatim to core; no dangling imports, no cycles.

Single-run semantics confirmed: pure mutating rounds consume each call exactly once; terminal mixed exits execute the leftover compress once in the downstream fallback rewriter; fetchWithRetry retries transport only. Read-only triggers never re-request (pinned by tests).

Byte fidelity: no-trigger response returns the same object reference — now pinned by a new test (unknown top-level keys survive untouched; the loop only re-serializes when a trigger fires). Pre-existing caveat (NOT this PR): the server's non-streaming branch always round-trips the response through parse/stringify even with no trigger — response-side only, semantically identical.

Rebase done (a13c275): base was 12 behind; master's #1419 threaded a 5th ctx arg into executeSearchContextTarget inside the local dispatch this PR deletes — kept the deletion (the shared core dispatch already carries the arg). Typecheck ✓ · targeted 97+4 pass / 0 fail.

Follow-up filed (pre-existing asymmetry, unchanged by this PR): in a MIXED response (text-trigger compress + real tool call), the JSON path silently drops the text-trigger compress (stripped, never executed, no marker) while the streaming loop executes it before the realCalls gate — residual JSON↔SSE drift in the spirit of #1439, worth its own fix rather than bundling here.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 独立复核了你这份 deep review,全部属实 — 逐项验证结果:

补一条行为变更披露(PR body 未列,属统一化的连带效果):JSON 路径的 search_context 旧实现是 4 参调用(无 CCR v3 的 planAware 上下文),现在经共享分发带上了 ctx —— 非流式文本协议的 search_context 结果质量对齐 streaming。old→new:无 ctx → 有 ctx;原因:去重后走同一分发函数。属本 PR 意图内的 parity 改善,非回归,但按规范显式列出。

结论维持:可合并(mergeable/clean,CI 全绿)。合并操作留给人类:#1448

中文摘要:对 deep review 的全部论断做了独立复核(rebase 距离、#1419 五参冲突处理、identity 测试、CI 13/13、#1459 立项及两侧代码的 mixed-response 不对称)均属实,另补披露一条 search_context 获得 CCR v3 上下文的连带行为变更——PR 可合并,待人工执行。

@ranxianglei
ranxianglei merged commit cae30bb into master Sep 27, 2026
13 checks passed
@ranxianglei ranxianglei mentioned this pull request Sep 27, 2026
@ranxianglei
ranxianglei deleted the 2026-09-27_unify-responses-json-loop branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant