Skip to content

fix(chain-echo): strip echoed bili-chain carriers from model-visible paths - #1584

Closed
ranxianglei wants to merge 3 commits into
masterfrom
2026-09-28_chain-echo-filter
Closed

ranxianglei wants to merge 3 commits into
masterfrom
2026-09-28_chain-echo-filter

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What & why

#1565: when the bare <bili-chain digest="sha256:<hex>"/> checkpoint stamp (added by #1421's cross-instance content checkpointing) leaks into model-visible context, models echo it back verbatim — visible tag chatter, polluted history, and completed turns that end up empty once the echo is stripped.

This fix makes the proxy own the echo lifecycle instead of requiring users to disable chain checkpoints (BILI_CHAIN_CONTENT=0):

  • new src/loop/chain-echo-filter.ts — streaming-tolerant echo-span machine for <bili-chain>/<bili-chain-processed> carriers: well-formed spans plus the three malformed variants observed in bug(chain-checkpoint): 裸戳记进入模型上下文后被复述,并可能造成可见污染与空完成轮 #1565 (leading space inside quotes, non-hex byte, space after =), truncated-at-EOF openings, broken-attr openings (swallowed to close), with a split-at-every-byte-position invariant (streamed deltas ≡ whole-text strip). Plus one-shot walkers for all four wire shapes, byte-exact echoed-input item removal, and post-verdict history scrubbing.
  • adapters (openai/responses/anthropic/google) — nest the filter into the existing tag/marker stream filters; report chainEcho on done.
  • loop/core.ts — an echo-only terminal turn (zero visible output, no reasoning, but a stripped carrier) now fires the existing one-shot continuation retry alongside sawThinking. Genuinely bare turns remain untouched (pinned by test E2).
  • adapter-responses — response.completed echoing the request back loses its byte-exact carrier input items (near-variants preserved verbatim).
  • server.ts — stampOutboundWithTag's result feeds chainCarrierTag into pickAdapter and the non-stream rewriters; inbound history scrub runs in each prepare lane only when the request continues into this instance's pipeline (chainSkip relays stay byte-verbatim); verdict matrix unchanged.
  • plugin.ts — both plugin pipes and the non-stream JSON pipe get the same treatment (count-only telemetry; retry latch via the existing sawStrippedEcho flag).
  • non-stream rewriters (stream.ts / stream-openai / stream-google / stream-responses) — chain-echo stripping + carrier-item removal.
  • tests — new tests/chain-echo.test.ts (unit machine incl. split property, walker parity, exact-vs-near input removal, history scrub, P0-1 integration) + degenerate-retry E1/E2 pins.
  • docs — BILI_CHAIN_CONTENT rows in CONFIGURATION.md / CONFIGURATION.zh-CN.md and the config.ts doc comment updated from stale "advisory-only since bug(chain): ACP tag literals in normal history can permanently force fresh proxy sessions into passthrough #1357" wording to current stamp/detect/enforce semantics.

Behavior changes (old → new)

Path Old New
Model prose echoing a carrier forwarded verbatim to client/history span stripped (all 4 wires, streaming + non-stream, both compression modes)
response.completed echoing request input echoed input forwarded as-is byte-exact carrier items removed; near-variants kept verbatim
Inbound assistant history containing carriers (post-verdict, continuing locally) forwarded as-is spans stripped; user text kept verbatim (count-only log); tool args/results untouched
Completed turn whose only output was a stripped carrier delivered as an empty completed turn one continuation retry via the existing #732/#821 mechanism; a second degeneration follows the normal error/warn path (no infinite retry)
Telemetry no chain-echo logs count-only, ≤1 log per category per response/request — never tag content/digest/user text

Unchanged: verdict matrix (valid / recent-mismatch / stale / invalid), carrier format & digest, parser, retry nudge text, all thresholds, tool-call argument bytes (#1039), and tag-free SSE/JSON bytes (chain probes skip response.input, so frames without echoes are never rebuilt).

Config surface

None added. The existing BILI_CHAIN_CONTENT=0 / "chainContentDetection": false remains the escape hatch and disables stamping AND detection; the docs are corrected to state that (they previously described pre-#1421 advisory-only behavior). No new env var, CLI flag, or config field.

Pre-flight (rebased onto master @ v0.1.166)

  • npm run typecheck ✅
  • npm test ✅ 3051 pass / 0 fail / 12 skip (+17 new tests vs base)
  • npm run build ✅
  • E2E tests/e2e/e2e-codex.test.ts: local zero-token preflight (E2E_CHECK=1) passes; the full run could not be executed locally because the local sglang upstream was down at commit time (connection refused on the e2e port). This diff touches request-pipeline hot files, so CI's e2e job will run it automatically.

Fixes #1565

…paths (#1565)

A bare <bili-chain digest="sha256:<hex>"/> checkpoint stamp leaking into model-visible context gets echoed back verbatim by models: visible tag chatter, polluted history, and completed turns that end up empty once the echo is removed. Root fixes, all gated behind the existing BILI_CHAIN_CONTENT switch (no new config surface):

- src/loop/chain-echo-filter.ts (new): streaming-tolerant echo-span machine for <bili-chain>/<bili-chain-processed> carriers (well-formed plus the three observed malformed variants, truncated-at-EOF, broken-attrs swallow, byte-split invariant), one-shot walkers for all four wire shapes, byte-exact echoed-input item removal, and post-verdict history scrubbing (assistant prose stripped, user text kept+counted, tool args/results untouched).
- adapters (openai/responses/anthropic/google): nest the filter into the existing tag/marker stream filters; report chainEcho on done.
- loop/core.ts: an echo-only terminal turn (zero visible output, no reasoning, but a stripped carrier) fires the existing one-shot continuation retry alongside sawThinking; genuinely bare turns remain untouched.
- adapter-responses: response.completed echoing the request back loses its byte-exact carrier input items (near-variants preserved).
- server.ts: the stampOutboundWithTag result feeds chainCarrierTag into pickAdapter and the non-stream rewriters; inbound history scrub runs in each prepare lane only when the request continues into this instance's pipeline (chainSkip relays stay byte-verbatim); verdict matrix unchanged.
- plugin.ts: both plugin pipes and the non-stream JSON pipe get the same treatment (count-only telemetry; retry latch via the existing sawStrippedEcho flag).
- non-stream rewriters (stream.ts / stream-openai / stream-google / stream-responses): chain-echo stripping + carrier-item removal.
- tests: tests/chain-echo.test.ts (new: unit machine incl. split-at-every-byte-position property, walker parity, exact-vs-near input removal, history scrub, P0-1 integration) and degenerate-retry E1/E2 pins.

Fixes #1565
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-28_chain-echo-filter (5e805e6)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1515-pr.1584.7813706

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1584.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Picking up the review of PR #1584 (2026-09-28_chain-echo-filter). Will work through the standard gates: original issue #1565 direction check → regression inventory → code review → test validity → diff cleanliness. First pass findings coming.

ework-agent added 2 commits September 28, 2026 19:53
pipePluginResponsesWithStrip gained a chainCarrier param for P0-1
(response.input echo removal), but none of its three call sites in
forward() passed it, so the removal was dead code on the plugin-mode
streaming path. Wire chainCarrierTag at all three sites.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Review complete (supersedes the earlier ack). Full verdict below.

① Direction gate (原 issue): PASS. The PR implements #1565's prescribed plan point-for-point: P0-1 byte-exact carrier removal from echoed response.input (no global replace), P0-2 bounded streaming state machine across all four wires + terminal lifecycle events + non-stream JSON, P0-3 one-shot continuation retry for echo-only completed turns, P1 post-verdict inbound history scrub (user slots kept verbatim, assistant stripped, messages never deleted) + count-only telemetry. Non-goals respected: no #1397 parser/digest/carrier-format changes, verdict matrix untouched, no new config surface (config.ts is a doc-comment fix; CONFIGURATION.md zh/en corrects the stale "advisory-only since #1357" line — accurate, verified).

② Regression audit — existing behaviors carried by the touched paths, each checked against pre-change code:

③ Code review — found and fixed directly (pushed to the PR branch):

  1. P0-1 dead wiring. pipePluginResponsesWithStrip gained the chainCarrier param, but none of its three call sites in forward() (src/server.ts:5609, 5729, 5769) passed it — so the response.input echo removal never ran on the plugin-mode streaming path, where outbound stamping still happens (the stamp gate doesn't exclude plugin sessions). The bug(chain-checkpoint): 裸戳记进入模型上下文后被复述,并可能造成可见污染与空完成轮 #1565 exposure existed there too. Fixed: chainCarrierTag ?? undefined appended at all three sites (commit 134d6d27).
  2. Off-by-one indents: src/loop/core.ts:632–636 (retry-block comment/if/first conditions at 17/21 spaces instead of 16/20) and src/loop/adapter-anthropic.ts:498 (done-yield at 16 spaces inside a 20-space block) (commit e7f68ff4).

Flagged, not fixed (author/owner call):
3. pipePluginJson gap (src/plugin.ts:2517): non-stream plugin JSON responses strip render-tags/markers but not chain echoes — all four sibling non-stream proxy lanes (rewrite*JsonResponse) were extended, this one wasn't. Same root cause as #1565 P0-2 ("non-stream JSON across all wires"). Fix is small: add containsChainEchoText to the probe + apply the per-wire chain strippers (Google needs per-part handling à la stream-google.ts). Fold into this PR or follow-up — your call.
4. Split-invariant edge (verified empirically): for malformed openings whose non-angle attr run exceeds 587 chars (past CHAIN_TAG_OPEN_CAP=600), whole-text keeps the tag verbatim while streaming drops the prefix and leaks tail residue (603-char case: whole = unchanged, 1-char chunks → "x>"). Real carriers are ≤534 chars, so real traffic is unaffected — but the docstring claim at src/loop/chain-echo-filter.ts:212–214 ("guaranteed identical") is strictly false. Suggest correcting the docstring or pinning current over-long behavior in a test.
5. Minor telemetry nit: the extended non-stream warn logs (stream.ts / stream-openai.ts / stream-google.ts / stream-responses.ts) print up to 120 chars of matched text when only a chain carrier matches; #1565 P1 asks for count-only on the new category. Low sensitivity (the digest rides every stamped egress to the provider anyway, and this extends the pre-existing approved preview convention) — left as-is; trivial per-site change if you want strict compliance.

④ Test validity: strong. tests/chain-echo.test.ts: split-property test across every split point of realistic traffic; the three malformed variants observed in #1565; #1039 byte-exact tool-arg pins per wire; near-variant preservation proving byte-exact-only removal; P0-1 exercised end-to-end through runCompressLoop + createResponsesAdapter with the single-completion-frame assertion. tests/loop-degenerate-retry.test.ts: E1 (echo-only → exactly one retry, nudge present, content delivered) and E2 (genuinely bare → no retry). Known gap: no test drives pipePluginResponsesWithStrip with the carrier param — my wiring commit activates already-tested logic, and server-level integration is the e2e-codex suite's job, which re-runs on the new head.

⑤ Diff cleanliness: tight after the two nits — 19 files, every line serves #1565; no whitespace churn, no unrelated files, no lockfile drift.

Verification: CI was fully green on 5e805e6e (13/13 checks incl. e2e + hermetic registry). Local re-run on the updated head e7f68ff4: typecheck ✅, 3051 pass / 0 fail / 12 skip (identical to baseline) ✅, build + dist-import-annotations ✅. CI re-triggers on the two pushed commits.

Verdict: mergeable once CI is green on the updated head — the pushed commits are +3 lines of wiring (activating tested logic) plus whitespace. Item 3 (pipePluginJson) is the only substantive open question; decide fold-in vs follow-up around merge time.

中文摘要:审查了 #1584(剥离模型回显的 bili-chain 载体):实现方向与 #1565 方案完全一致,回归面逐项清点无意外;我已在 PR 分支直接补两个 commit——把 P0-1 的 carrier 参数接到三个 plugin 管道调用点(此前是死代码)+ 修正两处缩进错位;另指出两点待你定夺:pipePluginJson(非流式插件 JSON)漏了 chain 剥离,以及超长(>534 字符)畸形开标签下流式与整文处理不一致(不影响真实载体)。CI 全绿、本地复验通过,可以合并。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(chain-checkpoint): 裸戳记进入模型上下文后被复述,并可能造成可见污染与空完成轮

1 participant