Repository navigation
fix: strip refs-only residue adjacent to render tags in the echo filter (#2023) - #2025
Conversation
…er (#2023) The filter only treated a body of EXACTLY ONE bare ref as tag content (#1720 REF_LIKE), so refs-only text adjacent to render-tag structure was released as prose: orphan ref+close units, multi-ref/range pair bodies, EOF tails after an opening, and chunk-boundary splits all leaked mNNNNN remnants to the client reasoning stream. - Generalize single-ref to refs-run (REFS_RUN/REFS_ONLY: lists, en/em-dash ranges, comma-separated range lists) at every decision point: PAIRED body, swallow inner decision, over-budget release, flush EOF branch, PARTIAL_TAIL hold (incl. run+tag-fragment composite and bare-m/partial ref prefixes so any split position holds toward the unit). - New orphan unit: a refs run directly against an UNMATCHED close is dead markup and is stripped with the close. Whole-text applies it only to depth-zero closes (orphanCloseSpans greedy pairing); streaming only when no opening is live — so a ref cited inside a prose-wearing pair survives (#1720) in both modes. - The [\w>] lookbehind extends across chunk boundaries via lastEmitted: word-like tokens (xm01233) and tag-terminator residue (#644 over-cap drop) are never orphan-stripped or held. Tests: tests/issue2023-refs-residue.test.ts pins every shape whole-text AND streaming (char-by-char equivalence + explicit chunk splits), plus #1720 protection pins, #644 lossless pins, and documented residual pins.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.2059-pr.2025.22a1b90This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr2025.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…ounting (#2023) Review finding on the #2025 head: the three new PARTIAL_TAIL refs-run alternatives flow into mayStartRenderTag, which isOrphanMarkupText used as its render-tag probe. A released tail shaped like a refs run (a genuine citation held to EOF) was therefore counted as dead markup residue, so a tool-call-free turn whose only visible output was a bare ref citation read as degenerate and burned the one-shot retry (#732/#821) — and could end in the #870 in-band error instead of delivering the citation. isOrphanMarkupText now probes with TAG_PARTIAL_TAIL (PARTIAL_TAIL's \x3c-prefixed alternatives only): byte-identical classification to pre-#2023 for every tag-shaped tail, refs-run tails flip to prose. Tagged echoes need no help from this path — their drop already sets sawStrippedEcho upstream. mayStartRenderTag stays broad (fast-path gating must keep engaging on refs-run tails so split units are still stripped). Pinned: unit classification + a full openai-lane pipe test proving a bare-citation answer reaches the client with zero refetches; both fail on the previous head. #870 unclosed-tag retry pin stays green.
[bot] 🏷 ## PR review — 结论:发现并修复了一处未披露的跨模块语义副作用,已补提交到本分支,CI 绿后可合 做了什么:拉取分支核对代码(未重做)。逐条验证了 diff 触碰路径的全部既有行为、流式状态机各决策点、whole-text/streaming 等价性,审了测试有效性。审查中发现一处未列在 PR 行为变更表里的回归,按「小问题直接修」政策已在 PR 分支补一个 additive commit: 发现的问题(审查新增)
修法:把两个探针解耦—— 结构化七问
测试与 diff 质量新套件(tests/issue2023-refs-residue.test.ts)是真测试:字节精确断言、全形状 char-by-char streaming≡whole-text 等价性 harness、显式分块切分、#1720/#644 保护钉、文档化残留在册钉,无恒真断言。diff 干净(模块 + 其测试文件,我补的 commit 同样收敛到 2 文件)。 状态与判定
|
[bot] 🏷 CI 全绿确认:head 更正上一条中的一处状态:等待期间 master 前进了 8 个提交(合入的 #2019 provider protocol field 与 #2020 opencode-v2 routed-warn)。已核对:两者触碰路径( 最终判定:可合并(人工操作):#2025 —— 修复完整、披露充分(含审查中补修的 degenerate-turn 记账项)、CI 绿、diff 干净。方向二(#2023 floor 3,压缩确认标记减少 refs 枚举)仍待你拍板,不阻塞本 PR。 |
…tack compat) One-line compat for the agreed #2025 -> #2211 -> #2067 merge order: #2025 folds the master REF_LIKE constant into REF_TOKEN/REFS_RUN, which would leave this call dangling (silent tsc break once stacked, each PR green alone). Standalone const carries the identical regex /^\s*m\d{4,}\s*$/; behavior unchanged.
Closes #2023
What & Why
The echo filter only treated a body of EXACTLY ONE bare ref as tag content (#1720 REF_LIKE), so refs-only text adjacent to render-tag structure was released as prose — which is how mNNNNN remnants leaked into the client reasoning stream after compress confirmation markers (which enumerate ref ranges).
Leak shapes confirmed on master (reproduced in-process before fixing):
m01233</acp>— close stripped alone, ref leaked (whole-text + streaming)Fix (one mechanism, every decision point)
PAIRED正则吞掉最多 64 个字符的助手正文 #1720 protection preserved).Behavior changes (old -> new)
Unchanged (pinned by tests): well-formed single-ref pairs; prose-wearing pairs keep their content incl. cited refs (#1720); word-like xm01233; kebab elements; #644 lossless release of MIXED over-cap tails; lone trailing ref with NO tag context at EOF still released (genuine citation).
Both compression modes reasoned: the filter acts on model PROSE fields only (content/reasoning/thinking/summary) — tool-call arguments stay byte-exact (#1039); plugin mode and proxy mode consume the same createTagEchoFilter/stripAcpTags paths.
Documented residuals (out of scope, pre-existing)
<ac/<ac>render-tag fragments to the client (bare truncated open tag survives both process() and flush()) #1755 — a ref glued to a TRUNCATED close (m01233</ac) still leaks, same as before this fix;Tests
New tests/issue2023-refs-residue.test.ts: every shape pinned whole-text AND streaming (char-by-char equivalence over all shapes + explicit chunk splits), plus #1720 protection pins, #644 lossless pins, residual pins. Targeted family green locally (200 tests across the 9 suites touching this module); full suite via CI.
Config surface: none.