Skip to content

fix: strip case-drifted render tags; uncapped open-side attrs in tag-echo filter (#1731) - #1741

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-30_tag-echo-case-and-caps
Sep 30, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-30_tag-echo-case-and-caps

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #1731

What changed

src/loop/tag-echo-filter.ts — two independent defects in the model-emitted render-tag stripper:

Defect 1 (case). Every name pattern was built from lowercase literals only, so uppercase/mixed-case echoes (m00001, <AcP …>…, lone ) passed through verbatim. The fix bakes case folding into buildAcplikeName() as per-letter character classes instead of an i flag — deliberately flag-free, because stripAcpTags and flush() rebuild their regexes from .source, where a flag would be silently dropped at every rebuild site (the second-order trap called out in the issue). The raw-wire gate (RENDER_TAG_DETECT), the streaming partial-tail gate (third branch made case-aware), and the Responses-adapter replay-drop regex all consume the same alternation and become case-aware without further edits.

Defect 2 (caps defined a bypass). Open-side attr caps (PAIRED/LONE_OPEN {{0,256}}, BROKEN_ATTRS {{0,512}}) meant a longer attr run escaped every open-tag matcher while the loose close still went — the observed residue "open survived, close deleted". Caps removed on the open side only: [^<>] cannot cross a bracket, so the matchers stay linear for arbitrarily long runs (measured: single 200 KB attr run ≈ 0.5–2 ms; 500 unterminated heads ≈ 0.4 ms; pinned by a new perf-guard test). Close-side {{0,32}} bounds are UNCHANGED — they are load-bearing for #644 (an unbounded close-side tail ate real content after a malformed close; a dedicated test requires the bound).

Tests: the two old pins that encoded the bypass ("long tag-like spans are prose, not tags", "prose with a tag-like opening beyond the attr bound survives intact") are superseded by #1731 tests — long-attr paired tags stripped whole-text and at every split position, case-drifted pairs/lone opens/closes stripped plus a full split sweep, gate engagement on upper-case tag-head tails, prose guards under case drift (Caption/app/"the ACPI spec"/#include <ACPI/acpi.h> all survive), and the ReDoS guard.

Behavior changes (old → new)

  1. Name matching is now case-insensitive everywhere (was lowercase-only). Uppercase/mixed-case echoes are stripped instead of leaking to the client. Consequence: a bare angle-bracketed uppercase token like is now stripped where it previously survived — it is indistinguishable from a casing-drifted tag head, the same trade the filter already made for lowercase. Prose containing the letters without angle brackets still survives (covered by tests).
  2. Open-side attr caps removed (256/512 → unbounded). A terminated long-attr opening is now always decided by its body shape (缺陷:PAIRED 正则吞掉最多 64 个字符的助手正文 #1720): ref-like body → whole span stripped; prose body → tags go, prose stays. Previously such openings leaked verbatim or half-leaked (open kept, close deleted = orphan markup). Performance measured linear; see perf-guard test.
  3. No other wire/message-shape change. Both compression modes considered: the filter operates on model-output text in both modes; plugin mode suppresses injection upstream of this path.

Config surface: none — no config field added, renamed, or re-semantics'd.

Scope notes

Pre-flight (rebased onto current master 27c8774)

  • npm run typecheck — clean
  • npm test — 3326 pass / 0 fail / 17 skip
  • npm run build — success
  • The issue's repro matrix (all six rows) re-run against this code: every previously-leaking row now strips; the lowercase control row is unchanged.

@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-30_tag-echo-case-and-caps (1d3942f)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1689-pr.1741.1009c43

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1741.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of PR #1741 (Fixes #1731)

Pulled 2026-09-30_tag-echo-case-and-caps @ 1d3942fa: merge-base == current master tip 27c8774a, mergeable_state: clean, single commit, 2 files, +104/−30 — every line relates to the PR's purpose.

① Original-issue direction. #1731 names two independent defects (lowercase-only name patterns, with the .source flag-drop second-order trap; open-side attr caps silently defining a bypass while the loose close still went). The PR fixes both at the producer (name construction, cap removal), not at the symptom layer — correct layer. Scope notes verified against history: the #1720 payload-length proxy is indeed already fixed upstream by merged #1721 (in master's ancestry), and the orphan-ref residual is correctly deferred (kernel-owned content, not markup).

② Regression review — existing behaviors on the touched paths:

  1. Lowercase tag stripping (偶尔重复输出 <acp tokens="177" type="text">m00155</acp> #206/[bug] #644 follow-up (post-#646, v0.1.97): typo'd render-tag echo (<acip …) bypasses the strip regex and passes through as a 43-char degenerate end_turn turn — agent stalls mid-orchestration until manual nudge #673): unchanged — the letter classes subsume lowercase; all pre-existing pins green.
  2. Prose guards (acpi/acpi.h includes, caption, app, ACPI spec): held, and re-pinned under case drift — <ACPI/acpi.h> survives because / right after the name disqualifies it; <app>/<Caption> never enter the mutation set at all.
  3. Long-attr open handling (>256 chars): intentional change, properly disclosed — caps removed, a terminated open is now decided by body shape per 缺陷:PAIRED 正则吞掉最多 64 个字符的助手正文 #1720. The two deleted pins encoded the bypass itself as expected behavior; removing them is correct, not lost coverage.
  4. [bug] #361 residual, form 3: turn whose entire text is a render-tag echo strips to an EMPTY message — agent records a marker-only turn, stalls until manual nudge (v0.1.92, post-#365) #644 close-side {0,32} bound: UNCHANGED in LONE_CLOSE/TRUNC_CLOSE/PARTIAL_TAIL-close; the dedicated incident-shape tests (tests/tag-echo.test.ts:135 et al.) still pass.
  5. Streaming gates (RENDER_TAG_DETECT/mayStartRenderTag/third partial-tail branch): extended to upper-case prefixes; the false-positive cost is one no-op filter pass, and the filter is content-preserving on prose, so byte-identical passthrough for pure-prose chunks is unaffected.
  6. Responses-adapter replay drop (src/loop/adapter-responses.ts:134 RENDER_TAG_RE): consumes the shared ACP_NAME_ALT, so it became case-aware automatically — consistent with disclosed change fix: keep system prompt byte-stable for prefix cache #1 ("case-insensitive everywhere").
  7. Tool-call arguments ([bug] 插件透传会剥掉工具调用参数里的 render tag(arguments 流挂了 tag-echo 过滤器) #1039 invariant): untouched; the invariant block is preserved verbatim.
  8. Both compression modes: the filter operates on model-output text fields identically in both modes; no mode-specific code touched; no wire/message-shape change.
    Verdict: no unintended regressions found; the two behavior deltas are exactly the disclosed ones.

③ Code review.

  • buildAcplikeName() is an exact case-folding of the old set: insert-letter set {a,c,p,i} preserved as [aAcCpPiI], longest-first order preserved (all 4-letter before 3-letter). Order-ambiguity check: a 3-letter and a 4-letter alternative can never both satisfy the post-name constraint (\s/>/$) at the same position, so alternation order cannot change outcomes.
  • Flag-free-by-construction is the load-bearing choice, and I verified the trap is real: every .source rebuild site (stripAcpTags lines 243–247, flush lines 676/681) does new RegExp(X.source[, "g"]) — an i flag would have been silently dropped at each rebuild, exactly the second-order failure the issue calls out. Correctly avoided.
  • Linearity argument holds: each uncapped matcher is a single [^<>]* (class cannot cross a bracket) followed by a fixed anchor (>, (?=<), $); no nested quantifiers. Streaming memory stays bounded by TAG_OPEN_CAP=4096 (definite tails dropped past budget) and IMITATION_SWALLOW_CAP=4096.
  • Tie-break verified: when PAIRED and LONE_OPEN match at the same index, the candidate loop's strict-less-than keeps PAIRED first, so ref-like bodies still take the whole-span drop and prose bodies fall to the [bug] #361 residual, form 3: turn whose entire text is a render-tag echo strips to an EMPTY message — agent records a marker-only turn, stalls until manual nudge (v0.1.92, post-#365) #644 release rule — streaming and whole-text results are equivalent, and the split-sweep tests pin that at every boundary.
  • Cross-spelling case-drift pairing (<AcP …></aCP>) works because NAME appears independently on both sides of PAIRED.

④ Test validity. The new tests have teeth: exact-equality assertions, full split-position sweeps (the strongest streaming-boundary pin available), gate-engagement checks, prose-survival guards, and a ReDoS guard whose thresholds (500 ms / 1000 ms) sit 100–1000× above the claimed measurements — an exponential blow-up would trip them. Deleted pins are documented as superseded in-test. One minor, non-blocking gap: no explicit case-drift variant of the #644 ">32-char malformed close" test — the mechanism is identical through the shared NAME, so I accept it.

⑤ Diff cleanliness. Single purposeful commit; no unrelated files, no whitespace churn, no lockfile/generated churn.

Pre-flight (reproduced locally on the rebased head): npm run typecheck clean; npm test 3326 pass / 0 fail / 17 skip (matches the PR's claim); CI: all 13 check runs green, including the real-client e2e (triggered because src/loop/** is hot-file territory), the hermetic registry e2e, the ubuntu/windows × node 22/24 matrix, and build-artifact.

Behavior changes (confirming the PR's disclosure): (1) name matching is case-insensitive everywhere — a bare angle-bracketed uppercase token like <ACPI> is now stripped where it previously survived (documented trade, identical to the lowercase one); (2) open-side attr caps 256/512 → unbounded — long-attr openings no longer leak, performance measured linear and pinned by the perf-guard test. Both are properly flagged in the PR body. Config surface: none.

Gate note: per §7.4, a change to the stripping surface (which model-output bytes reach the client) stays human-merge regardless of how clean the rest is. Nothing here needs a fix-up commit; no separate issue warranted.

一句话中文摘要:审查了 PR #1741(修 #1731 tag-echo 过滤器大小写盲区 + 开标签属性上限绕过)——两处根因都在源头修复(名字模式大小写折叠直接烤进交替式、规避 .source 重建丢 flag 的二阶陷阱;开侧上限移除且保住 #644 闭侧 {0,32} 边界),既有行为逐项回归清单干净、行为变更已如实披露,本地 typecheck/测试(3326/0/17)与 CI 全绿(含 e2e),diff 干净,可以合并(按规约由人工执行合并)。

@ranxianglei
ranxianglei merged commit c8b62af into master Sep 30, 2026
13 checks passed
ranxianglei pushed a commit that referenced this pull request Sep 30, 2026
…ICT_PORT pipe

Review finding on the env-surface retirement: #1714's Class B inventory
lists BILI_STRICT_PORT, but it was left readable from process.env in every
proxy (server.ts port-retry opt-out, #964). It is a claude SessionStart
hook -> launcher -> child pipeline var, so like BILI_MITM_DOMAINS it moves
to the internal BILI_LAUNCHER_* namespace and joins the RETIRED boot-scan
map — a stale 'export BILI_STRICT_PORT=1' now warns instead of silently
disabling port retry on all instances (#1706 failure class). Also rebased
onto current master (no file overlap with #1741).
@ranxianglei
ranxianglei deleted the 2026-09-30_tag-echo-case-and-caps branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tag-echo 过滤器漏掉大写 <ACP …> 与超长属性形态——渲染标签原样到达客户端

1 participant