Skip to content

[PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender - #7377

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_mdrender-linear-link-scan
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_mdrender-linear-link-scan

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Summary

Fixes #7358 — the shared web Markdown renderer's inline link/image regexes backtracked one character at a time from every opening bracket, so a document full of unclosed [ / ![ cant / [x]( rescanned the same suffix once per marker (O(n²)). Rendering is synchronous on the server (handoff/blog) and the same factory is inlined into the panel IIFE, so a moderate malformed document stalled the calling event loop.

What changed

  • src/web/mdrender.ts: replaced the two regex substitutions (/\[([^\]]+)\]\(([^)\s]+)\)/g and its image twin) with a single linear scanner scanDelimited that shares forward pointers for the first ] and the first ), so every character is visited O(1) times. The two URL-scheme callbacks (image: https? or /__bili/blog-assets/; link: https? or relative) are kept verbatim.
  • tests/mdrender.test.ts: 16 exact-semantics cases + two bounded-time cases (200k unclosed labels under a 2 s runInNewContext timeout, server and minified detached factory).

Behavior

Match semantics are byte-identical to the old regexes — verified over a 20,096-input corpus (20,000 fixed-seed punctuation samples + 96 hand-crafted edge cases) with zero mismatches:

  • the label ends at the first ], the url at the first );
  • a structurally-complete [label](url) is always consumed;
  • a bad URL scheme degrades to the fallback (label text for links, drop for images) — the javascript: / data: vectors stay blocked exactly as before.

Measured on the issue's fixtures:

input before after
[ x 40,000 928 ms 1.8 ms
![ cant x 10,000 847 ms 8.2 ms
[x]( x 10,000 279 ms 4.5 ms

Output is unchanged (the literal text in a paragraph).

Verification

  • npm run typecheck clean, npm run build green.
  • tests/mdrender.test.ts + tests/mdrender-entities.test.ts: 49 pass.
  • The two bounded-time regression tests fail on the old renderer and pass on the fix (verified by temporarily reverting mdrender.ts).
  • Old-vs-new diff over the 20,096-input corpus: 0 mismatches.

Notes

  • No config surface change, no wire/message-shape change, no kernel/ change.
  • No model-facing text change (web panel renderer only).
  • Security posture preserved: the URL-scheme filtering callbacks are unchanged and byte-identical.

@ranxianglei ranxianglei added bug Something isn't working p1 perf scope/ui labels Nov 20, 2026
@ranxianglei ranxianglei changed the title fix: replace quadratic link/image regexes with a linear scanner in mdrender [PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender Nov 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

(ai: vllm-qwen/qwen3.8-27b)

This PR is a duplicate of #7359 — please close it and merge #7359 instead.

The issue reporter (@Afloat16) had already submitted #7359 fixing this exact issue (#7358) before I started. I opened this PR in error: I should have screened open [PR] rows for this thread first (one-fix-PR-per-thread rule) and added to the existing PR instead of opening a parallel one.

Comparison of the two:

Recommendation: close this PR, merge #7359.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Status update (verified live, 2026-11-20 ~17:00Z) — no change since my last comment; still waiting on your decision.

Current live state:

Recommendation (unchanged): close #7377 (this PR) and merge #7359. The technical fix is already complete and verified in #7359; #7377 adds nothing #7359 lacks — its only extra is a broader 16-case exact-semantics test set, which I can fold into #7359 as additive commits if you want (otherwise #7359's existing suite is sufficient).

Both remaining actions are yours — I can't close or merge PRs:

  1. Close [PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender #7377 (this PR).
  2. Approve + merge [PR] fix(web): avoid repeated scans of malformed Markdown links #7359.

8问(给审核者):

  1. 是什么问题: 本线程是一个重复 PR——我(agent)开了 [PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender #7377,但外部贡献者 @Afloat16 的 [PR] fix(web): avoid repeated scans of malformed Markdown links #7359 更早修了同一个 issue [Bug] Malformed Markdown links repeatedly scan overlapping suffixes and block rendering #7358,需要决定保留哪个。
  2. 什么原因导致的: 我开 [PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender #7377 前没有先筛 open [PR] 行(违反 one-fix-PR-per-thread 规则),误开了一个并行重复 PR。
  3. 影响范围与严重程度: 无外部/用户影响——两个 PR 都未合并,线上行为不变。影响仅在流程层面:同一 issue 两个 open PR 在合并时可能互相踩踏。严重度:流程/卫生问题,非用户可见 bug。
  4. 怎么修复的: 保留 [PR] fix(web): avoid repeated scans of malformed Markdown links #7359(外部贡献、与 master 同步、CI 全绿、已独立验证字节一致),关闭 [PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender #7377(我的重复、base 已 stale)。技术修复本身已在 [PR] fix(web): avoid repeated scans of malformed Markdown links #7359 完成并验证,无需重做。
  5. 是否有回归问题: 无——[PR] fix(web): avoid repeated scans of malformed Markdown links #7359 的修复与旧渲染行为字节一致(20,043 输入语料 0 失配),URL scheme 过滤保持不变,CI 全绿(57 success / 27 skipped / 0 failure)。
  6. 是否破坏缓存: 不涉及——仅 web 面板 Markdown 渲染器,无缓存/重试/游标影响。
  7. 是否改了设计方向: 无——同为「线性正向扫描器替换二次正则」方案,无架构/边界/数据流变更。
  8. 是否引入了新的配置: 无——无新 env/config/默认值。

待你决策(仅你能做): ① 关闭 #7377(本 PR);② 批准并合并 #7359。可选:是否把 #7377 额外的 16 个精确语义测试以增量 commit 并入 #7359(否则 #7359 现有测试已足够)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

This is a re-sync of my own prior comment (the duplicate declaration) — no new request. State is unchanged: #7359 is green and verified, #7377 (this PR) is the stale duplicate. I'm waiting on your close of #7377 + approve/merge of #7359, and won't post further until then.

…-time regression (#7358)

The linear link/image scanner fix for #7358 is already on master (619b529 'fix(web): avoid repeated scans of malformed Markdown links'). This PR's original code change (scanDelimited) is a behaviorally-identical duplicate of master's inlineLinks and was dropped during the conflict rebase; what remains is the additive regression suite (16 exact-semantics cases + 2 bounded-time cases). All 45 tests pass against master's inlineLinks.
@ranxianglei
ranxianglei force-pushed the 2026-11-20_mdrender-linear-link-scan branch from f53d916 to fe95880 Compare November 21, 2026 03:52
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-20_mdrender-linear-link-scan (fe95880)

Latest build — install this exact version

npm install -g billion-context@0.0.11140-pr.7377.5a8497e

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
fe95880 billion-context@0.0.11140-pr.7377.5a8497e build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7377.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

Conflict resolved — PR is now mergeable (rebased onto master, force-pushed).

What I did this run:

  1. Diagnosed the conflict. The PR (old head f53d916, branch 2026-11-20_mdrender-linear-link-scan) was cut from an older master. Since then master merged 619b5299 "fix(web): avoid repeated scans of malformed Markdown links" (author Afloat) — the same [Bug] Malformed Markdown links repeatedly scan overlapping suffixes and block rendering #7358 fix this PR implements, but as a different function (inlineLinks vs this PR's scanDelimited). Both are behaviorally-identical linear scanners, so the two edits to src/web/mdrender.ts collided.

  2. Resolved by keeping master's code + this PR's tests. Rebased onto master (e691e6e). For src/web/mdrender.ts I kept master's inlineLinks (already merged, already verified) and dropped this PR's redundant scanDelimited. For tests/mdrender.test.ts (master didn't touch it) I kept this PR's additions. Net result: the PR is now tests-only — 64 lines of additive regression coverage sitting on top of a fix that's already on master.

  3. Verified locally. All 45 tests in tests/mdrender.test.ts pass against master's inlineLinks (16 exact-semantics cases + 2 bounded-time cases: 200k / 20k / 20k unclosed labels under a 2 s runInNewContext timeout, server + minified detached factory). The 4 tests/mdrender-entities.test.ts cases still pass. Targeted typecheck clean.

  4. Pushed. Force-pushed rebased head fe95880 (used --force-with-lease pinned to the verified old head f53d916; no concurrent push was present). Amended the commit message to reflect that the commit is now tests-only. GitHub now reports mergeable: true, mergeable_state: blocked — i.e. clean (no conflict), waiting on review/approval, which is yours.

Important context — this PR is a duplicate. The #7358 fix is already on master (the merged #7359 / 619b5299). After the conflict rebase, this PR adds nothing but regression tests. So you have two options:

I can't close or merge — that's your call. I've left it in a clean, mergeable state either way.

Status-freshness note: the mirror check-cache still shows the old head f53d916 (not yet refreshed to fe95880), so I can't quote the new CI run's result from here — I verified the targeted tests + typecheck locally instead. Full CI will run on fe95880.


8问(给审核者):

  1. 是什么问题: 上游 PR [PR] fix: replace quadratic link/image regexes with a linear scanner in mdrender #7377 与 master 冲突(dirty)——它和 master 上已合并的 [Bug] Malformed Markdown links repeatedly scan overlapping suffixes and block rendering #7358 修复(619b5299)各自独立实现了同一个「线性扫描器替换二次正则」改动,两处对 src/web/mdrender.ts 的编辑互相碰撞。
  2. 什么原因导致的: PR 基于一份较旧的 master 切出;此后 master 合并了 [PR] fix(web): avoid repeated scans of malformed Markdown links #7359(619b5299,inlineLinks),与本 PR 的 scanDelimited 是同一修复的两个等价实现,于是在同一文件同一区域产生内容冲突。
  3. 影响范围与严重程度: 无用户/线上影响——修复本身已在 master 上,本 PR 合并前线上行为不变。冲突只阻塞了 PR 的合并状态。严重度:流程/卫生问题,非用户可见 bug。
  4. 怎么修复的: rebase 到最新 master;src/web/mdrender.ts 保留 master 的 inlineLinks、丢弃本 PR 冗余的 scanDelimited;tests/mdrender.test.ts 保留本 PR 的增量测试。净效果 = 本 PR 变成「纯测试」PR(64 行回归测试)。
  5. 是否有回归问题: 无——保留的是 master 已验证的 inlineLinks(与旧渲染器字节一致);本 PR 的 45 个测试 + 4 个 entities 测试全部通过,定向 typecheck 干净,未改动任何既有行为。
  6. 是否破坏缓存: 不涉及——仅 web 面板 Markdown 渲染器与测试,无缓存/重试/游标影响。
  7. 是否改了设计方向: 无——沿用 master 既定的「线性正向扫描器」方案,无架构/边界/数据流变更。
  8. 是否引入了新的配置: 无——无新 env/config/默认值。

待你决策(仅你能做): ① 关闭本 PR(与已合并的 #7359 / 619b5299 重复);或 ② 合并它(把 16 个精确语义 + 有界时间回归测试并入 master)。我已把它留在干净的、可合并状态,两种方式都行。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working duplicate This issue or pull request already exists p1 perf scope/ui

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Malformed Markdown links repeatedly scan overlapping suffixes and block rendering

1 participant