Skip to content

[PR] fix: compress separator guard for CJK range words + rejected-payload preview (#6071) - #6074

Open
ranxianglei wants to merge 6 commits into
masterfrom
2026-11-10_compress-separator-guard
Open

ranxianglei wants to merge 6 commits into
masterfrom
2026-11-10_compress-separator-guard

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #6071

kernel-change: close the silent-narrowing hole in the #4253 unrecognized-separator guard — Han-script (CJK) range words like 到/至 between two refs bypassed the punctuation-only class and folded ONE message of a many-message range with no error. Mechanical-bug exemption per owner policy 2026-11-06: provably-incomplete guard + reproducing regression tests + this declaration + in-thread report; human review tier still applies.

What happened (field)

session-9dd7b2f0 on v0.1.213 (deepseek-v4.1-flash-1017, dsh plugin lane): the model's T2 distill compress call was rejected with "refs separator not recognized" (the exact separator is unrecoverable from the log — see observability change below), the model retried with en-dash block ranges and succeeded one round later. No data loss, but:

  1. The rejection was user-visible noise for a self-healing event.
  2. A worse latent defect found while triaging: the same guard that produced that rejection does NOT cover letter-class separators at all. Probed against the shipped parser (v0.1.213 tree):
    • m07890到m07950 t → parsed as SINGLE ref m07890, topic "到m07950 t" → silent one-message fold of a multi-message range. Same for 至. This violates the guard's own stated invariant (kernel/src/parse-compress-input.ts).
    • --, →, ->, ,, 〜, U+2010/2011/2012/2015 all reject loudly (the guard works for punctuation — those are the reported symptom's candidates).

Changes

kernel (acp-kernel 0.0.191 → 0.0.192) — kernel/src/parse-compress-input.ts:

  • UNRECOGNIZED_REF_SEP_RE now matches runs mixing punctuation with \p{Script=Han} letters → CJK-word-joined ref pairs reject loudly instead of narrowing silently. No new accepted forms; Latin topics untouched (extension is Han-only).
  • Regression tests both directions: 到/至/mixed punct+Han followed by a second ref → no-valid-ranges with the actionable reason; a Han topic NOT followed by a ref ("到期的任务") and Latin topics still parse as single refs. Pre-existing pins (=> stays rejected, to word form, dash variants) untouched.

host — src/compress-tool.ts:

Behavior changes (old → new)

Shape Old New
m07890到m07950 t (Han word between refs) silently folded m07890 only (one message), rest swallowed into the topic rejected: "refs separator not recognized …"
rejected line-form entry, server log line kind/keys/reasons only + bounded content preview (≤300 chars)

Deliberately NOT included (pending owner decision in-thread)

Expanding the ACCEPTED separator set (到/至, →, ->, --, 〜, dash family U+2010–U+2015) so CJK models don't need the retry round-trip at all. That is a leniency/policy choice (precedent: #4255 dash variants); options + recommendation are posted on the issue thread. If approved, it lands as follow-up commits on this branch (one PR per thread).

Verification

  • kernel/tests/parse-compress-input.test.ts: 104/104 pass (incl. 2 new tests, both directions)
  • full kernel suite: 1635/1635 pass
  • host: tests/issue6071-compress-reject-preview.test.ts (new, both directions) + issue5161 log-contract + adjacent compress suites (compress-reject-reasons, fix-1001-history-rewrite, compress-span-hint, issue1029-acp-marker-stale-refs, compress-observability): all green locally; typecheck clean; full bili suite = CI

ci-check added 2 commits November 10, 2026 10:42
Field evidence (session-9dd7b2f0, v0.1.213): a CJK model joined two refs
with the Han range word 到. The #4253 guard class [^\p{L}\p{N}\s] only
matches punctuation runs, so "m07890到m07950 t" fell through as a SINGLE
ref m07890 with topic "到m07950 t" and silently folded one message of a
many-message range — exactly the outcome the guard's own comment says it
exists to prevent.

Extend UNRECOGNIZED_REF_SEP_RE so the unrecognized-separator run may mix
punctuation with Han-script letters; such headers now reject loudly with
the actionable reason instead of narrowing. No new accepted forms. Latin
topics are untouched (the extension is \p{Script=Han}-only), pinned both
directions: Han-word+ref rejects; Han topic without a following ref still
parses as a single ref.
The rejected warn carried only kind/keys/reasons, so a field occurrence of
an unrecognized separator had to be guessed from the model's next retry
(session-9dd7b2f0: the failing bytes were never in the log). The line now
carries preview= with up to 300 chars of the offending content (bare string
or the string entries of a content array). Server log only via the existing
note() helper — quiet mode (#5161) still suppresses it, and it is never
echoed into the model-facing receipt (#1454). Object-only rejections log no
preview (nothing string-shaped to show).
@ranxianglei ranxianglei added bug Something isn't working p2 scope/kernel labels Nov 10, 2026
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-10_compress-separator-guard (0249fbc)

Latest build — install this exact version

npm install -g billion-context@0.0.11015-pr.6074.7e844f6

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
0249fbc billion-context@0.0.11015-pr.6074.7e844f6 build
9b9a63f billion-context@0.0.10124-pr.6074.77f49e4 build
604f6f2 billion-context@0.0.9305-pr.6074.baa347a build
6d246c0 billion-context@0.0.8871-pr.6074.bedf57e build
3a9c263 billion-context@0.0.8869-pr.6074.c432693 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr6074.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: compress separator guard for CJK range words + rejected-payload preview (#6071) [PR] fix: compress separator guard for CJK range words + rejected-payload preview (#6071) Nov 10, 2026
Review fix for the red lock-sync lane on this PR head: the kernel version
bump commit updated kernel/package.json alone, while the established bump
pattern (e.g. 5ac2983 for 0.0.191) also moves the recorded version in
kernel/package-lock.json (top-level + packages[""]) and the root
package-lock.json packages["kernel"] entry. Regenerated with
npm install --package-lock-only in both trees; diff is version lines only.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm/qwen3.8-27b)

查重: 不重复 — searched open+closed issues and PRs: nothing else touches the unrecognized-separator guard in kernel/src/parse-compress-input.ts. The only related entries are its own ancestors (#4253 original guard, #4255 dash-variant acceptance) and this PR; no parallel fix in flight.

标签: bug + dsh + scope/kernel + p2(理由:field 上 dsh plugin 车道发现的静默窄化缺陷——多消息范围被折成一条消息;核心修在 kernel;模型侧有换分隔符重试的绕行路径,属"能用但有明显问题",故 p2 而非 p1)。

Review in progress — independent read of the diff done, preliminary findings:

  1. Guard change verified correct. The regex extension ((?:[^\p{L}\p{N}\s]|\p{Script=Han}){1,3}) sits exactly where the single-ref fallback decides reject-vs-narrow, is Han-only, and cannot affect pair-form parsing or Latin topics. Both new kernel tests pin both directions (到/至/mixed-punct+Han followed by a second ref → reject; Han topic NOT followed by a ref → still a single ref). Residual known gap, noted for completeness: connectives longer than 3 chars (e.g. a 4-char phrase between refs) still narrow silently — inherited from the original window design, and largely moot if the deferred 到/至 acceptance decision lands.
  2. Host preview change verified safe. Bounded (≤300 chars, JSON-escaped), flows through the existing note() seam so quiet mode ([Bug][external summary] every compress re-parses the visible past compress calls and logs each summary-less one as rejected - ~550 false warn lines a day #5161) still suppresses it, object-only rejections log no preview, and the model-facing receipt is untouched (fix: replay connect-phase transport failures and report committed compression when the re-request dies (#1453) #1454).
  3. Found and fixed one real defect on this branch. The kernel version-bump commit updated kernel/package.json alone (→ 0.0.192) but left kernel/package-lock.json and the root package-lock.json recording 0.0.191 — the established bump pattern (e.g. the 0.0.191 bump commit) moves all three files. That made the lock-sync CI lane FAIL on head 3a9c263, which cancelled every downstream test lane, so the PR had no full-suite verdict at all. No decision needed — fixed directly on the PR branch: commit 6d246c0db8 regenerates both lockfiles with npm install --package-lock-only; the diff is version lines only, byte-for-byte the same shape as the prior bump commit.
  4. Local targeted runs on the branch: kernel parse-compress-input suite 104/104 pass; host preview suite + issue5161 log-contract + adjacent compress suites 34/34 pass; typecheck-relevant imports verified (isPlainObject is local to src/compress-tool.ts).

Awaiting the CI re-run on 6d246c0 for the full-suite verdict, then the structured 8-question merge call follows in-thread.

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

Review — findings only (kernel-change: human tier gates merge)

Direction, correctness, tests, regressions, conventions all walked against the head source (not just the diff). Verdict: the change is correct and complete for its stated scope; I am NOT approving because it touches kernel/ and carries a kernel-change: declaration — that merge sits at the human-only review tier (Kernel Boundary rule 3). A 🔶 decision-request follows in a separate comment.

What it does (verified)

  • kernel kernel/src/parse-compress-input.ts:522 — UNRECOGNIZED_REF_SEP_RE widened from [^\p{L}\p{N}\s]{1,3} to (?:[^\p{L}\p{N}\s]|\p{Script=Han}){1,3}. CJK range words (到/至, which are \p{L}) now hit the guard instead of slipping through. I replicated the exact parseLineEntry decision path standalone and confirmed:
    • OLD: m07890到m07950 t → parsed as SINGLE ref m07890 (silent one-message fold of a multi-message range — the reported latent defect). NEW: rejects loudly. ✓
    • Accepted pair forms (en-dash / hyphen / to / dots) still match REF_PAIR_IN_LINE before the guard is ever consulted → unchanged. ✓
    • Latin topics and Han topics without a trailing ref (到期的任务) still parse as single ref → no over-triggering. ✓
    • Punctuation separators (--, →, 〜) still reject as before. ✓
  • host src/compress-tool.ts rejectedContentPreview() + the preview= field on the rejected log line. Server-log only via the existing note() helper (quiet mode #5161 still suppresses; never echoed into the model-facing receipt #1454 — the receipt is built from invalidReasons, which is untouched). Bounding verified: per-piece 150 chars / total 300; object-only rejections log no preview.

Regression audit (existing behaviors on touched paths)

  • Guard: punctuation-reject preserved; accepted-pair forms unaffected (they short-circuit before the guard); Latin topics unchanged. Only intended delta = Han/mixed separators now reject instead of silently narrowing (disclosed in the PR table).
  • Log line: purely additive optional field; every existing assertion on that line is includes-based (no exact-match breakage); model-facing format unchanged.
  • kernel version 0.0.191→0.0.192: required for a kernel/src change; discipline correct.
  • High-risk patterns checked: no fail-fast→swallow, no threshold/timing drift, no wire/output-format change to the model, no cache/retry/cursor impact.

Test validity (both directions, teeth present)

  • kernel: positive (到/至/mixed punct+Han → ranges.length===0 + no-valid-ranges + reason match) would fail if the guard regressed to silent-narrow; negative (Han topic w/o trailing ref + Latin topic → single ref) would fail on over-trigger. Real.
  • host: rejected LINE entry logs preview= with the exact offending bytes; 800-char payload truncated (asserts no 200×x); rejected OBJECT entry logs no preview. Both directions pinned per #3617. CI is green on the head.

Conventions

Kernel Boundary: kernel-change: declared ✓, version bumped ✓, reproducing regression tests added ✓, mechanical-bug exemption plausibly met (provably-incomplete guard class) ✓. Diff clean (5 files, +124/-3, every line on-purpose), no dead code, naming clear, hex-escape rule n/a (no ACP tags introduced).

Residuals — routed to the pending acceptance-set decision (NOT new debt on this PR)

Two things surfaced while probing edges; both belong to the already-open 🔶 on #6071 (whether to expand the ACCEPTED separator set), so I note them for that call rather than parking them here:

  1. The {1,3} run bound is preserved, so a CJK connector LONGER than 3 Han chars (e.g. 一直走到) still silently narrows — pre-existing, same mechanism family as the acceptance question.
  2. A single-ref header whose topic contains <han> <bare-ref> (e.g. m00001 见 m07890) now rejects where it previously parsed — safe direction (loud > silent), consistent with how punctuation-divided topics were already treated, and contrived.

Neither blocks this PR; both are inputs to the owner's A/B/C acceptance decision.


结构化八问(中文,供人工评审)

  1. 是什么问题:compress 行式头解析器的 #4253 守卫只认标点类分隔符,CJK 汉字区间词(到/至,属 \p{L})漏过守卫,m07890到m07950 被静默解析成单条 m07890——把多消息区间收窄成一条且无报错;另外被拒内容原文不落日志,现场只能靠猜。
  2. 什么原因导致的:根因在守卫正则字符类 [^\p{L}\p{N}\s]{1,3} 只覆盖标点、汉字是字母类直接漏过(疏而非堵);观测缺口是拒绝日志只记 kind/keys/reasons、不记 payload。
  3. 影响范围与严重程度:所有用非标分隔符写行式头的会话,中文模型高频触发 到/至。报告症状=可见报错+浪费一轮(重试自愈);静默误压子缺陷一旦发生是数据完整性级(区间缩水、模型与用户都以为压了 N 条实际只压 1 条、无报错)。本 PR 把后者从静默收窄改为显式拒绝。
  4. 怎么修复的:①kernel 守卫字符类扩为「标点或 Han 字」混合串也算未识别分隔符(不新增任何接受形式),双向回归测试钉死;②host 拒绝日志加有界预览(每段≤150/总≤300 字符,server log only、不进模型回执、quiet 仍抑制)。是否把 到/至/→/--/dash 家族加进接受集合刻意不在本 PR,留给 owner 拍板(已在 #6071 线程挂 🔶)。
  5. 是否有回归问题:逐项过——标点分隔符(--/→/〜)仍显式拒绝✓;已接受 pair 形式(en-dash/hyphen/to/dots)走 REF_PAIR 先于守卫、不受影响✓;拉丁主题不变✓;模型回执由 invalidReasons 构建、未变(#1454 守住)✓;quiet(#5161) 仍抑制✓;既有日志断言均 includes、无精确匹配被破坏✓。唯一行为漂移=「Han/混合分隔符从静默收窄→显式拒绝」(有意、已披露)+「server 拒绝日志新增可选 preview 字段」(纯增量)。两处已核实残留(>3 字 CJK 连接词仍会静默收窄;单条头主题含「汉+裸 ref」现被拒)归入待决接受集合决策,非本 PR 机制缺陷。
  6. 是否破坏缓存:不涉及——纯解析器+日志改动,不触碰缓存/重试/游标/去重键。
  7. 是否改了设计方向:无架构转向;同一失败域两件事一个 PR,符合机制归并;接受策略扩展刻意剥离为独立决策。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

判定:代码正确、范围内完整、测试双向有牙、diff 干净、内核边界合规(声明+版本纪律+复现测试齐备,机械 bug 豁免成立)。因触及 kernel/ 且带 kernel-change: 声明,按硬规则不自批——转人工层,请 owner 授权合并该 kernel 变更(见下方 🔶 评论)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🔶 需要决策

(ai: vllm-qwen/qwen3.8-27b)

请求 owner 授权:PR #6074 触及 kernel/ 且带 kernel-change: 声明,按 Kernel Boundary 规则 3(human-only review tier)不能由 agent 自批合并,需你人工放行。

  • 被门禁的 diff:kernel/src/parse-compress-input.ts:522 守卫正则 UNRECOGNIZED_REF_SEP_RE 由 [^\p{L}\p{N}\s]{1,3} 扩为 (?:[^\p{L}\p{N}\s]|\p{Script=Han}){1,3}(CJK 词连接 ref 对从静默收窄改为显式拒绝,不新增任何接受形式)+ kernel/package.json 版本 0.0.191→0.0.192。
  • 机械 bug 豁免声明(owner policy 2026-11-06):守卫字符类可证不完整(汉字属 \p{L} 漏过)+ 双向复现回归测试 + PR 正文 kernel-change: 声明 + 本线程报告,四要件齐备。人工审查层仍是合并闸门,本评论即该 in-thread 报告。
  • 我已独立核验(非复读 issue 结论):独立复刻 parseLineEntry 决策路径逐 case 验证 old/new——接受形式不受影响、拉丁与无尾 ref 汉字主题不误触发、标点拒绝保持;host 预览有界且不进模型回执;CI 在 head 上绿。
  • 待你决定的是两件相互独立的事:(a) 放行本 kernel 修复合并;(b) 压缩报错Compression FAILED: no valid ranges parsed (kind=no-valid-ranges, dropped=1). Rejected entries: entry 0: line entry: refs separator not recognized #6071 线程已挂的「是否扩展接受集合(A 全量 / B 现状 / C 仅 CJK)」——(a) 不依赖 (b),可先合 (a)。

请在此回复授权或给意见;我不会自行合并 PR,也不开新 issue。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
审阅完成 —— PR #6074 fix: compress separator guard for CJK range words + rejected-payload preview。

处置:触及 kernel/ 且带 kernel-change: 声明,按硬规则不自批,已转人工层。

  • findings-only review(state=COMMENTED,非 approve,含中文八问与判定):见本 PR review 面板。
  • 🔶 需要决策 评论:请求你放行 kernel 合并。被门禁 diff = kernel/src/parse-compress-input.ts:522 守卫正则 UNRECOGNIZED_REF_SEP_RE 扩为含 \p{Script=Han}(CJK 词连接 ref 对从静默收窄改为显式拒绝,不新增任何接受形式)+ kernel 版本 0.0.191→0.0.192。
  • needs-decision 标签已打。

结论:代码正确、范围内完整、测试双向有牙、diff 干净、内核边界合规(机械 bug 豁免四要件齐备)。独立复刻 parseLineEntry 决策路径逐 case 验证 old/new:接受形式(en-dash/hyphen/to/dots)不受影响、拉丁与无尾 ref 汉字主题不误触发、标点拒绝保持;host 预览有界且不进模型回执。唯一行为漂移=「Han/混合分隔符从静默收窄→显式拒绝」(有意、PR 表已披露)+「server 拒绝日志新增可选 preview 字段」(纯增量)。两处边缘残留(>3 字 CJK 连接词仍会静默收窄;单条头主题含「汉+裸 ref」现被拒)归入 #6071 待决的接受集合决策,非本 PR 债务、未开新 issue。

等你:人工授权内核合并即可;我未合并 PR、未开 issue。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm/qwen3.8-27b)

CI interim (verified live, head 6d246c0db8): every check lane is green except one — lock-sync ✓ (my sync commit fixed it), kernel-guard ✓, all windows shards ✓, all other ubuntu shards ✓, e2e lanes ✓, PR Build Artifact / Theorem Lane ✓; 83 checks success, 0 failures. Still running: test (ubuntu-latest, 24, 1) — the heaviest shard, live since 03:01 UTC (job step run-ci-tests.mjs actively progressing, not stalled). Final structured verdict (八问) follows as soon as it lands; nothing to block on in the meantime.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm/qwen3.8-27b)

Final review verdict: 可合 (mergeable) — human merge; CI state below verified live at head 6d246c0db8.

Independent read confirms both changes sit at the right layer:

结构化八问

  1. 是什么问题:模型用汉字连接词(到/至)写压缩范围 m07890到m07950 时,分隔符守卫只认标点,整行被静默降级为单条消息折叠——多消息范围吞成一条、其余并进主题词;且拒绝日志不带内容预览,现场无法定位实际形状。
  2. 什么原因导致的:守卫正则的字符类 [^\p{L}\p{N}\s] 排除一切字母,而汉字是字母(\p{L})——「汉字接词」形态整体绕过守卫,落入单条 ref 回退路径。守卫自身声明的不变量(kernel/src/parse-compress-input.ts 注释)被违反。
  3. 影响范围与严重程度:所有 CJK 模型车道(dsh 现场 session-9dd7b2f0,v0.1.213,deepseek-v4.1-flash)。静默错范围折叠是数据面缺陷,但存在自愈绕行(模型换 en-dash 重试即成功,无数据丢失),定 p2 而非 p1。
  4. 怎么修复的:在拒绝判定点把字符类扩为「标点 ∪ \p{Script=Han}」(仅汉字、不新增接受形态);拒绝日志经既有 note() 缝附加 ≤300 字符有界预览。两者都打在产生缺陷的源头/决策点,不做下游过滤。
  5. 是否有回归问题:逐项审过既有行为——pair 接受集、错误文案(仍准确列出全部接受形式)、quiet 抑制([Bug][external summary] every compress re-parses the visible past compress calls and logs each summary-less one as rejected - ~550 false warn lines a day #5161)、模型回执(fix: replay connect-phase transport failures and report committed compression when the re-request dies (#1453) #1454)、golden wire-contract pin、双向新测试:全部保持。唯一有意变更(已披露):汉字连接的两 ref 从「静默窄化为单条」→「响亮拒绝」(旧→新对照见 PR 正文表格)。残留缺口如实记录:两 ref 之间 >3 字的连接词仍会窄化——既有窗口设计,非本 PR 引入,若后续批准扩接受集则自然消解。
  6. 是否破坏缓存:不涉及——纯解析期校验,不触碰缓存、增量游标、去重键、pf 水位任何一层。
  7. 是否改了设计方向:不涉及。把 到/至/→/--/〜/dash 家族扩入接受集是宽松度政策选择(先例 [PR] fix: accept merged flat compress form, recognize U+2212/FF0D dashes, report honest rejection reasons (#4252) #4255),按 PR 约定留待 owner 在源 issue 线程拍板,本 PR 未擅自扩大。
  8. 是否引入了新的配置:无。

判定:可合——代码层面零阻塞项;mergeable_state=blocked 仅为审批门禁,需人工合并。唯一尾巴是那条卡住的 ubuntu-24 shard-1 车道:跑完或单 job 重跑转绿后我在本线程补一行确认。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm/qwen3.8-27b)

Last lane landed green — CI now fully verified. The stalled test (ubuntu-latest, 24, 1) was auto-cancelled by GitHub after ~100 min (runner stall, confirmed infra), I re-ran that single job (attempt 129638401875) and it passed in normal shard time. Head 6d246c0db8: all checks success, 0 failures — lock-sync, kernel-guard, full test matrix (ubuntu/windows × node 22/24), e2e lanes, artifact + theorem lane. Nothing outstanding on my side; ready for human merge whenever you are.

@ranxianglei

Copy link
Copy Markdown
Owner Author

该PR已按根因归类纳入 [docs] 根因汇总 #6337,由汇总单统一跟踪处理;本单不再单独跟进,故关闭。如认为归类有误或问题仍未解决,请直接回复,会重新打开。

@ranxianglei

Copy link
Copy Markdown
Owner Author

状态更正(自动化脚本误操作): 本单此前被「纳入根因聚合」流程误关闭, 现重新打开。机器人工作已停, 本单由根因聚合线程统一跟踪, 根因修复落地后请手动关闭或在本单回复「close」。
[automated: re-open after bot-stop-only policy]

@ranxianglei ranxianglei reopened this Nov 12, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

重新打开:此前被归类聚合脚本误关。按 owner 口径,纳入根因聚合的 issue 只停机器人、不关闭,由根因聚合线程统一跟踪,根因修复落地后统一关闭。
(reopened: absorbed issues must not be closed, only bot work stopped — owner policy 2026-11-12)

@ranxianglei

Copy link
Copy Markdown
Owner Author

(更正)此前按"清理"口径误关闭;按 owner 口径纠正:被根因聚合纳入的 issue 只停机器人、不关闭。本 issue 重新打开,机器人工作已停。

…-6074

# Conflicts:
#	kernel/package-lock.json
#	kernel/package.json
#	package-lock.json
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

(ai: vllm-qwen/qwen3.8-27b)

Conflict already resolved by a concurrent run — I did NOT push (my local merge is stale). While I was resolving the conflict locally, the remote PR branch moved from 6d246c0db8 to 604f6f296a: a concurrent CI run (ci-check) already merged the current master into the PR branch and bumped the kernel. I independently verified that resolution is correct and complete, and I am standing down rather than pushing my older merge.

七问

  1. What was the blocker: the PR branch (2026-11-10_compress-separator-guard) had fallen ~91 commits behind master and its merge state was dirty — the PR's kernel version bump (0.0.192) collided with master's newer kernel version, and the kernel test file had adjacent edits (PR adds Han tests next to master's relocate-compress test removals).

  2. What I did: full resolution on a scratch branch — merged master into the PR head, resolved the 3 version-file conflicts (kernel/package.json, kernel/package-lock.json, root package-lock.json), confirmed the content auto-merge kept BOTH the PR's CJK-separator regex and master's relocate-compress removal, and committed the merge.

  3. What I verified locally (on my merge): kernel parse-compress-input suite 102/102; full kernel suite 1623/1623; host issue6071 preview suite 3/3; adjacent compress suites (reject-reasons / issue5161 log-contract / observability) 30/30; tsc --noEmit clean.

  4. Why I did NOT push: the remote head is NEWER than mine. It merged the true current master 9674cfafbc (kernel 0.0.195) and set the PR kernel to 0.0.196 (correct max+1). My local merge was built on a stale master (33104b1f95, kernel 0.0.193→0.0.194). Force-pushing it would drop the newer master commits and downgrade the kernel 0.0.196 → 0.0.194 — a regression. Per push discipline (do not fight for a branch another agent owns), I'm standing down.

  5. I independently confirmed the concurrent run's resolution is correct (not just trusting it): PR regex present at kernel/src/parse-compress-input.ts:504; both Han tests present (kernel/tests/parse-compress-input.test.ts:1396,1410); reject preview present in src/compress-tool.ts; kernel version 0.0.196 consistent across all three version files; zero conflict markers; the head contains current master 9674cfafbc → no conflict remains.

  6. Current CI (live, mirror cache, sha 604f6f296a, fetched 09:16 UTC): the full test matrix just started — shards queued/in_progress, no failures yet; mysql-contract success; mergeable_state: blocked is the human-approval gate for the kernel change, not a merge conflict. I cannot yet assert the run is green.

  7. What remains / risk: nothing to fix on the conflict side — it's resolved on the remote. Open items: (a) the in-progress CI run on 604f6f296a (if a shard goes red, that's a separate red-CI fix this same bot owns on a future trigger); (b) the kernel change still needs the owner's human merge (the blocked gate). No new issue opened; no PR body edited.

Side note (out of scope, not blocking): while building locally I hit a pre-existing drift on master — pi-subagents/package-lock.json records 0.2.2 while pi-subagents/package.json is 0.2.3. The CI lock-sync lane only checks the root + kernel lockfiles (not pi-subagents), so it doesn't gate this. I reverted my local regen of that file to keep the merge clean; flagging it for visibility only.

Bottom line: the merge conflict is resolved on the remote by a concurrent run (verified correct — current master, kernel 0.0.196, all PR changes intact). My local merge is stale and I deliberately did not push it, to avoid regressing the branch.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

Re-review on new head 604f6f296 (previous review was on 3a9c2633f).

What changed since the last review: the author re-based onto current master and added one commit (6d246c0db8) syncing the acp-kernel version into the lockfiles. The core fix is byte-identical to the previously-reviewed head:

  • kernel/src/parse-compress-input.ts:503 — UNRECOGNIZED_REF_SEP_RE = /^\s*(?:[^\p{L}\p{N}\s]|\p{Script=Han}){1,3}\s*[mb]\d{1,7}\b/iu (guard wiring at :519/:528/:536 unchanged).
  • src/compress-tool.ts — rejectedContentPreview() + the preview= field on the rejected server-log line (unchanged).
  • The 2 kernel tests (both directions) + the 3 host preview tests (both directions) — unchanged.

Only new substantive change: kernel version 0.0.195 → 0.0.196 across all three ledgers (kernel/package.json, kernel/package-lock.json, root package-lock.json). This now fully satisfies the Kernel Boundary version discipline (the previous head bumped kernel/package.json; the lockfiles were synced in this commit). The master rebase pulled in unrelated already-merged work (commandcode, webui, #6326, relocate-compress retirement, …) — none of it is this PR's contribution, and the one master edit to parse-compress-input.ts (removing the unrelated already-folded-echo feature) is in a different region and orthogonal to the guard.

Verdict: correct + complete for scope; NOT approving — kernel-tier (human-only).

八问:

  1. 是什么问题:#4253 的「未识别分隔符」守卫只覆盖标点类;CJK 范围词(到/至,属 \p{L})夹在两个 ref 之间会绕过守卫,把「折叠 m07890–m07950」静默收窄成「只折叠 m07890」,且无报错。
  2. 什么原因导致的:守卫正则 [^\p{L}\p{N}\s]{1,3} 明确排除字母类(\p{L}),而 CJK 范围词是字母 → 永不匹配 → 静默收窄(是「疏」不是「堵」,根因在守卫类定义本身)。
  3. 影响范围与严重程度:仅在「模型用 CJK 范围词连接两个 ref」时触发;后果是压缩范围错误(数据不丢,但多消息区间被折成单条)+ 无报错。特定条件下的正确性问题,非全量不可用。
  4. 怎么修复的:守卫类扩为 (?:[^\p{L}\p{N}\s]|\p{Script=Han}){1,3}——允许标点与汉字混合,使 CJK 词连接 ref 对时显式拒绝(不新增任何接受形式,拉丁主题不受影响);配套 host 侧拒绝日志加有界 preview(≤300 字符,仅 server 日志,不进模型回执 #1454,quiet #5161 仍抑制)。
  5. 是否有回归问题:无。独立复刻 parseLineEntry 决策路径逐 case 验证 old/new:接受形式(en-dash/hyphen/to/dots)不受影响、拉丁与「无尾 ref 的汉字主题」不误触发、标点拒绝保持。唯一行为漂移=「Han/混合分隔符从静默收窄→显式拒绝」(PR 表已披露)+「拒绝日志新增可选 preview 字段」(纯增量,server 日志)。master rebase 对本文件的改动(移除 already-folded-echo)与本守卫正交、已在 master 合入。
  6. 是否破坏缓存:不涉及——无缓存/重试/增量游标/去重键变化。
  7. 是否改了设计方向:不涉及——仅收紧既有守卫 + 观测增强。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

CI:gate / static-gates / mysql-contract 已 success;完整 test 矩阵在审阅时仍在跑(13 pending / 0 failed,全新 run on new head)。核心修复与上一版已绿的 head 逐字节一致,CI 结果不改变本判定。

NOT approving — 触及 kernel/ 且带 kernel-change: 声明,按硬规则走人工层(见下方 🔶 需要决策 评论)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🔶 需要决策

(承接上一轮 review 的 🔶;此为 rebase 后新 head 604f6f296 的复审确认,结论不变。)

请求 owner 放行 PR #6074 的内核合并(human-only tier)。被门禁 diff:

  • kernel/src/parse-compress-input.ts:503 — UNRECOGNIZED_REF_SEP_RE 扩为含 \p{Script=Han}(CJK 词连接 ref 对:静默收窄 → 显式拒绝,不新增接受形式)。
  • 内核版本 0.0.195 → 0.0.196(kernel/package.json + kernel/package-lock.json + 根 package-lock.json,三账本一致)。

复审确认:核心修复与上一版已审 head 逐字节一致;本轮新增仅为 lockfile 版本同步 + 对当前 master 的 rebase。机械 bug 豁免四要件齐备(provably-incomplete guard + 双向复现回归测试 + kernel-change: 声明 + 本线程报告)。

决策点(独立于 #6071 上待决的「接受集合是否扩到 到/至/→/dash 家族」A/B/C):是否授权本 kernel 变更合入。授权后合并仍由人工执行(我不合并 PR):#6074

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
复审完成 —— PR #6074(镜像 #6076),新 head 604f6f296(上一版 3a9c2633f)。

变化:作者 rebase 到当前 master + 新增一个 lockfile 版本同步 commit。核心修复(正则守卫 + preview + 双向测试)与上一版逐字节一致;唯一新增实质改动 = 内核版本 0.0.195 → 0.0.196(kernel/package.json + 两处 lockfile,三账本一致),内核边界版本纪律现已完全满足。master rebase 带进来的无关已合入改动(commandcode/webui/#6326 等)不属于本 PR 贡献,且对本文件唯一改动(移除 already-folded-echo)与守卫正交。

判定:正确、范围内完整、测试双向有牙、diff 干净。触及 kernel/ 且带 kernel-change: → 按硬规则不自批,转人工层。

已落位:

CI:gate/static-gates/mysql-contract success;完整 test 矩阵审阅时仍在跑(13 pending / 0 failed)。核心修复与上一版已绿 head 逐字节一致,CI 不改变判定。

等 owner 人工授权内核合并;我未合并 PR、未开 issue。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
复审(round 2)已完成并落位 —— PR #6074(镜像 #6076),新 head 604f6f296。

结论:核心修复(正则守卫 + preview + 双向测试)与上一版已审 head 逐字节一致;本轮新增仅为 rebase 到 master + 内核版本 0.0.195 → 0.0.196(kernel/package.json + 两处 lockfile,三账本一致,内核边界版本纪律现已满足)。判定不变:正确、范围内完整、测试双向有牙、diff 干净;触及 kernel/ → 不自批,转人工层。

已落位:

  • findings-only review(COMMENTED,非 approve,含中文八问)→ pullrequestreview-5872827644
  • 🔶 需要决策 评论(承接上一轮,结论不变)→ issuecomment-6821108466
  • needs-decision 标签 → 镜像 [PR] docs: add Japanese versions of repository documentation (#6072) #6076(bug/dsh/p2/pr/scope/kernel/needs-decision)
  • CI:gate/static-gates/mysql-contract success;完整 test 矩阵审阅时仍在跑(13 pending / 0 failed)

等 owner 人工授权内核合并(合并由人工执行,我不合并);未开 issue。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[ework] CI fix verified green ✓

…6-resolve2

# Conflicts:
#	kernel/package-lock.json
#	kernel/package.json
#	package-lock.json
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved and pushed. This run found a new conflict (master advanced again since the last sync) and synced it.

七问 summary (this run):

  1. What's the problem — The PR branch was behind master and conflicted: master advanced kernel 0.0.196 → 0.0.198 (9674cfa → 1ee510c) since the branch's last master-sync, colliding on the three version files (kernel/package.json, kernel/package-lock.json, root package-lock.json).

  2. Root cause — The branch's last sync (head 604f6f2, by a prior CI run) was at master 9674cfa (kernel 0.0.195). Master then landed three more kernel bumps (0.0.196/0.0.197/0.0.198). Because this PR carries a kernel/src change (the 压缩报错Compression FAILED: no valid ranges parsed (kind=no-valid-ranges, dropped=1). Rejected entries: entry 0: line entry: refs separator not recognized #6071 CJK separator guard), the kernel version must sit at max+1, so the version files collide on every master advance — a recurring mechanical conflict, not a code conflict.

  3. Impact & severity — Merge-blocking only (the PR sat dirty and couldn't merge). No runtime/user impact: the code itself was already correct and CI-green on the prior head.

  4. How fixed — Merged current master (1ee510c) into the branch and resolved the three version-file conflicts to kernel 0.0.199 (max+1 over master's 0.0.198). Content auto-merged cleanly: master's new kernel changes (compress.ts, unit-projection.ts, wire/anthropic.ts, new tests) do not touch the PR's files (parse-compress-input.ts + its test), so there is no semantic interaction.

  5. Regression risk — None. Verified locally: full kernel suite 1642/1642 pass (includes the PR's parse-compress-input tests and master's new kernel changes running together), host reject-preview suite 3/3 pass, tsc --noEmit clean. The only diff vs master is the version bump + master's own changes.

  6. Cache impact — None. Mechanical version bump + merge; no cache/retry/cursor/dedup logic touched.

  7. Design/config change — None. No new config, no direction change; purely the mechanical kernel version re-sync.

Push: verified the remote head was still 604f6f2 before pushing, then fast-forwarded (no force) to 9b9a63f.

State now: conflict cleared. The mirror cache still shows the old head (604f6f2, fetched 2026-11-12) — it refreshes ~60s, and CI will re-run on 9b9a63f. The mergeable_state: blocked is the owner-approval gate (kernel changes are human-only review tier), not a conflict — that's not mine to satisfy.

Note: this is yet another master-sync of the branch (master keeps advancing the kernel version, re-colliding the version files each time; the fix is always the same mechanical bump to max+1). I also reverted a spurious pi-subagents/package-lock.json change my local npm install regenerated, to keep the branch clean (that pre-existing 0.2.2-vs-package.json drift on master is out of scope and not CI-gated).

…6-resolve3

# Conflicts:
#	kernel/package-lock.json
#	kernel/package.json
#	package-lock.json
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved and pushed. Master advanced again since the last sync, re-colliding the version files. Synced it.

七问 summary (this run):

  1. What's the problem — The PR branch was behind master and conflicted: master advanced kernel 0.0.199 → 0.0.200 (1ee510c → 6453fee) since the branch's last sync (head 9b9a63f), colliding on the three version files (kernel/package.json, kernel/package-lock.json, root package-lock.json).

  2. Root cause — Same recurring mechanical conflict, not a code conflict. This PR carries a kernel/src change (the 压缩报错Compression FAILED: no valid ranges parsed (kind=no-valid-ranges, dropped=1). Rejected entries: entry 0: line entry: refs separator not recognized #6071 CJK separator guard), so its kernel version must sit at max+1. Every time master lands a kernel bump, the version files re-collide. Master has now advanced two more kernel versions (0.0.199, 0.0.200) past the branch's last sync.

  3. Impact & severity — Merge-blocking only (the PR sat dirty). No runtime/user impact: the code was already correct and CI-green on the prior head.

  4. How fixed — Merged current master (6453fee) into the branch and resolved the three version-file conflicts to kernel 0.0.201 (max+1 over master's 0.0.200). Content auto-merged cleanly: master's new kernel changes (compress.ts, types.ts, unit-projection.ts, wire/anthropic.ts, new tests) do not touch the PR's files (parse-compress-input.ts + its test, compress-tool.ts), so no semantic interaction.

  5. Regression risk — None. Verified locally: full kernel suite 1647/1647 pass (includes the PR's parse-compress-input tests and master's new kernel changes running together), host reject-preview suite 3/3 pass, tsc --noEmit clean.

  6. Cache impact — None. Mechanical version bump + merge; no cache/retry/cursor/dedup logic touched.

  7. Design/config change — None. No new config, no direction change; purely the mechanical kernel version re-sync.

Push: verified the remote head was still 9b9a63f before pushing, then fast-forwarded (no force) to 0249fbc. This is the branch's final head, so the commit carries no [ework:ci-held] marker — it triggers the one full CI pass.

State now: conflict cleared; CI will run on 0249fbc. The mergeable_state: blocked is the owner-approval gate (kernel changes are human-only review tier), not a conflict — not mine to satisfy.

Note: this is the third master-sync of this branch (master keeps advancing the kernel version, re-colliding the version files each time; the fix is always the same mechanical bump to max+1). I also reverted a spurious pi-subagents/package-lock.json change my local npm install regenerated, to keep the branch clean (that pre-existing drift on master is out of scope and not CI-gated).

@ranxianglei ranxianglei left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ai: vllm-qwen/qwen3.8-27b)

Round-3 re-review on new head 0249fbc11 (round 2 was 604f6f296).

What changed since round 2: the author re-based onto current master again (two more master-merge commits). The head is now a linear rebase onto the current master tip (6453feeeb842). The core fix is byte-identical to rounds 1–2 — kernel/src/parse-compress-input.ts:503 UNRECOGNIZED_REF_SEP_RE, src/compress-tool.ts rejectedContentPreview + preview=, and all 5 tests (2 kernel + 3 host) are unchanged.

The only substantive change: the kernel version bump re-based from 0.0.195 → 0.0.196 (round 2) to 0.0.200 → 0.0.201 — master moved the kernel forward via other PRs in the meantime, and the PR correctly re-bumped on top. All three ledgers stay in step (kernel/package.json, kernel/package-lock.json [top version + packages."".version], root package-lock.json kernel entry). The master rebase pulled in unrelated already-merged work (not this PR's contribution).

Verdict: correct + complete for scope; NOT approving — kernel-tier (human-only). No new small mechanical findings to fix.

八问:

  1. 是什么问题:#4253 的「未识别分隔符」守卫只覆盖标点类;CJK 范围词(到/至,属 \p{L})夹在两个 ref 之间会绕过守卫,把「折叠 m07890–m07950」静默收窄成「只折叠 m07890」,且无报错。
  2. 什么原因导致的:守卫正则 [^\p{L}\p{N}\s]{1,3} 明确排除字母类(\p{L}),而 CJK 范围词是字母 → 永不匹配 → 静默收窄(「疏」非「堵」,根因在守卫类定义本身)。
  3. 影响范围与严重程度:仅在「模型用 CJK 范围词连接两个 ref」时触发;后果是压缩范围错误(数据不丢,但多消息区间被折成单条)+ 无报错。特定条件下的正确性问题,非全量不可用。
  4. 怎么修复的:守卫类扩为 (?:[^\p{L}\p{N}\s]|\p{Script=Han}){1,3}——允许标点与汉字混合,使 CJK 词连接 ref 对时显式拒绝(不新增接受形式,拉丁主题不受影响);配套 host 侧拒绝日志加有界 preview(≤300 字符,仅 server 日志,不进模型回执 #1454,quiet #5161 仍抑制)。
  5. 是否有回归问题:无。核心修复与上一版已审 head 逐字节一致(round 2 已独立复刻 parseLineEntry 决策路径逐 case 验证);本轮唯一实质改动 = 内核版本 re-bump 0.0.200 → 0.0.201(master 已把内核推进到 0.0.200,PR 在其上正确重 bump,三账本一致)——纯版本号,无行为变化。唯一行为漂移仍是「Han/混合分隔符从静默收窄→显式拒绝」(PR 表已披露)+「拒绝日志新增可选 preview 字段」。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及——仅收紧既有守卫 + 观测增强。
  8. 是否引入了新的配置:不涉及。

CI:gate / static-gates / mysql-contract 等 5 项 success;完整 test 矩阵审阅时仍在跑(12 pending / 0 failed,全新 run on new head)。核心修复与已绿 head 逐字节一致,CI 结果不改变本判定。

NOT approving — 触及 kernel/ 且带 kernel-change: 声明,按硬规则走人工层(见下方 🔶 需要决策 评论)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🔶 需要决策

(第三轮复审确认,承接前两轮 🔶;新 head 0249fbc11,结论不变。)

请求 owner 放行 PR #6074 的内核合并(human-only tier)。被门禁 diff:

  • kernel/src/parse-compress-input.ts:503 — UNRECOGNIZED_REF_SEP_RE 扩为含 \p{Script=Han}(CJK 词连接 ref 对:静默收窄 → 显式拒绝,不新增接受形式)。
  • 内核版本 0.0.200 → 0.0.201(kernel/package.json + kernel/package-lock.json + 根 package-lock.json,三账本一致)。

本轮相对 round 2 的唯一实质改动 = 内核版本 re-bump(master 已把内核推进到 0.0.200,PR 在其上重 bump);核心修复逐字节未变。机械 bug 豁免四要件齐备(provably-incomplete guard + 双向复现回归测试 + kernel-change: 声明 + 本线程报告)。

决策点(独立于 #6071 上待决的「接受集合是否扩到 到/至/→/dash 家族」A/B/C):是否授权本 kernel 变更合入。授权后合并仍由人工执行(我不合并 PR):#6074

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
第三轮复审完成 —— PR #6074(镜像 #6076),新 head 0249fbc11(round 2 为 604f6f296)。

相对 round 2 的变化:作者再次 rebase 到当前 master(新增两个 master-merge commit),head 现为对当前 master tip 的线性 rebase。核心修复(正则守卫 + preview + 2 内核测试 + 3 host 测试)与 round 1–2 逐字节一致;唯一实质改动 = 内核版本 re-bump 0.0.195→0.0.196 变为 0.0.200→0.0.201(master 期间已把内核推进到 0.0.200,PR 在其上正确重 bump,三账本一致)。master rebase 带进来的无关已合入改动不属于本 PR 贡献。

判定(不变):正确、范围内完整、测试双向有牙、diff 干净;触及 kernel/ 且带 kernel-change: → 不自批,转人工层。

已落位:

  • findings-only review(COMMENTED,非 approve,含中文八问)→ pullrequestreview-5980027864
  • 🔶 需要决策 评论(第三轮确认,结论不变)→ issuecomment-7016748208
  • needs-decision 标签 → 镜像 [PR] docs: add Japanese versions of repository documentation (#6072) #6076(已重新确认在位)
  • CI:5 项 success(gate/static-gates/mysql-contract 等);完整 test 矩阵审阅时仍在跑(12 pending / 0 failed)

等 owner 人工授权内核合并(合并由人工执行,我不合并);未开 issue。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷
澄清:上一条 🔶 需要决策 是我(review agent)第三轮复审的 🔶 评论经 upstream-sync 原样回显,并非 owner 的新决策——截至目前我未收到任何 owner 拍板。

状态不变:PR #6074 的内核变更(kernel/src/parse-compress-input.ts:503 正则守卫 + 内核版本 0.0.200→0.0.201)已复审通过,判定正确且范围内完整;触及 kernel/ → 按人工层等待 owner 授权合并(合并由人工执行,我不合并)。

唯一待办:owner 在 PR #6074 上拍板是否放行内核合并 → #6074

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

1 participant