Repository navigation
[PR] fix: make streaming tag-echo verdicts chunk-split independent (#4169/#4107) - #4210
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.5014-pr.4210.d1fa4e1This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr4210.tgz
npm install -g packageInstall instructions are refreshed on each push. |
…ag echoes take main-loop verdicts (#4169) (ai: vllm-qwen/qwen3.8-27b)
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复。 检索 open+closed issue/PR(tag-echo / nested bare open / streaming parity / stripAcpTags / #4107 / #4079),无同机制在途条目;本 PR 与 #4163(分块漂移)机制不同,#4169 正文已写明区分。 开始审核 PR #4210。初步状态:代码 diff 已拉取(单 commit,3 文件,+57/-3);CI 有 3 个 test 分片红—— |
…4169) The #4169 close-resolution re-adjudication left four more split-dependent shapes in the seeded verif domain; each is closed at its own decision point so streaming takes whole-text verdicts on the same bytes: - definite-tail prefix: force the dead-unit gate when a held tail is guaranteed to resolve into a recorded drop (third shape); - degenerate pair complete in the same buffer after an open: drop the unit whole instead of letting #1720 early-drop's buffer content decide it (fourth shape); - non-definite tail prefix: defer settled prose containing a dead unit (preHeld) until the tail resolves — prose tails keep the gate closed (#1760), tag tails open it (fifth shape); - bare-session EOF payload: re-run the main loop over it so complete structures that arrived while the session held take their normal verdicts (sixth shape); - degenerate unit ahead of a later acplike close: consume it before the loose-close resolution race (seventh shape). tests/verif-tag-echo-filter.test.ts: re-enable the full P1 domain and flip C#4107/C#4107-2 from pinned-divergence to assert-equal per the pin instruction. P5's duality detectors are aligned to removal semantics (column-0 marker anchor; word-glued residue preserved by design) and a documented exclusion covers the known-open #4106 gated-pass-ordering facet. (ai: vllm-qwen/qwen3.8-27b)
3f90a2a to
57dd887
Compare
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复(首条回复已述,维持)。 审核完成。审核期间作者把分支 rebase 到 master 并追加了第二个 commit( 结构化八问:
行为变更披露(旧 → 新):除 PR 正文表格列出的 #4169 形状外,第二个 commit 把更多输入类从「分块相关的任意输出」变为「whole-text 一致的确定输出」(dead 单位在各剩余形状下一致地死、prose 内引用的 ref 一致地活)。旧值 = 同一输入在不同切法下输出不同;新值 = 唯一输出且与 stripAcpTags 语义一致。这正是修复目的所在。 验证证据:本地在最终 head 跑全部 12 个 tag-echo-filter 相关测试文件(253/253 绿,node 22)+ typecheck 干净 + process()/flush() 全文精读(每个机制对照 whole-text pass 顺序逐一核对)。一个对后续有用的发现:C#4107 形状一的泄漏只在 Node 24 上表现(node 22 上 dead-ref attestation 根本不触发)——旧 pin 在 master 上是依赖 shard 分配的潜在 flake,已写进 pin 注释供后续参考。 协调说明:作者的第二个 commit 顺带关闭了 #4107 的已知形状(P1 全量域绿)——若日后还有未发现形状,#4107 可留观察,无需另立;#4220(#4164)与本 PR 无关、互不阻塞。审核期间我独立复现了同样的 pin 翻转(本地 WIP,与作者最终实现方向一致),未推送任何东西。 判定:CI 全绿、修的是根因(判决时机)、测试有牙(P1 全量域 + 逐 split 点硬断言 + pin 翻转 + 双方向门控)、diff 收敛在本目的相关的 4 个文件——可以人工合并:#4210 |
…tion (ranxianglei#4210) subsumes it, corpus 21/21
…anxianglei#6557) TOOL_TYPED_OPEN's tail [^<>]*> read a self-closing imitation (<acp type="tool:…"/>) as an OPENING, so the whole-text path made it own the rest of the field (or pair it with a later orphan close) while the streaming path killed it in place via SELF_CLOSE — a whole-text/streaming divergence on the same input, the ranxianglei#4178/ranxianglei#4210 split-invariance class this channel exists to prevent, and a loss of the prose that follows the imitation. Exclude the self-closing form from the opening match (the character before the final > must not be /); it then falls through to the SELF_CLOSE pass, killed in place — the same verdict streaming already reaches, so both paths agree again. A refs-only body with a non-acplike degenerate close still owns the rest of the field in both paths: the corrupted close is not the span's terminator, consistent with the wrapped-turn EOF rule. +4 regression tests: self-close dies in place with following prose surviving (whole-text, both shapes), streaming split-invariance for the self-close shape, and a guard that a type="text" self-close keeps its following prose (SELF_CLOSE semantics unchanged). Reviewer fix-up on the PR branch: found by whole-text vs streaming differential repro during review; one-line regex change, no state-machine change.
Closes #4169
Fixes #4107
修什么
#4169:streaming 过滤器(createTagEchoFilter)与整段参考实现(stripAcpTags)对同一字节给出不同结果——attrs payload 里嵌套的裸开标签在 streaming 存活、整段剥掉。按 #4107 pin 的指示重开完整 verif 种子域后,同机制家族(流式判决依赖分块位置 / 哪个闭合赢得解析竞争)又暴露出更多形状。本 PR 在各自的判决点关闭全部形状,让 streaming 对同一字节取整段判决。
行为变更(旧→新,全部 = streaming 对齐整段)
A <acp tokens="1">junk<acp>text</acp>B.旧流式保留内层裸开 → 新A junktextB.(= 整段)PAIRED正则吞掉最多 64 个字符的助手正文 #1720 early-drop 依赖缓冲内容)不变项(既有钉逐字节验证):refs-only body 整体 drop、attested imitation 丢弃、prose body 内引用的 ref 存活(#1720)、EOF 未闭合 attrs 尾 = tag 内容(#644)、word-glued token 保全(#1760/#2862)、marker 行 column-0 锚定语义、artifact 计数(#2405c)。
测试
stripAcpTagson the same text will additionally remove the dead refere | 标签清理不幂等:同一文本第二次 stripAcpTags 会额外删掉死引用闭合(m3117</cPa) #4106 gated-pass-ordering 面(已在 Tag cleanup is not idempotent: a second call tostripAcpTagson the same text will additionally remove the dead refere | 标签清理不幂等:同一文本第二次 stripAcpTags 会额外删掉死引用闭合(m3117</cPa) #4106 单独报告,不在此修)与其他在途 PR 的关系
stripAcpTagson the same text will additionally remove the dead refere | 标签清理不幂等:同一文本第二次 stripAcpTags 会额外删掉死引用闭合(m3117</cPa) #4106 修复:输入侧 attestation 门控 + 有界不动点):本 PR 不动 stripAcpTags,无机制冲突;若 [PR] fix(#4106): tag cleanup is now idempotent — one call reaches the fixed point #4148 先合入,本 PR P5 的 gated-unit 排除即冗余(rebase 时删除),C#4106/P2 区域以 [PR] fix(#4106): tag cleanup is now idempotent — one call reaches the fixed point #4148 为准<acp …>标签会漏到用户可见输出(附根因、量化与修法建议) #2190)语义不变,本 PR 补的是它们未覆盖的残留形状