Repository navigation
[PR] fix: keep ordering-repair separators out of compression range analysis - #5143
Conversation
…ession persistence (#4599) The four native-lane e2e suites SIGTERM the hermetic proxy and then wait a fixed 500ms before reading persisted sessions from disk. That races the product's own shutdown order: SIGTERM -> server.close() drains keep-alive connections first (SHUTDOWN_DRAIN_GRACE_MS = 10s backstop) and only then flushAllSessions() runs, so on a slow socket teardown the 500ms read lands pre-flush and the suite flakes ("proxy must persist the session"). Seen in the e2e-native-opencode scheduled lane at 5594c13 (run 40751086427 red, same-sha rerun green). Process exit implies flushAllSessions completed, so waiting for the recorded instance pids to actually exit makes the disk assertions deterministic: - e2e-native-opencode / e2e-subagent-sessions: reuse the file's existing awaitProxyExit() bounded poll (20s budget > 10s drain grace). - e2e-native-pi / e2e-native-dsh: extract liveInstancePids() and poll it until empty (same 20s budget). e2e-pi-codex-ws's fire-and-forget variant is untouched: it only runs in teardown hooks and gates no disk assertion. Verified locally: tsc --noEmit clean; the previously flaking subtest passes 4/4 consecutive hermetic runs (zero tokens). Full-suite verdict: remote CI.
#4599) Master red at 1fad21d (run 40852517737, 6 shards): two mechanisms, both in the #5107/#5115/#5120 hermetic data-root cluster. 1. preload-test-env.mjs pinned BILI_CONFIG_FILE to an absent file. configFile() (src/paths.ts) consults that env BEFORE the XDG default, so the pin shadowed every fixture that re-points XDG_CONFIG_HOME at its own dir and writes the default config path: the claude-native attach-gate / lane-priority / policy / url file-layer assertions failed on all shards, and the #3436 compat checks resolved mitm.domains=[] -> spawn path -> 60s fake-child timeout. The XDG_CONFIG_HOME pin already keeps the default config path inside the throwaway root; DELETE the env instead of pinning so fixtures are un-shadowed AND a developer's ambient pin is dropped. The isolation probe now asserts configFile() resolves UNDER the sandboxed XDG_CONFIG_HOME (same property, sibling-check style) instead of equality with the removed pin. 2. test-suite-lock.mjs snapshotSessionsTree keyed entries by raw path.relative(): Windows yields 'anthropic\\s.json' while the diff, the banner and the tests all compare '/'-separated paths (4 failures on the windows s2 shards). Normalize keys to '/' at the source. Verified locally with the CI preload chain (targeted): claude-native + #3436 + test-suite-lock suites 96/96 pass, 0 fail. Windows separator behavior holds by construction (split(path.sep).join('/')); full verdict = remote CI.
#5141) repairResponsesAssistantOrdering inserts acp_turn_sep_* user markers into the Responses SEND view only; they never exist in the original history (the apply view). liveCompressionRanges / uncoveredHistoryRuns / the post-compress recovery path re-run processTurn on the send view while screening candidates against the apply view, so any candidate unit that endpoints on a marker ref throws BoundaryNotFoundError in projectUnit and screenRecommendedUnits demotes it to a gap silently — large amounts of offerrable history vanish from the acp_status offer ("little left to fold" while ~180K was recoverable). Analysis entry points now strip outbound-only markers before processTurn (withoutTurnSeparators, zero-copy when absent). Real-message refs are id-stable across views, so every offered endpoint resolves in the apply view again. Ordering repair, tool pairing and turn-integrity protection are untouched — only the analyzed array class changes. Behavior change: ranges previously dropped behind separator endpoints reappear in acp_status / compress-receipt offers; the "little left to fold" pressure note stops firing while a large pending mass is actually offerrable.
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.6778-pr.5143.bcf580eThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr5143.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重: 不重复 — open/closed issue 与 PR 中均无修复此缺陷(排序修复分隔符污染压缩范围分析)的在途工作;最近邻 #5141(cold fork snapshot 恢复,不同机制)、#4266(fork 家族)均不重叠。 标签: bug + codex + scope/kernel + p2(Responses 车道 ordering-repair 分隔符静默吞掉 acp_status/compress offer — 功能性缺陷;prepare 自动压缩路径不受影响、无数据丢失/缓存泄露 → p2)。 编号澄清(避免找错 PR):本镜像 issue 对应 GitHub PR #5143( CI 初查(单次拉取):静态门禁全绿(kernel-guard / config-surface-guard / lane-guard / lock-sync / docs-sync / build-artifact…);test 分片:windows-22 的 shard 1、2 已红,其余仍在跑。完整结论随结构化 review 跟进。 |
(ai: vllm-qwen/qwen3.8-27b) 审核结论(结构化八问)已独立拉取分支通读全部 6 个文件的 shipped bytes(非照抄 PR 描述),并清点所有
测试有效性(④):4 个 pin 全部有牙——复现前置条件双向断言(有标记会话断言标记确实进了发送视图;控制组断言确实没有,#3617 门的双向纪律)、验收契约为「每个 offer 端点必须在 apply 视图可解析」、质量覆盖断言(mid1..o3 / f1..o3 span)、双表面一致性断言(offer 与 UNCOVERED 审计不得互相矛盾)。作者另做了修复前验证:临时回退调用点后 offer 坍缩到分隔符前的头部( 代码整洁(⑤/⑤b):命名达意、单一 helper 三处复用无重复、复用既有前缀常量无魔法串、注释讲原因且带 issue 出处、缩进风格随各文件既有惯例、无新依赖、无死代码。全项过。 CI 状态:首 head 遗留提醒(首条回复已提,再强调一次):PR 正文 判定:代码审查通过,§7.4 自动合入门禁各项满足(单机制 fix、回归测试钉住原症状、不触 kernel/config/wire/persistence/ref-id 逻辑),待全量 CI 矩阵转绿即可合并(合并动作仍归人)。 |
(ai: vllm-qwen/qwen3.8-27b) CI 终判:head 收工判定:可合并(合并动作归 owner)。合并前唯一待办仍是上一条提醒的编号问题——正文 |
…ments cite #5139, not #5141 Owner correction on #5144: the Codex separator-endpoint bug report is GitHub issue #5139 (kuviki), not #5141 (an unrelated in-flight external PR). Renames tests/issue5141-turn-separator-ranges.test.ts -> tests/issue5139-turn-separator-ranges.test.ts and re-points every in-code citation. No behavior change: comments, test names/session-id prefix only.
(ai: vllm-qwen/qwen3.8-27b) 已核实并修正:GitHub #5139 = @kuviki 的 Codex bug 报告「临时分隔符被选作压缩端点,acp_status 静默漏报 ~180K」(open),确为本 PR 的真正目标 issue;#5141 是无关的在途外部 PR。已做两处处理:
说明:原始 fix commit 的 subject 里仍带 新 head |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.912-master.58e61a3 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #5139
Root cause
repairResponsesAssistantOrdering()(src/server.ts) insertsacp_turn_sep_*user markers into the Responses send view only to keep assistant ordering legal upstream; they never exist in the original history (the apply view).The status/recovery entry points analyze the send view while screening candidates against the apply view:
liveCompressionRanges()(src/acp-status.ts):processTurn({ messages: <send view>, applyView: <original> })uncoveredHistoryRuns()(src/uncovered.ts) — same wiring via the acp_status audit sectionpostCompressTail)During that analysis run the markers get transient refs, and the segmenter can start a candidate unit at a marker (a marker is a user message; user messages start new groups). Screening then runs
projectUnit()against the apply view, where the marker id does not exist →BoundaryNotFoundError→screenRecommendedUnits()demotes the whole unit to a gap silently. Every range behind such an endpoint disappears from the offer while the uncovered-history audit still shows it going upstream raw — exactly the reported split (~182K uncovered vs ~12K offered, "little left to fold"), while prepare-time logs correctly saw pendingT1≈180K because prepare runs on the original history without markers.Fix
Strip outbound-only markers from the analyzed array before processTurn — new
withoutTurnSeparators()in src/turn-separator.ts (zero-copy when absent), applied at the three analysis entry points above (+ re-export from session.ts). Real-message refs are id-stable across views, so every offered endpoint resolves in the apply view again. Matches the reporter's offline experiment: removing the separators from the analysis view restored 10 ranges / 199,434-token candidates.Untouched: the ordering repair itself (send wire unchanged), tool pairing, turn-integrity protection,
kernel/, config surface.Behavior change (intended)
Tests
New
tests/issue5139-turn-separator-ranges.test.ts(4 pins, wired exactly like the relay: send view asmessages, original ascompressMessages):withoutTurnSeparators(no-op zero-copy + strip);Pre-fix verification by temporarily reverting the call sites: the offer collapses to the pre-separator head (
["m00001-m00003"]) — the reported symptom reproduced. Related suites green locally (issue564 ordering, issue2627 sep coverage, issue4331 uncovered, acp-status*, compress-recovery-ranges, ccr-v2, marker-render-contract, search-plan-aware, …); full-suite verdict on CI.