Skip to content

fix: dead-ref tombstones — stop the self-amplifying compress failure loop (#2362) - #2370

Merged
ranxianglei merged 7 commits into
masterfrom
2026-10-08_dead-ref-tombstones
Oct 9, 2026
Merged

ranxianglei merged 7 commits into
masterfrom
2026-10-08_dead-ref-tombstones

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

kernel-change: dangling-endpoint detection, gate receipts and the reachability tombstone live inside the kernel compression pipeline (boundaries + applyCompression); per-ref reachability is kernel-owned state and cannot be expressed from the host side.

Closes #2362

What this fixes

A compress request whose endpoint ref is known to the session but backs no visible or folded message (client history rewrite, host-native compaction) failed with "…leaving your old refs dangling. Run acp_status, then call the compress tool again using only the refs it reports." — inviting a retry that can never succeed, while every failed attempt added more hidden orphan refs (hiddenOrphanRefs). Net effect: the same range stayed permanently un-compressible and the diagnostics self-contradicted ("unknown ranges in request=0/1" while telling the model to retry).

Mechanism

Kernel (v0.0.108) — master had already advanced the kernel to 0.0.107 via other merged PRs; this branch bumps to 0.0.108.

  • New optional state field deadRefs?: string[] — tombstone-with-clear: marked by applyCompression when a requested endpoint dangles (ref known, no visible message, no active block covers it); cleared per-ref by processTurn once the backing message reappears inbound; refs are NEVER removed from messageRefs (kernel contract: numbers never re-issue).
  • Failure receipts name the dead refs and forbid retrying: "These refs are now recorded as DEAD: no range that includes them can compress now or later, whatever the neighbors. Do not retry this range in any form — run acp_status and target only the live refs it reports."
  • Mixed-batch skip warnings say DEAD instead of "already compressed" when the refs genuinely dangle; covered refs keep the old verdict verbatim.
  • Gate-rejected batches return the cloned state (was: the original input state) so the marking survives; cloneState carries deadRefs through.
  • Carry-through completeness (found via CI on this branch): syncBlocks (kernel/src/sync.ts — runs every request as pipeline node Use kernel renderTags:"text-only" + forbid tag echoing in prompt #3) now carries deadRefs; without it the tombstone died at the next request boundary and the model re-learned each dead ref by failing again.

Host

  • compressibleSpanHint (src/stream.ts) subtracts state.deadRefs from the advertised live span (multi-span output; all-dead branch drops the "Retry NOW" push). No-dead output is byte-identical to before.
  • handleAcpStatus (src/acp-status.ts) appends a DEAD REFS — … section when non-empty — receipts point the model at acp_status, so the dead set is visible on the canonical steering surface.
  • Persistence & fork carry-through: mergeState (src/persist.ts, restart hydration) merges deadRefs; the subagent fork (src/plugin.ts) inherits it with inverse-prefix semantics (a dead ref whose message IS in the forked prefix revives; refs outside it stay dead).
  • Machine-readable refusal reasons: ProxyToolResult.reason set at every compress refusal site (parse:{kind} / gate:{cannot-anchor|already-compressed|none-resolved|too-small|other} / exception) and carried onto the [plugin] … executed via plugin (… outcome=…, reason=…) execution log line — previously outcome=refused never said why on the anchor line (compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 log-sweep finding F2).

Behavior changes (old → new)

  1. Dangling-endpoint error: "…the message content changed (or the message was filtered out of the view) and now carries a new ref, leaving your old refs dangling. Run acp_status, then call the compress tool again using only the refs it reports." → "…refs R are known to this session but no longer back any visible or folded message — the client rewrote or dropped those messages (an edit reissues a new ref; host-native compaction or a bulk history rewrite drops them outright), and no active block covers them. These refs are now recorded as DEAD: no range that includes them can compress now or later, whatever the neighbors. Do not retry this range in any form — run acp_status and target only the live refs it reports." (substrings "cannot be anchored" and "no active block covers them" preserved)
  2. Mixed-batch skip warning for a genuinely dangling consumed range: "Skipped range (X..Y) — already compressed …" → "Skipped range (X..Y) — its refs R are DEAD (the client rewrote or dropped those messages); this range can never compress, do not retry it."
  3. New optional persisted state field deadRefs (additive; old sessions load fine; auto-clears when the messages reappear; now also survives restart hydration and subagent forks).
  4. A compress call rejected at the min-range gate now returns the cloned state instead of the original input state — the only delta is newly-marked deadRefs.
  5. acp_status gains a DEAD REFS section only when non-empty; receipt live-span hints subtract dead sub-spans (byte-identical otherwise).
  6. [plugin] execution log line gains , reason=<class> on refused compress calls (log-only; applied/partial outcomes unchanged; all e2e assertions use prefix regexes — verified compatible).

No config-surface change, no env vars, no wire-format change, KEEP_LAST_ORPHANED unchanged.

Commits (rebased onto master@625989406, which includes merged #2364)

  1. 2881083e8 kernel: record dead refs for unreachable endpoints and stop the retry-amplification loop (compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362) — types/compress/tests, kernel 0.0.107 → 0.0.108
  2. f777b5a5a fix: surface dead refs in acp_status and carve them out of the live-span hint (compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362)
  3. bf978cb97 fix: carry a machine-readable refusal reason onto the [plugin] execution line (compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362)
  4. acee3c3f8 fix: carry deadRefs through syncBlocks, restart hydration and subagent forks (compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362) — found when CI's targeted suite caught the per-request drop

Conflict resolution during rebase was textual union of intent (no semantic overlap with #2364's guard/receipt changes).

Tests

  • NEW kernel/tests/dead-refs.test.ts (6 tests): marking + receipt wording; retry stability (dead set does not grow); tombstone lift on full/partial revival; mixed batch (live half folds, dead half warns DEAD); covered-not-dangling guard keeps "already compressed".
  • Updated acp_status 缝合报告:Compressible ranges/Nudge 读冻结的 ctx.nudge,压缩成功后旧区间与新块同框并存 #387 drift test in kernel/tests/compress.test.ts to assert the new receipt wording + deadRefs.
  • Extended tests/compress-span-hint.test.ts: dead-carved multi-span, all-dead branch, receipt embedding.
  • Post-rebase targeted runs green on the new head: kernel 48/48 (dead-refs + compress + orphan-endpoint-snap + regression-bili-470), host batch 94/94 (span-hint + acp-status + estimator + issue1307-side-snapshot + fold-reconcile + plugin lanes); root typecheck clean; build clean. Full suites run in CI.

Relation to #2360 / #2388

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-08_dead-ref-tombstones (d9ce7ad)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2645-pr.2370.f5f8297

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2370.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 已完成对 PR #2370(dead-ref tombstones,#2362)的审查。这是 kernel 变更(v0.0.105→0.0.106),按 Kernel Boundary 属人工门禁、禁止自动合并。

结论:暂不可合并 —— 1 个机械性阻塞(需 rebase + 重定版本)+ 1 个真实回归 bug + 1 处披露不实。以下按七问作答。


① 是什么问题
一个 compress 请求,其端点 ref「本会话认识」但已不对应任何可见/折叠消息(客户端改写历史 / host-native compaction 丢消息)时,报错却引导模型「再试一次」——而这次重试永远不可能成功;每次失败还新增 hiddenOrphanRefs,使同一区间永久不可压缩且诊断自相矛盾(自放大失败循环)。

② 什么原因导致的
根因在 kernel:它没有「某个 ref 永久不可达」的概念。悬空端点一律报「内容变了,重试」;且 min-range/gate 拒绝路径返回的是原始 input.state(丢弃副作用),没有任何持久化「别再试」信号可累积。分层:#2360(DSH native compaction 在 bili 视野外落地)是初始触发(产生悬空 ref);本 PR 修的是放大器(错误判定 + 状态不持久),这一层选得对——无论哪个 host 丢了消息都成立。

③ 怎么修复的
kernel 引入持久化的按-ref 可达性墓碑 deadRefs(tombstone-with-clear):端点悬空时由 applyCompression 标记;processTurn 一旦该 ref 的消息重新出现在入站视图即按-ref 清除;ref 本体永不从 messageRefs 删除(守住「id 永不复用」契约)。所有 receipt + acp_status 改为报 DEAD 并禁止重试;gate 拒绝的批次返回克隆态以保留标记;host 侧把 dead 子区间从广告 live span 中挖掉,并在 acp_status 增加 DEAD REFS 段。

④ 是否有回归问题 —— 有,三处(按严重度):

  1. [阻塞·机械] 版本碰撞:master 已由 分析缓存泄露问题又一个日志 #2302(c979301a5)把 kernel 提到 0.0.106,本 PR 也 bump 到 0.0.106。rebase 到当前 master 后,版本 hunk 变成空操作 → rebase 后的 kernel 提交含 kernel/src 改动却无版本 bump(相对新父提交),kernel-guard Rule 2(ci.yml:112)必然红。已在临时 worktree 实证(rebase 干净应用、无文本冲突,但版本行消失)。须 rebase + 把 kernel 重定为 0.0.107 + 重新生成 package-lock.json(现 lock 还是 0.0.104→0.0.106 的陈旧跳变,掩盖了 desync)。
  2. [真 bug·中] src/stream.ts compressibleSpanHint 的「全 DEAD」分支误触发:当 (boundary, highest] 内没有 live ref 时进入该分支——这既包括「全部 DEAD」,也包括「全部落在 active block 里(boundary≥highest、零 dead)」这种正常折叠态;后者会被误报成 "every ref … is DEAD … can never compress"。调用点 src/stream.ts:472(无条件)与 :570(存在 active block 时)均可达。违反 §7.3「Honest output(fix: resolve issues #150 #151 #152 #154 (24h issue batch) #155):退化态不许发误导性信息」,也打破 PR 自己「No-dead output byte-identical」的承诺;现有测试未覆盖此边界。修法:把 DEAD 措辞用 dead.size > 0 门控;spans 空且 dead.size===0 时回退到中性提示(active block 之上无 live raw refs),绝不宣称 DEAD;补一条 no-dead/all-blocked 回归测试。
  3. [披露·低] Behavior feat: OpenAI Responses API support (/v1/responses) #4 描述不实:「gate 拒绝只多带 deadRefs」不准确——allocateRunId(state)(kernel/src/state.ts:41)会在 gate 之前(compress.ts:347)把 state.nextRunId +1,gate 拒绝现在返回克隆态会同时持久化 nextRunId+1。影响:每次被拒批次消耗一个 run 号 → 持久化单调计数器出现空档;runId 只是打在 block 上的唯一性 token,空档无害,但这是未披露的行为变化,PR 文案应更正(行为本身可保留)。

确认无回归的项:acp-status.ts 纯增量、deadRefs 为空时 byte-identical;covered-not-dangling 保持「already compressed」原话(有负向测试钉住);consumedRanges/danglingBySpec 谓词与旧的重算等价;id-never-reused 契约保留;KEEP_LAST_ORPHANED 未动;无配置/env/wire-format 变更。

⑤ 是否破坏缓存:否。deadRefs 只存在于持久化会话态,不改模型可见 wire(head/system 前缀字节稳定)、tool-call arguments、SSE 结构;前缀缓存不受影响(nextRunId 差异也只落会话文件,不上 wire)。

⑥ 是否改了设计方向:无架构转向。加一个加法式可选 state 字段 + 复用既有 receipt/acp_status 引导面。kernel-boundary 声明到位且理由成立(按-ref 可达性是 kernel 自有状态,host 侧表达不了)。与 #2360(初始触发)解耦得当。

⑦ 是否引入新配置:无。不加 env/config key/CLI flag(符合 config-surface 纪律)。


补充:

合并判定:不可合并(人工门禁)。 合并前必需:

  1. rebase 到当前 master + kernel 重定 0.0.107 + 重新生成 lock【阻塞】;
  2. 修 compressibleSpanHint 全-DEAD 误触发(用 dead.size>0 门控)+ 补边界回归测试【真 bug】;
  3. 更正 Behavior feat: OpenAI Responses API support (/v1/responses) #4 文案,披露 nextRunId 差异【披露】。

之后重跑 CI 至绿。

我可以在你确认后:(a) 立即推 (1)——纯机械、我已在临时 rebase 里备好;(b) 起草 (2)(3) 补丁供你审。(2)(3) 都动到 kernel-gated 代码 / 引导面行为决策,故等你拍板再动手。

@loalissm-cell

Copy link
Copy Markdown

Reproduced the reviewer's regression (2) from a second environment and it is not an edge case on this box — it is the common path. Cleaning that up first because it changes the priority of the fix.

Environment

  • proxy 0.1.187 (stale: false), 587 bili session files under ~/.local/share/billion-context/sessions/responses/
  • health: conflicts {sessions:0, events:0}, error/fatal 0

Measurement

Scanned all 587 sessions, reading payload.state:

quantity result
state.deadRefs non-empty 0 sessions
state.messageRefs size 2, in all 587 — p50 = 2, p90 = 2, max = 2
active blocks = 0 123 sessions
active blocks >= 1 464 sessions

Two things follow.

1. The "all DEAD" branch is reached on ordinary healthy sessions, not on poisoned ones. messageRefs holds 2 entries on every session on this box — including long-running ones (28–104 requests, 10–57 messages). So the live span (boundary, highest] is never the large window the branch was presumably sized for. Whenever the last two refs are both inside an active block — i.e. boundary >= highest — the span is empty and dead.size === 0. That is exactly the state a session sits in right after a successful fold, and the permanent state of the 123 sessions that never folded at all.

2. Therefore the PR's own regression promise does not hold here. "No-dead output is byte-identical to before" is true only if spans empty + dead.size === 0 is rare. Measured, it is the state the majority of sessions pass through every turn, and the only state of 21% of them. On this box the patch would replace the neutral hint with every ref ... is DEAD ... can never compress on the most-travelled path.

Victim ranking, which is the part I would weight: the two affected states are "clean session that never needed to fold" and "healthy session that just folded successfully". Both are good states. The failure direction is falsely reporting poisoning, which is worse than staying silent, and it lands on the users whose compression is working — the ones who would have no reason to distrust the message. That is the §7.3 Honest output concern the review already raised, but the blast radius is the whole box rather than a corner.

A regression case that would pin it

The review noted tests/compress-span-hint.test.ts covers no-dead only in the non-empty-span shape. Suggest asserting the empty-span/no-dead shape explicitly, in the negative direction:

  • input: spans empty, dead.size === 0, at least one active block covering the range
  • assert: output does not contain DEAD or can never compress, and is byte-identical to the pre-patch neutral hint
  • and a companion case with spans empty + dead.size > 0 to confirm the two shapes do not collapse into one string

I can supply the fixtures if useful.

On the two other release blockers

(1) version bump — no objection to rebasing and re-cutting kernel at 0.0.107; nothing on our side depends on the current value.

(3) nextRunId side effect — we measured this and it is inert in practice, so I would keep the behaviour and only correct the disclosure. On this box runId is consumed only as a per-block uniqueness token: the render path reads block.runId directly, there is no run-scoped range computation and no migration keyed on it, so gaps in the monotonic counter have no observable consequence. Worth stating in the PR text, not worth changing code for.

Thanks for catching (2) before merge — on this box it would have been the noisiest possible way to ship a correct fix.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Correction — I was wrong on item (2); withdrawing it.

Re-reading the actual diff (not just the receipt wording) overturns my earlier read. The "all-DEAD false positive" I reported does not exist:

  • The neutral early-return if (highest <= boundary) at src/stream.ts:98 ("No raw refs are directly compressible right now — compress a run of ACTIVE blocks…") predates this PR — it is byte-identical in master. So whenever the newest refs sit inside an active block (boundary >= highest), execution stops there and never reaches the new all-DEAD branch — regardless of how few entries messageRefs holds.
  • The new all-DEAD branch (src/stream.ts:123) is reached only when highest > boundary and the span builder yields zero live sub-ranges, which requires every ref in (boundary, highest] to be in deadRefs. With dead.size === 0 the builder always emits the full (boundary, highest] range, so the branch cannot fire. It fires only when the PR's own mechanism has genuinely marked those endpoints dead — the intended case, correctly worded.

So the escalation in the previous comment rests on the same misread: it assumes boundary >= highest falls through to the all-DEAD branch, but it hits the neutral early-return first. (Its underlying observation — messageRefs staying tiny under host-native compaction — is plausible and matches applyCompactionArchive pruning byRaw/byRef to live raw ids at src/session.ts:1016-1047; but that state routes to the neutral hint, not a false DEAD.)

Net effect on the "byte-identical otherwise" promise: it holds, with one exception — when exactly one ref sits above the boundary (boundary+1 === highest, no dead refs), old emitted mNNN–mNNN and new emits mNNN. Cosmetic cleanup, not a regression.

Corrected pre-merge list (item 2 withdrawn):

  1. [still blocking] version collision — master already owns kernel 0.0.106 (分析缓存泄露问题又一个日志 #2302); rebase + recut kernel at 0.0.107 + regenerate lock. Real and empirically proven.
  2. span-hint all-DEAD misfire — withdrawn; no such defect.
  3. [disclosure] Behavior feat: OpenAI Responses API support (/v1/responses) #4 text — gate-rejection return also persists nextRunId+1 (benign; keep behavior, fix wording). I agree with the commenter this is inert in practice — disclosure-only.

Verdict unchanged in kind: human-gated kernel PR, not mergeable until (1) lands; (3) is a one-line text fix. I hold for owner (@ranxianglei) sign-off before pushing anything.

Two notes for the record: (a) @loalissm-cell is flagged unverified/outside by the platform, so I re-derived every claim from source rather than taking either account's word; (b) thanks for pushing on this — it caught a real error in my first pass.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 上游冲突已解决并 push(rebase 到当前 master;未合并,内核改动 human-gated)

系统转发「merge state: dirty」。核对后确认是真实冲突:master 在本 PR base 之后又合入了 6 个提交——全部是 pi-subagents 入仓(#2384/#2386/#2380)+ 文档/CI。按「上游 PR 冲突唤醒」流程 rebase → 解冲突 → push。

结构化回答

  1. 是什么问题:PR 分支落后于 master、与目标分支产生合并冲突,需 rebase 到最新 master 才能合并。
  2. 什么原因导致:master 前进后,与本 PR 唯一交叉的文件是 package-lock.json——两边都改了依赖树(master 加 pi-subagents 条目,本 PR 改 acp-kernel 版本行),同区域文本冲突。其余 12 个源码文件零重叠。
  3. 影响范围与严重程度:仅机械性冲突;rebase 本身不改变任何运行时行为。严重度:无。
  4. 怎么修复的:把 4 个 PR 提交 rebase 到当前 master tip 081234441;唯一的 package-lock.json 冲突按「取并集」解决——保留 master 的 pi-subagents 条目 + 本 PR 的 acp-kernel 0.0.108。用 --force-with-lease(锁定到已验证的前置状态 acee3c3f8,不会误伤任何更新提交)推回分支。
  5. 是否有回归问题:无。rebase 后本地定向验证全绿:typecheck ✓、build ✓、定向测试 54/54 通过(dead-refs 6/6、compress、compress-span-hint、acp-status 全套);diffstat 恰好是本 PR 应有的 12 个文件(+403/-18),无夹带无关改动。(说明:我在一个过期的本地快照上曾看到 dead-refs 一条单测失败,经 A/B 核实真实 head acee3c3f8 上并不存在,属快照陈旧假象,非回归。)
  6. 是否破坏缓存:不涉及。rebase 未触碰缓存 / 重试 / 去重键逻辑。
  7. 是否改了设计方向:不涉及。仍是原 PR 设计(kernel dead-ref 墓碑 + acp_status 呈现 + span-hint 排除)。
  8. 是否引入新配置:不涉及。无新 env / config / 默认值。

版本披露(按行为变更披露要求):acp-kernel 版本 0.0.107 → 0.0.108。原因:master 当前 kernel 停在 0.0.107(#2377 随 v0.1.188 发布),本 PR 改了 kernel/src/**,按 Kernel Boundary 规则 2 必须 bump 到下一个空位 0.0.108。这是 kernel 自身版本号,非 bili 根包版本——根包仍为 0.1.188,未动。lockfile 相应记录 0.0.108。

当前状态:head 已更新到 17e5a4541;API 显示 mergeable: true、mergeable_state: blocked——冲突已清除(不再是 dirty),blocked 仅因 CI / 评审尚未完成,属预期。内核改动 human-gated,不自动合并,请人工合入;CI 会跑全量门禁(我本地只跑定向测试,遵循本地测试纪律)。

结论:冲突解决完毕,可进入 CI + 人工评审/合并流程。

@ranxianglei

Copy link
Copy Markdown
Owner Author

冲突 ci失败

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI 已定位:4 个 test job 全红 = bili 套件 3 条测试失败,全部在 PR 自身内容里(非本次 rebase 引入)

先排除嫌疑:我上次 rebase 只动了 package-lock.json,源码与 PR 原 head acee3c3f8 逐字节一致。CI 里内核套件 1268/1268 全绿、e2e/hermetic/build-artifact/各 guard 全绿,红的只有 bili 套件的 3 条。bisect 证明:base 625989406 全绿,PR 第一个提交 2881083e8(核心内核改动「record dead refs」)一上来就变红并持续到 head——即作者提交时未跑全量套件带出来的。

逐条根因

①② tests/issue2360-compress-receipt.test.ts(2 条)=陈旧断言,非回归
#2362 在内核改写了「cannot be anchored」gate 文案:旧 Run acp_status, then call the compress tool again using only the refs it reports.(大写 R)→ 新 dead-ref 文案 These refs are now recorded as DEAD … Do not retry this range in any form — run acp_status and target only the live refs it reports.(小写 run)。

③ tests/plugin-status-provenance.test.ts:168「HTTP failed manual tools do not publish a new context observation」=真回归,别改测试
断言:失败 manual tool(compress m99998..m99999 / decompress not-a-block)前后 [contextTokens, source, at, contextGeneration] 完全不变;实际只有第 4 项 contextGeneration 漂移。contextGeneration = forkHash({sessionId, sessionRevision, generation: observation.generation, tokens, source, limit, model})(src/plugin.ts:1394),前 3 项未变 → 漂的是 observation.generation。bisect 锁定是第一个内核提交 2881083e8 引入:失败的 compress/decompress 现在会 bump context-observation 的 generation。
这条测试名就是设计不变式:失败 tool 不得发布新 context observation(否则客户端把每次失败调用误判成「上下文变了」→ 多余重取/缓存失效)。这是 PR 核心内核改动「修 A 坏 B」的真回归——把测试改成能过=掩盖 bug(违反疏不堵)。修法=改代码:让失败路径不 mutate 喂给 observation.generation 的状态(大概率是 dead-ref 标记/某次 state mutation 落进了失败分支),而不是动测试。

结构化八问

  1. 是什么问题:rebase 后 CI 红,实为 PR 内容里 3 条测试失败(2 陈旧断言 + 1 真回归)。
  2. 什么原因导致:compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 内核改写 cannot-anchor 文案(①②)+核心内核提交使失败 tool bump contextGeneration(③)。
  3. 影响范围与严重度:①②仅测试层、无运行时影响;③影响所有 plugin 客户端的 provenance/缓存语义——特定条件下(发生失败 compress/decompress 时)的真实缺陷,会放大无效重取/缓存失效。
  4. 怎么修复:①②更新两条断言到 compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 新文案;③改代码让失败路径不 bump observation.generation(先定位 mutation 点,保住 dead-ref 功能)。
  5. 是否有回归:③本身就是本 PR 新引入的回归(修 dead-ref 循环时破了 provenance 不变式);①②无回归。
  6. 是否破坏缓存:③会——spurious generation bump 让客户端按「上下文变化」处理,破坏缓存/provenance 稳定性;这正是必须修代码而非改测试的原因。
  7. 是否改设计方向:否,仍是 dead-ref 墓碑方案;③是它实现层的 bug。
  8. 是否引入新配置:否。

建议与分工:这是 human-gated 内核 PR,且 ③ 动的是你内核里的真回归,我不擅自改方向。①②我可以立刻补两条断言(独立标注 commit);③想请你先定修法方向——我倾向「失败路径不落任何喂给 observation.generation 的 state mutation」,定位到具体行再动手。你点头我就把三条一起修好推上去;或你说「直接修」我就照此执行。

当前状态:head 17e5a4541 冲突已解(mergeable:true / blocked),CI 因上述 3 条仍红,合并前需先修。

@ranxianglei

Copy link
Copy Markdown
Owner Author

你找到ci红色根本原因并修复,rebase master最后报告状态

awork added 7 commits October 9, 2026 00:11
…-amplification loop (#2362)

A compress request whose endpoint ref is known to the session but backs no visible or folded message (client history rewrite / host-native compaction) used to fail with 'cannot be anchored ... leaving your old refs dangling. Run acp_status and call again' — inviting a retry that can never succeed while every failed attempt added more hidden orphan refs. Now such refs are recorded as DEAD (state.deadRefs, tombstone-with-clear: marked by applyCompression on a dangling endpoint, cleared per-ref by processTurn once the message reappears inbound, never removed from messageRefs so ref numbers stay un-reissuable). Failure receipts name the dead refs and forbid retrying the range; mixed-batch skip warnings say DEAD instead of 'already compressed'; rejected batches return the cloned state so the marking survives; cloneState carries deadRefs.
…pan hint (#2362)

Failure receipts point the model at acp_status, so the dead set must be visible on every surface that steers the next compress: compressibleSpanHint subtracts state.deadRefs from the advertised live span (multi-span output, all-dead branch drops the retry push), and handleAcpStatus appends a DEAD REFS section when non-empty. No-dead output is byte-identical to before.
…s in cloneState

applyCompression bumped nextRunId unconditionally and cloneState dropped lastPassIds, so a failed (zero-block) manual compress returned a state that differed only in internal bookkeeping. The host adopts res.state and markDirty()s after every tool, so forkSnapshot's parentRevision re-hashed and contextGeneration churned on every failed tool call (tests/plugin-status-provenance #3). Allocate the run id only when a block is actually created, and make cloneState faithfully preserve lastPassIds.
…receipt tests

#2362 renamed the cannot-anchor gate sentence but the armed-breaker scrub list kept only the old wording, so an armed receipt carried BOTH 'run acp_status and target only the live refs' and 'do not poll acp_status' - reintroducing the contradiction #2146/#2360 removed. Add the new sentence to KERNEL_RETRY_GUIDANCE and update the two stale assertions to match the new kernel wording.
master independently consumed 0.0.108 (#2378); recut so the branch's kernel/src changes carry a version delta vs master (Kernel Boundary rule 2).
@ranxianglei
ranxianglei force-pushed the 2026-10-08_dead-ref-tombstones branch from 14dd146 to d9ce7ad Compare October 8, 2026 16:14
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

#2370 已修复并 rebase 到最新 master —— 结构化八问

冲突已解、CI 红色的根因已定位并修复,分支已 rebase 到当前 master tip(d3be99116)并 force-push(head d9ce7ad34),无冲突。

  1. 是什么问题:两件事——(a) 上游 master 合并了 pi-subagents vendoring(billion-context-pi-subagents 直接纳入本项目 像acp-kernel一样 #2384/feat: vendor billion-context-pi-subagents in-repo under pi-subagents/ (#2384) #2386)、web-summary-card(fix: card-wrap the shared summary service section on the web config page #2430)等,与本分支在 package-lock.json、src/acp-status.ts 撞车产生 merge-conflict;(b) CI 红——3 个 bili 测试挂掉。

  2. 什么原因导致:

    • ①②(tests/issue2360-compress-receipt)= 陈旧断言:compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 把内核 cannot-anchor 指引句从大写 R Run acp_status… 改成小写死引用措辞,但测试仍断言旧文案;且武装断路器的 scrub 清单(src/stream.ts 的 KERNEL_RETRY_GUIDANCE)只留了旧句 → 武装态回执里同时出现「run acp_status」和「do not poll acp_status」两条矛盾指令。
    • ③(tests/plugin-status-provenance Use kernel renderTags:"text-only" + forbid tag echoing in prompt #3「失败的手动工具不得发布新 context 观察」)= 真回归:核心死引用提交让 applyCompression 在「没压出任何块」时也改了内部簿记——allocateRunId 在函数顶部无条件自增 nextRunId;cloneState 漏拷 lastPassIds。宿主在失败判断之前就采纳 res.state 再 markDirty,而 provenance 指纹对整份 session.state 做哈希,于是任何这类簿记漂移都被客户端当成「上下文变了」。我此前 review 把 nextRunId 自增当良性,低估了——它其实有真实消费者(provenance 指纹)。
  3. 影响范围与严重程度:③ 潜在中等——任何按 contextGeneration 指纹判定的客户端,在每次失败的 compress/decompress 后都会收到一次虚假「上下文变更」信号(可能触发无谓 refetch / 缓存失效),非数据丢失;② 中等——武装断路态下给模型发了互相矛盾的 acp_status 指令,正是 dsh内部问题 出现这个错误后,重启dsh可以发消息 但陷入无限压缩失败 #2146/bili 插件模式下 allowDshCompaction 在 responses 线路上完全失效 —— DSH 原生压缩落地、折叠基底被摧毁、compress 连片失败 #2360 刚消除的困惑,且恰好落在 compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 新增的死引用失败路径上;冲突本身阻塞合入。

  4. 怎么修:冲突 = rebase 到最新 tip(master 期间 081234441→c2dc81b79→d3be99116 动了三次,故做了两轮 rebase),package-lock 取 acp-kernel 版本、acp-status.ts 用 union-of-intent 保留双方改动。③ = 在生产者处根治:run-id 改为惰性分配(只有真正建块才调用 allocateRunId)+ cloneState 恢复忠实克隆(补拷 lastPassIds)→ 零块失败现在返回字节一致的 state,指纹稳定(state-diff 探针实测 pre/post 完全相同:nextRunId 1→1、lastPassIds 两侧都在、sessRev/obsGen 不变)。①② = 把改名后的死引用句补进武装 scrub 清单(恢复单指令行为)+ 3 处测试断言对齐内核新文案(并把那条「武装态应被 scrub」断言重新变回有牙)。版本:内核 0.0.108→0.0.109(master 已被 feat: 分层压缩触发阈值 compress.tierNudgeTokens(T1/T2/T3 独立设置) #2378 占用 0.0.108),lock 单行重生成。

  5. 是否回归:过了一遍既有行为——惰性 run-id 不改变真实建块的 runId 语义(dead-refs 6/6、compress 38/38 绿);cloneState 补 lastPassIds 纯增量(state-prune / handoff / ccr / token-snapshot / ranges-consistency / refold / orphan-fixes / regression-bugfixes 共 181/181 绿);scrub 清单只收窄武装态展示、不动非武装回执与 operator log;wire 形状不变(deadRefs 为增量可选字段)。高危模式逐项:无 fail-fast 变吞错、无阈值/时序漂移、输出格式变化仅限 feat: OpenAI Responses API support (/v1/responses) #4 披露的那句指引文案。残留未修(待你定夺):已知悬空端点的失败 compress 仍会写 deadRefs(它在被哈希的 state 里)→ 该特定场景仍可能扰动 parentRevision(现测试只覆盖未知 ref 场景);要彻底免疫需把纯内部字段(deadRefs/nextRunId/lastPassIds)排除出 forkSnapshot 哈希,但那触碰 fork 身份语义(plugin.ts PARENT_REVISION_CONFLICT),属你的决策范围。另注:cloneState 仍漏 imageFullRestored/imageShrinks(图像会话潜在数据丢失,超出本 PR 范围)。

  6. 是否破坏缓存:否。前缀缓存头部 system 消息未动;本修复净减少了指纹误抖(对缓存是正向);deadRefs 增量;重试/去重/游标语义未改。

  7. 是否改设计方向:否。compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible #2362 的死引用墓碑机制原样保留,只是不再让它泄漏进失败时的无关内部簿记,并把宿主侧 scrub/测试对齐内核改名后的措辞。

  8. 是否引入新配置:不涉及(无新 env/config/默认值)。

收工判定:本地 typecheck 干净、承重测试 26/26(provenance/receipt/span-hint/acp-status)+ 内核定向 38/38 + 181/181 全绿;全量回归 + e2e 交给 CI(按本地测试纪律未在本地跑全量)。这是 kernel 变更(Kernel Boundary:永不自动合并)——我不合并,请 review 后自行合并:#2370 。仓库合并节奏很快(本会话内 master 动了三次),若合入前 master 又前进,可能还需再一次机械 rebase,需要的话我来做。第 5 条的两个残留项若要立项跟进,我可以补带「来源」标记的 issue,等你点头。

@ranxianglei
ranxianglei merged commit 293c32d into master Oct 9, 2026
27 checks passed
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.165-master.293c32d (rolling master tag).

npm install -g billion-context@0.0.165-master.293c32d  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@ranxianglei ranxianglei mentioned this pull request Oct 9, 2026
@ranxianglei
ranxianglei deleted the 2026-10-08_dead-ref-tombstones branch October 15, 2026 05:32
ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
…drator

CompressionState clone/hydrate points rebuild via explicit field lists, so each new optional field silently drops wherever the list lagged (deadRefs hit three such points in #2362/#2370). Make that drift loud instead of silent:

Guard (two layers): a compile-time field inventory mapped over keyof CompressionState with optionality stripped (-?), so adding a field without listing it is a TYPE error, plus a runtime EXACT match between each point's actually-carried keys and its documented expected-missing set. Covers syncBlocks, cloneState, mergeCompressionState (kernel) and mergeState (host). The subagent fork is targeted inheritance, documented rather than asserted (needs the session harness).

Fix: mergeCompressionState (the standalone acp-kernel hydrator) carried neither hiddenOrphanRefs nor lastPassIds, so a real consumer lost them on restart. deadRefs (#2362) hit the same gap when it landed — the host mergeState carries all three, so the hydrator matches it. Bare passthrough like terminalStreak (none of them is seeded by createInitialState).

syncBlocks keeps its two documented gaps (hiddenOrphanRefs self-heals post-pipeline; lastPassIds is inert — consumed before it runs, re-stamped after); cloneState and both hydrators carry every field, pinned against regression.

Rebased onto master: #3266 removed imageFullRestored/imageShrinks from CompressionState, so inventories and gap sets track the post-removal field set; cloneState's original gaps were closed upstream by the #2362 work.

Bumps acp-kernel 0.0.126 -> 0.0.127 (kernel/src changed).
ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
…drator

CompressionState clone/hydrate points rebuild via explicit field lists, so each new optional field silently drops wherever the list lagged (deadRefs hit three such points in #2362/#2370). Make that drift loud instead of silent:

Guard (two layers): a compile-time field inventory mapped over keyof CompressionState with optionality stripped (-?), so adding a field without listing it is a TYPE error, plus a runtime EXACT match between each point's actually-carried keys and its documented expected-missing set. Covers syncBlocks, cloneState, mergeCompressionState (kernel) and mergeState (host). The subagent fork is targeted inheritance, documented rather than asserted (needs the session harness).

Fix: mergeCompressionState (the standalone acp-kernel hydrator) carried neither hiddenOrphanRefs nor lastPassIds, so a real consumer lost them on restart. Bare passthrough like terminalStreak (neither is seeded by createInitialState).

No runtime behavior change: the syncBlocks/cloneState gaps are left as-is; their safety proofs (self-heal / inert / deferred-eval) are documented in the guard. Carrying image*/lastPassIds there is deliberately deferred per the issue.

Bumps acp-kernel 0.0.108 -> 0.0.109 (kernel/src changed).
ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
…drator

CompressionState clone/hydrate points rebuild via explicit field lists, so each new optional field silently drops wherever the list lagged (deadRefs hit three such points in #2362/#2370). Make that drift loud instead of silent:

Guard (two layers): a compile-time field inventory mapped over keyof CompressionState with optionality stripped (-?), so adding a field without listing it is a TYPE error, plus a runtime EXACT match between each point's actually-carried keys and its documented expected-missing set. Covers syncBlocks, cloneState, mergeCompressionState (kernel) and mergeState (host). The subagent fork is targeted inheritance, documented rather than asserted (needs the session harness).

Fix: mergeCompressionState (the standalone acp-kernel hydrator) carried neither hiddenOrphanRefs nor lastPassIds, so a real consumer lost them on restart. deadRefs (#2362) hit the same gap when it landed — the host mergeState carries all three, so the hydrator matches it. Bare passthrough like terminalStreak (none of them is seeded by createInitialState).

syncBlocks keeps its two documented gaps (hiddenOrphanRefs self-heals post-pipeline; lastPassIds is inert — consumed before it runs, re-stamped after); cloneState and both hydrators carry every field, pinned against regression.

Rebased onto master: #3266 removed imageFullRestored/imageShrinks from CompressionState, so inventories and gap sets track the post-removal field set (and pick up deadRefs); cloneState's original gaps were closed upstream by the #2362 work.

Bumps acp-kernel 0.0.126 -> 0.0.127 (kernel/src changed).
ranxianglei pushed a commit that referenced this pull request Oct 15, 2026
…drator

CompressionState clone/hydrate points rebuild via explicit field lists, so each new optional field silently drops wherever the list lagged (deadRefs hit three such points in #2362/#2370). Make that drift loud instead of silent:

Guard (two layers): a compile-time field inventory mapped over keyof CompressionState with optionality stripped (-?), so adding a field without listing it is a TYPE error, plus a runtime EXACT match between each point's actually-carried keys and its documented expected-missing set. Covers syncBlocks, cloneState, mergeCompressionState (kernel) and mergeState (host). The subagent fork is targeted inheritance, documented rather than asserted (needs the session harness).

Fix: mergeCompressionState (the standalone acp-kernel hydrator) carried neither hiddenOrphanRefs nor lastPassIds, so a real consumer lost them on restart. Bare passthrough like terminalStreak (neither is seeded by createInitialState).

No runtime behavior change: the syncBlocks/cloneState gaps are left as-is; their safety proofs (self-heal / inert / deferred-eval) are documented in the guard. Carrying image*/lastPassIds there is deliberately deferred per the issue.

Bumps acp-kernel 0.0.108 -> 0.0.109 (kernel/src changed).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

compress failure is self-amplifying: failed calls are hidden as orphan refs, making the same range permanently un-compressible

2 participants