Skip to content

[PR] Guard open objectives against self-declared completion in compression summaries (#3538) - #3660

Open
ranxianglei wants to merge 5 commits into
masterfrom
2026-10-18_issue3538-objective-status-guard
Open

ranxianglei wants to merge 5 commits into
masterfrom
2026-10-18_issue3538-objective-status-guard

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

kernel-change: the write-side objective-status gap from #3538 lives in the kernel: (a) the compression prompt rules (compression-rules.ts) are the surface that tells the model how to carry and re-judge open objectives, so the "status change needs in-range evidence" gate can only be stated there — the host has no prompt surface; (b) the deterministic anti-loss backstop must run where summaries are stored/rewritten (compress.ts applyRefolds / covered-span paths), which is also kernel. The host-side receipt reminder ships as a separate non-kernel commit.

(acp-kernel version: 0.0.131 → 0.0.132 on top of current master.)

What this fixes (#3538)

In a long pi-native session on v0.1.197, a model's compress summaries kept the Open objectives: heading but rewrote a still-open acceptance criterion into "delivered" / "unreachable per current analysis", after which the assistant concluded "All work is delivered and committed." and closed out the task. Root cause (verified at tag v0.1.197): that build already carries #2918's read-side wording ("objectives remain binding until the user completes, cancels, or supersedes them"), yet nothing on the WRITE side constrained who may change an objective's status — the model's own "unreachable" analysis counted as a valid completion mark, and no pre-store check existed for dropped objective entries.

Three parts, all minimal:

  1. Prompt gate (two sentences total, per owner instruction to keep prompt edits lean) — kernel/src/compression-rules.ts:
    • T1 rule: a status change needs in-range evidence — the user confirmed completion, the acceptance criteria were met, or the user cancelled/superseded the objective; the model's own "unreachable/blocked" analysis is NOT closing evidence → record it as blocked (needs verification) and keep the entry.
    • T2 rule: an "unreachable/blocked per analysis" note is not a completed-or-superseded mark.
  2. Deterministic anti-loss backstop — kernel/src/compress.ts: when a distillation (T2/T3) or refold drops a ref-backed Open objectives: line that a consumed source block still carried, the original line is appended back into the stored summary verbatim and the receipt warns naming the refs ("…remain open: do not treat this fold as completing them"). Strictly deterministic subset — a ref present anywhere in the new summary passes untouched (no semantic judgment), whole-line append avoids fabricating text, and it runs post-hoc after length validation.
  3. Receipt reminder — src/stream.ts (host): every successful compress receipt gains one static line: "Compression only folds history — it completes no task. Before declaring done, re-check every still-unmet acceptance criterion in your Open objectives: entries." Model-facing receipt only; the log copy stays unchanged (日志清理隐私信息 #1718 excerpt-free log invariant). Plugin and proxy lanes both funnel through this path (external-summary lane included).

Behavior changes (old → new)

Surface Old New Why
Successful compress receipt header + fingerprint lines + #2544 steering (+ #1387 stop tail) + one static task-state line a completed fold was misread as a completed task (#3538 step 4)
Kernel defaultPrompts no evidence requirement for objective status changes + two sentences (T1 + T2) write-side gap: model analysis could close an objective
Distill/refold storage a model-dropped objective line was silently lost dropped ref-backed lines appended back verbatim + named receipt warning deterministic floor under the prompt gate
Prompt fixtures + wire-contract golden pinned old prompt text regenerated — defaultPrompts changed by exactly those two sentences; kernel.json diff confined to the howToCompressRules/tier2 text goldens pin the behavior surface; regeneration justified by this PR
Nudge text on the wire rules embedded in the nudge carried the old wording nudge now carries the same two sentences (~1 KB constant offset per nudged request body — verified uniform per-turn byte delta in the regenerated goldens) renderNudgeText embeds howToCompressRules/tier2DistillRules; one-time prefix-cache invalidation for sessions receiving a nudge after upgrade (same class as #2918)
Storage-render goldens (4 wires) pinned pre-change turn shas/bytes/state digests regenerated under the suite's pinned BILI_IMAGE_RESCALE=0 shape; diff is sha/byte/digest lines only, no structural change the nudge lives in persisted session state and every post-nudge wire body; deterministic consequence of the prompt change
Nudge text on the wire rules embedded in the nudge carried the old wording nudge now carries the same two sentences (~1 KB per nudged request body, constant offset — verified uniform per-turn byte delta in the goldens) renderNudgeText embeds howToCompressRules/tier2DistillRules; one-time prefix-cache invalidation for sessions that receive a nudge after upgrade (same class as #2918)
Storage-render goldens (4 wires) pinned pre-change turn shas/bytes/state digests regenerated with BILI_IMAGE_RESCALE=0 (the suite's pinned shape); diff is sha/byte/digest lines only, no structural change the nudge lives in the persisted session state and every post-nudge wire body; deterministic consequence of the prompt change

One-time prefix-cache invalidation on upgrade (same class as #2918, ~500 B). No new config/env keys. No changes to refusal semantics, message-ref/id logic, or persistence format.

Tests

  • NEW tests/issue3538-post-compress-receipt.test.ts — success receipt carries the note, log copy does not, refused receipts do not.
  • EXTENDED kernel/tests/open-objectives.test.ts — distillation dropping a ref → line carried back verbatim + warning names it; keeping the ref → summary byte-identical, no warning; in-place refold → same guarantee. Nudge text now also pins the evidence rule.
  • UPDATED pin — tests/compress-fingerprint-refold.test.ts exact receipt-shape assertion includes the new static line.
  • Local: kernel component suite 1297/1297 green, targeted host files green, typecheck clean. Full-suite verdict = remote CI on the pushed head.

Not closing #3538: the P1 sub-item (real-model eval lane measuring objective-preservation rate across DeepSeek Flash / GPT / no-compression baseline) remains as a follow-up tracked in that issue; the GPT-vs-DeepSeek contrast stays an uncontrolled observation until then.

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-18_issue3538-objective-status-guard (a9e7ef4)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4407-pr.3660.491323d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3660.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title Guard open objectives against self-declared completion in compression summaries (#3538) [PR] Guard open objectives against self-declared completion in compression summaries (#3538) Oct 18, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-18_issue3538-objective-status-guard branch 2 times, most recently from abe5b71 to 9199591 Compare October 18, 2026 07:48
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复 — 本 PR 修 #3538(压缩摘要里模型自判“完成”导致任务被提前收尾)的写侧缺口。open/closed 里没有同症状同机制的并列条目(#3592 是 pi compress carrier identity,不同问题)。

标签: bug pi scope/kernel p2(缺陷、复现于 pi-native、核心改动在内核、可用但有明显正确性问题 → p2)。已打好。

我独立审了 diff(没有照抄 issue 结论),并把改动触及的整个爆炸半径在本地跑绿了。先给结论:代码与测试本身没毛病,可合的前提只剩一次干净的 CI。

结构化八问

1. 是什么问题
长会话里,压缩摘要把仍开放的验收标准改写成了“已交付 / 按当前分析不可达”,模型据此宣布 “All work delivered” 提前收尾。读侧其实已有措辞(#2918 “objectives remain binding until…”),但写侧没有任何东西约束“谁能改一个 objective 的状态”——模型自己的“不可达”分析被当成了有效完成标记,而且落库前没有对“丢掉的 objective 条目”做任何校验。

2. 什么原因导致的
因果分三层:①提示词只说“要把开放目标带着走”,没说“状态变更需要什么证据”;②落库路径(distill/refold 用新摘要覆盖旧块时)对模型是否真把 ref-backed 的 Open objectives: 行带过来零校验;③折叠完成的回执本身长得像“任务完成了”。PR 三层正好对上三个洞:提示词证据门(T1+T2 各一句)、内核确定性兜底(ref 缺失就逐字回填 + 点名告警)、成功回执加一句静态提醒。方向打到根因,不是堵症状。

3. 影响范围与严重程度
触发条件:长会话 + 发生压缩 + 模型倾向提前收尾(复现于 pi-native,但机制在内核、全客户端通用)。严重度 p2:不是不可用,是特定条件下会错误地宣告任务完成(没做完的说做完了)。属于“能用但有明显正确性问题”。

4. 怎么修的

  • 提示词(kernel/src/compression-rules.ts):T1 加“状态变更需范围内证据——用户确认完成 / 验收达标 / 用户取消或取代;你自己的‘不可达’分析不算关闭证据,记为 blocked (needs verification) 并保留条目”;T2 加“‘unreachable/blocked per analysis’ 不算已完成标记”。
  • 内核兜底(kernel/src/compress.ts):openObjectiveCarryLines 抽出源块 Open objectives: 行上的每个 mNNNNN ref;替换后的新摘要里缺哪个 ref,就把那一整行逐字追加回去,并推一条点名 ref 的告警。刻意只做“ref 是否存在”这一维——ref 只要出现就不动(存在 ≠ 语义上已关闭,状态措辞归提示词管),整行追加避免伪造文本。落在两条覆盖路径:in-place refold(applyRefolds)与 distillation/merge(applySingleRange)。
  • 主机侧(src/stream.ts):成功回执加一行静态提醒,仅模型可见;日志副本保持无摘录(日志清理隐私信息 #1718 不变)。plugin 与 proxy 两 lane 都走这条 applyRanges 成功分支。

5. 是否有回归
我把爆炸半径整段跑绿,逐项过了既有行为:

  • 成功回执形状:多一行静态提醒 → 已在行为表披露(old→new)。本地跑了全部 38 个 applyRanges 消费方文件(284 用例)+ 6 个 nudge/prompt 敏感文件(67 用例),全绿——没有“精确断言回执全文 / 行数”的用例被这行打破(失败态回执不受影响,新行只在成功分支)。
  • defaultPrompts:多那两句 → golden tests/golden/wire-contract/kernel.json 已重新生成,diff 严格限定在 howToCompressRules / tier2DistillRules 两处,wire-contract 套件本地绿。
  • distill/refold 存储:由“静默丢掉”变“回填 + 告警” → 有意变更,已披露。
  • 未触碰:ref/id 逻辑、持久化格式、拒绝语义、缓存键。
  • 两个有意的设计取舍(非回归,记录供 owner 权衡,我认为本 PR 不必改):

6. 是否破坏缓存
不动增量游标 / 去重键。升级时有一次性前缀缓存失效(~500B,与 #2918 同类);回执那行骑在“折叠本就重写下游历史”的既有失效上,零额外成本。

7. 是否改了设计方向
没有。仍是“提示词约束 + 内核确定性兜底”双层,边界与数据流都没转向。

8. 是否引入新配置
没有。无新 env / config / 默认值。

关键卡点:CI 红,需一次干净重跑

pushed head 上 6 个 test shard 挂了 4 个(ubuntu 22/2、ubuntu 24/1、windows 22/1、windows 22/2),而同名 shard 在当前 master HEAD 上是绿的(除 ubuntu 24/1 仍在跑),失败 check-run 没有任何 annotation。我这边拿不到 CI 日志(token 无 checks-log 权限,/logs 端点 404),所以没法 100% 定性;但结合「master 同 shard 绿」+「本 PR 触达的全部本地用例(内核全套 1303 + 回执/prompt 爆炸半径 351 = 1654)全绿」,几乎可以断定是环境型 flake(端口占用 #394 / Windows spawn EINVAL #3617-C / runner OOM 这一类),不是本 PR 引入的回归。

处置建议:重跑失败 shard。转绿即可人合并;若重跑后仍有具体测试用例(而非环境错)失败,那才需要追真正回归——但我手上的证据指向 flake。

合规核对(Kernel Boundary)

PR body 首行声明 kernel-change: ✓;kernel/package.json 版本 0.0.131→0.0.132 ✓;内核提交用 kernel: 前缀 ✓;两层测试(内核单元 + 宿主)本地均绿 ✓;golden 重新生成并在 PR 说明理由 ✓。本 PR 触碰 kernel/,属 human-only review 档,禁止 auto-merge;它又改了 prompt(行为)+ wire-contract golden,同样不在 auto-merge 范围。合并由人来。

判定

代码与测试:通过,质量达标(命名达意、注释讲为什么、正负向断言都有牙、diff 收敛到 12 个相关文件无跑题)。唯一未闭环的是 CI 红——请重跑失败 shard,确认转绿后由人工合并;若重跑仍红且有真实用例失败再回来追。我不会代合并(kernel 档 + 系统规则都禁止 agent 合并)。

@ranxianglei ranxianglei added bug Something isn't working p2 pi scope/kernel labels Oct 18, 2026
@ranxianglei
ranxianglei force-pushed the 2026-10-18_issue3538-objective-status-guard branch from 9199591 to a9e7ef4 Compare October 18, 2026 16:37
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

⚠️ 上游冲突已处理:rebase 到最新 master(6f2b113bd,v0.1.201 线)并 force-push,分支头现在是 a9e7ef4b9。

冲突面(PR 在途期间 master 合入了 #3658 protocol-stable-core、#3699、lock-sync 等):

  • 内核版本撞车:master 已把 acp-kernel 从 0.0.131 推进到 0.0.133,与本 PR 的 0.0.131→0.0.132 bump 冲突;
  • 四个 storage-render golden 双方都改过(master 侧是 wire identity 变更,本 PR 侧是提示词措辞变更)。

解法:

  1. 版本冲突 → kernel/package.json + package-lock.json 直接定到 0.0.134(master 已在 0.0.133,本 PR 仍需一次 bump),对应 commit 消息同步改为 "kernel: bump version to 0.0.134";
  2. golden 冲突 → 没有手工文本合并:取 master 基线后用官方再生脚本 scripts/update-storage-render-goldens.ts 从合并后的源树重新生成。已核验再生结果相对 master 只有两类变化——每轮 bytes 统一 +308(正是新增提示词的长度)及相应 digest/sha 重算,无其他语义漂移;
  3. 其余文件(compression-rules.ts / compress.ts / stream.ts / prompt fixtures / wire-contract golden / 测试)自动合并干净,无语义碰撞。

本地定向验证:typecheck 干净 · build 绿(含 dist 注解检查)· 内核单元套件 1313/1313 · 宿主定向测试 4 个文件 43 用例全绿(golden-storage-render、wire-contract、compress-fingerprint-refold、issue3538-post-compress-receipt)。全量判定 = 新 head 上的远程 CI。

结构化八问

  1. 是什么问题:PR 分支与 master 冲突(merge state dirty),无法合并。
  2. 什么原因导致的:master 在此期间合入了内核变更([PR] protocol-stable core identity: same logical conversation, same ids in every dialect (#3642) #3658 等),acp-kernel 版本先行推进到 0.0.133,且双方都动了同一批 pinned artifacts(golden)。
  3. 影响范围与严重程度:仅本 PR 的可合并性受阻,无用户面影响。
  4. 怎么修复的:rebase(非 merge);版本冲突取 0.0.134;golden 用官方再生脚本从合并树重生成,不做手工拼合。
  5. 是否有回归问题:无新增行为——rebase 后 diff 与原 PR 逐文件一致(16 个文件,461+/159-,形状与 rebase 前完全相同);唯一差异是内核版本号 0.0.132→0.0.134(master 先行的机械后果,非行为变更)。本 PR 触碰的既有行为(成功回执形状、prompt golden、carry-forward 兜底、refold 指纹断言)已由上面重跑的定向套件覆盖。
  6. 是否破坏缓存:不变。仍只是升级时一次性前缀缓存失效(~500B,与 fix(#2750): make Open objectives binding until user closes them; notifications re-authorize nothing #2918 同类)。
  7. 是否改了设计方向:没有。
  8. 是否引入新配置:没有。

判定:冲突已解除,分支现在可合;请等新 head 的 CI 转绿后由人工合并(kernel 档,agent 不代合并)。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 pi scope/kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant