Skip to content

[PR] fix(#4665): carry identity + retired_at across archive/restore - #4672

Merged
ranxianglei merged 8 commits into
masterfrom
2026-10-25_4665-archive-restore-row-state
Oct 31, 2026
Merged

ranxianglei merged 8 commits into
masterfrom
2026-10-25_4665-archive-restore-row-state

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #4665

What

The archive dump SELECT and restore INSERT predated #4390's v8 row-state columns (identity, retired_at). Consequences after an archive→restore:

This is a regression against pre-v8 behavior, where the content_hash-based JOIN let restore self-heal.

Fix (both backends)

  • sqlite (src/storage/store.ts) + mysql (src/storage/mysql.ts): dump SELECT now carries identity, retired_at; restore writes both back verbatim.
  • Legacy-dump fallback (archives written before this fix lack both fields): identity is recomputed from the CAS body with identityOfStoredContent — the exact formula the v8 migration backfill used; retire state is unrecoverable there and restores as live (documented boundary).
  • Retired rows continue to be dumped (historian bytes stay byte-exact); they now come back retired.

Behavior change disclosure

Old → new, all intended by the fix:

surface old (v0.1.207-era) new
restored row identity '' (JOIN dead → re-mint/duplicates) stored value verbatim; legacy dumps: recomputed from CAS body
restored row retired_at always NULL (resurrection) stored value verbatim; legacy dumps: NULL (unrecoverable)
archive JSONL line shape 7 message columns + identity, retired_at (additive; readers tolerate absence via the fallback)

No schema change, no config surface change, no kernel/pi-subagents/ change.

Evidence

  • New tests/issue4665-archive-restore-identity.test.ts: round-trip (dump carries fields → restore preserves seq/identity/retire state → re-ingest mints nothing), legacy-dump fallback path, mysql-lane twin (DSN-gated, runs in CI). Both sqlite tests FAIL on the pre-fix code (verified via stash) and pass with the fix.
  • Targeted local runs green: issue4665 suite, storage-archive-gc, storage-review-p2s, storage-backend-contract, storage-ingest-watermark, media-cas (48 pass / 0 fail; mysql lanes skip locally, run in CI).
  • Full-suite verdict comes from the CI test lanes on this head.

The archive dump SELECT and restore INSERT predated #4390's v8 row-state columns: every restored row landed with identity='' (the full-ingest JOIN key can never match again -> next full save re-mints every message beside the restored copy = duplicate messages) and retired_at=NULL (a rolled-back row resurrects as live). Both backends now carry both fields verbatim; pre-fix dumps lacking them fall back to recomputing identity from the CAS body (the v8 migration backfill formula) and restore retired rows as live (state unrecoverable there).
@ranxianglei ranxianglei changed the title fix(#4665): carry identity + retired_at across archive/restore [PR] fix(#4665): carry identity + retired_at across archive/restore Oct 25, 2026
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-25_4665-archive-restore-row-state (4a413e8)

Latest build — install this exact version

npm install -g billion-context@0.0.6158-pr.4672.2e3e7ca

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
4a413e8 billion-context@0.0.6158-pr.4672.2e3e7ca build
1246f56 billion-context@0.0.6071-pr.4672.fdbc17a build
2f4b0e8 billion-context@0.0.5971-pr.4672.ebda92c build
8bbedda billion-context@0.0.5888-pr.4672.4e0aaeb build
45fa4c2 billion-context@0.0.5820-pr.4672.c39c3f0 build
5c8719b billion-context@0.0.5725-pr.4672.87de81f build
b9f6112 billion-context@0.0.5713-pr.4672.42484b3 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr4672.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 检索 open+closed 的 issue/PR(按 archive/restore/identity/retired_at 失败签名):本 PR 是 #4665 的唯一修复 PR;相邻条目均不同机制——#4664/#4670 是 export 车道漏带 retired 行(读侧),#4390/#4395 是本缺陷的引入方(v8 行状态列迁移)。无同根因在途工作。

标签: bug + scope/storage + p1 + review中(存储层数据完整性缺陷:恢复后重复消息/回滚行复活,envelope-only 保存下永久;按 10-19 政策属数据完整性类 p1)。

开始独立审核:先审原 issue 方向、再清点 dump/restore 既有行为、逐行读 diff、验测试有效性、看 CI。结论稍后按八问体例给出。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

审核结论(独立审,非复读 issue)

八问

  1. 是什么问题:会话经历 archive→restore 后,恢复出来的每一行都丢了行状态——identity 落空、retired_at 落 NULL。后果是服务历史里出现重复消息(下次 full save 把每条消息重新发号一份,挨着恢复的副本),且被回滚过的行复活成活消息。
  2. 什么原因导致的:archive 的 dump SELECT 与 restore INSERT 写于 Four storage-layer defects: identity hash contaminated with positional metadata (truncation triggers identity swap) / sw | 存储层四缺陷:身份哈希掺位置元数据(截断即换血)/清扫删除回退历史/渲染接缝图片只比长度/fork 谱系死列 #4390 之前,v8 引入的两个行状态列(identity/retired_at)从未进入归档 JSONL。Four storage-layer defects: identity hash contaminated with positional metadata (truncation triggers identity swap) / sw | 存储层四缺陷:身份哈希掺位置元数据(截断即换血)/清扫删除回退历史/渲染接缝图片只比长度/fork 谱系死列 #4390 把 full-ingest 的 JOIN 键从 content_hash 迁到 identity 之后,空 identity 让 JOIN 永久失效;pre-v8 靠 content_hash 自愈的路径已不存在,于是回归显形。是「状态在归档边界丢失」,不是重建逻辑本身有错。
  3. 影响范围与严重程度:所有走过归档→恢复路径的会话(GC 保留期回收或手动 archive 后 restore)。触发条件明确(必须先 restore);一旦触发,用户可见历史损坏(重复消息 + 回滚行复活),envelope-only save 的会话重复永不收敛——p1 数据完整类,定级正确。
  4. 怎么修复的:在产生缺陷的决策点修(dump/restore 通道),不在下游打掩码——dump 原样携带两字段、restore 原样写回;对缺字段的旧 dump(identity 可从 CAS body 重算)用 v8 backfill 的同一公式重算,retire 状态不可恢复则如实按 live 恢复并文档化边界。sqlite/mysql 双后端对称改。修法通过 litmus test:同一根因换个形状出现(比如将来再加一个行状态列),只要 dump/restore 带上它即成立。
  5. 是否有回归问题:审过的既有行为逐项过——① archive.jsonl 全仓只有两个消费方(archiveSession 写 / restoreArchived 读),行格式变更为纯增量,旧代码读新 dump 会忽略多余字段(回滚部署也优雅降级,不崩);② restore 本来就不先删行(依赖 archiveSession 已清场),语义未动;③ mysql BIGINT NULL 经 DSN 默认池解析为 JS number(epoch ms < 2^53 精确),round-trip 类型检查成立,且严格 typeof === "number" 与代码库既有 num() 容错口径一致;④ 无 fail-fast 变吞错、无阈值/时序/输出格式漂移。测试有效性:断言有牙——我独立复验了「有牙」:把两个源文件回退到父提交后 issue4665 两条 sqlite 用例必挂,恢复修复即绿。定向域(issue4665/archive-gc/review-p2s/backend-contract/ingest-watermark/media-cas)本地 38 pass / 0 fail(mysql 道 DSN 门控,CI 跑)。
  6. 是否破坏缓存:不涉及——没碰任何缓存/重试/增量游标/去重键策略;本修的正是 seq/identity 稳定性本身。
  7. 是否改了设计方向:没有。
  8. 是否引入了新的配置:没有(无 schema 变更、无 config surface 变更、无 kernel/pi-subagents 变更,PR 披露表与代码事实一致)。

CI 状态与处置:head b9f6112 上 6 个 test 分片红 = 继承性失败(master 8b24d1f 自身同批分片红,#4599 floor 139;逐片日志核对,失败用例全部落在已知 born-red 族:storage golden/engine-proofs ×6、plugin-fork HTTP fork revision、orphan-recovery cold load,与本 PR diff 零重叠)。ci-fixes 已包含于 master(merge no-op),故把分支同步到 master d1b46fc(merge commit 5c8719b3e,无冲突;合并后 head 本地 typecheck 干净 + 定向套件绿,已 push)。等哨兵 ci-fixes 收敛、CI 在新 head 转绿即可合。

判定:代码侧 ready-to-merge(方向正确、双后端对称、回归钉住、diff 干净、行为变更已如实披露);唯一阻塞 = 继承性 CI 红,非本 PR 代码问题。已在 #4599 留一行状态。

ci-check added 6 commits October 26, 2026 01:33
master window commits 293ae65/de5e03659 bumped kernel/package.json to
0.0.165 without regenerating the lockfiles; lock-sync re-resolution then
rewrites the stale recorded versions (root 0.0.163, kernel 0.0.164) and
fails. Pure recorded-version sync via npm install --package-lock-only;
no dependency or semantic change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant