Skip to content

[PR] fix: reject corrupt positional affinity snapshot carriers - #6047

Merged
ranxianglei merged 4 commits into
ranxianglei:masterfrom
Afloat16:2026-11-09_fix-affinity-snapshot-position
Nov 10, 2026
Merged

ranxianglei merged 4 commits into
ranxianglei:masterfrom
Afloat16:2026-11-09_fix-affinity-snapshot-position

Conversation

@Afloat16

@Afloat16 Afloat16 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Problem

Filtering malformed middle slots from persisted sketch/role arrays shifts later similarity evidence onto the wrong message.

Changes

Validate positional carriers as a whole and discard malformed similarity evidence while keeping exact hashes.

Regression coverage

Invalid middle sketches, sparse array holes and role flags degrade their complete positional carrier; exact replay and valid round-trip remain intact.

Validation

Passed against exact fork commit caf14bb702346eae823ede9b46d501952b3cca4d in an isolated GitHub Actions checkout: validation job. The targeted regression and related suites, npm run typecheck, and npm run build all passed on Node 22. This is targeted verification, not the full upstream CI matrix.

node --import ./tests/preload-storage-baseline.mjs --import ./tests/preload-test-env.mjs --import tsx --test tests/affinity-snapshot-position.test.ts tests/prefix-affinity-simhash.test.ts
npm run typecheck
npm run build

Related work

Original feature #2266; malformed carrier handling is the new case.

Closes #6037.

@ranxianglei ranxianglei changed the title fix: reject corrupt positional affinity snapshot carriers [PR] fix: reject corrupt positional affinity snapshot carriers Nov 9, 2026
ci-check added 2 commits November 10, 2026 00:10
…anxianglei#6037)

Same defect class as the sketches/userFilters fix in this PR: the old
filter-based validation dropped a malformed middle slot and shifted every
later fragment onto the wrong source message, so the append-continuation
rung compared stored vs incoming fragments positionally out of sync
(worst case: a true continuation misses and mints a fresh session id).
Whole-carrier validation now matches the other two carriers; exact-hash
resolution is untouched either way.

Regression coverage: a corrupted middle tail fragment degrades the whole
carrier while exact replay still reattaches via 'prefix'; a valid
tailFrags list round-trips byte-identical. Both fail/pass correctly
against the pre-fix code.
…ianglei#6037)

The local lists hold the raw persisted slots BEFORE validation, so
'valid'/'validU' misnamed them. Renamed to slots/flags to match the
frags naming in the tailFrags block. No behavior change.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Reviewed PR #6047 (closes #6037, external contributor). Verdict: mergeable — human merge. The fix is correct, minimal, and well-tested; I verified it end-to-end in an isolated checkout rather than trusting "should work."

What I checked (evidence)

  • Typecheck — passes under the project strict tsconfig. I reproduced the exact changed lines under tsc --strict: the .every(<type guard>) form narrows the array soundly (every-true ⇒ every element matches), so the string[]/boolean[] assignment is legal. (My first instinct was that unknown[] → string[] would fail; it does not — verified empirically.)
  • Targeted suites — tests/affinity-snapshot-position.test.ts (4 new) + tests/prefix-affinity-simhash.test.ts (21) = 25/25 green on the PR head.
  • Teeth — run against the true PR base, 3 of the 4 new cases FAIL (middle-sketch, role-flag, sparse-hole); the valid round-trip case passes on both base and fix. So the tests reproduce the original bug and don't over-degrade healthy carriers.
  • Full CI matrix (on the pushed head) — 30/30 checks green, 0 failed / 0 pending.
  • Rebase — src/prefix-affinity.ts is byte-identical between the PR base and current master, so this merges cleanly, and the bug is still live on master today.

【结构化八问】

  1. 是什么问题:持久化匿名亲和快照里按位置存储的 simhash sketches 与角色位 userFlags 数组,若中间某格损坏(非 16 位 hex / 稀疏空洞),旧代码用 .filter 逐格过滤会把坏格丢掉、其后所有格整体左移一格,导致后续位置的相似度证据对到错误的消息上。
  2. 什么原因导致的:importSnapshot(src/prefix-affinity.ts:1214-1228)把「按位置对齐」的载体当「可任意删格」的集合做逐元素校验;而消费端 simhashAdopt(:876 逐下标比 Hamming、:901-905 所有权锚点按下标取 user 位)严格按下标消费——丢一格即整体错位。根因是校验语义(集合式 filter)与消费语义(位置式)不匹配。
  3. 影响范围与严重程度:仅当「重启后从持久化快照加载」且该快照 sketch/flag 数组含坏格时触发;正常流量不受影响。最坏后果是匿名会话被错误采纳(两个不同对话复用同一会话对象 → 折叠状态串话),下一轮 note() 会按真实字节重新锚定、可自愈,无持久数据丢失。p2(特定条件才触发、需持久文件损坏)。
  4. 怎么修复的:改为整载体校验——Array.from(...) + .every(类型守卫),任一格非法或存在稀疏空洞即整体判无效(保持 undefined),该条目降级为精确哈希解析;itemHashes/tailHash 等精确哈希原样保留,精确回放不受影响。Array.from 是关键:把稀疏洞物化成 undefined 让 .every 能抓到(.filter/裸 .every 都会跳过原始空槽)。
  5. 是否有回归问题:合法载体字节级往返不变(往返用例在基线与修复版均通过);只有原本就非法的载体行为改变(「错位的部分存活」→「安全地整体降级」),属本 PR 目标本身、非回归。未触碰 wire/消息形/config schema/持久化格式版本/kernel。相邻隐患:tailFrags(:1245)仍是旧的 .filter,同机制但更轻——其消费档被字节级 itemHashes 前置门住(:711-713),错序只会漏判续写→新铸会话,不会误采纳,且不在本 issue「sketch/role arrays」字面范围内,见下方补充。
  6. 是否破坏缓存:不涉及缓存/重试/增量游标/去重键。采纳只影响「选哪个已跟踪会话对象」,转发上游的线字节仍是客户端原样;精确哈希路径完全不变。
  7. 是否改了设计方向:无架构转向。反而把 sketch/flag 校验对齐到本就已用于 itemHashes(:1204 .some 整载体拒绝)的纪律上,方向一致。
  8. 是否引入新配置:无新增 env/config/默认值。

Additional finding (same mechanism, out of this issue's literal scope): tailFrags at src/prefix-affinity.ts:1245 is still parsed with the old .filter, and it's consumed positionally by the append-continuation rung (prefix check at :718-720) — the same shift-on-corruption pattern. Severity is lower than the two fixed arrays because that rung sits behind a byte-exact item-hash gate (:711-713), so a shifted fragment yields a missed continuation (fresh mint), never a wrong adoption; and it only bites on a corrupted persisted snapshot. Recommendation: fold in the parallel ~3-line fix here (identical mechanism, and this PR already establishes the reject-whole pattern) or track it separately — your call. It does not block merging this PR.

Not auto-merge eligible: it touches anonymous session-resolution logic (human-gated per §7.4). Otherwise clean, single-file, pure fix: with real regression teeth — ready for you to merge.

@ranxianglei
ranxianglei merged commit 9895b69 into ranxianglei:master Nov 10, 2026
48 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.1141-master.9895b69 (rolling master tag).

npm install -g billion-context@0.0.1141-master.9895b69  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Reject corrupt positional affinity snapshot carriers

2 participants