Skip to content

fix: stop legacy-id migration destroying anonymous pfa sessions (#499 prerequisite) - #502

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-03_pfa-restart-reseed
Sep 4, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-03_pfa-restart-reseed

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Sep 3, 2026 •

Copy link
Copy Markdown
Owner

Scope change (was: full restart-resume fix)

This PR originally shipped the complete #499 restart-resume fix (persist chain snapshots in session metadata + reseed() + initSessions() hook). PR #504 now covers that half via a dedicated affinity-table snapshot (src/affinity-persist.ts, hydrate on boot / flush on shutdown) — a strictly more complete mechanism (preserves lastSeen for TTL, independent of session-file lifecycle, covers the whole table). Keeping both would mean two competing persistence paths for the same table, so this PR was shrunk to its unique remaining piece: the bug neither PR addressed.

The bug: migrateLegacyIds() destroys every anonymous session on each boot

The #286 one-time migration re-keys sessions whose stored label ≠ their id, deleting all but the newest sibling per label. Anonymous prefix-affinity sessions (#309) all carry the shared display label "prefix-affinity" ≠ their pfa- id — which trips the migration's self-termination invariant. Result, verified empirically before fixing:

before migrateLegacyIds(): [pfa-bc1e….json, pfa-8588….json]   ← two anonymous sessions
after  migrateLegacyIds(): [prefix-affinity.json]              ← one file, newest state only

On every startup, all but the newest anonymous session's saved compression state is silently deleted, and the survivor is re-keyed away from its pfa- id.

Why this is a prerequisite for #504 P1a

P1a restores the routing table across restarts — it tells the proxy "this replay belongs to pfa-8588…". But the folded view itself lives in the session state file, which the migration deletes/re-keys at boot before any request arrives. Without this guard, post-restart resolution lands on a non-existent session id → fresh empty session → raw history forwarded → 0% cache again. The two fixes are complementary: P1a restores routing, this guard keeps the routed-to state alive.

Change

One guard in migrateLegacyIds(): skip ids with the pfa- prefix (src/persist.ts). True legacy derived-hash sessions are still re-keyed exactly as before.

Tests

New regression test in tests/proxy-persist.test.ts (same style as the existing #286 migration test): two pfa sessions with the shared label + one true legacy session → after migration both pfa files survive intact (state round-trips) while the legacy session is still re-keyed under its label.

Pre-flight: typecheck ✅ · tests/proxy-persist.test.ts 22/22 ✅ · full suite 1000/1001 ✅ (the one failure, launcher.test.ts codex PATH resolution, pre-exists on master in environments where /usr/bin/codex is installed) · no build impact (no dist-affecting surface beyond persist.ts).

Merge order: this can land before or alongside #504 — zero file overlap.

@github-actions

github-actions Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-03_pfa-restart-reseed (3be6b6d)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-502

Each push to this PR publishes a new version under the pr-502 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr502.tgz
npm install -g package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · qwen3.8-27b

[bot] 🏷 Reviewing this PR now — pulling the branch, re-running pre-flight (typecheck/test/build), and auditing the three fix pieces (migrateLegacyIds pfa exclusion, metadata persistence, boot-time re-seed) plus the new tests. Will report findings here shortly.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 ework agent · qwen3.8-27b

[bot] 🏷 Review complete — approved, merge-ready. ✅

Root-cause audit (both defects verified against code)

Defect 1 — migrateLegacyIds data loss: confirmed. Every anonymous session gets label "prefix-affinity" (src/server.ts:1131), violating the #286 migration's label === id self-termination invariant. All pfa- files therefore funnel into one byLabel bucket each boot: newest wins, losers deleted via removeLegacyFile, survivor rekeyed to id "prefix-affinity" — an id the resolver can never mint again, so even the survivor becomes permanently orphaned. The guard if (id.startsWith("pfa-")) continue (src/persist.ts:224) is correct and precise: pfa- ids only exist since #309 (current format), so no legacy derived-hash session can collide with the prefix. True legacy rekeying is preserved (regression-tested in test #15).

Defect 2 — missing re-seed: confirmed. trackedChains is process-memory only with no disk rebuild path; post-restart the replay has grown past the creation-time depth, resolution mints a fresh id, and the persisted CompressionState is orphaned. Resume path verified logically: seeded entry (depth=d, tailHash=T) + client replays ≥d byte-identical messages ⇒ hashes[d-1] === T ⇒ prefix match at depth d ⇒ original pfa- id ⇒ saved folded view applies. Persisting itemHashes too is what makes tail-window reattach survive restarts — correct call.

Code quality

  • reseed() shape-validates (pfa- prefix, integer depth > 0, non-empty tailHash, string-filtered itemHashes capped at MAX_STORED_ITEMS=128) and routes through note(), so LRU/TTL semantics are unchanged. No import cycle (session.ts → prefix-affinity.ts → node:crypto only).
  • Seeding from all loaded sessions, not just the memory-capped subset, is right: evicted sessions reload on demand via loadSync and must stay routable.
  • No schema change (free-form metadata); old files without the record simply skip re-seed — backward compatible.
  • Mode-safety: the whole change lives behind anonAffinity != null (headerless/stateless clients only). Plugin mode and client-provided-identity paths are untouched — both compression modes unaffected.

Pre-flight (re-run independently)

PR head: typecheck ✅ · npm test 951/952 · build ✅ · tests/prefix-affinity.test.ts 16/16 incl. the 5 new tests. The single failure (launcher.test.ts "resolveClientCommand: codex/claude resolve to themselves") reproduces identically on master in this environment (has /usr/bin/codex) — verified by checking out the master tree, so your pre-existing claim holds.

Merge-readiness vs current master (branch base is 29 commits behind, incl. #404 timestamp restore, #473, #453 backstop): simulated merge is clean, zero conflicts. I actually merged it locally and re-ran everything on the combined tree: typecheck ✅ · 994/995 (same single pre-existing launcher failure) · build ✅. Merged initSessions() composes correctly with #404's freshness-based cap (re-seed still iterates all loaded, independent of the cap).

Minor observations (non-blocking)

  1. reseed() → note() stamps lastSeen: Date.now() on every seeded chain, refreshing the 7-day TTL clock at boot — chains older than TTL_MS resurrect post-restart instead of expiring. I read this as deliberate and arguably more consistent (sessions never age out of persistence, and a long-running process keeps routing forever, so post-restart matching that is good). If you ever want TTL honored across restarts, pass the persisted savedAt through — 重启伪造时间戳:buildSession 用 Date.now() 覆盖 lastSeen,面板"最后活动"全员开机时刻、fallback=latest 在 ~245 路平局里随机挂会话 #404 now gives you exactly that on the Session (lastSeen/restored). Side benefit for the >256-sessions edge case: boot-time LRU eviction among equal lastSeen currently falls to disk-load order; real timestamps would make it deterministic.
  2. E2E skip is justified: codex sends body.session_id → clientProvided → never enters the anonymous branch.

No issues found that warrant a separate bug report. LGTM from my side — merge is yours to execute.

…erequisite)

migrateLegacyIds() re-keys sessions whose stored label differs from their id
(#286 cleanup), deleting all but the newest sibling per label. Anonymous
prefix-affinity sessions (#309) all carry the shared display label
"prefix-affinity" != their pfa- id, so every boot re-keyed them all to the
single id "prefix-affinity" and deleted every sibling but the newest — silent
loss of saved compression state for anonymous clients on every restart.

Exclude pfa- ids from the migration. True legacy derived-hash sessions are
still re-keyed as before (regression-tested alongside the guard).

Prerequisite for #504 P1a affinity-snapshot persistence: without it, routing
restored across restarts lands on session files that no longer exist, so the
folded view cannot be rebuilt.
@ranxianglei
ranxianglei force-pushed the 2026-09-03_pfa-restart-reseed branch from d1b73aa to 3be6b6d Compare September 3, 2026 15:49
@ranxianglei ranxianglei changed the title fix: resume anonymous prefix-affinity sessions across restarts (#499) fix: stop legacy-id migration destroying anonymous pfa sessions (#499 prerequisite) Sep 3, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

Independent review (second session) — defect confirmed real, fix correct for the intended path; one theoretical hardening nit below.

Defect confirmed (traced against master): anonymous pfa sessions get meta.label = "prefix-affinity" (server.ts ~1153) while their id is pfa-<hash> (prefix-affinity.ts:270). In migrateLegacyIds, label ≠ id → classified legacy → ALL pfa sessions collide on the single label "prefix-affinity" → every boot deletes all but the newest sibling (removeLegacyFile) and re-keys the survivor to id "prefix-affinity" — which no future affinity lookup will ever request (ids are pfa-<hash>). Silent loss of persisted compression state on EVERY boot with ≥1 pfa file. This also silently defeats #504's affinity-table hydration (the restored pfa- id points at a deleted/re-keyed file), so this PR is indeed the missing prerequisite. I checked the label audit: "prefix-affinity" is the only shared display label in the codebase (pi anonymous stays empty → handled by the unlabeled branch, left in place) — the pfa- guard covers the full bug class. Test coverage (siblings + a real legacy rekey alongside) is right.

Nit (theoretical clobber, 2-line reorder): the guard if (id.startsWith("pfa-")) continue; runs BEFORE the label === id → claimed.add(id) branch, so a new-format session whose client-chosen id happens to start with pfa- is never added to claimed — a legacy file labeled exactly that same string could then be re-keyed ONTO it and overwrite it in the rekey loop. Requires a client sending pfa--prefixed conversation ids in both headers plus a matching legacy label — vanishingly unlikely — but the reorder closes it for free:

if (label === id) { claimed.add(id); continue; }
if (id.startsWith("pfa-")) continue;

Anonymous pfa sessions (label "prefix-affinity" ≠ id) still skip identically; self-labeled pfa- ids get claimed. Not blocking from my side — merge as-is or fold in the reorder, both fine.

Combined-tree pre-flight (this + #500 on current master): typecheck ✅, 1020/1022 (2 known env fails).

@ranxianglei
ranxianglei merged commit 4606219 into master Sep 4, 2026
6 checks passed
@ranxianglei
ranxianglei deleted the 2026-09-03_pfa-restart-reseed branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant