Skip to content

test: add model-switch phase to the identity proof matrix (#2636) - #2692

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-10_model-switch-proof
Oct 10, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-10_model-switch-proof

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Follow-up to #2644 (fix, merged) — extends the identity proof matrix that #2480 introduced, per the post-mortem question "之前的证明形同虚设?" (no issue needed, per owner instruction in #2636).

Why the old proof missed #2636

tests/identity-proof.test.ts's churn operators only ever proved same-shape, new-shell churn: id prefix swap (alt_), tool-arg key reordering and whitespace jitter. Every synthetic assistant message was pure text (no reasoning), every user message text-only (no images). #2636's failure class is same-content, new-shape churn from cross-model re-serialization (pi's transformMessages): reasoning moves from a structured reasoning_content field into inline text (a pair of cores becomes one merged core, seamless "" join), and images on text-only targets become a placeholder line. No existing pass could re-anchor that — 211/211 covered ids went unmatched in the live incident.

What this adds

A model-switch phase on the chat wire: reasoning-bearing assistant turns (scratch-{t}-final markers) + image-bearing user turns, folded, then the history is replayed with the pi-style transformation (churnModelSwitch: reasoning_content merged into content, image_url parts replaced by the placeholder). Pure shape churn — ids/params untouched.

  • positive — coverage survives: summary carrier present, covered thinking and covered tool result never re-enter the wire, first post-churn body stays below the unfolded reference.
  • control (reconcile=off) — this churn class re-bills the unfolded history before a self-healed re-fold (round-1 = 112041B carrying scratch-2-final, vs 95219B unfolded ref); the control scans all post-churn bodies because the healed round-2 alone looks clean.

Proof-strength fix found on the way

The matrix previously judged bodies[length-1] after the churn turn. That is exactly the body a fresh server-side compress re-folds: a broken reconcile can replay unfolded on round-1 and still show a folded last body. Both the positive phase and the new control now judge the first post-churn body (positive: true round-1 = 76516B, previously the healed round-2 = 53598B stood in for it).

Evidence

  • node --import tsx --test tests/identity-proof.test.ts — 8/8 (4 wires + old control + cross-wire swap + new positive + new control)
  • fold-reconcile ×2 + cache-proof — 85/85
  • npm run typecheck clean, npm run build ok
  • MESSAGE-IDENTITY.md + MESSAGE-IDENTITY.zh-CN.md matrix sections updated in sync

Refs #2636 (fix shipped in #2644).

The #2480 proof matrix only proved same-shape churn (id prefix swap,
tool-arg key reordering/whitespace): every synthetic assistant was pure
text with no reasoning, every user message had no images. #2636 showed
the real failure class is cross-model re-serialization that CHANGES the
shape of the same content — reasoning moved from a structured field into
inline text (pair of cores -> one merged core), images replaced by a
placeholder line — and none of the existing passes could re-anchor it.

Extend tests/identity-proof.test.ts with a model-switch phase on the
chat wire: reasoning-bearing assistant turns (scratch-N-final markers)
plus image-bearing user turns, then the history is replayed with the
pi-style transformation (reasoning_content joined seamlessly into
content, image parts replaced by the placeholder). Assertions:

- positive: coverage survives the shape churn; the covered thinking and
  covered tool result never re-enter the wire; judged on the FIRST
  post-churn body — a fresh server-side compress may re-fold a failed
  replay, and the healed round-2 must never stand in for the retention
  proof (the old bodies[length-1] assertion was masking exactly that).
- control (reconcile=off): the churn re-bills the unfolded history
  (round-1 carries scratch-2-final, 112041B vs 95219B unfolded ref)
  before a self-healed re-fold — scanned across all post-churn bodies.

Docs: MESSAGE-IDENTITY(.zh-CN).md matrix section extended in sync.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-10_model-switch-proof (8d8983b)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2902-pr.2692.6c0af8b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2692.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收到,开始处理这个 [PR] issue:定位对应 PR、核对代码与 CI 状态,按 review 规范给出八问结论。稍后更新。

@ranxianglei
ranxianglei merged commit 44d5ac9 into master Oct 10, 2026
13 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.230-master.44d5ac9 (rolling master tag).

npm install -g billion-context@0.0.230-master.44d5ac9  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

@ranxianglei
ranxianglei deleted the 2026-10-10_model-switch-proof branch October 15, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant