Skip to content

test: plugin-mode cache-friendliness matrix for all four wires (#1553) - #1571

Merged
ranxianglei merged 1 commit into
masterfrom
2026-09-28_plugin-cache-matrix
Sep 28, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-09-28_plugin-cache-matrix

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #1553

What

The plugin-mode twin of #1548's proxy-mode cache-friendliness suite (tests/cache-friendly-proxy.test.ts), landed as a permanent gate. Test-only: one new file, tests/cache-friendly-plugin.test.ts (~627 lines); zero src/ changes.

Because the two compression modes have different summary carriers (plugin mode re-sends its own compress call+result from persisted history; proxy mode synthesizes them ephemerally), a green proxy-mode matrix proves nothing about plugin mode — hence the separate symmetric suite #1553 ordered.

Invariants pinned per wire (responses / openai-chat / anthropic / google)

One session per wire, 8 consecutive upstream-bound request payloads compared as flat unit lists (text blocks + structured tool units in serialization order):

  • P1 — growth turns are append-stable: the previous body's unit list is an EXACT byte-prefix of the next; mid-history never mutates; the ≤4 volatile tail slots (chain checkpoint / nudge / imgNote / retrieval note) may only appear as a contiguous tail run.
  • P2 — fold transitions (client re-sends its compress pair after a /__bili/plugin/tool fold): the pre-fold span is replaced by exactly the expected carrier units (+ retained first user message) and nothing else; the client's pair stays visible at its chronological position every turn; coexistence pin: each fold summary occurs exactly 3× on every post-fold turn while its pair rides inbound (in-place carrier + fold spec echoed in the re-sent tool_call args + tail quote inside the persistently re-sent tool result).
  • P3 — multi-fold: two consecutive folds each satisfy P2; both carriers persist byte-stably afterwards.
  • P4 — marker/tag hygiene: every content unit carries a well-formed <acp> head tag, refs are never reused across turns, q/a marker lines are byte-stable within their ref, summary carriers stay untagged, and the system field (system/systemInstruction) is byte-identical across all turns.
  • P5 — no transient artifacts: zero acp_loop_* ids in any forwarded body (plugin mode has none by construction — pinned so a regression cannot appear silently).
  • Geometry subtest (anthropic): superset-swallow fold REPLACES the swallowed summary, partial-overlap folds coexist, non-covered content survives monotonically, and KEEP_LAST_ORPHANED=2 prunes the oldest pair exactly when a third arrives.

Finding surfaced while building → #1567

The in-place acp_summary carrier is not stripped when the client's own compress pair rides inbound history — the plugin endpoint mints plugin_<ts> callIds that can never equal the client's tool_use id, so stripKernelSummaries's source-keyed match never fires in plugin mode. That diverges from #1553's invariant-2a wording and #1551's "(plugin-mode) behavior unchanged" parenthetical. Filed #1567 with the full trace; the suite pins ACTUAL behavior and carries the intended-strip assertion skip-gated behind BILI_PLUGIN_STRIP_FIXED=1 (off by default) so it flips on automatically once #1567 lands.

Wire-specific normalization notes

  • anthropic/google merge consecutive same-role turns into ONE entry holding several blocks/parts — extraction follows serialized block order so the unit sequence mirrors the forwarded bytes.
  • google part-shapes carry no call ids — tool-unit identity is derived from payload fingerprints, and pair presence is pinned through the fold topic embedded in the re-sent functionCall args.

Behavior changes

None — test-only workstream. No config surface touched. No wire behavior changed.

Pre-flight

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-28_plugin-cache-matrix (72697cd)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.1523-pr.1571.831e41d

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr1571.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review complete — mergeable, with one cross-PR coordination caveat.

Pulled head 5501b25c and reviewed against live master. Verified: base sha 1c60633c == current master tip (no stale-base risk), diff is exactly one new file tests/cache-friendly-plugin.test.ts (+627/−0), zero src/ changes, no version bump, no config surface touched.

① Original issue (#1553): delivers exactly what was ordered. P1–P5 + the anthropic geometry subtest map onto the ordered invariants (append-stability, fold-transition carrier replacement, multi-fold, marker/tag hygiene, zero transient artifacts). Two deltas vs the issue wording, both justified: the volatile-tail budget is ≤4 not ≤3 because #1419's retrieval note ([billion-context] Earlier acp_retrieve…) landed on master after #1553 was worded; and the invariant-2a strip premise turned out false — handled precisely per #1553's non-goals clause: defect filed separately (#1567, with the required 来源 marker citing #1553) and the intended-strip case kept skip-gated behind BILI_PLUGIN_STRIP_FIXED=1.

② Regression review (existing-behavior inventory): since the diff is purely additive test code, the inventory is short and each item checks out:

  • Wire behavior (both compression modes): untouched — no src/ lines changed.
  • Config surface / persistence format / session binding: untouched.
  • npm test: only gains a permanent gate (5 top-level tests, 4 skip-gated subtests off by default); glob tests/*.test.ts picks it up automatically.
  • Hot-file contention: none — fix: strip redundant in-place acp_summary for plugin-mode folds (#1567) #1570 touches src/plugin.ts, src/server.ts, tests/plugin-protocol.test.ts; no textual overlap with this file.

③ Code review: read the full file. The flat-unit-list discipline is stronger than the proxy-mode twin's element-level compare (it dissolves anthropic/google same-role merging, so the unit sequence mirrors forwarded bytes), and the fold-transition check does exact whole-body reconstruction (assertFoldTransition), not spot-checks. Occurrence counting is per marker, not per containing element — the exact lesson from #1553 floor 4 about the anthropic counting artifact. Ports come from listen(0); process-global state (XDG_STATE_HOME, store, registry, plugin state) is reset per subtest and restored in finally; tags use hex escapes; no as any/@ts-ignore/console.error. Two minor notes, neither blocking:

  • canon() strips slot-kind units (chain/nudge/imgnote/retrnote) from the compared core regardless of position, so P1's "contiguous tail run" is effectively asserted as count(≤4)+kinds only. A hypothetical regression injecting a slot-shaped unit mid-history could slip past assertPrefix if surrounding positions still line up. Symmetric with the proxy twin's design and structurally unlikely (slot injections are tail-only in src/), noted for the ledger.
  • The "flips on automatically once plugin mode: in-place acp_summary never stripped when the client's own compress pair rides inbound — double summary carrier on every post-fold turn #1567 lands" phrasing overstates slightly: when the fix lands, the 4 gated subtests do flip on, but the main coexistence pins (summary occurs exactly 3×, marker count ≥1) will need manual re-pinning to the post-strip values in the same change — the gate alone doesn't keep the suite green across the fix.

④ Test validity (teeth check): ran BILI_PLUGIN_STRIP_FIXED=1 locally on the pre-fix tree → exactly the 4 expected "in-place carrier must be stripped while the client's pair rides inbound" subtests fail (one per wire, at t4), everything else green. So the gate is a real assertion, not a tautology: it fails correctly today and turns into coverage the moment #1570's fix lands. Ran the default suite twice back-to-back: identical results (5 pass / 4 skip / 0 fail) — deterministic as claimed. npm run typecheck clean locally.

CI: all 12 checks green on head 5501b25c (incl. test (windows-latest, 22) which was still running at pickup).

⚠ Coordination caveat — merge order with #1570. #1570 (the #1567 strip fix, open, same base 1c60633c) changes exactly the behavior this suite pins. If #1570 lands while this suite still pins coexistence, master goes red on the 3×/marker assertions. Recommended sequence: merge #1571 first (it pins current actual behavior, explicitly sanctioned by #1553's non-goals), then when landing #1570 update this suite in the same PR (re-pin occurrence counts to the post-strip values and turn BILI_PLUGIN_STRIP_FIXED on permanently or drop the gate). Alternatively hold #1570 until #1571 is in and rebase it. Either way the two must land as a coordinated pair — flagging so the merge of either doesn't silently break the other.


中文摘要:审查了 plugin-mode 缓存友好矩阵测试 PR(#1571,纯测试、单文件 +627 行)——断言有牙(本地开启 BILI_PLUGIN_STRIP_FIXED=1 后恰好 4 个预期子测试失败、关闭时全绿且两次运行结果一致)、CI 12 项全绿、无 src 改动无行为变更,可以合并;唯一注意点是它与在途修复 PR #1570 存在合并顺序耦合(本 PR 钉住的是修复前的实际行为),建议先合本 PR、#1570 落地时同步更新本套件的计数断言。

@ranxianglei

Copy link
Copy Markdown
Owner Author

Deep review complete — merge as-is (rebased onto current master, clean; head pushed).

Design verified: P1 append-stability / P2a exact fold reconstruction / P2b occurrence=3 reality pin / P3 multi-fold survival / P4 no-transient-leak / P5 marker hygiene — occurrence-counting from day one (the anthropic element-counting artifact lesson from #1548 is baked in).

Teeth (mutation-verified):

Gates: plugin matrix 5 pass + 4 skip-by-design; combined with proxy matrix 10 pass + 4 skip; typecheck ✓. Test-only (one file, zero src changes).

Follow-through owed by #1567's fix PR: flip the BILI_PLUGIN_STRIP_FIXED gate (unskip or default-on) so P2c becomes the permanent contract — noted there.

@ranxianglei
ranxianglei force-pushed the 2026-09-28_plugin-cache-matrix branch from 5501b25 to 72697cd Compare September 28, 2026 11:50
@ranxianglei
ranxianglei merged commit 21ca064 into master Sep 28, 2026
12 checks passed
ranxianglei pushed a commit that referenced this pull request Sep 29, 2026
…und (#1567 hardening)

The unconditional prefix strip could lose a summary outright: if the client
prunes or fails to re-send its compress pair, the in-place anchor was
stripped with no carrier left (zero carriers). Restore the old predicate's
self-verification — strip per-block only when that exact fold's pair is
present in the (post-prepare) inbound history, matched by tool name plus the
folded range quoted in the call args (flat {startId,endId} or
{content:[...]}). hideConsumedCompressCalls (KEEP_LAST_ORPHANED) runs before
the strip, so a kernel-hidden pair also keeps its anchor — fail-safe in both
directions.

Matrix (#1571) re-pinned to the intended behavior (0 carriers while pairs
ride; exactly 2 pair quotes) and supplemented with two hardening scenarios:
client drops a re-sent pair → anchor re-carries, bytes stable; 3rd concurrent
fold → oldest pair pruned by KEEP_LAST_ORPHANED, oldest anchor kept.
@ranxianglei
ranxianglei deleted the 2026-09-28_plugin-cache-matrix branch October 15, 2026 05:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[test] Plugin-mode cache-friendliness invariants — same matrix as #1548's proxy-mode suite, as a permanent gate

1 participant