Repository navigation
feat: chain checkpoint step 2 — shadow recognition (parser, JCS digest, carrier contract) - #1397
Conversation
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-1397Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr1397.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Reviewing PR #1397 (chain checkpoint step 2 — shadow recognition) now: reading the parent issue thread for the confirmed decisions, checking CI, then doing the full code/regression/test-validity review against current master. Will report the verdict here shortly. |
…t, carrier contract) First-processor-wins idempotent interop (#1395, step 2 of the #1357 protocol): recognition-only infrastructure with ZERO forwarding behavior change. - src/chain-checkpoint.ts: strict bili-chain tag parser (own namespace, single line, tag-echo-exempt), per-wire reserved-carrier contract (trailing user-run slots whose ENTIRE content strictly matches the syntax — ownership-based, no global magic-string scanning), dependency-free RFC 8785 JCS canonicalization, SHA-256 request digest over the carrier-stripped body, zero-normalize stamp math, and the digest×timestamp verdict matrix (none | valid | recent-mismatch | stale | invalid) exposed as a request-local sidecar that never enters CoreMessage[]/CompressionState/CCR/session snapshots. - src/server.ts: additive shadow evaluation right after body parse — logs [chain-shadow] verdicts only (info for valid, warn otherwise); gated by the existing chainContentDetection kill-switch and skipped under x-bili-hop. No request is forwarded, rejected, or altered by this change. - tests/chain-checkpoint-step2.test.ts: parser/JCS/carrier/digest/verdict synthetic fixtures plus three real-proxy integration tests proving a valid checkpoint still gets fully processed (no passthrough) in step 2. Defaults per #1395: MAX_FUTURE_SKEW=2min, RECENT_CHECKPOINT_WINDOW=10min, both env-overridable (BILI_CHAIN_MAX_FUTURE_SKEW_MS / BILI_CHAIN_RECENT_WINDOW_MS). Generation + enforcement land in step 3 as a separate human-reviewed PR. Refs #1395
e54f39a to
9d9c583
Compare
[bot] 🏷 Rebased onto current master + note on the one failing CI job Master moved while this was open (
On the earlier
Watching the fresh run on |
…es (#1397) insertCheckpointCarrier's responses bare-string branch emitted a single message with two input_text parts; extraction's carrier contract (singleText) requires exactly one part per slot, so a stamped string-input body was never recognized downstream (verdict 'none') and computeCheckpointDigest hashed the unstripped body, breaking the round-trip property for that shape. Normalize to the message-array form instead so both sides hash an identical post-strip shape. Inert in step 2 (no live stamping yet); pins the step 3 generation path. Adds regression test incl. empty-string edge.
[bot] 🏷 PR #1397 review — found & fixed one real bug; branch is now green. Verdict: mergeable (merging stays human-only per repo rules). Duplicate screen first: searched open+closed issues/PRs for chain-checkpoint work — only parent design issue #1395 and this PR; no parallel work. Review below is against current master (the author rebased the branch onto v0.1.154 mid-review; I rebased my additions onto the new head, plain fast-forward push, no force). ① Direction (原 issue) ✓. Implements exactly what #1395 assigned to step 2: strict ② Regression review (既有行为清点) — each touched-path behavior, verdict:
③ Code review — three findings:
④ Test validity (质量关) — assertions have teeth throughout: exact log-line matching, captured-upstream-body comparisons, count asserts, injected-clock verdict matrix (incl. "verified digest outranks fresher mismatched", unknown-version gating, latest-issuedAt-wins), per-wire carrier contract (responses ⑤ Diff cleanliness — 5 files, +788/−0, every line purpose-related (new module, 16-line server hook, tests, two env-var doc rows ×2 languages). No lock-file, generated, or whitespace churn. Commits I appended to the branch (per review norms — small findings fixed in place, each labeled):
Behavior-change disclosure (旧值→新值):
Verification: CI 13/13 success on head 修复了 chain checkpoint step 2 中 responses 字符串型 input 打戳后永不被识别(两段式消息不符合载体槽契约、且摘要哈希了未剥离正文、破坏回合性质)的问题,并补齐两个新环境变量的文档;影子模式"零转发行为变更"已逐项核实成立,CI 全绿,可以合并。 |
[bot] 🏷 Verified the two review-pass commits now on this branch (
Full CI is green on head |
…or-wins enforcement (#1421) Owner-approved design (issue #1421, decision 2026-09-26): adopt carrier merge (a) with per-wire split on role-alternation-strict wires. Stamp shape per wire (insertCheckpointCarrier): - anthropic: merge as an extra trailing text part of the last user message (string content normalizes to parts); standalone append only when the body does not end in a user message. - google: merge into the last content's parts when it is not model-role (mandatory — mirrors appendGoogleNudge; the append form died inside bili's own pipeline because coreToGoogle fuses same-side runs). - openai: standalone trailing user message (unchanged from step 2). - responses: standalone trailing user message inserted before any trailing compaction_trigger (unchanged; #283 invariant preserved). Restamping an already-stamped body replaces the trailing carrier instead of stacking (idempotent re-stamp). Recognition relaxation (extractChainCarriers), matching each wire's emitted shape — still whole-part matching, ownership semantics unchanged: - anthropic: whole-content OR a LAST part that entirely is the tag. - google: ANY part that entirely is the tag (a later hop's nudge merge can push the tag off the end of parts). - openai/responses: unchanged. Strip is surgical: part-level removal for merged carriers, whole-slot for standalone ones; emptied entries are dropped. Enforcement (server.ts, first-processor-wins): - valid / recent-mismatch / stale+digest-matched → forward verbatim, pipeline skipped (log [chain] info/warn). - stale without digest match → strip the stale carrier(s), process normally (the request leaves re-stamped fresh by this instance). - invalid → never trusted, process normally + warn. - new ChainCheckpointContext.selectedMatched splits the stale verdict. - kill switch: both enforcement and outbound stamping respect opts.chainContentDetection !== false (BILI_CHAIN_CONTENT=0). Outbound stamping (forward()): every request that actually ran the kernel (prepared with non-empty processedMessages, not side passthrough) leaves with a <bili-chain> stamp computed over the exact bytes forwarded (placed after compat roles + output steering so the digest covers them). Best-effort: a stamp failure logs debug and forwards unstamped. requestId = first 8 hex of the digest (deterministic, constant-size). New stampOutbound() export; module header now carries the identification-contract-freeze warning carried forward from #1397 plus the #1421 per-wire shape decision. Legacy advisory text updated from 'until signed chain proof lands' to 'until both sides ship the request checkpoint'. Tests: new tests/chain-checkpoint-step3.test.ts (per-wire stamp shapes, restamp idempotency, recognition relaxation, google must-fail regression via real coreToGoogle fusion, round-trip invariant matrix, integration E1-E7 incl. kill switch and side-request no-stamp). Step-2 test file updated to step-3 semantics (T1/T2 now assert byte-identical verbatim forward; google two-part case is now a carrier under the any-part rule). Eight unrelated test rigs pin exact wire shapes and opt out via chainContentDetection:false.
What & why
#1395 Step 2 of the first-processor-wins Chain Checkpoint protocol (split from #1357): recognition-only infrastructure with ZERO forwarding behavior change — per the landing order in the issue, this PR ships the parser + per-wire carrier contract + JCS normalization/digest + request-level sidecar + shadow verdict logging only. Generation + enforcement are Step 3 (separate human review).
Decisions implemented as confirmed in the #1357 thread ((a) trailing control channel + ownership-based carrier judgment, (b) RFC 8785 JCS):
src/chain-checkpoint.ts(new): strict<bili-chain v="1" processor=… issued-at=… request-id=… digest="sha256:…"/>parser (own namespace, single line ≤512 chars, tag-echo-exempt by design); per-wire reserved-carrier contract — checkpoint is recognized ONLY in the trailing user-run slots (after the last non-user entry) whose ENTIRE content strictly matches the syntax (string content, or exactly one text part); nothing else in the body is ever scanned (no global magic-string search), so model/user-generated tag-shaped text is never a carrier; dependency-free RFC 8785 JCS canonicalization (recursive code-point key sort, arrays in order, JS shortest-round-trip numbers incl.-0, JSON.stringify escaping) with SHA-256 over the carrier-stripped body; zero-normalize stamp math (computeCheckpointDigest) pinned now so Step 3's round-trip property is already tested; verdict matrixnone | valid | recent-mismatch | stale | invalidfrom the digest×timestamp quadrants (a verified digest outranks any fresher mismatched timestamp; unknown versions never trusted; future-beyond-skew with a match = replay/skew → stale), exposed as a request-localChainCheckpointContextsidecar that never enters CoreMessage[]/CompressionState/CCR/session snapshots.src/server.ts(+16 lines): additive shadow evaluation immediately after body parse — logs[chain-shadow] …verdict=…only (infofor valid,warnotherwise), gated by the existingchainContentDetectionkill-switch (BILI_CHAIN_CONTENT=0) and skipped whenx-bili-hopis present (that path already decides decisively). Perf guard: positional extraction first, JCS+SHA-256 computed only when candidates exist.tests/chain-checkpoint-step2.test.ts(new, synthetic fixtures first per the issue): parser accept/reject matrix, JCS invariants (key-order, array order, number/string forms, code-point vs UTF-16 key sort), carrier contract per wire (incl. responsescompaction_triggerstays-last, google part strictness), digest stability (carrier-stripped ≡ carrier-free; key-order invariant), zero-normalize round-trip on all four wires, full verdict matrix with injected clock, env overrides, and three real-proxy integration tests proving a valid checkpoint is STILL fully processed (no passthrough) in step 2.Behavior changes (disclosure)
[chain-shadow]log lines when a chain checkpoint is present in a carrier slot (silent for plain requests).BILI_CHAIN_MAX_FUTURE_SKEW_MS(default 120000),BILI_CHAIN_RECENT_WINDOW_MS(default 600000).Coexistence note
This is purely additive next to the legacy ACP-artifact fallback gate (#1357/#1364 territory) — it does not touch that logic, so it coexists with whichever state of #1364 lands. If #1364 rewrites the same region, the later-landing PR rebases; no semantic conflict expected (different code paths).
Pre-flight
npm run typecheck— cleannpm test— 2804 tests, 2792 pass, 0 fail, 12 skipped (gated e2e)npm run build— successRefs #1395 (multi-step issue — not closed by this PR; Step 3 follows as its own human-reviewed PR)