Repository navigation
[PR] feat: add I1/I2 outbound-invariant judge for cache-stability attribution - #3866
ranxianglei wants to merge 4 commits into
Conversation
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4591-pr.3866.11b6357This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3866.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 全仓 open+closed 按主题/失败签名检索(invariant judge / cache-stability / I1 I2 / outbound invariant):仅本 PR 命中;#3842 是其父设计 issue(owner 已拍板方向),不是并行实现;无其他在途 PR 做同一件事。 标签: 本 PR 是 #3842 owner 批准路线(判官挂真实流量 + 通用宿主行为生成器,PBT/SMT 层已否决)的闭环交付,现在进入 review:对照批准方向核代码、逐项过既有行为回归清单、审测试有效性(真测试还是假测试)、查 pushed head 的 CI 状态,随后按八问体例给结论。 |
…und cores (#3842) Reconciles inbound vs outbound message cores on the live request path to enforce two cache-stability invariants machine-checkably: I1 (no fold) requires every inbound message to reappear at egress in order, core-identical, modulo tail growth plus a closed exemption list (chain carriers, acp_summary carriers, reasoning drops, crush envelopes); I2 (fold) confines all divergence to [earliest fold coverage .. end) with the frozen prefix identical and the carrier position pinned. Exit cores are re-parsed from the exact outbound body through its own wire codec, so the comparison reflects what crossed the wire and is wire-neutral (same-wire and cross-wire unified via the kernel canonical core). Render-tag scaffolding (renderTags=text-only) is normalized away on both sides, so the exemption list closes without documenting a carve-out. The judge is advisory and disarmed by default (zero hot-path cost); it is armed in tests today, and production arming on real traffic is deliberately left as an owner decision (perf/event-noise tradeoff and the point at which a config knob would enter). CI enforcement is active: a seeded host-behavior generator (append/edit/retry/model-switch/restart) drives the real proxy through the judge with zero false positives, and injected rewrite/reorder/drop/mid-insertion faults are all caught. Tests: unit oracle (positive controls + five-shape negatives + fold boundary) and a real-proxy e2e generator + defect fixtures. Fixes nothing upstream; the referenced defects (#3805/#2483/#3691/#3769/#3676) stay scheduled separately per the issue.
…entity (#3842) Review findings on the I1/I2 judge (PR #3866): - The call site sat right after runPrepare(), BEFORE preflight compression. A fold-CREATING request was judged on its pre-fold bytes, so I2 never reconciled the one traffic class it exists for. Move the call to the resolved prepared value (after preflight / reforge), where prepared.body is the exact byte stream relayed upstream. - The scan skipped every exit carrier while the entry side only skipped reasoning: a stateful host that re-sends previously-injected acp_summary / chain-checkpoint carriers false-positived on every steady-state post-fold turn (even entry==exit). Carrier entries now match their own twin by identity, and the gap budget counts non-carrier elements only. - The sink delivered violations only, so folded-but-clean verdicts were invisible and the live I2 path could not be asserted. Every judged request now reports its verdict (test-only surface; prod stays disarmed). Tests: unit fixtures for carrier echo (summary + chain), five-consecutive carrier gap, and the carrier-text-drift design boundary; e2e gains a real preflight fold through the live path (tiny window) - zero false positives across the folded stream, and post-fold prefix corruption fires.
Pre-existing on master (introduced by #3811): the function has no importers outside its own file, so the CI unused-exports gate fails on every head that reaches it - which skips all downstream test lanes. In-scope fix for this PR's merge commit to go green.
Pre-existing on master: the generated table still carried a superseded storage.unified row (pre-#3818 'unset (shadow)' default) plus an older sqliteAutoDownload row missing the #3788 cooldown text. The generator now emits only the current rows; docs-sync gate passes again. No config surface change - regenerated output only.
7a08a0a to
f729104
Compare
(ai: vllm-qwen/qwen3.8-27b) 中间状态(非最终结论):复审发现 2 个真实缺陷 + 2 个阻塞 CI 的既有问题,已在本地修复并验证;push 排队等 GitHub egress 恢复。
本地验证:定向测试 20/20 绿、tsc 干净、两个 gate 干净。全量套件按 §7.2 交给 CI。生产行为零变化:judge 默认仍 disarmed,无新配置面。 当前阻塞:GitHub git egress 暂时不可达(API 侧正常),后台重试 push 进行中;push 成功后补八问结论 + CI 结果。 |
(ai: vllm-qwen/qwen3.8-27b) 复审完成:原实现有 2 处「接线了但没生效」的真实缺陷 + 1 处范围缺口,已全部修复并推送(
CI 与附带处置:
判定:可合并。代码、测试、CI 全部闭环;唯一遗留决策项是生产是否启用 judge(建议暂缓,先小范围武装观察事件量与 CPU 开销再定)。 |
…entity (ranxianglei#3842) Review findings on the I1/I2 judge (PR ranxianglei#3866): - The call site sat right after runPrepare(), BEFORE preflight compression. A fold-CREATING request was judged on its pre-fold bytes, so I2 never reconciled the one traffic class it exists for. Move the call to the resolved prepared value (after preflight / reforge), where prepared.body is the exact byte stream relayed upstream. - The scan skipped every exit carrier while the entry side only skipped reasoning: a stateful host that re-sends previously-injected acp_summary / chain-checkpoint carriers false-positived on every steady-state post-fold turn (even entry==exit). Carrier entries now match their own twin by identity, and the gap budget counts non-carrier elements only. - The sink delivered violations only, so folded-but-clean verdicts were invisible and the live I2 path could not be asserted. Every judged request now reports its verdict (test-only surface; prod stays disarmed). Tests: unit fixtures for carrier echo (summary + chain), five-consecutive carrier gap, and the carrier-text-drift design boundary; e2e gains a real preflight fold through the live path (tiny window) - zero false positives across the folded stream, and post-fold prefix corruption fires.
…ontinuity; bound the exemption list (ranxianglei#3938 PR-A) - Arm by default (ranxianglei#3938 decision 2): violations log at warn, clean verdicts at debug, never blocks. Knob diagnostics.invariantJudge / BILI_INVARIANT_JUDGE=0 to disarm (CONFIGURATION en+zh). - Move the main-lane hook from the pre-send site (handle.ts) to the universal egress chokepoint (relay.ts noteForwardedBody block): judges the EXACT bytes after compat-role / steering / chain-stamp / shed rewrites. - New loop-lane continuity check (loop/core.ts fetchUpstream): rounds 2+ have no client inbound, so I1's entry ground truth is the previous outbound; clipped/oversized forensics silently skip. - ranxianglei#3842 count-bounded exemption list: carrier/reasoning/crush volume rides every verdict (exempt ledger) and beyond the cap (128 default) is itself an exemption-overflow violation. - e2e now proves all three wires live (openai + anthropic + google): zero false positives on legitimate growth, teeth on injected mid-history defects. Rebased from ranxianglei#3866 (its 2 content commits ride as the first two commits of this branch).
Closes #3842
What this is
Implements the owner-approved route from #3842 (the corrected direction after the PBT/SMT layer was rejected): judge I1/I2 on real traffic + a general host-behavior generator, with the existing cache-proof scripts demoted to a regression archive.
Two machine-checkable invariants, reconciled on the live request path:
[earliest fold coverage .. end); the frozen prefix is identical; the carrier position is pinned (drift = violation).Why exit cores are re-parsed from the body
The outbound body is built from
rebuiltMessages, notprocessedMessages(which is bookkeeping only). Re-parsingprepared.bodythrough its own wire codec makes the comparison reflect exactly what crossed the wire, and unifies same-wire vs cross-wire through the kernel canonical core (#3658). Render-tag scaffolding (renderTags=text-only) is normalized away on both sides, so the exemption list closes without documenting a carve-out — no change to outbound bytes.Enforcement split (deliberate)
tests/invariant-judge-e2e.test.tsarms the judge and drives the real proxy through a seeded host-behavior generator (append / edit-prior / retry / model-switch / restart). Legit stream → zero false positives; injected rewrite/reorder/drop/mid-insertion faults → all caught. These are the acceptance fixtures proving the generator + oracle find the shape family.maybeRunInvariantJudge; it is a no-op until armed. Arming on prod traffic is left as an owner decision because it trades CPU (a codec re-parse per request) against event volume, and it is the point at which a config knob would legitimately enter.Config surface
No new config field, env var, or CLI flag is introduced. The judge is enabled only through the existing
_set…ForTestidiom (tests). This keeps the change inside the owner-gated config-surface discipline — production enablement, if wanted, gets its own design pass first.Behavior changes
None in production (advisory + disarmed). Only additions: one module, one guarded call site in
src/server/handle.ts, two test files. The call site is guarded (prepared && protocol && !countTokens && !responsesCompact) and the judge never throws or mutates the body.Regression review
handle.ts: the added line runs afterrunPrepare()and before relay; when disarmed it returns immediately (one boolean check). No path/timing/output change for production traffic.bodyorprocessedMessages.Tests
tests/invariant-judge.test.ts— unit oracle: positive controls (identity, render-tag normalization, tail append, carriers, reasoning drop, crush envelope, tool-call id re-mint) + five-shape negatives + fold boundary. Green locally (targeted).tests/invariant-judge-e2e.test.ts— real-proxy generator + defect fixtures. Green locally (targeted).