Repository navigation
test: wire-contract anti-regression system — parity fakes, goldens, unified ledger - #1305
Conversation
…nified ledger (#1304) Consolidates the #1301 served-surface gate and the #1302 kernel-export gate into one wire-contract suite and extends it per owner decision on #1304 (items 1+2+3, plus the 4-protocol scripted-client matrix phase): - tests/wire-contract-fakes.ts: WIRE_RULES ledger (WC-001..WC-006, each with production/doc provenance) + strictest-known validation for all four wires (anthropic /v1/messages, openai chat, responses, google v1beta) + four validation-parity fake upstreams on listen(0) with canned per-wire replies. - tests/wire-contract.test.ts: four layers — ledger self-check (every rule id has a live enforcement clause), A kernel exports, B bili-built surfaces, C plugin manifest (default + opt-in), D live proxy-mode forward of one client-shaped turn per protocol through the matching fake (asserts the forwarded tool set is exactly bili's intended surface and the fake's strictest validation accepts it). The anthropic lane reproduces #1299 end-to-end. Supersedes tests/kernel-pin-wire-schema.test.ts and tests/issue1299-anthropic-top-level.test.ts (all their assertions kept). - tests/wire-contract-golden.test.ts + tests/golden/wire-contract/*.json: byte-equal golden snapshots of every advertised tool schema at both layers (kernel exports, bili surfaces, plugin manifest default+opt-in); manifest version normalized to <VERSION> so release commits don't churn it. - scripts/update-wire-contract-goldens.ts: explicit regeneration path. - AGENTS.md §7.3: institutional rule — every upstream rejection/validation constraint becomes a permanent wire-contract rule inside the fixing PR; the ledger only grows.
📦 Built Package ArtifactBranch: Option A — Install from npm PR tag (recommended)npm install -g billion-context@pr-1305Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pr1305.tgz
npm install -g packageInstall instructions are refreshed on each push. |
[bot] 🏷 Review started. Pulling branch |
…align new ledger bullet)
…autocrlf) CI run 36102348905: all 4 golden tests failed on windows-latest because GitHub Actions checkouts materialize committed-LF files as CRLF in the worktree; runtime-generated canonical text uses LF. Normalize disk content to LF so the gate is platform-deterministic while still catching any real content drift.
…yte-equal gates pass on Windows CI
[bot] 🏷 ## Review verdict: mergeable — CI fully green on Reviewed against current master ( ① Direction (vs #1304) — PASSScope matches the owner-decoped items exactly: 1+2+3 + phase-(a) matrix, pure test/CI infra, zero runtime changes, deterministic ② Regression audit — both deleted gates preserved assertion-by-assertionI recovered both superseded files from
③ Code review — cleanFakes bind loopback only on ④ Test-tooth check — verified, not assumedSimulated the #1299 payload shape ( ⑤ Diff cleanliness — two issues found, both fixed directly on the branch
Post-fix CI on Behavior changes: none (rule 4 checked — no threshold/default/timing/format drift; test/CI signal only). Not merged — that part is yours: #1305 中文摘要:审查了 wire-contract 测试体系 PR——方向符合 #1304 决议、两个被删旧门禁的断言逐条保留且更强、测试有牙(模拟 #1299 负载确实触发 WC-001);发现并直接在分支上修了两个问题:AGENTS.md 既有列表项的缩进漂移,以及 Windows CI 上 golden 字节比对因 autocrlf 转 CRLF 必挂的确定性 bug(补了 |
…enai SSE lane Review of #1305 (mutation-verified) found the consolidation dropped the old #1302 assertion for the OpenAI chat and Responses tool shapes: top-level combinators were only checked on anthropic (WC-001) and google (WC-006), so an anyOf on COMPRESS_TOOL_OPENAI.parameters passed every rule test and only the golden byte-diffs caught it. Restores the ban as part of WC-004/WC-005 with provenance; re-running the mutation now fails 6 rule tests (was 0). Also: stream:true on the openai-chat smoke lane (activates the fake's SSE branch and exercises the real src/stream-openai.ts parse path — previously dead code), sse() no longer emits an empty event: line for OpenAI chunks, and AGENTS.md now points WIRE_RULES at tests/wire-contract-fakes.ts (where it is defined).
|
Independent deep review + fixes (head Review method: throwaway worktree, mutation testing against the historical bug shapes, golden idempotence checks, deleted-file assertion diffing. ✅ Headline — the system bites
🔴 Found & fixed: one dropped assertion (mutation-verified)The consolidation lost the old #1302 combinator ban for the OpenAI chat / Responses shapes — combinators were only checked on anthropic/google. Mutation C ( Fix in Minor fixes in the same commit
Noted, not blocking (follow-ups)
Gates on |
Symmetry with the WC-004/WC-006 probes: without a combinator-shaped responses body, deleting the restored WC-005 top-level-combinator loop would go undetected by the suite (the other two responses probes hit the name/type clauses only).
[bot] 🏷 Round-2 review of ✅ Your finding is correct — and my round-1 call was too lenientI re-derived the old #1302 gate's exact semantics from the deleted file (
And because Layers A/B/C/D all route through the same ✅ Minor fixes audited, all sound
🔧 One small fix I pushed directly (
|
…ate on Windows CI The two Windows test failures are pre-existing on the base 98fb91c (run 38034641775 fails the identical assertion): tests/issue2605-dsh-retry-patch.test.ts byte-pins '- id: compaction-basic\n config:\n auto: false\n' against the repo-checked-out dsh.bundle.patch.yml, and Git-for-Windows' default autocrlf=true converts it to CRLF at checkout, so includes() fails. The runtime-written overlay lane (test 2) is LF by construction and passes. Exactly the #1305 class: goldens got 'text eol=lf' for this; the yml was never covered. One attribute line, no behavior change on Linux.
…2605) tests/issue2605-dsh-retry-patch.test.ts reads the patch file raw and byte-matches multi-line rows; Git-for-Windows autocrlf=true converts it to CRLF at checkout, failing both windows CI shards on every run since #2610 landed (master red at 98fb91c, run 38034641775). Same mechanism as the #1305 golden pin above. In-passing fix carried on #2625.
…t includes (ranxianglei#2631) ranxianglei#2605's bundle test asserts byte-exact multi-line literals against the shipped dsh.bundle.patch.yml; Git-for-Windows checks it out CRLF (autocrlf) and the includes fail on every Windows leg since ranxianglei#2610 merged — master's test (windows-latest, 22|24) legs have been red since 07:30Z, blocking every PR's required checks (ranxianglei#2631). Same failure class as ranxianglei#1305 (golden JSONs): pin eol=lf in .gitattributes, and belt-and-suspenders normalize CRLF after read in the test so checkouts that bypass gitattributes still pass.
…anxianglei#1295) windows-latest shard 4 (run 43029697326) failed one sub-test: not ok 2458 - trae plugin pipe: a clean captured turn is relayed byte-verbatim (no synthesized error) The assertion is `assert.equal(out, REAL_TURN)` where REAL_TURN is tests/fixtures/trae-2026-11-04-two-tool-calls.sse.txt read straight off disk. The committed blob is LF, but .gitattributes had no rule for this path, so Git-for-Windows' autocrlf=true checked it out CRLF. The pipe normalizes SSE terminators to LF (src/sse-util.ts:normalizeSseLineEndings), so on Windows the comparison was LF-output vs CRLF-fixture: every single line differed, which is why the assert dumped the whole 9.6 KB turn as a 90-for-91 line replacement. Pinned the fixture to eol=lf, the same remedy the repo already applies to tests/golden/wire-contract/*.json (ranxianglei#1305), dsh.bundle.patch.yml (ranxianglei#2631), tests/golden/web-trajectory/*.svg (ranxianglei#2489), tests/golden/web-hitrate/*.svg (ranxianglei#3484) and tests/fixtures/vertex-anthropic/* (run 38982246862). No content change: the blob was already LF, so this only fixes the checkout. Linux and macOS legs were green throughout; nothing else moved.
What
Implements owner-decoped items 1+2+3 of #1304 plus the phase-(a) scripted-client matrix (4 protocols × proxy mode) — pure test/CI infrastructure, zero runtime code changes, no release-workflow changes.
Item 1 — validation-parity fake upstreams (
tests/wire-contract-fakes.ts)WIRE_RULESledger: WC-001..WC-006, each citing where the constraint was learned (WC-001/002 = the [bug] 0.1.148 Claude 请求因 compress 顶层 anyOf schema 被 Anthropic 400 拒绝 #1299 production 400 text + acp-kernel 重启伪造时间戳:buildSession 用 Date.now() 覆盖 lastSeen,面板"最后活动"全员开机时刻、fallback=latest 在 ~245 路平局里随机挂会话 #404/ACP_PASSTHROUGH 一键静默关死全部压缩:无日志/无横幅/面板不可见且不可清除,同族 env 旋钮全部生效于无声 #405; WC-003/004/005 = provider API references; WC-006 = the kernel's documented deliberate policy of keeping the Google wire a plain object schema).listen(0)(deterministic, loopback-only) enforcing the strictest known validation per wire and replying with canned per-wire responses (Anthropic SSE, OpenAI chat JSON/SSE, Responses full event sequence, Gemini v1beta SSE frames).startServerproxy into the matching fake and asserts: 200 round-trip, exactly one upstream request, zero validator violations, and the forwarded tool set is exactly bili's intended surface (+ client tools). The anthropic lane reproduces [bug] 0.1.148 Claude 请求因 compress 顶层 anyOf schema 被 Anthropic 400 拒绝 #1299 end-to-end (a reintroduced top-level combinator surfaces as a relayed 400).absorb/acp_rule/acp_retrieve, still legal under the fake's validation.Item 2 — golden schema snapshots (
tests/golden/wire-contract/*.json)Byte-equal snapshots of every advertised tool schema at both layers: kernel exports (all arrays + individual constants incl. optional absorb/image_full/retrieve variants), bili-built surfaces (
BILI_ACP_TOOLS_*, search_context/decompress extensions, RULE_TOOL*, retrieveToolsFor, IMAGE_FULL_TOOL_GOOGLE), and the plugin manifest (default + opt-in). Manifestversionis normalized to<VERSION>so release commits don't churn it. Drift fails with instructions; regeneration is explicit:node --import tsx scripts/update-wire-contract-goldens.ts, with justification required in the PR that changes a snapshot.Item 3 — unified gate + institutional rule (
tests/wire-contract.test.ts, AGENTS.md §7.3)Consolidates the #1302 kernel-export gate (
tests/kernel-pin-wire-schema.test.ts) and the #1301 served-surface gate (tests/issue1299-anthropic-top-level.test.ts) into one module — both files are deleted here and every assertion they made is preserved (layer A = kernel exports incl. the 'Do NOT pin this kernel version' failure text; layer B = bili surfaces incl. the compress call-form property check; layer C = manifest; plus a ledger self-check asserting every rule id has a live enforcement clause). AGENTS.md §7.3 gains the permanent rule: every upstream rejection/validation constraint discovered in production or docs becomes a wire-contract rule inside the fixing PR; the ledger only grows.Behavior changes
None. No runtime code touched; no config/persistence/wire-format changes. The only observable effect is new CI signal: future pin bumps or tool-surface changes that emit a wire-illegal schema now fail
npm testinstead of reaching production.Pre-flight (on current master, dc8bdea)
npm run typecheck✅tsc --noEmitover the new test files: clean (repo-wide root-tsconfig noise in pre-existing test files is untouched)npm test: 2569 tests, 0 failures ✅npm run build✅node --import tsx --test tests/*.test.tsglob, so every PR (incl. pin-bump PRs) inherits them automatically.