bench: add counterbalanced performance regression runner

[morluto]

Summary

LeanToken's benchmark examples measure one revision at a time, but they do not produce a same-host base/head regression decision. This adds an opt-in runner that:

• builds clean detached base and head worktrees with Rust 1.95 in release mode;
• collects at least 10 counterbalanced AB/BA pairs for retrieval p50/p95 and selected indexing p50/p95 measurements;
• rejects toolchain, source, corpus, report-schema, operation-count, or observable-response mismatches before comparing timing;
• retains raw reports, logs, provenance, Benchstat input/output, and the final decision as one artifact;
• runs only through manual workflow dispatch, while ordinary CI tests the comparator contract.

The orchestration borrows the clean-worktree and counterbalanced-run pattern from justrach/codedb#675, but delegates medians, confidence intervals, and significance testing to pinned Benchstat rather than implementing statistics locally. Collection is paired to reduce order bias; Benchstat's comparison remains its independent-sample Mann-Whitney test.

The branch also initializes continuation_cursor in the Windows-only junction-containment fixture. That initializer was missed when the field became required on main, so the Windows test target otherwise failed to compile.

Decision rule

Thresholds live in benchmarks/paired_performance.json and are set per operation. A row fails only when the head median exceeds both its relative and absolute regression thresholds and Benchstat reports a significant change.

• NOISE: relative threshold exceeded, absolute threshold not exceeded
• INCONCLUSIVE: both materiality thresholds exceeded, significance not established
• FAIL: both materiality thresholds exceeded and significance established

hot_path_bounds hashes the complete final response for each timed operation after timing. The runner uses one fixed disposable repository path across samples so repository identity is stable and the hashes are directly comparable. Indexing parity covers the frozen corpus and observable operation counts.

Testing

• cargo fmt --all -- --check
• cargo clippy --all-targets --all-features -- -D warnings
• cargo test --all-features --lib --bins --test integration
• cargo test-extras
• cargo test --all-features --doc
• PYTHONDONTWRITEBYTECODE=1 python3 -m unittest scripts/test_paired_performance.py
• Ruff formatting/lint, ShellCheck, JSON/YAML parsing, and git diff --check
• Two independent hot_path_bounds executions produced identical observable-response hashes
• Reduced-corpus 10-pair end-to-end smoke produced 20 raw sample directories, passed parity, and produced a passing Benchstat decision

The reduced-corpus smoke validates orchestration only; it is not performance evidence for LeanToken.

Impact

• MCP schema change (snapshot diff required)
• Storage migration
• CLI interface change
• Token budget behavior change
• No breaking changes

The CLI change is an additive --repository-root option on the benchmark example, used to keep response identity stable across processes. Product CLI behavior is unchanged.

Suggested review order

1. benchmarks/paired_performance.json and benchmarks/run_paired_performance.sh
2. scripts/paired_performance.py and its behavior tests
3. exact response parity in examples/hot_path_bounds.rs
4. the manual workflow, CI wiring, and documentation

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.