fix(heldout): declare decision-latency repetitions per context - #1096
Conversation
Paired ranking timings must fail closed without a positive per-context repetition count, and the declared count must be the actual timing loop.
Remove the hidden 200-repetition timing default. The harness run still writes 200 as this run's choice and records latency_repetitions_per_context.
ADR 0047 is Proposed. Production route/conduct defaults stay locked.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: Seongho Bae <me@seonghobae.me>
|
Normal integration checkpoint b8a7b3b: retains both predecessor8ccf011dbf74f7140dd29fc2661234117c7cb36b and repaired parent318370e84aa4a2a2d280493057a6b19b23484a7e. Only the gap-baseline insertion conflicted; both entries were preserved. Required latency repetitions remain in place, while censored no-detection semantics and parent static-analysis repairs are retained. Integrated boundary and ADR identifier tests:23 passed in33.65s, exit0. This is local focused evidence, not protected-main merge, current-head hosted success, release or measured buyer latency. Draft/parent-first review boundaries remain unchanged. |
Stack
Draft successor of #1095 (
74c27e7e). Parents #1093, #1092, #1091, #1090, #1074, and psychometric KPI owner #1067 stay open. This PR does not close #1095.Exact head:
8ccf011dbf74f7140dd29fc2661234117c7cb36b.Problem
Paired held-out ranking timings hid
LATENCY_REPETITIONS = 200, so buyer-facing decision-latency p50/p95 used a repository-selected Monte Carlo sample size.Change
repetitions_per_contexton_measure_paired_latency. Missing, boolean, or non-positive values fail closed.latency_repetitions_per_context.Verification
Local smallest tests:
tests/test_psychometric_routing.py,tests/test_psychometric_benchmark_boundaries.py,tests/test_planning_adr_identifiers.py)This is declaration-contract evidence, not buyer-held-out accuracy, p95 latency, or protected merge.
Remaining no-heuristics work
Other harness sample sizes remain later slices. No production policy or release is authorized. Parent #1067 still needs independent review.