diff --git a/CHANGELOG.md b/CHANGELOG.md index 590d1029..f6dee629 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,9 @@ Notable changes to house-rules. Versions follow [semantic versioning](https://se before the first run. Results always carry the run count and range, never a bare average. The expected wall-clock time is stated before the first run. The JavaScript reference no longer asks for at least 10 runs when timing startup. +- `hr-agent-guidance-audit` now says its pass/fail rule screening (4 or more runs, 9 per arm for a + smaller gap) is a deliberate exception to the 3-runs-per-arm default for performance claims in + `hr-benchmarking`. - The `hr-code-review` panel recommendations gain an entry for Muse Wonder Full 1.0 on the Bug Hunt board: two runs each at xhigh (41.5 of 105) and max (42.0), no published price, and reachable only diff --git a/skills/hr-agent-guidance-audit/references/prune-workflow.md b/skills/hr-agent-guidance-audit/references/prune-workflow.md index 0e603c74..d054f14e 100644 --- a/skills/hr-agent-guidance-audit/references/prune-workflow.md +++ b/skills/hr-agent-guidance-audit/references/prune-workflow.md @@ -40,9 +40,12 @@ request does not need. confirm none of your own rule files appear; a few known extras (Devin's own global `AGENTS.md`, OpenCode's installed-skill list) are listed in the README. A leak makes the "without" arm obey the rule anyway. - - Use `--runs 4` or more. Four runs separate a rule only when the arms differ - by 4 and 0; a smaller gap needs 9 runs per arm before you act on it - (8 of 9 against 3 of 9 gave p = 0.05). + - Use `--runs 4` or more. This pass/fail screening is a deliberate exception + to the 3-runs-per-arm default for performance claims in `hr-benchmarking`, + because a count of passes needs more runs than a timing range does. Four + runs separate a rule only when the arms differ by 4 and 0; a smaller gap + needs 9 runs per arm before you act on it (8 of 9 against 3 of 9 gave + p = 0.05). - Read the output: "rule helps" keeps the rule; "no difference" on all models makes it a removal candidate; "fails with the rule" means fix the scenario first, because it tests nothing. A scenario that fails in both