Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ Notable changes to house-rules. Versions follow [semantic versioning](https://se
before the first run. Results always carry the run count and range, never a bare average. The
expected wall-clock time is stated before the first run. The JavaScript reference no longer asks
for at least 10 runs when timing startup.
- `hr-agent-guidance-audit` now says its pass/fail rule screening (4 or more runs, 9 per arm for a
smaller gap) is a deliberate exception to the 3-runs-per-arm default for performance claims in
`hr-benchmarking`.

- The `hr-code-review` panel recommendations gain an entry for Muse Wonder Full 1.0 on the Bug Hunt
board: two runs each at xhigh (41.5 of 105) and max (42.0), no published price, and reachable only
Expand Down
9 changes: 6 additions & 3 deletions skills/hr-agent-guidance-audit/references/prune-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,12 @@ request does not need.
confirm none of your own rule files appear; a few known extras (Devin's own
global `AGENTS.md`, OpenCode's installed-skill list) are listed in the
README. A leak makes the "without" arm obey the rule anyway.
- Use `--runs 4` or more. Four runs separate a rule only when the arms differ
by 4 and 0; a smaller gap needs 9 runs per arm before you act on it
(8 of 9 against 3 of 9 gave p = 0.05).
- Use `--runs 4` or more. This pass/fail screening is a deliberate exception
to the 3-runs-per-arm default for performance claims in `hr-benchmarking`,
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
because a count of passes needs more runs than a timing range does. Four
runs separate a rule only when the arms differ by 4 and 0; a smaller gap
needs 9 runs per arm before you act on it (8 of 9 against 3 of 9 gave
p = 0.05).
- Read the output: "rule helps" keeps the rule; "no difference" on all
models makes it a removal candidate; "fails with the rule" means fix the
scenario first, because it tests nothing. A scenario that fails in both
Expand Down
Loading