Skip to content

feat(bench): Print how far each forced decision cleared its bound - #395

Merged
aywrite merged 1 commit into
masterfrom
instruments/forced-scout-gap
Oct 10, 2026
Merged

aywrite merged 1 commit into
masterfrom
instruments/forced-scout-gap

Conversation

@aywrite

@aywrite aywrite commented Oct 10, 2026

Copy link
Copy Markdown
Owner

A forced row said which shortcut was taken, not how narrowly. The search is fail soft, so each decision's own search leaves a score on the far side of its bound. That distance is what root flips are read against.

Each row now carries a gap column after alpha_gap, zero at the bound:

  • trusted scout: alpha less the scout's score;
  • null move: the pass's score less beta;
  • reverse futility: the margin's floor less beta;
  • skip: -, since it searched nothing.

What a reviewer should know

  • The search is unchanged. Bench 11,088,536, the same as master. The scout score and the pass's score were already in hand at the call sites and are only passed through to the cold recorders.
  • Row format: twenty-one columns before the fen, where there were twenty. No script in scripts/ parses forced rows. docs/INSTRUMENTS.md describes the new column.
  • Tests: tests/forced_session.rs reads the column and checks that it is a non-negative integer on every kind but a skip, and - on a skip. Release and debug tests, clippy with and without machine-test, and fmt are clean.
  • Two readings used it: the trusted scout (55,158 rows over 10,000 game positions) and the null move (69,138 rows). In both, the root's move flips far more often near the bound (4.5% and 5.8% at the smallest gaps, under 1% at the largest). Valued five plies deeper, though, the flips are as often gains as harms, so neither reading led to a search change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01A57SbXV3vACDuNUX5zSKPQ


Generated by Claude Code

A `forced` row said which shortcut was taken, not how narrowly. The
search is fail soft, so each decision's own search leaves a score on the
far side of its bound, and its distance from that bound is the obvious
thing to read root flips against: a scout that came back just under
alpha, or a pass that just cleared beta, is the one a deeper search is
most likely to overturn.

Each row now carries `gap` after `alpha_gap`: for a trusted scout the
alpha less its scout's score, for the null move the pass's score less
beta, and for reverse futility the margin's floor less beta, each zero at
the bound. A skip searched nothing and prints `-`. The search is
unchanged and the bench is 11,088,536. The session test reads the new
column and checks that it is never negative and is `-` only on a skip.

Bench: 11088536
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A57SbXV3vACDuNUX5zSKPQ
@github-actions

Copy link
Copy Markdown
Contributor

Speed against 2d7fe54b

Measured on AMD EPYC 7763 64-Core Processor.
Both sides built with rustc 1.99.0 (b940084d7 2026-09-28).

Both sides built and run on this runner in this job, the way
scripts/speed.sh measures a perf commit. Each round runs
both sides on a layout of its own, the same compiled code with
its code and data shuffled and moved, so the 95% interval carries
where the code landed as well as the run. The verdict holds the
interval against a 1% threshold. The default layout, the one a
release ships, is measured after as a diagnostic. The node
counts are the search's: they move when the search does, and a
speed change leaves them alone.

round     base nps  candidate nps  change
    1      4095426        4166124   +1.7%
    2      4195972        4184195   -0.3%
    3      4180029        4157671   -0.5%
    4      4202621        4141803   -1.4%
    5      4186108        4205123   +0.5%
    6      4200405        4300827   +2.4%
    7      4152081        4176373   +0.6%
    8      4261874        4211102   -1.2%
    9      4281970        4217204   -1.5%
   10      4136172        4257846   +2.9%
   11      4150143        4200161   +1.2%
   12      4175677        4157059   -0.4%
   13      4256276        4219955   -0.9%
   14      4204641        4177596   -0.6%
   15      4202977        4133549   -1.7%
   16      4151936        4143636   -0.2%
   17      4152849        4162210   +0.2%
   18      4167262        4185104   +0.4%
   19      4164328        4242770   +1.9%
   20      4183110        4124027   -1.4%
   21      4076532        4194059   +2.9%
   22      4187662        4153151   -0.8%
   23      4199619        4192626   -0.2%
   24      4213677        4266260   +1.2%
   25      4200888        4213009   +0.3%
   26      4167909        4130990   -0.9%
   27      4150600        4202879   +1.3%
   28      4676069        4242903   -9.3%
   29      4798779        4728693   -1.5%
   30      4514010        4625915   +2.5%
   31      4700059        4622181   -1.7%
   32      4818000        4380379   -9.1%
   33      4801258        4801117   -0.0%
   34      4798779        4697216   -2.1%
   35      4855158        4649772   -4.2%
   36      4749380        4810218   +1.3%
   37      4792348        4787345   -0.1%
   38      4769970        4696800   -1.5%
   39      4786010        4862876   +1.6%
   40      4733425        4786967   +1.1%

              nodes    time  median nps  faster half
base       11088536  2.64 s     4201754      4570864
candidate  11088536  2.63 s     4212056      4520513
change                            +0.2%        -1.1%

paired change -0.1%, 95% interval -0.7% to +0.4%
diagnostic, on the default layout alone -2.2%, 95% interval -3.6% to +0.0%, 13 rounds

no change beyond ±1.0%: the whole interval is inside it

Speed: -0.1% (bench nps, 95% interval -0.7% to +0.4%, 40 interleaved rounds over shuffled layouts vs 2d7fe54b)

The instructions each side's bench executed, counted under
cachegrind. The count repeats to within a few hundred
instructions, so a small change here is a real one, but it
prices instructions only: cache misses, mispredicted branches
and where the code lands are the speed's to show.

instructions against 2d7fe54b, one cachegrind run a side, bench

             instructions     nodes  per node
base       16,519,608,737  11088536    1489.8
candidate  16,519,610,926  11088536    1489.8
change             +0.00%              +0.00%

within ±0.7%, as far as edits not made for speed have moved the count

@aywrite
aywrite merged commit 5d1bb6e into master Oct 10, 2026
21 checks passed
@aywrite
aywrite deleted the instruments/forced-scout-gap branch October 10, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants