Skip to content

feat(rung4): the boost stage — one sampled play-out per targeted infoset after the walk's rows, at (1 − α)·K·arrival with the walk's rows turned to α, so expected banked weight and regret are plain sampling's; 35 s an iteration at B = 500, never exact - #94

Open
AlecROndo wants to merge 59 commits into
mainfrom
AlecROndo/pr-4-boost-stage

Conversation

@AlecROndo

@AlecROndo AlecROndo commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

PR 4 of the targeted-sampling plan for rung 4: the boost stage, wired into train.step. After a seat's walks and before the merge, the stage draws this iteration's B target rows from PR 3's frozen TargetSet, computes one sampled boosted row per target under the σ the walks just read (PR 1's BoostTargets.targets with a Sampling), turns the walk's own rows at those infosets down to α, and offers the boosted rows after the natural ones at weight (1 − α) · K · arrival (PR 2's column). A targeted infoset's expected banked weight and regret are plain sampling's; whether its noise falls is what the race will measure (it does where one play-out's variance is under a walk sample's over K · p). A run without an aim is today's run bit for bit.

Branched from #93's head (PR 3), with main merged in twice — once at the start and once after #93 landed as a squash, with this branch's side taken where PR 4 had edited PR 3's lines — never rebased. The diff against main is byte-identical to PR 4's diff over PR 3's head.

Decisions, settled by Alec on 2026-11-09

  • α = 0. The walk's rows at a targeted infoset are offered at weight 0: inert in the fit, still offered so within a phase the walk's rows keep their reservoir stream positions.
  • B = 500, R = 25. The plan's 2,000 costs about five table iterations under sampling; 500 is about one. Both are now TargetSpec's defaults.
  • Draws ∝ arrival; leak on the frozen exact average. PR 3's provisional choices, confirmed. The leak source is now a TargetSpec field, leak, so a resume refuses a change to it by name.
  • No EXACT anywhere in the boost stage. Every boosted row is a sampled play-out, Sampling() by default: the whole opponent range in one vectorised sum, one second board card, one replacement per throw, one opponent action line. DeepConfig refuses an exact Sampling and inject refuses it again; EXACT is the tests' oracle only. The opponent-range count is a flag, so the mini-walk form (one dealt hand per play-out, M play-outs) is one argument away. Alec's rule: boosting is extra sampling at high-variance nodes, never brute-forcing the tree from a node.

What it adds

  • src/drawmaha_solver/deepcfr/boost.py (new, NumPy only, never imports the grader)
    • inject(rows, targets, *, drawn, exact, sigma, seat, t, seed, sampling, alpha, walks) -> Injected: the phase's advantage offer with one boosted row per treatable drawn row appended. Natural rows first, in walk order, at weight α where their row is treated and 1 elsewhere; boosted rows after, at (1 − α) · K · arrival in float32. A drawn row is treated all or nothing: one with zero arrival, or whose float32 weight would equal exactly 1 or α (PR 3's noise score tells walk rows from boosted rows by weight alone), is left untreated and counted in skipped. Refuses an exact Sampling, a repeated or unsorted drawn, a bad α or K.
    • Boost(refresher, exact=None): due(targets, t), refresh(t=, table=, memories=, spec=) (refuses a set built for another t or another spec), phase(targets, offered, table=, seat=, t=, seed=, sampling=, alpha=, walks=), which reads σ from the table handed to it and draws rows_for(t, seat, seed). The whole game's BoostTargets is built on first use (~5 s).
  • train.py
    • DeepConfig.target: TargetSpec | None = None and DeepConfig.boost_sampling: Sampling = Sampling(); both in ADDED_LATER (None, the default sampling) and SAMPLE_FIELDS; both travel as nested dicts in state.json and are rebuilt by config_from_saved, which refuses a nested dict missing a field or carrying an unknown one by name.
    • step(..., boost=None): materialise → observe → (seat 0, when due) refresh → collect → inject → merge (with the weight) → fit. Without a boost the merge call is unchanged.
    • DeepRun.targets; save_run/load_run carry targets-t{t:05d}.npz when the run has an aim; an old state without it loads with None.
    • run_deep(..., boost=None) refuses an aim without a boost and a boost without an aim, before the collector opens.
    • StepStats and the progress line gain seconds.refresh, seconds.boost[2], rows.boosted[2], rows.skipped[2], rows.turned_down[2] (zeros without an aim).
  • targeting.py: TargetSpec.leak ("average" | "current"), budget=500, refresh=25; the Refresher protocol is __call__(*, t, table, memories) — whatever else a refresher needs it is given when built, so the loop hands over only what it has.
  • scripts/deepcfr/targeting_refresh.py: GraderRefresher(spec, *, walks, average=None, seed=0, exact=None) reads spec.leak; average is required when the aim follows the average's leak and unused otherwise.
  • scripts/deepcfr/run_deep_cfr.py: --boost {none,random,infoset,line,line_class} and --boost-budget / -refresh / -alpha / -leak / -pool / -noise-cap / -round-two-only, plus the sampling of one boosted row, --boost-playouts / -opponent-range / -board-cards / -opponent-draws / -hero-draws / -opponent-actions (an int or all). An exact sampling is refused at the command line. With an arm named, main builds GraderRefresher around the run's ExactAverage and hands the loop a Boost. (--target was already the iteration to reach, hence --boost.)
  • Docs: deepcfr/AGENTS.md (the no-EXACT rule, the resume set, the config fields), deepcfr/__init__.py (eleven modules), scripts/AGENTS.md, train.py's iteration sketch and timings, reservoir.py's and targeting.py's docstrings (the boosted row is sampled, not exact), one README sentence, and the plan docs/superpowers/plans/2026-11-09-targeted-sampling-pr4-boost-stage.md.

Measured

Whole-game smoke (MINIDRAWMAHA_FULL_DEEP=1, the line arm at B = 500, K = 200, the oracle learner, leak="current"), this Mac:

seconds
refresh (two grader walks + the pool's arrivals) 99.1
boost, seat 0 / seat 1, t = 1 22.7 / 12.3
boost, seat 0 / seat 1, t = 2 21.7 / 11.9

About 35 s per iteration at B = 500, 500 rows a seat boosted and none skipped; at R = 25 the refresh adds about 4 s per iteration. That is half again over the 23 s PR 3's per-row costs predicted for B = 500 (and about one and a half of the plan's 18 to 23 s table iterations); B stays 500, as decided, and the gap is recorded in train.py's timings. The plan's B = 2,000 would be about 140 s. Seat 0 costs more because its draw lands more often on round-1 rows, whose play-out crosses the second board card and the draws.

An aim flag (--boost-budget and the like) with --boost left at none is refused at the command line, a fresh-context review finding: it would otherwise be today's run with nothing in the log.

Tests

  • uv run pytest (whole default suite, no torch): 1,932 passed, 25 skipped at the head after the review rounds (1a45946's code; the last commits are docstrings), 20 min 32 s with another suite run competing for the machine; 1,921 passed in 10 min 32 s at 72c3946 before the review fixes added eleven CLI and guard tests.
  • The oracle pin test_the_oracle_loop_is_lcfr_bit_for_bit passes unchanged.
  • New tests/deepcfr/test_boost.py (24 tests):
    • inject against hand-worked numbers: order, the weights at α = 0 and 0.5, a drawn row with no natural rows, the zero-arrival skip, the exactly-1-or-α skip at both α, the refusal of EXACT and of an exact-by-counts Sampling before any target is computed, repeated/unsorted/non-integer drawn rows, fewer rows than B.
    • Boost: due, refresh's refusals, phase reading σ from the table it is handed and drawing this iteration's rows, the lazy whole-game oracle.
    • The gate: on a restricted deal (seat 0 one hole, any opponent) with the oracle learner and the real BoostTargets, one iteration at 24 seeds (50 under MINIDRAWMAHA_FULL_TARGET=1) with and without the boost, same walks: at every targeted row (two opening rows and the four busiest later rows) the paired difference in banked regret is within 4 s.e. of zero at every legal column, the banked weight difference is within 4 s.e. of zero, the boosted weight is K · arrival to the bit every seed, and the boosted rows' mean is within 4 s.e. of the EXACT regret. Mutation checks (not committed): a K + 1 weight and natural rows left at weight 1 each fail it. About 30 s.
  • tests/deepcfr/test_train.py (+17): the config fields and their refusals, ADDED_LATER/SAMPLE_FIELDS, the nested JSON round trip, an old state without the fields, the refresh cadence (t = 1, 3, 5 at R = 2), boosted rows after the natural ones with the policy memory untouched (real BoostTargets), σ read after each phase's write (a stamping learner), the progress line, the target set in the resume and marks (not re-graded; refreshed on resume once expired), stop-and-restart equals straight, the two run_deep refusals, the CLI flags and the exact-sampling refusal, main's wiring, and the gated whole-game smoke.
  • test_targeting.py, test_targeting_refresh.py: leak, the new defaults, the refresher's constructor-time average.

Departures from the appendix

  • Flags are --boost…, not --target… (--target is the iteration to reach).
  • Refresher takes no average; GraderRefresher is built with it.
  • TargetSpec.leak exists so the leak source is part of the saved run.
  • No EXACT in the stage, per Alec; the appendix's "exact row" is a sampled boosted row throughout.
  • A boosted row whose weight would read as a walk row's is skipped rather than refused, so a fluke never kills a 15-hour run; it is counted and reported.

Not in this PR

PR 5, the race (launcher, grading, verdict, pre-registration), which should also record, per targeted row, one play-out's variance σ_b² against a walk sample's over the visits, σ_w² / (K · p): the boost is unbiased everywhere and quieter only where that inequality holds, and the race must tell "the boost helps" from "the boost is merely unbiased". Its pre-registration should also check that rows.skipped stays near 0 and read rows.turned_down (the walk's rows made inert at α = 0, the boost's cost in reservoir slots once eviction starts) at the chosen K. Nothing has been run with the boost beyond the smoke.

Pre-push review (fresh-context reviewer subagent), dispositioned

No correctness bug found; the weight arithmetic, the float32 guard, the σ timing, the offer order, the streams and the resume state were each re-derived and checked against the tests. Findings:

  1. Fixed. targeting_refresh.py's docstring still said the boost stage "computes exactly"; it now says one sampled play-out per row.
  2. Fixed. The cost claims written before the smoke (B = 500 "about one table iteration", 2,000 "five") now quote the measured 35 s and ~140 s, in TargetSpec's docstring, train.py's timings and a test comment.
  3. Fixed. train.py's sketch counted two boost lines; it has three.
  4. Fixed. An aim flag with --boost left at none was silently dropped; the parser now refuses it by name (7 new CLI cases).
  5. Fixed after the bot's round 1. step refuses a boost when the run has no aim, by name.
  6. Fixed after the bot's round 1. α = 1 is refused by inject and TargetSpec.

Set aside by the reviewer and ruled on: modal_rung4.py has no --boost path (PR 5's); the first refresh grades an exact average of one observation (a consequence of leak="average"); rows.advantage in the log now counts boosted rows among the offers, with rows.boosted beside it.

…every opponent hand and every physical entry of its key, and the walk's arrival rate, from the rules alone

PR 1 of the targeted-sampling plan. `minidrawmaha/solver/exact_target.py`
computes, for any table row, the regret the walk's samples there estimate
(the mean over the opponent's hands of u − ⟨σ, u⟩, weighted by the deck and
the opponent's own play, normalised by the arrival) and the probability one
walk visits the row. It re-derives the public tree, key maps, draw maps and
showdown from rules/ and never imports the grader, which grades what this
will feed.

Measured: a round-2 row costs 0.8 ms, the arrival alone 0.3 ms; a draw row
45–85 ms and a round-1 row 0.2–0.6 s, because the value below expands over
the traverser's own replacements and the second board card.

Tests: the back-solved reach equals the grader's carried vector on every
board of every point (1e-10, full and restricted deals); arrival at a seat's
first decision is the grader's frequency and sums to one; regret and
arrival equal a world-by-world walk through MiniState on a small restricted
game (1e-9) and the mean and visit count of 6,000 Collector walks row by
row within 4 s.e. and pooled; the module's import block holds no grader.
…, 5,190 vectors, re-derived by a test; the sampled tests select rows by expected count, so a row cannot enter because it fluctuated high

Review follow-ups on #89. The docstring counted the memo's level-1 points
alone (29) where the chance nodes on the paths are memoised too (66 in
all); the test now walks the ancestor closure and pins 66, 20 and 5,190.
`reach()` hands out a copy. The pooled z² windows are a 4-sd band on the
chi-square mean, √(2/n), instead of fixed numbers. The gated 200,000-walk
test kept rows by observed count ≥ 100, which let seat 1's draw rows in
only when they had fluctuated high (mean z +1.8 on two seeds); rows are
now kept by expected count ≥ 80 against an observed cutoff of 40, and
both seats read mean z within 0.01 of zero over thousands of rows. The
module docstring names the double-visit caveat and no longer points at a
plan outside the repo without saying where it is.
… the mean of z², and cap a single row at six standard errors — the deal alone dealt one picture 292 times against 220 at seed 12

The pooled mean is the sharp bias test (a tenth of a standard error in
every row is caught at 1,600 rows); the per-row cap is loose because of
about 3,000 rows one passes 5 in a run out of a thousand, and the deal
itself did it at seed 12 in a count that involves nothing of this module.
Run with MINIDRAWMAHA_FULL_TARGET=1: both seats pass.
…e band's assumptions written beside it — the "caught at 1,600 rows" line was a half-truth; the band is reached there and reliable at about 3,000
…9 at n = 10, biting from n ≈ 45 — and the arrival seeds checked named
…caller says otherwise; a file without the column loads as ones

PR 2 of the targeted-sampling plan, first of three commits. The reservoir's
row grows from (row, t, target) to (row, t, target, weight): 24 to 28 bytes,
a 50M-row memory 1.2 to 1.4 GB. The weight rides its row's slot through
Algorithm R's eviction, save and load; merge without a weight stamps ones;
a file with no weight key is from before the column and loads with ones.
Refused by name: a negative, non-finite, mis-shaped weight on merge, and a
weight array of the wrong length or dtype on load.

The learner seam still takes three arrays, so tests/deepcfr/test_train.py
is red until the next commit, which threads the column through it.
…y's loss bit for bit and a row of weight zero is inert

Second of PR 2's three commits. _weighted_loss scales each row's squared
error by (t · w) / mean(t · w): with w ≡ 1 that is t / mean(t) in the same
bits, since t · 1.0 is t in float32. fit_from_scratch and batch_loss take
the weight as a fourth positional array, None meaning ones, and refuse a
weight that is not 1-D, not the memory's length, non-finite, negative, or
all zero (nothing to fit, as an empty memory is). A batch whose every row
weighs 0 divides by a clamped mean and has loss 0, not NaN. fit_policy
takes the column positionally so the policy memory's contents() unpack
straight in; its weights are all one.
…cle banks Σ t′ · w · r in LCFR's order, so weight one keeps the pin bit for bit

Last of PR 2's three commits. Learner.fit takes the memory's four arrays,
the weight positional with no default so no learner can drop it by
accident; step is unchanged, the four-tuple from contents() flowing
through. NetLearner hands the weight to fit_from_scratch and the held
batch's loss, after the pot scaling and the regret rules act on the target:
the weight says how loudly a row counts, never what it says. The oracle
multiplies each sample by its weight in float64 before the per-t′ np.add.at
sum, where × 1.0 is exact, so the LCFR pin passes unchanged; a weight of 2
banks what the row offered twice would, a weight of 0 nothing. Oracle model
files carry the weight and load as ones without it.
… refused by name — the ones default ran ahead of the checks it relied on; and the weight-one pin is against the old loss formula, not the code itself

From the fresh-context review before the push. Reservoir.load sized the
default weight from a negative `offered` before refusing it; merge and
fit_from_scratch sized it from a 0-d rows before the 1-D check; a 0-d
weight on load indexed its own empty shape. Each now fails on the by-name
ValueError the old code gave. test_weight_one_is_the_unweighted_fit_bit_for_bit
compared the fit to itself (both paths built the same ones); it now
restates the pre-column loss, t / mean(t) over the legal squared error,
and pins batch_loss at weight one and by default against it to the bit,
then the whole fit.
…ight for overflow, on merge, on load and in the fit — a float64 weight past float32's range passed the finite check and was stored as inf; and the oracle's "offered twice" claim holds to the bit only when the two are alone at the column

- `_validate_offer` casts to float32 first and checks finite, non-negative and t × weight finite after; a bool or non-numeric weight is refused by name.
- `Reservoir.load` refuses a weight that is NaN, negative or overflows t × weight, naming the file, as `merge` never writes one.
- `fit_from_scratch` refuses a t × weight that overflows float32, the loss's scale.
- `OracleLearner` docstring: weight 2 equals the row offered twice exactly only when nothing else sits at the column in that t′; otherwise up to rounding, since `np.add.at` sums one at a time. A new test pins two rows of widths 2 and 3 with weights 3 and 2 against the rows offered three and two times, so a weight landing on a neighbour's columns would show.
- `batch_loss` says why it defaults the weight while `Learner.fit` does not; `NetLearner.fit` notes the held loss is a log line nothing decides on.
…high-noise row — board card, both seats' replacements, the opponent's actions and range — each unbiased against the exact sum, which stays as EXACT, with the arrival always exact and one stream per (seed, t, seat, row)

exact_target.py becomes boost_target.py (ExactTargets → BoostTargets). targets()
takes a required Sampling; EXACT reproduces the previous output bit for bit.
… one branch at a time, averaged, is the exact sum to 1e-10 — and the range draw is tested where a key's two entries face different ranges; the stream's tail is two words, not one, and seed and t take any integer
…le's rows — 141 lines and 3,870 line × hand classes, from the rules alone
… √(E[r²]/p)/|E[r]|, from the memory's own walk rows only — an exact row is a mean, not a sample
…nfoset and take them until the pool is 20 × B reached rows; infoset takes the B best outright
…rrival by exponential keys, reproducible from (seed, t, seat, stream) and never the same row twice
…rozen target set, asking for arrivals at the pool alone; the Refresher protocol is the seam PR 4 calls
… — None, or a chance count at its fan — so it needs no seed and refuses extra play-outs; counts take any integer type, as seed and t do; the stream comment says when six words hold
…ound 1's and hold 19% of LCFR-2M's exploitability on 0.07% of its reached infosets; the 20 leakiest hold 57% on a quarter
…ds as zero; a real negative is still refused — the LCFR-2M release has gains at −7e-12
…, by choice, the current strategy), weighs the pool by the walk's arrival, and is the only importer of the meter on the trainer's side
@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 49s —— View job


Review of PR #94 (boost stage)

I read boost.py in full and the train.py diff. I did not read the tests, targeting*.py or the CLI, and I ran no tests. I found no outright correctness bug. The findings below are ranked by severity, each with a confidence level.

1. "Removes noise, never adds emphasis" is not guaranteed (medium severity, medium confidence).

  • boost.py:13-24 and the module docstring say the weight and regret are plain sampling's in expectation. I re-derived that, and it holds: α·Kp + (1−α)·Kp = Kp.
  • The variance claim does not follow. A boosted row is one sampled play-out carrying weight W = (1−α)·K·p.
  • Its contribution to the refit has variance W²·σ_b², where σ_b² is the variance of one play-out.
  • The Kp natural rows it replaces had variance Kp·σ_w², where σ_w² is the variance of one walk sample.
  • The boost therefore only quiets the infoset if σ_b² < σ_w²/(Kp). That is a per-row condition, and it is hardest to meet on the highest-arrival rows, which are the busiest and most likely to be targeted.
  • With the default Sampling() (whole opponent range, one second card, one action line), σ_b² could be well below σ_w². Nothing here measures it.
  • The gate test only checks that the mean is unbiased. It does not check that the variance falls.
  • Suggestion: soften the docstring to "unbiased; noise falls when σ_b² < σ_w²/(Kp)", or add a variance check to the gate. PR 5 would be a natural place for one.

2. "The reservoir sees the same offers as a plain run" is overbroad (low severity, high confidence).

  • This appears in boost.py:26-33, the deepcfr/AGENTS.md rule, and probably elsewhere.
  • Natural rows keep their stream positions within a phase, because boosted rows come after them.
  • Each phase still adds up to B extra offers, so the offer count, and therefore every later phase's Algorithm R draws, differ from a plain run's.
  • At α = 0 the inert natural rows also occupy reservoir slots at zero weight, which dilutes effective capacity.
  • The bit-for-bit claim is true only for target=None. Please say so, and say that the two memories are equal in expectation, not row for row.

3. The weight guard is not randomised (low severity, medium confidence; boost.py:164).

  • A row skipped because its float32 weight equals exactly 1 or α is skipped on every iteration at that σ, not at random.
  • It is unbiased, as the docstring says (the row is "not targeted"). It is also a systematic hole in which rows can be boosted.
  • This is negligible for K = 200. At small K, such as a K = 1 test with arrival = 1, it could skip a row that was drawn deliberately.
  • It is already counted in skipped, so I would only mention the systematic nature.

4. Deferred items (agree with the PR's disposition).

  • α = 1 pays for B boosted rows of weight 0. It is cheap to refuse in inject (alpha < 1) rather than defer. The weight-0 rows would also pass the weight != alpha check, since α = 1 ≠ 0.
  • step with boost but no config.target raises a bare AttributeError. A one-line ValueError guard at the top of step would match the run_deep refusals.

5. Tests (low severity, medium confidence).

  • The gate uses 24 seeds and a 4 s.e. band over several rows and columns. That is loose enough that a small bias, such as a few percent of weight, could pass.
  • The mutation checks you describe (K+1 weight, natural rows left at weight 1) are not committed. Consider committing them as parametrised "must fail" tests, or at least recording the effect sizes they detect.

6. Pedagogy and scope nits.

  • DeepConfig.boost (a Sampling) next to the Boost class, the boost= argument and the boost.py module is easy to misread. boost_sampling would be clearer.
  • Docstrings cite "Alec's rule (2026-11-06…)". The design record is better served by stating the reason (the exact subtree sum is intractable and sidesteps what the network is for) and leaving the person's name out.
  • The cost claim "35 s an iteration, never exact" in the title is a single-machine smoke measurement. The docstrings do label it as measured on a Mac, which is fine.

What I checked and found fine.

  • The offer order: natural rows first, boosted rows after.
  • isin treatment of natural rows applies only to treated rows.
  • The drawn validation (sorted, distinct, integer).
  • The α, K and Sampling refusals.
  • The refresher's built_at and spec checks.
  • Saving targets before state.json, with state.json written last as the commit point.
  • ADDED_LATER and the nested-config rebuild with named refusals.

Not run: uv run pytest. The PR reports 1,921 passed, and CI runs no tests.
· branch AlecROndo/pr-4-boost-stage

…lay-out's variance is under a walk sample's over K·p, a boosted run matches a plain one in expectation not row for row, α = 1 is refused, step refuses a boost without an aim, and the sampling field is boost_sampling
@AlecROndo

Copy link
Copy Markdown
Owner Author

Round 1 of the review, dispositioned in c3a0135.

  1. "Removes noise, never adds emphasis" — fixed. Agreed: the mean is unbiased by construction, the variance is not a guarantee. boost.py's docstring now states the condition — the boosted row enters at W = (1 − α) · K · p with variance W² · σ_b² against the K · p walk rows' K · p · σ_w², so the infoset is quieter only where σ_b² < σ_w² / (K · p), hardest at the busiest rows — and says the race (PR 5) measures it rather than this module assuming it. targeting.py's and the README's "quieter" wording is now "the opponent's hidden hand summed out", and deepcfr/AGENTS.md carries the condition. PR 1's boost_target.py already said a low-k form is not lower-noise by construction; this PR now agrees everywhere.
  2. "The reservoir sees the same offers as a plain run" — fixed. Now: within a phase the natural rows keep their stream positions; across phases each boosted phase adds up to B offers, so later draws and the resident set differ, and the two memories agree in expectation, not row for row (to t = 200 at the run's capacity nothing is evicted, so every offer of either run is resident). Only target=None is bit for bit. Fixed in boost.py, reservoir.py and deepcfr/AGENTS.md.
  3. The weight guard is deterministic, not random — said. boost.py's docstring now notes the skip repeats at the same σ until σ moves the weight off 1 or α, harmless at a measure-zero coincidence in float32 and worth knowing when a test draws a row deliberately at K = 1. It stays counted in rows.skipped.
  4. α = 1 and the step guard — fixed, not deferred. inject and TargetSpec refuse α = 1 by name (α in [0, 1)), and step refuses a boost when run.config.target is None with the same message run_deep gives; three tests.
  5. Tests — the weight is pinned exactly, not only to 4 s.e. The gate asserts the boosted banked weight equals K · arrival in float32 at every targeted row on every seed, which is what catches a K + 1 weight (a 2.5% bias that did sit inside 4 s.e. of the visit count at arrival 1/12 in the uncommitted mutation check, and the reason the exact pin was added — the plan's ledger records it). The 4 s.e. band covers the regret, whose resolving power is stated in the test's comment (about 0.8 standard deviations of one iteration's banked regret at 24 seeds; 50 under MINIDRAWMAHA_FULL_TARGET=1). Committing "must fail" mutation tests would mean monkeypatching the stage's internals to test the test; not done.
  6. Names and voice — fixed. DeepConfig.boost is DeepConfig.boost_sampling (the --boost-* sampling flags map to it; boost.Boost and step(boost=) are the stage). The rule is stated by its reason — the exact subtree sum costs a hundredfold at a round-1 row and, as a census of the tree, sets aside what the network is for — with the date it was settled, not a name. The title's 35 s is the Mac smoke, as the docstring labels it.

Tests: tests/deepcfr/test_boost.py, test_targeting.py, test_targeting_refresh.py, test_train.py and tests/test_agent_docs.py: 250 passed, 2 skipped.

@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 22s —— View job


Review of PR #94 (boost stage), updated for the current head

I read boost.py in full and checked the train.py guards. I did not re-read the tests, targeting*.py or the CLI, and I ran no tests. I found no correctness bug.

The earlier review on this PR was written against an older commit. These are now resolved in the head:

  • The variance caveat is stated correctly (boost.py:23-30: quieter only where σ_b² < σ_w²/(K·p)).
  • The reservoir-offer claim is scoped to target=None (boost.py:32-43).
  • The deterministic weight skip is documented (boost.py:55-58).
  • α = 1 is refused (boost.py:168).
  • step raises a named ValueError when given a boost without an aim (train.py:497).
  • The sampling field is now boost_sampling.

What I re-derived and found correct

  • Weight: α·Kp + (1−α)·Kp = Kp. The expected banked weight is plain sampling's.
  • Regret: the boosted row is an unbiased sample of the same conditional mean.
  • Offer order: natural rows first, boosted rows after. np.isin only turns down natural rows at treated infosets.
  • Float32 guard: it compares the float32 weight to float32 1 and α, which is the same representation targeting.noise sees.
  • σ timing: σ is read from the table right after learner.write.

Remaining findings

  1. The boost's noise reduction is still unmeasured (low severity, high confidence).

    • The docstring is honest about it, but the gate test only checks bias.
    • The PR title and body say "the noise" is what changes. That is true only conditionally, so consider rewording the title's implication in the PR description.
    • PR 5 should record σ_b² against σ_w²/(K·p) per targeted row. Without that, the race cannot separate "boost helps" from "boost is merely unbiased".
  2. α = 0 makes natural rows inert but they still occupy reservoir slots (low severity, medium confidence).

    • Zero-weight rows dilute the reservoir's effective capacity. This holds at the 50M capacity once eviction starts.
    • The docstring says "inert, kept for the count", and the design is deliberate. Please add a sentence noting the capacity cost.
  3. Zero-arrival rows are still computed (low severity, high confidence; boost.py:176).

    • exact.targets runs the play-out for every drawn row, then drops those with arrival 0.
    • If BoostTargets exposes arrival cheaply, filtering first saves the play-out. The PR reports none skipped on the whole game, so this is minor.
  4. The weight guard can skip a deliberate row (low severity, medium confidence).

    • At α = 0, a float32 weight that underflows to 0 equals α, so it is skipped. That is fine.
    • It is counted in skipped, so it is not silent. No change needed beyond the existing docstring note.
  5. Test strength (low severity, medium confidence).

    • The gate uses 24 seeds and a 4 s.e. band over many row and column cells. That is loose against a small bias of a few percent of weight.
    • The K+1 and natural-rows-at-1 mutations are not committed. Consider committing them as "must fail" tests, or recording the effect sizes in the test docstring.
  6. Formatting nits (low severity, high confidence).

    • boost.py:9 and boost.py:70 hold long lines in a docstring that is otherwise wrapped to about 78 columns.
    • The module docstring cites "settled 2026-11-06, restated 2026-11-09". The reason is already stated in the sentence that follows, so the dates can go. A docstring is a design record, and the dates are in the plan.

Pedagogy. The weight derivation and the all-or-nothing treatment argument are correct and clear.

Not run: uv run pytest. The PR reports 1,921 passed, and CI runs no tests.
· branch AlecROndo/pr-4-boost-stage

@AlecROndo

AlecROndo commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner Author

Round 2, dispositioned in 6d23ead (docstring only; an earlier version of this comment named c3a0135 before the commit had landed).

  1. Noise reduction unmeasured — the PR body no longer says the noise is what changes; it says whether the noise falls is what the race will measure, and the title claims only the weight, the regret, the cost and never-exact. Recording σ_b² against σ_w² / (K · p) per targeted row is on the PR 5 list in the body.
  2. Inert rows hold slots — said in boost.py: once eviction starts, the weight-0 rows cost their share of effective capacity, at most about B · t of the offers, under 1% of a 50M-row memory at t = 2,000.
  3. Zero-arrival rows computed — they are not: BoostTargets.targets skips the play-out when the arrival is 0 (boost_target.py, if total <= 0.0: continue), so such a row costs only its reach. inject's docstring now says so.
  4. Underflow to 0 at α = 0 — agreed, no change.
  5. Test strength — as in round 1: the boosted weight is pinned exactly (float32) at every targeted row on every seed, which catches any wrong factor; the 4 s.e. band is for the regret, with its resolving power stated in the test. Not committing mutation tests, which would monkeypatch the stage to test the test.
  6. Formatting — the long lines are wrapped and the dates are gone from boost.py's docstring; the reason stands on its own.

…pacity, that a zero-arrival row's play-out is never run, and wraps its lines
@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 29s —— View job


Review of PR #94 (boost stage), at head 6d23ead

I read boost.py in full. I did not re-read train.py, the CLI, the refresher or the tests this round, and I ran no tests. I found no correctness bug in the stage itself.

Resolved since the earlier rounds. I checked each of these against the current boost.py:

  • the variance caveat (lines 24-31)
  • the reservoir wording scoped to target=None (lines 38-44)
  • the deterministic weight skip (lines 59-62)
  • the α = 1 refusal (line 173)
  • the zero-arrival play-out now skipped by BoostTargets.targets, per the inject docstring
  • the boost_sampling rename

Re-derived and correct.

  • Expected banked weight is α·Kp + (1−α)·Kp = Kp.
  • Natural rows go first and boosted rows after, so natural rows keep their stream positions within a phase.
  • The float32 guard compares the float32 weight against np.float32(1.0) and np.float32(alpha), which is the representation targeting.noise sees.
  • drawn must be sorted and distinct, so no row is boosted twice.

Findings

  1. The capacity-cost arithmetic in the docstring looks wrong (low severity, medium confidence; boost.py:44-47).

    • The text says the inert natural rows are "about B · t of the offers at most, under 1% of a 50M-row memory at t = 2,000".
    • B · t = 500 · 2,000 = 1M, which is 2% of 50M, not under 1%.
    • The bound itself is also suspect. A treated infoset can hold about K·p natural rows per iteration (up to about K = 200 at a high-arrival row), not one. So the inert count per phase is roughly K·Σp over the drawn rows, not B.
    • Please recompute it, or quote the measured rows.advantage − rows.boosted share from the smoke. The repo's rule is that the reviewer re-derives docstring math, so a wrong figure here counts as a bug.
  2. The skip rule changes which rows can be targeted (low severity, medium confidence).

    • Skipped rows are counted in rows.skipped. Near weight == 1 (a K·arrival of about 1, so a very low-arrival row at small K) the skip is systematic.
    • Those rows silently fall out of the aim. The race should check that skipped stays near 0 at the chosen K = 200, and the PR 5 pre-registration should say so.
  3. The noise reduction is still unmeasured (informational).

    • The docstring is now honest about this.
    • PR 5 should log σ_b² against σ_w²/(K·p) per targeted row. Without it, "boost helps" cannot be separated from "boost is merely unbiased".
  4. The gate tests bias only, with 24 seeds and a 4 s.e. band (low severity, medium confidence).

    • The exact weight pin covers the weight. The regret band cannot resolve a few-percent bias, and the comment says so.
    • The mutation checks stay uncommitted, which is a defensible choice.
  5. Style nits (low severity, high confidence).

    • boost.py mixes ~78-column docstring wrapping with 110+ column code and docstring lines, for example lines 105, 174, 198 and 271. Please wrap them or confirm the project's line-length setting.
    • The module docstring says "never the exact sum over the tree" in a few places. The inject error text already states the reason, so the module docstring could drop one repeat.

Not verified. I did not check whether BoostTargets.targets returns arrival as the traverser-side expected visits per walk, which is what the K · arrival weight assumes. The gate test's exact weight pin is what covers this, so I trusted it.

Not run: uv run pytest. The PR reports 1,921 passed, and CI runs no tests.
· branch AlecROndo/pr-4-boost-stage

…ogress line reports them as rows.turned_down, in place of a capacity figure the docstring had wrong
@AlecROndo

Copy link
Copy Markdown
Owner Author

Round 3, dispositioned in 3ec9080.

  1. The capacity figure was wrong — fixed, and made measurable. You are right on both counts: B · t at t = 2,000 is 2% of 50M, not under 1%, and the inert count is the walk's rows at the treated infosets, about K · Σp over the drawn rows per phase, not one per drawn row. The docstring no longer quotes a number. Instead Injected carries turned_down, the natural rows set to α in that phase, StepStats and the progress line report it as rows.turned_down, and the docstring says the race reads that share rather than guessing it. Two tests pin the count (3 of 4 natural rows at the two treated infosets by hand; and, through step with the real oracle, the number of the plain run's rows at the drawn rows).
  2. The skip near weight 1 is systematic — only at exact float32 equality: K · arrival ≈ 1 is arrival ≈ 1/K, a common row at K = 200, and landing on 1.0 to the bit is a measure-zero coincidence, repeated at the same σ until σ moves it. It is counted in rows.skipped, and the PR body's PR 5 list now says the pre-registration should check rows.skipped stays near 0 at the chosen K.
  3. Noise reduction unmeasured — on the PR 5 list in the body, as per-row σ_b² against σ_w² / (K · p).
  4. Gate strength — as before; the weight is pinned exactly, the regret band's power is stated.
  5. Style — the lines over 120 columns I wrote are wrapped (the repo sets no line length; a few longer pre-existing lines in train.py are left as they were). One repeat of "never the exact sum" dropped with the dates in the previous round; the remaining two are the module's statement and inject's refusal, each in its own place.

@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 22s —— View job


Review of PR #94 (boost stage), at head 3ec9080

I read the diffs of 6d23ead and 3ec9080 in boost.py and train.py, and grepped the tests that cover turned_down. I did not re-read the rest of the PR and ran no tests. I found no correctness bug.

Resolved since my last round

  • Capacity arithmetic. The wrong "B · t, under 1%" figure is gone. The docstring now says the inert count is the walk's visits to the drawn rows, about K · Σp per phase. That matches my round-3 re-derivation.
  • Turned-down count. Injected.turned_down is np.isin(rows, drawn[treated]).sum(). It counts exactly the natural rows set to α, and the at_treated mask is reused for the weight assignment, so the two cannot drift apart.
  • Wiring. The count is threaded through StepStats and the progress line. Tests cover it in test_boost.py (lines 92 and 113, hand-worked) and test_train.py (lines 274, 777, 817 and 821: the zero case, the stats-to-line match, and a plain run).
  • Names, dates and wrapping. The dates and the person's name are out of the boost.py docstring, and the long lines are wrapped.
  • Injected constructor. turned_down is a new required field with no default, but inject is the only constructor (grep found no other Injected( call), so nothing breaks.

Remaining findings, all low severity

  1. Skip rule, still open for PR 5 (medium confidence). The skip is deterministic, so a row near K · arrival ≈ 1 falls out of the aim at every iteration. rows.skipped already reports it. The PR 5 pre-registration should state "skipped near 0 at K = 200" as a check.
  2. Noise reduction unmeasured (informational). The docstring and PR body are honest about it. PR 5 should log σ_b² against σ_w²/(K·p) per targeted row.
  3. rows.advantage semantics (low confidence). It now counts boosted rows among the offers. The log carries rows.boosted beside it, so this is fine. A one-line note in the log schema docs would stop a later reader subtracting twice.
  4. Gate strength (low). The regret band is about 0.8 σ of resolving power at 24 seeds. The exact weight pin covers the weight bugs that matter, and the uncommitted mutation checks are a defensible choice.
  5. Style (low). boost.py still has some long code lines next to wrapped docstrings. Confirm that matches the repo's line-length setting.

Not verified: that BoostTargets.targets returns arrival as the traverser-side expected visits per walk. The exact weight pin in the gate test covers it indirectly.

Not run: uv run pytest. The PR reports 1,921 passed, and CI runs no tests.
· branch AlecROndo/pr-4-boost-stage

@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 26s —— View job


Review of PR #94 (boost stage), at head 1d03e30

I read the diff of the two newest commits (3ec9080, 1d03e30) in boost.py, plus the stat counts in the commit message. I did not re-read the rest of train.py, the CLI or the tests, and I ran no tests. I found no correctness bug.

Round 3 is resolved correctly.

  • The wrong capacity figure is gone. The docstring no longer quotes B · t. It says the inert count is the walk's visits to the drawn rows, about K · Σp per phase. That matches my re-derivation: each of K walks reaches a row with probability p, so K · p natural rows per row.
  • turned_down is computed correctly. at_treated = np.isin(rows, drawn[treated]) is the same mask that sets the natural weights to α. So turned_down is exactly the number of natural rows at weight α. Rows skipped as untreated are correctly excluded.
  • No ordering or dtype change. The weights and the offer order are untouched.

Remaining findings, all low severity.

  1. The phase docstring now understates its draw (confidence medium; boost.py ~line 279). 1d03e30 shortened "rows drawn for (t, seat, seed)" to "rows drawn for t". The draw still depends on the seat and the seed: the PR body says phase draws rows_for(t, seat, seed). Seat 0 and seat 1 get different rows at the same t. Only the line width forced the change, and the shorter text is slightly wrong.

    • Suggestion: "rows drawn for (t, seat)", or reflow the line.
  2. turned_down at α > 0 is not "inert" (confidence high). The Injected docstring says turned_down counts "the memory slots the boost makes inert at α = 0". At α > 0 the same rows are down-weighted but not inert. The count is still right, so only the wording is narrow. Consider "slots made inert at α = 0, down-weighted otherwise".

  3. Test strength for turned_down is thin (confidence medium). The commit message says two tests pin it:

    • A hand-worked case with 3 of 4 natural rows at two treated infosets.
    • A step test with the real oracle.

    Neither visibly covers a skipped row whose natural rows must stay at weight 1 and so be excluded from turned_down. That is the one branch where turned_down could differ from boosted > 0. If the hand-worked test already includes a skipped drawn row, ignore this.

  4. Carried over, informational. PR 5 still needs to record σ_b² against σ_w²/(K·p) per targeted row. It also needs to check rows.skipped near 0, and now rows.turned_down as a share of rows.advantage, at the chosen K. The PR body already lists all three.

  5. Style. The repo sets no line length, so the wrapped lines are fine. A few definition lines in boost.py (for example the Injected fields and the treated comment) are still long. That is cosmetic.

Not run: uv run pytest. The PR reports 1,921 passed, and CI runs no tests. The last two commits changed tests, so the author should re-run the touched files (test_boost.py, test_train.py) before merge.
· branch AlecROndo/pr-4-boost-stage

…ned_down's that α > 0 down-weights rather than silences
@AlecROndo

Copy link
Copy Markdown
Owner Author

Round 4, dispositioned in bd08963 (docstring only).

  1. phase's docstring — now says the rows are targets.rows_for(t, seat, seed) and that the two seats draw different rows at the same t; the summary line no longer tries to carry it.
  2. turned_down at α > 0 — reworded: slots made inert at α = 0, merely down-weighted at α > 0.
  3. A skipped row's natural rows excluded from turned_down — covered: test_a_zero_arrival_row_is_skipped_and_its_natural_rows_keep_weight_one asserts (boosted, skipped, turned_down) == (1, 1, 2) with four natural rows at two drawn infosets, one of them skipped, so the skipped row's natural row stays at weight 1 and out of the count.
  4. PR 5's checks — rows.skipped near 0, σ_b² against σ_w² / (K · p) per targeted row, and rows.turned_down as a share of rows.advantage are all in the body's PR 5 list, and have been handed to the PR 5 agent directly.
  5. Style — the repo sets no line length; the remaining long lines are definitions, left as they are.

The touched test files were re-run after every commit (test_boost.py and test_train.py, 120 passed on 3ec9080); the whole default suite is running on the final head and its count will be added to the body.

@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 16s —— View job


Review of PR #94 (boost stage), at head bd08963

I read the diff of bd08963, which touches only boost.py docstrings. I did not re-read the rest of the PR and ran no tests. I found no correctness bug.

Resolved from my last round

  • Boost.phase's docstring now says the rows are targets.rows_for(t, seat, seed) and that the two seats draw different rows at the same t. That matches the code.
  • Injected.turned_down now reads "inert at α = 0, merely down-weighted at α > 0". That is accurate.

Still open, all low severity and all for PR 5, as the PR body already says

  1. Test strength for turned_down (low confidence). I did not check whether a test covers a skipped drawn row whose natural rows stay at weight 1 and so must not be counted in turned_down. If the hand-worked test has no such row, one case would close the gap.
  2. PR 5 should check these:
    • σ_b² against σ_w²/(K·p) per targeted row.
    • rows.skipped near 0 at K = 200.
    • rows.turned_down as a share of rows.advantage.
  3. rows.advantage now counts boosted rows (informational). rows.boosted sits beside it in the log. A one-line note in the log docs would stop a later reader subtracting twice.

Since the last two commits changed only docstrings, the 1,921-pass result still applies. I ran no tests myself.
· branch AlecROndo/pr-4-boost-stage

@AlecROndo

Copy link
Copy Markdown
Owner Author

Round 5, dispositioned in c10d5e9 (one docstring line).

  1. turned_down with a skipped row — covered, as in round 4: test_a_zero_arrival_row_is_skipped_and_its_natural_rows_keep_weight_one has four natural rows at two drawn infosets, one drawn row skipped, and asserts (boosted, skipped, turned_down) == (1, 1, 2), so the skipped row's natural row stays at weight 1 and out of the count.
  2. PR 5's checks — in the body's PR 5 list and handed to the PR 5 agent.
  3. rows.advantage counts boosted offers — said in train.py's log paragraph now: it is the memory's count of offers ever made, boosted rows included, with rows.boosted beside it.

@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 32s —— View job


Review of PR #94 (boost stage), at head c10d5e9

I read the diff of c10d5e9 (a train.py docstring) and checked it against the code that writes the log (train.py:562 and train.py:1058). I ran no tests. This commit changes no behaviour, so the 1,921-pass result still applies. I found no correctness bug.

One docstring problem in the new text (low severity, medium-high confidence; train.py:120-123)

The new sentence says rows.advantage is "each seat's memory's count of offers ever made, boosted rows included". That matches the code: line 1058 writes memory.offered, which is cumulative over the run.

The next clause is muddled: "the walk's own rows of the iteration are rows' complement, rows.boosted beside it, so a reader does not subtract the boosted rows twice."

  • rows.advantage is cumulative, but rows.boosted is the current iteration's count (stats.boosted).
  • Subtracting one from the other gives nothing meaningful. A reader would first have to difference consecutive rows.advantage values, or sum rows.boosted over all t, and take the LAST line per t when a run resumed.
  • "rows' complement" is also unclear. I read it as the rows variable at line 562, which holds the count of the walk's rows. That count is not logged under that name in the snippet I saw.

Suggested wording: "rows.advantage is each seat's memory's cumulative count of offers, boosted rows included. rows.boosted, rows.skipped and rows.turned_down are this iteration's counts, so the iteration's offers are the difference of consecutive rows.advantage values, and the walk's own rows are that difference minus rows.boosted."

If that is wrong because memory.offered is per-iteration, the "ever made" wording is the part that needs fixing instead. Either way the two sentences disagree.

Open items for PR 5, unchanged from earlier rounds and already in the PR body

  • Record σ_b² against σ_w²/(K·p) per targeted row.
  • Check that rows.skipped stays near 0 at K = 200.
  • Read rows.turned_down as a share of the offers.

No other new findings.
· branch AlecROndo/pr-4-boost-stage

@AlecROndo

Copy link
Copy Markdown
Owner Author

Round 6, dispositioned in 1a45946 (one docstring paragraph). Agreed: rows.advantage is cumulative (memory.offered) and rows.boosted, rows.skipped, rows.turned_down are the iteration's counts, and my sentence mixed the two. It now reads as you suggested: an iteration's offers are the difference of consecutive rows.advantage values, and the walk's own rows that iteration are that difference minus rows.boosted.

@claude

claude Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Claude finished @AlecROndo's task in 14s —— View job


Review of PR #94 (boost stage), at head 1a45946

I read the diff of 1a45946, which changes one docstring in train.py, and checked it against the log writer at train.py:1060. I ran no tests. The commit changes no behaviour, so the 1,921-pass result still applies. I found no correctness bug.

Resolved from my last round. The rows.advantage paragraph (train.py:120-125) now says what the code does:

  • rows.advantage is memory.offered, which is cumulative.
  • rows.boosted, rows.skipped and rows.turned_down are this iteration's counts.
  • An iteration's offers are the difference of consecutive rows.advantage values. The walk's own rows are that difference minus rows.boosted.

One caveat, low severity and medium confidence. The difference rule needs consecutive log lines. The same paragraph says a resumed run re-appends iterations, and a reader should take the last line per t. Differencing must happen after that dedup, and the first iteration has no predecessor. The wording is correct as written. You could add "after taking the last line per t" if you want it airtight.

Still open for PR 5, all already in the PR body

  • Record σ_b² against σ_w²/(K·p) per targeted row.
  • Check that rows.skipped stays near 0 at K = 200.
  • Read rows.turned_down as a share of the offers.

No other new findings.
· branch AlecROndo/pr-4-boost-stage

@AlecROndo

Copy link
Copy Markdown
Owner Author

Round 7: agreed and no change — the paragraph already tells a reader to take the last line per t, and the difference rule applies after that; the first iteration's predecessor is zero offers. Nothing else open on this PR; the three PR 5 checks are in the body and with the PR 5 agent.

This branch was successfully deployed

1 active deployment
Preview — 1a45946f Deployed Nov 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant