Repository navigation
feat(rung4): targeting ranks groups by leak × noise per reached infoset and draws B rows without replacement ∝ arrival — on LCFR-2M the line arm's draws land mostly on round-1 and draw rows, ~100× dearer than round 2's - #93
Conversation
…every opponent hand and every physical entry of its key, and the walk's arrival rate, from the rules alone PR 1 of the targeted-sampling plan. `minidrawmaha/solver/exact_target.py` computes, for any table row, the regret the walk's samples there estimate (the mean over the opponent's hands of u − ⟨σ, u⟩, weighted by the deck and the opponent's own play, normalised by the arrival) and the probability one walk visits the row. It re-derives the public tree, key maps, draw maps and showdown from rules/ and never imports the grader, which grades what this will feed. Measured: a round-2 row costs 0.8 ms, the arrival alone 0.3 ms; a draw row 45–85 ms and a round-1 row 0.2–0.6 s, because the value below expands over the traverser's own replacements and the second board card. Tests: the back-solved reach equals the grader's carried vector on every board of every point (1e-10, full and restricted deals); arrival at a seat's first decision is the grader's frequency and sums to one; regret and arrival equal a world-by-world walk through MiniState on a small restricted game (1e-9) and the mean and visit count of 6,000 Collector walks row by row within 4 s.e. and pooled; the module's import block holds no grader.
…, 5,190 vectors, re-derived by a test; the sampled tests select rows by expected count, so a row cannot enter because it fluctuated high Review follow-ups on #89. The docstring counted the memo's level-1 points alone (29) where the chance nodes on the paths are memoised too (66 in all); the test now walks the ancestor closure and pins 66, 20 and 5,190. `reach()` hands out a copy. The pooled z² windows are a 4-sd band on the chi-square mean, √(2/n), instead of fixed numbers. The gated 200,000-walk test kept rows by observed count ≥ 100, which let seat 1's draw rows in only when they had fluctuated high (mean z +1.8 on two seeds); rows are now kept by expected count ≥ 80 against an observed cutoff of 40, and both seats read mean z within 0.01 of zero over thousands of rows. The module docstring names the double-visit caveat and no longer points at a plan outside the repo without saying where it is.
… the mean of z², and cap a single row at six standard errors — the deal alone dealt one picture 292 times against 220 at seed 12 The pooled mean is the sharp bias test (a tenth of a standard error in every row is caught at 1,600 rows); the per-row cap is loose because of about 3,000 rows one passes 5 in a run out of a thousand, and the deal itself did it at seed 12 in a count that involves nothing of this module. Run with MINIDRAWMAHA_FULL_TARGET=1: both seats pass.
…e band's assumptions written beside it — the "caught at 1,600 rows" line was a half-truth; the band is reached there and reliable at about 3,000
…9 at n = 10, biting from n ≈ 45 — and the arrival seeds checked named
…width drops under 1, not from 45
…caller says otherwise; a file without the column loads as ones PR 2 of the targeted-sampling plan, first of three commits. The reservoir's row grows from (row, t, target) to (row, t, target, weight): 24 to 28 bytes, a 50M-row memory 1.2 to 1.4 GB. The weight rides its row's slot through Algorithm R's eviction, save and load; merge without a weight stamps ones; a file with no weight key is from before the column and loads with ones. Refused by name: a negative, non-finite, mis-shaped weight on merge, and a weight array of the wrong length or dtype on load. The learner seam still takes three arrays, so tests/deepcfr/test_train.py is red until the next commit, which threads the column through it.
…y's loss bit for bit and a row of weight zero is inert Second of PR 2's three commits. _weighted_loss scales each row's squared error by (t · w) / mean(t · w): with w ≡ 1 that is t / mean(t) in the same bits, since t · 1.0 is t in float32. fit_from_scratch and batch_loss take the weight as a fourth positional array, None meaning ones, and refuse a weight that is not 1-D, not the memory's length, non-finite, negative, or all zero (nothing to fit, as an empty memory is). A batch whose every row weighs 0 divides by a clamped mean and has loss 0, not NaN. fit_policy takes the column positionally so the policy memory's contents() unpack straight in; its weights are all one.
…cle banks Σ t′ · w · r in LCFR's order, so weight one keeps the pin bit for bit Last of PR 2's three commits. Learner.fit takes the memory's four arrays, the weight positional with no default so no learner can drop it by accident; step is unchanged, the four-tuple from contents() flowing through. NetLearner hands the weight to fit_from_scratch and the held batch's loss, after the pot scaling and the regret rules act on the target: the weight says how loudly a row counts, never what it says. The oracle multiplies each sample by its weight in float64 before the per-t′ np.add.at sum, where × 1.0 is exact, so the LCFR pin passes unchanged; a weight of 2 banks what the row offered twice would, a weight of 0 nothing. Oracle model files carry the weight and load as ones without it.
… refused by name — the ones default ran ahead of the checks it relied on; and the weight-one pin is against the old loss formula, not the code itself From the fresh-context review before the push. Reservoir.load sized the default weight from a negative `offered` before refusing it; merge and fit_from_scratch sized it from a 0-d rows before the 1-D check; a 0-d weight on load indexed its own empty shape. Each now fails on the by-name ValueError the old code gave. test_weight_one_is_the_unweighted_fit_bit_for_bit compared the fit to itself (both paths built the same ones); it now restates the pre-column loss, t / mean(t) over the legal squared error, and pins batch_loss at weight one and by default against it to the bit, then the whole fit.
…ight for overflow, on merge, on load and in the fit — a float64 weight past float32's range passed the finite check and was stored as inf; and the oracle's "offered twice" claim holds to the bit only when the two are alone at the column - `_validate_offer` casts to float32 first and checks finite, non-negative and t × weight finite after; a bool or non-numeric weight is refused by name. - `Reservoir.load` refuses a weight that is NaN, negative or overflows t × weight, naming the file, as `merge` never writes one. - `fit_from_scratch` refuses a t × weight that overflows float32, the loss's scale. - `OracleLearner` docstring: weight 2 equals the row offered twice exactly only when nothing else sits at the column in that t′; otherwise up to rounding, since `np.add.at` sums one at a time. A new test pins two rows of widths 2 and 3 with weights 3 and 2 against the rows offered three and two times, so a weight landing on a neighbour's columns would show. - `batch_loss` says why it defaults the weight while `Learner.fit` does not; `NetLearner.fit` notes the held loss is a log line nothing decides on.
…high-noise row — board card, both seats' replacements, the opponent's actions and range — each unbiased against the exact sum, which stays as EXACT, with the arrival always exact and one stream per (seed, t, seat, row) exact_target.py becomes boost_target.py (ExactTargets → BoostTargets). targets() takes a required Sampling; EXACT reproduces the previous output bit for bit.
… one branch at a time, averaged, is the exact sum to 1e-10 — and the range draw is tested where a key's two entries face different ranges; the stream's tail is two words, not one, and seed and t take any integer
…le's rows — 141 lines and 3,870 line × hand classes, from the rules alone
… √(E[r²]/p)/|E[r]|, from the memory's own walk rows only — an exact row is a mean, not a sample
…nfoset and take them until the pool is 20 × B reached rows; infoset takes the B best outright
…rrival by exponential keys, reproducible from (seed, t, seat, stream) and never the same row twice
…rozen target set, asking for arrivals at the pool alone; the Refresher protocol is the seam PR 4 calls
…hecked on every import statement
… — None, or a chance count at its fan — so it needs no seed and refuses extra play-outs; counts take any integer type, as seed and t do; the stream comment says when six words hold
…ound 1's and hold 19% of LCFR-2M's exploitability on 0.07% of its reached infosets; the 20 leakiest hold 57% on a quarter
…nto AlecROndo/pr-3-targeting
…that draws nothing
…ds as zero; a real negative is still refused — the LCFR-2M release has gains at −7e-12
…, by choice, the current strategy), weighs the pool by the walk's arrival, and is the only importer of the meter on the trainer's side
…nto AlecROndo/pr-3-targeting
…one-board-card rows, whose exact rows cost about a hundred times a round-2 row's
…und-1 lines hold 22% of LCFR-2M's leak on 0.13% of its reached infosets, read by a test from the committed file
…l decisions, and the row-cost finding behind round_two_only
…ne, √(E[r²]/p − μ²)/|μ|, so a line every walk reaches scores its scatter σ/|μ| rather than √(σ² + μ²)/|μ|; the tests now pin p's memory share and make only the noise break a tie
…rough the table's own index, and a pool landing exactly on pool × B is pinned to take no further group
…he meter nor the analysis; float rows, non-int draws and a bad K are refused by name before any walk; a saved set reads back every spec field; σ is read only when an arrival is asked for; the package docs count ten modules
…nto AlecROndo/pr-3-targeting
…/|μ|, and the random arm's measured refresh, eight minutes a seat
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @AlecROndo's task in 46s —— View job Review of PR #93 (targeting)I read Summary: I found no correctness bug in the selection, the draws or save/load. I have one medium-severity concern about the visit rate Math I re-derived (all hold)
Findings1. Medium (confidence ~60%):
2. Low–medium (confidence ~90%, already acknowledged in the PR): walk rows are identified by weight alone —
3. Low: noise is dominated by the cap on 4. Nit: docstring wording. 5. Nit: unvalidated 6. Note: caches. Two Scope
|
…y, because size and offered both count them — said in the docstring and pinned over 400 eviction streams; an infoset set's unread weights must be ones
|
Round 1 of the review, dispositioned in 2306ed8.
Tests: |
|
Claude finished @AlecROndo's task in 25s —— View job Review of PR #93 (targeting), re-check after the latest pushI read Retraction of the earlier finding 1 (
Still open (unchanged from the earlier review):
The selection, draws and save/load logic looks correct. I found nothing new. |
|
Round 2, against 2306ed8, plus one docstring line in 6616976.
No code change beyond the docstring line. |
|
Claude finished @AlecROndo's task in 11s —— View job Re-check of PR #93 after 6616976This push only changes a docstring in
Still open from the earlier review (unchanged):
|
PR 3 of the targeted-sampling plan for rung 4: targeting, which decides which table rows the boost stage (PR 4) will compute exactly. Nothing is wired into
train.steporDeepConfig; the oracle pin is unchanged.Branched from #91's head (it carries #89), and #91's later heads are merged in, never rebased. While #89 and #91 are open, the diff shows their files too.
What it adds
src/drawmaha_solver/deepcfr/targeting.py(NumPy only, never imports the grader)TargetSpec: granularity,alpha(required, no default), budget B, refresh R,noise_cap,pool,stream, and the opt-inround_two_only.infoset(a row),line(a public point, 141),line_class(point × inner × outer category, 3,870 non-empty groups), andrandom(the whole seat). These are pinned by tests.infosettakes the B best rows.lineandline_classtake groups best first until their reached rows reachpool × B; the group that crosses is in.randompools the whole seat.TargetSet, frozen for R iterations.rows_for(t, seat, seed)draws ≤ B distinct rows ∝ the pool's arrival weight using exponential keys (successive sampling). The draws are deterministic in (seed, t, seat, stream), on a stream whose fifth word is "TARG". A set used outside its window is refused.save/loadround-trip it as an atomic.npz.target_set(...), the orchestrator, and theRefresherprotocol PR 4 calls.scripts/deepcfr/targeting_refresh.py:GraderRefresher, the only importer of the meter on the trainer's side.leak="average", the default, or"current"), lays the gains and frequencies on table rows, and weighs the pool byBoostTargets.arrivalsunderBoostTargets.sigma(table).analysis.py:gains_by_row,scatter_by_row,line_profile, andminidraw-exploit-by-reach --by-line, which writesfigures/rung3/exploit_by_line.json/.png(LCFR-2M).exploit_by_reach.jsonand the README's sentence to the file.deepcfr/__init__.py,deepcfr/AGENTS.md(the no-grader rule and the no-repeat rule),scripts/AGENTS.md,figures/AGENTS.md, one README sentence, and the plandocs/superpowers/plans/2026-11-06-targeted-sampling-pr3-targeting.md.Departures from the appendix sketch, each for a reason:
group_of_row.rows_forcan refuse a stale iteration.Decisions provisional — Alec to confirm
Alec was asleep when this was built, so the plan's proposals are the defaults. Every one is a parameter, never hard-wired.
B = 2,000 rows per seat per iteration, R = 25 iterations between refreshes:
TargetSpec.budgetandTargetSpec.refresh, the plan's proposals as they stood. Item 4 shows the 2,000 is cheap only on round-2 rows.Leak of what:
GraderRefresher(leak="average"), the frozen exact average, which is what the run is graded on.leak="current"grades the table's current strategy, which is what the regrets follow. They cost the same two grader walks.Reach for draws = the walk's arrival (
BoostTargets.arrivalsunder the table's σ), not the grader's play frequency. The boost's row weight multiplies the arrival, so drawing ∝ arrival spends the budget where the most sampled weight is replaced. It also means the random arm needs no grader. Eligible rows for the aimed arms are rows with frequency > 0, so the pool-size rule counts exactly the rows that can be drawn.B counts rows, but rows differ in cost a hundredfold. Measured with the real refresher on LCFR-2M (the release as the table, empty memories, so every noise is at the cap), the line arm's pool is about 45k rows per seat:
Cost per row of
BoostTargets.targetsunder LCFR-2M's σ, measured on 12 rows per stage from the leakiest lines:EXACTSampling()(the walk's own sampling below the row, against the whole range;seed=0, t=0)A quieter first pass over 12 other seat-0 rows read 181 / 101 / 1.1 ms for
EXACT. Per iteration, at those draw counts:EXACT): 284 s + 245 s ≈ 530 s.round_two_only=True,EXACT: 2,000 round-2 rows per seat ≈ 4.5 s + 2.0 s ≈ 6.5 s, assuming the round-2 rows it would draw cost what these do.Sampling()target and round-2 rows exact: 52 s + 38 s ≈ 89 s.For scale: a table (Tier-1) iteration is about 18 s by the plan's estimate, and a 4 × 256 network iteration about 190 s.
The random control is not cheaper under the plan's rule. Its whole-seat pool puts 10–12% of the draw weight on round-1 rows and 13–17% on draw rows. At t = 1 it draws 196 / 231 / 1,573 rows (seat 0) and 219 / 372 / 1,409 (seat 1), about 89 s + 82 s ≈ 170 s per iteration with
EXACT. Its refresh (arrivals over both whole seats, no grader) measured 16 min 52 s.The plan's rule stays the default.
round_two_only(option a) is here, opt-in. The other options are not built: (b) budget in cost rather than rows; (c) a separate, smaller B for one-board-card rows; (d) above, which is PR 4's to wire.α is not settled.
TargetSpec.alphahas no default. This PR reads it only to tell the walk's rows from exact rows in the noise score, which is why PR 4 must not inject an exact row whose weight equals 1 or α.Tests
uv run pytest(whole default suite, no torch): 1,872 passed, 24 skipped in 9 min 54 s, on the head merged with feat(rung4): every memory row carries a weight; the refit and the oracle weigh a row by t × weight, and weight one is today's run bit for bit #91's cbb5308.MINIDRAWMAHA_FULL_GRADE=1 uv run pytest tests/deepcfr/test_targeting_refresh.py -k real: 1 passed in 3 min 8 s. This is a real refresh on the whole game: the pools are reached rows and the draws are reproducible.tests/deepcfr/test_targeting.pycovers:round_two_only, and round-off gains;deepcfr/imports the grader.tests/deepcfr/test_targeting_refresh.py: the wiring with stand-ins for the grader walk and the oracle. One real end-to-end refresh is behindMINIDRAWMAHA_FULL_GRADE.tests/minidrawmaha/test_exploit_by_line.py: the scatter checked againstPackedTable.row_of, the profile's partition, the committed file againstexploit_by_reach.json, and the README numbers.tests/deepcfr/test_train.py::test_the_oracle_loop_is_lcfr_bit_for_bitpasses unchanged.Pre-push review (fresh-context reviewer subagent), dispositioned
The reviewer found no correctness bug in the selection, the draws, the scatter or save/load, and checked every quoted number against the committed JSON. Its findings:
answer.analysisby name, and checks in a fresh interpreter that importingtargetingloads neither the meter nor the analysis.deepcfr/AGENTS.mdnow says the name check is not transitive.average,distill,policyandtrainalready reachexploitabilitythroughanswer.strategy's row-sum tolerance constant; that predates this PR.deepcfr/__init__.pynow says ten modules and listsdistill.deepcfr/AGENTS.md's stale "the package itself is complete" is updated.PackedTable.row_of.t/seed,seat=True, and a badwalksorspec(checked before any grader walk) are refused by name.loadrebuilds the spec fromdataclasses.fields(TargetSpec)and refuses a file missing a field.infosetnever reads it. Noted, not changed:groups()caches about 50 MB per granularity and seat.Regenerate