diff --git a/docs/superpowers/plans/2026-11-06-boost-target-modular-sampling.md b/docs/superpowers/plans/2026-11-06-boost-target-modular-sampling.md new file mode 100644 index 00000000..40504a61 --- /dev/null +++ b/docs/superpowers/plans/2026-11-06-boost-target-modular-sampling.md @@ -0,0 +1,95 @@ +# Boost target: modular sampling from a high-noise row — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Turn PR #89's exact target into a sampling framework. At a targeted row, it runs M extra play-outs from that row. Each source of randomness below the row is sampled k-of-n or enumerated, chosen per call. The result is unbiased for the regret the walk's samples estimate. Exact enumeration remains as the all-enumerate special case and the test oracle. + +**Architecture:** One value type, `Sampling`, names a count per source: opponent range, board cards, opponent draws, hero draws, opponent actions, and the number of play-outs. The existing downward recursion (`_value`, `_action_values`, `_push`) takes a private `_Draws` object. That object either returns everything (exact) or a random subset with its unbiasing scale. The upward back-solve (`_reach`) and the arrival stay exact always: the arrival is the cheap half, and an exact denominator under a sampled numerator keeps the ratio unbiased. Randomness is one `default_rng([seed, t, seat, row, TAG])` per row, so a row's result never depends on how rows are batched or split across workers. + +**Tech Stack:** Python 3.12, NumPy 2.5 (`Generator.multinomial` with 3-D `pvals`), pytest, `uv`. + +**Spec:** the chat design agreed 2026-11-06 (Alec): "sample extra starting from node x because node x is high variance … modular … board, opponent draws, hero draws, opp actions … unbiased". This builds on `~/Desktop/Claude/poker/targeted-sampling-plan/targeted-sampling-plan.md` §3 (row weight K · arrival · t, unchanged). + +## Global Constraints + +- The module imports `rules/` and the packed table only: never `exploitability.py`, `deepcfr`, or torch. The static import test stays. +- `Sampling` fields: `playouts: int = 1`, `opponent_range: int | None = None`, `board_cards: int | None = 1`, `opponent_draws: int | None = 1`, `hero_draws: int | None = 1`, `opponent_actions: int | None = 1`. `None` means enumerate all. `EXACT = Sampling(1, None, None, None, None, None)`. +- Chance sources (board, opponent draws, hero draws) sample k of n **without** replacement with scale n/k; k ≥ n is enumeration. +- Opponent actions are k i.i.d. draws from σ per opponent state, weighted count/k; `None` means split by σ. +- Opponent range is k i.i.d. hands per entry, drawn from that entry's compatible weight, each carrying W/k; `None` means the whole range. +- The arrival is always exact; `targets()` returns the same arrival as `arrivals()`, bit for bit. +- `targets(..., sampling=EXACT)` equals the pre-change output bit for bit. +- Stream: `np.random.default_rng([seed, t, seat, row, _STREAM_TAG])`; a non-EXACT `sampling` without `seed` and `t` is refused by name. +- Rename: `solver/exact_target.py` → `solver/boost_target.py`, `ExactTargets` → `BoostTargets`, `tests/minidrawmaha/test_exact_target.py` → `test_boost_target.py`; update `minidrawmaha/AGENTS.md`. +- The default suite stays torch-free and under its current time budget. Sampled unbiasedness tests use the restricted `SMALL`/`WIDE` deal sets. + +## Review Focus + +1. A sampled dimension that never actually samples, because the row has nothing of that kind below it. Every unbiasedness case must also assert that the sampled spread is > 0. +2. Hero-draw sampling must not leak into the hero's own **actions**. Hero actions are always enumerated under the hero's mix; only the replacement card is sampled. +3. Pruning a zero-weight opponent branch must not change the EXACT path. This is pinned by the bit-for-bit test. +4. `opponent_range = k` at a row whose compatible weight is zero (an unreachable row) must return zero regret, not NaN or a sampling error. +5. The same row requested twice in one call, or in two calls with rows in a different order, must give identical sampled results. + +--- + +### Task 1: rename and the `Sampling` value type + +**Files:** +- Rename: `src/drawmaha_solver/minidrawmaha/solver/exact_target.py` → `boost_target.py`; `tests/minidrawmaha/test_exact_target.py` → `test_boost_target.py` +- Modify: `src/drawmaha_solver/minidrawmaha/AGENTS.md` (the `exact_target` line) + +**Interfaces — Produces:** `Sampling` (frozen dataclass, fields above), `EXACT`, `BoostTargets` (the renamed class). `BoostTargets.targets(sigma, *, seat, rows, sampling: Sampling, seed: int | None = None, t: int | None = None) -> tuple[np.ndarray, np.ndarray]`. `sampling` is required, so every caller states its choice. + +- [ ] Step 1: Snapshot today's outputs (not committed). Save `targets` on `random_table(9)` for 40 rows spread over round-1, draw and round-2 points, both seats, to `.context/exact_snapshot.npz`. +- [ ] Step 2: `git mv` both files, update imports and the AGENTS.md line, and add `sampling=EXACT` at every existing test call site. Run `uv run pytest tests/minidrawmaha/test_boost_target.py tests/test_agent_docs.py`: PASS. +- [ ] Step 3: Failing test `test_sampling_refuses_counts_that_are_not_positive_ints`: `Sampling(board_cards=0)`, `Sampling(hero_draws=-1)`, `Sampling(opponent_actions=True)`, `Sampling(playouts=0)` and `Sampling(playouts=2, …all None)` each raise `ValueError`, and the message names the field. +- [ ] Step 4: Failing test `test_a_sampled_call_without_seed_and_t_is_refused`. +- [ ] Step 5: Implement `Sampling.__post_init__` and the refusal in `targets`, then run the tests: PASS. Commit `feat(rung4): the boost target names what it samples — one Sampling value, EXACT the oracle`. + +### Task 2: the `_Draws` seam and the exact path through it + +**Files:** `boost_target.py` (`_value`, `_action_values`, `_push`, `targets`) + +**Interfaces — Produces:** `_Draws(sampling, rng)` with: +- `board(kids: (nb,14)) -> (kids', scale)` +- `replacements(n_rows, fan=12) -> (pick | None, scale)`, used by both hero and opponent draws with their own field +- `actions(pvals: (nb,S,w)) -> weights (nb,S,w)` +- `range(u, mask, k) -> u'` + +On `EXACT`, every method is the identity, with no RNG and no array copy. + +- [ ] Step 1: Failing test `test_exact_sampling_reproduces_the_snapshot_bit_for_bit`. This is a temporary local test reading `.context/exact_snapshot.npz`; it's deleted before the commit, and its committed successor is Step 3 of this task. +- [ ] Step 2: Thread `draws: _Draws` through `_action_values` and `_value`, and give `_push` an optional `pick`. Skip an opponent child whose pushed weight has no positive entry (it returns zeros). Run: PASS bit for bit. +- [ ] Step 3: Committed test `test_a_count_at_or_above_the_fan_is_enumeration`: `Sampling(board_cards=14, opponent_draws=12, hero_draws=12, opponent_actions=None, opponent_range=None)` equals `EXACT` with `np.array_equal`, on a draw row and a round-1 row. +- [ ] Step 4: Commit `feat(rung4): the play-out takes its draws from one seam; exact is the seam returning everything, bit for bit`. + +### Task 3: the five samplers, each unbiased and non-vacuous + +**Files:** `boost_target.py` (`_Draws` bodies, the playouts loop, the per-row stream), `test_boost_target.py` + +- [ ] Step 1: Failing test `test_each_sampled_source_is_unbiased_and_actually_samples`, parametrized over `opponent_range=1`, `board_cards=1`, `opponent_draws=1`, `hero_draws=1`, `opponent_actions=1`, and `Sampling()` (all at once). Each case uses a row where that source is live, under `build(holes=WIDE)`, a random positive table, 400 seeds (t = seed). Assertions: + - for every action column, |mean − exact| < 4 · s.e. + - `assert_pooled_fair` on the pooled z + - the per-column spread is > 0 for at least one column (Review Focus 1) + - `arrival` equals `arrivals()` exactly +- [ ] Step 2: Implement the samplers: + - **Board:** `rng.permuted` over the 14 children per batch row, take the first k, scale 14/k into `u`. + - **Replacements:** k of 12 per (row, hole) or (row, hero state), via argsort of uniforms, scale 12/k. + - **Actions:** `rng.multinomial(k, pvals) / k`; rows of invalid states get uniform `pvals`, and their weight is already zero. + - **Range:** per entry, mask the states that clash with the hero's cards, draw k states ∝ `u·mask` with `rng.choice`, and put W/k on each draw (with multiplicity); W = 0 returns zeros. + - **Play-outs:** loop M times per row, averaging the numerator; divide once by the exact arrival. + + Run: PASS. +- [ ] Step 3: Failing test, then passing: `test_a_rows_sample_depends_on_seed_t_seat_row_only`. The same rows in reversed order, split over two calls, and with a duplicate give equal results; t + 1 gives a different result (Review Focus 5). +- [ ] Step 4: Failing test, then passing: `test_more_playouts_shrink_the_spread`. Across 200 seeds, the variance at `playouts=8` is within [1/16, 1/4] of the variance at `playouts=1`. +- [ ] Step 5: Failing test, then passing: `test_an_unreachable_row_is_zero_under_any_sampling`, with `opponent_range=1` (Review Focus 4). +- [ ] Step 6: Commit `feat(rung4): any source below a boosted row can be sampled k of n — each unbiased against the exact sum, on a per-row stream`. + +### Task 4: cost table, docstring, PR, and PR 2 + +- [ ] Step 1: Extend the benchmark test to print ms per row for `EXACT` vs `Sampling()` at round-1, draw and round-2 rows. Keep the existing round-2 ceiling, applied to `EXACT`. +- [ ] Step 2: Rewrite the module docstring: what is sampled, why the arrival stays exact (ratio bias), why k-of-n without replacement for chance and i.i.d. for σ, the stream, and the measured costs. Every number in it must be recomputed by a test or printed by the benchmark. +- [ ] Step 3: `uv run pytest`, the full default suite: PASS. Push to `AlecROndo/canary-handoff-pr-1-build`, retitle #89 and rewrite its body. +- [ ] Step 4: PR 2: in a temporary detached worktree on `origin/AlecROndo/pr-2-handoff-prompt`, `git merge origin/AlecROndo/canary-handoff-pr-1-build` (merge, never rebase), run `uv run pytest`, and push `HEAD:AlecROndo/pr-2-handoff-prompt`. Workspace `bariloche`'s local branch then needs a `git pull`; say so. +- [ ] Step 5: Drive #89 and #91 to merge-ready: checks green, `claude-review` findings dispositioned. diff --git a/src/drawmaha_solver/minidrawmaha/AGENTS.md b/src/drawmaha_solver/minidrawmaha/AGENTS.md index 56f5ded6..97aa5019 100644 --- a/src/drawmaha_solver/minidrawmaha/AGENTS.md +++ b/src/drawmaha_solver/minidrawmaha/AGENTS.md @@ -13,7 +13,14 @@ Three subpackages, and one module that stays alone on purpose: arrays), `infoset_table` (a ledger window over it), `mccfr` (external sampling, one hand per seat), `lockstep` (W hands per seat against one frozen σ; the reference), `parallel` (the same in W processes over shared - memory), `lockstep_run` (resume, grid checkpoints, progress). + memory), `lockstep_run` (resume, grid checkpoints, progress), and + `boost_target` (rung 4's boost target: extra samples of a high-noise row's + regret, each source below the row — the board card and both seats' + replacements k of n, the opponent's actions and range k draws — or + enumerated, as a `Sampling` names, unbiased, with `EXACT`, the sum over every opponent hand + and every physical entry of the key, as their limit; and the exact + probability a walk arrives there; rules-only, and it must never import + `exploitability.py`, which grades what it feeds). - `answer/` — everything built on the solved game. `strategy` (the frozen LCFR average, a release asset pinned by SHA-256), `analysis` (`minidraw-analysis`, writes `figures/rung3/`), `play` (`minidraw-play`), diff --git a/src/drawmaha_solver/minidrawmaha/solver/boost_target.py b/src/drawmaha_solver/minidrawmaha/solver/boost_target.py new file mode 100644 index 00000000..8830f431 --- /dev/null +++ b/src/drawmaha_solver/minidrawmaha/solver/boost_target.py @@ -0,0 +1,1112 @@ +"""The boost target: extra samples of a high-noise row's regret, unbiased, with the exact sum as their limit. + +Rung 4 learns every infoset from the regret samples the walks bank there, +`r = u − ⟨σ, u⟩` with one opponent hand and one line of opponent play drawn +per visit (`mccfr.traverse`, `lockstep.walk`). At a rare round-2 row a visit +comes once in thousands of iterations and its sample swings by chips, almost +all of it the opponent's hidden hand. This module draws more samples of that +same quantity at a chosen row: it starts at the row, faces the opponent's +whole range (or k hands of it), and plays +the hand out with each source of randomness below — the board card, both +seats' replacements, the opponent's actions — sampled k of n or enumerated, +as a `Sampling` names. With every source enumerated (`EXACT`) the target is +the exact conditional mean the walk's samples estimate, and that is what the +tests pin the sampled forms to. A later stage (the "boost") banks the target +at the weight the walk's samples would have carried there, so a targeted +infoset is learned from a few good samples instead of a handful of noisy +ones. Against the whole range the opponent's hidden hand — nearly all of a +walk sample's spread — is gone, but a low-k form is not lower-noise than a +walk sample by construction: a k-of-n fan here is public, so it can keep a +branch that is dead for the hand at hand (the board card among a throw's 12 +replacements, a held card among the 14 second-board cards) and scales the +live ones up to pay for it, where the walk deals only live cards. Unbiased +either way; how much noise a form leaves is measured, not assumed. + +**What is sampled, and why each stays unbiased.** Every sampled source +replaces a sum over n branches by k of them, so that the expectation over +the draw is the sum: + +- a chance fan (the second board card, 14 branches; a throw's replacement, + 12 per hand, for either seat) keeps k of its n branches without + replacement, each scaled n/k: each branch is kept with probability k/n; +- the opponent's actions send each opponent state's weight down k actions + drawn from its σ at 1/k each, instead of splitting it by σ: the expected + share of an action is σ(a); +- the opponent range draws k states per physical entry in proportion to + their weight, among those sharing no card with the entry's own state, each + carrying 1/k of the entry's total; +- `playouts` independent play-outs are averaged. + +The traverser's own actions are never sampled, as the walk never samples +them: the regret is defined against each of them. The arrival — the ratio's +denominator below — is always exact. It is the cheap half (the opponent's +reach, back-solved upward, never sampled), and an exact denominator under an +unbiased numerator keeps the ratio unbiased; a sampled one would not, since +the mean of a ratio is not the ratio of the means. Each row draws from +`default_rng([seed, t, seat, row, _STREAM_TAG])`, its own stream, independent +of the walk's, so a row's target depends on (seed, t, seat, row) and never on +which rows share the call. + +**What is computed.** For a row standing for the key I of seat `seat`: + + regret(I, a) = Σ_{b,h,o} w(b,h,o) · (v(a | b,h,o) − v(I | b,h,o)) / Σ_{b,h,o} w(b,h,o) + arrival(I) = Σ_{b,h,o} w(b,h,o) + w(b,h,o) = deal(h) · r_b[o] · compat_{b,h}[o] + +where (b, h) runs over every PHYSICAL entry of the key — a board row in the +order it came, and the seat's physical (hole, thrown card) — and o over the +opponent's physical states. `r_b[o]` is the reach of the deck and the +opponent at the row's public point on board b: the opponent's deal weight, +the chance probability of every card dealt along the path, and the +opponent's σ at each of its decisions, pushed through its draw. `compat` +is one where the two seats' cards are disjoint. `deal(h)` is the +traverser's own deal probability — seat 0 carries 1/100,100 per allowed +pair and seat 1 carries 1 (the grader's convention in `_weights`, restated +here) — and at a post-draw state it is the sum over the three pre-draw holes +that could have become it, because the key forgets which card was drawn and +the walk arrives through any of them. `v(a | b,h,o)` is the traverser's +expected chips below action a with both seats at σ and the deck averaged. + +**Why the ratio, and why the traverser's own σ never enters the weight.** +The walk enumerates the traverser's own actions, so how often it arrives at +I is the deck's and the opponent's doing alone: `arrival` is exactly the +expected number of visits per walk, and `regret` is the conditional mean of +the banked sample given a visit — `E[r | reach]`. Normalising by Σ w is what +makes the exact row commensurable with the sampled ones: a sampled row's +target is one draw of r, so its expectation is this ratio, and the boost's +row weight, `K · arrival · t`, restores the total weight the K walks would +have put there. Dropping the denominator, or weighting every target 1, +would be a "teleport" that over-weights the rare rows it lands on, which is +the bias the targeted-sampling plan (kept, like the rung-3 plan, outside +the repo on Alec's machine under `~/Desktop/Claude/poker/`) exists to +avoid. The traverser's own σ enters only inside `v(I) = ⟨σ, v(·)⟩`, the +regret's baseline, as it does in the walk. One caveat on "visits": a walk +can bank a row twice (about one traversal in a thousand, through the suit +relabelling), so `arrival` is the expected number of visits per walk, a +probability but for those, and `regret` is the mean banked sample per +visit. + +**Why every physical entry is summed.** A key is a suit orbit of one +picture — up to six physical (board, state) pairs, fewer when the picture +has a suit symmetry. In the full game a global suit relabelling maps the +whole game onto itself and σ is read in key space, so the six entries +carry identical numbers and one representative times the orbit size would +do. The module sums them anyway: a restricted deal set (`build(holes=)`, +which PR 4's oracle test and this module's own tests use) is not closed +under relabelling, and there the entries face different opponent ranges +and the sum is load-bearing. The orbit shortcut is an optimisation left for +the day the full game's cost matters, with the sum as its oracle. + +**Card removal, applied once and in the same place.** The numerator's +weights and the denominator's are the same `compat` vector, computed by +`_compat` from the two seats' card sets. Below the row nothing further is +masked by hand: a replacement the deck could not deal — the board card, a +card the other seat holds — leads to a terminal where the two seats share a +card and pays zero through the same compat, and the chance node's public +1/|stub| is then exactly the probability of each live replacement, because +the stub's size is public (the walk relies on the same fact). The board-2 +deal masks the opponent's states the same way the grader's levels do. + +**The frame.** Every vector is indexed by a seat's physical state, 455 +holes in deck order × 16 slots (slot 0 for nothing thrown, 1 + c for card +c), 7,280 entries, with the board as a batch axis: 15 rows after the first +card, 210 ORDERED pairs after the second. Canonical keys enter at one place, +a map from (board row, state) to a position in `private_keys(shape)`, used +to gather σ rows; the same map inverted lists each key's physical entries. +That is the grader's frame, re-derived here from `rules/` alone. + +**Rules-only, on purpose.** This module imports `rules/` and the packed +table and nothing from `exploitability.py`, which grades what the boost +feeds; a tree, a key map or a showdown table shared between the two would +let one bug certify itself. The tests hold the two descriptions to each +other instead: the back-solved reach equals the grader's carried vector on +every board of every point to 1e-10, and the regret and arrival equal a +world-by-world walk through `MiniState` and the mean of thousands of real +walks. + +**Cost, measured on this Mac (2026-11-06, a table with every action +positive, so nothing below a row prunes).** The description is built once +per process, 5–7 s — the key maps, the draw maps, the public tree with +parent links, the showdown ranks — and holds no showdown matrix: shares and +compatibility are computed per terminal from the holes' ranks and card sets, +folded over the opponent's 455 holes. `sigma` over the whole table takes +0.12 s. Exactly, a round-2 row costs about 2 ms, a draw row 160–230 ms and a +round-1 row 0.5–0.75 s, because the value below the last two expands over +the traverser's own replacements (36 states per throw) and the second board +card (14 boards). Under the walk's own sampling against the whole range +(`Sampling()`) a draw row +costs about 25–32 ms and a round-1 row about 100–130 ms: the board card and +the hero's replacements are what sampling cuts. A round-2 row costs more +sampled than exact, about 3–4 ms: against a whole range the opponent's +actions are split by σ in one array operation, and drawing them per state +only adds a draw and noise — action sampling earns its keep when +`opponent_range` is a few hands, where most lines carry no weight and are +skipped. `tests/minidrawmaha/test_boost_target.py` prints the three costs on +every run. The opponent's reach is memoised per call at every node with one +board card out that lies on the path to a decision (66: the 29 level-1 +decision points and 37 chance nodes, 15 boards each) and at every round-2 +opening decision (20: one per round-1 line and pair of draw counts, 210 +boards each), 58 KB a (node, board) pair, so the memo is at most +66 × 15 + 20 × 210 = 5,190 vectors, 302 MB — a count the test file +re-derives from the tree; a call over 2,240 round-2 rows peaked at about +260 MB. +""" + +from __future__ import annotations + +import numbers +from collections.abc import Iterable, Sequence +from dataclasses import dataclass +from functools import cache +from itertools import combinations, permutations +from typing import Any + +import numpy as np + +from drawmaha_solver.minidrawmaha.rules.cards import DECK, Card +from drawmaha_solver.minidrawmaha.rules.enumeration import ( + PublicPoint, + private_keys, + public_decision_points, +) +from drawmaha_solver.minidrawmaha.rules.game import ( + BETTING_ACTIONS, + DRAW_ACTIONS, + Action, + MiniState, + NodeKind, + canonical_picture, + chip_state, + draw_order, + throw_count, +) +from drawmaha_solver.minidrawmaha.rules.hands import HOLE_CARDS, inner_score, outer_score +from drawmaha_solver.minidrawmaha.solver.packed_table import PackedTable + +Hole = tuple[Card, ...] +SEATS = (0, 1) + +# The widest ledger: the draw's four throws. Regret rows are padded to it. +TARGET_WIDTH = max(len(DRAW_ACTIONS), len(BETTING_ACTIONS)) + +# The tail of a boost stream's seed, ASCII "BOOST": 39 bits, which +# SeedSequence takes as two 32-bit words. For seed, t, seat and row below +# 2³² a row's stream is six words, so it cannot equal a walk's four-word +# stream or the reservoir's five-word "RESV" one, and its last two words +# tell it from either at any size. +_STREAM_TAG = int.from_bytes(b"BOOST", "big") + +# --------------------------------------------------------------------------- +# What the play-out below a boosted row samples +# --------------------------------------------------------------------------- + +_SOURCES = ("opponent_range", "board_cards", "opponent_draws", "hero_draws", "opponent_actions") + +# The fans a chance count is read against: the second board card can be any +# of the 14 cards the first left, and a throw's replacement any of the 12 +# cards not in the thrower's hole (dead ones pay zero below). +_BOARD_FAN = len(DECK) - 1 +_REPLACEMENT_FAN = len(DECK) - HOLE_CARDS + +@dataclass(frozen=True, slots=True) +class Sampling: + """How many of each random thing below a boosted row one play-out takes; `None` takes all of them. + + - `playouts` — independent play-outs from the row, averaged. + - `opponent_range` — `None` plays against the whole range; k draws k + opponent hands per physical entry, with replacement, in proportion to + their weight, each carrying 1/k of the entry's total. + - `board_cards` — k of the 14 cards the second board card can be, + without replacement, each scaled 14/k. + - `opponent_draws`, `hero_draws` — k of the 12 replacements a throw can + bring, per hand, without replacement, each scaled 12/k. + - `opponent_actions` — `None` splits every opponent hand's weight over its + actions by σ; k sends it down k actions drawn from σ, 1/k each. + + A chance count at or above its fan is that source enumerated. The + defaults are the walk's own sampling below the row against the whole + opponent range; `EXACT` enumerates everything and is the oracle. The + traverser's own actions are never sampled — the walk enumerates them, and + the regret is defined against them — only the cards its throws bring. + """ + + playouts: int = 1 + opponent_range: int | None = None + board_cards: int | None = 1 + opponent_draws: int | None = 1 + hero_draws: int | None = 1 + opponent_actions: int | None = 1 + + def __post_init__(self) -> None: + for name in ("playouts", *_SOURCES): + value = getattr(self, name) + if value is None and name != "playouts": + continue + if isinstance(value, bool) or not isinstance(value, numbers.Integral) or value < 1: + allowed = "a positive int" if name == "playouts" else "a positive int, or None for all" + raise ValueError(f"Sampling.{name} must be {allowed}, got {value!r}") + if self.is_exact and self.playouts != 1: + raise ValueError( + f"Sampling draws nothing (every count None, or a chance count at or above its fan), so every play-out is the same; playouts must be 1, got {self.playouts}" + ) + + @property + def is_exact(self) -> bool: + """Whether nothing is drawn: every count `None`, or a chance count at or above its fan.""" + return ( + self.opponent_range is None + and self.opponent_actions is None + and all( + k is None or k >= fan + for k, fan in ( + (self.board_cards, _BOARD_FAN), + (self.opponent_draws, _REPLACEMENT_FAN), + (self.hero_draws, _REPLACEMENT_FAN), + ) + ) + ) + +EXACT = Sampling( + playouts=1, opponent_range=None, board_cards=None, opponent_draws=None, hero_draws=None, opponent_actions=None +) + +@dataclass(frozen=True, slots=True, eq=False) +class _Draws: + """The play-out's one source of randomness: under `EXACT` every method hands back everything and the RNG is never touched.""" + + sampling: Sampling + rng: np.random.Generator | None + + def subset(self, k: int | None, n: int, shape: tuple[int, ...]) -> np.ndarray | None: + """k of n positions for every index of `shape`, without replacement; `None` for all n.""" + if k is None or k >= n: + return None + return np.argsort(self.rng.random((*shape, n)), axis=-1)[..., :k] + + def actions(self, probs: np.ndarray) -> np.ndarray: + """Per opponent state, the share of its weight each action carries: σ itself, or k draws from σ at 1/k each.""" + k = self.sampling.opponent_actions + if k is None: + return probs + cdf = np.cumsum(probs, axis=-1) + cdf[..., -1] = 1.0 # a row summing to 1 − ulp must still pick an action + columns = np.arange(probs.shape[-1]) + counts = np.zeros_like(probs) + for _ in range(k): + pick = (self.rng.random(probs.shape[:-1])[..., None] < cdf).argmax(axis=-1) + counts += pick[..., None] == columns + return counts / k + + def opponents(self, u: np.ndarray, clash: np.ndarray) -> np.ndarray: + """Per entry, the opponent's weighted vector, or k hands drawn from its live weight at 1/k of the total each.""" + k = self.sampling.opponent_range + if k is None: + return u + live = np.where(clash, 0.0, u) + totals = live.sum(axis=1) + out = np.zeros_like(u) + for e, total in enumerate(totals): + if total > 0.0: + picks = self.rng.choice(u.shape[1], size=k, p=live[e] / total) + out[e] = np.bincount(picks, minlength=u.shape[1]) * (total / k) + return out + +_EXACT_DRAWS = _Draws(EXACT, None) + +# --------------------------------------------------------------------------- +# The frame: physical states and board rows +# --------------------------------------------------------------------------- + +HOLES: tuple[Hole, ...] = tuple(combinations(DECK, HOLE_CARDS)) +SLOTS = 1 + len(DECK) +N_STATES = len(HOLES) * SLOTS + +_HOLE_INDEX = {hole: index for index, hole in enumerate(HOLES)} +_BOARD_SETS = tuple(combinations(DECK, 2)) +_SET_INDEX = {pair: index for index, pair in enumerate(_BOARD_SETS)} +_LEVEL_ROWS = (1, len(DECK), len(DECK) * (len(DECK) - 1)) + +def state_index(hole: Hole, thrown: Hole) -> int: + """The row of `(hole, thrown)` in a seat's vector; `thrown` is empty or one card.""" + slot = 0 if not thrown else 1 + DECK.index(thrown[0]) + return _HOLE_INDEX[tuple(sorted(hole))] * SLOTS + slot + +def board_row(board: tuple[Card, ...]) -> int: + """A board's batch row: deck order for one card, `(first, second)` ordered for two. + + Ordered pairs, 210 rows rather than 105 sets, because round 1 was played + against the first card alone and the two orders carry different reach + into round 2; only the showdown, which cannot tell them apart, is per set. + """ + if len(board) == 1: + return DECK.index(board[0]) + first, second = DECK.index(board[0]), DECK.index(board[1]) + return first * (len(DECK) - 1) + (second if second < first else second - 1) + +def _state_cards() -> tuple[np.ndarray, np.ndarray]: + """Per state: which deck cards it accounts for (hole and thrown), and whether it is a real state.""" + holds = np.zeros((N_STATES, len(DECK)), dtype=bool) + valid = np.zeros(N_STATES, dtype=bool) + for h, hole in enumerate(HOLES): + for slot in range(SLOTS): + s = h * SLOTS + slot + for card in hole: + holds[s, DECK.index(card)] = True + if slot == 0: + valid[s] = True + elif not holds[s, slot - 1]: + holds[s, slot - 1] = True + valid[s] = True + return holds, valid + +# --------------------------------------------------------------------------- +# The description: levels, key maps, draw maps, the public tree, the showdown ranks +# --------------------------------------------------------------------------- + +@dataclass(frozen=True, slots=True, eq=False) +class _Level: + """The boards at one board count: `ok[b, s]` is 1 when state s avoids board b; `parent[b]` is the board above b and `children_of[p]` the boards of this level under p.""" + + ok: np.ndarray + parent: np.ndarray + children_of: np.ndarray + +@dataclass(frozen=True, slots=True, eq=False) +class _KeyMap: + """(board row, state) → position in one shape's `private_keys`, −1 outside its support; and the inverse.""" + + index: np.ndarray # int32 (n_boards, N_STATES) + n_keys: int + entry_start: np.ndarray # int64 (n_keys + 1,): entries of key k are entry_*[start[k]:start[k+1]] + entry_board: np.ndarray # int32 + entry_state: np.ndarray # int32 + +@dataclass(frozen=True, slots=True, eq=False) +class _DrawMap: + """One throw as a map from pre-draw holes to post-draw states: `targets[b, k, j]` on board b, hole k, replacement j.""" + + source: np.ndarray # int32 (455,): the pre-draw state of each hole + targets: np.ndarray # int32 (15, 455, 12) + +@dataclass(frozen=True, slots=True, eq=False) +class _Edge: + """The actions of a decision that lead to one child, with the draw map of each throw (None for a bet or standing pat).""" + + child: _Node + actions: tuple[int, ...] + draws: tuple[_DrawMap | None, ...] + +@dataclass(frozen=True, slots=True, eq=False) +class _Node: + """One public node. `point` is a decision's index among the 141 table points; `deals_board` tells a board deal from a replacement.""" + + index: int + kind: NodeKind + level: int + discards: tuple[int, int] + player: int | None + actions: tuple[Any, ...] + point: int | None + edges: tuple[_Edge, ...] + child: _Node | None + prob: float + deals_board: bool + fold_winner: int | None + stake: float + pot: float + committed: tuple[float, float] + +@dataclass(frozen=True, slots=True, eq=False) +class _Description: + """Everything that does not depend on the deal set, built once per process.""" + + points: tuple[PublicPoint, ...] + row_offset: np.ndarray + column_offset: np.ndarray + width: np.ndarray + holds: np.ndarray # bool (N_STATES, 15) + valid: np.ndarray # bool (N_STATES,) + hole_of: np.ndarray # int32 (N_STATES,) + hole_holds: np.ndarray # float64 (455, 15): which cards each hole holds + levels: tuple[_Level, _Level, _Level] + set_of_row: np.ndarray # int32 (210,): level-2 row → board set, for the showdown alone + keys: dict[tuple[int, int], _KeyMap] + root: _Node + parent: dict[int, tuple[_Node, _Edge | None]] + decision_of_point: tuple[_Node, ...] + inner: np.ndarray # int32 (455,): dense rank of each hole's inner hand + outer: np.ndarray # int32 (105, 455): dense rank of each hole's outer hand with each board set, 0 where they clash + +def _levels(holds: np.ndarray, valid: np.ndarray) -> tuple[_Level, _Level, _Level]: + n = len(DECK) + # `parent` and `children_of` both live on the CHILD level: `parent[b]` is + # the board above b, `children_of[p]` every board of this level under p. + zero = _Level( + ok=valid[None, :].astype(np.float64), + parent=np.zeros(1, np.int32), + children_of=np.zeros((0, 0), np.int32), + ) + one = _Level( + ok=(valid[None, :] & ~holds.T).astype(np.float64), + parent=np.zeros(n, np.int32), + children_of=np.arange(n, dtype=np.int32)[None, :], + ) + ok = np.zeros((n * (n - 1), N_STATES)) + parent = np.zeros(n * (n - 1), np.int32) + for first in range(n): + for second in range(n): + if first != second: + row = board_row((DECK[first], DECK[second])) + ok[row] = valid & ~holds[:, first] & ~holds[:, second] + parent[row] = first + two = _Level( + ok=ok, + parent=parent, + children_of=np.stack([np.flatnonzero(parent == first) for first in range(n)]).astype(np.int32), + ) + return zero, one, two + +def _key_map(level: int, discards: int) -> _KeyMap: + """(board row, state) → position in `private_keys(level, discards)`, built per ORDERED board row, and its inverse.""" + keys = private_keys(board_cards=level, discards=discards) + position = {key: k for k, key in enumerate(keys)} + boards = [(card,) for card in DECK] if level == 1 else list(permutations(DECK, 2)) + index = np.full((_LEVEL_ROWS[level], N_STATES), -1, np.int32) + for board in boards: + b = board_row(board) + for h, hole in enumerate(HOLES): + if set(hole) & set(board): + continue + if discards == 0: + index[b, h * SLOTS] = position[canonical_picture(hole, (), board)] + continue + for card in DECK: + if card in hole or card in board: + continue + index[b, h * SLOTS + 1 + DECK.index(card)] = position[ + canonical_picture(hole, (card,), board) + ] + flat = index.ravel() + order = np.argsort(flat, kind="stable") + order = order[flat[order] >= 0] + sorted_keys = flat[order] + start = np.searchsorted(sorted_keys, np.arange(len(keys) + 1)).astype(np.int64) + return _KeyMap( + index=index, + n_keys=len(keys), + entry_start=start, + entry_board=(order // N_STATES).astype(np.int32), + entry_state=(order % N_STATES).astype(np.int32), + ) + +def _draw_maps() -> tuple[_DrawMap, _DrawMap, _DrawMap]: + """The three one-card throws as maps, one board card out, every replacement listed. + + Which physical card `THROW_LOW` names depends on the board through the + canonical relabelling, so the thrown card is read off `draw_order`. + """ + n = len(DECK) + replacements = n - HOLE_CARDS + targets = np.zeros((n, len(HOLES), HOLE_CARDS, replacements), np.int32) + for b, board_card in enumerate(DECK): + for h, hole in enumerate(HOLES): + ordered = draw_order(hole=hole, discarded=(), board=(board_card,)) + for k, thrown in enumerate(ordered): + kept = tuple(card for card in hole if card != thrown) + spare = [card for card in DECK if card not in hole] + for j, drawn in enumerate(spare): + targets[b, h, k, j] = state_index(kept + (drawn,), (thrown,)) + source = (np.arange(len(HOLES)) * SLOTS).astype(np.int32) + return tuple(_DrawMap(source=source, targets=targets[:, :, k, :]) for k in range(HOLE_CARDS)) + +def _public_tree( + draws: tuple[_DrawMap, ...], points: tuple[PublicPoint, ...] +) -> tuple[_Node, dict[int, tuple[_Node, _Edge | None]], tuple[_Node, ...]]: + """The public tree with edges and parent links, walked from one representative deal. + + Replacement deals are told apart from board deals (the census merges + them), and a decision is tied to its table point by public signature. + """ + by_signature = { + (point.board_cards, point.draws, point.betting): index for index, point in enumerate(points) + } + root_state = MiniState(holes=(DECK[0:HOLE_CARDS], DECK[HOLE_CARDS : 2 * HOLE_CARDS])) + nodes: dict[tuple, _Node] = {} + order: list[_Node | None] = [] + parent: dict[int, tuple[_Node, _Edge | None]] = {} + decision_of_point: dict[int, _Node] = {} + + def visit(state: MiniState) -> _Node: + signature = ( + state.kind, + len(state.board), + state.draws, + state.betting, + tuple(len(hole) for hole in state.holes), + ) + if signature in nodes: + return nodes[signature] + index = len(order) + order.append(None) + fields: dict[str, Any] = dict( + index=index, + kind=state.kind, + level=len(state.board), + discards=tuple(len(pile) for pile in state.discards), + player=None, + actions=(), + point=None, + edges=(), + child=None, + prob=0.0, + deals_board=False, + fold_winner=None, + stake=0.0, + pot=0.0, + committed=(0.0, 0.0), + ) + if state.kind is NodeKind.TERMINAL: + chips = chip_state(state.betting) + line = state.betting[-1] + if line and line[-1] is Action.FOLD: + loser = (len(line) - 1) % 2 + fields.update(fold_winner=1 - loser, stake=float(chips.committed[loser])) + else: + fields.update( + pot=float(chips.pot), + committed=(float(chips.committed[0]), float(chips.committed[1])), + ) + node = _Node(**fields) + elif state.kind is NodeKind.CHANCE: + outcomes = state.chance_outcomes() + child = visit(state.apply_chance(outcomes[0][0])) + fields.update( + child=child, + prob=outcomes[0][1], + # A short hole means the deck owes a replacement; otherwise a board card. + deals_board=all(len(hole) == HOLE_CARDS for hole in state.holes), + ) + node = _Node(**fields) + parent[child.index] = (node, None) + else: + actions = state.legal_actions() + grouped: dict[int, tuple[list[int], list[_DrawMap | None]]] = {} + children: dict[int, _Node] = {} + for k, action in enumerate(actions): + child = visit(state.apply(action)) + children[child.index] = child + columns, maps = grouped.setdefault(child.index, ([], [])) + columns.append(k) + maps.append( + draws[DRAW_ACTIONS.index(action) - 1] + if state.is_draw_decision() and throw_count(action) > 0 + else None + ) + edges = tuple( + _Edge(child=children[child_index], actions=tuple(columns), draws=tuple(maps)) + for child_index, (columns, maps) in grouped.items() + ) + fields.update( + player=state.current_player, + actions=actions, + point=by_signature[(len(state.board), state.draws, state.betting)], + edges=edges, + ) + node = _Node(**fields) + for edge in edges: + parent[edge.child.index] = (node, edge) + decision_of_point[node.point] = node + nodes[signature] = node + order[index] = node + return node + + root = visit(root_state) + if len(decision_of_point) != len(points): + raise AssertionError("the public tree and the enumerator disagree on the decision points") + return root, parent, tuple(decision_of_point[p] for p in range(len(points))) + +def _dense_rank(scores: Sequence[Any]) -> np.ndarray: + """Scores as integers with the same order, so a sign of a difference is a comparison.""" + ladder = {score: rank for rank, score in enumerate(sorted(set(scores)))} + return np.array([ladder[score] for score in scores], dtype=np.int32) + +def _showdown_ranks() -> tuple[np.ndarray, np.ndarray]: + """Per hole, the dense rank of its inner hand, and of its outer hand with each board set (0 where the hole clashes).""" + inner = _dense_rank([inner_score(hole) for hole in HOLES]) + outer = np.zeros((len(_BOARD_SETS), len(HOLES)), np.int32) + for s, board in enumerate(_BOARD_SETS): + live = [h for h, hole in enumerate(HOLES) if not set(hole) & set(board)] + outer[s, live] = _dense_rank([outer_score(HOLES[h], board) for h in live]) + return inner, outer + +@cache +def _description() -> _Description: + points = public_decision_points() + sizes = [len(private_keys(board_cards=p.board_cards, discards=p.discards)) for p in points] + widths = [p.width for p in points] + holds, valid = _state_cards() + hole_of = (np.arange(N_STATES) // SLOTS).astype(np.int32) + set_of_row = np.zeros(_LEVEL_ROWS[2], np.int32) + for first in DECK: + for second in DECK: + if first != second: + set_of_row[board_row((first, second))] = _SET_INDEX[tuple(sorted((first, second)))] + root, parent, decision_of_point = _public_tree(_draw_maps(), points) + inner, outer = _showdown_ranks() + return _Description( + points=points, + row_offset=np.concatenate([[0], np.cumsum(sizes)]).astype(np.int64), + column_offset=np.concatenate( + [[0], np.cumsum([s * w for s, w in zip(sizes, widths, strict=True)])] + ).astype(np.int64), + width=np.asarray(widths, dtype=np.int64), + holds=holds, + valid=valid, + hole_of=hole_of, + hole_holds=holds[::SLOTS].astype(np.float64), + levels=_levels(holds, valid), + set_of_row=set_of_row, + keys={shape: _key_map(*shape) for shape in ((1, 0), (2, 0), (2, 1))}, + root=root, + parent=parent, + decision_of_point=decision_of_point, + inner=inner, + outer=outer, + ) + +# --------------------------------------------------------------------------- +# The deal set +# --------------------------------------------------------------------------- + +def _weights(holes: tuple[Iterable[Hole], Iterable[Hole]] | None) -> tuple[np.ndarray, np.ndarray]: + """Per-seat deal probability at pre-draw states, zero elsewhere: uniform over the allowed disjoint pairs. + + P(r, h) = weight[0][r] · weight[1][h] · [r ∩ h = ∅]; 1/100,100 in the + full game, carried by seat 0 with seat 1 at 1 — the walk's `random_deal` + distribution, and the grader's convention. + """ + allowed = ( + (set(HOLES), set(HOLES)) + if holes is None + else (set(map(tuple, map(sorted, holes[0]))), set(map(tuple, map(sorted, holes[1])))) + ) + for seat, chosen in enumerate(allowed): + unknown = chosen - set(HOLES) + if unknown: + raise ValueError(f"seat {seat} cannot be dealt {sorted(unknown)}") + pairs = sum(1 for r in allowed[0] for h in allowed[1] if not set(r) & set(h)) + if pairs == 0: + raise ValueError("no deal gives both seats a hole from their allowed sets") + out = [] + for seat, chosen in enumerate(allowed): + weight = np.zeros(N_STATES) + weight[::SLOTS] = [1.0 if hole in chosen else 0.0 for hole in HOLES] + if seat == 0: + weight /= pairs + out.append(weight) + return out[0], out[1] + +def _own_deal(weight: np.ndarray) -> dict[tuple[int, int], np.ndarray]: + """The traverser's own deal factor per state, per shape: the pre-draw weight, or the sum over the three pre-draw holes of a post-draw state.""" + post = np.zeros(N_STATES) + for h, hole in enumerate(HOLES): + for slot in range(1, SLOTS): + thrown = DECK[slot - 1] + if thrown in hole: + continue + post[h * SLOTS + slot] = sum( + weight[state_index(tuple(c for c in hole if c != drawn) + (thrown,), ())] + for drawn in hole + ) + return {(1, 0): weight, (2, 0): weight, (2, 1): post} + +# --------------------------------------------------------------------------- +# The targets +# --------------------------------------------------------------------------- + +_Sigma = Sequence[np.ndarray] + +@dataclass(frozen=True, eq=False) +class BoostTargets: + """Sampled (or exact) regret and exact arrival for any table row, under a strategy read from the materialised table. + + boost = BoostTargets.build() # once per process + sigma = boost.sigma(table) # the walk's σ, per point + regret, arrival = boost.targets(sigma, seat=0, rows=rows, sampling=Sampling(), seed=seed, t=t) + regret, arrival = boost.targets(sigma, seat=0, rows=rows, sampling=EXACT) # the oracle + + `regret` is `(n, 4)` float64, zero past each row's width; `arrival` is + `(n,)`, the expected number of visits one walk with traverser `seat` + pays the row (a probability, but for the rare walk that reaches a row + twice through two throws). + `arrivals` is the second number alone, at a fraction of the cost. + `build(holes=)` restricts what each seat may be dealt, uniformly over + the pairs that remain, as `exploitability.minidrawmaha_game(holes=)` does + and as a test's `deal` must then sample; the default is every deal. + """ + + description: _Description + weight: tuple[np.ndarray, np.ndarray] + own_deal: tuple[dict[tuple[int, int], np.ndarray], dict[tuple[int, int], np.ndarray]] + + @classmethod + def build(cls, holes: tuple[Iterable[Hole], Iterable[Hole]] | None = None) -> BoostTargets: + """The description (shared, built once) with this deal set's weights. The default is cached.""" + if holes is None: + return _full_game() + weights = _weights(holes) + return cls(_description(), weights, (_own_deal(weights[0]), _own_deal(weights[1]))) + + @property + def points(self) -> tuple[PublicPoint, ...]: + return self.description.points + + def entries_of(self, shape: tuple[int, int], position: int) -> tuple[np.ndarray, np.ndarray]: + """The physical (board row, state) entries of key `position` in `shape`'s `private_keys`.""" + keys = self.description.keys[shape] + span = slice(keys.entry_start[position], keys.entry_start[position + 1]) + return keys.entry_board[span], keys.entry_state[span] + + # -- σ ------------------------------------------------------------------ + + def sigma(self, table: PackedTable) -> tuple[np.ndarray, ...]: + """Regret matching over the whole-game table, one `(n_keys, width)` array per point, float64. + + The positive part of each row over its total, uniform where nothing + is positive: `RegretMatcher.strategy`, vectorised with one `reduceat`, + which adds a row left to right where the ledger's `sum` may pair, so + a row can differ from the walk's by one ulp. Refuses a table that is + not the whole game in the whole game's order. + """ + d = self.description + widths = table.widths() + if ( + len(table) != d.row_offset[-1] + or table.cumulative_regret.size != d.column_offset[-1] + or not np.array_equal(widths, np.repeat(d.width, np.diff(d.row_offset))) + ): + raise ValueError( + "the exact target reads the whole game's layout and was handed a table of " + f"{len(table):,} rows that is not the whole game in the whole game's order" + ) + positive = np.maximum(table.cumulative_regret, 0.0) + starts = np.cumsum(widths) - widths + totals = np.add.reduceat(positive, starts) + per_column = np.repeat(totals, widths) + uniform = np.repeat(1.0 / widths, widths) + with np.errstate(invalid="ignore", divide="ignore"): + flat = np.where(per_column > 0.0, positive / per_column, uniform) + return tuple( + flat[d.column_offset[p] : d.column_offset[p + 1]].reshape(-1, int(d.width[p])) + for p in range(len(d.points)) + ) + + # -- the targets -------------------------------------------------------- + + def targets( + self, + sigma: _Sigma, + *, + seat: int, + rows: np.ndarray, + sampling: Sampling, + seed: int | None = None, + t: int | None = None, + ) -> tuple[np.ndarray, np.ndarray]: + """Regret `(n, 4)` and arrival `(n,)` — expected visits per walk — for table `rows`, all of them `seat`'s, under `sigma`. + + The regret is exact under `EXACT` and an unbiased estimate of it + otherwise; the arrival is exact under any `sampling`. A sampled call + names `seed` and `t`, and row r draws from + `default_rng([seed, t, seat, r, _STREAM_TAG])` alone, so its target + does not depend on which rows share the call or in what order — and + a row asked for twice gets the same sample twice: more samples of one + row are `playouts`, not repeats. A row the + walk cannot reach (zero arrival) gets zero regret, not NaN; a row of + the other seat raises, as does one outside the table. + """ + if not isinstance(sampling, Sampling): + raise TypeError(f"sampling must be a Sampling (EXACT for the exact sum), got {sampling!r}") + if not sampling.is_exact: + for name, value in (("seed", seed), ("t", t)): + if isinstance(value, bool) or not isinstance(value, numbers.Integral) or value < 0: + raise ValueError(f"a sampled target needs a non-negative int {name}, got {value!r}") + seed, t = int(seed), int(t) + rows = self._rows(seat, rows) + d = self.description + regret = np.zeros((rows.size, TARGET_WIDTH)) + arrival = np.zeros(rows.size) + memo: dict[tuple[int, int, int], np.ndarray] = {} + for i, (node, position, boards, states, u, total) in enumerate( + self._entries(sigma, seat, rows, memo) + ): + arrival[i] = total + if total <= 0.0: + continue + draws = ( + _EXACT_DRAWS + if sampling.is_exact + else _Draws(sampling, np.random.default_rng([seed, t, seat, int(rows[i]), _STREAM_TAG])) + ) + # Opponent states sharing a card with the entry's hero state; only + # a range draw needs them, since the play-out's compat masks them. + clash = ( + None + if sampling.opponent_range is None + else (d.holds.astype(np.uint8) @ d.holds[states].T.astype(np.uint8)).T > 0 + ) + mix = sigma[node.point][position] + summed = np.zeros(len(mix)) + for _ in range(sampling.playouts): + facing = u if clash is None else draws.opponents(u, clash) + values = self._action_values(sigma, seat, node, facing, states[:, None], boards, draws)[:, 0, :] + summed += (values - (values @ mix)[:, None]).sum(axis=0) + regret[i, : len(mix)] = summed / (sampling.playouts * total) + return regret, arrival + + def arrivals(self, sigma: _Sigma, *, seat: int, rows: np.ndarray) -> np.ndarray: + """The arrival alone for table `rows`: the reach and the compatibility sum, no value below.""" + rows = self._rows(seat, rows) + memo: dict[tuple[int, int, int], np.ndarray] = {} + return np.fromiter( + (total for *_, total in self._entries(sigma, seat, rows, memo)), dtype=np.float64, count=rows.size + ) + + def reach(self, sigma: _Sigma, *, carried: int, point: int, board: int) -> np.ndarray: + """Seat `carried`'s reach at `point` on one board, over its physical states: deal weight × chance × its own σ along the path. + + What the grader carries down for the responder's opponent; here the + row's opponent is `carried = 1 − seat`. Hero's actions never enter. + """ + if carried not in SEATS: + raise ValueError(f"carried must be one of {SEATS}, got {carried!r}") + # A copy: at the root the vector is the deal weight itself. + return self._reach(sigma, carried, self.description.decision_of_point[point], board, {}).copy() + + def _rows(self, seat: int, rows: np.ndarray) -> np.ndarray: + if seat not in SEATS: + raise ValueError(f"seat must be one of {SEATS}, got {seat!r}") + d = self.description + rows = np.asarray(rows, dtype=np.int64) + if rows.ndim != 1: + raise ValueError(f"rows must be a 1-D array of table rows, got shape {rows.shape}") + if rows.size and (rows.min() < 0 or rows.max() >= d.row_offset[-1]): + raise IndexError(f"a row is outside the table's {d.row_offset[-1]:,} rows") + return rows + + def _entries(self, sigma: _Sigma, seat: int, rows: np.ndarray, memo: dict): + """Per row: its node and position, its physical entries, the weighted opponent vector at each, and the arrival.""" + d = self.description + own = self.own_deal[seat] + for row in rows.tolist(): + p = int(np.searchsorted(d.row_offset, row, side="right") - 1) + position = row - int(d.row_offset[p]) + node = d.decision_of_point[p] + if node.player != seat: + raise ValueError(f"row {row} belongs to seat {node.player}, not seat {seat}") + shape = (node.level, node.discards[seat]) + boards, states = self.entries_of(shape, position) + reach = np.stack([self._reach(sigma, 1 - seat, node, int(b), memo) for b in boards]) + u = own[shape][states][:, None] * reach + compat, _ = self._showdown(node.level, boards, states[:, None], u, shares=False) + yield node, position, boards, states, u, float(compat.sum()) + + # -- the back-solve ---------------------------------------------------- + + def _reach(self, sigma: _Sigma, carried: int, node: _Node, b: int, memo: dict) -> np.ndarray: + d = self.description + key = (carried, node.index, b) + if key in memo: + return memo[key] + if node is d.root: + return self.weight[carried] + above, edge = d.parent[node.index] + if above.kind is NodeKind.CHANCE: + level = d.levels[node.level] + if above.deals_board: + base = self._reach(sigma, carried, above, int(level.parent[b]), memo) + else: + base = self._reach(sigma, carried, above, b, memo) + out = above.prob * base * level.ok[b] + elif above.player != carried: + out = self._reach(sigma, carried, above, b, memo) + else: + base = self._reach(sigma, carried, above, b, memo) + index = d.keys[(above.level, above.discards[carried])].index[b] + valid = index >= 0 + safe = np.where(valid, index, 0) + table = sigma[above.point] + out = np.zeros(N_STATES) + for column, draw in zip(edge.actions, edge.draws, strict=True): + played = base * np.where(valid, table[safe, column], 0.0) + if draw is None: + out += played + else: + out += _push(played[None, :], draw, np.array([b]))[0] + # Memoised through the board-2 deal: the 66 level-1 nodes on the + # paths (decisions and chance nodes) have 15 boards each and the 20 + # round-2 opening decisions 210 each; the rest of round 2 is at most + # two opponent steps away and is recomputed, so a call's memo is at + # most 5,190 vectors whatever rows it is handed. + if node.level <= 1 or (above.kind is NodeKind.CHANCE and above.deals_board): + memo[key] = out + return out + + # -- the value below ---------------------------------------------------- + # + # The recursion carries, per board b_i: the opponent's weighted vector + # u_i (over its 7,280 states) and a SET of hero states h_i (the same + # count on every board). Hero's own moves never touch u; a throw widens + # the set twelvefold per throw, the board-2 deal widens the boards + # fourteenfold, and values come back as one number per (board, hero + # state). Keeping u per board rather than per hero state is what makes a + # round-1 row affordable: the opponent's σ gathers and pushes below are + # done once per board, not once per hero continuation. + + def _action_values( + self, sigma: _Sigma, hero: int, node: _Node, u: np.ndarray, h: np.ndarray, b: np.ndarray, draws: _Draws + ) -> np.ndarray: + """Hero's weighted value below each action of `node`: `(nb, nh, width)` for boards b, hero states h (nb, nh), opponent vectors u (nb, S).""" + d = self.description + nb, nh = h.shape + values = np.zeros((nb, nh, len(node.actions))) + for edge in node.edges: + if all(draw is None for draw in edge.draws): + below = self._value(sigma, hero, edge.child, u, h, b, draws) + for column in edge.actions: + values[..., column] = below + continue + # The throw edge: the three throws' replacement states, stacked, + # walk the one subtree below in one pass and are split after. + # Under sampling each (board, hero state) keeps k of its twelve + # replacements per throw, scaled 12/k; hero's throws themselves + # are all walked. + hole = d.hole_of[h] + fan = edge.draws[0].targets.shape[-1] + parts = [] + for draw in edge.draws: + replacements = draw.targets[b[:, None], hole] # (nb, nh, 12) + pick = draws.subset(draws.sampling.hero_draws, fan, (nb, nh)) + parts.append(replacements if pick is None else np.take_along_axis(replacements, pick, axis=-1)) + kept = parts[0].shape[-1] + stacked = np.concatenate(parts, axis=-1) # (nb, nh, 3 × kept) + below = self._value(sigma, hero, edge.child, u, stacked.reshape(nb, -1), b, draws) + below = below.reshape(nb, nh, len(edge.draws), kept).sum(axis=-1) + if kept < fan: + below *= fan / kept + for j, column in enumerate(edge.actions): + values[..., column] = below[..., j] + return values + + def _value( + self, sigma: _Sigma, hero: int, node: _Node, u: np.ndarray, h: np.ndarray, b: np.ndarray, draws: _Draws + ) -> np.ndarray: + """Hero's weighted expected chips from `node`, `(nb, nh)`, both seats at σ, the deck averaged (or sampled, by `draws`).""" + d = self.description + nb, nh = h.shape + if node.kind is NodeKind.TERMINAL: + compat, share = self._showdown(node.level, b, h, u, shares=node.fold_winner is None) + if node.fold_winner is not None: + sign = 1.0 if node.fold_winner == hero else -1.0 + return sign * node.stake * compat + return node.pot * share - node.committed[hero] * compat + if node.kind is NodeKind.CHANCE: + if node.deals_board: + level = d.levels[node.child.level] + kids = level.children_of[b] # (nb, 14) + full = kids.shape[1] + pick = draws.subset(draws.sampling.board_cards, full, (nb,)) + if pick is not None: + kids = np.take_along_axis(kids, pick, axis=1) + fan = kids.shape[1] + spread = node.prob * u[:, None, :] * level.ok[kids] + if fan < full: + spread *= full / fan + below = self._value( + sigma, hero, node.child, + spread.reshape(-1, N_STATES), np.repeat(h, fan, axis=0), kids.ravel(), draws, + ) + return below.reshape(nb, fan, nh).sum(axis=1) + return self._value( + sigma, hero, node.child, node.prob * u * d.levels[node.level].ok[b], h, b, draws + ) + if node.player == hero: + index = d.keys[(node.level, node.discards[hero])].index[b[:, None], h] # (nb, nh) + valid = index >= 0 + mix = np.where(valid[..., None], sigma[node.point][np.where(valid, index, 0)], 0.0) + return (self._action_values(sigma, hero, node, u, h, b, draws) * mix).sum(axis=-1) + index = d.keys[(node.level, node.discards[node.player])].index[b] # (nb, S) + valid = index >= 0 + safe = np.where(valid, index, 0) + share = draws.actions(sigma[node.point][safe]) # (nb, S, width) + total = np.zeros((nb, nh)) + for edge in node.edges: + pushed = np.zeros_like(u) + for column, draw in zip(edge.actions, edge.draws, strict=True): + played = u * np.where(valid, share[..., column], 0.0) + if draw is None: + pushed += played + continue + pick = draws.subset(draws.sampling.opponent_draws, draw.targets.shape[-1], (nb, len(HOLES))) + pushed += _push(played, draw, b, pick) + # A line no opponent state takes is worth nothing below; under + # action sampling most are, and walking them would cost the + # whole subtree for a zero. + if pushed.any(): + total += self._value(sigma, hero, edge.child, pushed, h, b, draws) + return total + + # -- the showdown and compatibility ------------------------------------ + + def _showdown( + self, level: int, b: np.ndarray, h: np.ndarray, u: np.ndarray, *, shares: bool + ) -> tuple[np.ndarray, np.ndarray | None]: + """Per (board i, hero state j): Σ_o u_i[o]·compat(h_ij, o) and, with `shares`, the same weighted by hero's pot share. + + Compatibility is the two card sets being disjoint — hero's hole and + thrown card against the opponent's hole and thrown card — and a hero + state that holds a card of its own board is no state and scores + zero. Folded over the opponent's HOLES: u_i summed over the 16 slots + of each hole gives the hole's total reach, and the slots of the cards + hero holds are subtracted back out, because an opponent who threw one + of hero's cards cannot be there. Shares come from the dense ranks of + both halves: ¼(2 + sign(inner) + sign(outer)). + """ + d = self.description + nb, nh = h.shape + folded = u.reshape(nb, len(HOLES), SLOTS) + total = folded.sum(axis=-1) # (nb, 455) + mine = d.holds[h].astype(np.float64).transpose(0, 2, 1) # (nb, 15, nh) + shared = d.hole_holds @ mine # (nb, 455, nh): cards an opponent hole shares with hero + excluded = folded[:, :, 1:] @ mine # (nb, 455, nh): reach of that hole's states that threw a card of hero's + terms = np.where(shared == 0.0, total[:, :, None] - excluded, 0.0) + live = (d.valid[h] & (d.levels[level].ok[b[:, None], h] > 0.0)).astype(np.float64) + compat = terms.sum(axis=1) * live + if not shares: + return compat, None + hole = d.hole_of[h] # (nb, nh) + sets = d.set_of_row[b] # (nb,) + inner = np.sign(d.inner[hole][:, None, :] - d.inner[None, :, None]) + outer = np.sign(d.outer[sets[:, None], hole][:, None, :] - d.outer[sets][:, :, None]) + share = 0.25 * (2.0 + inner + outer) + return compat, (terms * share).sum(axis=1) * live + +def _push(played: np.ndarray, draw: _DrawMap, b: np.ndarray, pick: np.ndarray | None = None) -> np.ndarray: + """Each pair's vector through a throw on its level-1 board: every pre-draw hole spreads over its twelve replacements, or over the k of them `pick` keeps, scaled 12/k.""" + m = played.shape[0] + targets = draw.targets[b] # (m, 455, 12) + full = targets.shape[2] + if pick is not None: + targets = np.take_along_axis(targets, pick, axis=2) + offsets = (np.arange(m) * N_STATES)[:, None, None] + weights = np.repeat(played[:, draw.source], targets.shape[2], axis=1).ravel() + if targets.shape[2] < full: + weights *= full / targets.shape[2] + return np.bincount((offsets + targets).ravel(), weights=weights, minlength=m * N_STATES).reshape( + m, N_STATES + ) + +@cache +def _full_game() -> BoostTargets: + weights = _weights(None) + return BoostTargets(_description(), weights, (_own_deal(weights[0]), _own_deal(weights[1]))) diff --git a/tests/AGENTS.md b/tests/AGENTS.md index 84fe5be7..fd5c973e 100644 --- a/tests/AGENTS.md +++ b/tests/AGENTS.md @@ -25,7 +25,8 @@ loop. them. Loosening one to make a test pass is a finding to report, not a fix. - **Slow and heavy work is gated by an environment variable named in the skip reason**: `MINIDRAWMAHA_FULL_TABLE`, `MINIDRAWMAHA_FULL_GRADE`, - `MINIDRAWMAHA_FULL_CALIBRATION`, `MINIDRAWMAHA_FULL_DEEP`. Tests that need + `MINIDRAWMAHA_FULL_CALIBRATION`, `MINIDRAWMAHA_FULL_DEEP`, + `MINIDRAWMAHA_FULL_TARGET`. Tests that need torch `importorskip` it. The default suite must keep passing without torch, Modal, OpenSpiel or a network download. - **Committed numbers are recomputed, not transcribed.** `minidrawmaha/test_play.py` diff --git a/tests/minidrawmaha/test_boost_target.py b/tests/minidrawmaha/test_boost_target.py new file mode 100644 index 00000000..9bfabe26 --- /dev/null +++ b/tests/minidrawmaha/test_boost_target.py @@ -0,0 +1,943 @@ +"""The boost target: extra samples of a row's regret, each unbiased for the exact sum the tests first pin down. + +The module under test is rung 4's boost target. Exactly (`EXACT`), for a +table row it sums, over every physical entry of the key and every opponent +hand, the regret the walk would bank there, weighted by how often the walk +arrives; sampled, it estimates that sum from play-outs that keep k of n of +each source below the row. The claims, in +order: it imports the rules and never the grader; `sigma` is the ledger's +own σ row by row; the back-solved opponent reach equals the grader's carried +vector on every board of every public point, in the full game and in a +restricted-deal one (test a); at a seat's first decision the arrival IS the +grader's frequency, and a seat's first decisions sum to one; on a small +restricted game the regret and the arrival equal a world-by-world +expectation walked through `MiniState` (test b, exact form); on a wider +restricted game they match the mean and the visit count of thousands of +`Collector` walks, row by row within four standard errors and pooled (tests +b and c); rows of the wrong seat are refused and an unreachable row is zero; +and the cost per round-2 row is recorded (test d, a benchmark). Then the +sampled forms (e): counts that are not positive ints are refused by name; a +count at or above its fan is enumeration, bit for bit; each chance source +kept one branch at a time and averaged over its branches is the exact sum +to 1e-10, in the full game and a restricted one, which pins every n/k scale +with no noise; each source alone, and all of them together, is unbiased +against the exact sum at a row where it is live, and actually moves the +target there; the range draw is unbiased where a key's two physical entries +face wide and different ranges, before and after a throw; a row's sample depends on +(seed, t, seat, row) only; play-outs shrink the spread as 1/M; an +unreachable row is zero under any sampling; and sampling a draw or round-1 +row costs a fraction of its exact sum. + +Helpers are restated rather than imported from other test modules, as the +suite's rule has it. +""" + +from __future__ import annotations + +import ast +import os +import time +from collections.abc import Mapping +from pathlib import Path + +import numpy as np +import pytest + +from drawmaha_solver.deepcfr.collect import Collector +from drawmaha_solver.minidrawmaha import exploitability as grader +from drawmaha_solver.minidrawmaha.rules.cards import DECK, SUIT_RELABELLINGS, parse_cards, relabel +from drawmaha_solver.minidrawmaha.rules.enumeration import ( + PublicPoint, + private_keys, + public_decision_points, +) +from drawmaha_solver.minidrawmaha.rules.game import ( + Action, + InfoSet, + MiniState, + NodeKind, + canonical_picture, + random_deal, +) +from drawmaha_solver.minidrawmaha.solver.boost_target import ( + HOLES, + N_STATES, + SLOTS, + EXACT, + BoostTargets, + Sampling, + board_row, + state_index, +) +from drawmaha_solver.minidrawmaha.solver.packed_table import PackedTable + +REPO = Path(__file__).parents[2] +MODULE = REPO / "src" / "drawmaha_solver" / "minidrawmaha" / "solver" / "boost_target.py" +FULL_TARGET = os.environ.get("MINIDRAWMAHA_FULL_TARGET") == "1" + +# --------------------------------------------------------------------------- +# Helpers (restated) +# --------------------------------------------------------------------------- + +def key_on(point: PublicPoint, private) -> InfoSet: + hole, thrown, board = private + return InfoSet( + player=point.player, hole=hole, discarded=thrown, board=board, + draws=point.draws, betting=point.betting, + ) + +def shape_of(point: PublicPoint): + return private_keys(board_cards=point.board_cards, discards=point.discards) + +def random_table(seed: int) -> PackedTable: + """The whole game with N(0, 1) regrets: every row plays a different, mostly sparse σ.""" + table = PackedTable.whole_game() + table.cumulative_regret[:] = np.random.default_rng(seed).standard_normal( + table.cumulative_regret.size + ) + return table + +def positive_table(seed: int) -> PackedTable: + """The whole game with regrets in (0.05, 1.05): every action has positive probability, so every line is reachable.""" + table = PackedTable.whole_game() + table.cumulative_regret[:] = 0.05 + np.random.default_rng(seed).random(table.cumulative_regret.size) + return table + +def point_index() -> dict[tuple, int]: + """Public signature → position among the 141 points, the table's order.""" + return { + (point.board_cards, point.draws, point.betting): index + for index, point in enumerate(public_decision_points()) + } + +def grader_node_to_point(game: grader.VectorGame) -> dict[int, int]: + """The grader's own decision-node indices → table point indices, by public signature.""" + by_signature = point_index() + return { + node.index: by_signature[(node.level, node.public[0], node.public[1])] + for node in game.decisions + } + +def grader_sigma(game: grader.VectorGame, sigma) -> dict[int, np.ndarray]: + """`sigma` re-keyed by the grader's node index, as `_snapshot` would hand it to the walk.""" + to_point = grader_node_to_point(game) + return {node.index: sigma[to_point[node.index]] for node in game.decisions} + +def grader_reach(game: grader.VectorGame, sigma, *, carried: int) -> dict[int, np.ndarray]: + """The grader's forward pass, stopping at nothing: seat `carried`'s reach at every node, all boards. + + `_walk`'s chance and opponent branches verbatim (the opponent here is + `carried`); the other seat's nodes pass the reach straight down, since its + actions never enter a counterfactual reach. + """ + tables = grader_sigma(game, sigma) + out: dict[int, np.ndarray] = {} + + def forward(node, reach): + out[node.index] = reach + if node.kind is NodeKind.TERMINAL: + return + if node.kind is NodeKind.CHANCE: + child = node.child + if node.deals_board: + level = game.levels[child.level] + forward(child, node.prob * reach[level.parent] * level.ok) + else: + forward(child, node.prob * reach * game.levels[node.level].ok) + return + if node.player != carried: + for edge in node.edges: + forward(edge.child, reach) + return + keys = game.keys[(node.level, node.discards[node.player])] + valid = keys.index >= 0 + safe = np.where(valid, keys.index, 0) + table = tables[node.index] + for edge in node.edges: + pushed = np.zeros_like(reach) + for column, draw in zip(edge.actions, edge.draws, strict=True): + played = reach * np.where(valid, table[safe, column], 0.0) + if draw is None: + pushed += played + else: + grader._push(pushed, played, draw) + forward(edge.child, pushed) + + forward(game.root, game.weight[carried][None, :]) + return out + +def grader_board_row(level: int, b: int) -> int: + """My board index → the grader's batch row for the same cards (a guard against a convention drift).""" + if level == 1: + return grader.board_row((DECK[b],)) + first = b // (len(DECK) - 1) + rest = b % (len(DECK) - 1) + second = rest if rest < first else rest + 1 + return grader.board_row((DECK[first], DECK[second])) + +class RowProfile(Mapping): + """A profile over the whole game read from flat σ, float64, for the grader's functions.""" + + def __init__(self, table: PackedTable, flat: np.ndarray): + self._table, self._flat = table, flat + + def __getitem__(self, key): + _, start, width = self._table.slot_of(key) + return self._flat[start : start + width] + + def __iter__(self): + return iter(self._table) + + def __len__(self): + return len(self._table) + +# -- a restricted deal the walks can sample ------------------------------------ + +_ALLOWED: list[tuple[tuple, tuple]] = [] + +def restricted_deal(rng: np.random.Generator) -> MiniState: + """Uniform over `_ALLOWED` (seat-0 hole, seat-1 hole) pairs; module-level so the collector may pickle it.""" + return MiniState(holes=_ALLOWED[int(rng.integers(len(_ALLOWED)))]) + +def allowed_pairs(seat0, seat1) -> list[tuple[tuple, tuple]]: + return [ + (tuple(sorted(a)), tuple(sorted(b))) + for a in seat0 + for b in seat1 + if not set(a) & set(b) + ] + +# -- the world-by-world expectation ------------------------------------------ + +def world_by_world(table: PackedTable, key: InfoSet, pairs) -> tuple[np.ndarray, float]: + """E[sampled target | the walk reaches `key`] and P(reach), by enumerating every world. + + Walks `MiniState` from every allowed deal (each at 1/len(pairs)), following + every chance outcome at its probability and every action of the other + seat at its σ, and every action of the key's seat unweighted — the walk + enumerates its own actions. At the key, the exact value of each action + (`ev`: chance averaged, both seats at σ) stands in for the walk's sampled + continuation, whose expectation it is. Nothing here shares code with the + module under test: it reads σ through the ledger and the rules through + the state machine. + """ + seat = key.player + width = len(key.legal_actions()) + numerator = np.zeros(width) + denominator = 0.0 + + def ev(state: MiniState) -> float: + if state.kind is NodeKind.TERMINAL: + return state.returns()[seat] + if state.kind is NodeKind.CHANCE: + return sum(q * ev(state.apply_chance(cards)) for cards, q in state.chance_outcomes()) + sigma = table[state.infoset()].strategy() + return sum( + s * ev(state.apply(action)) + for action, s in zip(state.legal_actions(), sigma, strict=True) + if s > 0.0 + ) + + def consistent(state: MiniState) -> bool: + """Could this history still reach `key`? Its public record must be a prefix of the key's, and the seat's own cards must fit.""" + if len(state.board) > len(key.board) or len(state.draws) > len(key.draws): + return False + if state.draws != key.draws[: len(state.draws)]: + return False + for round_index, line in enumerate(state.betting): + if round_index >= len(key.betting) or key.betting[round_index][: len(line)] != line: + return False + if round_index < len(state.betting) - 1 and line != key.betting[round_index]: + return False + own = set(state.holes[seat]) | set(state.discards[seat]) + # The seat's cards so far, in the key's suit frame, must be a subset of the key's picture. + return any( + own <= set(relabel(frame, key.hole)) | set(relabel(frame, key.discarded)) + and tuple(relabel(frame, key.board))[: len(state.board)] == state.board + for frame in SUIT_RELABELLINGS + ) + + def arrive(state: MiniState, p: float) -> None: + nonlocal denominator + if state.kind is NodeKind.TERMINAL or not consistent(state): + return + if state.kind is NodeKind.CHANCE: + for cards, q in state.chance_outcomes(): + arrive(state.apply_chance(cards), p * q) + return + actions = state.legal_actions() + if state.current_player == seat: + if state.infoset() == key: + utilities = np.array([ev(state.apply(action)) for action in actions]) + sigma = table[key].strategy() + numerator[:] += p * (utilities - sigma @ utilities) + denominator += p + return # perfect recall: the key is not below itself + for action in actions: + arrive(state.apply(action), p) + return + sigma = table[state.infoset()].strategy() + for action, s in zip(actions, sigma, strict=True): + if s > 0.0: + arrive(state.apply(action), p * s) + + for holes in pairs: + arrive(MiniState(holes=holes), 1.0 / len(pairs)) + return (numerator / denominator if denominator > 0 else numerator), denominator + +def rows_at(table: PackedTable, keys) -> np.ndarray: + return np.asarray([table.row_of(key) for key in keys], dtype=np.int64) + +def assert_pooled_fair(z: np.ndarray) -> None: + """The z of many rows, pooled: mean within 4/√n of 0, mean of z² within 4√(2/n) of 1. + + The mean of z is the sharp bias test: at n rows its standard deviation + is 1/√n, so a bias of a tenth of a standard error in every row reaches + the band at n ≈ 1,600 and is caught reliably by about 3,000 rows. The + widths take the z's as independent and normal, which they are not + quite: rows on one path share walks (a positive correlation, so the + regret band is a little narrower than four true standard deviations), + visit counts over rows come from one multinomial (negative, so that + band is conservative), and a z built from a plug-in standard error over + 40-odd samples has heavier tails than normal. The seeds are fixed, so + the runs are deterministic. The arrival bands were checked at the + gated test's seed 12 for both seats and at seeds 13, 14, 15, 16, 17, + 18 and 19 for seat 1 (2026-11-05, mean z within 0.01 of zero on every + fair selection); they are a calibrated regression gate, not a + distributional guarantee. At small n the z² band is wide: 1 ± 1.79 at + n = 10; its low side bites once the half-width drops under 1, from + n = 33 (1 ± 0.98), and is 1 ± 0.84 at n = 45. + """ + n = z.size + assert abs(np.mean(z)) < 4 / np.sqrt(n), (n, np.mean(z)) + assert abs(np.mean(z**2) - 1.0) < 4 * np.sqrt(2 / n), (n, np.mean(z**2)) + +# --------------------------------------------------------------------------- +# Layering +# --------------------------------------------------------------------------- + +def test_the_module_imports_the_rules_and_never_the_grader_or_the_net(): + # The boost feeds the trainer; a bug shared with the grader would cancel + # and certify itself. The check is static, on the import block, so a + # review of the module starts where the root instructions say it should. + tree = ast.parse(MODULE.read_text()) + imported = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + imported |= {alias.name for alias in node.names} + elif isinstance(node, ast.ImportFrom): + imported.add(node.module or "") + assert not any("exploitability" in name for name in imported), imported + assert not any("deepcfr" in name for name in imported), imported + assert not any(name.startswith("torch") for name in imported), imported + assert all( + name.startswith(("drawmaha_solver.minidrawmaha.rules", "drawmaha_solver.minidrawmaha.solver.packed_table")) + for name in imported + if name.startswith("drawmaha_solver") + ), imported + +# --------------------------------------------------------------------------- +# The description +# --------------------------------------------------------------------------- + +def test_build_is_one_object_per_deal_set_over_a_shared_description(): + full = BoostTargets.build() + assert BoostTargets.build() is full # the default is cached: the description takes seconds to build + assert len(full.points) == 141 + # The reach memo's bound, as the module states it: every node with one + # board card out on the path from a decision to the root, and every + # round-2 opening decision (one per round-1 line and pair of draw counts). + d = full.description + memoised = {} + for node in d.decision_of_point: + while node is not d.root: + above, _ = d.parent[node.index] + if node.level <= 1 or (above.kind is NodeKind.CHANCE and above.deals_board): + memoised[node.index] = node + node = above + level_one = [node for node in memoised.values() if node.level <= 1] + opening = [node for node in memoised.values() if node.level == 2] + assert len(level_one) == 66 and sum(node.kind is NodeKind.DECISION for node in level_one) == 29 + assert len(opening) == 20 and all(node.kind is NodeKind.DECISION for node in opening) + assert len(level_one) * 15 + len(opening) * 210 == 5_190 + # Every physical state of the frame is one of (hole, nothing thrown | a card not in the hole). + assert N_STATES == len(HOLES) * SLOTS == 455 * 16 + assert state_index(parse_cards("2c 2d 2h"), ()) == 0 # the deck is ordered by rank, then suit + assert state_index(parse_cards("2c 2d 2h"), parse_cards("3c")) == 1 + DECK.index(parse_cards("3c")[0]) + assert board_row(parse_cards("2c")) == 0 and board_row(parse_cards("2d 2c")) == 14 + # Every key of every shape has its physical entries, and the entry counts + # are the frame's: boards × states the board allows. + counts = {(1, 0): 15 * 364, (2, 0): 210 * 286, (2, 1): 210 * 286 * 10} + for shape, expected in counts.items(): + n_keys = len(private_keys(board_cards=shape[0], discards=shape[1])) + total = 0 + sizes = set() + for position in range(n_keys): + boards, states = full.entries_of(shape, position) + assert boards.shape == states.shape and boards.size >= 1 + total += boards.size + sizes.add(boards.size) + assert total == expected + # A key is a suit orbit of one picture: 6 relabellings over its stabiliser. + assert sizes <= {1, 2, 3, 6} + +def test_sigma_is_the_ledgers_strategy_row_by_row(): + full = BoostTargets.build() + table = random_table(1) + sigma = full.sigma(table) + assert len(sigma) == 141 + for p, point in enumerate(public_decision_points()): + shape = shape_of(point) + assert sigma[p].shape == (len(shape), point.width) and sigma[p].dtype == np.float64 + for position in (0, len(shape) // 3, len(shape) - 1): + key = key_on(point, shape[position]) + np.testing.assert_allclose(sigma[p][position], table[key].strategy(), rtol=1e-15, atol=0) + with pytest.raises(ValueError, match="whole game"): + full.sigma(PackedTable.listed([key_on(public_decision_points()[0], shape_of(public_decision_points()[0])[0])])) + +# --------------------------------------------------------------------------- +# (a) the back-solved reach is the grader's carried reach +# --------------------------------------------------------------------------- + +RESTRICTED = ( + [parse_cards("2c 3c 5d"), parse_cards("2d 3d 5c"), parse_cards("4h 5h 6h")], + [parse_cards("2d 4h 6h"), parse_cards("3h 4c 4d"), parse_cards("2c 2d 6c"), parse_cards("3c 5h 6d")], +) + +@pytest.mark.parametrize("holes", [None, RESTRICTED], ids=["full", "restricted"]) +@pytest.mark.parametrize("carried", [0, 1]) +def test_reach_equals_the_graders_carried_reach_on_every_board(holes, carried): + exact = BoostTargets.build(holes=holes) + game = grader.minidrawmaha_game(holes=holes) + table = random_table(2) + sigma = exact.sigma(table) + reference = grader_reach(game, sigma, carried=carried) + to_point = grader_node_to_point(game) + checked = 0 + for node in game.decisions: + p = to_point[node.index] + for b in range(reference[node.index].shape[0]): + mine = exact.reach(sigma, carried=carried, point=p, board=b) + theirs = reference[node.index][grader_board_row(node.level, b)] + np.testing.assert_allclose(mine, theirs, rtol=1e-10, atol=1e-300) + checked += bool(theirs.any()) + assert checked > 1000 # most (point, board) pairs carry reach under a random table + +# --------------------------------------------------------------------------- +# arrival at a first decision is the grader's frequency +# --------------------------------------------------------------------------- + +@pytest.mark.parametrize("seat", [0, 1]) +def test_arrival_at_a_seats_first_decision_is_the_graders_frequency_and_sums_to_one(seat): + # Before a seat has acted, own reach is the deal alone, so the grader's + # `frequency` (own × compat) at its opening points is exactly the walk's + # arrival; and every walk reaches one of them. + exact = BoostTargets.build() + game = grader.minidrawmaha_game() + table = random_table(3) + sigma = exact.sigma(table) + reach = grader_reach(game, sigma, carried=1 - seat) + to_point = grader_node_to_point(game) + opening = [ + node for node in game.decisions + if node.player == seat and node.level == 1 and len(node.public[1][0]) == seat + ] + assert len(opening) == 1 + seat + total = 0.0 + for node in opening: + p = to_point[node.index] + keys = game.keys[(1, 0)] + valid = keys.index >= 0 + _, compat = game.showdown.sums(1, reach[node.index], discards=0, shares=False) + frequency = game.weight[seat][None, :] * compat + per_key = np.zeros(keys.n_keys) + np.add.at(per_key, keys.index[valid], frequency[valid]) + rows = np.arange(len(per_key)) + table.row_of(key_on(public_decision_points()[p], shape_of(public_decision_points()[p])[0])) + arrival = exact.arrivals(sigma, seat=seat, rows=rows) + assert arrival.shape == (len(rows),) + np.testing.assert_allclose(arrival, per_key, rtol=1e-10, atol=1e-300) + total += arrival.sum() + # The full computation agrees with the arrival-only path (a round-1 + # row's value walks the whole game below it, so a handful of rows). + some = rows[:: max(1, len(rows) // 4)] + regret, with_values = exact.targets(sigma, seat=seat, rows=some, sampling=EXACT) + assert regret.shape == (len(some), 4) and np.isfinite(regret).all() + np.testing.assert_array_equal(with_values, arrival[:: max(1, len(rows) // 4)]) + assert total == pytest.approx(1.0, abs=1e-12) + +# --------------------------------------------------------------------------- +# (b) exact form: a world-by-world expectation on a small restricted game +# --------------------------------------------------------------------------- + +SMALL = ( + # Two suit relabellings of one hole for seat 0: one key, two physical entries. + [parse_cards("2c 3c 5d"), parse_cards("2d 3d 5c")], + # An opponent set that is NOT closed under that relabelling, so the two + # entries face different ranges and the sum over entries is load-bearing. + [parse_cards("2d 4h 6h"), parse_cards("3h 4c 4d"), parse_cards("2c 2h 6c")], +) + +def small_game_keys(seat: int) -> list[InfoSet]: + """One round-2 stood-pat row, one post-draw row, and (seat 1) one draw row, all reachable from SMALL.""" + points = public_decision_points() + picks = [] + hero = SMALL[seat][0] + if seat == 1: + # Seat 1's draw after seat 0 stood pat: the value below it expands over + # 37 hero continuations and 7 boards, which the exact walk can afford. + draw = next(p for p in points if p.player == 1 and p.is_draw_decision and p.draws[0].count == 0) + picks.append((draw, (hero, (), parse_cards("4c")))) + pat = next(p for p in points if p.player == seat and p.board_cards == 2 and p.discards == 0 and len(p.betting[1]) == seat) + picks.append((pat, (hero, (), parse_cards("4c 6d")))) + threw = next(p for p in points if p.player == seat and p.board_cards == 2 and p.discards == 1 and len(p.betting[1]) == seat) + kept = hero[:2] + (parse_cards("6d")[0],) + picks.append((threw, (kept, hero[2:], parse_cards("4c 3h")))) + return [key_on(point, canonical_key(private)) for point, private in picks] + +def canonical_key(private): + hole, thrown, board = private + return canonical_picture(tuple(sorted(hole)), tuple(sorted(thrown)), board) + +@pytest.mark.parametrize("seat", [0, 1]) +def test_regret_and_arrival_equal_a_world_by_world_expectation(seat): + pairs = allowed_pairs(*SMALL) + exact = BoostTargets.build(holes=SMALL) + table = positive_table(4) + sigma = exact.sigma(table) + keys = small_game_keys(seat) + rows = rows_at(table, keys) + regret, arrival = exact.targets(sigma, seat=seat, rows=rows, sampling=EXACT) + for key, row_regret, row_arrival in zip(keys, regret, arrival, strict=True): + expected, reach = world_by_world(table, key, pairs) + assert reach > 0, str(key) + width = len(key.legal_actions()) + assert row_arrival == pytest.approx(reach, rel=1e-9, abs=0), str(key) + np.testing.assert_allclose(row_regret[:width], expected, rtol=1e-9, atol=1e-12, err_msg=str(key)) + assert not row_regret[width:].any() + # The seat-0 keys stand for two physical entries whose weights differ, + # so a representative entry would have failed above. + if seat == 0: + for key in keys: + own = [ + world_by_world(table, key, [pair for pair in pairs if pair[0] == hole])[1] + for hole in SMALL[0] + ] + assert own[0] != pytest.approx(own[1], rel=1e-3) + +# --------------------------------------------------------------------------- +# (b) and (c) sampled: thousands of walks against the exact numbers +# --------------------------------------------------------------------------- + +WIDE = ([parse_cards("2c 3c 5d")], list(HOLES)) # one hero hole for seat 0, any opponent + +def test_regret_and_arrival_match_the_sampled_walks_row_by_row_and_pooled(): + # Seat 0 holds one hole, so its rows are reached often enough to test: + # 6,000 walks reach the opening row every time and the draw rows a few + # hundred times each. Per row the sample mean is within 4 s.e. of the + # exact regret and the visit count within 4 s.e. of the exact arrival; + # pooled, the mean of z² over rows sits near 1 — a bias of half a + # standard error in every row would read 1.25 and a convention bug + # (swapped reach, a dropped entry, the wrong chance factor) far above. + walks = 6_000 + _ALLOWED[:] = allowed_pairs(*WIDE) + exact = BoostTargets.build(holes=WIDE) + table = random_table(5) + sigma = exact.sigma(table) + with Collector(seed=11, workers=0, deal=restricted_deal) as collector: + collector.table.cumulative_regret[:] = table.cumulative_regret + offered = collector.collect(seat=0, t=1, walks=walks) + rows, counts = np.unique(offered.advantage_rows, return_counts=True) + enough = counts >= 40 + rows, counts = rows[enough], counts[enough] + regret, arrival = exact.targets(sigma, seat=0, rows=rows, sampling=EXACT) + # Rows are kept by their EXPECTED count, 80 or more, so the observed + # cutoff of 40 sits 4.5 standard errors below every kept row's mean and + # the selection cannot bias z: a row chosen by its observed count alone + # is there because it fluctuated high. + fair = walks * arrival >= 80 + rows, counts, regret, arrival = rows[fair], counts[fair], regret[fair], arrival[fair] + assert rows.size >= 8 + widths = table.widths()[rows] + z_regret, z_visits = [], [] + for row, count, width, exact_regret, exact_arrival in zip(rows, counts, widths, regret, arrival, strict=True): + samples = offered.advantage_targets[offered.advantage_rows == row][:, :width].astype(np.float64) + mean = samples.mean(axis=0) + se = samples.std(axis=0, ddof=1) / np.sqrt(count) + live = se > 0 + z = (mean - exact_regret[:width])[live] / se[live] + assert (np.abs(z) < 4).all(), (row, mean, exact_regret[:width], se) + z_regret += z.tolist() + # Visits: binomial over walks (a row reached twice in one walk is rarer than one in a thousand). + se_visits = np.sqrt(walks * exact_arrival * (1 - exact_arrival)) + zv = (count - walks * exact_arrival) / se_visits + assert abs(zv) < 4, (row, count, walks * exact_arrival) + z_visits.append(zv) + for z in (np.asarray(z_regret), np.asarray(z_visits)): + assert z.size >= 8 + assert_pooled_fair(z) + +@pytest.mark.skipif( + not FULL_TARGET, reason="200,000 whole-game walks on 8 workers, ~2 min; set MINIDRAWMAHA_FULL_TARGET=1" +) +@pytest.mark.parametrize("seat", [0, 1]) +def test_arrival_is_the_visit_rate_of_two_hundred_thousand_whole_game_walks(seat): + walks = 200_000 + exact = BoostTargets.build() + table = random_table(6) + sigma = exact.sigma(table) + with Collector(seed=12, workers=8, deal=random_deal) as collector: + collector.table.cumulative_regret[:] = table.cumulative_regret + offered = collector.collect(seat=seat, t=1, walks=walks) + rows, counts = np.unique(offered.advantage_rows, return_counts=True) + enough = counts >= 40 + rows, counts = rows[enough], counts[enough] + arrival = exact.arrivals(sigma, seat=seat, rows=rows) + # Kept by EXPECTED count, 80 or more: the observed cutoff of 40 is then + # 4.5 standard errors below every kept row's mean, so the selection + # cannot bias z. (Selecting on the observed count alone put seat 1's + # draw rows, whose means sit near 80, at a mean z of +1.8 on two seeds: + # the rows that cleared the cutoff were the ones that fluctuated high. + # Seat 1's commonest rows, its two opening points, expect about 110.) + fair = walks * arrival >= 80 + rows, counts, arrival = rows[fair], counts[fair], arrival[fair] + assert rows.size >= 50 + z = (counts - walks * arrival) / np.sqrt(walks * arrival * (1 - arrival)) + # The sharp tests are the pooled ones (`assert_pooled_fair`). The + # per-row cap is loose on purpose: of some 3,000 rows the largest |z| + # passes 5 in a run out of a thousand, and the deal alone can do it — at + # seed 12 one seat-1 picture (3c 3d 4h with 2c on the board, six + # relabellings) was dealt 292 times against an expected 220, 4.9 + # standard errors, in a count that involves no σ and nothing of this + # module, and 221, 219, 222 times on three other seeds. + assert (np.abs(z) < 6).all(), (np.abs(z).max(), rows[np.argmax(np.abs(z))]) + assert_pooled_fair(z) + +# --------------------------------------------------------------------------- +# Refusals and the unreachable row +# --------------------------------------------------------------------------- + +def test_rows_of_the_other_seat_and_outside_the_table_are_refused(): + exact = BoostTargets.build() + table = random_table(7) + sigma = exact.sigma(table) + points = public_decision_points() + seat1 = next(i for i, point in enumerate(points) if point.player == 1) + row_of_seat_1 = table.row_of(key_on(points[seat1], shape_of(points[seat1])[0])) + with pytest.raises(ValueError, match="seat"): + exact.targets(sigma, seat=0, rows=np.array([row_of_seat_1]), sampling=EXACT) + with pytest.raises(ValueError, match="seat"): + exact.targets(sigma, seat=2, rows=np.array([0]), sampling=EXACT) + for bad in (-1, len(table)): + with pytest.raises(IndexError): + exact.targets(sigma, seat=0, rows=np.array([bad]), sampling=EXACT) + +def test_an_unreachable_row_has_zero_arrival_and_zero_regret(): + # Seat 1 never bets or raises: every seat-0 round-1 row facing a bet is + # unreachable, and the exact target says so with zeros rather than NaN. + exact = BoostTargets.build() + table = random_table(8) + points = public_decision_points() + for p, point in enumerate(points): + if point.player == 1 and not point.is_draw_decision and Action.POT in point.actions: + column = point.actions.index(Action.POT) + for position in range(len(shape_of(point))): + _, start, width = table.slot_of(key_on(point, shape_of(point)[position])) + table.cumulative_regret[start + column] = -1.0 + if not (table.cumulative_regret[start : start + width] > 0).any(): + table.cumulative_regret[start + point.actions.index(Action.CHECK_CALL)] = 1.0 + sigma = exact.sigma(table) + facing = next( + i for i, point in enumerate(points) + if point.player == 0 and point.board_cards == 1 and not point.is_draw_decision + and point.betting[0][-1:] == (Action.POT,) + ) + rows = np.arange(len(shape_of(points[facing]))) + table.row_of(key_on(points[facing], shape_of(points[facing])[0])) + regret, arrival = exact.targets(sigma, seat=0, rows=rows, sampling=EXACT) + assert not arrival.any() and not regret.any() + assert np.isfinite(regret).all() + +# --------------------------------------------------------------------------- +# (e) sampling below the row: unbiased against the exact sum, seeded per row +# --------------------------------------------------------------------------- + +def only(**sampled) -> Sampling: + """A Sampling that enumerates every source but the ones named.""" + return Sampling(**{ + "opponent_range": None, "board_cards": None, "opponent_draws": None, + "hero_draws": None, "opponent_actions": None, **sampled, + }) + +def row_on(table: PackedTable, point: int, position: int = 7) -> np.ndarray: + points = public_decision_points() + return np.array([table.row_of(key_on(points[point], shape_of(points[point])[position]))]) + +def sampled(exact, sigma, seat, rows, sampling, seeds) -> np.ndarray: + """One sampled target per seed, `(len(seeds), 4)`, t equal to the seed.""" + return np.stack([ + exact.targets(sigma, seat=seat, rows=rows, sampling=sampling, seed=0, t=s)[0][0] for s in seeds + ]) + +@pytest.mark.parametrize( + "bad", + [dict(board_cards=0), dict(hero_draws=-1), dict(opponent_actions=True), dict(opponent_range=1.5), dict(playouts=0)], + ids=lambda d: next(iter(d)), +) +def test_sampling_refuses_counts_that_are_not_positive_ints(bad): + field = next(iter(bad)) + with pytest.raises(ValueError, match=f"Sampling.{field}"): + Sampling(**bad) + +def test_sampling_counts_take_any_integer_type(): + assert Sampling(board_cards=np.int64(2), playouts=np.int32(3)).board_cards == 2 + +def test_exact_with_more_than_one_playout_is_refused(): + with pytest.raises(ValueError, match="playouts must be 1"): + only(playouts=2) + +def test_a_sampled_call_names_its_seed_and_t(): + exact = BoostTargets.build() + table = positive_table(3) + sigma = exact.sigma(table) + rows = row_on(table, 4) + with pytest.raises(ValueError, match="seed"): + exact.targets(sigma, seat=0, rows=rows, sampling=Sampling(), t=0) + with pytest.raises(ValueError, match="t,|non-negative int t"): + exact.targets(sigma, seat=0, rows=rows, sampling=Sampling(), seed=0) + with pytest.raises(TypeError, match="Sampling"): + exact.targets(sigma, seat=0, rows=rows, sampling=None) + +@pytest.mark.parametrize("point", [2, 0], ids=["draw row", "round-1 row"]) +def test_a_count_at_or_above_its_fan_is_enumeration_bit_for_bit(point): + exact = BoostTargets.build() + table = positive_table(3) + sigma = exact.sigma(table) + rows = row_on(table, point) + full = only(board_cards=14, opponent_draws=12, hero_draws=99) + # The fans `is_exact` reads are the description's own. + assert exact.description.levels[2].children_of.shape[1] == 14 + assert all( + draw.targets.shape[-1] == 12 + for node_edges in [n.edges for n in exact.description.decision_of_point if n.edges] + for edge in node_edges for draw in edge.draws if draw is not None + ) + assert full.is_exact + with pytest.raises(ValueError, match="playouts must be 1"): + only(board_cards=14, playouts=2) + want = exact.targets(sigma, seat=0, rows=rows, sampling=EXACT) + got = exact.targets(sigma, seat=0, rows=rows, sampling=full, seed=0, t=0) + assert np.array_equal(want[0], got[0]) and np.array_equal(want[1], got[1]) + +# Each source alone at a row where it is live, then all of them at once. The +# draw-row cases walk the whole draw and round 2 below them, about 0.2 s a +# call when nothing prunes, so they take fewer seeds. At these seeds the +# smallest relative bias the 4.5-s.e. gate resolves (4.5 · s.e. / |mean|, +# measured 2026-11-06 at these rows) is about 3% for the opponent's actions, +# 8% for the range and 2–8% for the opponent's replacements; for the hero's +# replacements, the board card and the mixed forms it is 13% to over 100%, +# because those sources move the target by more than its size. Their scale +# factors are pinned exactly instead, by the branch oracle below; these +# cases check that the forms compose and that each one actually samples. +UNBIASED = [ + ("opponent_range", only(opponent_range=1), 4, 0, 400), + ("opponent_range after a throw", only(opponent_range=1), 29, 0, 400), + ("opponent_actions", only(opponent_actions=1), 4, 0, 400), + ("board_cards", only(board_cards=1), 2, 0, 200), + ("hero_draws, seat 0", only(hero_draws=1), 2, 0, 60), + ("hero_draws, seat 1", only(hero_draws=1), 3, 1, 60), + ("opponent_draws", only(opponent_draws=1), 2, 0, 60), + ("the walk's sampling, whole range", Sampling(), 2, 0, 200), + ("everything at one", Sampling(opponent_range=1), 2, 0, 200), + ("two of each, two play-outs", Sampling(playouts=2, opponent_range=2, board_cards=2, opponent_draws=2, + hero_draws=2, opponent_actions=2), 2, 0, 100), +] + +@pytest.mark.parametrize(("sampling", "point", "seat", "n"), [c[1:] for c in UNBIASED], ids=[c[0] for c in UNBIASED]) +def test_each_sampled_source_is_unbiased_and_actually_samples(sampling, point, seat, n): + exact = BoostTargets.build() + table = positive_table(3) + sigma = exact.sigma(table) + rows = row_on(table, point) + want, arrival = exact.targets(sigma, seat=seat, rows=rows, sampling=EXACT) + assert arrival[0] > 0.0 + draws = sampled(exact, sigma, seat, rows, sampling, range(n)) + spread = draws.std(axis=0, ddof=1) + live = spread > 1e-9 # above float rounding, which a z would blow up + # Not vacuous: the source really varied the target at this row. + assert live.any(), "the sampled source never moved the target here" + z = (draws.mean(axis=0)[live] - want[0][live]) / (spread[live] / np.sqrt(n)) + assert (np.abs(z) < 4.5).all(), z + # A column the sample never moves is the exact one, bit for bit or near. + np.testing.assert_allclose(draws[:, ~live], np.broadcast_to(want[0][~live], draws[:, ~live].shape), atol=1e-9) + _, sampled_arrival = exact.targets(sigma, seat=seat, rows=rows, sampling=sampling, seed=0, t=0) + assert sampled_arrival[0] == arrival[0] == exact.arrivals(sigma, seat=seat, rows=rows)[0] + +def small_draw_row(table: PackedTable, seat: int) -> np.ndarray: + """A draw row reachable from SMALL: seat 0's (both throws below it, two physical entries) or seat 1's.""" + if seat == 1: + return rows_at(table, small_game_keys(1)[:1]) + point = public_decision_points()[2] + return rows_at(table, [key_on(point, canonical_key((SMALL[0][0], (), parse_cards("4c"))))]) + +# Each chance source crosses a path below a draw row once, and the target is +# linear in it, so keeping branch j at every position, scaled n, and +# averaging over j = 0 … n − 1 is the exact sum with no noise at all: a +# scale factor off by one branch in twelve fails at 1e-10 instead of hiding +# inside a standard error. +ORACLE = [ + ("board_cards", 14, None, 0), ("hero_draws", 12, None, 0), ("opponent_draws", 12, None, 0), + ("hero_draws", 12, None, 1), + ("board_cards", 14, SMALL, 0), ("hero_draws", 12, SMALL, 0), ("opponent_draws", 12, SMALL, 0), + ("board_cards", 14, SMALL, 1), +] + +@pytest.mark.parametrize(("source", "fan", "holes", "seat"), ORACLE, + ids=[f"{c[0]}, {'small' if c[2] else 'full'}, seat {c[3]}" for c in ORACLE]) +def test_a_chance_source_kept_one_branch_at_a_time_averages_to_the_exact_sum(source, fan, holes, seat, monkeypatch): + from drawmaha_solver.minidrawmaha.solver import boost_target + exact = BoostTargets.build(holes=holes) + table = positive_table(3) + sigma = exact.sigma(table) + if holes is None: + rows = row_on(table, 2 if seat == 0 else 3) + else: + rows = small_draw_row(table, seat) + want, arrival = exact.targets(sigma, seat=seat, rows=rows, sampling=EXACT) + assert arrival[0] > 0.0 + branch = [0] + monkeypatch.setattr( + boost_target._Draws, "subset", + lambda self, k, n, shape: None if k is None or k >= n else np.full((*shape, 1), branch[0]), + ) + total = np.zeros_like(want) + for j in range(fan): + branch[0] = j + total += exact.targets(sigma, seat=seat, rows=rows, sampling=only(**{source: 1}), seed=0, t=0)[0] + assert np.abs(want).max() > 0.0 + np.testing.assert_allclose(total / fan, want, rtol=1e-10, atol=1e-13) + +# SMALL's seat-0 hole and its clubs ↔ diamonds relabelling against every +# hole holding more clubs than diamonds: a set the relabelling maps onto its +# complement, so the key's two physical entries face wide and different +# ranges (SMALL's three opponent holes leave one live hand at these rows, +# and a range draw there has nothing to vary). +LOPSIDED = ( + SMALL[0], + [hole for hole in HOLES if sum(c.suit == 0 for c in hole) > sum(c.suit == 1 for c in hole)], # suit 0 = c, 1 = d +) + +def test_the_range_draw_is_unbiased_where_two_entries_face_different_ranges(): + # At a stood-pat row and at one after a throw (where the thrown card + # enters the clash mask): a draw that mixed one entry's total or mask + # into the other's would shift the mean. + exact = BoostTargets.build(holes=LOPSIDED) + table = positive_table(4) + sigma = exact.sigma(table) + rows = rows_at(table, small_game_keys(0)) + want, arrival = exact.targets(sigma, seat=0, rows=rows, sampling=EXACT) + assert (arrival > 0.0).all() + n = 600 + draws = np.stack([ + exact.targets(sigma, seat=0, rows=rows, sampling=only(opponent_range=1), seed=0, t=s)[0] for s in range(n) + ]) # (n, rows, 4) + spread = draws.std(axis=0, ddof=1) + live = spread > 1e-9 + assert live[0].any() and live[1].any(), spread + z = (draws.mean(axis=0)[live] - want[live]) / (spread[live] / np.sqrt(n)) + assert (np.abs(z) < 4.5).all(), z + +def test_a_rows_sample_depends_on_seed_t_seat_and_row_only(): + exact = BoostTargets.build() + table = positive_table(3) + sigma = exact.sigma(table) + rows = np.concatenate([row_on(table, 4, 7), row_on(table, 2, 11), row_on(table, 4, 30)]) + at = lambda r, t=5: exact.targets(sigma, seat=0, rows=r, sampling=Sampling(opponent_range=1), seed=1, t=t)[0] + together = at(rows) + assert np.array_equal(at(rows[::-1]), together[::-1]) + assert np.array_equal(np.concatenate([at(rows[:1]), at(rows[1:])]), together) + assert np.array_equal(at(rows[[0, 0, 1, 2]]), together[[0, 0, 1, 2]]) + assert not np.array_equal(at(rows, t=6), together) + +def test_more_playouts_shrink_the_spread_as_one_over_m(): + exact = BoostTargets.build() + table = positive_table(3) + sigma = exact.sigma(table) + rows = row_on(table, 4) + one = sampled(exact, sigma, 0, rows, only(opponent_actions=1), range(200)).var(axis=0, ddof=1) + eight = sampled(exact, sigma, 0, rows, only(opponent_actions=1, playouts=8), range(200)).var(axis=0, ddof=1) + live = one > 0.0 + ratio = eight[live] / one[live] + assert ((1 / 16 < ratio) & (ratio < 1 / 4)).all(), ratio + +def test_an_unreachable_row_is_zero_under_any_sampling(): + # The same construction as the exact test above: seat 1 never bets, so a + # seat-0 round-1 row facing a bet has no weight to draw a range from. + exact = BoostTargets.build() + table = random_table(8) + points = public_decision_points() + for p, point in enumerate(points): + if point.player == 1 and not point.is_draw_decision and Action.POT in point.actions: + column = point.actions.index(Action.POT) + for position in range(len(shape_of(point))): + _, start, width = table.slot_of(key_on(point, shape_of(point)[position])) + table.cumulative_regret[start + column] = -1.0 + if not (table.cumulative_regret[start : start + width] > 0).any(): + table.cumulative_regret[start + point.actions.index(Action.CHECK_CALL)] = 1.0 + sigma = exact.sigma(table) + facing = next( + i for i, point in enumerate(points) + if point.player == 0 and point.board_cards == 1 and not point.is_draw_decision + and point.betting[0][-1:] == (Action.POT,) + ) + rows = np.arange(4) + table.row_of(key_on(points[facing], shape_of(points[facing])[0])) + regret, arrival = exact.targets(sigma, seat=0, rows=rows, sampling=Sampling(opponent_range=1), seed=0, t=0) + assert not arrival.any() and not regret.any() + assert np.isfinite(regret).all() + +# --------------------------------------------------------------------------- +# (d) cost per row: a benchmark, recorded, with a loose ceiling +# --------------------------------------------------------------------------- + +def test_a_round_two_row_costs_a_few_milliseconds(): + exact = BoostTargets.build() + table = random_table(9) + sigma = exact.sigma(table) + rng = np.random.default_rng(0) + points = public_decision_points() + offsets = np.cumsum([0] + [len(shape_of(point)) for point in points]) + round_two = [p for p, point in enumerate(points) if point.player == 0 and point.board_cards == 2] + rows = np.concatenate([ + offsets[p] + rng.choice(len(shape_of(points[p])), size=10, replace=False) for p in round_two + ]) + exact.targets(sigma, seat=0, rows=rows[:5], sampling=EXACT) # warm the description + started = time.perf_counter() + exact.targets(sigma, seat=0, rows=rows, sampling=EXACT) + per_row = (time.perf_counter() - started) / rows.size + print(f"\nexact target: {per_row * 1e3:.2f} ms per round-2 row over {rows.size} rows") + # The plan's figure is 1 ms a row and the measured one 0.8 ms. The + # ceiling is fifty times that: wide enough that a laptop busy with a + # training run (0.95 ms measured beside a 9-worker LCFR run) cannot + # cross it, narrow enough to catch a regression to the grader's + # whole-board update, which is about a second a row. + assert per_row < 0.050 + +def test_sampling_below_a_draw_or_round_one_row_costs_a_fraction_of_the_exact_sum(): + # The board card's fourteen branches and the hero's thirty-six + # replacements are what an exact draw or round-1 row pays for; the walk's + # sampling keeps one of each. Printed for the record; the assertion is a + # ratio, which a busy laptop slows on both sides alike. + exact = BoostTargets.build() + table = positive_table(3) + sigma = exact.sigma(table) + lines = [] + for point, name in ((4, "round 2"), (2, "draw"), (0, "round 1")): + rows = np.concatenate([row_on(table, point, position) for position in (3, 7, 11)]) + exact.targets(sigma, seat=0, rows=rows[:1], sampling=EXACT) # warm + cost = {} + for label, sampling in (("exact", EXACT), ("walk's sampling", Sampling())): + started = time.perf_counter() + exact.targets(sigma, seat=0, rows=rows, sampling=sampling, seed=0, t=0) + cost[label] = (time.perf_counter() - started) / rows.size + lines.append(f"{name:>8}: exact {cost['exact'] * 1e3:7.1f} ms, walk's sampling {cost["walk's sampling"] * 1e3:6.1f} ms a row") + if point != 4: + assert cost["walk's sampling"] < cost["exact"] / 2, (name, cost) + print("\nboost target, a positive table:\n" + "\n".join(lines))