Repository navigation
Conversation
…ampled walk — μ = 0 is today's walk bit for bit, and on Leduc it makes CFR+'s current strategy converge Claude-Session: https://claude.ai/code/session_014D1ZwBFFxJAkajfcQPe6bx
…ds and costs 5–11% before that — the current strategy already beats the average at every mark without it
The friction experiment script gains --save-table (the finished table's regrets,
strategy sums, widths and reference, for row-level analysis), and the graded
record of every arm lands in figures/rung3/friction_mini.jsonl: CFR+ W = 10,
CFR+ W = 100 and LCFR W = 10 at μ ∈ {0, 10} to 1M hands per seat, μ = 0.1 to
50k, μ ∈ {1, 100} for CFR+ and LCFR W = 10 to 500k. One seed per arm.
Claude-Session: https://claude.ai/code/session_014D1ZwBFFxJAkajfcQPe6bx
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Claude finished @AlecROndo's task in 1m 26s —— View job Review of PR #95 (friction mini experiment)
I did not run the test suite. I only read the code and the jsonl. Numbers checkEvery figure I recomputed from the jsonl matches the PR body:
Findings
Overall
|
| One departure from the paper, forced by sampling. In the paper the friction enters | ||
| every infoset's value with full weight, every iteration. External sampling banks a | ||
| row only when it is visited, and the payoff part of what it banks is weighted, in | ||
| expectation, by the chance-and-opponent reach of the row. The friction here is added | ||
| undivided on each visit, so it carries the same reach weight: at every row the tax | ||
| and the payoff keep the ratio they have in the paper, but the tax is reach-weighted. | ||
| Dividing by the visit probability instead would recover the paper's weighting and | ||
| hand rare rows enormous, noisy shoves. The paper's γ (a 1e-10 mix toward uniform) |
There was a problem hiding this comment.
Docstring claim looks wrong (confidence: medium-high, severity: medium — pedagogy). "at every row the tax and the payoff keep the ratio they have in the paper" does not follow from what is written.
In the paper, a row's counterfactual regret is weighted by the opponent and chance reach π₋ᵢ, but the friction term μ(x̂ − r̂) is not. Their ratio at a row is therefore (tax)/(π₋ᵢ·payoff).
Here both are added undivided per visit. In expectation both are scaled by the visit probability, so the ratio is the paper's multiplied by π₋ᵢ. The tax is weaker at rarely reached rows than in the paper, by the factor π₋ᵢ.
The next sentence ("dividing by the visit probability would recover the paper's weighting") says the same thing, so the two sentences contradict each other. This is also a plausible mechanism for the null result at rarely reached rows, and the PR's own row diagnostics point that way. Suggest rewording to: "the tax is down-weighted at row I by π₋ᵢ(I) relative to the paper, so μ is an effective μ·π₋ᵢ".
Line 3 (confidence: medium, severity: low): "A CFR run's CURRENT strategy circles the equilibrium forever" is overbroad. This PR's own data shows the current strategy beating the average at every mark. "need not converge, and in the worst case circles" is safer.
Line 13 (confidence: low, severity: low): "the paper reports that zeroing them stops convergence" — I could not verify this against arXiv 2308.11256 from here. Please cite the section or soften the claim.
| --out runs/friction-mini/cfrplus-w10-mu0.1.jsonl | ||
|
|
||
| Trains one run in this process (`train_lockstep`; friction is not parallelised) to | ||
| `--hands` hands per seat. Every `--reset-hands` hands per seat it moves the friction's |
There was a problem hiding this comment.
Confidence: high, severity: low. The docstring mentions --hands, but no such flag exists. The budget is the last of --marks. Also, the run terminates at marks[-1], so "Trains to ..." should say so.
| @@ -0,0 +1,130 @@ | |||
| """Friction toward a reference (RTCFR+-style) in the sampled walk. | |||
|
|
|||
| * **Off is today.** `friction=None` and friction at μ = 0 bank exactly today's regrets. | |||
There was a problem hiding this comment.
Test gaps (confidence: medium, severity: medium).
- "Off is today" compares
friction=Nonewith μ=0, i.e. two new code paths. It does not compare against the pre-PR walk. That holds by construction, because theNonebranch is the old code, but "bit for bit" is not pinned to any frozen bytes. It is also only tested on dict tables withlcfrandcfr+, not on aPackedTableor the vanilla rule. - No test checks that
traverse's reach plumbing (own_reach * sigma[k],reference_reach * reference[k]) matches a hand-computed tax over a small tree.friction_adjustmentis tested in isolation. The only end-to-end tests are "equals at t=1" and "!=after 20 iterations". The second asserts only inequality, so a wrong sign or a missing reach would pass. A one-decision toy game with known regrets would pin this down. Friction.new's negative or non-finite μ refusal is untested.test_the_reset...sampleskeys[::37]and only key 0 falls in the first 40 all-negative columns, so the uniform-fallback branch is barely exercised.- The gated Leduc test is one seed, a bare
<with no margin, and μ=0.05 with no justification. It is skipped by default, so the default suite never checks that friction does anything useful.
…π₋ᵢ against the paper, and tests pin its sign, μ refusals and the uniform reset branch Review follow-up on #95: the module docstring said the sampled tax keeps the paper's tax-to-payoff ratio; it keeps a constant ratio, i.e. μ acts as μ·π₋ᵢ(I). New tests: strong friction toward a fixed uniform reference leaves Leduc play nearer uniform than plain play; negative or non-finite μ is refused; the reset check covers the all-negative (uniform) rows. The script docstring drops a nonexistent --hands flag. Claude-Session: https://claude.ai/code/session_014D1ZwBFFxJAkajfcQPe6bx
|
Addressed in cb0f4aa:
|
|
Claude finished @AlecROndo's task in 35s —— View job Re-review of PR #95 after the two follow-up commitsI read the code and did not run the test suite. Earlier findings, status
Remaining nits (all low)
Verdict: the math and docstrings now agree with the code, and the sign convention matches the paper's loss-side formulation. I have no blocking findings. The one-seed caveat is the main thing I'd still address before merge. |
What
RTCFR+-style friction (Meng et al. 2023, arXiv 2308.11256) in the sampled tabular walk on mini-Drawmaha. The goal is to see whether a pull toward a moving reference makes the current strategy converge, under three inner rules: CFR+ (W = 10), mini-batch CFR+ (W = 100) and LCFR (W = 10).
solver/friction.py:Friction(mu, table, reference)and a vectorised reset. At the traverser's spots the tax μ·(own reach·σ − reference reach·reference) is subtracted from each action's value. It is added undivided on each visit, so it is reach-weighted like the payoffs.mccfr.traverseandlocksteptake an optional friction.friction=Noneand μ = 0 are today's walk bit for bit (tested).save_lockstepandParallelLocksteprefuse a friction run by name.scripts/minidrawmaha/friction_experiment.py: runs one arm, resetting the reference every 10k hands. It grades the current and average strategies at each mark;--save-tablewrites the arrays for row analysis.figures/rung3/friction_mini.jsonl: every graded mark of every arm (one seed each).Findings (exploitability, chips per hand)
Tests
uv run pytest: 1882 passed, 25 skipped. The gated Leduc gate (MINIDRAWMAHA_FULL_CALIBRATION=1) passes: on Leduc, friction makes CFR+'s current strategy beat plain CFR+'s.https://claude.ai/code/session_014D1ZwBFFxJAkajfcQPe6bx