A Boxoban policy trained with a win-only reward: no box-on-target hint, no curriculum, no search, and a 0.73M-parameter network (conv encoder + MinGRU). It was trained for 3B steps on mainline PufferLib with a small patch. On the official Boxoban unfiltered/test split it solves 80.7% of levels (sampled actions, 150 steps) and 74.7% greedy at DeepMind's 120-step limit.
Official Boxoban splits, one attempt per level. Values are solve rate % with a Wilson 95% interval. Sources: results/unit18/.
| Model (3B steps, same trainer and training levels) | unfiltered/test, 150 steps: greedy / sampled | unfiltered/test, 120 steps: greedy / sampled | medium/valid (20k): sampled | hard: sampled |
|---|---|---|---|---|
No hints (this repo, configs/boxoban_b_ours.ini) |
77.1 / 80.7 [78.1, 83.0] | 74.7 [71.9, 77.3] / 78.6 | 21.2 [20.6, 21.7] | 8.0 [7.1, 9.0] |
Puffer's shaped reward (configs/boxoban_m1_unf.ini) |
72.4 / 74.4 [71.6, 77.0] | 70.8 [67.9, 73.5] / 72.2 | 15.3 [14.9, 15.9] | 5.9 [5.1, 6.7] |
- Same budget, same levels, Puffer's shaped reward: the shaped baseline is mainline's shipped Boxoban setup (+0.25 per box on target, −1 on timeout, its own 9.9M-parameter network) trained on the same levels. Hint-free is +6.3 points on test (sampled; paired McNemar p = 1.3e-5) and +3.9 greedy at 120 steps (p = 0.015). It also solves more medium (21.2 vs 15.3) and hard (8.0 vs 5.9) levels.
- Longer run: an 8B-step run of the same recipe on Suarez's simplecl branch reaches 79.9% greedy at 120 steps (80.5% sampled at 150):
results/unit18/table_u18.md. - Limit, small budgets: at the Sokoban Speedrun's budget (0.5B steps) the hint matters. Removing only the box hint and the timeout penalty from the speedrun's record setup gives 0.0% and 5.8% (2 seeds) against 72.8% shaped (3 seeds). The hint-free run is still climbing when the budget ends. So the hint buys a fast start, not a higher ceiling:
results/unit19/. - Other grid games: the same conv encoder, swapped into mainline's default config, helps on snake (mean length at death 118–119 vs 104, 2 seeds; at 3× the shipped learning rate, chosen by a learning-rate check) and trash_pickup (38.7 vs 37.5 collected, 2 seeds), and ties on maze (seeds split 2–1):
results/unit17_games/. - Training note: V-trace's trace clip must be ≤ 1 with prioritized replay. On the earlier trainer,
vtrace_c_clip 5made value targets explode:docs/vtrace_bug.md.
git clone https://github.com/inductiveML/sokoban-no-hints && cd sokoban-no-hints
scripts/setup.sh # Python deps, official Boxoban levels (~34 MB download), checkpoint via Git LFS
scripts/eval.sh # scores checkpoints/b_ours_3b.bin on all splits; checks every level against results/unit18- Eval time: a few minutes on one GPU. It prints the table above and
REPRODUCED EXACTLYwhen every level's result and action sequence matches the shipped files. - Requirements: Python ≥ 3.10, PyTorch with CUDA, numpy, scipy. The shipped files were made with torch 2.11 / CUDA 12.8 on an RTX 6000 Ada. A different GPU or library version may change a few greedy ties.
scripts/setup.sh --trainer # PufferLib 5.0 @ ea7f9893 + patches/pufferlib_ea7f9893.patch, built; training level file
scripts/train.sh b_ours # 3B steps -> runs/b_ours/ckpt/boxoban/b_ours/0000002999975936.bin
scripts/eval.sh runs/b_ours/ckpt/boxoban/b_ours/0000002999975936.bin my_b_ours- Levels:
scripts/make_levels.pybuilds the training file (849,981 levels) from the official download and checks its sha256 against the file the shipped checkpoint was trained on. - Time: 3B steps took 4.8 h on an RTX 6000 Ada (48 GB, shared with other jobs; 110–200K steps/s, about 7.5 GB VRAM). Building the trainer needs CUDA and a C compiler.
- Shaped baseline:
scripts/train.sh m1_unf; it took 2.0 h on the same GPU. - Config check:
scripts/check_config.pyshows the shipped configs equal the runs' logged effective configs.
More detail: docs/protocol.md.
patches/pufferlib_ea7f9893.patch conv encoder (src/convgrid.cu) + Boxoban timeout_penalty / random_start_tick flags
configs/ boxoban_b_ours.ini (shipped model), boxoban_m1_unf.ini (shaped baseline)
scripts/ setup.sh, make_levels.py, train.sh, eval.sh, evaluate.py, make_table.py,
check_config.py, vtrace_bug.py
sokoban_eval/ evaluation harness (level parser, batched env, policy port, protocol)
checkpoints/ b_ours_3b.bin (Git LFS) + SHA256SUMS
data/sets/ fixed level lists (medium/valid sample, held-out lists used for the exclusion)
results/ unit18 (official splits), unit17_games, unit17_b_ours, unit19 (small-budget limit)
docs/ protocol.md, vtrace_bug.md, outside_results.md
- Joseph Suarez and PufferLib: the trainer, the Boxoban environment and the shaped baseline configuration. This repo patches mainline PufferLib 5.0.
- DeepMind:
- The Boxoban levels.
- DRC (Guez et al. 2019, "An investigation of model-free planning"), which reports 99% on unfiltered test after 1e9 steps with a shaped reward (+10 solve, ±1 per box, −0.01 per step) and a 120-step cap.
- @dkta0, "Playing Sokoban backwards": a win-only reward with a reverse curriculum. The blog gives "61%" at 3B steps for PufferLib's shaped reward. Its 72.2% reverse-curriculum figure is from an X post we could not read, so it is unverified here.
- Sokoban Speedrun, non-LLM track: greedy, 120 steps, scored on unfiltered/test, with Puffer's shaped reward. Its record scores 74.2% (lower 95% CI 0.715) in 12:38 on one H100.
Protocols and quotes for these numbers: docs/outside_results.md.