Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Sokoban without hints

A Boxoban policy trained with a win-only reward: no box-on-target hint, no curriculum, no search, and a 0.73M-parameter network (conv encoder + MinGRU). It was trained for 3B steps on mainline PufferLib with a small patch. On the official Boxoban unfiltered/test split it solves 80.7% of levels (sampled actions, 150 steps) and 74.7% greedy at DeepMind's 120-step limit.

Results

Official Boxoban splits, one attempt per level. Values are solve rate % with a Wilson 95% interval. Sources: results/unit18/.

Model (3B steps, same trainer and training levels) unfiltered/test, 150 steps: greedy / sampled unfiltered/test, 120 steps: greedy / sampled medium/valid (20k): sampled hard: sampled
No hints (this repo, configs/boxoban_b_ours.ini) 77.1 / 80.7 [78.1, 83.0] 74.7 [71.9, 77.3] / 78.6 21.2 [20.6, 21.7] 8.0 [7.1, 9.0]
Puffer's shaped reward (configs/boxoban_m1_unf.ini) 72.4 / 74.4 [71.6, 77.0] 70.8 [67.9, 73.5] / 72.2 15.3 [14.9, 15.9] 5.9 [5.1, 6.7]
  • Same budget, same levels, Puffer's shaped reward: the shaped baseline is mainline's shipped Boxoban setup (+0.25 per box on target, −1 on timeout, its own 9.9M-parameter network) trained on the same levels. Hint-free is +6.3 points on test (sampled; paired McNemar p = 1.3e-5) and +3.9 greedy at 120 steps (p = 0.015). It also solves more medium (21.2 vs 15.3) and hard (8.0 vs 5.9) levels.
  • Longer run: an 8B-step run of the same recipe on Suarez's simplecl branch reaches 79.9% greedy at 120 steps (80.5% sampled at 150): results/unit18/table_u18.md.
  • Limit, small budgets: at the Sokoban Speedrun's budget (0.5B steps) the hint matters. Removing only the box hint and the timeout penalty from the speedrun's record setup gives 0.0% and 5.8% (2 seeds) against 72.8% shaped (3 seeds). The hint-free run is still climbing when the budget ends. So the hint buys a fast start, not a higher ceiling: results/unit19/.
  • Other grid games: the same conv encoder, swapped into mainline's default config, helps on snake (mean length at death 118–119 vs 104, 2 seeds; at 3× the shipped learning rate, chosen by a learning-rate check) and trash_pickup (38.7 vs 37.5 collected, 2 seeds), and ties on maze (seeds split 2–1): results/unit17_games/.
  • Training note: V-trace's trace clip must be ≤ 1 with prioritized replay. On the earlier trainer, vtrace_c_clip 5 made value targets explode: docs/vtrace_bug.md.

Quickstart

git clone https://github.com/inductiveML/sokoban-no-hints && cd sokoban-no-hints
scripts/setup.sh                 # Python deps, official Boxoban levels (~34 MB download), checkpoint via Git LFS
scripts/eval.sh                  # scores checkpoints/b_ours_3b.bin on all splits; checks every level against results/unit18
  • Eval time: a few minutes on one GPU. It prints the table above and REPRODUCED EXACTLY when every level's result and action sequence matches the shipped files.
  • Requirements: Python ≥ 3.10, PyTorch with CUDA, numpy, scipy. The shipped files were made with torch 2.11 / CUDA 12.8 on an RTX 6000 Ada. A different GPU or library version may change a few greedy ties.

Retrain

scripts/setup.sh --trainer       # PufferLib 5.0 @ ea7f9893 + patches/pufferlib_ea7f9893.patch, built; training level file
scripts/train.sh b_ours          # 3B steps -> runs/b_ours/ckpt/boxoban/b_ours/0000002999975936.bin
scripts/eval.sh runs/b_ours/ckpt/boxoban/b_ours/0000002999975936.bin my_b_ours
  • Levels: scripts/make_levels.py builds the training file (849,981 levels) from the official download and checks its sha256 against the file the shipped checkpoint was trained on.
  • Time: 3B steps took 4.8 h on an RTX 6000 Ada (48 GB, shared with other jobs; 110–200K steps/s, about 7.5 GB VRAM). Building the trainer needs CUDA and a C compiler.
  • Shaped baseline: scripts/train.sh m1_unf; it took 2.0 h on the same GPU.
  • Config check: scripts/check_config.py shows the shipped configs equal the runs' logged effective configs.

More detail: docs/protocol.md.

Layout

patches/pufferlib_ea7f9893.patch   conv encoder (src/convgrid.cu) + Boxoban timeout_penalty / random_start_tick flags
configs/                           boxoban_b_ours.ini (shipped model), boxoban_m1_unf.ini (shaped baseline)
scripts/                           setup.sh, make_levels.py, train.sh, eval.sh, evaluate.py, make_table.py,
                                   check_config.py, vtrace_bug.py
sokoban_eval/                      evaluation harness (level parser, batched env, policy port, protocol)
checkpoints/                       b_ours_3b.bin (Git LFS) + SHA256SUMS
data/sets/                         fixed level lists (medium/valid sample, held-out lists used for the exclusion)
results/                           unit18 (official splits), unit17_games, unit17_b_ours, unit19 (small-budget limit)
docs/                              protocol.md, vtrace_bug.md, outside_results.md

Related work and credits

  • Joseph Suarez and PufferLib: the trainer, the Boxoban environment and the shaped baseline configuration. This repo patches mainline PufferLib 5.0.
  • DeepMind:
    • The Boxoban levels.
    • DRC (Guez et al. 2019, "An investigation of model-free planning"), which reports 99% on unfiltered test after 1e9 steps with a shaped reward (+10 solve, ±1 per box, −0.01 per step) and a 120-step cap.
  • @dkta0, "Playing Sokoban backwards": a win-only reward with a reverse curriculum. The blog gives "61%" at 3B steps for PufferLib's shaped reward. Its 72.2% reverse-curriculum figure is from an X post we could not read, so it is unverified here.
  • Sokoban Speedrun, non-LLM track: greedy, 120 steps, scored on unfiltered/test, with Puffer's shaped reward. Its record scores 74.2% (lower 95% CI 0.715) in 12:38 on one H100.

Protocols and quotes for these numbers: docs/outside_results.md.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages