Skip to content

docs(fold): milestone 2 accepted — proof runs, evaluation, width decision - #120

Merged
briney merged 2 commits into
mainfrom
docs/fold-m2c-results
Nov 13, 2026
Merged

briney merged 2 commits into
mainfrom
docs/fold-m2c-results

Conversation

@briney

@briney briney commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Closes #48

This PR records the milestone-2c proof runs, their offline evaluation and the pair-width decision. It accepts milestone 2.

Decision: pair width 256. At step 20000, w256 beats w128 on the held-out sets (oplm fold compare, paired over targets, bootstrap 95 % CI):

set n w128 TM w256 TM Δ 95 % CI
pooled 975 0.755 0.771 +0.016 [0.013, 0.020]
CAMEO 745 0.806 0.821 +0.015 [0.011, 0.019]
CASP14 230 0.590 0.611 +0.020 [0.012, 0.030]
  • The pooled lower bound (0.0127) clears the plan's 0.01 bar for paying w256's 1.58× step cost.
  • w256 also leads on lDDT (+0.022 pooled), at 10k (+0.017), and at matched compute: w128 at 30k trails w256 at 20k by 0.016.

Deviations from the plan:

  • Runs stopped early. Both runs were stopped at w128 34.2k and w256 21.6k instead of 50k. The answer was already clear, and the ESM-C checkpoints don't carry into M3. Stopping saved about 2,000 GPU-hours.
  • Acceptance step. Acceptance compares 20k against 10k, not 50k against 10k.

Acceptance:

  1. Held-out TM improves (met). It improves from 10k to 20k on CAMEO and CASP14 for both runs, with every CI above 0. w128's CASP14 score dips from 20k to 30k, which is noted in the README.
  2. Resume exercised (met). Task 8 evidence is in docs/fold/m2/runs/.
  3. Width: 256.
  4. Gates. Global batch 128 and the warmup are recorded.

Finding (#119): both runs trained at an effective peak LR of 3.33e-4, not 1e-3. The fold configs inherit adamw_lr_mult: 0.333 from the MLM base recipe (483e91a, which landed after the configs were written). The width comparison is like for like, but spec §6.3's 1e-3 was never tried. M3 must set adamw_lr_mult: 1.0 and settle its LR.

Also in this PR:

  • docs/fold/b200-m2c-eval.sbatch wrote an empty pip-freeze.txt: it ran python -m uv inside the venv, which doesn't have uv. The freeze step now runs before activation, with --python. This run's freeze was rebuilt from the venv's dist-info.

Files:

  • docs/fold/m2/README.md: new section "Milestone 2c: proof runs and the width decision".
  • docs/fold/m2/eval/: per-target CSVs, summaries, status, versions and valid-curves.csv.
  • docs/FOLD.md: status line, plus §10 Results.

🤖 Generated with Claude Code

briney and others added 2 commits November 13, 2026 09:13
…for both widths

Both runs (jobs 35927 w128, 35928 w256) drained, requeued and resumed into
the same W&B run. SIGTERM to the ranks replaced the plan's scancel USR1,
which does not reach or survive in the ranks on this cluster (#113).

Refs #48

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…accepted

Both proof runs stopped early (w128 at 34.2k, w256 at 21.6k): at step
20000 w256 beats w128 on CAMEO + CASP14 by +0.016 TM pooled, 95 % CI
[0.013, 0.020], and the ESM-C checkpoints do not carry into M3.
Acceptance uses 20k against 10k instead of 50k against 10k. Width 256.

Records that both runs trained at an effective LR of 3.33e-4 (inherited
adamw_lr_mult 0.333, #119), and fixes the eval script's empty
pip-freeze.txt (uv is not installed in the eval venv).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@briney
briney merged commit b82e7ad into main Nov 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M2c cluster proof runs: tag fold-m2c-runs, then cluster Tasks 8-9

1 participant