Skip to content

boltz-2 1.0.2 - #45

Merged
nateprice8 merged 5 commits into
mainfrom
flyloft/92868998
Nov 20, 2026
Merged

nateprice8 merged 5 commits into
mainfrom
flyloft/92868998

Conversation

@nateprice8

Copy link
Copy Markdown
Contributor

Submit boltz-2 1.0.2: CUDA 12.8 base of the same PyTorch 2.7.1 so the image runs on this cluster's B200s (1.0.1's CUDA 12.6 base has no sm_100 kernels); all other pins unchanged.

Built by flyloft campaign 92868998; its test's record is results/boltz-2/1.0.2.txt.

…so the image runs on Blackwell (B200, sm_100) -- 1.0.1's CUDA 12.6 base ships no sm_100 kernels, so every prediction on a B200 died at the first CUDA call with 'no kernel image is available for execution on the device'; every other pin (boltz 2.2.1, cuEquivariance 0.6.1, the weights revision) is unchanged
…tage

The candidate's own test run proved the Blackwell fix works -- the first
fixture predicted cleanly on the B200 (Number of failed examples: 0) --
but the test, unchanged from 1.0.1's shape of two separate invocations
at five diffusion samples, was killed by the 30-minute tool-test limit
during the second fixture: on sm_100 the first complex past
cuequivariance's 100-residue crossover pays the Triton kernels' first-use
compile on top of a second 3 GB checkpoint load and the image import,
and that compile had not finished when the limit hit.

Fold both fixtures through one boltz predict on their directory -- one
checkpoint load and one compile session, the same shape a batch screen
uses -- and cut to one diffusion sample per fixture; stamp every stage
so a future timeout names its stage. check.py globs the directory input's
boltz_results_<dir>/predictions/<stem>/ layout and drops the
five-sample ranking assert, vacuous at one sample (1.0.1's test covered
ranking at five samples on an L40S; nothing this version changes touches
it). SKILL.md's Compute section and Triton gotchas now describe the
once-per-process compile and the batching rule.
…, on every GPU

Two runs of the test on a B200 (fd77eed9, 50bcfb7f) were cancelled at the
30-minute limit still inside the 198-residue fixture's prediction: past
cuequivariance's 100-residue crossover, the Triton kernels' first-use compile
for sm_100 did not finish inside 29 minutes, while the 66-residue fixture
(under the crossover, plain PyTorch) predicted in 6 seconds. The test now
exports CUEQ_TRIMUL_FALLBACK_THRESHOLD=1000000 and
CUEQ_TRIATTN_FALLBACK_THRESHOLD=1000000 so both fixtures run the
plain-PyTorch ops -- the same code --no_kernels runs -- with no compile
involved; assertions are unchanged. The SKILL documents the compile cost and
both ways around it (raise the thresholds, or budget the compile once per
process; TRITON_CACHE_DIR lets later processes reuse the kernels). The
Containerfile is untouched.
@nateprice8
nateprice8 marked this pull request as ready for review November 20, 2026 00:26
@nateprice8
nateprice8 merged commit eabb342 into main Nov 20, 2026
2 checks passed
@nateprice8
nateprice8 deleted the flyloft/92868998 branch November 20, 2026 00:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant