Skip to content

ablm-eval 1.0.3 - #47

Merged
smburbach merged 2 commits into
mainfrom
flyloft/1d8801b2
Nov 20, 2026
Merged

smburbach merged 2 commits into
mainfrom
flyloft/1d8801b2

Conversation

@smburbach

Copy link
Copy Markdown
Member

Submit ablm-eval 1.0.2: AbLM-Eval at 32a3961 (BALM at a91d7b4 through its pin), the shed test unchanged, the skill carrying the measured table and classification outcomes.

Built by flyloft campaign 1d8801b2; its test's record is results/ablm-eval/1.0.3.txt.

…pute figures are the in-image measurements. The 1.0.2 checks ran inside the published image on its own digest (ABLMEVAL_MODE=image), and they differ from the venv twin's numbers the 1.0.2 skill carried: balm-moe's mean per-residue log-probability measured -0.2821 in the image against the twin's -0.2848 -- the only number that moved; the eight models the AbLM-Eval 32a3961 bump leaves alone reproduce exactly, in the twin, in the image, and against the 1.0.1 image's table -- and the wall times and classification metrics are re-measured. The skill now says the numbers were measured inside the published image, carries the image's all-models table (balm-moe <cls> -0.2821 at 15.0 s; the eight paired models in one config 145.1 s cold, balm-unpaired 22.6 s), notes balm-moe's log-probability is good to the third decimal, and states the in-image classification outcome (8M 106.0 s, balm-moe 748.3 s, both trained and wrote scores on the default training arguments, metrics labeled as from that short run). The launch-variable gotcha stays and is now confirmed in the image: the in-image classification run arrived with LOCAL_RANK=0 and RANK=0 and no WORLD_SIZE, and transformers' TrainingArguments failed exactly there until they were unset. The test paragraph cites the 1.0.2 record's run, about two minutes with the --overwrite exercise inside it. The Containerfile and test are unchanged; the image rebuilds to the same digest.
@smburbach
smburbach marked this pull request as ready for review November 20, 2026 01:47
@smburbach
smburbach merged commit 5921823 into main Nov 20, 2026
2 checks passed
@smburbach
smburbach deleted the flyloft/1d8801b2 branch November 20, 2026 01:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant