Repository navigation
ablm-eval 1.0.3 - #47
Merged
Merged
Conversation
…pute figures are the in-image measurements. The 1.0.2 checks ran inside the published image on its own digest (ABLMEVAL_MODE=image), and they differ from the venv twin's numbers the 1.0.2 skill carried: balm-moe's mean per-residue log-probability measured -0.2821 in the image against the twin's -0.2848 -- the only number that moved; the eight models the AbLM-Eval 32a3961 bump leaves alone reproduce exactly, in the twin, in the image, and against the 1.0.1 image's table -- and the wall times and classification metrics are re-measured. The skill now says the numbers were measured inside the published image, carries the image's all-models table (balm-moe <cls> -0.2821 at 15.0 s; the eight paired models in one config 145.1 s cold, balm-unpaired 22.6 s), notes balm-moe's log-probability is good to the third decimal, and states the in-image classification outcome (8M 106.0 s, balm-moe 748.3 s, both trained and wrote scores on the default training arguments, metrics labeled as from that short run). The launch-variable gotcha stays and is now confirmed in the image: the in-image classification run arrived with LOCAL_RANK=0 and RANK=0 and no WORLD_SIZE, and transformers' TrainingArguments failed exactly there until they were unset. The test paragraph cites the 1.0.2 record's run, about two minutes with the --overwrite exercise inside it. The Containerfile and test are unchanged; the image rebuilds to the same digest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Submit ablm-eval 1.0.2: AbLM-Eval at 32a3961 (BALM at a91d7b4 through its pin), the shed test unchanged, the skill carrying the measured table and classification outcomes.
Built by flyloft campaign 1d8801b2; its test's record is results/ablm-eval/1.0.3.txt.