Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions results/ablm-eval/1.0.2.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
tool: ablm-eval
version: 1.0.2
image: ghcr.io/brineylab/toolshed/ablm-eval@sha256:8c932603546af070394b0747658992b298f6874913d8b43b61233389d64c88a9
digest: sha256:8c932603546af070394b0747658992b298f6874913d8b43b61233389d64c88a9
runner: sbatch --partition=flyloft --gres=gpu:1 --time=00:30:00
host: sunk.6af0ac-eu-south-03b-briney-b200.coreweave.app
started: 2026-11-20T01:08:02Z
exit: 0
---
Job 1077: SUNK job dashboard: https://cks-grafana.coreweave.com/d/slurm-job-metrics/slurm-job-metrics?var-slurm_cluster=slurm&var-job_id=1077
pyxis: importing docker image: ghcr.io#brineylab/toolshed/ablm-eval@sha256:8c932603546af070394b0747658992b298f6874913d8b43b61233389d64c88a9
pyxis: imported docker image: ghcr.io#brineylab/toolshed/ablm-eval@sha256:8c932603546af070394b0747658992b298f6874913d8b43b61233389d64c88a9
node: slurm-b200-204-039
GPU 0: NVIDIA B200 (UUID: GPU-e3ae7af6-08de-017b-aedd-15ab9be767f6)
┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃ name ┃ repository ┃ revision ┃ separator ┃
┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ balm-paired │ brineylab/BALM-paired │ 78e6a59276db │ </s> │
│ balm-unpaired │ brineylab/BALM-unpaired │ 8fd18a5b47bf │ single chains only │
│ ft-esm │ brineylab/ft-ESM │ 6594c8b61de1 │ <cls><cls> │
│ balm-moe │ brineylab/BALM-MoE │ 51842e69ca3f │ <cls> │
│ currab │ brineylab/CurrAb │ 92e28534663e │ <cls> │
│ preferential-250k │ brineylab/preferential-250k │ 72ebb7b3e0f1 │ <cls><cls> │
│ uniform-250k │ brineylab/uniform-250k │ bb2fd2338300 │ <cls><cls> │
│ uniform-350k │ brineylab/uniform-350k │ 43a1e4fd35cf │ <cls><cls> │
│ scaling-350m-full │ brineylab/350M_full_checkpoint-500000 │ 9cd9e8af5138 │ <cls><cls> │
└───────────────────┴───────────────────────────────────────┴──────────────┴────────────────────┘
tiny: brineylab/8M_full_checkpoint-500000 @ 01e6532f9091
Fetching 7 files: 0%| | 0/7 [00:00<?, ?it/s]Fetching 7 files: 14%|█▍ | 1/7 [00:00<00:01, 5.69it/s]Fetching 7 files: 100%|██████████| 7/7 [00:01<00:00, 4.05it/s]Fetching 7 files: 100%|██████████| 7/7 [00:01<00:00, 4.10it/s]
Running Task #1: Mini Per Pos Inference
Loading weights: 0%| | 0/107 [00:00<?, ?it/s]Loading weights: 100%|██████████| 107/107 [00:00<00:00, 15506.55it/s]
Generating train split: 0 examples [00:00, ? examples/s]Generating train split: 8 examples [00:00, 2877.49 examples/s]
Map: 0%| | 0/8 [00:00<?, ? examples/s]Map: 100%|██████████| 8/8 [00:00<00:00, 2022.81 examples/s]
Map: 0%| | 0/8 [00:00<?, ? examples/s]Map: 100%|██████████| 8/8 [00:00<00:00, 1230.63 examples/s]
Running inference: 0%| | 0/8 [00:00<?, ?it/s]Running inference: 12%|█▎ | 1/8 [00:00<00:02, 3.07it/s]Running inference: 50%|█████ | 4/8 [00:00<00:00, 10.06it/s]Running inference: 88%|████████▊ | 7/8 [00:00<00:00, 14.24it/s]Running inference: 100%|██████████| 8/8 [00:00<00:00, 12.40it/s]
python 3.12.3
token ids of the first paired input: [ 0 9 7 16 4 7 9 8 6 6 6 4 7 16 14 6 6 8 4 10 4 8 23 5
5 8 6 18 11 18 8 8 19 5 20 8 22 7 10 16 5 14 6 15 6 4 9 22
7 8 5 12 8 6 8 6 6 8 11 19 19 6 13 8 7 15 6 10 18 11 12 8
10 13 17 8 15 17 11 4 19 4 16 20 17 8 4 10 5 9 13 11 5 7 19 19
23 5 15 5 5 20 14 5 10 14 6 12 7 19 19 19 19 19 19 20 13 7 22 6
15 6 11 11 7 11 7 8 8 0 0 13 12 16 20 11 16 8 14 8 8 4 8 5
8 7 6 13 10 7 11 12 11 23 10 5 8 16 6 12 17 17 19 4 5 22 19 16
16 15 14 6 15 7 14 15 4 4 12 19 5 5 8 11 4 16 8 6 7 14 8 10
18 8 6 8 6 8 6 11 13 18 11 4 11 12 8 8 4 16 14 9 13 7 5 11
19 19 23 16 15 19 17 8 5 14 4 11 18 6 6 6 8 15 7 9 12 15 2]
mean per-residue log-probability: -0.311014 (reference -0.311014, tolerance 0.0001)
tiny: brineylab/8M_full_checkpoint-500000 @ 01e6532f9091
Fetching 7 files: 0%| | 0/7 [00:00<?, ?it/s]Fetching 7 files: 100%|██████████| 7/7 [00:00<00:00, 1027.26it/s]
Running Task #1: Mini Per Pos Inference
Loading weights: 0%| | 0/107 [00:00<?, ?it/s]Loading weights: 100%|██████████| 107/107 [00:00<00:00, 14165.92it/s]
Running inference: 0%| | 0/8 [00:00<?, ?it/s]Running inference: 12%|█▎ | 1/8 [00:00<00:02, 2.92it/s]Running inference: 50%|█████ | 4/8 [00:00<00:00, 9.74it/s]Running inference: 88%|████████▊ | 7/8 [00:00<00:00, 13.94it/s]Running inference: 100%|██████████| 8/8 [00:00<00:00, 12.07it/s]
python 3.12.3
token ids of the first paired input: [ 0 9 7 16 4 7 9 8 6 6 6 4 7 16 14 6 6 8 4 10 4 8 23 5
5 8 6 18 11 18 8 8 19 5 20 8 22 7 10 16 5 14 6 15 6 4 9 22
7 8 5 12 8 6 8 6 6 8 11 19 19 6 13 8 7 15 6 10 18 11 12 8
10 13 17 8 15 17 11 4 19 4 16 20 17 8 4 10 5 9 13 11 5 7 19 19
23 5 15 5 5 20 14 5 10 14 6 12 7 19 19 19 19 19 19 20 13 7 22 6
15 6 11 11 7 11 7 8 8 0 0 13 12 16 20 11 16 8 14 8 8 4 8 5
8 7 6 13 10 7 11 12 11 23 10 5 8 16 6 12 17 17 19 4 5 22 19 16
16 15 14 6 15 7 14 15 4 4 12 19 5 5 8 11 4 16 8 6 7 14 8 10
18 8 6 8 6 8 6 11 13 18 11 4 11 12 8 8 4 16 14 9 13 7 5 11
19 19 23 16 15 19 17 8 5 14 4 11 18 6 6 6 8 15 7 9 12 15 2]
mean per-residue log-probability: -0.311014 (reference -0.311014, tolerance 0.0001)
Job 1077: SUNK job dashboard: https://cks-grafana.coreweave.com/d/slurm-job-metrics/slurm-job-metrics?var-slurm_cluster=slurm&var-job_id=1077
Expand Down
8 changes: 5 additions & 3 deletions tools/ablm-eval/Containerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# AbLM-Eval (github.com/smburbach/AbLM-Eval, MIT) at commit 641cff8: evaluation
# AbLM-Eval (github.com/smburbach/AbLM-Eval, MIT) at commit 32a3961: evaluation
# of antibody language models -- inference, per-position inference, mutation
# prediction, naturalness, classification (fine-tuning a classification head
# over folds) and MoE routing -- from a YAML config, on the lab's published
Expand Down Expand Up @@ -40,9 +40,11 @@ RUN pip uninstall -y --break-system-packages torchaudio torchvision \
# (any pyarrow 21 or later needs numpy 2 and breaks the import). pyyaml,
# matplotlib, seaborn and tqdm are pinned at the versions the same set resolved
# to in the campaign's Python 3.12 build of it. BALM rides AbLM-Eval's own
# dependency pin, git 36a8001.
# dependency pin, git a91d7b4 -- the commit that fixes BalmTokenizer on
# transformers 5.23 (which passes special tokens as AddedToken objects), so
# balm-moe loads.
RUN pip install --no-cache-dir --break-system-packages \
"ablm-eval @ git+https://github.com/smburbach/AbLM-Eval@641cff87aa688967d61a5beaacb8c9b51b29c78e" \
"ablm-eval @ git+https://github.com/smburbach/AbLM-Eval@32a396150cc16303f7c06b7f0631f103f493b728" \
numpy==1.26.4 \
pyarrow==20.0.0 \
transformers==5.23.0 \
Expand Down
Loading