Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions datasets/hello2-6a463bf0/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
name: hello2-6a463bf0
description: "300 synthetic antibody pairs of random amino acids in three splits: the repository's smoke dataset. Reach for it to check that reading a dataset works from where you are, never to train."
metadata:
version: 1.0.0
tools: datascience@1.0.0
license: mit
---
# hello2-6a463bf0

Hub: `brineylab/hello2-6a463bf0`, tag `v1.0.0`

## Files

`train.parquet`, `eval.parquet`, `test.parquet`: 240, 30, and 30 rows, one per pair, columns `pair_id` (`hello-0000` to `hello-0299`), `sequence_heavy` (110 to 130 amino acids), `sequence_light` (105 to 115). The letters are uniform draws from the twenty amino acids: no sequence is an antibody.

## Reading it

```python
from huggingface_hub import snapshot_download
import polars as pl
path = snapshot_download("brineylab/hello2-6a463bf0", repo_type="dataset", revision="v1.0.0")
train = pl.read_parquet(f"{path}/train.parquet")
```

## Recipe

No source: `recipe/build` draws `pairs = 300` pairs from `random.Random(seed = 1)`, shuffles them, and splits them `[0.8, 0.1, 0.1]` in that order. It runs in `datascience@1.0.0`. The test checks the three files, their columns, their row counts, and that no `pair_id` is in two splits.

## Compute

Seconds on a laptop, one CPU, kilobytes of disk.

## Gotchas

- It exists to exercise filing, checking, promoting, and reading; its sequences mean nothing.
23 changes: 23 additions & 0 deletions datasets/hello2-6a463bf0/recipe/build
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/usr/bin/env -S python3 -I
"""recipe/build <out>: `pairs` synthetic antibody pairs from params.toml's seed, shuffled and split by
`split` into train, eval, and test parquet files in <out>."""
import random
import sys
import tomllib
from pathlib import Path

import polars as pl

AA = "ACDEFGHIKLMNPQRSTVWY"
params = tomllib.loads((Path(__file__).parent / "params.toml").read_text())
out = Path(sys.argv[1])
out.mkdir(parents=True, exist_ok=True)
rng = random.Random(params["seed"])
chain = lambda lo, hi: "".join(rng.choice(AA) for _ in range(rng.randint(lo, hi)))
rows = [(f"hello-{i:04d}", chain(110, 130), chain(105, 115)) for i in range(params["pairs"])]
rng.shuffle(rows)
n_train, n_eval = (round(f * len(rows)) for f in params["split"][:2])
splits = {"train": rows[:n_train], "eval": rows[n_train:n_train + n_eval], "test": rows[n_train + n_eval:]}
for split, part in splits.items():
pl.DataFrame(part, schema=["pair_id", "sequence_heavy", "sequence_light"], orient="row").write_parquet(out / f"{split}.parquet")
print(f"{split}.parquet {len(part)} rows")
3 changes: 3 additions & 0 deletions datasets/hello2-6a463bf0/recipe/params.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
seed = 1
pairs = 300
split = [0.8, 0.1, 0.1]
26 changes: 26 additions & 0 deletions datasets/hello2-6a463bf0/recipe/test
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env -S python3 -I
"""recipe/test <out>: exit 0 iff <out> holds hello's three splits, with its columns and row counts and no
pair_id in two splits."""
import sys
from pathlib import Path

import polars as pl

out, seen, problems = Path(sys.argv[1]), {}, []
for split, rows in (("train", 240), ("eval", 30), ("test", 30)):
path = out / f"{split}.parquet"
if not path.is_file():
problems.append(f"{split}.parquet is missing")
continue
df = pl.read_parquet(path)
if df.columns != ["pair_id", "sequence_heavy", "sequence_light"]:
problems.append(f"{split}.parquet has columns {df.columns}")
if df.height != rows:
problems.append(f"{split}.parquet has {df.height} rows, not {rows}")
for pid in df["pair_id"]:
if pid in seen:
problems.append(f"{pid} is in {seen[pid]} and {split}")
seen[pid] = split
print(f"{split}.parquet {df.height} rows")
print("\n".join(problems) or "ok")
sys.exit(1 if problems else 0)
26 changes: 26 additions & 0 deletions results/hello2-6a463bf0/1.0.0.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
dataset: hello2-6a463bf0
version: 1.0.0
hub: brineylab-dev/hello2-6a463bf0
branch: candidate-22a46dde029a3283d7b17a1b29a48d7f01821605
commit: d13fe0c7e780743b714f780ce5dbbbf7148f2d46
tree: 22a46dde029a3283d7b17a1b29a48d7f01821605
tools: datascience@1.0.0
runner: sbatch --partition=hpc-low --cpus-per-task=4 --mem=8G --time=00:20:00
host: sunk.6af0ac-us-east-13a-briney-rtx6000pro.coreweave.app
started: 2026-11-07T00:52:59Z
exit: 0
---
pyxis: importing docker image: ghcr.io#brineylab/toolshed/datascience:1.0.0
pyxis: imported docker image: ghcr.io#brineylab/toolshed/datascience:1.0.0
train.parquet 240 rows
eval.parquet 30 rows
test.parquet 30 rows
train.parquet 240 rows
eval.parquet 30 rows
test.parquet 30 rows
ok
file eval.parquet 6602 fc311054c3da6268f2dcb7cb9a751697a5542f6fdd0ea5befe7bb7a6a39a234d
file test.parquet 6505 98c0a809593d563ea422663e21185e8b08631be9b7a593a3eda2d0034a0b5234
file train.parquet 35145 5382f5f36ce977c410e4401ef5dc4c8755e8aa66d5cee8b4e217809473da5b72
commit: d13fe0c7e780743b714f780ce5dbbbf7148f2d46
Loading