diff --git a/datasets/oplm-antibody-sequences/SKILL.md b/datasets/oplm-antibody-sequences/SKILL.md new file mode 100644 index 0000000..20c6d75 --- /dev/null +++ b/datasets/oplm-antibody-sequences/SKILL.md @@ -0,0 +1,111 @@ +--- +name: oplm-antibody-sequences +description: "Human antibody heavy and light chains from AntiRef v2022.12.14 (OAS, healthy donors' blood), laid out as OPLM's sequence data: AntiRef96 clusters with their AntiRef99 members, AntiRef99 representatives, and held-out AntiRef90 clusters with locus and an IMGT CDR mask. Reach for it to add antibodies to OPLM training or to measure CDR3 loss." +metadata: + version: 1.0.0 + tools: datascience@1.2.0 + license: cc-by-4.0 +--- +# oplm-antibody-sequences + +Hub: `brineylab/oplm-antibody-sequences`, tag `v1.0.0` + +A pilot: does a decay with antibodies improve OPLM's antibody performance, judged on CDR3, against a decay on protein sequences only? It is built from the published AntiRef as it stands (Briney 2023, *Bioinformatics Advances*, [10.1093/bioadv/vbad109](https://doi.org/10.1093/bioadv/vbad109)). A rebuild from current OAS, with donors and germline annotations, waits on the pilot. + +## Files + +The layout is `brineylab/oplm-sequence-data`'s, one folder per training source: + +``` +antiref96/train/shard-00000.parquet … one row per AntiRef96 cluster +antiref99/train/shard-00000.parquet … one row per AntiRef99 representative +eval_singletons.parquet eval_gte2.parquet test_singletons.parquet test_gte2.parquet +``` + +- **`antiref96/train/`**, uniref70's columns: + - `sequence_id`: the AntiRef96 representative's AntiRef UUID; + - `sequence`: its amino acids; + - `member_ids`, `member_sequences`: the AntiRef99 representatives inside the cluster, the representative's own entry first. +- **`antiref99/train/`**, deepclust70's columns: `sequence_id`, `sequence`. +- **Shards:** 1,000,000 rows each (the last fewer), row groups of 10,000, zstd. Rows are in a seeded global shuffle (`md5('1:')`), so any subset of shards is a uniform sample. +- **Eval and test** each hold 5,000 representatives of held-out AntiRef90 clusters: `_singletons` from clusters of one AntiRef100 sequence, `_gte2` from clusters of two or more. Heavy and light chains are mixed. Columns: + - `sequence_id`, `sequence`; + - `locus`: `IGH`, `IGK`, or `IGL`; + - `cdr_mask_aa`: abstar 0.9.0's CDR mask, one digit per residue of `sequence`: `0` framework, `1` CDR1, `2` CDR2, `3` CDR3. It uses IMGT regions (CDR1 27–38, CDR2 56–65, CDR3 105–117, insertions with their base position) from ANARCII 2.0.8's numbering. + + Every member of a held-out cluster is out of both train sets. + +`sequence` is the variable domain, FWR1 through FWR4: the span abstar's `sequence_vdj_aa` covers. + +## Reading it + +```python +from huggingface_hub import snapshot_download +path = snapshot_download("brineylab/oplm-antibody-sequences", repo_type="dataset", revision="v1.0.0") +``` + +In OPLM, a `data.train` source is a folder of shards: +- **AntiRef96 representatives:** `antiref96/train` with `sample_members: false`. +- **AntiRef96 clusters sampled to one AntiRef99 member:** `antiref96/train` with `sample_members: true`. +- **AntiRef99 representatives:** `antiref99/train`. + +The four held-out files are `type: sequence` evals. The CDR3 eval, which scores only positions where `cdr_mask_aa` is `3` and runs per `locus`, is briney/oplm#85. + +## Recipe + +**Source:** AntiRef v2022.12.14 on Zenodo (CC BY 4.0), pulled 2026-11-09: +- record 7474336, `antiref_cluster-manifest.csv.gz` (14,054,641,207 bytes, MD5 `18fcf6376702a2d23e2acdc32f1fd40a`): one row per AntiRef100 sequence, with its representative at 100, 99, 98, 96, 94, 92, and 90% identity; +- record 7474658, `antiref100.fasta.gz` (18,496,427,276 bytes, MD5 `4a82d4be0c493377668fac9730aca866`): each sequence's UUID and amino acids. + +The recipe refuses a download whose MD5 differs. AntiRef's clustering was MMseqs2 linclust, nested: each threshold clustered the previous one's representatives, with `-c 0.8 --cov-mode 0`. + +**Steps**, one job in `datascience@1.2.0`: + +1. **Stage** the manifest and the FASTA as parquet. Every manifest ID must have a sequence, and every sequence a manifest row. +2. **Hold out.** + - Put the AntiRef90 clusters in the seeded order, `md5(':')`, and walk singletons and gte2 separately. + - Number each representative with ANARCII (`seq_type="antibody"`, `mode="accuracy"`, IMGT). Keep it when its chain type is H, K, or L, every residue is numbered, and it has a CDR3. + - The first `heldout` kept are eval, the next `heldout` test. Candidates that fail numbering stay in train; the build prints how many. + - Before any of this, the build numbers one known heavy chain and stops if its mask moved. +3. **Train.** Drop every manifest row in a held-out cluster. AntiRef99 rows are the distinct `antiref99`. AntiRef96 rows are the distinct `antiref96`, with their distinct `antiref99` as members; the build stops if a representative is not its own first member. Both are written in the seeded order. + +`params.toml`: +- `seed = 1`, `heldout = 5000`, `shard_rows = 1000000`; +- `limit = 0`: whole AntiRef90 clusters, for a developer's dry run. 0 is all, and the published version has 0. + +**The test** checks: +- the files, their columns, and `heldout` rows in each held-out file; +- the shards' names and sizes, and each train folder against its floor; +- no `sequence_id` in two held-out files or in a held-out file and a train shard (first and last); +- each mask's length, digits, and single CDR3 run; +- `locus`; +- representative-first members. + +## Compute + +Measured on the first full build, on a workstation under podman with 16 CPUs and 100 GB of memory: +- **Time:** about 37 minutes. + - Download, 32 parallel range requests: 11 min. + - Stage: 9 min. + - Hold out (ANARCII on 16 CPUs): 5 min. + - AntiRef99: 3 min. + - AntiRef96: 9 min. +- **Memory:** the container's peak reached its 100 GB limit. That peak counts page cache, and DuckDB spills past 80% of the limit, so 100 GB is enough but not much more than enough; ask 128 GB. +- **Disk:** + - `.work/`: 89 GB (32.5 GB of downloads and 62.5 GB staged); + - DuckDB's spill: up to about 90 GB; + - the output: 40 GB (AntiRef96 24 GB, AntiRef99 16 GB). +- **Rows:** AntiRef96 210,559,202 in 211 shards; AntiRef99 315,983,206 in 316 shards. The test's floors are 90% of these. + +## Gotchas + +- **Zenodo is slow to CoreWeave.** On 2026-11-10 it served CoreWeave at 5–7 KB/s per connection and the Scripps network at about 520 KB/s, while CoreWeave reached other hosts at hundreds of MB/s. The 32.5 GB download would take about two days on CoreWeave, so build on bcr. +- **Held out by AntiRef90 cluster membership only.** No identity screen was run. + - A train sequence can share a held-out sequence's V gene and a similar CDR3 at under 90% identity. + - Because linclust is greedy and nested over six levels, a train sequence can even be 90% or more identical to a held-out representative and sit in another AntiRef90 cluster. + - Antibody loss on these sets flatters the model. Compare runs on the same sets; don't read absolute values. +- Two clustering thresholds live in one dataset, against the datasets design's rule (D3), by the lab's decision for this pilot: every train option shares one set of eval and test files. +- No screen against OPLM's uniref70 or deepclust70 either: an antibody from UniProt may sit in both. +- AntiRef's filter: OAS unpaired, human, PBMC, no disease, no vaccine. There is no donor, no nucleotide sequence, no germline call, and no SHM mask; a donor holdout needs a rebuild. +- Light chains are low-diversity and easy for a model; read heavy-only CDR3 loss for the signal. +- In a production decay, 3% of rows exceeds AntiRef99's 316M, so the antibodies repeat. A pilot that subsamples takes a subset of shards. diff --git a/datasets/oplm-antibody-sequences/recipe/build b/datasets/oplm-antibody-sequences/recipe/build new file mode 100755 index 0000000..cccbde1 --- /dev/null +++ b/datasets/oplm-antibody-sequences/recipe/build @@ -0,0 +1,250 @@ +#!/usr/bin/env -S python3 -I +"""recipe/build : OPLM's antibody sequences from AntiRef v2022.12.14 (Zenodo 7474336 and 7474658). Held-out +AntiRef90 clusters, singletons and gte2, are numbered with ANARCII and written as eval and test with `locus` and an +abstar-format `cdr_mask_aa`; every other cluster goes to two train sets, AntiRef96 clusters with their AntiRef99 +members and AntiRef99 representatives, sharded as OPLM's uniref70 and deepclust70 are. + +Stages under /.work/, each skipped when a rerun finds it finished. Prints `held out failed numbering` lines, +then `rows` and `locus` lines last. +""" +import gzip +import hashlib +import itertools +import os +import shutil +import sys +import tomllib +import urllib.request +from concurrent.futures import ThreadPoolExecutor +from pathlib import Path + +import duckdb +import pyarrow as pa +import pyarrow.compute as pc +import pyarrow.parquet as pq +from anarcii import Anarcii + +# file: (Zenodo record, bytes, MD5) +SOURCE = {"antiref_cluster-manifest.csv.gz": (7474336, 14054641207, "18fcf6376702a2d23e2acdc32f1fd40a"), + "antiref100.fasta.gz": (7474658, 18496427276, "4a82d4be0c493377668fac9730aca866")} +STREAMS = 32 +LEVELS = ["antiref100", "antiref99", "antiref98", "antiref96", "antiref94", "antiref92", "antiref90"] +REGIONS = [(27, 38, "1"), (56, 65, "2"), (105, 117, "3")] # IMGT; everything else is framework, 0 +LOCI = {"H": "IGH", "K": "IGK", "L": "IGL"} +STRATA = {"singletons": "size = 1", "gte2": "size >= 2"} +# a heavy chain whose IMGT regions are FWR1 25, CDR1 8, FWR2 17, CDR2 8, FWR3 38, CDR3 21 (with insertions), FWR4 11 +KNOWN = ("QVQLVQSGAEVKKPGASVKVSCKASGYTFTSYAMYWVRQAPGQRLEWMGWINARNGNTKYSQKFQGRVTITRDTSASTAYMELSSLRSEDTAVYYCARSPYSSSWPGYYYYYYGMDVWGQGTTVTVSS", + "00000000000000000000000001111111100000000000000000222222220000000000000000000000000000000000000033333333333333333333300000000000") + +params = tomllib.loads((Path(__file__).parent / "params.toml").read_text()) +out = Path(sys.argv[1]).resolve() +work = out / ".work" +cpus = len(os.sched_getaffinity(0)) # the job's CPUs, not the node's +seed = params["seed"] + + +def order(column: str) -> str: + """The seeded order: md5 of ':', stable across tools and versions, sortable out of core.""" + return f"md5('{seed}:' || {column})" + + +def connect() -> duckdb.DuckDBPyConnection: + con = duckdb.connect() + con.execute(f"SET threads = {cpus}; SET temp_directory = '{work / 'duckdb'}'") + return con + + +def fetch_range(url: str, part: Path, lo: int, hi: int) -> None: + request = urllib.request.Request(url, headers={"Range": f"bytes={lo}-{hi}"}) + with urllib.request.urlopen(request) as r, part.open("r+b") as f: + if r.status != 206: + raise RuntimeError(f"{url} answered {r.status} to a range request") + f.seek(lo) + while chunk := r.read(1 << 22): + f.write(chunk) + if f.tell() != hi + 1: + raise RuntimeError(f"{url} bytes {lo}-{hi} ended at {f.tell()}") + + +def download(dest: Path) -> None: + """Each file in STREAMS parallel ranges: Zenodo serves about 1 MB/s to one stream, 17 MB/s to 32.""" + dest.mkdir(parents=True, exist_ok=True) + for name, (record, size, md5) in SOURCE.items(): + path = dest / name + if path.is_file() and path.stat().st_size == size: + continue + url, part = f"https://zenodo.org/api/records/{record}/files/{name}/content", path.with_suffix(".part") + with part.open("wb") as f: + f.truncate(size) + bounds = [(i * size // STREAMS, (i + 1) * size // STREAMS - 1) for i in range(STREAMS)] + with ThreadPoolExecutor(STREAMS) as pool: + list(pool.map(lambda b: fetch_range(url, part, *b), bounds)) + digest = hashlib.md5() + with part.open("rb") as f: + while chunk := f.read(1 << 24): + digest.update(chunk) + if digest.hexdigest() != md5: + part.unlink() + sys.exit(f"MD5 {digest.hexdigest()} != {md5} for {name}") + part.rename(path) + + +def stage(work: Path, limit: int) -> None: + """manifest.parquet (the seven levels) and sequences.parquet (sequence_id, sequence). With `limit`, only the first + `limit` AntiRef90 clusters in the seeded order, whole, so every representative's own row comes along.""" + manifest, sequences = work / "manifest.parquet", work / "sequences.parquet" + con = connect() + if not manifest.is_file(): + columns = ", ".join(f"'{c}': 'VARCHAR'" for c in LEVELS) + csv = f"read_csv('{work / 'antiref_cluster-manifest.csv.gz'}', header = true, columns = {{{columns}}})" + select = (f"WITH m AS (SELECT * FROM {csv}), keep AS (SELECT DISTINCT antiref90 FROM m ORDER BY {order('antiref90')} " + f"LIMIT {limit}) SELECT m.* FROM m SEMI JOIN keep USING (antiref90)") if limit else f"SELECT * FROM {csv}" + con.execute(f"COPY ({select}) TO '{manifest}.part' (FORMAT parquet, COMPRESSION zstd)") + os.rename(f"{manifest}.part", manifest) + if not sequences.is_file(): + wanted = set(con.sql(f"SELECT antiref100 FROM '{manifest}'").fetchnumpy()["antiref100"]) if limit else None + schema = pa.schema([("sequence_id", pa.string()), ("sequence", pa.string())]) + with gzip.open(work / "antiref100.fasta.gz", "rt") as f, pq.ParquetWriter(f"{sequences}.part", schema, compression="zstd") as w: + ids, seqs = [], [] + for line in f: + if line.startswith(">"): + sid = line[1:].strip() + continue + if wanted is None or sid in wanted: + ids.append(sid) + seqs.append(line.strip()) + if len(ids) == 1_000_000: + w.write_table(pa.table([ids, seqs], schema=schema)) + ids, seqs = [], [] + w.write_table(pa.table([ids, seqs], schema=schema)) + os.rename(f"{sequences}.part", sequences) + lacking = con.sql(f"SELECT count(*) FROM '{manifest}' ANTI JOIN '{sequences}' s ON antiref100 = s.sequence_id").fetchall()[0][0] + if lacking: + sys.exit(f"{lacking} manifest ids lack a sequence") + if not limit: + unlisted = con.sql(f"SELECT count(*) FROM '{sequences}' ANTI JOIN '{manifest}' m ON sequence_id = m.antiref100").fetchall()[0][0] + if unlisted: + sys.exit(f"{unlisted} sequences lack a manifest row") + + +def cdr_mask(numbering: list, sequence: str) -> str | None: + """abstar's cdr_mask_aa from ANARCII's IMGT numbering: one digit per residue, or None when the numbered residues + are not exactly `sequence` or there is no CDR3.""" + residues = [(pos, aa) for (pos, _ins), aa in numbering if aa != "-"] + if "".join(aa for _, aa in residues) != sequence: + return None + mask = "".join(next((c for lo, hi, c in REGIONS if lo <= pos <= hi), "0") for pos, _ in residues) + return mask if "3" in mask else None + + +def heldout_sets(work: Path, n: int) -> dict[str, list[dict]]: + """The first n numbered representatives of each stratum, in the seeded order, are eval; the next n test.""" + anarcii = Anarcii(seq_type="antibody", mode="accuracy", cpu=True, ncpu=cpus) + known = anarcii.number({"known": KNOWN[0]})["known"] + if known["chain_type"] != "H" or cdr_mask(known["numbering"], KNOWN[0]) != KNOWN[1]: + sys.exit(f"ANARCII's known answer moved: {known['chain_type']} {cdr_mask(known['numbering'], KNOWN[0])}") + con, sets = connect(), {} + for stratum, where in STRATA.items(): + rows = con.cursor().execute( + f"WITH c AS (SELECT antiref90, count(*) AS size FROM '{work / 'manifest.parquet'}' GROUP BY antiref90) " + f"SELECT c.antiref90, s.sequence FROM c JOIN '{work / 'sequences.parquet'}' s ON s.sequence_id = c.antiref90 " + f"WHERE {where} ORDER BY {order('c.antiref90')}") + kept, failed = [], 0 + while len(kept) < 2 * n and (pool := rows.fetchmany(int(2.4 * n))): + numbered = anarcii.number(dict(pool)) + for sid, sequence in pool: + if len(kept) == 2 * n: + break + result = numbered.get(sid, {"chain_type": "F"}) + mask = cdr_mask(result["numbering"], sequence) if result["chain_type"] in LOCI else None + if mask is None: + failed += 1 + else: + kept.append({"sequence_id": sid, "sequence": sequence, "locus": LOCI[result["chain_type"]], "cdr_mask_aa": mask}) + if len(kept) < 2 * n: + sys.exit(f"stratum {stratum}: {len(kept)} of {2 * n} after {len(kept) + failed} candidates") + print(f"held out failed numbering {stratum}: {failed}") + sets[f"eval_{stratum}"], sets[f"test_{stratum}"] = kept[:n], kept[n:] + return sets + + +def write_shards(batches: pa.RecordBatchReader, folder: Path, shard_rows: int) -> None: + tmp = folder.with_suffix(".part") + shutil.rmtree(tmp, ignore_errors=True) + tmp.mkdir(parents=True) + def write(table: pa.Table, i: int) -> None: + pq.write_table(table, tmp / f"shard-{i:05d}.parquet", row_group_size=10_000, compression="zstd") + + # DuckDB's reader hands back its own batch sizes (79,872 rows from a sort), so shards are cut here + held, i = [], 0 + for batch in batches: + held.append(batch) + while sum(b.num_rows for b in held) >= shard_rows: + table = pa.Table.from_batches(held) + write(table.slice(0, shard_rows), i) + held, i = table.slice(shard_rows).to_batches(), i + 1 + if sum(b.num_rows for b in held): + write(pa.Table.from_batches(held), i) + tmp.rename(folder) + + +def clusters(reader: pa.RecordBatchReader): + """One row per AntiRef96 cluster from rows (antiref96, antiref99, sequence) sorted so that each cluster's rows are + together, its representative first. A batch's last cluster may go on in the next batch, so it waits for it.""" + held = reader.schema.empty_table() + for batch in itertools.chain(reader, [None]): + t = held if batch is None else pa.concat_tables([held, pa.Table.from_batches([batch])]).combine_chunks() + if not t.num_rows: + continue + c = t["antiref96"].combine_chunks() + starts = [0, *pc.add(pc.indices_nonzero(pc.not_equal(c[1:], c[:-1])), 1).to_pylist()] + end = t.num_rows if batch is None else starts.pop() + held, t = t.slice(end), t.slice(0, end) + if not starts: + continue + ids, seqs, offsets = t["antiref99"].combine_chunks(), t["sequence"].combine_chunks(), pa.array([*starts, end], pa.int32()) + stray = pc.not_equal(c.take(starts), ids.take(starts)) + if pc.any(stray).as_py(): + sys.exit(f"AntiRef96 representative {c.take(starts).filter(stray)[0]} is not among its own AntiRef99 members") + yield pa.record_batch({"sequence_id": c.take(starts), "sequence": seqs.take(starts), + "member_ids": pa.ListArray.from_arrays(offsets, ids), + "member_sequences": pa.ListArray.from_arrays(offsets, seqs)}) + + +def train(work: Path, shard_rows: int) -> None: + """Every manifest row outside the held-out AntiRef90 clusters, as AntiRef96 clusters with their AntiRef99 members + (representative first) and as AntiRef99 representatives, each in the seeded order.""" + con = connect() + con.execute(f"CREATE VIEW kept AS SELECT * FROM '{work / 'manifest.parquet'}' ANTI JOIN '{work / 'heldout.parquet'}' h " + f"USING (antiref90)") + con.execute(f"CREATE VIEW seqs AS SELECT * FROM '{work / 'sequences.parquet'}'") + if not (out / "antiref99" / "train").is_dir(): + write_shards(con.sql(f"SELECT d.antiref99 AS sequence_id, s.sequence FROM (SELECT DISTINCT antiref99 FROM kept) d " + f"JOIN seqs s ON s.sequence_id = d.antiref99 ORDER BY {order('d.antiref99')}").to_arrow_reader(), + out / "antiref99" / "train", shard_rows) + if not (out / "antiref96" / "train").is_dir(): + # one sort and a streaming group: an ordered list() over 300M sequences spilled 90 GB and stalled + rows = con.sql(f"SELECT m.antiref96, m.antiref99, s.sequence FROM (SELECT DISTINCT antiref96, antiref99 FROM kept) m " + f"JOIN seqs s ON s.sequence_id = m.antiref99 " + f"ORDER BY {order('m.antiref96')}, m.antiref96, m.antiref99 = m.antiref96 DESC, {order('m.antiref99')}") + write_shards(clusters(rows.to_arrow_reader()), out / "antiref96" / "train", shard_rows) + + +download(work) +stage(work, params["limit"]) +if not (work / "heldout.parquet").is_file(): + for name, rows in heldout_sets(work, params["heldout"]).items(): + pq.write_table(pa.Table.from_pylist(rows), out / f"{name}.parquet", compression="zstd") + ids = [r["sequence_id"] for name in ("eval", "test") for s in STRATA for r in pq.read_table(out / f"{name}_{s}.parquet").to_pylist()] + pq.write_table(pa.table({"antiref90": ids}), work / "heldout.parquet") +train(work, params["shard_rows"]) + +con = connect() +for folder in ("antiref96", "antiref99"): + print(f"rows {folder}/train: {con.sql(f"SELECT count(*) FROM '{out / folder / 'train'}/*.parquet'").fetchall()[0][0]}") +for name in ("eval", "test"): + for s in STRATA: + path = out / f"{name}_{s}.parquet" + print(f"rows {path.name}: {pq.ParquetFile(path).metadata.num_rows}") + for locus, count in con.sql(f"SELECT locus, count(*) FROM '{path}' GROUP BY locus ORDER BY locus").fetchall(): + print(f"locus {path.name} {locus}: {count}") diff --git a/datasets/oplm-antibody-sequences/recipe/params.toml b/datasets/oplm-antibody-sequences/recipe/params.toml new file mode 100644 index 0000000..ce9b566 --- /dev/null +++ b/datasets/oplm-antibody-sequences/recipe/params.toml @@ -0,0 +1,5 @@ +pulled = "2026-11-09" +seed = 1 +heldout = 5000 +shard_rows = 1000000 +limit = 0 # AntiRef90 clusters, whole, for a developer's dry run; 0 is all, and the published version has 0 diff --git a/datasets/oplm-antibody-sequences/recipe/test b/datasets/oplm-antibody-sequences/recipe/test new file mode 100755 index 0000000..5f797f7 --- /dev/null +++ b/datasets/oplm-antibody-sequences/recipe/test @@ -0,0 +1,82 @@ +#!/usr/bin/env -S python3 -I +"""recipe/test : exit 0 iff holds a version of oplm-antibody-sequences. Reads parquet footers, the four +held-out files, and the first and last shard of each train folder; one line per failure, naming the check and the +first offending sequence_id.""" +import sys +import tomllib +from pathlib import Path + +import polars as pl +import pyarrow.parquet as pq + +params = tomllib.loads((Path(__file__).parent / "params.toml").read_text()) +FLOOR = {"antiref96": 189_503_281, "antiref99": 284_384_885} # rows: 90% of the first full build's +HELDOUT = ["eval_singletons", "eval_gte2", "test_singletons", "test_gte2"] +TRAIN_COLUMNS = {"antiref96": ["sequence_id", "sequence", "member_ids", "member_sequences"], + "antiref99": ["sequence_id", "sequence"]} +HELDOUT_COLUMNS = ["sequence_id", "sequence", "locus", "cdr_mask_aa"] + +out, problems = Path(sys.argv[1]), [] +held = {} +for name in HELDOUT: + path = out / f"{name}.parquet" + if not path.is_file(): + problems.append(f"files: {name}.parquet is missing") + continue + df = pl.read_parquet(path) + if df.columns != HELDOUT_COLUMNS: + problems.append(f"columns: {name}.parquet has {df.columns}, not {HELDOUT_COLUMNS}") + continue + if df.height != params["heldout"]: + problems.append(f"rows: {name}.parquet has {df.height}, not {params['heldout']}") + bad = { + "locus": ~pl.col("locus").is_in(["IGH", "IGK", "IGL"]), + "cdr_mask_aa length": pl.col("cdr_mask_aa").str.len_chars() != pl.col("sequence").str.len_chars(), + "cdr_mask_aa digits": ~pl.col("cdr_mask_aa").str.contains(r"^[0-3]+$"), + "cdr_mask_aa one CDR3": ~pl.col("cdr_mask_aa").str.contains(r"^[0-2]*3+[0-2]*$"), + } + for check, expr in bad.items(): + hit = df.filter(expr.fill_null(True)) + if hit.height: + problems.append(f"{check}: {hit.height} rows in {name}.parquet, first {hit['sequence_id'][0]}") + held[name] = set(df["sequence_id"]) + print(f"{name}.parquet {df.height} rows") + +edges = {} +for folder, columns in TRAIN_COLUMNS.items(): + shards = sorted((out / folder / "train").glob("*.parquet")) + if not shards: + problems.append(f"files: {folder}/train has no shards") + continue + names = [p.name for p in shards] + if names != [f"shard-{i:05d}.parquet" for i in range(len(shards))]: + problems.append(f"files: {folder}/train's shards are not shard-00000.parquet onward without gaps") + counts = [pq.ParquetFile(p).metadata.num_rows for p in shards] + if any(n != params["shard_rows"] for n in counts[:-1]): + problems.append(f"rows: a {folder}/train shard before the last is not {params['shard_rows']} rows") + if sum(counts) < FLOOR[folder]: + problems.append(f"rows: {folder}/train has {sum(counts)}, under its floor {FLOOR[folder]}") + for p in {shards[0], shards[-1]}: + df = pl.read_parquet(p) + if df.columns != columns: + problems.append(f"columns: {folder}/train/{p.name} has {df.columns}, not {columns}") + continue + edges[f"{folder}/train/{p.name}"] = set(df["sequence_id"]) + if folder == "antiref96": + bad = df.filter( + (pl.col("member_ids").list.first() != pl.col("sequence_id")) + | (pl.col("member_sequences").list.first() != pl.col("sequence")) + | (pl.col("member_ids").list.len() != pl.col("member_sequences").list.len()) + ) + if bad.height: + problems.append(f"members: {bad.height} rows in {folder}/train/{p.name}, first {bad['sequence_id'][0]}") + print(f"{folder}/train {len(shards)} shards {sum(counts)} rows") + +seen = {} +for where, ids in [*held.items(), *edges.items()]: + for name, other in seen.items(): + if not (where in edges and name in edges) and ids & other: + problems.append(f"sequence_id: {next(iter(ids & other))} is in {name} and {where}") + seen[where] = ids +print("\n".join(problems) or "ok") +sys.exit(1 if problems else 0) diff --git a/results/oplm-antibody-sequences/1.0.0.txt b/results/oplm-antibody-sequences/1.0.0.txt new file mode 100644 index 0000000..d20e1e0 --- /dev/null +++ b/results/oplm-antibody-sequences/1.0.0.txt @@ -0,0 +1,577 @@ +--- +dataset: oplm-antibody-sequences +version: 1.0.0 +hub: brineylab/oplm-antibody-sequences +branch: candidate-3cacf1d0d75b5c946fca7eba41f06d1913b9917f +commit: eca1d2f3bfffb9f607e8e033d21ef6f18db98ed6 +tree: 3cacf1d0d75b5c946fca7eba41f06d1913b9917f +tools: datascience@1.2.0 +runner: sbatch --partition=cpu --cpus-per-task=16 --mem=256G --time=03:00:00 +host: bcr.scripps.edu +started: 2026-11-10T20:49:02Z +exit: 0 +--- +pyxis: importing docker image: ghcr.io#brineylab/toolshed/datascience:1.2.0 +pyxis: imported docker image: ghcr.io#brineylab/toolshed/datascience:1.2.0 +held out failed numbering singletons: 6 +Sequence could not be numbered. Contains an invalid residue: 'n' +Sequence could not be numbered. Contains an invalid residue: 'n' +Sequence could not be numbered. Contains an invalid residue: 'n' +held out failed numbering gte2: 5 +rows antiref96/train: 210559202 +rows antiref99/train: 315983206 +rows eval_singletons.parquet: 5000 +locus eval_singletons.parquet IGH: 2216 +locus eval_singletons.parquet IGK: 1897 +locus eval_singletons.parquet IGL: 887 +rows eval_gte2.parquet: 5000 +locus eval_gte2.parquet IGH: 2875 +locus eval_gte2.parquet IGK: 1305 +locus eval_gte2.parquet IGL: 820 +rows test_singletons.parquet: 5000 +locus test_singletons.parquet IGH: 2228 +locus test_singletons.parquet IGK: 1911 +locus test_singletons.parquet IGL: 861 +rows test_gte2.parquet: 5000 +locus test_gte2.parquet IGH: 2801 +locus test_gte2.parquet IGK: 1377 +locus test_gte2.parquet IGL: 822 +eval_singletons.parquet 5000 rows +eval_gte2.parquet 5000 rows +test_singletons.parquet 5000 rows +test_gte2.parquet 5000 rows +antiref96/train 211 shards 210559202 rows +antiref99/train 316 shards 315983206 rows +ok +file antiref96/train/shard-00000.parquet 118918481 c79e7e20e3b6fe3cef97b4be368d11ac9da5ccb664665bde311afcb65442f43a +file antiref96/train/shard-00001.parquet 118794041 a278f8a9ed49aa556cb6dfb1a544f32efd5cc1ead36dc29068d44da0135e9744 +file antiref96/train/shard-00002.parquet 119315189 14b8b52b4f11ea859ef9fe06b7bb32342567e5fe90189e63c3564234f0ac9438 +file antiref96/train/shard-00003.parquet 118851122 ae8c54c8e32ee51423bc03eebaa60652bb9b40619325488c928087c784ca62c9 +file antiref96/train/shard-00004.parquet 119127859 56d6dfaf37e656389352f45d6d6e9b3a6172dc97c52781842d6772d20a86e1c4 +file antiref96/train/shard-00005.parquet 118942240 8e7149a4f6a97a6d63aea8a1c4dfc70de4022da10e8708601dec64c6fe4ac2c7 +file antiref96/train/shard-00006.parquet 118853605 5f9136a322e519915b0220ec43c0a3724926a67bc4e03c944ca3bc34ce73d7e2 +file antiref96/train/shard-00007.parquet 119115202 d95e1d7a775514f234dae693fefc7bab4bfbcbe11a43bec3bf5b4f1f0d30ced7 +file antiref96/train/shard-00008.parquet 118571699 d4249a1875a5a02d2d9755f70699258358c0068ec0c90637298453aef9d9b287 +file antiref96/train/shard-00009.parquet 119082870 9f682f6bcff87c8ca5fe455baafca03b7c7a580fc8da436e035350d65cc61eba +file antiref96/train/shard-00010.parquet 118800663 400fdee0487767af3b7cc71c0cae8a1f3f84b98f79b6b3785fa25f99be3315f1 +file antiref96/train/shard-00011.parquet 118697567 32508cb3a65d814eaf343412c4974382a9415053ead434f7fd346b450bb23fda +file antiref96/train/shard-00012.parquet 119087492 2d385fbf65225c59e1a4e9d60e509360c09ba6a76d0cafd12ee0c5852b9e7e33 +file antiref96/train/shard-00013.parquet 118900476 49cb81fde2ef27f160e3f68db1ffad236e177954a70a9ee797c8d4af88a5fa3f +file antiref96/train/shard-00014.parquet 118493004 26c1daaed181b58dc1c623d19e422a2d75b4b1322b6a9bc4ee02418d50538d66 +file antiref96/train/shard-00015.parquet 118841143 9397d13bd3fecde794c4b03223cd3eaf15541c35bdcfdb477552dbbb009d7313 +file antiref96/train/shard-00016.parquet 119076570 aa688d236db1c73f0722a14220b40184d93a19ce9aba802294dcbf767f7299af +file antiref96/train/shard-00017.parquet 119085646 8127b49530b18fd412cd598aadb797d0b91100caa91c917091e54dda288a0850 +file antiref96/train/shard-00018.parquet 118787562 5611b9ff093999b3cecdbc629825966497a4a54f9d4667e2dc080022f6b9633c +file antiref96/train/shard-00019.parquet 118532495 c80e9598ff5d55b2e2e79bec788d0c5d25cf160eef9cb6ee1d6cc894df342012 +file antiref96/train/shard-00020.parquet 119315308 e800b3b365dfad781ca6b34ca66e2321627476c5550bb0428c32e3ae3a0e086d +file antiref96/train/shard-00021.parquet 118875745 7c9d60eaab2460f1a4c121b352973a80b1f2c7be7e7a6f801513f40af5920d31 +file antiref96/train/shard-00022.parquet 118655480 cf8f65af31adbfa7acb0c64e56bf9a0dbb4ffc0f4dfc0bbc29c0cb75a4e9eb1d +file antiref96/train/shard-00023.parquet 119206417 24e632c81a60ea9fa8e192dffa9dd8c89e13e3a2c05e558a85c73394a116cfb7 +file antiref96/train/shard-00024.parquet 119295073 25ee8aa5ddeea81afded01871fb6436b5016d0475c322920b20d2e605b4d192f +file antiref96/train/shard-00025.parquet 118993291 3babd06802804fa9576f59d16d4a1aa198af3957d1a7ff0bf7a71fe2ae8835ab +file antiref96/train/shard-00026.parquet 118782673 3a511717552dda620d7255305b68776f0c9bfe3e41805275506127c51aa831c4 +file antiref96/train/shard-00027.parquet 118828266 8d116d5af50f04d13c303f7042c632ce246454b61c40496dc1f8b85d4a548c76 +file antiref96/train/shard-00028.parquet 118905275 d62024516b56dda835b67c9fa9c63921ca8a60fd96c627935660249ba5e9fccb +file antiref96/train/shard-00029.parquet 118853136 17923552e25405856ee6055018ae8d96a781f0100263e65659c05a29418d414e +file antiref96/train/shard-00030.parquet 119657279 fd567cbd2d3bbf3811f614d60b7f2341986348addae0397b4eae487efce363b6 +file antiref96/train/shard-00031.parquet 119211616 af593e9f4daaeb87dcad93596a7bb79d74d71fd8c3e33ace1bd39d3afa447830 +file antiref96/train/shard-00032.parquet 118973491 ea8e49ada67fe9d5ce436fcaf984f588bb87374b7c999aee0f87e559112e0ae0 +file antiref96/train/shard-00033.parquet 119394931 202e77c041218948748541b7b57756008c180cc85e15593ab3c0246c404120b1 +file antiref96/train/shard-00034.parquet 118716911 0d19d753260aad759c68833e58ade1d4ceaeff2dbff02dacaf6dc9755fea56b3 +file antiref96/train/shard-00035.parquet 118890414 b2e7df4c69543957b9088bc52b1f67685f84dc82b3c0e35f7511237e9913e366 +file antiref96/train/shard-00036.parquet 118991195 6286ea5d6e74eebe643503b9d533c496bc9e82a2c04123d883d95d25704188b4 +file antiref96/train/shard-00037.parquet 118745841 b383e61ec2e46adf573bbcfc8cc9cdd2f1554a66554c0e1fb0d421a856561bf0 +file antiref96/train/shard-00038.parquet 119384642 64e78364e75eb868f1fdcf4d3491892d27bb6a8ddfe801c2bcdad774e5572a9c +file antiref96/train/shard-00039.parquet 118806476 79324b84f708434b21ad9ebdb3b6d70f8fc8bdc1f50c3e5635b205fc6a2fd57b +file antiref96/train/shard-00040.parquet 118984484 21df91138a67b3bcdd0932da416734f84fe86c497b4b0e4c25cda5d7f2b06387 +file antiref96/train/shard-00041.parquet 119016344 4282d844cbdca1552b24c0e5cf1ce691d3c4e55ed5f6acfa90c65a6ec994380a +file antiref96/train/shard-00042.parquet 118896793 dc5b7e4091d720a16fc642af4168eed37eed039cc6a9f54e2ea4752456faebb5 +file antiref96/train/shard-00043.parquet 119238983 f36f2f4ea6bdb382a7de2e27b99284605aa479741d369b34341215b6d392b7f6 +file antiref96/train/shard-00044.parquet 118680845 34bed188eeb2d2fcb2ff2cdccc0cdaee173a033387ba67b250c460925940bbfa +file antiref96/train/shard-00045.parquet 118943311 4d181b45a7350242e992e71cc29caac6659ec5be5b6d18f19e9c24e075e60e3f +file antiref96/train/shard-00046.parquet 119408045 ca4d4b9e6e372ab9821e689bb7ec0c357f2c1537df490670d2396085884a6158 +file antiref96/train/shard-00047.parquet 119010812 19e9e0701d4305f6a83de8f55f41389d421e13972f4fefedc4102232205b099f +file antiref96/train/shard-00048.parquet 118793607 95e47a75916f9e6d6c5820d28eed429b487c5632e635e9a5e82a45202977d880 +file antiref96/train/shard-00049.parquet 118790142 8b6f1b526fbc6264a4ea4515521fbed2ca9c6b5d24b7651263db6dfe2edf38f3 +file antiref96/train/shard-00050.parquet 118796509 760b433c2e35dda8961ac34cba8ed9711adc7ec37cea9657a7954942ec3ca69f +file antiref96/train/shard-00051.parquet 119045115 414aadbdc119a07a5ea321750ae87b55b436e6d8aaef095b142fd4cb5bcacc07 +file antiref96/train/shard-00052.parquet 118876201 460c312c4af0d953ae506414698b5da67e70a72e1b897f97f7079c8f6ab59e40 +file antiref96/train/shard-00053.parquet 120165809 1e2eeb861282bf8bde3ba922b42b15f67034743e1fd01b299fd8a022c8a8ca12 +file antiref96/train/shard-00054.parquet 119246795 dc6c02a8986a9cd963b0225fb92376dc7aa8e70f3c1c48734df4ba64420854f8 +file antiref96/train/shard-00055.parquet 119034146 9f495aec1803178b9788b9067825edb95abb3a8ba3524323f67da8b6a5a87cc6 +file antiref96/train/shard-00056.parquet 119099870 d7026fa49e20506f323c038d5019753bc5ee68a446b3279560bd95303537abb6 +file antiref96/train/shard-00057.parquet 118792853 c8b9b3ec41563cf625591f0fff85b59bb437cd92f6125a366a03db93c30cefee +file antiref96/train/shard-00058.parquet 119117274 e9cc260f353ae08baaaf30f0236ca8513454d62c4554a705c4a1007ea9b7a011 +file antiref96/train/shard-00059.parquet 118855140 021862a0e11ba0e9adaa43ffe18ae0c2db9ff0bcfe24feb344f4c7b3dcfa2197 +file antiref96/train/shard-00060.parquet 118980494 e05a359aab9485cab7dc7a7297be01f3e2709298524bf8bbb4e096fa2c5f80e8 +file antiref96/train/shard-00061.parquet 118572604 9d6579467a8f439e0be231becb57fb348631ce7b9098052b83238bb5cf5a8278 +file antiref96/train/shard-00062.parquet 118983417 45036686ea338f1868487c7af21c65589c76e912460a7d64d1dd987c9782201e +file antiref96/train/shard-00063.parquet 119282232 0db4cd09b29b463489ee20ad40b8829c211ce1b9af01fa2d6f0ddc16a5050f9f +file antiref96/train/shard-00064.parquet 119426066 ae337419c418f6ba21af75b4effe976b98999e24b7a6c4ce6ab921fc110d219c +file antiref96/train/shard-00065.parquet 119107601 92113a399ab549a391af8c9eb8f0562a4d6ebafc6eaf139c38e5fcbe28eb3e9c +file antiref96/train/shard-00066.parquet 119100331 2d1c1f838da97f4560a663e1a777a0f562c3d7be88a1d4f05727ddaa42a47eec +file antiref96/train/shard-00067.parquet 118965769 f7108ad75ce02005702d0879592377c0b6594634656a2a88eb05fa650fb16807 +file antiref96/train/shard-00068.parquet 119231307 e082cf2f44076ab96566e5bb0d140cbbb6bf8a38aceff0314a546a20709cc671 +file antiref96/train/shard-00069.parquet 118775660 1b0473ef3c755cc35d384409729e48be536734c81d41f5417a5cfc057c3f3773 +file antiref96/train/shard-00070.parquet 118905689 8af07f0ca99a097b1f2f79a400885c60b4ef23e64e881642da463f138218bf6d +file antiref96/train/shard-00071.parquet 119190004 4c8a5de8915205d29c7f2ea355322b4d6c4c132c15709eb137ed571e6eabadb1 +file antiref96/train/shard-00072.parquet 119168705 1711baaaf7e579425348ebef741453da0baf9254ab4f6dcb3c779893c59f2635 +file antiref96/train/shard-00073.parquet 118840280 733cb0df2464ab1ae468d1627ebd75ff9b0386decab2ee6bf4e3f1737b1d60c8 +file antiref96/train/shard-00074.parquet 119076945 78a65dd387bab9c2b2f13ad7199b5ed0540c0bbd359f81e229979b48063b0f0b +file antiref96/train/shard-00075.parquet 119185379 20b328de1bfc10c250a80c2230961ba66fa23b5445ddcc29d83708696387ecb6 +file antiref96/train/shard-00076.parquet 119074335 58d3142ca8faa1be5b298227ffcc6624b7a5b745097def4c2723c4ada45bc6e1 +file antiref96/train/shard-00077.parquet 119035362 e4e9a30e0cf6c8a05cb3d35f9e7d299008e2af15c1753cd58d1fac50928b1cf9 +file antiref96/train/shard-00078.parquet 119197183 8c3b4e1d3460a09ab1b16edde8b9fa4326fb36d312620ac9b48b2d4536128e18 +file antiref96/train/shard-00079.parquet 119000929 9ac4e7602871312d6b1fc4ddbc27ba4272fe7c2924e57a64c0072bcdc1ecc0e3 +file antiref96/train/shard-00080.parquet 118752524 36b11a0af1cda3110155d95e1cd9f28bac1a404f074b9ae53d86e0291b93a722 +file antiref96/train/shard-00081.parquet 119225683 cd873ad61f27ee3428df71f0f4ff4a579fab896e6c973b0ffe976c291f4439e5 +file antiref96/train/shard-00082.parquet 118967485 f6000688c7c682bf83cf6d9d9e3229d84dc249c16bf403e1a875928f43489d97 +file antiref96/train/shard-00083.parquet 118873385 6977dcbd13cabb17f5334830225d478f5a212650190b35c9d4183d2ac7d0ef49 +file antiref96/train/shard-00084.parquet 118265370 e2f7e3c5fc14ad19f023ca75bc86a806219f92dd92a279c96d241ac2243a6a67 +file antiref96/train/shard-00085.parquet 119332417 6224b5d4fd9a15e4d1d914c0840712e279e3b56fc52869549ce4a9b2642bec7c +file antiref96/train/shard-00086.parquet 119243272 4215416f8fa843121c80127567c51eda2363858bcf5532ee9c06b5e57ca5f420 +file antiref96/train/shard-00087.parquet 119111240 34d89c08f2bc64fdc35dc31c01372f4352b15eb7d3d3e0260456738b2953511a +file antiref96/train/shard-00088.parquet 119092927 4da4cc8b6f9067200424fc860b9175738a777ce1be3bd77c7aadd2ee6a93639e +file antiref96/train/shard-00089.parquet 119176080 1b401f72ca8486c7476f562c2a7fc427bc02d3f2a281b9195c24c5282ffa4e93 +file antiref96/train/shard-00090.parquet 119389133 04e6d5b8070f94245ece3ab7b976383f0bed6b9194a3d19380dd25ffe4c7a994 +file antiref96/train/shard-00091.parquet 118960965 84c306702e9215ae890ea9960f5c7506f431e7c98e6f880697b48aebb2a43f34 +file antiref96/train/shard-00092.parquet 118908696 8062a14c00d010df03b0a118a6ae103bc1190cf5187036cca41b34f010b24b97 +file antiref96/train/shard-00093.parquet 119600738 e3ba6504ebde0961fe76067892b436662b63d94ce941765491f650de81a1fd67 +file antiref96/train/shard-00094.parquet 119071888 69b360d0431eb8299fe75998688b09f22a58d750b4633c59b4495ded3de0cacd +file antiref96/train/shard-00095.parquet 118822827 01086314b86bd780af5d0cb9e8bcf5f9036ca88a9684c5841dd089ff24374912 +file antiref96/train/shard-00096.parquet 118702577 ec68cdede1558a289f28b12dc1a2e407b8382f965d07570d435741a9d2bdec58 +file antiref96/train/shard-00097.parquet 119311673 add33e0a4dd95e11c6ff1e47a57fd68e2722d36003c5d7436cecda11f7843a44 +file antiref96/train/shard-00098.parquet 118736548 b53bf906f911876a91c81ec83838f20c874bdff5b867585b894920052d83e2c9 +file antiref96/train/shard-00099.parquet 118843567 159c2e8629a795c33aa7d884febd732ea725c86833e1e25914c57da370c2d149 +file antiref96/train/shard-00100.parquet 118623361 e8516d627fe40e99fe7c4dacaf8e1e27452b6fa99f2a17955eb5a1a582e7080c +file antiref96/train/shard-00101.parquet 119358204 94f5de9f7339263c8c107fda1b127824d46a4c4cf6585cd60d6e601a0925a944 +file antiref96/train/shard-00102.parquet 119319636 45ad7acff964f456d36dff895a4664b86e0c1108525d0ef7f98a9704478d5c38 +file antiref96/train/shard-00103.parquet 118612415 d107ae2d1b19940053afd73249fcbda73fed0f2187e071ef0a0cb9a8b25c14a7 +file antiref96/train/shard-00104.parquet 118764404 71b60540b09069c7e68f0ed3cc5321f66005765c63d92a851a00bdb2d6f12191 +file antiref96/train/shard-00105.parquet 118582936 60f900be4142695bfdd7dd88e295075abafed177edf04675ac7780d7bb984b7f +file antiref96/train/shard-00106.parquet 119207517 56cfe59407f72110e6eb2c503c8f941086be0be33fc92d5d810885f6f5d0197b +file antiref96/train/shard-00107.parquet 119187067 6ad9623cc5f72849ea74ee87be934839c4e0c8f23f8d55180bd5936a73af12a4 +file antiref96/train/shard-00108.parquet 118766434 423d4313491dcb67d20d36e02015bd8423e9493e6c143578f9a5b24d8919d505 +file antiref96/train/shard-00109.parquet 119037932 7e638f13e2a1cdd089a747ec348600d7e1c6a676cfcc45b654e0a0c48b3a4419 +file antiref96/train/shard-00110.parquet 119407472 425ba9c195cbc529c200d1250e745ce702ec4c799a4d7fe234713b8ce49b98d1 +file antiref96/train/shard-00111.parquet 119109702 8f6c180171930e9b0c308580e6e8e92a40242e0874b8ffbf54e3d65cbffb4c50 +file antiref96/train/shard-00112.parquet 119081199 f0897a44a71fdda942f281597e73209144dce79f8874f164a22b67b72349eecd +file antiref96/train/shard-00113.parquet 119376760 6c16afd900efe65ebf14865bfce120f5f0da24c0a154679242b3e8b12f6e5653 +file antiref96/train/shard-00114.parquet 119149957 b58fb34ff2e9f6734e155b3e20a84c6891d50d6ec0317232b7a3fb99b5dfecb3 +file antiref96/train/shard-00115.parquet 118964787 c209d1c43c46a7ba1fe65867a2f30165e83f03d732fec21687f5397fb9e7b02f +file antiref96/train/shard-00116.parquet 118714104 d4eeeabed33e570719760b693f73a987e5828746b01a774e86e92604beef4a01 +file antiref96/train/shard-00117.parquet 118970294 07a7422bb0305bfeb1ba0bac1b8853bcc74f5ea675c708c95ebeb76c2b884883 +file antiref96/train/shard-00118.parquet 118849672 063c012acee5a45419c2abff4f8da31c60ee8ff556449969ce6cfcd2a5b5d1d8 +file antiref96/train/shard-00119.parquet 118597255 32d18395a2f809dc8077e71c7e027c7b1bedc1fbdba255c757538c92fc8f672d +file antiref96/train/shard-00120.parquet 118588149 818302f92e45f3e354f263e7e0b25232ae16921f0973b4b07a01198166c525ad +file antiref96/train/shard-00121.parquet 118705750 492d1c875abb34ce87747bd8b48a3f1146bde658c63864f5c376a243aec67666 +file antiref96/train/shard-00122.parquet 119285733 0e55d6187d4aefe73b5ac85998d7b9c41994b16aca3c2a3c0694b3cb69b32d39 +file antiref96/train/shard-00123.parquet 118940149 3b1f5546d7d17e6dccacdd66a7bc08c369a29e6da8969ea09df4f0be7ee5266c +file antiref96/train/shard-00124.parquet 119251943 2a3fe385e78f949cce039ed5dee59bab7c6111954349251548de8898524384d3 +file antiref96/train/shard-00125.parquet 118563180 4b78fa5b2b6a43d4281242df57ccdde8dff873a5ef03d02a23400d8144e40509 +file antiref96/train/shard-00126.parquet 119216936 6a2a83224c40343f72929fc61891ef75d95c92c7f47a27e9c8799c0c2f920b89 +file antiref96/train/shard-00127.parquet 118866347 c30c6cc8579136442d2848bc9bcbba926ef628c69241498a928e8034f3bc6c6b +file antiref96/train/shard-00128.parquet 118662180 3fa5d7e88ce54671b78199a3763100b44b6e0505a705348f3d62aab875feafc0 +file antiref96/train/shard-00129.parquet 119163542 9b543eae05e6627da5242256f77c5765290b136822d0c1a96ffb976149bfd51e +file antiref96/train/shard-00130.parquet 118866624 c8f3f0fd03c9b3b539c57f29bd09b898afc5b1cdcc1f4c667bf21e1c0d09008c +file antiref96/train/shard-00131.parquet 119118081 bf2eebba4bcb9160e20f8522e209f17dcea664174cc732f869207f631ff189d7 +file antiref96/train/shard-00132.parquet 118551722 8881123248ad3c502fad773b11de2529df3e37ed24d55b6fd21fe67f899d90d3 +file antiref96/train/shard-00133.parquet 119677018 631ffff506cc0e46742bea84f52b31dc5c180b77f664f9bccf18c706556b0d45 +file antiref96/train/shard-00134.parquet 119126170 3ff3e6f79e544c3e39c502427f4eed0ab48693c3f9874446ec0d6b589867ff4f +file antiref96/train/shard-00135.parquet 118663240 a974852204424ee3f0562dd9376c251d9de2815559f84b1207710255423c8229 +file antiref96/train/shard-00136.parquet 119119583 277181c2a8584375008331748126ae4d2003df1d75612c5a5a2be9e7750dd4a2 +file antiref96/train/shard-00137.parquet 119187766 ac5a7561dc0f5a43e91f36ec4987a0b2b6876f7c8b3b51683716c587e9ca093b +file antiref96/train/shard-00138.parquet 118697230 c531a805632bcca4f21c79d72b5abda2bb234dd661553607aadb2adcc747d743 +file antiref96/train/shard-00139.parquet 118750663 4b95b6bc4bce2a652d0f894c88b8c74a5fdb2b4eb44a39245f5fae517479ab04 +file antiref96/train/shard-00140.parquet 118894109 4832afa19e68ee2c8bdff32ade1baf261103ebddb13efc277f119975952fcfcb +file antiref96/train/shard-00141.parquet 119198552 fa32e9813774abfdebdd2ff7449a76d942aa85a5f922fb3b797044cee7003752 +file antiref96/train/shard-00142.parquet 119161616 e35dee802512bf5853a65f6d5a3a74d7cf02f8ea84d6151180b32671db198116 +file antiref96/train/shard-00143.parquet 119024901 d4cbdf4ea68704209eafe010fdd10cd8d1865c9abe4a0aa3db4b528c9efdbae7 +file antiref96/train/shard-00144.parquet 119016295 8e499863ef8ebd93ba6c37bb9e310e34e614d55f499d1855eaec9c6ad0b6f0fc +file antiref96/train/shard-00145.parquet 118934401 5abca2a6bec66a9ec1ff715aba0937caecd94fd16b2eb9bd9c317ea1d0c589f2 +file antiref96/train/shard-00146.parquet 118952281 4712b9ee509ce719bb5f30b9b1bc5fb07d1cce70aff9c747e07c14b58c2b460d +file antiref96/train/shard-00147.parquet 118611196 c7b6ead1e79e60ebc09ddbcd0ec6763ed124ff5e7b43c821a4dae72c73153bee +file antiref96/train/shard-00148.parquet 118804229 865f6add00ec5d9fe64219e41af618f382a4e85b6675e3f10f8ea15042329a4f +file antiref96/train/shard-00149.parquet 119649727 68d0bd0925fb1af6182e167d304c6ad75a3346d9a9ef2118ccb48dba816d33ff +file antiref96/train/shard-00150.parquet 119148534 cba4934a52dc2454ed6bfb7c36886b383189027bf79684d3dcbfa5dc8cc3f603 +file antiref96/train/shard-00151.parquet 119019157 df189e42bb73dabbc1adf5ae95b34bb18036e58b7163f46247c1984f81d30158 +file antiref96/train/shard-00152.parquet 118946890 72cec367fba8021dda32e9b1af177d37a75e416936d9a4f11d5a9f38cadf3a9f +file antiref96/train/shard-00153.parquet 119034222 bd3fcec5f1d7e8a6853f1d49c82ebc5340063507538b7cbdb6efc6106b275c85 +file antiref96/train/shard-00154.parquet 118523347 00674e5b8892d303409d90d2249cb16faf1dc651485de37235b8004b98f9cd14 +file antiref96/train/shard-00155.parquet 119538639 9fb34cc25611e4a67366d6d4d6c15965986bf67267021788aa3b4abd40543ad3 +file antiref96/train/shard-00156.parquet 118714178 9ea657ce9f0b38bd6280608cca529c692ce0497098b868094e6c93a442c7e9dd +file antiref96/train/shard-00157.parquet 119578928 c0fc66d653c4100fe8e8eba68fd1ed5d82d3327b07c723238c1d4d225cd02fff +file antiref96/train/shard-00158.parquet 118924141 8babc552d9734db66c3adf04990ea1037ee293a7ccf6ac6d373c3ca61642a622 +file antiref96/train/shard-00159.parquet 118611626 b85b1f3cb2949736f6f0c61bbd397ead497ddc1005d64bb7defd564699d70e1e +file antiref96/train/shard-00160.parquet 118436757 2764199a2c853b6858a390d0a8400a18e3bb8edae6bf53d1cd904751d0c2b816 +file antiref96/train/shard-00161.parquet 118804575 3efc9a429ab163fe4548346e5d4380745be025f6578907b86aa297c109998b7d +file antiref96/train/shard-00162.parquet 119068972 de54f6b8cc2674f5d735381f1149819ecb233da67307a106dcccb1c0514d8211 +file antiref96/train/shard-00163.parquet 119046165 7a843057880eae790ef5c9700565c3b0575e93808646e3b568645f781b5b327b +file antiref96/train/shard-00164.parquet 118661708 bbdb3b770079b22b7689c0d80bbdaa212ffbf1cf4087979bd10e876a674e9a06 +file antiref96/train/shard-00165.parquet 118681036 e6fe97cf6f3dc12a1b7db500f35e6d2a69cd635d8d23a46b6305113260c076a6 +file antiref96/train/shard-00166.parquet 119068262 95bf0eb7d4f330043a379e5bb51c1a17fc467ba58908981ba68b8e90633d5e8f +file antiref96/train/shard-00167.parquet 119166399 b5cb05a9976d4f7d63cf37d4165920ad174bb4f5a1181571ec5f9a6c4fed1533 +file antiref96/train/shard-00168.parquet 118831277 95440ea3b8f66ef8e4c138e9fb7633cbd6dd3d5526cf9707104d96806ec9c68e +file antiref96/train/shard-00169.parquet 118827166 b812d9df0d49b7fa6a9ae02ba30bd5db6b69835ad329560635dedd8e4a78f174 +file antiref96/train/shard-00170.parquet 119062019 4c18990c8512eb70a5d256a21ca656a9a625d4da245f0a503eb0572cb06c7711 +file antiref96/train/shard-00171.parquet 119099994 b652a215567b32ea5736c26078a431f3832729d84b4c845de54b953386097e6e +file antiref96/train/shard-00172.parquet 118787943 e21c217091320db2f3954da8180bbe1bf2da777b91603c3e31e7eef5e90c8d98 +file antiref96/train/shard-00173.parquet 119112504 1e0e1544b33055065abce1e9a1a73327d5fbac89a12539ab95ba1a5b7ec007f7 +file antiref96/train/shard-00174.parquet 118958643 1b185036ae7bcc950ae98253abae8b5d7ebdd5608f6db4bf14d6b1a60807fba7 +file antiref96/train/shard-00175.parquet 119335127 00b910f962dbb04159e2b229ce45eef61301adfa0188e0b2d869c26649be3688 +file antiref96/train/shard-00176.parquet 118931782 eee7c719b1ba5cf0a6ac06e1ab6f34337898cc0aefb12d2305113e9875aea04b +file antiref96/train/shard-00177.parquet 118865554 a255c0d1c076f40c560e33729e59837c61436a15a26da560ccf3dd4873a9a86c +file antiref96/train/shard-00178.parquet 118670894 1dc8a29d8f9bf3088484435bec38efc0e13eb39f529e5e61186f06973c49a494 +file antiref96/train/shard-00179.parquet 119287936 f58ad77bbe695062731abf390b3e9a1582ebffcef029d954e67533d76a23faf8 +file antiref96/train/shard-00180.parquet 118924856 342d67c5655cc5af93ec8a0db2bd19649654d01be306892d503502fc19c0804c +file antiref96/train/shard-00181.parquet 119368633 dac923d45da14307d894047489e9664dbe7c03041bb23b5b961cf57924940eac +file antiref96/train/shard-00182.parquet 119139472 1d4a89eeff41672393051aa8c43374741bd3b89476c9a1c2b8b102b73326cc7b +file antiref96/train/shard-00183.parquet 119474513 d8f37987c9da7e18e1b6c5916c6973960b31a976db5dd816bc19253647d67889 +file antiref96/train/shard-00184.parquet 119587346 04177d39d55d3b0b22dbdb9c462f5a66d98b71b81dfa37efa6aac9cf2c79c94d +file antiref96/train/shard-00185.parquet 118431724 be8513f7e395147a3abb20d981200cfefaea13c8625f0ef41f3ff8100db6358a +file antiref96/train/shard-00186.parquet 118787749 dcdc506870032989b0dd5ab3e4713d966c05719629af9098ab308f524cd89525 +file antiref96/train/shard-00187.parquet 119891496 5513c3502f19fe10208c91a72e0ae4d8715995f1711aaeaad6f528aee5d728da +file antiref96/train/shard-00188.parquet 118909759 4a35ca79ea806ed511ab2a7034cc023df489ee2184da8a041adb44970bafb542 +file antiref96/train/shard-00189.parquet 119549590 4f5dbc91fadfce4ecaf1a0cd60ffe21a9b38b60f3d4042055fc8c9e46418426f +file antiref96/train/shard-00190.parquet 119081373 e8a2f17a1f8e085ef0b60546068e9dae5f4704d630676ab4f5313b54e4192524 +file antiref96/train/shard-00191.parquet 118627763 a0d11371e49db9be5a74779be46ed68729d8a00c04d95efc48abbbbae3db2e58 +file antiref96/train/shard-00192.parquet 119376744 8c84020de713f069aad796ac5fdfeec75112393bc14517abd6cbe2e8026e5518 +file antiref96/train/shard-00193.parquet 119334833 502acc4c532528090eb486ee55aa740d6281defa952e41889abe1cda0633ef7b +file antiref96/train/shard-00194.parquet 118896614 c03f91718a08688a0e1b25ba18c333880a5af1c7dd2bcc234c9a735e4deb852f +file antiref96/train/shard-00195.parquet 119209305 aef21d1c9cb565c088d9736f8a7ec6097650536529de4d7115a3c0381a228f98 +file antiref96/train/shard-00196.parquet 119150120 2e06b147c466f2c228dda423d08cd343e602693e198fcf82157a27bb74852fc4 +file antiref96/train/shard-00197.parquet 119258475 f8ccce9eb59cfab07dfc71d3964eecb64312f741e7fb803e47d4df7f0622ac68 +file antiref96/train/shard-00198.parquet 119362166 efe987b085355965fc712a459497ae551acc4a371916b520ed27d1de4ea85c10 +file antiref96/train/shard-00199.parquet 118515568 f070edbe239586263d81b431172ea8fb55a988fb565751295b35d52690491837 +file antiref96/train/shard-00200.parquet 118894174 cfd38bc489817faa884af1714e55c17d9381562ba4c6954b860ca73a25347117 +file antiref96/train/shard-00201.parquet 119088233 2c9821566e7adf5383a980c4f404fb472afcea67f9d80d440d8ffadaa7ddf261 +file antiref96/train/shard-00202.parquet 119465568 7e8a1f07029be08dc3b6f69c2955452f33159b46a754f3efc5188494ab8ba97e +file antiref96/train/shard-00203.parquet 118772295 c58e713564a972637cbe58842f608d4315238747600afafabaaf12a7498187f6 +file antiref96/train/shard-00204.parquet 119225005 dae88621c4a98f010ce6a926a20ae89f24edfeefe5f1ed78ad6f808c0bd0c0c2 +file antiref96/train/shard-00205.parquet 119508074 4e7794ab0acc66a7baced599a11a7fe2c347aac313b060d173343ce009714693 +file antiref96/train/shard-00206.parquet 118898293 65b9c013f31f35350771535e44fff50e7ea0afb52b36d5be583cf7e382427556 +file antiref96/train/shard-00207.parquet 118977195 b7241ad72c17d3e76444f3258f6331ffcaaf8098a2ed4a3e2877a95e94be7cae +file antiref96/train/shard-00208.parquet 119266833 501cc4a403c2708ccb956a055d110c0ed2e585d0bce897453ac3e9c6000f94d8 +file antiref96/train/shard-00209.parquet 118846115 f1dd66c07f91881e766fff99f454ad33ecdfd9ad1d1148bbc559b79c3b30d643 +file antiref96/train/shard-00210.parquet 66600244 0581e3c09da8963cd84c07b1aa331eaf61bf47d9596be8e51186608094a0c49a +file antiref99/train/shard-00000.parquet 52503090 38f18981ea388db851ff8a5a20fcdb110c2746da8d222482fab9519ee49b144f +file antiref99/train/shard-00001.parquet 52438859 a0dc3f3cabbc73b9c5d1c055f6564c96abf585e0c460a29536b298662a6043fa +file antiref99/train/shard-00002.parquet 52467988 188987fe752ba8b6f29bac307a07def2ea05ade970182d89e75641433df9907d +file antiref99/train/shard-00003.parquet 52484138 e1ce05aca04b3d3e4ef7fa1c973cdb7c169e9dfdb1b9d856bcbe8ca97f70cb98 +file antiref99/train/shard-00004.parquet 52435267 4d1cdecf63db82ed9f8a6751f4f1058b7159be51b26e68a50dc9096d01b84a2c +file antiref99/train/shard-00005.parquet 52484609 5d375c15550de4f0bc4cc6ff9e6e5238c25b3fd3e7f65fc8db5ba384b8b468c3 +file antiref99/train/shard-00006.parquet 52465778 8989a321f26d817d04f859d9997e7e93a0d3c6f745dcf31867b61ebb5277cb75 +file antiref99/train/shard-00007.parquet 52413405 2d9425ed97185ba30abfdbb793053a3cb122fb509496c9b78b10bc6a3c2094fd +file antiref99/train/shard-00008.parquet 52472834 03e303a187e672264ae2eadc455f89294d213773506d71a23f7b484a3269ac56 +file antiref99/train/shard-00009.parquet 52434484 0435e52d63d7650af6ab93b6ba435a1e371cef47d720f76da832063935b4f140 +file antiref99/train/shard-00010.parquet 52471610 e357921e48ef9bd3ce911914b747954c55a5d36bf09635932aa753af76dd32a4 +file antiref99/train/shard-00011.parquet 52443438 31b977ef073be7e7b6f9b3e8c14a321e75e0f87361002a4f5da96e6b6d9218bc +file antiref99/train/shard-00012.parquet 52471734 0dd5a7f819a9f57ea72a04852dfd47e298c234582c5399a01ac186ae85157900 +file antiref99/train/shard-00013.parquet 52501198 a746dbd2dd68a17ac02c7a0787c244d22a7299b0ed2b189c49bc3a42e6a50bae +file antiref99/train/shard-00014.parquet 52504894 786bcaedbf6ef12ffba2d9696550e1c924717a4166e50a3408f102b8056f49a8 +file antiref99/train/shard-00015.parquet 52486789 b32762813fb76463d6e8422abd1d8ce97356614a065bf7cd977b2c0ec28b1013 +file antiref99/train/shard-00016.parquet 52460230 c0a9171b755fd1846f2c5b7293ea4b5eb94db286f324677308cd7f62108e32f3 +file antiref99/train/shard-00017.parquet 52466757 f8d797d22e78240c74932eed808851ff42ccae7e9e1e92ba6b33d7c723216e2c +file antiref99/train/shard-00018.parquet 52505374 fbdf16e4634165c646d70d388e3f2ae0e2217bd8fa6e42a6e5249f2836c0b0e9 +file antiref99/train/shard-00019.parquet 52507453 4c1130205f8f4dd72d01627d3922ccf906458226e557e5a9e863314f1a05f28a +file antiref99/train/shard-00020.parquet 52480206 cf37e95bd4d19605112c8304c48cb5f03dd42773afae92e11b15f512af686b49 +file antiref99/train/shard-00021.parquet 52465528 4753f7e78d60a9204aaf88688de52c416e0692413fa4c5a487660f91ed9b7f79 +file antiref99/train/shard-00022.parquet 52464634 4f59f2822cb59082f6c7eb52ed23478a6c5b3e3c52ea6762736bf6e86cfda647 +file antiref99/train/shard-00023.parquet 52519763 5e27b3b39e0efb01355a14fe0bd03f864df6af7a30ac6131a1c2dcab5f021b4b +file antiref99/train/shard-00024.parquet 52514168 8a01060837a225fabb54b6cfc11e44f5db1cc90968c78a1162af33d16d1b834e +file antiref99/train/shard-00025.parquet 52457829 a11ef73d31779741e7d38d812692c2ffad800602190327197a79ed015dcb278f +file antiref99/train/shard-00026.parquet 52485298 09a8f3c123b72bb4798924f3a9634570b8091715ffb58b9b18e09756023a7c02 +file antiref99/train/shard-00027.parquet 52490649 a1f73a77f77d196e5bf51c544577c72e6b42c629223304ea5a141d908a8d278c +file antiref99/train/shard-00028.parquet 52480099 bd0f96acd57cdfb3a9a2bdc680c4c5024b1ed9327bdcb4bcc9fc746a2acdc1c0 +file antiref99/train/shard-00029.parquet 52473901 fd01f1ee0b016b4df2d6eb2990e21e32374a7cc6540b1fa4291c803d33ffca64 +file antiref99/train/shard-00030.parquet 52479472 fc5f769d6acb126422c9d9cb7cb034441f394405f39363ae3a9ea5b6a27c1337 +file antiref99/train/shard-00031.parquet 52450518 00cdde48f91a045d420e0eadb983ca67d8d89c2a269bf98ca913a1fedd46daff +file antiref99/train/shard-00032.parquet 52494844 a69a09af76a0c4e55d0319268911b55e7ee3532b76bcc3da70da05e273d019ad +file antiref99/train/shard-00033.parquet 52522002 9503bd8db3c209ebc169d177524e8dc0463c417348f03df3b1d6739dfd7c178b +file antiref99/train/shard-00034.parquet 52471706 b527a5e5d372be6a8b9a79ff419b5f449ec050f6c39d6b93b6c26fb62e964fcc +file antiref99/train/shard-00035.parquet 52503048 a7c648211c3a33cb890a2d9fcd0a5410055a13a75b87266807bc1d194e077aee +file antiref99/train/shard-00036.parquet 52461536 ca71be7af265d2804e1ce1f7944dcfa3c9cfd672f182de66711c22699a059d33 +file antiref99/train/shard-00037.parquet 52458062 91bded10324152907a85fbbff1c586009724e4a7c0d4285708c42f073f822bb2 +file antiref99/train/shard-00038.parquet 52484628 a4b0fbedefc18e97120b261e0d780ce0c6b948ddf92cebb51b1eccb4bc151e7d +file antiref99/train/shard-00039.parquet 52455855 40dfb063e6d75fa325cf2d0213a9a3a2154cde8864b2f3b5c5a1e73c01072490 +file antiref99/train/shard-00040.parquet 52489101 6dd074e23f7c0a86c29103ed30f2d03e67ee1f220cc3db595093ba741623ddeb +file antiref99/train/shard-00041.parquet 52477721 44e62b996b259221582007398845d89aa510b35eeb2c693c400e398290b4b256 +file antiref99/train/shard-00042.parquet 52458440 576055bbadf67d459623eba2bc1d541ca068df81ed9fc643c06995e0a85176ec +file antiref99/train/shard-00043.parquet 52496775 83656c5ec856a94e145a74c16795b3473ebdf785b5e3376bdbc0064bfbefaf4c +file antiref99/train/shard-00044.parquet 52450229 13e75ae3db26ed5154d15da609044680c90cbf6b1b655bcdfad91d502175c068 +file antiref99/train/shard-00045.parquet 52507775 fbe99fb672b8cceddba49e68d68145f30e037284dabbfb42d1de4847518ff57b +file antiref99/train/shard-00046.parquet 52443865 c1aa23405bf49a2d63bc97a4d3bdcd9df1d413f48814623268aa8adb5b35606f +file antiref99/train/shard-00047.parquet 52518705 7f9aed60ebd6a4d289ec75708fca0ff012c1fcca9ea56cf4e1db4ad4cb1f799d +file antiref99/train/shard-00048.parquet 52440132 5590784b4ac37ad9f38559828044a3435690694b4015d82cd84587f705098d8e +file antiref99/train/shard-00049.parquet 52469435 ef8e22fd46564cd53b62049980a890896ba49a97415d752ddab2f3b040f9a832 +file antiref99/train/shard-00050.parquet 52486772 6947811f1f19e260731e62dd20a959ad8b1d3f55a022bb8d905420c9bd98356c +file antiref99/train/shard-00051.parquet 52505701 f23a03cb52139937c661dd28a16076e8ba5084e3f5219e5760bf4be0d13a33ad +file antiref99/train/shard-00052.parquet 52467290 cb2941fcbe8bcc747019444698812e895c529623f0764a1a7e64d73466545c74 +file antiref99/train/shard-00053.parquet 52497788 e880aaa5638262a33ce0dfa20f4a034ec90b59784428203df96aa67bb2b36ebf +file antiref99/train/shard-00054.parquet 52492211 a8c1c45b9606c19b9b6bc6b4fdc4b2e896e9dfa0b54ff00aa099cb63a31129bd +file antiref99/train/shard-00055.parquet 52503886 1aa62ade823a20bc9dfbfc23d1702a5fd51ffdab56fc65db03d7d03506900176 +file antiref99/train/shard-00056.parquet 52456406 1594608a98ab00103169e0c407cd3e031efbf344f661a82bd90a2a17e41dd899 +file antiref99/train/shard-00057.parquet 52451162 96c81b5f43c5282a7b73ca7def0c0129a9b844367ae5ce03b9afbaee17e6991b +file antiref99/train/shard-00058.parquet 52493727 5c0ba481e83c917913232382b0554c22c8b3a5ee9ba8d4aea6ad974fa623d4aa +file antiref99/train/shard-00059.parquet 52473561 6fab847702c25283273531a0d1c7f468e08896e737f3020e6b0ad5956391bb96 +file antiref99/train/shard-00060.parquet 52423808 8a3ebd11938efb220280dbf2eaa978a8113f31cb5b269108073bb59188f96337 +file antiref99/train/shard-00061.parquet 52509377 d8816acba91128ef1c6f41c4cde65963e9b673f480d55bf6eb86a035b43adaeb +file antiref99/train/shard-00062.parquet 52511881 f30e5d00a9cef68926803079ec7493fad4a11c1ad2d916341efd51f4b9628643 +file antiref99/train/shard-00063.parquet 52446319 b9c4c9d4949790c881a6eb5f7ded99868026d2869164ebfb77fd224afb5884ec +file antiref99/train/shard-00064.parquet 52508548 47ad9c7cbd367fcffe576a5c9695aef368cb5ad3535476f514292063f6be5ec0 +file antiref99/train/shard-00065.parquet 52490722 57e000594ab072c97e37440f4a4c2d8ab923e8d830fdd099ffcedd27a684e4e3 +file antiref99/train/shard-00066.parquet 52464364 548e61d80451deda08cb97fbdb8bcb95389326649113d87a7ae7091a1ca8ad18 +file antiref99/train/shard-00067.parquet 52449879 f110938de83b6be3c2e46dc8fd972401585d38cd171abb926a28c501b159b1c6 +file antiref99/train/shard-00068.parquet 52447054 649a3fe1c2a187c0df660cc18252bcb2173df27cd82153475da1caa1175ce8ba +file antiref99/train/shard-00069.parquet 52514282 14183d26a2e1c791e5ceaa016cfc294d04e528b740469b12a3695733d464a5ef +file antiref99/train/shard-00070.parquet 52462033 d2a7887f8d6934a8eff45251e99de0a1a19b3183cb2a0525082cc3af6600dacf +file antiref99/train/shard-00071.parquet 52491369 b5c406987cfed7f5537d77ed560f3f42e7a8df39067a103b8f324295fe8f1d98 +file antiref99/train/shard-00072.parquet 52483663 64ed33de3ee0c5e90ed60e13664200bd410038544a709ff342563e903795d60d +file antiref99/train/shard-00073.parquet 52468756 e1e6ecde51a311a6ff4e373653c41a8287ffdbf7417d5c4413e0904206539f87 +file antiref99/train/shard-00074.parquet 52500037 28fec7865ec24e81eab6c3e4d15dbe14646d4cd7b239e2fcb9ede1e45ed129d6 +file antiref99/train/shard-00075.parquet 52489392 41203c53affceaf61a1d492a02111e25e32f01df5b1a42d92cd5b91427d27eb2 +file antiref99/train/shard-00076.parquet 52467203 1e18de574606e7d598379c29131beaff260b644ac091c660da552ea11c2820fc +file antiref99/train/shard-00077.parquet 52445066 643d44aae9b1b3e0ae159ab9a9f7c3b4cb4c1ab891bcd175dd3debd783761b02 +file antiref99/train/shard-00078.parquet 52506856 cb0757302cc8cf174de362c3702ad75350076baa105eefaaeef688046fa7325e +file antiref99/train/shard-00079.parquet 52480670 a0774dd163797158e4e7f045e55a21d7e9809695fc445f8eb8fa8019b31cda53 +file antiref99/train/shard-00080.parquet 52475485 bc2cbc848f9f61511cf4fff9162c8721fecd2c42a8018db45177d90f5ddccaa0 +file antiref99/train/shard-00081.parquet 52485135 b8ca92072946396e7b9c6baa46ffd9f81b9d3f7508ef22f1b16f7ce4764d532c +file antiref99/train/shard-00082.parquet 52448485 adbdccd4166c4f6dd132f07ac95a20630b2566f9b8f5c26a0d29c5d89222e856 +file antiref99/train/shard-00083.parquet 52512621 64cb373464f6ba79e6204046f3a8afa4bc80353f59536c9d5eb2ba1c3257b8f3 +file antiref99/train/shard-00084.parquet 52414431 32074eb594409e0813d28e073844d8650a7b12158488e194c244061d7b90ba81 +file antiref99/train/shard-00085.parquet 52474612 26e57b928d0e42b1cd23cdc7ff8c6894e49f79f4696c0c42b78eb1c1feb3061c +file antiref99/train/shard-00086.parquet 52493300 2b61fadc08952dc8b7e548142cf6c40e5ecee90afb8cc648c9adeb5456378de6 +file antiref99/train/shard-00087.parquet 52466467 fd6eafa19c3895c5cdc3ad8e880f53310d4cddeed0b2f3dd122a556ca69ee7d8 +file antiref99/train/shard-00088.parquet 52494590 d31868e5053d7e9e97925dacfa59612ecc231c001be2da30969d11a973cbfd4c +file antiref99/train/shard-00089.parquet 52503724 1d3cd9bbf5f0a1ef562ddf0fa1e858072e90ba303895405f0d0e4d6f09ac84f6 +file antiref99/train/shard-00090.parquet 52503699 3b7c9a0f5a3f7124023b0ffe2c275d11f96cba0f6059f840b4bbac4c9f5f2dcc +file antiref99/train/shard-00091.parquet 52487280 aef00e110a545bdd79492d7bc0402dd36d609ca2d5d9ec89e85f4b60013ea042 +file antiref99/train/shard-00092.parquet 52437795 b97a0edcd491f69ffc578d8b7044a2368b82e17d3b6dc356a8f5474d881896a5 +file antiref99/train/shard-00093.parquet 52455244 d0da228051da74a3aca501b892a5b2c9a198731aac9d6ad444093e566688648b +file antiref99/train/shard-00094.parquet 52469664 7bc8f24c10d462e3c6f7320774b9bf13a810eaf665f815f5a13b0739a68424c7 +file antiref99/train/shard-00095.parquet 52489627 3d779060418b3c5a08620a744378a6dc01d94ed0bc511c95c43c1e175882d7e5 +file antiref99/train/shard-00096.parquet 52520088 d6f7b0e46d40826109f31ec11ab819ebb4afc8af5b42e9f2409014412c57450a +file antiref99/train/shard-00097.parquet 52446787 d9dc8d013870af202741e6ae8bdb53caf7217d586f023af9e2b56c0459d2ae5f +file antiref99/train/shard-00098.parquet 52475756 d252896cf1d011d0e291ced0cf287119aee1726e5c33664d10c1a07a338278c8 +file antiref99/train/shard-00099.parquet 52472291 265f355b6a168424a6a87ac8edce237878d8b289b0a718840029267c9f34b60d +file antiref99/train/shard-00100.parquet 52513728 facb33f7dafa72e107c8f3062aac93825d8c61a0d46a0fca16bdcfdf3f4bb2e7 +file antiref99/train/shard-00101.parquet 52444971 2f33f816bd6b07200785dfac9e4ab7dde9c7bc30ce13a8ff028e5a6f232c98f8 +file antiref99/train/shard-00102.parquet 52447088 0c0f309134c4a97585117980193f23f0005ba307a7d080c62e28c86fa81b2f06 +file antiref99/train/shard-00103.parquet 52514052 8520c5b05b573d9591531847513e20ca3fddbdc6f7717a244841784e5481ae9d +file antiref99/train/shard-00104.parquet 52524334 a357fc4cbf535a6b89fce84a4561349159f92c33a6872c9b770ae21ae938cec7 +file antiref99/train/shard-00105.parquet 52483299 bca62561ade388a68ce2ea8bf0e25cc02cdc00f6be841dd57c6c30eb6ae1878e +file antiref99/train/shard-00106.parquet 52491137 006e2da2407d264dcc2b67c9898d613ff5d0acb44c56d8604405f0f45368e568 +file antiref99/train/shard-00107.parquet 52474079 fad8ef13f26f1981d1e658b531a4f3d99abc99451248e7e72a97e0452617f51f +file antiref99/train/shard-00108.parquet 52458078 d9634135b2b358100d410fe9ebebc31337a3bb0afae6caeb1b2e2042dcf113c6 +file antiref99/train/shard-00109.parquet 52495596 c98c6d0f0c67959338ae2d3a4a5ee7201345caaf8131e87ef9126a0e81d9562c +file antiref99/train/shard-00110.parquet 52473405 87be82a69c051fdad17fd87ef41ba57c5d373459ee9e9e8a7f0f93d46606b2a6 +file antiref99/train/shard-00111.parquet 52498051 6c284257211b4b5541a1703af744299be16da0261cb3fb5a02c694b80cfdc06a +file antiref99/train/shard-00112.parquet 52466441 f86658f0a2d59da8f77fcac2bc3f00dd245c4e606d3f645a64a555866b054563 +file antiref99/train/shard-00113.parquet 52498462 086251435e154252f297dade854474c9f695e23c34e3c58af36f0d4d9c0fa5d0 +file antiref99/train/shard-00114.parquet 52477668 ef216819a2cf130127afe8a2e9fd2a6c6bd39ffaed34853dd002c4d36a1afcab +file antiref99/train/shard-00115.parquet 52479604 486fdc9943eed02b9557799be23fac711fabe810508086ad373ef161cc360b7d +file antiref99/train/shard-00116.parquet 52466153 849b17e3b56309077c19dece9825bbbd8c411e9bd49b594cdfb4c48ad71736f6 +file antiref99/train/shard-00117.parquet 52498415 779c89932bd736022e8472a2837cd30a78399c7ac6eec8fd7901d69f84d36410 +file antiref99/train/shard-00118.parquet 52506670 6cb4394803d255cddfcc791e488667532eea4380975666b73664e173f1f44a77 +file antiref99/train/shard-00119.parquet 52432078 2898464c9ac4c9a10f8adf5cdeaf7eb2733bd8c17808344a9c629fe5082ff315 +file antiref99/train/shard-00120.parquet 52506886 cff55a581c4a1f4fc829782ccc5ad7302ee7415cc2167e3a6610eebd3cf6a793 +file antiref99/train/shard-00121.parquet 52497117 a8493f1efadf731752595fb8e3aa83bc4cc55bb72217a572ccf41329127fb833 +file antiref99/train/shard-00122.parquet 52456055 74cd8c75ba19b974d860ea24739dd28fdfc3ea4c0ab37a5e5610d19d77def120 +file antiref99/train/shard-00123.parquet 52450868 ed54307dffae7becf84eef2f5073cc1e624ca4c54190f096c8c14fe19671f1aa +file antiref99/train/shard-00124.parquet 52472904 58231c13b2b51a3ab5485f123dd99f3d10e2654f5f4af05c76edf2ed09755138 +file antiref99/train/shard-00125.parquet 52448947 97d45f589aafd42720ad4ff50cd8be0de658e7cf64ce2fad7ea9e2a255964d03 +file antiref99/train/shard-00126.parquet 52464733 767a6a771d5094952c26f7ba62c70a76f62a62c8a43619c4c7a294c1bb87f3a8 +file antiref99/train/shard-00127.parquet 52496015 a661f477235c6dffe56eacc5b98badc63349ff5a7e1b71ecda449db458a7fa06 +file antiref99/train/shard-00128.parquet 52437125 c92fb3a99c4ab43fa58303133102181569ec80472d8b9726d4913ede747f259b +file antiref99/train/shard-00129.parquet 52457640 bc03612b6f7eac47e5d4ca8fbb63cf18b75b38a7fbe304d5ff6da510bfca5add +file antiref99/train/shard-00130.parquet 52478123 70557d6928c0aacdc3d41c5e02d0bcdc05c6368e4f63e7e9aba4c347ba09da7e +file antiref99/train/shard-00131.parquet 52489872 774aff9bc75bc28893ab4caeceb7a190d6e02915c2154ba6b70221730ed1a726 +file antiref99/train/shard-00132.parquet 52467668 4d588356cb55622c073189b7798e840f7a0d5bb0204bb755afe956cb39b88c53 +file antiref99/train/shard-00133.parquet 52463741 a382086cbeca8fce7c418d4cff72a07743da465987493bad1970dfd43a50f6fc +file antiref99/train/shard-00134.parquet 52439389 b24681d5efc2ff37abccc23e15ac3d8640b6bce9d2760de71c284c1ec996899c +file antiref99/train/shard-00135.parquet 52466115 c691a81abb73243062aba61454fe84fdd166649a16bdee2b98da8a49416a2d36 +file antiref99/train/shard-00136.parquet 52435241 034fac2cd4d2212241c5138b7b5123e96b286b3affd31ace307be71f189f8027 +file antiref99/train/shard-00137.parquet 52456756 956ef966596c26c0ee8e6fdb42fe1d62a66d463affe3de60c62c6c30edc33aba +file antiref99/train/shard-00138.parquet 52451449 6811724b064440ccc1c17ecbbc5186280374f4b0c0b5f74112e831bcd9b70fb6 +file antiref99/train/shard-00139.parquet 52442165 287e40d8ab07033a67f60ce330757d20275d54075729fdd7afb017d21505c0b5 +file antiref99/train/shard-00140.parquet 52490640 53205b4495bda1225aa1ab05ec164dca1b0396c276bde6bd90a5077c330495a7 +file antiref99/train/shard-00141.parquet 52496003 212b32d603c891cef7c13afd6bf5f293b648486e6c84019c080614a4b7c40712 +file antiref99/train/shard-00142.parquet 52512857 eed330f3d588159b7ec1da7ca5014b2c29cd720d62575aadfe6f3dddd435fc8a +file antiref99/train/shard-00143.parquet 52496081 3f028dc494e948c33e794a3e4d96eb7b94d7adea2c9aa8a6a1574094c084a11a +file antiref99/train/shard-00144.parquet 52448578 c07015e7ac5a38c76f7edac0a09ddd603ecef19414ec756b612d1d6b1daacc75 +file antiref99/train/shard-00145.parquet 52428629 f564f277de2331e6f85e5e4e55c89cd95bf3530e39a595e9a1ff2d33de6988a9 +file antiref99/train/shard-00146.parquet 52452503 7a81f89b3d1cc6ef052918c55d963a79a60a87809893e994eed016ee4d0bea73 +file antiref99/train/shard-00147.parquet 52473970 cdf39d91b3a0dc0ea9e166431b0b63011db78490c340b78979ed27cde2fbf9d0 +file antiref99/train/shard-00148.parquet 52475925 04633ccd461f9f06e7e92dbec0ed4ba463d8a30fccfd65864ce02bf47a73b174 +file antiref99/train/shard-00149.parquet 52471489 4a00d09df55f15b440cb9b95bac432057a29b67853dabc8a06bd7756c676b222 +file antiref99/train/shard-00150.parquet 52498080 8815dff2de53ab10fff67fbd04d2fa9ea1f08b13f709cead36b7bffd923b93d0 +file antiref99/train/shard-00151.parquet 52452041 83ca50613834688ca2a2005696522e6d7ab5c34dd14711bddf65f79104ce2488 +file antiref99/train/shard-00152.parquet 52463621 a4af02f2d67bdfb967cd498e58dab8fe657822499a6ba9d78107e3e22e492227 +file antiref99/train/shard-00153.parquet 52452794 bb4f02e2c51da88499e9690a1b754d02e2cd363fd4a800e13947d4840d3b168f +file antiref99/train/shard-00154.parquet 52485035 647b9acddd51e55bcbcfd23e66cf854200cf7f9484895d4303cf701c4e082cd6 +file antiref99/train/shard-00155.parquet 52502763 02ac78d19b1c87969eb68c847717579aa7bb7ad996fbf2904a1c7742e97bd09a +file antiref99/train/shard-00156.parquet 52496347 a5e9a71393a5a57712adfe5775a11ef90213ff41f2068914ea6ba94a0dc10766 +file antiref99/train/shard-00157.parquet 52459531 2dab47c11f1e4f4d4f5cc498462280747e1c72c21a44257f9533de649b030ccb +file antiref99/train/shard-00158.parquet 52487266 f86c6c49b97b7262298a6eb493e65437febd25ba71af0c9806d09b651c06007c +file antiref99/train/shard-00159.parquet 52481357 a42f622d5518ada38255481d0fe750492db6a39839010bcbcadfef2a269fc581 +file antiref99/train/shard-00160.parquet 52442516 2be5dca522636fa30700d7e2130d96432ee538124606069b2086493cccd9a62a +file antiref99/train/shard-00161.parquet 52458890 fb6746e3d201e675f516e9e74d5311c2f45f4658014e27c95d4c0f9befd0691e +file antiref99/train/shard-00162.parquet 52494162 4a09a182666934e4cbfbc34c015586456f3e68eb222c1da93befce5eceedfd2c +file antiref99/train/shard-00163.parquet 52489750 8309d722cf96179364bace43b238903f6a37d0fb38f5235aa1256cfcb2faf821 +file antiref99/train/shard-00164.parquet 52446823 ec532a439dbcf1315f85ff0366eb28d0bca3a5f4bf657e5b9c3a244da16270bf +file antiref99/train/shard-00165.parquet 52528075 d8ec804da954d1baa85d007082bf20eac60c11e4b0629854bf181fca165807c5 +file antiref99/train/shard-00166.parquet 52445785 1c5486f95ca8b1ffedbf10e22cc6fe485af962a466612252fae556d8f8187cde +file antiref99/train/shard-00167.parquet 52470742 912cdb8fc8cf9aaca59a7eeb4177a9de17a9d27246ca31ca973aec0f868659d6 +file antiref99/train/shard-00168.parquet 52453141 49e24dab93ad29c4470005dc5a57b44e943ede807197a697a4a517eb806b3fc0 +file antiref99/train/shard-00169.parquet 52468628 ecbc4387ca794a14f6f9b9071a2a5861c2fc47ce38b62207ea789e9bac6dda2a +file antiref99/train/shard-00170.parquet 52507467 d8c34c089e1cfb1fb6362b9b17e6f34497a4e083750ff6dfb8091d145a7baaf8 +file antiref99/train/shard-00171.parquet 52458677 e69b4b21fdcddfbfc8f06e082653b9d6518fd38f9b168b1e3b08364fb3906ef7 +file antiref99/train/shard-00172.parquet 52474299 6950ce21e4c88ae0a0cdbabe2f1a8e5fc94a6dcc96f05e4886079cfccd8c7520 +file antiref99/train/shard-00173.parquet 52466260 89f427f936ddaf8f15d030cb72b1a72e1227f02ec2008ff2cedec988699c7f4f +file antiref99/train/shard-00174.parquet 52484689 4d46fbb13069f260d92a86b963b91d557b54185592d09bca3773e0f4bc34b40f +file antiref99/train/shard-00175.parquet 52479833 93281b1cc891f0064faaffca5fd61d511b267c036f3d18a4c1702af4aa3d5dec +file antiref99/train/shard-00176.parquet 52475407 8e9afe57d9c186a01e8dade98dcf806e04edb733c1bd98f04bcceb3e81085014 +file antiref99/train/shard-00177.parquet 52473779 0d24d04fd39de3f34adfa549265bbc4016c4ff63c3fca980ff9bd58fb3c07087 +file antiref99/train/shard-00178.parquet 52410677 ba13983f7fc61ae8737819f755ab84aa7866213b240584012556d7ff22ec8791 +file antiref99/train/shard-00179.parquet 52506177 511fc15dcf977fa96d39d91b0c10b32f009ff196a03b9464b2c7d6c5c548d873 +file antiref99/train/shard-00180.parquet 52466908 ed066aa9f17c4d152f4ee92eb13f1966cc5327bbced0068b3416a272e2252b29 +file antiref99/train/shard-00181.parquet 52453333 bdb1cbd03aad4a14861520af5446575359ea13b927d76b7f867ef0de00874e6b +file antiref99/train/shard-00182.parquet 52444775 a6a210c755e61587d222b3336c6ee5a7a64b376970ad840e1ea4e313cd715d97 +file antiref99/train/shard-00183.parquet 52469809 e8261e7b0a158c5c0818ebb3eb7a699ba9fed6ecf2a825f3bf6b581b5c3e4155 +file antiref99/train/shard-00184.parquet 52449671 f3a96ef6251f77a53827e42b3be748291c4b4ed2941ad087e874ee2dfec7a228 +file antiref99/train/shard-00185.parquet 52518861 627f51af5338a60e4751fb75f5bf0fd330427ba3ca7543b6f1331fe46382b35a +file antiref99/train/shard-00186.parquet 52493971 c8dd90b0a3ca920ffbdc8bb601f990db507f2301012d64af8e6de5e16d77b23f +file antiref99/train/shard-00187.parquet 52476587 eaa7c4b837bfb4602334db218d5fa173c4369c50ded1e472422412878d963ea3 +file antiref99/train/shard-00188.parquet 52496390 53e6c00eebc29b950740de535586e388c91878d020e8881072b61e2deeac1539 +file antiref99/train/shard-00189.parquet 52536340 485c48d7e56cbf92fa95cf45df27328c8b862113599e4b5011e4334c8f791418 +file antiref99/train/shard-00190.parquet 52512226 4478b8d84d2c2a990e4ac1e0d8489d5f09524777a982e520f934ce388057b449 +file antiref99/train/shard-00191.parquet 52452060 f0c9967d78e36280154871d1cb5a516145731fc09fe7ef7384649f136104aa6f +file antiref99/train/shard-00192.parquet 52470622 3edb77af7e88475dd5d7ef09aaf280a6c90c472a823fac92d352353409b6b5df +file antiref99/train/shard-00193.parquet 52500720 7ed8bc47eed39009d366361a116b873e2c4a4eb531169a1e363457e38764f388 +file antiref99/train/shard-00194.parquet 52479975 1badc0a563a8a341e3356a9a13acde6923ed5872271e2868faa1481d3e4bf39c +file antiref99/train/shard-00195.parquet 52487880 b07bef62d91e4519a3e50483c5bd8fb1e516b3357ec447a6a20ba708417a4769 +file antiref99/train/shard-00196.parquet 52526022 d1d79243aee60bcb65d07de313d6ae0b6a3d23a039623d1f31474b0a1c76fbbe +file antiref99/train/shard-00197.parquet 52530278 a0503f838c39894f7a4a9b8874cc1df41131fe236ccc45ab7989fadb9b989d49 +file antiref99/train/shard-00198.parquet 52438727 62686c777d0051f4925e76581c988f96d99efa8cb9fe696ca05f64b287265768 +file antiref99/train/shard-00199.parquet 52448471 efb22e707bbf13a05e097ea764757ddac70785b56359478bbf86d31083618b53 +file antiref99/train/shard-00200.parquet 52496285 1603ec83a37463f2f8adcfc3f86e004b9c4b3c979354d5a88fe35260a58d98aa +file antiref99/train/shard-00201.parquet 52435299 be1c5d2d1db7fd3575a7fa8135692ae1a092ee5e6cfdcf4ec3e0da4e9fe97272 +file antiref99/train/shard-00202.parquet 52493221 41c4f8b99f63940cc09b2d734fbcc34110778d569e785f255d5190809a1ce719 +file antiref99/train/shard-00203.parquet 52448645 bf9368a10cbcd8bd5c8b3ba432c83ae241895cbdfe1a96edec26e1dde31dce6a +file antiref99/train/shard-00204.parquet 52496266 0854c2c873d51e1af25dec555eda2f45b5267308b91b6266439292c514361e50 +file antiref99/train/shard-00205.parquet 52503286 f1917be9ca3678d04a6f1a6bfc903cb57116a781225bd538d0584a0e2b8c7a29 +file antiref99/train/shard-00206.parquet 52454077 462040b332fb17695307a8590c3fb478c58bce551564476920af835ab0fc6084 +file antiref99/train/shard-00207.parquet 52465032 b4efdfa2575ddaddf6880ea8c1c01c3343d6fde8868b24860898365a8351c951 +file antiref99/train/shard-00208.parquet 52466168 a729bc5a08558c1697bb7a2f0a15a40019b5bc9cd1f5bfc3e953f36d891d04af +file antiref99/train/shard-00209.parquet 52495224 ebd94bb761ed99dee1511fe95b84bd24bbb18934eda42a332796e31af1d306ab +file antiref99/train/shard-00210.parquet 52501216 b91676e329c36849f0939d3b1923d2d3d6a9e84464becc5ef20ba811f315bd82 +file antiref99/train/shard-00211.parquet 52467128 089787543879ac32a77e2278895851691c3cadb58aac63b98574ffbeb330f63c +file antiref99/train/shard-00212.parquet 52477712 96c409690dd1c108078a161333cb950545fa956d61ea86114339f63f903f1b66 +file antiref99/train/shard-00213.parquet 52470053 c91c298aaa36716125f6634d3023b93c92e4c46890f04374fb1f1a2a0f6cbc0c +file antiref99/train/shard-00214.parquet 52473685 2f3eea46e952e71aff675665cb5e0251b7763f45f0dcedf39a70ff2174d808e4 +file antiref99/train/shard-00215.parquet 52495084 ca2ea7cc97d2dc4139b8ec96e3a1858fc71e063e9d0d6456ee8c252d7f6b4a32 +file antiref99/train/shard-00216.parquet 52532523 f91cb5ef0cc0438ef603b9fc22dfd3aed084935f7f75ad9c58619535af90da09 +file antiref99/train/shard-00217.parquet 52447642 275a9f0c28fdf4933f81fcbe826b28580bbdcbfc2eacadf36d645275588beba2 +file antiref99/train/shard-00218.parquet 52455579 ba6a34b1683c2f006d9fe2a85fe7889eb599a9ea8d7763d866b37a32093d0f62 +file antiref99/train/shard-00219.parquet 52463764 8b32af042ebcb4a53c8859294f55ca182ba9e55a890f288e691975ff5368a36d +file antiref99/train/shard-00220.parquet 52504241 b41cb5af37fe3b34cbdc3b184e54f35ed352963b5a0d8aa276505f68ce7a2de4 +file antiref99/train/shard-00221.parquet 52499489 110f68194bcd60602cb537a56c1d8e7f4f5689e88708485f218cb4c0dd86e7a6 +file antiref99/train/shard-00222.parquet 52462060 2daea4d12a5d108ed6c770cec6185be6d727b34500551e703436989ed5c489e2 +file antiref99/train/shard-00223.parquet 52452744 547574d1279e6bd755d8677e7f2ee51767e2f43762f95d3096e4f742c9aba50f +file antiref99/train/shard-00224.parquet 52458847 6957d1ce24ddeef1f308e7f0765fce0ed54efabdb34ed92883ec3a39ca6b34c5 +file antiref99/train/shard-00225.parquet 52505415 b7152763d9983fc1917d06746dde29b324e320792d1d9b5efff95f09a5d67de9 +file antiref99/train/shard-00226.parquet 52464403 c8f0d2fa65cd55a85605d3829c8645e204f1d26c88e228b1b88f99f5e0810d3b +file antiref99/train/shard-00227.parquet 52516191 3223c35b8ffe0dad2ac1f155388fc4219479f5144219cadab1e4bdf397896595 +file antiref99/train/shard-00228.parquet 52467823 cf8110fcdb2707794eaa0517c637350defaf385d68e1e147628c9d748767c6e6 +file antiref99/train/shard-00229.parquet 52467232 2dcb4a5cf807e42292682568ad8f3b7f74c5f34fcf1df9d15a66d7c206c47015 +file antiref99/train/shard-00230.parquet 52458460 6e4faf9bf0c02e9368f60f585db7e30815f5b1fcd81fef0aa934dd89e9b82f65 +file antiref99/train/shard-00231.parquet 52568241 ad27cb58a983b51c07498b5e5aee9909e4b7ad31b9a229234421f6bd18d899d2 +file antiref99/train/shard-00232.parquet 52499208 f69073eadc0998eb89a432dc6c344c67c9587f34350e019dc5fc6c958b6808e3 +file antiref99/train/shard-00233.parquet 52448044 5e81ed6dbd1556fd9d08e47357d9852c6b254d62a3c1ca14eccbf1b6d7716a01 +file antiref99/train/shard-00234.parquet 52514248 0efc3d9d17f4773cf67c6d5386e28f8c5cec48fee1c1041edcd7edd824a53120 +file antiref99/train/shard-00235.parquet 52475903 9176851ddd6672ab78b5919c1ae8e1f9c7cc730b25bb0efd3ee5ff052e1072c0 +file antiref99/train/shard-00236.parquet 52451036 13035d921f92e60cd3f6fc831ea259efa854812229d7d34a4785b0cae9982e99 +file antiref99/train/shard-00237.parquet 52495883 df958c9e4afb2d9c0a1e9e81633e0475bd68e82160a402bcdfa9016b4bb140cb +file antiref99/train/shard-00238.parquet 52473836 beb7f46534fdfc70107fced2e3e6213428ef9f9ca2795720f82b3f12736212c0 +file antiref99/train/shard-00239.parquet 52417725 29b66a3c0697a07e2a5cd1d1a51f1956a34f1724adb18d5f7e59c4092cd0e3c9 +file antiref99/train/shard-00240.parquet 52464145 106648b4c96c21ad82414cc8ed7af25e6199bd1dd85995875f1cf3bb23478688 +file antiref99/train/shard-00241.parquet 52439866 6d92add61fe8c28c083d3a839d0ce3f99a5210358be514d1bc8874cc975a87ff +file antiref99/train/shard-00242.parquet 52500089 1e9f31329a5b18db36df0d27de3cd1e9c66165b959d9e46384d9831824bebb94 +file antiref99/train/shard-00243.parquet 52470659 905f82ca0e62260e457fa1250347e32e9cf9f03dcff06f4ae261314d95d6f296 +file antiref99/train/shard-00244.parquet 52458688 7608f3bd5a5a95168434da56408f8a996416b395a77d97c5418df04dc0212cea +file antiref99/train/shard-00245.parquet 52474923 d0b6f55d71602dc729acbdc6d015d449eff927aa42ef388e0375f865ad026622 +file antiref99/train/shard-00246.parquet 52460547 c51f7297748c0e046a420c5481473a6cbd3c714f4930af8299a1e537927c9722 +file antiref99/train/shard-00247.parquet 52509064 0bbb4f4b2a1db82334595d134469d0d11aa291d03275e41fb8a731f07497a409 +file antiref99/train/shard-00248.parquet 52479847 bc345aacbeb255c3ef5a42a1b24bcdf77ddb0ae8cb40627e000399a410cd1087 +file antiref99/train/shard-00249.parquet 52460398 6124eabc906cf7eababf501ecb43d8c69f25a2000452cc3825a1caa8afd4cc16 +file antiref99/train/shard-00250.parquet 52449471 defb5b233efd075f0108f3760ec0d0b4ae746622e113f09c008253cd67018cb3 +file antiref99/train/shard-00251.parquet 52495255 b38676b6906c94069715d200f2f15a00f2804b237687089dadbba1a1f3c6a804 +file antiref99/train/shard-00252.parquet 52464768 a54aa3a97ff7e52a4a2945668183e28874bce1951f1dd0fca852d0b5c26a6de1 +file antiref99/train/shard-00253.parquet 52477196 f16ea82a2b52a9412adf21de652f7888fe07b145a51f7e1e47302865c533fd38 +file antiref99/train/shard-00254.parquet 52504258 cc021af060e8226dfafc9688f03c305385f577e8ef5524d0555e5be1528762ba +file antiref99/train/shard-00255.parquet 52465994 090f32387ca67c536b1264c3133178fd3ef5be0d7f1a061c3b1279afae3b0b83 +file antiref99/train/shard-00256.parquet 52500187 7998f1c7ea9ddedfcc8d8c2101b00e21ada4f2fea1a5ea590c8f38688705406a +file antiref99/train/shard-00257.parquet 52527164 e757e240ad8d0933993f92926639d2bfc27bb2766092bc30318f8020f6baf5f1 +file antiref99/train/shard-00258.parquet 52438417 71c543ef073e85abc3e9c3cd7bfe4c6ef8f9181cbf643f57da4a81b0909f32c2 +file antiref99/train/shard-00259.parquet 52452179 471be2c503e4eb7713764782e8489f27ca0269082020ef29cdae6c01beeaf0e1 +file antiref99/train/shard-00260.parquet 52488986 3c60bee0bbe1535aaf3cb3e47f373fb7d6822dee376b8cfc73852d6caf455831 +file antiref99/train/shard-00261.parquet 52489285 4b4d4859e768f3aefa93221ab98b589434794e8bb2ca527a38bbb68e29c7e6a1 +file antiref99/train/shard-00262.parquet 52492078 f120dce99ae72660682ad79d6225e93fe50c27745b99121df2482248f53c1bcb +file antiref99/train/shard-00263.parquet 52475304 7cdd1ca5aa6656f67b7c93f82e7c46d1d286b0d882cdb0b86faf3e8857acc678 +file antiref99/train/shard-00264.parquet 52482658 324d4c43ad8b6d4acabd6fbb9a1c028658c8d056ada3fa318384437c7bab1f9b +file antiref99/train/shard-00265.parquet 52443660 02bc3575dbc069a8e6c338b2802ac0947b3e0c13bb7032ca1e91bb94c70a40b4 +file antiref99/train/shard-00266.parquet 52512019 5eabbf367de9887943fb15caa3421b3c5639a9e7bdb2bfea8277fa60e90ab687 +file antiref99/train/shard-00267.parquet 52506602 b97ed37ae7af725edfbd8f0a378fb97b6fae623963de272e403c81a1ded5e215 +file antiref99/train/shard-00268.parquet 52477274 fbe3de34b2d21911d69177b6708f6a4a73f32bf02aa52d5acd9b225f5dd4a592 +file antiref99/train/shard-00269.parquet 52481418 586974c535d22431630d014cf4954519fa68f803b0493f9f76f607c63e4f44a0 +file antiref99/train/shard-00270.parquet 52465947 6a99b50841ee334dfc6d5708044e53eb224d9c3ad12369764b6cf5a4434088a0 +file antiref99/train/shard-00271.parquet 52480924 5528fe2151f21b02408daeebd65ce74b2485eccbc149df7dae4e466d9a0fedc1 +file antiref99/train/shard-00272.parquet 52524648 21188119e6ee96c3fdecdad1dcd7fac8e261b2b5d9c63bb699106c2ea3c23ba1 +file antiref99/train/shard-00273.parquet 52492505 3cf6ec4d50f612b275e28f1d73c7be1818ca79a3ca9aa66270f06e3d55750f85 +file antiref99/train/shard-00274.parquet 52503356 a87de8686ebd7d0eb25d89e4d8c8dd0d1403ba72d736da51e48b461c79dcca98 +file antiref99/train/shard-00275.parquet 52463384 f8bfb9c8a3d1df98fc621cc1e1d306fe88e228ec4c9992ff9a862a623cab9334 +file antiref99/train/shard-00276.parquet 52506273 813417b10a4a075bc65bcef87f5b1872418f0ca4b4da40427fd33531f4ad4c2a +file antiref99/train/shard-00277.parquet 52443574 f517ed9afedcfc2e33b946066a763d7580740e399398472d5b7c42d18e017d1c +file antiref99/train/shard-00278.parquet 52448738 d024dd013811ca6d5e22a1ddf6d983387bcd7dafd6f76c74b8d4b11b49278f83 +file antiref99/train/shard-00279.parquet 52448428 8e3b7bfbfe871404646c67cd740277ba7b463f14d6663189b608ede83b0f3d22 +file antiref99/train/shard-00280.parquet 52475208 3a214a4ea38cf241dc8e22f16b55ca4ed86251a691a906c2117874d6863e8c51 +file antiref99/train/shard-00281.parquet 52454765 9c5314be99b3306eddaad779464e198cc1429d7bf683d964e38507ead94c64cf +file antiref99/train/shard-00282.parquet 52465377 866f3d7ca84c8ef01f5ed9049b8748fd634a3b16569923241f6c1eca3217368a +file antiref99/train/shard-00283.parquet 52523780 b01b328bf0578da5e3883a1bcc343ef3a48223a288b7a89709bc12deb97ce690 +file antiref99/train/shard-00284.parquet 52464699 6d2724b2879c5e67b662c0ba3c5fdcd871e9f44b6339b999c34c98af3df81919 +file antiref99/train/shard-00285.parquet 52440975 40634a7b2703fb801ceae86ecd70f706a4ae0803972be3e88cffd7504218bba9 +file antiref99/train/shard-00286.parquet 52509744 1eba471b075008c4463ce0a64763c905a685060bdad47af9a7b7ed6e346770b4 +file antiref99/train/shard-00287.parquet 52463639 f95e048b151c60d4b2354ee1855030452343b665a60523abf4fbdd93911c92bd +file antiref99/train/shard-00288.parquet 52491787 234d2d23cdf8bb37376b3e37c29a2c61c5bfe5e0fadb1f42a15d634cae99ef3a +file antiref99/train/shard-00289.parquet 52500470 acd6583ce33a9724fd39d8a974ec223cd8447ef04a74a9584ca28144123e1404 +file antiref99/train/shard-00290.parquet 52436281 d7f091baebfc388d28b130518fc8a314e3e2abbbb5ccbc57e6c6c8fb6d7bccf7 +file antiref99/train/shard-00291.parquet 52459708 970fa4429d39159a932c783069274ccb60f4267d762c9c158eb87c2364271175 +file antiref99/train/shard-00292.parquet 52469623 cf7039800599ee17d7e5f381bef1d63b7e3277d23b0183af3ac1d1353fd595b8 +file antiref99/train/shard-00293.parquet 52472111 f8bd31ae5155c2e6a9194a8a8d5cf948bc22d5453ab65d6a7592017a2ee0e355 +file antiref99/train/shard-00294.parquet 52448295 79142dc868fe2766268829876d7e18540f607e5e453eedf3e5d98e28a7a72eae +file antiref99/train/shard-00295.parquet 52468307 e5c3610ea3d933e62173e4c93bc75f50c26782cefc735dc229f1c12e4c468ee8 +file antiref99/train/shard-00296.parquet 52469412 422d1f2b290bb3c234a349bd8102be30fa844925b4369210b3a2cdc04f5618d6 +file antiref99/train/shard-00297.parquet 52483974 65188924df63ee15de932892cca10c58ea0bd94619d628b8329783ed55962db2 +file antiref99/train/shard-00298.parquet 52466746 dcb4d18ebd9e6fb7c9ec92f196b1f85baffb7302da3884b0dc783dbff3b3c2de +file antiref99/train/shard-00299.parquet 52431796 f570c57686053a3d3119cbf635492892017fd74475e6ebb679fb8bdc7b57f972 +file antiref99/train/shard-00300.parquet 52459896 c7014af649858708c25ac11b7b258df0235f32c053cda7a027b47725c818a72e +file antiref99/train/shard-00301.parquet 52493631 5d9b3709d38ded4de29ffd873dafb9ad6b9f416f03a613f45fcbfc199b10434b +file antiref99/train/shard-00302.parquet 52439922 c4f7fe5c07cbe0d115363c15f1a3a4078717963047301cac3f488c606b5355e1 +file antiref99/train/shard-00303.parquet 52514697 0ca6b9f2561c0cb014a96ad722f953252a796b604034b27abf8db2ce3355b4a0 +file antiref99/train/shard-00304.parquet 52472195 f1b40cfa43368a77f74a4d9ae5cf0a2d7f54ffddb5878cec725e37d7b4d45900 +file antiref99/train/shard-00305.parquet 52533472 cbbc5a9172aec52f1e454a3d9d4a7b49a914b132d0774efcd4ec16b11ad74ee6 +file antiref99/train/shard-00306.parquet 52428668 a9af5e3da3c23f53a0dda0e815866ac8f61efc124225a2946932196e3bba8ef9 +file antiref99/train/shard-00307.parquet 52473251 f3fff726a3d213abb15ee68f0bc3b895c294e34aeb6690d7a6b87d3dc8d23eea +file antiref99/train/shard-00308.parquet 52476400 dffb9869aa44b3c7875d1c005b03ac9d1262fa3032b559b0e3b4952ef2ce653c +file antiref99/train/shard-00309.parquet 52483937 e78de7ab5fa74cc41a68b59a5ccc3b4dbb80e38dc2175bb4ff7b05f8f45fc8de +file antiref99/train/shard-00310.parquet 52448727 926dd3833648aa026f384588b0a2f8a74d433cf9c5cfa8a20a5d5075af48c0ff +file antiref99/train/shard-00311.parquet 52488389 da944b04b284efd34d9abd3a1b52c57135a9ca436b477d1a62892be98614731f +file antiref99/train/shard-00312.parquet 52505608 c7ebcc97ea80d53947e53764d437a66e2c8ba9b065e2c3d5d46b3e807f5da3a5 +file antiref99/train/shard-00313.parquet 52435761 15a05a795927e988e8da057257eb20ab70257b1a43a948cc9ca3a6199126808b +file antiref99/train/shard-00314.parquet 52491845 742fd91da65e28f965590d3a6a443e581b42ece69d00265dc0a69dcffe71bc91 +file antiref99/train/shard-00315.parquet 51610643 a2d2498132653189e78f48f1e3bacc924b4027208325b1b1d64559f318c1724f +file eval_gte2.parquet 259750 862f73fe8c1faadcf98c50c5b6c83742dac6e6fc36c1b1331ee4896e6ee28c99 +file eval_singletons.parquet 261751 f28027d3b65fc11e1ac790674d5be1885775973c072da9dfd4e4e91899648552 +file test_gte2.parquet 257271 9456f11b468f016492fc58f03817de55a8b385879a73ded08ee50a92073aa9d5 +file test_singletons.parquet 261764 cc6381e9e0fedb8c447925c9d369ef57e219237ab61f60055166b27ed8688758 +commit: eca1d2f3bfffb9f607e8e033d21ef6f18db98ed6