Repository navigation
Fix V(D)J boundary consistency, unify public output (0.9.0 breaking), add corpus CI - #135
Merged
Merged
Conversation
GitHub rejects workflows that reference the runner context in job-level env, so the corpus job never ran. Derive CORPUS_OUTPUT and MPLCONFIGDIR from RUNNER_TEMP in a step and guard against the pattern in tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add sequence_vdj and sequence_vdj_aa, make OUTPUT_SCHEMA the single public field order (AIRR-required fields first), and replace to_airr_row with to_public_row, which maps every schema coordinate pair to 1-based closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Annotation workers now build each record's public row once, so work Parquets, dataframe and Sequence returns, final Parquet, and AIRR TSV all read the same values. Writers only encode; the per-row Parquet conversion loop is gone. A row whose conversion fails becomes an isolated record failure. Tests now assert public meanings: 1-based closed coordinates, sequence as the input, and the V(D)J in sequence_vdj/sequence_vdj_aa. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The consistency audit now slices and checks region adjacency with 1-based closed coordinates, matching 0.9 public Parquet. The corpus builder notes that its source Parquets predate the change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Recorded with --record-baseline and reviewed against the previous baseline for all 49,999 annotated records: the only differences are the new sequence_vdj/sequence_vdj_aa columns (sequence_vdj equals the oriented 1-based V(D)J slice) and every *_start shifted by exactly one. A full corpus check then passed with zero differences. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a 0.9.0 breaking-change warning with a before/after record to the output-format guide, a new changelog page, and the sequence_vdj fields. Python API, README, AGENTS, and edge-case docs now state that every output shares 1-based closed coordinates and AIRR sequence meanings, with Antibody internals the only 0-based representation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The audit now requires sequence_vdj, which only 0.9+ Parquet carries, so 0.8.x files with 0-based coordinates fail clearly instead of flagging every record; it also checks sequence_vdj against the oriented slice. The changelog notes this and that invalid-interval records now fail individually in every output format. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
baltic 1.0 turned baltic.leaf into a module. abutils 0.6 uses it in a typing.Union annotation, which Python 3.10 rejects at import time, so `import abstar` fails on 3.10 before any test runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abutils 0.6.1 constrains baltic<1 itself, so abstar no longer needs its own baltic stopgap. Drop the arbitrary <0.7 cap on abutils, and move the lowest-dependency CI profile and the corpus profile to abutils 0.6.1. The corpus check with abutils 0.6.1 passed with no differences from the baseline (49,999 annotated, 1 expected failure, 0 inconsistent). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The <2 and <26 caps excluded unreviewed releases rather than a known incompatibility. With polars 2.1.0 and pyarrow 26.0.1 on Python 3.12, and polars 2.0.0 and pyarrow 25.0.1 on Python 3.10, the full suite passes (1689 passed, 4 skipped), and the corpus check matches the baseline (49,999 annotated, 1 expected failure, 0 inconsistent). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Incomplete junction anchors and region boundaries could abort annotation or produce inconsistent V(D)J region assemblies and masks. Recover boundaries conservatively from retained alignment evidence, partition amino-acid regions in the continuous query frame, and synchronize the FWR3/CDR3 endpoint. Ambiguous recovery remains an explicit record failure.
Per-sequence annotation errors now preserve diagnostics and a failure index while other records continue;
strict=True/--strictrestores abort-on-record-error behavior. Add focused biological regressions, native-Parquet auditing and exact run comparison tools, anddocs/EDGE-CASES.mdwith grouped handling procedures and examples.Replace the nightly corpus workflow with one push/PR job using a committed 50,000-sequence human BCR cohort: 40,000 stratified background records and 10,000 difficult records. Authenticate source identities and provenance, enforce conservation and independent consistency checks, and compare all 168 public output fields exactly against the reviewed regression baseline. Preserve duplicate IDs and require one individually documented ambiguous-boundary failure. Retain reports and diagnostics as workflow artifacts, document assignment-drift investigation and baseline review, and exclude the corpus from Python distributions.
Validation:
The five-minute CI target still needs a hosted GitHub measurement; local timing excludes dependency installation. The frozen corpus baseline establishes regression behavior, not independent biological truth. The earlier full-source-corpus recovery investigation and its five separate assignment-reproducibility differences are documented in
docs/superpowers/reports/2026-09-07-junction-anchor-validation.md.Unified public output (breaking change for 0.9.0)
Every public output (AIRR TSV, Parquet, and the dataframe or
Sequenceobjects returned byabstar.run()) now has the same fields, in the same order, with the same meanings, built by one conversion (abstar.annotation.airr.to_public_row) that runs once per record in the annotation workers. Writers only encode.*_startis one greater than in 0.8.x; every*_endis unchanged.sequenceis the original input everywhere, including returnedSequenceobjects. The assembled V(D)J (no constant region) and its translation are the newsequence_vdj/sequence_vdj_aafields, used for every locus; the masks index them.scripts/audit_annotation_consistency.pyuses 1-based coordinates, checkssequence_vdj, and rejects 0.8.x Parquet.output_formats.rst, and a new changelog page (0.9.0, unreleased).version.pyis not bumped.Also fixes the corpus workflow, which GitHub had rejected because it used the
runnercontext in job-levelenv; the corpus job now runs (about 5m22s on hosted runners).Validation for this part:
sequence_vdjequals the oriented 1-based V(D)J slice) and every*_startshifted by exactly one. A full corpus check then passed with zero differences.🤖 Generated with Claude Code