Skip to content

v2 lane C: support columns, per-stage drop counts, abstar failure table, QC report - #19

Merged
bnemoz merged 4 commits into
mainfrom
v2/support-counts-qc
Nov 18, 2026
Merged

bnemoz merged 4 commits into
mainfrom
v2/support-counts-qc

Conversation

@bnemoz

@bnemoz bnemoz commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

PairPlex v2, lane C: carry support into the outputs, count drops at every stage, add a QC report. This covers fixes 9 and 10 (the PairPlex-side parts), plus "report drop counts at every stage", "count empty reads instead of skipping them silently" and the AbStar failure table, all from Core fixes to do.md.

Pairing is unchanged. All logic is in the new pairplex/qc.py. run() gets only small hooks, so lane B (pairing block) and lane D (parser) can merge without conflicts.

What changes

1. Support columns (support_columns=True)

  • Unpaired outputs gain umis, reads, cluster_fraction, barcode and contig. contig is contig-N, where N is the rank by reads within the barcode. These are joined from metadata/<well>.csv on the contig name, which is the sequence_id.
  • Paired outputs gain umis:0/1, reads:0/1, cluster_fraction:0/1, barcode:0/1 and contig:0/1. These are the same objects, so assign_pairs carries them through.
  • umi / umi:0/1 are dropped instead of filled.
    • That column is abstar's per-read UMI, parsed from the read header via umi_pattern. PairPlex never sets umi_pattern.
    • A consensus contig stands for many molecules, so there is no single UMI to report. Putting a count into a UMI-sequence field would contradict abstar/AIRR semantics.
    • The count is now in umis, the name abutils already uses for 10x data.
    • The AIRR-standard names would be umi_count / consensus_count. This is easy to rename if preferred.
  • Pair._chain_selector sorts by umis only when a barcode has several heavies or lights. The 1H1L rule removes those, so the emitted pairs stay the same.

2. qc/stage_counts.csv (per well and a TOTAL row)

  • Reads:
    • input reads;
    • reads with a valid barcode;
    • reads dropped by orientation/barcode.
  • Barcode groups:
    • barcode groups;
    • groups dropped by min_cluster_reads, and their reads;
    • empty sequences that greedy_clustering skips. These are counted only; behaviour is unchanged.
  • Clusters:
    • clusters;
    • clusters failing reads / umis / fraction, each counted independently;
    • clusters passing.
  • Annotation:
    • consensus sequences sent to abstar;
    • rows returned;
    • annotated;
    • failed annotation, split into unassigned and record errors (abstar drops these with no row);
    • productive, unproductive, no junction.
  • Chain composition of barcodes from assign_pairs, taken before the 1H1L rule: 1H1L, 1H0L, 0H1L, 1H2L, 2H1L, 2H2L, 0H0L, other.
    • 0H0L was added because barcodes whose contigs all fail annotation would otherwise dominate "other".
  • Emitted pairs.

Everything is counted from frames that run() already holds. Nothing is re-parsed.

3. qc/annotation_failures.csv

Columns: well, kind, mode, count, plus TOTAL rows. On by default because it is cheap. Kinds:

  • record_error: abstar reports these only through a RuntimeWarning that points to its failures.tsv. PairPlex captures that warning, reads the file and then re-emits the warning. The mode is stage/category exception: message.
  • unassigned: abstar's failure_reason.
  • unproductive: one count per productivity_issues item.
  • no_junction.

4. qc/report.json + qc/report.md (written at the end of run())

  • Fragment fraction. Share of passing consensus sequences that fail annotation or are truncated, by locus. Truncated means the V alignment starts more than 30 nt into germline V, or there is no junction. abstar's complete_vdj is reported next to it.
  • UMI error signature. Hamming-1 UMI pairs within a barcode, compared with a null where barcodes are shuffled across (barcode, UMI) rows.
    • The count is exact and linear: each Hamming-1 pair shares exactly one key with one position masked.
    • Wells with more than 200k distinct (barcode, UMI) rows are subsampled by barcode.
  • Ambient proxy. Share of distinct productive IGH junctions seen under ≥2 and under ≥10 barcodes across the run.
  • Leakage f by chain (heavy, light, IGK, IGL). Computed with pooled_pairing.estimate_leakage on the run's own per-well outputs when there are ≥8 wells.
  • Pair confidence distribution. Only when a confidence column exists, in pairs/pairs.parquet or in the paired outputs.
  • Clonal light-chain coherence.
    • Clone = heavy V + CDRH3 aa.
    • Purity is cell-weighted over clones with ≥2 cells; the per-clone mean is also reported.
    • Chance = Σp² over light V.
    • IGKV/IGLV D- duplicates are normalised, as in the harness.
    • The report carries an explicit caveat: purity only makes sense for memory/expanded repertoires.

Flags

  • New run() kwargs, documented in the docstring: support_columns=True, qc=True, legacy=False.
  • legacy=True forces both new options off.
  • CLI: --support_columns/--no_support_columns, --qc/--no_qc, --legacy.
  • No new required options.

Local checks (2 simulated wells, 2 workers)

  • legacy=True: the paired/unpaired parquets are identical to v2-base, columns included.
  • Default:
    • the paired/unpaired parquets are identical to v2-base on every shared column;
    • only umi was removed and the support columns were added;
    • the metadata is identical.
  • Accounting is consistent: clusters_pass == consensus_to_annotation and annotation_returned + failed_record_error == consensus_to_annotation.
  • Unit tests: pairplex/tests/test_qc.py has 21 tests on small synthetic frames, one or more per counter and metric. The full pairplex suite gives 37 passed.

Benchmark

C-support-qc-v1 (default kwargs, all 4 datasets) vs v2-base: OVERALL PASS. Runtime was 17386 s -> 19116 s (1.10x).

Simulated sets:

[SAME] sim_cal / per_well
   precision            0.593 ->      0.593
   cell_recall          0.126 ->      0.126
   emitted               1413 ->       1413
[PASS] sim_cal / pooled
   precision            0.627 ->      0.628
   cell_recall          0.152 ->      0.152
   emitted               1585 ->       1585
   n_conf0.9               41 ->         41
   prec_conf0.9         0.878 ->      0.878
[SAME] sim_imp / per_well
   precision            0.865 ->      0.865
   cell_recall          0.199 ->      0.199
   emitted               1632 ->       1632
[PASS] sim_imp / pooled
   precision            0.967 ->      0.967
   cell_recall          0.871 ->      0.871
   emitted               6368 ->       6369
   n_conf0.9             4387 ->       4387
   prec_conf0.9         0.987 ->      0.988

Real sets (clonal purity delta):

set verdict purity delta
real_d02 / per_well PASS +0.00 pt
real_d02 / pooled PASS +0.01 pt
real_d07 / per_well PASS +0.00 pt
real_d07 / pooled PASS +0.05 pt

Lineage retention on the real sets:

  • per_well: PASS on both, cells recovered 1.000 at every clone size.
  • pooled: PASS on both, cells recovered 0.98–1.00, 0 clones lost.

Why not every set is SAME. v2-base is not deterministic from run to run, and this PR does not change pairing logic.

  • parse_barcodes chunks are collected with as_completed. The row order of the parsed reads in a well therefore changes between runs. I checked two local runs of v2-base code: same reads, different order.

  • That order feeds greedy-clustering ties and POA input order. Between the two benchmark runs, 12/96 (sim_cal) and 28/96 (sim_imp) wells have a few clusters with the same read counts but a 1-nt different consensus or a different UMI count.

  • Those small consensus differences propagate to a few annotated contigs and so to pooled pairs. Per-well paired outputs are identical in all 192 sim wells.

  • This PR's hooks only read those frames: they never reorder or filter them.

  • A sims-only rerun of unmodified v2-base (C-v2base-rerun-sim) measures this noise floor: the unmodified code shows the same pattern against v2-base:

    • per_well is SAME on both sims;
    • sim_cal pooled: precision 0.627 -> 0.628, emitted 1585 -> 1585;
    • sim_imp pooled: precision 0.967 -> 0.968, recall 0.871 -> 0.872, emitted 6368 -> 6370.

    The shifts seen here are therefore inside run-to-run noise.

  • Making the parse order deterministic (sort chunks, or sort reads by seq_id) would make future benchmarks exact. That change belongs with lane D (parser).

Notes for other lanes / upstream

  • abstar: complete_vdj is False for every IGK contig on the sim (554/554), including contigs that start at germline position 2. The fragment metric therefore uses v_germline_start instead.
  • abstar: when a run has record failures, abstar keeps a /tmp/abstar-* diagnostics directory per call. That uses pod ephemeral storage on large runs.
  • abstar: about 1% of consensus sequences fail with ValueError: CDR3 coordinates fall outside the assembled V(D)J sequence and are dropped with no row. They are now visible in annotation_failures.csv.
  • Lane D (parser / clustering): cluster_fraction uses the full partition size as its denominator, including the empty reads that clustering skips.
  • Lane D (parser / clustering): in run(), the parquet_chunks == [] branch continues without updating the progress bar or cleaning up.

🤖 Generated with Claude Code

bnemoz and others added 4 commits November 17, 2026 16:18
Adds pairplex/umi.py:collapse_umis_directional (UMI-tools directional adjacency,
neighbour generation, N-containing UMIs absorbed but never absorbing). process_droplet
collapses UMIs per droplet partition: umi_raw keeps the raw UMI, metadata umis is the
corrected count and umis_raw the raw count; min_cluster_umis filters on corrected counts.

run(umi_collapse='directional'|'none'), CLI --umi-collapse; run(legacy=True) / --legacy
forces 'none' (v2-base behaviour).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e, QC report

- annotated outputs carry cluster support: umis, reads, cluster_fraction,
  barcode, contig (unpaired) and umis:0/1, reads:0/1, ... (paired); the
  always-null abstar `umi` column is dropped (a consensus contig has no
  single UMI; the count lives in `umis`)
- qc/stage_counts.csv: drop counts at every stage, per well and TOTAL
  (reads, barcode groups, empty sequences skipped by greedy_clustering,
  clusters failing each filter, annotation outcomes, chain composition,
  emitted pairs)
- qc/annotation_failures.csv: abstar record errors (from its failures.tsv
  warning), unassigned reasons, productivity issues, missing junctions
- qc/report.{json,md}: fragment fraction by locus, UMI Hamming-1 excess
  vs shuffled null, heavy-junction multi-barcode share, leakage f by
  chain (>= 8 wells), pair confidence distribution, clonal light-chain
  purity with chance and a memory-only caveat
- new run() kwargs support_columns=True, qc=True, legacy=False (legacy
  forces both off and reproduces v2-base); matching CLI options
- pairing is unchanged; logic lives in pairplex/qc.py behind small hooks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…x/barcode_prefer_exact alongside umi_collapse; legacy forces all)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… all v2 options; abstar always gets an explicit mmseqs_threads)

Stacked on #18 (v2/umi-collapse-perwell): merge #18 first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bnemoz
bnemoz merged commit 6681cd8 into main Nov 18, 2026
8 checks passed
bnemoz added a commit that referenced this pull request Nov 18, 2026
- one legacy flag forcing every v2 option (now also pairing='exact')
- qc chain-composition counts taken before the pairing rule; emitted_pairs from pair_well
- umis:0/1 keep #19's meaning (the contig's own UMI count); the ranking support summed over
  merged same-clonotype contigs moves to chain_umis:0/1 (also in pairs/pairs.parquet)

Stacked on #19 (which is stacked on #18): merge #18, then #19, then this.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bnemoz added a commit that referenced this pull request Nov 18, 2026
legacy also forces pooling='per_well'; the per-well-only options (umi_collapse, pairing,
min_dominance, collapse_clonal, support_columns, qc) are documented as not applying in
cross-well mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant