Skip to content

fix(parser): strip only the TSO's G-run (tso_g_max=3), not every leading G - #15

Merged
bnemoz merged 2 commits into
mainfrom
v2/tso-g-strip
Nov 17, 2026
Merged

bnemoz merged 2 commits into
mainfrom
v2/tso-g-strip

Conversation

@bnemoz

@bnemoz bnemoz commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Problem

parse_barcodes takes the insert as s[36:].lstrip("G"). Read layout: barcode 16 + UMI 10 + TSO TTTCTTATATGGG, where positions 26–35 are TTTCTTATAT and the TSO's rGrGrG run starts at position 36. lstrip("G") removes the TSO run and also every genuine G at the 5′ end of the insert.

In SimPlex the cDNA starts directly at FR1 codon 1. That codon begins with G for many genes (e.g. GAC ATC…, GAA ATT…, GAG GTG…), so v2-base cuts the first V base off those chains. In real data the insert starts in the 5′UTR, so the bases lost there sit outside V(D)J.

Fix

  • strip_tso_g(sequence, tso_g_max) in utils.py removes at most tso_g_max leading Gs. parse_barcodes(..., tso_g_max=3) uses it.
  • New run() arguments, documented in the docstring:
    • tso_g_max: int = 3 (CLI --tso_g_max);
    • legacy: bool = False (CLI --legacy). legacy=True forces lstrip("G").
  • pairplex correct-barcodes also calls parse_barcodes, so it gets the new default too. It has no legacy flag.

G-run length at position 36 (real data, barcode-first orientation)

Method: stream the reads of one real memory-B-cell well; the file holds 111,307 reads, all of them parsed. Use the same orientation logic as parse_barcodes; 99.998% of reads are barcode-first. Count the run of Gs starting at position 36. The "exact TSO" column keeps only reads whose positions 26–35 read exactly TTTCTTATAT. A second well (first 200k reads) gives the same shape (table below).

G-run   all parsed reads (n=111,307)            exact TSO prefix (n=19,793)
  0   44.8% ###########################        21.1% #############
  1   24.0% ##############                     20.0% ############
  2    9.6% ######                             12.0% #######
  3    5.3% ###                                11.2% #######
  4    8.5% #####                              18.5% ###########
  5    5.5% ###                                12.0% #######
  6    0.9% #                                   2.0% #
  7    1.0% #                                   2.4% #
  8    0.3%                                     0.6%
  9    0.2%                                     0.2%
 10+   0.0%                                     0.0%

Second well, exact-TSO reads (n=24,642): 0: 22.4%, 1: 21.5%, 2: 12.8%, 3: 10.2%, 4: 16.1%, 5: 11.7%, 6: 2.3%, 7: 2.3%, 8+: 0.6%.

Reading the distribution

  • Runs of 0–2 are mostly sequencing errors, not short TSO runs. Positions 26–39 are low-diversity cycles: every read carries the same TSO there. Their mean base quality is about Q23, against Q36–39 on either side. Only 18% of reads have an exact TTTCTTATAT. G is still the most common base at positions 36–39 (55%, 47%, 48%, 68%), well above the ~25–30% G seen from position 42 on.
  • Among runs of 3 or more, the mode is 4 and 5 is common. Position 39, the 4th G, is G in 68% of reads, and position 40 in 45%. Runs of 6 or more are rare (≤5% of exact-TSO reads). So the TSO's 3 rGs are usually followed by 1–2 more Gs, which matches the reverse-transcribed cap / non-templated addition.
  • Why k = 3 and not 4: the costs are lopsided.
    • With k = 3, the reads with a 4th or 5th G (about 30% of exact-TSO reads) keep 1–2 extra Gs at the very 5′ end of the insert. That is the 5′UTR, so annotation and V calls are unaffected, and a one-base end difference barely moves the clustering identity (distance / min length on reads of about 500 nt).
    • With k = 4, one genuine G is lost whenever the run is exactly the TSO's 3 Gs (about 11% of exact-TSO reads) or the insert starts with G. In SimPlex, where the insert starts at FR1, k = 4 would cut FR1 codon 1 just as lstrip did.
    • So 3 is the number of Gs the TSO itself contributes, and the only value that never removes a templated base. Use tso_g_max=4 to strip one non-templated G as well.

Tests

New file pairplex/tests/test_tso_g_strip.py:

  • strip_tso_g edge cases (k = 0, 3, 4, None; short runs; empty strings);
  • parse_barcodes on constructed reads (…TTTCTTATATGGG + GGACAGTC keeps GGACAGTC at k = 3);
  • run() default keeps a genuine 5′ G, and legacy=True restores lstrip, even when tso_g_max is set as well;
  • invalid tso_g_max values are rejected.

Full suite: 36 passed.

Smoke run on 2 SimPlex wells with 2 workers: with legacy=True the metadata, annotated paired/unpaired tables and consensus sequences match v2-base exactly (consensus FASTA records come out in a different order because of as_completed). With the default, the consensus set changes slightly: well000 has 893 → 892 metadata rows and 736 → 735 annotated sequences.

Benchmark

Pending: D3-tso-g3-v1 (default tso_g_max=3) vs v2-base. Results will be added here.

🤖 Generated with Claude Code

bnemoz and others added 2 commits November 17, 2026 16:26
parse_barcodes did s[36:].lstrip("G"), which also removed genuine 5'
Gs of the insert after the TSO's rGrGrG. Strip at most tso_g_max Gs
(default 3). New run() options tso_g_max (CLI --tso_g_max) and legacy
(CLI --legacy); legacy=True restores lstrip("G") exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e tso_g_max/legacy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bnemoz
bnemoz merged commit c3be220 into main Nov 17, 2026
8 checks passed
bnemoz added a commit that referenced this pull request Nov 17, 2026
- legacy=True now forces both tso_g_max=None and barcode_prefer_exact=False
- parse_barcodes takes tso_g_max, prefer_exact and return_counts together
- fix a silent semantic conflict: the new exact-hit orientation path still used
  s[36:].lstrip("G"), which would have bypassed #15 for nearly every read; it now uses
  strip_tso_g(s[36:], tso_g_max). Added tests covering both orientation paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant