Skip to content

fix: always pass an explicit mmseqs thread count to abstar.run - #12

Merged
bnemoz merged 2 commits into
mainfrom
v2/mmseqs-threads
Nov 17, 2026
Merged

bnemoz merged 2 commits into
mainfrom
v2/mmseqs-threads

Conversation

@bnemoz

@bnemoz bnemoz commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

For wells with at least 1000 consensus sequences, run() passed mmseqs_threads=None to abstar.run. MMseqs2 then starts one thread per online CPU of the node. It ignores OMP_NUM_THREADS and CPU affinity, and it crashes whenever its thread count is larger than an abstar sub-query batch. We saw this on the 240-core pod: 256 threads against 67 sequences, and 32 threads against a C-region batch of 31.

Fix

  • New run() argument mmseqs_threads: int = 1, with CLI option --mmseqs_threads (≥1). It is documented in the docstring and is now always passed to abstar.run. The < 1000 / None branch is gone.
  • Values that are not a positive int raise ValueError at the start of run().
  • pairplex correct-barcodes (correct_barcodes.py) had the same bug, and hits it sooner: it annotates the raw input reads and passed None once a file had ≥1000 reads. It now passes mmseqs_threads=1 (second commit).
  • The proper fix belongs in abstar: cap the threads by batch size. That should be reported upstream.

Results unchanged, so no benchmark

The thread count does not change annotation output. The benchmark harness already rewrites None to 1, so v2-base benchmark numbers were produced with 1 thread anyway. This PR is therefore checked with tests and a smoke run only, without a benchmark. This option is not tied to legacy: forcing None back would only bring the crash back, with no change to output.

Tests

  • pairplex/tests/test_run_annotation.py mocks abstar.run:
    • the default passes 1 and an explicit value is passed through;
    • a well with ≥1000 consensus sequences (510 synthetic barcodes) still gets mmseqs_threads=1. On v2-base code this test fails with assert None == 1;
    • invalid values (0, -2, None, 1.5) are rejected;
    • correct_barcodes.run on 1212 reads passes mmseqs_threads=1. On v2-base this fails with assert None == 1.
  • Full suite: 24 passed.

Smoke run (2 SimPlex wells, PAIRPLEX_MAX_WORKERS=8)

mmseqs_threads abstar.run per well (s) consensus seqs whole run (s)
1 10.7 / 10.6 736 / 773 33.1
8 8.7 / 8.6 736 / 773 27.9

Annotated outputs (sequence_id, V/J genes, junction; paired and unpaired) are identical at 1 and 8 threads. Most of the per-well annotation time is fixed overhead, so 1 thread costs about 2 s per well here.

🤖 Generated with Claude Code

bnemoz and others added 2 commits November 17, 2026 16:20
run() passed mmseqs_threads=None for wells with >= 1000 consensus
sequences. MMseqs2 then uses every online CPU of the node (ignoring
OMP_NUM_THREADS and CPU affinity) and crashes when its thread count
exceeds the size of an abstar sub-query batch (seen: 256 threads vs 67
sequences, 32 threads vs a C-region batch of 31).

New run() argument mmseqs_threads (default 1, CLI --mmseqs_threads),
always passed explicitly. Annotation is not the bottleneck. Results do
not depend on the thread count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
correct_barcodes.run annotates the raw input and passed
mmseqs_threads=None once the input had >= 1000 reads: the same MMseqs2
all-node-CPUs crash as in run(). Tests: correct-barcodes case added;
the large-well test now uses 2 workers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bnemoz
bnemoz merged commit 7dc8330 into main Nov 17, 2026
8 checks passed
bnemoz added a commit that referenced this pull request Nov 17, 2026
…e tso_g_max/legacy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bnemoz added a commit that referenced this pull request Nov 17, 2026
…alongside barcode_prefer_exact/legacy)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant