Repository navigation
Stream dense seed context during mate and SA recovery - #110
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dense regional inputs made mate/SA recovery retain millions of ordinary alignment objects and unrelated query names. Stream that context and keep recovery objects only for pointer-bearing templates and records with diagnostics. Every original context record and duplicate remains in the output; unchanged outputs share the seed BAM, and changed outputs are verified against the complete record multiset.
Partner queries include all pointer-bearing seed names in every round, including candidates needed by later recursive links. A second seed pass preserves partners without their own outgoing pointers. Recovery request schema 3 separates the changed queries from existing cached results. Version: 0.15.3.
Validation: ruff; 405 tests, including dense-context preservation and seed partners without pointers. Public Sid 10x retries acquired all 773,288 IGKC records in 23 seconds (~1.1 GiB peak RSS) and all 5,669,578 FTH1/FTL records in 235 seconds (~4.2 GiB peak RSS), without acquisition limits. No downsampling, region narrowing or quality/tag changes. Hash/count provenance still scales with the number of distinct records.
Fixes #109. Related: openvax/isovar#436. Pipe cleanup issue #106 remains separate and open; this PR does not claim to fix it.