Verify real Sid RNA phasing through protein effects

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Change Log
 
+## [v10.11.6](https://github.com/openvax/varcode/tree/v10.11.6) (2026-10-05)
+
+- Exercise real Sid RNA evidence through `VariantCollection.effects()` for
+  NTF3 compound codons and MAP2 complex haplotypes (#464). Pin independent
+  NCBI coding/protein references, retain complete available sequencing
+  templates, and verify individual predictions, phase uncertainty and joint
+  effect provenance. Keep corrected, deletion-only and historical MAP2
+  alleles distinct; RNA support does not replace their DNA predictions.
+
 ## [v10.11.5](https://github.com/openvax/varcode/tree/v10.11.5) (2026-10-05)
 
 - Test RNA read phasing and molecular phase resolution on real Sid RNA

tests/README.md

@@ -227,7 +227,7 @@ cache); later runs reuse them offline.
 in [#528](https://github.com/openvax/varcode/issues/528). It uses the published
 `openvax-v2` bundle, with manifest SHA256
 `b42dbce529cb5724153f035fca3783a41e673b31610a30851b2ef96d4ba43734`.
-The test exports three named members into a pytest temporary directory,
+The phasing checks export three named members into a pytest temporary directory,
 preserving their original alignment records and indexed BAM format:
 
 | Source label | Target/member suffix | Records | Templates | Supplementary records |
@@ -294,3 +294,103 @@ python -m pytest -q tests/test_osteosarc_rna_phasing.py
 
 The first run can download the verified 28 MB shared bundle; later runs reuse
 it offline. No sibling checkout or BAM copies in `tests/data` are required.
+
+## Real Sid RNA protein effects
+
+The same module passes `MolecularPhaseResolver(RNAReadPhasingSource(...))` to
+`VariantCollection.effects()` for NTF3 and complex MAP2 cases, following
+[#464](https://github.com/openvax/varcode/issues/464). It checks every individual
+DNA prediction against the collection without RNA phasing, including other
+transcripts, and verifies the selected transcript's complete protein against
+independent expectations. NTF3 runs with `fast`, `protein_diff` and the opt-in
+`transcript_model`; MAP2 runs with the two established point-edit annotators.
+Joint legacy effects expose their edited cDNA, and the transcript model's
+realized candidates expose theirs; the tests check their CDS, affected codon
+and stop codon. Joint membership, protein and phase provenance survive JSON.
+The experimental model currently loses its realized candidates/hypotheses
+and their cDNA on JSON round-trip; this reproduced limitation is tracked in
+[#574](https://github.com/openvax/varcode/issues/574). Its full CDS/codon checks
+use the in-memory products, and its round-trip checks cover the preserved
+protein, membership and phase source. The audit also reproduced a support-cache
+key defect ([#575](https://github.com/openvax/varcode/issues/575)); returned
+counts are correct here, but repeated calls rescan the BAM.
+
+`tests/data/sid_rna_effect_references.json` is a separate offline reference
+asset (SHA256 `d9e21cc96450d4a14e7b41d0900df48792c9f64b103b9ce01ab5cc841508d29e`).
+On 2026-10-05, NCBI EFetch supplied these versioned primary records:
+
+| Gene | RefSeq mRNA / protein | GenBank CDS, one-based inclusive | Ensembl 81 transcript |
+|---|---|---|---|
+| NTF3 | [NM_002527.5](https://www.ncbi.nlm.nih.gov/nuccore/NM_002527.5) / NP_002518.1 | 84–857 | ENST00000331010 |
+| MAP2 | [NM_002374.4](https://www.ncbi.nlm.nih.gov/nuccore/NM_002374.4) / NP_002365.3 | 321–5804 | ENST00000360351 |
+
+The acquired CDS and `/translation` protein strings match those Ensembl
+transcripts exactly. Independently retrieved GRCh38 chromosome windows
+`NC_000012.12:5494300–5494500` and `NC_000002.12:209694750–209694830` each match
+one interval of the corresponding RefSeq CDS, at zero-based offsets 85 and
+2579. The asset retains these strings, URLs and response/sequence hashes.
+To repeat acquisition, fetch each recorded URL, extract its CDS interval and
+`/translation`, and compare the sequence hashes. GenBank descriptive metadata
+can change while the accession's sequence remains the same.
+
+The expected edits were applied directly to these RefSeq CDS strings and
+translated independently with [NCBI's standard genetic code](https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi#SG1),
+without importing Varcode, PyEnsembl or using upstream peptide annotations.
+The asset records fixed CDS edit offsets, codons, literal protein replacements,
+complete mutant CDS/protein hashes, lengths and stops. Tests read those frozen
+expectations; they do not compute a second prediction from Varcode.
+
+- NTF3's zero-based CDS offsets 166 and 167 give `AAG>AGG` (`p.K56R`) and
+  `AAG>AAT` (`p.K56N`) individually. Together they give `AAG>AGT` (`p.K56S`),
+  with local peptide `DVSENY` instead of `DVKENY`. Both short-read and ONT cis
+  evidence produce this joint effect while retaining both individual effects.
+- The real ONT trans companion `12:5494466 G>A` changes `CCG>CCA` at proline
+  84. Its silent effect and the separate `p.K56R` prediction remain present.
+  Increasing the compound pair's minimum to 10 leaves its nine observations
+  insufficient: phase is unknown, and the two individual predictions remain.
+
+The MAP2 reads use three additional `openvax-v2` members under
+`isovar/osteosarc/figure_comparisons/corpus/context-evidence.json.gz#/map2/sources/`:
+
+| Member suffix | Source | Regional member records | Complete available records / templates | Q20 compound templates |
+|---|---|---:|---:|---:|
+| `11/original_records` | `IPISRC044_tumor_T1_ucla_rna.md` | 123 | 125 / 78 | 4 |
+| `12/original_records` | `IPISRC044_tumor_T2_ucla_rna.md` | 25 | 26 / 18 | 2 |
+| `3/original_records` | `IPISRC044_T2_sclrs_ONT_dedup` | 32 | 33 / 32 | 1 |
+
+These are original regional acquisitions retained in the neutral bundle. The
+fixture selects their RG/QNAMEs, then copies **all** available records of those
+templates from the bundle source into an indexed temporary BAM. No alignment
+fields change; record digest multiplicities must equal the source's complete
+template subset. The ONT input retains one supplementary record, which the
+default evidence filter excludes. The two bulk inputs have no supplementary
+records. Source products are RNA products, checked by their manifest URLs.
+
+The corrected GRCh38 MAP2 allele is explicitly decomposed into
+`2:209694769 C>A`, `2:209694770 T>G` and deletion of
+`2:209694773–209694800 GCTACTGTGTGTTCAATAAGTACACAGT`. The existing
+`register_haplotype()` API tests this hypothesis's original sequence between
+five-base genomic flanks: `AAGACAGGGCCCAT`. Independent aligned-position,
+query-sequence and quality inspection found four T1 bulk, two T2 bulk and one
+T2 ONT supporting RG/QNAME templates. The bulk witnesses encode a `28N` gap;
+the ONT witness encodes `22D2M6D`. Exact context allows the same RNA sequence
+to support the hypothesis without assigning a genomic deletion mechanism.
+Default MAPQ/Q20, pairing, flag, five-base edge and two-fragment filters remain
+in force; ONT uses the existing 5% error allowance. Missing or different context
+stays unknown and supplies no reference/trans evidence.
+
+The bulk evidence yields a joint 904-aa protein: reference residues 1–866
+followed by `RAHCHHLFKTVRIYQGRVVPFTKALMIKFEEIWPQTFH`, then `TGA`. The single
+ONT compound witness leaves phase unknown and yields no joint prediction.
+Separate predictions for the two substitutions and deletion remain in all cases.
+Additional checks retain the corrected complex allele, the 28-base deletion
+alone (904 aa, with `LAH` instead of `RAH` at the junction), and the historical
+22-base deletion (906 aa, with `STQSH` at the junction) as distinct DNA predictions.
+Only the corrected compound sequence matches these RNA witnesses. Zero matches
+for the other hypotheses do not remove their effects or establish their absence.
+
+These proteins are reference-frame predictions for observed local sequence
+combinations. The tests do not reconstruct full RNA molecules, establish an
+expressed isoform, validate tumor haplotypes, estimate allele fractions from the
+bounded bundle, or pool processed libraries as independent molecules. No runtime
+dependency, RNA assembly implementation or global phasing policy changes.

tests/data/README.md

@@ -13,6 +13,7 @@ No BAM/CRAM files or shared RNA bundles belong here. See
 |---|---|
 | `osteosarc_variants.json`, `osteosarc_snapshot_2026-09-18t.zip` | [Snapshot identity, receipts, archive hash and offline export recipe](../README.md#regenerate-from-the-verified-snapshot). `../collect_osteosarc_variants.py` exports the reviewed snapshot with the pinned Osteosarc version. |
 | `osteosarc_observed_junctions.json` | Each record retains its source page, ONT read name, reference identity and source-sequence SHA256. [Shared read verification](../README.md#shared-test-data-openvax-v2) uses the external bundle; only the bounded junction windows are stored here. |
+| `sid_rna_effect_references.json` | Versioned NCBI RefSeq CDS/protein records and GRCh38 genomic windows, acquired 2026-10-05 independently of Varcode/PyEnsembl. URLs, accession versions, response and sequence hashes, CDS edits and manually audited protein expectations are recorded in the file. [RNA effect verification](../README.md#real-sid-rna-protein-effects) explains the audit. |
 | `osteosarc_esvee_somatic.vcf`, `osteosarc_esvee_shared_svs.vcf` | Published Sid esvee records with FORMAT/sample columns removed. The subset selection and reference releases are documented in [the fusion tests](../test_osteosarc_fusions.py) and [shared SV tests](../test_osteosarc_shared_svs.py). |
 | `real_callers/*.vcf` | [Pinned GATK source and synthetic Strelka2/VEP construction notes](real_callers/README.md). |
 | `spec_examples/*.vcf` | [VCF specification sections and source](spec_examples/README.md). |
@@ -25,8 +26,8 @@ No BAM/CRAM files or shared RNA bundles belong here. See
 
 The original external acquisition recipes for the legacy inputs were not
 recorded. Their repository history is the available provenance; no new upstream
-identity or scientific validation is inferred here. All fixture payloads in
-this release are byte-for-byte those in
+identity or scientific validation is inferred here. The legacy fixture payloads
+are byte-for-byte those in
 [commit 6f29cf8](https://github.com/openvax/varcode/tree/6f29cf8c5201189ae2803ad319aacca8053f7ddf/tests/data).
 To recover those exact payloads, use the raw files under that commit's
 `tests/data/` tree and verify them against `SHA256SUMS`. The new inventory notes

tests/data/SHA256SUMS

@@ -1,4 +1,4 @@
-b91532598e8ef0034ac5f9db42ad23b80d4881aac57e278c1c70db57c77c25f1  README.md
+12f256e28c8420d5f0e6a22a89c9fb714f4f96f85c0474d6978f1df7b1ffa9f7  README.md
 e569c37ac74f13111be67d230f0bf03891a419af31afb04c7e3a28212cba8bac  dbnsfp_validation_set.csv
 a3d5571678f7a4717bec59325fe77131d54d814b27f99c494d944d78a07de501  different-samples.1.vcf
 b2cb6090f568cfacdb0df4a9c280161923c255712fa368ac433863a623d126a5  different-samples.2.vcf
@@ -26,6 +26,7 @@ eabc38358591c380ddb676e2d53a564cb01a33e428b67186a009f5b3cad4ce93  real_callers/s
 91c8c7dd7e9915a73fd0e709dde8d5cc74d4f8fa3af14df0dc63dfd400771674  real_callers/vep_annotated_csq.vcf
 a3d5571678f7a4717bec59325fe77131d54d814b27f99c494d944d78a07de501  same-samples.1.vcf
 a3d5571678f7a4717bec59325fe77131d54d814b27f99c494d944d78a07de501  same-samples.2.vcf
+d9e21cc96450d4a14e7b41d0900df48792c9f64b103b9ce01ab5cc841508d29e  sid_rna_effect_references.json
 94bf15f0cca60a2d2e14526cc1da7a3933aaedea18f57ba4ada75462ad8943b6  simple.1.vcf
 6910e2b188142b0a9702aac57cc88d29987c47e9d4adc1db0bc24b3543d86b47  simple.2.vcf
 5d988950e60a08f40fb406d7d40a4e2bf2c4df402c73395753db39933d8409ea  somatic_hg19_14muts.space_in_sample_name.vcf