Minor release 0.1.0

[northwestwitch]

[0.1.0]

Added

• Use the rust-htslib for reading VCF files (Use rust-htslib to read VCF files #1)
• First Variant model and parsing modules (Added first Variant model and coordinates parser #2)
• Parse alleles and missing docstrings (Parse alleles and add docstrings #3)
• Variant type enum (clinical or research) key/values passed to main function (Pass and save variant type (clinical or research) #5)
• Custom issues and pull requests templates (Custom issues and pull requests templates  #9)
• Pass case _id from CLI (Pass case #10)
• Parse variant.FILTER field (Parse variant.FILTER field from VCF #11)
• Parse variant.QUAL field (Parse and collect INFO.QUAL #12)
• Parse compounds field (Parse compounds #14)
• Parse the 4 variant IDs: simple_id, variant_id, display_name, document_id (Parse the 4 types of variant IDs #15)
• Parse rank_score and norm_rank_scorefrom VCF (Rank score and normalised rank score from VCF (genmod) #16)
• Infer complete coordinate data (Parse end_chrom, length, sub_category and mate_id #17)
• Parse cytobands and collect cytoband start and end for each variant (Parse cytobands and add start and stop to variants #18)
• Parse genmod genetic models (Added parsing of genmod genetic models #19)
• Parsing specific variant fields: str_swegen_mean and str_swegen_std (STRs) and somatic_score (cancer and cancer_sv variants) (Parse as mongo docs, STRs and cancer variants specific fields #20)
• Parse a number of STR-related fields (Parse many STRs fields in one function #21)
• Parse SCOUT_CUSTOM INFO field (Parse custom field #22)
• Parse strs source fields (SourceDisplay, Source, SourceId) (Parse STR source fields #23)
• Parse mei info: name and polarity (Parse MEI info field #24)
• Parse genotype info (Parse samples genotypes #25)
• Parse eventual dbSNP IDs (Parse dbSNP IDs #27)
• Parse INFO.MATEID as mate_id, when available (Parse INFO.MATEID when available #28)
• Parse AZLENGTH and AZQUAL values, when available (Parse autozygosity values #31)
• Parse fusion variants INFO field (Parse fusion info #32)
• Set fusion genes from fusion variant INFO (Build fusion genes #33)
• Parse MitoMap-associated diseases (Parse MitoMap diseases #34)
• Parse HmtVar annotations (Parse HmtVar annotations #35)
• Parsing Genes and transcript annotations from VEP annotations - basic parsing (Parse basic VEP info (genes and transcripts IDs) #36)
• Create list of genes from parsed VEP transcripts (Create genes from parsed VEP transcripts #40)
• Create list of variant HGNC ids from variant genes (Set HGNC IDs from parsed VEP genes #41)
• Parse clnsig from VEP transcripts (Parse clnsig from VEP transcripts #42)
• Parse oncogenicity clnsig from ONC, ONCREVSTAT, ONCDN, CLNVID INFO fields (Parse oncogenicity clnsig from INFO fields #43)
• Parse external databases' frequencies from INFO field and parsed VEP transcripts (Parse frequencies from INFO field #44)
• Parse LoqusDB frequencies and metadata (Parse LoqusDB frequencies and header metadata #45)
• Added GitHub Action for linting, tests and changelog enforcement (Add GitHub lint and test automation #47)
• Parse severity predictions keys/values: CADD, Spidex and Revel scores (Parse severity predition keys #53)
• Parse conservation: GERP, PhastCons and PhyloP (Parse conservation scores: GERP, PhastCons and PhyloP #54)
• Parse callers (Parse callers #55)
• Parse RankResult INFO field (Parse RankResult INFO field #56)
• Parse SV-specific frequencies key/values (Parse SV-specific key/values #57)
• Parse MEI-specific frequencies key/values (Parse MEI frequencies #58)
• Parse cancer-specific MSK_MVL as mvl_tag (Parse cancer-specific MSK_MVL as mvl_tag #59)
• Parse Mivmir and Gicam scores for SNVs (Parse Mivmir and Gicam scores for SNVs #60)
• Introduce a loading orchestrator which is responsible to load all types of variants for a case (Variant loader orchestrator #62)
• Collect gene_to_panels from database, based on case panels found on case config file (Fetch gene_to_panels before variant parsing  #65)
• Build an HGNC ID → gene document mapping reflecting genes in genome build provided on case loading config file (Collect hgncid_to_gene info before parsing VCFs #67)
• Tests for loader.hgncid_to_gene and loader.gene_to_panels (Add tests for DB loader #69)
• Add gene panel info to parsed variants (Add gene panels info to parsed variants #70)
• Add gene symbols info to parsed variants (Add gene symbols to parsed variants #71)
• Complete gene and transcript variant info using gene DB collection (Complete gene and transcript variant info using gene DB collection #77)
• Retain panther_domain when parsing variant transcripts (Make sure that keys removed from transcripts were added at the variant level before #78)
• Check that provided institute exists in database prior to the parsing and loading of variants (Check that case owner exists in database before doing any parsing #81)
• Added an optional --config argument to the CLI (Add an optional --config argument to the CLI #84)
• Load variants in database in batches of 10000 (Load variants in batches of 1000 #82, Exact variant counts and load by batches of 5K #93, Preserve genomic regions when batching VCF variants #95)
• Save institute key on loaded variants documents (Save institute key on loaded variants documents #87)
• Update variants' rank after loading them (Update variants rank after loading #94)
• Show progress when loading variant categories (Add a progress bar when loading variant categories #97)
• Variant loading filter based on rank score and category (Add variant loading filter based on rank score and category #98)
• Parse custom images for STR variants (Always load variants causative in other cases #102)
• Support loading VCFs by category and research status (Allow selecting VCF categories and research data #111)
• Tests for clinical and research VCF category selection (Add tests for selecting category or research #113)
• Improved documentation on README file (Improved documentation #115)
• Clean up loaded variants on failure (Clean up loaded variants on failure #122)
• A Codecov push step on the automatic tests GitHub action (Split actions and add push to Codecov #123)
• Codecov, license and test status badges on README page (Add Codecov and License badges on README #124, Add tests status badge #132)
• A Release GitHub workflow, which build and publish standalone binaries for Linux x86_64, macOS x86_64, and macOS ARM64 (Create a release workflow to publish binaries #144)
• A CLI version flag (Add a CLI version flag #145)

Changed

• Structure of parsed entities, to reflect documents saved in the database (Parse as mongo docs, STRs and cancer variants specific fields #20)
• Improved and simplified code (Parse custom field #22)
• Refactored VEP annotations parsing module (Refactored and simplified the VEP parsing module #38)
• Refactor and simplify main function to accept and parse case config file (Pass only case config file as args #64)
• Switch to async Loader (Switch to an async loader #68)
• Set the same value for rank_score and fusion_score for fusion variants (Fixed parsing/printing and and rank_score key for fusion variants #74)
• Print exact count of inserted variants by category (Exact variant counts and load by batches of 5K #93)
• Preserve genomic regions when batching VCF variants (Preserve genomic regions when batching VCF variants #95)
• Update compound data for batched variants (Update compound data for batched variants #96)
• Always load variants which are pathogenic in ClinVar (Always load pathogenic variants #99)
• Always load managed variants (Always load managed variants #100)
• Always load any variant found to be causative in other cases (Samples field modified with fields removed? #101)
• Split linter and tests GitHub actions (Split actions and add push to Codecov #123)
• Assign rank scores regardless of case ID, with test (Decouple rank_score from case _id and add tests #127)
• Decouple genetic model parsing from case ID (Decouple genetic model parsing from case ID #130)
• Decouple compound parsing from case ID (Decouple compound parsing from case ID #131)

Fixed

• Normalize chromosome names by stripping the chr prefix (Normalize chromosome name by stripping "chr" #4)
• Parsing of end chromosome (Fixed parsing of variant sub_category and end chromosome #29)
• Parsing of variant sub-category (non SNVs) and end chromosome (Fixed parsing of variant sub_category and end chromosome #29)
• Parsing of multi-transcript genes from VEP CSQ field (Parse annotations - fixes and improvements #37)
• Fixed bug when parsing fusion variants (Fixed parsing/printing and and rank_score key for fusion variants #74)
• Renamed score compound key to combined_score (Rename compound's score key to combined_score #80)
• Clippy warning about a function with too many arguments (Fix function has too many args warning #86)
• Gene parsing for STRs (Fix parsing of genes for STRs #105)
• Loading of fusion variants (Fix loading of fusion variants #108)
• Parsing of fusion variants: FOUND_DB, ORIENTATION, genes, hgnc_ids, hgnc_symbols, genotypes, callers (Fix parse fusions #110)
• Reintroduced STRs and MEIs catetegories of parsed VCF, accidentally removed in a previous commit (Reintroduced accidentally removed VCF categories #125)
• Variant position, end and length are now parsed and stored as i32 (Parse position, end and length as i32 numbers #137)
• HGNC IDs in transcripts are stored as i32 and not i64 (Store HGNC IDs in transcripts as i32 #138)
• Variant batch sizing issue causing problem in updating compounds (Fix problem in batch sizing causing compounds issue #139)
• Gene exons should be set to empty string when None (Gene exons should be set to empty string when None #140)
• Remove compounds key from variant document if it's an empty array (Remove compounds key when list is empty #141)
• Fix parsing of 1000G frequencies (left and right) (Fix parsing of frequencies #142)

Review

• Tests executed by CR
• "Merge and deploy" approved by

This version is a

• MAJOR - when you make incompatible API changes
• MINOR - when you add functionality in a backwards compatible manner
• PATCH - when you make backwards compatible bug fixes or documentation/instructions

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.90%. Comparing base (b8a89f4) to head (f99b074).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #146   +/-   ##
=======================================
  Coverage   63.90%   63.90%           
=======================================
  Files          44       44           
  Lines        3687     3687           
=======================================
  Hits         2356     2356           
  Misses       1331     1331           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.