Skip to content

code_lists.md

Latest commit

 

History

History
75 lines (60 loc) · 5.97 KB

code_lists.md

File metadata and controls

75 lines (60 loc) · 5.97 KB
Raw

phageFACTor — where the curated word-lists / patterns live in the code

Companion to scoring_curation_DAG.svg. Paths are relative to the repo root.

Two files:

  • scripts/lib/lexicon.py — all vocabulary (word-lists, patterns, the category↔keyword map). The definition site: config.py, foldseek_scoring.py, utils.py, 03_compare_annotations.py and 04_curate_annotations.py import from it and define no vocabulary of their own.
  • scripts/lib/rules.py — the ordered rules that consume it (name upgrades, description → PHROG category, short_name generation).

config.py holds paths and thresholds; it re-exports three lexicon names (UNINFORMATIVE_STRINGS, GENERIC_WORDS, COMPLEMENTARY_CATEGORY_MAP) so from config import ... call sites keep working.

Search the symbol name rather than trusting a line number.

lexicon.py, by section

§ What it controls Symbols
A · informativeness the gate that drops most hits, plus the three ranking maluses UNINFORMATIVE_STRINGS (exact blocklist) · UNCHAR_PATTERN (hypothetical / DUF / bare folds / accession-shaped names) · GENERIC_DOMAIN (×0.75, "family protein", "domain-containing") · GENERIC_NAMES (×0.75, UPF/Yabc/orf labels) · GENERIC_STRUCTURAL_PRODUCTS (real but unspecific: endolysin, major capsid protein) · GENERIC_WORDS (tokens dropped before the Jaccard) · GENERIC_DESCRIPTORS (exact-membership set for the generic-vs-specific gate)
B · taxonomy kingdom assignment without an NCBI taxonomy dependency BACTERIA_KEYWORDS · EUKARYOTE_KEYWORDS · ARCHAEA_KEYWORDS · VIRUS_KEYWORDS · EUKARYOTIC_DESC (description-level euka signal) · is_eukaryotic_taxname() (word-boundary matcher shared by 03 and by 02d's parse-time filter)
C · phage terms the boosts and the wrapper malus PHAGE_SPECIFIC (×2.0) · PHAGE_CONTEXT (×1.5) · GP_WRAPPER (×0.40) · PHAGE_WRAPPER_ONLY (anchored: blocks a bare gpNN from winning)
D · defense defense / toxin-antitoxin flag DEFENSE_PATTERN (scoring) · DEFENSE_DESC (curation gate) · EUKA_RESCUE (keeps Sir2/TIR/cGAS/Argonaute-type names that section B would otherwise demote)
E · promiscuous folds fold ≠ function: flag + cap at BORDERLINE PROMISCUOUS_FOLDS
F · PDB titles "Crystal structure of X" → X; drop ORF/DARPin/SeMet titles PDB_CRYSTAL_PREFIX · PDB_TRAILING_QUAL · PDB_UNINFORMATIVE · PDB_TITLE · extract_pdb_description()
G · characterized domains the positive counterpart of E — domain-level folds that do carry a corroborated activity (SPOR, PIN, LysM, HNH, GIY-YIG…), each with activity + PMID RELEVANT_DOMAINS · relevant_domain()
H · curation gates the two deliberately different widths used in step 04 FS_GENERIC (narrow) · FS_GENERIC_BROAD (wide) · GENE_LABEL (drives an upgrade, not a malus)
I · functional equivalence category ↔ keyword map: the complementary agreement label, and (symmetrically) the synonym groups used by _shares_function COMPLEMENTARY_CATEGORY_MAP

Public API: classify(desc) -> (tier, tags) and is_informative_desc(desc). tier (0 uninformative / 1 low / 2 proper) exists only so the rule-based classifier is directly comparable with inPhormer, which uses the same three levels. No curation decision reads tier — every predicate reads tags, because the callers need different widths and each width drives a different consequence (drop vs ×0.75 malus vs upgrade vs gate).

Ordered rules: scripts/lib/rules.py

The other half. A lexicon entry is a set member — order is irrelevant, and adding one widens a test by exactly that one string. A rule is (pattern → action) and first match wins, so a rule added at the top can shadow every rule below it. Mixing the two in one file makes that difference invisible, so they are separate.

Symbol Used by What it does
UPGRADE_RULES step 04 (desc pattern, top-3 pattern, replacement) — swap a generic winning name for a specific one already present in the top-3 hits. Supports \1 backreferences.
FUNC_CAT_RULES step 04 desc → PHROG category, for genes with no PHold call at all.
CAT_KEYWORDS · PHROG_CANON · phrog_category() step 05 desc → PHROG category at the output layer, for genome-mode pharokka passthrough rows. Overlaps FUNC_CAT_RULES but is not identical: it runs on real product names and carries the defense/other/unknown function fallbacks.
make_short_name() (_HEAD_NOUNS · _DEFENSE_SHORT · _DECORATION · _LEADING · _GENE_SYMBOL) steps 05, 07 final_product → a concise short_name. A pure post-pass: it cannot change which annotation was chosen. Manual exceptions go in input/overrides.tsv.

Elsewhere on purpose

Symbol File Why it stays
_RE_REG · _RE_INT · _RE_STR · _RE_REP · _RE_REG_EXCLUDE scripts/lib/synteny_hint.py neighbourhood classes for the optional C1/Cro synteny hint; used nowhere else.
_SUBDB_SOURCES · _SUBDB_FILE_RE 01p_merge_phold_proteins.py · 03_compare_annotations.py phold sub-DB filenames, i.e. an interface to another tool.

Notes

  • Section A is where most hits are removed (every hypothetical / DUF / unknown structural homolog).
  • A dropped hit and a demoted hit are not the same thing: C's boosts and A's ×0.75 maluses are multiplicative on the ranking score only; E caps confidence; B and D set flags or demote one step.
  • Eukaryotic hits are touched at three stages, from one list: is_eukaryotic_taxname() removes them before scoring (local search only), EUKARYOTIC_DESC / the same keyword list flag or demote them after, and EUKA_RESCUE brings back the ones that are meaningful in phage defence (Sir2, TIR, cGAS, Argonaute). Matching is on word boundaries, not substrings: Campylobacter conciSUS is not a pig.