BacCurate

BacCurate turns heterogeneous public sequencing metadata into a standardized resource for comparative genomics, genomic epidemiology, and One Health research.

This repository contains the source code that extracts and harmonizes BioSample metadata. The dataset itself, and the documentation, are on the website.

Samples covered

As of the 2026-07-09 BioSample snapshot, ~1.5 million BioSample records are covered, including all ESKAPEE pathogens:

• Enterococcus faecium
• Enterococcus faecalis
• Staphylococcus aureus
• Klebsiella pneumoniae species complex
• Acinetobacter baumannii
• Pseudomonas aeruginosa
• Enterobacter spp.
• Escherichia coli

Methodology in brief

1. The NCBI BioSample XML dump is parsed and the records of the registered taxa are indexed.
2. The attribute-value pairs of interest from each record are selected.
3. Values are resolved against the reference vocabularies in data/reference/:
   • Isolation source: Large Language Model (LLM) assisted mapping to a purpose-built ontology of 103 terms (ontology/terms.tsv), with SSSOM mappings to external ontologies.
   • Host organism: mapped to NCBI taxonomy (taxonomy/) taxids and scientific names.
   • Geographic location: mapped to the INSDC Geographic Location Name List and Natural Earth map units for coordinate-derived countries.
   • Collection dates: parsed with custom-built rules and normalized to ISO 8601.

LLMs are only used for isolation-source standardization, everything else is deterministic and driven by hand-curated rules.

For more information on the methods and the interpretation of the output data, read the full documentation here (still WIP).

Installation

Requires Python 3.12 or later.

1. Clone the repository

git clone https://github.com/ELTEbioinformatics/BacCurate.git
cd BacCurate

2. Install dependencies and setup environment

Using uv:

uv sync

If you'd rather not install uv, pip install -e . also works in a standard venv:

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install -e .

Note

The commands below use uv. With a venv activated instead, drop the uv run prefix.

Verify the installation:

uv run baccurate --help

3. Configuring API credentials (optional)

By default, isolation-source standardization utilizes an OpenAI compatible API.

Create a .env file in the repository root with your API credentials:

API_KEY="sk-..."
SERVER="https://api.openai.com/v1"
LLM_MODEL="gpt-4o-mini"

Use --skip-llm to process without LLMs.

4. Setting up input data

The starting dataset is assembled from:

• NCBI BioSample metadata (biosample_set.xml.gz)
• AllTheBacteria metadata (sylph/GTDB species profiling)
• NCBI Genbank metadata
• NCBI RefSeq metadata
• NCBI SRA metadata

Place these in data/raw.

The NCBI taxonomy dump (nodes.dmp, names.dmp, merged.dmp) goes in data/reference/taxonomy/. Then run:

uv run python scripts/parse_biosample_xml.py
uv run python scripts/filter_sequence_accessions.py
uv run python scripts/build_biosample_index.py

Usage

Running all pipelines for each taxon

uv run baccurate

Outputs are in output/<timestamp>/.

Specifying taxa

You can provide taxon keywords defined in config/taxa.yaml to only process select taxa.

# Acinetobacter baumannii and Escherichia coli
uv run baccurate abaumannii ecoli

Specifying pipelines

The --standardize option accepts host, date, loc, and iso (host organism, collection date, geographical location and isolation-source).

# Collection date and geographical location standardization
uv run baccurate abaumannii --standardize date loc

Re-extracting metadata

Extraction (selecting the attribute-value pairs of interest from the raw BioSample metadata) runs only if output/extracted_metadata.tsv does not exist.

To re-extract, either delete it or specify a new filename:

uv run baccurate --extracted-metadata extracted_metadata_new.tsv

Debug mode

Enable verbose logging:

uv run baccurate --debug