Enable retrieving fasta file from a seqcol server DRS endpoint

[nsheff]

If this could do from the CLI:

refget cache ABCXYZ // stores it into local disk cache
refget extract ABCXYZ --regions query.bed  // look up sequences for a set of intervals

I need to retrieve sequences from a remote server.
Two options:

1. I can use refgenie to get a fasta asset.
2. I can create a new DRS endpoint for just the fasta file.

Option 1: Add DRS endpoint to the refget seqcol server

where is the pointer from Digest to Fasta file location stored? In the database... a new table? Files?

• add Files pydantic model
• add Files agent. given a digest, retrieve the DRSInfo, etc. basically make the files agent into a DRS provider
• add to the data_loaders, a data loader that can inset Files information.
• the endpoint is super easy, just use the files agent.

Option 2: Just use refgenie idea

Maybe let refgenie handle the distribution of the sequence data?

Differences

• refgenie asset is indexed by the asset digest, not the seqcol digest
• refgenie asset includes not just the fasta file but other stuff.

Decision

I should use refgenie.

• they'll be storing the fasta files anyway.
• refget benefits from the refgenie content delivery networks, automatically
• a good use case of refgenie

Implementation

How would I use this then? how do I pass information about the file from refgenie to the local refget extractor?

• refget extract is happening in Python;

• add an optional dependency of refgenie? Can it operate like a plugin?

• refget cache looks up the refgenie asset, pulls it out, and creates the refget-rs on-disk representation? so there's a $REFGETCACHE location where the seqcols are stored.

• refget extract thenis a lightweight python wrapper around the rust extraction command?

[nsheff]

I've gone in a different direction; the new RefgetStore solves this now.

RefgetStore (implemented in gtars, exposed via refget CLI) is a local disk-based storage system that handles sequence collections and their underlying sequences. It provides:

Cache a collection from remote (metadata + lazy sequence loading)

refget store pull ABC123 --server https://seqcolapi.example.com/store

Extract sequences for a set of intervals

refget store export ABC123 --bed query.bed -o output.fa

Or query remote directly without caching

refget store export ABC123 --server https://example.com/store --bed query.bed

How it works:

1. refget store pull connects to a remote RefgetStore, downloads collection metadata immediately, and caches sequence data on-demand (or eagerly with --eager)
2. refget store export --bed extracts subsequences for BED intervals from locally cached collections
3. The cache lives at ~/.refget/store/.remote_cache/ by default

Why not refgenie?

The original idea was to leverage refgenie's asset distribution, but RefgetStore is simpler and more direct. Now, refgenie will be able to be adapted to use this as a source of sequence data.

The DRS endpoint idea is still viable for serving FASTA files, but for the CLI use case of "cache + extract regions," RefgetStore handles it natively.