Skip to content

subsample: Improve default search paths, allow customization #1918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
victorlin merged 8 commits into from
Nov 18, 2025
Merged

subsample: Improve default search paths, allow customization #1918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4efd65d
Remove numbering from comments
victorlin Oct 31, 2025
e867ba1
Mark filepath values in schema
victorlin Oct 3, 2025
4d42c4b
Load schema outside of config parsing
victorlin Oct 3, 2025
8e2cc7a
Add CLI option for custom search paths
victorlin Oct 24, 2025
17a8a18
Add config's parent directory as first default search path
victorlin Oct 21, 2025
3e39a8c
Add env var for custom search paths
victorlin Oct 21, 2025
4441afa
Add helper function to get resolved filepaths
victorlin Oct 24, 2025
1828dde
Update changelog
victorlin Oct 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,18 @@
### Features

* augur.io.read_metadata: Added a new parameter `keep_id_as_column` to keep the resolved id column as a column in addition to setting it as the DataFrame index. [#1917][] (@victorlin)
* subsample: Filepaths in the config file can now be relative to the config file's parent directory in addition to the current working directory. Custom directories can also be specified using a new command line option `--search-paths` or environment variable `AUGUR_SEARCH_PATHS`. [#1897][] (@victorlin)
* A helper function – `augur.subsample.get_referenced_files` – has been added to optimize usage of `augur subsample` in Snakemake workflows. This is experimental and not yet part of the public API. [#1918][] (@victorlin)

### Bug fixes

* filter: Previously, `--query`, `--exclude-where`, and `--include-where` did not work for the id column (`strain`, `name`, or other from `--metadata-id-columns`). This has been fixed. [#1915][] (@corneliusroemer, @victorlin)
* export v2: Support export of URLs for non-string values. [#1926][] (@joverlee521)

[#1897]: https://github.com/nextstrain/augur/issues/1897
[#1915]: https://github.com/nextstrain/augur/issues/1915
[#1917]: https://github.com/nextstrain/augur/pull/1917
[#1918]: https://github.com/nextstrain/augur/pull/1918
[#1926]: https://github.com/nextstrain/augur/pull/1926

## 32.0.0 (21 October 2025)
Expand Down
35 changes: 22 additions & 13 deletions augur/data/schema-subsample-config.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,18 @@
"exclude": {
"oneOf": [
{
"type": "string"
"type": "string",
"format": "filepath"
},
{
"type": "array",
"items": {
"type": "string"
"type": "string",
"format": "filepath"
}
}
],
"description": "File(s) with list of strains to exclude. Paths must be relative to the\nworking directory."
"description": "File(s) with list of strains to exclude."
},
"exclude_all": {
"type": "boolean",
Expand Down Expand Up @@ -58,16 +60,18 @@
"include": {
"oneOf": [
{
"type": "string"
"type": "string",
"format": "filepath"
},
{
"type": "array",
"items": {
"type": "string"
"type": "string",
"format": "filepath"
}
}
],
"description": "File(s) with list of strains to include regardless of priorities,\nsubsampling, or absence of an entry in sequences. Paths must be relative\nto the working directory."
"description": "File(s) with list of strains to include regardless of priorities,\nsubsampling, or absence of an entry in sequences."
},
"include_where": {
"oneOf": [
Expand Down Expand Up @@ -136,16 +140,18 @@
"exclude": {
"oneOf": [
{
"type": "string"
"type": "string",
"format": "filepath"
},
{
"type": "array",
"items": {
"type": "string"
"type": "string",
"format": "filepath"
}
}
],
"description": "File(s) with list of strains to exclude. Paths must be relative to the\nworking directory."
"description": "File(s) with list of strains to exclude."
},
"exclude_all": {
"type": "boolean",
Expand Down Expand Up @@ -178,16 +184,18 @@
"include": {
"oneOf": [
{
"type": "string"
"type": "string",
"format": "filepath"
},
{
"type": "array",
"items": {
"type": "string"
"type": "string",
"format": "filepath"
}
}
],
"description": "File(s) with list of strains to include regardless of priorities,\nsubsampling, or absence of an entry in sequences. Paths must be relative\nto the working directory."
"description": "File(s) with list of strains to include regardless of priorities,\nsubsampling, or absence of an entry in sequences."
},
"include_where": {
"oneOf": [
Expand Down Expand Up @@ -263,7 +271,8 @@
},
"group_by_weights": {
"type": "string",
"description": "TSV file defining weights for grouping. Path must be relative to the\nworking directory. Requirements:\n\n(1) Lines starting with '#' are treated as comment lines.\n(2) The first non-comment line must be a header row.\n(3) There must be a numeric ``weight`` column (weights can take on any\n non-negative values).\n(4) Other columns must be a subset of grouping columns, with\n combinations of values covering all combinations present in the\n metadata.\n(5) This option only applies when grouping columns and a total sample\n size are provided.\n(6) This option can only be used when probabilistic sampling is allowed.\n\nNotes:\n\n(1) Any grouping columns absent from this file will be given equal\n weighting across all values *within* groups defined by the other\n weighted columns.\n(2) An entry with the value ``default`` under all columns will be\n treated as the default weight for specific groups present in the\n metadata but missing from the weights file. If there is no default\n weight and the metadata contains rows that are not covered by the\n given weights, augur filter will exit with an error."
"format": "filepath",
"description": "TSV file defining weights for grouping. Requirements:\n\n(1) Lines starting with '#' are treated as comment lines.\n(2) The first non-comment line must be a header row.\n(3) There must be a numeric ``weight`` column (weights can take on any\n non-negative values).\n(4) Other columns must be a subset of grouping columns, with\n combinations of values covering all combinations present in the\n metadata.\n(5) This option only applies when grouping columns and a total sample\n size are provided.\n(6) This option can only be used when probabilistic sampling is allowed.\n\nNotes:\n\n(1) Any grouping columns absent from this file will be given equal\n weighting across all values *within* groups defined by the other\n weighted columns.\n(2) An entry with the value ``default`` under all columns will be\n treated as the default weight for specific groups present in the\n metadata but missing from the weights file. If there is no default\n weight and the metadata contains rows that are not covered by the\n given weights, augur filter will exit with an error."
},
"probabilistic_sampling": {
"type": "boolean",
Expand Down
9 changes: 3 additions & 6 deletions augur/filter/arguments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,7 @@
"2010-XX-01")."""),

"exclude": dedent("""\
File(s) with list of strains to exclude. Paths must be relative to the
working directory."""),
File(s) with list of strains to exclude."""),

"exclude_where": dedent("""\
Exclude sequences matching these conditions. Ex: "host=rat" or
Expand All @@ -57,8 +56,7 @@

"include": dedent("""\
File(s) with list of strains to include regardless of priorities,
subsampling, or absence of an entry in sequences. Paths must be relative
to the working directory."""),
subsampling, or absence of an entry in sequences."""),

"include_where": dedent("""\
Include sequences with these values. ex: host=rat. Multiple values are
Expand Down Expand Up @@ -104,8 +102,7 @@
when a total sample size is provided."""),

"group_by_weights": dedent("""\
TSV file defining weights for grouping. Path must be relative to the
working directory. Requirements:
TSV file defining weights for grouping. Requirements:

(1) Lines starting with '#' are treated as comment lines.
(2) The first non-comment line must be a header row.
Expand Down
Loading
Loading