Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 83 additions & 42 deletions sphinx/user-guide/cli.rst
Original file line number Diff line number Diff line change
@@ -1,66 +1,107 @@
==========
Vortex CLI
==========
.. _vtx-command:

Vortex comes with the ``vtx`` CLI tool to execute a section. The CLI takes a path to a
yaml config file that describes the arguments to pass to a section (``toolbox.input``,
``toolbox.output``, ``toolbox.rload``).
===================
The ``vtx`` command
===================

The config can also be passed via stdin if the path is not provided.
Vortex comes with a command line program ``vtx`` that fetches or
stores files from/into the local and/or remote data tree.

.. note::

The `vtx` program is in fact a thin wrapper around the
:py:func:`vortex.input` and :py:func:`vortex.output` functions.

Usage
-----

.. code:: bash

vtx [-h] [--addon [ADDON ...]] [--log-level LOG_LEVEL] {get,put} [path]

The ``vtx`` command accepts two subcommands:

- ``get``: fetch a resource (runs :py:func:`vortex.input`).
- ``put``: store a resource (runs :py:func:`vortex.output`).

The subcommand takes an optional path to a YAML config file.
If the path is not provided, the config is read from stdin.

Options
-------

``-h``, ``--help``
^^^^^^^^^^^^^^^^^^

Show help message and exit

``-a``, ``--addon``
^^^^^^^^^^^^^^^^^^^

Addon to load. Multiple addons can be provided

``--log-level``
^^^^^^^^^^^^^^^

Logging level

Example
^^^^^^^
-------

From a YAML file:

.. code:: bash

vtx get config.yaml

Or from standard input directly:

.. code:: bash

cat < EOF | vtx
section: input
cat < EOF | vtx get
args:
now: true
local: file.grib
model: arpege
vapp: arpege
vconf: 4dvarfr
experiment: OPER
geometry: glob025
kind: gridpoint
nativefmt: grib
cutoff: prod
date: 2026-01-01
local: "file.grib"
model: "arpege"
vapp: "arpege"
vconf: "4dvarfr"
experiment: "OPER"
geometry: "glob025"
kind: "gridpoint"
nativefmt: "grib"
cutoff: "prod"
date: "202601010600"
term: 0
namespace: vortex.archive.fr
block: forecast
origin: historic
namespace: "vortex.archive.fr"
block: "forecast"
origin: "historic"
EOF


Addons
^^^^^^

Addons can be loaded by listing them in the config:
Addons can be loaded by listing them in the configuration:

.. code:: bash

cat < EOF | vtx
section: input
cat < EOF | vtx get
addons:
- kind: grib
- kind: ectrans
args:
now: true
local: file.grib
model: arpege
vapp: arpege
vconf: 4dvarfr
experiment: OPER
geometry: glob025
kind: gridpoint
nativefmt: grib
cutoff: prod
date: 2026-01-01
local: "file.grib"
model: "arpege"
vapp: "arpege"
vconf: "4dvarfr"
experiment: "OPER"
geometry: "glob025"
kind: "gridpoint"
nativefmt: "grib"
cutoff: "prod"
date: "202601010600"
term: 0
namespace: vortex.archive.fr
block: forecast
origin: historic
storetube: ectrans
namespace: "vortex.archive.fr"
block: "forecast"
origin: "historic"
storetube: "ectrans"
EOF
8 changes: 7 additions & 1 deletion sphinx/user-guide/fetch-write.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,16 @@ Fetching and writing data files
===============================


VORTEX facilitates the transfer of data files between an arbitrary
*vortex* facilitates the transfer of data files between an arbitrary
location and the current working directory. This location can either
be a leaf of the :doc:`data tree <data-layout>` or a specific path.

.. seealso::

This chapter describes how to fetch and write data files
programmatically, using *vortex* as a library. This can also be done
directly from the command line using :ref:`the vtx command <vtx-command>`.

Fetching files from the data tree
---------------------------------

Expand Down
80 changes: 43 additions & 37 deletions src/vortex/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,29 +16,34 @@

def main() -> None:
"""
Run a vortex section via CLI.
The argument of the section is provided via a yaml config file or via stdin.
Fetch/store a vortex resource from the command line.

The config is a valid yaml document can contain the following keys:
Files are fetched with the ``get`` subcommand and stored with the ``put``
subcommand. The vortex resource description is provided via a yaml config
file or via stdin.

Example:

..code:: bash

vtx get desc.yaml

The config is a valid YAML document can contain the following keys:

..code:: yaml

---
section: ... # (input or output)
args: ... # the section's arguments
args: ... # the resource descrition
addons: ... # a list of addons to load.

Only ``args`` is required. ``section`` will be ``input`` by default, or can be
overriden using the ``--section`` argument to the script.

Only ``args`` is required.
``addons`` is a list of addon arguments to pass to ``footprints.proxy.addon``.

For example:

..code:: yaml

---
section: input
args:
remote: "path/to/my/file.txt"
local: "file.txt"
Expand All @@ -49,26 +54,10 @@ def main() -> None:
- kind: grib

:note: You can provide multiple yaml document separated via ``---``
to execute multiple sections.
to execute multiple resources.
"""
parser = argparse.ArgumentParser(description="Save resource using vortex.")
parser.add_argument(
"path",
type=str,
nargs="?",
default=None,
help=(
"Path to the config file to run. "
"If not provided, will get the file from stdin."
),
)
parser.add_argument(
"--section",
"-s",
type=str,
default=None,
choices=["input", "output"],
help="Section to use (input or output).",
parser = argparse.ArgumentParser(
description="Fetch and store resources using vortex."
)
parser.add_argument(
"--addon",
Expand All @@ -77,8 +66,27 @@ def main() -> None:
help="Addon to load. Multiple addons can be provided.",
)
parser.add_argument(
"--log-level", type=str, default="NOTSET", help="Log level."
"--log-level", type=str, default="INFO", help="Log level."
)

subparsers = parser.add_subparsers(dest="subcommand", required=True)

for name, help_text in [
("get", "Fetch data from data tree(s)"),
("put", "Store data from data tree(s)"),
]:
sub = subparsers.add_parser(name, help=help_text)
sub.add_argument(
"path",
type=str,
nargs="?",
default=None,
help=(
"Path to the config file to run. "
"If not provided, will get the file from stdin."
),
)

args = parser.parse_args()

LOG.setLevel(args.log_level)
Expand All @@ -92,23 +100,21 @@ def main() -> None:
if yaml_str:
documents = yaml.safe_load_all(yaml_str.strip())

action = "input" if args.subcommand == "get" else "output"
for document in documents:
section = document.get("section", "input")
addons = document.get("addons", [])
if args.section is not None:
section = args.section
for addon in args.addon or []:
addons.append({"kind": addon})
vortex_cli(section, document.get("args", {}), addons)
vortex_cli(action, document.get("args", {}), addons)


def vortex_cli(
section: Literal["input", "output"],
action: Literal["input", "output"],
args: dict[str, Any],
addons: list[dict[str, Any]] | None = None,
) -> None:
"""
Execute a section and loads specified addons.
Execute a vortex section and loads specified addons.

:param section: The section to load.
:param args: The section's arguments.
Expand All @@ -124,7 +130,7 @@ def vortex_cli(

footprints.proxy.addon(**addon, shell=t.sh)

if section == "input":
if action == "input":
toolbox.input(now=True, **args)
elif section == "output":
elif action == "output":
toolbox.output(now=True, **args)
Loading