Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,9 @@ jobs:
enable-cache: true
cache-suffix: py${{ matrix.python }}
cache-dependency-glob: pyproject.toml
- name: Test without optional dependencies and without pyyaml
- name: Test without optional dependencies
run: |
uv pip install .[coverage]
uv pip uninstall pyyaml types-PyYAML
pytest --cov --cov-report=term --cov-report=xml --junit-xml=junit.xml
mv coverage.xml coverage_py${{ matrix.python }}_bare.xml
mv junit.xml junit_py${{ matrix.python }}_bare.xml
Expand Down Expand Up @@ -218,11 +217,10 @@ jobs:
with:
name: package
path: dist
- name: Test without optional dependencies and without pyyaml
- name: Test without optional dependencies
run: |
cd dist
uv pip install $(ls jsonargparse-*.whl)[test-no-urls] $(ls jsonargparse_tests-*.whl)
uv pip uninstall pyyaml
python -m jsonargparse_tests
- name: Test with all optional dependencies
run: |
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,14 @@ Changed
- Config objects always include metadata, i.e. ``clone(with_meta=False)`` is now
the only way to strip it (`#969
<https://github.com/mauvilsa/jsonargparse/pull/969>`__).
- ``pyyaml`` is no longer a required dependency, install the ``yaml`` extra to
have it. Without it the default ``parser_mode`` and dump format is ``json``,
and explicitly using ``yaml`` raises an informative ``ImportError`` (`#970
<https://github.com/mauvilsa/jsonargparse/pull/970>`__).
- The ``json`` dump format is now indented, so that the print config argument
gives a more readable output. Use the new ``json_compact`` format for the
previous single line output (`#970
<https://github.com/mauvilsa/jsonargparse/pull/970>`__).

Removed
^^^^^^^
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ among them:
Development environment
-----------------------

All requirements of the project are defined in ``pyproject.toml``. The basic
runtime requirements are in ``dependencies``. Requirements for optional
features, as well as for testing, development and documentation building
(``test``, ``dev`` and ``doc``), are in ``[project.optional-dependencies]``.
All requirements of the project are defined in ``pyproject.toml``. There are no
required runtime dependencies. Requirements for optional features, as well as
for testing, development and documentation building (``test``, ``dev`` and
``doc``), are in ``[project.optional-dependencies]``.

The recommended way to work with the source code is to clone the repository,
create a virtual environment, activate it, and install the development
Expand Down
11 changes: 6 additions & 5 deletions DOCUMENTATION.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1331,7 +1331,8 @@ Configuration files

jsonargparse can parse configuration files (config files). The dot notation
hierarchy of the arguments (see :ref:`nested-namespaces`) defines the structure
expected in these files. The default format is YAML. To change it, use the
expected in these files. The default format is YAML, or JSON when the ``yaml``
extra is not installed, see :ref:`installation`. To change it, use the
``parser_mode`` parameter of the parser, e.g.
``ArgumentParser(parser_mode="toml")``.

Expand Down Expand Up @@ -1419,8 +1420,8 @@ comma, e.g. ``--print_config=comments,skip_default``:

From Python, a config object is serialized with the :meth:`dump
<.ArgumentParser.dump>` and :meth:`save <.ArgumentParser.save>` methods. The
supported formats are ``yaml``, ``toml``, ``json``/``json_compact``,
``json_indented`` and ``parser_mode``, the default, which uses the format of the
supported formats are ``yaml``, ``toml``, ``json``/``json_indented``,
``json_compact`` and ``parser_mode``, the default, which uses the format of the
parser. More formats are added with :func:`.set_dumper`, for example to dump
with PyYAML's ``default_flow_style``:

Expand All @@ -1441,8 +1442,8 @@ with PyYAML's ``default_flow_style``:
Custom loaders
--------------

The ``yaml`` parser mode (see :py:meth:`.ArgumentParser.__init__`) loads with a
subclass of `yaml.SafeLoader
The ``yaml`` parser mode (see :py:meth:`.ArgumentParser.__init__`) requires the
``yaml`` extra and loads with a subclass of `yaml.SafeLoader
<https://pyyaml.org/wiki/PyYAMLDocumentation#loader>`__ that has three
differences:

Expand Down
16 changes: 8 additions & 8 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -185,14 +185,14 @@ You can install using `pip <https://pypi.org/project/jsonargparse/>`__ as:

pip install jsonargparse

By default, the only dependency installed with ``jsonargparse`` is `PyYAML
<https://pypi.org/project/PyYAML/>`__. However, several optional features can be
enabled by specifying one or more of the following extras (optional
dependencies): ``signatures``, ``jsonschema``, ``jsonnet``, ``urls``,
``fsspec``, ``toml``, ``ruamel``, ``omegaconf``, ``shtab``, and ``argcomplete``.
Additionally, the ``all`` extras can be used to enable all optional features
(excluding tab completion ones). To install ``jsonargparse`` with extras, use
the following syntax:
``jsonargparse`` has no required dependencies. Optional features are enabled by
specifying one or more of the following extras (optional dependencies):
``signatures``, ``yaml``, ``jsonschema``, ``jsonnet``, ``urls``, ``fsspec``,
``toml``, ``ruamel``, ``omegaconf``, ``shtab``, and ``argcomplete``. The
``yaml`` extra installs `PyYAML <https://pypi.org/project/PyYAML/>`__, without
which config files are parsed as JSON. Additionally, the ``all`` extras can be
used to enable all optional features (excluding tab completion ones). To install
``jsonargparse`` with extras, use the following syntax:

.. code-block:: bash

Expand Down
9 changes: 6 additions & 3 deletions jsonargparse/_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
fsspec_support,
import_fsspec,
import_jsonnet,
import_pyyaml,
omegaconf_apply,
pyyaml_available,
)
Expand Down Expand Up @@ -791,8 +792,8 @@ def dump(

Args:
namespace: The configuration object to dump.
format: The output format: ``yaml``, ``json``, ``json_indented``, ``toml``, ``parser_mode`` or ones added
via :func:`.set_dumper`.
format: The output format: ``yaml``, ``json``, ``json_compact``, ``toml``, ``parser_mode`` or ones
added via :func:`.set_dumper`.
skip_unset: Whether to exclude entries whose value is the configured None/Unset value.
skip_default: Whether to exclude entries whose value is the same as the default.
skip_validation: Whether to skip parser checking.
Expand Down Expand Up @@ -904,7 +905,7 @@ def save(
Args:
namespace: The configuration object to save.
path: Path to the location where to save config.
format: The output format: ``yaml``, ``json``, ``json_indented``, ``parser_mode`` or ones added via
format: The output format: ``yaml``, ``json``, ``json_compact``, ``parser_mode`` or ones added via
:func:`.set_dumper`.
skip_unset: Whether to exclude entries whose value is the configured None/Unset value.
skip_validation: Whether to skip parser checking.
Expand Down Expand Up @@ -1533,6 +1534,8 @@ def parser_mode(self, parser_mode: str):
raise ValueError(f"The only accepted values for parser_mode are {accepted}.")
if parser_mode == "jsonnet":
import_jsonnet("parser_mode=jsonnet")
elif parser_mode == "yaml":
import_pyyaml("parser_mode=yaml")
self._parser_mode = parser_mode
if self._subcommands_action:
for subparser in self._subcommands_action._name_parser_map.values():
Expand Down
16 changes: 8 additions & 8 deletions jsonargparse/_loaders_dumpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from ._common import Unset, get_parsing_setting, load_value_mode, parent_parser
from ._optionals import (
import_jsonnet,
import_pyyaml,
import_toml_dumps,
import_toml_loads,
omegaconf_support,
Expand Down Expand Up @@ -54,7 +55,7 @@ def get_yaml_default_loader():
if yaml_default_loader:
return yaml_default_loader

import yaml
yaml = import_pyyaml("get_yaml_default_loader")

class DefaultLoader(getattr(yaml, "CSafeLoader", yaml.SafeLoader)):
pass
Expand Down Expand Up @@ -92,8 +93,7 @@ def remove_implicit_resolver(cls, tag_to_remove):


def yaml_load(stream):
import yaml

yaml = import_pyyaml("yaml_load")
value = yaml.load(stream, Loader=get_yaml_default_loader())
if isinstance(value, dict) and value and all(v is None for v in value.values()):
if len(value) == 1 and stream.strip() == next(iter(value)) + ":":
Expand Down Expand Up @@ -159,7 +159,7 @@ def get_loader_exceptions(mode: str | None = None) -> tuple[type[Exception], ...
mode = get_load_value_mode()
if mode not in loader_exceptions:
if mode == "yaml":
loader_exceptions[mode] = (__import__("yaml").YAMLError,)
loader_exceptions[mode] = (import_pyyaml("get_loader_exceptions").YAMLError,)
elif mode == "json":
loader_exceptions[mode] = (__import__("json").JSONDecodeError,)
elif mode == "toml":
Expand Down Expand Up @@ -247,8 +247,7 @@ def replace_unset(data):


def yaml_dump(data):
import yaml

yaml = import_pyyaml("yaml_dump")
return yaml.safe_dump(data, **dump_yaml_kwargs)


Expand Down Expand Up @@ -278,7 +277,7 @@ def toml_dump(data):

dumpers: dict[str, Callable] = {
"yaml": yaml_dump,
"json": json_compact_dump,
"json": json_indented_dump,
"json_compact": json_compact_dump,
"json_indented": json_indented_dump,
"toml": toml_dump,
Expand All @@ -302,7 +301,8 @@ def check_valid_dump_format(dump_format: str):

def dump_using_format(parser: ArgumentParser, data: dict, dump_format: str, with_comments: bool = False) -> str:
if dump_format == "parser_mode":
dump_format = parser.parser_mode if parser.parser_mode in dumpers else "yaml"
default_format = "yaml" if pyyaml_available else "json"
dump_format = parser.parser_mode if parser.parser_mode in dumpers else default_format
if with_comments:
if f"{dump_format}_comments" not in dumpers:
if dump_format == "yaml":
Expand Down
6 changes: 6 additions & 0 deletions jsonargparse/_optionals.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,12 @@ def missing_package_raise(package, importer):
raise ImportError(f"{package} package is required by {importer} :: {ex}") from ex


def import_pyyaml(importer):
with missing_package_raise("PyYAML", importer):
import yaml
return yaml


def import_toml_loads(importer):
if find_spec("tomllib"):
import tomllib
Expand Down
19 changes: 9 additions & 10 deletions jsonargparse_tests/test_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -565,17 +565,17 @@ def test_dump_formats(dump_parser):
if pyyaml_available:
assert dump_parser.dump(cfg) == "op1: 123\nop2: abc\n"
assert dump_parser.dump(cfg, format="yaml") == dump_parser.dump(cfg)
assert dump_parser.dump(cfg, format="json") == '{"op1":123,"op2":"abc"}'
assert dump_parser.dump(cfg, format="json_indented") == '{\n "op1": 123,\n "op2": "abc"\n}\n'
assert dump_parser.dump(cfg, format="json") == '{\n "op1": 123,\n "op2": "abc"\n}\n'
assert dump_parser.dump(cfg, format="json_indented") == dump_parser.dump(cfg, format="json")
assert dump_parser.dump(cfg, format="json_compact") == '{"op1":123,"op2":"abc"}'
pytest.raises(ValueError, lambda: dump_parser.dump(cfg, format="invalid"))


def test_dump_skip_default_simple(dump_parser):
dump = dump_parser.dump(dump_parser.get_defaults(), skip_default=True)
expected = "{}\n" if pyyaml_available else "{}"
assert dump == expected
assert dump == "{}\n"
dump = dump_parser.dump(Namespace(op1=123, op2="xyz"), skip_default=True)
expected = "op2: xyz\n" if pyyaml_available else '{"op2":"xyz"}'
expected = "op2: xyz\n" if pyyaml_available else '{\n "op2": "xyz"\n}\n'
assert dump == expected


Expand All @@ -585,13 +585,12 @@ def test_dump_skip_default_nested(parser):
parser.add_argument("--g2.op1", type=int, default=987)
parser.add_argument("--g2.op2", type=str, default="xyz")
dump = parser.dump(parser.get_defaults(), skip_default=True)
expected = "{}\n" if pyyaml_available else "{}"
assert dump == expected
assert dump == "{}\n"
dump = parser.dump(parser.parse_args(["--g1.op1=0"]), skip_default=True)
expected = "g1:\n op1: 0\n" if pyyaml_available else '{"g1":{"op1":0}}'
expected = "g1:\n op1: 0\n" if pyyaml_available else '{\n "g1": {\n "op1": 0\n }\n}\n'
assert dump == expected
dump = parser.dump(parser.parse_args(["--g2.op2=pqr"]), skip_default=True)
expected = "g2:\n op2: pqr\n" if pyyaml_available else '{"g2":{"op2":"pqr"}}'
expected = "g2:\n op2: pqr\n" if pyyaml_available else '{\n "g2": {\n "op2": "pqr"\n }\n}\n'
assert dump == expected


Expand Down Expand Up @@ -841,7 +840,7 @@ def test_save_path_content(parser, tmp_cwd):
parser.save_path_content.add("the.path")
parser.save(cfg, out_yaml)

expected = "the:\n path: file.txt\n" if pyyaml_available else '{"the":{"path":"file.txt"}}'
expected = "the:\n path: file.txt\n" if pyyaml_available else '{\n "the": {\n "path": "file.txt"\n }\n}\n'
assert out_yaml.read_text() == expected
assert out_file.read_text() == "file content"

Expand Down
9 changes: 2 additions & 7 deletions jsonargparse_tests/test_jsonnet.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
ArgumentError,
ArgumentParser,
)
from jsonargparse._optionals import jsonnet_support, pyyaml_available
from jsonargparse._optionals import jsonnet_support
from jsonargparse_tests.conftest import (
get_parser_help,
json_or_yaml_load,
Expand Down Expand Up @@ -177,12 +177,7 @@ def test_action_jsonnet_save_config_metadata(parser, tmp_path):
# rewrite the config to make sure that ext_vars is after jsonnet
main_cfg = json_or_yaml_load(output_config.read_text())
main_cfg = {k: main_cfg[k] for k in ["jsonnet", "ext_vars"]}
if pyyaml_available:
import yaml

output_config.write_text(yaml.safe_dump(main_cfg, sort_keys=False))
else:
output_config.write_text(json.dumps(main_cfg))
output_config.write_text(json.dumps(main_cfg))

# parse using saved config and verify result is the same
cfg2 = parser.parse_args([f"--cfg={output_config}"])
Expand Down
27 changes: 26 additions & 1 deletion jsonargparse_tests/test_loaders_dumpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,31 @@
pytest.raises(ValueError, lambda: ArgumentParser(parser_mode="invalid"))


@skip_if_no_pyyaml
def test_default_parser_mode_yaml():
assert ArgumentParser().parser_mode == "yaml"


@pytest.mark.skipif(pyyaml_available, reason="PyYAML package should not be installed")
def test_without_pyyaml_default_parser_mode_json():
assert ArgumentParser().parser_mode == "json"


@pytest.mark.skipif(pyyaml_available, reason="PyYAML package should not be installed")
def test_without_pyyaml_parser_mode_yaml_error():
with pytest.raises(ImportError) as ctx:
ArgumentParser(parser_mode="yaml")
ctx.match("PyYAML package is required by parser_mode=yaml")


@pytest.mark.skipif(pyyaml_available, reason="PyYAML package should not be installed")
def test_without_pyyaml_dump_yaml_error(parser):
parser.add_argument("--int", type=int, default=1)
with pytest.raises(ImportError) as ctx:

Check warning on line 76 in jsonargparse_tests/test_loaders_dumpers.py

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Refactor this exception test to have only one invocation possibly throwing an exception.

See more on https://sonarcloud.io/project/issues?id=mauvilsa_jsonargparse&issues=AaB_bd3Bo-nPJxlEaw3Y&open=AaB_bd3Bo-nPJxlEaw3Y&pullRequest=970
parser.dump(parser.get_defaults(), format="yaml")
ctx.match("PyYAML package is required by yaml_dump")


def test_get_loader():
from jsonargparse._loaders_dumpers import jsonnet_load

Expand Down Expand Up @@ -82,7 +107,7 @@
parser.add_argument("--int", type=int, default=1)
parser.dump_header = ["line 1", "line 2"]
dump = parser.dump(parser.get_defaults(), format="json")
assert dump == '{"int":1}'
assert dump == '{\n "int": 1\n}\n'


def test_dump_header_invalid(parser):
Expand Down
18 changes: 18 additions & 0 deletions jsonargparse_tests/test_optionals.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,12 @@
import_fsspec,
import_jsonnet,
import_jsonschema,
import_pyyaml,
import_requests,
import_ruamel,
jsonnet_support,
jsonschema_support,
pyyaml_available,
ruamel_support,
url_support,
)
Expand All @@ -28,9 +30,25 @@
get_parser_help,
skip_if_docstring_parser_unavailable,
skip_if_fsspec_unavailable,
skip_if_no_pyyaml,
skip_if_requests_unavailable,
)

# pyyaml support


@skip_if_no_pyyaml
def test_pyyaml_support_true():
import_pyyaml("test_pyyaml_support_true")


@pytest.mark.skipif(pyyaml_available, reason="PyYAML package should not be installed")
def test_pyyaml_support_false():
with pytest.raises(ImportError) as ctx:
import_pyyaml("test_pyyaml_support_false")
ctx.match("test_pyyaml_support_false")


# jsonschema support


Expand Down
Loading