Support for NewType, LiteralString, NamedTuple, TypedDict ReadOnly keys and subscripted generic type aliases

CHANGELOG.rst

@@ -46,6 +46,19 @@ Added
   which raises an exception for all parameters that don't have a type
   annotation, not only the required ones (`#965
   <https://github.com/mauvilsa/jsonargparse/pull/965>`__).
+- Support for ``NamedTuple`` as a type. The value is an object with the fields
+  as keys or an array of positional values, parsing gives an instance of the
+  named tuple and dumping gives an object. It has a ``--*.help`` option that
+  shows the accepted fields, is accepted by ``add_class_arguments`` and works
+  subscripted when generic, see :ref:`type-hints` (`#967
+  <https://github.com/mauvilsa/jsonargparse/pull/967>`__).
+- ``TypedDict`` now accepts ``ReadOnly`` for its keys (`#967
+  <https://github.com/mauvilsa/jsonargparse/pull/967>`__).
+- Support for ``NewType`` and ``LiteralString`` as types. Previously they were
+  not validated, i.e. any value was accepted. Now a ``NewType`` is validated as
+  the supertype it stands for and a ``LiteralString`` as a ``str``, in both
+  cases the help showing the name as in the source code, see :ref:`type-hints`
+  (`#967 <https://github.com/mauvilsa/jsonargparse/pull/967>`__).
 
 Fixed
 ^^^^^
@@ -90,6 +103,19 @@ Fixed
   subscripted, e.g. ``os.PathLike[str]`` for a registered ``PathLike``. Now the
   registration of the unsubscripted type is used (`#966
   <https://github.com/mauvilsa/jsonargparse/pull/966>`__).
+- A subscripted generic ``TypeAliasType``, e.g. ``Alias[int]`` for ``type
+  Alias[T] = list[T]``, raised ``Unsupported type hint``. Unsubscripted, its
+  type parameters now also stand for their default, constraints or bound, the
+  same as any other ``TypeVar`` (`#967
+  <https://github.com/mauvilsa/jsonargparse/pull/967>`__).
+- ``Namespace.as_dict`` did not convert the namespaces nested in a ``dict`` or
+  ``list`` that also holds values which are not namespaces, e.g. a ``TypedDict``
+  with one key of a class type and another of a simple type. Dumping such a
+  config as json failed with ``Object of type Namespace is not JSON
+  serializable`` (`#967 <https://github.com/mauvilsa/jsonargparse/pull/967>`__).
+- ``add_class_arguments`` given a subscripted generic class, e.g.
+  ``SomeClass[int]``, did not instantiate it, giving a ``Namespace`` instead of
+  an instance (`#967 <https://github.com/mauvilsa/jsonargparse/pull/967>`__).
 
 Changed
 ^^^^^^^

DOCUMENTATION.rst

@@ -448,23 +448,26 @@ Types can be nested with any complexity. Notes about the support:
   class that owns the method, e.g. a nested class referred to without qualifying
   it.
 
-- Fully supported types are: ``str``, ``bool`` (see :ref:`boolean-arguments`),
-  ``int``, ``float``, ``Decimal``, ``complex``, ``bytes``/``bytearray`` (Base64
-  encoding), ``range``, ``list`` (see :ref:`list-append`), ``Deque``,
-  ``Iterable``, ``Sequence``, ``MutableSequence``, ``Collection``,
-  ``Container``, ``Reversible``, ``Any``/``object``, ``Union``/``Optional`` (see
-  :ref:`union-types`), ``Literal``, ``Type``, ``Enum``, ``PathLike``, ``UUID``,
-  ``timedelta``, the restricted types of :ref:`restricted-numbers` and
-  :ref:`restricted-strings`, and the path and URL types of :ref:`parsing-paths`
-  and :ref:`parsing-urls`.
+- Fully supported types are: ``str``/``LiteralString``, ``bool`` (see
+  :ref:`boolean-arguments`), ``int``, ``float``, ``Decimal``, ``complex``,
+  ``bytes``/``bytearray`` (Base64 encoding), ``range``, ``list`` (see
+  :ref:`list-append`), ``Deque``, ``Iterable``, ``Sequence``,
+  ``MutableSequence``, ``Collection``, ``Container``, ``Reversible``,
+  ``Any``/``object``, ``Union``/``Optional`` (see :ref:`union-types`),
+  ``Literal``, ``Type``, ``Enum``, ``PathLike``, ``UUID``, ``timedelta``, the
+  restricted types of :ref:`restricted-numbers` and :ref:`restricted-strings`,
+  and the path and URL types of :ref:`parsing-paths` and :ref:`parsing-urls`.
 
 - ``dict``, ``Mapping``, ``MutableMapping``, ``MappingProxyType``,
   ``OrderedDict`` and ``TypedDict`` are supported, but only with ``str`` or
   ``int`` keys, see :ref:`dict-items`.
 
 - ``TypedDict`` accepts ``Required`` and ``NotRequired`` to mark single keys as
-  required or optional, and ``Unpack`` to type ``**kwargs`` precisely, see PEP
-  `692 <https://peps.python.org/pep-0692/>`__. A ``--*.help`` option, e.g.
+  required or optional, ``ReadOnly`` (PEP `705
+  <https://peps.python.org/pep-0705/>`__) which only marks a key as not mutable
+  and thus changes neither its type nor its requiredness, and ``Unpack`` to
+  type ``**kwargs`` precisely, see PEP `692
+  <https://peps.python.org/pep-0692/>`__. A ``--*.help`` option, e.g.
   ``--data.help``, shows the accepted keys. It takes no value, unless the
   ``TypedDict`` is in a union with other types that have their own help, in
   which case the value is the name of the typed dict, e.g. ``--data.help
@@ -491,6 +494,15 @@ Types can be nested with any complexity. Notes about the support:
   a list when parsing, since subclass specs are not hashable, and becomes a set
   on :meth:`instantiate <.ArgumentParser.instantiate>`.
 
+- ``NamedTuple`` is supported. The value is either an object with the fields as
+  keys or an array of positional values, fields with a default can be omitted,
+  and parsing gives an instance of the named tuple. It is always dumped as an
+  object, so that the fields are named. A ``--*.help`` option shows the accepted
+  fields, and :meth:`add_class_arguments <.ArgumentParser.add_class_arguments>`
+  accepts a ``NamedTuple``, both the same as for a ``TypedDict``. A generic
+  ``NamedTuple`` works unsubscripted and subscripted, e.g. ``SomeTuple[int]``. A
+  field of an untyped ``collections.namedtuple`` accepts any value.
+
 - ``None`` is written as ``null``, as JSON/YAML define it. For the same reason
   the help shows ``NoneType`` as ``null``, e.g. a parameter with type and
   default ``Optional[str] = None`` is shown as ``type: Union[str, null],
@@ -558,7 +570,15 @@ Types can be nested with any complexity. Notes about the support:
 - ``TypeAliasType`` is supported. Values are parsed as the aliased type and the
   help shows the alias as the argument type. This includes aliases defined with
   the `PEP 695 <https://peps.python.org/pep-0695/>`__ ``type X = ...`` statement
-  (Python 3.12+) and aliases created with ``typing_extensions.TypeAliasType``.
+  (Python 3.12+) and aliases created with ``typing_extensions.TypeAliasType``. A
+  generic alias, e.g. ``type X[T] = list[T]``, is parsed as its target with the
+  type parameters substituted by what it is subscripted with, e.g. ``X[int]``
+  behaves as ``list[int]``. Unsubscripted, its type parameters stand for their
+  default, constraints or bound, the same as any other ``TypeVar``.
+
+- ``NewType`` is supported. Values are parsed as the supertype it stands for,
+  including a ``NewType`` of a ``NewType``, and the help shows the name given in
+  the source code.
 
 
 .. _union-types:
@@ -635,13 +655,13 @@ an argument of type ``Union[int, list[int]]``, ``--val=1`` gives ``1``, while
 Unvalidated types
 -----------------
 
-A :ref:`signature parameter <classes-methods-functions>` or a ``TypedDict`` key
-can have a type that jsonargparse can't validate. The argument is still added,
-with only the parts of the type that can't be validated replaced by a type that
-accepts any value. The help shows these parts as ``Unvalidated<...>``, keeping
-the name used in the source code. For example, a class with a parameter
-``items: list[SomeType] = []`` for which ``SomeType`` can't be validated is
-shown in the help as:
+A :ref:`signature parameter <classes-methods-functions>`, a ``TypedDict`` key or
+a ``NamedTuple`` field can have a type that jsonargparse can't validate. The
+argument is still added, with only the parts of the type that can't be validated
+replaced by a type that accepts any value. The help shows these parts as
+``Unvalidated<...>``, keeping the name used in the source code. For example, a
+class with a parameter ``items: list[SomeType] = []`` for which ``SomeType``
+can't be validated is shown in the help as:
 
 .. code-block:: text
 
@@ -3441,8 +3461,8 @@ which subclasses accept each one. For example:
     $ example.py --cls other.module.SubclassA --cls.param2 <TAB><TAB>
     Expected type: int; Accepted by subclasses: SubclassA
 
-Analogously, for subclasses-disabled types and ``TypedDict``, the fields or keys
-are completed, as well as the values that they accept, e.g.:
+Analogously, for subclasses-disabled types, ``TypedDict`` and ``NamedTuple``,
+the fields or keys are completed, as well as the values that they accept, e.g.:
 
 .. code-block:: bash
 

jsonargparse/_actions.py

@@ -31,6 +31,7 @@
     get_import_path,
     import_object,
     indent_text,
+    iter_to_or_str,
     iter_to_set_str,
     load_config_path_context,
     merge_config,
@@ -353,15 +354,15 @@ def __init__(self, typehint=None, **kwargs):
             super().__init__(**kwargs)
 
     def update_init_kwargs(self, kwargs):
-        from ._typehints import get_help_types, is_protocol, is_typed_dict
+        from ._typehints import get_help_types, is_namedtuple, is_protocol, is_structured_value_type, is_typed_dict
 
         self._typehint = kwargs.pop("_typehint")
         self._help_types = get_help_types(self._typehint)
-        typed_dicts = [t for t in self._help_types if is_typed_dict(t)]
-        # a subscripted generic typed dict is a generic alias instead of a class, see get_typed_dict_type
-        assert self._help_types and all(isinstance(b, type) for b in self._help_types if b not in typed_dicts)
+        # a subscripted generic typed dict or named tuple is a generic alias instead of a class
+        structured = [t for t in self._help_types if is_structured_value_type(t)]
+        assert self._help_types and all(isinstance(b, type) for b in self._help_types if b not in structured)
         # a single type means that the help refers to it, so no value is expected
-        single_type = len(self._help_types) == 1 and (is_subclasses_disabled(self._help_types[0]) or bool(typed_dicts))
+        single_type = len(self._help_types) == 1 and (is_subclasses_disabled(self._help_types[0]) or bool(structured))
         self._basename = iter_to_set_str(t.__name__ for t in self._help_types)
 
         if len(self._help_types) == 1:
@@ -374,13 +375,15 @@ def update_init_kwargs(self, kwargs):
             self._kind = "subclass of"
             if any(is_protocol(b) for b in self._help_types):
                 self._kind = "subclass or implementer of protocol"
-            if typed_dicts:
-                # a typed dict is given by name, since it doesn't accept a class path
-                if len(typed_dicts) == len(self._help_types):
+            if structured:
+                # a typed dict or named tuple is given by name, since it doesn't accept a class path
+                kinds = ["typed dict"] if any(is_typed_dict(t) for t in structured) else []
+                kinds += ["named tuple"] if any(is_namedtuple(t) for t in structured) else []
+                if len(structured) == len(self._help_types):
                     kwargs["metavar"] = "NAME"
-                    self._kind = "typed dict"
                 else:
-                    self._kind = "class or typed dict"
+                    kinds.insert(0, "class")
+                self._kind = iter_to_or_str(kinds)
             msg = f"the given {self._kind} "
 
         kwargs["default"] = SUPPRESS
@@ -393,15 +396,15 @@ def __call__(self, *args, **kwargs):
         return self.print_help(args)
 
     def resolve_help_type(self, value, option_string):
-        from ._typehints import implements_protocol, is_typed_dict, resolve_class_path_by_name
+        from ._typehints import implements_protocol, is_structured_value_type, resolve_class_path_by_name
 
         if self.nargs == 0 or (self.nargs == "?" and value is None):
             return self._help_types[0]
-        typed_dict = next((t for t in self._help_types if is_typed_dict(t) and t.__name__ == value), None)
-        if typed_dict:
-            return typed_dict
-        # typed dicts excluded since they don't have subclasses that a class path could refer to
-        class_types = tuple(t for t in self._help_types if not is_typed_dict(t))
+        structured = next((t for t in self._help_types if is_structured_value_type(t) and t.__name__ == value), None)
+        if structured:
+            return structured
+        # structured value types excluded, they don't have subclasses that a class path could refer to
+        class_types = tuple(t for t in self._help_types if not is_structured_value_type(t))
         val_class = None
         if class_types:
             try:

jsonargparse/_common.py

@@ -21,9 +21,12 @@
     capture_typing_extension_shadows,
     get_alias_target,
     get_annotated_base_type,
+    get_new_type_supertype,
     is_alias_type,
     is_annotated,
     is_attrs_class,
+    is_literal_string,
+    is_new_type,
     is_pydantic_model,
     typing_extensions_import,
 )
@@ -637,6 +640,10 @@ def get_unaliased_type(cls):
             new_cls = get_annotated_base_type(new_cls)
         if is_alias_type(new_cls):
             new_cls = get_alias_target(new_cls)
+        if is_new_type(new_cls):
+            new_cls = get_new_type_supertype(new_cls)
+        if is_literal_string(new_cls):
+            new_cls = str
         origin = get_unsubscripted_alias_origin(new_cls)
         if origin is not None:
             new_cls = origin

jsonargparse/_completions.py

@@ -31,8 +31,8 @@
     get_typed_dict_key_type,
     get_typehint_origin,
     is_single_subclass_or_closed_type,
+    is_structured_value_type,
     is_subclass,
-    is_typed_dict,
     type_to_str,
 )
 from ._util import NoneType, Path, import_object, merge_config, unique
@@ -370,11 +370,12 @@ def get_choices_state(typehint) -> tuple[list[str], bool, bool]:
             choices = add_subactions_and_get_subclass_choices(typehint, prefix, parser, skip, added_subclasses)
             return choices, True, False
 
-        if is_typed_dict(typehint) or (
+        if is_structured_value_type(typehint) or (
             is_single_subclass_or_closed_type(typehint, origin) and is_subclasses_disabled(typehint)
         ):
-            # a dataclass-like type is only inlined as a group when not in a union and a typed
-            # dict never is, so their init args or keys need to be added as options to complete them
+            # a dataclass-like type is only inlined as a group when not in a union and a typed dict
+            # or named tuple never is, so their init args, keys or fields need to be added as
+            # options to complete them
             added_subclasses.add(typehint)
             add_subactions_and_get_subclass_choices(typehint, prefix, parser, skip, added_subclasses, closed_type=True)
             return [], False, True
@@ -451,8 +452,9 @@ def add_subactions_and_get_subclass_choices(
 def get_help_class_choices(typehint) -> list[str]:
     choices: list[str] = []
     for help_type in get_help_types(typehint) or []:
-        if is_typed_dict(help_type):
-            choices.append(help_type.__name__)  # typed dicts don't accept a class path, only their name
+        if is_structured_value_type(help_type):
+            # a typed dict or named tuple doesn't accept a class path, only its name
+            choices.append(help_type.__name__)
         else:
             choices += [p for p in get_all_subclass_paths(help_type) if p not in choices]
     return choices

jsonargparse/_completions_jsonschema.py

@@ -27,16 +27,18 @@
     callable_origin_types,
     get_all_subclass_paths,
     get_callable_return_type,
+    get_namedtuple_annotations,
     get_typed_dict_annotations,
     get_typed_dict_required_keys,
     get_typehint_origin,
+    is_namedtuple,
     is_single_subclass_or_closed_type,
     is_single_subclass_type,
     literal_types,
     mapping_origin_types,
-    not_required_required_types,
     sequence_origin_types,
     tuple_set_origin_types,
+    typed_dict_key_qualifiers,
     typed_dict_meta_types,
 )
 from ._util import NoneType, get_import_path, import_object
@@ -396,7 +398,7 @@ def typehint_schema(self, typehint, action) -> dict:
 
         if typehint in {Any, object}:
             return {}
-        if origin in not_required_required_types:  # requiredness comes from the TypedDict, not the type
+        if origin in typed_dict_key_qualifiers:  # requiredness and mutability come from the TypedDict
             return self.typehint_schema(typehint.__args__[0], action)
         if typehint is uuid.UUID:
             return dict(uuid_schema)
@@ -409,6 +411,8 @@ def typehint_schema(self, typehint, action) -> dict:
             return {"enum": list(typehint.__members__)}
         if type(typehint) in typed_dict_meta_types:
             return self.typed_dict_schema(typehint, action)
+        if is_namedtuple(typehint):
+            return self.namedtuple_schema(typehint, action)
         if root in literal_types:
             return {"enum": [json_value(arg) for arg in typehint.__args__]}
         if origin is Union:
@@ -475,6 +479,23 @@ def typed_dict_schema(self, typehint, action) -> dict:
                 add_required(schema, key)
         return schema
 
+    def namedtuple_schema(self, typehint, action) -> dict:
+        """Describes both forms accepted for a NamedTuple: an object of fields or an array of values."""
+        annotations = get_namedtuple_annotations(typehint)
+        defaults = typehint._field_defaults
+        obj_schema = new_object(get_doc_short_description(typehint), schema_key=False)
+        for field, annotation in annotations.items():
+            obj_schema["properties"][field] = self.typehint_schema(annotation, action)
+            if field not in defaults:
+                add_required(obj_schema, field)
+        array_schema = {
+            "type": "array",
+            "prefixItems": [obj_schema["properties"][f] for f in annotations],
+            "items": False,
+            "minItems": len(annotations) - len(defaults),
+        }
+        return anyof_schema([obj_schema, array_schema])
+
     # classes
 
     def class_ref(self, class_type, action, subclass: bool) -> dict:

jsonargparse/_namespace.py

@@ -169,16 +169,7 @@ def __bool__(self) -> bool:
 
     def as_dict(self) -> dict[str, Any]:
         """Converts the nested namespaces into nested dictionaries."""
-        dic = {}
-        for key, val in vars(self).items():
-            if isinstance(val, Namespace):
-                val = val.as_dict()
-            elif isinstance(val, dict) and val != {} and all(isinstance(v, Namespace) for v in val.values()):
-                val = {k: v.as_dict() for k, v in val.items()}
-            elif isinstance(val, list) and val != [] and all(isinstance(v, Namespace) for v in val):
-                val = [v.as_dict() for v in val]
-            dic[del_clash_mark(key)] = val
-        return dic
+        return {del_clash_mark(key): _value_as_dict(val) for key, val in vars(self).items()}
 
     def as_flat(self) -> argparse.Namespace:
         """Converts the nested namespaces into a single argparse flat namespace."""
@@ -260,6 +251,17 @@ def pop(self, key: str, default: Any = None) -> Any:
 clash_mark = "\u200b"
 
 
+def _value_as_dict(val):
+    """Converts the namespaces nested in a value into dictionaries, including in containers."""
+    if isinstance(val, Namespace):
+        return val.as_dict()
+    if isinstance(val, dict):
+        return {k: _value_as_dict(v) for k, v in val.items()}
+    if isinstance(val, list):
+        return [_value_as_dict(v) for v in val]
+    return val
+
+
 def add_clash_mark(key: str) -> str:
     if key in clash_names:
         key = clash_mark + key