feat!: Unify X and layers

.github/workflows/test-cpu.yml

@@ -41,6 +41,7 @@ jobs:
     needs: get-environments
     runs-on: ubuntu-latest
     strategy:
+      fail-fast: false
       matrix:
         env: ${{ fromJSON(needs.get-environments.outputs.envs) }}
         io_mark: ["zarr_io", "not zarr_io", "dask_distributed"] # dask_distributed should not be run with -n auto as it uses a client with processes

docs/concatenation.rst

@@ -30,6 +30,7 @@ Let's start off with an example:
         obsm: 'X_pca', 'X_umap'
         varm: 'PCs'
         obsp: ...
+        layers: None
 
 If we split this object up by clusters of observations, then stack those subsets we'll obtain the same values – just ordered differently.
 
@@ -42,6 +43,7 @@ If we split this object up by clusters of observations, then stack those subsets
         var: 'n_counts', 'means', 'dispersions', 'dispersions_norm', 'highly_variable'
         obsm: 'X_pca', 'X_umap'
         varm: 'PCs'
+        layers: None
 
 Note that we concatenated along the observations by default, and that most elements aligned to the observations were concatenated as well.
 A notable exception is :attr:`~anndata.AnnData.obsp`, which can be re-enabled with the `pairwise` keyword argument.
@@ -168,6 +170,7 @@ First, our example case:
         uns: 'pca'
         obsm: 'X_pca'
         varm: 'PCs'
+        layers: None
 
 Now we will split this object by the categorical `"blobs"` and recombine it to illustrate different merge strategies.
 
@@ -184,6 +187,7 @@ Now we will split this object by the categorical `"blobs"` and recombine it to i
         uns: 'pca'
         obsm: 'X_pca', 'qc'
         varm: 'PCs', '0_qc'
+        layers: None
 
 `adatas` is now a list of datasets with disjoint sets of observations and a common set of variables.
 Each object has had QC metrics computed, with observation-wise metrics stored under `"qc"` in `.obsm`, and variable-wise metrics stored with a unique key for each subset.
@@ -193,16 +197,19 @@ Taking a look at how this affects concatenation:
     AnnData object with n_obs × n_vars = 640 × 30
         obs: 'blobs'
         obsm: 'X_pca', 'qc'
+        layers: None
     >>> ad.concat(adatas, merge="same")
     AnnData object with n_obs × n_vars = 640 × 30
         obs: 'blobs'
         obsm: 'X_pca', 'qc'
         varm: 'PCs'
+        layers: None
     >>> ad.concat(adatas, merge="unique")
     AnnData object with n_obs × n_vars = 640 × 30
         obs: 'blobs'
         obsm: 'X_pca', 'qc'
         varm: 'PCs', '0_qc', '1_qc', '2_qc', '3_qc', '4_qc'
+        layers: None
 
 Note that comparisons are made after indices are aligned.
 That is, if the objects only share a subset of indices on the alternative axis, it's only required that values for those indices match when using a strategy like `"same"`.

docs/conf.py

@@ -181,6 +181,7 @@ def res(
     ("py:class", "anndata.acc.GenericAlias"),
     ("py:obj", "typing.R"),
     ("py:class", "_M"),
+    ("py:class", "anndata.utils.Default"),
 ]
 
 # -- Social cards ---------------------------------------------------------

docs/release-notes/1707.breaking.md

@@ -0,0 +1 @@
+Remove `dtype` argument from {class}`~anndata.AnnData` constructor. {user}`flying-sheep`

docs/release-notes/1707.feat.md

@@ -0,0 +1,3 @@
+Move {attr}`~anndata.AnnData.X` into {attr}`anndata.AnnData.layers` under the `None` key.
+As result, `layers: X` shows up in `AnnData`’s text representation, `None` appears in `.layers.items()`/`.keys()`, {func}`~anndata.concat` now works when some `AnnData`s have no `X`.
+{user}`flying-sheep` & {user}`ilan-gold`

src/anndata/_core/access.py

@@ -7,10 +7,10 @@
     from anndata import AnnData
 
 
-class ElementRef(NamedTuple):
+class ElementRef[K: (str, str | None)](NamedTuple):
     parent: AnnData
     attrname: str
-    keys: tuple[str, ...] = ()
+    keys: tuple[K, ...] = ()
 
     def __str__(self) -> str:
         return f".{self.attrname}" + "".join(f"[{x!r}]" for x in self.keys)