Why is there a need to wrap dask array views in DaskArrayView

[joshua-gould]

Question

I don't understand why anndata wraps dask arrays with DaskArrayView. Just using regular dask arrays give the expected behavior:

import dask.array as da
import numpy as np

x = da.zeros((2, 2))
x2 = x[[0, 1]]
x2[0, 0] = 1
np.testing.assert_equal(x2.compute(), np.array([[1, 0], [0, 0]]))
np.testing.assert_equal(x.compute(), np.zeros((2, 2)))

Perhaps views of numpy arrays should follow numpy view vs. copy semantics (https://numpy.org/doc/stable/user/basics.copies.html) as other libraries like xarray?:

import numpy as np
import xarray as xr
xarray1 = xr.DataArray(np.zeros((2, 2)))
xarray2 = xarray1[0:1]
xarray2[0, 0] = 1
assert xarray1[0, 0] == 1

ad1 = anndata.AnnData(np.zeros((2, 2)))
ad2 = ad1[0:1]
ad2.X[0, 0] = 1
assert ad1.X[0, 0] == 0

Thanks.

[ilan-gold]

Great question, I've wondered the same thing myself (the class predates my time here). I agree with you about the view/copy semnatics (I'm the one who implemented xarray in here, so there I didn't make a class for exactly that reason). BTW re: CoW I have #2327 in flight - does that seem reasonable to you?

[joshua-gould]

#2327 looks good, thanks.

[ilan-gold]

https://scverse.zulipchat.com/#narrow/channel/315789-data-structures/topic/Setting.20.60X.60.20on.20a.20view.20i.2Ee.2E.2C.20.60adata.5Bsubset.5D.2EX.20.3D.20Value.60/near/573044416

I've openly pondered about this on zulip. The behavior we are trying to encode with the view mechanism has two purposes in general, setting aside that as you point out, this CoW behavior already "works" with dask:

1. When you do adata = ad.AnnData(x) and then subset = adata[0], doing X = subset.X followed by X[0, 0] = 0 has the side effect of intiailizing the entire subset object (i.e., it is no longer a view). So this side-effect explains the view-behavior as far as I can tell (and not for CoW with dask)
2. Uniform behavior across items i.e., CoW for numpy as you point out

I think this is a moderately weak use-case considering that we are now supporting jax which isn't even mutable. And dataframes are CoW as of pandas>=3. So I'd consider removing the entire view mechanism. But I'd like to hear your thoughts since you've been around this a lot longer than I have

[joshua-gould]

I'm in favor of removing the view mechanism as I think it causes more issues than it solves.

[joshua-gould]

• Unable to rechunk a view of a 2-d sliced AnnData with a dask array #2330
• Error calling numpy nan functions on views #2149

Here's another example, where anndata makes an unnecessary copy which doubles the amount of memory required:

import anndata
import numpy as np

X = np.ones((10, 10))
indices = np.array([0, 4, 5])
assert X[indices].base is None  # not a view

d1 = anndata.AnnData(X=X)
d2 = d1[[0, 4, 5]]
d2.X[0, 0] = 0  # already a copy, no need to copy again