Problem caused by `within_flit()` when building own package with flit

[LustigePerson]

Hello,

we are using anndata and scanpy in our FASTGenomics reader module.
We are also using flit to built this package. During build flit tries to load scanpy, which then loads anndata, which ultimately crashes because AnnData cannot be imported due to the within_flit() function called in the anndata/init.py.

Are you aware of any way to overcome this issue?

[ivirshup]

Ping @flying-sheep

As another point of reference, scirpy also uses flit for it's build system but is able to work with anndata>0.7.6. @grst, maybe you've already worked out a solution here?

[grst]

I don't think I did anything specific, except maybe that I use the within_flit function myself:
https://github.com/icbi-lab/scirpy/blob/0f0b28243ee9db5e0bf1ffac24d78dbed83cb628/scirpy/__init__.py

[flying-sheep]

That hack means that during flit’s get_docstring_and_version_via_import, the module will be empty (except for __version__). How does that turn out to be a problem for you @LustigePerson?

I guess using the same hack really saves you from it, since it prevents your module from importing anything.

[LustigePerson]

@flying-sheep Well as I said, during flit build all imports are executed. This includes import scanpy. The error then occurs when scanpy is trying to import AnnData from anndata.
But perhaps it help that you say it occurs during get_docstring_and_version_via_import, I will have a look if flit allows to not do this for all packages.
For now I fixed it by just using an older anndata version for build. But I could also try to use the same "hack".

[flying-sheep]

But I could also try to use the same "hack".

You should. The only shortcoming of flit is that it doesn’t have great support for dynamic retrieval of versions:

Besides parsing a string literal, it only supports importing the base package namespace. That’s a hen-and-egg problem, since by that point the package’s dependencies are not necessarily available.

@takluyver only needs to merge one of pypa/flit#382 and pypa/flit#374, or implement a feature fixing pypa/flit#257 and pypa/flit#271 to make this limitation go away.

[LustigePerson]

Thannks, for the clarification. I'll try and close this issue!