Wheels do not explicitly specify the torch version

[nbren12]

I get this error in my current environment after `pip install torch-harmonics'

│ ImportError: /scratch/nbrenowitz/healda/.venv/lib/python3.12/site-packages/torch_harmonics/attention/_C.cpython-312-x86_64-linux-gnu.so: undefined        │
│ symbol: _ZNR5torch7Library4_defEON3c1014FunctionSchemaEPNS1_12OperatorNameERKSt6vectorIN2at3TagESaIS8_EENS_17_RegisterOrVerifyE

I think this is because the torch ABI is not compatible between torch minor versions. I am currently using torch 2.12.1, python 312, and cuda 13.

If you want to fix, e2grid has some code to encode the pytorch version in the wheel name, but it's a bit awkward.

[azrael417]

Hello Noah,

thanks for bringing this up. We had a long discussion with how to achieve this in a clean fashion, as we do not want to support #cuda versions x # pytorch versions kitmaker projects. As long as there is no good solution for this, we decided to support only the oldest torch available for any given CUDA versions.

What is missing in the wheels, is to tighten the support of the wheel for exactly this CUDA version, because seemingly PyTorch does not guarantee ABI stability even across minor versions. In upcoming releases, we will add that constraint and also update the compatibility matrix https://github.com/NVIDIA/torch-harmonics#prebuilt-wheels accordingly.

Let me know if that is a solution.

Best regards
Thorsten

[nbren12]

Makes sense. Wheels are tricky.

In case it’s helpful here is how I specified the correct PyTorch constraint at build time: https://github.com/NVlabs/earth2grid/blob/8fdff5a78d324f8d25afe224915301b3169bffe2/setup.py#L29. There were some special cases related to prereleases that I debugged for this. It came up when using from within the ngc PyTorch container. Not sure if that addresses the abi compatibility issues you have been encountering within minor versions.

[bonevbs]

Thank you @nbren12 for the suggestions. What we should add is a cleaner way for the precompiled wheels to fail when a newer pytroch is installed.

Unfortunately, the main issue with shipping compiled torch extensions as wheels is that the current PEP does not cleanly support variants and simultaneously enforces sandboxing. So neither is dynamically compiling an option nor can it resolve dependencies regarding the installed PyTorch version.

As @azrael417 already mentioned, trying to support every possible torch+CUDA combination will result in combinatorial explosion so we limit ourselves to common combinations.

[nbren12]

What we should add is a cleaner way for the precompiled wheels to fail when a newer pytroch is installed.

The behavior of the e2grid wheels is to overwrite any existing incompatible pytorch with version compatible with version the wheel was built against. Not sure if that is what you want, but it does resolve this issue and make the metadata in the wheel more accurate about what pytorch version it can be used with.

setup(install_requires=["torch{major}.{minor+1},torch>={major}.{minor}", ... ])

I agree supporting every python x torch x cuda version is tough, and that the PEP is not clear and that every pytorch extension does it differently.