Support AMD GPUs

[jammm]

Also uploading a patch for https://gitlab.inria.fr/bkerbl/simple-knn.git

0001-Support-AMD-GPUs.patch

With these changes, following the same steps as the one in the README.md should work fine. (in some cases you may have to add a --no-build-isolation to the pip install command, but that's it)

It builds fine for any RDNA GPU, but I'm unable to confirm it running as I'm not sure which example model to use. @heldJan @rvandeghen any models you can share that I can quickly check?

[heldJan]

Hi @jammm, thank you for your interest in Triangle Splatting. I uploaded the Garden scene for you, let me know if it works.

https://filesender.belnet.be/?s=download&token=df6bfefa-a03d-4c59-a19b-b53269ad9097

[jammm]

@heldJan thanks! I just tried it but it's reporting an issue:

(venv) PS D:\jam\graphics\triangle-splatting> python create_video.py -m .\garden\
Looking for config file in .\garden\cfg_args
Config file found: .\garden\cfg_args
Creating video for .\garden\
Loading trained model at iteration 30000
Traceback (most recent call last):
  File "D:\jam\graphics\triangle-splatting\create_video.py", line 87, in <module>
    scene = Scene(args=dataset,
            ^^^^^^^^^^^^^^^^^^^
  File "D:\jam\graphics\triangle-splatting\scene\__init__.py", line 60, in __init__
    assert False, "Could not recognize scene type!"
           ^^^^^
AssertionError: Could not recognize scene type!

Is this because it's unable to find a sparse or transforms_train.json ?

[heldJan]

You have to add the path to the dataset using: -s path/to/MipNerf/garden/

[jammm]

Same error. I must be missing something stupid. Is there a command I could use?

[jammm]

The zip doesn't contain either of sparse or transforms_train.json. Could that be the issue?

[heldJan]

Those files are inside the MipNeRF360 dataset.

The command to create the video should be:
python create_video.py -m <path_to_model> -s <path_to_scene>

where <path_to_model> is the path to the model I sent you, and <path_to_scene> is the path to MipNeRF360/garden, which contains the images of the garden scene, sparse data, etc.

[jammm]

Those files are inside the MipNeRF360 dataset.

The command to create the video should be: python create_video.py -m <path_to_model> -s <path_to_scene>

where <path_to_model> is the path to the model I sent you, and <path_to_scene> is the path to MipNeRF360/garden, which contains the images of the garden scene, sparse data, etc.

Ah okay. My bad. The zip you sent me has the checkpoint, and I should download the mipnerf360 dataset for the rest. Thanks! I'll download it and report back.

[jammm]

So it seems to be training and loss is trending downwards, which is promising. Currently training on a framework desktop containing Strix Halo (Radeon 8060S)

However, if I try to make it render a video using the existing checkpoint, it looks for "triangles_points" which doesn't exist in the state_dict:

(venv) PS D:\jam\graphics\triangle-splatting> python create_video.py -m .\garden\ -s .\garden\
Looking for config file in .\garden\cfg_args
Config file found: .\garden\cfg_args
Creating video for .\garden\
Loading trained model at iteration 30000
Reading camera 185/185
Loading Training Cameras
[ INFO ] Encountered quite large input images (>1.6K pixels width), rescaling to 1.6K.
 If this is not desired, please explicitly specify '--resolution/-r' as 1
Loading Test Cameras
Traceback (most recent call last):
  File "D:\jam\graphics\triangle-splatting\create_video.py", line 87, in <module>
    scene = Scene(args=dataset,
            ^^^^^^^^^^^^^^^^^^^
  File "D:\jam\graphics\triangle-splatting\scene\__init__.py", line 89, in __init__
    self.triangles.load(os.path.join(self.model_path,
  File "D:\jam\graphics\triangle-splatting\scene\triangle_model.py", line 258, in load
    shapes = point_cloud_state_dict["triangles_points"]
             ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
KeyError: 'triangles_points'

The state_dict has dict_keys(['convex_points', 'sigma', 'active_sh_degree', 'features_dc', 'features_rest', 'opacity'])

[rvandeghen]

This checkpoint was trained with an older version. This new one should be correct: https://filesender.belnet.be/?s=download&token=635e7912-f128-4fd8-ab53-7b31bd0d7694
Sorry for the inconvenience.

[jammm]

No worries at all. Appreciate the quick reply! I’ll check in asap.

[jammm]

Just tried it but it seems like the .pt file is corrupted:

(venv) PS D:\jam\graphics\triangle-splatting> python create_video.py -m .\garden\ -s .\garden\
Looking for config file in .\garden\cfg_args
Config file found: .\garden\cfg_args
Creating video for .\garden\
Loading trained model at iteration 30000
Reading camera 185/185
Loading Training Cameras
Loading Test Cameras
point_cloud_state_dict  .\garden\point_cloud\iteration_30000\point_cloud_state_dict.pt True
Traceback (most recent call last):
  File "D:\jam\graphics\triangle-splatting\create_video.py", line 87, in <module>
    scene = Scene(args=dataset,
            ^^^^^^^^^^^^^^^^^^^
  File "D:\jam\graphics\triangle-splatting\scene\__init__.py", line 89, in __init__
    self.triangles.load(os.path.join(self.model_path,
  File "D:\jam\graphics\triangle-splatting\scene\triangle_model.py", line 256, in load
    point_cloud_state_dict = torch.load(os.path.join(path, 'point_cloud_state_dict.pt'))
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\jam\graphics\triangle-splatting\venv\Lib\site-packages\torch\serialization.py", line 1486, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\jam\graphics\triangle-splatting\venv\Lib\site-packages\torch\serialization.py", line 771, in __init__
    super().__init__(torch._C.PyTorchFileReader(name_or_buffer))
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

[rvandeghen]

We are converging 😅
https://filesender.belnet.be/?s=download&token=03de093a-f86e-4986-899a-b6e6e11f1dbf

[jammm]

Success! https://drive.google.com/file/d/1c0WV-8KqvPuhw9zQABLizTOQFs0DtPim/view?usp=sharing
It seems to work well. Inference at around 6it/s on strix halo.

If this PR (and the patch for simple-knn) builds fine on a CUDA machine, it should be good to merge.