RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:5 and cuda:0! (when checking argument for argument weight in method wrapper_CUDA__cudnn_convolution)

[Qigqi]

I get this error when I multi-GPU use llava's inference code.How should I fix the code to make it work on multiple gpu's? Thanks,a lot.@John-Ge
embedding_output = self.vision_tower.embeddings(images)

[John-Ge]

It seems that this error does not come from our code. This error may come from deployment of model and data. The data and model are not on the same device. You could check the device of the model and data.

[Qigqi]

Before embedding_output = self.vision_tower.embeddings(images)this code image’s feature and model are on the same device，When running into this code image‘s feature move to the other device.I don't know what happened.