Bug: Sys GMRES example fails on some CUDA machines

[shakedregev]

Description how to reproduce the bug

./examples/sysGmres.exe   -m ../matrixmarket/fidap012.mtx   -r ../matrixmarket/fidap012_rhs1.mtx   -b cuda -i fgmres
Matrix file: ../matrixmarket/fidap012.mtx
RHS file: ../matrixmarket/fidap012_rhs1.mtx
sysGmres with CUDA backend
solver.setMatrix returned status: 0
solver.preconditionerSetup returned status: 0
[ERROR] CUDA error in function cudaMemcpy(dst, src, sizeof(T) * n, cudaMemcpyDeviceToHost) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:133, error# 716
[ERROR] Dot product failed with error code 15
[ERROR] CUDA error in function cudaMemcpy(dst, src, sizeof(T) * n, cudaMemcpyDeviceToDevice) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:150, error# 716
[ERROR] Dot product failed with error code 15
[ERROR] Dot product failed with error code 15
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
solver.solve returned status: 0
	 Initial relative residual norm ||b-A*x||/||b|| : 1.0000000000000000e+00
	 Final relative residual norm   ||b-A*x||/||b|| : 0.0000000000000000e+00
	 Number of iterations                           : 1
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716
[ERROR] CUDA error in function cudaFree(v) at /home/szb/ReSolve_dir/ReSolve/resolve/cuda/CudaMemory.hpp:43, error# 716

Re::Solve version

develop

System and environment details

Linux milan2.ftpn.ornl.gov 6.8.0-136-generic #136-Ubuntu SMP PREEMPT_DYNAMIC Wed Jul  1 21:53:05 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
gcc --version
gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0

This works fine on HIP and apparently some other CUDA machines
See original flagging here

Additional information

No response

[shakedregev]

@pelesh - @kakeueda made the original PR, but his machine does not catch this.
This may just be a milan2 thing, as we previously did tests on pdesx (but now it's down).

[pelesh]

@pelesh - @kakeueda made the original PR, but his machine does not catch this. This may just be a milan2 thing, as we previously did tests on pdesx (but now it's down).

This looks like incompatible CUDA driver. More information is needed CUDA version, CUDA capability/hardware.

[pelesh]

Not a Re::Solve bug.