Repository navigation
PETSc.jl and GPUs #102
Description
In order to use PETSc on GPUs, we'll also need to wrap the necessary data types for CUDA and OpenCL. We'll need to provide the compiled binaries as well; https://www.mcs.anl.gov/petsc/features/gpus.html
Activity
ViralBShah commentedon Oct 16, 2024
@boriskaus I believe we do not build
PETSc_jllwith GPU support right? Is it something that users expect nowadays? Again, a first step towards getting this eventually is having support in the JLL.boriskaus commentedon Oct 16, 2024
No we don't build it with GPU support (I imagine that doing that with BinaryBuilder is a nightmare).
If people are interested in this (which more and more are), they should perhaps build the correct version of PETSc themselves and link that one to
PETSc.jl, instead ofPETSc_jll. A crucial factor here is that we must ensure that the correct version of PETSc is being linkedViralBShah commentedon Oct 16, 2024
We do it increasingly in BinaryBuilder, but it definitely increases complexity. I agree that for now, this is a future feature, and people who need it should use their own PETSc build.
Of course, contributions to do this are welcome (there are several Yggdrasil examples with GPU enabled packages now).
b-fg commentedon Mar 14, 2025
Hi, I was looking into this package as an alternative option for our GMG in WaterLily.jl. GPU support in PETSc.jl would be very much appreciated, so I was wondering if there are plans in this direction for the short/mid term?
boriskaus commentedon Mar 14, 2025
PETScitself works with GPU support and we recently managed to scale one of our codes to a large GPU machine. This was done using our own build ofPETSc. I imagine that installing GOPU support though binarybuilder is rather complicated, so I don't think I will touch that anywhere soon.Also, this has many layers of abstraction (PETSc, for example, creates a copy of every vector on the GPU and CPU), and you certainly loose efficiency with every abstraction layer.
Unless you really need to use the nonlinear solvers of
PETSc, I therefore imagine that it will be much simpler to use either ParallelStencil.jl or Chmy.jl basis for your GMG solvers, which both have excellent (parallel) GPU support on different flavors of GPUs. Since this is written in Julia it is closer to the GPU chip, so I imagine that you loose less efficiency (although I haven't measured that)b-fg commentedon Mar 14, 2025
Thanks for the details. I was looking for an alternative to our GMG, which already works with GPUs. The only constraints I had in mind were support for MPI, automatic differentiation, and GPU. The packages you mention offer support for finite differences/general stencils, but not as linear algebra solvers, right?
ViralBShah commentedon Mar 14, 2025
Actually it may not be too hard to add gpu support. Lots of packages do that nowadays. It is worth a shot at least.
ViralBShah commentedon Oct 4, 2025
The GPU support in BB is now pretty good with lots of libraries built with GPU support. So this should be pretty straightforward now.
sloisel commentedon Dec 2, 2025
Hi, just to let you know that GPU support (and Strumpack specifically) would be kindly appreciated in this. I tried to set up Github builder CI to use self-made PETSc builds and it's been a nightmare. I think this is beyond the reach of mere mortals.
ViralBShah commentedon Sep 16, 2026
Given the recent improvements, this should be relatively easily doable with a PetSC_GPU recipe now, and the AGENTS.md file updates teach claude etc. how to do this well.