Adds BLT_ALLOW_MISSING_MPI_WRAPPER option to allow Fortran configs with MPI to not have an MPI_Fortran_COMPILER

[kennyweiss]

This PR adds the BLT_ALLOW_MISSING_MPI_WRAPPER option to enable users to configure MPI without providing some MPI__COMPILER variables for some enabled languages.

This came up in an msan config where we need Fortran to be enabled (ENABLE_FORTRAN=TRUE) as well as MPI (ENABLE_MPI=TRUE), but we don't need an MPI wrapper for Fortran -- and more importantly, we don't have an MPI wrapper for Fortran that is compatible with our C and C++ MPI wrappers.

[tomstitt]

thanks @kennyweiss !

[kennyweiss]

Fun wrinkle from the openmp_mpi_fortran_tests CI:

CMake Error at /usr/tce/backend/installations/linux-rhel8-x86_64/gcc-10.3.1/cmake-3.23.1-mdfqd2l7c33zg7xcvqizwz25vqmp7jfw/share/cmake-3.23/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find MPI (missing: MPI_C_FOUND C) (found version "3.1")
      Reason given by package: MPI component 'C' was requested, but language C is not enabled.  

Good test!

I was wondering if I'd need extra logic about checking for a valid C and/or C++ MPI compiler/wrapper.
Should hopefully be an easy fix.

[white238]

I chatted with @kennyweiss directly but putting summary here:

I am concerned this is changing a long standing behavior that all enabled languages needed an MPI compiler wrapper due to not listing components on the find(mpi) call prior to this. I think it failed before if a user didn't set all the wrappers (or possibly defaulted to a system one).

My recommendation is to add a sanity check for all wrappers being provided in the normal case (improvement over what is there today), then the niche case requiring the user to set a new variable, something like BLT_ALLOW_MISSING_MPI_WRAPPER (or some better name), to skip the sanity check.

Thoughts?

[cyrush]

I think implicitly needing all wrappers was a mistake.

That was what led to issues with fortran flags being fused with c flags and undermining clang builds.

Also you don't need compiler wrappers -- you need the knowledge of how to compile and link (often comes from the wrappers, but not always).

FindMPI will work if you provide flags as well. This check for fortran wrapper my undermine that.

It would be better if BLT had and option for which languages to use with MPI.

That is what CMake's MPI has evolved to.

[white238]

@cyrush that makes sense. We can always pivot later. shall we merge then?

Adding some documentation to the MPI section that we will use information only from the given compiler wrappers would be good though.

[cyrush]

I think current fix is needed, and we can improve more in the future.

[kennyweiss]

I updated the branch based on @white238's suggestion. Specifically, by default, you need to provide a MPI_<lang>_COMPILER for each enabled language when ENABLE_MPI=ON.
Not providing these is a config error.

However, you can override that by setting BLT_ALLOW_MISSING_MPI_WRAPPER=ON.

Based on @cyrush's comment, we might not want to require users to provide a MPI_<lang>_COMPILER for each language.

Tag: @cyrush @white238

Edit: In the meantime, I have patched blt in my local build, so this fix is not blocking me. We should get the behavior correct before merging.