Adding scorep build option

[HendriceH]

Added scorep profling preset with option to automatically build scorep via cmake.
Since the compiler wrapper scorep-g++ or it's clang variant are required for the configure of NeoFOAM, this needs to happen in a two step process.

1. on first call of cmake --preset instrumented, look for scorep existance, if not found exit the configuration after external project at
2. Run cmake --build --preset instrumented to compile scorep
3. Run cmake --preset instrumented again and this time the new scorep is found and NeoFOAM configures normally
4. Run cmake --build --preset instrumented to compile NeoFOAM and dependencies

Status update:

• Reduce code in the main cmake
• scorep is compiled with --no-cuda, as this crashed the profiling in all cases due to asynchronos memcopy.
• the preset is now called "instrumented" since scorep is installed into build/scorep, this enables to recompile NeoFOAM multiple times without reinstalling scorep
• profiling only works with scorep is injected via NVCC_WRAPPER_DEFAULT_COMPILER
• when running the instrumented neoIcoFoam, a directory scorep-<timestap> should appear that can be visualised in cube
• One might need to set SCOREP_ENABLE_PROFILING=1 and SCOREP_ENABLE_TRACING=0 when running the executable

[github-actions]

Deployed test documentation to https://exasim-project.com/NeoFOAM/Build_PR_191

[chihtaw]

It doesn't work for me.
Score-P is built successfully, but the configure step for NeoFOAM fails with the error shown below:

-- Configuring done (101.8s)
You have changed variables that require your cache to be deleted.
Configure will be re-run and you may have to reset some variables.
The following variables have changed:
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_C_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++
CMAKE_CXX_COMPILER= /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-g++

-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc
-- Check for working C compiler: /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc - broken
CMake Error at /storage/apps/opt/spack/opt/spack/linux-x86_64_v3/cmake-3.30.8-aqhna7hiskf4dk2zvxsgq3lblvrdsavt/share/cmake-3.30/Modules/CMakeTestCCompiler.cmake:67 (message):
  The C compiler

    "/storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: '/storage/home/ctwang/Code/NeoFOAM/build/scorep/CMakeFiles/CMakeScratch/TryCompile-GY1lAa'
    
    Run Build Command(s): /usr/bin/ninja-build -v cmTC_82161
    [1/2] /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc    -o CMakeFiles/cmTC_82161.dir/testCCompiler.c.o -c /storage/home/ctwang/Code/NeoFOAM/build/scorep/CMakeFiles/CMakeScratch/TryCompile-GY1lAa/testCCompiler.c
    FAILED: CMakeFiles/cmTC_82161.dir/testCCompiler.c.o 
    /storage/home/ctwang/Code/NeoFOAM/build/scorep/scorep/bin/scorep-gcc    -o CMakeFiles/cmTC_82161.dir/testCCompiler.c.o -c /storage/home/ctwang/Code/NeoFOAM/build/scorep/CMakeFiles/CMakeScratch/TryCompile-GY1lAa/testCCompiler.c
    [Score-P] ERROR: This installation does not support 'cuda' instrumentation
    ninja: build stopped: subcommand failed.
    
    

  

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:45 (project)

[greole]

Needs a license header

Suggested change

	# Check whether scorep is present and set flag
	# SPDX-License-Identifier: Unlicense
	#
	# SPDX-FileCopyrightText: 2025 NeoFOAM authors
	# Check whether scorep is present and set flag

[chihtaw]

Since Score-P is compiled without cuda support, can it profile NeoFOAM running on NVIDIA GPUs?

[HendriceH]

It profiles the Kokkos functions, so indirectly the CUDA kernels. One could play around enabling CUDA in the scorep build and then setting the SCOREP_WRAPPER_INSTRUMENTER_FLAGS to --no-kokkos and --cuda and see if the produces valid profiling. I got cuMemcopy errors when Kokkos and CUDA were enabled.

[greole]

It profiles the Kokkos functions, so indirectly the CUDA kernels.

I think it is a first step and ok for me. Besides that, is the update of src/NeoN required here? We dont use it in our CI and i would rather recommend usind -DNeoFOAM_NEON_DIR to use a custom version of Neon.,

[HendriceH]

It profiles the Kokkos functions, so indirectly the CUDA kernels.

I think it is a first step and ok for me. Besides that, is the update of src/NeoN required here? We dont use it in our CI and i would rather recommend usind -DNeoFOAM_NEON_DIR to use a custom version of Neon.,

I think it snuck in at some point as a submoduel. Maybe we can .gitignore the src/NeoN directory and take out the submodule entirely?

[greole]

I think it snuck in at some point as a submoduel. Maybe we can .gitignore the src/NeoN directory and take out the submodule entirely?

That would be my preference. But I guess @HenningScheufler would like to keep it.

[chihtaw]

@HendriceH Did you manage to profile neoIcoFoam using the Score-P build provided in this PR?
I tried it for cavity case on NVIDIA GPU. The simulation was aborted near the end time with the message:

Running ../../build/instrumented/bin/neoIcoFoam on /storage/home/ctwang/Code/NeoFOAM/tutorials/cavity
/storage/home/ctwang/spack/opt/spack/linux-zen3/openfoam-2412-aigjozo3df3dv6wpno7674in3pftbha6/bin/tools/RunFunctions: line 235: 3094694 Aborted                 (core dumped) $appRun $appArgs "$@" > $logFile 2>&1

And the log file of neoIcoFoam ends with

[Score-P] src/measurement/thread/create_wait/scorep_thread_create_wait_generic.c:640: Warning: Thread after main (location=1)
[Score-P] src/measurement/thread/create_wait/scorep_thread_create_wait_generic.c:640: Warning: Thread after main (location=2)
munmap_chunk(): invalid pointer

Score-P did generate some profiling data, but the simulation didn't get finalized properly with Score-P.

[chihtaw]

@HendriceH Can you get any info about GPU activities from the profiling data generated by Score-P which is built via the CMake preset in this PR?