Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -646,6 +646,11 @@ jobs:
export PATH="/usr/local/cuda-12.8/bin:$PATH"
cmake --build --preset cuda --parallel 4
df -h /
- name: No device lambda in two of the extension's files (#815)
working-directory: native/build/presets/cuda
run: |
python3 ../../../ci/check_lambda_tags.py \
$(find CMakeFiles/solprop_native_core.dir CMakeFiles/_core.dir -name '*.o')

device-compile-hip:
needs: changes
Expand Down
1 change: 1 addition & 0 deletions changelog.d/815-radial-device-gemm.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fixed a segmentation fault in the native session's `advance` on a CUDA build for compute capability 8.6 (`AMPERE86`, such as an NVIDIA A40): every radial run crashed in its first step, envelope or field, with or without a collar or statistics, with every stepper. The device GEMM's kernel is now a named functor; as an extended host-device lambda instantiated alike in several source files, a Release build called a null copy function. Other device targets and the other routes were not affected ([#815](https://github.com/LupoLab/solprop/issues/815)).
11 changes: 10 additions & 1 deletion docs/dev/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -1597,7 +1597,16 @@ kernel behind a wait on the previous such launch, which kept the host from queui
trial's kernels ahead. A plain range reduction on HIP takes launch bounds of 256
threads with the hint, since Kokkos gives a light-weight HIP reduction its largest block.
CUDA builds pass functors as grid constants and are unchanged; host builds dispatch as
Kokkos does. The device Adams passes hold a trial's phi-functions and history in registers,
Kokkos does. A kernel in a shared header that several source files instantiate with the
same types is a named functor, not an extended `__host__ __device__` lambda: nvcc keeps
such a lambda's copy and delete functions in per-file static variables, set where the
lambda is constructed, while the link keeps one file's copy of the inline dispatch that
copies it. `device_dense_gemm` was such a lambda, and on a Release CUDA build for
compute capability 8.6 every radial run called a null copy function in its first step
([#815](https://github.com/LupoLab/solprop/issues/815)).
`native/ci/check_lambda_tags.py` fails a CUDA build's object files on an extended lambda
that more than one of them holds.
The device Adams passes hold a trial's phi-functions and history in registers,
dispatched on its order and width (one to eight samples): indexed per-thread arrays live
in scratch memory. An accepted Adams step ends without a fence on a device, whose
execution instances are its one stream. A device holds the two held plasma currents'
Expand Down
104 changes: 104 additions & 0 deletions native/ci/check_lambda_tags.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
#!/usr/bin/env python3
# SPDX-FileCopyrightText: 2026 John C. Travers and contributors
#
# SPDX-License-Identifier: GPL-3.0-or-later

"""Fail on a device lambda that two CUDA object files instantiate (#815).

nvcc gives an extended ``__host__ __device__`` lambda a wrapper whose copy, call and
delete functions are static members of ``__nv_hdl_helper``, in an anonymous namespace:
each object file holds its own, set where that file constructs the lambda. The host code
that copies the wrapper (Kokkos's dispatch) is an inline template with external
linkage, so when two files instantiate the same lambda the link keeps one file's copy
of it, which reads that file's members. A run that constructed the lambda only in the
other file then calls a null function: the radial route's device GEMM, instantiated
alike in three files, crashed so in a Release build on an NVIDIA A40. A kernel in a
shared header whose views can match between files takes a named functor instead.

``python native/ci/check_lambda_tags.py OBJECT...`` lists each object file's helper
symbols with ``nm`` and exits 1, naming every lambda's tag that more than one file
holds, unless ``ALLOWED`` lists it with the reason it never runs. It also fails when it
finds no helper symbol at all, so that a change in nvcc's naming cannot make it pass by
finding nothing.

Standard library only; it needs GNU ``nm`` (binutils) and object files that a CUDA
build compiled, at any optimisation level: the members exist whether or not a link
happens to keep the wrong copy. It sees only the kernels the build's device target
instantiates: the CUDA device-compile lane builds for compute capability 9.0, which
compiles no ``device_dense_gemm``, so the lambda behind #815 shows only in a build for
8.6 (``AMPERE86``), checked by hand over that build's objects.
"""

from __future__ import annotations

import re
import subprocess
import sys
from pathlib import Path

__all__ = ["ALLOWED", "HELPER", "helper_tags", "main", "shared_tags"]

HELPER = re.compile(r"__nv_hdl_helper<(.*)>::fp_copier$")
"""A demangled extended lambda's copy-function static; its group is the lambda's tag."""
ALLOWED: tuple[str, ...] = (
# Kokkos-FFT scales a transform's output in a lambda unless the normalisation is
# none; the backend passes `KokkosFFT::Normalization::none` to every transform and
# scales explicitly, so the lambda is instantiated but never constructed.
r"KokkosFFT::Impl::normalize_impl<",
)
"""Shared tags allowed, each a pattern searched for in the tag."""


def helper_tags(path: Path) -> set[str]:
"""Return the tags of the extended lambdas whose helper members `path` holds."""
listing = subprocess.run(
["nm", "--demangle", "--format=posix", str(path)],
check=True,
capture_output=True,
text=True,
).stdout
tags: set[str] = set()
for line in listing.splitlines():
# POSIX format: name, type, value, size; the demangled name holds spaces.
name = line.rsplit(" ", 3)[0] if line.count(" ") >= 3 else line
if match := HELPER.search(name):
tags.add(match[1])
return tags


def shared_tags(objects: list[Path]) -> tuple[int, dict[str, list[Path]]]:
"""Return the helper count, and each tag several of `objects` hold with them."""
holders: dict[str, list[Path]] = {}
count = 0
for path in objects:
for tag in helper_tags(path):
holders.setdefault(tag, []).append(path)
count += 1
return count, {tag: paths for tag, paths in holders.items() if len(paths) > 1}


def main(argv: list[str] | None = None) -> int:
"""Check the object files named; return 1 on a lambda two of them hold."""
args = sys.argv[1:] if argv is None else argv
if not args:
print("check_lambda_tags: name the object files", file=sys.stderr)
return 2
count, shared = shared_tags([Path(arg) for arg in args])
if not count:
print("check_lambda_tags: no extended lambda helper found", file=sys.stderr)
return 1
failed = False
for tag, paths in sorted(shared.items()):
files = ", ".join(sorted(path.name for path in paths))
if any(re.search(pattern, tag) for pattern in ALLOWED):
print(f"allowed in several files ({files}): {tag}")
else:
print(f"lambda in several files ({files}): {tag}", file=sys.stderr)
failed = True
if not failed:
print(f"{count} extended lambda helpers in {len(args)} files, none shared")
return 1 if failed else 0


if __name__ == "__main__":
sys.exit(main())
58 changes: 40 additions & 18 deletions native/include/solprop_native/gemm.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,45 @@ inline constexpr bool device_dense_gemm_enabled =
false;
#endif

/** `c = a * b + beta * c`, real, on a device: a thread takes `Rows` rows of one column
of C, so that each element of B it reads serves all of them. Each element is the
/** `device_dense_gemm`'s kernel: item `item` takes `Rows` rows of one column of C, so
that each element of B it reads serves all of them.

A named functor rather than an extended lambda (solprop #815). nvcc keeps a
`__host__ __device__` lambda's copy, call and delete functions in per-file static
variables that the lambda's construction sets, while the host code that copies it
(Kokkos's dispatch) is an inline template shared between files: this kernel's views
are the same in several files, so a Release link kept one file's copy of the
dispatch, and on a run that had constructed the lambda only in another file it
called a null copy function. A functor's copy is its own. */
template <std::size_t Rows, class AView, class BView, class CView>
struct DeviceDenseGemm {
AView a;
BView b;
CView c;
std::size_t rows;
std::size_t columns;
std::size_t inner;
double beta;

KOKKOS_FUNCTION void operator()(const std::size_t item) const {
const std::size_t column = item % columns;
const std::size_t first = (item / columns) * Rows;
std::size_t row_of[Rows];
for (std::size_t r = 0; r < Rows; ++r)
row_of[r] = Kokkos::min(first + r, rows - 1);
double value[Rows] = {};
for (std::size_t index = 0; index < inner; ++index) {
const double right = b(index, column);
for (std::size_t r = 0; r < Rows; ++r)
value[r] += a(row_of[r], index) * right;
}
for (std::size_t r = 0; r < Rows && first + r < rows; ++r)
c(first + r, column) =
beta == 0.0 ? value[r] : value[r] + beta * c(first + r, column);
}
};

/** `c = a * b + beta * c`, real, on a device (`DeviceDenseGemm`). Each element is the
custom kernel's sum (`custom_gemm`), its products added in order from zero. */
template <class ExecutionSpace, class AView, class BView, class CView>
void device_dense_gemm(const ExecutionSpace& execution_space, const AView& a,
Expand All @@ -200,22 +237,7 @@ void device_dense_gemm(const ExecutionSpace& execution_space, const AView& a,
launch::parallel_for(
"solprop_native.transform.device_dense_gemm",
Kokkos::RangePolicy<ExecutionSpace>(execution_space, 0, blocks * columns),
KOKKOS_LAMBDA(const std::size_t item) {
const std::size_t column = item % columns;
const std::size_t first = (item / columns) * Rows;
std::size_t row_of[Rows];
for (std::size_t r = 0; r < Rows; ++r)
row_of[r] = Kokkos::min(first + r, rows - 1);
double value[Rows] = {};
for (std::size_t index = 0; index < inner; ++index) {
const double right = b(index, column);
for (std::size_t r = 0; r < Rows; ++r)
value[r] += a(row_of[r], index) * right;
}
for (std::size_t r = 0; r < Rows && first + r < rows; ++r)
c(first + r, column) =
beta == 0.0 ? value[r] : value[r] + beta * c(first + r, column);
});
DeviceDenseGemm<Rows, AView, BView, CView>{a, b, c, rows, columns, inner, beta});
}

// The engine's KokkosBlas call: vendor or device GEMM where the build has one, and
Expand Down
96 changes: 96 additions & 0 deletions tests/test_check_lambda_tags.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# SPDX-FileCopyrightText: 2026 John C. Travers and contributors
#
# SPDX-License-Identifier: GPL-3.0-or-later

"""The CUDA build check that fails on a device lambda two files instantiate (#815)."""

import importlib.util
import subprocess
from pathlib import Path

SCRIPT = Path(__file__).parent.parent / "native" / "ci" / "check_lambda_tags.py"
spec = importlib.util.spec_from_file_location("check_lambda_tags", SCRIPT)
assert spec is not None and spec.loader is not None
check_lambda_tags = importlib.util.module_from_spec(spec)
spec.loader.exec_module(check_lambda_tags)


def _helper(function: str, slot: str = "fp_copier") -> str:
"""Return `nm --demangle --format=posix`'s line for a lambda's helper static."""
return (
"(anonymous namespace)::__nv_hdl_helper<__nv_dl_tag<void (*)(Kokkos::Cuda "
"const&, Kokkos::View<double**, Kokkos::LayoutRight, Kokkos::CudaSpace, "
f"Kokkos::MemoryTraits<1u> > const&, double), &(void {function}<Kokkos::Cuda, "
"Kokkos::View<double**, Kokkos::LayoutRight, Kokkos::CudaSpace, "
"Kokkos::MemoryTraits<1u> > >(Kokkos::Cuda const&, Kokkos::View<double**, "
"Kokkos::LayoutRight, Kokkos::CudaSpace, Kokkos::MemoryTraits<1u> > const&, "
f"double)), 1u>, void, unsigned long>::{slot} b 0000000000000018 "
"0000000000000008\n"
)


GEMM = _helper("solprop_native::device_dense_gemm")
GATHER = _helper("solprop_native::GeneralWorkspace::synthesise_retained")
NORMALIZE = _helper("KokkosFFT::Impl::normalize_impl")
OTHER = (
"solprop_native::Problem::validate() const T 0000000000001000 0000000000000040\n"
)


def _objects(tmp_path, monkeypatch, listings):
"""Return object files whose `nm` listings are `listings`, in order."""
by_path = {}
for index, listing in enumerate(listings):
path = tmp_path / f"unit{index}.cpp.o"
path.write_bytes(b"")
by_path[str(path)] = listing

def nm(command, **_):
return subprocess.CompletedProcess(command, 0, stdout=by_path[command[-1]])

monkeypatch.setattr(check_lambda_tags.subprocess, "run", nm)
return list(by_path)


def test_each_file_s_own_lambdas_pass(tmp_path, monkeypatch, capsys):
objects = _objects(tmp_path, monkeypatch, [GEMM + OTHER, GATHER])
assert check_lambda_tags.main(objects) == 0
assert (
"2 extended lambda helpers in 2 files, none shared" in capsys.readouterr().out
)


def test_a_lambda_two_files_hold_fails_and_is_named(tmp_path, monkeypatch, capsys):
# The radial route's device GEMM, instantiated alike in two files, called a null
# copy function from the file whose dispatch the link kept (#815).
objects = _objects(tmp_path, monkeypatch, [GEMM, GATHER + GEMM])
assert check_lambda_tags.main(objects) == 1
error = capsys.readouterr().err
assert "lambda in several files (unit0.cpp.o, unit1.cpp.o)" in error
assert "solprop_native::device_dense_gemm" in error


def test_an_allowed_library_lambda_is_reported_not_failed(
tmp_path, monkeypatch, capsys
):
objects = _objects(tmp_path, monkeypatch, [NORMALIZE + GEMM, NORMALIZE])
assert check_lambda_tags.main(objects) == 0
assert "allowed in several files" in capsys.readouterr().out


def test_the_other_slots_are_not_counted_twice(tmp_path, monkeypatch):
# A lambda's call and delete static members name the same tag as its copy member.
listing = GEMM + _helper("solprop_native::device_dense_gemm", "fp_caller")
objects = _objects(tmp_path, monkeypatch, [listing])
assert check_lambda_tags.main(objects) == 0


def test_finding_no_helper_fails(tmp_path, monkeypatch, capsys):
# A change in nvcc's naming must not make the check pass on nothing.
objects = _objects(tmp_path, monkeypatch, [OTHER])
assert check_lambda_tags.main(objects) == 1
assert "no extended lambda helper found" in capsys.readouterr().err


def test_naming_no_file_is_a_usage_error():
assert check_lambda_tags.main([]) == 2
Loading