Skip to content

Strip /usr/include from apt pybind11's interface include dirs#121

Open
whitesscott wants to merge 1 commit into
NVIDIA:mainfrom
whitesscott:fix-pybind11-usr-include-leak
Open

Strip /usr/include from apt pybind11's interface include dirs#121
whitesscott wants to merge 1 commit into
NVIDIA:mainfrom
whitesscott:fix-pybind11-usr-include-leak

Conversation

@whitesscott

Copy link
Copy Markdown

Summary

  • On Debian/Ubuntu hosts, the apt pybind11-dev CMake config sets pybind11::pybind11_headers INTERFACE_INCLUDE_DIRECTORIES to ${_IMPORT_PREFIX}/include, where _IMPORT_PREFIX=/usr. That leaks /usr/include into every target using pybind11_add_module — in this repo, _edgellm_runtime.
  • nvcc translates it to -isystem /usr/include on the host g++ command line, which reorders g++'s system include chain and puts /usr/include before /usr/include/c++/<ver>. libstdc++'s <cmath> then fails to satisfy #include_next <math.h> and the build dies during device_link_stub.cu.o:
    /usr/include/c++/13/cmath: fatal error: math.h: No such file or directory
       #include_next <math.h>
    
  • Fix: after find_package(pybind11 CONFIG REQUIRED), remove the literal /usr/include entry from pybind11::pybind11_headers's interface include dirs. The real pybind11 headers still resolve; only the bogus root include is dropped.
  • No-op for pip-installed pybind11 (its _IMPORT_PREFIX is under the venv's site-packages, not /usr).

Reproduction

  • Host: Ubuntu 24.04 aarch64 (Jetson Thor), gcc 13.3.0, CUDA 13.2.
  • Apt package: pybind11-dev 2.11.1-2 at /usr/lib/cmake/pybind11/.
  • Configure:
    cmake .. \
        -DTRT_PACKAGE_DIR=/usr \
        -DCUDA_CTK_VERSION=13.2 \
        -DCMAKE_TOOLCHAIN_FILE=cmake/aarch64_linux_toolchain.cmake \
        -DEMBEDDED_TARGET=jetson-thor \
        -DENABLE_CUTE_DSL=ALL \
        -DENABLE_NVTX_PROFILING=ON \
        -DBUILD_PYTHON_BINDINGS=ON
    
  • Without this patch: build fails compiling experimental/pybind/device_link_stub.cu with the <cmath> / math.h error above.
  • With this patch: experimental/pybind/CMakeFiles/_edgellm_runtime.dir/includes_CUDA.rsp no longer contains a bare -isystem /usr/include, and _edgellm_runtime.cpython-312-aarch64-linux-gnu.so links successfully.
  • Also verified that pip-installed pybind11 3.0.4 (via -Dpybind11_DIR="$(python -m pybind11 --cmakedir)") continues to work with this patch — the workaround is guarded on the presence of /usr/include in the interface list.

Test plan

  • Configure + build with apt pybind11-dev on Ubuntu 24.04 aarch64 (Jetson Thor) — _edgellm_runtime target builds.
  • Configure + build with pip-installed pybind11 via -Dpybind11_DIR=$(python -m pybind11 --cmakedir) — still builds, no regression.
  • python -c "import _edgellm_runtime" succeeds against the resulting .so.
  • Spot-check that no other target consuming pybind11::* targets is affected on x86_64.

Related

🤖 Generated with Claude Code

The Debian/Ubuntu `pybind11-dev` package ships a CMake config in which
`pybind11::pybind11_headers` has

    INTERFACE_INCLUDE_DIRECTORIES = ${_IMPORT_PREFIX}/include

and, for the apt install, `_IMPORT_PREFIX` resolves to `/usr`, so
`/usr/include` is propagated to every target that consumes pybind11 via
`pybind11_add_module` (in this repo, `_edgellm_runtime`).

nvcc translates that into `-isystem /usr/include` on the host g++ command
line for each CUDA source in the target.  g++ then reorders its system
include chain, placing `/usr/include` before the libstdc++ header directory
`/usr/include/c++/<ver>`.  libstdc++'s `<cmath>` relies on
`#include_next <math.h>` to reach the C library header; with `/usr/include`
moved to the front, nothing after `<cmath>` in the search order still
contains `math.h`, and the build fails during device_link_stub.cu.o with:

    /usr/include/c++/13/cmath: fatal error: math.h: No such file or directory
       #include_next <math.h>

Reproduced on Ubuntu 24.04 aarch64 (Jetson Thor) with `pybind11-dev`
2.11.1 and CUDA 13.2 + gcc 13.3.  Pip-installed pybind11 is unaffected
because its `_IMPORT_PREFIX` points at the venv's site-packages, not `/usr`.

Fix: after `find_package(pybind11 CONFIG REQUIRED)`, strip `/usr/include`
from the `pybind11::pybind11_headers` target's INTERFACE_INCLUDE_DIRECTORIES.
The real pybind11 headers live in a subdirectory of `_IMPORT_PREFIX/include`
(specifically `pybind11/`) and are still exposed via the include-path
resolution rules; only the bogus root include is removed.  No-op for pip
installs of pybind11.

Verified by:
 - `rm -rf build && cmake ..` (no -Dpybind11_DIR override, so apt pybind11
   at /usr/lib/cmake/pybind11 is picked): configure succeeds and the
   generated experimental/pybind/CMakeFiles/_edgellm_runtime.dir/includes_CUDA.rsp
   no longer contains a bare `-isystem /usr/include`.
 - `make -j _edgellm_runtime`: device_link_stub.cu.o now compiles and the
   `_edgellm_runtime.cpython-312-aarch64-linux-gnu.so` links successfully.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@whitesscott whitesscott requested a review from a team July 2, 2026 02:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant