Skip to content

Commit 779c361

Browse files
committed
fix(arc): keep the sysroot's pkg-config files visible after actions/setup-python
actions/setup-python exports PKG_CONFIG_PATH as its own directory alone, replacing the sysroot's entry the job-started hook put there, so a later step could no longer find the sysroot's .pc files. The hook now also sets PKG_CONFIG_LIBDIR to the sysroot's directory followed by pkg-config's default search path. Nothing rewrites that variable, and pkg-config searches it after PKG_CONFIG_PATH, so the sysroot stays reachable and the distribution's own files still resolve. It is skipped when the image has no pkg-config.
1 parent 187450b commit 779c361

5 files changed

Lines changed: 43 additions & 3 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -98,7 +98,7 @@ jobs:
9898
run: |
9999
set -euo pipefail
100100
sudo apt-get update -qq
101-
sudo apt-get install -y -qq ansible
101+
sudo apt-get install -y -qq ansible pkg-config
102102
cd ansible
103103
for playbook in ../playbooks/*.yml; do
104104
echo "Syntax-checking ${playbook}"

‎roles/github_runner_arc/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ A capability adds something to a profile's runner pods when they start, so the r
111111

112112
A tool-cache capability, `{image, path?}`, brings a tool. For each one the runner pod gets an init container, named `capability-<name>`, that copies the tool from the image into an `emptyDir` mounted on the runner container at `github_runner_arc_tool_cache_path`, and `RUNNER_TOOL_CACHE` is set to that path. The runner hands `RUNNER_TOOL_CACHE` to every step, and setup actions that consult the Actions tool cache before downloading, such as `actions/setup-node` and `terraform-linters/setup-tflint`, then find the tool there and skip the download, with no change to the workflow. The init containers run as the runner's user, so the copies belong to it and a setup action can still add another version beside them. `path`, `<tool>/<version>` optionally followed by `/<subdirectory>`, also puts the tool on `PATH` before the job's first step, for a tool no setup action looks up (Terraform, whose `hashicorp/setup-terraform` always downloads, or the AWS CLI).
113113

114-
A system-library capability, `{sysroot_image, env?}`, brings shared libraries, their headers and their programs, for what cannot live in the tool cache: an MPI implementation, a solver library, a database client library, anything a compiler or the dynamic loader has to find. For each one the runner pod gets an init container, also named `capability-<name>`, that copies the image's relocatable prefix into an `emptyDir` mounted on the runner container at `github_runner_arc_sysroot_path` (`/opt/sysroot` by default); every system-library capability of a profile shares that one prefix, the init containers running in the profile's order, so a later capability's file replaces an earlier one's of the same name. Before the job's first step the job-started hook prepends the prefix's directories to the search paths a build and the loader use, `bin` to `PATH`, `lib` to `LD_LIBRARY_PATH` and `LIBRARY_PATH`, `include` to `CPATH`, `lib/pkgconfig` to `PKG_CONFIG_PATH` and the prefix itself to `CMAKE_PREFIX_PATH`, and then sets each variable in `env`, a mapping of name to single-line string for anything else the libraries need, such as the relocation variable Open MPI reads. This takes the place of an `apt-get install` layer in a runner image.
114+
A system-library capability, `{sysroot_image, env?}`, brings shared libraries, their headers and their programs, for what cannot live in the tool cache: an MPI implementation, a solver library, a database client library, anything a compiler or the dynamic loader has to find. For each one the runner pod gets an init container, also named `capability-<name>`, that copies the image's relocatable prefix into an `emptyDir` mounted on the runner container at `github_runner_arc_sysroot_path` (`/opt/sysroot` by default); every system-library capability of a profile shares that one prefix, the init containers running in the profile's order, so a later capability's file replaces an earlier one's of the same name. Before the job's first step the job-started hook prepends the prefix's directories to the search paths a build and the loader use, `bin` to `PATH`, `lib` to `LD_LIBRARY_PATH` and `LIBRARY_PATH`, `include` to `CPATH`, `lib/pkgconfig` to `PKG_CONFIG_PATH` (and, with the `pkg-config` default search path after it, to `PKG_CONFIG_LIBDIR`, because `actions/setup-python` replaces `PKG_CONFIG_PATH` and leaves `PKG_CONFIG_LIBDIR` alone; this is skipped when the image has no `pkg-config`) and the prefix itself to `CMAKE_PREFIX_PATH`, and then sets each variable in `env`, a mapping of name to single-line string for anything else the libraries need, such as the relocation variable Open MPI reads. This takes the place of an `apt-get install` layer in a runner image.
115115

116116
A hook capability, `{job_started_hook}`, is a bash script the runner runs before each job's first step, for anything small: configuring git, writing a proxy setting to `GITHUB_ENV`, checking a mount.
117117

‎roles/github_runner_arc/files/job-started.sh‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/usr/bin/env bash
22
# Runner job-started hook (ACTIONS_RUNNER_HOOK_JOB_STARTED) for a profile with capabilities. The github_runner_arc role mounts it, with the profile's tool paths and hook scripts, from a ConfigMap at a fixed directory (see the role README's Capabilities section), and points the runner at it the same way the runner image points at its job-completed hook. The runner runs it as the runner user before the job's first step, with the job's default environment variables and its environment files (GITHUB_PATH, GITHUB_ENV).
33
#
4-
# First it puts each tool path on PATH: every line of tool-paths is <tool>/<version>, optionally followed by /<subdirectory>, and the directory added is <tool>/<version>/<arch>[/<subdirectory>] under RUNNER_TOOL_CACHE, where <arch> is the one architecture the capability's image put there (the one with a <arch>.complete marker beside it). Next it applies sysroot-env, the system-library capabilities' environment: a NAME+=directory line prepends the directory to NAME's current value (through GITHUB_PATH for PATH), and a NAME=value line sets NAME. Each is also exported here, so the scripts after it see it too. Then it runs each script in job-started.d in name order, each with bash -e, as the runner runs a hook.
4+
# First it puts each tool path on PATH: every line of tool-paths is <tool>/<version>, optionally followed by /<subdirectory>, and the directory added is <tool>/<version>/<arch>[/<subdirectory>] under RUNNER_TOOL_CACHE, where <arch> is the one architecture the capability's image put there (the one with a <arch>.complete marker beside it). Next it applies sysroot-env, the system-library capabilities' environment: a NAME+=directory line prepends the directory to NAME's current value (through GITHUB_PATH for PATH), and a NAME=value line sets NAME. Each is also exported here, so the scripts after it see it too. A PKG_CONFIG_PATH line also sets PKG_CONFIG_LIBDIR to that directory followed by pkg-config's own default search path, because actions/setup-python replaces PKG_CONFIG_PATH with its own directory instead of extending it, which would otherwise hide the sysroot's .pc files from every later step, whereas nothing rewrites PKG_CONFIG_LIBDIR and pkg-config searches it after PKG_CONFIG_PATH. Then it runs each script in job-started.d in name order, each with bash -e, as the runner runs a hook.
55
#
66
# Unlike the job-completed hook, a failure here fails the job: a job whose tools or setup did not arrive should stop before its first step rather than fail obscurely later.
77
set -euo pipefail
@@ -46,6 +46,19 @@ if [ -f "${hooks_dir}/sysroot-env" ]; then
4646
continue
4747
fi
4848
current="${!name:-}"
49+
if [ "$name" = PKG_CONFIG_PATH ]; then
50+
libdir_default="${PKG_CONFIG_LIBDIR:-}"
51+
# Without pkg-config there is no default to keep and nothing reads the variable; a pkg-config installed later in the job finds the sysroot through PKG_CONFIG_PATH alone.
52+
if [ -z "$libdir_default" ] && command -v pkg-config >/dev/null; then
53+
libdir_default="$(pkg-config --variable=pc_path pkg-config)"
54+
fi
55+
if [ -n "$libdir_default" ]; then
56+
libdir="${value}:${libdir_default}"
57+
echo "Setting PKG_CONFIG_LIBDIR"
58+
echo "PKG_CONFIG_LIBDIR=${libdir}" >>"$GITHUB_ENV"
59+
export PKG_CONFIG_LIBDIR="$libdir"
60+
fi
61+
fi
4962
value="${value}${current:+:${current}}"
5063
fi
5164
echo "Setting ${name}"

‎tests/system_libraries/checks/common.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ check_search_paths() {
3333
# Fails unless pkg-config resolves the module's library directory inside the sysroot, which shows its .pc file was made relocatable. The payloads' .pc files name their prefix as ${pcfiledir}/../.., which pkg-config does not simplify, so each -L directory is resolved before it is compared.
3434
check_pkg_config() {
3535
local module="$1" flags flag
36+
# A step such as actions/setup-python replaces PKG_CONFIG_PATH, so the module must resolve without it.
37+
PKG_CONFIG_PATH=/nonexistent/lib/pkgconfig pkg-config --exists "$module"
3638
flags="$(pkg-config --cflags --libs "$module")"
3739
echo "pkg-config ${module}: ${flags}"
3840
for flag in $flags; do

‎tests/unit/test_capabilities.py‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -434,6 +434,31 @@ def test_the_sysroot_env_prepends_search_paths_and_sets_variables_for_the_steps_
434434
self.assertEqual(github_env.read_text().splitlines(), ["LD_LIBRARY_PATH=/opt/sysroot/lib:/usr/local/lib", "CPATH=/opt/sysroot/include", "OPAL_PREFIX=/opt/sysroot", "EXAMPLE_FLAGS=a=b c"])
435435
self.assertEqual(self.log.read_text().splitlines(), ["/opt/sysroot/lib:/usr/local/lib|/opt/sysroot/include|/opt/sysroot|a=b c|/opt/sysroot/bin"])
436436

437+
def test_the_sysroot_pkg_config_files_survive_a_step_that_replaces_pkg_config_path(self) -> None:
438+
# actions/setup-python exports PKG_CONFIG_PATH as its own directory alone (src/find-python.ts), so the sysroot's .pc files must be reachable through a variable it leaves alone.
439+
self.assertIsNotNone(shutil.which("pkg-config"), "this test needs pkg-config on PATH")
440+
sysroot = self.dir / "sysroot"
441+
(sysroot / "lib" / "pkgconfig").mkdir(parents=True)
442+
(sysroot / "lib" / "pkgconfig" / "fixture.pc").write_text("Name: fixture\nDescription: fixture\nVersion: 1\n")
443+
python = self.dir / "python" / "lib" / "pkgconfig"
444+
python.mkdir(parents=True)
445+
github_env = self.dir / "github_env"
446+
github_env.touch()
447+
(self.hooks / "sysroot-env").write_text(f"PKG_CONFIG_PATH+={sysroot}/lib/pkgconfig\n")
448+
env = {key: value for key, value in os.environ.items() if key not in {"PKG_CONFIG_PATH", "PKG_CONFIG_LIBDIR"}}
449+
env.update({"RUNNER_TOOL_CACHE": str(self.cache), "GITHUB_PATH": str(self.github_path), "GITHUB_ENV": str(github_env)})
450+
completed = subprocess.run(["bash", "-e", str(self.hooks / "job-started.sh")], env=env, capture_output=True, text=True, stdin=subprocess.DEVNULL)
451+
self.assertEqual(completed.returncode, 0, completed.stderr)
452+
# The next steps start from the hook's GITHUB_ENV, then setup-python overwrites PKG_CONFIG_PATH.
453+
step_env = {key: value for key, value in env.items() if key != "GITHUB_ENV"}
454+
step_env.update(line.split("=", 1) for line in github_env.read_text().splitlines())
455+
self.assertEqual(subprocess.run(["pkg-config", "--exists", "fixture"], env=step_env).returncode, 0)
456+
step_env["PKG_CONFIG_PATH"] = str(python)
457+
self.assertEqual(subprocess.run(["pkg-config", "--exists", "fixture"], env=step_env).returncode, 0)
458+
# The distribution's own .pc files stay reachable, which a bare PKG_CONFIG_LIBDIR of the sysroot would lose.
459+
default_path = subprocess.run(["pkg-config", "--variable=pc_path", "pkg-config"], env=env, capture_output=True, text=True, check=True).stdout.strip()
460+
self.assertTrue(step_env["PKG_CONFIG_LIBDIR"].endswith(f":{default_path}"), step_env["PKG_CONFIG_LIBDIR"])
461+
437462
def test_a_sysroot_env_without_github_env_fails_the_job(self) -> None:
438463
(self.hooks / "sysroot-env").write_text("OPAL_PREFIX=/opt/sysroot\n")
439464
completed = self.run_hook()

0 commit comments

Comments
 (0)