Skip to content

Commit b8bf03c

Browse files
committed
feat(tool-images): add GLPK and MariaDB Connector/C system-library images
Jobs that build against GLPK or a MySQL client library had no capability to supply them on the stock runner image. Both are now reference system-library images following the sysroot payload contract, built on the runner image's Ubuntu release. GLPK uses its own arithmetic rather than GMP and ships no pkg-config file, as the distribution's development package does not either, so a build finds it through the CPATH and LIBRARY_PATH the hook sets. MariaDB Connector/C stands in for the distribution's MySQL client development package: libmariadb.pc, the libmysqlclient names, and mariadb_config with a mysql_config copy, both of which find their prefix from their own path. Its authentication plugins are built into the library, since it would load them from an absolute directory a relocated payload cannot provide; the build fails if one is left to load. The relocation suite checks both from /opt/sysroot: their programs in the stock image, and C programs compiled against them in the build runner image, including the plugins MySQL 8 servers ask for.
1 parent 7aa3ca4 commit b8bf03c

9 files changed

Lines changed: 219 additions & 15 deletions

File tree

‎.github/workflows/publish-tool-images.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ jobs:
3636
- {name: openmpi, kind: sysroot}
3737
- {name: highs, kind: sysroot}
3838
- {name: libpq, kind: sysroot}
39+
- {name: glpk, kind: sysroot}
40+
- {name: libmariadb, kind: sysroot}
3941
- {name: build, kind: runner}
4042
platform:
4143
- {arch: amd64, runner: ubuntu-latest}
@@ -130,6 +132,8 @@ jobs:
130132
- {name: openmpi, kind: sysroot}
131133
- {name: highs, kind: sysroot}
132134
- {name: libpq, kind: sysroot}
135+
- {name: glpk, kind: sysroot}
136+
- {name: libmariadb, kind: sysroot}
133137
- {name: build, kind: runner}
134138
steps:
135139
- uses: actions/checkout@v7

‎.github/workflows/system-libraries.yml‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
11
name: System libraries
22

3-
# Builds each reference system-library capability image in tool-images/ (Open MPI, HiGHS, libpq) natively on amd64 and arm64 and proves it works relocated (tests/system_libraries/run.sh): its payload is copied to /opt/sysroot as the role's init container copies it, the role's own job-started hook applies the environment the role renders for it, and inside the stock actions-runner image the payload's programs run from there, and inside the reference build runner image (tool-images/build) a program compiled against the payload does (for Open MPI, a two-process mpirun). Only on changes that can affect them. Builds nothing it publishes.
3+
# Builds each reference system-library capability image in tool-images/ (Open MPI, HiGHS, libpq, GLPK, MariaDB Connector/C) natively on amd64 and arm64 and proves it works relocated (tests/system_libraries/run.sh): its payload is copied to /opt/sysroot as the role's init container copies it, the role's own job-started hook applies the environment the role renders for it, and inside the stock actions-runner image the payload's programs run from there, and inside the reference build runner image (tool-images/build) a program compiled against the payload does (for Open MPI, a two-process mpirun). Only on changes that can affect them. Builds nothing it publishes.
44

55
on:
66
pull_request:
77
paths:
88
- tool-images/openmpi/**
99
- tool-images/highs/**
1010
- tool-images/libpq/**
11+
- tool-images/glpk/**
12+
- tool-images/libmariadb/**
1113
- tool-images/build/**
1214
- tests/system_libraries/**
1315
- tests/capabilities/render.yml
@@ -32,7 +34,7 @@ jobs:
3234
strategy:
3335
fail-fast: false
3436
matrix:
35-
payload: [openmpi, highs, libpq]
37+
payload: [openmpi, highs, libpq, glpk, libmariadb]
3638
platform:
3739
- {arch: amd64, runner: ubuntu-latest}
3840
- {arch: arm64, runner: ubuntu-24.04-arm}

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ An Unraid host runs the same cluster role as any other Docker host (it ships Doc
4747

4848
### Capabilities
4949

50-
A profile can also add tools, libraries and setup to its runner pods when they start, rather than baking them into a runner image someone has to build, publish and patch: `capabilities` names entries of `github_runner_arc_capabilities`. A tool-cache capability is a public, pinned image holding a tool in the Actions tool-cache layout; an init container copies it into the runner's `RUNNER_TOOL_CACHE`, where `actions/setup-node`, `setup-tflint` and other setup actions that consult the cache find it and skip their download, and it can also be put on `PATH` for a tool no setup action looks up. A system-library capability is a public, pinned image holding a relocatable install prefix (shared libraries, headers, pkg-config files, programs) for what cannot live in the tool cache, such as an MPI implementation, a solver library or a database client library; an init container copies it into a sysroot (`/opt/sysroot` by default), and the job-started hook puts the sysroot on `PATH`, `LD_LIBRARY_PATH`, `LIBRARY_PATH`, `CPATH`, `PKG_CONFIG_PATH` and `CMAKE_PREFIX_PATH` and sets any relocation variable the library needs. A hook capability is a script run before each job's first step, through a job-started hook mounted from a ConfigMap. A profile with its own `runs_on_label` then works as a named runner, for example one with Terraform, TFLint and the AWS CLI, or one with Open MPI and a solver library, that workflows select with `runs-on`, all from one values file and the stock `actions-runner` image. `tool-images/` has Dockerfiles for reference tool images (Node.js, Terraform, TFLint, the AWS CLI) reference system-library images (Open MPI, HiGHS, libpq) and a reference build runner image (the stock runner image with a C toolchain, for jobs that compile against those libraries), which `.github/workflows/publish-tool-images.yml` publishes when started by hand or by a `tool-images-*` tag; `roles/github_runner_arc/README.md`'s Capabilities section has the tool-image and payload contracts, how each reference library is declared and proven to work from a relocated prefix, how to build an image of your own, and where capabilities apply.
50+
A profile can also add tools, libraries and setup to its runner pods when they start, rather than baking them into a runner image someone has to build, publish and patch: `capabilities` names entries of `github_runner_arc_capabilities`. A tool-cache capability is a public, pinned image holding a tool in the Actions tool-cache layout; an init container copies it into the runner's `RUNNER_TOOL_CACHE`, where `actions/setup-node`, `setup-tflint` and other setup actions that consult the cache find it and skip their download, and it can also be put on `PATH` for a tool no setup action looks up. A system-library capability is a public, pinned image holding a relocatable install prefix (shared libraries, headers, pkg-config files, programs) for what cannot live in the tool cache, such as an MPI implementation, a solver library or a database client library; an init container copies it into a sysroot (`/opt/sysroot` by default), and the job-started hook puts the sysroot on `PATH`, `LD_LIBRARY_PATH`, `LIBRARY_PATH`, `CPATH`, `PKG_CONFIG_PATH` and `CMAKE_PREFIX_PATH` and sets any relocation variable the library needs. A hook capability is a script run before each job's first step, through a job-started hook mounted from a ConfigMap. A profile with its own `runs_on_label` then works as a named runner, for example one with Terraform, TFLint and the AWS CLI, or one with Open MPI and a solver library, that workflows select with `runs-on`, all from one values file and the stock `actions-runner` image. `tool-images/` has Dockerfiles for reference tool images (Node.js, Terraform, TFLint, the AWS CLI) reference system-library images (Open MPI, HiGHS, GLPK, libpq, MariaDB Connector/C) and a reference build runner image (the stock runner image with a C toolchain, for jobs that compile against those libraries), which `.github/workflows/publish-tool-images.yml` publishes when started by hand or by a `tool-images-*` tag; `roles/github_runner_arc/README.md`'s Capabilities section has the tool-image and payload contracts, how each reference library is declared and proven to work from a relocated prefix, how to build an image of your own, and where capabilities apply.
5151

5252
## Build, test, and smoke-test
5353

@@ -63,7 +63,7 @@ A profile can also add tools, libraries and setup to its runner pods when they s
6363
kubectl get pods -n arc-runners-<org> # zero runner pods when idle is expected, not a bug
6464
```
6565
- **Confirm no disk accumulates across repeated runner jobs:** the Docker host's disk usage (inside the VM, on a host that runs Docker in one) must stay flat across repeated `test-arc-runner.yml` runs.
66-
- **Test capabilities:** `tests/capabilities/run.sh` builds the reference Node.js tool image and libpq system-library image, renders a profile with capabilities through the ARC role and the scale-set chart, and runs a pod from the chart's runner template in a throwaway kind cluster to check that the tool reaches the tool cache and `PATH` and the library reaches the sysroot and its search paths; `.github/workflows/capabilities.yml` runs it on pull requests. `tests/system_libraries/run.sh [openmpi|highs|libpq]...` proves each reference system-library image works relocated: it copies the payload to `/opt/sysroot` as the role's init container does, applies the role's job-started hook, and runs the payload's programs in the stock `actions-runner` image and a program compiled against it in the reference build runner image (`tool-images/build`, the stock image with `build-essential` and `pkg-config`), including a two-process `mpirun` for Open MPI; `.github/workflows/system-libraries.yml` runs it on amd64 and arm64 on pull requests. The images under `tool-images/` build with `docker buildx build --platform linux/arm64,linux/amd64 tool-images/<name>`; `.github/workflows/publish-tool-images.yml` publishes them (see Releases), and the release workflow does not.
66+
- **Test capabilities:** `tests/capabilities/run.sh` builds the reference Node.js tool image and libpq system-library image, renders a profile with capabilities through the ARC role and the scale-set chart, and runs a pod from the chart's runner template in a throwaway kind cluster to check that the tool reaches the tool cache and `PATH` and the library reaches the sysroot and its search paths; `.github/workflows/capabilities.yml` runs it on pull requests. `tests/system_libraries/run.sh [openmpi|highs|libpq|glpk|libmariadb]...` proves each reference system-library image works relocated: it copies the payload to `/opt/sysroot` as the role's init container does, applies the role's job-started hook, and runs the payload's programs in the stock `actions-runner` image and a program compiled against it in the reference build runner image (`tool-images/build`, the stock image with `build-essential` and `pkg-config`), including a two-process `mpirun` for Open MPI; `.github/workflows/system-libraries.yml` runs it on amd64 and arm64 on pull requests. The images under `tool-images/` build with `docker buildx build --platform linux/arm64,linux/amd64 tool-images/<name>`; `.github/workflows/publish-tool-images.yml` publishes them (see Releases), and the release workflow does not.
6767
- **Lint:** `.github/workflows/ci.yml` runs Shellcheck, actionlint, yamllint, hadolint, an Ansible syntax check, ansible-lint (at its `production` profile), and a collection build check on every push and pull request, plus commitlint (on every commit in a pull request, and on the pull request's own title) on pull requests specifically - all gated by one `required-checks` job so a repository ruleset only ever has to require that one check regardless of how many lint jobs exist. hadolint's warning-level pinning advice is visible but non-blocking (`--failure-threshold error`): this repo deliberately floats several images/packages on `latest` rather than pinning. yamllint's config lives in `.yamllint`, relaxing the three default rules that conflict with this project's own conventions (long single-line comments, no leading `---`, GitHub Actions' `on:` key).
6868
- **Commit messages:** `.husky/commit-msg` runs commitlint locally (after `npm ci`), the same check CI's commitlint job applies, so a message semantic-release would mishandle is rejected before it is pushed. The machine-wide hooks dispatcher hands off to it; without one, `git config core.hooksPath .husky` enables it.
6969

0 commit comments

Comments
 (0)