Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ concurrency:
jobs:
test:
name: Build and test
runs-on: ubuntu-24.04
# Self-hosted Dockiva runner (scripts/ci-runner/README.md); fork PRs and
# the CI_RUNNER=hosted fallback use GitHub-hosted ubuntu-24.04.
runs-on: ${{ (github.event.pull_request.head.repo.fork || vars.CI_RUNNER == 'hosted') && 'ubuntu-24.04' || 'dockiva' }}
steps:
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
with:
Expand Down
37 changes: 37 additions & 0 deletions scripts/ci-runner/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Self-hosted GitHub Actions runner for Dockiva (linux/arm64).
# One container = one job: run-dockiva-runner.sh starts it with a single-use
# JIT config and removes it when the job ends.
FROM ubuntu:24.04

ARG RUNNER_VERSION=2.338.2
ARG RUNNER_SHA256=670adb2f268fe21dcbb58b3db0e0aa9d5b3bddfe2c651481336e561ff86de945

# ponytail: Dockiva <= 0.6.7 keeps container files on case-insensitive APFS, so
# linux-libc-dev's xt_CONNMARK.h / xt_connmark.h pairs collide and dpkg fails.
# Skip the uppercase netfilter target headers; drop this once Dockiva ships
# the case-sensitive engine store (engine af61a4e).
RUN printf '%s\n' \
'path-exclude=/usr/include/linux/netfilter/xt_[A-Z]*' \
'path-exclude=/usr/include/linux/netfilter_ipv4/ipt_[A-Z]*' \
'path-exclude=/usr/include/linux/netfilter_ipv6/ip6t_[A-Z]*' \
> /etc/dpkg/dpkg.cfg.d/dockiva-case-insensitive

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
build-essential ca-certificates curl git jq pkg-config sudo tar unzip \
xz-utils zstd \
&& rm -rf /var/lib/apt/lists/*

WORKDIR /home/runner
RUN curl -fsSL -o runner.tgz \
"https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-arm64-${RUNNER_VERSION}.tar.gz" \
&& echo "${RUNNER_SHA256} runner.tgz" | sha256sum -c - \
&& tar xzf runner.tgz && rm runner.tgz \
&& ./bin/installdependencies.sh \
&& rm -rf /var/lib/apt/lists/*

# Root, not a sudo user: on Dockiva Engine, once apt drops to _apt (which sets
# no_new_privs), later sudo calls in the container can't regain root. Each job
# gets a fresh container, so root here doesn't outlive the job.
ENV RUNNER_ALLOW_RUNASROOT=1
ENTRYPOINT ["./run.sh"]
78 changes: 78 additions & 0 deletions scripts/ci-runner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# Self-hosted CI runner on Dockiva

CI's `Build and test` job (`.github/workflows/ci.yml`) runs on a self-hosted
GitHub Actions runner with the label `dockiva`. The runner is GitHub's
`actions/runner` (linux/arm64) in a Dockiva container on a maintainer's Mac.

## How it works

- `Dockerfile` builds `dockiva-gh-runner:<version>`: Ubuntu 24.04 and the
pinned runner release (SHA-256 checked). The runner runs as root (`sudo`
still works for steps that call it); see the Dockerfile for why.
- `run-dockiva-runner.sh` loops on the Mac: it asks GitHub for a single-use
JIT runner config (`generate-jitconfig`) with the Mac's `gh` login, runs one
job in a fresh container (`docker run --rm`), then starts over. Each job gets
a clean container, and no GitHub credential lives in the image.
- A launchd agent keeps the loop running. When Dockiva is stopped, the loop
waits; jobs stay queued on GitHub until a runner comes back (GitHub fails a
job that waits more than 24 hours).

## Which runs use it

```yaml
runs-on: ${{ (github.event.pull_request.head.repo.fork || vars.CI_RUNNER == 'hosted') && 'ubuntu-24.04' || 'dockiva' }}
```

- Pull requests from forks always run on GitHub-hosted `ubuntu-24.04`, so
outside code never runs on the Mac.
- Pushes to `main` and pull requests from branches in this repository run on
Dockiva.
- Fallback: set the repository variable `CI_RUNNER` to `hosted` to send every
run to `ubuntu-24.04` (for example, when the Mac is away for a while):

```sh
gh variable set CI_RUNNER --body hosted --repo Tomnerb/Dockiva
gh variable delete CI_RUNNER --repo Tomnerb/Dockiva # back to Dockiva
```

Runs that are already queued keep their runner; re-run them after the switch.

`release.yml` stays on GitHub-hosted runners: its macOS and Windows jobs can't
run in Linux containers, and its protected environments sign and publish.

## Setup (once per Mac)

Needs Dockiva running (docker context `dockiva`) and `gh` logged in as a user
with admin access to the repository (`gh auth status`; scope `repo`).

```sh
cd scripts/ci-runner
sed "s|__SCRIPT__|$PWD/run-dockiva-runner.sh|" dev.dockiva.gh-runner.plist \
> ~/Library/LaunchAgents/dev.dockiva.gh-runner.plist
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/dev.dockiva.gh-runner.plist
```

The first start builds the image (about 2 minutes). Check it:

```sh
tail -f /tmp/dockiva-gh-runner.log
gh api repos/Tomnerb/Dockiva/actions/runners --jq '.runners[] | [.name, .status, .busy] | @tsv'
```

Stop it:

```sh
launchctl bootout gui/$(id -u)/dev.dockiva.gh-runner
```

Upgrade the runner: change `RUNNER_VERSION` and `RUNNER_SHA256` in the
Dockerfile (checksum from the release notes), and `IMAGE` in the script.

## Limits

- The job container gets 8 GiB (`MEMORY` in the script; Dockiva's default is
2 GiB, too little for a cold cgo `go test ./...`).
- One runner, one job at a time: concurrent runs queue.
- The runner is arm64; GitHub-hosted `ubuntu-24.04` is amd64.
- Repository settings keep "Require approval for first-time contributors"
on, as a second guard for fork PRs.
14 changes: 14 additions & 0 deletions scripts/ci-runner/dev.dockiva.gh-runner.plist
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<!-- Installed by scripts/ci-runner/README.md; __SCRIPT__ is replaced with the script's absolute path. -->
<plist version="1.0">
<dict>
<key>Label</key><string>dev.dockiva.gh-runner</string>
<key>ProgramArguments</key>
<array><string>/bin/sh</string><string>__SCRIPT__</string></array>
<key>RunAtLoad</key><true/>
<key>KeepAlive</key><true/>
<key>StandardOutPath</key><string>/tmp/dockiva-gh-runner.log</string>
<key>StandardErrorPath</key><string>/tmp/dockiva-gh-runner.log</string>
</dict>
</plist>
32 changes: 32 additions & 0 deletions scripts/ci-runner/run-dockiva-runner.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/sh
# Keeps one ephemeral GitHub Actions runner (label "dockiva") available in a
# Dockiva container. Each loop: mint a single-use JIT config with the host's
# `gh` login, run one job in a fresh container, remove the container.
# No GitHub credential is stored in the image or container.
set -u

REPO=${REPO:-Tomnerb/Dockiva}
CONTEXT=${DOCKER_CONTEXT:-dockiva}
IMAGE=${IMAGE:-dockiva-gh-runner:2.338.2}
# Dockiva gives containers 2 GiB by default; a cold cgo `go test ./...` needs more.
MEMORY=${MEMORY:-8g}
HERE=$(cd "$(dirname "$0")" && pwd)
export PATH="/opt/homebrew/bin:/usr/local/bin:$PATH"

docker_() { docker --context "$CONTEXT" "$@"; }

while :; do
# Dockiva not running: wait for it. Queued jobs wait on GitHub meanwhile.
if ! docker_ info >/dev/null 2>&1; then sleep 30; continue; fi
if ! docker_ image inspect "$IMAGE" >/dev/null 2>&1; then
docker_ build -t "$IMAGE" "$HERE" || { sleep 60; continue; }
fi

name="dockiva-$(hostname -s)-$(date +%s)"
jit=$(gh api -X POST "repos/$REPO/actions/runners/generate-jitconfig" \
-f name="$name" -F runner_group_id=1 -f work_folder=_work \
-f 'labels[]=dockiva' --jq .encoded_jit_config) || { sleep 60; continue; }

echo "$(date '+%F %T') $name waiting for a job"
docker_ run --rm -m "$MEMORY" --name "$name" "$IMAGE" --jitconfig "$jit" || sleep 10
done