Skip to content

Commit 224dfac

Browse files
committed
test(mesh): cover both sides of the faulty Compose range
A compose_faulty scenario runs the cluster role under Compose 2.38.2 and asserts it refuses before writing to the host. The hosted scenario also runs on 2.39.0, the first fixed release, so the rerun check covers it.
1 parent b99ae54 commit 224dfac

3 files changed

Lines changed: 47 additions & 9 deletions

File tree

‎.github/workflows/litestream.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ jobs:
3535
- uses: actions/checkout@v7
3636

3737
- name: Install the pinned Docker Compose
38-
# Always the pinned version, not whatever the runner has: ubuntu-latest's own Compose recreated every node container when the role ran a second time, which the pinned version does not. The user plugin directory takes precedence over the system one.
38+
# Always the pinned version, not whatever the runner has: ubuntu-latest has shipped Compose releases the cluster role refuses (see its tasks/main.yml). The user plugin directory takes precedence over the system one.
3939
run: |
4040
set -euo pipefail
4141
docker version --format 'Docker Engine {{.Server.Version}}'

‎.github/workflows/mesh-integration.yml‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
name: Mesh integration
22

3-
# Brings up three k3s nodes in Docker and joins them through Headscale with the cluster role (tests/mesh/run.sh), once per scenario: headscale_hosted, headscale_in_cluster, and the headscale_existing policy check. Only on changes that can affect how nodes join the mesh.
3+
# Brings up three k3s nodes in Docker and joins them through Headscale with the cluster role (tests/mesh/run.sh), once per scenario: headscale_hosted, headscale_in_cluster, and the headscale_existing policy check, plus the role's refusal of a Docker Compose release that recreates the containers it has just built. Only on changes that can affect how nodes join the mesh.
44
#
55
# Runs on GitHub-hosted ubuntu-latest, whose Docker daemon runs the privileged containers the test needs (k3s, and Tailscale inside it). The work directory sits under runner.temp, which the job always has write access to.
66

@@ -25,20 +25,26 @@ permissions:
2525

2626
jobs:
2727
mesh:
28-
name: Mesh (${{ matrix.scenario }})
28+
name: Mesh (${{ matrix.scenario }}, Compose ${{ matrix.compose }})
2929
runs-on: ubuntu-latest
3030
timeout-minutes: 45
3131
strategy:
3232
fail-fast: false
3333
matrix:
34-
scenario: [existing, hosted, in_cluster]
35-
env:
36-
COMPOSE_VERSION: v5.5.1
34+
# The hosted scenario also runs on Compose 2.39.0, the first release after the range the cluster role refuses (docker/compose#13047), so its rerun check proves that release keeps every node running. compose_faulty runs on 2.38.2, the last release in that range, and checks the role refuses it.
35+
include:
36+
- {scenario: existing, compose: v5.5.1}
37+
- {scenario: hosted, compose: v5.5.1}
38+
- {scenario: in_cluster, compose: v5.5.1}
39+
- {scenario: hosted, compose: v2.39.0}
40+
- {scenario: compose_faulty, compose: v2.38.2}
3741
steps:
3842
- uses: actions/checkout@v7
3943

40-
- name: Install the pinned Docker Compose
41-
# Always the pinned version, not whatever the runner has: ubuntu-latest's own Compose recreated every node container when the role ran a second time, which the pinned version does not. The user plugin directory takes precedence over the system one.
44+
- name: Install Docker Compose ${{ matrix.compose }}
45+
# Always the matrix's version, not whatever the runner has, so a runner image update never changes what a scenario tests. The user plugin directory takes precedence over the system one.
46+
env:
47+
COMPOSE_VERSION: ${{ matrix.compose }}
4248
run: |
4349
set -euo pipefail
4450
docker version --format 'Docker Engine {{.Server.Version}}'

‎tests/mesh/run.sh‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/usr/bin/env bash
2-
# Mesh integration test for the github_runner_cluster role: stands up three k3s nodes in Docker on one machine, each in its own copy of the Compose project, and has the role join them through Headscale. Scenarios: hosted headscale_hosted: the role runs Headscale under Compose beside the first node. in_cluster headscale_in_cluster: the role runs Headscale inside the cluster, on the bootstrap server. existing headscale_existing: the policy check fails, with the entry to add, against a server whose policy lacks autoApprovers, and passes once it has one. Every scenario uses automatic server selection (three hosts, no overrides, so three servers). The cluster scenarios assert that etcd has three members, that every node is Ready, and that pods on different nodes reach each other over the mesh.
2+
# Mesh integration test for the github_runner_cluster role: stands up three k3s nodes in Docker on one machine, each in its own copy of the Compose project, and has the role join them through Headscale. Scenarios: hosted headscale_hosted: the role runs Headscale under Compose beside the first node. in_cluster headscale_in_cluster: the role runs Headscale inside the cluster, on the bootstrap server. existing headscale_existing: the policy check fails, with the entry to add, against a server whose policy lacks autoApprovers, and passes once it has one. Every scenario uses automatic server selection (three hosts, no overrides, so three servers). compose_faulty the role refuses a Docker Compose release that recreates the containers it has just built, before writing anything (run it under such a release; it is not in the default set). The cluster scenarios assert that etcd has three members, that every node is Ready, and that pods on different nodes reach each other over the mesh.
33
#
44
# Usage: tests/mesh/run.sh <scenario>... (default: all three). Needs Docker with Compose, and Python 3 with ansible-core (ANSIBLE_PLAYBOOK overrides which ansible-playbook runs). Creates only Docker objects named grtest-*, and removes them again on exit unless GRTEST_KEEP=1.
55
set -euo pipefail
@@ -274,6 +274,37 @@ scenario_cluster() {
274274
fi
275275
}
276276

277+
# The role must refuse a Docker Compose release that recreates the containers it has just built (see the check in the cluster role's tasks/main.yml) before it writes anything to the host. The workflow installs such a release for this scenario; every other scenario runs under a good one and so also shows the check lets that through.
278+
scenario_compose_faulty() {
279+
local output node
280+
reset_environment
281+
node="$(node_name 1)"
282+
mkdir -p "${work}/${node}"
283+
cat > "${work}/inventory.yml" <<EOF
284+
all:
285+
vars:
286+
ansible_connection: local
287+
ansible_python_interpreter: $(command -v python3)
288+
children:
289+
github_runner_cluster:
290+
vars:
291+
github_runner_cluster_k3s_token: ${k3s_token}
292+
hosts:
293+
${node}:
294+
github_runner_cluster_dir: ${work}/${node}
295+
github_runner_cluster_compose_project: ${node}
296+
EOF
297+
log "The role must refuse Docker Compose $(docker compose version --short) before touching the host"
298+
if output=$(run_role 2>&1); then
299+
echo "$output"; fail "the role ran with a Compose release it should refuse"
300+
fi
301+
echo "$output" | grep -F "Fail if this host's Docker Compose recreates the containers it has just built" >/dev/null || { echo "$output"; fail "the role failed, but not at the Compose version check"; }
302+
echo "$output" | grep -F 'Upgrade the Docker Compose plugin on this host' >/dev/null || { echo "$output"; fail "the failure did not say to upgrade Compose"; }
303+
[ -z "$(ls -A "${work}/${node}")" ] || fail "the role wrote to the cluster directory before refusing: $(ls -A "${work}/${node}")"
304+
echo "$output" | grep -F 'Upgrade the Docker Compose plugin' | head -n 1
305+
log "It did"
306+
}
307+
277308
scenario_existing() {
278309
reset_environment
279310
local fixtures="${repo_root}/tests/mesh/fixtures/headscale-existing" api_url api_key output
@@ -322,6 +353,7 @@ for scenario in "${scenarios[@]}"; do
322353
case "$scenario" in
323354
hosted | in_cluster) scenario_cluster "$scenario" ;;
324355
existing) scenario_existing ;;
356+
compose_faulty) scenario_compose_faulty ;;
325357
# Re-checks a cluster a previous run kept with GRTEST_KEEP=1 and the same GRTEST_WORK, without rebuilding it.
326358
assert) assert_cluster ;;
327359
*) fail "unknown scenario ${scenario}" ;;

0 commit comments

Comments
 (0)