Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .github/actions/gnome-e2e/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@
# - Runner must have KVM enabled (ubuntu-latest on GHA has it)
# - OCI image must be a bootc/ostree image (uses bootc install to-disk)
# - gnome-ponytail-daemon must be baked into the OCI image
# (suite: extensions checks this and other prerequisites before boot via
# scripts/guest_preflight.py and records results/guest-provenance.json)
#
# Usage:
# - uses: projectbluefin/testsuite/.github/actions/gnome-e2e@main
Expand Down Expand Up @@ -131,6 +133,7 @@ runs:
shell: bash
env:
GNOME_E2E_IMAGE: ${{ inputs.image }}
GNOME_E2E_SUITE: ${{ inputs.suite }}
run: |
fallocate -l 30G disk.raw

Expand Down Expand Up @@ -185,6 +188,20 @@ runs:
echo "GNOME_E2E_ROOT_UUID=${ROOT_UUID}" >> "$GITHUB_ENV"
echo "GNOME_E2E_KVER=${KVER}" >> "$GITHUB_ENV"

# Extension-validation lane (#908): record which guest is under test and
# fail closed before boot when the image lacks a lane prerequisite
# (stock GNOME OS ships no gnome-ponytail-daemon). Cleanup below still
# runs; the step fails after it.
PREFLIGHT_RC=0
if [[ "${GNOME_E2E_SUITE}" == "extensions" ]]; then
IMAGE_DIGEST=$(sudo podman image inspect --format '{{.Digest}}' "${GNOME_E2E_IMAGE}" 2>/dev/null || true)
mkdir -p results
sudo python3 _testsuite/scripts/guest_preflight.py "${D}" \
--image-ref "${GNOME_E2E_IMAGE}" --image-digest "${IMAGE_DIGEST}" \
--output results/guest-provenance.json || PREFLIGHT_RC=$?
sudo chown -R "$(id -u):$(id -g)" results
fi

EFI_PART="${LOOP}p2"
sudo mkdir -p /mnt/efi
sudo mount "${EFI_PART}" /mnt/efi
Expand Down Expand Up @@ -243,6 +260,10 @@ runs:
sudo podman rmi "${GNOME_E2E_IMAGE}" 2>/dev/null || true
ls -lh disk.raw vmlinuz initramfs.img
df -h .
if [[ ${PREFLIGHT_RC} -ne 0 ]]; then
echo "::error::Guest preflight failed (rc=${PREFLIGHT_RC}); see results/guest-provenance.json"
exit 1
fi

# ── Boot VM ───────────────────────────────────────────────────────────────
- name: Boot VM with QEMU + KVM
Expand Down
6 changes: 3 additions & 3 deletions docs/skills/ci-ops/e2e-workflow/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,8 +103,7 @@ protecting anything.

## Job-summary scenario counts come from `scripts/e2e_summary.py`

Never derive `passed` by subtraction. `passed = total - failed - skipped` is wrong
on two counts:
Never derive `passed` by subtraction. `passed = total - failed - skipped` is wrong on two counts:

- `results.json` `elements` include **`background`** entries alongside `scenario`
ones, so `len(elements)` overstates the scenario total.
Expand Down Expand Up @@ -155,7 +154,7 @@ the checkout's namespace package whatever `sys.path` says, forcing the fail-clos
path. That load catches bare `Exception` on purpose — `exec_module` runs the module's top level, so a corrupted
copy raises `SyntaxError` or anything else, and this reporting-only step must exit 0 rather than fail the job.

**Gate extension-validation run** follows `Summarise results` for `suite: extensions`: it loads the sibling results helper by file path and requires at least one passing scenario with no errors, while emitting a JSON report for either verdict.
**Gate extension-validation run** follows `Summarise results` for `suite: extensions`: it loads the sibling results helper by file path and requires at least one passing scenario with no errors, while emitting a JSON report for either verdict. Earlier, before boot, the install step runs `scripts/guest_preflight.py` over the mounted deployment: it writes `results/guest-provenance.json` (image ref, resolved digest, guest `os-release`) and fails closed when a prerequisite such as `gnome-ponytail-daemon` is missing.

## Sparse checkout is non-cone — every script must be listed explicitly

Expand Down Expand Up @@ -496,4 +495,5 @@ Load these when you hit the specific topic:
- [Installer suite assertions, and why an env-var gate made one unreachable.](references/installer-suite.md)
- [Migration inputs `migration-target` / `extra-tags` and the startup_failure they cause when missing.](references/migration-inputs.md)
- [GNOME OS guest lane for the developer extension-validation service and its mandatory gate.](references/gnome-extensions-validation.md)
- [GNOME OS image provenance (2-week tag expiry), verified guest inventory, and the extensions pre-boot preflight](references/gnomeos-guest-provenance.md)
- [Which units the fast e2e lane masks, why, and which still need a startup-qualification lane.](references/startup-service-inventory.md)
Original file line number Diff line number Diff line change
Expand Up @@ -19,35 +19,28 @@ and exercised with a qecore/Behave input+AT-SPI scenario.
> **No run has completed yet.** Do **not** claim GNOME OS extension validation as done until a
> green run exists and its evidence is attached.

## Pinned provenance (do not float)
## Provenance, image inventory and pre-boot preflight

Base image `quay.io/gnome_infrastructure/gnome-build-meta:gnomeos-nightly`. The lane consumes the
**digest reference**, never the tag:
`gnomeos-*` tags and digests expire two weeks after push, so the lane resolves a digest per run
and the action records it, with the guest `os-release` identity, in `results/guest-provenance.json`.
Stock GNOME OS nightly carries every lane prerequisite except `gnome-ponytail-daemon`, and
`scripts/guest_preflight.py` fails `suite: extensions` closed on that before boot. Evidence,
commands and the full inventory: [gnomeos-guest-provenance.md](gnomeos-guest-provenance.md).

| Tag | Digest |
|---|---|
| `gnomeos-nightly` (pinned for the lane) | `sha256:57eeef917d057e37d8d5824fab195ebf3b0ad49fc1ca191b7bc02b82418ed981` |
| `gnomeos-51` / `gnomeos-50` / `gnomeos-latest` | reference only |

Resolved 2026-09-30; proves **existence**, not a successful boot with this action's provisioner.
Re-derive after any change with `skopeo inspect docker://…:gnomeos-nightly | jq -r '.Digest'`.

## Guest inventory
## Provisioned surface

The lane reuses the `gnome-e2e` provisioner (bootc/ostree → `bootc install to-disk` → QEMU/KVM
boot → GDM autologin → ponytail + qecore-headless), pointed at `gnomeos-nightly` instead of a
Bluefin/Dakota image. Provisioned surface:
boot → GDM autologin → ponytail + qecore-headless), pointed at a GNOME OS digest instead of a
Bluefin/Dakota image:

- GNOME OS nightly (pinned digest) as a bootc/ostree deployment, with kernel + initramfs
extracted from the deployment for direct QEMU boot
- GNOME OS as a bootc/ostree deployment, with kernel + initramfs extracted from the deployment
for direct QEMU boot
- GDM autologin for the `bluefin-test` user; passwordless sudo for test automation
- **real** GNOME Wayland session (Mutter) + AT-SPI bus (at-spi2-core)
- `gnome-ponytail-daemon` (virtual input) driven by **qecore-headless** — **absent from stock
`gnomeos-nightly`**; required baked into the OCI image (`action.yml:14`) and no overlay
supplying it exists yet, so it is an input to the lane, not a lane output
GNOME OS**; the action expects it in the image (`action.yml` header) and no overlay supplying
it exists yet, so it is an input to the lane, not a lane output
- SSH (port 2222 → 22) as `bluefin-test` for the harness to drive the guest
- No Fedora dnf/RPM assumption — GNOME OS has no package manager; tooling must ship in a test
overlay/sysext

## The four hive extensions

Expand Down Expand Up @@ -77,21 +70,24 @@ behaviour features plus a cross-cutting `extension_lifecycle.feature`.

`gnome-e2e` is not a boot-only action: the same invocation provisions the guest **and** runs
`tests/<suite>/features/` inside it under `qecore-headless` (`action.yml:384`). `suite` must
therefore be set — it defaults to `smoke` (`action.yml:38-40`), which would boot GNOME OS and run
the **Bluefin** smoke suite, testing the wrong thing. Pass the **pinned digest**, not a tag: the
therefore be set — it defaults to `smoke` (`action.yml:40-42`), which would boot GNOME OS and run
the **Bluefin** smoke suite, testing the wrong thing. Pass a **digest resolved at dispatch time**, not a tag: the
action hands `inputs.image` verbatim to `podman pull` / `bootc install`.

```yaml
- uses: ./.github/actions/gnome-e2e
with:
# requires an image that carries gnome-ponytail-daemon (blocker 1 above)
image: quay.io/gnome_infrastructure/gnome-build-meta@sha256:57eeef917d057e37d8d5824fab195ebf3b0ad49fc1ca191b7bc02b82418ed981
# requires an image that carries gnome-ponytail-daemon (blocker 1 above);
# digest from a not-yet-written `resolve` job (sketch: gnomeos-guest-provenance.md); expires in 2 weeks
image: quay.io/gnome_infrastructure/gnome-build-meta@${{ needs.resolve.outputs.digest }}
suite: extensions # uses tests/extensions/ (landed with #914)
```

The action runs `bootc install to-disk`, extracts kernel/initramfs, boots a KVM QEMU VM, waits for
The action runs `bootc install to-disk`, runs the guest preflight (fails here today on stock GNOME
OS: no `gnome-ponytail-daemon`), extracts kernel/initramfs, boots a KVM QEMU VM, waits for
SSH then a live GNOME session, copies `tests/extensions` (plus `tests/shared`) into the guest, runs
behave there, and writes `results/` + `vm-serial.log` into the workspace.
behave there, and writes `results/` (including `guest-provenance.json`) + `vm-serial.log` into
the workspace.

### 2. Confirm a real session (canary)

Expand Down Expand Up @@ -176,25 +172,29 @@ and must never reach the host:
- **No privileged host mounts** — the VM boots from a raw disk written by `bootc install`; the
host filesystem is not writable from the guest.
- **Credentials are guest-local and disposable** — the harness SSH key (`authorized_keys` in the
`bluefin-test` home, `action.yml:215`) and the `bluefin-test` autologin let the harness drive
the guest. Extension code runs in `gnome-shell` **as `bluefin-test`**, who also has
`NOPASSWD:ALL` sudo (`action.yml:224-226`), so extension code **can** read that key and escalate
**inside the guest**. Accepted: the key is generated per run, authorises nothing but this
throwaway VM, and grants no reach back to the host. Never reuse a long-lived or host-valid
credential here.
`bluefin-test` home, `action.yml:215`) and the `bluefin-test` autologin let the harness drive the
guest. Extension code runs in `gnome-shell` **as `bluefin-test`**, who also has `NOPASSWD:ALL`
sudo (`action.yml:224-226`), so extension code **can** read that key and escalate **inside the
guest**. Accepted: the key is generated per run, authorises nothing but this throwaway VM, and
grants no reach back to the host. Never reuse a long-lived or host-valid credential here.

A host socket, key, or mount exposed to the guest desktop is a regression: stop and file an issue.

## Status

- [x] Official GNOME OS nightly pinned (digest above)
- [x] Guest inventory + four hive extensions documented
- [x] Provenance contract: tested build recorded per run in
`results/guest-provenance.json` (static pins expire after 2 weeks)
- [ ] Provenance contract: resolve job that pins the digest per run
(sketched in `gnomeos-guest-provenance.md`; no workflow does this yet)
- [x] Guest inventory verified against the image layers; four hive extensions documented
- [x] Pre-boot preflight (`scripts/guest_preflight.py`) fails closed on missing prerequisites
- [x] Mandatory service gate implemented and unit-tested (`scripts/extension_validation.py`)
- [x] Security posture recorded: guest-local, disposable credentials; no host reach
- [x] Gate wired into the lane: `gnome-e2e` runs `scripts/extension_validation.py` as a
mandatory step when `suite == 'extensions'` (#991). No repo workflow/recipe calls the
action with `suite: extensions` yet — a consumer runs it directly.
- [ ] Repeatable boot + load + run commands validated in the lab (**unverified**;
tests/extensions landed with #914; blocked on a `gnome-ponytail-daemon` overlay)
tests/extensions landed with #914; blocked on a `gnome-ponytail-daemon` overlay, which the
preflight now reports by name)
- [ ] At least one green extension-validation run captured and attached
- [ ] Lane closed
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
---
name: gnomeos-guest-provenance
description: "GNOME OS image provenance (2-week tag expiry), verified guest inventory, and the extensions pre-boot preflight"
metadata:
type: reference
audience: agents
maturity: draft
---
# GNOME OS guest provenance and preflight (#908)

Companion to [gnome-extensions-validation.md](gnome-extensions-validation.md). Everything here
was checked against the published image's layers; **none of it proves a boot.**

## Provenance: pin per run, not in this file

Base image `quay.io/gnome_infrastructure/gnome-build-meta`, tag `gnomeos-nightly` (or
`gnomeos-51` / `gnomeos-50` for a stable branch). **Every `gnomeos-*` tag expires two weeks after
it is pushed** — the image config carries the label `quay.expires-after: 2w`, and the Quay tag API
shows each `gnomeos-nightly` revision with a 14-day `end_ts`. A digest copied into a doc or
workflow is unpullable within a fortnight: the `sha256:57eeef…ed981` pin recorded here on
2026-09-30 returned `MANIFEST_UNKNOWN` by 2026-10-27. Stable-branch tags move too (`gnomeos-51`
was re-pushed three times between 2026-10-25 and 2026-10-27).

So the lane resolves the tag to a digest **at dispatch time**, pulls that digest, and the
`gnome-e2e` action records what it actually installed in `results/guest-provenance.json`:
the image reference, the digest podman resolved, and the guest's `os-release` identity
(`ID=org.gnome.os`, `IMAGE_VERSION`, `GNOMEOS_COMMIT`, the gnome-build-meta commit). That file,
not a doc table, is the provenance record. A long-lived pin would need a mirror into a registry
we control, which is lab infrastructure, not this repo.

```bash
# resolve without pulling (Quay anonymous token)
TOKEN=$(curl -s "https://quay.io/v2/auth?service=quay.io&scope=repository:gnome_infrastructure/gnome-build-meta:pull" | jq -r .token)
curl -sI -H "Authorization: Bearer $TOKEN" \
-H 'Accept: application/vnd.oci.image.manifest.v1+json' \
https://quay.io/v2/gnome_infrastructure/gnome-build-meta/manifests/gnomeos-nightly \
| grep -i docker-content-digest
# or: skopeo inspect docker://quay.io/gnome_infrastructure/gnome-build-meta:gnomeos-nightly | jq -r .Digest
```

### Resolving the digest in a workflow

No workflow does this yet. A sketch of a `resolve` job whose output feeds the `gnome-e2e`
`image:` input (the e2e job declares `needs: resolve`):

```yaml
jobs:
resolve:
runs-on: ubuntu-latest
outputs:
digest: ${{ steps.digest.outputs.digest }}
steps:
- id: digest
run: |
d=$(skopeo inspect docker://quay.io/gnome_infrastructure/gnome-build-meta:gnomeos-nightly | jq -r .Digest)
echo "digest=$d" >> "$GITHUB_OUTPUT"
```

## Guest inventory (verified against the image, not booted)

Inventoried 2026-10-27 from `gnomeos-nightly` @ `sha256:9dd3a858…e824c`
(`IMAGE_VERSION=nightly.20261027.1`, `GNOMEOS_COMMIT=9c38eabc…`) by listing its four OCI layers
(Freedesktop SDK minimal + platform, GNOME SDK platform, GNOME OS) and running
`scripts/guest_preflight.py` over a skeleton of that tree. The image config sets
`containers.bootc=1`.

| `gnome-e2e` step | Needs | GNOME OS nightly |
|---|---|---|
| `bootc install to-disk` | `bootc` in the image | `usr/bin/bootc`, `usr/lib/bootc/install/` ✅ (no `bootupd`; the action already tolerates that) |
| direct QEMU boot | kernel + initramfs in the deployment | `usr/lib/modules/7.3.0/{vmlinuz,initramfs.img}` ✅ |
| harness user | appendable `/etc/passwd`, `/bin/bash` | `etc/passwd` (system users only), `usr/bin/bash` ✅ |
| harness SSH | `sshd`, `sshd.service`, `scp`, `sftp-server` | ✅ (unit runs `ssh-keygen -A` itself) |
| harness admin | `sudo` | `usr/bin/sudo` + `etc/sudoers` ✅ |
| session | GDM, gnome-shell, Mutter, GJS | ✅ `etc/systemd/system/display-manager.service` → `gdm.service` |
| test stack install | `python3` + `ensurepip` | Python 3.14 with bundled pip wheel ✅ (no system pip) |
| AT-SPI | PyGObject, pyatspi, `at-spi-bus-launcher` | ✅ |
| virtual input | `uinput` module | `uinput.ko.zst` + `modules-load.d/uinput.conf` ✅ |
| virtual input | **`gnome-ponytail-daemon`** | **absent** ❌ |
| extensions | `gnome-extensions`, `gdbus` | ✅ |

Not verifiable without a boot, so still open: whether GNOME OS's first-boot
`gnome-initial-setup` (`usr/libexec/gnome-initial-setup` is present) interferes with GDM
autologin of the injected `bluefin-test` user, and whether `bootc install to-disk` completes on
this image with `--bootloader systemd`. Record both from the first lab run's `vm-serial.log`.

There is no package manager (no `dnf`/`rpm`); missing tooling must arrive in a test overlay or
sysext, recorded separately from the base digest.

## Pre-boot preflight (`scripts/guest_preflight.py`)

For `suite: extensions` the action runs the preflight over the mounted ostree deployment **before
the VM boots**. It writes `results/guest-provenance.json` and fails the job as a provisioning
error when any prerequisite above is missing (or the guest has no `os-release`). Today it fails
stock GNOME OS on exactly one item, `gnome-ponytail-daemon` — that is the correct, named failure
in place of a qecore error ten minutes later. It records `is_gnome_os` from `os-release`
`ID` but does not reject other images, so a Bluefin/Dakota extensions run stays possible and is
never labelled GNOME OS. The final path component is checked with `lexists` and never resolved,
and hits under a parent directory that resolves outside the guest tree are discarded, so
absolute guest symlinks are not followed onto the host.
16 changes: 7 additions & 9 deletions docs/skills/ci-ops/ops/references/fedora-version-targets.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Three Fedora versions appear in this repo. They are not interchangeable.
| **`behave --dry-run` CI container** (`pr-validate.yml`) | `fedora:41` (pinned digest) | qecore/dogtail/GObject ABI target; PyGObject from Ubuntu breaks |
| **Test runner image** (`container/Containerfile.runner`) | `fedora-minimal:latest` (rebuilt weekly) | Base for the runner container shipped to the VM |
| **OS under test (gnomeos)** | `gnomeos-latest` (current Fedora / GNOME release) | `quay.io/gnome_infrastructure/gnome-build-meta` |
| **OS under test (extension-validation lane)** | `gnomeos-nightly` (pinned digest) | reproducible GNOME OS build for the developer extension validation lane (#908) |
| **OS under test (extension-validation lane)** | `gnomeos-nightly` (digest resolved per run; tags expire after 2 weeks) | GNOME OS build for the developer extension validation lane (#908); provenance recorded in `results/guest-provenance.json` |
| **OS under test (Bluefin)** | Fedora 41 based (stable/gts/lts) | Do NOT test against F42 — Bluefin does not ship it |

**Never use F42**: no Bluefin or Bazzite image is based on Fedora 42.
Expand Down Expand Up @@ -49,15 +49,13 @@ one moving part in the lane: the base image digest.
> `docs/skills/ci-ops/e2e-workflow/references/gnome-extensions-validation.md` for the
> open blockers.

**Pinned digest (do not float):** the lane passes this digest reference — not the
`gnomeos-nightly` tag — to the `gnome-e2e` action's `image` input.
**Resolve the digest at dispatch time:** `gnomeos-*` digests expire two weeks after push, so
no static pin is kept here. The lane resolves `gnomeos-nightly` to a digest per run and passes
that `@sha256:…` reference — not the tag — to the `gnome-e2e` action's `image` input; see
[gnomeos-guest-provenance.md](../../e2e-workflow/references/gnomeos-guest-provenance.md).

```
quay.io/gnome_infrastructure/gnome-build-meta@sha256:57eeef917d057e37d8d5824fab195ebf3b0ad49fc1ca191b7bc02b82418ed981
```

Sibling tags for reference only (not pinned for the lane): `gnomeos-51`,
`gnomeos-50`, `gnomeos-latest`. If the nightly digest changes, re-derive it with:
Sibling tags for reference only: `gnomeos-51`, `gnomeos-50`, `gnomeos-latest`.
Resolve the current nightly digest with:

```bash
skopeo inspect docker://quay.io/gnome_infrastructure/gnome-build-meta:gnomeos-nightly \
Expand Down
Loading
Loading