Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,9 @@ URL-wide check would select Bluefin scenarios on Dakota.
Bluefin-family names are `bluefin`, `bazzite`, `utah`, and `common`. Common's PR
e2e image is named `common`, but it overlays common's `shared` and `bluefin`
trees onto its base image. Utah is Bluefin built on Fedora Hummingbird. Dakota
remains separate and selects `@dakota_only` scenarios. In the common suite,
`@skip_utah` excludes a `@bluefin` scenario on Utah only: it marks content that
Bluefin's common pin still ships but Utah's newer pin does not (the Searchlight
dconf scenario, until #1076 replaces it).
remains separate and selects `@dakota_only` scenarios. Content that reaches
images at different times (one image's common pin lagging another's) is gated on
the content itself, not the image name: `Skip unless the image ships "<path>"`.

The smoke suite runs inside the GUI runner container, so it only sees
`BASE_IMAGE` if `e2e.yml` passes it into that container (`-e BASE_IMAGE` and
Expand Down
2 changes: 1 addition & 1 deletion docs/skills/test-authoring/behave/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,7 @@ For commands that produce multiline output (e.g. `flatpak install ... 2>&1; echo
In the SSH-driven `common` suite, validate Bluefin desktop identity overrides with the API that matches the schema type:

- Use `gsettings get` for regular schemas shipped via `zz0-bluefin-modifications.gschema.override` (for example `org.gnome.desktop.interface accent-color` or `org.gnome.desktop.app-folders folder-children`).
- Use `dconf read` for relocatable schemas and extensions without XML schemas (for example custom media-key keybindings under `/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/`, Search Light under `/org/gnome/shell/extensions/search-light/`, and Ptyxis profile palette keys under `/org/gnome/Ptyxis/Profiles/<uuid>/`).
- Use `dconf read` for relocatable schemas and extensions without XML schemas (for example custom media-key keybindings under `/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/` such as Compass at `.../custom-keybindings/compass/`, and Ptyxis profile palette keys under `/org/gnome/Ptyxis/Profiles/<uuid>/`).

This keeps common-suite assertions aligned with how Bluefin actually ships those defaults in `projectbluefin/common`.

Expand Down
8 changes: 4 additions & 4 deletions docs/skills/test-authoring/gnome/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,10 +254,10 @@ The smoke suite's UUID-specific extension checks use the same
`_run_host(...)` helper so they work both inside the VM and from the Fedora
runner container over SSH.

Bluefin's 9 bundled extensions each get a named scenario in
`tests/smoke/features/bluefin_extensions.feature`. Tag the
`search-light@icedman.github.com` scenario `@bluefin` and guard it with
`... is installed, otherwise skip on GNOME 51+` (missing on older shells fails).
Bluefin's bundled extensions each get a named scenario in
`tests/smoke/features/bluefin_extensions.feature`; guard ones newer GNOME stops
shipping with `@bluefin` and `... is installed, otherwise skip on GNOME 51+`.
Super+Space is Compass (`org.tunaos.compass` Flatpak): `compass_launcher.feature`.

Use the distinct step phrase `GNOME extension "{uuid}" is enabled` (not
bazzite's `Extension "{uuid}" is enabled`) to avoid cross-suite step collisions.
Expand Down
4 changes: 2 additions & 2 deletions docs/skills/test-authoring/suite-map/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ Set `image` (e.g. `ghcr.io/projectbluefin/dakota:latest`), `suites: lifecycle`,

<!-- coverage-snapshot:start -->

554 scenarios across 81 feature files: 439 active, 0 quarantined, 115 `@future`/`@pending`/`@hardware_blocked`
555 scenarios across 82 feature files: 440 active, 0 quarantined, 115 `@future`/`@pending`/`@hardware_blocked`

| Suite | Scenarios | Active | Quarantined | Pending/Future | Notes |
|---|---|---|---|---|---|
Expand All @@ -180,7 +180,7 @@ Set `image` (e.g. `ghcr.io/projectbluefin/dakota:latest`), `suites: lifecycle`,
| lifecycle | 24 | 22 | 0 | 2 | bootc upgrade / rollback / migration; switch is `@future` (needs a valid alternate image ref; pin activated with settled-deployment barrier) |
| nvidia | 12 | 0 | 0 | 12 | `@future` / `@hardware_blocked` until GPU passthrough exists in the lab |
| security | 7 | 7 | 0 | 0 | cosign verify: projectbluefin (bluefin, lts, dakota) + ublue-os (latest, LTS, DX, nvidia, GTS, DX-nvidia, negative) |
| smoke | 221 | 175 | 0 | 46 | 39 `@pending` flatpak-permission audits blocked on CI never seeding system Flatpaks; MIME handler coverage (Firefox/Papers/Loupe/Text Editor/video); GNOME accessibility (AT-SPI daemon, high-contrast toggle, a11y panel); display fractional/integer scaling via Mutter DisplayConfig; Bluefin desktop identity (Wayland, hardware accel, Dash to Dock); GNOME regression guards in gnome_regression.feature; Dakota sudo-rs privilege and PAM checks |
| smoke | 222 | 176 | 0 | 46 | 39 `@pending` flatpak-permission audits blocked on CI never seeding system Flatpaks; MIME handler coverage (Firefox/Papers/Loupe/Text Editor/video); GNOME accessibility (AT-SPI daemon, high-contrast toggle, a11y panel); display fractional/integer scaling via Mutter DisplayConfig; Bluefin desktop identity (Wayland, hardware accel, Dash to Dock); GNOME regression guards in gnome_regression.feature; Dakota sudo-rs privilege and PAM checks |
| software | 35 | 27 | 0 | 8 | Bazaar GUI (`@bazaar_ui`: launch, view content per tab, typed search with a visible result, Ctrl+Q quit) + CLI presence/info/remote + config YAML parse (non-empty mappings; empty dir fails) active on images that carry Bazaar; CLI (Flathub remote + permissions DB) active on all images; Flatpak per-app permission management active on all images; FlatPark remote + Compass (`org.tunaos.compass`) system install/permission/export/uninstall lifecycle active (x86_64 only) and the shipped `compass@tunaos.org` default gated on the image enabling it (#1073); upstream GNOME Software scenarios are `@future` (#847) |
| vanilla-gnome | 15 | 15 | 0 | 0 | Baseline GNOME Shell parity check; runs on any GNOME image; org.gnome.desktop.a11y.interface reduced-motion + keyboard-focus-visible-timeout gsettings round-trips (@requires_gnome_51, skip on GNOME <= 50 via runtime Shell version probe) |

Expand Down
16 changes: 11 additions & 5 deletions tests/common/features/common_dconf.feature
Original file line number Diff line number Diff line change
Expand Up @@ -84,12 +84,18 @@ Feature: Bluefin common dconf and GSettings defaults
* SSH command return code is "0"
* SSH command output stripped "is" "'xdg-terminal-exec'"

# Bluefin's pinned common still ships Searchlight; Utah's newer pin does not (#1076).
@skip_utah
Scenario: Searchlight extension is configured
* Run SSH command: "dconf read /org/gnome/shell/extensions/search-light/shortcut-search"
# Images get Compass when their common pin reaches projectbluefin/common#1599.
Scenario: Compass launcher keybinding is configured
* Skip unless the image ships "/usr/share/flatpak/preinstall.d/compass.preinstall"
* Run SSH command: "gsettings get org.gnome.settings-daemon.plugins.media-keys custom-keybindings"
* SSH command return code is "0"
* SSH command output contains "/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/compass/"
* Run SSH command: "dconf read /org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/compass/binding"
* SSH command return code is "0"
* SSH command output stripped "is" "'<Super>space'"
* Run SSH command: "dconf read /org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/compass/command"
* SSH command return code is "0"
* SSH command output stripped "is" "['<Super>space']"
* SSH command output contains "org.tunaos.compass toggle"

# Pending: the Ptyxis palette is user-session dconf state, not image state.
@pending
Expand Down
4 changes: 0 additions & 4 deletions tests/common/features/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -213,10 +213,6 @@ def before_scenario(context, scenario):
f"(IMAGE={os.environ.get('IMAGE', 'unknown')})"
)
return
# Temporary: drop @skip_utah once #1076 replaces the scenarios it gates.
if "skip_utah" in scenario_tags and getattr(context, "is_utah_image", False):
scenario.skip("Skipping @skip_utah scenario: Utah's common pin no longer ships it")
return
if "requires_brew" in scenario_tags and not _has_brew(context):
scenario.skip("Homebrew not present on this image")
return
Expand Down
13 changes: 13 additions & 0 deletions tests/shared/ssh_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,19 @@ def run_ssh_command(context, cmd):
run_ssh(context, cmd)


@step('Skip unless the image ships "{path}"')
def skip_unless_image_ships(context, path):
"""Skip the scenario on images that predate the feature under test.

For content that reaches images at different times (one image's pin may
lag another's): the scenario runs, and must pass, wherever the marker file
exists.
"""
run_ssh(context, f"test -e {shlex.quote(path)}")
if context.ssh_rc != 0:
context.scenario.skip(f"image does not ship {path}")


@step('Run long SSH command: "{cmd}"')
def run_long_ssh_command(context, cmd):
"""Run a command that may take many minutes (bootc switch/upgrade pulling OCI layers)."""
Expand Down
7 changes: 0 additions & 7 deletions tests/smoke/features/bluefin_extensions.feature
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,3 @@ Feature: Bluefin GNOME extension presence
Scenario: Custom Command List extension is enabled
* GNOME Shell is accessible via AT-SPI
* GNOME extension "custom-command-list@storageb.github.com" is enabled

@bluefin @extensions
Scenario: Search Light extension is enabled
# Broken on GNOME 51: GNOME 51 images do not ship it, so skip there only.
* GNOME extension "search-light@icedman.github.com" is installed, otherwise skip on GNOME 51+
* GNOME Shell is accessible via AT-SPI
* GNOME extension "search-light@icedman.github.com" is enabled
20 changes: 20 additions & 0 deletions tests/smoke/features/compass_launcher.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
@smoke_suite @compass
Feature: Compass launcher
Compass (org.tunaos.compass) is preinstalled from FlatPark and bound to
Super+Space through a GNOME custom keybinding.

Scenario: Compass launcher Flatpak is preinstalled
* Compass Flatpak is installed system-wide within 600 seconds

Scenario: Super+Space toggles the Compass launcher
* Compass Flatpak is installed system-wide within 600 seconds
* Compass user service is wanted by graphical-session.target
* Compass user service is stopped and its engine is gone within 30 seconds
* Compass user service is started with one serving engine within 60 seconds
* Compass user service is restarted with one serving engine within 60 seconds
* GNOME Shell is accessible via AT-SPI
* Key combo: "<Super>space"
* Compass diagnostics are logged
* Compass launcher window is visible within 60 seconds
* Key combo: "<Super>space"
* Compass launcher window is hidden within 30 seconds
174 changes: 174 additions & 0 deletions tests/smoke/features/steps/steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -892,3 +892,177 @@ def flatpak_no_excessive_permissions(context, app_id: str) -> None:
assert not re.search(r"filesystems=(host|home)", perms_out), (
f"{app_id} has excessive filesystem access:\n{perms_out}"
)


# ── Compass launcher (org.tunaos.compass Flatpak, Super+Space) ──

_COMPASS_VISIBLE_JS = (
"global.get_window_actors().some(a => { const w = a.meta_window; "
"const ids = [w.get_wm_class(), w.get_wm_class_instance(), "
"w.get_gtk_application_id(), w.get_sandboxed_app_id(), w.get_title()]"
".join(' ').toLowerCase(); "
"return ids.includes('compass') && a.visible && !w.minimized "
"&& w.showing_on_its_workspace(); })"
)
_WINDOWS_JS = (
"JSON.stringify(global.get_window_actors().map(a => { const w = a.meta_window; "
"return [w.get_wm_class(), w.get_gtk_application_id(), "
"w.get_sandboxed_app_id(), w.get_title(), a.visible, w.minimized]; }))"
)


COMPASS_PREINSTALL = "/usr/share/flatpak/preinstall.d/compass.preinstall"


@step("Compass Flatpak is installed system-wide within {timeout:d} seconds")
def compass_flatpak_installed_within(context, timeout: int) -> None:
# flatpak-preinstall downloads a ~71 MB extra-data AppImage at boot.
# The VM e2e lane masks flatpak-preinstall.service (no first-boot network
# pulls), so Compass can never appear there: skip instead of timing out.
# Images get Compass when their common pin reaches projectbluefin/common#1599;
# where the preinstall file ships, Compass must install.
_, shipped, _ = _run_host("test -f " + COMPASS_PREINSTALL)
if shipped != 0:
context.scenario.skip(f"image does not ship {COMPASS_PREINSTALL}")
return
state, _, _ = _run_host("systemctl is-enabled flatpak-preinstall.service")
if state == "masked":
context.scenario.skip("flatpak-preinstall.service is masked in this lane")
return
deadline = time.monotonic() + timeout
while True:
out, rc, err = _run_host("flatpak info --system org.tunaos.compass")
if rc == 0:
return
assert time.monotonic() < deadline, (
f"org.tunaos.compass not installed after {timeout}s: {err or out}"
)
sleep(10)


_SESSION = "source /tmp/session.env 2>/dev/null; "


@step("Compass user service is wanted by graphical-session.target")
def compass_service_wanted_by_session(context) -> None:
enabled, _, _ = _run_host(_SESSION + "systemctl --user is-enabled compass.service")
exec_start, _, _ = _run_host(_SESSION + "systemctl --user show -p ExecStart compass.service")
print(f"compass.service is-enabled={enabled!r} {exec_start}", flush=True)
wanted_by, rc, err = _run_host(_SESSION + "systemctl --user show -p WantedBy compass.service")
assert rc == 0 and "graphical-session.target" in wanted_by, (
f"compass.service not wanted by graphical-session.target: {wanted_by!r} {err}"
)


def _compass_engine_ps() -> tuple[list[str], str]:
"""Return (engine lines, full compass ps listing).

An engine is the sandboxed binary itself (argv[0] basename ``compass``,
e.g. /app/extra/compass/usr/bin/compass) running ``serve --no-hotkey``;
the bwrap and ``flatpak run`` wrappers carry other argv[0]s.
"""
out, _, _ = _run_host("ps -eo pid=,ppid=,user=,args=")
listing = [line for line in out.splitlines() if "compass" in line.lower()]
engines = []
for line in listing:
argv = line.split()[3:]
if (argv and os.path.basename(argv[0]) == "compass"
and "serve" in argv and "--no-hotkey" in argv):
engines.append(line)
return engines, "\n".join(listing)


def _compass_ping_rc() -> tuple[int, str]:
out, rc, err = _run_host(_SESSION + "timeout 15 flatpak run org.tunaos.compass ping")
return rc, (err or out)


def _systemctl_user(verb: str) -> None:
_, rc, err = _run_host(_SESSION + f"systemctl --user {verb} compass.service", timeout=60)
assert rc == 0, f"systemctl --user {verb} compass.service failed: {err}"


@step("Compass user service is stopped and its engine is gone within {timeout:d} seconds")
def compass_service_stopped(context, timeout: int) -> None:
_systemctl_user("stop")
deadline = time.monotonic() + timeout
while True:
ping_rc, ping_out = _compass_ping_rc()
engines, listing = _compass_engine_ps()
if ping_rc != 0 and not engines:
return
if time.monotonic() >= deadline:
raise AssertionError(
f"Compass engine survived 'systemctl --user stop' for {timeout}s: "
f"ping rc={ping_rc} {ping_out}; engines={len(engines)}\n{listing}"
)
sleep(2)


@step("Compass user service is {verb} with one serving engine within {timeout:d} seconds")
def compass_service_serves_one_engine(context, verb: str, timeout: int) -> None:
action = {"started": "start", "restarted": "restart"}[verb]
_systemctl_user(action)
deadline = time.monotonic() + timeout
while True:
active, _, _ = _run_host(_SESSION + "systemctl --user is-active compass.service")
ping_rc, ping_out = _compass_ping_rc()
engines, listing = _compass_engine_ps()
if active == "active" and ping_rc == 0 and len(engines) == 1:
break
if time.monotonic() >= deadline:
status, _, _ = _run_host(
_SESSION + "systemctl --user status --no-pager -l compass.service 2>&1"
)
raise AssertionError(
f"compass.service not serving one engine {timeout}s after {action}: "
f"is-active={active!r} ping rc={ping_rc} {ping_out}; "
f"engines={len(engines)}\n{listing}\n{status}"
)
sleep(2)
if action == "restart":
restarts, _, _ = _run_host(_SESSION + "systemctl --user show -p NRestarts compass.service")
assert restarts == "NRestarts=0", (
f"compass.service crash-looped after restart: {restarts!r}\n{listing}"
)


_COMPASS_DIAG_CMDS = (
"ps -eo pid,ppid,user,args | grep -i [c]ompass",
"systemctl --user status compass.service --no-pager -l 2>&1 | tail -30",
"journalctl --user -b --no-pager 2>&1 | grep -i -E 'compass|media-keys|custom' | tail -80",
"systemctl --user show-environment | grep -E 'WAYLAND|DISPLAY|XDG_SESSION|DBUS'",
"timeout 20 flatpak run org.tunaos.compass doctor 2>&1 | tail -60",
"timeout 10 flatpak run org.tunaos.compass logs 2>&1 | tail -80",
)


def _compass_diagnostics(include_toggle: bool) -> str:
cmds = list(_COMPASS_DIAG_CMDS)
if include_toggle:
# Mutates launcher state, so only run once the assertion has failed.
cmds.append("timeout 15 flatpak run org.tunaos.compass toggle 2>&1; echo rc=$?")
parts = []
for cmd in cmds:
out, rc, err = _run_host(_SESSION + cmd, timeout=40)
parts.append(f"$ {cmd} [rc={rc}]\n{out}\n{err}".rstrip())
report = "\n".join(parts)
print(f"── Compass diagnostics ──\n{report}", flush=True)
return report


@step("Compass diagnostics are logged")
def compass_diagnostics_are_logged(context) -> None:
_compass_diagnostics(include_toggle=False)


@step("Compass launcher window is {state} within {timeout:d} seconds")
def compass_window_state_within(context, state: str, timeout: int) -> None:
assert state in ("visible", "hidden"), f"unknown state {state!r}"
if _wait_eval_bool(_COMPASS_VISIBLE_JS, state == "visible", retries=timeout, delay=1):
return
windows = _shell_eval(_WINDOWS_JS)
raise AssertionError(
f"Compass window not {state} after {timeout}s; windows: {windows}\n"
+ _compass_diagnostics(include_toggle=True)
)
20 changes: 0 additions & 20 deletions tests/unit/test_common_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -249,26 +249,6 @@ def test_allows_dakota_only_on_dakota_image(self):
assert scenario.skip_message is None


class TestCommonEnvironmentSkipUtah:
def test_skips_skip_utah_on_utah_image(self):
m = _import_common_environment()
context = _ctx(is_bluefin_image=True, is_utah_image=True, has_brew=True)
scenario = _Scenario(["bluefin", "skip_utah"])

m.before_scenario(context, scenario)

assert "Skipping @skip_utah scenario" in scenario.skip_message

def test_allows_skip_utah_on_bluefin_image(self):
m = _import_common_environment()
context = _ctx(is_bluefin_image=True, is_utah_image=False, has_brew=True)
scenario = _Scenario(["bluefin", "skip_utah"])

m.before_scenario(context, scenario)

assert scenario.skip_message is None


class TestCommonEnvironmentRequiresToggleAction:
def test_skips_when_recipe_lacks_action_value(self):
m = _import_common_environment(run_ssh_returncode=1)
Expand Down
2 changes: 1 addition & 1 deletion tests/unit/test_gnome_extensions_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ def fake_run_host(cmd):


class TestExtensionInstalledOrSkip:
UUID = "search-light@icedman.github.com"
UUID = "gsconnect@andyholmes.github.io"

def _run(self, installed: bool, major):
m = _import_gnome_extensions_steps()
Expand Down
Loading
Loading