Skip to content
Closed
2 changes: 1 addition & 1 deletion docs/skills/test-authoring/behave/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -383,7 +383,7 @@ For commands that produce multiline output (e.g. `flatpak install ... 2>&1; echo
In the SSH-driven `common` suite, validate Bluefin desktop identity overrides with the API that matches the schema type:

- Use `gsettings get` for regular schemas shipped via `zz0-bluefin-modifications.gschema.override` (for example `org.gnome.desktop.interface accent-color` or `org.gnome.desktop.app-folders folder-children`).
- Use `dconf read` for relocatable schemas and extensions without XML schemas (for example custom media-key keybindings under `/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/`, Search Light under `/org/gnome/shell/extensions/search-light/`, and Ptyxis profile palette keys under `/org/gnome/Ptyxis/Profiles/<uuid>/`).
- Use `dconf read` for relocatable schemas and extensions without XML schemas (for example custom media-key keybindings under `/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/` such as Compass at `.../custom-keybindings/compass/`, and Ptyxis profile palette keys under `/org/gnome/Ptyxis/Profiles/<uuid>/`).

This keeps common-suite assertions aligned with how Bluefin actually ships those defaults in `projectbluefin/common`.

Expand Down
8 changes: 4 additions & 4 deletions docs/skills/test-authoring/gnome/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,10 +244,10 @@ The smoke suite's UUID-specific extension checks use the same
`_run_host(...)` helper so they work both inside the VM and from the Fedora
runner container over SSH.

Bluefin's 9 bundled extensions each get a named scenario in
`tests/smoke/features/bluefin_extensions.feature`. Tag the
`search-light@icedman.github.com` scenario `@bluefin` and guard it with
`... is installed, otherwise skip on GNOME 51+` (missing on older shells fails).
Bluefin's bundled extensions each get a named scenario in
`tests/smoke/features/bluefin_extensions.feature`; guard ones newer GNOME stops
shipping with `@bluefin` and `... is installed, otherwise skip on GNOME 51+`.
Super+Space is Compass (`org.tunaos.compass` Flatpak): `compass_launcher.feature`.

Use the distinct step phrase `GNOME extension "{uuid}" is enabled` (not
bazzite's `Extension "{uuid}" is enabled`) to avoid cross-suite step collisions.
Expand Down
4 changes: 2 additions & 2 deletions docs/skills/test-authoring/suite-map/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ Set `chunked_enabled: true` once `ghcr.io/projectbluefin/bluefin:latest` ships z

<!-- coverage-snapshot:start -->

571 scenarios across 82 feature files: 455 active, 0 quarantined, 116 `@future`/`@pending`/`@hardware_blocked`
572 scenarios across 83 feature files: 456 active, 0 quarantined, 116 `@future`/`@pending`/`@hardware_blocked`

| Suite | Scenarios | Active | Quarantined | Pending/Future | Notes |
|---|---|---|---|---|---|
Expand All @@ -181,7 +181,7 @@ Set `chunked_enabled: true` once `ghcr.io/projectbluefin/bluefin:latest` ships z
| lifecycle | 33 | 30 | 0 | 3 | bootc upgrade / rollback / migration; switch is `@future` (needs a valid alternate image ref; pin activated with settled-deployment barrier) |
| nvidia | 12 | 0 | 0 | 12 | `@future` / `@hardware_blocked` until GPU passthrough exists in the lab |
| security | 15 | 15 | 0 | 0 | cosign verify: projectbluefin (bluefin, lts, dakota) + ublue-os (latest, LTS, DX, nvidia, GTS, DX-nvidia, negative) |
| smoke | 221 | 175 | 0 | 46 | 39 `@pending` flatpak-permission audits blocked on CI never seeding system Flatpaks; MIME handler coverage (Firefox/Papers/Loupe/Text Editor/video); GNOME accessibility (AT-SPI daemon, high-contrast toggle, a11y panel); display fractional/integer scaling via Mutter DisplayConfig; Bluefin desktop identity (Wayland, hardware accel, Dash to Dock); GNOME regression guards in gnome_regression.feature; Dakota sudo-rs privilege and PAM checks |
| smoke | 222 | 176 | 0 | 46 | 39 `@pending` flatpak-permission audits blocked on CI never seeding system Flatpaks; MIME handler coverage (Firefox/Papers/Loupe/Text Editor/video); GNOME accessibility (AT-SPI daemon, high-contrast toggle, a11y panel); display fractional/integer scaling via Mutter DisplayConfig; Bluefin desktop identity (Wayland, hardware accel, Dash to Dock); GNOME regression guards in gnome_regression.feature; Dakota sudo-rs privilege and PAM checks |
| software | 35 | 27 | 0 | 8 | Bazaar GUI (`@bazaar_ui`: launch, view content per tab, typed search with a visible result, Ctrl+Q quit) + CLI presence/info/remote + config YAML parse (non-empty mappings; empty dir fails) active on images that carry Bazaar; CLI (Flathub remote + permissions DB) active on all images; Flatpak per-app permission management active on all images; FlatPark remote + Compass (`org.tunaos.compass`) system install/permission/export/uninstall lifecycle active (x86_64 only) and the shipped `compass@tunaos.org` default gated on the image enabling it (#1073); upstream GNOME Software scenarios are `@future` (#847) |
| vanilla-gnome | 15 | 15 | 0 | 0 | Baseline GNOME Shell parity check; runs on any GNOME image; org.gnome.desktop.a11y.interface reduced-motion + keyboard-focus-visible-timeout gsettings round-trips (@requires_gnome_51, skip on GNOME <= 50 via runtime Shell version probe) |

Expand Down
14 changes: 9 additions & 5 deletions tests/common/features/common_dconf.feature
Original file line number Diff line number Diff line change
Expand Up @@ -84,12 +84,16 @@ Feature: Bluefin common dconf and GSettings defaults
* SSH command return code is "0"
* SSH command output stripped "is" "'xdg-terminal-exec'"

# Bluefin's pinned common still ships Searchlight; Utah's newer pin does not (#1076).
@skip_utah
Scenario: Searchlight extension is configured
* Run SSH command: "dconf read /org/gnome/shell/extensions/search-light/shortcut-search"
Scenario: Compass launcher keybinding is configured
* Run SSH command: "gsettings get org.gnome.settings-daemon.plugins.media-keys custom-keybindings"
* SSH command return code is "0"
* SSH command output contains "/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/compass/"
* Run SSH command: "dconf read /org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/compass/binding"
* SSH command return code is "0"
* SSH command output stripped "is" "'<Super>space'"
* Run SSH command: "dconf read /org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/compass/command"
* SSH command return code is "0"
* SSH command output stripped "is" "['<Super>space']"
* SSH command output contains "org.tunaos.compass toggle"

# Pending: the Ptyxis palette is user-session dconf state, not image state.
@pending
Expand Down
7 changes: 0 additions & 7 deletions tests/smoke/features/bluefin_extensions.feature
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,3 @@ Feature: Bluefin GNOME extension presence
Scenario: Custom Command List extension is enabled
* GNOME Shell is accessible via AT-SPI
* GNOME extension "custom-command-list@storageb.github.com" is enabled

@bluefin @extensions
Scenario: Search Light extension is enabled
# Broken on GNOME 51: GNOME 51 images do not ship it, so skip there only.
* GNOME extension "search-light@icedman.github.com" is installed, otherwise skip on GNOME 51+
* GNOME Shell is accessible via AT-SPI
* GNOME extension "search-light@icedman.github.com" is enabled
20 changes: 20 additions & 0 deletions tests/smoke/features/compass_launcher.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
@smoke_suite @compass
Feature: Compass launcher
Compass (org.tunaos.compass) is preinstalled from FlatPark and bound to
Super+Space through a GNOME custom keybinding.

Scenario: Compass launcher Flatpak is preinstalled
* Compass Flatpak is installed system-wide within 600 seconds

Scenario: Super+Space toggles the Compass launcher
* Compass Flatpak is installed system-wide within 600 seconds
* Compass user service is wanted by graphical-session.target
* Compass user service is stopped and its engine is gone within 30 seconds
* Compass user service is started with one serving engine within 60 seconds
* Compass user service is restarted with one serving engine within 60 seconds
* GNOME Shell is accessible via AT-SPI
* Key combo: "<Super>space"
* Compass diagnostics are logged
* Compass launcher window is visible within 60 seconds
* Key combo: "<Super>space"
* Compass launcher window is hidden within 30 seconds
165 changes: 165 additions & 0 deletions tests/smoke/features/steps/steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -892,3 +892,168 @@ def flatpak_no_excessive_permissions(context, app_id: str) -> None:
assert not re.search(r"filesystems=(host|home)", perms_out), (
f"{app_id} has excessive filesystem access:\n{perms_out}"
)


# ── Compass launcher (org.tunaos.compass Flatpak, Super+Space) ──

_COMPASS_VISIBLE_JS = (
"global.get_window_actors().some(a => { const w = a.meta_window; "
"const ids = [w.get_wm_class(), w.get_wm_class_instance(), "
"w.get_gtk_application_id(), w.get_sandboxed_app_id(), w.get_title()]"
".join(' ').toLowerCase(); "
"return ids.includes('compass') && a.visible && !w.minimized "
"&& w.showing_on_its_workspace(); })"
)
_WINDOWS_JS = (
"JSON.stringify(global.get_window_actors().map(a => { const w = a.meta_window; "
"return [w.get_wm_class(), w.get_gtk_application_id(), "
"w.get_sandboxed_app_id(), w.get_title(), a.visible, w.minimized]; }))"
)


@step("Compass Flatpak is installed system-wide within {timeout:d} seconds")
def compass_flatpak_installed_within(context, timeout: int) -> None:
# flatpak-preinstall downloads a ~71 MB extra-data AppImage at boot.
# The VM e2e lane masks flatpak-preinstall.service (no first-boot network
# pulls), so Compass can never appear there: skip instead of timing out.
state, _, _ = _run_host("systemctl is-enabled flatpak-preinstall.service")
if state == "masked":
context.scenario.skip("flatpak-preinstall.service is masked in this lane")
return
deadline = time.monotonic() + timeout
while True:
out, rc, err = _run_host("flatpak info --system org.tunaos.compass")
if rc == 0:
return
assert time.monotonic() < deadline, (
f"org.tunaos.compass not installed after {timeout}s: {err or out}"
)
sleep(10)


_SESSION = "source /tmp/session.env 2>/dev/null; "


@step("Compass user service is wanted by graphical-session.target")
def compass_service_wanted_by_session(context) -> None:
enabled, _, _ = _run_host(_SESSION + "systemctl --user is-enabled compass.service")
exec_start, _, _ = _run_host(_SESSION + "systemctl --user show -p ExecStart compass.service")
print(f"compass.service is-enabled={enabled!r} {exec_start}", flush=True)
wanted_by, rc, err = _run_host(_SESSION + "systemctl --user show -p WantedBy compass.service")
assert rc == 0 and "graphical-session.target" in wanted_by, (
f"compass.service not wanted by graphical-session.target: {wanted_by!r} {err}"
)


def _compass_engine_ps() -> tuple[list[str], str]:
"""Return (engine lines, full compass ps listing).

An engine is the sandboxed binary itself (argv[0] basename ``compass``,
e.g. /app/extra/compass/usr/bin/compass) running ``serve --no-hotkey``;
the bwrap and ``flatpak run`` wrappers carry other argv[0]s.
"""
out, _, _ = _run_host("ps -eo pid=,ppid=,user=,args=")
listing = [line for line in out.splitlines() if "compass" in line.lower()]
engines = []
for line in listing:
argv = line.split()[3:]
if (argv and os.path.basename(argv[0]) == "compass"
and "serve" in argv and "--no-hotkey" in argv):
engines.append(line)
return engines, "\n".join(listing)


def _compass_ping_rc() -> tuple[int, str]:
out, rc, err = _run_host(_SESSION + "timeout 15 flatpak run org.tunaos.compass ping")
return rc, (err or out)


def _systemctl_user(verb: str) -> None:
_, rc, err = _run_host(_SESSION + f"systemctl --user {verb} compass.service", timeout=60)
assert rc == 0, f"systemctl --user {verb} compass.service failed: {err}"


@step("Compass user service is stopped and its engine is gone within {timeout:d} seconds")
def compass_service_stopped(context, timeout: int) -> None:
_systemctl_user("stop")
deadline = time.monotonic() + timeout
while True:
ping_rc, ping_out = _compass_ping_rc()
engines, listing = _compass_engine_ps()
if ping_rc != 0 and not engines:
return
if time.monotonic() >= deadline:
raise AssertionError(
f"Compass engine survived 'systemctl --user stop' for {timeout}s: "
f"ping rc={ping_rc} {ping_out}; engines={len(engines)}\n{listing}"
)
sleep(2)


@step("Compass user service is {verb} with one serving engine within {timeout:d} seconds")
def compass_service_serves_one_engine(context, verb: str, timeout: int) -> None:
action = {"started": "start", "restarted": "restart"}[verb]
_systemctl_user(action)
deadline = time.monotonic() + timeout
while True:
active, _, _ = _run_host(_SESSION + "systemctl --user is-active compass.service")
ping_rc, ping_out = _compass_ping_rc()
engines, listing = _compass_engine_ps()
if active == "active" and ping_rc == 0 and len(engines) == 1:
break
if time.monotonic() >= deadline:
status, _, _ = _run_host(
_SESSION + "systemctl --user status --no-pager -l compass.service 2>&1"
)
raise AssertionError(
f"compass.service not serving one engine {timeout}s after {action}: "
f"is-active={active!r} ping rc={ping_rc} {ping_out}; "
f"engines={len(engines)}\n{listing}\n{status}"
)
sleep(2)
if action == "restart":
restarts, _, _ = _run_host(_SESSION + "systemctl --user show -p NRestarts compass.service")
assert restarts == "NRestarts=0", (
f"compass.service crash-looped after restart: {restarts!r}\n{listing}"
)


_COMPASS_DIAG_CMDS = (
"ps -eo pid,ppid,user,args | grep -i [c]ompass",
"systemctl --user status compass.service --no-pager -l 2>&1 | tail -30",
"journalctl --user -b --no-pager 2>&1 | grep -i -E 'compass|media-keys|custom' | tail -80",
"systemctl --user show-environment | grep -E 'WAYLAND|DISPLAY|XDG_SESSION|DBUS'",
"timeout 20 flatpak run org.tunaos.compass doctor 2>&1 | tail -60",
"timeout 10 flatpak run org.tunaos.compass logs 2>&1 | tail -80",
)


def _compass_diagnostics(include_toggle: bool) -> str:
cmds = list(_COMPASS_DIAG_CMDS)
if include_toggle:
# Mutates launcher state, so only run once the assertion has failed.
cmds.append("timeout 15 flatpak run org.tunaos.compass toggle 2>&1; echo rc=$?")
parts = []
for cmd in cmds:
out, rc, err = _run_host(_SESSION + cmd, timeout=40)
parts.append(f"$ {cmd} [rc={rc}]\n{out}\n{err}".rstrip())
report = "\n".join(parts)
print(f"── Compass diagnostics ──\n{report}", flush=True)
return report


@step("Compass diagnostics are logged")
def compass_diagnostics_are_logged(context) -> None:
_compass_diagnostics(include_toggle=False)


@step("Compass launcher window is {state} within {timeout:d} seconds")
def compass_window_state_within(context, state: str, timeout: int) -> None:
assert state in ("visible", "hidden"), f"unknown state {state!r}"
if _wait_eval_bool(_COMPASS_VISIBLE_JS, state == "visible", retries=timeout, delay=1):
return
windows = _shell_eval(_WINDOWS_JS)
raise AssertionError(
f"Compass window not {state} after {timeout}s; windows: {windows}\n"
+ _compass_diagnostics(include_toggle=True)
)
2 changes: 1 addition & 1 deletion tests/unit/test_gnome_extensions_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ def fake_run_host(cmd):


class TestExtensionInstalledOrSkip:
UUID = "search-light@icedman.github.com"
UUID = "gsconnect@andyholmes.github.io"

def _run(self, installed: bool, major):
m = _import_gnome_extensions_steps()
Expand Down
Loading