Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/skills/test-authoring/suite-map/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ Set `image` (e.g. `ghcr.io/projectbluefin/dakota:latest`), `suites: lifecycle`,
| `@bluefin` | Scenario runs on Bluefin-family image names (`bluefin`, `bazzite`, `utah`, `common`); smoke and common read `BASE_IMAGE` before `IMAGE`, so composed runs keep their family |
| `@dakota_only` | Scenario runs only when the image name contains `dakota`; smoke and common read `BASE_IMAGE` before `IMAGE`. Only the image **name** is matched, so the `projectbluefin` org name cannot false-positive |
| `@skip_utah` | Common scenario skips when the image name contains `utah`: it asserts content Bluefin's common pin still ships but Utah's newer pin does not. Remove it when the scenario is replaced (#1076) |
| `@dakota_family` | Smoke scenario runs only on Dakota-family image names (`dakota`, `utah`): images that follow Dakota's desktop defaults, such as its default-enabled extension set. Keep Dakota-specific UUIDs or content under `@dakota_only` |
| `@dakota_family` | Smoke scenario runs only on Dakota-family image names (`dakota`, `utah`): images that follow Dakota's desktop defaults, such as its default-enabled extension set. Keep Dakota-specific UUIDs or content under `@dakota_only`; content Utah ships too, such as bling's `syncthing-toggle@projectbluefin.io` and its `start-stop-only` default, belongs under `@dakota_family` |
| `@bluefin_classic` | Smoke scenario asserts Bluefin-classic desktop defaults (for example copyous or the Quick Settings extensions enabled by default) and skips on Dakota-family images. Utah is in both families, so a plain `@bluefin` scenario still runs there |
| `@dx_only` / `@developer_suite` | DX variant only |
| `@nvidia_only` | NVIDIA variant only |
Expand Down Expand Up @@ -200,7 +200,7 @@ that issue rather than file another, and close it when it no longer applies. Det

<!-- coverage-snapshot:start -->

554 scenarios across 81 feature files: 439 active, 0 quarantined, 115 `@future`/`@pending`/`@hardware_blocked`
553 scenarios across 81 feature files: 438 active, 0 quarantined, 115 `@future`/`@pending`/`@hardware_blocked`

| Suite | Scenarios | Active | Quarantined | Pending/Future | Notes |
|---|---|---|---|---|---|
Expand All @@ -217,7 +217,7 @@ that issue rather than file another, and close it when it no longer applies. Det
| lifecycle | 24 | 22 | 0 | 2 | bootc upgrade / rollback / migration; switch is `@future` (needs a valid alternate image ref; pin activated with settled-deployment barrier) |
| nvidia | 12 | 0 | 0 | 12 | `@future` / `@hardware_blocked` until GPU passthrough exists in the lab |
| security | 7 | 7 | 0 | 0 | cosign verify: projectbluefin (bluefin, lts, dakota) + ublue-os (latest, LTS, DX, nvidia, GTS, DX-nvidia, negative) |
| smoke | 221 | 175 | 0 | 46 | 39 `@pending` flatpak-permission audits blocked on CI never seeding system Flatpaks; MIME handler coverage (Firefox/Papers/Loupe/Text Editor/video); GNOME accessibility (AT-SPI daemon, high-contrast toggle, a11y panel); display fractional/integer scaling via Mutter DisplayConfig; Bluefin desktop identity (Wayland, hardware accel, Dash to Dock); GNOME regression guards in gnome_regression.feature; Dakota sudo-rs privilege and PAM checks |
| smoke | 220 | 174 | 0 | 46 | 39 `@pending` flatpak-permission audits blocked on CI never seeding system Flatpaks; MIME handler coverage (Firefox/Papers/Loupe/Text Editor/video); GNOME accessibility (AT-SPI daemon, high-contrast toggle, a11y panel); display fractional/integer scaling via Mutter DisplayConfig; Bluefin desktop identity (Wayland, hardware accel, Dash to Dock); GNOME regression guards in gnome_regression.feature; Dakota sudo-rs privilege and PAM checks |
| software | 35 | 27 | 0 | 8 | Bazaar GUI (`@bazaar_ui`: launch, view content per tab, typed search with a visible result, Ctrl+Q quit) + CLI presence/info/remote + config YAML parse (non-empty mappings; empty dir fails) active on images that carry Bazaar; CLI (Flathub remote + permissions DB) active on all images; Flatpak per-app permission management active on all images; FlatPark remote + Compass (`org.tunaos.compass`) system install/permission/export/uninstall lifecycle active (x86_64 only) and the shipped `compass@tunaos.org` default gated on the image enabling it (#1073); upstream GNOME Software scenarios are `@future` (#847) |
| vanilla-gnome | 15 | 15 | 0 | 0 | Baseline GNOME Shell parity check; runs on any GNOME image; org.gnome.desktop.a11y.interface reduced-motion + keyboard-focus-visible-timeout gsettings round-trips (@requires_gnome_51, skip on GNOME <= 50 via runtime Shell version probe) |

Expand Down
9 changes: 0 additions & 9 deletions tests/smoke/features/bluefin_new_extensions.feature
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,6 @@ Feature: Newly enabled Bluefin GNOME extension stability
* GNOME Shell remains accessible and does not crash during the stress run
* GNOME Shell memory usage remains bounded after repeated copy and paste operations

@bluefin @extensions
Scenario: Syncthing Toggle Quick Settings control is enabled
* GNOME Shell is accessible via AT-SPI
* GNOME extension "syncthing-toggle@rehhouari.github.com" is enabled
* Quick Settings contains a toggle labeled "Sync Folder"
* The pill and header use the label "Sync Folder"
* The toggle starts and stops the Syncthing service without shell errors
* The extension honors "start-stop-only = true"

@extensions @pending
# @pending: 'The Bluetooth battery panel icon is rendered' soft-passes when
# dogtail is unavailable or gnome-shell is absent from the AT-SPI tree.
Expand Down
7 changes: 4 additions & 3 deletions tests/smoke/features/dakota_extensions.feature
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ Feature: Dakota GNOME Shell extension presence and configuration
are present and valid, and that dconf schema overrides match the shipped specification.
The default-enabled set is @dakota_family: Utah enables the same set
(projectbluefin/utah#1252). The installed-only checks name Dakota's
shipped UUIDs and stay @dakota_only.
shipped UUIDs and stay @dakota_only, except bling's Sync Folder toggle,
which Utah ships with the same start-stop-only default.

# ---------------------------------------------------------------------------
# Default-enabled extensions (from disable-ext-validator.bst)
Expand Down Expand Up @@ -94,8 +95,8 @@ Feature: Dakota GNOME Shell extension presence and configuration
* GNOME Shell is accessible via AT-SPI
* GNOME extension "quicksettings-audio-devices-renamer@marcinjahn.com" is installed

@dakota_only @extensions
Scenario: Syncthing Toggle extension is installed and configured on Dakota
@dakota_family @extensions
Scenario: Syncthing Toggle extension is installed and configured on Dakota-family images
* GNOME Shell is accessible via AT-SPI
* GNOME extension "syncthing-toggle@projectbluefin.io" is installed
* Dconf path "/org/gnome/shell/extensions/syncthing-toggle/start-stop-only" has value "true"
Expand Down
64 changes: 0 additions & 64 deletions tests/smoke/features/steps/bluefin_new_extensions_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
# Extension identifiers (issue #793 / projectbluefin/common#1087)
# ---------------------------------------------------------------------------
COPYOUS_UUID = "copyous@boerdereinar.dev"
SYNCTHING_TOGGLE_UUID = "syncthing-toggle@rehhouari.github.com"
BLUETOOTH_BATTERY_METER_UUID = "Bluetooth-Battery-Meter@maniacx.github.com"
AUDIO_HIDER_UUID = "quicksettings-audio-devices-hider@marcinjahn.com"
AUDIO_RENAMER_UUID = "quicksettings-audio-devices-renamer@marcinjahn.com"
Expand Down Expand Up @@ -168,18 +167,6 @@ def _extension_gsettings_value(uuid: str, key: str):
return None, None


def _systemd_unit_start_stop(unit: str) -> tuple[bool, bool, str]:
"""Try to start then stop a systemd (user) unit. Returns
(unit_available, both_succeeded, detail)."""
show_out, show_rc, _ = _run_host(f"systemctl --user show -p LoadState {unit} 2>&1")
if show_rc != 0 or "LoadState=not-found" in show_out or "LoadState=" not in show_out:
return False, False, show_out
_, start_rc, start_err = _run_host(f"systemctl --user start {unit} 2>&1")
sleep(1)
_, stop_rc, stop_err = _run_host(f"systemctl --user stop {unit} 2>&1")
return True, (start_rc == 0 and stop_rc == 0), (start_err or stop_err)


# ---------------------------------------------------------------------------
# Copyous — clipboard manager stress/AT-SPI coverage
# ---------------------------------------------------------------------------
Expand Down Expand Up @@ -392,57 +379,6 @@ def gnome_shell_memory_usage_remains_bounded(context) -> None:
)


# ---------------------------------------------------------------------------
# Syncthing Toggle
# ---------------------------------------------------------------------------

@step('Quick Settings contains a toggle labeled "{label}"')
def quick_settings_contains_a_toggle_labeled(context, label: str) -> None:
assert _quick_settings_contains_text(label), (
f'Quick Settings does not expose a toggle labeled "{label}"'
)


@step('The pill and header use the label "{label}"')
def the_pill_and_header_use_the_label(context, label: str) -> None:
assert _quick_settings_contains_text(label), (
f'Quick Settings pill/header does not use the label "{label}"'
)


@step("The toggle starts and stops the Syncthing service without shell errors")
def the_toggle_starts_and_stops_the_syncthing_service(context) -> None:
marker = _journal_snapshot_marker()
available, ok, detail = _systemd_unit_start_stop("syncthing.service")
if not available:
print(
"WARNING: syncthing.service not present on this image — "
"soft-skipping start/stop exercise",
flush=True,
)
else:
assert ok, f"Failed to start/stop syncthing.service cleanly: {detail}"
_assert_no_new_shell_errors(marker, extra_pattern=re.escape(SYNCTHING_TOGGLE_UUID))


@step('The extension honors "{setting}"')
def the_extension_honors_setting(context, setting: str) -> None:
match = re.match(r"\s*([\w-]+)\s*=\s*(\S+)\s*", setting)
assert match, f"Could not parse setting expression: {setting!r}"
key, expected = match.group(1), match.group(2)
value, schema = _extension_gsettings_value(SYNCTHING_TOGGLE_UUID, key)
if value is None:
print(
f"WARNING: could not resolve gsettings schema for {SYNCTHING_TOGGLE_UUID} "
f"key {key!r} — soft pass (schema id is not standardized for community extensions)",
flush=True,
)
return
assert value.strip().lower() == expected.lower(), (
f"{schema} {key} is {value!r}, expected {expected!r}"
)


# ---------------------------------------------------------------------------
# Bluetooth Battery Meter
# ---------------------------------------------------------------------------
Expand Down
91 changes: 5 additions & 86 deletions tests/unit/test_bluefin_new_extensions_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,17 +71,16 @@ def _ctx(**attrs):
# ---------------------------------------------------------------------------

class TestConstants:
def test_seven_extension_uuids_defined(self, mod):
def test_six_extension_uuids_defined(self, mod):
uuids = {
mod.COPYOUS_UUID,
mod.SYNCTHING_TOGGLE_UUID,
mod.BLUETOOTH_BATTERY_METER_UUID,
mod.AUDIO_HIDER_UUID,
mod.AUDIO_RENAMER_UUID,
mod.TILING_ASSISTANT_UUID,
mod.TAILSCALE_UUID,
}
assert len(uuids) == 7
assert len(uuids) == 6
assert mod.COPYOUS_UUID == "copyous@boerdereinar.dev"


Expand Down Expand Up @@ -197,36 +196,6 @@ def test_returns_none_when_no_schema_resolves(self, mod):
assert schema is None


class TestTheExtensionHonorsSetting:
def test_soft_passes_when_schema_unresolvable(self, mod, capsys):
context = _ctx()
with patch.object(mod, "_extension_gsettings_value", return_value=(None, None)):
mod.the_extension_honors_setting(context, "start-stop-only = true")
assert "WARNING" in capsys.readouterr().out

def test_passes_when_value_matches(self, mod):
context = _ctx()
with patch.object(
mod, "_extension_gsettings_value",
return_value=("true", "org.gnome.shell.extensions.syncthing-toggle"),
):
mod.the_extension_honors_setting(context, "start-stop-only = true")

def test_raises_when_value_differs(self, mod):
context = _ctx()
with patch.object(
mod, "_extension_gsettings_value",
return_value=("false", "org.gnome.shell.extensions.syncthing-toggle"),
):
with pytest.raises(AssertionError, match="expected 'true'"):
mod.the_extension_honors_setting(context, "start-stop-only = true")

def test_rejects_unparsable_setting_expression(self, mod):
context = _ctx()
with pytest.raises(AssertionError, match="Could not parse"):
mod.the_extension_honors_setting(context, "not a key value pair")


class TestSymbolicIndicatorColor:
def test_strips_gvariant_type_prefix_before_comparing(self, mod):
context = _ctx()
Expand All @@ -246,69 +215,19 @@ def test_raises_when_value_mismatches(self, mod):
mod.the_extension_uses_symbolic_indicator_color(context, "level-indicator-color = 0")


# ---------------------------------------------------------------------------
# systemd unit helper
# ---------------------------------------------------------------------------

class TestSystemdUnitStartStop:
def test_unit_not_found_reports_unavailable(self, mod):
with patch.object(mod, "_run_host", return_value=("LoadState=not-found", 0, "")):
available, ok, _detail = mod._systemd_unit_start_stop("syncthing.service")
assert available is False
assert ok is False

def test_available_unit_starts_and_stops_successfully(self, mod):
responses = iter([
("LoadState=loaded", 0, ""),
("", 0, ""),
("", 0, ""),
])
with patch.object(mod, "_run_host", side_effect=lambda *a, **kw: next(responses)), \
patch.object(mod.time, "sleep"):
available, ok, _detail = mod._systemd_unit_start_stop("syncthing.service")
assert available is True
assert ok is True

def test_available_unit_reports_failure_detail(self, mod):
responses = iter([
("LoadState=loaded", 0, ""),
("", 1, "start failed"),
("", 0, ""),
])
with patch.object(mod, "_run_host", side_effect=lambda *a, **kw: next(responses)), \
patch.object(mod.time, "sleep"):
available, ok, detail = mod._systemd_unit_start_stop("syncthing.service")
assert available is True
assert ok is False
assert detail == "start failed"


# ---------------------------------------------------------------------------
# Quick Settings text scan
# ---------------------------------------------------------------------------

class TestQuickSettingsContainsText:
def test_returns_true_when_eval_bool_true(self, mod):
with patch.object(mod, "_eval_bool", return_value=True) as eval_bool:
assert mod._quick_settings_contains_text("Sync Folder") is True
assert "Sync Folder" in eval_bool.call_args[0][0]
assert mod._quick_settings_contains_text("Tailscale") is True
assert "Tailscale" in eval_bool.call_args[0][0]

def test_returns_false_when_eval_bool_raises(self, mod):
with patch.object(mod, "_eval_bool", side_effect=AssertionError("boom")):
assert mod._quick_settings_contains_text("Sync Folder") is False


class TestQuickSettingsContainsALabeledToggle:
def test_passes_when_present(self, mod):
context = _ctx()
with patch.object(mod, "_quick_settings_contains_text", return_value=True):
mod.quick_settings_contains_a_toggle_labeled(context, "Sync Folder")

def test_raises_when_absent(self, mod):
context = _ctx()
with patch.object(mod, "_quick_settings_contains_text", return_value=False):
with pytest.raises(AssertionError, match="Sync Folder"):
mod.quick_settings_contains_a_toggle_labeled(context, "Sync Folder")
assert mod._quick_settings_contains_text("Tailscale") is False


# ---------------------------------------------------------------------------
Expand Down
Loading