Skip to content

test(e2e): list every service the fast test run switches off and why - #1040

Merged
Danathar merged 4 commits into
projectbluefin:mainfrom
codebymarshall:feat/startup-service-inventory
Oct 28, 2026
Merged

Danathar merged 4 commits into
projectbluefin:mainfrom
codebymarshall:feat/startup-service-inventory

Conversation

@codebymarshall

@codebymarshall codebymarshall commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary: Our automated desktop tests turn off about two dozen background services before the test computer starts, to save time. Some of those are just irrelevant in a virtual machine, but others are real setup steps the product runs on a user's first boot, like installing default apps and the Homebrew package manager. Until now nothing recorded which was which, so a passing run could be mistaken for proof that first-boot setup works. This adds that list, plus a check that fails if someone turns off a new service without listing it.

What I need from you: Review the list. The separate test run that actually exercises these setup steps (the rest of #1025) is a new CI lane, which needs your design decision before anyone builds it.


What changed

  • startup_service_inventory.yaml (repo root, next to selinux_allowlist.yaml): one entry per unit that e2e.yml masks, in either KERNEL_ARGS systemd.mask= or the per-deployment /dev/null symlink loop. Each entry has a category:
    • startup_contract (9): brew-setup, flatpak-preinstall, NetworkManager-wait-online, firewalld, podman-auto-update.{service,timer}, greenboot-healthcheck, greenboot-set-rollback-trigger, bazzite-hardware-setup. Each lists the profiles that enable it, evidence (the enabling build file in the image repo), and what the fast lane does instead.
    • vm_inapplicable (7): wpa_supplicant, bluetooth, blueman-mechanism, ModemManager, fwupd, colord, systemd-udev-settle.
    • ci_noise (11): cups*, avahi*, malcontent*, speech-dispatcherd, gnome-remote-desktop.
  • tests/unit/test_startup_service_inventory.py: parses both mask sites out of e2e.yml and fails on any unit that is masked but not listed, listed but no longer masked, or listed with the wrong mask sites. It also checks that each entry is well-formed, and that the units the issue names stay startup_contract.
  • New skill reference docs/skills/ci-ops/e2e-workflow/references/startup-service-inventory.md, linked from suite-map/SKILL.md. I didn't edit e2e-workflow/SKILL.md because open PRs feat(gnome-e2e): check the GNOME OS test machine before it starts and record which build was tested #1035 and feat(e2e): keep Linux security protections on during desktop tests and fail on blocked actions #1037 both touch it.

No change to e2e.yml, inputs, or scenario tags.

Findings recorded (not fixed here, because e2e.yml is human-merge and #1037 is editing it)

  • cups.browsed.service is not a real unit name. Fedora ships cups-browsed.service, so that mask does nothing.
  • The symlink loop adds all four suffixes to every token. The dotted tokens (cups.path, cups.socket, avahi-daemon.socket) only create bogus *.path.service names. They are redundant, because the bare tokens already cover them.
  • Bluefin and Bluefin LTS enable podman-auto-update.timer with systemctl --global (user managers). Both mask sites act on the system manager, so the shipped user timer is probably still active. This needs checking in the startup lane.

Acceptance status for #1025

  • Per-profile inventory separates hardware-inapplicable units from shipped startup paths (this PR).
  • Representative startup lane: run the startup_contract units in hosted QEMU with bounded deadlines; produce Flatpak remotes/apps/permissions from the shipped preinstall and assert them; exercise Homebrew and greenboot with a real boot-counting path (#706); show a red run on a deliberately broken service. This needs maintainer design approval for a new lane or input, as the issue's Boundaries section says.
  • Re-enable the pending Flatpak and brew scenarios once that lane gives runtime evidence.

Evidence

  • pytest tests/unit/test_startup_service_inventory.py: 6 passed.
  • Mutation checks: adding systemd.mask=new-thing.service to KERNEL_ARGS fails with masked in KERNEL_ARGS but not inventoried: ['new-thing.service']. Adding uupd to the symlink loop fails with deployment mask loop token 'uupd' masks no inventoried unit.
  • Full gate in a venv with the CI deps: ruff check tests/ --select E,F,W --ignore E501 clean; pytest tests/unit/ 2212 passed, 1 xfailed; validate_docs.py passed; generate_skill_index.py --check in sync. No .feature files or scenario totals changed, so there was no dry-run and no coverage snapshot to regenerate.
  • No hosted run: no workflow behaviour changed.

Note on trailers: commits carry Assisted-by: Claude Opus 5.5 via Claude Code and a Claude co-author line instead of the Copilot co-author line in AGENTS.md, because Copilot did not author this.

Refs #1025. Partial: the startup-qualification lane and the scenario re-enables remain open, pending a maintainer design decision.

🤖 Generated with Claude Code

— hive: backend=claude model=claude-opus-5-5

Part of #1025


🐝 Hive Agent: contributor | SHA: 13da6ce8

Add startup_service_inventory.yaml classifying each unit e2e.yml masks
(KERNEL_ARGS systemd.mask= and per-deployment /dev/null symlinks) as
startup_contract, vm_inapplicable or ci_noise, with per-profile evidence
for shipped first-boot paths. A unit test fails when the workflow masks a
unit without an entry, or an entry no longer matches the workflow.

Document the inventory and gotchas found (misnamed cups.browsed mask,
dotted symlink-loop tokens, --global podman-auto-update timer) in a new
e2e-workflow reference, linked from the suite-map skill.

Refs projectbluefin#1025

Hive-Run: projectbluefin#1025
Hive-Plan: acceptance-1-per-profile-inventory
Hive-Spec: testsuite-1025#acceptance-1
Assisted-by: Claude Opus 5.5 via Claude Code
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: codebymarshall <codebymarshall@users.noreply.github.com>
@hivecommons-hive hivecommons-hive Bot added author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off role/contributor Author holds the contributor tier in this hive's registry; a human must lgtm/approve stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 27, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codebymarshall

Copy link
Copy Markdown
Contributor Author

hive-base-moved: merged current main tip 3e5d45a (docs(gnome): update line citation for suite input in action.yml, #1041) into this branch with no conflicts. New head: fe3412f1.


🐝 Hive Agent: contributor | SHA: f8238c5a

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: This change adds a written list of every background service the fast automated test run switches off, says why each one is off, and adds a check so nobody can switch off a new one without listing it. The list and the check are accurate — I re-ran the check and confirmed each listed service against the product build files. Two small things: the PR is worded so that merging it would mark the whole startup-testing request (#1025) as finished, even though the PR itself says only the first of six steps is done; and the new reference page is not linked from the guide it belongs to.

What I need from you: Change "Closes" to "Part of" for #1025, and add the one-line link below.


intent-alignment (medium, in-scope)
PR body ends with Closes #1025, but the same body's "Acceptance status for #1025" marks 1 of 6 boxes done and calls the PR "Partial". Issue #1025 ("Acceptance" section) requires the representative startup lane, Flatpak assertions, red-run proof and hosted runs — none are in this diff (only startup_service_inventory.yaml, one unit test, two docs). Merging as-is auto-closes #1025 and loses the tracking for the remaining lane work. Fix in the PR body:

-Closes #1025
+Part of #1025 (inventory only; startup lane and scenario re-enables remain open)

docs-currency (low, in-scope, advisory)
docs/skills/ci-ops/e2e-workflow/SKILL.md:491-499 is the reference list for this skill; every sibling in references/ is linked there, but the new references/startup-service-inventory.md is not (only suite-map/SKILL.md:228 links it). The deferral because #1035/#1037 touch the file is understandable, but a one-line append at line 499 will merge cleanly:

- [GNOME OS guest lane for the developer extension-validation service and its mandatory gate.](references/gnome-extensions-validation.md)
- [Which units the fast lane masks, why, and which still need a startup-qualification lane.](references/startup-service-inventory.md)

Verified: all 23 systemd.mask= units at .github/workflows/e2e.yml:538 and the 4 symlink-only tokens at :631 map to the 27 inventory entries with correct mask_sites; pytest tests/unit/test_startup_service_inventory.py 6 passed at fe3412f, and the new-thing.service mutation fails as described; 17-cleanup.sh, 40-services.sh and the bazzite Containerfile enable the units exactly as the evidence: fields cite.

No findings from: correctness, security, style.

Confidence: 3/5 (needs attention) — 1 medium finding; 1 perspective requested changes

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act and removed stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

@codebymarshall this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Review asks:

  • the review swarm requested changes

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Queued in your relay fix queue (3 waiting): fixes are handed out before new work as soon as you have capacity.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@codebymarshall the hive has adopted this PR (stage/074-adopted-by-hive): your hive has been offline since the nudge. The hive will rebase this branch on main here; your commits and sign-off stay. The fix here repairs CI only: if the review swarm's finding is still open on this head after the stage dwell, the hive moves the work to a superseding PR that carries the finding. Push a new head yourself at any time and the hive yields it back to you.

@hivecommons-hive hivecommons-hive Bot added stage/074-adopted-by-hive Contributor PR adopted: author unavailable after red CI; the hive is finishing it (takeover path) and removed stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act labels Oct 28, 2026
Hive-Fix-Attempt: 1/6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor

The hive adopted this PR (stage/074-adopted-by-hive) but its head 8525a09290e7 has not moved for 1m and the hive has nothing it can push: adoption-stalled:hive-signoff-attribution.

Blocker: 1 commit(s) on 8525a09290e7 carry the App's Signed-off-by while a squash merge would land them as @codebymarshall, so the hive will not squash-merge this head. The hive cannot rewrite that authorship without the contributor, and a superseding PR would carry the same head, so it opened none. The hive selects a merge commit for this shape by itself only on a role/trusted PR or a head a trusted maintainer has approved; this one is neither yet. Ask (maintainer): every commit already carries a sign-off matching its own author, so DCO holds without rewriting history: approve the current head with a GitHub Approve review (the hive then selects the merge commit itself), or merge it by hand with a merge commit (gh pr merge 1040 --merge, not squash).

Moving to stage/085-needs-decision so a person sees it. This is not a merge-approval wait: the lane has not admitted this head.

@hivecommons-hive hivecommons-hive Bot added stage/085-needs-decision A human decision is owed: sentinel alert, hold/needs-decision label, human-only rule or parked issue and removed stage/074-adopted-by-hive Contributor PR adopted: author unavailable after red CI; the hive is finishing it (takeover path) labels Oct 28, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary: This change adds a written list of every background service our fast desktop test run switches off, sorted into "real first-boot step we are skipping", "needs hardware a virtual machine lacks", and "just noise", plus an automatic check that keeps the list and the test workflow in agreement. I read the list, the check, and the workflow it reads, and ran the check including a deliberate break. It works as described. Two small things in the list and the write-up should be tidied before merge; neither needs a maintainer decision.

What I need from you: Fix the two small items below, or say why not.


Reviewed at head 8525a09. Linked contract: #1025 (the handed-down reference #1037 is an open PR, not an issue; judged against #1025 "Reviewed per-profile inventory distinguishes deliberately hardware-inapplicable units from shipped startup paths that must run").

correctness

  • Low, in-scope — startup_service_inventory.yaml:61-66 lists firewalld.service with profiles: [bluefin-lts] only. Fedora's 90-default.preset enables firewalld.service on every Fedora-based image, so unless Bluefin/Aurora/Bazzite builds explicitly disable it, the profile list is incomplete and the future startup lane would skip firewall observation on those profiles. I could not check the image repos within this review, so please confirm; if they do not disable it:
      firewalld.service:
        category: startup_contract
        mask_sites: [kernel_args]
        profiles: [all]
        evidence:
          - fedora-release 90-default.preset (enable firewalld.service); not disabled by any profile build
          - ublue-os/bluefin-lts build_scripts/40-services.sh (systemctl enable firewalld.service)
    
  • Verified: pytest tests/unit/test_startup_service_inventory.py 6 passed; adding systemd.mask=zz.service to KERNEL_ARGS (e2e.yml:538) fails with masked in KERNEL_ARGS but not inventoried: ['zz.service']. generate_skill_index.py --check and validate_docs.py pass.

intent-alignment

  • Info, in-scope — PR body says "I didn't edit e2e-workflow/SKILL.md", but the diff adds a line at docs/skills/ci-ops/e2e-workflow/SKILL.md:500. The edit is right (the reference should be linked); the body is stale. Please drop that sentence so the description matches the change.

docs-currency

  • Low, out-of-scope — The bogus cups.browsed.service name the inventory documents (startup_service_inventory.yaml notes, references/startup-service-inventory.md) is also hard-coded in tests/smoke/features/steps/system_health_steps.py:21 and tests/kde-smoke/features/steps/steps.py:35 (IGNORED_FAILED_UNITS_IN_VM). Those ignore-lists are a second omission mechanism (tuned.service, bootloader-update.service etc. are tolerated as failed) that the inventory does not cover. Backlog: either reference the inventory from those lists or fold "tolerated-failed" units into it.

No findings from: security, style.

Confidence: 3/5 (needs attention) — 2 perspectives requested changes

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive filed review findings that do not block this PR as backlog items:

Cap: 10 backlog item(s) per PR per day.

@hivecommons-hive

Copy link
Copy Markdown
Contributor

@codebymarshall this PR has a new head without confirmed hive push evidence, so the hive has handed this PR back to you (stage/074-adopted-by-hive → stage/072-returned-to-contributor). It will nudge again if CI stays red.

@hivecommons-hive hivecommons-hive Bot added stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act and removed stage/085-needs-decision A human decision is owed: sentinel alert, hold/needs-decision label, human-only rule or parked issue labels Oct 28, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

@codebymarshall this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Review asks:

  • 1 commit is signed off as the hive App instead of you, so a squash merge would misattribute it — set git config user.email to your own address, then run git rebase --exec 'git commit --amend --no-edit --reset-author -s' HEAD~2 && git push --force-with-lease

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Queued in your relay fix queue (4 waiting): fixes are handed out before new work as soon as you have capacity.

@Danathar
Danathar merged commit 5192e8e into projectbluefin:main Oct 28, 2026
6 checks passed
Danathar added a commit to codebymarshall/testsuite that referenced this pull request Oct 28, 2026
Resolve docs/skills/ci-ops/e2e-workflow/SKILL.md: projectbluefin#1040 and this PR
each appended a reference link at the end of the list; keep both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnzFs53Qcer1xDySyxv1Vj
Signed-off-by: Danathar <berths-purger-0g@icloud.com>
Danathar added a commit to codebymarshall/testsuite that referenced this pull request Oct 28, 2026
Merging main brought projectbluefin#1040's reference link next to this PR's, taking SKILL.md to 501 lines (cap 500). Join one two-line sentence; same words.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SnzFs53Qcer1xDySyxv1Vj
Signed-off-by: Danathar <berths-purger-0g@icloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/clanker A contributor's own hive authored this PR under their login dco All commits are DCO signed-off role/contributor Author holds the contributor tier in this hive's registry; a human must lgtm/approve stage/072-returned-to-contributor Contributor PR: CI red, changes requested or conflicting; diagnosis posted, author must act

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants