Skip to content

feat(e2e): record unrequired startup services in the startup lane - #1074

Merged
hivecommons-hive[bot] merged 1 commit into
mainfrom
scanner/fix-1025
Oct 31, 2026
Merged

hivecommons-hive[bot] merged 1 commit into
mainfrom
scanner/fix-1025

Conversation

@hivecommons-hive

@hivecommons-hive hivecommons-hive Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Summary: The first-boot test run only watches the setup steps we already know an image turns on. For the current Bluefin image that is just two steps, so we have no record of whether it also sets up Homebrew, the firewall or automatic container updates. This makes the run also look at those other steps and write down what it saw, without letting them pass or fail the run. That gives the evidence needed to decide which steps each image must be held to.


Why

The startup lane (#1051) requires only the units the inventory lists for a profile. The dakota entry is deliberately narrow (NetworkManager-wait-online, flatpak-preinstall) and the reference doc says to add brew-setup or the podman-auto-update user units "once a run shows dakota ships them". Nothing in the lane could produce that evidence: unlisted units were neither masked nor collected, so they ran unseen.

How

  • startup_lane.py: new observed_units() — non-deferred startup_contract units the profile does not list. plan appends them (after the required units) to SYSTEM_UNITS/USER_UNITS and their produces paths to PRODUCES, so the existing collector and journal dump pick them up with no workflow change. evaluate adds one observed row per unit: not shipped (not-found), no state, or "shipped, and as a required unit would be pass/fail/pending: …" plus produced-path presence. observed is outside the verdict, so the gate and the negative control are unchanged. Deferred units (greenboot) stay masked and are never observed.
  • Tests: plan contents for bluefin/dakota, observed ∩ required/deferred = ∅, a shipped brew-setup on dakota is recorded as would-pass with paths, a failed observed unit leaves the verdict green, required units never appear as observed. Without the change these fail (observed_units/OBSERVED missing, plan lists differ).
  • Inventory header and the skill reference document the rule: add a profile only when an observed row shows the unit shipped and would pass.

This PR touches scripts/startup_lane.py, so the lane's own PR trigger will produce a hosted dakota run whose summary carries the first observed rows.

Fixes #1025 — remaining: promoting observed units into profiles from hosted evidence, re-enabling the pending Flatpak/brew scenarios on lane evidence, greenboot boot-counting (#706), user-session first-login paths (#1026), and release wiring (#1021/#1022, maintainer decision).

— hive: agent=scanner backend=copilot model=claude-opus-5.5 copilot=1.0.88 requested_by=@castrojo

Collect every non-deferred startup_contract unit a profile does not require and report it as observed, without changing the verdict, so a hosted run shows whether e.g. dakota ships brew-setup or the podman-auto-update user units before the inventory requires them.

Refs #1025

Assisted-by: Claude via GitHub Copilot CLI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author

hive/awaiting-ci: CI pending — sweep will merge when green.


🐝 Hive Agent: scanner | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: 74a1ebe00051a45c647c2a3f32dae84814e1796b

— hive: agent=scanner backend=copilot model=claude-opus-5.5 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot added author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 31, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me. Read scripts/startup_lane.py, scripts/startup_lane_collect.sh and .github/workflows/startup-qualification.yml at merge-base to check the "never changes the verdict" claim — it holds.

Why the verdict is safe

  • Report.verdict (scripts/startup_lane.py:75-81) only looks for FAIL and PENDING in the status set; OBSERVED is a new distinct string, so it can never flip the gate or the negative control.
  • Produced-path rows are only emitted inside the required_units loop (startup_lane.py:425-427); describe_observed folds path presence into the detail text instead, so an observed unit's missing path cannot become its own FAIL row. test_dakota_records_shipped_brew_setup_without_requiring_it pins that.
  • STARTUP_DEADLINE_S and CHECKS still derive from required only, so polling duration and the flatpak check are unchanged.

The one new runtime path I checked: dakota's USER_UNITS goes from empty to podman-auto-update.service podman-auto-update.timer, so systemctl --user show (startup_lane_collect.sh:27) now runs on dakota for the first time. If the ssh user has no user bus there, the error text is captured via 2>&1 under set -u (no -e), parse_units (startup_lane.py:234-239) drops any block without Id=, and the unit lands as "no state collected" — which the new test asserts for podman-auto-update.timer. The journal step at startup-qualification.yml:251-254 already has || true. So no new failure mode.

Issue link: the body says Fixes #1025 and then lists what remains (promoting observed units, pending Flatpak/brew scenarios, #706, #1026, #1021/#1022). #1025 has six acceptance boxes and is labelled hive/partially-addressed; on merge, Fixes will close it with most boxes unticked. If the intent is for #1025 to stay open as the tracker, this should be Refs #1025 — a maintainer call, not a code defect. Not blocking.

Confidence: 5/5 (safe)

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 6c2075a into main Oct 31, 2026
8 checks passed
@hivecommons-hive

Copy link
Copy Markdown
Contributor Author
Evidence bundle `#1074f0a94e5` sha256:2089d5ab860f7eaa6e20e6a17a8c64fa786a0607df0bd9902f63ca9deecdb60b — download from the dashboard or `hivectl review evidence #1074`.

@hivecommons-hive hivecommons-hive Bot removed role/trusted Author is trusted for unattended merge (trusted-author role or registry/claim tier in require_tiers) stage/050-review-pending Review swarm requested; awaiting a verdict for the current head labels Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author/hive This repository's own hive (the App or one of its agent lanes) authored this PR dco All commits are DCO signed-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(e2e): exercise shipped startup services and system Flatpak initialization in GHA

1 participant