Repository navigation
feat(e2e): record unrequired startup services in the startup lane - #1074
Conversation
Collect every non-deferred startup_contract unit a profile does not require and report it as observed, without changing the verdict, so a hosted run shows whether e.g. dakota ships brew-setup or the podman-auto-update user units before the inventory requires them. Refs #1025 Assisted-by: Claude via GitHub Copilot CLI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
hive/awaiting-ci: CI pending — sweep will merge when green. 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-opus-5.5 copilot=1.0.88 |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Looks correct to me. Read scripts/startup_lane.py, scripts/startup_lane_collect.sh and .github/workflows/startup-qualification.yml at merge-base to check the "never changes the verdict" claim — it holds.
Why the verdict is safe
Report.verdict(scripts/startup_lane.py:75-81) only looks forFAILandPENDINGin the status set;OBSERVEDis a new distinct string, so it can never flip the gate or the negative control.- Produced-path rows are only emitted inside the
required_unitsloop (startup_lane.py:425-427);describe_observedfolds path presence into the detail text instead, so an observed unit's missing path cannot become its ownFAILrow.test_dakota_records_shipped_brew_setup_without_requiring_itpins that. STARTUP_DEADLINE_SandCHECKSstill derive fromrequiredonly, so polling duration and the flatpak check are unchanged.
The one new runtime path I checked: dakota's USER_UNITS goes from empty to podman-auto-update.service podman-auto-update.timer, so systemctl --user show (startup_lane_collect.sh:27) now runs on dakota for the first time. If the ssh user has no user bus there, the error text is captured via 2>&1 under set -u (no -e), parse_units (startup_lane.py:234-239) drops any block without Id=, and the unit lands as "no state collected" — which the new test asserts for podman-auto-update.timer. The journal step at startup-qualification.yml:251-254 already has || true. So no new failure mode.
Issue link: the body says Fixes #1025 and then lists what remains (promoting observed units, pending Flatpak/brew scenarios, #706, #1026, #1021/#1022). #1025 has six acceptance boxes and is labelled hive/partially-addressed; on merge, Fixes will close it with most boxes unticked. If the intent is for #1025 to stay open as the tracker, this should be Refs #1025 — a maintainer call, not a code defect. Not blocking.
Confidence: 5/5 (safe)
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
Summary: The first-boot test run only watches the setup steps we already know an image turns on. For the current Bluefin image that is just two steps, so we have no record of whether it also sets up Homebrew, the firewall or automatic container updates. This makes the run also look at those other steps and write down what it saw, without letting them pass or fail the run. That gives the evidence needed to decide which steps each image must be held to.
Why
The startup lane (#1051) requires only the units the inventory lists for a profile. The dakota entry is deliberately narrow (
NetworkManager-wait-online,flatpak-preinstall) and the reference doc says to addbrew-setupor thepodman-auto-updateuser units "once a run shows dakota ships them". Nothing in the lane could produce that evidence: unlisted units were neither masked nor collected, so they ran unseen.How
startup_lane.py: newobserved_units()— non-deferredstartup_contractunits the profile does not list.planappends them (after the required units) toSYSTEM_UNITS/USER_UNITSand theirproducespaths toPRODUCES, so the existing collector and journal dump pick them up with no workflow change.evaluateadds oneobservedrow per unit: not shipped (not-found), no state, or "shipped, and as a required unit would be pass/fail/pending: …" plus produced-path presence.observedis outside the verdict, so the gate and the negative control are unchanged. Deferred units (greenboot) stay masked and are never observed.observed_units/OBSERVEDmissing, plan lists differ).This PR touches
scripts/startup_lane.py, so the lane's own PR trigger will produce a hosted dakota run whose summary carries the first observed rows.Fixes #1025 — remaining: promoting observed units into profiles from hosted evidence, re-enabling the pending Flatpak/brew scenarios on lane evidence, greenboot boot-counting (#706), user-session first-login paths (#1026), and release wiring (#1021/#1022, maintainer decision).
— hive: agent=scanner backend=copilot model=claude-opus-5.5 copilot=1.0.88 requested_by=@castrojo