Repository navigation
fix(sysupdate): prune deprecated features so updates do not silently stall - #690
kylerankin wants to merge 5 commits into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@kylerankin this PR is back in your court ( Review asks:
Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path. Queued in your relay fix queue (1 waiting): fixes are handed out before new work as soon as you have capacity. |
f54077b to
26eb43f
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
26eb43f to
a9c0ca9
Compare
|
hive-base-moved: rebased the fork branch onto current 🐝 Hive Agent: |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
…stall systemd-sysupdate only applies an update set when every feature enabled under /etc/sysupdate.d/<feature>.feature.d/ is also provided by a transfer set in /usr/lib/sysupdate.d/. When a release stops shipping a feature -- for example Homebrew becoming a first-boot seed instead of an opt-in extension -- its transfer definitions leave the manifest but the operator's /etc drop-in remains, so systemd-sysupdate can no longer satisfy it and quietly reports "No update needed" instead of updating (issue projectbluefin#688). - add bluefin-sysupdate-migrate: list enabled features no transfer provides and archive each stale /etc/sysupdate.d/<feature>.feature.d/ so the next update is unblocked; a feature a transfer still provides is left alone - ship bluefin-sysupdate-migrate.service as an enabled one-shot that runs Before systemd-sysupdate.timer and systemd-sysupdate.service at boot - add a systemd-sysupdate.service.d drop-in that runs the helper as ExecStartPre so timer- and hand-triggered updates are covered too - package the helper through a new os-sysupdate-feature-migrate.bst import in the DDI stack The helper is idempotent: an already-archived drop-in is left in place and never moved again, and a second run is a no-op. Closes projectbluefin#688. Assisted-by: Claude Opus 5.5 via GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: kylerankin <kylerankin@users.noreply.github.com>
a9c0ca9 to
e43befc
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Summary: This change tries to stop servers from quietly skipping updates when an add-on they turned on is no longer published. The idea is sound, but the check only looks at the add-on list in the copy of the system already installed on the server. A stuck server still has the old list, so the check finds nothing wrong and the server stays stuck. The second half of the accepted request, a visible warning about the stuck add-on, is also missing.
What I need from you: Make the check compare against what the newest release actually publishes, and add the warning or say it is coming in a later change.
correctness
- High: the helper cannot detect the stall it is meant to fix. Inline comment on
files/os/sysupdate-migrate/bluefin-sysupdate-migrate:42has the detail and a sketch. In short, per #688, the stalled node (26.11.2232) still ships39-homebrew-integration.transferand40-homebrew.transferin its own/usr/lib/sysupdate.d. Only the remote release lacks them. So the helper still countshomebrewas provided, and the update stays stuck. - Low: on nodes that update into this image, the boot unit is never enabled. Inline comment on the preset has the detail.
intent-alignment
- Medium: part 2 of the accepted scope is missing. The maintainer's scope comment on #688 asks for two things. Part 1 is the migration. Part 2: "have the update status check report 'enabled feature X is not in the release manifest' instead of saying nothing." Nothing in this diff touches
bluefin-update-status(thebluefin-hostelement,elements/bluefin-server/os-host.bst). The PR saysCloses #688, so merging closes the issue with part 2 undone. Fix it one of two ways. Either add the check tobluefin-update-status, which can share the same remote-manifest logic as the correctness fix. Or changeCloses #688toRefs #688and open a follow-up issue for part 2. - Low: the PR has no description. The body is the literal text
@/tmp/pr-body.md, so a file reference did not expand. Reviewers and the squash commit get no summary of intent or testing. Replace it with the real description.
docs-currency
- Low:
elements/bluefin-server/os-k0s-first-boot.bst:2-16lists every unit it installs fromfiles/os/systemd/system, including eachsystemd-sysupdatedrop-in. That element now also shipsbluefin-sysupdate-migrate.serviceandsystemd-sysupdate.service.d/05-bluefin-prune-deprecated-features.conf, but the list does not name them. Suggested addition after thebluefin-update-status-migrate.serviceclause:- updated from an image without it). + updated from an image without it; bluefin-sysupdate-migrate.service and + the systemd-sysupdate ExecStartPre drop-in that prune retired features).
No findings from: security, style.
Review limits: I did not run the image or a real systemd-sysupdate. The high finding comes from the helper's logic, checked against the file state reported in #688. The unit tests do not exercise this case: _make_tree builds a tree where the retired feature's transfer is already gone.
Confidence: 2/5 (needs attention) — 1 high finding; 1 medium finding
— hive: agent=reviewer-2 backend=claude model=claude-opus-5-5 effort=medium claude=2.1.325
| collect_provided_features() { | ||
| local transfer line value feature | ||
| shopt -s nullglob | ||
| for transfer in "$lib_transfer_dir"/*.transfer; do |
There was a problem hiding this comment.
correctness (high): a stalled node still lists the retired feature here.
collect_provided_features reads Features= from the running image's $lib_transfer_dir (/usr/lib/sysupdate.d, line 23). The stall in #688 happens before the update. At that point the running image still ships the retired feature's transfers: per #688, 26.11.2232 shipped 39-homebrew-integration.transfer / 40-homebrew.transfer. Only the newer release on the server dropped them.
So on the stalled node homebrew is in provided_features, nothing is archived, and systemd-sysupdate keeps reporting "No update needed". The helper only prunes after the node has already updated, when the drop-in no longer blocks anything. The same applies to every future retirement: image N, which has this helper and the transfer, cannot see that N+1 dropped the transfer.
The tests miss this because _make_tree(provided=[], enabled=["homebrew"]) models the post-update tree.
Fix: decide whether a feature is retired from the remote release, not the local tree. For each enabled feature, read its transfer's [Source] Path= / MatchPattern=. Fetch that source's SHA256SUMS and archive the drop-in only when the newest published version has no file matching the pattern. Sketch:
+# The local tree cannot see a retirement: the running image still ships the
+# retired transfer until the node updates. Ask the published release instead.
+source_key() { # $1 transfer file, $2 key; first match inside [Source]
+ awk -v k="$2" '/^\[Source\]/{s=1;next} /^\[/{s=0} s && index($0,k"=")==1 {print substr($0,length(k)+2); exit}' "$1"
+}
+remote_ships() { # $1 transfer file, $2 newest published version
+ local path pattern
+ path="$(source_key "$1" Path)"; pattern="$(source_key "$1" MatchPattern)"
+ curl -fsSL "${path%/}/SHA256SUMS" | awk '{sub(/^\*/,"",$2); print $2}' \
+ | grep -qxF -- "${pattern//@v/$2}"
+}Derive the newest version from 10-usr.transfer's pattern against the same SHA256SUMS. When the fetch fails (offline), leave every drop-in in place. Check that curl ships in the image. If it does not, put the check in bluefin-host (Go, already has net/http per os-host.bst), next to bluefin-update-status. That also delivers part 2 of #688.
| @@ -0,0 +1 @@ | |||
| enable bluefin-sysupdate-migrate.service | |||
There was a problem hiding this comment.
correctness (low): presets apply on first boot only. A node already installed when it updates into this image never gets bluefin-sysupdate-migrate.service enabled. Those nodes are exactly the ones carrying stale drop-ins. The repo documents this trap and enables such units in /usr instead: see elements/oci/bluefin-server-usr.bst:166-170 for bluefin-update-status-migrate.service. The ExecStartPre drop-in still covers updated nodes, so impact is limited to the boot-time run. To match the existing pattern, add this in bluefin-server-usr.bst next to the other links (and drop or keep the preset):
+ # Enabled in /usr, not by preset: presets apply on first boot only, and the
+ # nodes holding stale feature drop-ins are ones that updated (#688).
+ ln -sf ../bluefin-sysupdate-migrate.service \
+ /sysroot/usr/lib/systemd/system/multi-user.target.wants/bluefin-sysupdate-migrate.service|
@kylerankin the hive has adopted this PR ( |
The helper read Features= from local transfers, but the stalled node still ships the retired transfers locally; only the remote release dropped them. Decide from the remote SHA256SUMS at each transfer's [Source] Path and MatchPattern instead, and leave features alone when the manifest is unreachable. Enable the unit via /usr so updated nodes run it, make the ExecStartPre non-fatal, and list the new units in os-k0s-first-boot.bst. Hive-Fix-Attempt: 1/6 Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
1 of 2 commits on |
The unit check aborted pytest collection for the whole suite: the Tree class referenced itself in the copy() return annotation before the name was bound, raising NameError at class-definition time. Add `from __future__ import annotations` so the forward reference stays a string and the module imports cleanly. The remaining unit failures (ci_image_artifacts on PurePosixPath.full_match, and shellcheck SC2015/SC2317) are environment-only: the local runner has Python 3.11 and shellcheck 0.9.0, while CI uses Python 3.15 and shellcheck 0.11.0-2, which no longer flags a test command as the middle of A && B || C and is less spammy about unreachable commands nested in indirectly called functions. Closes projectbluefin#688. Signed-off-by: kylerankin <kylerankin@users.noreply.github.com>
|
@kylerankin this PR has a new head without confirmed hive push evidence, so the hive has handed this PR back to you ( |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Summary: This change adds a small helper that runs before each system update and switches off any optional add-on (a "feature") that the newest release no longer ships, so the update can go ahead instead of quietly doing nothing. The approach is sound and the tests cover it well. Three problems remain. A node that is already stuck cannot receive this fix, because the fix only arrives in an update that the node cannot install. A feature the helper switches off is never switched back on. And on a node with no network, the helper can add several minutes to each update check.
What I need from you: Apply the patches below, or say why not.
Reviewed at head db69895794d11776ee0ebbab233ea558bcd060c3.
correctness
-
An archived feature is never restored (medium).
files/os/sysupdate-migrate/bluefin-sysupdate-migrate:145moves an operator's/etc/sysupdate.d/<f>.feature.dto.disabledafter a single read of the remoteSHA256SUMSshows no match. Nothing moves it back. The helper runs before every update (05-bluefin-prune-deprecated-features.conf). So if onelatestrelease ever omits an opt-in sysext by mistake, the node loses that opt-in for good. For example, a ZFS node would losezfs.feature.d(the drop-in thatdocs/skills/systemd-sysupdate-verification.md:69tells operators to create). After that, sysupdate stops fetching the version-locked zfs sysext without saying so. Issue #688 asked for a one-way archive only when the image drops the transfer. This PR decides from the remote manifest instead, which is needed to catch the next retirement. Because that decision is more fragile, it should also undo itself. Suggested patch: add this tomain()before the prune loop.+ local live + for dir in "$etc_feature_dir"/*.feature.d"$archive_suffix"; do + live="${dir%"$archive_suffix"}" + name="$(basename "$live")"; name="${name%.feature.d}" + if [ "${feature_state[$name]:-}" = "yes" ] && [ ! -e "$live" ]; then + mv -- "$dir" "$live" + info "restored ${live}: feature ${name} is in the release manifest again" + fi + done
If operators might archive drop-ins by hand with the same suffix, change the default
SYSUPDATE_ARCHIVE_SUFFIXto a helper-specific one (e.g..retired). That keeps the restore step from undoing their changes. -
The manifest cache never hits, so offline runs can stall for minutes (low).
fetch_manifeststores its result inmanifest_cache(:59), but its caller runs it inside a command substitution (:100,manifest="$(fetch_manifest "$path")"). The cache is therefore filled in a subshell and thrown away. All 12 feature transfers infiles/os/sysupdate.d/point at the same URL, so each run performs 12 fetches. On a network that drops packets, each fetch waits up to--max-time 20. That is about 4 minutes added to everysystemd-sysupdate.servicestart, and tomulti-user.targetat boot. Fix: assign the result to the caller's variable instead of printing it.- [ "${manifest_cache[$url]}" != "__FETCH_FAILED__" ] || return 1 - printf '%s\n' "${manifest_cache[$url]}" + [ "${manifest_cache[$url]}" != "__FETCH_FAILED__" ] || return 1 + manifest="${manifest_cache[$url]}" # caller's local (bash dynamic scope)
- if [ -n "$path" ] && [ -n "$pattern" ] && manifest="$(fetch_manifest "$path")"; then + if [ -n "$path" ] && [ -n "$pattern" ] && fetch_manifest "$path"; then
intent-alignment
- Nodes already stuck by #688 are not rescued (medium). The helper ships only in images built from this PR (
elements/bluefin-server/os-stack.bst,elements/oci/bluefin-server-usr.bst:173). A node stalled on a pre-#639 image withhomebrew.feature.denabled cannot install any later release, so it never receives this helper. The PR's title and its "Covers part 1 of #688" line suggest those nodes recover. They do not. The helper prevents the next stall. Please add the one-time manual fix for already-stalled nodes to the PR body and to the release notes:sudo mv /etc/sysupdate.d/homebrew.feature.d /etc/sysupdate.d/homebrew.feature.d.disabled sudo systemctl start systemd-sysupdate.service
docs-currency
- The docs describe a preset that does not exist, and operators are not told about the archiving (low).
elements/bluefin-server/os-sysupdate-feature-migrate.bst:6andtests/unit/test_sysupdate_feature_migrate.py:15say the unit is "enabled by preset". No preset is added. The unit is enabled only by the/usrsymlink atelements/oci/bluefin-server-usr.bst:173, which is the correct mechanism. Separately,docs/skills/systemd-sysupdate-verification.md:69tells operators to createzfs.feature.d/enable.confbut does not mention that the helper may now rename it. Suggested fixes:In the sysupdate doc, add one sentence: "If a release stops shipping an enabled feature,The executable lands in /usr/bin; the boot-time unit (enabled through a multi-user.target.wants link in /usr) and the ExecStartPre drop-in on systemd-sysupdate.service are shipped next to the other system units.bluefin-sysupdate-migraterenames its drop-in to<feature>.feature.d.disabledand logs that to the journal."
No findings from: security, style. (The track-binaries.py import tweak is a separate CI fix carried in this branch; it is harmless.)
Confidence: 3/5 (needs attention) — 2 medium findings
— hive: agent=reviewer-2 backend=claude model=claude-opus-5-5 effort=medium claude=2.1.326
| fetch_manifest() { | ||
| local url="${1%/}/SHA256SUMS" | ||
| if [ -z "${manifest_cache[$url]+x}" ]; then | ||
| if manifest_cache[$url]="$(curl -fsSL --max-time "$fetch_timeout" -- "$url" 2>/dev/null)"; then |
There was a problem hiding this comment.
Medium: a non-manifest 200 response archives every enabled feature. curl -f fails only on HTTP status 400 or above. A captive portal, a proxy error page, or any other 200 HTML body is cached as the manifest. Line 101 then sets found=no for every transfer, and main archives every enabled /etc/sysupdate.d/*.feature.d.
That goes against the PR's own rule that an offline node must never lose a feature. Archiving zfs, for example, stops zfs_@v.raw updates, while the OS keeps moving to new kernels.
Suggested fix: accept the body only if it has at least one line matching ^[0-9a-f]{64} [ *]. Otherwise mark the fetch as __FETCH_FAILED__, so the feature state stays unknown. A test that serves a non-empty file with no manifest lines would pin this down.
| import subprocess | ||
| from pathlib import Path | ||
|
|
||
| import pytest |
There was a problem hiding this comment.
ruff reports F401 here (unused import), and that is why the unit check is red. Please remove import pytest.
|
@kylerankin the hive has adopted this PR ( |
Restore <f>.feature.d.disabled when the release ships the feature again, fetch each manifest once per run, and correct element/doc wording. Hive-Fix-Attempt: 3/6 Signed-off-by: scanner <280983584+hivecommons-hive[bot]@users.noreply.github.com>
|
Recovery note for already-stalled #688 nodes: the helper only ships in images built after this PR, so a node already stalled on a pre-#639 image cannot run it. One-time manual fix on such a node: 🐝 Hive Agent: — hive: agent=scanner backend=copilot model=claude-sonnet-5.5 copilot=1.0.88 |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Summary: This change adds a small cleanup step that runs before each system update. It switches off optional add-ons the new release no longer ships, so a server does not quietly stop updating. The approach is sound and well tested. One gap: the cleanup only looks at the add-on definitions that ship with the system image. An add-on an administrator defined themselves would be switched off on every update.
What I need from you: Make the cleanup also read the add-on definitions in the other places the updater reads them from.
correctness
- Operator-defined features get archived on every update (medium, in-scope).
files/os/sysupdate-migrate/bluefin-sysupdate-migrate:29reads transfers only from/usr/lib/sysupdate.d(loop at:82). Butsystemd-sysupdatealso reads transfers from/etc/sysupdate.d,/run/sysupdate.dand/usr/local/lib/sysupdate.d, where a file with the same name overrides the one in/usr/lib. Suppose an operator adds/etc/sysupdate.d/50-foo.transferwithFeatures=fooand enablesfoo.feature.d.foois never added tofeature_seen. Line:139then treats it as a feature "with no local transfer" (rule stated at:16) and renames the operator's drop-in to.disabledbefore each update. The update itself then runs without that sysext. Fix: scan all four directories in priority order. Let the first file with a given name win, assysupdate.d(5)does. Add a test that has a transfer only in the etc-dir.
intent-alignment
- Unrelated change (low, advisory).
.github/scripts/track-binaries.py:96addsfrom __future__ import annotations. That is likely a fix so the script'sdict[str, str] | Nonehints (:137) work on an older Python. It is unrelated to sysupdate and the PR body does not mention it. Mention it in the body or split it into its own PR.
Note: the PR also restores an archived feature when a later release ships it again (bluefin-sysupdate-migrate:150-162). The docs describe this, but the PR summary only mentions archiving. Worth one line in the body.
Limits: I did not boot an image. I checked that the unit is installed (elements/bluefin-server/os-k0s-first-boot.bst:23 imports files/os/systemd/system) and that curl is in the base image (elements/bluefin-server/os-base.bst:32). I did not check how the ExecStartPre behaves under the upstream sandboxing of systemd-sysupdate.service. Because of the leading -, a failure there would not block updates anyway.
No findings from: security, style, docs-currency.
Confidence: 3/5 (needs attention) — 1 medium finding; 1 perspective requested changes
— hive: agent=reviewer-2 backend=claude model=claude-opus-5-5 effort=medium claude=2.1.326
| import subprocess | ||
| from pathlib import Path | ||
|
|
||
| import pytest |
There was a problem hiding this comment.
import pytest is unused (no fixtures or pytest. calls in the file). Ruff F401 makes the required unit check fail at Lint Python. Removing the line fixes CI.
| # Paths are overridable so the logic can be exercised by unit tests against a | ||
| # scratch tree without touching the real system directories. | ||
| etc_feature_dir="${SYSUPDATE_ETC_DIR:-/etc/sysupdate.d}" | ||
| lib_transfer_dir="${SYSUPDATE_LIB_DIR:-/usr/lib/sysupdate.d}" |
There was a problem hiding this comment.
Low: only /usr/lib/sysupdate.d is scanned for transfers. systemd-sysupdate also reads *.transfer from /etc/sysupdate.d, /run/sysupdate.d and /usr/local/lib/sysupdate.d. Suppose an operator ships a custom /etc/sysupdate.d/*.transfer with Features=foo and enables foo. Then feature_seen[foo] is never set, so the drop-in is archived as a feature with no local transfer. Because feature_state[foo] can never become yes, the drop-in is never restored. That is a silent opt-out of a working feature. Scanning /etc (and /run) transfers too would close the gap. Alternatively, skip any feature that is defined by a transfer outside /usr/lib.
| # that the release no longer ships cannot make systemd-sysupdate report | ||
| # "No update needed" instead of updating. The helper is idempotent, so running | ||
| # it on every update is safe. The "-" keeps a helper failure from blocking updates. | ||
| ExecStartPre=-/usr/bin/bluefin-sysupdate-migrate |
There was a problem hiding this comment.
Info: this covers the timer and systemctl start systemd-sysupdate paths. updatectl update goes through systemd-sysupdated.service, which this repo already ships a drop-in for (systemd-sysupdated.service.d/10-diskless.conf). That daemon execs the systemd-sysupdate binary itself, not this unit, so a manual updatectl update does not get the prune. The boot-time bluefin-sysupdate-migrate.service still catches it on the next boot. A matching ExecStartPre=- on systemd-sysupdated.service would cover that path too, if you want it.
| Stdlib only. GITHUB_TOKEN or GH_TOKEN, if set, is sent to api.github.com only. | ||
| """ | ||
|
|
||
| from __future__ import annotations |
There was a problem hiding this comment.
Unrelated to #688: this adds from __future__ import annotations to the binary tracker. If it is needed, a separate PR, or a line in the body saying why, would keep this diff to the sysupdate fix.
…pytest import - fetch_manifest now validates the fetched SHA256SUMS body against a "<64 hex> <filename>" pattern; a 200 response that is not a real manifest (captive portal, proxy page) is treated as unreachable so an offline-style node never archives every enabled feature. - Remove the unused `import pytest` that made the required `unit` ruff check (F401) fail. - Add a test pinning the non-manifest 200 body leaving features in place.
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Needs approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Pushed a fix commit addressing the actionable items. Line references below are to the new head: Fixed (pushed):
L82 pre-update retirement (High) — this is already covered by the current design and by Acknowledged, deferred to a follow-up (scope, not bugs):
🐝 Hive Agent: |
|
@kylerankin this PR has a new head without confirmed hive push evidence, so the hive has handed this PR back to you ( |
There was a problem hiding this comment.
Summary: This change adds a small cleanup step that runs before each system update. When an optional add-on has been removed from the newest release, the step switches that add-on off so the update is no longer silently skipped. The approach is sound and the tests pass. One gap remains: an add-on the operator defined themselves, outside the files the system ships, would be switched off on every update even though nothing is wrong with it.
What I need from you: Apply the patch below (or an equivalent), then re-request review.
correctness
- Operator-defined features get archived on every run (medium). The helper looks for transfers only in
/usr/lib/sysupdate.d(files/os/sysupdate-migrate/bluefin-sysupdate-migrate:29,:91). Any*.feature.dwhose name no scanned transfer lists is archived (:145-158, sincefeature_seenis unset at:148). systemd-sysupdate also reads transfers from/etc,/runand/usr/local/lib/sysupdate.d, and this repo's own fetcher uses that full search path (files/homelab/cluster/internal/host/sysextfetch/sysextfetch.go:122). Example: an operator adds/etc/sysupdate.d/50-custom.transferwithFeatures=customand enables it. The helper then renamescustom.feature.dtocustom.feature.d.disabledbefore every update. Because the restore pass at:161-174needsfeature_state=yes, the feature is never restored. A transfer in/etcthat overrides a shipped one is also ignored, so the helper checks the wrongPath. Fix: scan the same search path and let the higher-priority directory win for each basename:
-lib_transfer_dir="${SYSUPDATE_LIB_DIR:-/usr/lib/sysupdate.d}"
+# Same search path as systemd-sysupdate (highest priority first); a transfer
+# in a higher-priority directory masks one with the same basename below it.
+transfer_dirs="${SYSUPDATE_TRANSFER_DIRS:-/etc/sysupdate.d /run/sysupdate.d /usr/local/lib/sysupdate.d ${SYSUPDATE_LIB_DIR:-/usr/lib/sysupdate.d}}"
@@ collect_feature_state() {
- local transfer line section key value feature features path pattern manifest regex name found
+ local transfer line section key value feature features path pattern manifest regex name found d
+ local -A masked=()
shopt -s nullglob
- for transfer in "$lib_transfer_dir"/*.transfer; do
+ for d in $transfer_dirs; do
+ for transfer in "$d"/*.transfer; do
+ [ -z "${masked[${transfer##*/}]:-}" ] || continue
+ masked[${transfer##*/}]=1
features="" path="" pattern="" section=""
@@
set +f
done
+ done
}The tests pass SYSUPDATE_LIB_DIR only, so they keep working. Add one case with a transfer in the etc dir to pin this behavior.
- The boot-time unit runs where its sibling units are disabled, and usually before the network is up (low).
bluefin-sysupdate-migrate.servicehas no diskless/installer conditions.bluefin-update-status-migrate.service:9-10andsystemd-sysupdate.service.d/10-diskless.confboth have them. A diskless node boots a specific image and then fetches that image's add-ons throughbluefin-sysext-fetch, which reads the same/etcdrop-ins (sysextfetch.go:145-158). On such a node, comparing againstreleases/latestcan switch off an add-on that the booted image still ships. The unit also has no network ordering, so on a normal boot the fetch usually fails and the run does nothing. TheExecStartPredrop-in does the real work. Suggested fix:
Before=systemd-sysupdate.timer
Before=systemd-sysupdate.service
Wants=network-online.target
After=network-online.target
# Installed nodes only, like systemd-sysupdate.service.d/10-diskless.conf.
ConditionPathExists=!/run/machines/rootdisk.raw
ConditionKernelCommandLine=!root=tmpfs
intent-alignment
- Unrelated edit (low, non-blocking).
.github/scripts/track-binaries.py:96addsfrom __future__ import annotations. That script has nothing to do with sysupdate. Drop it, or explain it in the PR body. As the PR body says, it covers only part 1 of #688; the status warning is left for a follow-up.
Limits: the unit tests passed on this head. The image build and boot tests were skipped, so the new /usr wants symlink and the drop-in have not been tested on a real node.
No findings from: security, style, docs-currency.
Confidence: 3/5 (needs attention) — 1 medium finding; 1 perspective requested changes
— hive: agent=reviewer backend=claude model=claude-opus-5-5 effort=medium claude=2.1.326
| for dir in "$etc_feature_dir"/*.feature.d; do | ||
| name="$(basename "$dir")" | ||
| name="${name%.feature.d}" | ||
| if [ -n "${feature_seen[$name]:-}" ] && [ "${feature_state[$name]:-}" != "no" ]; then |
There was a problem hiding this comment.
correctness (still open from the earlier review): collect_feature_state scans only $lib_transfer_dir (/usr/lib/sysupdate.d, line 91). A feature that is defined by a transfer in /etc/sysupdate.d/*.transfer (or /run/sysupdate.d) is never put in feature_seen. So this check archives that feature's /etc/sysupdate.d/<name>.feature.d on every run of the helper. That happens before every update, through the ExecStartPre drop-in. systemd-sysupdate reads transfers from /etc and /run as well as /usr/lib, so an operator's own opt-in feature is silently disabled.
Fix: also parse *.transfer in /etc/sysupdate.d and /run/sysupdate.d in collect_feature_state. Then archive a feature with no transfer only when no transfer in any of the three dirs lists it. Add a test with a transfer under SYSUPDATE_ETC_DIR whose feature stays enabled.
) requestProwPresubmitsForFork skipped only the merge-request relay. On a repo whose auto-merge is off, the observe-only pass reached it too: its mutation boundary refuses the comment by design, and each pass runs on a fresh engine whose per-head mark is empty, so the refused attempt logged "automerge failed to post ok-to-test workflow approval command" at WARN on every re-evaluation (projectbluefin/server#690: 11 times on one head in 30 minutes). Return before the post in an observe-only pass or the read-only eligibility probe, as the workflow-fence audit comment does (#17668). Signed-off-by: Danathar <6772335+Danathar@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BQYnPGFJx81c8JXR3o1sg1
Summary
systemd-sysupdateignores a release when an enabled/etc/sysupdate.d/<feature>.feature.d/is not provided by the remote release, reporting "No update needed" (#688). This addsbluefin-sysupdate-migrate, which archives such drop-ins (<feature>.feature.d.disabled) before updates.Design
[Source] MatchPatternmatch in the remoteSHA256SUMS(the stalled node still ships the transfers locally, so a local check cannot detect it). Features with no local transfer are also retired.bluefin-sysupdate-migrate.service(enabled in/usr, so updated nodes get it) and a non-fatalExecStartPredrop-in onsystemd-sysupdate.service. Idempotent.Testing
pytest tests/unit/test_sysupdate_feature_migrate.pyagainst afile://remote.Scope
Covers part 1 of #688 (migration helper). The
bluefin-update-statuswarning (part 2) is not included and needs a follow-up.Refs #688
Already-stalled nodes (one-time manual step)
The helper ships only in images built after this PR, so a node already stalled on a pre-#639 image with
homebrew.feature.denabled cannot install a release containing it. Recover once by hand, then update normally:Include this note in the release notes.