diff --git a/docs/RELEASES.md b/docs/RELEASES.md index 2937eaa4d..27941295c 100644 --- a/docs/RELEASES.md +++ b/docs/RELEASES.md @@ -389,7 +389,7 @@ CI never holds the signing key and never sees a factory image. The maintainer do - **Users follow `stable`, builders and testers `development`** (decided, ยง12.9). Release images point `[omamobile]` at `stable` on `pkgs.omamobile.org`: the device package's `omamobile-mirrorlist` becomes that URL instead of a local `file://` path. - **The move from the original key.** The original package key (2026-10-12, `6C84FE3D240709E70E27CC94B301F5EA62B249FE`, `keys/gnupg`) still signs packages, the database and bundles. `omamobile-keyring` 20261026 trusts both keys and is itself signed by the original, so phones accept the release key with an ordinary update. Then: switch the build tools to the release key (`keys/gnupg.release`), and once published phones have that keyring, move the original key to `omamobile-revoked`. `tools/install-phone` accepts either key's signature on a bundle until then. - Rotation and revocation go through `omamobile-keyring`'s `omamobile-trusted` and `omamobile-revoked`, as Arch's keyring does. -- **Publishing** (`tools/repo-publish`): `pkgs.omamobile.org` has two channels, `development/aarch64` and `stable/aarch64`. `repo-publish development` builds what changed (`build-packages --all`), checks every package's checksum and signature against `omamobile-keyring`'s trusted keys, and uploads packages first, then the index, then the pruning; each channel keeps the current and the previous index's files. `repo-publish stable` promotes development's signed files unchanged, after asking. The maintainer's laptop runs it daily from a clean `origin/main` checkout (`--from-main`, `tools/systemd/omamobile-publish.timer`), and `--remind` opens a "Ready for stable" issue when development has differed from stable for three days. Experimental phones' device packages are published to development only. The server account needs the `deployers` group. +- **Publishing** (`tools/repo-publish`): `pkgs.omamobile.org` has two channels, `development/aarch64` and `stable/aarch64`. `repo-publish development` builds what changed (`build-packages --all`), checks every package's checksum and signature against `omamobile-keyring`'s trusted keys, and uploads packages first, then the index, then the pruning; each channel keeps the files of the current index and of the last one before it with a different package set, also after a publish that changed nothing. `repo-publish stable` promotes development's signed files unchanged, after asking. The maintainer's laptop runs it daily from a clean `origin/main` checkout (`--from-main`, `tools/systemd/omamobile-publish.timer`), and `--remind` opens a "Ready for stable" issue when development has differed from stable for three days. Experimental phones' device packages are published to development only. The server account needs the `deployers` group. - Never commit, paste or upload a key, a passphrase or `keys/`. CI gets no key. - **Contributors' builds** (review lesson L-7). A port's own packages are signed with a local key that only that contributor's machine trusts, and everything lands under `out//` and `keys//`, never in the maintainer's `keys/` or `out/repo`: diff --git a/tests/all b/tests/all index 06bb61eef..7417b0fe9 100755 --- a/tests/all +++ b/tests/all @@ -20,8 +20,9 @@ # a scratch server, tools/extract-camera-data's adrp decoding, # tools/phone-smoke's dmesg error classification, the OWL conformance # report's differences on the host, bin/lineage-ref sync's branch handling -# against stand-ins for docker and repo, and tools/repo-publish --from-main's -# copy of the prebuilt binaries into a fixture's publish tree. +# against stand-ins for docker and repo, tools/repo-publish --from-main's +# copy of the prebuilt binaries into a fixture's publish tree, and +# tools/repo-publish's staging. set -euo pipefail cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." for f in tools/install-phone tools/phone-provision tools/nv-backup tools/build-rootfs tools/build-packages tools/make-bundle \ @@ -38,7 +39,7 @@ sh -n device/flame/initramfs/build bash -n tools/build-flame-probe echo "ok - scripts parse" python3 tests/bt-uart-gpio.py -for t in tests/pr-check tests/kernel-pr-check tests/phone-smoke tests/alarm-wake-test tests/install-phone tests/build-rootfs tests/rootfs-resolve tests/phone-provision tests/nv-backup tests/initramfs-grow tests/images-layout tests/boot-update-parity tests/stage-boot-files tests/initramfs-reproducible tests/device-overlays tests/simplefb-trigger tests/initramfs-steps tests/storage-guard tests/usb-product tests/boot-method tests/modules-push tests/make-bundle-keys tests/make-bundle tests/release-signing tests/pinned-packages tests/flame-probe packages/omamobile-firstboot/test/all packages/omamobile-device-felix/test/all packages/omamobile-device-lynx/test/all packages/omamobile-device-panther/test/all tests/local-builds tests/phone-install-package tests/publish-kernel tests/fetch-toolchain tests/release-upload tests/extract-camera-data tests/release tests/lineage-ref tests/repo-publish; do +for t in tests/pr-check tests/kernel-pr-check tests/phone-smoke tests/alarm-wake-test tests/install-phone tests/build-rootfs tests/rootfs-resolve tests/phone-provision tests/nv-backup tests/initramfs-grow tests/images-layout tests/boot-update-parity tests/stage-boot-files tests/initramfs-reproducible tests/device-overlays tests/simplefb-trigger tests/initramfs-steps tests/storage-guard tests/usb-product tests/boot-method tests/modules-push tests/make-bundle-keys tests/make-bundle tests/release-signing tests/pinned-packages tests/flame-probe packages/omamobile-firstboot/test/all packages/omamobile-device-felix/test/all packages/omamobile-device-lynx/test/all packages/omamobile-device-panther/test/all tests/local-builds tests/phone-install-package tests/publish-kernel tests/fetch-toolchain tests/release-upload tests/extract-camera-data tests/release tests/lineage-ref tests/repo-publish tests/repo-publish-stage; do echo "# $t" "$t" done diff --git a/tests/repo-publish-stage b/tests/repo-publish-stage new file mode 100755 index 000000000..8bef0715b --- /dev/null +++ b/tests/repo-publish-stage @@ -0,0 +1,222 @@ +#!/bin/bash +# SPDX-License-Identifier: MIT +# tools/repo-publish's stage() (no keys, no signing, no server): it brings a +# channel directory to a new index and keeps the files of the last index +# that listed a different package set, so a phone can go back one version +# even after a publish that changed nothing, or after a stage that failed +# part way. Fictional packages and indexes only; stage() is taken from the +# script and run on its own. +set -euo pipefail +cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." +fail() { printf 'not ok - %s\n' "$1" >&2; exit 1; } +pass() { printf 'ok - %s\n' "$1"; } +command -v bsdtar >/dev/null || { pass "# skip: no bsdtar"; exit 0; } +t=$(mktemp -d) +trap 'rm -rf "$t"' EXIT + +{ + grep -m1 '^DB=' tools/repo-publish + grep -m1 '^db_files() ' tools/repo-publish + grep -m1 '^pkg_files() ' tools/repo-publish + grep -m1 '^save_sets() ' tools/repo-publish + sed -n '/^stage() {$/,/^}$/p' tools/repo-publish +} > "$t/fn.sh" +if ! grep -q '^stage() {$' "$t/fn.sh" || ! grep -q '^db_files() ' "$t/fn.sh" || ! grep -q '^pkg_files() ' "$t/fn.sh" || + ! grep -q '^save_sets() ' "$t/fn.sh" || ! bash -n "$t/fn.sh"; then + fail "stage() and the helpers it uses are in tools/repo-publish" +fi +# shellcheck source=/dev/null +. "$t/fn.sh" + +# make_repo NAME PACKAGE...: a built repository in $t/NAME, as +# tools/build-packages leaves out/repo: each package with its .sig, and a +# (fictionally signed) index listing them +make_repo() { + local p f sum d=$t/$1 + shift + rm -rf "$d" "$t/db" && mkdir -p "$d" "$t/db" + for p in "$@"; do + echo "fictional $p" > "$d/$p-aarch64.pkg.tar.zst" + echo "fictional signature" > "$d/$p-aarch64.pkg.tar.zst.sig" + sum=$(sha256sum < "$d/$p-aarch64.pkg.tar.zst" | cut -d' ' -f1) + mkdir -p "$t/db/$p" + printf '%%FILENAME%%\n%s\n\n%%SHA256SUM%%\n%s\n' "$p-aarch64.pkg.tar.zst" "$sum" > "$t/db/$p/desc" + done + for f in omamobile.db omamobile.files; do + (cd "$t/db" && bsdtar -czf "$d/$f.tar.gz" -- *) + echo "fictional signature" > "$d/$f.tar.gz.sig" + ln -sfn "$f.tar.gz" "$d/$f" + ln -sfn "$f.tar.gz.sig" "$d/$f.sig" + done +} +ch=$t/channel/aarch64 +# run_stage NAME: stage $t/NAME into the channel, in a shell of its own so +# set -e stops stage() at a failure as in the script (in a caller's `if` +# or `||` it would be ignored). FAULT, if set, is run first: a function +# that stands in for a command and fails. +run_stage() { FAULT=${FAULT:-} bash -c 'set -euo pipefail; . "$1"; eval "$FAULT"; stage "$2" "$3"' _ "$t/fn.sh" "$t/$1" "$ch" > "$t/log" 2>&1; } +# stages NAME WHAT: run_stage NAME, which must succeed +stages() { run_stage "$1" || { cat "$t/log" >&2; fail "$2"; }; } +# The package files (and signatures) wanted, and those the channel has +want() { local p; for p in "$@"; do printf '%s\n' "$p-aarch64.pkg.tar.zst" "$p-aarch64.pkg.tar.zst.sig"; done | sort; } +have() { (cd "$ch" && printf '%s\n' *.pkg.tar.*) | sort; } +# expect WHAT PACKAGE...: the channel holds exactly these packages +expect() { + local what=$1 + shift + diff <(want "$@") <(have) >&2 || fail "$what" + pass "$what" +} +# kept WHAT PACKAGE...: these packages are all still there +kept() { + local what=$1 + shift + [[ -z $(comm -23 <(want "$@") <(have)) ]] || fail "$what" + pass "$what" +} +# expect_index NAME: the channel's index files are NAME's +expect_index() { + diff <(db_files "$t/$1") <(db_files "$ch") >&2 || fail "the channel's index is $1's" + local f s + for f in omamobile.db omamobile.files; do + for s in "" .sig; do + if ! cmp -s "$ch/$f.tar.gz$s" "$t/$1/$f.tar.gz$s" || ! cmp -s "$ch/$f$s" "$t/$1/$f.tar.gz$s"; then + fail "$f$s and $f.tar.gz$s are $1's" + fi + done + done +} + +# v1 -> v2 -> v2 -> v3: base stays, app moves 1 -> 2 -> 3 +make_repo v1 base-1-1 app-1-1 +make_repo v2 base-1-1 app-2-1 +make_repo v3 base-1-1 app-3-1 +stages v1 "stages v1" +expect_index v1 +expect "the first publish holds v1" base-1-1 app-1-1 +stages v2 "stages v2" +expect_index v2 +expect "v2 keeps v1's files" base-1-1 app-1-1 app-2-1 +stages v2 "stages v2 again" +expect_index v2 +expect "publishing v2 again still keeps v1's files" base-1-1 app-1-1 app-2-1 +stages v2 "stages v2 a third time" +expect "and so does a third publish of v2" base-1-1 app-1-1 app-2-1 +stages v3 "stages v3" +expect_index v3 +expect "v3 keeps v2's files and drops v1's" base-1-1 app-2-1 app-3-1 +stages v3 "stages v3 again" +expect "publishing v3 again keeps v2's files" base-1-1 app-2-1 app-3-1 + +# A package removed from the set stays for one version, as the rollback +make_repo v4 base-1-1 +stages v4 "stages v4" +expect_index v4 +expect "a set without app keeps v3's app as the way back" base-1-1 app-3-1 +stages v4 "stages v4 again" +expect "and still after publishing that set again" base-1-1 app-3-1 +make_repo v5 base-2-1 +stages v5 "stages v5" +expect "the next change drops the removed package" base-1-1 base-2-1 + +# Going back to an older set is a change too: the set it replaces stays +stages v4 "stages v4 after v5" +expect_index v4 +expect "staging an older set keeps the newer one it replaces" base-1-1 base-2-1 +stages v5 "stages v5 after v4" +expect_index v5 + +# A file no index lists goes, with its signature, also on a no-op +echo stray > "$ch/stray-1-1-aarch64.pkg.tar.zst" +echo stray > "$ch/stray-1-1-aarch64.pkg.tar.zst.sig" +stages v5 "stages v5 with a stray file" +expect "a no-op drops a file neither set lists" base-1-1 base-2-1 + +# A failed stage (a package or signature the new index lists is missing) +# changes neither the index nor what is kept; the files cp copied before it +# stopped go at the next stage, also when that one changes nothing +make_repo v6 base-2-1 app-6-1 +mv "$t/v6/app-6-1-aarch64.pkg.tar.zst" "$t/app-6-1.held" +if run_stage v6; then fail "refuses an index whose package is missing"; fi +expect_index v5 +kept "a failed stage keeps the index and every file" base-1-1 base-2-1 +stages v5 "stages v5 after the failed v6" +expect_index v5 +expect "the no-op after it drops what the failed stage copied" base-1-1 base-2-1 +mv "$t/v5/base-2-1-aarch64.pkg.tar.zst.sig" "$t/base-2-1.sig.held" +if run_stage v5; then fail "refuses a package without its signature"; fi +expect_index v5 +kept "a failed no-op stage keeps the index and every file" base-1-1 base-2-1 +mv "$t/base-2-1.sig.held" "$t/v5/base-2-1-aarch64.pkg.tar.zst.sig" +mv "$t/app-6-1.held" "$t/v6/app-6-1-aarch64.pkg.tar.zst" +stages v6 "stages v6 once its package is there" +expect_index v6 +expect "the next good stage holds v6 and v5 and nothing else" base-2-1 app-6-1 +extra=$(cd "$ch" && shopt -s nullglob dotglob && printf '%s\n' * | grep -vE '\.pkg\.tar\.|^omamobile\.(db|files)(\.tar\.gz)?(\.sig)?$' || true) +[[ -f $ch.sets && -z $extra ]] || fail "the record is beside the channel, which holds only packages and the index: $extra" +pass "the record of the sets is beside the channel, so it isn't uploaded" +rm -rf "$t/channel" +rm "$t/v6/app-6-1-aarch64.pkg.tar.zst" +if run_stage v6; then fail "refuses v6 in a fresh channel"; fi +[[ ! -e $ch/omamobile.db && ! -s $ch.sets ]] || fail "a failed first stage leaves no index" +pass "a failed first stage leaves no index and an empty record" + +# A stage that fails while copying the index leaves the database on disk +# new and the record as it was: the retry decides from the record, so it +# keeps the set before it. The same when the record can't be replaced. +rm -rf "$t/channel" +stages v1 "stages v1 in a new channel" +stages v2 "stages v2 in a new channel" +mv "$t/v3/omamobile.files.tar.gz" "$t/files.held" +if run_stage v3; then fail "refuses v3 without its .files archive"; fi +diff <(db_files "$t/v3") <(db_files "$ch") >&2 || fail "the failed v3 got as far as the database" +mv "$t/files.held" "$t/v3/omamobile.files.tar.gz" +stages v3 "retries v3" +expect_index v3 +expect "a retry after a failed index copy keeps v2's files" base-1-1 app-2-1 app-3-1 +cp "$ch.sets" "$t/sets.before" +if FAULT='mv() { return 75; }' run_stage v4; then fail "stops when the record can't be replaced"; fi +cmp -s "$ch.sets" "$t/sets.before" || fail "a record that can't be replaced stays as it was" +stages v4 "retries v4" +expect "a retry after a failed record keeps v3's files" base-1-1 app-3-1 +cp "$ch.sets" "$t/sets.before" +# shellcheck disable=SC2016 # FAULT is expanded by the stage's own shell +if FAULT='sed() { [[ $2 != "s/^./current &/p" ]] || return 75; command sed "$@"; }' run_stage v5; then + fail "stops when the record's current set can't be written" +fi +cmp -s "$ch.sets" "$t/sets.before" || fail "a record whose current set can't be written stays as it was" +stages v5 "retries v5" +stages v5 "stages v5 again" +expect "a retry after a half-written record keeps v4's files" base-1-1 base-2-1 +if FAULT='find() { return 1; }' run_stage v4; then fail "stops when the channel can't be listed for pruning"; fi +kept "a stage that can't list the channel deletes nothing" base-1-1 base-2-1 +FAULT='grep() { return 2; }' stages v5 "stages v5 with a failing grep" +expect "a failing grep can't make the prune delete what it keeps" base-1-1 base-2-1 + +# A channel staged before the record existed (v1 and v2's files, v2's +# index) gets one from what it has before anything is copied: a failed +# stage adds nothing to it, a no-op keeps v1, the next change rotates +rm -rf "$t/channel" +stages v1 "stages v1 for a channel without a record" +stages v2 "stages v2 for a channel without a record" +rm "$ch.sets" +if FAULT='pkg_files() { return 75; }' run_stage v2; then fail "stops when the channel's files can't be listed"; fi +if FAULT="bsdtar() { [[ \$2 != '$ch/'* ]] || return 75; command bsdtar \"\$@\"; }" run_stage v2; then + fail "stops when the channel's index can't be read" +fi +if ((EUID)); then + chmod 300 "$ch" + if run_stage v2; then chmod 755 "$ch"; fail "stops when the channel's directory can't be read"; fi + chmod 755 "$ch" +fi +[[ ! -e $ch.sets ]] || fail "a failed read records nothing" +pass "a channel whose files, index or directory can't be read gets no record" +mv "$t/v3/app-3-1-aarch64.pkg.tar.zst.sig" "$t/sig.held" +if run_stage v3; then fail "refuses v3 without app's signature"; fi +[[ -f $ch/app-3-1-aarch64.pkg.tar.zst ]] || fail "the failed v3 copied app-3-1 before it stopped" +mv "$t/sig.held" "$t/v3/app-3-1-aarch64.pkg.tar.zst.sig" +stages v2 "stages v2 after the failed v3" +expect_index v2 +expect "without a record, a no-op keeps v1 and drops what the failed v3 copied" base-1-1 app-1-1 app-2-1 +stages v3 "stages v3 after that" +expect "and the next change keeps only the set it replaces" base-1-1 app-2-1 app-3-1 diff --git a/tools/repo-publish b/tools/repo-publish index 283f4e38d..e87ee3045 100755 --- a/tools/repo-publish +++ b/tools/repo-publish @@ -27,7 +27,7 @@ # on GitHub, unless one is open already # # Each channel is out/publish//aarch64 here, mirrored to the server. -# It keeps the files of the current index and of the one before it, so a +# It keeps the files of the current index and the last different one, so a # phone can go back one version. Upload order: packages and signatures, then # the index, then the pruning, so a phone never sees an index that names a # file not there yet. Nothing is uploaded unless every package the index lists @@ -111,6 +111,10 @@ stb=$pub/stable/$ARCH # The package file names an index lists, one per line db_files() { [[ -f $1/$DB ]] && bsdtar -xOf "$1/$DB" --include='*/desc' | awk '/^%FILENAME%$/ { getline; print }' | sort; } +# The package file names a channel directory holds, one per line +pkg_files() { find "$1" -mindepth 1 -maxdepth 1 -name '*.pkg.tar.*' -printf '%f\n' | sed 's/\.sig$//' | sort -u; } +# save_sets FILE CURRENT PREVIOUS: record two package sets in FILE at once +save_sets() { { sed -n 's/^./current &/p' <<<"$2" && sed -n 's/^./previous &/p' <<<"$3"; } > "$1.new" && mv -f -- "$1.new" "$1"; } # "file sha256" for each package an index lists db_sums() { bsdtar -xOf "$1/$DB" --include='*/desc' | @@ -175,28 +179,47 @@ verify() { echo " $n packages and the index: checksums good, signed by the release key" } -# Bring a channel directory to a new index: the new files, plus the ones the -# previous index listed, nothing else. +# Bring a channel directory to a new index: the new files, plus those of the +# last different package set before it, nothing else. $to.sets, beside the +# channel (never uploaded), records the set last staged and the one before +# it. Every run decides from that record, never from the index on disk, and +# replaces it in one rename once the new index is in place, so a stage that +# fails anywhere changes nothing a retry decides by. Staging the same set +# again keeps the one before it, so a phone can still go back. A channel +# staged before the record existed starts it, before copying anything, from +# its index and every package file it has. Each read and write here is +# checked on its own, so one that fails stops the stage instead of +# recording, or keeping, less. stage() { local from=$1 to=$2 f mkdir -p "$to" - local previous - previous=$(db_files "$to" || true) + local new current="" previous + new=$(db_files "$from") + if [[ -f $to.sets ]]; then + current=$(sed -n 's/^current //p' "$to.sets") + previous=$(sed -n 's/^previous //p' "$to.sets") + else + if [[ -f $to/$DB ]]; then current=$(db_files "$to"); fi + previous=$(pkg_files "$to") + save_sets "$to.sets" "$current" "$previous" + fi + [[ $new == "$current" ]] || previous=$current while read -r f; do cp -p "$from/$f" "$from/$f.sig" "$to/" - done < <(db_files "$from") + done <<<"$new" for f in omamobile.db omamobile.files; do cp -pL "$from/$f.tar.gz" "$from/$f.tar.gz.sig" "$to/" cp -pL "$from/$f.tar.gz" "$to/$f" cp -pL "$from/$f.tar.gz.sig" "$to/$f.sig" done - local keep - keep=$( { db_files "$to"; if [[ -n $previous ]]; then echo "$previous"; fi; } | sort -u) - for f in "$to"/*.pkg.tar.*; do - [[ -e $f ]] || continue - f=${f##*/}; f=${f%.sig} - grep -qxF "$f" <<<"$keep" || rm -f "$to/$f" "$to/$f.sig" - done + save_sets "$to.sets" "$new" "$previous" + local -A keep=() + local have + while read -r f; do [[ -z $f ]] || keep[$f]=1; done <<<"$new"$'\n'"$previous" + have=$(pkg_files "$to") + while read -r f; do + [[ -z $f || -n ${keep[$f]:-} ]] || rm -f "$to/$f" "$to/$f.sig" + done <<<"$have" } write_index() {