From 43df7c7b2ea22d3b849a691b5c374f9df966838e Mon Sep 17 00:00:00 2001 From: Victor Vogel Date: Sun, 1 Nov 2026 21:59:47 +0000 Subject: [PATCH 1/2] tools/publish-kernel: stop when the published tip can't be read The publisher fetched the published branch in an AND list and took any failure for a missing branch: a network error left pub empty, which picked a full rebuild and an unleased `git push --force`, so one failed read could replace every published commit id that pull requests are based on. Read the tip with `git ls-remote --exit-code` instead: exit 2 means the remote answered without the branch, and any other failure, or a fetch that fails, stops the run. A published tip that matches no local commit (rewritten history) now stops unless `--rebuild` is given. Every push sends the checked replay by commit id with `--force-with-lease` on the tip the run fetched (no tip: the branch must not exist), so neither a published branch that moved during the run nor a local publish/ moved by another run is ever pushed over it. publish/ moves only after a successful push, so it keeps naming the published head. tests/publish-kernel runs the tool against a bare repository in a scratch folder (the remote URL rewritten to it, every Git protocol but file refused): a first publication, a fast-forward, a failed fetch, an unreadable remote, a rewritten history with and without --rebuild, a published branch that moves before the push on both paths, a local publish/ that moves before the push, and a hidden tip. The old tool fails seven of the nine. Co-Authored-By: Claude Opus 5.5 --- device/cheetah/KERNEL-PUBLISH.md | 20 +++-- docs/RELEASES.md | 2 +- tests/all | 5 +- tests/publish-kernel | 140 +++++++++++++++++++++++++++++++ tools/publish-kernel | 33 ++++++-- 5 files changed, 182 insertions(+), 18 deletions(-) create mode 100755 tests/publish-kernel diff --git a/device/cheetah/KERNEL-PUBLISH.md b/device/cheetah/KERNEL-PUBLISH.md index ac2445cc5..fccb20254 100644 --- a/device/cheetah/KERNEL-PUBLISH.md +++ b/device/cheetah/KERNEL-PUBLISH.md @@ -4,11 +4,12 @@ Status (2026-10-19): **published.** `omamobile` is on GitHub (linux-omarchy-mobile, 146 commits on v7.3-rc6) and moves forward without rewriting published commits: `tools/publish-kernel` fast-forwards when the -published tip's tree matches a local commit. Community kernel PRs are reviewed -against it (felix's board file and UFS are merged). First patch sent -upstream: "power: supply: max17042_battery: infer the status when no -supplier is described" (linux-pm, 2026-10-19), Reviewed-by Hans de Goede, -waiting for the maintainer. +published tip's tree matches a local commit, and otherwise stops unless +given `--rebuild`; it always stops when it can't read the published tip. +Community kernel PRs are reviewed against it (felix's board file and UFS +are merged). First patch sent upstream: "power: supply: max17042_battery: +infer the status when no supplier is described" (linux-pm, 2026-10-19), +Reviewed-by Hans de Goede, waiting for the maintainer. Earlier status (2026-10-12): **prepared, not pushed.** Branch `omamobile-next` in `~/android/mainline/wt-next`, built in `~/android/mainline/out-next` @@ -303,10 +304,13 @@ tools/publish-kernel # defaults: branch omamobile, tag v `tools/publish-kernel` replays the 123 commits on the squashed v7.3-rc6 base in a throwaway worktree, checks the tree matches `omamobile`, and force-pushes `publish/omamobile` to `github.com/derekross/linux-omarchy-mobile`. -Pass: "Published omamobile (123 commits on v7.3-rc6)". Rollback: the +Pass: "Published omamobile (123 commits on v7.3-rc6)". (Since 2026-11-01 the +tool fast-forwards a published branch and stops instead of rewriting one: a +rebuild like this one takes `tools/publish-kernel --rebuild`, and its push +is leased on the published tip the run read.) Rollback: the previous published head is `publish/omamobile` before the run -(`056bfe463`); `git push --force 056bfe463:refs/heads/omamobile` -restores it. +(`056bfe463`); `git push --force-with-lease=refs/heads/omamobile: 056bfe463:refs/heads/omamobile` restores it. Afterwards: `git worktree remove ~/android/mainline/wt-next` and `git branch -d omamobile-next` once merged. diff --git a/docs/RELEASES.md b/docs/RELEASES.md index 00c63cd51..342b603e2 100644 --- a/docs/RELEASES.md +++ b/docs/RELEASES.md @@ -287,7 +287,7 @@ A release never contains a kernel or package that isn't in `[omamobile]` at the ## 8. The kernel branches -- **`omamobile-int`** is the snapshot the phone runs and that releases are built from. Before each release it is published as a snapshot (`tools/publish-kernel omamobile-int`) and tagged `omamobile-int-`, so anyone can rebuild a release's kernel. +- **`omamobile-int`** is the snapshot the phone runs and that releases are built from. Before each release it is published as a snapshot (`tools/publish-kernel omamobile-int`; when `omamobile-int` was rebuilt since the last snapshot, the tool stops and the rebuilt branch needs `--rebuild`) and tagged `omamobile-int-`, so anyone can rebuild a release's kernel. - **`omamobile`** stays the curated, upstream-ready series. Before each release, a curation pass moves whatever has passed on the phone since the last one out of `omamobile-int` into `omamobile` as clean commits (the `omamobile-next` process in [KERNEL-PUBLISH](../device/cheetah/KERNEL-PUBLISH.md)), and `omamobile` is published. The release notes list how many commits `omamobile-int` still carries beyond `omamobile`; that number should shrink over time. - Board DTS files for new devices land in the device's topic branch first, then in both branches as above. diff --git a/tests/all b/tests/all index 4afae7bc4..0558e9bf3 100755 --- a/tests/all +++ b/tests/all @@ -10,7 +10,8 @@ # smoke test of the newest root image in out/ if there is one, the felix, # lynx and panther device packages' own tests, tools/alarm-wake-test's # check, and a contributor's local builds (tools/local-key-init, -# tools/fetch-published, --local). +# tools/fetch-published, --local), and tools/publish-kernel against a +# scratch remote. set -euo pipefail cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." for f in tools/install-phone tools/phone-provision tools/build-rootfs tools/build-packages tools/make-bundle \ @@ -24,7 +25,7 @@ sh -n device/flame/initramfs/init sh -n device/flame/initramfs/build bash -n tools/build-flame-probe echo "ok - scripts parse" -for t in tests/pr-check tests/kernel-pr-check tests/alarm-wake-test tests/install-phone tests/build-rootfs tests/phone-provision tests/initramfs-grow tests/images-layout tests/device-overlays tests/simplefb-trigger tests/initramfs-steps tests/storage-guard tests/usb-product tests/boot-method tests/make-bundle-keys tests/flame-probe packages/omamobile-firstboot/test/all packages/omamobile-device-felix/test/all packages/omamobile-device-lynx/test/all packages/omamobile-device-panther/test/all tests/local-builds tests/phone-install-package; do +for t in tests/pr-check tests/kernel-pr-check tests/alarm-wake-test tests/install-phone tests/build-rootfs tests/phone-provision tests/initramfs-grow tests/images-layout tests/device-overlays tests/simplefb-trigger tests/initramfs-steps tests/storage-guard tests/usb-product tests/boot-method tests/make-bundle-keys tests/flame-probe packages/omamobile-firstboot/test/all packages/omamobile-device-felix/test/all packages/omamobile-device-lynx/test/all packages/omamobile-device-panther/test/all tests/local-builds tests/phone-install-package tests/publish-kernel; do echo "# $t" "$t" done diff --git a/tests/publish-kernel b/tests/publish-kernel new file mode 100755 index 000000000..ac1f6a6fd --- /dev/null +++ b/tests/publish-kernel @@ -0,0 +1,140 @@ +#!/bin/bash +# SPDX-License-Identifier: MIT +# tools/publish-kernel against a stand-in kernel tree and a bare repository +# in a scratch folder, never GitHub: the publisher's remote URL is rewritten +# to the local repository, every Git protocol but file is refused, and the +# test stops before running anything unless the URL resolves into the +# scratch folder. A shim in front of git fails a fetch, or moves a branch +# just before the push, to stand in for a network error or a second +# publisher. +set -euo pipefail +here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +t=$(mktemp -d) +trap 'rm -rf "$t"' EXIT +url=$(sed -n 's/^remote=//p' "$here/tools/publish-kernel") +[[ -n $url ]] || { echo "not ok - tools/publish-kernel names its remote" >&2; exit 1; } +export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=$t/gitconfig GIT_SSH_COMMAND=false + +realgit=$(command -v git) +mkdir "$t/bin" +cat > "$t/bin/git" <&2; exit 128 ;; + push/move) "$realgit" -C "\$MOVE_REPO" update-ref "\$MOVE_REF" "\$MOVE_TO" ;; +esac +exec "$realgit" "\$@" +EOF +chmod +x "$t/bin/git" + +fail() { echo "# $*" >&2; exit 1; } +fixture() { # name: a kernel tree (tag v1.0, two commits on omamobile) and an empty remote + k=$t/$1/linux r=$t/$1/remote.git + printf '[user]\n\tname = A Human\n\temail = human@example.org\n[protocol]\n\tallow = never\n[protocol "file"]\n\tallow = always\n[url "%s"]\n\tinsteadOf = %s\n' \ + "$r" "$url" > "$GIT_CONFIG_GLOBAL" + [[ $r == "$t"/* && $(git ls-remote --get-url "$url") == "$r" ]] || { + echo "Bail out! $url does not resolve to the scratch remote" >&2; exit 2; } + [[ $(git ls-remote ssh://example.invalid/none 2>&1) == *"transport 'ssh' not allowed"* ]] || { + echo "Bail out! Git protocols other than file are allowed" >&2; exit 2; } + git init -q --bare "$r" + git init -q -b omamobile "$k" + commit base && git -C "$k" tag v1.0 + commit one && commit two +} +commit() { echo "$1" >> "$k/file"; git -C "$k" add file; git -C "$k" commit -q -m "$1"; } +tip() { git -C "$r" rev-parse -q --verify refs/heads/omamobile || true; } +pubref() { git -C "$k" rev-parse -q --verify refs/heads/publish/omamobile || true; } +publish() { # ok|fail [pattern] [-- args...]: the output must contain pattern + local want=$1 pattern= status=0 out; shift + [[ ${1:-} == -- ]] || { pattern=$1; shift; }; shift + out=$(KERNEL_DIR=$k "$here/tools/publish-kernel" "$@" 2>&1) || status=$? + [[ ($want == ok && $status == 0) || ($want == fail && $status != 0) ]] || { echo "$out" >&2; fail "exit $status, expected $want"; } + [[ -z $pattern ]] || grep -c -- "$pattern" <<<"$out" >/dev/null || { echo "$out" >&2; fail "doesn't say '$pattern'"; } +} +published() { fixture "$1"; publish ok -- omamobile v1.0; p1=$(tip); } +rewrite() { echo changed >> "$k/file"; git -C "$k" commit -q -a --amend -m two; } +same_tree() { [[ $(git -C "$k" rev-parse "$1^{tree}") == $(git -C "$k" rev-parse "omamobile^{tree}") ]]; } + +case_first() { + fixture first + publish ok -- omamobile v1.0 + [[ -n $(tip) && $(tip) == $(pubref) ]] || fail "no published branch" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - a first publication creates the branch" +} +case_ff() { + published ff + commit three + publish ok -- omamobile v1.0 + [[ $(git -C "$r" rev-parse "$(tip)^") == "$p1" ]] || fail "not one commit on the old tip" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - a new local commit is published as a fast-forward" +} +case_fetch_fails() { + published fetchfail + commit three + PATH=$t/bin:$PATH FAKE_GIT=fail-fetch publish fail "Can't fetch" -- omamobile v1.0 + [[ $(tip) == "$p1" ]] || fail "the published branch was replaced" + [[ $(pubref) == "$p1" ]] || fail "publish/omamobile moved" + echo "ok - a failed fetch of the published branch stops before anything changes" +} +case_unreadable() { + published unreadable + commit three + mv "$r" "$r.gone" + publish fail "Can't read" -- omamobile v1.0 + mv "$r.gone" "$r" + [[ $(tip) == "$p1" && $(pubref) == "$p1" ]] || fail "something changed" + echo "ok - a remote that can't be read is not taken for a missing branch" +} +case_rewritten() { + published rewritten + rewrite + publish fail --rebuild -- omamobile v1.0 + [[ $(tip) == "$p1" && $(pubref) == "$p1" ]] || fail "the published branch was replaced" + publish ok -- --rebuild omamobile v1.0 + ! git -C "$r" merge-base --is-ancestor "$p1" "$(tip)" || fail "not rebuilt" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - a rewritten local history stops, and --rebuild replaces the published branch" +} +moved() { # name prepare [args...]: a second publisher moves the branch back just before the push + published "$1" + $2; shift 2 + local back; back=$(git -C "$r" rev-parse "$p1^") + PATH=$t/bin:$PATH FAKE_GIT=move MOVE_REPO=$r MOVE_REF=refs/heads/omamobile MOVE_TO=$back publish fail "stale info" -- "$@" omamobile v1.0 + [[ $(tip) == "$back" ]] || fail "the moved branch was overwritten" + [[ $(pubref) == "$p1" ]] || fail "publish/omamobile names a commit that wasn't published" +} +case_moved_ff() { + moved movedff "commit three" + echo "ok - a fast-forward doesn't overwrite a branch that moved during the run" +} +case_moved_rebuild() { + moved movedrebuild rewrite --rebuild + echo "ok - --rebuild doesn't overwrite a branch that moved during the run" +} +case_local_moved() { + published localmoved + commit three + PATH=$t/bin:$PATH FAKE_GIT=move MOVE_REPO=$k MOVE_REF=refs/heads/publish/omamobile MOVE_TO=$(git -C "$k" rev-parse v1.0) \ + publish ok -- omamobile v1.0 + [[ $(git -C "$r" rev-parse "$(tip)^") == "$p1" ]] || fail "pushed something other than the checked replay" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - the push sends the checked replay, not whatever publish/omamobile points at by then" +} +case_hidden() { + published hidden + git -C "$r" config uploadpack.hideRefs refs/heads/omamobile + commit three + publish fail "stale info" -- omamobile v1.0 + [[ $(tip) == "$p1" && $(pubref) == "$p1" ]] || fail "the published branch or publish/omamobile changed" + echo "ok - a branch that looks missing but exists at push time is not overwritten" +} + +failed=0 +for c in first ff fetch_fails unreadable rewritten moved_ff moved_rebuild local_moved hidden; do + set +e; ( set -e; "case_$c" ); s=$?; set -e + [[ $s != 2 ]] || exit 2 + [[ $s == 0 ]] || { echo "not ok - $c"; failed=1; } +done +exit "$failed" diff --git a/tools/publish-kernel b/tools/publish-kernel index dff306e4c..870792213 100755 --- a/tools/publish-kernel +++ b/tools/publish-kernel @@ -7,8 +7,13 @@ # commit holding the exact upstream tag's tree, with our commits replayed on # top unchanged. The trees are identical to the local branch, which keeps the # real upstream history for sending patches. -# tools/publish-kernel [branch] [upstream-tag] (defaults: omamobile v7.3-rc6) +# tools/publish-kernel [--rebuild] [branch] [upstream-tag] +# (defaults: omamobile v7.3-rc6) +# --rebuild allows replacing a published branch whose tip matches no commit +# on the local branch (local history was rewritten); without it that stops. set -euo pipefail +rebuild= +[[ ${1:-} != --rebuild ]] || { rebuild=1; shift; } # The kernel clone, its worktrees and build outputs live in KERNEL_BASE: # ~/omamobile-kernel, or ~/android/mainline on setups from before 2026-10-29. @@ -37,15 +42,31 @@ individual commits; they apply unchanged on top of the upstream tag.") # full replay gives every commit a new id. So when the published tip has the # tree of a commit on the local branch, replay only the commits after it, on # top of the published tip, and push a fast-forward. Only when no such commit -# exists (local history rewritten) is the branch rebuilt and force-pushed. -pub= -git fetch -q "$remote" "refs/heads/$branch" 2>/dev/null && pub=$(git rev-parse FETCH_HEAD) +# exists (local history rewritten) is the branch rebuilt, and only with +# --rebuild. A failed read is not a missing branch: ls-remote exits 2 only +# when the remote answered without the branch, and anything else stops the +# run. The push leases the tip seen here (no tip: the branch must not exist), +# so a branch that moved during the run is never overwritten. +status=0 +refs=$(git ls-remote --exit-code "$remote" "refs/heads/$branch") || status=$? +[[ $status == 0 || $status == 2 ]] || { echo "Can't read $branch from $remote (git ls-remote exit $status); not publishing" >&2; exit 1; } +pub=$(awk -v r="refs/heads/$branch" '$2 == r { print $1 }' <<<"$refs") from= if [[ -n $pub ]]; then + git fetch -q "$remote" "refs/heads/$branch" || { echo "Can't fetch $branch ($pub) from $remote; not publishing" >&2; exit 1; } + pub=$(git rev-parse FETCH_HEAD) ptree=$(git rev-parse "$pub^{tree}") for c in $(git rev-list "$tag..$branch"); do [[ $(git rev-parse "$c^{tree}") == "$ptree" ]] && { from=$c; break; } done + if [[ -z $from ]]; then + [[ -n $rebuild ]] || { + echo "The published $branch ($pub) matches no commit on the local $branch; not publishing." >&2 + echo "To replace it, which changes every published commit id: $0 --rebuild $branch $tag" >&2 + exit 1 + } + echo "Replacing the published $branch ($pub): every published commit id changes" >&2 + fi fi # Replay in a throwaway worktree, so no checkout in use changes. @@ -60,12 +81,10 @@ fi published=$(git -C "$work" rev-parse HEAD) [[ -z $(git diff "$branch" "$published") ]] || { echo "Replayed tree differs from $branch; not publishing" >&2; exit 1; } +git push --force-with-lease="refs/heads/$branch:$pub" "$remote" "$published:refs/heads/$branch" git branch -f "publish/$branch" "$published" if [[ -n $from ]]; then - git push "$remote" "publish/$branch:refs/heads/$branch" echo "Published $branch to $remote: $(git rev-list --count "$pub..$published") new commit(s), fast-forward" else - echo "No published commit matches $branch's history: rebuilding, which changes every published commit id" >&2 - git push --force "$remote" "publish/$branch:refs/heads/$branch" echo "Published $branch ($(git rev-list --count "$tag..$branch") commits on $tag) to $remote" fi From 395498692484795da2016e234e9401270260163c Mon Sep 17 00:00:00 2001 From: Victor Vogel Date: Mon, 2 Nov 2026 01:02:31 +0000 Subject: [PATCH 2/2] publish-kernel: isolate the offline remote fixture Select a local remote directly in the publisher test and clear inherited Git config overrides before its protocol guards run. The default publish destination is unchanged. Date the new behavior in the status summary. Co-Authored-By: GPT-6.1 Sol --- device/cheetah/KERNEL-PUBLISH.md | 7 ++++--- tests/publish-kernel | 15 +++++++-------- tools/publish-kernel | 3 ++- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/device/cheetah/KERNEL-PUBLISH.md b/device/cheetah/KERNEL-PUBLISH.md index fccb20254..092df0bac 100644 --- a/device/cheetah/KERNEL-PUBLISH.md +++ b/device/cheetah/KERNEL-PUBLISH.md @@ -3,9 +3,10 @@ Status (2026-10-19): **published.** `omamobile` is on GitHub (linux-omarchy-mobile, 146 commits on v7.3-rc6) and moves forward without -rewriting published commits: `tools/publish-kernel` fast-forwards when the -published tip's tree matches a local commit, and otherwise stops unless -given `--rebuild`; it always stops when it can't read the published tip. +rewriting published commits: `tools/publish-kernel` (since 2026-11-01) +fast-forwards when the published tip's tree matches a local commit, and +otherwise stops unless given `--rebuild`; it always stops when it can't read +the published tip. Community kernel PRs are reviewed against it (felix's board file and UFS are merged). First patch sent upstream: "power: supply: max17042_battery: infer the status when no supplier is described" (linux-pm, 2026-10-19), diff --git a/tests/publish-kernel b/tests/publish-kernel index ac1f6a6fd..dd191d808 100755 --- a/tests/publish-kernel +++ b/tests/publish-kernel @@ -1,8 +1,8 @@ #!/bin/bash # SPDX-License-Identifier: MIT # tools/publish-kernel against a stand-in kernel tree and a bare repository -# in a scratch folder, never GitHub: the publisher's remote URL is rewritten -# to the local repository, every Git protocol but file is refused, and the +# in a scratch folder, never GitHub: PUBLISH_REMOTE names the local +# repository, every Git protocol but file is refused, and the # test stops before running anything unless the URL resolves into the # scratch folder. A shim in front of git fails a fetch, or moves a branch # just before the push, to stand in for a network error or a second @@ -11,8 +11,7 @@ set -euo pipefail here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) t=$(mktemp -d) trap 'rm -rf "$t"' EXIT -url=$(sed -n 's/^remote=//p' "$here/tools/publish-kernel") -[[ -n $url ]] || { echo "not ok - tools/publish-kernel names its remote" >&2; exit 1; } +unset GIT_ALLOW_PROTOCOL GIT_CONFIG_PARAMETERS GIT_CONFIG_COUNT export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=$t/gitconfig GIT_SSH_COMMAND=false realgit=$(command -v git) @@ -30,10 +29,10 @@ chmod +x "$t/bin/git" fail() { echo "# $*" >&2; exit 1; } fixture() { # name: a kernel tree (tag v1.0, two commits on omamobile) and an empty remote k=$t/$1/linux r=$t/$1/remote.git - printf '[user]\n\tname = A Human\n\temail = human@example.org\n[protocol]\n\tallow = never\n[protocol "file"]\n\tallow = always\n[url "%s"]\n\tinsteadOf = %s\n' \ - "$r" "$url" > "$GIT_CONFIG_GLOBAL" - [[ $r == "$t"/* && $(git ls-remote --get-url "$url") == "$r" ]] || { - echo "Bail out! $url does not resolve to the scratch remote" >&2; exit 2; } + export PUBLISH_REMOTE=$r + printf '[user]\n\tname = A Human\n\temail = human@example.org\n[protocol]\n\tallow = never\n[protocol "file"]\n\tallow = always\n' > "$GIT_CONFIG_GLOBAL" + [[ $PUBLISH_REMOTE == "$t"/* && $(git ls-remote --get-url "$PUBLISH_REMOTE") == "$r" ]] || { + echo "Bail out! PUBLISH_REMOTE does not resolve to the scratch remote" >&2; exit 2; } [[ $(git ls-remote ssh://example.invalid/none 2>&1) == *"transport 'ssh' not allowed"* ]] || { echo "Bail out! Git protocols other than file are allowed" >&2; exit 2; } git init -q --bare "$r" diff --git a/tools/publish-kernel b/tools/publish-kernel index 870792213..ece186b3f 100755 --- a/tools/publish-kernel +++ b/tools/publish-kernel @@ -11,6 +11,7 @@ # (defaults: omamobile v7.3-rc6) # --rebuild allows replacing a published branch whose tip matches no commit # on the local branch (local history was rewritten); without it that stops. +# PUBLISH_REMOTE overrides the destination (for an offline test or a fork). set -euo pipefail rebuild= [[ ${1:-} != --rebuild ]] || { rebuild=1; shift; } @@ -24,7 +25,7 @@ fi linux=${KERNEL_DIR:-$KERNEL_BASE/linux} branch=${1:-omamobile} tag=${2:-v7.3-rc6} -remote=git@github.com:derekross/linux-omarchy-mobile.git +remote=${PUBLISH_REMOTE:-git@github.com:derekross/linux-omarchy-mobile.git} cd "$linux" [[ -z $(git status --porcelain --untracked-files=no) ]] || { echo "$linux has uncommitted changes" >&2; exit 1; }