diff --git a/device/cheetah/KERNEL-PUBLISH.md b/device/cheetah/KERNEL-PUBLISH.md index ac2445cc5..092df0bac 100644 --- a/device/cheetah/KERNEL-PUBLISH.md +++ b/device/cheetah/KERNEL-PUBLISH.md @@ -3,12 +3,14 @@ Status (2026-10-19): **published.** `omamobile` is on GitHub (linux-omarchy-mobile, 146 commits on v7.3-rc6) and moves forward without -rewriting published commits: `tools/publish-kernel` fast-forwards when the -published tip's tree matches a local commit. Community kernel PRs are reviewed -against it (felix's board file and UFS are merged). First patch sent -upstream: "power: supply: max17042_battery: infer the status when no -supplier is described" (linux-pm, 2026-10-19), Reviewed-by Hans de Goede, -waiting for the maintainer. +rewriting published commits: `tools/publish-kernel` (since 2026-11-01) +fast-forwards when the published tip's tree matches a local commit, and +otherwise stops unless given `--rebuild`; it always stops when it can't read +the published tip. +Community kernel PRs are reviewed against it (felix's board file and UFS +are merged). First patch sent upstream: "power: supply: max17042_battery: +infer the status when no supplier is described" (linux-pm, 2026-10-19), +Reviewed-by Hans de Goede, waiting for the maintainer. Earlier status (2026-10-12): **prepared, not pushed.** Branch `omamobile-next` in `~/android/mainline/wt-next`, built in `~/android/mainline/out-next` @@ -303,10 +305,13 @@ tools/publish-kernel # defaults: branch omamobile, tag v `tools/publish-kernel` replays the 123 commits on the squashed v7.3-rc6 base in a throwaway worktree, checks the tree matches `omamobile`, and force-pushes `publish/omamobile` to `github.com/derekross/linux-omarchy-mobile`. -Pass: "Published omamobile (123 commits on v7.3-rc6)". Rollback: the +Pass: "Published omamobile (123 commits on v7.3-rc6)". (Since 2026-11-01 the +tool fast-forwards a published branch and stops instead of rewriting one: a +rebuild like this one takes `tools/publish-kernel --rebuild`, and its push +is leased on the published tip the run read.) Rollback: the previous published head is `publish/omamobile` before the run -(`056bfe463`); `git push --force 056bfe463:refs/heads/omamobile` -restores it. +(`056bfe463`); `git push --force-with-lease=refs/heads/omamobile: 056bfe463:refs/heads/omamobile` restores it. Afterwards: `git worktree remove ~/android/mainline/wt-next` and `git branch -d omamobile-next` once merged. diff --git a/docs/RELEASES.md b/docs/RELEASES.md index 00c63cd51..342b603e2 100644 --- a/docs/RELEASES.md +++ b/docs/RELEASES.md @@ -287,7 +287,7 @@ A release never contains a kernel or package that isn't in `[omamobile]` at the ## 8. The kernel branches -- **`omamobile-int`** is the snapshot the phone runs and that releases are built from. Before each release it is published as a snapshot (`tools/publish-kernel omamobile-int`) and tagged `omamobile-int-`, so anyone can rebuild a release's kernel. +- **`omamobile-int`** is the snapshot the phone runs and that releases are built from. Before each release it is published as a snapshot (`tools/publish-kernel omamobile-int`; when `omamobile-int` was rebuilt since the last snapshot, the tool stops and the rebuilt branch needs `--rebuild`) and tagged `omamobile-int-`, so anyone can rebuild a release's kernel. - **`omamobile`** stays the curated, upstream-ready series. Before each release, a curation pass moves whatever has passed on the phone since the last one out of `omamobile-int` into `omamobile` as clean commits (the `omamobile-next` process in [KERNEL-PUBLISH](../device/cheetah/KERNEL-PUBLISH.md)), and `omamobile` is published. The release notes list how many commits `omamobile-int` still carries beyond `omamobile`; that number should shrink over time. - Board DTS files for new devices land in the device's topic branch first, then in both branches as above. diff --git a/tests/all b/tests/all index 4afae7bc4..0558e9bf3 100755 --- a/tests/all +++ b/tests/all @@ -10,7 +10,8 @@ # smoke test of the newest root image in out/ if there is one, the felix, # lynx and panther device packages' own tests, tools/alarm-wake-test's # check, and a contributor's local builds (tools/local-key-init, -# tools/fetch-published, --local). +# tools/fetch-published, --local), and tools/publish-kernel against a +# scratch remote. set -euo pipefail cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." for f in tools/install-phone tools/phone-provision tools/build-rootfs tools/build-packages tools/make-bundle \ @@ -24,7 +25,7 @@ sh -n device/flame/initramfs/init sh -n device/flame/initramfs/build bash -n tools/build-flame-probe echo "ok - scripts parse" -for t in tests/pr-check tests/kernel-pr-check tests/alarm-wake-test tests/install-phone tests/build-rootfs tests/phone-provision tests/initramfs-grow tests/images-layout tests/device-overlays tests/simplefb-trigger tests/initramfs-steps tests/storage-guard tests/usb-product tests/boot-method tests/make-bundle-keys tests/flame-probe packages/omamobile-firstboot/test/all packages/omamobile-device-felix/test/all packages/omamobile-device-lynx/test/all packages/omamobile-device-panther/test/all tests/local-builds tests/phone-install-package; do +for t in tests/pr-check tests/kernel-pr-check tests/alarm-wake-test tests/install-phone tests/build-rootfs tests/phone-provision tests/initramfs-grow tests/images-layout tests/device-overlays tests/simplefb-trigger tests/initramfs-steps tests/storage-guard tests/usb-product tests/boot-method tests/make-bundle-keys tests/flame-probe packages/omamobile-firstboot/test/all packages/omamobile-device-felix/test/all packages/omamobile-device-lynx/test/all packages/omamobile-device-panther/test/all tests/local-builds tests/phone-install-package tests/publish-kernel; do echo "# $t" "$t" done diff --git a/tests/publish-kernel b/tests/publish-kernel new file mode 100755 index 000000000..dd191d808 --- /dev/null +++ b/tests/publish-kernel @@ -0,0 +1,139 @@ +#!/bin/bash +# SPDX-License-Identifier: MIT +# tools/publish-kernel against a stand-in kernel tree and a bare repository +# in a scratch folder, never GitHub: PUBLISH_REMOTE names the local +# repository, every Git protocol but file is refused, and the +# test stops before running anything unless the URL resolves into the +# scratch folder. A shim in front of git fails a fetch, or moves a branch +# just before the push, to stand in for a network error or a second +# publisher. +set -euo pipefail +here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +t=$(mktemp -d) +trap 'rm -rf "$t"' EXIT +unset GIT_ALLOW_PROTOCOL GIT_CONFIG_PARAMETERS GIT_CONFIG_COUNT +export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=$t/gitconfig GIT_SSH_COMMAND=false + +realgit=$(command -v git) +mkdir "$t/bin" +cat > "$t/bin/git" <&2; exit 128 ;; + push/move) "$realgit" -C "\$MOVE_REPO" update-ref "\$MOVE_REF" "\$MOVE_TO" ;; +esac +exec "$realgit" "\$@" +EOF +chmod +x "$t/bin/git" + +fail() { echo "# $*" >&2; exit 1; } +fixture() { # name: a kernel tree (tag v1.0, two commits on omamobile) and an empty remote + k=$t/$1/linux r=$t/$1/remote.git + export PUBLISH_REMOTE=$r + printf '[user]\n\tname = A Human\n\temail = human@example.org\n[protocol]\n\tallow = never\n[protocol "file"]\n\tallow = always\n' > "$GIT_CONFIG_GLOBAL" + [[ $PUBLISH_REMOTE == "$t"/* && $(git ls-remote --get-url "$PUBLISH_REMOTE") == "$r" ]] || { + echo "Bail out! PUBLISH_REMOTE does not resolve to the scratch remote" >&2; exit 2; } + [[ $(git ls-remote ssh://example.invalid/none 2>&1) == *"transport 'ssh' not allowed"* ]] || { + echo "Bail out! Git protocols other than file are allowed" >&2; exit 2; } + git init -q --bare "$r" + git init -q -b omamobile "$k" + commit base && git -C "$k" tag v1.0 + commit one && commit two +} +commit() { echo "$1" >> "$k/file"; git -C "$k" add file; git -C "$k" commit -q -m "$1"; } +tip() { git -C "$r" rev-parse -q --verify refs/heads/omamobile || true; } +pubref() { git -C "$k" rev-parse -q --verify refs/heads/publish/omamobile || true; } +publish() { # ok|fail [pattern] [-- args...]: the output must contain pattern + local want=$1 pattern= status=0 out; shift + [[ ${1:-} == -- ]] || { pattern=$1; shift; }; shift + out=$(KERNEL_DIR=$k "$here/tools/publish-kernel" "$@" 2>&1) || status=$? + [[ ($want == ok && $status == 0) || ($want == fail && $status != 0) ]] || { echo "$out" >&2; fail "exit $status, expected $want"; } + [[ -z $pattern ]] || grep -c -- "$pattern" <<<"$out" >/dev/null || { echo "$out" >&2; fail "doesn't say '$pattern'"; } +} +published() { fixture "$1"; publish ok -- omamobile v1.0; p1=$(tip); } +rewrite() { echo changed >> "$k/file"; git -C "$k" commit -q -a --amend -m two; } +same_tree() { [[ $(git -C "$k" rev-parse "$1^{tree}") == $(git -C "$k" rev-parse "omamobile^{tree}") ]]; } + +case_first() { + fixture first + publish ok -- omamobile v1.0 + [[ -n $(tip) && $(tip) == $(pubref) ]] || fail "no published branch" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - a first publication creates the branch" +} +case_ff() { + published ff + commit three + publish ok -- omamobile v1.0 + [[ $(git -C "$r" rev-parse "$(tip)^") == "$p1" ]] || fail "not one commit on the old tip" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - a new local commit is published as a fast-forward" +} +case_fetch_fails() { + published fetchfail + commit three + PATH=$t/bin:$PATH FAKE_GIT=fail-fetch publish fail "Can't fetch" -- omamobile v1.0 + [[ $(tip) == "$p1" ]] || fail "the published branch was replaced" + [[ $(pubref) == "$p1" ]] || fail "publish/omamobile moved" + echo "ok - a failed fetch of the published branch stops before anything changes" +} +case_unreadable() { + published unreadable + commit three + mv "$r" "$r.gone" + publish fail "Can't read" -- omamobile v1.0 + mv "$r.gone" "$r" + [[ $(tip) == "$p1" && $(pubref) == "$p1" ]] || fail "something changed" + echo "ok - a remote that can't be read is not taken for a missing branch" +} +case_rewritten() { + published rewritten + rewrite + publish fail --rebuild -- omamobile v1.0 + [[ $(tip) == "$p1" && $(pubref) == "$p1" ]] || fail "the published branch was replaced" + publish ok -- --rebuild omamobile v1.0 + ! git -C "$r" merge-base --is-ancestor "$p1" "$(tip)" || fail "not rebuilt" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - a rewritten local history stops, and --rebuild replaces the published branch" +} +moved() { # name prepare [args...]: a second publisher moves the branch back just before the push + published "$1" + $2; shift 2 + local back; back=$(git -C "$r" rev-parse "$p1^") + PATH=$t/bin:$PATH FAKE_GIT=move MOVE_REPO=$r MOVE_REF=refs/heads/omamobile MOVE_TO=$back publish fail "stale info" -- "$@" omamobile v1.0 + [[ $(tip) == "$back" ]] || fail "the moved branch was overwritten" + [[ $(pubref) == "$p1" ]] || fail "publish/omamobile names a commit that wasn't published" +} +case_moved_ff() { + moved movedff "commit three" + echo "ok - a fast-forward doesn't overwrite a branch that moved during the run" +} +case_moved_rebuild() { + moved movedrebuild rewrite --rebuild + echo "ok - --rebuild doesn't overwrite a branch that moved during the run" +} +case_local_moved() { + published localmoved + commit three + PATH=$t/bin:$PATH FAKE_GIT=move MOVE_REPO=$k MOVE_REF=refs/heads/publish/omamobile MOVE_TO=$(git -C "$k" rev-parse v1.0) \ + publish ok -- omamobile v1.0 + [[ $(git -C "$r" rev-parse "$(tip)^") == "$p1" ]] || fail "pushed something other than the checked replay" + same_tree "$(tip)" || fail "published tree differs" + echo "ok - the push sends the checked replay, not whatever publish/omamobile points at by then" +} +case_hidden() { + published hidden + git -C "$r" config uploadpack.hideRefs refs/heads/omamobile + commit three + publish fail "stale info" -- omamobile v1.0 + [[ $(tip) == "$p1" && $(pubref) == "$p1" ]] || fail "the published branch or publish/omamobile changed" + echo "ok - a branch that looks missing but exists at push time is not overwritten" +} + +failed=0 +for c in first ff fetch_fails unreadable rewritten moved_ff moved_rebuild local_moved hidden; do + set +e; ( set -e; "case_$c" ); s=$?; set -e + [[ $s != 2 ]] || exit 2 + [[ $s == 0 ]] || { echo "not ok - $c"; failed=1; } +done +exit "$failed" diff --git a/tools/publish-kernel b/tools/publish-kernel index dff306e4c..ece186b3f 100755 --- a/tools/publish-kernel +++ b/tools/publish-kernel @@ -7,8 +7,14 @@ # commit holding the exact upstream tag's tree, with our commits replayed on # top unchanged. The trees are identical to the local branch, which keeps the # real upstream history for sending patches. -# tools/publish-kernel [branch] [upstream-tag] (defaults: omamobile v7.3-rc6) +# tools/publish-kernel [--rebuild] [branch] [upstream-tag] +# (defaults: omamobile v7.3-rc6) +# --rebuild allows replacing a published branch whose tip matches no commit +# on the local branch (local history was rewritten); without it that stops. +# PUBLISH_REMOTE overrides the destination (for an offline test or a fork). set -euo pipefail +rebuild= +[[ ${1:-} != --rebuild ]] || { rebuild=1; shift; } # The kernel clone, its worktrees and build outputs live in KERNEL_BASE: # ~/omamobile-kernel, or ~/android/mainline on setups from before 2026-10-29. @@ -19,7 +25,7 @@ fi linux=${KERNEL_DIR:-$KERNEL_BASE/linux} branch=${1:-omamobile} tag=${2:-v7.3-rc6} -remote=git@github.com:derekross/linux-omarchy-mobile.git +remote=${PUBLISH_REMOTE:-git@github.com:derekross/linux-omarchy-mobile.git} cd "$linux" [[ -z $(git status --porcelain --untracked-files=no) ]] || { echo "$linux has uncommitted changes" >&2; exit 1; } @@ -37,15 +43,31 @@ individual commits; they apply unchanged on top of the upstream tag.") # full replay gives every commit a new id. So when the published tip has the # tree of a commit on the local branch, replay only the commits after it, on # top of the published tip, and push a fast-forward. Only when no such commit -# exists (local history rewritten) is the branch rebuilt and force-pushed. -pub= -git fetch -q "$remote" "refs/heads/$branch" 2>/dev/null && pub=$(git rev-parse FETCH_HEAD) +# exists (local history rewritten) is the branch rebuilt, and only with +# --rebuild. A failed read is not a missing branch: ls-remote exits 2 only +# when the remote answered without the branch, and anything else stops the +# run. The push leases the tip seen here (no tip: the branch must not exist), +# so a branch that moved during the run is never overwritten. +status=0 +refs=$(git ls-remote --exit-code "$remote" "refs/heads/$branch") || status=$? +[[ $status == 0 || $status == 2 ]] || { echo "Can't read $branch from $remote (git ls-remote exit $status); not publishing" >&2; exit 1; } +pub=$(awk -v r="refs/heads/$branch" '$2 == r { print $1 }' <<<"$refs") from= if [[ -n $pub ]]; then + git fetch -q "$remote" "refs/heads/$branch" || { echo "Can't fetch $branch ($pub) from $remote; not publishing" >&2; exit 1; } + pub=$(git rev-parse FETCH_HEAD) ptree=$(git rev-parse "$pub^{tree}") for c in $(git rev-list "$tag..$branch"); do [[ $(git rev-parse "$c^{tree}") == "$ptree" ]] && { from=$c; break; } done + if [[ -z $from ]]; then + [[ -n $rebuild ]] || { + echo "The published $branch ($pub) matches no commit on the local $branch; not publishing." >&2 + echo "To replace it, which changes every published commit id: $0 --rebuild $branch $tag" >&2 + exit 1 + } + echo "Replacing the published $branch ($pub): every published commit id changes" >&2 + fi fi # Replay in a throwaway worktree, so no checkout in use changes. @@ -60,12 +82,10 @@ fi published=$(git -C "$work" rev-parse HEAD) [[ -z $(git diff "$branch" "$published") ]] || { echo "Replayed tree differs from $branch; not publishing" >&2; exit 1; } +git push --force-with-lease="refs/heads/$branch:$pub" "$remote" "$published:refs/heads/$branch" git branch -f "publish/$branch" "$published" if [[ -n $from ]]; then - git push "$remote" "publish/$branch:refs/heads/$branch" echo "Published $branch to $remote: $(git rev-list --count "$pub..$published") new commit(s), fast-forward" else - echo "No published commit matches $branch's history: rebuilding, which changes every published commit id" >&2 - git push --force "$remote" "publish/$branch:refs/heads/$branch" echo "Published $branch ($(git rev-list --count "$tag..$branch") commits on $tag) to $remote" fi