Skip to content

tools: repo-publish --from-main publishes a tree with no prebuilt files - #289

Closed
only21mil wants to merge 1 commit into
derekross:mainfrom
only21mil:build/c123-empty-prebuilt-inventory
Closed

only21mil wants to merge 1 commit into
derekross:mainfrom
only21mil:build/c123-empty-prebuilt-inventory

Conversation

@only21mil

Copy link
Copy Markdown
Contributor

What and why

tools/repo-publish --from-main copies the gitignored prebuilt binaries (packages/*/…/prebuilt/) into out/publish-tree with git ls-files | grep -z /prebuilt/ | rsync, then hands over to the clean tree's publisher. In a checkout with no prebuilt files (a fresh clone, or one where nothing has been built), grep finds no match and exits 1. Under set -euo pipefail that ends the script before the hand-over, with no message, so nothing is built or published. Related: #109. The build machine has its prebuilts, so this isn't the 10-28 failure there; it bites any checkout without them.

git now selects the files itself with the pathspec ':(glob)packages/**/prebuilt/**'. On a test tree of awkward names it picks exactly the files the grep did, and it exits 0 when there are none; rsync then gets an empty list and copies nothing. A failing git ls-files or rsync still stops the script before the publisher runs.

tests/repo-publish (new, in tests/all) runs the block in a fixture repository. The real git lists the inventory and the real rsync copies it; a stand-in git answers fetch, checkout and clean, and a stand-in publisher in the fixture's out/publish-tree records its arguments. Nothing is fetched, built, signed or published, and no worktree is created. It covers:

  • nothing ignored;
  • ignored files, none under prebuilt/;
  • prebuilt names with a space and a newline, at two depths;
  • git ls-files failing at once (a corrupt index) or after some names;
  • a failing rsync.

CI's Arch container gets rsync; without it the new test would always skip there.

Found by: Sol 6.1 High / Priority; Terra 6.1 Max / Priority; Sol 6.1 xhigh / Priority (r2). Records: SOL-RELEASE-BUILD-008, TERRA-BUILD_INSTALL-015, SOL-XHIGH-R2:release_build:SOL61XH-R2-RB-027.

Device and build

  • Phone: none: tools only (tools/repo-publish runs on the maintainer's machine, never on a phone)
  • Kernel release (uname -r): n/a
  • Packages changed, with versions: none

Checks (no phone needed)

  • tools/pr-check passes (against the base, af8c0f0)
  • tests/all passes. Skipped on this laptop for missing tools: the Docker end-to-end run, AOSP mkbootimg, dtc, lz4, the rootfs smoke test
  • tests/packages: not run, no package touched
  • Each changed package has its version bumped: n/a, no package changed
  • Commits have area prefixes and bodies that say why; AI help is a Co-Authored-By: trailer

Also: the new test fails at its first case against the parent's tools/repo-publish; mutants that swallow the git or rsync status each fail it.

On the phone

  • I ran this on my own phone: not applicable, tools only (CONTRIBUTING §6.5)
  • What I did not test: a real --from-main run (it fetches origin and publishes), the CI container itself, and an environment with GIT_LITERAL_PATHSPECS=1 or GIT_ICASE_PATHSPECS=1 set. The pathspec selects nothing under the first and also takes Prebuilt/ under the second; the publish timer sets neither.
  • Nothing in this change writes outside slot B and userdata, weakens the storage guard, or reads registers from userspace

Generated with Claude Opus 5.5 and GPT-6.1 Sol in T3 Code Nightly

--from-main copies the gitignored prebuilt binaries into
out/publish-tree with git ls-files | grep -z /prebuilt/ | rsync. In a
checkout that has none (a fresh clone, or one where nothing has been
built), grep finds no match and exits 1; under set -euo pipefail that
ends the script with no message before it hands over to the clean
tree, so nothing is built or published.

git now selects the files itself with the pathspec
':(glob)packages/**/prebuilt/**', which picks the same names the grep
did and exits 0 when there are none; rsync then gets an empty list and
copies nothing. A failing git ls-files or rsync still stops the script
before the publisher runs.

tests/repo-publish runs the block in a fixture repository: the real git
lists the inventory and the real rsync copies it, a stand-in git
answers fetch, checkout and clean, and a stand-in publisher in the
fixture's out/publish-tree records its arguments. It covers no ignored
files, ignored files with none under prebuilt/, prebuilt names with a
space and a newline at two depths, a git ls-files that fails at once
(a corrupt index) or after some names, and a failing rsync. CI's Arch
container gets rsync so the test runs there.

Tested on the laptop (no phone; the script never runs on one):
tests/repo-publish passes on this commit and fails at its first case on
the parent; mutants that swallow the git or rsync status each fail it;
the pathspec and the old grep select the same files from a tree of
awkward names. tools/pr-check and tests/all pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derekross

Copy link
Copy Markdown
Owner

Thanks. This is right: grep -z exits 1 on no match, and under pipefail that ended --from-main silently in any checkout without prebuilts. The pathspec selects the same files and exits 0 on none.

I ran tests/repo-publish in a container with no network before approving CI. The fixture keeps everything inside its temporary directory: the stand-in git answers fetch, checkout, clean and rev-parse, the stand-in publisher only records its arguments, and nothing is signed or uploaded. All four groups pass. The workflow change only adds rsync to the packages CI installs, which the new test needs; nothing else in checks.yml changes.

Since this is the publisher, I'll also do one --from-main --dry-run on the build machine myself after merging. tests/all conflicts with other open PRs on the same line, and I'll resolve that at merge.

@derekross

Copy link
Copy Markdown
Owner

A note on the red CI run here: the failure is omamobile-audio's c9 rise test, a timing flake that #315 has since fixed on main. Your change has nothing to do with it.

#313 changed tools/repo-publish on main to sign with the release key, but it didn't touch the --from-main block. Your change still merges cleanly there (only tests/all conflicts, as expected). On the merged tree tests/repo-publish passes all four groups. I'll merge it and then do the --from-main --dry-run myself, as planned.

derekross added a commit that referenced this pull request Nov 7, 2026
The check greps tools/repo-publish's source for the old grep -zE filter,
which #289's pathspec replaces. On a checkout with cheetah's boot files
staged, the old filter and the new pathspec list the same files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
derekross added a commit that referenced this pull request Nov 7, 2026
…es (#289) (#476)

repo-publish --from-main copies the gitignored prebuilt binaries and boot files into its clean origin/main checkout. It listed them with git ls-files piped through grep -z, and grep exits 1 on an empty list, so under pipefail a checkout with none stopped the publish. A git pathspec now picks them (packages/**/prebuilt/** and packages/**/boot-files/**), and an empty list copies nothing. tests/repo-publish covers the empty inventory, NUL-separated names, and an inventory or copy that fails partway, each stopping before anything is published. On a checkout with cheetah's boot files staged, the old filter and the new pathspec list the same files.

Co-Authored-By: Victor Vogel <satsdev@only21mil.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@derekross

Copy link
Copy Markdown
Owner

Merged through #476 with the conflicts against main resolved (and its boot-files/ copy kept), you credited as co-author. Thank you, @only21mil!

@derekross derekross closed this Nov 7, 2026
@only21mil

Copy link
Copy Markdown
Contributor Author

You're welcome my friend.

Glad I can help!

robertzas pushed a commit to robertzas/omarchy-mobile that referenced this pull request Nov 14, 2026
…es (derekross#289) (derekross#476)

repo-publish --from-main copies the gitignored prebuilt binaries and boot files into its clean origin/main checkout. It listed them with git ls-files piped through grep -z, and grep exits 1 on an empty list, so under pipefail a checkout with none stopped the publish. A git pathspec now picks them (packages/**/prebuilt/** and packages/**/boot-files/**), and an empty list copies nothing. tests/repo-publish covers the empty inventory, NUL-separated names, and an inventory or copy that fails partway, each stopping before anything is published. On a checkout with cheetah's boot files staged, the old filter and the new pathspec list the same files.

Co-Authored-By: Victor Vogel <satsdev@only21mil.xyz>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants