Skip to content

tools: move NCM transfer profile and refresh Bluejay review status - #666

Merged
derekross merged 5 commits into
derekross:mainfrom
latentoperator:followup/ncm-transfer-20261111
Nov 15, 2026
Merged

derekross merged 5 commits into
derekross:mainfrom
latentoperator:followup/ncm-transfer-20261111

Conversation

@latentoperator

@latentoperator latentoperator commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Add the attended USB NCM transfer check moved out of linux-omarchy-mobile #225. It reads sixteen exact 2 MiB zero streams, verifies each SHA-256 under one 20-second deadline, and stops at the first failure. It writes no phone files or partitions.

Every transfer uses tools/phone-ssh --fresh, preserving its key, host-key pin/alias and direct-interface settings. --peer and --interface are required; the interface must be a literal name, so off and auto cannot disable binding. Direct binding defaults OMAMOBILE_SSH_CONFIG to /dev/null; an incompatible explicit config fails with a diagnostic. Failures identify the transfer and failed check and include SSH's stderr. Timeouts and interruption stop the SSH process group, including its proxy.

Validation: policy, repository tests/all and every package's tests pass on the rebased head. The new tests/ncm-transfer fixtures cover sixteen successful streams, pinned fresh SSH settings, early SSH/size/digest failures, argument refusals, the shared deadline and cancellation. Independent review also exercised real sleeping SSH descendants under timeout, SIGINT, SIGTERM and SIGHUP. The exact tool passed all sixteen transfers over the attached Pixel 6a's USB link using its normal pinned identity/known-host settings, with no wrapper.

The phone ran 7.3.0-rc6-omamobile-bluejay-input-gated0. These transfers qualify the host tool, not the revised NCM driver's aggregation-retry path; USB tracing is needed to establish that path ran. Pixel 7 Pro and laptop-sleep qualification remain with kernel #225. This PR performs no kernel installation or boot. Bluejay's documentation records the transfer command and the fixed November 11 source/build checkpoint; current revision and CI details stay in the kernel PRs.

November 13–14 follow-up

The same exact tool subsequently passed sixteen 2 MiB SHA-checked reads on the revised Bluejay restricted-pool boot (7.3.0-rc6-omamobile-bluejay-review-20261111). This adds ordinary revised-kernel transfer evidence, not proof that a full-pool aggregation retry executed. The previous-head CI run 43449538659 completed successfully.

Rebased onto main on November 14, retaining both tests/chroot-extractors and tests/ncm-transfer after #676. The documentation also corrects the passive-package coordination status: the proposed boot-update.conf ownership follow-up was not taken, and #346 stays passive. Policy, the complete tests/all suite and all 32 package suites pass locally; independent review found no remaining blocker. Current-head hosted CI passed on a4217e82274ce09c5745ba043bdb85715a283092 (Policy and Tests: tests/all plus every package).

Second upstream rebase — November 14, 2026

Rebased again after the install-link test additions, preserving tests/chroot-extractors, tests/rootfs-smoke-link and every other upstream suite. The NCM suite now runs beside tests/phone-ssh rather than at the append-only tail, avoiding the repeated tail conflict. The transfer tool and its fixtures are unchanged from the previously green head.

Policy, the complete final tests/all run and all 33 package suites pass locally. Independent review found no remaining blocker. The previous-head green CI above remains historical. The first hosted run failed in unchanged upstream tests/stage-boot-files after #775. Upstream #776 fixed the named-only behavior and test. This PR has now rebased onto that fixed base (259be39), with all five topic patches unchanged by range-diff. Policy, the corrected upstream stage suite and the NCM suite pass again locally. Fresh PR-event CI passed on d891c66def3858676a751c2a1d54f7d55582625a against the fixed base (Policy and complete Tests: tests/all plus every package). The failed old run was not reused.

@latentoperator latentoperator changed the title tools: move the attended NCM transfer profile out of kernel selftests tools: move NCM transfer profile and refresh Bluejay review status Nov 11, 2026
@derekross

Copy link
Copy Markdown
Owner

Thanks, Chris. This is where I wanted the SSH transfer check to live, and the tool is careful in the ways that matter: it only reads /dev/zero on the phone, it stops at the first failure, and a peer or interface that starts with a dash can't turn into an ssh option. I ran it against a stand-in ssh and it behaves as described. The refreshed table in UPSTREAM matches what I see on the kernel PRs (#225 and #227 to #230 on fc7086a68fca with passing CI, #231 merged).

Two changes before I merge, both in tools/ncm-transfer:

  1. The header says SPDX-License-Identifier: GPL-2.0. New files here are MIT (CONTRIBUTING §7) unless they carry an upstream licence, and this one is yours, so please make it MIT.
  2. Please run the transfers through tools/phone-ssh --fresh (or take the same settings). As it is, the tool calls ssh in batch mode with its own options, so the key, the host-key handling and the interface binding that phone-ssh gets from OMAMOBILE_SSH_KEY, OMAMOBILE_SSH_KNOWN_HOSTS and OMAMOBILE_SSH_INTERFACE don't apply. Your description says your own run needed a wrapper for the key and known-hosts options, and I'd hit the same thing on the Pixel 7 Pro when I run it for kernel bt-uart-test: unwind HCI resources and stop on failed identity reads #225. While you're there, please show why a transfer failed: ssh's output is captured and dropped, so a refused host key, a missing key and a short read all print the same FAIL: SSH, size or digest check line. Naming the failed check and printing ssh's stderr is enough.

Smaller things, take them or leave them:

  • The stand-in tests you describe would be welcome in the tree as tests/ncm-transfer with a fake ssh, on its own line at the end of tests/all.
  • Exit code 4 and SKIP make sense in a selftest; here I'd make --peer and --interface required arguments.
  • The UPSTREAM paragraph about the November 11 replacements will be out of date at the next push to those PRs. The base commit and "built, not installed or booted" are the parts that last; the rest can stay in the kernel PRs.

The 7 Pro and laptop-sleep run for the driver stays with kernel #225.

@latentoperator
latentoperator force-pushed the followup/ncm-transfer-20261111 branch from f09b63c to 8ca0de9 Compare November 13, 2026 13:35
@latentoperator

latentoperator commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author

The November 12 review requests are addressed on 8ca0de97d5901d2333aeb0cf2612d41302eb81d5: MIT licence, transfers through the shared tools/phone-ssh --fresh path with its pinned settings, and useful failure diagnostics. The optional stand-in fixtures are now in tests/ncm-transfer and tests/all; peer/interface are required. The tool also rejects off/auto interface bypasses and cleans up the SSH process group on deadline/cancellation.

Policy, full tests/all, all 32 package suites, independent review and all sixteen live transfers passed on this rebased head. The tool subsequently passed the same sixteen 2 MiB SHA-checked reads on the revised Bluejay w0 kernel as well as the historical desktop kernel. Neither run establishes a full-pool aggregation retry; the Pixel 7 Pro/laptop-sleep driver qualification stays with kernel #225.

This PR was closed without merging; reopening it to keep the requested tooling follow-up reviewable. Reopening triggered CI run 43449538659: Policy has passed and Tests (tests/all, every package) is running at this publication check. Hosted tests are not yet claimed green. No new code change accompanies this comment.

@derekross

Copy link
Copy Markdown
Owner

Thank you, Chris. Both changes are in, and the rest went further than I asked. The transfers now go through tools/phone-ssh --fresh, so my usual key and host-key settings apply. A failure names the transfer and the check that failed and shows ssh's own message, and refusing off and auto as interface names is a nice touch. I read tests/ncm-transfer and ran it here: it drives the real phone-ssh against a stand-in ssh, so it can't reach a phone, and all eight checks pass. Killing the whole process group on a timeout or a dropped session is a good addition too. CI is green on this head.

One wording point in device/bluejay/UPSTREAM.md, which I can also fix when I merge: the paragraph about the "passive policy follow-up" says it is coordinated on #346 and needs #563 first. #563 has merged since, and we agreed on #346 to keep that follow-up out of the package, so as written it reads like a plan that's still on. Something like "a proposed follow-up that would make the package own boot-update.conf was not taken; #346 stays passive" would keep the record straight.

Your new line at the end of tests/all will meet a couple of other PRs that add theirs in the same place. That's a two-line resolution on my side at merge, so nothing to do there. Thanks again for moving this out of the kernel tree so cleanly.

@derekross

Copy link
Copy Markdown
Owner

Thanks, @latentoperator. This was ready, but #676 just merged and appended to tests/all too, so the two now conflict on its last lines. Please rebase on main and keep both lines (tests/chroot-extractors and tests/ncm-transfer), and I'll merge it once CI is green.

@latentoperator
latentoperator force-pushed the followup/ncm-transfer-20261111 branch 2 times, most recently from a4217e8 to 4d372ff Compare November 14, 2026 17:37
Keep the USB-interface-pinned SSH reproduction in mobile tooling as requested in linux-omarchy-mobile derekross#225. The kernel selftest remains host-only; document what the transfer profile does and does not establish.

Assisted-by: LLM
Refresh the review table after the November 11 signed replacements. Distinguish source/build results from the still-unbooted runtime stack and document the focused passive package coordination and migration dependency.

Assisted-by: LLM
Use phone-ssh for identity, host-key and direct-interface settings. Report the failed transfer/check and SSH diagnostics, stop the SSH process group on timeout or interruption, and reject helper control tokens as interface names.

Add offline fixtures covering successful transfers, early failures, settings, argument validation, deadlines and cancellation. The attached Bluejay also passed sixteen verified transfers on its historical kernel.
The boot-update.conf ownership follow-up was not accepted on derekross#346. Keep the package passive and describe the attended migration and smoke report that still gate it.

Assisted-by: OpenAI Codex
Keep this host SSH transport test beside tests/phone-ssh rather than at the append-only tail, where new upstream suites repeatedly conflict with it. Retain all upstream test entries.

Assisted-by: OpenAI Codex
@latentoperator
latentoperator force-pushed the followup/ncm-transfer-20261111 branch from 4d372ff to d891c66 Compare November 14, 2026 17:49
@derekross

Copy link
Copy Markdown
Owner

Thanks, Chris. The rebase is just what I asked for: both suites are still there, and tools/ncm-transfer and its test are byte for byte the ones I read last time. I ran tests/ncm-transfer again on this head merged onto today's main and it passes, and CI is green. The new UPSTREAM sentence about the boot-update.conf follow-up reads right too, thank you.

Moving the line up next to tests/phone-ssh is fine with me. It's on its own line, so it won't meet anyone else's addition.

One small thing for a later Bluejay docs change, not this one: the UPSTREAM table still lists kernel #226 as "awaiting maintainer merge", and it merged on the 13th.

Merging now.

@derekross
derekross merged commit 1681f5e into derekross:main Nov 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants