From 2f3aebcbd211533627f77ef19813d76defd9e8fb Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Tue, 10 Nov 2026 17:36:04 -0643 Subject: [PATCH 1/8] device/tegu SLEEP-PLAN, bringup: what is left brought up to date, the loop's start check names its run Items 2 and 3 of "What is left" were overtaken: the sleep hook merged as omamobile-device-tegu 0.2.11, and sleep with the modem up was run with the fixes on 2026-11-09 (MODEM-PLAN has the record, #618 to #620 what is open). The loop read the last "series unit:" line of a log that grows across runs, so a start that printed none read the previous run's; the line now carries the run number. modem-sleep-test.sh says what "detached" means. --- device/tegu/SLEEP-PLAN.md | 14 ++++++++++---- device/tegu/bringup/modem-sleep-test.sh | 3 ++- device/tegu/bringup/sleep-series-loop.sh | 7 ++++--- 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index 20e6eda90..a856bca29 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -125,10 +125,16 @@ missing pull, NFC polling (stopped by the hook throughout), the modem core doesn't answer an NMI either, it is stuck below the kernel or stopped, and the series goes on with the PMIC commits alone, without #224's two. -2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): its series is in (the - last note below, 40 of 40); it waits for its merge. -3. Sleep with the modem up (M6): never run with the fixes; the one try - before them (10-27) also had NFC polling. +2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): merged on 2026-11-10 as + `omamobile-device-tegu` 0.2.11; its series is the 40 of 40 in the notes + below. +3. Sleep with the modem up (M6): run with the fixes on 2026-11-09, five + boots, and it neither hung nor reset, but one attempt costs the phone + service until a reboot. The record is [MODEM-PLAN](MODEM-PLAN.md), "Suspend + with the modem on, with the sleep fixes"; what is open is in + [#618](https://github.com/derekross/omarchy-mobile/issues/618), + [#619](https://github.com/derekross/omarchy-mobile/issues/619) and + [#620](https://github.com/derekross/omarchy-mobile/issues/620). 4. The root cause under the masks: zuma's PMU data without `pmu_cpuhp`, compared with gs101/gs201 before idle states come to a PR. 5. The DEVICES cell to "works" once item 1's no-wake is found and fixed, diff --git a/device/tegu/bringup/modem-sleep-test.sh b/device/tegu/bringup/modem-sleep-test.sh index 28d637a55..2542a8a89 100755 --- a/device/tegu/bringup/modem-sleep-test.sh +++ b/device/tegu/bringup/modem-sleep-test.sh @@ -9,7 +9,8 @@ # CP's and the phone service's state after; "omamobile-sit status" shows the last state # read, so check sitd's journal for "poll failed" as well. Stops when the CP isn't ONLINE, # sitd isn't active or a UFS line appears. Attended, on the charger (AGENTS.md, "Suspend -# with the modem on"). Run detached: +# with the modem on"). Run detached from the SSH session, so a lost USB link doesn't end +# it, and with someone at the palmtop: # systemd-run --unit=x-modem-sleep /root/modem-sleep-test.sh [N] [S] # and when it is done, the last step is always "systemctl reboot", never reboot-to, so the # modem stops cleanly and the phone service comes back with the next boot. diff --git a/device/tegu/bringup/sleep-series-loop.sh b/device/tegu/bringup/sleep-series-loop.sh index 75f0fea23..d890dcee6 100755 --- a/device/tegu/bringup/sleep-series-loop.sh +++ b/device/tegu/bringup/sleep-series-loop.sh @@ -31,12 +31,13 @@ start_series() { # The first sleep waits for the graphical session: a suspend while the # shell is still starting after a boot left the session closed and the # splash on the screen (2026-11-07 13:16). - $PS --fresh 'systemctl unmask sleep.target suspend.target >/dev/null 2>&1; for i in $(seq 36); do [ -d /run/user/1000/hypr ] && pgrep -x Hyprland >/dev/null && break; sleep 5; done; echo "hyprland after boot: $(pgrep -x Hyprland >/dev/null && echo up || echo not up), uptime $(cut -d. -f1 /proc/uptime) s"; systemd-run --unit=x-audio-loop --wait -p TimeoutStartSec=300 /usr/local/bin/omamobile-tegu-audio-up >/dev/null 2>&1; systemctl reset-failed x-sleep-linkwait 2>/dev/null; sleep 30; systemd-run --unit=x-sleep-linkwait -E WFI='"$WFI"' -E RPM='"${RPM:-}"' -E N='"$N"' -E S='"$S"' bash /root/sleep-linkwait.sh; sleep 0.3; echo "series unit: $(systemctl is-active x-sleep-linkwait)"' >> $H/loop.log 2>&1 + $PS --fresh 'systemctl unmask sleep.target suspend.target >/dev/null 2>&1; for i in $(seq 36); do [ -d /run/user/1000/hypr ] && pgrep -x Hyprland >/dev/null && break; sleep 5; done; echo "hyprland after boot: $(pgrep -x Hyprland >/dev/null && echo up || echo not up), uptime $(cut -d. -f1 /proc/uptime) s"; systemd-run --unit=x-audio-loop --wait -p TimeoutStartSec=300 /usr/local/bin/omamobile-tegu-audio-up >/dev/null 2>&1; systemctl reset-failed x-sleep-linkwait 2>/dev/null; sleep 30; systemd-run --unit=x-sleep-linkwait -E WFI='"$WFI"' -E RPM='"${RPM:-}"' -E N='"$N"' -E S='"$S"' bash /root/sleep-linkwait.sh; sleep 0.3; echo "series unit, run '"$run"': $(systemctl is-active x-sleep-linkwait)"' >> $H/loop.log 2>&1 # A unit that failed at its start (once: the script's path given as the command, with no # execute bit) stops the loop here. The check runs in the start's own command, because the # first sleep begins within a second or two; no line at all means the palmtop was already - # going to sleep, and the poll below judges that. - st=$(grep -o 'series unit: [a-z]*' $H/loop.log | tail -1 | cut -d' ' -f3) + # going to sleep, and the poll below judges that. The line carries the run number, since + # the log grows across runs and a start that printed none would read the run before's. + st=$(grep -o "series unit, run $run: [a-z]*" $H/loop.log | tail -1 | awk '{print $NF}') if [[ $st == failed || $st == inactive ]]; then log "the series unit is '$st' after its start: stop (a keep-awake inhibitor, if any, is still held)"; send "Palmtop sleep series: the series did not start ($st). Nothing is running; the palmtop is as it was."; exit 1; fi log "series started (run $run, WFI=$WFI)" } From 94def493a60e988ed59ec37ed85908083c2fd2d3 Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Wed, 11 Nov 2026 06:44:06 -0644 Subject: [PATCH 2/8] device/tegu SLEEP-PLAN, bringup, AGENTS: the tracing build's first evening, and a sleep with no alarm taken for a hang On 2026-11-10 the published tree plus kernel #224, with the function tracer, pm_print_times, pm_debug_messages and panic_on_rcu_stall, entered 96 sleeps and none hung: 94 woke on the alarm, and one, which the shell's auto-sleep started between two cycles with no alarm, lasted nine and a half hours until the Power key. From the laptop that looked like the no-wake, and it was reported as one before the journal was read. sleep-series.sh now holds a block inhibitor of its own for the run and sleeps past it, so only its own sleeps can happen, and refuses to start with the modem's driver loaded. It waits for the sleep itself (logind can hold one back 15 s for a delay inhibitor), and PMDEBUG=1 and TRACE=1 turn on the no-wake debugging: the ring filled before a panic can wrap it, the runtime watchdog off for the run. The plan has the four test crashes that showed why, and "What is left" item 1 says what runs next. --- AGENTS.md | 2 +- device/tegu/SLEEP-PLAN.md | 51 ++++++++++++++------- device/tegu/bringup/sleep-series-loop.sh | 6 +-- device/tegu/bringup/sleep-series.sh | 57 +++++++++++++++++++----- 4 files changed, 86 insertions(+), 30 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e60b56d4f..2305d4738 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -186,7 +186,7 @@ Each entry gives the gotcha, then the reason or the fix. Details are in the link ### Suspend and modem - **On the 9a, NFC must not poll through s2idle.** A suspend with neard polling reset the palmtop (2026-10-27); with polling stopped, 35 s2idle cycles with RTC wakes that day all woke (`7.3.0-rc6-omamobile-tegu-s2mpg15-m1`). The cause isn't known: nxp-nci has no PM callbacks, but neither does the 7 Pro's st-nci, so the controller's interrupt or the I2C bus during the freeze are the next suspects. A sleep hook on the palmtop, `omamobile-tegu-nfc` (not in the repository yet; it comes with tegu's device package), stops neard's poll loop before a systemd sleep and restarts it after; a sleep written straight to `/sys/power/state` (`rtcwake -m mem`) skips the hook, so stop polling by hand first. Without [linux-omarchy-mobile#131](https://github.com/derekross/linux-omarchy-mobile/pull/131) the ST54L also doesn't answer on I2C after a wake (`Error -6 on I2C send`, also after the driver's retry), or after about 2-3 s of idle at any time, until neard powers it off and on ([#132](https://github.com/derekross/omarchy-mobile/issues/132)): nxp-nci never turned its NFC mode on, as ST's HAL does after init. With #131 (the results below on `core.c` blob `77fc35b61da5`, #131's first version; the merged version, `c96704ddd480`, re-run for six power-ons and polling at 3 s and 30 s; the release, `7.3.0-rc6-omamobile-tegu-s2mpg15-m1-g991042e2eff3-dirty`, is the same as without the fix), NFC polls after 3 s to 3 minutes of idle and polled again after two RTC-woken s2idle cycles (2026-10-29): one with a hook that stops polling and powers the adapter off, and one without that hook, for which it isn't recorded whether polling was stopped, so whether NFC comes back after a wake with the stop-poll hook alone rests on that one cycle and needs a re-run with the hook's log. No power-off hook is needed for NFC; the stop-poll rule above stands on its 35 of 35. [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) "Suspend with the modem on" -- **On the 9a, a new sleep test runs attended first, and the direct freeze path is still not used.** Unattended sleep series are allowed since 2026-11-05 (#409) on the contributor's conditions and three of the maintainer's. The contributor's: on the charger with the cable in; the modem off for the boot; the script stops itself on any UFS error line or oops; a wake with dead storage (SSH still answers) gets `echo b > /proc/sysrq-trigger` remotely, and only a silent hang waits for the owner's buttons; the owner says yes before each run. (Their "slot B marked" condition is not part of it: marking B on the 9a stays on hold, see the boot chain notes above, so a series that resets the palmtop spends ABL tries and stays short.) The maintainer's: nothing new (a kernel, a module or a script) runs unattended before one attended run of it; the automatic sysrq recovery is capped per night, and the next boot's ext4 recovery is checked and recorded; and the recovery script is in the tree (`device/tegu/bringup/`, where the series scripts already are) before it runs unattended. The plan says which rule each step runs under, and the counts below are the early ones; [tegu SLEEP-PLAN](device/tegu/SLEEP-PLAN.md) keeps the current tally. Four sleeps have never woken (not from the RTC alarm, not from the power key; Power + Volume Up to reset, nothing in pstore): two direct `rtcwake -m freeze` runs (2026-10-28, 2026-10-29), one 30-minute `systemctl suspend` with `tegu-mif-sleep` loaded (2026-10-30, cable out) and the sixth 30 s `systemctl suspend` cycle on 2026-10-30 (the daily boot, no module; it left the USB bus); 35 short and seven longer systemd sleeps woke (two 30-minute and four 240 s ones on 2026-10-29, a cable-in 30-minute one with the module on 2026-10-30; the afternoon's cable-out run with the module makes eight). No single factor explains it, so keep `power-ab` on `--suspend systemd` (the sleep hooks run that way), and don't load `tegu-mif-sleep` at boot until the hang is understood. The shell's auto-sleep (the `omamobile-sleep` toggle, systemd path, no module) woke every time while it was on, including seven sleeps of 11 s to 22 minutes in a day of use on 2026-11-01 (lengths from the RTC); the package ships it off, and the owner turned it back on on the palmtop that day, so keeping it on is the owner's call. Two causes are known. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224) re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules, no console), none in two re-runs of 36 with modules loaded, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through). So a palmtop that stays silent after a sleep may be a running kernel with one core locked up, not a SoC that stayed down; what that core was doing isn't known. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; on a kernel from before it `rtcwake` fails and a sleep ends only on a key. Check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. The rule above stands. [tegu SLEEP-PLAN](device/tegu/SLEEP-PLAN.md) S1 +- **On the 9a, a new sleep test runs attended first, and the direct freeze path is still not used.** Unattended sleep series are allowed since 2026-11-05 (#409) on the contributor's conditions and three of the maintainer's. The contributor's: on the charger with the cable in; the modem off for the boot; the script stops itself on any UFS error line or oops; a wake with dead storage (SSH still answers) gets `echo b > /proc/sysrq-trigger` remotely, and only a silent hang waits for the owner's buttons; the owner says yes before each run. (Their "slot B marked" condition is not part of it: marking B on the 9a stays on hold, see the boot chain notes above, so a series that resets the palmtop spends ABL tries and stays short.) The maintainer's: nothing new (a kernel, a module or a script) runs unattended before one attended run of it; the automatic sysrq recovery is capped per night, and the next boot's ext4 recovery is checked and recorded; and the recovery script is in the tree (`device/tegu/bringup/`, where the series scripts already are) before it runs unattended. The plan says which rule each step runs under, and the counts below are the early ones; [tegu SLEEP-PLAN](device/tegu/SLEEP-PLAN.md) keeps the current tally. Four sleeps have never woken (not from the RTC alarm, not from the power key; Power + Volume Up to reset, nothing in pstore): two direct `rtcwake -m freeze` runs (2026-10-28, 2026-10-29), one 30-minute `systemctl suspend` with `tegu-mif-sleep` loaded (2026-10-30, cable out) and the sixth 30 s `systemctl suspend` cycle on 2026-10-30 (the daily boot, no module; it left the USB bus); 35 short and seven longer systemd sleeps woke (two 30-minute and four 240 s ones on 2026-10-29, a cable-in 30-minute one with the module on 2026-10-30; the afternoon's cable-out run with the module makes eight). No single factor explains it, so keep `power-ab` on `--suspend systemd` (the sleep hooks run that way), and don't load `tegu-mif-sleep` at boot until the hang is understood. The shell's auto-sleep (the `omamobile-sleep` toggle, systemd path, no module) woke every time while it was on, including seven sleeps of 11 s to 22 minutes in a day of use on 2026-11-01 (lengths from the RTC); the package ships it off, and the owner turned it back on on the palmtop that day, so keeping it on is the owner's call. Two causes are known. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224) re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules, no console), none in two re-runs of 36 with modules loaded, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through). So a palmtop that stays silent after a sleep may be a running kernel with one core locked up, not a SoC that stayed down; what that core was doing isn't known. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; on a kernel from before it `rtcwake` fails and a sleep ends only on a key. Check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. The rule above stands. A sleep series has to keep the shell's auto-sleep out: with it on, `omamobile-sleepd` puts a wake that leaves the screen off behind a secure lock back to sleep after 5 s, with no alarm, and from the laptop that sleep looks like a no-wake (2026-11-10, twice; the second lasted nine and a half hours and was reported as a hang). `device/tegu/bringup/sleep-series.sh` holds a block inhibitor of its own for that; before calling a silent palmtop hung, ask for one short press of the Power key and read the journal. [tegu SLEEP-PLAN](device/tegu/SLEEP-PLAN.md) S1 - **On the 9a, a sleep can end badly in two ways: UFS fails to come back and the root filesystem goes read-only, or the SoC hangs and leaves the USB bus.** The first seen once on 2026-10-30 (`link startup failed 1`, `Host init failed 1` x5, `ufshcd_wl_resume failed`, then `I/O error, dev sda` and ext4 `emergency_ro`; `spm_lvl` 5). The palmtop woke on its alarm but nothing on disk could be read: `systemctl suspend`/`reboot` did nothing, the power key could not light the screen, SSH kept answering from memory. A dark palmtop that ignores the power key may therefore be awake with dead storage, not asleep. After every s2idle, `dmesg | grep -E "link startup failed|Host init failed|I/O error"` first; if it matches, `echo b > /proc/sysrq-trigger` (binaries won't load), then check the ext4 recovery on the next boot. The dead storage is understood: gph5-0 and gph5-1 (the UFS reference clock and reset, `13060000.pinctrl`) read `CON 0x0` after such a wake, so the device gets no clock and no reset edge; linux-omarchy-mobile#224 re-applies the UFS pin state in `exynos_ufs_resume()`. On a kernel without it the check above stays. [tegu SLEEP-PLAN](device/tegu/SLEEP-PLAN.md) S1 - **UFS must be at `spm_lvl` 5 in system suspend.** At the core default 3, UFS failed its first hibern8 exit on resume and needed a host reset (it recovered, no I/O errors); at 5 the first real s2idle freeze resumed clean. gs201 defaults to 5 in the kernel (S3a, `d0c08c464ef8` on `omamobile`), and gs101 too since linux-omarchy-mobile#188 (2026-11-04), so a suspend test only checks that `spm_lvl` reads 5; on a kernel without those commits, `echo 5` first. [SUSPEND-PLAN](device/cheetah/SUSPEND-PLAN.md) S1-S3 - **Only s2idle, never `deep`.** The firmware advertises PSCI SYSTEM_SUSPEND, so mainline offers `deep`, but Linux has no gs201 SoC sleep sequence (SYS_SLEEP), and the phone wouldn't wake. `mem_sleep_default=s2idle` in the bootargs and device-cheetah's `sleep.conf` drop-in keep it on s2idle. [SUSPEND-PLAN](device/cheetah/SUSPEND-PLAN.md) §1, §5 diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index a856bca29..3918d6f94 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -73,9 +73,9 @@ boot unless said; `systemctl suspend` with an RTC alarm, 60 s unless said; "attended" means a person at the laptop watching the loop with the owner at home for a button press; the first run of every new kernel was watched that way, and the long unattended stretches ran under the loop's recovery budget, -with one exception, the last row: a new build's first run went overnight -with the owner asleep, started by hand over SSH and followed only by a poll -that could report, which is not what the rule asks): +with one exception, the overnight run of 11-10: a new build's first run went +overnight with the owner asleep, started by hand over SSH and followed only +by a poll that could report, which is not what the rule asks): | Kernel (`uname -v`) | Change under test | Cores in sleep | Result | Attended | | --- | --- | --- | --- | --- | @@ -96,6 +96,7 @@ that could report, which is not what the rule asks): | the same sources (`push1-g6c9420bfa77f`), modules installed, ramoops on | the re-run | WFI | **36 of 36** (60 s, `pm_wakeup_irq` 52 each, no UFS line) | yes | | the same sources, build #2 with `CONFIG_PHY_ZUMA_PCIE=m` (#274) | the same with Wi-Fi up and kept across the sleeps | WFI | **36 of 36**, wlan0 passing traffic after | the start yes, then the loop | | published `2f890a533` (11-10) + #224 + #236 + #237 (`push1-gff4fdf58035e`, build #4), modules installed, Wi-Fi kept, the hook of #591 installed | the newest tip, overnight | WFI | a 27-minute sleep ended by the Power key, then 212 woke and the 213th never did: CPU 7 stopped answering, the console kept | no (the owner asleep; his reset in the morning) | +| the same sources with the function tracer built in (`push1-dbg-gff4fdf58035e`, build #5), `pm_print_times`, `pm_debug_messages` and `kernel.panic_on_rcu_stall=1` on, modules installed, Wi-Fi kept, the hook of #591 installed | whether the no-wake comes back, with a console that would place it | WFI | 96 sleeps entered, no hang: 94 woke on the alarm (a test sleep, 90 with a 10 s alarm, three with a 35 s one); one, the shell's auto-sleep between two cycles with no alarm, lasted 9 h 33 min until the Power key; the 96th was cut by a button reset asked for in error | the first run yes, the owner at the palmtop; he left as the second began, and the long sleep was overnight | Before #96, one wake in 15 to 215 lost the storage; before #98, C2 in the sleep lost the palmtop after 6 to 135 cycles. @@ -112,19 +113,24 @@ missing pull, NFC polling (stopped by the hook throughout), the modem **What is left.** -1. The no-wake on the published kernel. On that tree 307 sleeps have woken - (21, 36, 36, two ended by the Power key, 212) and two have not: the 22nd - of 11-09's first run, with no console, and the 213th of 11-10's, whose - console shows CPU 7 not answering while the rest of the kernel runs - (failure 3 above). Next: the same series on a build that can show what - that core is doing and that recovers by itself, with pseudo-NMI - (`CONFIG_ARM64_PSEUDO_NMI=y`, `irqchip.gicv3_pseudo_nmi=1`), so the - stall report's backtrace reaches a core with interrupts masked, and +1. The no-wake on the published kernel. On that tree 402 sleeps have woken + (21, 36, 36, two ended by the Power key, 212, and the 95 of the evening + of 11-10) and two have not: the 22nd of 11-09's first run, with no + console, and the 213th of 11-10's, whose console shows CPU 7 not + answering while the rest of the kernel runs (failure 3 above). It hasn't + come back since. What is set up for it, in place of the pseudo-NMI build + planned here before (the maintainer's pointers on kernel #224): + `pm_print_times` and `pm_debug_messages`, so the console names each + device's suspend callback as it starts and returns; `kernel.panic_on_rcu_stall=1`, so the stall becomes a panic, a reboot - through `panic=10` and a console in pstore with no buttons. If the - core doesn't answer an NMI either, it is stuck below the kernel or - stopped, and the series goes on with the PMIC commits alone, without - #224's two. + through `panic=10` and a console in pstore with no buttons; and, on a + build with the function tracer, each core's last calls in the panic's + dump (`PMDEBUG=1` and `TRACE=1` in `sleep-series.sh`). The first evening + that way, 11-10, was 93 alarm wakes with the switches on and no hang + (the last note below). Next: more series the same way, to about 400 + wakes with the switches on, which is what the maintainer asks for before + #224 merges if the hang stays away; if it comes back, the console says + where it stopped. 2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): merged on 2026-11-10 as `omamobile-device-tegu` 0.2.11; its series is the 40 of 40 in the notes below. @@ -142,7 +148,10 @@ missing pull, NFC polling (stopped by the hook throughout), the modem have run from the published kernel. **Scripts** (`device/tegu/bringup/`): `sleep-series.sh` (on the palmtop: -N RTC-woken `systemctl suspend` cycles of S s, records each wake), +N RTC-woken `systemctl suspend` cycles of S s, records each wake; it holds a +sleep inhibitor of its own for the run, so the shell's auto-sleep can't +take the palmtop between two cycles, and `PMDEBUG=1` and `TRACE=1` turn on +the no-wake debugging of item 1 above), `sleep-series-loop.sh` (laptop: polls the series, captures a dead wake, recovers with sysrq under a budget of MAXREC recoveries per 24 hours, records the next boot's ext4 recovery, restarts the series; it unmasks `sleep.target` and `suspend.target` for the run and puts them back as it found them when it exits), @@ -4836,3 +4845,13 @@ What it says: So this no-wake isn't the SoC staying down, which is what failure 2 looked like on #97 (the last CPU going down, then the watchdog's reset). It is one core locked up with the rest of the kernel running, and nothing watches for that during a sleep: the cluster watchdog is stopped for it and systemd is frozen. What it doesn't say: what CPU 7 was doing. The backtrace IPI is an ordinary interrupt on this build (`CONFIG_ARM64_PSEUDO_NMI` is off), so a core with interrupts masked, or one stuck below the kernel, can't answer it. Nor whether the 22nd sleep of 11-09 was the same thing: that boot had no ramoops. "What is left" item 1 has the next run. + +**The tracing build's first evening: 94 alarm wakes, an unplanned sleep of nine and a half hours, and a false alarm (2026-11-10, 19:42, to 11-11, 06:19 MST palmtop clock).** `7.3.0-rc6-omamobile-push1-dbg-gff4fdf58035e`, build #5: the sources of the series above (the published tree with kernel #236 and #237, plus #224) with the function tracer added (`CONFIG_FUNCTION_TRACER`, `CONFIG_DYNAMIC_FTRACE`; pseudo-NMI is built in but not on the command line), its own modules installed, the published DTB with the `public-ramoops` and `modem-cpif` overlays, the modem boot unit masked for the boot, Wi-Fi kept, the cpuidle hook installed. The series is `sleep-series.sh` with what is now `PMDEBUG=1` and `TRACE=1`: `pm_print_times`, `pm_debug_messages`, `kernel.panic_on_rcu_stall=1`, and function and event tracing in 24 KB per CPU, dumped by a panic (`ftrace_dump_on_oops`). The owner was at the palmtop from 19:41 to about 20:45. + +- **Four test crashes first, and what they showed about the dump** (`echo c > /proc/sysrq-trigger`, awake, each on a fresh boot, between 19:46 and 19:58). In the first three the saved console ended about 3,460 kernel lines after its start, in the middle of the trace dump, with no "Rebooting in 10 seconds.." line, and the bootloader's record of the reset was the watchdog's (`APC Watchdog Early`), not the panic's reboot. The console in pstore starts at 2.5 s of uptime, so with the lines before that each ended at about record 4096 since boot, which is the number of records the kernel's log ring holds on this build (`CONFIG_LOG_BUF_SHIFT=17`): a panic in which the ring wraps for the first time lost everything after the wrap. For the fourth the ring was filled beforehand (5,000 lines written to `/dev/kmsg`) and systemd's runtime watchdog (10 s on the palmtop, from a bring-up drop-in) was turned off: the console has the whole dump (about 950 events for each of the four cores that were traced), the dump's end marker and "Rebooting in 10 seconds..", and the palmtop rebooted by itself 32 s after the crash. Two things changed at once in the fourth, so the watchdog's part isn't separated from the ring's; the series does both. A dump of 24 KB per CPU fits the 2 MB console with room for the sleep's own lines (about 64 KB of them per cycle with `pm_print_times`). +- **A test sleep, then the first run** (boot at 19:58:23). One `systemctl suspend` with a 25 s alarm at 19:59 for omarchy-mobile#619's `SuspendState=freeze` (MODEM-PLAN has it), before the switches were on. Then a series with a 10 s alarm from 20:00: 90 sleeps, 89 of them 19.1 to 19.8 s by the kernel log (why a 10 s alarm gives a sleep that long wasn't looked at), each woken by the alarm (`pm_wakeup_irq` 53), no UFS line, no ACPM timeout, no oops. The run stopped itself at the 90th with "no suspend happened": logind had held that sleep back for NetworkManager's delay inhibitor (up to 15 s here), the sleep began after the script's fixed 10 s wait had ended, and it lasted 6 s and woke on the alarm too. The script now waits for `suspend_stats/success` to move, up to 45 s. +- **The second run and the unplanned sleep.** A series with a 35 s alarm from 20:43, the owner leaving as it began. Two sleeps of 34.5 s woke on the alarm. The third did too, at 20:45:24, and seven seconds later the shell's auto-sleep, which is on on this palmtop, put the palmtop back to sleep, as it does 5 s after any wake that leaves the screen off behind a secure lock (`omamobile-sleepd: sleeping (dark resume, 5 s grace)`). No alarm was armed for that sleep. It lasted from 20:45:33 to 06:18:54, 9 hours 33 minutes, and ended at the owner's Power key (`PM: Triggering wakeup from IRQ 57`, `Power`), with no UFS line. In the first run the daemon logged nothing, most likely because the lock wasn't secure with the owner at the palmtop, and after the second run's first two wakes an SSH poll from the laptop happened to be open ("not sleeping: an SSH session is open"), so the daemon held off until the third. +- **The false alarm.** From the laptop a sleep with no alarm and a hang look the same: the USB device gone and no reconnect. It was taken for the no-wake, reported as that on kernel #224 the same night, and the owner was asked to reset the palmtop in the morning. His first press of the Power key woke it; the series went on, logged its third cycle and entered a fourth sleep at 06:19:02, and the Power + Volume Up reset landed in that one. The journal of the boot, read after it, has 96 `PM: suspend entry` and 95 `PM: suspend exit` lines, no RCU stall, no panic and no UFS error line. The report on #224 was corrected the same morning. +- **Why it happened, and the fix.** The series released the push script's keep-awake inhibitor (`x-awake`) at its first cycle, since a block inhibitor refuses its own sleeps too, and from then on nothing stood between the auto-sleep and the palmtop in the seconds after each wake. This is the trap of the unplanned first sleep in the note above, a second time. `sleep-series.sh` now holds a block inhibitor of its own from start to end and sleeps past it with `systemctl suspend -i`, so the only sleeps during a run are its own, each with an alarm; it refuses to start with the modem up (the unit active or `cpif` loaded), so the modem's inhibitor is never one of those it passes. Not yet run in that form. + +What it adds to the count: 93 sleeps woken by the alarm with the PM switches on, and one of nine and a half hours on this tree ended by the Power key, which the two Power-key presses of the morning of 11-10 could not do for the sleep that hung. diff --git a/device/tegu/bringup/sleep-series-loop.sh b/device/tegu/bringup/sleep-series-loop.sh index d890dcee6..6f9cf5fad 100755 --- a/device/tegu/bringup/sleep-series-loop.sh +++ b/device/tegu/bringup/sleep-series-loop.sh @@ -25,9 +25,9 @@ start_series() { for i in $(seq 40); do $PS --fresh true 2>/dev/null && break; sleep 10; done timeout 30 $PS --fresh "cat $L 2>/dev/null" > $H/series-before-run$run.log 2>/dev/null $PS --fresh 'cat > /root/sleep-linkwait.sh' < device/tegu/bringup/sleep-series.sh - # A block inhibitor left by the push scripts (x-awake) would refuse every sleep of the - # series. The series releases it itself once its first alarm is armed (sleep-series.sh); - # it is not released here, so a start that fails leaves the palmtop held awake. + # A block inhibitor left by the push scripts (x-awake) is left alone: the series holds one + # of its own and sleeps past both (sleep-series.sh), so a start that fails leaves the + # palmtop held awake, and the shell's auto-sleep can't take it between two cycles. # The first sleep waits for the graphical session: a suspend while the # shell is still starting after a boot left the session closed and the # splash on the screen (2026-11-07 13:16). diff --git a/device/tegu/bringup/sleep-series.sh b/device/tegu/bringup/sleep-series.sh index 389f65a6d..77029d0e1 100644 --- a/device/tegu/bringup/sleep-series.sh +++ b/device/tegu/bringup/sleep-series.sh @@ -6,12 +6,38 @@ # the NFC hook stops polling, console kept, cable in, modem not started this # boot. Records each sleep's RTC length, dmesg length, wake IRQ, UFS and ACPM # errors, and the link-wait lines (a slow wake rather than a dead one). -N=${N:-300}; S=${S:-60}; L=${L:-/var/log/omamobile/sleep-linkwait-20261107.log}; U=/sys/devices/platform/soc/13200000.ufs +N=${N:-300}; S=${S:-60}; W=${W:-20}; L=${L:-/var/log/omamobile/sleep-linkwait-20261107.log}; U=/sys/devices/platform/soc/13200000.ufs mkdir -p /var/log/omamobile; : > $L +# Debugging a no-wake (2026-11-10; off unless asked for). PMDEBUG=1 turns pm_print_times and pm_debug_messages +# on, so the console names each device's suspend callback as it starts and returns (the maintainer's pointer, +# kernel #224), and makes an RCU stall a panic (kernel.panic_on_rcu_stall), which panic=10 turns into a reboot +# with the console in pstore. TRACE=1, on a kernel with the function tracer, also runs function and event +# tracing in small per-CPU buffers that the panic dumps to the console. The dump has to fit the 2 MB ramoops +# console together with the sleep's own callback lines: it prints oldest first, and 24 KB per CPU is about 900 +# calls a core and under 1 MB of text for eight (measure it with an awake sysrq-c before a series). +# Either one first fills the kernel's log ring: three test crashes on the tracing build (sysrq-c, awake) lost +# everything after record 4096 since boot, the trace dump's tail and the "Rebooting" line with it, when the +# ring's first wrap fell inside the panic; with the ring filled beforehand the fourth kept the whole dump. +if [ "${PMDEBUG:-0}" = 1 ] || [ "${TRACE:-0}" = 1 ]; then + echo on > /proc/sys/kernel/printk_devkmsg 2>/dev/null; for i in $(seq 1 5000); do echo "ring fill $i" > /dev/kmsg; done + echo 1 > /sys/power/pm_print_times; echo 1 > /sys/power/pm_debug_messages + echo 1 > /proc/sys/kernel/panic_on_rcu_stall + # systemd's runtime watchdog (10 s on the palmtop) would reset a panicking kernel before the dump and the + # panic's own reboot are through (32 s in the test that kept its dump): off for the run, put back at the end. + WD=org.freedesktop.systemd1; wd_before=$(busctl get-property $WD /org/freedesktop/systemd1 $WD.Manager RuntimeWatchdogUSec | awk '{print $2}') + busctl set-property $WD /org/freedesktop/systemd1 $WD.Manager RuntimeWatchdogUSec t 0 +fi +if [ "${TRACE:-0}" = 1 ] && [ -d /sys/kernel/tracing ]; then + T=/sys/kernel/tracing; mountpoint -q $T || mount -t tracefs nodev $T + echo 0 > $T/tracing_on; echo ${TRACE_KB:-24} > $T/buffer_size_kb; echo function > $T/current_tracer + for e in sched/sched_switch irq/irq_handler_entry irq/irq_handler_exit ipi power/cpu_idle power/suspend_resume timer/hrtimer_expire_entry; do echo 1 > $T/events/$e/enable; done + echo 1 > $T/tracing_on; echo 1 > /proc/sys/kernel/ftrace_dump_on_oops +fi +echo "debug: pm_print_times $(cat /sys/power/pm_print_times), pm_debug_messages $(cat /sys/power/pm_debug_messages), panic_on_rcu_stall $(cat /proc/sys/kernel/panic_on_rcu_stall), panic timeout $(cat /proc/sys/kernel/panic) s, tracer $(cat /sys/kernel/tracing/current_tracer 2>/dev/null || echo none) $(cat /sys/kernel/tracing/buffer_size_kb 2>/dev/null) KB per CPU on $(cat /sys/kernel/tracing/tracing_on 2>/dev/null), ftrace_dump_on_oops $(cat /proc/sys/kernel/ftrace_dump_on_oops 2>/dev/null)" >> $L uid=$(id -u mobile) hy() { runuser -u mobile -- env XDG_RUNTIME_DIR=/run/user/$uid HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$uid/hypr | head -1) hyprctl dispatch "hl.dsp.dpms({ action = \"$1\" })" >/dev/null 2>&1; } echo N > /sys/module/printk/parameters/console_suspend -systemctl is-active -q omamobile-modem && { echo "the modem is up: stop" >> $L; exit 1; } +{ systemctl is-active -q omamobile-modem || [ -d /sys/module/cpif ]; } && { echo "the modem is up: stop" >> $L; exit 1; } # The cpuidle sleep hook (omarchy-mobile#591; an earlier hook of this name was withdrawn in #255): when it is # installed and its keep flag is absent, every sleep of the series runs in WFI whatever WFI= says. Noted, not refused. if [ -e /usr/lib/systemd/system-sleep/omamobile-tegu-cpuidle ]; then echo "cpuidle hook installed, keep flag $([ -e /etc/omamobile/cpuidle-keep-across-sleep ] && echo present || echo absent)" >> $L; fi @@ -23,6 +49,8 @@ state1_before=$(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/disable) # always-on is cut only by the system suspend, not by every idle period. rpm_before=$(cat $U/rpm_lvl) restore() { + [ -n "$wd_before" ] && busctl set-property org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager RuntimeWatchdogUSec t $wd_before + [ -n "$INH" ] && kill $INH 2>/dev/null for f in /sys/devices/system/cpu/cpu*/cpuidle/state1/disable; do echo $state1_before > $f; done echo 5 > $U/spm_lvl echo $rpm_before > $U/rpm_lvl @@ -30,6 +58,15 @@ restore() { echo "restored: spm_lvl $(cat $U/spm_lvl), state1 disable $state1_before, modem boot unit $(systemctl is-enabled omamobile-tegu-modem-boot.service 2>/dev/null) (was $modem_unit_before)" >> $L } trap restore EXIT +# Only the series' own sleeps, each with its alarm, may happen during a run. The shell's auto-sleep +# (omamobile-sleepd) puts a dark resume back to sleep 5 s after it once the lock is secure and no SSH session +# is open, with no alarm: it did between two cycles on 2026-11-10, and the palmtop slept nine and a half +# hours, until the Power key, which from the laptop looked like a no-wake. So the series holds a block +# inhibitor of its own for the whole run (a push script's x-awake has a time limit) and sleeps past it +# with "systemctl suspend -i"; the modem's inhibitor is not passed that way, since the series refuses +# to start with the modem up. +systemd-inhibit --what=sleep --who=sleep-series --why="only the series' own sleeps, each with an alarm" --mode=block sleep infinity & +INH=$! for f in /sys/devices/system/cpu/cpu*/cpuidle/state1/disable; do echo ${WFI:-1} > $f; done echo 5 > $U/spm_lvl echo ${RPM:-$rpm_before} > $U/rpm_lvl @@ -40,19 +77,19 @@ for c in $(seq 1 $N); do hy disable r0=$(cat /sys/class/rtc/rtc0/since_epoch) rtcwake -m no -s $S >/dev/null || { echo "rtcwake could not arm the alarm: stop" >> $L; hy enable; exit 1; } - # A keep-awake block inhibitor left by a push script (x-awake) would refuse every sleep. It is - # released here, by the series itself and only once the first alarm is armed: released by the - # starter before a start that then failed, it left the palmtop to the shell's auto-sleep with - # no alarm, until a key press (2026-11-10). - [ "$c" = 1 ] && systemctl stop x-awake 2>/dev/null ok0=$(cat /sys/power/suspend_stats/success) - systemctl suspend - sleep 20 + systemctl suspend -i + # logind holds a sleep back while a delay inhibitor is taken, up to 15 s here: NetworkManager did at the 90th + # sleep of the first run (2026-11-10), the sleep began 15 s after the request, and a fixed wait of 10 s (W=10) took that + # for "no suspend happened" while the palmtop slept and woke. So wait for the sleep itself, up to 45 s, and + # give S at least 30 s, so the alarm can't fire before a held-back sleep begins. + for i in $(seq 90); do [ "$(cat /sys/power/suspend_stats/success)" != "$ok0" ] && break; sleep 0.5; done + sleep $W hy enable # A sleep that never started (a block inhibitor held, a masked target) must not count as a cycle [ "$(cat /sys/power/suspend_stats/success)" != "$ok0" ] || { echo "cycle $c: no suspend happened (an inhibitor or a masked sleep target?): stop" >> $L; exit 1; } en=$(dmesg | grep -m1 "PM: suspend entry" | sed -E "s/^\[ *([0-9.]+)\].*/\1/"); ex=$(dmesg | grep -m1 "PM: suspend exit" | sed -E "s/^\[ *([0-9.]+)\].*/\1/") - echo "cycle $c $(date +%T): rtc $(( $(cat /sys/class/rtc/rtc0/since_epoch) - r0 - 20 )) s, dmesg $(awk -v a=$en -v b=$ex 'BEGIN{printf "%.1f", b-a}') s, wake irq $(cat /sys/power/pm_wakeup_irq 2>/dev/null), ufs $(dmesg | grep -cE "link startup failed|Host init failed|I/O error"), linkfail $(dmesg | grep -c "startup=1"), linkwait $(dmesg | grep -c "link-wait:"), acpm $(dmesg | grep -c "Timeout! ch"), oops $(dmesg | grep -c "Internal error"), spm $(cat $U/spm_lvl), c2-s2idle $(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/s2idle/usage 2>/dev/null) c2-run $(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/usage 2>/dev/null), ok $(cat /sys/power/suspend_stats/success) fail $(cat /sys/power/suspend_stats/fail)" >> $L + echo "cycle $c $(date +%T): rtc $(( $(cat /sys/class/rtc/rtc0/since_epoch) - r0 - W )) s, dmesg $(awk -v a=$en -v b=$ex 'BEGIN{printf "%.1f", b-a}') s, wake irq $(cat /sys/power/pm_wakeup_irq 2>/dev/null), ufs $(dmesg | grep -cE "link startup failed|Host init failed|I/O error"), linkfail $(dmesg | grep -c "startup=1"), linkwait $(dmesg | grep -c "link-wait:"), acpm $(dmesg | grep -c "Timeout! ch"), oops $(dmesg | grep -c "Internal error"), spm $(cat $U/spm_lvl), c2-s2idle $(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/s2idle/usage 2>/dev/null) c2-run $(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/usage 2>/dev/null), ok $(cat /sys/power/suspend_stats/success) fail $(cat /sys/power/suspend_stats/fail)" >> $L dmesg | grep -E "link-wait:|startup=1|link startup failed|Host init failed" | head -40 >> $L sync [ "$(dmesg | grep -cE "link startup failed|Host init failed|I/O error")" = 0 ] || { echo "UFS errors: stop" >> $L; break; } From 3e83a523c5ad97ee2234fa2fb716cc6c05394944 Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Wed, 11 Nov 2026 08:59:50 -0644 Subject: [PATCH 3/8] device/tegu bringup, SLEEP-PLAN: sleep-series.sh refuses first, puts everything back, looks before every sleep; its first run From the review of #659. The refusals (the modem up, an alarm under 30 s, kernel.panic 0 with the debug switches) and the trap now come before anything is set, and the trap puts back all the debug switches change, not only the watchdog. "systemctl suspend -i" passes every block inhibitor, so before each sleep the series checks for the modem and for any block-mode sleep inhibitor other than its own and x-awake's, and stops there. The runtime watchdog is raised to 60 s for a debug run rather than turned off: a fifth test crash kept its whole dump that way. The plan has that crash, the morning's 90 sleeps on the first form of the script, and the first run of this one with its lines, the inhibitor list and the values before and after. Nobody was at the palmtop for them, and the note says so. --- device/tegu/SLEEP-PLAN.md | 30 ++++++- device/tegu/bringup/sleep-series.sh | 118 +++++++++++++++++++--------- 2 files changed, 109 insertions(+), 39 deletions(-) diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index 3918d6f94..6ba1ac206 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -4848,10 +4848,36 @@ What it doesn't say: what CPU 7 was doing. The backtrace IPI is an ordinary inte **The tracing build's first evening: 94 alarm wakes, an unplanned sleep of nine and a half hours, and a false alarm (2026-11-10, 19:42, to 11-11, 06:19 MST palmtop clock).** `7.3.0-rc6-omamobile-push1-dbg-gff4fdf58035e`, build #5: the sources of the series above (the published tree with kernel #236 and #237, plus #224) with the function tracer added (`CONFIG_FUNCTION_TRACER`, `CONFIG_DYNAMIC_FTRACE`; pseudo-NMI is built in but not on the command line), its own modules installed, the published DTB with the `public-ramoops` and `modem-cpif` overlays, the modem boot unit masked for the boot, Wi-Fi kept, the cpuidle hook installed. The series is `sleep-series.sh` with what is now `PMDEBUG=1` and `TRACE=1`: `pm_print_times`, `pm_debug_messages`, `kernel.panic_on_rcu_stall=1`, and function and event tracing in 24 KB per CPU, dumped by a panic (`ftrace_dump_on_oops`). The owner was at the palmtop from 19:41 to about 20:45. -- **Four test crashes first, and what they showed about the dump** (`echo c > /proc/sysrq-trigger`, awake, each on a fresh boot, between 19:46 and 19:58). In the first three the saved console ended about 3,460 kernel lines after its start, in the middle of the trace dump, with no "Rebooting in 10 seconds.." line, and the bootloader's record of the reset was the watchdog's (`APC Watchdog Early`), not the panic's reboot. The console in pstore starts at 2.5 s of uptime, so with the lines before that each ended at about record 4096 since boot, which is the number of records the kernel's log ring holds on this build (`CONFIG_LOG_BUF_SHIFT=17`): a panic in which the ring wraps for the first time lost everything after the wrap. For the fourth the ring was filled beforehand (5,000 lines written to `/dev/kmsg`) and systemd's runtime watchdog (10 s on the palmtop, from a bring-up drop-in) was turned off: the console has the whole dump (about 950 events for each of the four cores that were traced), the dump's end marker and "Rebooting in 10 seconds..", and the palmtop rebooted by itself 32 s after the crash. Two things changed at once in the fourth, so the watchdog's part isn't separated from the ring's; the series does both. A dump of 24 KB per CPU fits the 2 MB console with room for the sleep's own lines (about 64 KB of them per cycle with `pm_print_times`). +- **Four test crashes first, and what they showed about the dump** (`echo c > /proc/sysrq-trigger`, awake, each on a fresh boot, between 19:46 and 19:58; a fifth the next morning). In the first three the saved console ended about 3,460 kernel lines after its start, in the middle of the trace dump, with no "Rebooting in 10 seconds.." line, and the bootloader's log after them has a watchdog reset (`RST_STAT: 0x1 - CLUSTER0_NONCPU_WDTRESET`): the panic never got to its own reboot. The console in pstore starts at 2.5 s of uptime, so with the lines before that each ended at about record 4096 since boot, which is the number of records the kernel's log ring holds on this build (`CONFIG_LOG_BUF_SHIFT=17`): a panic in which the ring wraps for the first time lost everything after the wrap. For the fourth the ring was filled beforehand (5,000 lines written to `/dev/kmsg`) and systemd's runtime watchdog (10 s on the palmtop, from a bring-up drop-in) was turned off: the console has the whole dump (about 950 events for each of the four cores that were traced), the dump's end marker and "Rebooting in 10 seconds..", and the palmtop rebooted by itself, SSH back 32 s after the crash, with `RST_STAT: 0x180000 - PIN_RESET | PO_RESET` in the bootloader's log. ("Reboot reason" reads `APC Watchdog Early` after all five, so that line doesn't tell a watchdog reset from the panic's own restart; `RST_STAT` does.) Two things changed at once in the fourth. The fifth (11-11, 08:36, on the boot of that morning's 90 sleeps, so with the ring long wrapped, all eight cores traced) had the watchdog raised to 60 s instead of off, the maintainer's suggestion: the whole dump again (883 to 958 events a core, 7,420 lines), the end marker, "Rebooting in 10 seconds..", `PIN_RESET | PO_RESET`, SSH back 31 s after the crash. So the series fills the ring and raises the watchdog to 60 s for the run, which keeps a net under an awake hang. Not tried: a wrapped ring with the watchdog left at 10 s, which by these timings would cut the panic's ten-second countdown short rather than the dump. A dump of 24 KB per CPU fits the 2 MB console with room for the sleep's own lines (about 64 KB of them per cycle with `pm_print_times`). - **A test sleep, then the first run** (boot at 19:58:23). One `systemctl suspend` with a 25 s alarm at 19:59 for omarchy-mobile#619's `SuspendState=freeze` (MODEM-PLAN has it), before the switches were on. Then a series with a 10 s alarm from 20:00: 90 sleeps, 89 of them 19.1 to 19.8 s by the kernel log (why a 10 s alarm gives a sleep that long wasn't looked at), each woken by the alarm (`pm_wakeup_irq` 53), no UFS line, no ACPM timeout, no oops. The run stopped itself at the 90th with "no suspend happened": logind had held that sleep back for NetworkManager's delay inhibitor (up to 15 s here), the sleep began after the script's fixed 10 s wait had ended, and it lasted 6 s and woke on the alarm too. The script now waits for `suspend_stats/success` to move, up to 45 s. - **The second run and the unplanned sleep.** A series with a 35 s alarm from 20:43, the owner leaving as it began. Two sleeps of 34.5 s woke on the alarm. The third did too, at 20:45:24, and seven seconds later the shell's auto-sleep, which is on on this palmtop, put the palmtop back to sleep, as it does 5 s after any wake that leaves the screen off behind a secure lock (`omamobile-sleepd: sleeping (dark resume, 5 s grace)`). No alarm was armed for that sleep. It lasted from 20:45:33 to 06:18:54, 9 hours 33 minutes, and ended at the owner's Power key (`PM: Triggering wakeup from IRQ 57`, `Power`), with no UFS line. In the first run the daemon logged nothing, most likely because the lock wasn't secure with the owner at the palmtop, and after the second run's first two wakes an SSH poll from the laptop happened to be open ("not sleeping: an SSH session is open"), so the daemon held off until the third. - **The false alarm.** From the laptop a sleep with no alarm and a hang look the same: the USB device gone and no reconnect. It was taken for the no-wake, reported as that on kernel #224 the same night, and the owner was asked to reset the palmtop in the morning. His first press of the Power key woke it; the series went on, logged its third cycle and entered a fourth sleep at 06:19:02, and the Power + Volume Up reset landed in that one. The journal of the boot, read after it, has 96 `PM: suspend entry` and 95 `PM: suspend exit` lines, no RCU stall, no panic and no UFS error line. The report on #224 was corrected the same morning. -- **Why it happened, and the fix.** The series released the push script's keep-awake inhibitor (`x-awake`) at its first cycle, since a block inhibitor refuses its own sleeps too, and from then on nothing stood between the auto-sleep and the palmtop in the seconds after each wake. This is the trap of the unplanned first sleep in the note above, a second time. `sleep-series.sh` now holds a block inhibitor of its own from start to end and sleeps past it with `systemctl suspend -i`, so the only sleeps during a run are its own, each with an alarm; it refuses to start with the modem up (the unit active or `cpif` loaded), so the modem's inhibitor is never one of those it passes. Not yet run in that form. +- **Why it happened, and the fix.** The series released the push script's keep-awake inhibitor (`x-awake`) at its first cycle, since a block inhibitor refuses its own sleeps too, and from then on nothing stood between the auto-sleep and the palmtop in the seconds after each wake. This is the trap of the unplanned first sleep in the note above, a second time. `sleep-series.sh` now holds a block inhibitor of its own from start to end and sleeps past it with `systemctl suspend -i`, so the only sleeps during a run are its own, each with an alarm; `-i` passes every block inhibitor, so the series refuses to start with the modem up (a modem unit active or `cpif` loaded) and looks again before every sleep: the modem up, or any block-mode sleep inhibitor other than its own and `x-awake`'s, ends the run. An alarm under 30 s is refused too (logind can hold a sleep back 15 s for a delay inhibitor, and an alarm that fires before the sleep begins leaves it with none). +- **The morning after: the series with its own inhibitor (2026-11-11, 07:00 to 08:42 MST), nobody at the palmtop.** The owner had handed the palmtop over and gone out; he was asked through his phone before the reviewed script's first run and didn't answer, so these runs were followed from the laptop only, which is not what the attended rule asks. First the script with the inhibitor and `-i` but before the review's changes: 90 sleeps with a 35 s alarm from 07:00 to 08:34, each 33.8 to 34.9 s by the kernel log and woken by the alarm (`pm_wakeup_irq` 53), no UFS line, no ACPM timeout, no oops, `suspend_stats` 90 and 0; after the wakes the auto-sleep logged "not sleeping: a sleep inhibitor is held", or "an SSH session is open" when the laptop's poll was in. It was stopped by hand after the 90th for the review's changes and the 60 s alarm. Then the fifth test crash (above), and on the fresh boot the script as it is here, `N=3 S=60 PMDEBUG=1 TRACE=1`, its first run: + + ``` + before: runtime watchdog 10000000, panic_on_rcu_stall 0, pm_print_times 0, pm_debug_messages 0, tracer nop, ftrace_dump_on_oops 0, console_suspend Y + debug: pm_print_times 1, pm_debug_messages 1, panic_on_rcu_stall 1, panic timeout 10 s, tracer function 27 KB per CPU on 1, ftrace_dump_on_oops 1 + runtime watchdog 60000000 (was 10000000) + cycle 1 08:38:56: rtc 63 s, dmesg 59.8 s, wake irq 53, ufs 0, linkfail 0, linkwait 0, acpm 0, oops 0, spm 5, c2-s2idle c2-run , ok 1 fail 0 + cycle 2 08:40:24: rtc 62 s, dmesg 58.8 s, wake irq 53, ufs 0, linkfail 0, linkwait 0, acpm 0, oops 0, spm 5, c2-s2idle c2-run , ok 2 fail 0 + cycle 3 08:41:53: rtc 62 s, dmesg 59.8 s, wake irq 53, ufs 0, linkfail 0, linkwait 0, acpm 0, oops 0, spm 5, c2-s2idle c2-run , ok 3 fail 0 + done + restored: pm_print_times 0, pm_debug_messages 0, panic_on_rcu_stall 0, ftrace_dump_on_oops 0, tracer nop, runtime watchdog 10000000 (was 10000000) + restored: spm_lvl 5, state1 disable , modem boot unit masked (was masked) + after: runtime watchdog 10000000, panic_on_rcu_stall 0, pm_print_times 0, pm_debug_messages 0, tracer nop, ftrace_dump_on_oops 0, console_suspend Y + ``` + + `systemd-inhibit --list` during the run (the block-mode ones; `linkwait` is the push script's `x-awake`), and what the auto-sleep logged in that boot: + + ``` + linkwait 0 root 969 systemd-inhibit sleep after test crash 5 block + sleep-series 0 root 3100 systemd-inhibit sleep only the series' own sleeps, each with an alarm block + omamobile-sleepd: not sleeping: a sleep inhibitor is held (sleep-series) 3 times + omamobile-sleepd: not sleeping: a sleep inhibitor is held (linkwait) once + omamobile-sleepd: not sleeping: an SSH session is open 3 times + ``` + + After it only `linkwait` was left. systemd's journal has "Watchdog running with a hardware timeout of 1min." at the start and "of 10s." at the end, so the driver takes 60 s. The "before" and "after" lines were read by hand around the run; the rest is the series' log. What it adds to the count: 93 sleeps woken by the alarm with the PM switches on, and one of nine and a half hours on this tree ended by the Power key, which the two Power-key presses of the morning of 11-10 could not do for the sleep that hung. diff --git a/device/tegu/bringup/sleep-series.sh b/device/tegu/bringup/sleep-series.sh index 77029d0e1..a92de7352 100644 --- a/device/tegu/bringup/sleep-series.sh +++ b/device/tegu/bringup/sleep-series.sh @@ -8,71 +8,116 @@ # errors, and the link-wait lines (a slow wake rather than a dead one). N=${N:-300}; S=${S:-60}; W=${W:-20}; L=${L:-/var/log/omamobile/sleep-linkwait-20261107.log}; U=/sys/devices/platform/soc/13200000.ufs mkdir -p /var/log/omamobile; : > $L -# Debugging a no-wake (2026-11-10; off unless asked for). PMDEBUG=1 turns pm_print_times and pm_debug_messages -# on, so the console names each device's suspend callback as it starts and returns (the maintainer's pointer, -# kernel #224), and makes an RCU stall a panic (kernel.panic_on_rcu_stall), which panic=10 turns into a reboot -# with the console in pstore. TRACE=1, on a kernel with the function tracer, also runs function and event -# tracing in small per-CPU buffers that the panic dumps to the console. The dump has to fit the 2 MB ramoops -# console together with the sleep's own callback lines: it prints oldest first, and 24 KB per CPU is about 900 -# calls a core and under 1 MB of text for eight (measure it with an awake sysrq-c before a series). -# Either one first fills the kernel's log ring: three test crashes on the tracing build (sysrq-c, awake) lost -# everything after record 4096 since boot, the trace dump's tail and the "Rebooting" line with it, when the -# ring's first wrap fell inside the panic; with the ring filled beforehand the fourth kept the whole dump. -if [ "${PMDEBUG:-0}" = 1 ] || [ "${TRACE:-0}" = 1 ]; then - echo on > /proc/sys/kernel/printk_devkmsg 2>/dev/null; for i in $(seq 1 5000); do echo "ring fill $i" > /dev/kmsg; done - echo 1 > /sys/power/pm_print_times; echo 1 > /sys/power/pm_debug_messages - echo 1 > /proc/sys/kernel/panic_on_rcu_stall - # systemd's runtime watchdog (10 s on the palmtop) would reset a panicking kernel before the dump and the - # panic's own reboot are through (32 s in the test that kept its dump): off for the run, put back at the end. - WD=org.freedesktop.systemd1; wd_before=$(busctl get-property $WD /org/freedesktop/systemd1 $WD.Manager RuntimeWatchdogUSec | awk '{print $2}') - busctl set-property $WD /org/freedesktop/systemd1 $WD.Manager RuntimeWatchdogUSec t 0 -fi -if [ "${TRACE:-0}" = 1 ] && [ -d /sys/kernel/tracing ]; then - T=/sys/kernel/tracing; mountpoint -q $T || mount -t tracefs nodev $T - echo 0 > $T/tracing_on; echo ${TRACE_KB:-24} > $T/buffer_size_kb; echo function > $T/current_tracer - for e in sched/sched_switch irq/irq_handler_entry irq/irq_handler_exit ipi power/cpu_idle power/suspend_resume timer/hrtimer_expire_entry; do echo 1 > $T/events/$e/enable; done - echo 1 > $T/tracing_on; echo 1 > /proc/sys/kernel/ftrace_dump_on_oops -fi -echo "debug: pm_print_times $(cat /sys/power/pm_print_times), pm_debug_messages $(cat /sys/power/pm_debug_messages), panic_on_rcu_stall $(cat /proc/sys/kernel/panic_on_rcu_stall), panic timeout $(cat /proc/sys/kernel/panic) s, tracer $(cat /sys/kernel/tracing/current_tracer 2>/dev/null || echo none) $(cat /sys/kernel/tracing/buffer_size_kb 2>/dev/null) KB per CPU on $(cat /sys/kernel/tracing/tracing_on 2>/dev/null), ftrace_dump_on_oops $(cat /proc/sys/kernel/ftrace_dump_on_oops 2>/dev/null)" >> $L +DBG=0; { [ "${PMDEBUG:-0}" = 1 ] || [ "${TRACE:-0}" = 1 ]; } && DBG=1 +# Refusals come first, before anything is changed. +modem_up() { systemctl is-active -q omamobile-modem omamobile-tegu-modem-boot 2>/dev/null || [ -d /sys/module/cpif ]; } +modem_up && { echo "the modem is up: stop" >> $L; exit 1; } +# An alarm that fires before a held-back sleep begins (logind waits up to 15 s for a delay inhibitor) leaves +# that sleep with no alarm. +[ "$S" -ge 30 ] || { echo "S=$S: an alarm under 30 s can fire before the sleep begins: stop" >> $L; exit 1; } +# With the debug switches a stall is a panic, and a panic has to end in a reboot. +[ $DBG = 1 ] && [ "$(cat /proc/sys/kernel/panic)" = 0 ] && { echo "kernel.panic is 0, a panic would wait for the buttons: stop" >> $L; exit 1; } uid=$(id -u mobile) hy() { runuser -u mobile -- env XDG_RUNTIME_DIR=/run/user/$uid HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$uid/hypr | head -1) hyprctl dispatch "hl.dsp.dpms({ action = \"$1\" })" >/dev/null 2>&1; } -echo N > /sys/module/printk/parameters/console_suspend -{ systemctl is-active -q omamobile-modem || [ -d /sys/module/cpif ]; } && { echo "the modem is up: stop" >> $L; exit 1; } +cs_before=$(cat /sys/module/printk/parameters/console_suspend) # The cpuidle sleep hook (omarchy-mobile#591; an earlier hook of this name was withdrawn in #255): when it is # installed and its keep flag is absent, every sleep of the series runs in WFI whatever WFI= says. Noted, not refused. if [ -e /usr/lib/systemd/system-sleep/omamobile-tegu-cpuidle ]; then echo "cpuidle hook installed, keep flag $([ -e /etc/omamobile/cpuidle-keep-across-sleep ] && echo present || echo absent)" >> $L; fi # The modem boot unit's state is put back as it was found, not forced on. modem_unit_before=$(systemctl is-enabled omamobile-tegu-modem-boot.service 2>/dev/null) -state1_before=$(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/disable) +# The published device tree has no idle states yet: then there is no state1 to set. +state1_before=$(cat /sys/devices/system/cpu/cpu0/cpuidle/state1/disable 2>/dev/null) +st1() { for f in /sys/devices/system/cpu/cpu*/cpuidle/state1/disable; do [ -e $f ] && echo $1 > $f; done; } # RPM (default: as found): UFS rpm_lvl for the run. Experiment #92 runs with 1 # (device active, link hibern8 in runtime suspend) so a VCC that is no longer # always-on is cut only by the system suspend, not by every idle period. rpm_before=$(cat $U/rpm_lvl) +# What the debug switches change is read here, before anything is set, and restore() puts all of it back. +T=/sys/kernel/tracing; WD=org.freedesktop.systemd1; EV="sched/sched_switch irq/irq_handler_entry irq/irq_handler_exit ipi power/cpu_idle power/suspend_resume timer/hrtimer_expire_entry" +wd() { busctl "$1"-property $WD /org/freedesktop/systemd1 $WD.Manager RuntimeWatchdogUSec ${2:+t $2}; } +if [ $DBG = 1 ]; then + ppt_before=$(cat /sys/power/pm_print_times); pdm_before=$(cat /sys/power/pm_debug_messages) + stall_before=$(cat /proc/sys/kernel/panic_on_rcu_stall); wd_before=$(wd get | awk '{print $2}') +fi +if [ "${TRACE:-0}" = 1 ] && [ -d $T ]; then + mountpoint -q $T || mount -t tracefs nodev $T + dump_before=$(cat /proc/sys/kernel/ftrace_dump_on_oops); tr_before=$(cat $T/current_tracer); on_before=$(cat $T/tracing_on) + kb_before=$(awk '{print $1}' $T/buffer_size_kb); ev_before=$(for e in $EV; do echo "$e=$(cat $T/events/$e/enable)"; done) +fi restore() { - [ -n "$wd_before" ] && busctl set-property org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager RuntimeWatchdogUSec t $wd_before [ -n "$INH" ] && kill $INH 2>/dev/null - for f in /sys/devices/system/cpu/cpu*/cpuidle/state1/disable; do echo $state1_before > $f; done + if [ -n "$tr_before" ]; then + echo 0 > $T/tracing_on; echo "$dump_before" > /proc/sys/kernel/ftrace_dump_on_oops + for x in $ev_before; do echo "${x##*=}" > $T/events/${x%%=*}/enable; done + echo "$tr_before" > $T/current_tracer; echo "$kb_before" > $T/buffer_size_kb; echo "$on_before" > $T/tracing_on + fi + if [ $DBG = 1 ]; then + echo "$stall_before" > /proc/sys/kernel/panic_on_rcu_stall + echo "$ppt_before" > /sys/power/pm_print_times; echo "$pdm_before" > /sys/power/pm_debug_messages + [ -n "$wd_before" ] && wd set "$wd_before" + echo "restored: pm_print_times $(cat /sys/power/pm_print_times), pm_debug_messages $(cat /sys/power/pm_debug_messages), panic_on_rcu_stall $(cat /proc/sys/kernel/panic_on_rcu_stall), ftrace_dump_on_oops $(cat /proc/sys/kernel/ftrace_dump_on_oops), tracer $(cat $T/current_tracer 2>/dev/null || echo none), runtime watchdog $(wd get | awk '{print $2}') (was $wd_before)" >> $L + fi + rtcwake -m disable >/dev/null 2>&1 + echo $cs_before > /sys/module/printk/parameters/console_suspend + st1 $state1_before echo 5 > $U/spm_lvl echo $rpm_before > $U/rpm_lvl [ "$modem_unit_before" = enabled ] && systemctl enable omamobile-tegu-modem-boot.service >/dev/null 2>&1 echo "restored: spm_lvl $(cat $U/spm_lvl), state1 disable $state1_before, modem boot unit $(systemctl is-enabled omamobile-tegu-modem-boot.service 2>/dev/null) (was $modem_unit_before)" >> $L } trap restore EXIT +echo N > /sys/module/printk/parameters/console_suspend +# Debugging a no-wake (2026-11-10; off unless asked for). PMDEBUG=1 turns pm_print_times and pm_debug_messages +# on, so the console names each device's suspend callback as it starts and returns (the maintainer's pointer, +# kernel #224), and makes an RCU stall a panic (kernel.panic_on_rcu_stall), which panic=10 turns into a reboot +# with the console in pstore. TRACE=1, on a kernel with the function tracer, also runs function and event +# tracing in small per-CPU buffers that the panic dumps to the console. The dump has to fit the 2 MB ramoops +# console together with the sleep's own callback lines: it prints oldest first, and 24 KB per CPU is about 900 +# calls a core and under 1 MB of text for eight (measure it with an awake sysrq-c before a series). +# Either one first fills the kernel's log ring: three test crashes on the tracing build (sysrq-c, awake) lost +# everything after record 4096 since boot, the trace dump's tail and the "Rebooting" line with it, when the +# ring's first wrap fell inside the panic; with the ring filled beforehand the dump was whole. +# systemd's runtime watchdog (10 s on the palmtop) would reset a panicking kernel before the dump and the +# panic's own reboot are through (32 s in the tests that kept their dump), so it is raised to WDSEC (60 s) for +# the run: still a net under an awake hang. WDSEC=0 turns it off. +if [ $DBG = 1 ]; then + echo on > /proc/sys/kernel/printk_devkmsg 2>/dev/null; for i in $(seq 1 5000); do echo "ring fill $i" > /dev/kmsg; done + echo 1 > /sys/power/pm_print_times; echo 1 > /sys/power/pm_debug_messages + echo 1 > /proc/sys/kernel/panic_on_rcu_stall + wd set $(( ${WDSEC:-60} * 1000000 )) +fi +if [ -n "$tr_before" ]; then + echo 0 > $T/tracing_on; echo ${TRACE_KB:-24} > $T/buffer_size_kb; echo function > $T/current_tracer + for e in $EV; do echo 1 > $T/events/$e/enable; done + echo 1 > $T/tracing_on; echo 1 > /proc/sys/kernel/ftrace_dump_on_oops +fi +echo "debug: pm_print_times $(cat /sys/power/pm_print_times), pm_debug_messages $(cat /sys/power/pm_debug_messages), panic_on_rcu_stall $(cat /proc/sys/kernel/panic_on_rcu_stall), panic timeout $(cat /proc/sys/kernel/panic) s, tracer $(cat /sys/kernel/tracing/current_tracer 2>/dev/null || echo none) $(cat /sys/kernel/tracing/buffer_size_kb 2>/dev/null) KB per CPU on $(cat /sys/kernel/tracing/tracing_on 2>/dev/null), ftrace_dump_on_oops $(cat /proc/sys/kernel/ftrace_dump_on_oops 2>/dev/null)" >> $L +[ $DBG = 1 ] && echo "runtime watchdog $(wd get | awk '{print $2}') (was $wd_before)" >> $L # Only the series' own sleeps, each with its alarm, may happen during a run. The shell's auto-sleep # (omamobile-sleepd) puts a dark resume back to sleep 5 s after it once the lock is secure and no SSH session # is open, with no alarm: it did between two cycles on 2026-11-10, and the palmtop slept nine and a half # hours, until the Power key, which from the laptop looked like a no-wake. So the series holds a block # inhibitor of its own for the whole run (a push script's x-awake has a time limit) and sleeps past it -# with "systemctl suspend -i"; the modem's inhibitor is not passed that way, since the series refuses -# to start with the modem up. +# with "systemctl suspend -i". That passes every block inhibitor, so before each sleep the series looks +# again: the modem up, or any block-mode sleep inhibitor other than its own and x-awake's, ends the run. systemd-inhibit --what=sleep --who=sleep-series --why="only the series' own sleeps, each with an alarm" --mode=block sleep infinity & INH=$! -for f in /sys/devices/system/cpu/cpu*/cpuidle/state1/disable; do echo ${WFI:-1} > $f; done +others() { + busctl --json=short call org.freedesktop.login1 /org/freedesktop/login1 org.freedesktop.login1.Manager ListInhibitors | python3 -c ' +import json, sys +mine = set(sys.argv[1:]) +print(" ".join("%s(pid %s)" % (i[1], i[5]) for i in json.load(sys.stdin)["data"][0] + if "sleep" in i[0].split(":") and i[3] == "block" and str(i[5]) not in mine))' $INH $(systemctl show -p MainPID --value x-awake 2>/dev/null) +} +st1 ${WFI:-1} echo 5 > $U/spm_lvl echo ${RPM:-$rpm_before} > $U/rpm_lvl -echo "$(uname -v): rpm_lvl $(cat $U/rpm_lvl) (was $rpm_before), ufs-vcc always-on: $([ -e /proc/device-tree/regulator-ufs-vcc/regulator-always-on ] && echo yes || echo no), spm_lvl $(cat $U/spm_lvl) ($(cat $U/spm_target_dev_state)/$(cat $U/spm_target_link_state)), state1 disable $(cat /sys/devices/system/cpu/cpu*/cpuidle/state1/disable | tr -d '\n'), modem boot unit $modem_unit_before, $(uname -r)" >> $L +echo "$(uname -v): rpm_lvl $(cat $U/rpm_lvl) (was $rpm_before), ufs-vcc always-on: $([ -e /proc/device-tree/regulator-ufs-vcc/regulator-always-on ] && echo yes || echo no), spm_lvl $(cat $U/spm_lvl) ($(cat $U/spm_target_dev_state)/$(cat $U/spm_target_link_state)), state1 disable $(cat /sys/devices/system/cpu/cpu*/cpuidle/state1/disable 2>/dev/null | tr -d '\n'), modem boot unit $modem_unit_before, $(uname -r)" >> $L [ "$(cat $U/spm_lvl)" = 5 ] || { echo "spm_lvl didn't take: stop" >> $L; exit 1; } for c in $(seq 1 $N); do + [ -e /run/sleep-series.stop ] && { rm -f /run/sleep-series.stop; echo "stop file: stop after $((c - 1)) cycles" >> $L; break; } + modem_up && { echo "cycle $c: the modem is up: stop" >> $L; exit 1; } + o=$(others); [ -z "$o" ] || { echo "cycle $c: another block-mode sleep inhibitor is held ($o): stop" >> $L; exit 1; } dmesg -C hy disable r0=$(cat /sys/class/rtc/rtc0/since_epoch) @@ -81,8 +126,7 @@ for c in $(seq 1 $N); do systemctl suspend -i # logind holds a sleep back while a delay inhibitor is taken, up to 15 s here: NetworkManager did at the 90th # sleep of the first run (2026-11-10), the sleep began 15 s after the request, and a fixed wait of 10 s (W=10) took that - # for "no suspend happened" while the palmtop slept and woke. So wait for the sleep itself, up to 45 s, and - # give S at least 30 s, so the alarm can't fire before a held-back sleep begins. + # for "no suspend happened" while the palmtop slept and woke. So wait for the sleep itself, up to 45 s. for i in $(seq 90); do [ "$(cat /sys/power/suspend_stats/success)" != "$ok0" ] && break; sleep 0.5; done sleep $W hy enable From c154389fee2b5ed65328b1b50dd47b410e1de55a Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Wed, 11 Nov 2026 16:38:28 -0644 Subject: [PATCH 4/8] device/tegu SLEEP-PLAN: 313 of 313 with the PM switches on and a 60 s alarm The reviewed sleep-series.sh ran its count on 2026-11-11: three sleeps in its first run and 310 from 08:43 to 16:20, each woken by the alarm, with no UFS line, stall or panic in the boot's journal. With the 93 of the evening before that is 406 alarm wakes with the switches on, the count asked for on kernel #224. The runs table, "What is left" item 1 and the note have it, with who was at the palmtop and when. --- device/tegu/SLEEP-PLAN.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index 6ba1ac206..38b5e21ca 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -97,6 +97,8 @@ by a poll that could report, which is not what the rule asks): | the same sources, build #2 with `CONFIG_PHY_ZUMA_PCIE=m` (#274) | the same with Wi-Fi up and kept across the sleeps | WFI | **36 of 36**, wlan0 passing traffic after | the start yes, then the loop | | published `2f890a533` (11-10) + #224 + #236 + #237 (`push1-gff4fdf58035e`, build #4), modules installed, Wi-Fi kept, the hook of #591 installed | the newest tip, overnight | WFI | a 27-minute sleep ended by the Power key, then 212 woke and the 213th never did: CPU 7 stopped answering, the console kept | no (the owner asleep; his reset in the morning) | | the same sources with the function tracer built in (`push1-dbg-gff4fdf58035e`, build #5), `pm_print_times`, `pm_debug_messages` and `kernel.panic_on_rcu_stall=1` on, modules installed, Wi-Fi kept, the hook of #591 installed | whether the no-wake comes back, with a console that would place it | WFI | 96 sleeps entered, no hang: 94 woke on the alarm (a test sleep, 90 with a 10 s alarm, three with a 35 s one); one, the shell's auto-sleep between two cycles with no alarm, lasted 9 h 33 min until the Power key; the 96th was cut by a button reset asked for in error | the first run yes, the owner at the palmtop; he left as the second began, and the long sleep was overnight | +| the same build, 11-11, `sleep-series.sh` holding its own sleep inhibitor (first form, before its review) | the inhibitor, and whether the auto-sleep stays out | WFI | **90 of 90** with a 35 s alarm (33.8 to 34.9 s, `pm_wakeup_irq` 53, no UFS line), stopped by hand for the review's changes | no (the owner out) | +| the same build, 11-11, the script as reviewed (`PMDEBUG=1 TRACE=1`, 60 s alarm) | the count with the PM switches on | WFI | **313 of 313** (3 in the script's first run, then 310 from 08:43 to 16:20; 58.8 to 60.0 s, `pm_wakeup_irq` 53 each, no UFS line, no stall, `suspend_stats` 313 and 0) | no: nobody at the palmtop for the first cycles; the owner saw it cycling through the day and watched a few cycles at about 11:25 | Before #96, one wake in 15 to 215 lost the storage; before #98, C2 in the sleep lost the palmtop after 6 to 135 cycles. @@ -113,9 +115,9 @@ missing pull, NFC polling (stopped by the hook throughout), the modem **What is left.** -1. The no-wake on the published kernel. On that tree 402 sleeps have woken - (21, 36, 36, two ended by the Power key, 212, and the 95 of the evening - of 11-10) and two have not: the 22nd of 11-09's first run, with no +1. The no-wake on the published kernel. On that tree 805 sleeps have woken + (21, 36, 36, two ended by the Power key, 212, the 95 of the evening of + 11-10, and 90 and 313 on 11-11) and two have not: the 22nd of 11-09's first run, with no console, and the 213th of 11-10's, whose console shows CPU 7 not answering while the rest of the kernel runs (failure 3 above). It hasn't come back since. What is set up for it, in place of the pseudo-NMI build @@ -125,12 +127,11 @@ missing pull, NFC polling (stopped by the hook throughout), the modem `kernel.panic_on_rcu_stall=1`, so the stall becomes a panic, a reboot through `panic=10` and a console in pstore with no buttons; and, on a build with the function tracer, each core's last calls in the panic's - dump (`PMDEBUG=1` and `TRACE=1` in `sleep-series.sh`). The first evening - that way, 11-10, was 93 alarm wakes with the switches on and no hang - (the last note below). Next: more series the same way, to about 400 - wakes with the switches on, which is what the maintainer asks for before - #224 merges if the hang stays away; if it comes back, the console says - where it stopped. + dump (`PMDEBUG=1` and `TRACE=1` in `sleep-series.sh`). With them on, + 406 sleeps have woken on the alarm and none has hung: 93 on the evening + of 11-10 and 313 of 60 s on 11-11 (the last two notes below), which is + the count the maintainer asked for before #224 merges if the hang + stayed away. If it comes back, the same switches say where it stopped. 2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): merged on 2026-11-10 as `omamobile-device-tegu` 0.2.11; its series is the 40 of 40 in the notes below. @@ -4879,5 +4880,8 @@ What it doesn't say: what CPU 7 was doing. The backtrace IPI is an ordinary inte ``` After it only `linkwait` was left. systemd's journal has "Watchdog running with a hardware timeout of 1min." at the start and "of 10s." at the end, so the driver takes 60 s. The "before" and "after" lines were read by hand around the run; the rest is the series' log. +- **The series for the count: 310 of 310 (2026-11-11, 08:43 to 16:20 MST).** The same boot and script, `N=310 S=60 PMDEBUG=1 TRACE=1`. Every sleep lasted 58.8 to 60.0 s by the kernel log and was woken by the alarm (`pm_wakeup_irq` 53), with no UFS line, no ACPM timeout and no oops; five cycles logged an RTC difference that doesn't fit (three about 20 s short, one 7 s and one about 40 s long; not looked into), while the kernel's clock has those sleeps at 60 s like the rest. The boot's kernel journal agrees with the series' log: 313 `PM: suspend entry` and 313 `PM: suspend exit` (the first run's three included), 313 `PM: Triggering wakeup from IRQ 53` and none from another interrupt, no `link startup failed`, `Host init failed`, `I/O error` or `ufshcd_err_handler` line, no RCU stall, no panic; `suspend_stats` 313 and 0. The auto-sleep asked for no sleep in that boot. At the end the "restored:" lines read as after the first run (the switches off, the tracer `nop`, the watchdog back at 10 s), and only `x-awake` held a block inhibitor. The owner wasn't at the palmtop when it began; he saw the screen cycling at several points through the day and watched a few cycles at about 11:25. -What it adds to the count: 93 sleeps woken by the alarm with the PM switches on, and one of nine and a half hours on this tree ended by the Power key, which the two Power-key presses of the morning of 11-10 could not do for the sleep that hung. +What the two days add to the count: with the PM switches on, 406 sleeps woken by the alarm (93 on 11-10, 313 on 11-11) and none hung; without counting toward that, 90 more with a 35 s alarm on 11-11. + +The long sleep of 11-10, nine and a half hours on this tree, ended at the first press of the Power key, which the two presses of the morning of 11-10 could not do for the sleep that hung. From 8a36a07b3412f8797195f9f020a41320000f0a7b Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Wed, 11 Nov 2026 18:00:15 -0644 Subject: [PATCH 5/8] device/tegu SLEEP-PLAN, DEVICES, bringup: the tally in the status line, 11-11's runs named as unattended, and sleep-series.sh keeps the log before it From the second review of #659. The plan's status line carries the count (805 woken and two not on that tree, 406 with the debug switches on) and says why 406 without a hang isn't a fix; the runs header names all of 11-11 as run without the owner's yes; "What is left" has the RTC's odd readings as an item. Kernel #224 merged on the count, and the 9a's Suspend cell says so. sleep-series.sh no longer empties its log before it can refuse and keeps the previous run's beside the new one, puts printk_devkmsg and the trace buffer's size back with the rest, and stops when the inhibitor list can't be read. Not run in this form yet. --- device/tegu/SLEEP-PLAN.md | 40 +++++++++++++++++++++-------- device/tegu/bringup/sleep-series.sh | 26 ++++++++++++++----- docs/DEVICES.md | 2 +- 3 files changed, 50 insertions(+), 18 deletions(-) diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index 38b5e21ca..efeeaaa4d 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -1,7 +1,7 @@ # Pixel 9a sleep: why s2idle drains so much -Status (2026-11-10): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), in review, re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules, no console), none in two re-runs of 36 with modules loaded, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 307 wakes and two no-wakes (11-09 and 11-10). The attended-first rule (S1, AGENTS.md) stands; the 11-10 run did not keep it (the run table says how it ran). +Status (2026-11-12): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The tally on that tree, with #224's two commits: 805 sleeps woken and those two not. It hasn't come back since: 406 sleeps on 11-10 and 11-11 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 of them 60 s sleeps on one boot, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate two or three no-wakes were to be expected in 406, none happens by chance about one time in thirteen, and the switches may change the timing. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 307 wakes and two no-wakes (11-09 and 11-10). The attended-first rule (S1, AGENTS.md) stands; the 11-10 run did not keep it (the run table says how it ran). S1 is the condensed account: the two causes, the fix, a run table, what is left and what was ruled out. The dated working notes that led there follow it, oldest first. @@ -73,9 +73,13 @@ boot unless said; `systemctl suspend` with an RTC alarm, 60 s unless said; "attended" means a person at the laptop watching the loop with the owner at home for a button press; the first run of every new kernel was watched that way, and the long unattended stretches ran under the loop's recovery budget, -with one exception, the overnight run of 11-10: a new build's first run went -overnight with the owner asleep, started by hand over SSH and followed only -by a poll that could report, which is not what the rule asks): +with two exceptions, neither of them what the rule asks. The overnight run +of 11-10: a new build's first run went overnight with the owner asleep, +started by hand over SSH and followed only by a poll that could report. And +all of 11-11, with the owner out and no yes from him for any of it: the +first form of the series script with its own inhibitor from 07:00, a +deliberate test crash at 08:36, the reviewed form's first run at 08:38 and +the 310 sleeps that followed): | Kernel (`uname -v`) | Change under test | Cores in sleep | Result | Attended | | --- | --- | --- | --- | --- | @@ -130,8 +134,12 @@ missing pull, NFC polling (stopped by the hook throughout), the modem dump (`PMDEBUG=1` and `TRACE=1` in `sleep-series.sh`). With them on, 406 sleeps have woken on the alarm and none has hung: 93 on the evening of 11-10 and 313 of 60 s on 11-11 (the last two notes below), which is - the count the maintainer asked for before #224 merges if the hang - stayed away. If it comes back, the same switches say where it stopped. + the count the maintainer asked for, and #224 merged on it. It doesn't + show the no-wake gone: without the switches that tree had two in about + 310 sleeps, so two or three were to be expected in 406, none happens + by chance about one time in thirteen, and the switches may change the + timing. If it comes back, the same switches say where it stopped; the + console of 11-10 and the tracing build are the tools for it. 2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): merged on 2026-11-10 as `omamobile-device-tegu` 0.2.11; its series is the 40 of 40 in the notes below. @@ -144,9 +152,19 @@ missing pull, NFC polling (stopped by the hook throughout), the modem [#620](https://github.com/derekross/omarchy-mobile/issues/620). 4. The root cause under the masks: zuma's PMU data without `pmu_cpuhp`, compared with gs101/gs201 before idle states come to a PR. -5. The DEVICES cell to "works" once item 1's no-wake is found and fixed, - and #224, the PMIC interrupt (kernel #236) and the hook are merged and - have run from the published kernel. +5. The DEVICES cell to "works" once item 1's no-wake is found and fixed. + #224 (merged 2026-11-12, 00:41Z), the PMIC interrupt (kernel #236) and the hook + are all merged; the 805 wakes above ran on the published tree with + #224's commits on top, and a run on the tree as published, with them + in it, is still to do. +6. The RTC around a sleep. Three things not looked into: in the 310 + sleeps of 11-11, five cycles read an RTC difference that doesn't fit + the 60 s the kernel's clock has for them (three about 20 s short, one + 7 s and one about 40 s long); on 11-10 a 10 s alarm gave sleeps of 19 + to 20 s, 89 times; and the read tear of the 11-04 note, a difference a + minute off when the read falls just after a minute rolls over. None + cost a wake, but the series' "rtc" column can't be relied on until + they are explained. **Scripts** (`device/tegu/bringup/`): `sleep-series.sh` (on the palmtop: N RTC-woken `systemctl suspend` cycles of S s, records each wake; it holds a @@ -4879,9 +4897,9 @@ What it doesn't say: what CPU 7 was doing. The backtrace IPI is an ordinary inte omamobile-sleepd: not sleeping: an SSH session is open 3 times ``` - After it only `linkwait` was left. systemd's journal has "Watchdog running with a hardware timeout of 1min." at the start and "of 10s." at the end, so the driver takes 60 s. The "before" and "after" lines were read by hand around the run; the rest is the series' log. + After it only `linkwait` was left. systemd's journal has "Watchdog running with a hardware timeout of 1min." at the start and "of 10s." at the end, so the driver takes 60 s. The "before" and "after" lines were read by hand around the run; the rest is the series' log. Changed in the script after these runs, from the review, and not run yet: the log of the run before is kept beside the new one and a refused start no longer empties it, `printk_devkmsg` and the trace buffer's size are put back with the rest (an unused buffer reads `7 (expanded: 1408)`, and the first run wrote the 7 back), and a list of inhibitors that can't be read stops the run. - **The series for the count: 310 of 310 (2026-11-11, 08:43 to 16:20 MST).** The same boot and script, `N=310 S=60 PMDEBUG=1 TRACE=1`. Every sleep lasted 58.8 to 60.0 s by the kernel log and was woken by the alarm (`pm_wakeup_irq` 53), with no UFS line, no ACPM timeout and no oops; five cycles logged an RTC difference that doesn't fit (three about 20 s short, one 7 s and one about 40 s long; not looked into), while the kernel's clock has those sleeps at 60 s like the rest. The boot's kernel journal agrees with the series' log: 313 `PM: suspend entry` and 313 `PM: suspend exit` (the first run's three included), 313 `PM: Triggering wakeup from IRQ 53` and none from another interrupt, no `link startup failed`, `Host init failed`, `I/O error` or `ufshcd_err_handler` line, no RCU stall, no panic; `suspend_stats` 313 and 0. The auto-sleep asked for no sleep in that boot. At the end the "restored:" lines read as after the first run (the switches off, the tracer `nop`, the watchdog back at 10 s), and only `x-awake` held a block inhibitor. The owner wasn't at the palmtop when it began; he saw the screen cycling at several points through the day and watched a few cycles at about 11:25. -What the two days add to the count: with the PM switches on, 406 sleeps woken by the alarm (93 on 11-10, 313 on 11-11) and none hung; without counting toward that, 90 more with a 35 s alarm on 11-11. +What the two days add to the count: with the PM switches on, 406 sleeps woken by the alarm (93 on 11-10, 313 on 11-11) and none hung; without counting toward that, 90 more with a 35 s alarm on 11-11. It leans the right way and no more: at the rate without the switches (two no-wakes in about 310) two or three were to be expected in 406, and none turns up by chance about one time in thirteen, so this can't tell "gone" from "the switches change the timing". The long sleep of 11-10, nine and a half hours on this tree, ended at the first press of the Power key, which the two presses of the morning of 11-10 could not do for the sleep that hung. diff --git a/device/tegu/bringup/sleep-series.sh b/device/tegu/bringup/sleep-series.sh index a92de7352..a8ec0205f 100644 --- a/device/tegu/bringup/sleep-series.sh +++ b/device/tegu/bringup/sleep-series.sh @@ -7,7 +7,7 @@ # boot. Records each sleep's RTC length, dmesg length, wake IRQ, UFS and ACPM # errors, and the link-wait lines (a slow wake rather than a dead one). N=${N:-300}; S=${S:-60}; W=${W:-20}; L=${L:-/var/log/omamobile/sleep-linkwait-20261107.log}; U=/sys/devices/platform/soc/13200000.ufs -mkdir -p /var/log/omamobile; : > $L +mkdir -p /var/log/omamobile DBG=0; { [ "${PMDEBUG:-0}" = 1 ] || [ "${TRACE:-0}" = 1 ]; } && DBG=1 # Refusals come first, before anything is changed. modem_up() { systemctl is-active -q omamobile-modem omamobile-tegu-modem-boot 2>/dev/null || [ -d /sys/module/cpif ]; } @@ -17,6 +17,10 @@ modem_up && { echo "the modem is up: stop" >> $L; exit 1; } [ "$S" -ge 30 ] || { echo "S=$S: an alarm under 30 s can fire before the sleep begins: stop" >> $L; exit 1; } # With the debug switches a stall is a panic, and a panic has to end in a reboot. [ $DBG = 1 ] && [ "$(cat /proc/sys/kernel/panic)" = 0 ] && { echo "kernel.panic is 0, a panic would wait for the buttons: stop" >> $L; exit 1; } +# Only a run that starts gets a fresh log: a refusal above is appended to the log that is there, and the +# previous run's log is kept beside the new one. +[ -s $L ] && mv -f $L $L.prev +: > $L uid=$(id -u mobile) hy() { runuser -u mobile -- env XDG_RUNTIME_DIR=/run/user/$uid HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$uid/hypr | head -1) hyprctl dispatch "hl.dsp.dpms({ action = \"$1\" })" >/dev/null 2>&1; } cs_before=$(cat /sys/module/printk/parameters/console_suspend) @@ -37,12 +41,16 @@ T=/sys/kernel/tracing; WD=org.freedesktop.systemd1; EV="sched/sched_switch irq/i wd() { busctl "$1"-property $WD /org/freedesktop/systemd1 $WD.Manager RuntimeWatchdogUSec ${2:+t $2}; } if [ $DBG = 1 ]; then ppt_before=$(cat /sys/power/pm_print_times); pdm_before=$(cat /sys/power/pm_debug_messages) + kmsg_before=$(cat /proc/sys/kernel/printk_devkmsg) stall_before=$(cat /proc/sys/kernel/panic_on_rcu_stall); wd_before=$(wd get | awk '{print $2}') fi if [ "${TRACE:-0}" = 1 ] && [ -d $T ]; then mountpoint -q $T || mount -t tracefs nodev $T dump_before=$(cat /proc/sys/kernel/ftrace_dump_on_oops); tr_before=$(cat $T/current_tracer); on_before=$(cat $T/tracing_on) - kb_before=$(awk '{print $1}' $T/buffer_size_kb); ev_before=$(for e in $EV; do echo "$e=$(cat $T/events/$e/enable)"; done) + # Before tracing is first used the size reads "7 (expanded: 1408)": 7 KB now, 1408 on first use. Writing 7 + # back would leave a tiny buffer for the rest of the boot, so the size it would have grown to is put back. + kb_before=$(sed -E 's/^[0-9]+ \(expanded: ([0-9]+)\)$/\1/' $T/buffer_size_kb) + ev_before=$(for e in $EV; do echo "$e=$(cat $T/events/$e/enable)"; done) fi restore() { [ -n "$INH" ] && kill $INH 2>/dev/null @@ -54,6 +62,7 @@ restore() { if [ $DBG = 1 ]; then echo "$stall_before" > /proc/sys/kernel/panic_on_rcu_stall echo "$ppt_before" > /sys/power/pm_print_times; echo "$pdm_before" > /sys/power/pm_debug_messages + echo "$kmsg_before" > /proc/sys/kernel/printk_devkmsg 2>/dev/null [ -n "$wd_before" ] && wd set "$wd_before" echo "restored: pm_print_times $(cat /sys/power/pm_print_times), pm_debug_messages $(cat /sys/power/pm_debug_messages), panic_on_rcu_stall $(cat /proc/sys/kernel/panic_on_rcu_stall), ftrace_dump_on_oops $(cat /proc/sys/kernel/ftrace_dump_on_oops), tracer $(cat $T/current_tracer 2>/dev/null || echo none), runtime watchdog $(wd get | awk '{print $2}') (was $wd_before)" >> $L fi @@ -99,15 +108,20 @@ echo "debug: pm_print_times $(cat /sys/power/pm_print_times), pm_debug_messages # hours, until the Power key, which from the laptop looked like a no-wake. So the series holds a block # inhibitor of its own for the whole run (a push script's x-awake has a time limit) and sleeps past it # with "systemctl suspend -i". That passes every block inhibitor, so before each sleep the series looks -# again: the modem up, or any block-mode sleep inhibitor other than its own and x-awake's, ends the run. +# again: the modem up, any block-mode sleep inhibitor other than its own and x-awake's, or a list that +# can't be read, ends the run. systemd-inhibit --what=sleep --who=sleep-series --why="only the series' own sleeps, each with an alarm" --mode=block sleep infinity & INH=$! others() { - busctl --json=short call org.freedesktop.login1 /org/freedesktop/login1 org.freedesktop.login1.Manager ListInhibitors | python3 -c ' + local j + j=$(busctl --json=short call org.freedesktop.login1 /org/freedesktop/login1 org.freedesktop.login1.Manager ListInhibitors) \ + || { echo "the inhibitor list could not be read"; return; } + printf '%s' "$j" | python3 -c ' import json, sys mine = set(sys.argv[1:]) print(" ".join("%s(pid %s)" % (i[1], i[5]) for i in json.load(sys.stdin)["data"][0] - if "sleep" in i[0].split(":") and i[3] == "block" and str(i[5]) not in mine))' $INH $(systemctl show -p MainPID --value x-awake 2>/dev/null) + if "sleep" in i[0].split(":") and i[3] == "block" and str(i[5]) not in mine))' $INH $(systemctl show -p MainPID --value x-awake 2>/dev/null) \ + || echo "the inhibitor list could not be read" } st1 ${WFI:-1} echo 5 > $U/spm_lvl @@ -117,7 +131,7 @@ echo "$(uname -v): rpm_lvl $(cat $U/rpm_lvl) (was $rpm_before), ufs-vcc always-o for c in $(seq 1 $N); do [ -e /run/sleep-series.stop ] && { rm -f /run/sleep-series.stop; echo "stop file: stop after $((c - 1)) cycles" >> $L; break; } modem_up && { echo "cycle $c: the modem is up: stop" >> $L; exit 1; } - o=$(others); [ -z "$o" ] || { echo "cycle $c: another block-mode sleep inhibitor is held ($o): stop" >> $L; exit 1; } + o=$(others); [ -z "$o" ] || { echo "cycle $c: another block-mode sleep inhibitor is held, or the list could not be read ($o): stop" >> $L; exit 1; } dmesg -C hy disable r0=$(cat /sys/class/rtc/rtc0/since_epoch) diff --git a/docs/DEVICES.md b/docs/DEVICES.md index d8de37503..a087ad73c 100644 --- a/docs/DEVICES.md +++ b/docs/DEVICES.md @@ -25,7 +25,7 @@ The status of each device, as verified on real hardware. Rows in the matrix have | Pixel 6 | oriole | gs101 (Tensor G1) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); native test image on one phone; slot B marked successful and `vbmeta_b` verification turned off, both without project approval; see notes) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); guarded `userdata` root on test image) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); connected output on test image) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); Panfrost render node present) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); touchscreen detected; input not checked by smoke) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); gpio-keys detected; input not checked by smoke) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); USB network and SSH on test image) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); link up; traffic not checked by smoke) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); controller powered; pairing not checked) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); charge and temperature reported) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); charging reported; control not checked) | no | no | no | no | no | no | no | no | no | no | no | | Pixel 4 | flame | SM8150 (Snapdragon 855) | testing (temporary RAM boot; [notes](../device/flame/BRINGUP-M1.md)) | testing (RAM probe; all UFS read-only) | testing (1080x2280 console; software desktop in separate Arch payload) | testing (separate Arch payload; FD640 contexts, llvmpipe desktop) | testing (separate Arch payload; events and PIN unlock verified) | no | testing (NCM + ACM + SSH; sustained transfers can disconnect) | testing (guarded RAM scan; association untested) | no | no | no | no | no | no | - | no | no | no | no | no | no | - | | Pixel 4 XL ([notes](../device/coral/NOTES.md)) | coral | SM8150 (Snapdragon 855) | WIP (direct ABL v2 boot on the test handset's slot A; slot-B install and §3.1 entry pending; [notes](../device/coral/BRINGUP-M1.md)) | testing (read-only `blockdev` guard + RAM `overlayfs`) | testing (1440x3040@60Hz S6E3HC2 DSC 1.1 KMS, DPMS, brightness) | testing (Adreno 640 / FD640 Mesa Freedreno, Hyprland) | testing (FTS SPI multitouch + read-only flash guard) | testing (power via `qcom-pon` + volume via `gpio-keys`) | testing (CDC ECM + ACM + SSH; stays up with host USB switched off) | testing (`WCN3990` `ath10k_snoc` + `rmtfs -r`, WPA2/DHCP/HTTPS; radio sessions ran before the modem baseline) | WIP (`WCN3990` `hci_qca` `hci0` up + BLE scan; audio/pairing open) | WIP (PM8150B ADC `VBAT_SNS` voltage + FG temp; aftermarket pack on test unit: no reply from `1-0036`) | testing (PM8150B SMB5 charger with `4.20 V` / `500 mA` kernel clamp) | testing (`CS40L25A` I2C click waveform) | testing (PM8150L flash LEDs + quick-settings tile) | no | - | testing (QDSP6 + dual `CS35L36` amps capped at `-26 dB` + `IA8508` SPI mic DSP) | testing (`SLPI` + `fastrpc` + RAM-backed `hexagonrpcd` + QMI SSC: 7 sensors + Soli radar) | WIP (`MPSS` boots with `rmtfs -r` + patched `tqftpserv`, SIM detected; no IPA/voice yet; radio sessions ran before the modem baseline) | no | testing (`CAMSS` + `IMX363`, `IMX481`, `IMX520`, dual `OV7251` NIR; `libcamera` softISP) | no | - | -| Pixel 9a ([notes](../device/tegu/NOTES.md)) | tegu | zumapro (Tensor G4) | works ([#28](https://github.com/derekross/omarchy-mobile/issues/28): native boot from slot B, the Arch root on `userdata`, `omamobile-device-tegu`; slot B was marked successful by hand on 2026-10-28 with the `devinfo` write approved on 2026-10-22, now on hold ([#125](https://github.com/derekross/omarchy-mobile/pull/125)), unmarked by the 10-31 fastboot recovery and marked again on 2026-10-31 during the hold ([record](../device/tegu/NOTES.md#2026-10-31-a-fastboot-flash-of-slot-b-cleared-its-successful-mark-and-b-was-marked-again)), and the package does not mark it; unpublished kernel: the SoC and board device tree is [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (UFS behind the read-only guard; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (120 Hz through gs-decon; the panel driver [kernel #89](https://github.com/derekross/linux-omarchy-mobile/pull/89), 120 Hz [kernel #88](https://github.com/derekross/linux-omarchy-mobile/pull/88), merged, the display DT, merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96); brightness, DPMS off and on and the 60/120 Hz switch on the boot of [kernel #125](https://github.com/derekross/linux-omarchy-mobile/pull/125), merged, which also checks the panel's ID; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on), [brightness record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (Panthor, the desktop renders, runtime suspend when idle; this boot runs at a fixed 160 MHz; the GPU's levels and its thermal zone are merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96) ([kernel #85](https://github.com/derekross/linux-omarchy-mobile/pull/85) and [kernel #87](https://github.com/derekross/linux-omarchy-mobile/pull/87) landed through it), not yet run from it; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (Synaptics S3908 over SPI, [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; survives sleep, [kernel #82](https://github.com/derekross/linux-omarchy-mobile/pull/82), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (gpio-keys: power and volume; the power key wakes from sleep; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (CDC NCM and SSH; reconnects after a replug, `omamobile-tegu-usb-rescue`, also after a replug the gadget didn't see ([#280](https://github.com/derekross/omarchy-mobile/pull/280); [the replug record](../device/tegu/NOTES.md#2026-10-28-what-the-device-tables-claims-rest-on)); [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (BCM4383 on PCIe channel 1 through the shared gs201 host and zuma PHY, [kernel #178](https://github.com/derekross/linux-omarchy-mobile/pull/178) on [kernel #156](https://github.com/derekross/linux-omarchy-mobile/pull/156), in review, and brcmfmac, [kernel #56](https://github.com/derekross/linux-omarchy-mobile/pull/56) in review; started at boot by `omamobile-device-tegu` 0.2.4 ([#238](https://github.com/derekross/omarchy-mobile/pull/238)) with the extractor's firmware; WPA2 with iwd, kept loaded across sleep by the package's sleep hook; [record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (hci_uart on the bootloader's address, BlueZ, A2DP; [kernel #94](https://github.com/derekross/linux-omarchy-mobile/pull/94); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (MAX77779 gauge with temperature, current and time estimates through UPower; [kernel #66](https://github.com/derekross/linux-omarchy-mobile/pull/66); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (the MAX77779 charger binds from the device tree, [kernel #161](https://github.com/derekross/linux-omarchy-mobile/pull/161), merged, built by [#227](https://github.com/derekross/omarchy-mobile/pull/227); the charge limit in `omamobile-device-tegu` 0.2.3 holds charging off at the limit through `charge_behaviour` and lets it run again 5 below; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (CS40L26A through feedbackd, with brownout prevention, [kernel #95](https://github.com/derekross/linux-omarchy-mobile/pull/95); its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (LM3644, 25 steps to 75 mA, quick-settings tile; its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | testing (Omarchy's Location switch serves fixes from GPS, Galileo, BeiDou and GLONASS through the modem's Kepler once the modem is up; `omamobile-device-tegu`'s boot unit, opt-in ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), starts gnssd with the modem: on 2026-11-03, after a reboot with nothing started by hand, a session through the switch's socket had 133 of 135 GGA sentences fixed, first fix 48 s after the client connected, 18 satellites; on 2026-11-09 the same from the in-tree drivers (the GNSS link driver, [kernel #185](https://github.com/derekross/linux-omarchy-mobile/pull/185), and the 9a's GNSS nodes, [kernel #237](https://github.com/derekross/linux-omarchy-mobile/pull/237), the `modem-cpif` overlay with the driver's config line, [#594](https://github.com/derekross/omarchy-mobile/pull/594), and the PCIe PHY's config line, [#274](https://github.com/derekross/omarchy-mobile/pull/274), all merged): four 150 s sessions, 497 of 507 GGA sentences fixed, up to 24 satellites; a 30-minute session on 2026-11-01 (omamobile-gnss 0.2.5's start, by hand, the modem link left to cycle) had 1763 of 1765, first fix 37 s, and a session in a moving car from [#114](https://github.com/derekross/omarchy-mobile/pull/114)'s package build fixed 145 of 153 times; needs [kernel #150](https://github.com/derekross/linux-omarchy-mobile/pull/150) and the PERIC0 clock fix [kernel #172](https://github.com/derekross/linux-omarchy-mobile/pull/172), both merged (without the latter the modem link hung the palmtop within minutes); not on by default because the modem isn't (the Modem cell); [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/TELEPHONY-PLAN.md), record with [#202](https://github.com/derekross/omarchy-mobile/pull/202)) | works (audio and sensor DSP through the GSA and Trusty, 93 services, started at boot by `omamobile-device-tegu` 0.2.3 from the device tree, [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162) in review; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (the TAS2572 amplifiers and the microphone through the AoC card and PipeWire, at boot with the AoC: a tone from the speaker, the microphone at -32 dBFS; speakers at a -10 dB ceiling; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (accelerometer, light, proximity, compass and gyroscope through the AoC and the sensor proxy, at boot with the AoC; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | testing (calls in and out on the speakerphone, texts both ways, mobile data over the device `omamobile-sit-data` finds, with the Phone and Messages apps on the bridge ([#356](https://github.com/derekross/omarchy-mobile/pull/356), [#364](https://github.com/derekross/omarchy-mobile/pull/364), [#367](https://github.com/derekross/omarchy-mobile/pull/367), [#371](https://github.com/derekross/omarchy-mobile/pull/371)); `omamobile-device-tegu`'s boot unit starts the modem, the phone service and the bridge at boot when the owner enables it ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), and is skipped on a phone that isn't provisioned or has no CP drivers; opt-in, not on by default, while marking slot B on the 9a is on hold ([#483](https://github.com/derekross/omarchy-mobile/pull/483)): with B unmarked a reset spends one of ABL's three tries, and the unit can't tell; it ran by itself on the owner's palmtop over a day of reboots on 2026-11-09 with 42 of 43 stops confirmed, one unexplained ([record](../device/tegu/TELEPHONY-PLAN.md#29-the-in-tree-drivers-through-the-starter-on-the-published-tip-plus-four-open-changes-2026-11-09)), and the NV partitions unchanged, on the owner's kernel with the bring-up modules, and on the in-tree drivers ([kernel #187](https://github.com/derekross/linux-omarchy-mobile/pull/187) and [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162), merged), which need the `modem-cpif` overlay and the GNSS link driver's config line ([#594](https://github.com/derekross/omarchy-mobile/pull/594)) and the PCIe PHY's config line ([#274](https://github.com/derekross/omarchy-mobile/pull/274)), all merged; the far end still hears an echo on the speakerphone ([#378](https://github.com/derekross/omarchy-mobile/pull/378)) and there's no earpiece route yet; [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/MODEM-PLAN.md#sim-test-results-2026-11-02-730-rc6-omamobile-tegu-s2mpg15-m1-g991042e2eff3-dirty)) | works (NXP ST54L through nxp-nci, its clock pad from the PMU ([kernel #104](https://github.com/derekross/linux-omarchy-mobile/pull/104) and [kernel #105](https://github.com/derekross/linux-omarchy-mobile/pull/105), merged, with [kernel #131](https://github.com/derekross/linux-omarchy-mobile/pull/131), merged), loaded at boot by `omamobile-device-tegu` 0.2.3 with neard and the link opener: polls with no I2C errors and read a payment card; polling stops before sleep; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (main, ultrawide and front from the device tree, [kernel #164](https://github.com/derekross/linux-omarchy-mobile/pull/164) in review, with [kernel #63](https://github.com/derekross/linux-omarchy-mobile/pull/63) and [kernel #103](https://github.com/derekross/linux-omarchy-mobile/pull/103) in review: their drivers load at boot with nothing by hand, libcamera lists all three, and each streams, main at 19 fps and the others at 30 fps at 1280x960; basic picture quality through libcamera's software ISP; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel); the main camera's contrast autofocus through its LC898129 ([kernel #213](https://github.com/derekross/linux-omarchy-mobile/pull/213) in review, and a lens node on top of kernel #164, not yet in a pull request; libcamera 0.7.2-4.23), close subjects checked, the far end not yet ([notes](../device/tegu/NOTES.md#2026-11-07-the-main-cameras-autofocus-lc898129))) | testing (s2idle when asked for, or through the shell's auto-sleep with the `omamobile-sleep` toggle on, which ships off; the power key lights the lock screen on the first press ([#268](https://github.com/derekross/omarchy-mobile/pull/268), [#275](https://github.com/derekross/omarchy-mobile/pull/275)), and the RTC wakes it; on 2026-11-01 thirty attended 120 s sleeps ([plan](../device/tegu/SLEEP-PLAN.md) S1) and a day of real use with auto-sleep on ([record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) all woke, but six test sleeps since 10-28 did not (one in 21 on the clock-fix kernel on 11-01, and one in 127 in an overnight series of 120 s sleeps on 11-01 to 11-02), so sleep tests stay attended ([plan](../device/tegu/SLEEP-PLAN.md) S1); two causes are known: dead storage after a wake (the UFS pads came back as plain inputs on about one wake in a hundred; [kernel #224](https://github.com/derekross/linux-omarchy-mobile/pull/224), in review, re-applies the pin state at resume), and a silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each, then 15 of 15 on the everyday path, twelve of them half-hour auto-sleeps (2026-11-07 and 11-08, [#409](https://github.com/derekross/omarchy-mobile/pull/409)); [#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it; open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules), none in two re-runs of 36, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes, where the console shows the kernel alive and CPU 7 not answering from just after the device suspend ([plan](../device/tegu/SLEEP-PLAN.md) S1, failure 3); the RTC alarm on the published kernel came with [kernel #236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10 (a kernel from before it has none, and a sleep there ends only on a key); works once that no-wake is found and fixed and kernel #224 is merged and has run from the published kernel; the drain is about 300 mA asleep, the same as awake with the screen off, because nothing below the CPUs sleeps yet; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (opt-in) (enrolls, matches, refuses other fingers and unlocks the lock screen through Trusty and `omamobile-fingerprint`, 2026-10-29; the sensor binds from the device tree, [kernel #165](https://github.com/derekross/linux-omarchy-mobile/pull/165) in review, with [kernel #126](https://github.com/derekross/linux-omarchy-mobile/pull/126), merged; opt-in as on the 7 Pro, because the service's first run writes Trusty's secure storage; [plan](../device/tegu/FINGERPRINT-PLAN.md), [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | +| Pixel 9a ([notes](../device/tegu/NOTES.md)) | tegu | zumapro (Tensor G4) | works ([#28](https://github.com/derekross/omarchy-mobile/issues/28): native boot from slot B, the Arch root on `userdata`, `omamobile-device-tegu`; slot B was marked successful by hand on 2026-10-28 with the `devinfo` write approved on 2026-10-22, now on hold ([#125](https://github.com/derekross/omarchy-mobile/pull/125)), unmarked by the 10-31 fastboot recovery and marked again on 2026-10-31 during the hold ([record](../device/tegu/NOTES.md#2026-10-31-a-fastboot-flash-of-slot-b-cleared-its-successful-mark-and-b-was-marked-again)), and the package does not mark it; unpublished kernel: the SoC and board device tree is [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (UFS behind the read-only guard; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (120 Hz through gs-decon; the panel driver [kernel #89](https://github.com/derekross/linux-omarchy-mobile/pull/89), 120 Hz [kernel #88](https://github.com/derekross/linux-omarchy-mobile/pull/88), merged, the display DT, merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96); brightness, DPMS off and on and the 60/120 Hz switch on the boot of [kernel #125](https://github.com/derekross/linux-omarchy-mobile/pull/125), merged, which also checks the panel's ID; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on), [brightness record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (Panthor, the desktop renders, runtime suspend when idle; this boot runs at a fixed 160 MHz; the GPU's levels and its thermal zone are merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96) ([kernel #85](https://github.com/derekross/linux-omarchy-mobile/pull/85) and [kernel #87](https://github.com/derekross/linux-omarchy-mobile/pull/87) landed through it), not yet run from it; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (Synaptics S3908 over SPI, [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; survives sleep, [kernel #82](https://github.com/derekross/linux-omarchy-mobile/pull/82), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (gpio-keys: power and volume; the power key wakes from sleep; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (CDC NCM and SSH; reconnects after a replug, `omamobile-tegu-usb-rescue`, also after a replug the gadget didn't see ([#280](https://github.com/derekross/omarchy-mobile/pull/280); [the replug record](../device/tegu/NOTES.md#2026-10-28-what-the-device-tables-claims-rest-on)); [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (BCM4383 on PCIe channel 1 through the shared gs201 host and zuma PHY, [kernel #178](https://github.com/derekross/linux-omarchy-mobile/pull/178) on [kernel #156](https://github.com/derekross/linux-omarchy-mobile/pull/156), in review, and brcmfmac, [kernel #56](https://github.com/derekross/linux-omarchy-mobile/pull/56) in review; started at boot by `omamobile-device-tegu` 0.2.4 ([#238](https://github.com/derekross/omarchy-mobile/pull/238)) with the extractor's firmware; WPA2 with iwd, kept loaded across sleep by the package's sleep hook; [record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (hci_uart on the bootloader's address, BlueZ, A2DP; [kernel #94](https://github.com/derekross/linux-omarchy-mobile/pull/94); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (MAX77779 gauge with temperature, current and time estimates through UPower; [kernel #66](https://github.com/derekross/linux-omarchy-mobile/pull/66); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (the MAX77779 charger binds from the device tree, [kernel #161](https://github.com/derekross/linux-omarchy-mobile/pull/161), merged, built by [#227](https://github.com/derekross/omarchy-mobile/pull/227); the charge limit in `omamobile-device-tegu` 0.2.3 holds charging off at the limit through `charge_behaviour` and lets it run again 5 below; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (CS40L26A through feedbackd, with brownout prevention, [kernel #95](https://github.com/derekross/linux-omarchy-mobile/pull/95); its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (LM3644, 25 steps to 75 mA, quick-settings tile; its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | testing (Omarchy's Location switch serves fixes from GPS, Galileo, BeiDou and GLONASS through the modem's Kepler once the modem is up; `omamobile-device-tegu`'s boot unit, opt-in ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), starts gnssd with the modem: on 2026-11-03, after a reboot with nothing started by hand, a session through the switch's socket had 133 of 135 GGA sentences fixed, first fix 48 s after the client connected, 18 satellites; on 2026-11-09 the same from the in-tree drivers (the GNSS link driver, [kernel #185](https://github.com/derekross/linux-omarchy-mobile/pull/185), and the 9a's GNSS nodes, [kernel #237](https://github.com/derekross/linux-omarchy-mobile/pull/237), the `modem-cpif` overlay with the driver's config line, [#594](https://github.com/derekross/omarchy-mobile/pull/594), and the PCIe PHY's config line, [#274](https://github.com/derekross/omarchy-mobile/pull/274), all merged): four 150 s sessions, 497 of 507 GGA sentences fixed, up to 24 satellites; a 30-minute session on 2026-11-01 (omamobile-gnss 0.2.5's start, by hand, the modem link left to cycle) had 1763 of 1765, first fix 37 s, and a session in a moving car from [#114](https://github.com/derekross/omarchy-mobile/pull/114)'s package build fixed 145 of 153 times; needs [kernel #150](https://github.com/derekross/linux-omarchy-mobile/pull/150) and the PERIC0 clock fix [kernel #172](https://github.com/derekross/linux-omarchy-mobile/pull/172), both merged (without the latter the modem link hung the palmtop within minutes); not on by default because the modem isn't (the Modem cell); [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/TELEPHONY-PLAN.md), record with [#202](https://github.com/derekross/omarchy-mobile/pull/202)) | works (audio and sensor DSP through the GSA and Trusty, 93 services, started at boot by `omamobile-device-tegu` 0.2.3 from the device tree, [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162) in review; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (the TAS2572 amplifiers and the microphone through the AoC card and PipeWire, at boot with the AoC: a tone from the speaker, the microphone at -32 dBFS; speakers at a -10 dB ceiling; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (accelerometer, light, proximity, compass and gyroscope through the AoC and the sensor proxy, at boot with the AoC; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | testing (calls in and out on the speakerphone, texts both ways, mobile data over the device `omamobile-sit-data` finds, with the Phone and Messages apps on the bridge ([#356](https://github.com/derekross/omarchy-mobile/pull/356), [#364](https://github.com/derekross/omarchy-mobile/pull/364), [#367](https://github.com/derekross/omarchy-mobile/pull/367), [#371](https://github.com/derekross/omarchy-mobile/pull/371)); `omamobile-device-tegu`'s boot unit starts the modem, the phone service and the bridge at boot when the owner enables it ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), and is skipped on a phone that isn't provisioned or has no CP drivers; opt-in, not on by default, while marking slot B on the 9a is on hold ([#483](https://github.com/derekross/omarchy-mobile/pull/483)): with B unmarked a reset spends one of ABL's three tries, and the unit can't tell; it ran by itself on the owner's palmtop over a day of reboots on 2026-11-09 with 42 of 43 stops confirmed, one unexplained ([record](../device/tegu/TELEPHONY-PLAN.md#29-the-in-tree-drivers-through-the-starter-on-the-published-tip-plus-four-open-changes-2026-11-09)), and the NV partitions unchanged, on the owner's kernel with the bring-up modules, and on the in-tree drivers ([kernel #187](https://github.com/derekross/linux-omarchy-mobile/pull/187) and [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162), merged), which need the `modem-cpif` overlay and the GNSS link driver's config line ([#594](https://github.com/derekross/omarchy-mobile/pull/594)) and the PCIe PHY's config line ([#274](https://github.com/derekross/omarchy-mobile/pull/274)), all merged; the far end still hears an echo on the speakerphone ([#378](https://github.com/derekross/omarchy-mobile/pull/378)) and there's no earpiece route yet; [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/MODEM-PLAN.md#sim-test-results-2026-11-02-730-rc6-omamobile-tegu-s2mpg15-m1-g991042e2eff3-dirty)) | works (NXP ST54L through nxp-nci, its clock pad from the PMU ([kernel #104](https://github.com/derekross/linux-omarchy-mobile/pull/104) and [kernel #105](https://github.com/derekross/linux-omarchy-mobile/pull/105), merged, with [kernel #131](https://github.com/derekross/linux-omarchy-mobile/pull/131), merged), loaded at boot by `omamobile-device-tegu` 0.2.3 with neard and the link opener: polls with no I2C errors and read a payment card; polling stops before sleep; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (main, ultrawide and front from the device tree, [kernel #164](https://github.com/derekross/linux-omarchy-mobile/pull/164) in review, with [kernel #63](https://github.com/derekross/linux-omarchy-mobile/pull/63) and [kernel #103](https://github.com/derekross/linux-omarchy-mobile/pull/103) in review: their drivers load at boot with nothing by hand, libcamera lists all three, and each streams, main at 19 fps and the others at 30 fps at 1280x960; basic picture quality through libcamera's software ISP; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel); the main camera's contrast autofocus through its LC898129 ([kernel #213](https://github.com/derekross/linux-omarchy-mobile/pull/213) in review, and a lens node on top of kernel #164, not yet in a pull request; libcamera 0.7.2-4.23), close subjects checked, the far end not yet ([notes](../device/tegu/NOTES.md#2026-11-07-the-main-cameras-autofocus-lc898129))) | testing (s2idle when asked for, or through the shell's auto-sleep with the `omamobile-sleep` toggle on, which ships off; the power key lights the lock screen on the first press ([#268](https://github.com/derekross/omarchy-mobile/pull/268), [#275](https://github.com/derekross/omarchy-mobile/pull/275)), and the RTC wakes it; on 2026-11-01 thirty attended 120 s sleeps ([plan](../device/tegu/SLEEP-PLAN.md) S1) and a day of real use with auto-sleep on ([record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) all woke, but six test sleeps since 10-28 did not (one in 21 on the clock-fix kernel on 11-01, and one in 127 in an overnight series of 120 s sleeps on 11-01 to 11-02), so sleep tests stay attended ([plan](../device/tegu/SLEEP-PLAN.md) S1); two causes are known: dead storage after a wake (the UFS pads came back as plain inputs on about one wake in a hundred; [kernel #224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged 2026-11-12, re-applies the pin state at resume), and a silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each, then 15 of 15 on the everyday path, twelve of them half-hour auto-sleeps (2026-11-07 and 11-08, [#409](https://github.com/derekross/omarchy-mobile/pull/409)); [#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it; open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules), none in two re-runs of 36, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes, where the console shows the kernel alive and CPU 7 not answering from just after the device suspend ([plan](../device/tegu/SLEEP-PLAN.md) S1, failure 3), and it hasn't come back in 406 sleeps with the PM debug switches on (2026-11-10 and 11-11, 313 of them 60 s sleeps on one boot), which leans the right way but doesn't show it gone; the RTC alarm on the published kernel came with [kernel #236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10 (a kernel from before it has none, and a sleep there ends only on a key); works once that no-wake is found and fixed and a series has run from the published kernel as it is, with kernel #224 in it; the drain is about 300 mA asleep, the same as awake with the screen off, because nothing below the CPUs sleeps yet; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (opt-in) (enrolls, matches, refuses other fingers and unlocks the lock screen through Trusty and `omamobile-fingerprint`, 2026-10-29; the sensor binds from the device tree, [kernel #165](https://github.com/derekross/linux-omarchy-mobile/pull/165) in review, with [kernel #126](https://github.com/derekross/linux-omarchy-mobile/pull/126), merged; opt-in as on the 7 Pro, because the service's first run writes Trusty's secure storage; [plan](../device/tegu/FINGERPRINT-PLAN.md), [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | | Pixel 8 Pro ([notes](../device/husky/NOTES.md)) | husky | zuma (Tensor G3) | WIP ([#1](https://github.com/derekross/omarchy-mobile/issues/1): slot B; from 2026-11-01 to 2026-11-07 a local unit marked slot B on Lef's phone; stopped pending [#252](https://github.com/derekross/omarchy-mobile/issues/252); Omarchy's image from the signed repository boots to the lock screen; the zuma device tree is merged ([kernel #13](https://github.com/derekross/linux-omarchy-mobile/pull/13), [#53](https://github.com/derekross/linux-omarchy-mobile/pull/53))) | testing (UFS at HS-G4 behind the read-only guard; Arch on `userdata`; [UFS-PLAN](../device/husky/UFS-PLAN.md)) | WIP (simple-framebuffer with the panel's size, [kernel #154](https://github.com/derekross/linux-omarchy-mobile/pull/154): console and Omarchy's lock screen; 120 Hz with brightness on a test kernel, [kernel #160](https://github.com/derekross/linux-omarchy-mobile/pull/160), started by the device package, [#228](https://github.com/derekross/omarchy-mobile/pull/228)) | testing (panthor on the Mali-G715 on a test kernel, [kernel #157](https://github.com/derekross/linux-omarchy-mobile/pull/157); [ROADMAP](../device/husky/ROADMAP.md)) | works (ST FTM5 over SPI, merged, [kernel #153](https://github.com/derekross/linux-omarchy-mobile/pull/153); verified on both phones, [notes](../device/husky/NOTES.md#2026-10-31-the-touchscreen)) | works (power and volume through `gpio-keys`, merged, [kernel #148](https://github.com/derekross/linux-omarchy-mobile/pull/148); verified on both phones, [notes](../device/husky/NOTES.md#2026-10-30-the-buttons)) | testing (USB and SSH) | testing (BCM4398 on a test kernel, [kernel #156](https://github.com/derekross/linux-omarchy-mobile/pull/156); firmware from the owner's image, [#214](https://github.com/derekross/omarchy-mobile/pull/214); [NOTES](../device/husky/NOTES.md)) | testing (BCM4398D0 on the APM UART, pairing and A2DP, on a test kernel, [kernel #199](https://github.com/derekross/linux-omarchy-mobile/pull/199); firmware from the owner's image; [ROADMAP](../device/husky/ROADMAP.md)) | testing (gauge with Google's ATL model through a bring-up overlay, temperature, MFD, pack EEPROM, [kernel #163](https://github.com/derekross/linux-omarchy-mobile/pull/163) and [#175](https://github.com/derekross/linux-omarchy-mobile/pull/175); the bus on by default waits for [kernel #205](https://github.com/derekross/linux-omarchy-mobile/pull/205); [notes](../device/husky/BATTERY-NOTES.md)) | testing (the charger driver and `tcpci_maxim` as a sink, by hand under a dead-man tool: sink at the Type-C 3 A level (input limit 0xc2 0x13 to 0x77); +1.5 A measured from a 30 W charger without the driver (1.485 A is the device-tree cap); no host mode, [#282](https://github.com/derekross/omarchy-mobile/pull/282); [notes](../device/husky/BATTERY-NOTES.md), [#1](https://github.com/derekross/omarchy-mobile/issues/1#issuecomment-6562547503)) | testing (CS40L26 on a test kernel, [kernel #200](https://github.com/derekross/linux-omarchy-mobile/pull/200)) | testing (LM3644, 50 mA cap, on a test kernel, [kernel #200](https://github.com/derekross/linux-omarchy-mobile/pull/200)) | no (first steps as drafts: [kernel #210](https://github.com/derekross/linux-omarchy-mobile/pull/210), [#211](https://github.com/derekross/linux-omarchy-mobile/pull/211)) | testing (loads and runs through the GSA and Trusty on a test kernel; plan and results [AOC-PLAN](../device/husky/AOC-PLAN.md)) | testing (stereo speakers and one microphone through UCM2 and PipeWire on a test kernel, [kernel #203](https://github.com/derekross/linux-omarchy-mobile/pull/203), [#204](https://github.com/derekross/linux-omarchy-mobile/pull/204), package [#453](https://github.com/derekross/omarchy-mobile/pull/453)) | no | no | testing (a test kernel reads tags; [kernel #208](https://github.com/derekross/linux-omarchy-mobile/pull/208), [#209](https://github.com/derekross/linux-omarchy-mobile/pull/209)) | no | no | no | | Pixel 8a ([notes](../device/akita/NOTES.md)) | akita | zuma (Tensor G3) | WIP ([#1](https://github.com/derekross/omarchy-mobile/issues/1): slot B into the initramfs with USB and SSH, no Arch root yet; `reboot-to bootloader` reaches fastboot; the first-light board file is merged, [kernel #192](https://github.com/derekross/linux-omarchy-mobile/pull/192); [BRINGUP-M1](../device/akita/BRINGUP-M1.md), [notes](../device/akita/NOTES.md#2026-11-03-first-light-m13-and-the-console-m14)) | WIP (UFS at HS-G4 behind the read-only guard, from the initramfs only, nothing run on `userdata`; merged, [kernel #206](https://github.com/derekross/linux-omarchy-mobile/pull/206); the device package and storage plan are in review, [#456](https://github.com/derekross/omarchy-mobile/pull/456)) | WIP (simple-framebuffer: the boot console over Omarchy's splash; merged with [kernel #192](https://github.com/derekross/linux-omarchy-mobile/pull/192); [notes](../device/akita/NOTES.md#2026-11-03-first-light-m13-and-the-console-m14)) | no | no | no | testing (USB and SSH from the initramfs; merged with [kernel #192](https://github.com/derekross/linux-omarchy-mobile/pull/192)) | no | no | no | no | no | no | no | no | no | no | no | no | no | no | no | | Pixel 7 ([notes](../device/panther/NOTES.md)) | panther | gs201 (Tensor G2) | WIP (slot B, not marked successful; Arch and Hyprland boot; `boot` and `init_boot` carry AVB footers, which a phone whose `vbmeta_b` is stock's needs to start slot B (device-panther 0.1.6, [#396](https://github.com/derekross/omarchy-mobile/pull/396), [notes](../device/panther/NOTES.md#2026-11-02-avb-footers-on-boot-and-init_boot)); `reboot-to bootloader` reaches fastboot with `reboot=warm`, merged, [kernel #198](https://github.com/derekross/linux-omarchy-mobile/pull/198); [BRINGUP-M1](../device/panther/BRINGUP-M1.md); board file merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108)) | testing (UFS behind the read-only guard; Arch on `userdata`; merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108)) | WIP (simple-framebuffer: console and the Hyprland desktop; brightness and display off/on over DCS through the S6E3FC3 panel driver and device-panther 0.1.4, merged, [kernel #177](https://github.com/derekross/linux-omarchy-mobile/pull/177); [DISPLAY-PLAN](../device/panther/DISPLAY-PLAN.md)) | testing (Panthor on the Mali-G710; Hyprland renders on it; merged through [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134), with the always-on rule in device-panther 0.1.1, [#145](https://github.com/derekross/omarchy-mobile/pull/145); [notes](../device/panther/NOTES.md#2026-10-28-the-gpu-and-the-haptics-power-off-and-usb-after-a-replug)) | WIP (FocalTech FT3658: touches drive Hyprland and unlock the PIN screen on a test kernel; the driver, [kernel #109](https://github.com/derekross/linux-omarchy-mobile/pull/109), and the touch node, [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134), are merged; stock's interrupt pull-up and CS delay, [kernel #166](https://github.com/derekross/linux-omarchy-mobile/pull/166), the driver's review fixes, [kernel #133](https://github.com/derekross/linux-omarchy-mobile/pull/133), and the kernel option with the touch notes, [#149](https://github.com/derekross/omarchy-mobile/pull/149), are merged) | testing (gpio-keys: power, volume; merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108)) | testing (USB and SSH; merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108); a replug comes back through `omamobile-usb-rescue` in device-panther 0.1.3, on the MAX77759's I2C bus, merged, [kernel #167](https://github.com/derekross/linux-omarchy-mobile/pull/167); [notes](../device/panther/NOTES.md#2026-10-29-usb-comes-back-after-a-replug-u1)) | no | testing (BCM4389 on the APM UART through hci_bcm with the phone's own patch, merged, [kernel #196](https://github.com/derekross/linux-omarchy-mobile/pull/196): a BLE mouse drives the shell, and headphones play over A2DP with their AVRCP buttons, on its author's Pixel 7; the Bluetooth address and the rest of the package side aren't merged yet; headset calls, suspend and power not tested) | WIP (MAX77759 fuel gauge with stock's battery model on a test kernel: percentage, capacity and temperature in upower and the bar; merged, [kernel #171](https://github.com/derekross/linux-omarchy-mobile/pull/171) and the driver fix [#170](https://github.com/derekross/linux-omarchy-mobile/pull/170); a second unit's Lishen pack isn't matched by that model yet, [notes](../device/panther/NOTES.md#2026-11-03-a-second-unit-angelrenefm); [BATTERY-PLAN](../device/panther/BATTERY-PLAN.md)) | no | testing (CS40L26 with the phone's own firmware, `tools/extract-vendor-firmware -d panther`; merged through [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134), with the feedbackd rule in device-panther 0.1.1, [#145](https://github.com/derekross/omarchy-mobile/pull/145); [notes](../device/panther/NOTES.md#2026-10-28-the-gpu-and-the-haptics-power-off-and-usb-after-a-replug)) | testing (LM3644 rear flash, 36 steps, from the node merged in [kernel #197](https://github.com/derekross/linux-omarchy-mobile/pull/197) and the bring-up overlay `torch.dtso`, [#426](https://github.com/derekross/omarchy-mobile/pull/426): lit on a second Pixel 7; the quick-settings tile not checked) | no | no | no | no | no | no | no (the package's camera start and DMA heap rule are merged, device-panther 0.1.9, [#443](https://github.com/derekross/omarchy-mobile/pull/443), inert until the camera device tree is proposed) | no (s2idle by default since [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134) and device-panther 0.1.1; not run yet) | WIP (the sensor's node, disabled, is merged, [kernel #207](https://github.com/derekross/linux-omarchy-mobile/pull/207); with a bring-up overlay and the package side, which is in review ([#458](https://github.com/derekross/omarchy-mobile/pull/458)), a finger enrolled and unlocked its author's Pixel 7, also after a reboot) | From 1eead1dd38cfd5da58615ede9e4125528181ef0f Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Thu, 12 Nov 2026 05:12:10 -0644 Subject: [PATCH 6/8] SLEEP-PLAN (tegu): 295 of 295 overnight, and the odd RTC readings explained The tracing build ran the reviewed series once more through the night of 11-11 to 11-12: 295 sleeps of 60 s, each woken by the alarm, no UFS line and no stall, the boot's journal agreeing with the log. With the PM switches on that makes 701 woken and none hung. The plan says what that does and doesn't show: at the rate seen without the switches none in 701 would be a one-in-ninety chance, but two no-wakes fix that rate poorly. The RTC differences that didn't fit are old reads. rtc-s5m waits for the PMIC's read update on the S2MPG10 and S2MPG12 and not on the 9a's S2MPG14, so a read can return the time of the read before it, or a mix of the two. bringup/rtc-pause-test.py shows it awake: 5 of 60 reads after a 20 s pause were old on the daily kernel, 2 of 60 on the tracing build and none with the S2MPG14 added to the wait. The kernel change is still to be sent. --- device/tegu/SLEEP-PLAN.md | 80 ++++++++++++++++++++------- device/tegu/bringup/rtc-pause-test.py | 34 ++++++++++++ 2 files changed, 94 insertions(+), 20 deletions(-) create mode 100755 device/tegu/bringup/rtc-pause-test.py diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index efeeaaa4d..23f2fa422 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -1,7 +1,7 @@ # Pixel 9a sleep: why s2idle drains so much -Status (2026-11-12): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The tally on that tree, with #224's two commits: 805 sleeps woken and those two not. It hasn't come back since: 406 sleeps on 11-10 and 11-11 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 of them 60 s sleeps on one boot, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate two or three no-wakes were to be expected in 406, none happens by chance about one time in thirteen, and the switches may change the timing. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 307 wakes and two no-wakes (11-09 and 11-10). The attended-first rule (S1, AGENTS.md) stands; the 11-10 run did not keep it (the run table says how it ran). +Status (2026-11-12): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The tally on that tree, with #224's two commits: 1,100 sleeps woken and those two not. It hasn't come back since: 701 sleeps from 11-10 to 11-12 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 and 295 of them 60 s sleeps on one boot each, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate four or five no-wakes were to be expected in 701 and none would be a one-in-ninety chance, but two no-wakes fix that rate poorly (they fit anything from about one in 1,300 sleeps to one in 40), and the switches may change the timing. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 307 wakes and two no-wakes (11-09 and 11-10). The attended-first rule (S1, AGENTS.md) stands; the 11-10 run did not keep it (the run table says how it ran). S1 is the condensed account: the two causes, the fix, a run table, what is left and what was ruled out. The dated working notes that led there follow it, oldest first. @@ -103,6 +103,7 @@ the 310 sleeps that followed): | the same sources with the function tracer built in (`push1-dbg-gff4fdf58035e`, build #5), `pm_print_times`, `pm_debug_messages` and `kernel.panic_on_rcu_stall=1` on, modules installed, Wi-Fi kept, the hook of #591 installed | whether the no-wake comes back, with a console that would place it | WFI | 96 sleeps entered, no hang: 94 woke on the alarm (a test sleep, 90 with a 10 s alarm, three with a 35 s one); one, the shell's auto-sleep between two cycles with no alarm, lasted 9 h 33 min until the Power key; the 96th was cut by a button reset asked for in error | the first run yes, the owner at the palmtop; he left as the second began, and the long sleep was overnight | | the same build, 11-11, `sleep-series.sh` holding its own sleep inhibitor (first form, before its review) | the inhibitor, and whether the auto-sleep stays out | WFI | **90 of 90** with a 35 s alarm (33.8 to 34.9 s, `pm_wakeup_irq` 53, no UFS line), stopped by hand for the review's changes | no (the owner out) | | the same build, 11-11, the script as reviewed (`PMDEBUG=1 TRACE=1`, 60 s alarm) | the count with the PM switches on | WFI | **313 of 313** (3 in the script's first run, then 310 from 08:43 to 16:20; 58.8 to 60.0 s, `pm_wakeup_irq` 53 each, no UFS line, no stall, `suspend_stats` 313 and 0) | no: nobody at the palmtop for the first cycles; the owner saw it cycling through the day and watched a few cycles at about 11:25 | +| the same build and script, the night of 11-11 to 11-12 (`PMDEBUG=1 TRACE=1`, 60 s alarm) | the count again, overnight | WFI | **295 of 295** (21:35 to 04:50; 58.7 to 60.0 s, `pm_wakeup_irq` 53 each, no UFS line, no stall, `suspend_stats` 295 and 0) | no (the owner asleep, with his yes for the night; the build and the script had both run before) | Before #96, one wake in 15 to 215 lost the storage; before #98, C2 in the sleep lost the palmtop after 6 to 135 cycles. @@ -119,9 +120,9 @@ missing pull, NFC polling (stopped by the hook throughout), the modem **What is left.** -1. The no-wake on the published kernel. On that tree 805 sleeps have woken +1. The no-wake on the published kernel. On that tree 1,100 sleeps have woken (21, 36, 36, two ended by the Power key, 212, the 95 of the evening of - 11-10, and 90 and 313 on 11-11) and two have not: the 22nd of 11-09's first run, with no + 11-10, 90 and 313 on 11-11, and 295 in the night after) and two have not: the 22nd of 11-09's first run, with no console, and the 213th of 11-10's, whose console shows CPU 7 not answering while the rest of the kernel runs (failure 3 above). It hasn't come back since. What is set up for it, in place of the pseudo-NMI build @@ -132,13 +133,19 @@ missing pull, NFC polling (stopped by the hook throughout), the modem through `panic=10` and a console in pstore with no buttons; and, on a build with the function tracer, each core's last calls in the panic's dump (`PMDEBUG=1` and `TRACE=1` in `sleep-series.sh`). With them on, - 406 sleeps have woken on the alarm and none has hung: 93 on the evening - of 11-10 and 313 of 60 s on 11-11 (the last two notes below), which is - the count the maintainer asked for, and #224 merged on it. It doesn't - show the no-wake gone: without the switches that tree had two in about - 310 sleeps, so two or three were to be expected in 406, none happens - by chance about one time in thirteen, and the switches may change the - timing. If it comes back, the same switches say where it stopped; the + 701 sleeps have woken on the alarm and none has hung: 93 on the evening + of 11-10, 313 of 60 s on 11-11 and 295 of 60 s in the night after + (the last notes below). The first 406 are the count the maintainer + asked for, and #224 merged on it. It doesn't show the no-wake gone: + without the switches that tree had two in about 310 sleeps, so four + or five were to be expected in 701 and none would happen by chance + about one time in ninety; but two no-wakes fix that rate poorly + (anything from about one in 1,300 sleeps to one in 40 fits them), and + at the low end 701 clean sleeps are an even chance. And the switches + may change the timing: the next run has the function tracer and + `pm_print_times` off and the stall panic kept, to see whether it + comes back without them. + If it comes back, the same switches say where it stopped; the console of 11-10 and the tracing build are the tools for it. 2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): merged on 2026-11-10 as `omamobile-device-tegu` 0.2.11; its series is the 40 of 40 in the notes @@ -154,17 +161,45 @@ missing pull, NFC polling (stopped by the hook throughout), the modem compared with gs101/gs201 before idle states come to a PR. 5. The DEVICES cell to "works" once item 1's no-wake is found and fixed. #224 (merged 2026-11-12, 00:41Z), the PMIC interrupt (kernel #236) and the hook - are all merged; the 805 wakes above ran on the published tree with + are all merged; the 1,100 wakes above ran on the published tree with #224's commits on top, and a run on the tree as published, with them in it, is still to do. -6. The RTC around a sleep. Three things not looked into: in the 310 - sleeps of 11-11, five cycles read an RTC difference that doesn't fit - the 60 s the kernel's clock has for them (three about 20 s short, one - 7 s and one about 40 s long); on 11-10 a 10 s alarm gave sleeps of 19 - to 20 s, 89 times; and the read tear of the 11-04 note, a difference a - minute off when the read falls just after a minute rolls over. None - cost a wake, but the series' "rtc" column can't be relied on until - they are explained. +6. The RTC around a sleep. The odd readings are explained, and the fix + is a kernel change still to be sent. In the 310 sleeps of 11-11, five + cycles read an RTC difference that doesn't fit the 60 s the kernel's + clock has for them (three 20 or 21 s short, one 7 s and one 39 s + long), and in the 295 of the night after, four (one 21 s short, one + 6 s and two 39 s long). The cause is in `rtc-s5m`: before it reads the time it sets the + PMIC's read-update bit, which copies the counters into the time + registers, and for the S2MPG10 and S2MPG12 it clears the bit, waits, + sets it and waits 1 ms for the copy; the 9a's S2MPG14 was left out of + that, so a read can return the time of the read before it. Awake, 60 + rounds of "wait 20 s, read twice, compare with the system clock" + (`bringup/rtc-pause-test.py`, 2026-11-11): on the daily kernel the first read + was old in five rounds (four by 20 s, one by 10 s), on the tracing + build in two (20 s), and on the same build with the S2MPG14 added to + the wait in none; the second read, a moment later, was right every + time, and 2,500 reads 5 to 90 ms apart showed none off, the read + before being that recent. In a series the old read is either the one + before the alarm is armed or the one 20 s after the wake, which gives + the differences 20 s short and 6 s long. The 39 s ones fit a read + that mixed the two times, the new minute with the old seconds (the + time of the wake plus a minute): in all three the wake was within + 20 s before a minute rolled over, or on it. The differences a minute + off in the night of 11-10, each just after a minute rolled over, fit + the other mix, the old minute with the new seconds, so the read tear + of the 11-04 note is most likely this too. That is a reading of the + numbers, not yet reproduced awake: `rtc-pause-test.py` with a third + argument reads just after the minute, and its one run so far, 35 + reads a second past the minute and a minute apart on the daily + kernel (11-12), returned none old or mixed. `rtcwake` sets its alarm relative + to its own read, so an old read also moves the alarm earlier by its + age, and one older than the interval would put it in the past; no + wake has been lost to that in these series, where the reads are a few + seconds apart. Until the fix is in the kernel the palmtop runs, the + series' "rtc" column is off in about one cycle in sixty and the + kernel log's length is the one to go by. Still not looked into: on + 11-10 a 10 s alarm gave sleeps of 19 to 20 s, 89 times. **Scripts** (`device/tegu/bringup/`): `sleep-series.sh` (on the palmtop: N RTC-woken `systemctl suspend` cycles of S s, records each wake; it holds a @@ -177,7 +212,8 @@ records the next boot's ext4 recovery, restarts the series; it unmasks `sleep.ta `sleep-kernel-run.sh` (laptop: a rewrite of the owner's kit launcher, pushes a built kernel and starts the loop), `sleep-autosleep-series.sh` and `sleep-autosleep-watch.sh` (the everyday path: the shell's auto-sleep does the sleeping, the script only arms the -alarms). +alarms), `rtc-pause-test.py` (on the palmtop, awake and read-only: whether +an RTC read after a pause returns an old time, item 6 above). The dated working notes that led here follow, oldest first; they are the long story and the record of what each day tried. @@ -4903,3 +4939,7 @@ What it doesn't say: what CPU 7 was doing. The backtrace IPI is an ordinary inte What the two days add to the count: with the PM switches on, 406 sleeps woken by the alarm (93 on 11-10, 313 on 11-11) and none hung; without counting toward that, 90 more with a 35 s alarm on 11-11. It leans the right way and no more: at the rate without the switches (two no-wakes in about 310) two or three were to be expected in 406, and none turns up by chance about one time in thirteen, so this can't tell "gone" from "the switches change the timing". The long sleep of 11-10, nine and a half hours on this tree, ended at the first press of the Power key, which the two presses of the morning of 11-10 could not do for the sleep that hung. + +**The night of 2026-11-11 to 11-12: 295 of 295 (21:35 to 04:50 MST).** The tracing build again (`7.3.0-rc6-omamobile-push1-dbg-gff4fdf58035e`, build #5, a fresh boot), the script in the form that ran the 313 (`N=295 S=60 PMDEBUG=1 TRACE=1`), the modem boot unit masked for the boot, on the charger, Wi-Fi kept. The owner said yes to the night before he went to sleep; nothing in it was new. Every sleep lasted 58.7 to 60.0 s by the kernel log and was woken by the alarm (`pm_wakeup_irq` 53), with no UFS line, no ACPM timeout and no oops. The boot's kernel journal agrees with the log: 295 `PM: suspend entry`, 295 `PM: suspend exit`, 295 `PM: Triggering wakeup from IRQ 53` and none from another interrupt, no `link startup failed`, `Host init failed`, `I/O error` or `ufshcd_err_handler` line, no RCU stall, no panic; `suspend_stats` 295 and 0. Four cycles logged an RTC difference that doesn't fit (cycle 48 21 s short, 91 6 s long, 148 and 181 39 s long), the old reads of "What is left" item 6. At the end the "restored:" lines read as before (the switches off, the tracer `nop`, the watchdog back at 10 s). The daily kernel went back at 04:53 and the modem was up and registered at 04:54. + +With it the count with the PM switches on is 701 sleeps woken by the alarm and none hung. Item 1 has what that does and doesn't say. diff --git a/device/tegu/bringup/rtc-pause-test.py b/device/tegu/bringup/rtc-pause-test.py new file mode 100755 index 000000000..beb4682bb --- /dev/null +++ b/device/tegu/bringup/rtc-pause-test.py @@ -0,0 +1,34 @@ +#!/usr/bin/env python3 +# On the palmtop, awake and read-only: does an RTC read after a pause return an old time? +# N rounds of "wait P s, read twice", every third with an alarm armed before the pause +# (rtcwake -m no), each read compared with the system clock. With a third argument A, each +# round also waits on until the RTC is A seconds past a minute, so the first read comes +# just after a minute rolls over (a read that mixes the old and the new time shows there +# as a minute off). +# python3 rtc-pause-test.py [rounds, default 60] [pause s, default 20] [A s past the minute] +import subprocess, sys, time +f = "/sys/class/rtc/rtc0/since_epoch" +N = int(sys.argv[1]) if len(sys.argv) > 1 else 60 +P = float(sys.argv[2]) if len(sys.argv) > 2 else 20 +A = float(sys.argv[3]) if len(sys.argv) > 3 else None + + +def rd(): + a = time.time(); r = int(open(f).read()); b = time.time(); return r - (a + b) / 2 + + +base = sorted(rd() for _ in range(50))[25] +print("%s %s; base offset %.2f s" % (open("/sys/class/rtc/rtc0/name").read().strip(), open("/proc/sys/kernel/osrelease").read().strip(), base), flush=True) +bad = 0 +for i in range(N): + mode = i % 3 + if mode == 1: subprocess.run(["rtcwake", "-m", "no", "-s", "3600"], capture_output=True) + time.sleep(P) + if A is not None: + # the RTC's whole seconds are the system clock plus base, floored + time.sleep((A - (time.time() + base)) % 60) + o1 = rd() - base; o2 = rd() - base + if mode == 1: subprocess.run(["rtcwake", "-m", "disable"], capture_output=True) + if abs(o1) >= 1.5 or abs(o2) >= 1.5: + bad += 1; print("round %d%s: first read after the pause %+.1f s, the next %+.1f s" % (i, " (alarm armed)" if mode == 1 else "", o1, o2), flush=True) +print("rounds %d of %.0f s%s (%d with an alarm armed before the pause), old or odd reads: %d" % (N, P, "" if A is None else ", read %.1f s past the minute" % A, (N + 1) // 3, bad)) From c396e157eaffc72a1b066f85808fe675bdb0424a Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Thu, 12 Nov 2026 07:44:07 -0644 Subject: [PATCH 7/8] device/tegu SLEEP-PLAN, bringup: what the night's run rests on, and an RTC test that says what it writes From the review of the last commit. rtc-pause-test.py is not read-only: every third round it arms a wake alarm and clears it, and the clear takes whatever alarm is armed. It now refuses to start when one is, holds a block sleep inhibitor of its own for the run, and its header and the plan's "Scripts" line say so. Run on the palmtop in this form: four rounds, then a start with an alarm armed, refused. The night's "Attended" cell and note name the two attended runs it rests on, the build's on 11-10 and the script's, the cycles the owner watched at about 11:25 on 11-11, with his yes for the night. The status line names 11-11's runs beside 11-10's and carries the 1,100 in its evidence. Item 6 names linux-omarchy-mobile#245 and the patched build's release, and says when that build booted and who was there. sleep-series.sh keeps every earlier log, under the time of its last line. --- device/tegu/SLEEP-PLAN.md | 26 ++++++++---- device/tegu/bringup/rtc-pause-test.py | 60 +++++++++++++++++---------- device/tegu/bringup/sleep-series.sh | 4 +- 3 files changed, 58 insertions(+), 32 deletions(-) diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index 23f2fa422..1773eaa05 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -1,7 +1,7 @@ # Pixel 9a sleep: why s2idle drains so much -Status (2026-11-12): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The tally on that tree, with #224's two commits: 1,100 sleeps woken and those two not. It hasn't come back since: 701 sleeps from 11-10 to 11-12 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 and 295 of them 60 s sleeps on one boot each, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate four or five no-wakes were to be expected in 701 and none would be a one-in-ninety chance, but two no-wakes fix that rate poorly (they fit anything from about one in 1,300 sleeps to one in 40), and the switches may change the timing. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 307 wakes and two no-wakes (11-09 and 11-10). The attended-first rule (S1, AGENTS.md) stands; the 11-10 run did not keep it (the run table says how it ran). +Status (2026-11-12): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The tally on that tree, with #224's two commits: 1,100 sleeps woken and those two not. It hasn't come back since: 701 sleeps from 11-10 to 11-12 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 and 295 of them 60 s sleeps on one boot each, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate four or five no-wakes were to be expected in 701 and none would be a one-in-ninety chance, but two no-wakes fix that rate poorly (they fit anything from about one in 1,300 sleeps to one in 40), and the switches may change the timing. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 1,100 wakes and two no-wakes (11-09 to 11-12). The attended-first rule (S1, AGENTS.md) stands; the overnight run of 11-10 and the runs of 11-11 did not keep it (the run table and its header say how each ran). S1 is the condensed account: the two causes, the fix, a run table, what is left and what was ruled out. The dated working notes that led there follow it, oldest first. @@ -103,7 +103,7 @@ the 310 sleeps that followed): | the same sources with the function tracer built in (`push1-dbg-gff4fdf58035e`, build #5), `pm_print_times`, `pm_debug_messages` and `kernel.panic_on_rcu_stall=1` on, modules installed, Wi-Fi kept, the hook of #591 installed | whether the no-wake comes back, with a console that would place it | WFI | 96 sleeps entered, no hang: 94 woke on the alarm (a test sleep, 90 with a 10 s alarm, three with a 35 s one); one, the shell's auto-sleep between two cycles with no alarm, lasted 9 h 33 min until the Power key; the 96th was cut by a button reset asked for in error | the first run yes, the owner at the palmtop; he left as the second began, and the long sleep was overnight | | the same build, 11-11, `sleep-series.sh` holding its own sleep inhibitor (first form, before its review) | the inhibitor, and whether the auto-sleep stays out | WFI | **90 of 90** with a 35 s alarm (33.8 to 34.9 s, `pm_wakeup_irq` 53, no UFS line), stopped by hand for the review's changes | no (the owner out) | | the same build, 11-11, the script as reviewed (`PMDEBUG=1 TRACE=1`, 60 s alarm) | the count with the PM switches on | WFI | **313 of 313** (3 in the script's first run, then 310 from 08:43 to 16:20; 58.8 to 60.0 s, `pm_wakeup_irq` 53 each, no UFS line, no stall, `suspend_stats` 313 and 0) | no: nobody at the palmtop for the first cycles; the owner saw it cycling through the day and watched a few cycles at about 11:25 | -| the same build and script, the night of 11-11 to 11-12 (`PMDEBUG=1 TRACE=1`, 60 s alarm) | the count again, overnight | WFI | **295 of 295** (21:35 to 04:50; 58.7 to 60.0 s, `pm_wakeup_irq` 53 each, no UFS line, no stall, `suspend_stats` 295 and 0) | no (the owner asleep, with his yes for the night; the build and the script had both run before) | +| the same build and script, the night of 11-11 to 11-12 (`PMDEBUG=1 TRACE=1`, 60 s alarm) | the count again, overnight | WFI | **295 of 295** (21:35 to 04:50; 58.7 to 60.0 s, `pm_wakeup_irq` 53 each, no UFS line, no stall, `suspend_stats` 295 and 0) | the owner's yes for the night, given before he went to sleep; he slept through it. The attended runs it rests on: the build's on the evening of 11-10, and the script's, in this form, the cycles the owner watched at about 11:25 on 11-11 (the screen coming on at the lock screen, going dark for about a minute and coming on again), in a run that had begun with nobody there | Before #96, one wake in 15 to 215 lost the storage; before #98, C2 in the sleep lost the palmtop after 6 to 135 cycles. @@ -165,7 +165,7 @@ missing pull, NFC polling (stopped by the hook throughout), the modem #224's commits on top, and a run on the tree as published, with them in it, is still to do. 6. The RTC around a sleep. The odd readings are explained, and the fix - is a kernel change still to be sent. In the 310 sleeps of 11-11, five + is [linux-omarchy-mobile#245](https://github.com/derekross/linux-omarchy-mobile/pull/245). In the 310 sleeps of 11-11, five cycles read an RTC difference that doesn't fit the 60 s the kernel's clock has for them (three 20 or 21 s short, one 7 s and one 39 s long), and in the 295 of the night after, four (one 21 s short, one @@ -178,9 +178,14 @@ missing pull, NFC polling (stopped by the hook throughout), the modem (`bringup/rtc-pause-test.py`, 2026-11-11): on the daily kernel the first read was old in five rounds (four by 20 s, one by 10 s), on the tracing build in two (20 s), and on the same build with the S2MPG14 added to - the wait in none; the second read, a moment later, was right every + the wait (`7.3.0-rc6-omamobile-push1-dbg-gc63fa9271bb8`, build #6) in + none; the second read, a moment later, was right every time, and 2,500 reads 5 to 90 ms apart showed none off, the read - before being that recent. In a series the old read is either the one + before being that recent. The patched build was new on the palmtop: + it booted at about 20:50 MST on 11-11 for this test alone, held awake + by a block inhibitor, entered no sleep, and was + replaced by the tracing build before the night's series; the owner was + at home and not at the palmtop. In a series the old read is either the one before the alarm is armed or the one 20 s after the wake, which gives the differences 20 s short and 6 s long. The 39 s ones fit a read that mixed the two times, the new minute with the old seconds (the @@ -196,7 +201,7 @@ missing pull, NFC polling (stopped by the hook throughout), the modem to its own read, so an old read also moves the alarm earlier by its age, and one older than the interval would put it in the past; no wake has been lost to that in these series, where the reads are a few - seconds apart. Until the fix is in the kernel the palmtop runs, the + seconds apart. Until #245 is in the kernel the palmtop runs, the series' "rtc" column is off in about one cycle in sixty and the kernel log's length is the one to go by. Still not looked into: on 11-10 a 10 s alarm gave sleeps of 19 to 20 s, 89 times. @@ -212,8 +217,11 @@ records the next boot's ext4 recovery, restarts the series; it unmasks `sleep.ta `sleep-kernel-run.sh` (laptop: a rewrite of the owner's kit launcher, pushes a built kernel and starts the loop), `sleep-autosleep-series.sh` and `sleep-autosleep-watch.sh` (the everyday path: the shell's auto-sleep does the sleeping, the script only arms the -alarms), `rtc-pause-test.py` (on the palmtop, awake and read-only: whether -an RTC read after a pause returns an old time, item 6 above). +alarms), `rtc-pause-test.py` (on the palmtop, awake: whether an RTC read after +a pause returns an old time, item 6 above. It is not read-only: every +third round it arms a wake alarm an hour ahead and clears it again, so it +refuses to start when an alarm is armed, holds a sleep inhibitor of its +own for the run, and must never run beside a series). The dated working notes that led here follow, oldest first; they are the long story and the record of what each day tried. @@ -4940,6 +4948,6 @@ What the two days add to the count: with the PM switches on, 406 sleeps woken by The long sleep of 11-10, nine and a half hours on this tree, ended at the first press of the Power key, which the two presses of the morning of 11-10 could not do for the sleep that hung. -**The night of 2026-11-11 to 11-12: 295 of 295 (21:35 to 04:50 MST).** The tracing build again (`7.3.0-rc6-omamobile-push1-dbg-gff4fdf58035e`, build #5, a fresh boot), the script in the form that ran the 313 (`N=295 S=60 PMDEBUG=1 TRACE=1`), the modem boot unit masked for the boot, on the charger, Wi-Fi kept. The owner said yes to the night before he went to sleep; nothing in it was new. Every sleep lasted 58.7 to 60.0 s by the kernel log and was woken by the alarm (`pm_wakeup_irq` 53), with no UFS line, no ACPM timeout and no oops. The boot's kernel journal agrees with the log: 295 `PM: suspend entry`, 295 `PM: suspend exit`, 295 `PM: Triggering wakeup from IRQ 53` and none from another interrupt, no `link startup failed`, `Host init failed`, `I/O error` or `ufshcd_err_handler` line, no RCU stall, no panic; `suspend_stats` 295 and 0. Four cycles logged an RTC difference that doesn't fit (cycle 48 21 s short, 91 6 s long, 148 and 181 39 s long), the old reads of "What is left" item 6. At the end the "restored:" lines read as before (the switches off, the tracer `nop`, the watchdog back at 10 s). The daily kernel went back at 04:53 and the modem was up and registered at 04:54. +**The night of 2026-11-11 to 11-12: 295 of 295 (21:35 to 04:50 MST).** The tracing build again (`7.3.0-rc6-omamobile-push1-dbg-gff4fdf58035e`, build #5, a fresh boot), the script in the form that ran the 313 (`N=295 S=60 PMDEBUG=1 TRACE=1`), the modem boot unit masked for the boot, on the charger, Wi-Fi kept. The owner said yes to the night before he went to sleep, and slept through it. Neither part was new, and each had had its attended run: the build on the evening of 11-10, the owner at the palmtop for the first run; the script in this form at about 11:25 on 11-11, when the owner watched a few of its cycles (the screen coming on at the lock screen, going dark for about a minute and coming on again), in the run of 313 that had begun with nobody there. Every sleep lasted 58.7 to 60.0 s by the kernel log and was woken by the alarm (`pm_wakeup_irq` 53), with no UFS line, no ACPM timeout and no oops. The boot's kernel journal agrees with the log: 295 `PM: suspend entry`, 295 `PM: suspend exit`, 295 `PM: Triggering wakeup from IRQ 53` and none from another interrupt, no `link startup failed`, `Host init failed`, `I/O error` or `ufshcd_err_handler` line, no RCU stall, no panic; `suspend_stats` 295 and 0. Four cycles logged an RTC difference that doesn't fit (cycle 48 21 s short, 91 6 s long, 148 and 181 39 s long), the old reads of "What is left" item 6. At the end the "restored:" lines read as before (the switches off, the tracer `nop`, the watchdog back at 10 s). The daily kernel went back at 04:53 and the modem was up and registered at 04:54. With it the count with the PM switches on is 701 sleeps woken by the alarm and none hung. Item 1 has what that does and doesn't say. diff --git a/device/tegu/bringup/rtc-pause-test.py b/device/tegu/bringup/rtc-pause-test.py index beb4682bb..e39673d38 100755 --- a/device/tegu/bringup/rtc-pause-test.py +++ b/device/tegu/bringup/rtc-pause-test.py @@ -1,10 +1,15 @@ #!/usr/bin/env python3 -# On the palmtop, awake and read-only: does an RTC read after a pause return an old time? -# N rounds of "wait P s, read twice", every third with an alarm armed before the pause -# (rtcwake -m no), each read compared with the system clock. With a third argument A, each -# round also waits on until the RTC is A seconds past a minute, so the first read comes -# just after a minute rolls over (a read that mixes the old and the new time shows there -# as a minute off). +# On the palmtop, awake: does an RTC read after a pause return an old time? +# N rounds of "wait P s, read twice", each read compared with the system clock. With a third +# argument A, each round also waits on until the RTC is A seconds past a minute, so the first +# read comes just after a minute rolls over (a read that mixes the old and the new time shows +# there as a minute off). +# It is not read-only: every third round arms a wake alarm an hour ahead before the pause +# (rtcwake -m no -s 3600) and clears it after the reads (rtcwake -m disable), which clears +# whatever alarm is armed at that moment. So it refuses to start when an alarm is armed, and it +# holds a block sleep inhibitor of its own from start to end, so that nothing sleeps under it +# with the alarm gone. Never run it beside a sleep series: the series arms the alarm its +# sleeps end on. # python3 rtc-pause-test.py [rounds, default 60] [pause s, default 20] [A s past the minute] import subprocess, sys, time f = "/sys/class/rtc/rtc0/since_epoch" @@ -12,23 +17,36 @@ P = float(sys.argv[2]) if len(sys.argv) > 2 else 20 A = float(sys.argv[3]) if len(sys.argv) > 3 else None +if open("/sys/class/rtc/rtc0/wakealarm").read().strip(): + sys.exit("a wake alarm is armed (/sys/class/rtc/rtc0/wakealarm isn't empty): something is counting on it, stop") +hold = subprocess.Popen(["systemd-inhibit", "--what=sleep", "--who=rtc-pause-test", "--why=arms and clears the wake alarm", + "--mode=block", "sleep", "infinity"]) +time.sleep(1) +if hold.poll() is not None: + sys.exit("could not take a sleep inhibitor: stop") + def rd(): a = time.time(); r = int(open(f).read()); b = time.time(); return r - (a + b) / 2 -base = sorted(rd() for _ in range(50))[25] -print("%s %s; base offset %.2f s" % (open("/sys/class/rtc/rtc0/name").read().strip(), open("/proc/sys/kernel/osrelease").read().strip(), base), flush=True) -bad = 0 -for i in range(N): - mode = i % 3 - if mode == 1: subprocess.run(["rtcwake", "-m", "no", "-s", "3600"], capture_output=True) - time.sleep(P) - if A is not None: - # the RTC's whole seconds are the system clock plus base, floored - time.sleep((A - (time.time() + base)) % 60) - o1 = rd() - base; o2 = rd() - base - if mode == 1: subprocess.run(["rtcwake", "-m", "disable"], capture_output=True) - if abs(o1) >= 1.5 or abs(o2) >= 1.5: - bad += 1; print("round %d%s: first read after the pause %+.1f s, the next %+.1f s" % (i, " (alarm armed)" if mode == 1 else "", o1, o2), flush=True) -print("rounds %d of %.0f s%s (%d with an alarm armed before the pause), old or odd reads: %d" % (N, P, "" if A is None else ", read %.1f s past the minute" % A, (N + 1) // 3, bad)) +try: + base = sorted(rd() for _ in range(50))[25] + print("%s %s; base offset %.2f s" % (open("/sys/class/rtc/rtc0/name").read().strip(), open("/proc/sys/kernel/osrelease").read().strip(), base), flush=True) + bad = 0 + for i in range(N): + mode = i % 3 + if mode == 1: subprocess.run(["rtcwake", "-m", "no", "-s", "3600"], capture_output=True) + try: + time.sleep(P) + if A is not None: + # the RTC's whole seconds are the system clock plus base, floored + time.sleep((A - (time.time() + base)) % 60) + o1 = rd() - base; o2 = rd() - base + finally: + if mode == 1: subprocess.run(["rtcwake", "-m", "disable"], capture_output=True) + if abs(o1) >= 1.5 or abs(o2) >= 1.5: + bad += 1; print("round %d%s: first read after the pause %+.1f s, the next %+.1f s" % (i, " (alarm armed)" if mode == 1 else "", o1, o2), flush=True) + print("rounds %d of %.0f s%s (%d with an alarm armed before the pause), old or odd reads: %d" % (N, P, "" if A is None else ", read %.1f s past the minute" % A, (N + 1) // 3, bad)) +finally: + hold.terminate() diff --git a/device/tegu/bringup/sleep-series.sh b/device/tegu/bringup/sleep-series.sh index a8ec0205f..d7a61c2d1 100644 --- a/device/tegu/bringup/sleep-series.sh +++ b/device/tegu/bringup/sleep-series.sh @@ -18,8 +18,8 @@ modem_up && { echo "the modem is up: stop" >> $L; exit 1; } # With the debug switches a stall is a panic, and a panic has to end in a reboot. [ $DBG = 1 ] && [ "$(cat /proc/sys/kernel/panic)" = 0 ] && { echo "kernel.panic is 0, a panic would wait for the buttons: stop" >> $L; exit 1; } # Only a run that starts gets a fresh log: a refusal above is appended to the log that is there, and the -# previous run's log is kept beside the new one. -[ -s $L ] && mv -f $L $L.prev +# previous run's log is kept beside the new one, under the time of its last line. +[ -s $L ] && mv -f $L $L.$(date -r $L +%Y%m%d-%H%M%S) : > $L uid=$(id -u mobile) hy() { runuser -u mobile -- env XDG_RUNTIME_DIR=/run/user/$uid HYPRLAND_INSTANCE_SIGNATURE=$(ls /run/user/$uid/hypr | head -1) hyprctl dispatch "hl.dsp.dpms({ action = \"$1\" })" >/dev/null 2>&1; } From 1a2b1a3e93e67d81d9159776baf523ee2021478a Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Fri, 13 Nov 2026 09:44:32 -0644 Subject: [PATCH 8/8] device/tegu SLEEP-PLAN, DEVICES, bringup: the no-wake came back on 11-12, and the RTC test's licence line The status line, "What is left" item 1 and the DEVICES cell still said the no-wake hadn't come back and called the run without the tracer the next one. It ran on 2026-11-12 and the no-wake came back twice; the console places CPU 7 in the noirq resume of an HSI0 USI, and linux-omarchy-mobile#247 proposes keeping those clocks on. One sentence in each place says so; the record follows in its own change. The DEVICES cell said 406 sleeps with the debug switches where the plan says 701. rtc-pause-test.py gets its SPDX line, and its header says that a dropped SSH session ends it without its cleanup, so an alarm can stay armed. --- device/tegu/SLEEP-PLAN.md | 15 +++++++-------- device/tegu/bringup/rtc-pause-test.py | 4 +++- docs/DEVICES.md | 2 +- 3 files changed, 11 insertions(+), 10 deletions(-) diff --git a/device/tegu/SLEEP-PLAN.md b/device/tegu/SLEEP-PLAN.md index 1773eaa05..b03f250ad 100644 --- a/device/tegu/SLEEP-PLAN.md +++ b/device/tegu/SLEEP-PLAN.md @@ -1,7 +1,7 @@ # Pixel 9a sleep: why s2idle drains so much -Status (2026-11-12): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing isn't known. The tally on that tree, with #224's two commits: 1,100 sleeps woken and those two not. It hasn't come back since: 701 sleeps from 11-10 to 11-12 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 and 295 of them 60 s sleeps on one boot each, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate four or five no-wakes were to be expected in 701 and none would be a one-in-ninety chance, but two no-wakes fix that rate poorly (they fit anything from about one in 1,300 sleeps to one in 40), and the switches may change the timing. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 1,100 wakes and two no-wakes (11-09 to 11-12). The attended-first rule (S1, AGENTS.md) stands; the overnight run of 11-10 and the runs of 11-11 did not keep it (the run table and its header say how each ran). +Status (2026-11-13): Two causes are known and fixed, and a third failure is open. Dead storage after a wake: the UFS pads came back as plain inputs; [linux-omarchy-mobile#224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged on 2026-11-12 (00:41Z), re-applies the pin state at resume. A silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each; [omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it. Open: on the published kernel, which has no idle states, a sleep has failed to wake twice: one in 22 on 2026-11-09 (a boot without modules, no console) and one in 213 overnight on 2026-11-10 on the newest tip with the fixes. That one left a console: the kernel stayed alive, and CPU 7 answered nothing from just after the device suspend until the owner's reset (an RCU stall on CPU 7 every 63 s; neither the alarm nor the Power key got through); what that core was doing wasn't known then. The tally on that tree, with #224's two commits: 1,100 sleeps woken and those two not. With the PM debug switches on it didn't come back: 701 sleeps from 11-10 to 11-12 with the PM debug switches on (`pm_print_times`, `pm_debug_messages`, a stall made a panic, the function tracer), 313 and 295 of them 60 s sleeps on one boot each, all woke on the alarm with no UFS line. That is not shown to be a fix: at the earlier rate four or five no-wakes were to be expected in 701 and none would be a one-in-ninety chance, but two no-wakes fix that rate poorly (they fit anything from about one in 1,300 sleeps to one in 40), and the switches may change the timing. With the function tracer and `pm_print_times` off it came back on 2026-11-12, twice; the console places CPU 7 in the noirq resume of an HSI0 USI, and [linux-omarchy-mobile#247](https://github.com/derekross/linux-omarchy-mobile/pull/247) proposes keeping those clocks on; the record follows. The main PMIC's interrupt, and with it the RTC alarm, came to the published `zumapro-tegu.dts` with [linux-omarchy-mobile#236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10; a kernel from before it has no alarm (`rtcwake` fails and a sleep ends only on a key), so check that `/sys/class/rtc/rtc0/wakealarm` exists before any RTC-woken sleep. Evidence: 300 of 300 RTC-woken 60 s sleeps twice (#96 WFI-only, #98 C2 at runtime, 2026-11-07/08), 15 of 15 wakes on the everyday path, the shell's auto-sleep behind the lock, twelve of them half-hour sleeps (#99, 2026-11-08), 12 of 12 half-hour auto-sleeps on #100 (11-09), and on the published tree with #224 and the PMIC interrupt 1,100 wakes and two no-wakes (11-09 to 11-12). The attended-first rule (S1, AGENTS.md) stands; the overnight run of 11-10 and the runs of 11-11 did not keep it (the run table and its header say how each ran). S1 is the condensed account: the two causes, the fix, a run table, what is left and what was ruled out. The dated working notes that led there follow it, oldest first. @@ -124,8 +124,8 @@ missing pull, NFC polling (stopped by the hook throughout), the modem (21, 36, 36, two ended by the Power key, 212, the 95 of the evening of 11-10, 90 and 313 on 11-11, and 295 in the night after) and two have not: the 22nd of 11-09's first run, with no console, and the 213th of 11-10's, whose console shows CPU 7 not - answering while the rest of the kernel runs (failure 3 above). It hasn't - come back since. What is set up for it, in place of the pseudo-NMI build + answering while the rest of the kernel runs (failure 3 above). With the + switches below on it didn't come back. What was set up for it, in place of the pseudo-NMI build planned here before (the maintainer's pointers on kernel #224): `pm_print_times` and `pm_debug_messages`, so the console names each device's suspend callback as it starts and returns; @@ -142,11 +142,10 @@ missing pull, NFC polling (stopped by the hook throughout), the modem about one time in ninety; but two no-wakes fix that rate poorly (anything from about one in 1,300 sleeps to one in 40 fits them), and at the low end 701 clean sleeps are an even chance. And the switches - may change the timing: the next run has the function tracer and - `pm_print_times` off and the stall panic kept, to see whether it - comes back without them. - If it comes back, the same switches say where it stopped; the - console of 11-10 and the tracing build are the tools for it. + may change the timing. With the function tracer and `pm_print_times` + off it came back on 2026-11-12, twice; the console places CPU 7 in the + noirq resume of an HSI0 USI, and [linux-omarchy-mobile#247](https://github.com/derekross/linux-omarchy-mobile/pull/247) + proposes keeping those clocks on; the record follows. 2. The sleep hook ([omarchy-mobile#591](https://github.com/derekross/omarchy-mobile/pull/591)): merged on 2026-11-10 as `omamobile-device-tegu` 0.2.11; its series is the 40 of 40 in the notes below. diff --git a/device/tegu/bringup/rtc-pause-test.py b/device/tegu/bringup/rtc-pause-test.py index e39673d38..e49c680a6 100755 --- a/device/tegu/bringup/rtc-pause-test.py +++ b/device/tegu/bringup/rtc-pause-test.py @@ -1,4 +1,5 @@ #!/usr/bin/env python3 +# SPDX-License-Identifier: MIT # On the palmtop, awake: does an RTC read after a pause return an old time? # N rounds of "wait P s, read twice", each read compared with the system clock. With a third # argument A, each round also waits on until the RTC is A seconds past a minute, so the first @@ -9,7 +10,8 @@ # whatever alarm is armed at that moment. So it refuses to start when an alarm is armed, and it # holds a block sleep inhibitor of its own from start to end, so that nothing sleeps under it # with the alarm gone. Never run it beside a sleep series: the series arms the alarm its -# sleeps end on. +# sleeps end on. A dropped SSH session ends it without its cleanup, so an alarm up to an hour +# ahead can stay armed; that only wakes the palmtop. # python3 rtc-pause-test.py [rounds, default 60] [pause s, default 20] [A s past the minute] import subprocess, sys, time f = "/sys/class/rtc/rtc0/since_epoch" diff --git a/docs/DEVICES.md b/docs/DEVICES.md index 0763ead84..fe6caa7c0 100644 --- a/docs/DEVICES.md +++ b/docs/DEVICES.md @@ -25,7 +25,7 @@ The status of each device, as verified on real hardware. Rows in the matrix have | Pixel 6 | oriole | gs101 (Tensor G1) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); native test image on one phone; slot B marked successful and `vbmeta_b` verification turned off, both without project approval; see notes) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); guarded `userdata` root on test image) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); connected output on test image) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); Panfrost render node present) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); touchscreen detected; input not checked by smoke) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); gpio-keys detected; input not checked by smoke) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); USB network and SSH on test image) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); link up; traffic not checked by smoke) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); controller powered; pairing not checked) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); charge and temperature reported) | WIP ([unpublished kernel/packages](../device/oriole/NOTES.md#2026-10-31-build-provenance-and-recovery-limits); charging reported; control not checked) | no | no | no | no | no | no | no | no | no | no | no | | Pixel 4 | flame | SM8150 (Snapdragon 855) | testing (temporary RAM boot; [notes](../device/flame/BRINGUP-M1.md)) | testing (RAM probe; all UFS read-only) | testing (1080x2280 console; software desktop in separate Arch payload) | testing (separate Arch payload; FD640 contexts, llvmpipe desktop) | testing (separate Arch payload; events and PIN unlock verified) | no | testing (NCM + ACM + SSH; sustained transfers can disconnect) | testing (guarded RAM scan; association untested) | no | no | no | no | no | no | - | no | no | no | no | no | no | - | | Pixel 4 XL ([notes](../device/coral/NOTES.md)) | coral | SM8150 (Snapdragon 855) | WIP (direct ABL v2 boot on the test handset's slot A; slot-B install and §3.1 entry pending; [notes](../device/coral/BRINGUP-M1.md)) | testing (read-only `blockdev` guard + RAM `overlayfs`) | testing (1440x3040@60Hz S6E3HC2 DSC 1.1 KMS, DPMS, brightness) | testing (Adreno 640 / FD640 Mesa Freedreno, Hyprland) | testing (FTS SPI multitouch + read-only flash guard) | testing (power via `qcom-pon` + volume via `gpio-keys`) | testing (CDC ECM + ACM + SSH; stays up with host USB switched off) | testing (`WCN3990` `ath10k_snoc` + `rmtfs -r`, WPA2/DHCP/HTTPS; radio sessions ran before the modem baseline) | WIP (`WCN3990` `hci_qca` `hci0` up + BLE scan; audio/pairing open) | WIP (PM8150B ADC `VBAT_SNS` voltage + FG temp; aftermarket pack on test unit: no reply from `1-0036`) | testing (PM8150B SMB5 charger with `4.20 V` / `500 mA` kernel clamp) | testing (`CS40L25A` I2C click waveform) | testing (PM8150L flash LEDs + quick-settings tile) | no | - | testing (QDSP6 + dual `CS35L36` amps capped at `-26 dB` + `IA8508` SPI mic DSP) | testing (`SLPI` + `fastrpc` + RAM-backed `hexagonrpcd` + QMI SSC: 7 sensors + Soli radar) | WIP (`MPSS` boots with `rmtfs -r` + patched `tqftpserv`, SIM detected; no IPA/voice yet; radio sessions ran before the modem baseline) | no | testing (`CAMSS` + `IMX363`, `IMX481`, `IMX520`, dual `OV7251` NIR; `libcamera` softISP) | no | - | -| Pixel 9a ([notes](../device/tegu/NOTES.md)) | tegu | zumapro (Tensor G4) | works ([#28](https://github.com/derekross/omarchy-mobile/issues/28): native boot from slot B, the Arch root on `userdata`, `omamobile-device-tegu`; slot B was marked successful by hand on 2026-10-28 with the `devinfo` write approved on 2026-10-22, now on hold ([#125](https://github.com/derekross/omarchy-mobile/pull/125)), unmarked by the 10-31 fastboot recovery and marked again on 2026-10-31 during the hold ([record](../device/tegu/NOTES.md#2026-10-31-a-fastboot-flash-of-slot-b-cleared-its-successful-mark-and-b-was-marked-again)), and the package does not mark it; unpublished kernel: the SoC and board device tree is [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (UFS behind the read-only guard; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (120 Hz through gs-decon; the panel driver [kernel #89](https://github.com/derekross/linux-omarchy-mobile/pull/89), 120 Hz [kernel #88](https://github.com/derekross/linux-omarchy-mobile/pull/88), merged, the display DT, merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96); brightness, DPMS off and on and the 60/120 Hz switch on the boot of [kernel #125](https://github.com/derekross/linux-omarchy-mobile/pull/125), merged, which also checks the panel's ID; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on), [brightness record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (Panthor, the desktop renders, runtime suspend when idle; this boot runs at a fixed 160 MHz; the GPU's levels and its thermal zone are merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96) ([kernel #85](https://github.com/derekross/linux-omarchy-mobile/pull/85) and [kernel #87](https://github.com/derekross/linux-omarchy-mobile/pull/87) landed through it), not yet run from it; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (Synaptics S3908 over SPI, [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; survives sleep, [kernel #82](https://github.com/derekross/linux-omarchy-mobile/pull/82), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (gpio-keys: power and volume; the power key wakes from sleep; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (CDC NCM and SSH; reconnects after a replug, `omamobile-tegu-usb-rescue`, also after a replug the gadget didn't see ([#280](https://github.com/derekross/omarchy-mobile/pull/280); [the replug record](../device/tegu/NOTES.md#2026-10-28-what-the-device-tables-claims-rest-on)); [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (BCM4383 on PCIe channel 1 through the shared gs201 host and zuma PHY, [kernel #178](https://github.com/derekross/linux-omarchy-mobile/pull/178) on [kernel #156](https://github.com/derekross/linux-omarchy-mobile/pull/156), in review, and brcmfmac, [kernel #56](https://github.com/derekross/linux-omarchy-mobile/pull/56) in review; started at boot by `omamobile-device-tegu` 0.2.4 ([#238](https://github.com/derekross/omarchy-mobile/pull/238)) with the extractor's firmware; WPA2 with iwd, kept loaded across sleep by the package's sleep hook; [record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (hci_uart on the bootloader's address, BlueZ, A2DP; [kernel #94](https://github.com/derekross/linux-omarchy-mobile/pull/94); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (MAX77779 gauge with temperature, current and time estimates through UPower; [kernel #66](https://github.com/derekross/linux-omarchy-mobile/pull/66); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (the MAX77779 charger binds from the device tree, [kernel #161](https://github.com/derekross/linux-omarchy-mobile/pull/161), merged, built by [#227](https://github.com/derekross/omarchy-mobile/pull/227); the charge limit in `omamobile-device-tegu` 0.2.3 holds charging off at the limit through `charge_behaviour` and lets it run again 5 below; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (CS40L26A through feedbackd, with brownout prevention, [kernel #95](https://github.com/derekross/linux-omarchy-mobile/pull/95); its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (LM3644, 25 steps to 75 mA, quick-settings tile; its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | testing (Omarchy's Location switch serves fixes from GPS, Galileo, BeiDou and GLONASS through the modem's Kepler once the modem is up; `omamobile-device-tegu`'s boot unit, opt-in ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), starts gnssd with the modem: on 2026-11-03, after a reboot with nothing started by hand, a session through the switch's socket had 133 of 135 GGA sentences fixed, first fix 48 s after the client connected, 18 satellites; on 2026-11-09 the same from the in-tree drivers (the GNSS link driver, [kernel #185](https://github.com/derekross/linux-omarchy-mobile/pull/185), and the 9a's GNSS nodes, [kernel #237](https://github.com/derekross/linux-omarchy-mobile/pull/237), the `modem-cpif` overlay with the driver's config line, [#594](https://github.com/derekross/omarchy-mobile/pull/594), and the PCIe PHY's config line, [#274](https://github.com/derekross/omarchy-mobile/pull/274), all merged): four 150 s sessions, 497 of 507 GGA sentences fixed, up to 24 satellites; a 30-minute session on 2026-11-01 (omamobile-gnss 0.2.5's start, by hand, the modem link left to cycle) had 1763 of 1765, first fix 37 s, and a session in a moving car from [#114](https://github.com/derekross/omarchy-mobile/pull/114)'s package build fixed 145 of 153 times; needs [kernel #150](https://github.com/derekross/linux-omarchy-mobile/pull/150) and the PERIC0 clock fix [kernel #172](https://github.com/derekross/linux-omarchy-mobile/pull/172), both merged (without the latter the modem link hung the palmtop within minutes); not on by default because the modem isn't (the Modem cell); [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/TELEPHONY-PLAN.md), record with [#202](https://github.com/derekross/omarchy-mobile/pull/202)) | works (audio and sensor DSP through the GSA and Trusty, 93 services, started at boot by `omamobile-device-tegu` 0.2.3 from the device tree, [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162) in review; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (the TAS2572 amplifiers and the microphone through the AoC card and PipeWire, at boot with the AoC: a tone from the speaker, the microphone at -32 dBFS; speakers at a -10 dB ceiling; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (accelerometer, light, proximity, compass and gyroscope through the AoC and the sensor proxy, at boot with the AoC; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | testing (calls in and out on the speakerphone, texts both ways, mobile data over the device `omamobile-sit-data` finds, with the Phone and Messages apps on the bridge ([#356](https://github.com/derekross/omarchy-mobile/pull/356), [#364](https://github.com/derekross/omarchy-mobile/pull/364), [#367](https://github.com/derekross/omarchy-mobile/pull/367), [#371](https://github.com/derekross/omarchy-mobile/pull/371)); `omamobile-device-tegu`'s boot unit starts the modem, the phone service and the bridge at boot when the owner enables it ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), and is skipped on a phone that isn't provisioned or has no CP drivers; opt-in, not on by default, while marking slot B on the 9a is on hold ([#483](https://github.com/derekross/omarchy-mobile/pull/483)): with B unmarked a reset spends one of ABL's three tries, and the unit can't tell; it ran by itself on the owner's palmtop over a day of reboots on 2026-11-09 with 42 of 43 stops confirmed, one unexplained ([record](../device/tegu/TELEPHONY-PLAN.md#29-the-in-tree-drivers-through-the-starter-on-the-published-tip-plus-four-open-changes-2026-11-09)), and the NV partitions unchanged, on the owner's kernel with the bring-up modules, and on the in-tree drivers ([kernel #187](https://github.com/derekross/linux-omarchy-mobile/pull/187) and [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162), merged), which need the `modem-cpif` overlay and the GNSS link driver's config line ([#594](https://github.com/derekross/omarchy-mobile/pull/594)) and the PCIe PHY's config line ([#274](https://github.com/derekross/omarchy-mobile/pull/274)), all merged; the far end still hears an echo on the speakerphone ([#378](https://github.com/derekross/omarchy-mobile/pull/378)) and there's no earpiece route yet; [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/MODEM-PLAN.md#sim-test-results-2026-11-02-730-rc6-omamobile-tegu-s2mpg15-m1-g991042e2eff3-dirty)) | works (NXP ST54L through nxp-nci, its clock pad from the PMU ([kernel #104](https://github.com/derekross/linux-omarchy-mobile/pull/104) and [kernel #105](https://github.com/derekross/linux-omarchy-mobile/pull/105), merged, with [kernel #131](https://github.com/derekross/linux-omarchy-mobile/pull/131), merged), loaded at boot by `omamobile-device-tegu` 0.2.3 with neard and the link opener: polls with no I2C errors and read a payment card; polling stops before sleep; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (main, ultrawide and front from the device tree, [kernel #164](https://github.com/derekross/linux-omarchy-mobile/pull/164) in review, with [kernel #63](https://github.com/derekross/linux-omarchy-mobile/pull/63) and [kernel #103](https://github.com/derekross/linux-omarchy-mobile/pull/103) in review: their drivers load at boot with nothing by hand, libcamera lists all three, and each streams, main at 19 fps and the others at 30 fps at 1280x960; basic picture quality through libcamera's software ISP; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel); the main camera's contrast autofocus through its LC898129 ([kernel #213](https://github.com/derekross/linux-omarchy-mobile/pull/213) in review, and a lens node on top of kernel #164, not yet in a pull request; libcamera 0.7.2-4.23), close subjects checked, the far end not yet ([notes](../device/tegu/NOTES.md#2026-11-07-the-main-cameras-autofocus-lc898129))) | testing (s2idle when asked for, or through the shell's auto-sleep with the `omamobile-sleep` toggle on, which ships off; the power key lights the lock screen on the first press ([#268](https://github.com/derekross/omarchy-mobile/pull/268), [#275](https://github.com/derekross/omarchy-mobile/pull/275)), and the RTC wakes it; on 2026-11-01 thirty attended 120 s sleeps ([plan](../device/tegu/SLEEP-PLAN.md) S1) and a day of real use with auto-sleep on ([record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) all woke, but six test sleeps since 10-28 did not (one in 21 on the clock-fix kernel on 11-01, and one in 127 in an overnight series of 120 s sleeps on 11-01 to 11-02), so sleep tests stay attended ([plan](../device/tegu/SLEEP-PLAN.md) S1); two causes are known: dead storage after a wake (the UFS pads came back as plain inputs on about one wake in a hundred; [kernel #224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged 2026-11-12, re-applies the pin state at resume), and a silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each, then 15 of 15 on the everyday path, twelve of them half-hour auto-sleeps (2026-11-07 and 11-08, [#409](https://github.com/derekross/omarchy-mobile/pull/409)); [#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it; open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules), none in two re-runs of 36, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes, where the console shows the kernel alive and CPU 7 not answering from just after the device suspend ([plan](../device/tegu/SLEEP-PLAN.md) S1, failure 3), and it hasn't come back in 406 sleeps with the PM debug switches on (2026-11-10 and 11-11, 313 of them 60 s sleeps on one boot), which leans the right way but doesn't show it gone; the RTC alarm on the published kernel came with [kernel #236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10 (a kernel from before it has none, and a sleep there ends only on a key); works once that no-wake is found and fixed and a series has run from the published kernel as it is, with kernel #224 in it; the drain is about 300 mA asleep, the same as awake with the screen off, because nothing below the CPUs sleeps yet; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (opt-in) (enrolls, matches, refuses other fingers and unlocks the lock screen through Trusty and `omamobile-fingerprint`, 2026-10-29; the sensor binds from the device tree, [kernel #165](https://github.com/derekross/linux-omarchy-mobile/pull/165) in review, with [kernel #126](https://github.com/derekross/linux-omarchy-mobile/pull/126), merged; opt-in as on the 7 Pro, because the service's first run writes Trusty's secure storage; [plan](../device/tegu/FINGERPRINT-PLAN.md), [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | +| Pixel 9a ([notes](../device/tegu/NOTES.md)) | tegu | zumapro (Tensor G4) | works ([#28](https://github.com/derekross/omarchy-mobile/issues/28): native boot from slot B, the Arch root on `userdata`, `omamobile-device-tegu`; slot B was marked successful by hand on 2026-10-28 with the `devinfo` write approved on 2026-10-22, now on hold ([#125](https://github.com/derekross/omarchy-mobile/pull/125)), unmarked by the 10-31 fastboot recovery and marked again on 2026-10-31 during the hold ([record](../device/tegu/NOTES.md#2026-10-31-a-fastboot-flash-of-slot-b-cleared-its-successful-mark-and-b-was-marked-again)), and the package does not mark it; unpublished kernel: the SoC and board device tree is [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (UFS behind the read-only guard; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (120 Hz through gs-decon; the panel driver [kernel #89](https://github.com/derekross/linux-omarchy-mobile/pull/89), 120 Hz [kernel #88](https://github.com/derekross/linux-omarchy-mobile/pull/88), merged, the display DT, merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96); brightness, DPMS off and on and the 60/120 Hz switch on the boot of [kernel #125](https://github.com/derekross/linux-omarchy-mobile/pull/125), merged, which also checks the panel's ID; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on), [brightness record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (Panthor, the desktop renders, runtime suspend when idle; this boot runs at a fixed 160 MHz; the GPU's levels and its thermal zone are merged with [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96) ([kernel #85](https://github.com/derekross/linux-omarchy-mobile/pull/85) and [kernel #87](https://github.com/derekross/linux-omarchy-mobile/pull/87) landed through it), not yet run from it; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (Synaptics S3908 over SPI, [kernel #81](https://github.com/derekross/linux-omarchy-mobile/pull/81), merged; survives sleep, [kernel #82](https://github.com/derekross/linux-omarchy-mobile/pull/82), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (gpio-keys: power and volume; the power key wakes from sleep; unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (CDC NCM and SSH; reconnects after a replug, `omamobile-tegu-usb-rescue`, also after a replug the gadget didn't see ([#280](https://github.com/derekross/omarchy-mobile/pull/280); [the replug record](../device/tegu/NOTES.md#2026-10-28-what-the-device-tables-claims-rest-on)); [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (BCM4383 on PCIe channel 1 through the shared gs201 host and zuma PHY, [kernel #178](https://github.com/derekross/linux-omarchy-mobile/pull/178) on [kernel #156](https://github.com/derekross/linux-omarchy-mobile/pull/156), in review, and brcmfmac, [kernel #56](https://github.com/derekross/linux-omarchy-mobile/pull/56) in review; started at boot by `omamobile-device-tegu` 0.2.4 ([#238](https://github.com/derekross/omarchy-mobile/pull/238)) with the extractor's firmware; WPA2 with iwd, kept loaded across sleep by the package's sleep hook; [record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) | works (hci_uart on the bootloader's address, BlueZ, A2DP; [kernel #94](https://github.com/derekross/linux-omarchy-mobile/pull/94); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (MAX77779 gauge with temperature, current and time estimates through UPower; [kernel #66](https://github.com/derekross/linux-omarchy-mobile/pull/66); unpublished kernel; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (the MAX77779 charger binds from the device tree, [kernel #161](https://github.com/derekross/linux-omarchy-mobile/pull/161), merged, built by [#227](https://github.com/derekross/omarchy-mobile/pull/227); the charge limit in `omamobile-device-tegu` 0.2.3 holds charging off at the limit through `charge_behaviour` and lets it run again 5 below; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (CS40L26A through feedbackd, with brownout prevention, [kernel #95](https://github.com/derekross/linux-omarchy-mobile/pull/95); its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (LM3644, 25 steps to 75 mA, quick-settings tile; its DT [kernel #96](https://github.com/derekross/linux-omarchy-mobile/pull/96), merged; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | testing (Omarchy's Location switch serves fixes from GPS, Galileo, BeiDou and GLONASS through the modem's Kepler once the modem is up; `omamobile-device-tegu`'s boot unit, opt-in ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), starts gnssd with the modem: on 2026-11-03, after a reboot with nothing started by hand, a session through the switch's socket had 133 of 135 GGA sentences fixed, first fix 48 s after the client connected, 18 satellites; on 2026-11-09 the same from the in-tree drivers (the GNSS link driver, [kernel #185](https://github.com/derekross/linux-omarchy-mobile/pull/185), and the 9a's GNSS nodes, [kernel #237](https://github.com/derekross/linux-omarchy-mobile/pull/237), the `modem-cpif` overlay with the driver's config line, [#594](https://github.com/derekross/omarchy-mobile/pull/594), and the PCIe PHY's config line, [#274](https://github.com/derekross/omarchy-mobile/pull/274), all merged): four 150 s sessions, 497 of 507 GGA sentences fixed, up to 24 satellites; a 30-minute session on 2026-11-01 (omamobile-gnss 0.2.5's start, by hand, the modem link left to cycle) had 1763 of 1765, first fix 37 s, and a session in a moving car from [#114](https://github.com/derekross/omarchy-mobile/pull/114)'s package build fixed 145 of 153 times; needs [kernel #150](https://github.com/derekross/linux-omarchy-mobile/pull/150) and the PERIC0 clock fix [kernel #172](https://github.com/derekross/linux-omarchy-mobile/pull/172), both merged (without the latter the modem link hung the palmtop within minutes); not on by default because the modem isn't (the Modem cell); [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/TELEPHONY-PLAN.md), record with [#202](https://github.com/derekross/omarchy-mobile/pull/202)) | works (audio and sensor DSP through the GSA and Trusty, 93 services, started at boot by `omamobile-device-tegu` 0.2.3 from the device tree, [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162) in review; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (the TAS2572 amplifiers and the microphone through the AoC card and PipeWire, at boot with the AoC: a tone from the speaker, the microphone at -32 dBFS; speakers at a -10 dB ceiling; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (accelerometer, light, proximity, compass and gyroscope through the AoC and the sensor proxy, at boot with the AoC; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | testing (calls in and out on the speakerphone, texts both ways, mobile data over the device `omamobile-sit-data` finds, with the Phone and Messages apps on the bridge ([#356](https://github.com/derekross/omarchy-mobile/pull/356), [#364](https://github.com/derekross/omarchy-mobile/pull/364), [#367](https://github.com/derekross/omarchy-mobile/pull/367), [#371](https://github.com/derekross/omarchy-mobile/pull/371)); `omamobile-device-tegu`'s boot unit starts the modem, the phone service and the bridge at boot when the owner enables it ([#372](https://github.com/derekross/omarchy-mobile/pull/372)), and is skipped on a phone that isn't provisioned or has no CP drivers; opt-in, not on by default, while marking slot B on the 9a is on hold ([#483](https://github.com/derekross/omarchy-mobile/pull/483)): with B unmarked a reset spends one of ABL's three tries, and the unit can't tell; it ran by itself on the owner's palmtop over a day of reboots on 2026-11-09 with 42 of 43 stops confirmed, one unexplained ([record](../device/tegu/TELEPHONY-PLAN.md#29-the-in-tree-drivers-through-the-starter-on-the-published-tip-plus-four-open-changes-2026-11-09)), and the NV partitions unchanged, on the owner's kernel with the bring-up modules, and on the in-tree drivers ([kernel #187](https://github.com/derekross/linux-omarchy-mobile/pull/187) and [kernel #162](https://github.com/derekross/linux-omarchy-mobile/pull/162), merged), which need the `modem-cpif` overlay and the GNSS link driver's config line ([#594](https://github.com/derekross/omarchy-mobile/pull/594)) and the PCIe PHY's config line ([#274](https://github.com/derekross/omarchy-mobile/pull/274)), all merged; the far end still hears an echo on the speakerphone ([#378](https://github.com/derekross/omarchy-mobile/pull/378)) and there's no earpiece route yet; [plan](../device/tegu/MODEM-PLAN.md), [record](../device/tegu/MODEM-PLAN.md#sim-test-results-2026-11-02-730-rc6-omamobile-tegu-s2mpg15-m1-g991042e2eff3-dirty)) | works (NXP ST54L through nxp-nci, its clock pad from the PMU ([kernel #104](https://github.com/derekross/linux-omarchy-mobile/pull/104) and [kernel #105](https://github.com/derekross/linux-omarchy-mobile/pull/105), merged, with [kernel #131](https://github.com/derekross/linux-omarchy-mobile/pull/131), merged), loaded at boot by `omamobile-device-tegu` 0.2.3 with neard and the link opener: polls with no I2C errors and read a payment card; polling stops before sleep; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | works (main, ultrawide and front from the device tree, [kernel #164](https://github.com/derekross/linux-omarchy-mobile/pull/164) in review, with [kernel #63](https://github.com/derekross/linux-omarchy-mobile/pull/63) and [kernel #103](https://github.com/derekross/linux-omarchy-mobile/pull/103) in review: their drivers load at boot with nothing by hand, libcamera lists all three, and each streams, main at 19 fps and the others at 30 fps at 1280x960; basic picture quality through libcamera's software ISP; [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel); the main camera's contrast autofocus through its LC898129 ([kernel #213](https://github.com/derekross/linux-omarchy-mobile/pull/213) in review, and a lens node on top of kernel #164, not yet in a pull request; libcamera 0.7.2-4.23), close subjects checked, the far end not yet ([notes](../device/tegu/NOTES.md#2026-11-07-the-main-cameras-autofocus-lc898129))) | testing (s2idle when asked for, or through the shell's auto-sleep with the `omamobile-sleep` toggle on, which ships off; the power key lights the lock screen on the first press ([#268](https://github.com/derekross/omarchy-mobile/pull/268), [#275](https://github.com/derekross/omarchy-mobile/pull/275)), and the RTC wakes it; on 2026-11-01 thirty attended 120 s sleeps ([plan](../device/tegu/SLEEP-PLAN.md) S1) and a day of real use with auto-sleep on ([record](../device/tegu/NOTES.md#2026-11-01-afternoon-and-evening-wi-fi-from-the-package-the-wake-and-the-replug-storage-idle-the-panel-id)) all woke, but six test sleeps since 10-28 did not (one in 21 on the clock-fix kernel on 11-01, and one in 127 in an overnight series of 120 s sleeps on 11-01 to 11-02), so sleep tests stay attended ([plan](../device/tegu/SLEEP-PLAN.md) S1); two causes are known: dead storage after a wake (the UFS pads came back as plain inputs on about one wake in a hundred; [kernel #224](https://github.com/derekross/linux-omarchy-mobile/pull/224), merged 2026-11-12, re-applies the pin state at resume), and a silent no-wake with C2 entered through s2idle's own path: with C2 disabled for the sleep (#96) or left to the ordinary idle path (#98) the RTC alarm woke the palmtop 300 of 300 times each, then 15 of 15 on the everyday path, twelve of them half-hour auto-sleeps (2026-11-07 and 11-08, [#409](https://github.com/derekross/omarchy-mobile/pull/409)); [#591](https://github.com/derekross/omarchy-mobile/pull/591) is the sleep hook for it; open: on the published kernel, which has no idle states, a sleep still fails to wake now and then: one in 22 on 2026-11-09 (a boot without modules), none in two re-runs of 36, and one in 213 overnight on 2026-11-10 on the newest tip with the fixes, where the console shows the kernel alive and CPU 7 not answering from just after the device suspend ([plan](../device/tegu/SLEEP-PLAN.md) S1, failure 3), it didn't come back in 701 sleeps with the PM debug switches on (2026-11-10 to 11-12), and with the function tracer and `pm_print_times` off it came back on 2026-11-12, twice: the console places CPU 7 in the noirq resume of an HSI0 USI, and [kernel #247](https://github.com/derekross/linux-omarchy-mobile/pull/247) proposes keeping those clocks on; the RTC alarm on the published kernel came with [kernel #236](https://github.com/derekross/linux-omarchy-mobile/pull/236), merged 2026-11-10 (a kernel from before it has none, and a sleep there ends only on a key); works once that no-wake is found and fixed and a series has run from the published kernel as it is, with kernel #224 in it; the drain is about 300 mA asleep, the same as awake with the screen off, because nothing below the CPUs sleeps yet; [record](../device/tegu/NOTES.md#2026-10-29-the-device-package-boot-and-what-this-row-rests-on)) | works (opt-in) (enrolls, matches, refuses other fingers and unlocks the lock screen through Trusty and `omamobile-fingerprint`, 2026-10-29; the sensor binds from the device tree, [kernel #165](https://github.com/derekross/linux-omarchy-mobile/pull/165) in review, with [kernel #126](https://github.com/derekross/linux-omarchy-mobile/pull/126), merged; opt-in as on the 7 Pro, because the service's first run writes Trusty's secure storage; [plan](../device/tegu/FINGERPRINT-PLAN.md), [record](../device/tegu/NOTES.md#2026-10-31-and-11-01-every-testing-row-on-a-published-style-kernel)) | | Pixel 8 Pro ([notes](../device/husky/NOTES.md)) | husky | zuma (Tensor G3) | WIP ([#1](https://github.com/derekross/omarchy-mobile/issues/1): slot B; from 2026-11-01 to 2026-11-07 a local unit marked slot B on Lef's phone; stopped pending [#252](https://github.com/derekross/omarchy-mobile/issues/252); Omarchy's image from the signed repository boots to the lock screen; the zuma device tree is merged ([kernel #13](https://github.com/derekross/linux-omarchy-mobile/pull/13), [#53](https://github.com/derekross/linux-omarchy-mobile/pull/53))) | testing (UFS at HS-G4 behind the read-only guard; Arch on `userdata`; [UFS-PLAN](../device/husky/UFS-PLAN.md)) | WIP (simple-framebuffer with the panel's size, [kernel #154](https://github.com/derekross/linux-omarchy-mobile/pull/154): console and Omarchy's lock screen; 120 Hz with brightness on a test kernel, [kernel #160](https://github.com/derekross/linux-omarchy-mobile/pull/160), started by the device package, [#228](https://github.com/derekross/omarchy-mobile/pull/228)) | testing (panthor on the Mali-G715 on a test kernel, [kernel #157](https://github.com/derekross/linux-omarchy-mobile/pull/157); [ROADMAP](../device/husky/ROADMAP.md)) | works (ST FTM5 over SPI, merged, [kernel #153](https://github.com/derekross/linux-omarchy-mobile/pull/153); verified on both phones, [notes](../device/husky/NOTES.md#2026-10-31-the-touchscreen)) | works (power and volume through `gpio-keys`, merged, [kernel #148](https://github.com/derekross/linux-omarchy-mobile/pull/148); verified on both phones, [notes](../device/husky/NOTES.md#2026-10-30-the-buttons)) | testing (USB and SSH) | testing (BCM4398 on a test kernel, [kernel #156](https://github.com/derekross/linux-omarchy-mobile/pull/156); firmware from the owner's image, [#214](https://github.com/derekross/omarchy-mobile/pull/214); [NOTES](../device/husky/NOTES.md)) | testing (BCM4398D0 on the APM UART, pairing and A2DP, on a test kernel, [kernel #199](https://github.com/derekross/linux-omarchy-mobile/pull/199); firmware from the owner's image; [ROADMAP](../device/husky/ROADMAP.md)) | testing (gauge with Google's ATL model through a bring-up overlay, temperature, MFD, pack EEPROM, [kernel #163](https://github.com/derekross/linux-omarchy-mobile/pull/163) and [#175](https://github.com/derekross/linux-omarchy-mobile/pull/175); the bus on by default waits for [kernel #205](https://github.com/derekross/linux-omarchy-mobile/pull/205); [notes](../device/husky/BATTERY-NOTES.md)) | testing (the charger driver and `tcpci_maxim` as a sink, by hand under a dead-man tool: sink at the Type-C 3 A level (input limit 0xc2 0x13 to 0x77); +1.5 A measured from a 30 W charger without the driver (1.485 A is the device-tree cap); no host mode, [#282](https://github.com/derekross/omarchy-mobile/pull/282); [notes](../device/husky/BATTERY-NOTES.md), [#1](https://github.com/derekross/omarchy-mobile/issues/1#issuecomment-6562547503)) | testing (CS40L26 on a test kernel, [kernel #200](https://github.com/derekross/linux-omarchy-mobile/pull/200)) | testing (LM3644, 50 mA cap, on a test kernel, [kernel #200](https://github.com/derekross/linux-omarchy-mobile/pull/200)) | no (first steps as drafts: [kernel #210](https://github.com/derekross/linux-omarchy-mobile/pull/210), [#211](https://github.com/derekross/linux-omarchy-mobile/pull/211)) | testing (loads and runs through the GSA and Trusty on a test kernel; plan and results [AOC-PLAN](../device/husky/AOC-PLAN.md)) | testing (stereo speakers and one microphone through UCM2 and PipeWire on a test kernel, [kernel #203](https://github.com/derekross/linux-omarchy-mobile/pull/203), [#204](https://github.com/derekross/linux-omarchy-mobile/pull/204), package [#453](https://github.com/derekross/omarchy-mobile/pull/453)) | no | no | testing (a test kernel reads tags; [kernel #208](https://github.com/derekross/linux-omarchy-mobile/pull/208), [#209](https://github.com/derekross/linux-omarchy-mobile/pull/209)) | no | no | no | | Pixel 8a ([notes](../device/akita/NOTES.md)) | akita | zuma (Tensor G3) | WIP ([#1](https://github.com/derekross/omarchy-mobile/issues/1): slot B into the initramfs with USB and SSH, no Arch root yet; `reboot-to bootloader` reaches fastboot; the first-light board file is merged, [kernel #192](https://github.com/derekross/linux-omarchy-mobile/pull/192); [BRINGUP-M1](../device/akita/BRINGUP-M1.md), [notes](../device/akita/NOTES.md#2026-11-03-first-light-m13-and-the-console-m14)) | WIP (UFS at HS-G4 behind the read-only guard, from the initramfs only, nothing run on `userdata`; merged, [kernel #206](https://github.com/derekross/linux-omarchy-mobile/pull/206); the device package and storage plan are in review, [#456](https://github.com/derekross/omarchy-mobile/pull/456)) | WIP (simple-framebuffer: the boot console over Omarchy's splash; merged with [kernel #192](https://github.com/derekross/linux-omarchy-mobile/pull/192); [notes](../device/akita/NOTES.md#2026-11-03-first-light-m13-and-the-console-m14)) | no | no | no | testing (USB and SSH from the initramfs; merged with [kernel #192](https://github.com/derekross/linux-omarchy-mobile/pull/192)) | no | no | no | no | no | no | no | no | no | no | no | no | no | no | no | | Pixel 7 ([notes](../device/panther/NOTES.md)) | panther | gs201 (Tensor G2) | WIP (slot B, not marked successful; Arch and Hyprland boot; `boot` and `init_boot` carry AVB footers, which a phone whose `vbmeta_b` is stock's needs to start slot B (device-panther 0.1.6, [#396](https://github.com/derekross/omarchy-mobile/pull/396), [notes](../device/panther/NOTES.md#2026-11-02-avb-footers-on-boot-and-init_boot)); `reboot-to bootloader` reaches fastboot with `reboot=warm`, merged, [kernel #198](https://github.com/derekross/linux-omarchy-mobile/pull/198); [BRINGUP-M1](../device/panther/BRINGUP-M1.md); board file merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108)) | testing (UFS behind the read-only guard; Arch on `userdata`; merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108)) | WIP (simple-framebuffer: console and the Hyprland desktop; brightness and display off/on over DCS through the S6E3FC3 panel driver and device-panther 0.1.4, merged, [kernel #177](https://github.com/derekross/linux-omarchy-mobile/pull/177); [DISPLAY-PLAN](../device/panther/DISPLAY-PLAN.md)) | testing (Panthor on the Mali-G710; Hyprland renders on it; merged through [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134), with the always-on rule in device-panther 0.1.1, [#145](https://github.com/derekross/omarchy-mobile/pull/145); [notes](../device/panther/NOTES.md#2026-10-28-the-gpu-and-the-haptics-power-off-and-usb-after-a-replug)) | WIP (FocalTech FT3658: touches drive Hyprland and unlock the PIN screen on a test kernel; the driver, [kernel #109](https://github.com/derekross/linux-omarchy-mobile/pull/109), and the touch node, [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134), are merged; stock's interrupt pull-up and CS delay, [kernel #166](https://github.com/derekross/linux-omarchy-mobile/pull/166), the driver's review fixes, [kernel #133](https://github.com/derekross/linux-omarchy-mobile/pull/133), and the kernel option with the touch notes, [#149](https://github.com/derekross/omarchy-mobile/pull/149), are merged) | testing (gpio-keys: power, volume; merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108)) | testing (USB and SSH; merged, [kernel #108](https://github.com/derekross/linux-omarchy-mobile/pull/108); a replug comes back through `omamobile-usb-rescue` in device-panther 0.1.3, on the MAX77759's I2C bus, merged, [kernel #167](https://github.com/derekross/linux-omarchy-mobile/pull/167); [notes](../device/panther/NOTES.md#2026-10-29-usb-comes-back-after-a-replug-u1)) | no | testing (BCM4389 on the APM UART through hci_bcm with the phone's own patch, merged, [kernel #196](https://github.com/derekross/linux-omarchy-mobile/pull/196): a BLE mouse drives the shell, and headphones play over A2DP with their AVRCP buttons, on its author's Pixel 7; the Bluetooth address and the rest of the package side aren't merged yet; headset calls, suspend and power not tested) | WIP (MAX77759 fuel gauge with stock's battery model on a test kernel: percentage, capacity and temperature in upower and the bar; merged, [kernel #171](https://github.com/derekross/linux-omarchy-mobile/pull/171) and the driver fix [#170](https://github.com/derekross/linux-omarchy-mobile/pull/170); a second unit's Lishen pack isn't matched by that model yet, [notes](../device/panther/NOTES.md#2026-11-03-a-second-unit-angelrenefm); [BATTERY-PLAN](../device/panther/BATTERY-PLAN.md)) | no | testing (CS40L26 with the phone's own firmware, `tools/extract-vendor-firmware -d panther`; merged through [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134), with the feedbackd rule in device-panther 0.1.1, [#145](https://github.com/derekross/omarchy-mobile/pull/145); [notes](../device/panther/NOTES.md#2026-10-28-the-gpu-and-the-haptics-power-off-and-usb-after-a-replug)) | testing (LM3644 rear flash, 36 steps, from the node merged in [kernel #197](https://github.com/derekross/linux-omarchy-mobile/pull/197) and the bring-up overlay `torch.dtso`, [#426](https://github.com/derekross/omarchy-mobile/pull/426): lit on a second Pixel 7; the quick-settings tile not checked) | no | no | no | no | no | no | no (the package's camera start and DMA heap rule are merged, device-panther 0.1.9, [#443](https://github.com/derekross/omarchy-mobile/pull/443), inert until the camera device tree is proposed) | no (s2idle by default since [kernel #134](https://github.com/derekross/linux-omarchy-mobile/pull/134) and device-panther 0.1.1; not run yet) | WIP (the sensor's node, disabled, is merged, [kernel #207](https://github.com/derekross/linux-omarchy-mobile/pull/207); with a bring-up overlay and the package side, which is in review ([#458](https://github.com/derekross/omarchy-mobile/pull/458)), a finger enrolled and unlocked its author's Pixel 7, also after a reboot) |