From 08e8f3fe3a752e2873f4529a0d72245265fd4451 Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sat, 14 Nov 2026 03:36:28 -0644 Subject: [PATCH 1/8] device/tegu MODEM-PLAN: the 9a sleeps with the modem up on a test kernel with the sleep guard (2026-11-13) The results of the day and the night on a test kernel with linux-omarchy-mobile#232, the maintainer's two patches from its review and the sleep guard turned on for zumapro: a refusal no longer costs the phone service, 299 of 300 sleeps in one boot were entered and woke with the modem registered, and what is still open (one hang at a link power-up while awake, RTC alarms that come early or late, the route change of #750, calls and texts during a sleep). The series script that ran is added to device/tegu/bringup. AGENTS.md gets one sentence; the rule for the daily kernel is unchanged. --- AGENTS.md | 2 +- device/tegu/MODEM-PLAN.md | 114 +++++++++++++++++++++- device/tegu/bringup/modem-sleep-series.sh | 67 +++++++++++++ 3 files changed, 181 insertions(+), 2 deletions(-) create mode 100755 device/tegu/bringup/modem-sleep-series.sh diff --git a/AGENTS.md b/AGENTS.md index 6f834cfbe..3ffb1e5d5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -194,7 +194,7 @@ Each entry gives the gotcha, then the reason or the fix. Details are in the link - **Some unattended RTC cycles wake early on the PMIC interrupt** (2 of 20 in S4, 4 of 31 in S5c, at 13-16 s instead of 30 s). Which PMIC event it is hasn't been read yet; count it, don't treat it as a failed cycle. [SUSPEND-PLAN](device/cheetah/SUSPEND-PLAN.md) S4, S5c - **Volume Up is the one key without an internal pull** (`PUD` 0x0, as on stock), and the first freeze woke 3 s in from it with nobody touching it. It didn't recur in the 51 RTC cycles of S4 and S5c, and it isn't proven to be a false edge; if spurious wakes recur, check gpa8-5's power-down pull before blaming the wake path. - **UFS ran its error handler around modem power changes** (PA LINERESET indications, recovered without a reset). Stock's DT shows no shared rail, clock or pin. M2.5 narrowed it down: power-on and a running link are clean, but every modem power-off is followed by one recovered UFS error-handler pass (a PA LINERESET, no I/O errors or resets). Stock powers the modem off only at shutdown, so the modem stays on once it is up (M2.6 is open), and a reboot with the modem up costs one such pass. Keep modem tests attended, don't power-cycle the modem, and check `dmesg | grep ufshcd_err_handler` after each step. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) M2, M2.5, M2.6 -- **Suspend with the modem on has never run on the 7 Pro, and is unqualified on the 9a**: its one try (2026-10-27) also had NFC polling, which alone resets the 9a in s2idle, so the modem's part was unknown. With the sleep fixes (2026-11-09, both link drivers) it neither hung nor reset: CPIF refuses most sleeps (`suspend_cp: abort suspend`, -16, while the CP holds `cp2ap_wakeup`), and the three that were entered woke on the modem within seconds. But one sleep attempt, entered or refused, leaves CPIF unable to reach the CP (`Can't set D0 state`, its command register reading 0xffff) and the phone service timing out until a reboot, while `omamobile-sit status` goes on showing the last state read. So with the modem up nothing passes or goes round its sleep inhibitor (`systemctl suspend -i`, `rtcwake -m`, a write to `/sys/power/state`) except an attended test on the charger that ends in a reboot; on the 7 Pro that test is the maintainer's. CPIF's PM paths are wake-armed in code, but M6 is design only. Keep modem tests on the charger, so upower's critical action doesn't suspend the phone mid-test. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) 6.3, [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) "Suspend with the modem on" +- **Suspend with the modem on has never run on the 7 Pro, and is unqualified on the 9a**: its one try (2026-10-27) also had NFC polling, which alone resets the 9a in s2idle, so the modem's part was unknown. With the sleep fixes (2026-11-09, both link drivers) it neither hung nor reset: CPIF refuses most sleeps (`suspend_cp: abort suspend`, -16, while the CP holds `cp2ap_wakeup`), and the three that were entered woke on the modem within seconds. But one sleep attempt, entered or refused, leaves CPIF unable to reach the CP (`Can't set D0 state`, its command register reading 0xffff) and the phone service timing out until a reboot, while `omamobile-sit status` goes on showing the last state read. So with the modem up nothing passes or goes round its sleep inhibitor (`systemctl suspend -i`, `rtcwake -m`, a write to `/sys/power/state`) except an attended test on the charger that ends in a reboot; on the 7 Pro that test is the maintainer's. On a 9a test kernel with [linux-omarchy-mobile#232](https://github.com/derekross/linux-omarchy-mobile/pull/232) and its sleep guard turned on for zumapro, 299 of 300 sleeps with the modem registered were entered and woke with the phone service kept (2026-11-13), and a refusal cost nothing; that kernel also hung once at a link power-up while awake, and none of it is published, so on the daily kernel the rule is unchanged. CPIF's PM paths are wake-armed in code, but M6 is design only. Keep modem tests on the charger, so upower's critical action doesn't suspend the phone mid-test. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) 6.3, [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) "Suspend with the modem on" - **Never `systemctl restart omamobile-modem` (or stop and start it) to recover a crashed modem.** An in-place restart left raven's CP in BOOTING, and a bring-up that never reaches ONLINE sets a latch that makes the next boot skip the modem. Reboot instead. If a boot skipped it, remove the latch (`rm /var/lib/omamobile/modem-bringup-started`) and `systemctl start omamobile-modem`; `/usr/lib/omamobile-vendor/omamobile-modem-start --retry` also clears it, but run by hand it brings the modem up outside the unit (a stray `rfsd` in your shell's session). From `omamobile-modem` 0.1.13 there is a second marker, `/var/lib/omamobile/modem-up`, written at ONLINE and removed by the unit's clean stop: a boot that hung or reset with the modem up leaves it. The next boot logs "ended without a clean stop" and, from 0.1.19, starts the modem all the same (0.1.13 to 0.1.18 skipped it for that boot); a second unclean end in a row refuses "(held)" until the markers are removed the same way (cheetah [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) M4.6). On the 7 Pro the stack starts at boot by default since device-cheetah 0.1.43 ([MODEM-PLAN](device/cheetah/MODEM-PLAN.md) M4.6b); to keep it off from the next boot, `systemctl mask omamobile-modem-boot.target` (`unmask` turns it back on; `disable` doesn't remove device-cheetah's vendor wants link); it also holds a sleep inhibitor until the reboot. [raven MODEM-PLAN](device/raven/MODEM-PLAN.md) D4. The same on the 9a: after the modem profile has stopped, starting it again in the same boot reset the palmtop silently, a watchdog reset with nothing in pstore (the one try was in the rfsd stage). [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) G3. **From `omamobile-modem` 0.1.10, `omamobile-vendor-chroot modem check` refuses once the profile has started this boot** ("the modem profile already started this boot"), so it is no longer a post-install health check while the modem is up: it is read-only and safe to run, but a refusal means the profile ran, not that anything is wrong. After a reboot it says "preconditions met" again. - **Linux never writes the modem's NV partitions.** The modem chroot profile (cbd, rfsd) refuses to start unless the NV partitions read `ro=1` and none is mounted, and works on checksum-verified RAM copies only. What the modem writes is lost at stop, except its writes to `efs/nv_normal.bin` (expected to hold the carrier profile, network and IMS data; the runbook confirms it) when the NV overlay is on (section 1): those are captured every 30 minutes and at the unit's stop, after the CP is off, and laid over the verified copy at the next start. `reboot-to` (and `bin/omamobile push`) skips that stop, so stop the modem first (section 2) or run `omamobile-modem nv-overlay capture`. Don't give the chroot a real block device. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) "NV overlay" diff --git a/device/tegu/MODEM-PLAN.md b/device/tegu/MODEM-PLAN.md index dd96bfe5f..439a3415d 100644 --- a/device/tegu/MODEM-PLAN.md +++ b/device/tegu/MODEM-PLAN.md @@ -1714,6 +1714,118 @@ the test), on `7.3.0-rc6-omamobile-push1-dbg-gff4fdf58035e`: error lines. It doesn't settle the Wi-Fi loss above, which also happened with one state. +### Suspend with the modem on, with the sleep guard: it sleeps and keeps the phone service (2026-11-13 and the night after) + +A test kernel only; nothing here is in a published kernel or a package yet, +so the modem's inhibitor and the rule in AGENTS.md stand as they are. + +The kernel: `7.3.0-rc6-omamobile-push1-gc50fd8fdf42a`, which is the +published `omamobile` at `2db2bc35beec` plus kernel #247 (the HSI0 USI +gates), linux-omarchy-mobile#232's four commits (`f4e5e4846c39`: the link +driver refuses a system sleep while the modem's link is up, keeps the PCI +core's sleep handling off the modem's root port and leaves the endpoint +out of system sleep), the maintainer's two patches from #232's review +(6.1: the refusal moved to a PM notifier before the freeze, the root port +a syscore device; 6.2: a ring trace) and one line of ours: all of it is +behind `sleep_guard`, which only gs201 sets, so `.sleep_guard = true` in +`zumapro_pcie_cp_data`. Userspace was main's with #697's phone packages. +The 9a has nothing that tells the modem the screen state, so each test +sent "screen off" through sitd's socket before a request and "screen on" +after, waited for the link to be down for 5 s, armed an RTC alarm and +passed the modem's inhibitor with `systemctl suspend -i`. On the charger, +cable in, home LTE with IMS voice and SMS. The owner was not at the +palmtop for these runs; from 2026-11-13 he has told us to run sleep tests +with the modem up without him, and every boot ended in `systemctl reboot`. +The series script is `device/tegu/bringup/modem-sleep-series.sh` (the copy +that ran logged to another file and lacked the check that `rtcwake` armed +the alarm; otherwise the same). + +- **The same day without the guard, for comparison** (the daily kernel #100 + with its out-of-tree link driver, one request per boot, five boots). As + they were: in one boot the link was down at the request and the modem + was lost as on 11-09 (`pcieport … D0 to D3hot, device inaccessible`, + `Data Link Layer Link Active not set in 100 msec`, `Can't set D0 state`, + 33 failed polls in four minutes); in the other the endpoint was in D0 at + the request and nothing was lost. With 0 written to the endpoint's + `d3cold_allowed` first, three boots, the link down at each request: none + lost the modem. That fits #232's reading (the root port marked D3cold, + and the resume waiting for a link that isn't coming). Both boots that + entered a sleep then failed to shut down: `systemctl reboot` logged + nothing more, the palmtop came back 57 s later and the modem's stop was + not recorded, which the modem's guard counted as unclean ends. The NV + partitions read SAME against the backup afterwards. +- **A refusal no longer costs the phone service.** 14 requests in one boot + were refused by the notifier (`sleep refused before the freeze: link up, + PHY out of isolation`), twelve of them under `ping -i 1 -I rmnet0`: 273 + of 273 and 200 of 200 replies, no failed poll in sitd. +- **It sleeps.** With the cellular data service stopped: three requests of + 45 s in that boot, all entered (45.2 s and 44.6 s to the alarm, one of + 15.1 s ended by `cp2ap_wakeup`), then twenty of 60 s in the next boot, + 19 entered (17 to the alarm, one ended at 27.7 s by `cp2ap_wakeup`, one + at 27.8 s by the PMIC's interrupt) and one refused. After each: the CP + ONLINE, sitd's polls answered, `ping -I rmnet0` 5 of 5. No `Unable to + change power state`, `Link Active`, `BME` or UFS error-handler line, the + root port in D0 and the endpoint in D3hot at the end, and both + `systemctl reboot`s stopped the modem cleanly (`CP OFFLINE`, the marker + removed), back in 25 s. +- **With Wi-Fi connected and the data service as on main, every request is + refused**: those 14 were all there were. At logind's sleep announcement NetworkManager takes + `wlan0` down, the bearer's metric-700 default route is the only one + left, whatever was talking over Wi-Fi goes out over `rmnet0` in the same + second, and the link is up when the notifier looks. omarchy-mobile#750 + (a draft) has `omamobile-sit-data` remove its route at the announcement + and put it back once Wi-Fi's is back, or after 10 s without one. Put + back at once after the wake it was worse than useless: 18 of 20 entered + and none reached the alarm, the answers to the few seconds of traffic + ending each next sleep early. +- **The night: 300 requests of 60 s in one boot, 299 entered and woke, 1 + refused** (7 h 40 min, Wi-Fi connected, the data service running with + #750's change copied over the packaged file for the test and put back + after). `suspend_stats` success 299, fail 1. 239 ended on the PMIC's + interrupt (the RTC alarm) and 60 on `cp2ap_wakeup`. After every one of + the 300: the CP ONLINE, sitd active, no failed poll, Wi-Fi up, no UFS + line; the default routes read Wi-Fi first with the bearer behind before + every request, and the service logged the route back behind Wi-Fi 300 + times. At the end: registered with IMS voice, `ping -I rmnet0` 5 of 5, + no bad line in the kernel log of the boot (the list above, plus oops, + warning and RCU stall). Entry took 0.52 to 0.68 s to the noirq stage and + resume 0.72 to 0.87 s by the kernel's stamps. The reboot after it + stopped the modem cleanly. +- **One hang, awake, on this build, not understood.** The evening before + the night run, a boot that had run 52 requests (49 entered, 3 refused) + stopped 20.4 s after a sleep had ended normally: the console's last + lines are the `cp2ap_wakeup` edge that starts every link power-up, and + the `zuma-pcie-phy … external PLL lock` line that follows about 12 ms + later in every other cycle never came. The watchdog reset the palmtop + about two minutes later (`APC Watchdog Early`, `RST_STAT: 0x1 - + CLUSTER0_NONCPU_WDTRESET`); nobody touched it. That was the boot's + 1,549th link power-up; the night's boot did 6,311 without one. It is on + the in-tree link driver's power-up path (`pcie-gs201-cp` with + `phy-zuma-pcie`), which the daily kernel doesn't use (it has the + out-of-tree copy), and whether it happens without #232 isn't known. + Reported on #232. The next boot started the modem after the one unclean + end the guard allows. +- **The alarm is not always what ends a 60 s sleep, and not always at + 60 s.** In the night's 239 alarm wakes, by the RTC from the arming: 223 + at 58 to 64 s, 12 at 28 to 36 s, 3 at 89 to 90 s and one at 121 s; five + more sleeps ran 83 to 93 s and were ended by the modem. The PMIC's + interrupt counted 522 in the boot, twice `rtc-alarm0`'s 261, so every + PMIC wake was an RTC alarm: the RTC raised one about 30 s early or 30 to + 60 s late now and then. The 300 modem-off sleeps on this kernel's base + earlier on 11-13 were all 59.2 to 60.3 s. Not traced; a second RTC timer beside + `rtcwake`'s is our guess (the kernel's alarmtimer programs the RTC at + suspend for the next userspace alarm), and the `rtc` and `alarmtimer` + trace events over a run would show it. The palmtop woke every time. + +What this leaves before the modem's inhibitor can go on the 9a: #232 in +the published kernel and the zumapro line after it (its own small pull +request, as #232's author asked); the hang above understood, or at least +counted on a build without #232; the screen state told to the modem by +the shell, not by hand (the 7 Pro's side of that is omarchy-mobile#556); +the route change (#750, or its equivalent in #556); and what no run here +tested: a call and a text arriving while the palmtop sleeps, a sleep of +hours, and a sleep off the charger. + ### M2: the link only (modem link driver, `allow_hw=1`; risk: medium) One step per run, the CPIF overlay not applied, the link overlay applied @@ -1903,7 +2015,7 @@ Preconditions, as SLEEP-PLAN's: | The bridge for apps | `omamobile-mmd` not installed | not part of this test (T2 follows a passing run) | | The link watcher | `omamobile-modem` 0.1.9 () keeps one open file on the boot device, which retires the old open-and-close watcher that woke the CP link every half second; 0.1.10 () adds the once-per-boot refusal | with the CP ONLINE the link idles and cycles on its own; on 2026-10-29 the palmtop hung eight times in a morning with the modem up for hours, the early ones tied to the watcher's open and close of the boot device (the console logs end on CPIF's `cp2ap_wakeup` wake toggles; `logs/tegu-review-20261029/hangs/`). The run uses 0.1.10 (SIM-1). Keep the session short (under an hour) and attended; `cpif-keep-link.ko` (G3-L1 step 1, untested) is the candidate fix, not part of this run | | Call audio | the AoC card is `google,tegu-aoc-snd-card` with its EP PCMs, but `omamobile-speaker-protection` is inactive and the amplifiers are TAS2572, which `voice_audio.py`'s earpiece checks (CS35L41 controls) don't know | **no audio routing exists on the 9a: calls are not part of this test.** `[audio] card = none` stays; a signalling-only call (cheetah's S10) is a later, separate step | -| Suspend | unqualified with the modem on (section "Suspend with the modem on"); the start script's inhibitor blocks `systemctl suspend` for the boot | charger in for the whole test | +| Suspend | unqualified with the modem on (section "Suspend with the modem on"); the start script's inhibitor blocks `systemctl suspend` for the boot. On a test kernel with linux-omarchy-mobile#232 and the sleep guard on for zumapro, 299 of 300 sleeps were entered and woke with the phone service kept (2026-11-13) | charger in for the whole test | | Identifiers | the ICCID (printed on the card), IMSI, IMEI, phone number, the carrier's name, its APN, IP addresses and message text never go into the repository, a log excerpt or a screenshot | sitd logs states and codes only; before quoting a log: `grep -nE '[0-9]{7,}|\+[0-9]|[0-9a-f]{2}(:[0-9a-f]{2}){5}|@' logs/` must show nothing you would quote | **What to expect, honestly:** SIM detection likely (GET_SIM_STATUS has diff --git a/device/tegu/bringup/modem-sleep-series.sh b/device/tegu/bringup/modem-sleep-series.sh new file mode 100755 index 000000000..78183d3db --- /dev/null +++ b/device/tegu/bringup/modem-sleep-series.sh @@ -0,0 +1,67 @@ +#!/bin/bash +# SPDX-License-Identifier: MIT +# On the palmtop: N systemd sleeps of S seconds with the modem up, for a kernel whose +# link driver has the sleep guard on for zumapro (linux-omarchy-mobile#232; MODEM-PLAN, +# "Suspend with the modem on, with the sleep guard"). On a kernel without it one sleep +# costs the phone service until a reboot: use modem-sleep-test.sh there, attended. +# Before each request the modem is told "screen off" through sitd's socket and the +# script waits until the link driver's last line is "link down" and 5 s old; after it, +# "screen on". The modem's block inhibitor is passed with "systemctl suspend -i", an RTC +# alarm is the wake. The cellular data service is started if it isn't running, and the +# default routes are logged before each request: with Wi-Fi connected and the service +# as on main, the bearer's route is the only one left at the freeze and every request +# is refused (omarchy-mobile#750). 20 s after each request: the CP's state, sitd, its +# failed polls, Wi-Fi and new UFS lines. Stops when the CP isn't ONLINE. On the charger. +# systemd-run --unit=x-modem-sleep-series /root/modem-sleep-series.sh [N] [S] [append] +# The last step afterwards is "systemctl reboot", never reboot-to. +L=/var/log/omamobile/modem-sleep-series.log; CP=/usr/lib/omamobile-vendor/omamobile-modem-cp +N=${1:-20}; S=${2:-60}; H=1.1.1.1 +U='link startup failed|Host init failed|I/O error, dev|ufshcd_err_handler' +say() { echo "$(date +%T) $*" >> $L; sync "$L" 2>/dev/null; } +screen() { python3 - "$1" <<'PY' 2>&1 | tail -1 +import json, socket, sys +s = socket.socket(socket.AF_UNIX); s.settimeout(15); s.connect('/run/omamobile-sit/socket') +s.sendall(json.dumps({'action': 'screen-state', 'on': sys.argv[1] == 'on'}).encode() + b'\n') +print(s.recv(4096).decode().strip()[:120]) +PY +} +st() { echo "cp $($CP state 2>&1 | tail -1), sitd $(systemctl is-active omamobile-sitd), poll failures $(journalctl -b -u omamobile-sitd --no-pager | grep -c 'poll failed'), wlan0 $(cat /sys/class/net/wlan0/operstate 2>/dev/null || echo gone)"; } +idle() { # wait until the link has been down for 5 s; prints how long it took + local t0=$SECONDS last + while ((SECONDS - t0 < 60)); do + last=$(journalctl -b -k --no-pager --since "-5 s" | grep -cE "gs201-pcie-cp .*(link up|link down)") + [[ $last == 0 ]] && journalctl -b -k --no-pager -n 400 | grep -E "gs201-pcie-cp .*(link up|link down)" | tail -1 | grep -c "link down" >/dev/null && { echo "idle after $((SECONDS - t0)) s"; return 0; } + sleep 1 + done + echo "not idle in 60 s"; return 1 +} +[[ -e /sys/class/rtc/rtc0/wakealarm ]] || { echo "no RTC wake alarm on this kernel" >&2; exit 1; } +[[ ${3:-} == append ]] || : > $L +systemctl is-active -q omamobile-sit-data || { systemctl start omamobile-sit-data; sleep 8; } +say "$(uname -r) $(uname -v | cut -d' ' -f1); the data service: $(systemctl is-active omamobile-sit-data), device $(cat /run/omamobile-sit/data-interface-kept 2>/dev/null), default via $(ip route show default | awk '{print $5}' | paste -sd,); $(st)" +sleep 10 +for n in $(seq 1 "$N"); do + s0=$(cat /sys/power/suspend_stats/success); f0=$(cat /sys/power/suspend_stats/fail); u0=$(journalctl -b -k --no-pager | grep -cE "$U") + for _ in $(seq 1 30); do ip route show default | grep -c wlan0 >/dev/null && break; sleep 1; done + say "sleep $n: default via $(ip route show default | awk '{print $5}' | paste -sd,); screen off -> $(screen off); link: $(idle); link-ups in the last 20 s: $(journalctl -b -k --no-pager --since '-20 s' | grep -c 'link up: Gen3')" + rtcwake -m no -s "$S" >/dev/null 2>&1 || { say "sleep $n: rtcwake could not arm the alarm: stop"; break; } + r0=$(cat /sys/class/rtc/rtc0/since_epoch) + systemctl suspend -i + for _ in $(seq 1 300); do [[ $(cat /sys/power/suspend_stats/success) != "$s0" || $(cat /sys/power/suspend_stats/fail) != "$f0" ]] && break; sleep 0.5; done + r1=$(cat /sys/class/rtc/rtc0/since_epoch); sleep 3; rtcwake -m disable >/dev/null 2>&1 + if [[ $(cat /sys/power/suspend_stats/success) != "$s0" ]]; then + irq=$(cat /sys/power/pm_wakeup_irq 2>/dev/null) + res="entered, $((r1 - r0)) s, wake irq $irq ($(grep -E "^ *$irq:" /proc/interrupts | awk '{print $NF}'))" + elif [[ $(cat /sys/power/suspend_stats/fail) != "$f0" ]]; then res="refused (fail $(cat /sys/power/suspend_stats/fail))" + else res="no change in suspend_stats after 150 s"; fi + say "sleep $n: $res; screen on -> $(screen on)" + sleep 20 + say "sleep $n, 20 s after: $(st), new ufs lines $(( $(journalctl -b -k --no-pager | grep -cE "$U") - u0 )), sitd's last line: $(journalctl -b -u omamobile-sitd --no-pager | tail -1 | cut -c40-120)" + [[ $($CP state 2>&1 | tail -1) == ONLINE ]] || { say "the CP is not ONLINE: stop"; break; } +done +IF=$(cat /run/omamobile-sit/data-interface-kept 2>/dev/null) +[[ -n $IF ]] && say "ping over $IF: $(ping -c 5 -W 3 -I "$IF" "$H" 2>&1 | grep -o '[0-9]* received')" +say "end: $(st)" +say "kernel lines of the boot:" +journalctl -b -k --no-pager | grep -E "sleep refused|Unable to change power state|Link Active not set|Can.t set D0|BME is not set|link-down recovery|suspend_cp: abort|failed to suspend" | sed -E 's/^.{16}[^ ]+ kernel: //' | cut -c1-110 | sort | uniq -c | sort -rn | head -20 >> $L +say "done" From 93e5e61f545e09bc4770bab161343b4b21664b37 Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sat, 14 Nov 2026 05:09:08 -0644 Subject: [PATCH 2/8] device/tegu MODEM-PLAN: the same sleeps with #556's package, nothing by hand (2026-11-14) 30 of 30 requests entered with omarchy-mobile#556's phone-service package and follow_screen on: sitd told the modem the screen state itself, the sleep hook ran each time, and the route change that was #750 is part of #556 now. --- device/tegu/MODEM-PLAN.md | 29 ++++++++++++++++------- device/tegu/bringup/modem-sleep-series.sh | 2 +- 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/device/tegu/MODEM-PLAN.md b/device/tegu/MODEM-PLAN.md index 439a3415d..91a4f29b7 100644 --- a/device/tegu/MODEM-PLAN.md +++ b/device/tegu/MODEM-PLAN.md @@ -1772,9 +1772,10 @@ the alarm; otherwise the same). refused**: those 14 were all there were. At logind's sleep announcement NetworkManager takes `wlan0` down, the bearer's metric-700 default route is the only one left, whatever was talking over Wi-Fi goes out over `rmnet0` in the same - second, and the link is up when the notifier looks. omarchy-mobile#750 - (a draft) has `omamobile-sit-data` remove its route at the announcement - and put it back once Wi-Fi's is back, or after 10 s without one. Put + second, and the link is up when the notifier looks. The fix, written + as omarchy-mobile#750 and now part of #556, has `omamobile-sit-data` + remove its route at the announcement and put it back once Wi-Fi's is + back, or after 10 s without one. Put back at once after the wake it was worse than useless: 18 of 20 entered and none reached the alarm, the answers to the few seconds of traffic ending each next sleep early. @@ -1817,14 +1818,26 @@ the alarm; otherwise the same). suspend for the next userspace alarm), and the `rtc` and `alarmtimer` trace events over a run would show it. The palmtop woke every time. +- **With omarchy-mobile#556's package nothing is done by hand** (2026-11-14, + its head `6eddef24` merged with #697 for the build, `[control] + follow_screen = yes`, the same kernel): 30 requests of 60 s, 30 entered, + 24 ended by the alarm and 6 by the modem. sitd told the modem "screen + off" once, 16 s after its start with the panel dark, and the sleep hook + logged "screen off before sleep told" 30 times; the link was already + idle at 26 of the 30 requests, and the whole 46-minute boot had 97 link + power-ups, where the modem on its "screen on" reports does about 49 a + minute. The route came back behind Wi-Fi 30 of 30 times. After each + sleep the CP was ONLINE with no failed poll, and `ping -I rmnet0` got 5 + of 5 at the end. + What this leaves before the modem's inhibitor can go on the 9a: #232 in the published kernel and the zumapro line after it (its own small pull request, as #232's author asked); the hang above understood, or at least -counted on a build without #232; the screen state told to the modem by -the shell, not by hand (the 7 Pro's side of that is omarchy-mobile#556); -the route change (#750, or its equivalent in #556); and what no run here -tested: a call and a text arriving while the palmtop sleeps, a sleep of -hours, and a sleep off the charger. +counted on a build without #232; omarchy-mobile#556 merged, which +carries both the screen state and the route change, with `follow_screen` +turned on for the 9a once a call with the screen dark has been tested; +and what no run here tested: a call and a text arriving while the palmtop +sleeps, a sleep of hours, and a sleep off the charger. ### M2: the link only (modem link driver, `allow_hw=1`; risk: medium) diff --git a/device/tegu/bringup/modem-sleep-series.sh b/device/tegu/bringup/modem-sleep-series.sh index 78183d3db..68e108350 100755 --- a/device/tegu/bringup/modem-sleep-series.sh +++ b/device/tegu/bringup/modem-sleep-series.sh @@ -10,7 +10,7 @@ # alarm is the wake. The cellular data service is started if it isn't running, and the # default routes are logged before each request: with Wi-Fi connected and the service # as on main, the bearer's route is the only one left at the freeze and every request -# is refused (omarchy-mobile#750). 20 s after each request: the CP's state, sitd, its +# is refused (fixed by omarchy-mobile#556). 20 s after each request: the CP's state, sitd, its # failed polls, Wi-Fi and new UFS lines. Stops when the CP isn't ONLINE. On the charger. # systemd-run --unit=x-modem-sleep-series /root/modem-sleep-series.sh [N] [S] [append] # The last step afterwards is "systemctl reboot", never reboot-to. From d49b9d6c2ccdac9d9c9bf1dd6b6814f123666d6e Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sat, 14 Nov 2026 06:38:23 -0644 Subject: [PATCH 3/8] device/tegu MODEM-PLAN: calls and texts wake the sleeping palmtop; one stale call (2026-11-14) Four calls and two texts that arrived in a sleep each woke the palmtop and reached the phone service within a second. One call was left as ringing-in (#729), and with follow_screen on that kept the modem busy and the palmtop awake. --- device/tegu/MODEM-PLAN.md | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/device/tegu/MODEM-PLAN.md b/device/tegu/MODEM-PLAN.md index 91a4f29b7..5b90690bc 100644 --- a/device/tegu/MODEM-PLAN.md +++ b/device/tegu/MODEM-PLAN.md @@ -1830,14 +1830,32 @@ the alarm; otherwise the same). sleep the CP was ONLINE with no failed poll, and `ping -I rmnet0` got 5 of 5 at the end. +- **A call and a text wake the sleeping palmtop** (2026-11-14, the same + build and package; the owner called and texted from another phone, the + palmtop untouched and no call answered). Four calls arrived in sleeps + of 82 s to 6 min 17 s: each woke it on `cp2ap_wakeup`, and sitd logged + "Incoming call ringing" 0.1 to 0.8 s after the kernel's suspend exit. + Two texts arrived 8 s and 15 s into a sleep and were stored 0.06 and + 0.08 s after the exit. Six sleep requests that morning were called off + (three refused with the link up, three ended in noirq by CPIF with the + CP's wake line pending), and calls and texts kept arriving after each. +- **One of those calls left a stale `ringing-in` in sitd**, which is + omarchy-mobile#729 (its fix for a declined call is #754): the modem + refused the hang-up with error 2 and sitd stopped itself 12 s later. + Until then, with `follow_screen` on, sitd never told the modem "screen + off" for the call it still held, the modem stayed at 48 link power-ups + a minute, and three sleep requests in a row were called off. Not seen + again in three more calls into a sleep. + What this leaves before the modem's inhibitor can go on the 9a: #232 in the published kernel and the zumapro line after it (its own small pull request, as #232's author asked); the hang above understood, or at least counted on a build without #232; omarchy-mobile#556 merged, which carries both the screen state and the route change, with `follow_screen` -turned on for the 9a once a call with the screen dark has been tested; -and what no run here tested: a call and a text arriving while the palmtop -sleeps, a sleep of hours, and a sleep off the charger. +turned on for the 9a; a stale call ended without the owner's help +(#729), since it keeps the palmtop awake; and what no run here tested: a +call answered after it woke the palmtop, a sleep of hours, and a sleep +off the charger. ### M2: the link only (modem link driver, `allow_hw=1`; risk: medium) From 4f17cba24b83c9fc3c0ea4cf7580fbd05c09d185 Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sun, 15 Nov 2026 08:46:36 -0644 Subject: [PATCH 4/8] device/tegu MODEM-PLAN, AGENTS, modem-sleep-series.sh: the runs were outside the rule; where data was checked The plan now says plainly that the 2026-11-13 and 11-14 sleeps with the modem up ran outside the rule in AGENTS.md, with their start times, and that the rule stands until the maintainer decides otherwise. For the night and the package runs it says where mobile data was checked (at the end, with Wi-Fi the first route), adds the night's count of "Reserve doorbell interrupt" and "PCI not powered on" lines (1,205 and 1,142, with data answering at the end; the daily kernel logs the same line awake with no sleep), and lists a run in the shape that lost data on the 7 Pro under what is left. The AGENTS sentence takes the maintainer's wording, so it doesn't go stale now that #258 has merged. The series script checks data 5 s after every wake (a ping and a DNS query sent out of the bearer's interface), takes WIFI=off for runs with the bearer as the only route, refuses to start and stops when the palmtop isn't charging, and counts the doorbell lines at the end. --- AGENTS.md | 2 +- device/tegu/MODEM-PLAN.md | 59 +++++++++++++++++------ device/tegu/bringup/modem-sleep-series.sh | 56 +++++++++++++++++---- 3 files changed, 91 insertions(+), 26 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index b5fa4b823..3bf13a0a0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -196,7 +196,7 @@ Each entry gives the gotcha, then the reason or the fix. Details are in the link - **Some unattended RTC cycles wake early on the PMIC interrupt** (2 of 20 in S4, 4 of 31 in S5c, at 13-16 s instead of 30 s). Which PMIC event it is hasn't been read yet; count it, don't treat it as a failed cycle. [SUSPEND-PLAN](device/cheetah/SUSPEND-PLAN.md) S4, S5c - **Volume Up is the one key without an internal pull** (`PUD` 0x0, as on stock), and the first freeze woke 3 s in from it with nobody touching it. It didn't recur in the 51 RTC cycles of S4 and S5c, and it isn't proven to be a false edge; if spurious wakes recur, check gpa8-5's power-down pull before blaming the wake path. - **UFS ran its error handler around modem power changes** (PA LINERESET indications, recovered without a reset). Stock's DT shows no shared rail, clock or pin. M2.5 narrowed it down: power-on and a running link are clean, but every modem power-off is followed by one recovered UFS error-handler pass (a PA LINERESET, no I/O errors or resets). Stock powers the modem off only at shutdown, so the modem stays on once it is up (M2.6 is open), and a reboot with the modem up costs one such pass. Keep modem tests attended, don't power-cycle the modem, and check `dmesg | grep ufshcd_err_handler` after each step. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) M2, M2.5, M2.6 -- **Suspend with the modem on is held on the 7 Pro, and unqualified on the 9a.** On the 7 Pro it passed M6's attended runbook on 2026-11-14 (kernel-2026.11.15), but twenty auto-sleeps that night lost mobile data from the first timer wake on and it stayed down awake until a reboot, so device-cheetah 0.1.59 no longer declares it verified and the modem's sleep inhibitor holds again ([SUSPEND-PLAN](device/cheetah/SUSPEND-PLAN.md)). The 9a's one try (2026-10-27) also had NFC polling, which alone resets the 9a in s2idle, so the modem's part was unknown. With the sleep fixes (2026-11-09, both link drivers) it neither hung nor reset: CPIF refuses most sleeps (`suspend_cp: abort suspend`, -16, while the CP holds `cp2ap_wakeup`), and the three that were entered woke on the modem within seconds. But one sleep attempt, entered or refused, leaves CPIF unable to reach the CP (`Can't set D0 state`, its command register reading 0xffff) and the phone service timing out until a reboot, while `omamobile-sit status` goes on showing the last state read. So with the modem up nothing passes or goes round its sleep inhibitor (`systemctl suspend -i`, `rtcwake -m`, a write to `/sys/power/state`) except an attended test on the charger that ends in a reboot; on the 7 Pro that test is the maintainer's. On a 9a test kernel with [linux-omarchy-mobile#232](https://github.com/derekross/linux-omarchy-mobile/pull/232) and its sleep guard turned on for zumapro ([#258](https://github.com/derekross/linux-omarchy-mobile/pull/258), open), 299 of 300 sleeps with the modem registered were entered and woke with the phone service kept, and `ping -I rmnet0` answered 5 of 5 at the end (2026-11-13); that kernel also hung once at a link power-up while awake, and #258 is in no release yet, so on the 9a the rule is unchanged. CPIF's PM paths are wake-armed in code; on the 7 Pro M6's runbook passed and its long check did not. Keep modem tests on the charger, so upower's critical action doesn't suspend the phone mid-test. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) 6.3, [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) "Suspend with the modem on" +- **Suspend with the modem on is held on the 7 Pro, and unqualified on the 9a.** On the 7 Pro it passed M6's attended runbook on 2026-11-14 (kernel-2026.11.15), but twenty auto-sleeps that night lost mobile data from the first timer wake on and it stayed down awake until a reboot, so device-cheetah 0.1.59 no longer declares it verified and the modem's sleep inhibitor holds again ([SUSPEND-PLAN](device/cheetah/SUSPEND-PLAN.md)). The 9a's one try (2026-10-27) also had NFC polling, which alone resets the 9a in s2idle, so the modem's part was unknown. With the sleep fixes (2026-11-09, both link drivers) it neither hung nor reset: CPIF refuses most sleeps (`suspend_cp: abort suspend`, -16, while the CP holds `cp2ap_wakeup`), and the three that were entered woke on the modem within seconds. But one sleep attempt, entered or refused, leaves CPIF unable to reach the CP (`Can't set D0 state`, its command register reading 0xffff) and the phone service timing out until a reboot, while `omamobile-sit status` goes on showing the last state read. So with the modem up nothing passes or goes round its sleep inhibitor (`systemctl suspend -i`, `rtcwake -m`, a write to `/sys/power/state`) except an attended test on the charger that ends in a reboot; on the 7 Pro that test is the maintainer's. On a test kernel with the sleep guard turned on for zumapro ([linux-omarchy-mobile#258](https://github.com/derekross/linux-omarchy-mobile/pull/258)), 299 of 300 sleeps with the modem registered were entered and woke with the phone service kept (2026-11-13; Wi-Fi connected, mobile data checked at the end), and a refusal cost nothing; that kernel also hung once at a link power-up while awake, so on the 9a the rule is unchanged until a release carries the guard and the 9a's own declaration is decided. CPIF's PM paths are wake-armed in code; on the 7 Pro M6's runbook passed and its long check did not. Keep modem tests on the charger, so upower's critical action doesn't suspend the phone mid-test. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) 6.3, [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) "Suspend with the modem on" - **Never `systemctl restart omamobile-modem` (or stop and start it) to recover a crashed modem.** An in-place restart left raven's CP in BOOTING, and a bring-up that never reaches ONLINE sets a latch that makes the next boot skip the modem. Reboot instead. If a boot skipped it, remove the latch (`rm /var/lib/omamobile/modem-bringup-started`) and `systemctl start omamobile-modem`; `/usr/lib/omamobile-vendor/omamobile-modem-start --retry` also clears it, but run by hand it brings the modem up outside the unit (a stray `rfsd` in your shell's session). From `omamobile-modem` 0.1.13 there is a second marker, `/var/lib/omamobile/modem-up`, written at ONLINE and removed by the unit's clean stop: a boot that hung or reset with the modem up leaves it. The next boot logs "ended without a clean stop" and, from 0.1.19, starts the modem all the same (0.1.13 to 0.1.18 skipped it for that boot); a second unclean end in a row refuses "(held)" until the markers are removed the same way (cheetah [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) M4.6). On the 7 Pro the stack starts at boot by default since device-cheetah 0.1.43 ([MODEM-PLAN](device/cheetah/MODEM-PLAN.md) M4.6b); to keep it off from the next boot, `systemctl mask omamobile-modem-boot.target` (`unmask` turns it back on; `disable` doesn't remove device-cheetah's vendor wants link); it also holds a sleep inhibitor until the reboot. [raven MODEM-PLAN](device/raven/MODEM-PLAN.md) D4. The same on the 9a: after the modem profile has stopped, starting it again in the same boot reset the palmtop silently, a watchdog reset with nothing in pstore (the one try was in the rfsd stage). [tegu MODEM-PLAN](device/tegu/MODEM-PLAN.md) G3. **From `omamobile-modem` 0.1.10, `omamobile-vendor-chroot modem check` refuses once the profile has started this boot** ("the modem profile already started this boot"), so it is no longer a post-install health check while the modem is up: it is read-only and safe to run, but a refusal means the profile ran, not that anything is wrong. After a reboot it says "preconditions met" again. - **Linux never writes the modem's NV partitions.** The modem chroot profile (cbd, rfsd) refuses to start unless the NV partitions read `ro=1` and none is mounted, and works on checksum-verified RAM copies only. What the modem writes is lost at stop, except its writes to `efs/nv_normal.bin` (expected to hold the carrier profile, network and IMS data; the runbook confirms it) when the NV overlay is on (section 1): those are captured every 30 minutes and at the unit's stop, after the CP is off, and laid over the verified copy at the next start. `reboot-to` (and `bin/omamobile push`) skips that stop, so stop the modem first (section 2) or run `omamobile-modem nv-overlay capture`. Don't give the chroot a real block device. [MODEM-PLAN](device/cheetah/MODEM-PLAN.md) "NV overlay" diff --git a/device/tegu/MODEM-PLAN.md b/device/tegu/MODEM-PLAN.md index 5b90690bc..3d7d38750 100644 --- a/device/tegu/MODEM-PLAN.md +++ b/device/tegu/MODEM-PLAN.md @@ -1733,9 +1733,20 @@ The 9a has nothing that tells the modem the screen state, so each test sent "screen off" through sitd's socket before a request and "screen on" after, waited for the link to be down for 5 s, armed an RTC alarm and passed the modem's inhibitor with `systemctl suspend -i`. On the charger, -cable in, home LTE with IMS voice and SMS. The owner was not at the -palmtop for these runs; from 2026-11-13 he has told us to run sleep tests -with the modem up without him, and every boot ended in `systemctl reboot`. +cable in, home LTE with IMS voice and SMS, and every boot ended in +`systemctl reboot`. + +**These runs were outside the rule as AGENTS.md has it.** With the modem +up, every request here went past its sleep inhibitor (`systemctl suspend +-i`) with nobody at the palmtop, on a kernel that was new that day and had +not had an attended run first. Their boots began, by the palmtop's journal +(MST): on 2026-11-13 the five comparison boots on the daily kernel from +15:10 to 16:04, the refusals and the three 45 s sleeps at 16:36, the twenty +at 17:02, the boot that hung at 17:51 and the night at 19:28; on 11-14 the +boots with #556's package and the calls at 04:09, 05:37, 06:29 and 06:56. +Until the maintainer decides otherwise, a sleep with the modem up on the 9a +runs as AGENTS.md says: attended, on the charger, ending in a reboot, and a +new kernel gets its attended run first. The series script is `device/tegu/bringup/modem-sleep-series.sh` (the copy that ran logged to another file and lacked the check that `rtcwake` armed the alarm; otherwise the same). @@ -1767,7 +1778,9 @@ the alarm; otherwise the same). change power state`, `Link Active`, `BME` or UFS error-handler line, the root port in D0 and the endpoint in D3hot at the end, and both `systemctl reboot`s stopped the modem cleanly (`CP OFFLINE`, the marker - removed), back in 25 s. + removed), back in 25 s. Mobile data was checked after every sleep here, + but with the data service stopped, so not in the shape that lost it on + the 7 Pro (below). - **With Wi-Fi connected and the data service as on main, every request is refused**: those 14 were all there were. At logind's sleep announcement NetworkManager takes `wlan0` down, the bearer's metric-700 default route is the only one @@ -1791,7 +1804,16 @@ the alarm; otherwise the same). no bad line in the kernel log of the boot (the list above, plus oops, warning and RCU stall). Entry took 0.52 to 0.68 s to the noirq stage and resume 0.72 to 0.87 s by the kernel's stamps. The reboot after it - stopped the modem cleanly. + stopped the modem cleanly. **Where data was checked:** once, at the end. + At every wake Wi-Fi was the first route and the bearer's was behind it, + so the night doesn't show whether data over the bearer answered just + after each wake, which is what failed on the 7 Pro. The boot's kernel + log, read again on 11-15, has 1,205 `Reserve doorbell interrupt` lines, + 1,142 of them `PCI not powered on`, the line the 7 Pro logged while its + data was down; here data answered at the end. The daily kernel logs the + same line awake with no sleep at all (23 to 97 in four boots of 7 to 44 + minutes on 11-13 and 11-14), so on the 9a the line alone doesn't mark + lost data. - **One hang, awake, on this build, not understood.** The evening before the night run, a boot that had run 52 requests (49 entered, 3 refused) stopped 20.4 s after a sleep had ended normally: the console's last @@ -1828,7 +1850,8 @@ the alarm; otherwise the same). power-ups, where the modem on its "screen on" reports does about 49 a minute. The route came back behind Wi-Fi 30 of 30 times. After each sleep the CP was ONLINE with no failed poll, and `ping -I rmnet0` got 5 - of 5 at the end. + of 5 at the end, with Wi-Fi the first route; data wasn't checked after + each wake. - **A call and a text wake the sleeping palmtop** (2026-11-14, the same build and package; the owner called and texted from another phone, the @@ -1847,15 +1870,19 @@ the alarm; otherwise the same). a minute, and three sleep requests in a row were called off. Not seen again in three more calls into a sleep. -What this leaves before the modem's inhibitor can go on the 9a: #232 in -the published kernel and the zumapro line after it (its own small pull -request, as #232's author asked); the hang above understood, or at least -counted on a build without #232; omarchy-mobile#556 merged, which -carries both the screen state and the route change, with `follow_screen` -turned on for the 9a; a stale call ended without the owner's help -(#729), since it keeps the palmtop awake; and what no run here tested: a -call answered after it woke the palmtop, a sleep of hours, and a sleep -off the charger. +What this leaves before the modem's inhibitor can go on the 9a: a release +kernel with the zumapro guard (#232 is in kernel-2026.11.15, #258 merged on +2026-11-15 and is in no release yet); the hang above understood, or at +least counted on a build without #232; `follow_screen` turned on for the +9a (#556 is in main since #791); a stale call ended without the owner's +help (#729), since it keeps the palmtop awake; and what no run here +tested: **mobile data in the shape that lost it on the 7 Pro** (#799): +Wi-Fi off for the sleeps so the bearer is the only route, RTC wakes, a +ping and a DNS lookup over `rmnet0` a few seconds after every timer wake, +and the boot's count of `Reserve doorbell interrupt` and `PCI not powered +on`. `modem-sleep-series.sh` with `WIFI=off` does all of that, checks data +5 s after every wake, and refuses to start, and stops, off the charger. +Then a call answered after it woke the palmtop, and a sleep of hours. ### M2: the link only (modem link driver, `allow_hw=1`; risk: medium) @@ -2046,7 +2073,7 @@ Preconditions, as SLEEP-PLAN's: | The bridge for apps | `omamobile-mmd` not installed | not part of this test (T2 follows a passing run) | | The link watcher | `omamobile-modem` 0.1.9 () keeps one open file on the boot device, which retires the old open-and-close watcher that woke the CP link every half second; 0.1.10 () adds the once-per-boot refusal | with the CP ONLINE the link idles and cycles on its own; on 2026-10-29 the palmtop hung eight times in a morning with the modem up for hours, the early ones tied to the watcher's open and close of the boot device (the console logs end on CPIF's `cp2ap_wakeup` wake toggles; `logs/tegu-review-20261029/hangs/`). The run uses 0.1.10 (SIM-1). Keep the session short (under an hour) and attended; `cpif-keep-link.ko` (G3-L1 step 1, untested) is the candidate fix, not part of this run | | Call audio | the AoC card is `google,tegu-aoc-snd-card` with its EP PCMs, but `omamobile-speaker-protection` is inactive and the amplifiers are TAS2572, which `voice_audio.py`'s earpiece checks (CS35L41 controls) don't know | **no audio routing exists on the 9a: calls are not part of this test.** `[audio] card = none` stays; a signalling-only call (cheetah's S10) is a later, separate step | -| Suspend | unqualified with the modem on (section "Suspend with the modem on"); the start script's inhibitor blocks `systemctl suspend` for the boot. On a test kernel with linux-omarchy-mobile#232 and the sleep guard on for zumapro, 299 of 300 sleeps were entered and woke with the phone service kept (2026-11-13) | charger in for the whole test | +| Suspend | unqualified with the modem on (section "Suspend with the modem on"); the start script's inhibitor blocks `systemctl suspend` for the boot. On a test kernel with linux-omarchy-mobile#232 and the sleep guard on for zumapro, 299 of 300 sleeps were entered and woke with the phone service kept (2026-11-13; Wi-Fi connected, mobile data checked at the end) | charger in for the whole test | | Identifiers | the ICCID (printed on the card), IMSI, IMEI, phone number, the carrier's name, its APN, IP addresses and message text never go into the repository, a log excerpt or a screenshot | sitd logs states and codes only; before quoting a log: `grep -nE '[0-9]{7,}|\+[0-9]|[0-9a-f]{2}(:[0-9a-f]{2}){5}|@' logs/` must show nothing you would quote | **What to expect, honestly:** SIM detection likely (GET_SIM_STATUS has diff --git a/device/tegu/bringup/modem-sleep-series.sh b/device/tegu/bringup/modem-sleep-series.sh index 68e108350..101a1a096 100755 --- a/device/tegu/bringup/modem-sleep-series.sh +++ b/device/tegu/bringup/modem-sleep-series.sh @@ -10,12 +10,18 @@ # alarm is the wake. The cellular data service is started if it isn't running, and the # default routes are logged before each request: with Wi-Fi connected and the service # as on main, the bearer's route is the only one left at the freeze and every request -# is refused (fixed by omarchy-mobile#556). 20 s after each request: the CP's state, sitd, its -# failed polls, Wi-Fi and new UFS lines. Stops when the CP isn't ONLINE. On the charger. -# systemd-run --unit=x-modem-sleep-series /root/modem-sleep-series.sh [N] [S] [append] +# is refused (fixed by omarchy-mobile#556). 5 s after each wake: a ping and a DNS lookup over the +# bearer's interface (rmnet0), so mobile data is checked after every sleep, not only at the +# end; 20 s after: the CP's state, sitd, its failed polls, Wi-Fi and new UFS lines. Stops +# when the CP isn't ONLINE. WIFI=off turns the Wi-Fi radio off for the run (back on at the +# end), so mobile data is the only route at each wake, the shape that lost data on the 7 Pro +# (linux-omarchy-mobile#258). Refuses to start, and stops, unless the palmtop is charging. +# The end counts the boot's "Reserve doorbell interrupt" and "PCI not powered on" lines. +# Attended, on the charger, and the boot ends in a reboot (AGENTS.md, the 9a sleep entry): +# [WIFI=off] systemd-run --unit=x-modem-sleep-series /root/modem-sleep-series.sh [N] [S] [append] # The last step afterwards is "systemctl reboot", never reboot-to. L=/var/log/omamobile/modem-sleep-series.log; CP=/usr/lib/omamobile-vendor/omamobile-modem-cp -N=${1:-20}; S=${2:-60}; H=1.1.1.1 +N=${1:-20}; S=${2:-60}; H=1.1.1.1; Q=example.com; BAT=/sys/class/power_supply/max77779-battery U='link startup failed|Host init failed|I/O error, dev|ufshcd_err_handler' say() { echo "$(date +%T) $*" >> $L; sync "$L" 2>/dev/null; } screen() { python3 - "$1" <<'PY' 2>&1 | tail -1 @@ -35,14 +41,44 @@ idle() { # wait until the link has been down for 5 s; prints how long it took done echo "not idle in 60 s"; return 1 } +# The battery's average current is positive while it charges (the 9a lists no charger supply). +charging() { (( $(cat $BAT/current_avg 2>/dev/null || echo 0) > 0 )); } +data() { # a ping and a DNS lookup over the bearer's interface + [[ -n $IF ]] || { echo "no bearer interface"; return; } + echo "ping $(ping -c 3 -W 3 -I "$IF" "$H" 2>&1 | grep -o '[0-9]* received'), dns $(dns)" +} +dns() { # one A query for $Q to the bearer's first DNS server, sent out of $IF itself: with Wi-Fi + # connected the bearer's link is not resolved's default route, so resolvectl won't use it + local srv; srv=$(resolvectl dns "$IF" 2>/dev/null | awk '{print $4}') + [[ -n $srv ]] || { echo "no server"; return; } + python3 - "$IF" "$srv" "$Q" <<'PY' 2>&1 | tail -1 +import socket, struct, sys +dev, srv, name = sys.argv[1:4] +fam = socket.AF_INET6 if ':' in srv else socket.AF_INET +q = struct.pack('>HHHHHH', 0x5a5a, 0x0100, 1, 0, 0, 0) +q += b''.join(bytes([len(x)]) + x.encode() for x in name.split('.')) + b'\0' + struct.pack('>HH', 1, 1) +s = socket.socket(fam, socket.SOCK_DGRAM); s.settimeout(4) +s.setsockopt(socket.SOL_SOCKET, socket.SO_BINDTODEVICE, dev.encode()) +try: + s.sendto(q, (srv, 53)); r = s.recv(512) + ok = r[:2] == q[:2] and r[3] & 0x0f == 0 and struct.unpack('>H', r[6:8])[0] > 0 + print('ok' if ok else 'bad answer') +except OSError as e: + print('fail (%s)' % (e.strerror or 'timeout')) +PY +} [[ -e /sys/class/rtc/rtc0/wakealarm ]] || { echo "no RTC wake alarm on this kernel" >&2; exit 1; } +charging || { echo "not charging: run this on the charger" >&2; exit 1; } [[ ${3:-} == append ]] || : > $L systemctl is-active -q omamobile-sit-data || { systemctl start omamobile-sit-data; sleep 8; } -say "$(uname -r) $(uname -v | cut -d' ' -f1); the data service: $(systemctl is-active omamobile-sit-data), device $(cat /run/omamobile-sit/data-interface-kept 2>/dev/null), default via $(ip route show default | awk '{print $5}' | paste -sd,); $(st)" +[[ ${WIFI:-} == off ]] && { nmcli radio wifi off; sleep 10; } +IF=$(cat /run/omamobile-sit/data-interface /run/omamobile-sit/data-interface-kept 2>/dev/null | head -1) +say "$(uname -r) $(uname -v | cut -d' ' -f1); the data service: $(systemctl is-active omamobile-sit-data), device ${IF:-none}, Wi-Fi radio $(nmcli radio wifi), default via $(ip route show default | awk '{print $5}' | paste -sd,); $(st); data: $(data)" sleep 10 for n in $(seq 1 "$N"); do s0=$(cat /sys/power/suspend_stats/success); f0=$(cat /sys/power/suspend_stats/fail); u0=$(journalctl -b -k --no-pager | grep -cE "$U") - for _ in $(seq 1 30); do ip route show default | grep -c wlan0 >/dev/null && break; sleep 1; done + charging || { say "sleep $n: not charging any more: stop"; break; } + [[ ${WIFI:-} == off ]] || for _ in $(seq 1 30); do ip route show default | grep -c wlan0 >/dev/null && break; sleep 1; done say "sleep $n: default via $(ip route show default | awk '{print $5}' | paste -sd,); screen off -> $(screen off); link: $(idle); link-ups in the last 20 s: $(journalctl -b -k --no-pager --since '-20 s' | grep -c 'link up: Gen3')" rtcwake -m no -s "$S" >/dev/null 2>&1 || { say "sleep $n: rtcwake could not arm the alarm: stop"; break; } r0=$(cat /sys/class/rtc/rtc0/since_epoch) @@ -55,13 +91,15 @@ for n in $(seq 1 "$N"); do elif [[ $(cat /sys/power/suspend_stats/fail) != "$f0" ]]; then res="refused (fail $(cat /sys/power/suspend_stats/fail))" else res="no change in suspend_stats after 150 s"; fi say "sleep $n: $res; screen on -> $(screen on)" - sleep 20 + sleep 5 + say "sleep $n, 5 s after: default via $(ip route show default | awk '{print $5}' | paste -sd,); data: $(data)" + sleep 15 say "sleep $n, 20 s after: $(st), new ufs lines $(( $(journalctl -b -k --no-pager | grep -cE "$U") - u0 )), sitd's last line: $(journalctl -b -u omamobile-sitd --no-pager | tail -1 | cut -c40-120)" [[ $($CP state 2>&1 | tail -1) == ONLINE ]] || { say "the CP is not ONLINE: stop"; break; } done -IF=$(cat /run/omamobile-sit/data-interface-kept 2>/dev/null) [[ -n $IF ]] && say "ping over $IF: $(ping -c 5 -W 3 -I "$IF" "$H" 2>&1 | grep -o '[0-9]* received')" -say "end: $(st)" +[[ ${WIFI:-} == off ]] && nmcli radio wifi on +say "end: $(st); in the boot's kernel log: Reserve doorbell interrupt $(journalctl -b -k --no-pager | grep -c 'Reserve doorbell interrupt'), PCI not powered on $(journalctl -b -k --no-pager | grep -c 'PCI not powered on')" say "kernel lines of the boot:" journalctl -b -k --no-pager | grep -E "sleep refused|Unable to change power state|Link Active not set|Can.t set D0|BME is not set|link-down recovery|suspend_cp: abort|failed to suspend" | sed -E 's/^.{16}[^ ]+ kernel: //' | cut -c1-110 | sort | uniq -c | sort -rn | head -20 >> $L say "done" From bb41d09dd85ecbf130e87fb0d189eca6d4e693cf Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sun, 15 Nov 2026 09:21:26 -0644 Subject: [PATCH 5/8] device/tegu MODEM-PLAN: the owner's palmtop runs sleeps with the modem up under #809 The maintainer agreed on #809 that on the owner's palmtop these runs may be unattended, on the charger, each boot ending in a reboot and a new kernel's first run short, so the section says which rule applies from 2026-11-15. --- device/tegu/MODEM-PLAN.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/device/tegu/MODEM-PLAN.md b/device/tegu/MODEM-PLAN.md index 3d7d38750..5a7596c65 100644 --- a/device/tegu/MODEM-PLAN.md +++ b/device/tegu/MODEM-PLAN.md @@ -1744,9 +1744,12 @@ not had an attended run first. Their boots began, by the palmtop's journal 15:10 to 16:04, the refusals and the three 45 s sleeps at 16:36, the twenty at 17:02, the boot that hung at 17:51 and the night at 19:28; on 11-14 the boots with #556's package and the calls at 04:09, 05:37, 06:29 and 06:56. -Until the maintainer decides otherwise, a sleep with the modem up on the 9a -runs as AGENTS.md says: attended, on the charger, ending in a reboot, and a -new kernel gets its attended run first. +From 2026-11-15 the owner's palmtop runs these under the maintainer's +decision in [#809](https://github.com/derekross/omarchy-mobile/issues/809): +unattended is allowed there, on the charger, every such boot ending in +`systemctl reboot`, a new kernel's first run at most 10 sleeps with its +result posted before a longer one, and the record saying which rule each +run ran under. Every other 9a keeps the rule as AGENTS.md has it. The series script is `device/tegu/bringup/modem-sleep-series.sh` (the copy that ran logged to another file and lacked the check that `rtcwake` armed the alarm; otherwise the same). From 1c92d4272d41287fa2b4b53a20b71ee9330d8273 Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sun, 15 Nov 2026 09:53:00 -0644 Subject: [PATCH 6/8] device/tegu bringup: modem-sleep-series.sh logs CPIF's own state around every wake One line before each request and around each data check: the FMT and raw queues, the wake lines, pcie_powered_on and reserve_doorbell, each pktproc queue's fore/rear and pass, and the mif_cp2ap_msg count, read as the 7 Pro's M6-D recorder reads them, so a 9a wake that loses data says which path stopped. --- device/tegu/bringup/modem-sleep-series.sh | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/device/tegu/bringup/modem-sleep-series.sh b/device/tegu/bringup/modem-sleep-series.sh index 101a1a096..e7d6d146e 100755 --- a/device/tegu/bringup/modem-sleep-series.sh +++ b/device/tegu/bringup/modem-sleep-series.sh @@ -17,6 +17,11 @@ # end), so mobile data is the only route at each wake, the shape that lost data on the 7 Pro # (linux-omarchy-mobile#258). Refuses to start, and stops, unless the palmtop is charging. # The end counts the boot's "Reserve doorbell interrupt" and "PCI not powered on" lines. +# Before each request and with each data check, one line of CPIF's own state, as the 7 Pro's +# M6-D recorder reads it (device/cheetah/bringup/m6-data-capture.sh): the FMT and raw queues' +# heads and tails, the wake lines, pcie_powered_on and reserve_doorbell, each pktproc queue's +# fore/rear and pass, and the mif_cp2ap_msg count, so a failed wake says whether uplink, +# downlink or the control channel stopped. # Attended, on the charger, and the boot ends in a reboot (AGENTS.md, the 9a sleep entry): # [WIFI=off] systemd-run --unit=x-modem-sleep-series /root/modem-sleep-series.sh [N] [S] [append] # The last step afterwards is "systemctl reboot", never reboot-to. @@ -47,6 +52,16 @@ data() { # a ping and a DNS lookup over the bearer's interface [[ -n $IF ]] || { echo "no bearer interface"; return; } echo "ping $(ping -c 3 -W 3 -I "$IF" "$H" 2>&1 | grep -o '[0-9]* received'), dns $(dns)" } +cpif() { # one line of CPIF's state (read-only sysfs and /proc/interrupts) + local c=/sys/devices/platform/cpif + [[ -r $c/legacy_queues ]] || { echo "no cpif state"; return; } + echo "fmt $(grep -oE '^FMT: tx head:[0-9]+ tail:[0-9]+|rx head:[0-9]+ tail:[0-9]+' $c/legacy_queues | head -2 | grep -oE '[0-9]+' | paste -sd/)" \ + "raw $(grep '^NORM_RAW' $c/legacy_queues | grep -oE '(head|tail):[0-9]+' | cut -d: -f2 | paste -sd/)" \ + "$(grep -oE '(cp2ap_wakeup|ap2cp_wakeup|pcie_powered_on|reserve_doorbell):[0-9]+' $c/legacy_queues | paste -sd' ')" \ + "dl $(awk '/fore\/rear\/done/ {f=$NF} /^ *pass:/ {printf "%s,%s ", f, $1}' $c/pktproc/status | sed 's/pass://g; s|fore/rear/done:||g')" \ + "ul $(awk '/fore\/rear:/ {f=$NF} /^ *pass:/ {printf "%s,%s ", f, $1}' $c/pktproc_ul/status | sed 's/pass://g; s|fore/rear:||g')" \ + "msg $(awk '/mif_cp2ap_msg/ {s=0; for (i = 2; i <= 9; i++) s += $i; print s}' /proc/interrupts)" +} dns() { # one A query for $Q to the bearer's first DNS server, sent out of $IF itself: with Wi-Fi # connected the bearer's link is not resolved's default route, so resolvectl won't use it local srv; srv=$(resolvectl dns "$IF" 2>/dev/null | awk '{print $4}') @@ -80,6 +95,7 @@ for n in $(seq 1 "$N"); do charging || { say "sleep $n: not charging any more: stop"; break; } [[ ${WIFI:-} == off ]] || for _ in $(seq 1 30); do ip route show default | grep -c wlan0 >/dev/null && break; sleep 1; done say "sleep $n: default via $(ip route show default | awk '{print $5}' | paste -sd,); screen off -> $(screen off); link: $(idle); link-ups in the last 20 s: $(journalctl -b -k --no-pager --since '-20 s' | grep -c 'link up: Gen3')" + say "sleep $n, before: cpif $(cpif)" rtcwake -m no -s "$S" >/dev/null 2>&1 || { say "sleep $n: rtcwake could not arm the alarm: stop"; break; } r0=$(cat /sys/class/rtc/rtc0/since_epoch) systemctl suspend -i @@ -92,7 +108,7 @@ for n in $(seq 1 "$N"); do else res="no change in suspend_stats after 150 s"; fi say "sleep $n: $res; screen on -> $(screen on)" sleep 5 - say "sleep $n, 5 s after: default via $(ip route show default | awk '{print $5}' | paste -sd,); data: $(data)" + say "sleep $n, 5 s after: default via $(ip route show default | awk '{print $5}' | paste -sd,); cpif $(cpif); data: $(data); cpif after the data check: $(cpif)" sleep 15 say "sleep $n, 20 s after: $(st), new ufs lines $(( $(journalctl -b -k --no-pager | grep -cE "$U") - u0 )), sitd's last line: $(journalctl -b -u omamobile-sitd --no-pager | tail -1 | cut -c40-120)" [[ $($CP state 2>&1 | tail -1) == ONLINE ]] || { say "the CP is not ONLINE: stop"; break; } From ad6ad0bc81d8acfa35b483e7e498e32eab620d1f Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sun, 15 Nov 2026 20:19:21 -0644 Subject: [PATCH 7/8] device/tegu: the modem sleep series stops on a UFS line or an oops, and the plan has the 11-14 night and #809's terms modem-sleep-series.sh now stops itself as #809 says: on a new UFS error line or an oops, both read from dmesg so the check works with storage gone, besides the CP and the charger. N defaults to 10, a trap turns Wi-Fi back on however the script ends, and each data check after a wake has sitd's data-call lines from that sleep beside it. The header's 7 Pro failure is #799. MODEM-PLAN lists the 11-14 to 11-15 night on push1-g716f6d8ef355 among the runs outside the rule, with its result (37 of 37 woke, rmnet0 0 of 5 with the bearer reported up, sitd 0.3.37), quotes all of #809's terms, and asks for data over rmnet0 after every wake with sitd's lines, not the doorbell counts. --- device/tegu/MODEM-PLAN.md | 39 +++++++++++++++++------ device/tegu/bringup/modem-sleep-series.sh | 34 +++++++++++++------- 2 files changed, 51 insertions(+), 22 deletions(-) diff --git a/device/tegu/MODEM-PLAN.md b/device/tegu/MODEM-PLAN.md index 5a7596c65..de0f61e70 100644 --- a/device/tegu/MODEM-PLAN.md +++ b/device/tegu/MODEM-PLAN.md @@ -1743,12 +1743,29 @@ not had an attended run first. Their boots began, by the palmtop's journal (MST): on 2026-11-13 the five comparison boots on the daily kernel from 15:10 to 16:04, the refusals and the three 45 s sleeps at 16:36, the twenty at 17:02, the boot that hung at 17:51 and the night at 19:28; on 11-14 the -boots with #556's package and the calls at 04:09, 05:37, 06:29 and 06:56. +boots with #556's package and the calls at 04:09, 05:37, 06:29 and 06:56, +and the boot from 10:31 on 11-14 to 07:23 on 11-15 on +`push1-g716f6d8ef355` (#232, #258's line and #247), with the modem up all +boot, the shell's auto-sleep doing the sleeping and the cable out for +10.8 h overnight. That night 37 of 37 sleep requests woke (24 on +`cp2ap_wakeup`, 11 on the alarm, two by the owner in the morning), with no +hang, UFS line or oops, and in the morning the CP was ONLINE and +registered with IMS, but a ping over `rmnet0` got 0 of 5 back with the +bearer reported up. `rmnet0` had carried almost nothing that boot and sitd +was 0.3.37, without #817's reconnect, so the night can't say whether the +internet bearer was dropped and not set up again, as on the 7 Pro (#817). +Beside it, the Wi-Fi night below ended with `rmnet0` answering 5 of 5. From 2026-11-15 the owner's palmtop runs these under the maintainer's -decision in [#809](https://github.com/derekross/omarchy-mobile/issues/809): -unattended is allowed there, on the charger, every such boot ending in -`systemctl reboot`, a new kernel's first run at most 10 sleeps with its -result posted before a longer one, and the record saying which rule each +decision in [#809](https://github.com/derekross/omarchy-mobile/issues/809), +whose terms, as AGENTS.md has them, are: W4HO-ham's modem-up sleep series +may run unattended, a new kernel's first among them, on the charger +(`modem-sleep-series.sh` refuses and stops off it); every boot that had a +sleep with the modem up ends in `systemctl reboot`, never `reboot-to`; a +new kernel's first run is at most 10 sleeps and its result is posted +before a longer one; the series stops itself on the CP not ONLINE, a UFS +error line or an oops, and a hang is left to the cluster watchdog or the +owner; the NV partitions stay read-only and are checked against the +backup after any boot that hung or reset; the record says which rule each run ran under. Every other 9a keeps the rule as AGENTS.md has it. The series script is `device/tegu/bringup/modem-sleep-series.sh` (the copy that ran logged to another file and lacked the check that `rtcwake` armed @@ -1880,11 +1897,13 @@ least counted on a build without #232; `follow_screen` turned on for the 9a (#556 is in main since #791); a stale call ended without the owner's help (#729), since it keeps the palmtop awake; and what no run here tested: **mobile data in the shape that lost it on the 7 Pro** (#799): -Wi-Fi off for the sleeps so the bearer is the only route, RTC wakes, a -ping and a DNS lookup over `rmnet0` a few seconds after every timer wake, -and the boot's count of `Reserve doorbell interrupt` and `PCI not powered -on`. `modem-sleep-series.sh` with `WIFI=off` does all of that, checks data -5 s after every wake, and refuses to start, and stops, off the charger. +Wi-Fi off for the sleeps so the bearer is the only route, RTC wakes, and +data over `rmnet0` (a ping and a DNS lookup) a few seconds after every +wake, with sitd's data-call lines from that sleep next to it, so a drop +and its reconnect (#817) show up wake by wake. `modem-sleep-series.sh` +with `WIFI=off` does all of that, and stops itself as #809 asks: off the +charger, on the CP not ONLINE, and on a UFS error line or an oops in +`dmesg`. Then a call answered after it woke the palmtop, and a sleep of hours. ### M2: the link only (modem link driver, `allow_hw=1`; risk: medium) diff --git a/device/tegu/bringup/modem-sleep-series.sh b/device/tegu/bringup/modem-sleep-series.sh index e7d6d146e..09965a1cd 100755 --- a/device/tegu/bringup/modem-sleep-series.sh +++ b/device/tegu/bringup/modem-sleep-series.sh @@ -12,22 +12,31 @@ # as on main, the bearer's route is the only one left at the freeze and every request # is refused (fixed by omarchy-mobile#556). 5 s after each wake: a ping and a DNS lookup over the # bearer's interface (rmnet0), so mobile data is checked after every sleep, not only at the -# end; 20 s after: the CP's state, sitd, its failed polls, Wi-Fi and new UFS lines. Stops -# when the CP isn't ONLINE. WIFI=off turns the Wi-Fi radio off for the run (back on at the -# end), so mobile data is the only route at each wake, the shape that lost data on the 7 Pro -# (linux-omarchy-mobile#258). Refuses to start, and stops, unless the palmtop is charging. -# The end counts the boot's "Reserve doorbell interrupt" and "PCI not powered on" lines. +# end, with sitd's data-call lines from that sleep ("Data-call list changed", "Internet +# bearer set up again", #817) beside it; 20 s after: the CP's state, sitd, its failed polls +# and Wi-Fi. WIFI=off turns the Wi-Fi radio off for the run (back on when the script ends, +# however it ends), so mobile data is the only route at each wake, the shape that lost data +# on the 7 Pro (omarchy-mobile#799, SUSPEND-PLAN M6-D). Stops itself, as omarchy-mobile#809 +# asks: off the charger (and refuses to start there), on the CP not ONLINE, and on a new UFS +# error line or an oops, both read from dmesg, which is in memory, so the check still works +# if storage is what failed. The end counts the boot's "Reserve doorbell interrupt" and +# "PCI not powered on" lines. # Before each request and with each data check, one line of CPIF's own state, as the 7 Pro's # M6-D recorder reads it (device/cheetah/bringup/m6-data-capture.sh): the FMT and raw queues' # heads and tails, the wake lines, pcie_powered_on and reserve_doorbell, each pktproc queue's # fore/rear and pass, and the mif_cp2ap_msg count, so a failed wake says whether uplink, # downlink or the control channel stopped. -# Attended, on the charger, and the boot ends in a reboot (AGENTS.md, the 9a sleep entry): +# On W4HO-ham's palmtop under #809's terms (AGENTS.md, the 9a's modem-up entry), everywhere +# else attended; on the charger either way, and the boot ends in a reboot. N defaults to 10, +# the most a new kernel's first run may have: # [WIFI=off] systemd-run --unit=x-modem-sleep-series /root/modem-sleep-series.sh [N] [S] [append] # The last step afterwards is "systemctl reboot", never reboot-to. L=/var/log/omamobile/modem-sleep-series.log; CP=/usr/lib/omamobile-vendor/omamobile-modem-cp -N=${1:-20}; S=${2:-60}; H=1.1.1.1; Q=example.com; BAT=/sys/class/power_supply/max77779-battery +N=${1:-10}; S=${2:-60}; H=1.1.1.1; Q=example.com; BAT=/sys/class/power_supply/max77779-battery U='link startup failed|Host init failed|I/O error, dev|ufshcd_err_handler' +# An oops, a BUG or a kernel panic path in dmesg +O='Internal error:|Unable to handle kernel|kernel BUG at|BUG: |Oops' +bad() { dmesg | grep -cE "$1"; } say() { echo "$(date +%T) $*" >> $L; sync "$L" 2>/dev/null; } screen() { python3 - "$1" <<'PY' 2>&1 | tail -1 import json, socket, sys @@ -86,12 +95,12 @@ PY charging || { echo "not charging: run this on the charger" >&2; exit 1; } [[ ${3:-} == append ]] || : > $L systemctl is-active -q omamobile-sit-data || { systemctl start omamobile-sit-data; sleep 8; } -[[ ${WIFI:-} == off ]] && { nmcli radio wifi off; sleep 10; } +[[ ${WIFI:-} == off ]] && { trap 'nmcli radio wifi on' EXIT; nmcli radio wifi off; sleep 10; } IF=$(cat /run/omamobile-sit/data-interface /run/omamobile-sit/data-interface-kept 2>/dev/null | head -1) say "$(uname -r) $(uname -v | cut -d' ' -f1); the data service: $(systemctl is-active omamobile-sit-data), device ${IF:-none}, Wi-Fi radio $(nmcli radio wifi), default via $(ip route show default | awk '{print $5}' | paste -sd,); $(st); data: $(data)" sleep 10 for n in $(seq 1 "$N"); do - s0=$(cat /sys/power/suspend_stats/success); f0=$(cat /sys/power/suspend_stats/fail); u0=$(journalctl -b -k --no-pager | grep -cE "$U") + s0=$(cat /sys/power/suspend_stats/success); f0=$(cat /sys/power/suspend_stats/fail); u0=$(bad "$U"); o0=$(bad "$O"); t0=$(date +%s) charging || { say "sleep $n: not charging any more: stop"; break; } [[ ${WIFI:-} == off ]] || for _ in $(seq 1 30); do ip route show default | grep -c wlan0 >/dev/null && break; sleep 1; done say "sleep $n: default via $(ip route show default | awk '{print $5}' | paste -sd,); screen off -> $(screen off); link: $(idle); link-ups in the last 20 s: $(journalctl -b -k --no-pager --since '-20 s' | grep -c 'link up: Gen3')" @@ -108,13 +117,14 @@ for n in $(seq 1 "$N"); do else res="no change in suspend_stats after 150 s"; fi say "sleep $n: $res; screen on -> $(screen on)" sleep 5 - say "sleep $n, 5 s after: default via $(ip route show default | awk '{print $5}' | paste -sd,); cpif $(cpif); data: $(data); cpif after the data check: $(cpif)" + say "sleep $n, 5 s after: default via $(ip route show default | awk '{print $5}' | paste -sd,); cpif $(cpif); data: $(data); cpif after the data check: $(cpif); sitd's data-call lines this sleep: $(journalctl -u omamobile-sitd --since "@$t0" --no-pager -o cat | grep -E 'Data-call list changed|Internet bearer' | cut -c1-160 | paste -sd'|' | sed 's/^$/none/')" sleep 15 - say "sleep $n, 20 s after: $(st), new ufs lines $(( $(journalctl -b -k --no-pager | grep -cE "$U") - u0 )), sitd's last line: $(journalctl -b -u omamobile-sitd --no-pager | tail -1 | cut -c40-120)" + say "sleep $n, 20 s after: $(st), new ufs lines $(( $(bad "$U") - u0 )), new oops lines $(( $(bad "$O") - o0 )), sitd's last line: $(journalctl -b -u omamobile-sitd --no-pager | tail -1 | cut -c40-120)" + (( $(bad "$U") > u0 )) && { say "a UFS error line in dmesg: stop"; dmesg | grep -E "$U" | tail -3 >> $L; break; } + (( $(bad "$O") > o0 )) && { say "an oops in dmesg: stop"; dmesg | grep -E "$O" | tail -3 >> $L; break; } [[ $($CP state 2>&1 | tail -1) == ONLINE ]] || { say "the CP is not ONLINE: stop"; break; } done [[ -n $IF ]] && say "ping over $IF: $(ping -c 5 -W 3 -I "$IF" "$H" 2>&1 | grep -o '[0-9]* received')" -[[ ${WIFI:-} == off ]] && nmcli radio wifi on say "end: $(st); in the boot's kernel log: Reserve doorbell interrupt $(journalctl -b -k --no-pager | grep -c 'Reserve doorbell interrupt'), PCI not powered on $(journalctl -b -k --no-pager | grep -c 'PCI not powered on')" say "kernel lines of the boot:" journalctl -b -k --no-pager | grep -E "sleep refused|Unable to change power state|Link Active not set|Can.t set D0|BME is not set|link-down recovery|suspend_cp: abort|failed to suspend" | sed -E 's/^.{16}[^ ]+ kernel: //' | cut -c1-110 | sort | uniq -c | sort -rn | head -20 >> $L From d9b11d2921b041eb77670918d254b47c1804f88a Mon Sep 17 00:00:00 2001 From: W4HO <87742609+W4HO-ham@users.noreply.github.com> Date: Sun, 15 Nov 2026 21:37:22 -0644 Subject: [PATCH 8/8] device/tegu: the modem sleep series counts a full battery on the cable as on the charger With the pack full the charger holds it and the average current is about 0, so the check that wanted it above 0 stopped tonight's run at sleep 47 with the cable in. The battery's status now decides (Charging or Full; Discharging off the cable), with the current as before for a kernel that doesn't report it. --- device/tegu/bringup/modem-sleep-series.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/device/tegu/bringup/modem-sleep-series.sh b/device/tegu/bringup/modem-sleep-series.sh index 09965a1cd..bb8039726 100755 --- a/device/tegu/bringup/modem-sleep-series.sh +++ b/device/tegu/bringup/modem-sleep-series.sh @@ -55,8 +55,11 @@ idle() { # wait until the link has been down for 5 s; prints how long it took done echo "not idle in 60 s"; return 1 } -# The battery's average current is positive while it charges (the 9a lists no charger supply). -charging() { (( $(cat $BAT/current_avg 2>/dev/null || echo 0) > 0 )); } +# On the charger: the battery charging, or full with the cable in (the charger then holds it +# and the current is about 0; a series stopped at sleep 47 on 2026-11-15 when the pack filled). +# Off the cable the status is Discharging. +charging() { case $(cat $BAT/status 2>/dev/null) in Charging | Full) return 0 ;; esac + (( $(cat $BAT/current_avg 2>/dev/null || echo 0) > 0 )); } data() { # a ping and a DNS lookup over the bearer's interface [[ -n $IF ]] || { echo "no bearer interface"; return; } echo "ping $(ping -c 3 -W 3 -I "$IF" "$H" 2>&1 | grep -o '[0-9]* received'), dns $(dns)"