Skip to content

omamobile-device-tegu 0.2.21: no service watchdogs, a 9a sleep runs with the kernel's clock ticking (#765) - #766

Merged
derekross merged 2 commits into
derekross:mainfrom
W4HO-ham:tegu-sleep-watchdog
Nov 15, 2026
Merged

derekross merged 2 commits into
derekross:mainfrom
W4HO-ham:tegu-sleep-watchdog

Conversation

@W4HO-ham

Copy link
Copy Markdown
Contributor

The stopgap from #765, with a sleep series behind it. Yours to refuse if you'd rather wait for the kernel to stop the clock.

What and why

On the 9a the kernel's clock runs through a sleep: s2idle stops CLOCK_MONOTONIC only when a cpuidle state is entered through its s2idle callback (tick_freeze()), the published device tree has no idle states, and where a kernel has them the cpuidle sleep hook keeps the cores in WFI for the sleep. systemd's service watchdogs count on that clock, so after a sleep longer than three minutes it aborted journald, logind, homed or userdbd at the thaw; with logind gone the lock screen took no swipe until a reboot. The journal lines and the mechanism are in #765.

omamobile-device-tegu 0.2.19 adds one file, /usr/lib/systemd/system/service.d/10-omamobile-tegu-sleep-watchdog.conf: WatchdogSec=0 for every system service. Its comment says when to remove it (when a 9a sleep stops the clock). It has to be a packaged file and not something a unit writes at boot: a running service keeps the watchdog it was started with, so a runtime drop-in and daemon-reload changed nothing in my first try.

The cost is the watchdogs themselves: a hung journald or logind is no longer restarted by systemd on the 9a. Nothing on the 9a relies on that today, and the other route to it (leaving them on) breaks the session at every long sleep.

AGENTS.md gets a note under "Suspend and modem" with the symptom and the check (journalctl -b | grep 'Watchdog timeout').

Tested

On my palmtop (Pixel 9a), 2026-11-14, 7.3.0-rc6-omamobile-push1-gc50fd8fdf42a (no cpuidle driver: current_driver reads none), the modem registered, with the same drop-in under /etc/systemd/system/service.d/ from boot (the package itself is not installed there yet):

sleeps of 3 min or more services aborted by the watchdog
without the drop-in (the boot in #765) 4 (3 min 41 s to 6 min 17 s) at least one after each: homed, journald twice, logind, coredumpd
with it, first boot 1 (4 min 15 s) none
with it, a series of seven 5-minute alarms 6 (190 s, and five of 301 to 302 s) none

With it WatchdogUSec read 0 for journald, logind, homed, userdbd, udevd, resolved, timesyncd and coredumpd; after the series: no "Watchdog timeout" line, NRestarts 0 for journald, logind, homed and userdbd, sddm quiet, Hyprland running, the modem ONLINE.

packages/omamobile-device-tegu/test/all (it now checks the drop-in holds [Service] and WatchdogSec=0 and nothing else, that it is the only one for all services, and that no unit of the package sets a watchdog), tests/all and tools/pr-check pass.

Not tested

  • The packaged path /usr/lib/systemd/system/service.d/; the test used /etc, which systemd reads the same way.
  • A kernel with idle states and the cpuidle hook. By the code the hook's sleep runs in WFI and so with the clock ticking too, which is why the drop-in is unconditional, but every long sleep I have on record there was before I knew to look for this.
  • Sleeps longer than five minutes.

The version is 0.2.19 here; #483's rebase names the same number, so one of them moves at merge.

…ith the kernel's clock ticking (derekross#765)

systemd's service watchdogs count on CLOCK_MONOTONIC, and on the 9a that clock runs through a sleep: s2idle stops it only when a cpuidle state is entered through its s2idle callback, and the 9a never gets there (no idle states in the published device tree; the cpuidle sleep hook keeps the cores in WFI where there are). After a sleep longer than three minutes systemd aborted journald, logind or homed at the thaw, and with logind gone the lock screen was unusable until a reboot.

One drop-in for every system service turns the watchdogs off until a 9a sleep stops the clock. The package test checks that it does that and nothing else.
@derekross

Copy link
Copy Markdown
Owner

Thank you for chasing this down to the clock, and for the journal in #765: four long sleeps, each costing a service, and none of the short ones, is clear. I read the kernel side in our tree, and it is as you say. tick_freeze() is only reached through a cpuidle state with an enter_s2idle callback, and without one the s2idle loop idles with the clock running. So systemd's watchdogs fire at the thaw. The drop-in is the right shape for a stopgap: packaged, so it is there at boot, and its comment says when it goes.

I'll decide this one myself, as you offered. The cost, no watchdog restart for a hung journald or logind, is real but small next to a lock screen that stops working after every long sleep, so I'm inclined to take it.

Three small things:

  • The AGENTS line points at SLEEP-PLAN S1, which has nothing on this yet. Either drop that link (Pixel 9a, a kernel without idle states: a sleep over three minutes ends with systemd's watchdog killing journald and logind, and the lock screen stops working #765 has the journal and the release) or add a short note to S1 with the date, the kernel release and the four sleeps.
  • When you next boot the package as built rather than the /etc copy, systemctl show -p WatchdogUSec systemd-logind reading 0 would close the "packaged path not tested" line.
  • One thing your reasoning doesn't cover: in cpuidle-psci.c the only enter_s2idle is for the OSI hierarchy (CPU nodes with power domains), and gs201's CPUs have none. So by the same reading the 7 Pro may not freeze the tick either. I'll check my 7 Pro's journal for "Watchdog timeout" after a long sleep.

The version may move at merge: #779 also bumps this package, and the two meet in test/all. I'll resolve that by keeping both blocks.

# Conflicts:
#	packages/omamobile-device-tegu/test/all
#	packages/omamobile-device-tegu/version
@W4HO-ham W4HO-ham changed the title omamobile-device-tegu 0.2.19: no service watchdogs, a 9a sleep runs with the kernel's clock ticking (#765) omamobile-device-tegu 0.2.21: no service watchdogs, a 9a sleep runs with the kernel's clock ticking (#765) Nov 15, 2026
@W4HO-ham

Copy link
Copy Markdown
Contributor Author

Thank you. The first is done, and the second is under way:

Good catch on cpuidle-psci.c. I hadn't looked at the 7 Pro's side, and your reading would put it in the same place.

@W4HO-ham

Copy link
Copy Markdown
Contributor Author

The packaged path, closed: after a clean reboot with omamobile-device-tegu 0.2.21-1 as built from this head (no /etc copy), systemctl show -p WatchdogUSec systemd-logind systemd-journald systemd-udevd reads WatchdogUSec=0 for all three (daily kernel, 2026-11-15).

@derekross

Copy link
Copy Markdown
Owner

Thank you, both are closed: the AGENTS line now points at #765, and the reading after a clean reboot with the package as built, WatchdogUSec=0 for logind, journald and udevd, is exactly what the "packaged path not tested" line needed. Main merged in with #779, both test blocks kept, 0.2.21 in the file and the title: nothing left for me to resolve.

I owe you a correction. My third point was wrong, and your first reasoning was right. dt_idle_states.c gives every plain device-tree idle state an enter_s2idle callback (its comment says enter() can also be the s2idle callback), and cpuidle-psci builds its states through it. The OSI code only swaps the last state's callback for the domain variant. gs201's CPUs have their C2 state, so on the 7 Pro s2idle does reach tick_freeze(), and the clock stops. The one way it wouldn't is a wakeup-latency limit below C2's exit latency, so I'll still read my 7 Pro's journal after a long sleep to confirm. Sorry for sending you after it.

Somnius's measurement on #765 shows the 8 Pro has the same problem, and zuma has no idle states either, so the 8a will too. Your package is the right place for the 9a's evidence. For husky and akita, either the same file in their device packages or your generator idea from #765 (the drop-in written only while cpuidle/current_driver reads none) would do. The generator would also leave the Tensor G1 and G2 phones alone by construction. That's for a follow-up, not this PR.

I'm taking this one, as I said I was inclined to. I'll merge it with a squash message for 0.2.21.

@derekross
derekross merged commit 2470b81 into derekross:main Nov 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants