Repository navigation
PCI: dwc: gs201-cp: suspend-to-idle with the modem online - #232
Conversation
|
Thank you, this is careful work: each of the three changes came from a A few things before it goes in:
A small one: the branch's This changes my phone's modem driver, so I'll run it on my 7 Pro together |
|
Thanks for the review! I'll fix all of these. |
|
Thanks for the reply. Nothing has been pushed yet, so the five points from my last comment still stand as written, and since then the base has moved under this file, which adds two things for the rebase:
When you push, please include the hash of the tested file again ( |
The modem stays powered across s2idle, as on stock: CPIF lets the AP sleep only after CP2AP_WAKEUP has dropped and the idle link has been powered down (it holds a wake lock while the link is up, and its suspend returns -EBUSY while the line is high), and stock's root complex does no link-down at suspend either, only isolating a link that is already down, which link_down does here. This driver had no sleep callbacks. Add noirq ones that refuse while the driver's own state says the link is up or the PHY is out of isolation, should those paths ever race. This is a late abort rather than a guard: the root port and the modem are this device's children, so the PM core has run their suspend_noirq, and the PCI core has handled them, before this callback runs; a refusal makes the PM core resume them and fails the sleep. The refusal is logged; the link-down edges at suspend and resume, with the PMU's isolation register, are debug messages. The isolation register is not used to refuse: a phone whose modem never started never isolated the PHY and must still sleep. Only gs201 sets the new sleep_guard match data, so on the Pixel 9a (zumapro), where sleep with the modem up has not been tried and the PHY stays out of isolation after the first link up, the callbacks do nothing and its behaviour is unchanged. On gs201 the modem link node is disabled in gs201.dtsi and only the Pixel 7 Pro (cheetah) enables it, through its device overlay; that is the board this changes. Seen on a Pixel 7 Pro with the modem up (7.3.0-rc6, before this change): the cp2ap_wakeup IRQ (gpa8-3, level) is wake-enabled by CPIF, and the link's MSIs are not. Assisted-by: LLM Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
…s root port The modem keeps its link down between transfers, and the root port's configuration space is unreachable while it is down. The first suspend-to-idle with the modem online (link down at the time) went: pcieport 0001:00:00.0: Unable to change power state from D0 to D3hot, device inaccessible pcieport 0001:00:00.0: Unable to change power state from D3cold to D0, device inaccessible pcieport 0001:00:00.0: Data Link Layer Link Active not set in 100 msec The failed D3hot left the port marked D3cold, so resume waited for the link, which only comes back when the modem driver asks for it, and marked the modem disconnected. When the modem driver raised the link 3 s later, every kernel config read of the modem returned 0xffff (a config read from user space still saw its vendor ID), and CPIF could not ring its doorbell again. Mark the root port PCI_DEV_FLAGS_NO_D3. It stays in D0, the PCI core skips its bus PM in suspend-to-idle (skip_bus_pm), and resume leaves the link to the modem driver, as at runtime. Only with sleep_guard (gs201), as the sleep callbacks: the Pixel 9a's root port is handled as before. Assisted-by: LLM Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
PCI_DEV_FLAGS_NO_D3 kept the root port out of D3hot, but it can still be marked D3cold. The read that does it is pci_power_up()'s read of the port's PM control register: with the link down, cp_root_map_bus() maps nothing, the read returns all ones, and pci_power_up() marks the port D3cold: pcieport 0001:00:00.0: Unable to change power state from D3cold to D0, device inaccessible The suspend-to-idle after that one started from D3cold, so resume again waited for the link, which only the modem driver raises, and marked the modem disconnected: pcieport 0001:00:00.0: Data Link Layer Link Active not set in 100 msec cpif: s51xx_pcie_restore_state: Can't set D0 state!!!! cpif: s51xx_pcie_send_doorbell_int: BME is not set(cnt=10) Give the devices below the port a zero d3cold_delay, so pci_bridge_wait_for_secondary_bus() returns at once and the link stays the modem driver's business, as at runtime. This makes the stale D3cold mark harmless rather than avoiding it, since the modem's power belongs to CPIF. Skipping the wait is safe because the modem is never powered down across s2idle, and the modem driver waits for its own link up and restores the endpoint's state before it touches the modem. The wait could only time out and mark a working modem disconnected, so the zero d3cold_delay must stay. Only with sleep_guard (gs201): the Pixel 9a's devices keep their d3cold_delay. With this and the two changes before it, on a Pixel 7 Pro with the modem registered on LTE: eight suspend-to-idle cycles in one boot, all resumed, woken by the RTC alarm or the modem's cp2ap_wakeup (an incoming SMS among them, received in the second it woke), mobile data working after each, no UFS error. Assisted-by: LLM Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
9bdcc18 to
64dc726
Compare
|
Thank you for the rework. I've read the new head commit by commit against my two comments, and most of it is done exactly as asked: the refusal is described as a late abort, the two per-cycle lines are debug messages, the third commit names the read that marks the port D3cold and says why the zero The new fourth commit is the interesting one, and finding that failure was valuable: a refused sleep that costs the modem until a reboot is exactly what the refusal must not do. Two things about it before this can go in. 1. What ran with it? The message describes the failure and the reasoning, but not a run with the fix. #556's plan has no entry for the refused sleep, and its only run on the rebased kernel is the registration test, with no sleep. So as far as I can see nothing has slept on this head yet. Could you add to #556's results what you have: the date and kernel release of the boot where the refusal hit, how the sleep was started, and, if you did run the fix afterwards, the kernel release, 2. The cause, as the message and the comment state it. They say the PCI core "had put the modem in D3hot over the live link". I don't think the code does that. The modem's driver ( One question that would help me understand the failure: the refusal printed "link up, PHY out of isolation", yet both config reads at the aborted resume came back all ones, which in this driver means Two small things while you're in there, neither required: the PMU isolation register is now read on every sleep only to feed a debug message, and the resume callback only prints one, so both could go; and commit 1 leaves one comment line near 100 columns that commit 4 reflows. Once the message is corrected I'll run the head on my 7 Pro together with #556: the modem up with no sleep first, then the freezes, with the loop that sleeps right after a modem wake so the refusal has a chance to fire, then the call half of M6.3. If no refusal shows in 20 cycles I'll record that path as not exercised and we'll decide then. The four commits go into the kernel before #556 turns sleep on. |
On a Pixel 7 Pro a suspend-to-idle was refused twice in a row: by the sleep callback (the link came up 8 ms in), then by the modem driver with the link down. That resume logged s51xx 0001:01:00.0: Unable to change power state from D3hot to D0, device inaccessible and the modem stopped answering until a reboot. Its driver has no PM ops, so the PCI core disabled, saved, powered up, restored and re-enabled it behind the driver's back, while the driver manages the modem's power and PCI state itself. Make the devices below the root port syscore devices, so the PM core skips them. PCI_DEV_FLAGS_NO_D3 would also refuse the driver's own runtime D3hot. With this: 60 refused attempts with data after each, and 8 quiet cycles woken by cp2ap_wakeup. Only with sleep_guard (gs201). Assisted-by: LLM Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
64dc726 to
f4e5e48
Compare
|
@ikegam, I ran this on my Pixel 7 Pro today (2026-11-13), and the refusal path lost the modem's data, so this goes back to you. Kernel:
So the downlink dies after a run of refusals even without the CPIF abort that commit 4 targets. I have the whole kernel log and sitd's journal of that boot; tell me what you want from them (they hold no identifiers I'd paste, but I'll cut them to the window around 480-530 s of monotonic time). The #556 waits on this one, and no more sleeps with the modem up on my phone until there's a new head. |
|
@ikegam, I had the log from this morning's run read against CPIF, your four commits and Google's driver. The short version: the refused sleeps probably didn't kill data on the AP side, and there are two candidates. The write-up and two patches are below (built and checkpatch-clean on top of your head merged with The analysislinux-omarchy-mobile#232: mobile data lost after refused sleeps (2026-11-13 run)Kernel under test: Labels: [code] read in the source, [log] in the boot's kernel log, 1. SummaryNo single cause can be proven from the code and the log, and the refusal is The two best candidates:
The fix proposed in section 6 takes every kernel action out of a refused 2. The log, read for the link and the cp2ap line[log] The cp2ap_wakeup handler runs once per edge, and each run logs the pair
3. What a refused attempt does, step by step [code]
So a refusal sends the modem no GPIO change, no shared-memory write and no Candidate B in detail. At each real sleep the link is down. The PCI core's The mark stays. 4. Why the AP side was still serviceable after 518 s
What is not known from this boot: whether the modem consumed the uplink One more AP-side possibility fits "packet path dead, IPC alive" [inferred]: 5. Stock (gs-6.1) on a suspend that fails late
6. Patches (against wt-modem-sleep HEAD
|
| UL fore | UL rear | DL rear | DL done | Means |
|---|---|---|---|---|
| rises | stays | stays | = rear | the modem stopped taking uplink: A, modem side. Next: the bearer state from sitd, then the hooks alone (step 2 below) |
| rises | follows | stays | = rear | uplink leaves, nothing comes back: the bearer or the network (A). Next: sitd's data call list |
| rises | follows | rises | stays | the AP stopped draining: a kernel bug. The napi state bits (SCHED 0x1) show whether NAPI is stuck |
| stays | stays | the AP stopped filling the UL ring: the netdev or the UL timer (not seen so far) |
7. Attended test on the maintainer's 7 Pro
The maintainer runs every step, on the charger, with NFC polling stopped and
auto-sleep off, as in B4. Before step 1, as AGENTS.md section 1 requires:
tools/nv-backup --check, spm_lvl 5, registered. Before every reboot or push:
systemctl stop omamobile-modem (AGENTS.md section 2).
Step 0 (read-only, low), on today's kernel, no sleep. Find the CPIF
platform device: ls -d /sys/devices/platform/*/pktproc. Then read
pktproc/status, pktproc_ul/status and the msi1-4 rows of
grep 11920000 /proc/interrupts twice, 10 s apart, with ping -i 1 -I rmnet2
running. Expect: DL rear and done advancing together, UL rear following fore,
the msi rows counting. Don't read /sys/kernel/debug/gs201-pcie-cp/status: it
reads ELBI whenever phy_inited is set, also while the PHY is isolated
(pcie-gs201-cp.c :2055-2061), and AGENTS.md says ELBI with the PHY isolated
hangs the SoC.
Step 1 (high): the run that failed, on the 6.1 + 6.2 build. The push is the
human's call. echo 1 > /sys/module/cpif/parameters/ring_trace. Two RTC sleeps
of 45 s, as in B4.1 and B4.2, so the boot has the history of the failed run.
Then the B4.3 loop: ping -i 1 -I rmnet2 and five times
rtcwake -m no -s 45; systemctl suspend, 15 s apart, then 60 s more of ping.
- Expect for each refused attempt:
gs201-pcie-cp 11920000.pcie: sleep refused before the freeze: link up, PHY out of isolation,
with noFreezing user space processes, noPM: noirq suspend of devices failed
and nocpif: s5100_pm_notifier: Suspend preparefor that attempt. CPIF's
notifier may run first and log it, and then "Resume done" follows at once.
Expect nopcieport 0001:00:00.0: Unable to change power stateline at the
real sleeps either. - Pass: the ping answers through the loop, and 3/3 at the end.
link on/off
trace lines show DL rear and done moving together. No UFS line, no
suspend_cp: abort suspend, nos51xx ... power stateline. - If data dies: read the trace lines around the death against the table in
6.2. Also readpktproc/statusandpktproc_ul/statusonce, and save the
sitd journal for the same minute. - Stop: no wake 2 minutes after a real sleep (don't send commands; the human
holds Power about 30 s, then Power); any UFS or I/O line; data dead (stop
sleeping, collect, stop the modem, reboot). - Rollback:
rm /etc/omamobile/modem-suspend-enable, stop the modem, reboot to
the previous slot B set.
Step 2 (medium), only if step 1 loses data with the UL or DL rear stuck (A):
the hooks alone, no kernel sleep. Same boot after a reboot, same ping. Five
times, 15 s apart: run /usr/lib/systemd/system-sleep/omamobile-modem-sit pre,
then the Wi-Fi unload and reload as omamobile-sleep does them. Write no
/sys/power/state. If data dies here too, the trigger is the SIT "screen off"
hold or the Wi-Fi reload, and the fix belongs in userspace or the modem, not
in #232. If it doesn't, it is the freeze itself: sitd and the modem daemons
stopped for 10-20 ms twice per attempt.
8. What would change these conclusions
- A DL rear that advances while done stays put (step 1) would make it a
kernel bug in CPIF's receive path, and the napi bits would show where. - The same failure with
ring_traceshowing nothing wrong on the AP, and
data back aftersystemctl restartof sitd alone (no modem restart, which
AGENTS forbids), would put it on the bearer.
Build check
KERNEL_DIR=~/omamobile-kernel/wt-m6fix KERNEL_OUT=~/omamobile-kernel/out-m6fix bin/omamobile kernel: exit 0 (2026-11-13). pcie-gs201-cp.o and cpif.ko built with no warning in either patched file (default warnings; the W=1 build of the two files is still to do in the gate). checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks on both diffs. Both diffs git apply --check cleanly on wt-modem-sleep HEAD.
|
Thanks for the run and the deep analysis! I'll run your 6.1 + 6.2 build on my 7 Pro first (step 1, on the charger) and post the ring trace here. Step 0 on the release kernel looks normal: DL rear and done move together with the ping. |
|
A second phone's result, in case it helps: this series on a Pixel 9a (zumapro, the same link driver and CPIF, a different modem), 2026-11-13. It slept with the modem registered and kept the phone service, which it has never done before. What ran. Refusals: 14 in one boot, data alive. Every request of the first two runs was refused by 6.1's notifier ( Real sleeps: 3 of 3, then 19 of 20.
The same day without the series, for comparison (the 9a's daily kernel with its out-of-tree copy of the link driver, one request per boot): two boots as they were: in one the link was down at the request and the modem was lost as on 11-09 ( What is particular to the 9a. With the cellular data fallback ( The ring trace. I have 6.2's lines for the whole first boot (about 4,300, every link power change, the refusals included). At the last link-off before the reboot DL rear and done were equal on every queue and UL fore, rear and done too. Say if you want the window around the refusals; it holds no identifiers. If this lands as it is, I'll send the zumapro line as its own small pull request with these results, or fold it in here if you'd rather, @ikegam. |
|
A follow-up that qualifies my result above: later the same evening the 9a hung once on that test kernel, awake, and the watchdog reset it. I don't think it is the sleep path, but it happened on this build and you should have it.
So it reads as a hang in the zumapro link's power-up (the PHY coming out of isolation and being set up), on the in-tree It doesn't change the sleep results above. It does mean the 9a's "sleeps with the modem up" rests on a link driver that hung once in that boot's 1,548 power-ups (two earlier boots of the same build that day, about an hour together, had none), which I'd want understood before |
|
Thanks again for the patches! I ran 6.1 + 6.2 on my 7 Pro (2026-11-14,
So the data loss didn't reproduce here. One more thing: with @W4HO-ham thanks for the 9a run, nice to see it match! I'd keep the zumapro line as its own small PR, since this one is checked on the 7 Pro only and the 9a hang is still open. |
|
The 9a's overnight count on the same test build, and one observation about sleep lengths. 300 requests, one boot, modem registered throughout (2026-11-13 evening to 11-14, 7 h 40 min,
The observation: 8 of the 299 sleeps ran 20 to 29 s past the alarm. They were asleep 80 to 89 s with the alarm set for 60 (three more ran 63 to 68 s). Five were ended by @ikegam agreed: the zumapro line goes in its own small pull request once this one has landed, with these counts. |
|
A correction to my reading of the late sleeps above, after looking at the counts again. I wrote that the alarm was raised on time and failed to end those sleeps. The numbers don't show that:
So the RTC raised an alarm about 30 s early in 12 sleeps and 30 or 60 s late in 4, and in the other late ones the modem woke the palmtop before any alarm came. That looks like a second RTC timer beside |
…ink is up With the modem online, systemd's sleeps were refused only in this driver's noirq callback: after every task was frozen and every device suspended, which the PM core then resumed again. On the maintainer's Pixel 7 Pro (2026-11-13) six such refusals in a row were followed by a downlink that received nothing until a reboot, and each real sleep logged "pcieport ... Unable to change power state from D3cold to D0": the root port had been marked D3cold by a sleep with the link down, so the next sleep, a refused one included, ran the PCI core's whole resume of it on a live link. Refuse earlier, from a PM_SUSPEND_PREPARE notifier, while the link is up or the PHY is out of isolation: a refusal then costs one notifier call, before the freeze. Stock gets the same from CPIF's wake lock under Android's autosleep, which systemd doesn't use. The noirq check stays, for a link that comes up during the freeze. Leave the root port out of system sleep too (dev_pm_syscore_device()), as the previous commit does for the modem. Its windows are this driver's and are restored at every link up (cp_root_win_restore()); PCI_DEV_FLAGS_NO_D3 stays for runtime PM. Tested on a second Pixel 7 Pro (2026-11-14, on top of omamobile at 2db2bc3): three real sleeps (9.7 s, 43.7 s to the RTC, 4.2 s), then five requests under a 1 s ping, all ten attempts refused before the freeze, ping 138 of 141 and 3 of 3 afterwards, no pcieport line. On a Pixel 9a with sleep_guard set for zumapro (W4HO): 14 refusals with data alive, then 22 of 23 sleeps entered. The notifier and the root port change are Derek Ross's, from his review of this series. Suggested-by: Derek Ross <derekross@gmail.com> Assisted-by: LLM Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
|
Pushed |
|
@ikegam, results from my Pixel 7 Pro tonight (2026-11-14), on
So I'd like #232 to carry the first patch (the early refusal in a PM notifier, and the root port as a syscore device); the |
|
@derekross, the fold is already on this branch, I think: @ikegam pushed The 9a has run both forms:
The awake hang at a link power-up from my second comment hasn't come back: none in over 15,000 power-ups since, on the two builds. |
|
Thanks both, Derek for the runs and the merge, and W4HO-ham for checking the fold line by line. |
What
Suspend-to-idle with the modem online on the Pixel 7 Pro (gs201), in the modem's PCIe link driver (
pcie-gs201-cp):PCI_DEV_FLAGS_NO_D3keeps the port in D0 and skips its bus PM in s2idle.d3cold_delaybelow the port makespci_bridge_wait_for_secondary_bus()return at once.Tested
Pixel 7 Pro, modem registered on LTE (IMS voice and SMS), on the charger,
7.3.0-rc6-omamobile-push1-g80f9610feb26(this series on the RTC power-off commit):suspend_statssuccess 8, fail 0), woken by the RTC alarm or the modem's cp2ap_wakeup;rmnet0) worked after every wake; no UFS error-handler line, no "Link Active" or "BME" line;tools/kernel-pr-check --build OUT --merge 1a9d1d4e9: merges cleanly, prefixes, checkpatch --strict (two long quoted log lines), policy, W=1 build of the touched file all ok.The userspace side (the modem's sleep inhibitor made optional, the screen state told to the modem) is in derekross/omarchy-mobile; results in
device/cheetah/MODEM-PLAN.md7.4.🤖 Generated with Claude Code