Skip to content

PCI: dwc: gs201-cp: suspend-to-idle with the modem online - #232

Merged
derekross merged 5 commits into
derekross:omamobilefrom
ikegam:omamobile-modem-suspend-pr
Nov 15, 2026
Merged

derekross merged 5 commits into
derekross:omamobilefrom
ikegam:omamobile-modem-suspend-pr

Conversation

@ikegam

@ikegam ikegam commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

What

Suspend-to-idle with the modem online on the Pixel 7 Pro (gs201), in the modem's PCIe link driver (pcie-gs201-cp):

  1. refuse system sleep while the modem's link is up: noirq sleep ops that return -EBUSY while the driver's state says the link is up or the PHY is out of isolation, and log both edges. CPIF already lets the AP sleep only once CP2AP_WAKEUP has dropped and the idle link is down; this is the link driver's own guard against a race.
  2. keep the PCI core's sleep handling off the root port: the root port's configuration space is unreachable with the link down, so the PCI core's D3hot attempt failed, left the port marked D3cold, and resume then waited for a link that only CPIF raises and marked the modem disconnected (every later config read 0xffff, no doorbell). PCI_DEV_FLAGS_NO_D3 keeps the port in D0 and skips its bus PM in s2idle.
  3. don't let resume wait for the modem's link: a failed power-up read at resume still marks the port D3cold, so the second sleep of a boot dropped the modem the same way. A zero d3cold_delay below the port makes pci_bridge_wait_for_secondary_bus() return at once.

Tested

Pixel 7 Pro, modem registered on LTE (IMS voice and SMS), on the charger, 7.3.0-rc6-omamobile-push1-g80f9610feb26 (this series on the RTC power-off commit):

  • eight s2idle cycles in one boot, all resumed (suspend_stats success 8, fail 0), woken by the RTC alarm or the modem's cp2ap_wakeup;
  • an SMS sent during sleep woke the phone and was stored and acknowledged in the second it woke;
  • mobile data (ping over rmnet0) worked after every wake; no UFS error-handler line, no "Link Active" or "BME" line;
  • without 2 and 3 the modem was lost after the first (without 2) or second (without 3) sleep, with the log lines quoted in the commits.

tools/kernel-pr-check --build OUT --merge 1a9d1d4e9: merges cleanly, prefixes, checkpatch --strict (two long quoted log lines), policy, W=1 build of the touched file all ok.

The userspace side (the modem's sleep inhibitor made optional, the screen state told to the modem) is in derekross/omarchy-mobile; results in device/cheetah/MODEM-PLAN.md 7.4.

🤖 Generated with Claude Code

@derekross

Copy link
Copy Markdown
Owner

Thank you, this is careful work: each of the three changes came from a
failure you saw on the phone, and the messages quote the log lines that
show it, which makes them easy to follow. I ran our gate merged onto
today's omamobile: it merges cleanly, checkpatch only notes the two long
quoted log lines (fine), and the W=1 build is clean. CI is waiting on my
approval.

A few things before it goes in:

  1. When the refusal runs. The comment says the check refuses "rather
    than let the PCI core suspend a live root port". The root port and the
    modem sit below this platform device and were registered after it, and
    the PM core runs suspend_noirq for children before their parent, so by
    the time this callback refuses, the PCI core has already handled the
    root port and the endpoint. With PCI_DEV_FLAGS_NO_D3 that's harmless,
    so it works as a late abort, which is still useful; could the comment
    and message say that, rather than describe it as a guard? (One cycle
    with /sys/power/pm_print_times set shows the order, if you want to
    check.)
  2. Quieter logs. With sleep on, "sleep: link down…" and "resume: link
    down" print on every cycle. Could those two be dev_dbg, keeping the
    refusal visible?
  3. What marks the port D3cold. The third commit makes the stale D3cold
    mark harmless rather than avoiding it, which I think is the right call
    here, since the modem's power belongs to CPIF. Could the message name the
    read that sets it (the power-up check through the guarded config ops,
    reading all-ones) and say why skipping the D3cold wait is safe, so nobody
    removes d3cold_delay = 0 later?
  4. The cycle count. The description says success 8; #556's MODEM-PLAN
    says success 7 for the same boot. Which is it?
  5. The tested tree. -g80f9610feb26 isn't public. Could you push it,
    or give git hash-object of the tested pcie-gs201-cp.c? It should
    match the head's 71dd0e353f8f.

A small one: the branch's coding-assistants.rst gives Assisted-by: LLM [TOOLS] as the format.

This changes my phone's modem driver, so I'll run it on my 7 Pro together
with #556 before merging: first with the modem up and no sleep (nothing
should change), then the M6 freezes, and the call half of M6.3, which needs
my SIM.

@ikegam

ikegam commented Nov 11, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review! I'll fix all of these.

@derekross

Copy link
Copy Markdown
Owner

Thanks for the reply. Nothing has been pushed yet, so the five points from my last comment still stand as written, and since then the base has moved under this file, which adds two things for the rebase:

  1. A conflict, a small one. omamobile gained the Pixel 9a's modem link and a doorbell helper in pcie-gs201-cp.c. The only textual conflict is in the include block: keep both lines, linux/pm.h before linux/property.h.
  2. The driver now also binds on the Pixel 9a (google,zumapro-pcie-cp, with a struct gs201_pcie_cp_data for what differs). As rebased, your sleep callbacks, PCI_DEV_FLAGS_NO_D3 and the zero d3cold_delay would apply there too, and none of that has run on a 9a, where sleep with the modem up isn't qualified at all. Could you put them behind the match data, for example a sleep_guard flag that only the gs201 entry sets, checked where cp_bus() sets the flags and at the top of both callbacks, and say in the message that the 9a is unchanged? Its owner can turn it on with a test of their own later. (On zumapro the PHY stays out of isolation after the first link up, so the "PHY out of isolation" wording in the refusal wouldn't be right there either.)

When you push, please include the hash of the tested file again (git hash-object drivers/pci/controller/dwc/pcie-gs201-cp.c), since it will change with the rebase, and tell me which of eight or seven cycles is right. I'll then run it on my 7 Pro together with #556: the modem up with no sleep first, then the M6 freezes, then the call half of M6.3.

The modem stays powered across s2idle, as on stock: CPIF lets the AP sleep
only after CP2AP_WAKEUP has dropped and the idle link has been powered
down (it holds a wake lock while the link is up, and its suspend returns
-EBUSY while the line is high), and stock's root complex does no link-down
at suspend either, only isolating a link that is already down, which
link_down does here.

This driver had no sleep callbacks. Add noirq ones that refuse while the
driver's own state says the link is up or the PHY is out of isolation,
should those paths ever race. This is a late abort rather than a guard:
the root port and the modem are this device's children, so the PM core
has run their suspend_noirq, and the PCI core has handled them, before
this callback runs; a refusal makes the PM core resume them and fails
the sleep. The refusal is logged; the link-down edges at suspend and
resume, with the PMU's isolation register, are debug messages. The
isolation register is not used to refuse: a phone whose modem never
started never isolated the PHY and must still sleep.

Only gs201 sets the new sleep_guard match data, so on the Pixel 9a
(zumapro), where sleep with the modem up has not been tried and the PHY
stays out of isolation after the first link up, the callbacks do
nothing and its behaviour is unchanged. On gs201 the modem link node is
disabled in gs201.dtsi and only the Pixel 7 Pro (cheetah) enables it,
through its device overlay; that is the board this changes.

Seen on a Pixel 7 Pro with the modem up (7.3.0-rc6, before this change):
the cp2ap_wakeup IRQ (gpa8-3, level) is wake-enabled by CPIF, and the
link's MSIs are not.

Assisted-by: LLM
Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
…s root port

The modem keeps its link down between transfers, and the root port's
configuration space is unreachable while it is down. The first
suspend-to-idle with the modem online (link down at the time) went:

  pcieport 0001:00:00.0: Unable to change power state from D0 to D3hot, device inaccessible
  pcieport 0001:00:00.0: Unable to change power state from D3cold to D0, device inaccessible
  pcieport 0001:00:00.0: Data Link Layer Link Active not set in 100 msec

The failed D3hot left the port marked D3cold, so resume waited for the
link, which only comes back when the modem driver asks for it, and
marked the modem disconnected. When the modem driver raised the link
3 s later, every kernel config read of the modem returned 0xffff (a
config read from user space still saw its vendor ID), and CPIF could not
ring its doorbell again.

Mark the root port PCI_DEV_FLAGS_NO_D3. It stays in D0, the PCI core
skips its bus PM in suspend-to-idle (skip_bus_pm), and resume leaves
the link to the modem driver, as at runtime.

Only with sleep_guard (gs201), as the sleep callbacks: the Pixel 9a's
root port is handled as before.

Assisted-by: LLM
Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
PCI_DEV_FLAGS_NO_D3 kept the root port out of D3hot, but it can still be
marked D3cold. The read that does it is pci_power_up()'s read of the
port's PM control register: with the link down, cp_root_map_bus() maps
nothing, the read returns all ones, and pci_power_up() marks the port
D3cold:

  pcieport 0001:00:00.0: Unable to change power state from D3cold to D0, device inaccessible

The suspend-to-idle after that one started from D3cold, so resume again
waited for the link, which only the modem driver raises, and marked the
modem disconnected:

  pcieport 0001:00:00.0: Data Link Layer Link Active not set in 100 msec
  cpif: s51xx_pcie_restore_state: Can't set D0 state!!!!
  cpif: s51xx_pcie_send_doorbell_int: BME is not set(cnt=10)

Give the devices below the port a zero d3cold_delay, so
pci_bridge_wait_for_secondary_bus() returns at once and the link stays
the modem driver's business, as at runtime. This makes the stale D3cold
mark harmless rather than avoiding it, since the modem's power belongs
to CPIF. Skipping the wait is safe because the modem is never powered
down across s2idle, and the modem driver waits for its own link up and
restores the endpoint's state before it touches the modem. The wait
could only time out and mark a working modem disconnected, so the zero
d3cold_delay must stay.

Only with sleep_guard (gs201): the Pixel 9a's devices keep their
d3cold_delay.

With this and the two changes before it, on a Pixel 7 Pro with the modem
registered on LTE: eight suspend-to-idle cycles in one boot, all resumed,
woken by the RTC alarm or the modem's cp2ap_wakeup (an incoming SMS among
them, received in the second it woke), mobile data working after each,
no UFS error.

Assisted-by: LLM
Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
@ikegam
ikegam force-pushed the omamobile-modem-suspend-pr branch from 9bdcc18 to 64dc726 Compare November 12, 2026 10:55
@derekross

Copy link
Copy Markdown
Owner

Thank you for the rework. I've read the new head commit by commit against my two comments, and most of it is done exactly as asked: the refusal is described as a late abort, the two per-cycle lines are debug messages, the third commit names the read that marks the port D3cold and says why the zero d3cold_delay must stay, the include conflict is gone, and sleep_guard keeps the Pixel 9a out of all of it. The eight-or-seven question is answered in #556's plan (eight; the earlier count was one short). Our gate passes on the merge with today's omamobile, and each of the four commits builds on its own with W=1. The CI run is still waiting for my approval; I'll start it.

The new fourth commit is the interesting one, and finding that failure was valuable: a refused sleep that costs the modem until a reboot is exactly what the refusal must not do. Two things about it before this can go in.

1. What ran with it? The message describes the failure and the reasoning, but not a run with the fix. #556's plan has no entry for the refused sleep, and its only run on the rebased kernel is the registration test, with no sleep. So as far as I can see nothing has slept on this head yet. Could you add to #556's results what you have: the date and kernel release of the boot where the refusal hit, how the sleep was started, and, if you did run the fix afterwards, the kernel release, git hash-object drivers/pci/controller/dwc/pcie-gs201-cp.c in that tree (the head's is 1d12db51af20), the number of cycles, and what followed any sleep refused: line? If you haven't run it, just say so. I'm not asking you for a new run: since the 11th our AGENTS.md keeps sleep with the modem up on a 7 Pro for my own attended test, so I'll run this head on my phone.

2. The cause, as the message and the comment state it. They say the PCI core "had put the modem in D3hot over the live link". I don't think the code does that. The modem's driver (s51xx) has no PM ops, and for such a device pci_pm_suspend() only calls pci_pm_default_suspend() (it disables the device), and pci_pm_suspend_noirq() only calls pci_save_state() and jumps past pci_prepare_to_sleep(), which is the one place a D3 write could come from. The "from D3hot" in your log line is pci_power_up() printing the state the kernel had on record before the sleep, and the one who sets D3hot there is CPIF itself, at its last link power-down. What the core does do to the modem is the disable and state save at suspend, and at resume pci_power_up(), pci_restore_state() and a re-enable, and I can believe any of those upsets CPIF. Making the modem a syscore device removes all of them, so the fix may well be right; but the commit should claim what we know. Could the message and the cp_bus() comment say that the PCI core's own suspend and resume of the modem ran behind the modem driver's back (naming those steps), and drop the D3hot sentences, unless you have a read of the modem's PM control register that shows it?

One question that would help me understand the failure: the refusal printed "link up, PHY out of isolation", yet both config reads at the aborted resume came back all ones, which in this driver means cfg_root_ok and cfg_ep_ok were off. Was the link in the middle of coming up, or of going down, at that moment? If CPIF was part-way through its own link-up when the sleep was aborted, the cause may be on its side, and then the run in point 1 is the only thing that tells us whether this commit cures it.

Two small things while you're in there, neither required: the PMU isolation register is now read on every sleep only to feed a debug message, and the resume callback only prints one, so both could go; and commit 1 leaves one comment line near 100 columns that commit 4 reflows.

Once the message is corrected I'll run the head on my 7 Pro together with #556: the modem up with no sleep first, then the freezes, with the loop that sleeps right after a modem wake so the refusal has a chance to fire, then the call half of M6.3. If no refusal shows in 20 cycles I'll record that path as not exercised and we'll decide then. The four commits go into the kernel before #556 turns sleep on.

On a Pixel 7 Pro a suspend-to-idle was refused twice in a row: by the
sleep callback (the link came up 8 ms in), then by the modem driver
with the link down. That resume logged

  s51xx 0001:01:00.0: Unable to change power state from D3hot to D0, device inaccessible

and the modem stopped answering until a reboot. Its driver has no PM
ops, so the PCI core disabled, saved, powered up, restored and
re-enabled it behind the driver's back, while the driver manages the
modem's power and PCI state itself. Make the devices below the root
port syscore devices, so the PM core skips them. PCI_DEV_FLAGS_NO_D3
would also refuse the driver's own runtime D3hot.

With this: 60 refused attempts with data after each, and 8 quiet
cycles woken by cp2ap_wakeup. Only with sleep_guard (gs201).

Assisted-by: LLM
Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
@ikegam
ikegam force-pushed the omamobile-modem-suspend-pr branch from 64dc726 to f4e5e48 Compare November 12, 2026 15:39
@derekross

Copy link
Copy Markdown
Owner

@ikegam, I ran this on my Pixel 7 Pro today (2026-11-13), and the refusal path lost the modem's data, so this goes back to you.

Kernel: github/omamobile at 2db2bc35beec merged with this branch's head f4e5e4846c39 (7.3.0-rc6-omamobile-push1-g025ae728a7c3). Userspace: #556's omamobile-modem 0.1.25 and omamobile-modem-sit 0.3.37 (built on main with #555 and #701), follow_screen = yes, the enable file and the runbook marker placed by hand. Auto-sleep off, NFC polling stopped, on the laptop's cable.

  • Kernel awake (B2): passed. Registered, voice, SMS and data (rmnet2) 3 of 3, systemctl suspend refused by the modem's inhibitor while the enable file was absent, no UFS line.
  • Screen state (B3): passed. "Screen off told to the modem", SET_SCREEN_STATE answered error 0.
  • First sleep (B4.1): passed. 45.0 s by the RTC alarm (PMIC IRQ), success 1 fail 0, ping 3/3 after, still registered. Resume logged pcieport 0001:00:00.0: Unable to change power state from D3cold to D0, device inaccessible.
  • Second sleep, same boot (B4.2): passed. Ended after about 2 s on cp2ap_wakeup (IRQ 160), ping 3/3, registered. The same pcieport line again.
  • Refusals (B4.3): failed. Five systemctl suspend with a 1 s ping on rmnet2: six refusals (sleep refused: link up, PHY out of isolation, failed to suspend noirq: error -16), then two real sleeps (each with the pcieport … D3cold to D0 line). The ping's last reply was about 5 s after the sixth refusal and before the next real sleep; from then on rmnet2 sent packets (TX counters rising) and received none (RX unchanged), ping -I rmnet2 0 of 5, while sitd's polls kept being answered and omamobile-sit status went on showing "data bearer: up". No s51xx … Unable to change power state line and no suspend_cp: abort suspend this time; no UFS line. A reboot brought data back; the NV partitions checked SAME afterwards.

So the downlink dies after a run of refusals even without the CPIF abort that commit 4 targets. I have the whole kernel log and sitd's journal of that boot; tell me what you want from them (they hold no identifiers I'd paste, but I'll cut them to the window around 480-530 s of monotonic time). The pcieport resume line after each real sleep may be part of it: the root port still gets marked D3cold on resume despite NO_D3.

#556 waits on this one, and no more sleeps with the modem up on my phone until there's a new head.

@derekross

Copy link
Copy Markdown
Owner

@ikegam, I had the log from this morning's run read against CPIF, your four commits and Google's driver. The short version: the refused sleeps probably didn't kill data on the AP side, and there are two candidates. The write-up and two patches are below (built and checkpatch-clean on top of your head merged with omamobile, not run on a phone yet). I'll run section 7 on my phone with the modem up when it's free; if you can run the debug patch on yours first, the ring pointers will tell us which side stopped.

The analysis

linux-omarchy-mobile#232: mobile data lost after refused sleeps (2026-11-13 run)

Kernel under test: 7.3.0-rc6-omamobile-push1-g025ae728a7c3 (github/omamobile
2db2bc35beec + #232 head f4e5e4846c39), tree ~/omamobile-kernel/wt-modem-sleep.
Log: logs/m6-refusal-kernel-2026-11-13.log (private; only the lines needed are
quoted, host name stripped). Line numbers below are in wt-modem-sleep unless a
file says Google's (gs-6.1 radio/samsung/s5300/ and
soc/gs/drivers/pci/controller/dwc/pcie-exynos-rc.c).

Labels: [code] read in the source, [log] in the boot's kernel log,
[inferred] reasoning from both, not observed.

1. Summary

No single cause can be proven from the code and the log, and the refusal is
probably not what killed data.
What a refused sleep does to CPIF in this tree
is close to nothing, and none of it lasts past the attempt (section 3). After
the failure the AP's side of the packet path was still working. The doorbells
reached the modem, the modem answered each one by raising the link, and every
link-up drained the downlink rings without being asked (section 4). Packets
stopped moving between the AP's rings and the modem about 5 s after the last
refusal, with no kernel event at that moment.

The two best candidates:

  • A (most likely) [inferred]: the modem's data plane stopped. Either the
    modem stopped taking uplink from the UL ring and stopped writing downlink to
    the DL rings, or the bearer behind rmnet2 went down without sitd hearing of
    it. Meanwhile its control IPC, its PCIe DMA into AP memory and its link power
    handshake all kept working. Nothing in the kernel's refusal path touches
    anything the modem can see (section 3). The things that did happen around
    each attempt and that the modem can see all come from userspace: the SIT
    hook's "screen off, hold 10 s" before every attempt, and the freeze of sitd
    and the modem daemons. The unload and reload of Wi-Fi happened at every
    attempt too, though the modem can't see that.
  • B (kernel, weaker) [code]: the PCI core's resume of the modem's root port
    on a live link.
    The two real sleeps had left the root port marked D3cold.
    The first refused attempt after them then ran the PCI core's whole
    resume_noirq of that port with the link up: power-up, pci_restore_state(),
    PME restore, and then pci_resume_bus() on the modem. This is the one
    kernel path that differs between this run and the contributor's 60 clean
    refusals, which came before any real sleep. It is ruled out as the direct
    cause, because data kept working for 35 s after it (481 s to 516 s).

The fix proposed in section 6 takes every kernel action out of a refused
sleep. It refuses at PM_SUSPEND_PREPARE, before anything is frozen, and it
makes the root port a syscore device. That removes B for certain, and it also
removes the "D3cold to D0" lines at every real sleep. It cannot remove A if A
comes from userspace or from the modem. So the attended test in section 7 is
built to tell them apart, with a ring trace (the debug patch in section 6.2)
that shows which side stops moving.

2. The log, read for the link and the cp2ap line

[log] The cp2ap_wakeup handler runs once per edge, and each run logs the pair
wake disabled/enabled for irq 160 (mif_disable_irq/mif_enable_irq,
modem_utils.c :579-609, :547-577). Filtered to the link and that IRQ:

[  479.816843] cpif: pcie_send_ap2cp_irq: Reserve doorbell interrupt: PCI not powered on
[  479.858316] gs201-pcie-cp 11920000.pcie: link up: Gen3 x2 (target Gen3), try 1
[  479.975205] gs201-pcie-cp 11920000.pcie: PHY isolated (PMU 0x3ec0 = 0x0)
[  480.060085] samsung-pinctrl 180e0000.pinctrl: wake disabled for irq 160
[  480.105631] gs201-pcie-cp 11920000.pcie: link up: Gen3 x2 (target Gen3), try 1
[  481.134836] gs201-pcie-cp 11920000.pcie: sleep refused: link up, PHY out of isolation
   ... five more refusals at 481.41, 496.17, 496.47, 511.15, 511.46, no irq 160 edge ...
[  518.200427] samsung-pinctrl 180e0000.pinctrl: wake disabled for irq 160
[  518.256101] gs201-pcie-cp 11920000.pcie: PHY isolated (PMU 0x3ec0 = 0x0)
[  519.041594] cpif: pcie_send_ap2cp_irq: Reserve doorbell interrupt: PCI not powered on
[  519.046711] samsung-pinctrl 180e0000.pinctrl: wake disabled for irq 160
[  519.085654] gs201-pcie-cp 11920000.pcie: link up: Gen3 x2 (target Gen3), try 1
[  519.192447] gs201-pcie-cp 11920000.pcie: PHY isolated (PMU 0x3ec0 = 0x0)
   ... the same three-line cycle about once a second from then on ...
  • The modem held CP2AP_WAKEUP high, and the link up, without a break from
    480.06 to 518.20, which is 38 s. All six refusals fell inside that window, so
    each one saw "link up", and the link never went down around any attempt.
    Before the loop, a ping kept the link up for about 100 ms per packet, with
    one hold of 2.7 s (467.55 to 470.22). [inferred] A ping every second inside
    the modem's idle timer kept the link up. That fits a working data path.
  • The ping's last reply came at about 516, and the modem let the link go
    at 518.20, about 2 s after the traffic stopped.
  • From 519 every ping does the same thing. The AP reserves the doorbell and
    wakes the modem by GPIO (pcie_send_ap2cp_irq, link_device.c :2692-2726,
    then s5100_try_gpio_cp_wakeup). The modem raises cp2ap_wakeup, the AP
    powers the link up and sends the reserved doorbell (s5100_poweron_pcie
    exit path, modem_ctrl_s5100.c :1953-1962), and the modem drops the line
    about 100 ms later. When a ping worked (466.5 s), a second link-up started
    by the modem followed about 160 ms later to deliver the reply. After 519 that
    second link-up is mostly missing. Twice (520.66, 526.24) a link-up started by
    the modem delivers only an unread oem_ipc0 frame.
  • There is no cpif error, no get_packet() error, no recv_net_skb fail,
    no Unbalanced enable, no WARN, no UFS line, no suspend_cp: abort suspend, and no s51xx ... Unable to change power state line.
  • The refusals logged no modem_ctrl_set_kerneltime: time = line, while each
    real sleep logs two (456.26, 526.19, from suspend_cp/resume_cp,
    modem_ctrl_s5100.c :2025, :2056). So CPIF's own noirq suspend
    (modem_suspend, modem_main.c :942-958) never ran in a refused attempt:
    the root complex refused first. [log+code]

3. What a refused attempt does, step by step [code]

Step Ran in the refusals? Changes the packet path?
s5100_pm_notifier PM_SUSPEND_PREPARE (modem_ctrl_s5100.c :2074-2080) yes ("Suspend prepare") sets pcie_pm_suspended, which defers cp2ap edges (:331-345). None came in any window: no "cp2ap_wakeup work pending" [log]. PM_POST_SUSPEND (:2081-2101) runs on a failed sleep too ("Resume done" every time [log]) and clears the flag. Identical to Google's (s5300 modem_ctrl_s5100.c :1978-2022). Nothing is left behind
Freezing user space and freezable kthreads yes sitd and the modem daemons stopped for about 10-20 ms, twice per systemctl suspend. Not kernel state
Device prepare/suspend/suspend_late yes the modem (s51xx) is a syscore device since f4e5e4846c39, so the PM core skips it in every phase. CPIF's platform device has only noirq ops. The root port gets pci_pm_suspend() (pci-driver.c :835-890): skip_bus_pm=false, PTM, pcieport's suspend. No port services are active (no IRQ, "of_irq_parse_pci: failed" [log], native AER/PME/DPC off in probe)
suspend_device_irqs() yes the four DL queue SPIs (msi1-4, GIC 390-393, level) are wake-enabled by set_pcie_msi_int() (:681-731), so they stay armed. One that fires in the window is held pending (irq_pm_check_wakeup) and replayed at resume_device_irqs() (kernel/irq/pm.c :65-100, :144-174; level IRQs re-fire when unmasked, resend.c :131). Block 0 (IPC) is a chained handler and isn't suspended at all (:70). Balanced, and the per-link-up kick (section 4) would recover even a lost one
root port pci_pm_suspend_noirq() (pci-driver.c :906-1000) yes pci_save_state() on a live link. Candidate B, below
gs201_pcie_cp_suspend_noirq() yes, refuses -EBUSY holds cp->lock only for the check
CPIF modem_suspend → suspend_cp (AP2CP_PDA_ACTIVE 0, kerneltime) no [log] nothing
root port pci_pm_resume_noirq() (:1002-1034) yes Candidate B, below
resume_device_irqs(), device resume, thaw, PM_POST_SUSPEND yes nothing that lasts

So a refusal sends the modem no GPIO change, no shared-memory write and no
config access to its endpoint. It doesn't change NAPI, the ring pointers or
cp_online, and it leaves no IRQ out of balance. Of everything above, only the
root port's handling could matter, and only in the state the two real sleeps
left it in.

Candidate B in detail. At each real sleep the link is down. The PCI core's
reads of the port's PMCSR then come back all ones, and pci_power_up() marks
the port D3cold (pci.c :1324-1352). That is the line logged at 421.76,
456.26, 526.21 and 541.29:

pcieport 0001:00:00.0: Unable to change power state from D3cold to D0, device inaccessible

The mark stays. pci_target_state() returns D3cold for a port already in
D3cold (pci.c :2672-2673), and __pci_set_power_state() returns early when
current_state == state (:1584) or the target is D3 with
PCI_DEV_FLAGS_NO_D3 (:1594). So at the first refused attempt after a real
sleep (481.13), the port enters suspend_noirq in D3cold. Its child is a
syscore device, so nothing sets the port's skip_bus_pm (:955-964). With
skip_bus_pm false and prev_state D3cold, pci_pm_resume_noirq() runs
pci_pm_default_resume_early() (:612-617): power-up, which succeeds with the
link up and sets D0; pci_restore_state(); pci_pme_restore(). It then runs
pci_pm_bridge_power_up_actions() (:619-641), which returns at once from the
wait because d3cold_delay is 0 (pci.c :4797-4802) and then calls
pci_resume_bus(), a runtime-resume request to the modem (:1212-1226).
Everything it writes is what it read 10 ms before, on a link that stayed up.
The port is D0 afterwards, so refusals 2-6 skipped all of it. Data worked
after it for 35 s. The contributor's 60 refusals in #556's plan came with no
earlier real sleep in the boot, so their port was always D0 and skipped. B is
the only kernel difference between the two runs that I found. The timing
rules it out as the direct cause, but it is a config write sequence on a live
link that nothing needs.

4. Why the AP side was still serviceable after 518 s

  • Downlink: every s5100_poweron_pcie() runs pktproc_kick_rx_queues()
    right after re-enabling the MSIs (modem_ctrl_s5100.c :1922-1923, kick at
    :740-758). For each active queue the kick calls pktproc_irq_handler(),
    which schedules NAPI if rear != done (link_rx_pktproc.c :1363-1386,
    usage :955-972). There were about 40 link-ups between 519 and 541. Any
    downlink frame the modem had written would have been polled within a
    second, even with every queue MSI lost, and polled frames reach
    rx_multi_pdp(), where rx_packets is counted (modem_io_device.c :422).
    RX stayed fixed. So either the modem wrote no DL descriptor, or NAPI was
    stuck scheduled. A stuck NAPI would need pktproc_poll() to return the full
    budget on every call (-EBUSY/-ENOMEM, :1340), which logs
    get_packet() error (:1017) or burns a CPU in softirq. No such line was
    logged. q->active is set once at pktproc_init and never cleared on this
    path (:1640, :1952). [code+log → inferred]
  • Uplink doorbell and wake: each ping's doorbell was reserved, the modem
    woke by GPIO and raised the link, and the doorbell went out on the link-up
    (section 2). [log]
  • Control: sitd's IPC polls were answered. They use the legacy IPC path,
    which is MSI block 0 (mif_cp2ap_msg, request_pcie_int, :760-836) plus
    the modem's DMA into AP shared memory. So the modem's PCIe writes into AP
    memory worked. [log+code]
  • IRQs: the cp2ap handler kept running (irq 160 pairs). The queue SPIs
    can't be seen without /proc/interrupts, but the kick in the first bullet
    makes their state moot for the question "did any DL arrive".

What is not known from this boot: whether the modem consumed the uplink
descriptors (the UL ring's rear_ptr), and whether it wrote any DL
descriptor (rear_ptr of DL0-DL3). The debug patch in section 6.2 logs both
at every link power change. Both can also be read today, read-only, from
/sys/devices/platform/<cpif>/pktproc/status and .../pktproc_ul/status
(link_rx_pktproc.c :1468-1530, link_tx_pktproc.c :248-259).

One more AP-side possibility fits "packet path dead, IPC alive" [inferred]:
the AP reading a stale cached rear_ptr and the modem missing the AP's
fore_ptr. The pktproc regions are mapped cached with hardware I/O coherency
(cached:1/1/1, iocc:1 [log]). But cp_set_iocc() rewrites the HSI1
shareability and the DBI coherency control at every link-up
(pcie-gs201-cp.c :400-408, :1235-1236, :1295), and no refusal step touches
HSI1's sysreg (the Wi-Fi reloads write HSI2's). I found no mechanism for this
one.

5. Stock (gs-6.1) on a suspend that fails late

  • The root complex never refuses. exynos_pcie_rc_suspend_noirq() isolates
    the PHY if the link is down, and otherwise only does work in the
    pcieon-sleep mode (pcie-exynos-rc.c :5821-5868). Resume undoes the same
    (:5870-5895). prepare/complete only switch PHY isolation and the
    pcieon-sleep enable count (:5897-5926).
  • CPIF's own late refusal is suspend_cp(), -EBUSY while CP2AP_WAKEUP is
    high (ours :2007-2031). PM_POST_SUSPEND replays a cp2ap edge deferred in
    the window (s5300 :1995-2015). On stock that path rarely runs: CPIF holds a
    wakeup source from s5100_poweron_pcie to s5100_poweroff_pcie (ours
    :1889-1890, :1819-1820), and Android's autosleep writes
    /sys/power/wakeup_count. With events_check_enabled set,
    pm_wakeup_pending() refuses the sleep at the freeze while the link is up
    (drivers/base/power/wakeup.c :871-892, checked in kernel/power/process.c
    :69 and drivers/base/power/main.c :1715, :1970). systemd doesn't write
    wakeup_count, so here the lock holds nothing off. That is why PCI: dwc: gs201-cp: suspend-to-idle with the modem online #232 needed
    a refusal in the RC at all, and why ours comes so late.
  • Stock spreads the queue IRQs over CPUs from the throughput monitor
    (s5300 cpif_tp_monitor.c :638-645 sets msi_irq_q_cpu[]). Ours never
    sets them, so resume_cp() → s5100_set_pcie_irq_affinity() (:1968-1991,
    :2041) pins all four DL queues and the IPC vector to CPU 0 after the first
    real sleep. That is harmless as far as I can see, but it is one more state
    that only a boot with real sleeps has.

6. Patches (against wt-modem-sleep HEAD 025ae728a7c3)

Both build in ~/omamobile-kernel/wt-m6fix (output out-m6fix; result at the
end of this file). Neither is committed or pushed.

6.1 Proposed fix: refuse before the freeze, and keep the root port out of system sleep

Reasoning:

  1. A PM_SUSPEND_PREPARE notifier that refuses while link_up || dbi_ok
    gives the semantics stock gets from CPIF's wake lock under autosleep: no
    sleep while the link is up. It fails the sleep before the freeze, before
    any device callback and before suspend_device_irqs(), so a refusal costs
    one notifier call. CPIF's PM_SUSPEND_PREPARE is undone by the robust
    chain's PM_POST_SUSPEND if it ran first. The noirq check stays as the
    backstop for a link that comes up during the freeze.
  2. dev_pm_syscore_device() on the root port, as f4e5e4846c39 did for the
    modem. The PM core then never reads the port's PMCSR with the link down
    (no D3cold mark, so the "Unable to change power state" line goes at every
    real sleep), and never runs the live-link resume of candidate B. The
    port's windows are already restored by this driver at every link-up
    (cp_root_win_restore()). PCI_DEV_FLAGS_NO_D3 stays, for runtime PM.
    d3cold_delay = 0 below the port stays too: it does no harm, and the
    earlier commit says why it must not go.
diff --git a/drivers/pci/controller/dwc/pcie-gs201-cp.c b/drivers/pci/controller/dwc/pcie-gs201-cp.c
index 8c5597da9d63..7ed4650b2c83 100644
--- a/drivers/pci/controller/dwc/pcie-gs201-cp.c
+++ b/drivers/pci/controller/dwc/pcie-gs201-cp.c
@@ -100,6 +100,7 @@
 #include <linux/pinctrl/consumer.h>
 #include <linux/platform_device.h>
 #include <linux/pm.h>
+#include <linux/suspend.h>
 #include <linux/property.h>
 #include <linux/regmap.h>
 #include <linux/seq_file.h>
@@ -281,6 +282,7 @@ struct gs201_pcie_cp {
 	struct pci_host_bridge *bridge;
 	struct resource msi_page;	/* memory-region: the modem's MSI page */
 	struct dentry *debugfs;
+	struct notifier_block pm_nb;	/* sleep_guard: the early refusal */
 	struct mutex lock;		/* serializes the steps */
 	raw_spinlock_t cfg_lock;	/* config accesses against link changes */
 	bool cfg_root_ok;		/* root port (DBI) accessible */
@@ -855,12 +857,20 @@ static int cp_bus(struct gs201_pcie_cp *cp)
 	 * own suspend and resume of it (disable and save, then the power-up
 	 * read, restore and re-enable) ran behind that driver's back when a
 	 * sleep was aborted, after which the driver could not reach the modem.
+	 * The root port is left out of system sleep the same way: once a
+	 * sleep with the link down had marked it D3cold, the next sleep, a
+	 * refused one included, ran the PCI core's whole resume of it on a
+	 * live link (power-up, config restore, PME restore, then a resume of
+	 * the bus below), and with the link down its suspend only saved and
+	 * read back all ones. Its windows are this driver's, restored at
+	 * every link up (cp_root_win_restore()).
 	 * All of this only where system sleep with the modem up has been
 	 * tried (sleep_guard, gs201).
 	 */
 	if (cp->data->sleep_guard) {
 		list_for_each_entry(rp, &cp->bridge->bus->devices, bus_list) {
 			rp->dev_flags |= PCI_DEV_FLAGS_NO_D3;
+			dev_pm_syscore_device(&rp->dev, true);
 			if (rp->subordinate)
 				pci_walk_bus(rp->subordinate,
 					     cp_ep_sleep_setup, NULL);
@@ -2175,6 +2185,39 @@ static int cp_get_sysreg(struct gs201_pcie_cp *cp)
 	return 0;
 }
 
+/*
+ * The same check before anything is frozen or suspended (sleep_guard): a
+ * sleep refused here costs one notifier call, where the noirq refusal
+ * (gs201_pcie_cp_suspend_noirq()) comes after every task was frozen and
+ * every device suspended, and resumes them all again. Stock gets the
+ * early refusal from CPIF's wake lock, held while the link is up; that
+ * lock holds a sleep off only for a caller that wrote
+ * /sys/power/wakeup_count (Android's autosleep), which systemd doesn't.
+ * The noirq check stays for a link that comes up while the system is
+ * being frozen.
+ */
+static int gs201_pcie_cp_pm_notify(struct notifier_block *nb,
+				   unsigned long event, void *unused)
+{
+	struct gs201_pcie_cp *cp = container_of(nb, struct gs201_pcie_cp,
+						pm_nb);
+	int ret = 0;
+
+	if (event != PM_SUSPEND_PREPARE)
+		return NOTIFY_DONE;
+
+	mutex_lock(&cp->lock);
+	if (cp->link_up || cp->dbi_ok) {
+		dev_info(cp->dev, "sleep refused before the freeze: link %s, PHY %s\n",
+			 cp->link_up ? "up" : "down",
+			 cp->dbi_ok ? "out of isolation" : "isolated");
+		ret = -EBUSY;
+	}
+	mutex_unlock(&cp->lock);
+
+	return notifier_from_errno(ret);
+}
+
 static int gs201_pcie_cp_probe(struct platform_device *pdev)
 {
 	struct device *dev = &pdev->dev;
@@ -2307,6 +2350,15 @@ static int gs201_pcie_cp_probe(struct platform_device *pdev)
 	gs201_cp = cp;
 	mutex_unlock(&gs201_cp_lock);
 
+	if (IS_ENABLED(CONFIG_PM_SLEEP) && cp->data->sleep_guard) {
+		cp->pm_nb.notifier_call = gs201_pcie_cp_pm_notify;
+		ret = register_pm_notifier(&cp->pm_nb);
+		if (ret) {
+			cp->pm_nb.notifier_call = NULL;
+			dev_warn(dev, "no early sleep refusal: %d\n", ret);
+		}
+	}
+
 	dev_info(dev, "modem PCIe ch0 bring-up: %u lanes, steps %s, modem lines %s\n",
 		 cp->num_lanes, allow_hw ? "enabled" : "disabled (allow_hw=0)",
 		 cp->has_lines ? "here" : "left to the modem driver");
@@ -2318,6 +2370,8 @@ static void gs201_pcie_cp_remove(struct platform_device *pdev)
 {
 	struct gs201_pcie_cp *cp = platform_get_drvdata(pdev);
 
+	if (cp->pm_nb.notifier_call)
+		unregister_pm_notifier(&cp->pm_nb);
 	mutex_lock(&gs201_cp_lock);
 	gs201_cp = NULL;
 	mutex_unlock(&gs201_cp_lock);
@@ -2351,14 +2405,12 @@ static void gs201_pcie_cp_remove(struct platform_device *pdev)
  * suspend either and only isolates a link that is already down, as
  * link_down does here. Should those paths ever race, this refuses the
  * sleep. It is a late abort, not a guard: the root port and the modem are
- * children of this device, so the PM core has already run their
- * suspend_noirq when this runs. The PCI core has handled the root port
- * by then (harmlessly, with PCI_DEV_FLAGS_NO_D3 on it) and leaves the
- * modem alone (a syscore device, see cp_bus()); the refusal makes the PM
- * core resume them and the sleep fails, with the modem driver's state of
- * the modem untouched. The PMU's isolation bit is only logged: a phone
- * whose modem never started has never isolated the PHY, and must still
- * sleep.
+ * children of this device, but both are syscore devices (see cp_bus()),
+ * so the PM core leaves them alone; the refusal fails the sleep with the
+ * modem driver's state of the link and the modem untouched. Most refusals
+ * come earlier, from gs201_pcie_cp_pm_notify(). The PMU's isolation bit
+ * is only logged: a phone whose modem never started has never isolated
+ * the PHY, and must still sleep.
  */
 static int gs201_pcie_cp_suspend_noirq(struct device *dev)
 {

What it does not do: it doesn't stop the systemd-sleep hooks. They run before
the kernel is asked (SIT "screen off" with its 10 s hold, the Wi-Fi unload and
reload), so a refused systemctl suspend still runs both. If A comes from
them, this patch leaves data dying, and the ring trace shows where.

Notes for the commit (not written as one, since nothing is verified yet): the
evidence is section 3's candidate B and the two-phone difference. The 9a is
unchanged (sleep_guard). checkpatch and W=1 are still to run in the gate.

6.2 Debug patch: the packet rings at every link power change

This decides between A's two halves and the AP-side alternatives. It is off by
default, and turned on with echo 1 > /sys/module/cpif/parameters/ring_trace.
It reads only the ring pointers in AP DRAM, which is safe with the link down.
It logs at every power-on (after the kick) and every power-off of the link
that gets past the skip checks:

diff --git a/drivers/soc/google/cpif/modem_ctrl_s5100.c b/drivers/soc/google/cpif/modem_ctrl_s5100.c
index 6e6824b6504b..43aff0c4682c 100644
--- a/drivers/soc/google/cpif/modem_ctrl_s5100.c
+++ b/drivers/soc/google/cpif/modem_ctrl_s5100.c
@@ -757,6 +757,54 @@ static void pktproc_kick_rx_queues(struct mem_link_device *mld)
 #endif
 }
 
+/*
+ * Debug (M6 refusal run): the packet rings at every link power change, so
+ * a log shows which side stopped moving. DL: the modem advances rear, the
+ * AP done (and fore, as it refills); UL: the AP advances fore, the modem
+ * rear. The rings are in the AP's DRAM, readable with the link down.
+ */
+static bool ring_trace;
+module_param(ring_trace, bool, 0644);
+MODULE_PARM_DESC(ring_trace, "Log the packet rings at every link power-on and power-off");
+
+static void pktproc_trace_rings(struct mem_link_device *mld, const char *why)
+{
+#if IS_ENABLED(CONFIG_CP_PKTPROC)
+	struct pktproc_adaptor *ppa = &mld->pktproc;
+	unsigned int i;
+
+	if (!ring_trace || !pktproc_check_support(ppa))
+		return;
+
+	for (i = 0; i < ppa->num_queue; i++) {
+		struct pktproc_queue *q = ppa->q[i];
+
+		if (!q || !pktproc_check_active(ppa, i))
+			continue;
+		mif_info("%s: DL%u fore/rear/done %u/%u/%u pass %lld napi %#lx\n",
+			 why, i, *q->fore_ptr, *q->rear_ptr, q->done_ptr,
+			 q->stat.pass_cnt, q->napi_ptr ? READ_ONCE(q->napi_ptr->state) : 0UL);
+	}
+#endif
+#if IS_ENABLED(CONFIG_CP_PKTPROC_UL)
+	{
+		struct pktproc_adaptor_ul *ppa_ul = &mld->pktproc_ul;
+		unsigned int j;
+
+		if (!ring_trace || !pktproc_check_support_ul(ppa_ul))
+			return;
+		for (j = 0; j < ppa_ul->num_queue; j++) {
+			struct pktproc_queue_ul *q = ppa_ul->q[j];
+
+			if (!q || !atomic_read(&q->active))
+				continue;
+			mif_info("%s: UL%u fore/rear/done %u/%u/%u\n", why, j,
+				 *q->fore_ptr, *q->rear_ptr, q->done_ptr);
+		}
+	}
+#endif
+}
+
 static int request_pcie_int(struct link_device *ld, struct platform_device *pdev)
 {
 #define DOORBELL_INT_MASK(x)	((x) | 0x10000)
@@ -1768,6 +1816,7 @@ static int s5100_poweroff_pcie(struct modem_ctl *mc, bool force_off)
 		}
 	}
 
+	pktproc_trace_rings(mld, "link off");
 	set_pcie_msi_int(ld, false);
 
 	if (mc->device_reboot) {
@@ -1921,6 +1970,7 @@ int s5100_poweron_pcie(struct modem_ctl *mc, bool boot_on)
 
 	set_pcie_msi_int(ld, true);
 	pktproc_kick_rx_queues(mld);
+	pktproc_trace_rings(mld, "link on");
 
 	if ((mc->s51xx_pdev != NULL) && mc->pcie_registered && (mc->phone_state != STATE_CRASH_EXIT)) {
 		/* DBG */

How to read it once data dies, with a ping every second on rmnet2:

UL fore UL rear DL rear DL done Means
rises stays stays = rear the modem stopped taking uplink: A, modem side. Next: the bearer state from sitd, then the hooks alone (step 2 below)
rises follows stays = rear uplink leaves, nothing comes back: the bearer or the network (A). Next: sitd's data call list
rises follows rises stays the AP stopped draining: a kernel bug. The napi state bits (SCHED 0x1) show whether NAPI is stuck
stays stays the AP stopped filling the UL ring: the netdev or the UL timer (not seen so far)

7. Attended test on the maintainer's 7 Pro

The maintainer runs every step, on the charger, with NFC polling stopped and
auto-sleep off, as in B4. Before step 1, as AGENTS.md section 1 requires:
tools/nv-backup --check, spm_lvl 5, registered. Before every reboot or push:
systemctl stop omamobile-modem (AGENTS.md section 2).

Step 0 (read-only, low), on today's kernel, no sleep. Find the CPIF
platform device: ls -d /sys/devices/platform/*/pktproc. Then read
pktproc/status, pktproc_ul/status and the msi1-4 rows of
grep 11920000 /proc/interrupts twice, 10 s apart, with ping -i 1 -I rmnet2
running. Expect: DL rear and done advancing together, UL rear following fore,
the msi rows counting. Don't read /sys/kernel/debug/gs201-pcie-cp/status: it
reads ELBI whenever phy_inited is set, also while the PHY is isolated
(pcie-gs201-cp.c :2055-2061), and AGENTS.md says ELBI with the PHY isolated
hangs the SoC.

Step 1 (high): the run that failed, on the 6.1 + 6.2 build. The push is the
human's call. echo 1 > /sys/module/cpif/parameters/ring_trace. Two RTC sleeps
of 45 s, as in B4.1 and B4.2, so the boot has the history of the failed run.
Then the B4.3 loop: ping -i 1 -I rmnet2 and five times
rtcwake -m no -s 45; systemctl suspend, 15 s apart, then 60 s more of ping.

  • Expect for each refused attempt:
    gs201-pcie-cp 11920000.pcie: sleep refused before the freeze: link up, PHY out of isolation,
    with no Freezing user space processes, no PM: noirq suspend of devices failed
    and no cpif: s5100_pm_notifier: Suspend prepare for that attempt. CPIF's
    notifier may run first and log it, and then "Resume done" follows at once.
    Expect no pcieport 0001:00:00.0: Unable to change power state line at the
    real sleeps either.
  • Pass: the ping answers through the loop, and 3/3 at the end. link on/off
    trace lines show DL rear and done moving together. No UFS line, no
    suspend_cp: abort suspend, no s51xx ... power state line.
  • If data dies: read the trace lines around the death against the table in
    6.2. Also read pktproc/status and pktproc_ul/status once, and save the
    sitd journal for the same minute.
  • Stop: no wake 2 minutes after a real sleep (don't send commands; the human
    holds Power about 30 s, then Power); any UFS or I/O line; data dead (stop
    sleeping, collect, stop the modem, reboot).
  • Rollback: rm /etc/omamobile/modem-suspend-enable, stop the modem, reboot to
    the previous slot B set.

Step 2 (medium), only if step 1 loses data with the UL or DL rear stuck (A):
the hooks alone, no kernel sleep.
Same boot after a reboot, same ping. Five
times, 15 s apart: run /usr/lib/systemd/system-sleep/omamobile-modem-sit pre,
then the Wi-Fi unload and reload as omamobile-sleep does them. Write no
/sys/power/state. If data dies here too, the trigger is the SIT "screen off"
hold or the Wi-Fi reload, and the fix belongs in userspace or the modem, not
in #232. If it doesn't, it is the freeze itself: sitd and the modem daemons
stopped for 10-20 ms twice per attempt.

8. What would change these conclusions

  • A DL rear that advances while done stays put (step 1) would make it a
    kernel bug in CPIF's receive path, and the napi bits would show where.
  • The same failure with ring_trace showing nothing wrong on the AP, and
    data back after systemctl restart of sitd alone (no modem restart, which
    AGENTS forbids), would put it on the bearer.

Build check

KERNEL_DIR=~/omamobile-kernel/wt-m6fix KERNEL_OUT=~/omamobile-kernel/out-m6fix bin/omamobile kernel: exit 0 (2026-11-13). pcie-gs201-cp.o and cpif.ko built with no warning in either patched file (default warnings; the W=1 build of the two files is still to do in the gate). checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks on both diffs. Both diffs git apply --check cleanly on wt-modem-sleep HEAD.

@ikegam

ikegam commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the run and the deep analysis! I'll run your 6.1 + 6.2 build on my 7 Pro first (step 1, on the charger) and post the ring trace here. Step 0 on the release kernel looks normal: DL rear and done move together with the ping.

@W4HO-ham

Copy link
Copy Markdown
Contributor

A second phone's result, in case it helps: this series on a Pixel 9a (zumapro, the same link driver and CPIF, a different modem), 2026-11-13. It slept with the modem registered and kept the phone service, which it has never done before.

What ran. omamobile at 2db2bc35beec plus kernel #247, this branch's four commits (f4e5e4846c39), the maintainer's 6.1 and 6.2 patches as posted above, and one line of mine, because everything here is behind sleep_guard and only gs201 sets it: .sleep_guard = true in zumapro_pcie_cp_data. Release 7.3.0-rc6-omamobile-push1-gc50fd8fdf42a. Userspace is main's, without #556: the screen state was told by hand through sitd's socket ("off" before each request, "on" after), and the modem's inhibitor was passed with systemctl suspend -i. On the charger, cable in, LTE with IMS voice and SMS, ring_trace on. The owner was not at the palmtop for these runs; he told me today to run them without him, and each boot ended in systemctl reboot.

Refusals: 14 in one boot, data alive. Every request of the first two runs was refused by 6.1's notifier (sleep refused before the freeze: link up, PHY out of isolation), seven of them with ping -i 1 -I rmnet0 running through all seven (273 of 273 replies), then two without a ping and five more with one (200 of 200). sitd's polls: none failed. No suspend_cp: abort suspend, no pcieport or s51xx power-state line. So the failure of B4.3 did not show here, but this is with 6.1, where a refusal never reaches the freeze; I did not run the head alone.

Real sleeps: 3 of 3, then 19 of 20.

  • Same boot, after the refusals: three requests of 45 s, all entered. 45.2 s and 44.6 s to the RTC alarm, one 15.1 s ended by cp2ap_wakeup (kernel log, entry to exit). After each: CP ONLINE, sitd's polls answered. After the three: ping -I rmnet0 5 of 5.
  • Next boot: twenty requests of 60 s. 19 entered: 17 ran 59.6 to 60.1 s to the alarm, one ended at 27.7 s on cp2ap_wakeup and one at 27.8 s on the PMIC's interrupt; one was refused before the freeze. suspend_stats success 19, fail 1. sitd: no failed poll in the boot. Ping after: 5 of 5.
  • In both boots: no Unable to change power state line, no Link Active, no BME, no UFS error-handler line, no oops; the root port read D0 and the endpoint D3hot at the end. Both systemctl reboots logged the modem's confirmed stop (CP OFFLINE, the marker removed), and the palmtop was back in 25 s.

The same day without the series, for comparison (the 9a's daily kernel with its out-of-tree copy of the link driver, one request per boot): two boots as they were: in one the link was down at the request and the modem was lost as on 11-09 (pcieport … D0 to D3hot, device inaccessible, Data Link Layer Link Active not set in 100 msec, Can't set D0 state, 33 failed polls in four minutes); in the other the endpoint was in D0 at the request and nothing was lost. In three more boots I first wrote 0 to the endpoint's d3cold_allowed; the link was down at the request in all three, and none lost the modem. That fits commit 2's reading, and it only covers a boot's first sleep, as commit 3 says. And both boots that entered a sleep there did not shut down afterwards: systemctl reboot logged nothing more, the palmtop came back 57 s later, and the modem's stop was not recorded. With the series the shutdown after 3 and after 19 sleeps was clean. The NV partitions read SAME against the backup after those two.

What is particular to the 9a. With the cellular data fallback (omamobile-sit-data) running, every request was refused: the 9a's Wi-Fi sleep hook takes wlan0 down for the sleep, the default route moves to the bearer, and the link is up when the notifier looks. With that service stopped, the link went idle 5 or 6 s after "screen off" (12 or 13 link-ups in the 20 s before), and the sleeps above went in. That is userspace and mine to sort out, not this pull request's.

The ring trace. I have 6.2's lines for the whole first boot (about 4,300, every link power change, the refusals included). At the last link-off before the reboot DL rear and done were equal on every queue and UL fore, rear and done too. Say if you want the window around the refusals; it holds no identifiers.

If this lands as it is, I'll send the zumapro line as its own small pull request with these results, or fold it in here if you'd rather, @ikegam.

@W4HO-ham

Copy link
Copy Markdown
Contributor

A follow-up that qualifies my result above: later the same evening the 9a hung once on that test kernel, awake, and the watchdog reset it. I don't think it is the sleep path, but it happened on this build and you should have it.

  • Same kernel (push1-gc50fd8fdf42a), a boot that had run 52 sleep requests in about 70 minutes (49 entered, 3 refused), all with the phone service and data intact. The last sleep had ended normally on cp2ap_wakeup 20.4 s earlier (PM: suspend exit at 4102.06).
  • The retained console's last kernel lines, at 4122.508, are the cp2ap_wakeup edge that starts every link power-up (samsung-pinctrl 154e0000.pinctrl: wake disabled for irq 83, wake enabled for irq 83). In every cycle before it the next line, about 12 ms later, is zuma-pcie-phy 12140000.phy: external PLL lock 1, …; here nothing follows. The bootloader's record: APC Watchdog Early, RST_STAT: 0x1 - CLUSTER0_NONCPU_WDTRESET.
  • No sleep was in progress or being requested, and no pcieport, s51xx, BME or UFS line anywhere in the console before it. The console holds the whole boot: 1,548 link power-ups before this one.

So it reads as a hang in the zumapro link's power-up (the PHY coming out of isolation and being set up), on the in-tree pcie-gs201-cp with phy-zuma-pcie, not in anything this series adds; the series' code isn't on that path. But I can't rule out an interaction, and I have no count yet for the same build without the series: the 9a's daily kernel uses its out-of-tree copy of the link driver, which has cycled this link for weeks without it. That comparison (this build without #232 and the two patches, the modem idling awake for some hours) is the next thing I'll run, and I'll put the result here. The NV partitions were checked SAME earlier in the day; the next boot started the modem after one counted unclean end, and its stop at the following reboot was confirmed.

It doesn't change the sleep results above. It does mean the 9a's "sleeps with the modem up" rests on a link driver that hung once in that boot's 1,548 power-ups (two earlier boots of the same build that day, about an hour together, had none), which I'd want understood before sleep_guard goes on for zumapro.

@ikegam

ikegam commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks again for the patches! I ran 6.1 + 6.2 on my 7 Pro (2026-11-14, 7.3.0-rc6-omamobile-push1-ge1f65b0ec720-dirty: 2db2bc35beec + f4e5e4846 + both diffs). On the charger, ring_trace=1, follow_screen = yes. My data link is rmnet0.

  • First sleeps (B4.1/B4.2): three 45 s RTC sleeps, wall clock 9.7 s, 43.7 s (the RTC) and 4.2 s (cp2ap_wakeup). Ping 3/3 after each.
  • Refusals (B4.3), same boot: five systemctl suspend 15 s apart under a 1 s ping. All 10 were refused before the freeze. Ping 138/141 (three single drops), 3/3 at the end, still registered.
  • No pcieport … D3cold to D0, abort suspend, s51xx or UFS line.

So the data loss didn't reproduce here. One more thing: with follow_screen off, every sleep that got past the check was woken by cp2ap_wakeup within about 1 s (7 of 7, wall clock). I have the ring trace and the kernel log if you want them.

@W4HO-ham thanks for the 9a run, nice to see it match! I'd keep the zumapro line as its own small PR, since this one is checked on the 7 Pro only and the 9a hang is still open.

@W4HO-ham

Copy link
Copy Markdown
Contributor

The 9a's overnight count on the same test build, and one observation about sleep lengths.

300 requests, one boot, modem registered throughout (2026-11-13 evening to 11-14, 7 h 40 min, 7.3.0-rc6-omamobile-push1-gc50fd8fdf42a: this branch's four commits, 6.1, 6.2 and .sleep_guard = true for zumapro). LTE with IMS voice and SMS, Wi-Fi connected, the cellular data service running with the route change of omarchy-mobile#750, on the charger. Each request: "screen off" through sitd's socket, wait for the link to go idle (5 to 15 s, median 10), a 60 s RTC alarm, systemctl suspend -i, "screen on", and a check 20 s later. The owner was not at the palmtop; he told me to run it without him.

  • 299 entered, 1 refused before the freeze (link up, PHY out of isolation); suspend_stats success 299, fail 1.
  • Woken by the PMIC's interrupt (the alarm) 239 times and by cp2ap_wakeup 60 times. By the kernel's own stamps, 249 were asleep 55 to 63 s, 39 under 55 s (27 ended by the modem, 12 by the PMIC) and 11 longer (below). Entry took 0.52 to 0.68 s to the noirq stage, resume 0.72 to 0.87 s.
  • After every one of the 300: CP ONLINE, sitd active, no failed poll, Wi-Fi up, no UFS line. At the end: still registered with IMS voice, ping -I rmnet0 5 of 5.
  • In the whole boot: no Unable to change power state, Link Active, Can't set D0, BME, abort suspend, UFS error-handler, oops, warning or RCU stall line.
  • 6,311 link power-ups in that boot and no hang. With the boot that hung at its 1,549th, that is one hang in over 7,800 power-ups on this build. The comparison I promised (the same build without the series, the modem idling for some hours) hasn't run, and at that rate a few hours of it couldn't tell the two builds apart, so I'll leave it unless one of you wants it. The hang stays open and unexplained.

The observation: 8 of the 299 sleeps ran 20 to 29 s past the alarm. They were asleep 80 to 89 s with the alarm set for 60 (three more ran 63 to 68 s). Five were ended by cp2ap_wakeup and three by the PMIC's interrupt at 86 s. The alarm's interrupt (rtc-alarm0) counted 261 in the boot, one for each sleep that lasted to the alarm, so the alarm was raised and handled each time; it just didn't end those eight sleeps when it was due. The palmtop woke every time. The 300 modem-off sleeps on this kernel's base two days ago were all 59.2 to 60.3 s, so it came either with the modem up, with Wi-Fi connected or with this series; I can't say which yet, and I'm not pinning it on the series. I mention it because sleep lengths are part of what gets measured here, and a late alarm with nothing else to wake the phone would look like a sleep that doesn't end.

@ikegam agreed: the zumapro line goes in its own small pull request once this one has landed, with these counts.

@W4HO-ham

Copy link
Copy Markdown
Contributor

A correction to my reading of the late sleeps above, after looking at the counts again. I wrote that the alarm was raised on time and failed to end those sleeps. The numbers don't show that:

  • The PMIC's interrupt counted 522 in the boot, exactly twice rtc-alarm0's 261, so every PMIC wake there was an RTC alarm.
  • 239 sleeps ended on the PMIC's interrupt, by the RTC from the moment the 60 s alarm was armed: 12 at 28 to 36 s, 223 at 58 to 64 s, 3 at 89 to 90 s and 1 at 121 s.

So the RTC raised an alarm about 30 s early in 12 sleeps and 30 or 60 s late in 4, and in the other late ones the modem woke the palmtop before any alarm came. That looks like a second RTC timer beside rtcwake's (the kernel's alarmtimer programs the RTC at suspend for whatever userspace alarm is next, and something in the modem's userspace may hold one), or the alarm time being set wrong now and then, rather than a wake that didn't get through. I haven't traced it; the rtc and alarmtimer trace events over a run would settle it. It has nothing to do with this series as far as I can see, and it doesn't touch the counts: 299 of 300 entered and woke.

…ink is up

With the modem online, systemd's sleeps were refused only in this driver's
noirq callback: after every task was frozen and every device suspended,
which the PM core then resumed again. On the maintainer's Pixel 7 Pro
(2026-11-13) six such refusals in a row were followed by a downlink that
received nothing until a reboot, and each real sleep logged
"pcieport ... Unable to change power state from D3cold to D0": the root
port had been marked D3cold by a sleep with the link down, so the next
sleep, a refused one included, ran the PCI core's whole resume of it on a
live link.

Refuse earlier, from a PM_SUSPEND_PREPARE notifier, while the link is up or
the PHY is out of isolation: a refusal then costs one notifier call, before
the freeze. Stock gets the same from CPIF's wake lock under Android's
autosleep, which systemd doesn't use. The noirq check stays, for a link
that comes up during the freeze.

Leave the root port out of system sleep too (dev_pm_syscore_device()), as
the previous commit does for the modem. Its windows are this driver's and
are restored at every link up (cp_root_win_restore());
PCI_DEV_FLAGS_NO_D3 stays for runtime PM.

Tested on a second Pixel 7 Pro (2026-11-14, on top of omamobile at
2db2bc3): three real sleeps (9.7 s, 43.7 s to the RTC, 4.2 s), then
five requests under a 1 s ping, all ten attempts refused before the
freeze, ping 138 of 141 and 3 of 3 afterwards, no pcieport line. On a
Pixel 9a with sleep_guard set for zumapro (W4HO): 14 refusals with data
alive, then 22 of 23 sleeps entered.

The notifier and the root port change are Derek Ross's, from his review
of this series.

Suggested-by: Derek Ross <derekross@gmail.com>
Assisted-by: LLM
Signed-off-by: Hiroyuki Ikegami <ikegam@mixallow.net>
@ikegam

ikegam commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Pushed fef165e74: your 6.1 as a commit on top (Suggested-by you, and the message says the code is yours; tell me if you'd rather be the author). 6.2 is left out as debug only. The message has the 7 Pro run from this morning and W4HO-ham's 9a numbers.

@derekross

Copy link
Copy Markdown
Owner

@ikegam, results from my Pixel 7 Pro tonight (2026-11-14), on kernel-2026.11.13 + your head f4e5e4846c39 + the two patches from my analysis comment (7.3.0-rc6-omamobile-push1-g21d56455a7c5), with #556's omamobile-modem 0.1.25 and omamobile-modem-sit 0.3.37, follow_screen = yes, the enable file and runbook marker by hand:

  • Two sleeps in one boot: passed (44 s on the RTC alarm; 9 s, woken by cp2ap_wakeup); ping over rmnet2 3/3 after each.
  • The refusal loop (B4.3): passed. With a 1 s ping on rmnet2, all ten requests were refused before the freeze (sleep refused before the freeze: link up, PHY out of isolation), and the ping ran 76/76 with no gap; 5/5 afterwards, IMS registered. On your head alone (11-13) data died after six noirq refusals; the PM_SUSPEND_PREPARE refusal is what changed.
  • The network wakes the phone (B4.4): a text woke it after 35 s on cp2ap_wakeup and arrived; a call woke it after 21 s on cp2ap_wakeup and rang; a second call answered with audio both ways, and systemctl suspend was refused while it was up.
  • No pcieport … D3cold lines after real sleeps any more (the root port out of system sleep), no UFS line.

So I'd like #232 to carry the first patch (the early refusal in a PM notifier, and the root port as a syscore device); the ring_trace debug patch can stay out. Can you fold it into your series (your sign-off, with an Assisted-by: for the analysis), or would you rather I push it as a commit on top for you to review? Then #232 and #556 can merge, and suspend with the modem up goes to the development channel.

@W4HO-ham

Copy link
Copy Markdown
Contributor

@derekross, the fold is already on this branch, I think: @ikegam pushed fef165e7496f ("refuse system sleep before the freeze while the link is up") on 11-14, with your Suggested-by: and an Assisted-by:. Its change to pcie-gs201-cp.c is your first patch line for line (I compared the added and removed lines against the 6.1 diff from your analysis comment: the same 63), and ring_trace isn't in it. So the head as it stands is the series plus 6.1, without 6.2.

The 9a has run both forms:

  • The setup of your 7 Pro test (f4e5e4846c39 + 6.1 + 6.2, plus the zumapro sleep_guard line): the counts in my comments above, 299 of 300 overnight with the modem registered, and calls and texts that woke the palmtop and rang or arrived.
  • This branch's head (fef165e7496f + the zumapro line, on omamobile at 82f31d3a7 with clk: samsung: zumapro: Keep the HSI0 USI2 and USI3 clocks on #247, release 7.3.0-rc6-omamobile-push1-g716f6d8ef355), since 11-14 10:31 MST: 10 sleeps entered and none refused or failed, 954 link power-ups, with no Unable to change power state, Link Active, Can't set D0, BME, abort suspend, UFS error-handler or oops line. Tonight it runs a night off the charger on the same build, with the modem registered and #556's screen-state hook; I'll post the standby current here in the morning.

The awake hang at a link power-up from my second comment hasn't come back: none in over 15,000 power-ups since, on the two builds.

@derekross
derekross merged commit da171ea into derekross:omamobile Nov 15, 2026
1 check passed
@ikegam

ikegam commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks both, Derek for the runs and the merge, and W4HO-ham for checking the fold line by line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants