diff --git a/docs/10-fps-limit.md b/docs/10-fps-limit.md index ae00beb..4701b7f 100644 --- a/docs/10-fps-limit.md +++ b/docs/10-fps-limit.md @@ -104,31 +104,133 @@ divisor cap reads its target exactly. took the slot; a subscriber whose peer is gone is dropped there, and the connection currently being served stays its own loop's to close. This is what makes a cap change made mid-session reach an already-connected shim at all. -- **gamescope's limiter enforces it; the shim declares FIFO while it is set.** On a - PC, gamescope's WSI layer declares FIFO to gamescope (`gamescope_swapchain. - set_present_mode`) while its limiter is engaged, gamescope latches one FIFO commit per - limited vblank (`steamcompmgr.cpp` `handle_done_commits_xwayland`), and the client is - paced by `wl_buffer.release`. The shim is that layer here: on gamescope's override - route it declares FIFO on every commit while the daemon's `FrameCapNote` carries a - cap, the guest's own mode otherwise, and mailbox while frame generation runs - (`icd_shim/src/proxy.rs` `live_present_mode`). `MESA_VK_WSI_PRESENT_MODE`, a container - default for the X11 path, no longer decides that route: honouring it declared every - commit mailbox, which the limiter never holds, and that is why an acquire gate on the - shim's own clock used to stand in for the cap. That time gate - (`scanout::next_acquire_after`) now runs only while frame generation sets the source - rate, and on the opt-in X11 Present route, which has no compositor declaration. - gamescope's fixed-rate limiter passes every `refresh / target`-th vblank with an integer - divisor, so a cap that does not divide the display's rate (40 or 45 on 60 Hz, 45 or 50 - on 120 Hz) runs at the next divisor rate above it; that is upstream gamescope's - behaviour and is left to it for now. +- **Who enforces it follows the present mode, and the mode follows the container.** + On gamescope's override route the shim declares to gamescope, per commit + (`gamescope_swapchain.set_present_mode`), the container's + `MESA_VK_WSI_PRESENT_MODE` (mailbox is the container default; the user may set + fifo, relaxed or immediate), or the guest's own mode when the env is absent + (`icd_shim/src/proxy.rs` `effective_present_mode`, `live_present_mode`). + gamescope's limiter holds only commits declared FIFO (`wlserver_surface_is_fifo`): + it latches one per limited vblank (`steamcompmgr.cpp` `handle_done_commits_xwayland`) + and the client is paced by `wl_buffer.release`. Any other mode is taken at the + next composite, so there the shim's acquire gate is the cap: + + | container mode | cap | declared | what limits the rate | + |---|---|---|---| + | mailbox (default) | none, guest asked for FIFO (in-game VSync on) | fifo (`source=guest`) | the display (FIFO) | + | mailbox (default) | none, guest asked for mailbox or immediate | mailbox | nobody | + | mailbox | at the display's rate | fifo (`source=cap-rate`) | the shim's acquire gate, every vblank; gamescope latches one FIFO commit per vblank | + | mailbox | below the display's rate | mailbox | the shim's acquire gate, on the display's vblank grid | + | fifo | none | fifo | the display (FIFO) | + | fifo | set | fifo | gamescope's divisor limiter | + | any, frame generation on | any | mailbox | the source-rate gate (`docs/17`) | + + FIFO-relaxed (`relaxed`) and immediate behave as mailbox in this table. No env: the guest's + own mode takes the env's place. "At the display's rate" is a cap within 0.5 % + of the measured rate (60 on a 59.97 Hz panel); a cap above it runs on the + gate's own clock under the container's mode. + + **With no cap a guest's FIFO request wins over the container's default.** Every + container carries `MESA_VK_WSI_PRESENT_MODE=mailbox` as a stamped default the user + did not choose. With no cap, honouring it turned a game's own VSync (a FIFO + request) into an uncapped mailbox session, which is neither master's behaviour nor + what the user asked for. A FIFO request (FIFO-relaxed too) is a specific wish and + is declared as asked (`source=guest`); a mailbox or immediate request means "do not + make me wait", and the container's mailbox is the substitute, as on the X11 route + (an uncapped immediate guest gets mailbox, not tearing). An env of `fifo` is an + explicit wish and wins in every case. Under a cap the table above applies whatever + the guest asked for. + + **The user's control is `MESA_VK_WSI_PRESENT_MODE`.** It sits in the container's + environment, written from the driver's present-mode setting, and takes Mesa's + names: `fifo`, `relaxed` (FIFO-relaxed), `mailbox`, `immediate`. Anything else, + `fifo_relaxed` included, is rejected as Mesa rejects it, the guest's own mode is + used, and the shim logs `Invalid MESA_VK_WSI_PRESENT_MODE value!` whether or not + `GNDRM_LOG_VKSHIM` is set, so a typo shows in a shipped session. It is + overridden in one case: a non-FIFO value under a cap at the display's rate is + declared FIFO (`source=cap-rate`, below). Mesa reads the same variable, so it + also sets the mode a zink guest's own swapchain asks for below the shim. + + **Why mailbox is not declared at the display's rate.** At that cap the panel + shows one frame per vblank under either mode, and under mailbox the shim has to + keep two commits out of one gamescope composite. It cannot see when gamescope + latches: gamescope takes commits at its wakeup, the target vblank minus its + rolling draw time and red zone (`vblankmanager.cpp` 142-199), and that draw time + is a peak hold that jumps on any slow draw (164-165) and decays per wakeup + (167); draw time runs to the page flip queued (`DRMBackend.cpp` 4465), on this + stack through the daemon's commit. The client sees only `presented` (stamped + with the wakeup's target vblank, `steamcompmgr.cpp` 8140, 8168) and `discarded`, + one inequality per frame. Measured on a Retroid Pocket 6 (DTD, cap 60, in-game + VSync off, a guest pacing itself at ~59.3 fps on a 59.98 Hz panel): mailbox with + the gate gave 58.3 fps on `SurfaceFlinger` with ~1 commit a second replaced in + gamescope's mailbox; a one-commit-per-window queue in the shim at a guessed + 2.5 ms boundary gave 59.0 with ~0.8 a second still replaced; FIFO on master + gives 59.22 on the same scene. Under FIFO gamescope latches one commit per + vblank at its own wakeup (`steamcompmgr.cpp` 8028-8049), its limiter's divisor + at this cap is 1, and the gate keeps the guest from getting ahead, so the FIFO + queue stays about one frame deep. Below the display's rate the gate's releases + are at least two vblanks apart and two frames in one composite are rare, so + mailbox stays (FIFO with gamescope's limiter quantized a 32 fps game to 30 at a + 60 cap on 120 Hz; mailbox with the gate did not). + + The gate (`icd_shim/src/vsync_gate.rs`) releases acquires on the vblank grid + gamescope composites from, not on a clock of its own: the daemon's `VsyncNote` + latch grid (measured period and phase) moved earlier by the lead the daemon + stamps flip events with (`stamp_lead_ns`). A cap at the display's rate releases + every vblank (60 on a 59.97 Hz panel is 59.97, one frame per latch); a cap that + divides the rate, every `refresh / cap`-th vblank (30 on 60, 60 on 120); any + other cap below the rate, the vblank nearest each instant of an ideal + `1e9 / cap` timeline, which averages the cap on vblank boundaries (45 on 60 + releases 3 of 4 vblanks); a cap above the rate, the cap's own clock. A frame + that arrives after its release point starts at once, and the next release is + the next point of the schedule, so a game below the cap keeps its own rate. A + gate on the shim's own `1e9 / N` clock, as before 7bdcbae, beat against the + panel: 57-59 fps on `SurfaceFlinger` at a 60 cap on a 59.97 Hz panel. + + The gate needs the stamp lead. Until a note carrying one has arrived (no note + yet, or a daemon that predates the field and sends 0), a capped swapchain whose + mode is not FIFO is declared FIFO and gamescope's limiter is the cap, as before + the container's mode was honoured here; the census reads `source=cap-compat`. + Released on the latch points themselves, frames start ~2 ms after gamescope's + wakeup and straddle its composite (Retroid Pocket 6, cap 60: 58.3 fps and 102 + frames replaced in gamescope's mailbox per minute at 60 Hz, 13 % of publishes + late at 120 Hz). A rollout that ships only the payload (the shim) therefore keeps + today's pacing until the app ships the daemon that sends the lead. A swapchain + created before the lead is known keeps the slot count of the guest's own request + until it is recreated. + + The census says which path ran. Ungated, once when frames first flow and again + whenever the declared mode, its source, the cap or the gate changes: + `present mode: declared=… requested=… source=env|guest|cap-rate|cap-compat + route=gamescope cap=… gate=vsync|clock|off lsfg=…` (`source=guest`: the guest's + own mode, no env or a FIFO request with no cap; `source=cap-rate`: declared FIFO + because the cap is the display's rate, with `gate=vsync`; `source=cap-compat`: + declared FIFO for want of the stamp lead, with `gate=off`; `gate=clock` is frame + generation's source rate or a cap above the display's rate). With + `GNDRM_LOG_VKSHIM=1` the same line repeats every 10 s with the counters a device + run checks the pacing against: `acquires= commits= declared_fifo= + declared_other= releases= feedback= presented= discarded= fallback= + gate_holds= gate_late= skipped= slot_waits= commit_lat_us= spacing=[…]`. They + are cumulative except the latency and the spacing: `gate_holds`, acquires the + gate held; `gate_late`, acquires that started after their release point (not + the daemon's `late=` on its `vsync: check` line, which counts publishes that + missed their latch); `skipped`, release points moved past for them; + `slot_waits`, acquires that passed the gate and then waited on a compositor + release for a free image; `commit_lat_us`, the smoothed acquire-to-commit time; + `spacing=[<15:… 15-16.2:… 16.2-17.2:… 17.2-20:… >20:…]`, the spacing of + consecutive commits in ms since the last line, the game's own cadence as it + reaches gamescope. Off the override route (the opt-in X11 + Present route) the env decides the mode as in Mesa and the gate runs on the + cap's own clock, as before. - **`-r` is not part of it, and no launch argument carries the limit.** The app sends the cap to the shim's gate; `--framerate-limit` is never built. Open points: -- **Divisor rounding bites the gamescope route only.** `nVblankDivisor = +- **Divisor rounding bites the FIFO route only.** `nVblankDivisor = nRefreshHz / nTargetFPS` (`steamcompmgr.cpp:7040`, and `:10621` for the window's - limited cycle) lands on the nearest divisor of the grid; the shim's floor is - exact (measured: 60.00 commits/s at a 60 cap). + limited cycle) lands on the nearest divisor of the grid, so with the container + set to fifo a 40 or 45 cap on 60 Hz runs at 60. Under mailbox the shim's gate + averages a non-divisor cap on vblanks instead. - **The client-visible refresh is the mode's, and the mode is the display's.** `-r` reaches `g_nOutputRefresh` only transiently (`drm_unset_mode`, `:3680`), before `drm_set_mode` (`:3728`) overwrites it with the mode's rate — diff --git a/docs/15-fps-limiter-pacing-and-vsync.md b/docs/15-fps-limiter-pacing-and-vsync.md index 321c9ed..0564bc6 100644 --- a/docs/15-fps-limiter-pacing-and-vsync.md +++ b/docs/15-fps-limiter-pacing-and-vsync.md @@ -11,15 +11,23 @@ cursor, scan-out and mode patches. | place | what it holds | set by | |---|---|---| -| the shim's declared present mode | FIFO while a cap is set, so gamescope's limiter holds each frame to its limited vblank | `FrameCapNote{fps}` from the daemon (`docs/10`) | -| gamescope's own limiter | `nRefreshHz / nTargetFPS`, a divisor of the mode's rate | `debug_set_fps_limit`, sent only when generation is off | +| the shim's declared present mode | the container's `MESA_VK_WSI_PRESENT_MODE` (else the guest's mode); with no cap, a guest's FIFO request over a non-FIFO env; FIFO under a cap at the display's rate; mailbox under frame generation | the container, the guest and the cap (`docs/10`) | +| gamescope's own limiter | `nRefreshHz / nTargetFPS`, a divisor of the mode's rate; holds FIFO commits only | `debug_set_fps_limit`, sent only when generation is off | +| the shim's acquire gate | the cap, on the display's vblank grid, unless the container's mode is FIFO | `FrameCapNote{fps}` and `VsyncNote` from the daemon | | the presenter's latch grid | the display's latch points, as measured | `VsyncNote` from the daemon (`docs/17`) | -They work together: the app hands gamescope the cap, gamescope's limiter passes every -`nRefreshHz / target`-th vblank (`nVblankDivisor = nRefreshHz / min(target, nRefreshHz)`), -and the shim's FIFO declaration is what makes the limiter hold the guest's commits. Under -frame generation the app disarms gamescope's limiter and the shim's time gate holds the -source rate instead (`docs/17`). +Which one holds the cap follows the declared mode. Declared FIFO (the container set +fifo, or no env and a FIFO guest), gamescope's limiter passes every +`nRefreshHz / target`-th vblank (`nVblankDivisor = nRefreshHz / min(target, nRefreshHz)`) +and holds the guest's commits to it. Declared mailbox, immediate or FIFO-relaxed (mailbox +is the container default), the limiter does not hold them and the shim's acquire gate +releases the guest's frames on the display's vblank grid. A container in one of those +modes under a cap at the display's rate is declared FIFO with the gate kept +(`source=cap-rate`): gamescope latches one commit per vblank at its own wakeup, which the +shim cannot see under mailbox (`docs/10`), and the gate keeps the FIFO queue about one +frame deep. Under frame generation the app +disarms gamescope's limiter and the shim's time gate holds the source rate instead +(`docs/17`). ## 2. The mode is the display's rate @@ -40,9 +48,9 @@ that rate is the display's. - **FIFO.** A present is requeued until the display releases a buffer, so the guest is paced to the panel whether or not it asked to be. On gamescope's override route the - mode declared per commit is FIFO while a cap is set, the guest's own otherwise, and - mailbox under frame generation; `MESA_VK_WSI_PRESENT_MODE` decides it only off that - route (`docs/10`). On that route the shim answers `vkWaitForPresentKHR` from + mode declared per commit is the container's `MESA_VK_WSI_PRESENT_MODE` (the guest's + own without it), cap or no cap, and mailbox under frame generation (`docs/10`). On + that route the shim answers `vkWaitForPresentKHR` from gamescope's presentation feedback, as Mesa's Wayland WSI does: every commit that carries a wait requests a `wp_presentation_feedback`, and the wait is answered on `presented` (at the refresh it reports) or `discarded`. The release comes a vblank @@ -58,12 +66,12 @@ that rate is the display's. unfocused game, or one behind a dialog, is therefore still answered once per vblank. Mesa blocks until the event comes; the shim answers a feedback that never comes 100 ms after the commit, a backstop for a compositor fault, not a path a working - session takes (the census's `fallback=` counts it). Without the global + session takes (the `fallback=` counter of the periodic `present mode:` line counts it; that line repeats with `GNDRM_LOG_VKSHIM=1`, while the ungated one prints at session start and on a change of the declared mode, its source, the cap or the gate, `docs/10`). Without the global (or off the override route) the wait is answered on the release, or two refresh periods after the commit when no release report arrives first. - **Mailbox and immediate.** A guest that asked for either does not wait, and neither - would one whose frames go out as mailbox (frame generation, or the env off the - override route while the guest believes it has FIFO), so those waits are answered at + would one whose frames go out as mailbox or immediate (frame generation, or the + container's env while the guest believes it has FIFO), so those waits are answered at the commit rather than holding the guest's frame-latency loop for two periods (`answers_present_wait_at_commit`). - **A replaced frame.** Under mailbox a frame can be replaced before it is shown. It is @@ -114,15 +122,32 @@ that rate is the display's. deadline stamp on that device). The stamps are earlier than the light by the lead plus the compositor's pipeline; consumers read differences between stamps (frame time), which a constant offset does not touch. -- **The guest is paced by gamescope's FIFO latch, not by a gate of its own.** With a - cap set, the shim declares FIFO on every commit, as gamescope's WSI layer does on a PC - while the limiter is engaged; gamescope latches one FIFO commit per limited vblank, on - the display's vblank stream above, and hands the previous buffer back with - `wl_buffer.release`, which is what frees the guest's next acquire. A gate on the shim's - own clock (exactly 1e9 / N ns) beat against the display's 59.97-59.99 Hz and, with the - env's mailbox declaration, left no queue to absorb a late frame. A cap that is not the - display's rate over a whole number runs at gamescope's next divisor rate above it - (`docs/10`). +- **Under FIFO the guest is paced by gamescope's FIFO latch; otherwise by a gate on + the display's vblank grid.** Declared FIFO with a cap set, gamescope latches one FIFO + commit per limited vblank, on the display's vblank stream above, and hands the + previous buffer back with `wl_buffer.release`, which is what frees the guest's next + acquire; a cap that is not the display's rate over a whole number runs at gamescope's + next divisor rate above it. Declared FIFO because the cap is the display's rate + (`source=cap-rate`), the gate below also runs, so the guest starts at most one frame + per vblank and the FIFO list stays about one frame deep. Declared anything else, + gamescope takes the newest commit at each composite and the shim's acquire gate is + the cap (`icd_shim/src/vsync_gate.rs`). + Its release points are the instants the flip events are stamped at: the daemon's + latch grid (`VsyncNote` anchor and measured period) minus the stamp lead + (`stamp_lead_ns`). gamescope wakes to composite just before each (wakeup = stamp + + interval − draw time − red zone), so a frame released there starts right after + gamescope took the previous one and has the period, less gamescope's margin, to reach + the next composite. A cap at the display's rate releases every vblank, a divisor cap + every `refresh / cap`-th, any other cap below the rate the vblank nearest each instant + of an ideal `1e9 / cap` timeline; a frame late for its point starts at once and the + schedule goes on, so a guest below the cap is never quantized to a divisor; a phase + push never lets two releases into one vblank. The gate this replaces ran on the shim's + own clock (exactly 1e9 / N ns) and beat against the display's 59.97-59.99 Hz + (57-59 fps at a 60 cap). A cap above the rate runs on the cap's own clock + (`gate=clock`, also frame generation's source rate). Without the stamp lead (no + note yet, or a daemon that predates it) the gate does not run: a capped swapchain is + declared FIFO and gamescope's limiter holds it (`source=cap-compat`), so a + payload-only rollout keeps today's pacing until the app ships the daemon. - **Fallbacks.** On API 29-32 Choreographer reports the vsync time only (`AChoreographer_postFrameCallback64`): its measured period is used with the latch grid's phase (`vsync.rs`, from `ASurfaceTransactionStats_getLatchTime`), or the vsync @@ -136,16 +161,24 @@ that rate is the display's. ## Behaviour changes in gamescope sessions -- **FIFO games without a cap are vsynced.** The env's mailbox used to be declared for - every swapchain; now a guest that asks for FIFO gets FIFO, as on a PC. -- **Mailbox and immediate games keep the mode they asked for** when no cap is set, and - are declared FIFO while one is. -- **`MESA_VK_WSI_PRESENT_MODE` is ignored on the gamescope route** (it still applies to - the X11 Present route), including a value the user picked in the driver settings; on a - PC it reaches only Mesa below gamescope's layer. -- **Caps that do not divide the display's rate** run at gamescope's divisor rate: - `floor(refresh / target)` vblanks, so 40 or 45 on 60 Hz runs at 60 and 50 on 120 Hz at - 60. +- **The container's present mode is declared, except under a cap at the display's + rate.** `MESA_VK_WSI_PRESENT_MODE` decides the mode on the gamescope route as on the + X11 Present route; without it the guest's own mode is declared. A container left at + the mailbox default runs, with no cap, FIFO for a guest that asked for FIFO (its + own VSync, `source=guest`) and mailbox for any other guest, and mailbox capped by the shim's + vblank-grid gate under a cap below the display's rate. Under a cap at the display's + rate it is declared FIFO with the gate kept (`source=cap-rate`), because the shim + cannot keep two mailbox commits out of one gamescope composite (`docs/10`). One set + to fifo runs FIFO under gamescope's limiter. +- **A game below the cap keeps its own rate under mailbox** (caps below the display's + rate). Under FIFO and gamescope's + limiter each frame is held to whole limited vblanks, so a game below the cap judders + between divisors of the display's rate, and one that waits on every present holds at + the next divisor below (refresh / 2 at a refresh-rate cap); the gate starts a late + frame at once instead. +- **Caps that do not divide the display's rate** run at gamescope's divisor rate under + FIFO: `floor(refresh / target)` vblanks, so 40 or 45 on 60 Hz runs at 60 and 50 on + 120 Hz at 60. Under mailbox the gate averages them on vblanks. - **Queue latency.** Under FIFO a guest faster than the cap fills its free slots, so up to `slots − 1` frames can wait in gamescope's FIFO list: at low caps that is up to that many limited periods of input latency, as on a PC. A guest that uses present wait @@ -154,15 +187,19 @@ that rate is the display's. mailbox discipline with the slot count it was created with (`minImageCount + 1`, one fewer than a swapchain created with generation on) until the guest recreates it. - **Toggling frame generation** sends the cap change and the generation settings - separately; between the two, for a moment, the guest's own mode is declared with no - gate, so a mailbox/immediate guest can run uncapped for a few frames. + separately; between the two, for a moment, the swapchain's mode is declared with no + cap, so a mailbox/immediate swapchain can run uncapped for a few frames. ## 4. What the daemon owns - The latch grid: `ASurfaceTransactionStats_getLatchTime` on every publish, the phase - tracked in `vsync.rs`, and a `VsyncNote` (`anchor_ns, period_ns, lead_ns`) pushed - whenever its phase moves by more than 300 µs or its period by more than 1 µs, and once - on subscribe. Its period is the one measured from Choreographer while that is locked + tracked in `vsync.rs`, and a `VsyncNote` (`anchor_ns, period_ns, lead_ns, + stamp_lead_ns`) pushed whenever its phase, lead or stamp lead moves by more than + 300 µs or its period by more than 1 µs, and once on subscribe. `stamp_lead_ns` is the + flip events' lead (below), so `anchor_ns - stamp_lead_ns` is gamescope's vblank grid, + the one the shim's acquire gate releases on. The daemon sends at least 1 ns while + the flip events follow a grid, so 0 means "not known" (no grid, or a daemon that + predates the field), and the shim then declares FIFO under a cap instead of gating. Its period is the one measured from Choreographer while that is locked (the mode's advertised rate otherwise), so frame generation extrapolates the display's real rate. Its anchor is also the flip timeline's phase where Choreographer gives no deadline, and its fallback. @@ -187,11 +224,26 @@ measures — the latch grid to the presenter, the vsync timeline in the flip eve ## 5. Measured on device +Retroid Pocket 6 (59.97-59.98 Hz), DTD title scene, cap 60, against master's FIFO +declaration on the same scene (59.22 fps, 1.3 % doubles): + +- **A cap at the display's rate, declared FIFO with the vblank-grid gate** + (`source=cap-rate gate=vsync`): in-game VSync off, 59.199 fps on `SurfaceFlinger` + and 1.3 % doubles over 60 s, shim commits equal to the daemon's publishes within + 0-2 per 10 s (the guest paces itself at ~59.3); VSync on, 59.983 fps and no + doubles. +- **The same cap declared mailbox with the gate** lost ~1 commit a second inside + gamescope's mailbox (58.3 fps, 2.8 % doubles), which is why it is declared FIFO + (`docs/10`). + Retroid Pocket Nova (120 Hz panel, a 60 Hz mode available), Hades II, Vulkan through -zink: +zink, measured before the vblank-grid gate, when the acquire gate ran on the cap's +own clock for every capped session (it now does so only for frame generation's +source rate, a cap above the display's rate and the X11 Present route; a capped +gamescope session is held as in §3): -- **The gate holds the cap.** Cap 40: 40.2 acquires a second over 30 s, 0.1 skipped - batches a second, `gate_avg_us` ≈ 22 ms against a 25 ms period. +- **The clock gate held the cap.** Cap 40: 40.2 acquires a second over 30 s, 0.1 + skipped batches a second, `gate_avg_us` ≈ 22 ms against a 25 ms period. - **Generation follows the multiplier, the display does not.** Cap 40 with frame generation at x2 / x3 / x4 produces and counts 80 / 120 / 160 frames a second, while the mode stays at the display's rate (120 Hz, or 60 Hz on a 60 Hz panel). diff --git a/docs/17-lsfg-frame-generation.md b/docs/17-lsfg-frame-generation.md index eb03fff..720fff2 100644 --- a/docs/17-lsfg-frame-generation.md +++ b/docs/17-lsfg-frame-generation.md @@ -90,7 +90,11 @@ drifting pattern. Three rules keep one frame per period, in order: 64 fps on 120 Hz, flips between x2 and none from batch to batch), and the display never reshapes it: a 40 cap runs 40 acquires a second on a 60 Hz panel as much as on a 120 Hz one, and the multiplier decides what is generated on - top. The panel's rate is the one the app reported (`LsfgConfig.refresh_mhz`), + top. The source rate is held by the acquire gate on the cap's own clock + (`scanout::next_acquire_after`, `gate=clock` in the census), not on the + display's vblank grid: the vblank-grid gate of `docs/10` covers a cap without + frame generation only, and generated frames are placed on the latch grid + below. The panel's rate is the one the app reported (`LsfgConfig.refresh_mhz`), not a measurement: while the daemon is the only thing publishing, the latch samples are spaced by its own rate, so a panel SurfaceFlinger runs below that rate (an LTPO panel that dropped to 60 Hz) is still paced at the reported rate diff --git a/docs/README.md b/docs/README.md index 3fd0f68..8f9847d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -44,7 +44,7 @@ winlator X11 path (which remains the default for normal launches). 12. [12-present-latency-budget.md](12-present-latency-budget.md) — the present-latency budget: what each frame on the paced path pays, and why the flip event is emitted with the publish. 13. [13-single-layer-windowing.md](13-single-layer-windowing.md) — the window/layer model on the single-layer path: what plays layer 0/1/N here, why the content-override mechanism exists, the per-patch layer audit, and the target architecture. 14. [14-guest-surface-commit-stall.md](14-guest-surface-commit-stall.md) — the guest-surface route's queued defect: the commit stream stalls when the guest's Vulkan swapchain comes up, the two candidate causes, and the instrumentation that would decide between them. -15. [15-fps-limiter-pacing-and-vsync.md](15-fps-limiter-pacing-and-vsync.md) — where a frame's rate is decided: gamescope's limiter on the shim's FIFO declaration, and the presenter's latch grid; and where vsync is real. +15. [15-fps-limiter-pacing-and-vsync.md](15-fps-limiter-pacing-and-vsync.md) — where a frame's rate is decided: the container's present mode, gamescope's limiter under FIFO or the shim's gate on the display's vblank grid otherwise, and the presenter's latch grid; and where vsync is real. 16. [16-hdr-verification-and-gaps.md](16-hdr-verification-and-gaps.md) — what HDR verification measured, and the gaps left open. 17. [17-lsfg-frame-generation.md](17-lsfg-frame-generation.md) — plan for a `gamenative_drm` LSFG backend: the app's own engine inside the ICD shim, reading the container's `Lossless.dll` and building its own shader cache, reusing `LsfgVkManager` and the quick-menu tab, with the artifacts and the measurements that decide it. 18. [18-lsfg-driver-facts.md](18-lsfg-driver-facts.md) — the properties the LSFG presenter rests on, each one silent when broken: queue-family matching, the engine writing only into its own images, the blit capture, destination depth, guest-only release pacing, the `Rgba8` storage binding, and the settings channel that carries the DLL path. diff --git a/icd_shim/src/lib.rs b/icd_shim/src/lib.rs index f836233..94a31fa 100644 --- a/icd_shim/src/lib.rs +++ b/icd_shim/src/lib.rs @@ -121,6 +121,7 @@ pub mod segv; pub mod thunk; pub mod vk; pub mod vk_dispatch_gen; +pub mod vsync_gate; pub mod wlproto; pub mod x11; diff --git a/icd_shim/src/proxy.rs b/icd_shim/src/proxy.rs index c09512b..f38e07a 100644 --- a/icd_shim/src/proxy.rs +++ b/icd_shim/src/proxy.rs @@ -233,13 +233,32 @@ const X11_TRACE_N: usize = 24; /// so it is bounded to the first few presents only. const X11_VIEW_N: u32 = 3; static RATE_STALL: Mutex> = Mutex::new(None); -// The ungated present-mode census (see `mode_census`): commits declared to -// gamescope as FIFO and as anything else, and every compositor release. Kept -// apart from the gated WSI census, whose counters only move while -// `GNDRM_LOG_VKSHIM` is set. +// The present-mode census's counters (see `mode_census`): commits declared to +// gamescope as FIFO and as anything else, and every compositor release. They +// count whether or not `GNDRM_LOG_VKSHIM` is set, unlike the WSI census's, so +// the periodic line reads the session from its start when the switch is on. static DECL_FIFO_N: AtomicUsize = AtomicUsize::new(0); static DECL_OTHER_N: AtomicUsize = AtomicUsize::new(0); static REL_ALL_N: AtomicUsize = AtomicUsize::new(0); +// Gated acquires that found no free image once past the gate and waited on a +// compositor release (the ungated census's `slot_waits=`). +static SLOT_WAIT_N: AtomicUsize = AtomicUsize::new(0); +// Spacing of consecutive guest-frame commits since the last census line, in +// buckets of <15, 15-16.2, 16.2-17.2, 17.2-20 and >20 ms: the guest's own +// cadence as it reaches gamescope (`commit_spacing_bucket`). +static COMMIT_GAPS: [AtomicUsize; 5] = [const { AtomicUsize::new(0) }; 5]; +static LAST_COMMIT_NS: AtomicU64 = AtomicU64::new(0); + +/// The `COMMIT_GAPS` bucket of a spacing of `gap_ns`. +fn commit_spacing_bucket(gap_ns: u64) -> usize { + match gap_ns { + 0..15_000_000 => 0, + 15_000_000..16_200_000 => 1, + 16_200_000..17_200_000 => 2, + 17_200_000..=20_000_000 => 3, + _ => 4, + } +} static RATE_MODE: Mutex> = Mutex::new(None); const MODE_CENSUS_EVERY: Duration = Duration::from_secs(10); static RATE_WAIT: Mutex> = Mutex::new(None); @@ -833,6 +852,7 @@ unsafe fn lsfg_destinations(sc: *mut GnSwapchain, count: usize) -> bool { acquired: false, in_flight: false, acq_seq: 0, + acquired_ns: 0, ino: 0, ahb: ptr::null_mut(), }); @@ -1709,6 +1729,21 @@ fn worker_main(sc: *mut GnSwapchain) { unsafe fn commit_real(sc: *mut GnSwapchain, idx: i32, fence_ms: u128) { unsafe { let s = (*sc).slots.as_mut_ptr().add(idx as usize); + let now = monotonic_ns(); + let last = LAST_COMMIT_NS.swap(now, Ordering::Relaxed); + if last != 0 && now > last { + COMMIT_GAPS[commit_spacing_bucket(now - last)].fetch_add(1, Ordering::Relaxed); + } + // How long the frame took from the gate's release to here, its render + // fence waited, for the census. + if (*s).acquired_ns != 0 { + let took = monotonic_ns().saturating_sub(core::mem::replace(&mut (*s).acquired_ns, 0)); + let est = (*sc).commit_latency_ns.load(Ordering::Relaxed); + (*sc).commit_latency_ns.store( + crate::vsync_gate::smooth_latency(est, took), + Ordering::Relaxed, + ); + } // Tell the daemon which present id this frame carries, before the commit // that names its buffer in an atomic commit. The worker may wait for an // allocation exchange in flight, which is bounded by that exchange's own @@ -1732,10 +1767,10 @@ unsafe fn commit_real(sc: *mut GnSwapchain, idx: i32, fence_ms: u128) { let mut feedback_token = 0u64; if (*s).wait_id != 0 { // A guest that asked for mailbox or immediate does not wait, and - // neither would one whose frames go out as mailbox (frame - // generation): both are answered here. A FIFO guest declared FIFO - // keeps the release-or-grace answer. - let at_commit = answers_present_wait_at_commit((*sc).present_mode) + // neither would one whose frames go out as mailbox or immediate + // (the container's mode, or frame generation): both are answered + // here. A FIFO guest declared FIFO keeps the release-or-grace answer. + let at_commit = answers_present_wait_at_commit((*sc).requested_mode) || answers_present_wait_at_commit(live_present_mode(sc)); let id = if at_commit { let _g = (*sc).shared.lock().unwrap(); @@ -1957,18 +1992,42 @@ unsafe extern "C" fn gn_vkCreateSwapchainKHR( )); } } - // On gamescope's override route the guest's own mode is the one this - // swapchain keeps (see `effective_present_mode`); what each commit declares - // is decided per present (`live_present_mode`). + // The container's mode, else the guest's, is the one this swapchain keeps + // (see `effective_present_mode`); what each commit declares is decided per + // present (`live_present_mode`). let gamescope_route = unsafe { !(*surf).swapchain_obj.is_null() }; - let mode = effective_present_mode(ci, gamescope_route); + let (mode, mode_source) = effective_present_mode(ci); // Mesa's wayland WSI derives minImageCount from the present mode // (wsi_common_wayland.c:1188): the extra image exists so one can be queued // for scan-out while the app renders the next. Mailbox therefore carries // one more slot than fifo; fifo keeps its existing count. Frame generation // presents as mailbox, so a swapchain made while it runs keeps that count. + // On the override route the container's mode decides the count only when + // the daemon's stamp lead is already known; before that a capped swapchain + // is declared FIFO (`live_present_mode`), so the guest's own request + // decides it, as it did before the container's mode was honoured there. A + // lead that arrives later changes the declaration on the next present but + // not the count, which holds until the guest recreates the swapchain. A + // cap at the display's rate declares FIFO but keeps this count: the acquire + // gate holds the guest to one frame per vblank, so the spare image does not + // deepen gamescope's FIFO queue. With no cap a guest that asked for FIFO + // over the container's default is declared FIFO (`choose_live_mode`), so + // its own request decides the count too, as on master. + let uncapped_vsync_request = mode_source == ModeSource::Env + && crate::scanout::cap_fps_note() == gndrm_uapi::wire::FPS_UNCAPPED + && matches!( + guest_present_mode(ci), + VK_PRESENT_MODE_FIFO_KHR | VK_PRESENT_MODE_FIFO_RELAXED_KHR + ); + let slot_mode = if gamescope_route + && (crate::vsync_gate::current_grid().is_none() || uncapped_vsync_request) + { + guest_present_mode(ci) + } else { + mode + }; let extra_slot = u32::from( - mode == VK_PRESENT_MODE_MAILBOX_KHR || (gamescope_route && crate::lsfg::live().0), + slot_mode == VK_PRESENT_MODE_MAILBOX_KHR || (gamescope_route && crate::lsfg::live().0), ); let mut n_slots = ci.minImageCount + 1 + extra_slot; if n_slots < 2 { @@ -1988,6 +2047,8 @@ unsafe extern "C" fn gn_vkCreateSwapchainKHR( extent: ci.imageExtent, fourcc, present_mode: mode, + requested_mode: guest_present_mode(ci), + mode_source, // The slot image carries the usage the guest asked for, plus TRANSFER_SRC: // the engine copies the presented frame out of it, and generation can be // turned on after the swapchain exists. @@ -1995,7 +2056,8 @@ unsafe extern "C" fn gn_vkCreateSwapchainKHR( slots: Vec::new(), guest_slots: 0, gen_no_grow: false, - next_acquire_ns: 0, + release: crate::vsync_gate::Schedule::default(), + commit_latency_ns: AtomicU64::new(0), worker: None, shared: Mutex::new(ScShared { jobs: VecDeque::with_capacity(GN_JOB_CAP), @@ -2027,6 +2089,7 @@ unsafe extern "C" fn gn_vkCreateSwapchainKHR( acquired: false, in_flight: false, acq_seq: 0, + acquired_ns: 0, ino: 0, ahb: ptr::null_mut(), }); @@ -2318,28 +2381,22 @@ unsafe fn gn_acquire_common( ); let deadline = (timeout != u64::MAX).then(|| Instant::now() + Duration::from_nanos(timeout)); // The time gate: a frame that may not start yet waits before it is given an - // image. On gamescope's override route it runs only while frame generation - // sets the source rate (the app disarms gamescope's limiter then); any other - // cap is gamescope's limiter's, which holds the FIFO commits this shim - // declares while a cap is set (`live_present_mode`). Off that route the cap - // has no compositor to hold it, and the gate keeps it as before. - let period = if unsafe { (*(*sc).surf).swapchain_obj.is_null() } { - crate::scanout::period_ns(crate::scanout::target_fps()) - } else { - crate::lsfg::source_cap().map_or(0, crate::scanout::period_ns) - }; + // image. Which clock it runs on, if any, is `acquire_pacing`'s rule. + let pacing = unsafe { acquire_pacing(sc) }; + let gated = pacing != crate::vsync_gate::Pacing::Off; let mut idx: i32 = -1; let mut timed_out = false; // First instant this acquire was parked by the deadline, 0 while it never // was: the census reads one park per acquire, not one per wait, because a // wake-up that finds the deadline still ahead parks again. let mut hold_from: u64 = 0; + let mut slot_waited = false; { let mut g = (*sc).shared.lock().unwrap(); loop { - if period != 0 && !g.stopping { + if gated && !g.stopping { let now_ns = monotonic_ns(); - let next_ns = (*sc).next_acquire_ns; + let next_ns = (*sc).release.next_ns; if next_ns != 0 && now_ns < next_ns { if timeout == 0 { break; @@ -2363,15 +2420,21 @@ unsafe fn gn_acquire_common( } unsafe { acquire_take_free(sc, &mut idx) }; if idx >= 0 { - if period != 0 { + if gated { // One timeline for the whole swapchain: the next frame may - // not start before this one plus a period. + // not start before the release this one was booked to. let now_ns = monotonic_ns(); - (*sc).next_acquire_ns = - crate::scanout::next_acquire_after((*sc).next_acquire_ns, now_ns, period); + (*(*sc).slots.as_mut_ptr().add(idx as usize)).acquired_ns = now_ns; + (*sc).release.book_paced(now_ns, pacing); } break; } + if gated && !slot_waited { + // Past the gate with no image free: this acquire waits on the + // compositor's release, not on the gate. + slot_waited = true; + SLOT_WAIT_N.fetch_add(1, Ordering::Relaxed); + } if g.stopping { break; } @@ -6203,39 +6266,215 @@ mod tests { } } + fn test_grid(period_ns: u64) -> crate::vsync_gate::Grid { + crate::vsync_gate::Grid { + anchor_ns: 1_000_000_000, + period_ns, + clock: crate::vsync_gate::GateClock::Vsync, + } + } + #[test] - fn the_declared_mode_is_fifo_while_a_cap_is_set_as_gamescopes_layer_does() { - let (m, f, i) = ( + fn the_declared_mode_follows_the_swapchain_the_cap_and_the_lead() { + let (m, f, i, r) = ( VK_PRESENT_MODE_MAILBOX_KHR, VK_PRESENT_MODE_FIFO_KHR, VK_PRESENT_MODE_IMMEDIATE_KHR, + VK_PRESENT_MODE_FIFO_RELAXED_KHR, + ); + let g60 = Some(test_grid(16_675_004)); // 59.97 Hz + let g120 = Some(test_grid(8_333_333)); + for mode in [m, i, r] { + // No cap: the swapchain's mode, lead or no lead. + assert_eq!( + choose_live_mode(mode, ModeSource::Guest, mode, false, 0, g60), + (mode, Declared::Mode) + ); + assert_eq!( + choose_live_mode(mode, ModeSource::Guest, mode, false, 0, None), + (mode, Declared::Mode) + ); + // A cap at the display's rate: FIFO. + assert_eq!( + choose_live_mode(mode, ModeSource::Guest, mode, false, 60, g60), + (f, Declared::CapRate) + ); + // A cap below it: the swapchain's mode. + assert_eq!( + choose_live_mode(mode, ModeSource::Guest, mode, false, 60, g120), + (mode, Declared::Mode) + ); + assert_eq!( + choose_live_mode(mode, ModeSource::Guest, mode, false, 30, g60), + (mode, Declared::Mode) + ); + // Capped without the lead: FIFO, gamescope's limiter. + assert_eq!( + choose_live_mode(mode, ModeSource::Guest, mode, false, 60, None), + (f, Declared::CapCompat) + ); + } + // A FIFO swapchain: FIFO, whatever the cap or the lead. + for (cap, grid) in [(0, g60), (60, g60), (30, g60), (60, None)] { + assert_eq!( + choose_live_mode(f, ModeSource::Guest, f, false, cap, grid), + (f, Declared::Mode) + ); + } + // Frame generation: mailbox, whatever the mode. + assert_eq!( + choose_live_mode(f, ModeSource::Guest, f, true, 60, g60), + (m, Declared::Generating) + ); + assert_eq!( + choose_live_mode(m, ModeSource::Guest, m, true, 0, None), + (m, Declared::Generating) ); - // Capped: FIFO whatever the guest asked for. - assert_eq!(choose_live_mode(f, false, true), f); - assert_eq!(choose_live_mode(m, false, true), f); - assert_eq!(choose_live_mode(i, false, true), f); - // Uncapped: the guest's own mode. - assert_eq!(choose_live_mode(f, false, false), f); - assert_eq!(choose_live_mode(m, false, false), m); - assert_eq!(choose_live_mode(i, false, false), i); - // Frame generation: mailbox, capped or not. - assert_eq!(choose_live_mode(f, true, true), m); - assert_eq!(choose_live_mode(f, true, false), m); } #[test] - fn the_env_mode_does_not_decide_the_gamescope_route() { - // SAFETY: no other test reads this variable. - unsafe { std::env::set_var("MESA_VK_WSI_PRESENT_MODE", "mailbox") }; + fn the_gate_on_the_gamescope_route_follows_the_declaration() { + use crate::vsync_gate::{GateClock, Pacing}; + let g = test_grid(16_675_004); + let grid = Some(g); + let (m, f) = (VK_PRESENT_MODE_MAILBOX_KHR, VK_PRESENT_MODE_FIFO_KHR); + // A FIFO swapchain, or FIFO for want of the lead: gamescope's limiter. + assert_eq!( + gamescope_route_pacing((f, Declared::Mode), None, 60, grid), + Pacing::Off + ); + assert_eq!( + gamescope_route_pacing((f, Declared::CapCompat), None, 60, None), + Pacing::Off + ); + // FIFO at the display's rate: the gate stays, one release per vblank. + assert_eq!( + gamescope_route_pacing((f, Declared::CapRate), None, 60, grid), + Pacing::Grid { + grid: g, + interval_ns: 16_675_004 + } + ); + // Mailbox, immediate, FIFO-relaxed under a cap: the gate, on the grid. + for mode in [ + m, + VK_PRESENT_MODE_IMMEDIATE_KHR, + VK_PRESENT_MODE_FIFO_RELAXED_KHR, + ] { + assert_eq!( + gamescope_route_pacing((mode, Declared::Mode), None, 30, grid).clock(), + GateClock::Vsync + ); + // Uncapped: nobody limits. + assert_eq!( + gamescope_route_pacing((mode, Declared::Mode), None, 0, grid), + Pacing::Off + ); + } + // Frame generation's source cap: the cap's own clock. + assert_eq!( + gamescope_route_pacing((m, Declared::Generating), Some(40), 0, grid), + Pacing::Clock { + period_ns: 25_000_000 + } + ); + } + + /// The rule table of docs/10: the container's mode (else the guest's), the + /// mode declared and why, and what limits the rate, on gamescope's + /// override route. + #[test] + fn the_container_mode_and_the_cap_decide_the_gamescope_route() { + use crate::vsync_gate::{GateClock, Grid, Pacing}; + let g60 = Some(test_grid(16_675_004)); // 59.97 Hz + let g120 = Some(test_grid(8_333_333)); let mut ci: VkSwapchainCreateInfoKHR = unsafe { core::mem::zeroed() }; - ci.presentMode = VK_PRESENT_MODE_FIFO_KHR; - assert_eq!(effective_present_mode(&ci, true), VK_PRESENT_MODE_FIFO_KHR); - assert_eq!(effective_present_mode(&ci, false), VK_PRESENT_MODE_MAILBOX_KHR); - ci.presentMode = 1000; - assert_eq!(effective_present_mode(&ci, true), VK_PRESENT_MODE_FIFO_KHR); + // `grid` is `crate::vsync_gate::current_grid()`: `None` with no note + // yet, or a note whose stamp lead is 0 (a daemon that predates it). + // `guest` is the guest's requested mode. The second field of the + // result is the census's `source=`. + let mut row = |env: Option<&str>, + guest: VkPresentModeKHR, + cap: u32, + lsfg_cap: Option, + grid: Option| { + // SAFETY: no other test reads this variable. + match env { + Some(v) => unsafe { std::env::set_var("MESA_VK_WSI_PRESENT_MODE", v) }, + None => unsafe { std::env::remove_var("MESA_VK_WSI_PRESENT_MODE") }, + } + ci.presentMode = guest; + let (mode, source) = effective_present_mode(&ci); + let requested = guest_present_mode(&ci); + let choice = choose_live_mode(mode, source, requested, lsfg_cap.is_some(), cap, grid); + let label = match (choice.1, source) { + (Declared::CapCompat, _) => "cap-compat", + (Declared::CapRate, _) => "cap-rate", + (Declared::GuestVsync, _) => "guest", + (_, ModeSource::Env) => "env", + (_, ModeSource::Guest) => "guest", + }; + let pacing = gamescope_route_pacing(choice, lsfg_cap, cap, grid); + (choice.0, label, pacing) + }; + let (m, f, i) = ( + VK_PRESENT_MODE_MAILBOX_KHR, + VK_PRESENT_MODE_FIFO_KHR, + VK_PRESENT_MODE_IMMEDIATE_KHR, + ); + let r = VK_PRESENT_MODE_FIFO_RELAXED_KHR; + let every = |g: Option, n: u64| Pacing::Grid { + grid: g.unwrap(), + interval_ns: n * g.unwrap().period_ns, + }; + let off = Pacing::Off; + // Capped, env mailbox, whatever the guest asked for: + for guest in [f, m, i] { + // 60 on 59.97 Hz: FIFO, the gate every vblank. + assert_eq!( + row(Some("mailbox"), guest, 60, None, g60), + (f, "cap-rate", every(g60, 1)) + ); + // 60 on 120 Hz, 30 on 60: mailbox, the gate every second vblank. + assert_eq!( + row(Some("mailbox"), guest, 60, None, g120), + (m, "env", every(g120, 2)) + ); + assert_eq!( + row(Some("mailbox"), guest, 30, None, g60), + (m, "env", every(g60, 2)) + ); + // No lead: FIFO, gamescope's limiter. + assert_eq!( + row(Some("mailbox"), guest, 60, None, None), + (f, "cap-compat", off) + ); + } + // env fifo, capped or not, whatever the guest asked for: FIFO, no gate. + for guest in [f, m, i] { + for cap in [0, 30, 60] { + assert_eq!(row(Some("fifo"), guest, cap, None, g60), (f, "env", off)); + } + } + // No cap, env mailbox: a guest that asked for FIFO (VSync on) or + // FIFO-relaxed gets it; one that asked for mailbox or immediate gets + // the env's mailbox. No gate either way. + assert_eq!(row(Some("mailbox"), f, 0, None, g60), (f, "guest", off)); + assert_eq!(row(Some("mailbox"), r, 0, None, g60), (r, "guest", off)); + assert_eq!(row(Some("mailbox"), f, 0, None, None), (f, "guest", off)); + assert_eq!(row(Some("mailbox"), i, 0, None, g60), (m, "env", off)); + assert_eq!(row(Some("mailbox"), m, 0, None, g60), (m, "env", off)); + // No env: the guest's mode. + assert_eq!(row(None, f, 60, None, g60), (f, "guest", off)); + assert_eq!(row(None, i, 0, None, g60), (i, "guest", off)); + // Frame generation: mailbox, the source-rate gate on its own clock. + let (live, _, pacing) = row(Some("fifo"), f, 0, Some(40), g60); + assert_eq!((live, pacing.clock()), (m, GateClock::Clock)); unsafe { std::env::remove_var("MESA_VK_WSI_PRESENT_MODE") }; + // An unknown guest mode falls back to fifo. + ci.presentMode = 1000; + assert_eq!(guest_present_mode(&ci), f); } - #[test] fn present_wait_is_answered_at_commit_only_for_unpaced_modes() { assert!(answers_present_wait_at_commit(VK_PRESENT_MODE_MAILBOX_KHR)); @@ -6702,6 +6941,10 @@ pub struct GnSlot { // Guest-side trace: the acquire sequence that filled this slot, so a // present can be tied back to the acquire that handed the guest the image. pub acq_seq: u64, + // When the gate let the guest's acquire of this slot go, `CLOCK_MONOTONIC` + // ns; 0 when no gate ran. Read once at the commit, for the census's + // acquire-to-commit latency. + pub acquired_ns: u64, // The guest's present id for the frame currently in this slot, 0 when it // chained none: read at the commit to mark the frame for the daemon, which // reports it back once the transaction presenting it latched. @@ -6736,10 +6979,14 @@ pub struct GnSwapchain { pub extent: VkExtent2D, pub fourcc: u32, pub usage: VkImageUsageFlags, - /// The present mode the shim honours for this swapchain (override wins over - /// the app's request; see `effective_present_mode`). Read by - /// `present_dispatch` on every present. + /// The present mode the shim honours for this swapchain (the container's + /// override wins over the app's request; see `effective_present_mode`). + /// Read by `present_dispatch` on every present. pub present_mode: VkPresentModeKHR, + /// The mode the guest asked for, whatever the override says. + pub requested_mode: VkPresentModeKHR, + /// Whether `present_mode` is the container's or the guest's. + pub mode_source: ModeSource, /// Every slot the shim owns: the guest's first (`guest_slots` of them), then /// the generation destinations, which the guest never sees. pub slots: Vec, @@ -6753,11 +7000,15 @@ pub struct GnSwapchain { pub next: *mut GnSwapchain, /// Set once a destination could not be made: the set stops growing. pub gen_no_grow: bool, - /// The instant the guest may start its next frame, in `CLOCK_MONOTONIC` ns: - /// 0 before the first, advanced by each acquire. One timeline per swapchain, - /// not per slot — a slot-local schedule would let a ring of N images run N - /// frames back to back. - pub next_acquire_ns: u64, + /// When the guest may start its next frame (`release.next_ns`, + /// `CLOCK_MONOTONIC` ns, 0 before the first), advanced by each gated + /// acquire. One timeline per swapchain, not per slot — a slot-local schedule + /// would let a ring of N images run N frames back to back. + pub release: crate::vsync_gate::Schedule, + /// The guest's acquire-to-commit time, smoothed over gated frames + /// (`crate::vsync_gate::smooth_latency`); 0 until one was measured. Written + /// by the present worker, read by the acquire. + pub commit_latency_ns: AtomicU64, } // ---------------------------------------------------------------- wayland @@ -9171,30 +9422,31 @@ fn parse_present_mode_override(val: Option<&str>) -> Option { "mailbox" => VK_PRESENT_MODE_MAILBOX_KHR, "immediate" => VK_PRESENT_MODE_IMMEDIATE_KHR, _ => { - crate::logf!(gndrm_obf::obf!("Invalid MESA_VK_WSI_PRESENT_MODE value!")); + // Ungated and once: the env is the user's present-mode control, and + // a typo must show in a shipped session's log without repeating on + // every query. + static SAID: AtomicBool = AtomicBool::new(false); + if !SAID.swap(true, Ordering::Relaxed) { + crate::logi!(gndrm_obf::obf!("Invalid MESA_VK_WSI_PRESENT_MODE value!")); + } return None; } }; Some(mode) } -/// The present mode a swapchain keeps: the guest's requested `presentMode`, -/// falling back to fifo (mesa wsi_common.c:1521). Off gamescope's override route -/// the MESA_VK_WSI_PRESENT_MODE override wins over it, as in Mesa. -/// -/// On the override route the env is not consulted. There the shim is what -/// gamescope's WSI layer is on a PC, and that layer declares the app's own mode -/// to gamescope whatever Mesa below it was told: the env reaches only Mesa. The -/// variable comes with every container as a default for the X11 path; honouring -/// it here declared every commit MAILBOX, which gamescope's limiter never holds. -fn effective_present_mode(ci: &VkSwapchainCreateInfoKHR, gamescope_route: bool) -> VkPresentModeKHR { - if !gamescope_route { - if let Some(m) = - parse_present_mode_override(std::env::var("MESA_VK_WSI_PRESENT_MODE").ok().as_deref()) - { - return m; - } - } +/// Where a swapchain's present mode came from, for the census. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ModeSource { + /// The container's `MESA_VK_WSI_PRESENT_MODE`. + Env, + /// The guest's `presentMode`. + Guest, +} + +/// The guest's requested `presentMode`, falling back to fifo (mesa +/// wsi_common.c:1521). +fn guest_present_mode(ci: &VkSwapchainCreateInfoKHR) -> VkPresentModeKHR { match ci.presentMode { VK_PRESENT_MODE_IMMEDIATE_KHR | VK_PRESENT_MODE_MAILBOX_KHR @@ -9204,6 +9456,19 @@ fn effective_present_mode(ci: &VkSwapchainCreateInfoKHR, gamescope_route: bool) } } +/// The present mode a swapchain keeps: the container's +/// `MESA_VK_WSI_PRESENT_MODE` when it names one, as in Mesa, else the guest's +/// requested mode ([`guest_present_mode`]). The same on both routes: on +/// gamescope's override route the env is the user's present-mode preference +/// (mailbox is the container default; fifo, fifo_relaxed and immediate are +/// the user's choice), and it is what the shim declares to gamescope. +fn effective_present_mode(ci: &VkSwapchainCreateInfoKHR) -> (VkPresentModeKHR, ModeSource) { + match parse_present_mode_override(std::env::var("MESA_VK_WSI_PRESENT_MODE").ok().as_deref()) { + Some(m) => (m, ModeSource::Env), + None => (guest_present_mode(ci), ModeSource::Guest), + } +} + /// Whether a present wait on a frame of a swapchain in effective `mode` is /// answered when the frame is committed. True for mailbox and immediate, where /// a guest that knew its real mode would not wait (DXVK skips the present wait @@ -9212,42 +9477,164 @@ fn answers_present_wait_at_commit(mode: VkPresentModeKHR) -> bool { mode == VK_PRESENT_MODE_MAILBOX_KHR || mode == VK_PRESENT_MODE_IMMEDIATE_KHR } -/// The mode a present on `sc` runs under and declares, decided per present the -/// way gamescope's WSI layer does on a PC: there the layer declares FIFO while -/// gamescope's limiter is engaged and the app's own mode otherwise, and -/// gamescope latches one FIFO commit per limited vblank, so the client is paced -/// by `wl_buffer.release` and gamescope's limiter is the cap. +/// Why a present declares the mode it does, on the override route. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Declared { + /// The swapchain's own mode ([`effective_present_mode`]). + Mode, + /// Mailbox for frame generation. + Generating, + /// FIFO for want of the daemon's stamp lead (`source=cap-compat`). + CapCompat, + /// FIFO because the cap is the display's rate (`source=cap-rate`). + CapRate, + /// No cap and a guest that asked for FIFO or FIFO-relaxed: its own mode + /// over the container's non-FIFO default (`source=guest`). + GuestVsync, +} + +/// The mode a present on `sc` runs under and declares, decided per present. /// /// On the override route: -/// - frame generation on: mailbox, as before. Generated frames are committed -/// with no declaration, and a real frame declared FIFO would queue behind -/// them. -/// - a cap in force (the daemon's `FrameCapNote`, the same value the app hands -/// gamescope's limiter): FIFO. -/// - otherwise: the guest's own mode. +/// - frame generation on: mailbox. Generated frames are committed with no +/// declaration, and a real frame declared FIFO would queue behind them. +/// - a cap set, a mode other than FIFO, and no stamp lead from the daemon yet +/// (`crate::vsync_gate::current_grid` is `None`: no note yet, or a daemon +/// that predates the field): FIFO, so gamescope's limiter is the cap. The +/// vblank-grid gate is not run without the lead: released on the latch +/// points themselves, frames straddle gamescope's composite and are +/// replaced in its mailbox. The census reads `source=cap-compat`. +/// - a cap at the display's rate and a mode other than FIFO: FIFO, with the +/// acquire gate kept on the vblank grid (`source=cap-rate`). At that cap one +/// frame per vblank is all the panel shows under either mode, and under +/// mailbox the shim would have to keep two commits out of one gamescope +/// composite without knowing when gamescope latches: its wakeup moves with +/// its rolling draw time (vblankmanager.cpp). A guest that paces itself just +/// below the display's rate had ~1 frame a second replaced in gamescope's +/// mailbox that way. Under FIFO gamescope latches one commit per vblank at +/// its own wakeup, its limiter's divisor is 1, and the gate keeps the guest +/// on the grid so the FIFO queue stays about one frame deep. +/// - no cap, the container's env naming a mode other than FIFO and a guest +/// that asked for FIFO or FIFO-relaxed: the guest's mode (`source=guest`); +/// see [`choose_live_mode`]. +/// - otherwise: the swapchain's mode ([`effective_present_mode`]). A FIFO +/// swapchain is held by gamescope's limiter, one commit per limited vblank; +/// under any other mode the acquire gate is the cap, on the display's vblank +/// grid (`acquire_pacing`). /// -/// Off the override route there is nothing to declare it to, and the mode is the -/// one the swapchain was made with. +/// The rule is read per present, so a lead or a cap that changes mid-session +/// moves the declaration on the next commit. Off the override route there is +/// nothing to declare it to, and the mode is the one the swapchain was made +/// with. pub(crate) unsafe fn live_present_mode(sc: *mut GnSwapchain) -> VkPresentModeKHR { - let requested = unsafe { (*sc).present_mode }; + unsafe { live_choice(sc) }.0 +} + +/// [`live_present_mode`] and why. +unsafe fn live_choice(sc: *mut GnSwapchain) -> (VkPresentModeKHR, Declared) { + let mode = unsafe { (*sc).present_mode }; if unsafe { (*(*sc).surf).swapchain_obj.is_null() } { - return requested; + return (mode, Declared::Mode); } choose_live_mode( - requested, + mode, + unsafe { (*sc).mode_source }, + unsafe { (*sc).requested_mode }, crate::lsfg::live().0, - crate::scanout::cap_in_force(), + crate::scanout::cap_fps_note(), + crate::vsync_gate::current_grid(), ) } -/// [`live_present_mode`]'s rule on the override route. -fn choose_live_mode(requested: VkPresentModeKHR, generating: bool, capped: bool) -> VkPresentModeKHR { +/// [`live_present_mode`]'s rule on the override route, for the swapchain's +/// `mode` and where it came from, the guest's `requested` mode, the daemon's +/// cap and the display's vblank grid (`None` without the stamp lead). +/// +/// With no cap a guest that asked for FIFO (or FIFO-relaxed) gets it even when +/// the container's env names another mode: every container carries +/// `MESA_VK_WSI_PRESENT_MODE=mailbox` as a stamped default the user did not +/// choose, and honouring it there turned a game's own VSync into an uncapped +/// mailbox session. A FIFO request is a specific wish; a mailbox or immediate +/// request is "do not make me wait", for which the container's mode is the +/// substitute, as on the X11 route. An env of `fifo` is an explicit wish and +/// wins in every case. +fn choose_live_mode( + mode: VkPresentModeKHR, + source: ModeSource, + requested: VkPresentModeKHR, + generating: bool, + cap_fps: u32, + grid: Option, +) -> (VkPresentModeKHR, Declared) { + let capped = cap_fps != gndrm_uapi::wire::FPS_UNCAPPED; + let vsync_request = + requested == VK_PRESENT_MODE_FIFO_KHR || requested == VK_PRESENT_MODE_FIFO_RELAXED_KHR; if generating { - VK_PRESENT_MODE_MAILBOX_KHR - } else if capped { - VK_PRESENT_MODE_FIFO_KHR + (VK_PRESENT_MODE_MAILBOX_KHR, Declared::Generating) + } else if mode == VK_PRESENT_MODE_FIFO_KHR { + (mode, Declared::Mode) + } else if !capped { + if source == ModeSource::Env && vsync_request { + (requested, Declared::GuestVsync) + } else { + (mode, Declared::Mode) + } } else { - requested + match grid { + None => (VK_PRESENT_MODE_FIFO_KHR, Declared::CapCompat), + Some(g) if crate::vsync_gate::at_display_rate(cap_fps, g) => { + (VK_PRESENT_MODE_FIFO_KHR, Declared::CapRate) + } + Some(_) => (mode, Declared::Mode), + } + } +} + +/// How the acquire gate paces `sc` now. +/// +/// - Off gamescope's override route (the X11 Present route) nothing holds the +/// cap but the gate, on the cap's own clock, as before. +/// - On the override route see [`gamescope_route_pacing`]. +unsafe fn acquire_pacing(sc: *mut GnSwapchain) -> crate::vsync_gate::Pacing { + if unsafe { (*(*sc).surf).swapchain_obj.is_null() } { + return crate::vsync_gate::Pacing::on_clock(crate::scanout::target_fps()); + } + gamescope_route_pacing( + unsafe { live_choice(sc) }, + crate::lsfg::source_cap(), + crate::scanout::cap_fps_note(), + crate::vsync_gate::current_grid(), + ) +} + +/// [`acquire_pacing`]'s rule on gamescope's override route, for the declared +/// mode and why ([`choose_live_mode`]), frame generation's source cap, the +/// daemon's cap and the display's vblank grid: +/// - frame generation sets the source rate: the gate holds it on the cap's own +/// clock (the app disarms gamescope's limiter then; generation paces the +/// output on the latch grid itself); +/// - a FIFO swapchain, or FIFO for want of the stamp lead: no gate. +/// gamescope's limiter holds FIFO commits, one per limited vblank, and is +/// the cap; +/// - FIFO because the cap is the display's rate: the gate, on the vblank grid, +/// one release per vblank, so the guest never gets ahead of gamescope's FIFO +/// latch and its queue stays about one frame deep; +/// - mailbox, immediate, FIFO-relaxed: gamescope's limiter does not hold them +/// (`wlserver_surface_is_fifo` is FIFO only), so the gate is the cap, on the +/// display's vblank grid (`crate::vsync_gate`). +fn gamescope_route_pacing( + (live, why): (VkPresentModeKHR, Declared), + source_cap: Option, + cap_fps: u32, + grid: Option, +) -> crate::vsync_gate::Pacing { + if let Some(cap) = source_cap { + return crate::vsync_gate::Pacing::on_clock(cap); + } + match why { + Declared::CapCompat => crate::vsync_gate::Pacing::Off, + Declared::Mode if live == VK_PRESENT_MODE_FIFO_KHR => crate::vsync_gate::Pacing::Off, + _ => crate::vsync_gate::pacing(cap_fps, grid), } } @@ -9272,23 +9659,79 @@ fn mode_name(mode: VkPresentModeKHR) -> &'static str { } } -/// Ungated, every 10 s while frames are presented: the mode declared to -/// gamescope and why, and the counts a device run checks the pacing against. -/// The launcher does not pass the container's environment to gamescope's -/// clients, so a gated line could not be read there. +/// What the ungated `present mode:` line reports: the mode declared to +/// gamescope and why, the cap and what holds it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +struct ModeLine { + declared: VkPresentModeKHR, + requested: VkPresentModeKHR, + source: &'static str, + gamescope_route: bool, + cap: u32, + gate: &'static str, + lsfg: bool, +} + +/// The last [`ModeLine`] printed. +static MODE_LINE: Mutex> = Mutex::new(None); + +/// The `present mode:` census. +/// +/// Ungated, once when frames first flow and again whenever the declared mode, +/// its source, the cap or the gate changes: a shipped session's log says which +/// path ran without repeating it. The counters a device run checks the pacing +/// against repeat every 10 s with `GNDRM_LOG_VKSHIM=1`, as the shim's stats +/// line does. unsafe fn mode_census(sc: *mut GnSwapchain) { - if !rate_ok(&RATE_MODE, MODE_CENSUS_EVERY) { + let route = unsafe { !(*(*sc).surf).swapchain_obj.is_null() }; + let (declared, why) = unsafe { live_choice(sc) }; + let line = ModeLine { + declared, + requested: unsafe { (*sc).requested_mode }, + source: match (why, unsafe { (*sc).mode_source }) { + (Declared::CapCompat, _) => "cap-compat", + (Declared::CapRate, _) => "cap-rate", + (Declared::GuestVsync, _) => "guest", + (_, ModeSource::Env) => "env", + (_, ModeSource::Guest) => "guest", + }, + gamescope_route: route, + cap: crate::scanout::cap_fps_note(), + gate: unsafe { acquire_pacing(sc) }.clock().name(), + lsfg: crate::lsfg::live().0, + }; + { + let mut last = MODE_LINE.lock().unwrap_or_else(|e| e.into_inner()); + if *last != Some(line) { + *last = Some(line); + crate::logi!(gndrm_obf::obf!( + "present mode: declared={} requested={} source={} route={} cap={} gate={} lsfg={}", + mode_name(line.declared), + mode_name(line.requested), + line.source, + if route { "gamescope" } else { "x11" }, + line.cap, + line.gate, + u8::from(line.lsfg), + )); + } + } + if !crate::vkshim_enabled() || !rate_ok(&RATE_MODE, MODE_CENSUS_EVERY) { return; } - let route = unsafe { !(*(*sc).surf).swapchain_obj.is_null() }; let (presented, discarded, fallback) = crate::present::feedback_counts(); - crate::logi!(gndrm_obf::obf!( - "present mode: declared={} requested={} route={} cap={} lsfg={} acquires={} commits={} declared_fifo={} declared_other={} releases={} feedback={} presented={} discarded={} fallback={}", - mode_name(unsafe { live_present_mode(sc) }), - mode_name(unsafe { (*sc).present_mode }), + let (late, skipped) = crate::vsync_gate::late_counts(); + // Since the last line, so each line reads one window's cadence. + let gaps = COMMIT_GAPS.each_ref().map(|b| b.swap(0, Ordering::Relaxed)); + crate::logf!(gndrm_obf::obf!( + "present mode: declared={} requested={} source={} route={} cap={} gate={} lsfg={} acquires={} commits={} declared_fifo={} declared_other={} releases={} feedback={} presented={} discarded={} fallback={} gate_holds={} gate_late={} skipped={} slot_waits={} commit_lat_us={} spacing=[<15:{} 15-16.2:{} 16.2-17.2:{} 17.2-20:{} >20:{}]", + mode_name(line.declared), + mode_name(line.requested), + line.source, if route { "gamescope" } else { "x11" }, - crate::scanout::cap_fps_note(), - u8::from(crate::lsfg::live().0), + line.cap, + line.gate, + u8::from(line.lsfg), N_ACQ.load(Ordering::Relaxed), N_COMMIT_OK.load(Ordering::Relaxed), DECL_FIFO_N.load(Ordering::Relaxed), @@ -9298,6 +9741,16 @@ unsafe fn mode_census(sc: *mut GnSwapchain) { presented, discarded, fallback, + GATE_HOLD_N.load(Ordering::Relaxed), + late, + skipped, + SLOT_WAIT_N.load(Ordering::Relaxed), + unsafe { (*sc).commit_latency_ns.load(Ordering::Relaxed) } / 1_000, + gaps[0], + gaps[1], + gaps[2], + gaps[3], + gaps[4], )); } @@ -9370,13 +9823,10 @@ unsafe extern "C" fn gn_vkGetPhysicalDeviceSurfacePresentModesKHR( )); return VK_ERROR_SURFACE_LOST_KHR; } - // The env's mode leads the list only where it decides the mode - // (`effective_present_mode`): not on gamescope's override route. - let override_mode = if unsafe { (*surf).swapchain_obj.is_null() } { - parse_present_mode_override(std::env::var("MESA_VK_WSI_PRESENT_MODE").ok().as_deref()) - } else { - None - }; + // The env's mode leads the list, as in Mesa: it decides the mode on both + // routes (`effective_present_mode`). + let override_mode = + parse_present_mode_override(std::env::var("MESA_VK_WSI_PRESENT_MODE").ok().as_deref()); let modes_list = offered_present_modes(&SERVED_PRESENT_MODES, override_mode); unsafe { if modes.is_null() { diff --git a/icd_shim/src/scanout.rs b/icd_shim/src/scanout.rs index 91f6522..e2d39ac 100644 --- a/icd_shim/src/scanout.rs +++ b/icd_shim/src/scanout.rs @@ -169,17 +169,18 @@ pub(crate) fn target_fps() -> u32 { CAP_FPS.load(Ordering::Relaxed) } -/// Whether the daemon's cap note sets a cap, whatever frame generation does: -/// the value the app also hands gamescope's limiter. -pub(crate) fn cap_in_force() -> bool { - CAP_FPS.load(Ordering::Relaxed) != FPS_UNCAPPED -} - /// The cap the daemon's note carries, [`FPS_UNCAPPED`] when none. pub(crate) fn cap_fps_note() -> u32 { CAP_FPS.load(Ordering::Relaxed) } +/// Adopt the daemon's vsync note: frame generation places its commits on the +/// latch grid, and the acquire gate releases on the vblank grid it gives. +fn apply_vsync(note: VsyncNote) { + crate::lsfg::apply_vsync(note); + crate::vsync_gate::apply_note(note); +} + /// How many frame-cap notes have been adopted on this connection, for the same /// census. pub(crate) fn notes_seen() -> usize { @@ -302,7 +303,7 @@ pub fn lsfg_config() -> Option { if let Some(note) = FrameCapNote::decode(&buf[..n as usize]) { apply_note(note); } else if let Some(note) = VsyncNote::decode(&buf[..n as usize]) { - crate::lsfg::apply_vsync(note); + apply_vsync(note); } else if let Some(done) = gndrm_uapi::wire::PresentDone::decode(&buf[..n as usize]) { crate::present::record(done.id, done.latch_ns); } else if gndrm_uapi::wire::FrameAcquired::decode(&buf[..n as usize]).is_some() { @@ -543,7 +544,7 @@ fn exchange_req(fd: RawFd, req: ScanoutAllocReq) -> Result if recv_fd >= 0 { libc::close(recv_fd); } - crate::lsfg::apply_vsync(note); + apply_vsync(note); if frames < MAX_BURST_FRAMES { continue; } @@ -710,7 +711,7 @@ pub(crate) fn poll_notes() { if let Some(note) = FrameCapNote::decode(&buf[..len]) { apply_note(note); } else if let Some(note) = VsyncNote::decode(&buf[..len]) { - crate::lsfg::apply_vsync(note); + apply_vsync(note); } else if let Some(config) = LsfgConfig::decode(&buf[..len]) { if !config.is_request() { crate::lsfg::apply(config); diff --git a/icd_shim/src/vsync_gate.rs b/icd_shim/src/vsync_gate.rs new file mode 100644 index 0000000..15a48ec --- /dev/null +++ b/icd_shim/src/vsync_gate.rs @@ -0,0 +1,577 @@ +//! The cap's acquire gate when gamescope's limiter does not hold the frames. +//! +//! gamescope's limiter holds only commits declared FIFO +//! (`wlserver_surface_is_fifo`): a mailbox, immediate or FIFO-relaxed commit is +//! taken at the next composite whatever the limiter says. When the container +//! asks for one of those modes and a cap is set, the shim's acquire gate is +//! the cap. A gate on a clock of its own (one release every `1e9 / cap` ns) +//! beats against the display: a 60 cap on a 59.97 Hz panel put two releases in +//! one vblank every half minute or so and ran 57-59 fps. This gate releases +//! frames on the display's vblank grid instead, the grid the daemon stamps its +//! flip events on and gamescope composites from. +//! +//! The grid is `anchor - stamp_lead + k * period` from the daemon's +//! [`VsyncNote`]: `SurfaceFlinger`'s latch points, measured, moved earlier by +//! the lead the daemon stamps flip events with. gamescope wakes to composite +//! just before each of those instants (wakeup = stamp + interval - draw time - +//! red zone), so a release on one starts the guest's frame right after +//! gamescope took the previous frame and gives it the whole period, less +//! gamescope's own margin, to reach the next composite. +//! +//! Which grid points release a frame: +//! - a cap at the display's rate (within [`SNAP_TOLERANCE`]): every vblank; +//! - a cap that divides the display's rate: every `refresh / cap`-th vblank; +//! - any other cap below the rate: the vblank nearest each instant of an ideal +//! `1e9 / cap` timeline, which averages the cap on vblank boundaries (45 on +//! 60 Hz releases 3 of every 4 vblanks); +//! - a cap above the rate: every vblank already gets a frame, so the gate runs +//! on the cap's own clock ([`crate::scanout::next_acquire_after`]). +//! +//! A frame that arrives after its release point is not held: it starts at once, +//! and the release after it is the next point of the schedule, so a guest +//! slower than the cap runs at its own rate rather than at a divisor of the +//! display's. +//! +//! The grid exists only once the daemon has sent a note with a stamp lead +//! ([`current_grid`]). Released on the latch points themselves (a daemon that +//! predates the field sends 0), a frame starts ~2 ms after gamescope's wakeup +//! and straddles the next composite: measured on a Retroid Pocket 6, 58.3 fps +//! and 102 frames replaced in gamescope's mailbox per minute at a 60 cap on a +//! 59.975 Hz panel, and 13 % of the daemon's publishes late at 120 Hz. Until +//! then the shim declares FIFO under a cap and gamescope's limiter holds it +//! (`proxy::choose_live_mode`). + +use std::sync::atomic::{AtomicU32, AtomicU64, Ordering}; + +use gndrm_uapi::wire::{VsyncNote, FPS_UNCAPPED}; + +const NANOS_PER_SECOND: u64 = 1_000_000_000; + +/// How close `refresh / cap` has to be to a whole number for the cap to be +/// taken as that divisor of the display's rate, as a fraction of it. A "60 Hz" +/// panel measures 59.94-59.99 Hz, 0.02-0.1 % off a 60 cap; 0.5 % keeps a 59 +/// cap (1.7 % off) on the averaging timeline, where it gets 59. +const SNAP_TOLERANCE: f64 = 0.005; + +/// The display's vblank grid as the daemon last reported it; a period of 0 +/// means no report yet. Atomics: written by whichever thread reads the scan-out +/// channel, read by every acquire. +static ANCHOR_NS: AtomicU64 = AtomicU64::new(0); +static PERIOD_NS: AtomicU32 = AtomicU32::new(0); +static STAMP_LEAD_NS: AtomicU32 = AtomicU32::new(0); + +/// Adopt the daemon's latch grid (see [`VsyncNote`]). +pub(crate) fn apply_note(note: VsyncNote) { + ANCHOR_NS.store(note.anchor_ns, Ordering::Relaxed); + STAMP_LEAD_NS.store(note.stamp_lead_ns, Ordering::Relaxed); + PERIOD_NS.store(note.period_ns, Ordering::Relaxed); +} + +/// The vblank grid releases sit on: the daemon's flip-stamp grid, `None` +/// until a note carrying a stamp lead has arrived (0 means the daemon did not +/// say; one that measures a lead always sends at least 1 ns). +pub(crate) fn current_grid() -> Option { + let period = u64::from(PERIOD_NS.load(Ordering::Relaxed)); + let lead = u64::from(STAMP_LEAD_NS.load(Ordering::Relaxed)); + if period == 0 || lead == 0 { + return None; + } + let anchor = ANCHOR_NS.load(Ordering::Relaxed); + Some(Grid { + anchor_ns: anchor.saturating_sub(lead), + period_ns: period, + clock: GateClock::Vsync, + }) +} + +/// What the acquire gate releases on, as the census names it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum GateClock { + /// No gate: nothing to cap, or gamescope's limiter holds the frames. + Off, + /// The display's vblank grid from the daemon's note. + Vsync, + /// The cap's own `1e9 / cap` clock: a cap above the display's rate, a + /// gate that is not this module's (frame generation's source rate, the X11 + /// Present route), or no grid known. + Clock, +} + +impl GateClock { + pub(crate) fn name(self) -> &'static str { + match self { + GateClock::Off => "off", + GateClock::Vsync => "vsync", + GateClock::Clock => "clock", + } + } +} + +/// Release instants `anchor_ns + k * period_ns`, `CLOCK_MONOTONIC`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) struct Grid { + pub anchor_ns: u64, + pub period_ns: u64, + pub clock: GateClock, +} + +impl Grid { + /// The last grid instant at or before `t_ns`. + fn at_or_before(&self, t_ns: u64) -> u64 { + let p = i128::from(self.period_ns.max(1)); + let off = i128::from(t_ns) - i128::from(self.anchor_ns); + (i128::from(self.anchor_ns) + off.div_euclid(p) * p) as u64 + } + + /// The grid instant nearest `t_ns` (the later one on a tie). + fn nearest(&self, t_ns: u64) -> u64 { + self.at_or_before(t_ns.saturating_add(self.period_ns / 2)) + } +} + +/// How an acquire is paced. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Pacing { + Off, + /// One release per `period_ns` on the shim's clock. + Clock { + period_ns: u64, + }, + /// Releases on `grid`, spaced by `interval_ns` of the ideal timeline. + Grid { + grid: Grid, + interval_ns: u64, + }, +} + +impl Pacing { + /// A cap of `fps` on the shim's own clock, off when uncapped. + #[must_use] + pub(crate) fn on_clock(fps: u32) -> Self { + match crate::scanout::period_ns(fps) { + 0 => Pacing::Off, + period_ns => Pacing::Clock { period_ns }, + } + } + + pub(crate) fn clock(self) -> GateClock { + match self { + Pacing::Off => GateClock::Off, + Pacing::Clock { .. } => GateClock::Clock, + Pacing::Grid { grid, .. } => grid.clock, + } + } +} + +/// How a cap of `cap_fps` is released on `grid` (see the module notes). +#[must_use] +pub(crate) fn pacing(cap_fps: u32, grid: Option) -> Pacing { + if cap_fps == FPS_UNCAPPED { + return Pacing::Off; + } + let cap_period = NANOS_PER_SECOND / u64::from(cap_fps); + let Some(grid) = grid.filter(|g| g.period_ns != 0) else { + return Pacing::Clock { + period_ns: cap_period, + }; + }; + let ratio = cap_period as f64 / grid.period_ns as f64; + let whole = ratio.round(); + if whole >= 1.0 && (ratio - whole).abs() <= whole * SNAP_TOLERANCE { + Pacing::Grid { + grid, + interval_ns: whole as u64 * grid.period_ns, + } + } else if ratio < 1.0 { + Pacing::Clock { + period_ns: cap_period, + } + } else { + Pacing::Grid { + grid, + interval_ns: cap_period, + } + } +} + +/// How late after its release point an acquire can start and still count as +/// on time: the wake-up of the gate's own wait, with room for a loaded CPU. +const ON_TIME_NS: u64 = 500_000; + +/// Late starts booked by [`Schedule::book`], and the release points they moved +/// past, for the census. +static LATE_N: AtomicU64 = AtomicU64::new(0); +static SKIPPED_N: AtomicU64 = AtomicU64::new(0); + +/// `(late, skipped)`: acquires that started after their release point, and the +/// release points the schedule moved past for them. +pub(crate) fn late_counts() -> (u64, u64) { + ( + LATE_N.load(Ordering::Relaxed), + SKIPPED_N.load(Ordering::Relaxed), + ) +} + +/// Whether a cap of `cap_fps` is the display's rate on `grid` (`refresh / cap` +/// snaps to 1): the gate then releases every vblank. +#[must_use] +pub(crate) fn at_display_rate(cap_fps: u32, grid: Grid) -> bool { + matches!( + pacing(cap_fps, Some(grid)), + Pacing::Grid { interval_ns, .. } if interval_ns == grid.period_ns + ) +} + +/// A swapchain's release schedule on the grid. +#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)] +pub struct Schedule { + /// The grid instant the next acquire may start at; 0 before the first. + pub next_ns: u64, + /// The ideal-timeline instant `next_ns` was rounded from. + ideal_ns: u64, +} + +impl Schedule { + /// Book an acquire that started at `now_ns` (at or after `next_ns`) and + /// return the instant the one after it may start. + /// + /// An acquire that started before the grid instant after its release point + /// (on time, or late by less than a period) is booked to that release + /// point, and the schedule goes on from there: the next release is + /// `interval` further along the ideal timeline, on the grid instant + /// nearest it. A start later than that, or the first, starts the timeline + /// again from the grid instant at or before it, so a stall carries no debt + /// and a late frame is never held for a whole slot. The release point is + /// read on the grid as it is now: the daemon re-pushes the phase as it + /// moves, and a point booked on the old phase must not let two releases + /// into one period of the new one. Starts later than [`ON_TIME_NS`] after + /// their point are counted (`gate_late`), with the points passed (`skipped`). + pub(crate) fn book(&mut self, now_ns: u64, grid: Grid, interval_ns: u64) -> u64 { + if grid.period_ns == 0 { + return self.book_clock(now_ns, interval_ns); + } + let at = grid.at_or_before(now_ns); + let due = (self.next_ns != 0).then(|| grid.nearest(self.next_ns)); + if let Some(due) = due.filter(|&d| now_ns >= d.saturating_add(ON_TIME_NS)) { + LATE_N.fetch_add(1, Ordering::Relaxed); + SKIPPED_N.fetch_add(at.saturating_sub(due) / grid.period_ns, Ordering::Relaxed); + } + let (slot, ideal) = match due { + Some(due) if at <= due => (due, self.ideal_ns), + _ => (at, at), + }; + let ideal_next = ideal.saturating_add(interval_ns); + // Never two releases on one grid instant. + let next = grid + .nearest(ideal_next) + .max(slot.saturating_add(grid.period_ns)); + self.ideal_ns = ideal_next; + self.next_ns = next; + next + } + + /// Book an acquire that started at `now_ns` on the shim's own clock + /// ([`crate::scanout::next_acquire_after`]). + pub(crate) fn book_clock(&mut self, now_ns: u64, period_ns: u64) -> u64 { + self.next_ns = crate::scanout::next_acquire_after(self.next_ns, now_ns, period_ns); + // `book` reads `ideal_ns` as an absolute instant: leaving it at 0 + // resolves `nearest(ideal_ns + interval)` near CLOCK_MONOTONIC zero and + // the schedule collapses to one release per vblank. + self.ideal_ns = self.next_ns; + self.next_ns + } + + /// Book an acquire that started at `now_ns` under `pacing`; `next_ns` is + /// left alone when there is no gate. + pub(crate) fn book_paced(&mut self, now_ns: u64, pacing: Pacing) { + match pacing { + Pacing::Off => {} + Pacing::Clock { period_ns } => { + self.book_clock(now_ns, period_ns); + } + Pacing::Grid { grid, interval_ns } => { + self.book(now_ns, grid, interval_ns); + } + } + } +} + +/// Fold one frame's acquire-to-commit time into a running estimate for the +/// census (1/8 weight; the first sample is taken as it is). +#[must_use] +pub(crate) fn smooth_latency(estimate_ns: u64, sample_ns: u64) -> u64 { + if estimate_ns == 0 { + sample_ns + } else { + estimate_ns - estimate_ns / 8 + sample_ns / 8 + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A 59.97 Hz panel, as the Retroid Pocket 6 measures. + const P60: u64 = 16_675_004; + /// A 120 Hz panel. + const P120: u64 = 8_333_333; + + fn grid(period: u64) -> Grid { + Grid { + anchor_ns: 1_000_000_000, + period_ns: period, + clock: GateClock::Vsync, + } + } + + /// Releases of a guest that is always ready: it asks again the instant it + /// is let go (plus a wake-up jitter), for `n` frames. + fn run_ready(g: Grid, interval: u64, n: usize) -> Vec { + let mut s = Schedule::default(); + let mut now = g.anchor_ns; + let mut out = Vec::with_capacity(n); + for _ in 0..n { + if s.next_ns != 0 && now < s.next_ns { + now = s.next_ns; + } + let start = now + 50_000; // the wait returns a hair late + out.push(start); + s.book(start, g, interval); + now = start + 1_000_000; // a frame's CPU work before the next acquire + } + out + } + + fn interval(p: Pacing) -> u64 { + match p { + Pacing::Grid { interval_ns, .. } => interval_ns, + other => panic!("expected a grid gate, got {other:?}"), + } + } + + /// The vblank index each release fell in. + fn vblanks(g: Grid, starts: &[u64]) -> Vec { + starts + .iter() + .map(|&t| (g.at_or_before(t) - g.anchor_ns) / g.period_ns) + .collect() + } + + #[test] + fn a_cap_at_the_display_rate_releases_once_per_vblank() { + let g = grid(P60); + // 60 on a 59.97 Hz panel: the display's rate, not 60.000. + let p = pacing(60, Some(g)); + assert_eq!(interval(p), P60); + let starts = run_ready(g, interval(p), 600); + let v = vblanks(g, &starts); + assert!(v.windows(2).all(|w| w[1] == w[0] + 1), "one per vblank"); + // 600 releases over 600 vblanks: the panel's rate exactly. + assert_eq!(v[599] - v[0], 599); + // Phase: each release is on its grid instant, not wherever a free clock + // drifted to. + for &t in &starts { + assert_eq!(t - g.at_or_before(t), 50_000); + } + } + + #[test] + fn thirty_on_sixty_releases_every_second_vblank() { + let g = grid(P60); + let p = pacing(30, Some(g)); + assert_eq!(interval(p), 2 * P60); + let v = vblanks(g, &run_ready(g, interval(p), 300)); + assert!(v.windows(2).all(|w| w[1] == w[0] + 2)); + } + + #[test] + fn sixty_on_a_hundred_and_twenty_releases_every_second_vblank() { + let g = grid(P120); + let p = pacing(60, Some(g)); + assert_eq!(interval(p), 2 * P120); + let v = vblanks(g, &run_ready(g, interval(p), 300)); + assert!(v.windows(2).all(|w| w[1] == w[0] + 2)); + } + + #[test] + fn forty_five_on_sixty_averages_the_cap_on_vblanks() { + let g = grid(16_666_667); + let p = pacing(45, Some(g)); + assert_eq!(interval(p), NANOS_PER_SECOND / 45); + let v = vblanks(g, &run_ready(g, interval(p), 451)); + // Every gap is one or two vblanks, never zero. + assert!(v.windows(2).all(|w| (1..=2).contains(&(w[1] - w[0])))); + // 450 intervals of 1/45 s span 600 vblanks of a 60 Hz panel. + let span = v[450] - v[0]; + assert!((599..=601).contains(&span), "span {span}"); + } + + #[test] + fn a_late_frame_starts_at_once_and_the_schedule_goes_on() { + let g = grid(P60); + let iv = interval(pacing(60, Some(g))); + let mut s = Schedule::default(); + let first = g.anchor_ns + 10_000; + let due = s.book(first, g, iv); + assert_eq!(due, g.anchor_ns + P60); + // The guest comes back 6 ms after its release point. The gate does not + // hold it (it is past `next_ns`), and the release after it is the next + // vblank, not a whole period from now. + let late = due + 6_000_000; + assert!(late >= s.next_ns); + assert_eq!(s.book(late, g, iv), due + P60); + // Late by more than a period: assigned to the vblank it is in, and the + // next release is the one after. + let very_late = due + P60 + 3 * P60 + 2_000_000; + assert_eq!(s.book(very_late, g, iv), due + 5 * P60); + } + + /// A guest that is always ready, booked under `first` for `n_first` + /// frames, then under a 30 cap on a 60 Hz grid: the gaps, in vblanks, + /// between the grid releases. + fn gaps_after(first: Pacing, n_first: usize) -> Vec { + let g = grid(16_666_667); + let then = pacing(30, Some(g)); + let mut s = Schedule::default(); + let mut now = g.anchor_ns + 1_000_000; + for _ in 0..n_first { + now = now.max(s.next_ns); + s.book_paced(now, first); + now += 1_000_000; + } + let mut starts = Vec::new(); + for _ in 0..7 { + now = now.max(s.next_ns) + 50_000; + starts.push(now); + s.book_paced(now, then); + now += 1_000_000; + } + vblanks(g, &starts) + .windows(2) + .map(|w| w[1] - w[0]) + .collect() + } + + #[test] + fn a_grid_cap_binds_after_frames_booked_on_the_clock() { + // The first grid release follows the clock's last due point, which is + // off the grid, so it may land a vblank later; every one after it is + // two vblanks apart. Before the fix the gaps were [0, 1, 1, 1, 1, 1]. + let check = |gaps: Vec| { + assert!((2..=3).contains(&gaps[0]), "{gaps:?}"); + assert_eq!(gaps[1..], [2, 2, 2, 2, 2], "{gaps:?}"); + }; + // Frame generation turned off at a cap below the display's rate: the + // source-rate gate's clock (40 fps) first, then 30 on 60. + check(gaps_after(Pacing::on_clock(40), 10)); + // A cap moved from above the panel's rate (90 on 60, the cap's own + // clock) to below it. + check(gaps_after(Pacing::on_clock(90), 10)); + // And with no clock booking at all. + assert_eq!(gaps_after(Pacing::Off, 0), [2, 2, 2, 2, 2, 2]); + } + + #[test] + fn a_phase_push_does_not_let_two_releases_into_one_vblank() { + let g = grid(P60); + let iv = interval(pacing(60, Some(g))); + let mut s = Schedule::default(); + let due = s.book(g.anchor_ns + 10_000, g, iv); + // The daemon re-anchors the grid 400 us later; the guest is released + // on the old point, which is now just before the new one. + let moved = Grid { + anchor_ns: g.anchor_ns + 400_000, + ..g + }; + let next = s.book(due + 20_000, moved, iv); + assert_eq!(next, due + 400_000 + P60); + // And 400 us earlier: the old point is just after the new one. + let earlier = Grid { + anchor_ns: g.anchor_ns - 400_000, + ..g + }; + let after = s.book(next + 20_000, earlier, iv); + assert_eq!(after, next - 800_000 + P60); + } + + #[test] + fn a_guest_below_the_cap_is_not_quantized_to_a_divisor() { + // 50 fps on a 60 Hz cap: every frame is late for its release point, so + // the gate never holds it, and it keeps its own 20 ms cadence. + let g = grid(16_666_667); + let iv = interval(pacing(60, Some(g))); + let mut s = Schedule::default(); + let mut t = g.anchor_ns; + for _ in 0..100 { + assert!(s.next_ns == 0 || t >= s.next_ns, "held at {t}"); + s.book(t, g, iv); + t += 20_000_000; + } + } + + #[test] + fn a_late_frame_on_a_divisor_cap_waits_for_no_more_than_the_schedule() { + // 30 on 60, the guest late by most of a vblank: it starts at once, and + // the one after it is due at the slot after the one it was given. + let g = grid(P60); + let iv = interval(pacing(30, Some(g))); + let mut s = Schedule::default(); + let due = s.book(g.anchor_ns, g, iv); + assert_eq!(due, g.anchor_ns + 2 * P60); + assert_eq!(s.book(due + P60 / 2, g, iv), due + 2 * P60); + } + + /// One test for the process-global grid, so no other test races it. + #[test] + fn the_grid_is_the_daemons_flip_stamp_grid_and_needs_its_lead() { + // No note yet: no grid. + assert_eq!(current_grid(), None); + let note = VsyncNote { + anchor_ns: 5_000_000_000, + period_ns: P60 as u32, + lead_ns: 9_000_000, + stamp_lead_ns: 0, + }; + // A note without a stamp lead (a daemon that predates it): no grid. + apply_note(note); + assert_eq!(current_grid(), None); + // With one: the latch grid moved earlier by the stamp lead, the + // instants gamescope's vblanks are stamped at. + apply_note(VsyncNote { + stamp_lead_ns: 2_500_000, + ..note + }); + let g = current_grid().expect("the daemon's grid"); + assert_eq!(g.clock, GateClock::Vsync); + assert_eq!(g.period_ns, P60); + assert_eq!(g.anchor_ns, 5_000_000_000 - 2_500_000); + apply_note(VsyncNote { + anchor_ns: 0, + period_ns: 0, + lead_ns: 0, + stamp_lead_ns: 0, + }); + // No grid at all: the cap's own clock. + assert_eq!( + pacing(30, None), + Pacing::Clock { + period_ns: NANOS_PER_SECOND / 30 + } + ); + } + + #[test] + fn caps_above_the_rate_and_none_are_not_on_the_grid() { + let g = grid(P120); + assert_eq!(pacing(FPS_UNCAPPED, Some(g)), Pacing::Off); + assert_eq!( + pacing(190, Some(g)), + Pacing::Clock { + period_ns: NANOS_PER_SECOND / 190 + } + ); + // 59 on 60 Hz is not the display's rate: 59, on vblanks. + let g60 = grid(16_666_667); + assert_eq!(interval(pacing(59, Some(g60))), NANOS_PER_SECOND / 59); + } +} diff --git a/modules/gndrm-android/src/vsync.rs b/modules/gndrm-android/src/vsync.rs index 0549da9..d41ec57 100644 --- a/modules/gndrm-android/src/vsync.rs +++ b/modules/gndrm-android/src/vsync.rs @@ -124,13 +124,17 @@ impl Tracker { /// Fold in one completion: the latch point `SurfaceFlinger` took the frame /// at and the instant the daemon's apply of it returned (0 when unknown). /// `pacing` says whether a shim is placing frames by the note, which is - /// when the lead can be measured. Returns the note to push when it moved. + /// when the lead can be measured. `stamp_lead` is how far ahead of a latch + /// the flip events are stamped now (0 when they follow no grid); it rides + /// in the note so the shim can find gamescope's vblank grid. Returns the + /// note to push when it moved. fn track( &mut self, latch_ns: u64, applied_ns: u64, period: u64, pacing: bool, + stamp_lead: u64, ) -> Option { if self.anchor_ns == 0 { self.anchor_ns = latch_ns; @@ -175,6 +179,7 @@ impl Tracker { anchor_ns: self.anchor_ns, period_ns: u32::try_from(period).unwrap_or(u32::MAX), lead_ns: u32::try_from(self.lead_ns).unwrap_or(u32::MAX), + stamp_lead_ns: u32::try_from(stamp_lead).unwrap_or(u32::MAX), }; let moved = match self.pushed { None => true, @@ -184,6 +189,7 @@ impl Tracker { u64::from(p.period_ns.abs_diff(note.period_ns)) > PERIOD_STEP_NS || phase > PUSH_STEP_NS || u64::from(p.lead_ns.abs_diff(note.lead_ns)) > PUSH_STEP_NS + || u64::from(p.stamp_lead_ns.abs_diff(note.stamp_lead_ns)) > PUSH_STEP_NS } }; if moved { @@ -195,7 +201,7 @@ impl Tracker { } /// The current note, recorded as pushed; `None` until a latch was seen. - fn note(&mut self, period: u64) -> Option { + fn note(&mut self, period: u64, stamp_lead: u64) -> Option { if self.anchor_ns == 0 || period == 0 { return None; } @@ -203,6 +209,7 @@ impl Tracker { anchor_ns: self.anchor_ns, period_ns: u32::try_from(period).unwrap_or(u32::MAX), lead_ns: u32::try_from(self.lead_ns).unwrap_or(u32::MAX), + stamp_lead_ns: u32::try_from(stamp_lead).unwrap_or(u32::MAX), }; self.pushed = Some(note); Some(note) @@ -227,6 +234,16 @@ fn period_ns(now_ns: u64) -> u64 { } } +/// How far ahead of its latch the flip events are stamped at `now_ns`, 0 while +/// they follow no display grid (`gndrm_daemon::vblank::display_grid`). At least +/// 1 ns while they do, so 0 always means "not known" to the shim, which does not +/// run its vblank-grid gate without it (a fixed `GNDRM_GS_LATCH_LEAD_US=0` would +/// otherwise read as an old daemon). +#[cfg(target_os = "android")] +fn stamp_lead_ns(now_ns: u64) -> u64 { + gndrm_daemon::vblank::display_grid(now_ns).map_or(0, |g| g.lead_ns.max(1)) +} + /// One publish's completion: the latch point `SurfaceFlinger` took it at and /// the instant the daemon's apply of it returned. Called from the completion /// callback, on `SurfaceFlinger`'s thread. @@ -237,10 +254,12 @@ pub(crate) fn sample(latch_ns: u64, applied_ns: u64) { return; } let pacing = crate::lsfg_pacing() && SUBSCRIBED.load(Ordering::Relaxed); + // Read before the tracker's lock: the display's state has a lock of its own. + let stamp_lead = stamp_lead_ns(latch_ns); let note = { let mut t = TRACKER.lock().unwrap_or_else(|e| e.into_inner()); let first = t.pushed.is_none(); - let note = t.track(latch_ns, applied_ns, period, pacing); + let note = t.track(latch_ns, applied_ns, period, pacing, stamp_lead); // The flip events' fallback timeline, and Choreographer's cross-check. gndrm_daemon::vblank::note_latch_grid(t.anchor_ns, period); if first && note.is_some() { @@ -272,11 +291,13 @@ pub(crate) fn sample(latch_ns: u64, applied_ns: u64) { /// The note to hand a shim now, `None` before any latch was seen. #[cfg(target_os = "android")] pub(crate) fn note() -> Option { - let period = period_ns(now_ns()); + let now = now_ns(); + let period = period_ns(now); + let stamp_lead = stamp_lead_ns(now); TRACKER .lock() .unwrap_or_else(|e| e.into_inner()) - .note(period) + .note(period, stamp_lead) } #[cfg(test)] @@ -309,7 +330,7 @@ mod tests { fn the_first_sample_anchors_the_grid_and_is_pushed() { let mut t = fresh(); let note = t - .track(1_000_000_000, 0, P, false) + .track(1_000_000_000, 0, P, false, 0) .expect("first sample pushes"); assert_eq!(note.anchor_ns, 1_000_000_000); assert_eq!(u64::from(note.period_ns), P); @@ -319,12 +340,12 @@ mod tests { #[test] fn samples_on_the_grid_do_not_push_again() { let mut t = fresh(); - t.track(1_000_000_000, 0, P, false); + t.track(1_000_000_000, 0, P, false, 0); for k in 1..200 { // Jitter of +-100 us, as `SurfaceFlinger`'s latch wakeups have. let jitter = if k % 2 == 0 { 100_000 } else { -100_000 }; let latch = (1_000_000_000 + k * P) as i64 + jitter; - assert_eq!(t.track(latch as u64, 0, P, false), None, "sample {k}"); + assert_eq!(t.track(latch as u64, 0, P, false, 0), None, "sample {k}"); } // The anchor stayed within the jitter of the grid. assert!(wrap(t.anchor_ns as i64 - 1_000_000_000, P as i64).unsigned_abs() < 150_000); @@ -333,12 +354,12 @@ mod tests { #[test] fn a_moved_latch_phase_is_adopted_after_a_run_of_samples() { let mut t = fresh(); - t.track(1_000_000_000, 0, P, false); + t.track(1_000_000_000, 0, P, false, 0); // `SurfaceFlinger` moves its latch 3 ms later from now on. let shift = 3_000_000; let mut pushed = None; for k in 1..=REPHASE_AFTER as u64 + 1 { - if let Some(n) = t.track(1_000_000_000 + k * P + shift, 0, P, false) { + if let Some(n) = t.track(1_000_000_000 + k * P + shift, 0, P, false, 0) { pushed = Some(n); } } @@ -355,9 +376,9 @@ mod tests { #[test] fn a_single_late_sample_is_ignored() { let mut t = fresh(); - t.track(1_000_000_000, 0, P, false); - assert_eq!(t.track(1_000_000_000 + P + 3_000_000, 0, P, false), None); - assert_eq!(t.track(1_000_000_000 + 2 * P, 0, P, false), None); + t.track(1_000_000_000, 0, P, false, 0); + assert_eq!(t.track(1_000_000_000 + P + 3_000_000, 0, P, false, 0), None); + assert_eq!(t.track(1_000_000_000 + 2 * P, 0, P, false, 0), None); assert_eq!(wrap(t.anchor_ns as i64 - 1_000_000_000, P as i64), 0); } @@ -366,13 +387,13 @@ mod tests { /// `pipeline` later, and is taken at the first latch point after that. fn settle(pipeline: u64) -> u64 { let mut t = fresh(); - t.track(1_000_000_000, 0, P, true); + t.track(1_000_000_000, 0, P, true, 0); for k in 1..400u64 { let target = 1_000_000_000 + k * P; let applied = target - t.lead_ns + pipeline; // The latch that actually takes it. let latch = if applied < target { target } else { target + P }; - t.track(latch, applied, P, true); + t.track(latch, applied, P, true, 0); } t.lead_ns } @@ -395,7 +416,7 @@ mod tests { #[test] fn the_lead_is_not_touched_while_nothing_paces_on_it() { let mut t = fresh(); - t.track(1_000_000_000, 0, P, false); + t.track(1_000_000_000, 0, P, false, 0); let lead = t.lead_ns; for k in 1..50u64 { t.track( @@ -403,17 +424,38 @@ mod tests { 1_000_000_000 + k * P - 100_000, P, false, + 0, ); } assert_eq!(t.lead_ns, lead); } + #[test] + fn the_stamp_lead_rides_in_the_note_and_a_move_is_pushed() { + let mut t = fresh(); + let first = t + .track(1_000_000_000, 0, P, false, 2_000_000) + .expect("first sample pushes"); + assert_eq!(first.stamp_lead_ns, 2_000_000); + // A wander inside the push step is not worth a push. + assert_eq!(t.track(1_000_000_000 + P, 0, P, false, 2_200_000), None); + // A move past it is. + let moved = t + .track(1_000_000_000 + 2 * P, 0, P, false, 2_400_000) + .expect("a moved stamp lead is pushed"); + assert_eq!(moved.stamp_lead_ns, 2_400_000); + assert_eq!( + t.note(P, 2_400_000).map(|n| n.stamp_lead_ns), + Some(2_400_000) + ); + } + #[test] fn note_is_none_until_a_latch_was_seen() { let mut t = fresh(); - assert_eq!(t.note(P), None); - t.track(1_000_000_000, 0, P, false); - assert!(t.note(P).is_some()); - assert_eq!(t.note(0), None); + assert_eq!(t.note(P, 0), None); + t.track(1_000_000_000, 0, P, false, 0); + assert!(t.note(P, 0).is_some()); + assert_eq!(t.note(0, 0), None); } } diff --git a/modules/gndrm-uapi/src/wire.rs b/modules/gndrm-uapi/src/wire.rs index e8e95b6..6038f34 100644 --- a/modules/gndrm-uapi/src/wire.rs +++ b/modules/gndrm-uapi/src/wire.rs @@ -750,16 +750,23 @@ pub const VSYNC_MAGIC: u32 = 0x474E_5653; /// The panel's latch grid, daemon -> shim, on the scan-out channel. /// /// Encoded little-endian, 24 bytes: -/// `magic:u32, reserved:u32, anchor_ns:u64, period_ns:u32, lead_ns:u32`. +/// `magic:u32, stamp_lead_ns:u32, anchor_ns:u64, period_ns:u32, lead_ns:u32`. /// /// `SurfaceFlinger` takes the newest buffer a layer holds at each of its latch /// points, which sit one panel period apart. `anchor_ns` is one such point in /// `CLOCK_MONOTONIC`, `period_ns` their spacing, and `lead_ns` how long before a /// latch point a frame has to be committed to the compositor to be the buffer /// that point takes. The daemon measures all three from its own publishes and -/// pushes a note whenever one of them moves. The period is the panel rate the -/// app reported; the phase and the lead are measured. A shim that never -/// receives one paces on its own clock, as before, and beats against the panel. +/// pushes a note whenever one of them moves. The period is the display's +/// measured vsync spacing while the daemon has one, the rate the app reported +/// otherwise; the phase and the lead are measured. A shim that never receives +/// one paces on its own clock, as before, and beats against the panel. +/// +/// `stamp_lead_ns` is how far ahead of its latch point the daemon stamps a +/// page-flip event, the instant gamescope schedules its next composite from: +/// `anchor_ns - stamp_lead_ns + k * period_ns` is the vblank grid gamescope +/// sees. It sat in a reserved field, so a daemon that predates it sends 0 and +/// a shim that predates it ignores it. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct VsyncNote { /// A latch point, `CLOCK_MONOTONIC` ns. @@ -768,6 +775,9 @@ pub struct VsyncNote { pub period_ns: u32, /// How far ahead of a latch point a commit has to be made to land on it, ns. pub lead_ns: u32, + /// How far ahead of its latch point a flip event is stamped, ns; 0 when the + /// daemon does not say. + pub stamp_lead_ns: u32, } impl VsyncNote { @@ -777,6 +787,7 @@ impl VsyncNote { pub fn encode(self) -> [u8; Self::LEN] { let mut b = [0u8; Self::LEN]; b[0..4].copy_from_slice(&VSYNC_MAGIC.to_le_bytes()); + b[4..8].copy_from_slice(&self.stamp_lead_ns.to_le_bytes()); b[8..16].copy_from_slice(&self.anchor_ns.to_le_bytes()); b[16..20].copy_from_slice(&self.period_ns.to_le_bytes()); b[20..24].copy_from_slice(&self.lead_ns.to_le_bytes()); @@ -793,6 +804,7 @@ impl VsyncNote { anchor_ns: u64::from_le_bytes(b[8..16].try_into().ok()?), period_ns: u32::from_le_bytes(b[16..20].try_into().ok()?), lead_ns: u32::from_le_bytes(b[20..24].try_into().ok()?), + stamp_lead_ns: u32::from_le_bytes(b[4..8].try_into().ok()?), }) } } @@ -930,10 +942,22 @@ mod tests { anchor_ns: 0x0123_4567_89ab_cdef, period_ns: 8_333_333, lead_ns: 4_000_000, + stamp_lead_ns: 2_500_000, }; let b = note.encode(); assert_eq!(b.len(), VsyncNote::LEN); assert_eq!(VsyncNote::decode(&b), Some(note)); + // A note from a daemon that predates the stamp lead: the field was + // reserved and zero, and reads as "not said". + let mut old = b; + old[4..8].fill(0); + assert_eq!( + VsyncNote::decode(&old), + Some(VsyncNote { + stamp_lead_ns: 0, + ..note + }) + ); assert_eq!(VsyncNote::decode(&b[..23]), None); assert_eq!(VsyncNote::decode(&FrameCapNote { fps: 60 }.encode()), None); assert_eq!(FrameCapNote::decode(&b), None);