Skip to content

android: A51 — the scan-out image keeps the pool buffer's row stride - #20

Merged
joshuatam merged 1 commit into
gamenativefrom
fix/gamescope-resolution-scanout
Oct 30, 2026
Merged

joshuatam merged 1 commit into
gamenativefrom
fix/gamescope-resolution-scanout

Conversation

@joshuatam

@joshuatam joshuatam commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

What this fixes

A composited frame was sheared whenever the session's scan-out width was not a multiple of 320 px.

The scan-out image is allocated from the session's pool, whose rows are gralloc's — padded for widths gralloc does not like (1024 px → 5120 B on Adreno), where a LINEAR Vulkan image is otherwise laid out at width * 4. Created through a DRM format-modifier list, Turnip picks the tight pitch. The compositor reads the buffer at the buffer's stride, so every row the GPU writes lands further along than the row read back: the frame is sheared, repeated across the width, with the tail of the buffer never written.

Only a composited frame writes this image. Direct scan-out (one screen-sized layer, no filter that needs the shader, no look, no ReShade) is the guest's own buffer — which is why the fault appeared exactly when NEAREST/FSR/NIS/SGSR forced vulkan_composite. Measured on a 1024x768 session into a 1280x960 panel: SurfaceFlinger reported the layer's stride as 5120 B while the composite wrote at 4096. At 1280 and 1920 wide the two pitches coincide, so no output this stack has run before could show it.

The change

The pitch is not derivable from the geometry, so A51 asks for it:

  • the shim owns the pool and answers gamenative_drm_scanout_pitch(w, h) (declared weak, resolved from gamenative-drm's libgndrm_shim.so);
  • when it answers, the image is created with an explicit LINEAR modifier at that stride instead of the modifier list — which also makes the AddFB2 export report the stride the compositor reads;
  • the symbol is weak, so a real KMS session keeps the driver's own layout;
  • the create is gated to output images with no imported dma-buf and no memory to reuse, and the modifier list is unlinked so the create chain carries exactly one layout authority (the external-memory and format-list nodes stay as the chain's tail).

android/patches/gamescope/0049-A51-scanout-image-at-the-pool-row-stride.patch applies on top of the 48-patch train, and the train re-applies cleanly around it.

android/check-payload-closure.sh also gains libnativewindow.so in its system allowlist: the ICD dlopens it for the guest's AHardwareBuffer receive path and it exists on every Android this payload runs on. That check was failing on a pre-existing edge, not on anything added here — RESULT: CLEAN again.

Verification

  • Cross-build for aarch64-linux-android succeeds with zero warnings; the train applies cleanly onto a clean 48-patch tree, and clean-train + A51 reproduces the working tree byte for byte.
  • In the built binary gamenative_drm_scanout_pitch is WEAK UND (as intended), and GLOBAL FUNC in the shim.
  • check-payload-closure.sh: RESULT: CLEAN.
  • On device (Retroid Pocket Nova, 1280x960@120Hz, Adreno 740 + Turnip 26.3.0-devel), every session logs the pitch it lays its scan-out image out at:
    • 1024x768 sessions: output image 1024x768 at the pool's row stride 5120 (tight 4096) — the padded case, and the frame is now coherent with SGSR forcing the composite;
    • 1280x960 / 1280x720 sessions: … 5120 (tight 5120) — unchanged, as they must be.
  • SurfaceFlinger agrees on the whole chain for the padded case: layer geomBufferSize=[0 0 1024 768], buffer stride:5120 bytes, layer transform 1.2500 → 1280x960, composition=DEVICE.

Six device cases (container screen size × Upscaling Resolution × filter) were run end to end through the app — the full matrix and screenshots are in utkarshdalal/GameNative#2283, which is the app side of this feature.

Companion PRs

Depends on the shim export landing in gamenative-drm; without it the symbol stays unresolved and this patch is a no-op (the driver's own layout, i.e. the previous behaviour).

A scan-out image is allocated from the session's pool, whose rows are gralloc's:
padded for widths gralloc does not like (1024 px -> 5120 B on Adreno), where a
LINEAR Vulkan image is otherwise laid out at width * 4. Created through a
DRM-format-modifier list, Turnip picks that tight pitch; the compositor reads the
buffer at the buffer's stride, so every row the GPU writes lands a little further
along than the row read back — the frame is sheared, repeated across the width,
with the tail of the buffer never written.

Only a composited frame writes this image. Direct scanout (one screen-sized
layer, no filter that needs the shader, no look, no ReShade) is the guest's own
buffer, which is why the fault appears exactly when NEAREST/FSR/NIS/SGSR forces
vulkan_composite — measured on a 1024x768 session into a 1280x960 panel, where
SurfaceFlinger reported the layer's stride as 5120 B while the composite wrote
at 4096. At 1280 and 1920 wide the two pitches coincide, so no output this stack
has run before could show it.

The pitch is not derivable from the geometry, so A51 asks for it: the shim owns
the pool and answers `gamenative_drm_scanout_pitch(w, h)` (gamenative-drm). When
it answers, the image is created with an explicit LINEAR modifier at that stride
instead of the modifier list, which also makes the AddFB2 export report the
stride the compositor reads. The symbol is weak, so a real KMS session keeps the
driver's own layout.

android/patches/gamescope/0049-A51-scanout-image-at-the-pool-row-stride.patch
  The patch itself (rendervulkan.cpp): the weak declaration beside BInit, and the
  explicit-modifier create — the modifier list unlinked so the create chain
  carries one layout authority, with the external-memory and format-list nodes
  kept as the chain's tail; gated to output images with no imported dma-buf and
  no memory to reuse. Applies on top of the 48-patch train, and the train
  re-applies cleanly around it.

android/check-payload-closure.sh
  libnativewindow.so joins the system allowlist: the ICD dlopens it for the
  guest's AHardwareBuffer receive path and it is present on every Android this
  payload runs on. The check was failing on that pre-existing edge, not on
  anything this change adds; RESULT is CLEAN again.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant