Skip to content

feat: Super resolution (AMD FSR 1.0) with Sharpening, on PC, PS5 and Xbox - #89

Open
xservzeroman wants to merge 3 commits into
feat/ps5-frame-pacingfrom
feat/fsr
Open

xservzeroman wants to merge 3 commits into
feat/ps5-frame-pacingfrom
feat/fsr

Conversation

@xservzeroman

Copy link
Copy Markdown

Super resolution with AMD FidelityFX Super Resolution 1.0, for PS5 streams first. Mockups are in docs/mockups/fsr/; fsr_compare.png is rendered by the shaders in this PR, run in WebGL 2.

Changelog

  • binding/video/FsrRenderer.kt:
    • Its own GL thread with EGL and an ES 3.0 context. The decoder renders into a SurfaceTexture (inputSurface).
    • Each frame takes three passes: copy the OES frame to an RGBA8 texture at the stream size; EASU to the screen surface's size; RCAS to the window.
    • It sets eglPresentationTimeANDROID from the SurfaceTexture timestamp, so frame pacing still applies.
    • attachOutput/detachOutput follow the screen surface (folds); frames are taken and dropped while no output is attached.
    • The shaders are ports of ffx_fsr1.h's 32-bit FsrEasuF/FsrEasuTapF/FsrEasuSetF and FsrRcasF (no denoise). FsrEasuCon is computed as floats. textureGather is four texelFetches in gather order, since ES 3.0 has no gather. The approximate reciprocals are exact ones with zero guards.
  • Ps5StreamActivity:
    • decoderSurface() uses FSR when Settings, Picture, Super resolution is FSR 1.0 and the screen surface is more than 1.1× the stream's height. Otherwise it uses the screen surface, as before, and also on any setup failure.
    • A surface change re-attaches FSR's output. surfaceDestroyed detaches it, and the decoder keeps FSR's surface.
    • FSR is released after the session is disposed.
  • Settings, Picture (both apps): Super resolution (Off/FSR 1.0) and, when it's on, Sharpening (0 to 100%, default 60%; RCAS stops = (1 - x) × 2).
  • About, NOTICE: AMD FidelityFX Super Resolution 1.0 (MIT), with its licence text.
  • Build: test234. Lint: 0 errors.

Tested

  • Off-device: both shaders compiled and ran in Chromium's WebGL 2 (GLSL ES 3.00, SwiftShader) with no GL error, upscaling a 1280×720 frame 3.4×. The output is upright and edge-adaptive.

Not tested yet (Saleh is away)

  • On the Odin and the Fold: PS5 at 1080p and 720p with FSR on. Check the picture and the frame rate, Decode time (FSR adds GPU time after decode), and fold and unfold during a stream.

🤖 Generated with Claude Code

…ening (test234)

FsrRenderer: the decoder renders into a SurfaceTexture; copy, EASU and RCAS passes (ports of
ffx_fsr1.h, MIT) draw to the screen surface with the decoder's presentation time. Only when the
stream is clearly smaller than the screen; a fold moves only FSR's output. The mockup comparison
is now rendered by these shaders in WebGL 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t235)

PC: MediaCodecDecoderRenderer.setSurfaceOverride, no setFixedSize while FSR is planned, off with HDR.
Xbox: FsrGlDrawer as SurfaceViewRenderer's GlDrawer, hardware scaler off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@xservzeroman

Copy link
Copy Markdown
Author

test235: PC and Xbox too.

  • FsrPasses now holds the shaders and the three passes, shared by FsrRenderer (PS5, PC) and the new FsrGlDrawer (Xbox). RCAS gained a uOrigin for drawing into a viewport that isn't at (0, 0).
  • PC.
    • MediaCodecDecoderRenderer.setSurfaceOverride. Game plans FSR when it's on, the stream isn't HDR and the stream is more than 1.1× smaller than the real display.
    • It then skips setFixedSize, and the aspect ratio too with Stretch, so the surface stays screen-sized.
    • FSR is created at the first surfaceChanged, detached in surfaceDestroyed, and released after conn.stop().
  • Xbox. SurfaceViewRenderer.init(..., CONFIG_PLAIN, FsrGlDrawer) with the hardware scaler off. OES frames under the size threshold go through FSR; YUV/RGB frames, and contexts without ES 3, use GlRectDrawer.
  • Re-checked: the refactored shaders compile and render in WebGL 2 with no GL error.
  • Lint: 0 errors.

🤖 Generated with Claude Code

@xservzeroman xservzeroman changed the title feat: Super resolution (AMD FSR 1.0) with Sharpening, PS5 first feat: Super resolution (AMD FSR 1.0) with Sharpening, on PC, PS5 and Xbox Oct 21, 2026
…atency keys stepwise (test236)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@xservzeroman

Copy link
Copy Markdown
Author

Tested on the JAL Meeting Room TV (AQUOS-TVX19B, Android 11, MediaTek m7322, Mali-470): PS5-423 at home, reached over PSN, 720p60, 2026-10-21.

test235 test236
PSN on OpenSSL 3: AIA, hole-punch, BANG/ECDH, streaminfo ✅ connected in 84 s, including waking the console ✅
Decoder OMX.MS.HEVC.Decoder low-latency keys refused (configure -10000), then all dropped levels 0, 1 and 2 refused; level 3 (priority) accepted
FSR eglChooseConfig failed, plain picture shown eglCreateContext ES 3 failed (EGL_BAD_CONFIG), plain picture shown
Quick menu 60 fps, 25 ms network, 0% lost, Decode time 20.5 ms 60 fps, 9 ms network, 0% lost, Decode time 20.3 ms
Put PS5 to rest ✅ ✅
  • The GPU is a Mali-470, OpenGL ES 2.0 only (ro.opengles.version 131072). FSR needs ES 3.0, so it can't run there, and the fallback to the plain picture is the right behaviour.
  • Afterwards the test-only 720p setting was removed again, and the TV was left on its home screen with PhoneLinkService running.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants