Image/Video Background Refactor #831
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Make clicks working
+ Fix UI layout resized incorrectly + Fix UI state updated incorrectly + Cleaning-up constants and unused/duplicate methods + Add option to select pip by hovering instead of clicking + Add event handlers for ItemIndex and ItemsCount changes
+ By preloading it to an empty grid, this issue can be fixed. Yeah, for no reason, even when the element has already been added to the ContentPresenter, the first frame of the element will report the offset and size to 0x0. So by tricking it to load into an empty grid, the element will force itself to load and reports the correct offset and size.
The countdown is currently using Timer for counting down the slideshow. This approach, however, uses interval event on each ticks so the duration of the count might be inaccurate (but not significant). The tick resolution is 100ms for now.
+ CodeQA + Simplify content placement on XAML + Fade-in/out countdown bar on entering and exiting + Fix content not updating while adding/removing items from Items collection
+ Implements Parallax effect on mouse hover + The image is currently using a dummy one and not yet compositing. Will be fully implemented later (ETA: at the end of this week, hopefully)
+ Bug fix on object -> double conversion, causing throw + Add IMediaCacheHandler interface for handling cache on media loading
Now we are able to perform direct frame drawing without dependency on CanvasBitmap or any SoftwareBitmap which requires back-to-back GPU->CPU->GPU frame copy. The GPU usage difference is negligible, as close as mainstream video player GPU usage.
Member
|
Fatal crash on region change |
Member
Author
As mentioned above, the code for these changes haven't been applied to main Collapse Launcher code yet and still on |
Member
|
As per @neon-nyan statement above, copilot code check is removed until PR is actually finished in Collapse side Cc @Cryotechnic |
+ This also occurs during element ``Unloaded`` event + The loader will perform media seeking to the last position if the previous source was the same
+ Copying all controls to main Collapse Launcher code. Adjustment in the background mechanism is still required and not done yet
Also use foreground binding
100% reviewed source file: 'en_US.json' on 'zh_CN'.
Removing old codes. Some codes will be replaced with new one later.
+ Use async-detached routine on image loading + Add PNG version of poster for GetPlaceholderBackgroundImageFrom + Add default placeholder image on MainPage + Add ParallaxResetOnUnfocused property on LayeredBackgroundImage
+ Fix a gradual memory leaks on both GPU and General memory due to ComMarshalling operation. + Make MediaPlayer.CopyFrameToVideoSurface able to run outside of UI thread (this reduces call overhead on UI threads significantly) + Implement context menu (right-click menu) for background image and selector pills at the top of the launcher UI. + Remove test project and codes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Main Goal
As described by the title, this PR contains major changes to Collapse Launcher's Image/Video Background mechanism to match current behavior on HoYoPlay which have multiple sequence of background and layered images.
Here's the current list of what have been implemented:
Control-based element.PipsPager(Implemented as:NewPipsPager)FlipView(Implemented asPanelSlideshow)This implementation of a
Slideshowelement added some customization whichFlipViewcan't accommodate, including:Transitionanimation for navigating between elements.SlideshowDuration.LayeredBackgroundImage)This custom control has major improvements than its previous background loading mechanism, including:
Built-in external decoder support for unsupported WIC (Windows Imaging Component) formats, including
.webp,.avifand.jxrviaPhotoScalerlibrary.Using direct Bitmap copy for drawing video frames via native
ISurfaceImageSourceNativeWithD2D.Previously, Collapse Launcher has to rely on software bitmaps to draw each frames into the
UIElementto pulls of the XAML-effects and compositions, including XAML Blur.The reason why Collapse Launcher has to perform this multiple frame copy is because
MediaPlayer, by default, doesn't support XAML effects and Composition and instead, each frames were drawn directly into different pipeline ofIDXGISwapChainwhich decoupled from the main WinUI's, causing the element above it feels a bit off (including no acrylic effects for any elements above it)This causes few overhead, in-terms of memory allocation or CPU cycles as it requires to perform multiple buffer copy steps:
SoftwareBitmap(CPU)MediaPlayer's frame buffer intoSoftwareBitmap(GPU -> CPU)VirtualCanvasImageSourceas source forImageto copy fromSoftwareBitmap(GPU)SoftwareBitmapbuffer intoVirtualCanvasImageSource(CPU -> GPU)Image's surface (GPU)Now, thanks to huge changes on
Hi3Helper.Win32submodule (and documentations provided by Win2D and DirectNAot), we managed to copy the frames fromMediaPlayerdirectly into backed-VirtualSurfaceImageSource/ISurfaceImageSourceNativeWithD2Dpipeline, loaded asImage's source, with XAML-effects and Composition support and without additional buffer copy on CPU side. Now, the process only require these steps:ISurfaceImageSourceNativeWithD2D->BeginDrawto createIDirect3DSurfacedrawing session (GPU) (Must be inside of UI Thread)MediaPlayer's frame buffer intoIDirect3DSurface(GPU -> GPU) (Must be inside of UI Thread)Image's surface viaISurfaceImageSourceNativeWithD2D->EndDraw(GPU) (Must be inside of UI Thread)(Update: 2026/01/13)
The call routines are now reduced. We are now reusing
IDirect3DSurfaceviaCanvasRenderTargetfor multiple draw routines instead of creating one every single time fromISurfaceImageSourceNativeWithD2D. Due to the nature ofCanvasRenderTargetbeing multi-threaded, we can now move the frame buffer copy routine to outside of the UI Thread. This significantly reduces the amount of overhead and thus, making the UI more responsive. The following call routines changed as follows:MediaPlayer's frame buffer intoIDirect3DSurface(GPU -> GPU) (Outside of UI Thread).ICanvasImageSource.CreateDrawingSession, Invalidate and draw buffers (GPU) (Must be inside of UI Thread).TODO List:
MediaPlayer,ISurfaceImageSourceNativeWithD2Dand underlayID3D11Device + ID2D1Device[Optional] Use fully-asynchronous drawing session onISurfaceImageSourceNativeWithD2D(Detaching draw session from UI-thread).Currently not possible due to SurfaceImageSource thread model bound to the UI-Thread and not set to Multi-thread mode.(Update: 2026/01/13) frame buffer copy routine can now be executed outside of the UI Thread. Though, buffers draw and invalidation still need to be called from inside of the UI Thread.PR Status :
Templates
Changelog Prefixes