Skip to content

perf(presets): keep large preset libraries light - #606

Merged
CaYatur merged 2 commits into
mainfrom
milkdrop/library-scale-574
Sep 22, 2026
Merged

CaYatur merged 2 commits into
mainfrom
milkdrop/library-scale-574

Conversation

@CaYatur

@CaYatur CaYatur commented Sep 22, 2026 •

Copy link
Copy Markdown
Owner

Part 1 of 2 for #574. Importing a ZIP or a library found on the machine means thousands of presets, and the store could not carry them. Measured in an isolated copy with the corpus (10,347 MilkDrop presets, 116 MB):

  • presets:list took 3.8 s;
  • one save held the main process for about 2.4 s;
  • the panel used 631 MB;
  • every web client received 116 MB on every change.

The user chose to scale the store before adding the importer (part 2).

Store (src/main/presets-store.js)

  • Files are read once, in the background at start-up, and kept. Saves and deletes update the cache.
  • list() compares only the folder's file names, so a file added or removed by hand still shows up. A file edited by hand shows up after a restart.
  • Bulk saves are written in batches with pauses, so the main process keeps serving audio, lights and IPC. They report progress and keep the pack's own order.

Change broadcast. A save or delete sends only what changed (presets-delta); the full list is sent only when a page opens. Pages apply it in the store's order: newest first, ties broken by id, so presets saved in the same millisecond no longer land in a machine-dependent order.

Web clients get MilkDrop presets without their sources.

  • The overlay engine fetches a source by id (/milkdrop/preset, behind the token) only when it is about to draw that preset.
  • The current preset stays on screen until the source arrives.
  • If the source cannot be fetched, the pick is dropped rather than falling back to the default preset.

Found on the way. Every preset change rebuilt the visualizer window's whole layer stack, so importing or deleting a MilkDrop preset restarted the MilkDrop picture on screen. The stack is now rebuilt only when a Studio preset changes. The MilkDrop panel no longer keeps a second copy of the list.

Measured after the change, same corpus: presets:list 0.3 s, a save 29 ms (91 ms with a window open), the panel 273 MB. With 2,000 presets and a web overlay open in a real (headless) browser:

  • the connect message was 271 KB instead of about 30 MB;
  • the overlay followed the window's auto advance preset for preset and fetched the sources of the seven presets it showed;
  • a delete took 68 ms while the window's MilkDrop kept drawing (same preset, 97 frames in 1.2 s);
  • the window, overlay and panel lists all dropped to 1,999.

Not done here. Visualizer windows still hold every source (141 MB with 10,347 presets), since they draw from them directly. The importer itself is part 2.

Tests. 2052 unit tests pass, 20 of them new. They cover:

  • the store against a temporary folder;
  • the change broadcast on the page;
  • the stream server stripping sources and serving them behind the token;
  • the web bridge;
  • the visualizer's rebuild rule;
  • the engine waiting for a source.

26 of 26 mutations are caught. A review found that a delete made while the background load runs could come back in the cache (get() would still find it); that is fixed and tested deterministically. npm run smoke passes, and the user's settings and 635 preset files are unchanged.

Refs #574, #560

Importing a ZIP or a library found on the machine means thousands of
presets, and the store could not carry them. Measured with the corpus
(10,347 MilkDrop presets, 116 MB) in an isolated copy: presets:list took
3.8 s, a single save held the main process for about 2.4 s, the panel
used 631 MB, and every web client received 116 MB on every change.

The store now reads the files once, in the background at start-up, and
keeps them; saves and deletes update the cache, and list() only compares
the folder's file names, so files added or removed by hand still show up.
Bulk saves are written in batches with pauses, report progress and keep
the pack's own order.

A save or delete sends only what changed (presets-delta); the full list
is sent only when a page opens. Pages apply changes in the store's order,
newest first with ties broken by id, so presets saved in the same
millisecond no longer land in a machine-dependent order.

Web clients get MilkDrop presets without their sources. The overlay
engine fetches a source by id (/milkdrop/preset, behind the token) only
when it is about to draw that preset, keeps the current one on screen
until it arrives, and drops the pick if it cannot be fetched.

Found on the way: every preset change rebuilt the visualizer window's
whole layer stack, so importing or deleting a MilkDrop preset restarted
the MilkDrop picture. The stack is now rebuilt only when a Studio preset
changes. The MilkDrop panel no longer keeps a second copy of the list.

Same corpus after the change: list 0.3 s, a save 29 ms (91 ms with a
window open), panel 273 MB. With 2,000 presets and a web overlay in a
real browser, the connect message went from about 30 MB to 271 KB.

Refs #574, #560
The background load builds its cache from a list of names taken when it
starts. A preset deleted after the load had read its file came back in
the cache: list() dropped it again on its next folder check, but get()
did not, so the web overlay could still be served a deleted preset's
source. Deletes made while the load runs are now removed from what it
installs, and get() falls back to the file for a preset saved during
the load. The race is tested deterministically, with the load's reads
held until the delete has happened.

The engine also says so once in the console if a page ever receives a
sourceless preset with no bridge to fetch the source; today only the
web overlay gets sourceless presets.

Refs #574, #560
@CaYatur
CaYatur merged commit 671912c into main Sep 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant