Repository navigation
MilkDrop Phase 2: visual rendering quality, user textures (img.ini), motion vectors, and smooth preset transitions #560
Description
Activity
- changed the title
[-]MilkDrop Phase 2: user textures (img.ini), motion vectors, and smooth preset transitions[/-][+]MilkDrop Phase 2: visual rendering quality, user textures (img.ini), motion vectors, and smooth preset transitions[/+]on Sep 5, 2026 Preliminary Investigation & Potential Observations for Phase 2 Implementation
Ahead of beginning active development and step-by-step resolution, a comprehensive exploratory analysis of the codebase and engine architecture was conducted. Below is a structured summary of potential observations and preliminary findings identified across the pipeline, intended to be verified and addressed step by step:
1. Potential Pipeline & Layer Stack Discrepancy (Observed Interaction with Standalone Mode)
- Layer Stack Synchronization: Preliminary inspection indicates that when
cfg.layerStack.enabled: true,resolve(cfg)primarily prioritizescfg.layers. Selecting a preset via the standalone MilkDrop panel updatescfg.visualizer.type = 'milkdrop', but does not appear to synchronize with the active visualizer layer incfg.layers. This potentially explains reports where selecting presets in the dedicated panel seems to have no visible effect when the layer stack is active. - Background Layer Collision: When the layer stack is disabled (
cfg.layerStack.enabled: false),synthesize(cfg)appears to instantiate a background layer (ly_bg) beneath the visualizer layer (ly_vis). Because MilkDrop renders an opaque, full-screen feedback and composite pipeline, the underlying background layer might cause redundant overhead or potential compositing/context conflicts. - Navigation Lifecycle: The standalone menu entry in
admin.jscurrently usesshow: () => v.type === 'milkdrop', which potentially hides the section if the user navigates while another mode or layer is selected, leading to navigation desynchronization.
2. Visual Rendering Quality & High-DPI Fidelity (Potential Findings)
- Supersampling Bounds: In
modes/milkdrop.js, the rendering scale factor is currently capped atMath.min(1, ...), which limits rendering to canvas dimensions. Adding configurable internal supersampling (e.g., 1.5× or 2.0× scale before downsampling) is observed as a potential enhancement for razor-sharp rendering on 1440p / 4K displays. - Blur Pyramid & Calibration: The 3-tier blur pass currently utilizes a 5-tap separable blur kernel, while
blur1_min..max,blur2_min..max,blur3_min..maxuniforms are currently fixed at defaults (0 and 1). Implementing MilkDrop 2's calibrated Gaussian weights and proper range scaling could potentially enhance neon glow and bloom fidelity. - Buffer Precision & Filtering: Configurable texture filtering (including anisotropic filtering via
EXT_texture_filter_anisotropic) and precision options (half-float vs 32-bit float) may help mitigate subtle color banding in dark feedback loops.
3. Motion Vectors (
motion_vectorsPass)- Variable bindings for
mv_x,mv_y,mv_dx,mv_dy,mv_l,mv_r,mv_g,mv_b,mv_aare not currently wired into the equation evaluation pools orPARAM_ALIAS. - A dedicated line/grid rendering pass before the composite shader appears to be the missing component for presets utilizing motion vector meshes.
4. Smooth Preset Transitions (Crossfade / Warp Dissolve)
- Currently, preset switches execute immediate pipeline replacements.
- Implementing a dual-buffer state machine with an interpolation shader (fRating / 1.0s–5.0s transition duration) will be tested to eliminate hard cuts during manual or automated preset cycling.
5. Interactive Mouse Input Routing
mouse_x,mouse_y, andmouse_downvariables are not currently captured from canvas events or forwarded toper_frame/per_pixelvariable pools. Routing normalized coordinates (0..1) will enable interactive preset equations.
6. HLSL to GLSL Translation Edge Cases (~8.3% Gap)
- Analysis indicates that the remaining compilation edge cases in the 10k+ preset corpus primarily stem from:
- Matrix-matrix multiplication semantics (
mul(mat, mat)) and vector-matrix overloads. - Complex swizzles across nested ternary expressions (
(cond ? a : b).xyz). - Missing intrinsics such as
tex2Dprojor bool-to-arithmetic negation patterns (-!x).
- Matrix-matrix multiplication semantics (
- Refining these edge cases is projected to push compilation rates above 95%.
7. Dynamic Mesh Grid Density
MESH_X(64) andMESH_Y(48) are currently static constants. Exposing configurable densities (e.g. 32×24 for performance, up to 96×72 for ultra-smooth warp geometry) with dynamic vertex buffer reallocation will be tested.
8. User / Sprite Textures (
img.ini& Sampler Binding)- Unrecognized sampler names (
sampler_fire,sampler_lichen, etc.) currently fallback to procedural noise. Adding asset resolution for preset-bundled sprites andimg.inireferences will complete texture mapping.
Next Steps
We will be addressing and validating each item step-by-step through dedicated test suites before merging any changes to ensure complete engine stability.
- Layer Stack Synchronization: Preliminary inspection indicates that when
- added 4 commits that reference this issue
on Sep 8, 2026 Phase 2 progress — measured, not estimated
Two harnesses were built first, because the 91.7% figure in the issue text left nothing reproducible behind:
scripts/milkdrop-compile-rate.js— parses every.milk, translates warp+comp, compiles each stage in a real WebGL2 context, groups failures by cause.scripts/milkdrop-render-rate.js— actually renders each preset for 20 frames and reads the pixels back. Compile rate cannot see a shader that compiles and then draws black, blows out to white, or freezes.
Both are seeded and repeatable. Numbers below are from the full 10,332-preset corpus (the earlier claim was measured on the 505-preset original pack only).
before now Shader stages compiling 95.4% 99.2% Presets with every stage clean 91.4% 98.5% Presets producing a live image 89.2% 93.0% Fell back to the fixed path 63 12 Item 6 — remaining shader translations: done, target exceeded
Asked for >95%; 99.2% across a corpus 20x larger than the original measurement. The work was HLSL's implicit conversions: at call sites of the preset's own functions (one template function alone took down 154 stages), multiple declarators converted separately, built-in names GLSL refuses to redeclare routed to helpers, function parameters no longer poisoning the global type table, and narrowing extended to
returnexpressions andif/whileconditions.Item 3 — motion vectors: done
92% of the corpus switches the grid on; visibility is
mv_a, which 8.6% set in the file and another 5.7% drive fromper_frame. Drawn after the shapes and waves and before the composite, so they land in the feedback buffer and flow with the picture.Item 8 — standalone vs layer stack: done
resolve()only readscfg.layersonce that list is non-empty and ignorescfg.visualizer.typeentirely, so with the stack on the panel's preset selection did nothing at all. The panel now points the stack's visualizer layer at MilkDrop; text and now-playing layers are skipped, and with several visualizer layers only the first is converted.Item 7 — mesh density: done
24x18 through 128x96 in the panel, aspect fixed at 4:3 as in MilkDrop's own list. Changing it rebuilds the vertex array, index buffer and VAO.
Item 5 — mouse input: done
mouse_x,mouse_y,mouse_downreach per-frame and the custom wave/shape pools. No preset in the corpus reads them, so the value is in user-written presets.Item 1 — visual quality: partly done
- Internal resolution scale (0.5x–2x) — done, clamped and still bounded by
maxSize. - Blur pyramid — progressive downscale with a separable Gaussian was already in place;
blur1_min/blur1_maxnow come from the preset'sb1n/b1xinstead of constants. - Configurable bilinear/anisotropic filtering and precision controls — not done. Feedback buffers are fp16 with linear filtering; no setting is exposed.
Item 4 — preset transitions: partly done
A dissolve of the previous preset's last frame over the new one, off by default, capped at 3s. This is not MilkDrop's dual-pipeline blend: MilkDrop runs both presets at once and blends their warp meshes and composite passes, while here the old picture is a frozen frame. Under about a second the difference is invisible; on long transitions it is noticeable. The dual pipeline needs two preset objects, two shader sets, two target pairs and two blur chains, and was not built.
Item 2 — user textures (
img.ini): not doneThe corpus ships zero image assets, so a loader could not be verified against a single real file. Unknown textures still fall back to seeded noise and the translator reports it as a soft note rather than failing silently.
Found by measurement, not in the issue
- Textured shapes (
shapecode_N_textured=1) had no support at all: the engine filled the polygon with the shape's own colour, almost always white. 40.3% of the corpus uses one and 12.1% make it big enough to cover the screen — this was the entire "blown out" class, a solid white screen, invisible to the compile measurement. - Rotation matrices (
rot_s/d/f/vf/uf/rand 1..4) were missing entirely; 28 presets read them. - MilkDrop did not work in the web overlay at all:
overlay.htmlloadedmodes/milkdrop.jsbut none of its three dependencies, sodraw()threw on the first frame. A test now requires the web overlay to load everything the desktop visualizer loads.
Phase 2, second round — per-sampler filtering, and the textures presets ask for
Two more items, both measured against the same 10,332-preset corpus as the previous comment. Commits: 17b8e07, a436ffa.
A correction to what I wrote last time about item 1
I reported the filtering half of item 1 as "not done" because no setting was exposed. That was only half the story, and the half that mattered was worse: the translator was discarding the filter prefix entirely.
MilkDrop offers one texture under several names, the prefix saying how to read it —
sampler_pw_mainis point-sampled and wrapped,sampler_fc_mainis filtered and clamped. Every spelling was being collapsed onto one uniform. Measured:Presets using at least one prefixed sampler 38.3% Presets reading one texture through two different prefixes 22.7% (2,349) Occurrences of sampler_pw_mainalone6,310, all filtered instead of point-sampled Those 2,349 presets wrote two different samplings and got the same answer twice. The compile measurement could never see it — the shaders compiled cleanly, they just read the wrong sampler state.
Each spelling now gets its own uniform and its own texture unit, with filter and wrap carried by a sampler object bound to that unit. Sampler objects are what makes this possible: the state lives on the unit, not the texture, so one texture can be read two ways in the same frame. Unit 0 is deliberately left alone, because
_bindMainsets the wrap there from the preset's ownwrapflag and a sampler object would override it silently.The unit budget was measured, not assumed: a preset asks for at most six variant units (three presets in 10,332), so units 10–15 suffice and fit inside the 16 WebGL2 guarantees — exactly 16 on the test machine. The limit is read from the driver, and a preset that would exceed it falls back to the canonical unit, which is the old behaviour: right texture, approximate filtering.
Proved by drawing the difference between two variants of one texture. If they collapse to one uniform the difference is exactly zero:
old new filter 0.00000 0.49669 wrap 0.00000 0.50152 control 0.00000 0.00000 (same variant twice — must stay zero)Item 2 — user textures: the
sampler_*half is doneA preset asks for its images by name:
sampler_wormswantsworms.jpg. Preset packs do not ship them, so the engine bound noise. Measured demand:Presets asking for at least one user texture 16.9% (1,748) Distinct texture names 66 ( wormsalone in 544 presets)Presets using rand00..15random slots242 Output → MilkDrop → Texture Pack now points at the
texturesfolder of a MilkDrop installation. Files are read in the main process and handed to the renderer as data rather than opening another privileged protocol — a preset asks for at most six textures and the images are small. The scope check refuses anything with a separator, a drive letter,.., or a non-image extension, and lives in its own module so a test can run it.rand00..15picks a random eligible texture per preset and slot, honouring MilkDrop's name-prefix filter (rand00_smalltiledonly draws from names startingsmalltiled). A file literally namedrand00.pngwins over the random pick — explicit beats implicit.texsize_<name>was silently zero. Presets declare it themselves and MilkDrop binds the line:float4 texsize_lichen; // auto-binds; .xy = (w,h); .zw = (1/w,1/h)Here it stayed an ordinary global, so a preset scaling by it got zero and drew the wrong thing without an error. The compile measurement could not see this either — the declaration is valid GLSL.
texsize_lichenappears in 74 places, ~180 user-texture size reads in total. The preset's own line is now dropped and a uniform put in its place, carrying the size of whichever texture is really bound.Loading is asynchronous and the draw loop cannot wait, so noise stays bound until the image arrives, and a missing file keeps the noise. Failing hard would turn 1,748 presets that are wrong-but-working today into black screens.
Proved against real files — a 64×32 pure red
worms.pngand a pure blueclouds.png, through the main process's own resolver:sampler_worms R=255 G=0 B=0 variance 0 sampler_fw_clouds R=0 G=0 B=255 variance 0 texsize_worms R=41 G=20 (64/400 and 32/400 of 255) sampler_rand00 R=0 G=0 B=255 picked a real texture control, no folder R=127 G=127 B=128 variance 29.67, noiseItem 2 — the
img.inihalf is not done, and here is whyimg.iniis MilkDrop's keyboard-triggered sprite overlay, not preset shading. The corpus contains zeroimg.inifiles and zero presets withimgNNkeys — 10,332 presets from the original and cream-of-the-crop packs. There is no demand to build against and nothing to verify a loader with. Saying "item 2 done" would be wrong; so would leaving it open without this reason.Measurements
Compile rate is byte-identical to the baseline: 16,212 / 16,346 stages, 99.2%; 98.5% of presets fully clean. That is the point — both changes add uniform declarations and stop a text substitution, and must not touch compilation.
Render rate 93.0% → 93.1%. It cannot move further on this corpus: there are no image files in it, so all 172 texture-substituted presets in the 900-preset sample still fall back to noise. The measurement is being honest, not the feature failing.
npm test1231,npm run smokePASS. The packaged self-test has not been re-run —src/changed substantially anddist/predates it. That gate is required before a release, not before these commits.What is left
Item 4's dual pipeline is the one real remaining gap. Transitions dissolve the previous preset's last frame; MilkDrop runs both presets at once and blends their warp meshes and composite passes. It needs two preset objects, two shader sets, two target pairs and two blur chains. Under about a second the difference is invisible; on long transitions it is noticeable, and the panel says so.
Item 1's precision control is deliberately not exposed: the feedback buffers are already fp16 with an 8-bit fallback, chosen because 8-bit quantisation error accumulates frame over frame and bands the dark tones. A setting there could only make it worse.
- added a commit that references this issue
on Sep 9, 2026 Phase 2, third round — item 1 closed, and five more values that compiled fine and drew the wrong picture
Same 10,332-preset corpus as the previous two comments. Commits: 19d2988, 56497da, 292a58d.
Item 1 is now done
Texture filtering & anisotropy. Noise and blur textures now carry mipmaps and
EXT_texture_filter_anisotropicat the driver maximum. Blur mipmaps are regenerated every frame after the framebuffer is released — an incomplete mip chain reads black, so the generation and theLINEAR_MIPMAP_LINEARfilter ship together.The blur pyramid was wrong in three separate ways. Its horizontal and vertical passes wrote to the same size; MilkDrop's ratios are
[0.5, 0.25] / [0.125, 0.125] / [0.0625, 0.0625], so our first level sat at twice MilkDrop's resolution and the "blurred" copy was not blurry enough. The kernel was a narrow five-tap Gaussian where MilkDrop uses a wide eight-weight kernel, collapsed into four bilinear tap pairs horizontally and two vertically. AndGetBlurNread back with* max + min— for the default 0..1 range that is right by accident, but a preset that narrowsb1n/b1xgot back a different number than it wrote. Levels are now written scaled into the preset's range and read back through the inverse, which also puts the full 8 bits of the texture inside the range the preset actually uses.The kernel moved into uniforms rather than a second shader, so the two passes differ by values only.
Four more, found by measuring rather than reading
was is sampler_noisevol_lq/hqdeclared sampler2D;tex3Ddiscarded zreal 3D textures, 32³ noise_mq/noise_hqidentical parameters MilkDrop's lattice at zoom 4 and 8, cubic interpolation roam/slow_roam0.3/0.7/1.1/1.5, range −1..1 0.3/1.3/5/20, range 0..1 hue_shaderone colour for the whole frame four corner colours, bilinear across the screen The volume-noise one is the largest: 2,217 presets (21.4% of the corpus) call
tex3D, 4,322 calls in total, and every one of them was reading a single slice while the preset believed it was moving through a volume.Fixing it exposed a second defect immediately. GLSL ES 3.00 has no implicit precision for
sampler3D, so without aprecision highp sampler3D;line every stage that reads volume noise fails to compile — measured at 57 of 57 stages in a 40-preset slice. That is the kind of thing the compile harness earns its keep on.The roam range mattered more than the frequencies: a multiplier running −1..1 instead of 0..1 can flip a preset's direction, and five presets that used to blow the feedback loop out to white now render.
Numbers
All six sit behind a "MilkDrop Uyumu" setting, default on. It changes values only — same shaders, same textures, same texture units — so there is one code path and one test surface. With it off, volume noise is still 3D and the hue corners are still four corners; they just all receive the same colour.
measurement before after presets producing a picture (900 sample, textures loaded) 93.1% 93.8% shader stages compiling 99.2% 99.2% unit tests 1231 1275 Compile rate is flat by design — none of this was a compile problem, which is exactly why the compile gate could not see any of it.
On a 300-preset sample the switch itself is worth a point: 95.3% on, 94.3% off. A path offered to the user should not be an unmeasured path, so the harness takes
--legacy.The texture pack, measured
--textures=<dir>now loads a pack into the measurement page and lets the engine's own loader resolve it, so item 2 can finally be measured against real images instead of noise. On the 900-preset sample: 172 presets ask for a texture, 131 (76.2%) get every one they asked for, 23 get some. The report separates "asked" (counted at translation time) from "got" (counted after drawing) — those were being conflated.Deliberately not done
rad/ang. Three sources give three different conventions and none of them is decisive. Changingangrotates or mirrors every radial preset in the corpus; that is too much blast radius for the evidence available.- Blur edge darkening (
b1ed). The engine does not track that value at all — a separate piece of work, not a value fix. - Item 4's dual-pipeline blend. Transitions ship (789139a) but as a dissolve over a frozen frame, not two presets running at once. Still open, as noted before.
- The harness deliberately does not clear feedback buffers between presets. It was tried: 93.1% → 85.8%, with the black class going from 14 to 81, because many presets only disturb an existing image. Carrying the picture over is also what the app and MilkDrop do. The cost is that a preset's class depends slightly on the one before it — measured at 4 flips in 900 across a large engine change — so runs are compared preset by preset, never by the headline percentage alone.
One thing not clean
npm run smokefailed once during this round, immediately after two back-to-back GPU measurement runs, with a zero-sized canvas indrawImage. It then passed four times in a row, cleanmainpassed at that moment too, and the same load pattern was reproduced deliberately without triggering it again. Recorded as transient rather than quietly dropped.Fidelity round 4 — letting the corpus pick the work
The previous rounds worked down this issue's list. This one started somewhere else: instead of picking items from the list, I diffed what 10,347 presets actually write against what the engine actually reads, and let the frequencies decide the order. That turned up six gaps that were not on the list at all, three of them affecting more of the corpus than anything still open on it.
The method is a script that walks every
.milk, collects every name assigned inper_frame/per_pixel/wavecode/shapecodeand every header key, and subtracts the set the engine reads throughP.get(...). A name a preset writes and the engine never reads is a silent gap: it compiles, it renders, it is simply not what the author drew.Ranked gap table (10,347 presets)
Gap Presets % State before fWarpScale≠ 18,265 79.9 Ignored; the warp used invented constants fWaveSmoothing> 08,171 79.0 Never applied b1edblur edge darkening7,094 68.6 Not implemented fWarpAnimSpeed≠ 14,556 44.0 Ignored bModWaveAlphaByVolume4,027 38.9 Never applied Visible border ( ob_*/ib_*)3,906 37.8 Never drawn fZoomExponent≠ 13,826 37.0 No alias — never reached the equations fWaveParam≠ 03,484 33.7 No alias — never reached the equations fShader> 02,273 22.0 Ignored bDarkenCenter711 6.9 Aliased, never read bRedBlueStereo59 0.6 Not implemented Separately, and not from the table — found while reading a reference port to get the warp constants right:
- The warp mesh ran in a vertically mirrored space. Ours is OpenGL's axis (
v = 0at the bottom); MilkDrop's runs top to bottom. Both are internally consistent, so nothing ever failed —dy,cyand the direction of rotation were simply seen in a mirror. 48.8% of presets usedyorcy, 49.5% write a non-zerorot, 30.0% readyinper_pixel.
What shipped
Six commits, each measured on its own against the same seeded 900-preset sample with the texture pack loaded.
1. Two header values never reached the equations —
57c90a9fZoomExponent→zoomexpandfWaveParam→wave_mystery. Two missing rows in the alias table, same bug class as thefDecaymiss the file's own comment already documents.zoomexpsits in the middle of the mesh transform,pow(zoom, pow(zoomexp, rad*2-1)); falling back to 1 removes the radial falloff entirely, so the zoom becomes uniform across the screen. 6,892 presets (66.6%) set one of the two in the header and never touch it in their equations, so for those the header was the only source. 93.8% → 94.0%.2. The mesh's vertical direction —
10ac544Measured rather than argued: a fixed shape at the centre,
dy = +0.02, no other motion. The trail's centre of mass went to 0.294 (up) and to 0.602 withdy = -0.02. MilkDrop's algebra says the opposite — the sample point shifts byv -= dy, and withvrunning downwards a positivedycarries the image down. After the fix the same experiment gives 0.701 and 0.393, mirrored exactly. The fix sits at two boundaries only: theythat goes into the equations and thevthat comes out to the texture. Shapes, waves and motion vectors were never wrong; only the mesh was. 94.0% → 93.4%, and each of the seven presets that changed class renders identically on its own with the switch either way.3. The warp ripple —
55d023cFour fixed numbers (5, 3, 4, 2) produce a standing wave. In MilkDrop none of the four frequencies is fixed — each is driven by its own cosine (11.68±4 at 1.413, 8.77±3 at 1.113, 10.54±3 at 1.233, 11.49±4 at 0.933), so the pattern keeps reweaving itself.
fWarpScaleenters as its reciprocal andfWarpAnimSpeedscales time; a scale of 0 is clamped, because an infinite frequency would fold the whole mesh (the corpus bottoms out at 0.01, where 1,862 presets sit at or near). 93.4% → 92.8%, all six class changes traced to the harness's feedback carry-over rather than to the change.4. Waveform smoothing and volume-driven alpha —
7d08517Three settings parsed and then dropped:
fWaveSmoothing(79.0%),bModWaveAlphaByVolume(38.9%), and a custom wave's ownsmoothing, whose default is 0.5 — so it is asked for even when the preset never writes it. The two filters are deliberately different: the basic waveform gets a single forward one-pole blend with the scale folded into the mix; a custom wave getssqrt(smoothing * 0.98)and two passes, forward then backward, so the curve softens without sliding. A test pins that difference by measuring the centre of mass of a single spike through each filter. 92.8% → 93.1%, three recovered, none regressed.5. Borders and centre darkening —
d7f8fbb99.7% of the corpus carries
ob_*andib_*in its header and the engine drew neither. Both go into the feedback target, after the motion vectors and before the composite — MilkDrop's own order, and the whole point: the border is not a frame painted on top, it is content injected at the edge every frame and then warped inward with everything else. Presets that looked frozen or blown out were missing that injection.The ring is four separate strips, not a big rectangle with a smaller one over it — with a translucent border the overlapping corners would blend twice and come out darker than the sides. The inner ring starts where the outer one ends.
Verified by rendering a synthetic preset and reading pixels: red outer ring at 4px, blue inner at 12px, body untouched at 40px, and 255 → 236 at the exact centre with the darkening on.
Measured separately: centre darkening on its own is 93.1% → 93.0%, neutral. Borders are 93.0% → 96.2%, 32 presets recovered.
6. A gate that depended on what was playing —
07b65e6Unrelated to MilkDrop but found by the release gates: the English-interface scan read the live Windows media session title, so
npm run smokefailed with "1 untranslated string" whenever the machine happened to be playing a track with a Turkish name.Numbers
Before After Presets rendering a live image (900 sample) 93.8% 96.2% Shader stage compile rate 99.2% 99.2% (untouched) Unit tests 1,275 1,318 The arc is not monotonic — 93.8 → 94.0 → 93.4 → 92.8 → 93.1 → 96.2 — and that is worth being explicit about. The harness measures "produces a varied image", not "matches MilkDrop". When a preset starts behaving the way MilkDrop makes it behave, it can leave the clean class: a real zoom exponent darkens some presets, a fully opaque black border bleeds inward when texture wrap is off. Every class change at every step was re-rendered on its own, and of the 17 presets that moved across the five engine steps, 13 render identically with the fidelity switch either way — the class change came from the picture inherited from the previous preset in the run, not from the change under test. That measurement is now recorded in the harness itself.
All of it sits behind the existing MilkDrop Fidelity switch, whose description in the panel was updated to say what it now covers.
Measured and declined
fShader(22.0%) — 2,273 presets write it, but only 632 (6.1%) have no composite shader and therefore genuinely need it. It blends MilkDrop 1's per-pixel lighting shader, which would have to be built from nothing before it could be blended. Cost is out of proportion to 6.1%.bRedBlueStereo(0.6%) — anaglyph output, 59 presets.b1ed(68.6%) — high count, but it is a blur-pyramid change and the pyramid was rewritten last round. Landing it here would put two independent changes in one measurement window.rad/angconventions — still deferred. Three sources give three conventions and there is no discriminator yet; a wrong convention breaks presets that currently work.- Item 4's dual-pipeline blend — still not built, for the reason given before: two complete pipelines running at once is the highest-blast-radius change in this issue and the harness measures single-pipeline output, so a regression would not show.
Queued for the next round
Found while reading references this round, none of them implemented:
- Transform order. MilkDrop zooms about the screen centre, then stretches about
(cx, cy), then warps, then rotates. We zoom about(cx, cy), rotate, stretch, and warp last. - Aspect is never applied in the mesh at all.
aspectx/aspectymay be inverted. Ours isGW >= GH ? GW/GH : 1(≥ 1 on a wide screen); the reference is≤ 1and puts the non-unit value on the other axis. 25.7% of presets read aspect in a shader and 11.2% in equations, so this needs verifying against MilkDrop's own documentation before anything is touched — it is a bigger blast radius than anything shipped this round.- Custom wave amplitude. Ours is
raw/128 * scaling; the reference israw * 0.004 * scaling * wave_scale— about 1.95× smaller and multiplied bywave_scale, which we drop entirely.
- The warp mesh ran in a vertically mirrored space. Ours is OpenGL's axis (
Fidelity round 5 — from MilkDrop 2's own source
The last round worked from a JavaScript port. This one works from Nullsoft's MilkDrop 2 C++ source (
vis_milk2/milkdropfs.cpp,plugin.cpp,state.cpp, and the shader headerinclude.fx). That matters more than it sounds: it settled a question that had been deferred for three rounds for lack of a discriminator, and it showed that one of last round's own numbers had been taken from a comment rather than from the code.Everything below is parameters and algorithms read from that source and re-implemented in our own structure. Nothing was copied.
What the primary source settled
radandang— deferred since round 2, now decided.plugin.cpp:1679-1684:rad = sqrt(x²·aspX² + y²·aspY²) ang = atan2(y·aspY, x·aspX) // the exact centre node pinned to 0radis not normalised — on a square screen it reaches √2 at the corner. Ours multiplied by 0.7071 and clamped to 1, so the top half of the exponent inpow(zoom, pow(zoomexp, rad*2-1))was never reached and the zoom opened towards the edge far more weakly than it should.angkeeps(-π, π]; ours shifted to0..2π, which is invisible through sin and cos but not through a non-integer multiplier or a comparison. 27.0% of presets readradinper_pixel, 13.3% readang.aspectx/aspectywere swapped. MilkDrop keeps two forms. The internal pair is always ≤ 1 — on a wide screenaspX = 1,aspY = H/W— and the mesh,rad,angand the shader'saspectare built from it. What a preset sees in its equations is the inverse (var_pf_aspectx = m_fInvAspectX), so it readsaspectx = 1,aspecty = W/H. We gave the larger number toaspectx. Not an inversion — a swap: a preset correcting for a wide screen corrected the wrong axis. 11.2% read it in equations, 25.7% in a shader, and the shader'saspectwas doubly wrong because it needs the internal pair in.xyand the inverses in.zw.Roam came from a comment. Last round I took the four frequencies from the comment in MilkDrop's shader header — "
~0.3, ~1.3, ~5, ~20". The code (milkdropfs.cpp:3736) has0.329 / 1.293 / 5.070 / 20.051with phases1.2 / 3.9 / 2.5 / 5.4, and the slow pair0.0050 / 0.0085 / 0.0133 / 0.0217with phases2.7 / 5.3 / 4.5 / 3.8. We had no phases at all, so all four components read 1 at t = 0 and a preset written to mix a slow component against a fast one saw one flat number on its first frame.bMotionVectorsOnis not a runtime flag.state.cpp:1402converts it tomv_aat load time (0 stays 0, anything else becomes 1), and an explicitmv_ain the file overrides it. 86 presets carry it and not one of them also writesmv_a, so for those it is the only source. 82 are off — our default of 0 got that right by accident — but 4 are on and drew no motion vectors at all. Implementing it as a runtime gate instead, which is what I tried first, would have silently switched motion vectors off for the 10,261 presets that do carrymv_a, because a preset without the key reads zero from the pool.What shipped
1. Aspect, roam and
pixelsx/pixelsy—67631a9The swap above, plus the exact roam constants and phases.
pixelsx/pixelsywere never in the pool at all, so a preset reading them saw zero and any line dividing by a pixel count went to infinity; 178 presets (1.7%) read them. Not behind the switch — the old value was not a behaviour, it was a hole. 96.2% → 96.2%, neutral: the harness renders at 240×180, which is 4:3, so a swap has little room to show there. It has the most effect at 16:9.2. The mesh transform —
02cb9baOne commit rather than four, because MilkDrop's node transform is one pipeline —
radfeeds the zoom exponent, the aspect pair appears in bothradand the initialu/v, and the aspect-undo at the end only means anything if the aspect-apply at the start is there.Besides
radandang: aspect never entered the mesh at all, and the order was different. MilkDrop zooms about the centre of the screen, then stretches, warps, rotates, translates, and undoes the aspect. Ours zoomed about(cx,cy), had rotation and stretch swapped, and pushed the warp to the end — presets usingrottogether withcx/cycame out somewhere else entirely.The vertical flip now lives in the formula's own
-yrather than in a separate step, which is the easiest way to get this commit wrong; the drift probe confirms the trail still goes down for a positivedy(0.701 / 0.393, unchanged from the round that fixed it).96.2% → 95.7%. All six presets that left the clean class were rendered again on their own: two are black with the switch off and produce a bright moving image with it on (
182.milk,beta106i - Airhandler), and the other four are black or near-black either way, differing in the third or fourth decimal.3. Blur edge darkening (
b1ed) —921b230A blur samples from outside itself at the edge and what sits there is the clamped edge texel, so the border reads brighter than it should and a frame glows. MilkDrop multiplies by a factor falling off with the square root of the distance to the edge — the square root presses the darkening against the border, where a linear curve would dim the whole frame. Only the first vertical pass gets it; MilkDrop's source notes why, and it is worth repeating: applied at every level, the very blurred levels grow thick black lines along their top and left edges.
7,094 presets (68.6%) write a value above zero and 1,544 (14.9%) write nothing and need MilkDrop's default of 0.25 — the pool's natural zero would have meant no darkening for exactly the presets whose authors were looking at the default. 95.7% → 95.8%, one recovered, none lost.
4. Custom wave amplitude and source —
40d0344MilkDrop multiplies a custom wave's samples by
0.004 * scaling * wave_scaleand its samples sit in ±128 units, so on a sample reduced to ±1 the factor is 0.512. We used 1: every custom wave was drawn about twice the size its author chose, andwave_scalenever reached them.A wave with
spectrum = 1asks for the frequency spectrum, not the waveform, and we handed both the same time data — a preset meaning to draw a frequency distribution drew a wiggle. 2,398 presets (23.2%) carry at least one. Our spectrum is normalised differently from MilkDrop's, so the size of those waves is approximate, but the source is now right. The harness now feeds synthetic frequency data too; without it that path would never run under measurement. 95.8% → 95.7%, and the one preset that changed class is dimmer and still moving on its own.5. The shader's clock, and
bMotionVectorsOn—608e879MilkDrop hands
timeto two places as two different things: in the equation language it is time since the application started, in a shader it is time since the preset started, wrapped at 10,000. We gave both the application clock. That costs twice — phase, sincesin(time)starts from the same place on every launch in MilkDrop and from wherever uptime put it here; and precision, since a shader float loses resolution at large values and an installation left running for a day went stepped. The wrap exists for exactly that. 95.7% → 95.7%.6. A harness that could take the screen down — in
921b230The render harness writes progress dots to stdout, and a caller that closes the pipe —
| head -3will do it — made that write throwEPIPE. In Electron's main process an uncaught exception becomes a modal error box in the middle of the screen, and the measurement died with it. A progress dot cannot change a result, so the stream's error event is swallowed and every write is guarded.Numbers
Round 4 end Round 5 end Presets rendering a live image (900 sample) 96.2% 95.7% Shader stage compile rate 99.2% 99.2% (untouched) Unit tests 1,318 1,354 The number went down, and that is worth stating plainly rather than burying. The harness measures whether a preset produces a varied image, not whether it matches MilkDrop. A real zoom exponent darkens some presets; a preset whose zoom now opens out the way MilkDrop opens it can saturate. Every class change at every step was re-rendered in isolation, and the two largest ones went the other way — black without the fidelity switch, a bright moving image with it.
What the number cannot show is the part that matters most here: on a 16:9 screen the aspect swap and the missing aspect in the mesh were distorting every preset, and the harness renders 4:3.
Measured and declined
fShader(22.0%) — only 632 presets (6.1%) have no composite shader and genuinely need it; it would mean building MilkDrop 1's per-pixel lighting shader from nothing before it could be blended.bRedBlueStereo(0.6%) — anaglyph output, 59 presets.mip_x/mip_y/mip_xy/mip_avgandGetMain()— in MilkDrop's shader header but used by zero presets in the corpus. Measured before spending the commit.- Item 4's dual-pipeline blend — still the one large thing left in this issue, and still the highest-blast-radius change in it.
What is left
The corpus diff no longer turns up an unread name above 1% that has not been either implemented or measured and declined. What remains is not a gap list but the two items above: the dual-pipeline blend, and the spectrum scale for custom waves, which needs our FFT normalised the way MilkDrop normalises its own before those waves are the right size rather than merely the right shape.
Two corrections to the round-5 write-up, and a re-run of the gap scan
1. The line-number citations above are now pinned. They quote
plugin.cpp,milkdropfs.cppandstate.cppwithout saying which source tree, and a line number means nothing without one. They are fromjecassis/foo_vis_milk2, which carries Nullsoft's MilkDrop 2 C++ source, at commit5b44cea79a8678a8f7e516e8c0cce0e4681f299b(2024-11-17):gh api "repos/jecassis/foo_vis_milk2/contents/vis_milk2/<file>?ref=5b44cea79a8678a8f7e516e8c0cce0e4681f299b" --jq '.content' | base64 -dAll three were re-fetched at that commit and re-checked today.
plugin.cpp:1679-1684is the unnormalisedradand the centre-nodeang = 0;milkdropfs.cpp:3736is0.329 / 1.293 / 5.070 / 20.051with phases1.2 / 3.9 / 2.5 / 5.4, and the slow pair two lines below it;state.cpp:1402isbMotionVectorsOnconverted tomv_awith the explicitmv_aoverriding it on the very next line. The files are read, then deleted — they never enter this repository, and nothing is copied from them.2.
bMotionVectorsOnis below the harness's resolution, and the 95.7% → 95.7% line should not be read as evidence about it. The rule turns motion vectors on for 4 presets out of 10,347. In a 900-preset sample that is 0.3 presets expected. The measurement was neutral because the change is invisible to it, not because it was measured and found neutral. The evidence for that commit is the corpus count and the source, not the render rate.3. The "no unread name above 1%" claim was re-derived, not re-asserted. It rested on a hand-written audit list. The scan was re-run the other way round: instead of listing every name presets write — which counts each author's own local variable (
beat,atime,zm) as a gap — it starts from MilkDrop's own built-in names and subtracts the ones the engine actually reads.Twenty-four names came back. Every one was checked against the source by hand, and all but two are artefacts of how the extraction matched:
b1n/b1x/b2n/… are read through a built key (bkey[i] + 'n'),usedotsandthickoutlineare read asuseDots/thickOutline,num_instand the shape's second colourr2/g2/b2/a2are read through the block parser, and the input names (aspectx,meshx,pixelsx,mouse_*) are supplied by the engine rather than read from it — the scan was asking the wrong direction for those.The two that survive:
fshader— already above, measured and declined.monitor(43.2%) — MilkDrop's debug readout. It is not a rendering input there either; it only appears when the user asks for the debug display. Not reading it is correct, and it is the only remaining name in this table that a preset writes in earnest.
So the claim stands, with a named exception rather than an assurance.
59 remaining items
Three open points from this thread are closed on
mainvia #600. The last item, sprites frommilk_img.ini, is tracked in #577 and comes next.The default wave's green and blue stay in order (a504a78).
milkdropfs.cpp:3101-3102in the pinned source writes green fromcband blue fromcg. This was deferred three times because the vertex struct was not at hand.- The struct is plain floats in RGBA order (support.h:71-75), so that fork really does draw the two channels swapped.
- The fork is a D3D11 port. The D3D9 code it was ported from sets the colour in one call,
D3DCOLOR_RGBA_01(cr, cg, cb, alpha1)(mvsoft74/BeatDrop 53d83ee, milkdropfs.cpp:3225). The fork still carries that line as a comment in every wave mode. - So the swap came in with the port. The engine already draws in order, and a test now pins it in all eight modes.
Item 8's remaining points (7acbd7c). With the layer stack off, the background was drawn every frame under an opaque MilkDrop layer. That was a cost, not a glitch.
- Covered layers are now skipped only when every condition below holds. Their canvases are hidden so the compositor skips them too. When the lights sample the background, it keeps drawing.
- The engine says its last frame covered the canvas.
- The blend is normal and the opacity is full.
- There is no transform, mask or per-layer effect.
- The audio is ready.
- Measured in an isolated copy at 1920×1080 with the frame-rate cap off, alternating new and old:
- gradient background: 253.6/260.7 fps against 246.2/246.6;
- aurora background: 264.8/266.9 against 234.8/238.5.
- A MilkDrop-over-aurora export matches frame for frame before and after the change: 60 of 60 frames, each tree run twice.
- The sidebar lock was already gone: the section has had no visibility condition since b6ef117.
The
--shotsgenerator (eecef9a). Its time data was three low sines; it now takes the time data and both channels fromsrc/shared/demo-audio.js. Three generator faults were fixed on the way:- the broadcast scenes had lost their logo;
- a ghost title from a running transition showed up;
- a full run would overwrite the two separately made MilkDrop pictures.
21 README pictures are regenerated. The tunnel pair keeps its previous render: the shared signal is 3.6 dB quieter, and the tunnel's brightness follows loudness. Changing the signal's loudness would move the MilkDrop measurement's baseline, so it is left as a separate decision.
- added a commit that references this issue
on Sep 22, 2026 The last item of this issue — sprites from
milk_img.ini— is done onmainvia #601; details are in #577.With #600's three points, every item and every deferred point in this thread is now either done on
mainor measured and declined with its reason written down. Declined, for reference:bRedBlueStereo,mip_*/GetMain()andmonitoras a render input.Two things from this thread carry on elsewhere:
- 134 shader stages (0.8%) that still do not compile, which move to the fidelity follow-ups in MilkDrop: fidelity follow-ups after #560 #580;
- the demo signal's loudness. It is 3.6 dB quieter than the old screenshot signal, which kept two README pictures on their previous render. It is left as its own decision, because changing it moves the MilkDrop measurement's baseline.
- added 4 commits that reference this issue
on Sep 22, 2026 Engine PRs from this round, for the record (several referenced only their own issue): #625 (expression front end as MilkDrop 2 reads it, #580), #627 (equations a third faster, bit-identical, #621), #629 (MilkDrop 2 / 3 preset format, #567), #630 (borders), #631 (blur chain), #632 (fixed warp wrap and snap point), #633 (composite rad/ang) — all #580 — and #634 (noise pixels generated once per page, #621).
- added a commit that references this issue
on Sep 30, 2026 For the record, these engine PRs referenced #560 but aren't listed in an earlier comment here. All are merged, and main CI is green on the current main,
4ee1cfddfebe9fd074fcd367976332edb5e13f78.PR merge what #603 9b027a0Follows the system's reduce-motion setting (#581) #604 be64171The sprite network-path test names its platform (#577) #605 1fdba69Favourites, tags and search by author (#576) #606 671912cLarge preset libraries stay light #607 15b5075Import from ZIP packs, folders and the machine (#574) #608 f91068dPreset thumbnails (#575), with engine buffer and light sampler fixes #611 82ab5ecFidelity: stage rule, fixed composite and defaults (#580) #612 17121feFidelity: hue colour and the fixed composite (#580) #613 d83c4f7Fidelity: missing keys, and values that stay in the file (#580) #614 7050cc9Fidelity: preset files read the way MilkDrop reads them (#580) #619 8d7ecc3Equations run by MilkDrop 2's compiler rules (#580) #620 732ef32Each block keeps MilkDrop 2's own variables (#580) #628 9a4eb22In-app preset editor (#578) #656 ca2edbfMilkDrop 3 double presets, .milk2(#567)
Follow-up to #559 and the core MilkDrop engine rewrite shipped in v3.1.2.
While v3.1.2 restored CSP-safe equation execution (via AST closure trees) and WebGL2 HLSL warp/composite pixel shader compilation for 91.7% of shader stages across 10,347 presets, several visual fidelity and engine features remain to be implemented in a dedicated quality pass:
1. Visual Rendering Quality & High-DPI Fidelity
sampler_blur1,sampler_blur2,sampler_blur3): Implement MilkDrop 2's progressive downscaled blur pyramid with authentic Gaussian kernels, giving presets their characteristic neon glow and bloom depth.2. User / Sprite Textures (
img.ini/sampler_main&sampler_noise_*).milk/img.ini.3. Motion Vectors (
motion_vectorspass)mv_x,mv_y,mv_dx,mv_dy,mv_l,mv_r,mv_g,mv_b,mv_a) before the composite shader pass.4. Smooth Preset Transitions (Crossfade / Warp Dissolve)
fRating, variable transition duration from 1.0s to 5.0s with blend shader).5. Interactive Mouse Input
mouse_x,mouse_y,mouse_down) into equation per-frame and per-vertex variables.6. Remaining Shader Translations (8.3%)
7. Mesh Grid Density & Performance Settings
8. Standalone Mode vs Layer Stack Pipeline Discrepancy & Stability
LayerStack(kind: 'visualizer',type: 'milkdrop') because it receives an isolated<canvas>and independent WebGL2 rendering surface.cfg.layerStack.enabled: true,resolve(cfg)ignorescfg.visualizer.type = 'milkdrop', causing user preset selection to have no visible effect.cfg.layerStack.enabled: false),synthesize(cfg)instantiatesly_bgbeneathly_vis, causing visual clipping or alpha/context collisions with MilkDrop's internal ping-pong FBOs.cfg.visualizer.type !== 'milkdrop', which desynchronizes navigation when switching presets.