Repository navigation
Serve MiniMax-H3 video as a sibling serve mode (M5 Pro) - #502
Draft
zaiyang-zhang wants to merge 23 commits into
Draft
zaiyang-zhang wants to merge 23 commits into
zaiyang-zhang wants to merge 23 commits into
Conversation
zaiyang-zhang
changed the title
zaiyang-zhang
force-pushed
the
h3-video
branch
from
October 21, 2026 00:04
03d0966 to
25f6725
Compare
The h3 engine runs MiniMax-H3 after its text encoder: request preparation, the VDN-H3 DiT (resident or streamed from SSD), the video VAE, the TAEH3 preview, the audio VAE and the mp4 writer, behind a C ABI (h3.h, ABI v2). dev/tools/vendor_video.sh copies it from a checkout: its host code into runtime/video, its kernels into the video group of splash.metallib (runtime/metal/kernels/video, its kernel ABI in runtime/metal/abi/video), its converters into install/video, with the include paths rewritten for those places and nothing else changed (runtime/video/VENDORED). The host code builds into an archive of its own with -ffp-contract=off, as h3 builds it, and the prep kernels without fast math, since both reproduce their references bit for bit; the mp4 writer links AVFoundation. Nothing calls the engine yet. The architecture check leaves the vendored engine's clocks alone: they time its phases for its statistics only. THIRD_PARTY_NOTICES lists the code it derives from and the model's license. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A video model's engine takes one new client frame, VideoRequest (the clip's variant, frames, size, steps, seed, outputs and the prompt's tokens), and answers it with two new events: VideoProgress, a phase's step out of its steps with its time and the engine's peak footprint, and VideoOutput, an mp4 (the preview while the clip runs, then the clip, which ends the request). Cancel and Error serve it as they serve a generation. Ready's vision byte becomes a features byte: FeatureVision as before, and FeatureVideo for an engine that serves a video model. A language model's engine refuses a VideoRequest as a request error and keeps serving. Protocol version 12; the golden vectors gain the two video requests and the three events, and both codecs and test peers read and write them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`serve-native MODEL_DIRECTORY` of an assembly whose record names an h3-video model runs the video engine (runtime/video/VideoRuntime.hpp) in place of the language model's: it reads the protocol on stdin, announces FeatureVideo, takes VideoRequests one at a time and Cancels, and runs every phase of a request in this process on the GPU. The prompt's tokens go through the text encoder, then the vendored engine's C ABI prepares the request, denoises with the DiT streamed from SSD (8 slots), decodes the TAEH3 preview when asked, the video and, when asked, the audio, and writes the mp4. Each phase reports its progress, the denoising each step; the preview and the clip arrive as VideoOutputs. A Cancel stops the request at the next step or phase boundary: the step callback throws through h3_denoise once the step's commands have completed. A device or command buffer failure ends the process, as the language model's engine does. The text encoder is Qwen3-VL-32B's language model through its first 50 decoder layers (hidden_states[50]), from the assembly's MLX checkpoint. Its weights load as the engine's block images of the checkpoint's own affine quantization, for the request alone, and run the engine's token gather, RMS norms and prefill linears; its Q/K norms, rotation and causal GQA attention are two kernels of the video group. On the golden prompt its hidden states are 0.44% relative RMS from MLX's (cosine 0.9999996), in 4.6 s with the 14 GB of weights loaded. Two engine options serve the identity tests: --hidden-states conditions every request on saved hidden states, and --keep writes each request's hidden states, final latents and frames. The engine's own log lines go to stderr. New Metal tests: the encoder's attention against a double CPU reference, and one DiT block's forward on synthetic weights, identical to h3's references. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A video model's assembly is a new target format, h3-video, beside mlx-affine and gguf, on the same chain: inspection, publish, and the immutable assembly the server holds. install/video.py holds everything of the kind. upstream.prepare hands a target of MiniMax-H3's layout (its FL2VA pipeline), or an installation of the kind, to video.prepare, and prepare_local_model a directory of it to video.prepare_local; the assembly's record names the target and the text encoder as its sources, and adds `video`: the license acknowledged and the images' origin. From the Hub, the target and its companion sources, OpenVDN's VDN-H3 stages, TAEH3 and Qwen3-VL-32B's MLX checkpoint, follow their default branches as a draft does. Before any weight download the installer quotes the MiniMax H3 Community License Agreement's territory clauses and asks for its acknowledgement, on the terminal or by SPLASH_ACCEPT_LICENSE, once per license text, recorded in the models root. The vendored converters then write the engine's images once into .converted/<key>, keyed by their sources and the commits converted, outside the installation lock; the images stay held until an assembly links them, and are collected, as GGUF metadata is, once none does. The video VAE's converter needs MLX and the MiniMax-H3 MLX port, which the installer asks for before downloading. From local files, a directory of images converted before prepares without converting, as h3 lays them out; the repository's layout converts with the companions the Hugging Face cache holds. Neither asks the Hub. The real converted images on an M5 Pro prepare in 34.7 s the first time (their digests) and 0.01 s again. assembly.py gains the kind's record keys and sources and a second derived root; packaging ships the module and the converters. Tests: the Hub install, a second start, the license refusal and record, refused options, missing sources, both local layouts and collection of held images, on fake sources with the converters stood in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`python -m server.server` of an h3-video assembly, which `splash serve
--model MiniMaxAI/MiniMax-H3` starts as it starts a language model's,
serves the video API alone (server/videos.py): POST /v1/videos, GET
/v1/videos, GET /v1/videos/{id}, GET /v1/videos/{id}/content and DELETE
/v1/videos/{id}, in OpenAI's shape, beside /health, /ready, /status and
/v1/models. A language model's routes answer 404, naming the video model:
chat and video are two instances. POST takes JSON or the multipart form
OpenAI's SDK sends; besides prompt, model, seconds (3, 10 or 15) and size
(1344x768), it takes Splash's variant, seed, steps, preview and audio.
The server tokenizes the prompt with the assembly's tokenizer and runs one
job at a time on the video engine, the others queued (four at most); a
job's progress follows the engine's phases and its denoising steps, and its
object records each phase's time and the engine's peak footprint. Finished
jobs' mp4s and previews stay in a store of 4 GB, the oldest evicted first.
DELETE cancels a job, a running one at the engine's next step. An engine
that fails is started again for the next job.
FrontendHandler's body reader splits from its JSON parsing, so that the
video routes read a form with it. Tests drive the API, through OpenAI's
SDK too, over a stand-in engine: a job's lifecycle and downloads, refusals
in the error shape, a language model's routes absent, a deleted running
job stopped at a step while the next runs, and the bounded queue and store.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
make test-video-real drives the engine process of a prepared MiniMax-H3 over the native protocol with the golden prompt's saved hidden states: the 3 s golden clip's final latents and video frames must equal the h3 engine's references byte for byte, with its preview and mp4 delivered; then a clip cancelled at its second denoising step ends as cancelled, and the engine serves on. make test-video-http-real serves the model as `splash serve` does and runs the golden clip through POST /v1/videos with the saved hidden states, again byte-identical, its preview and mp4 downloadable and a language model's routes absent; then a job deleted while it denoises, which must stop at the next step; then the clip of the server's tokenizer and the engine's text encoder, reported against the golden hidden states, latents (relative RMS) and frames (PSNR); with VIDEO_SECONDS=15 also a 15 s clip end to end. Each report gives every phase's time and the peak memory. VIDEO_REFERENCES names the references; the engine's --hidden-states and --keep options serve these tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README gains a Video section: the command, a request, and a 3 s and a 15 s clip's times on an M5 Pro. DEVELOPMENT.md gains a Video chapter: the sibling instance, the video API, the h3-video installation and its license acknowledgement, the engine process and its phases with their measured times and memory, and the tests; Native protocol names the Video frames, and Local model files a MiniMax-H3 directory. runtime/video/docs holds the design figure: the two serve modes, the shared model layer and the video pipeline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zaiyang-zhang
force-pushed
the
h3-video
branch
from
October 21, 2026 01:04
25f6725 to
13dbe91
Compare
zaiyang-zhang
marked this pull request as ready for review
October 21, 2026 01:04
splash generate --model MiniMaxAI/MiniMax-H3 --prompt ... --out clip.mp4 [--seconds 3|10|15] [--seed N] [--variant turbo|default] [--preview] [--no-audio] installs the model as splash serve does (the same installer run, license acknowledgement and SPLASH_ACCEPT_LICENSE), holding the installation lock shared, then executes python -m server.videos on the selection link: no HTTP server starts. That program holds the assembly, refuses a language model, starts the video engine as the server does and runs one clip on it, its flags turned into a POST /v1/videos body and checked by the route's own parse_request. Progress goes to stderr, phase by phase and step by step; --preview writes <out stem>.preview.mp4 as soon as it is decoded; the mp4 goes to --out. The status is 0 once it is written, 1 on a refusal or the engine's error, and 128 + the signal on Ctrl+C or SIGTERM, which cancel the clip at its next step. The server's job runner and generate share the prompt tokenization, the VideoRequest frame and the engine and tokenizer loading; the launcher's serve and generate share the installation lock and the exec. Tests drive generate over the stand-in engine, which gains a failing mode: the clip and its preview written, a language model and an invalid request refused, an engine error and a stop each failing it; and the launcher's command for it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zaiyang-zhang
marked this pull request as draft
October 21, 2026 18:39
The same behaviours in 1,028 added test lines instead of 1,808: one test per behaviour on shared fixtures. - install/test_video.py: one table of fake sources; the Hub install and its restart, the license, local images and the repository layout, and collection. - server/test_videos.py: a job through OpenAI's SDK and one refusal; splash generate's clip, its refusal of a language model and its stop. The stand-in engine only denoises and answers. - Protocol: the video golden vectors stay; round trips that the golden and stream tests already make are gone, and rejections keep one case per check. - video_real.py drives the engine through server.videos.VideoEngine and compares the kept latents with cmp against h3's references, as h3's smoke_golden.sh does; VIDEO_SECONDS sets the length of the clip of the server's own encoder, and VIDEO_REFERENCES defaults to H3_REFS. - The Metal tests use the shared bf16, buffer and check helpers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ComputeDispatch::gridThreads runs exactly that many threads in each dimension, Metal trimming the threadgroups at the grid's far edges (dispatchThreads), for kernels that index the grid without a bound of their own, as the video engine's do. It does not combine with an indirect grid. metal_backend_test checks the trimmed threadgroup and the refusal. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The h3 engine came with its own copy of what Splash already has; its host code in runtime/video/h3 now runs on Splash's instead, kernel arithmetic unchanged: - Metal: h3 metal/MetalBackend (device, queue, metallib, pipeline cache, buffers, command buffers) is gone. The engine allocates through Splash's MetalBackend, and engine/Encoder.hpp records a command's dispatches as the engine's kernels bind them (each argument at its own index, bound arguments kept from one dispatch to the next). One command runs a DiT block, a VAE clip or a preview chunk, the next encoded while it runs; the streamed slots of a block go back to the reader once its command completed. The register-layout check moves to ops/WindowAttention. - Readers: h3 model/Safetensors and model/Json are gone. Each converted directory (dit-<variant>/, fl2va/transformer/) is one SafetensorsCheckpoint, so the DiT manifest's and the FL2VA index's file maps are not read; blobs and the stream image are WeightSources read in slices on parallelFor; the decoders' manifests go through NSJSONSerialization (model/WeightFiles.hpp). - Errors: h3 engine/Status (codes, H3_TRY, Logger, H3_LOG) is gone. The engine throws as Splash does; CApi.mm returns each exception class as its h3_status (engine/Errors.hpp), and log lines go to stderr through logLine, so VideoRuntime no longer redirects stdout. - Clocks: AwakeClock, and the architecture check's exemption for the video engine's clocks is gone. - Removed with the h3 tools that used them: the VAE stage profile, the prep's intermediate dump, the per-step-class forward statistics (0 in h3_forward_stats) and the numpy readers nothing calls. The C ABI (h3.h) is unchanged but for comments. 7,400 host lines become 6,392. The synthetic forward still hashes to h3's references, and the golden 3 s clip's latents and frames stay byte-identical through the engine and through /v1/videos; a 15 s turbo clip denoises in 219.6 s per step (1,756.7 s for 8). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
runtime/video/VENDORED records the one-time import from h3 5b55ab7 and how the copy has diverged: the kernels, their ABI headers and the weight converters stay as h3 has them, while the host code is Splash's, on its Metal backend, weight readers and errors. vendor_video.sh copies those verbatim parts again from an h3 checkout and prints h3's host-code changes since the recorded commit, to port by hand. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
test_videos checks the bounds of the jobs' queue (503 video_queue_full) and of the store (the newest finished clip kept, the rest evicted), and that splash generate exits 1 with the engine's error. The stand-in engine takes a `fail` argument for the latter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A VideoRequest may carry keyframes: a u8 of VideoKeyframeFlag bits after the seed, and after the prompt tokens the first frame's and the last frame's pixels, uint8 RGB height x width x 3 each, already on the clip's canvas. Protocol.cpp checks each against the canvas (the size overflow checked) and server/protocol.py serializes them from VideoRequestFrame's first_frame and last_frame, empty by default. Protocol version 13; the golden vectors move to it and add a 4 x 2 clip with both frames. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
h3 e6deb5e adds FL2VA keyframe conditioning (ABI 3). vendor_video.sh copied its kernels and ABI headers as they are: the video VAE encoder (kernels/vae/encoder.metal, VaeEncoderABI.h), the keyframe prep kernels (prep_keyframe_latents, prep_cond_noise), the conditioning rows of dit_io_embed and the tags of Layout.h. The host changes are ported by hand onto Splash's copy, on its Metal backend, weight readers and exceptions: - ops/VaeEncoder: the encoder on 256 px tiles, a batch per command, weights through WeightSource and the manifest through NSJSONSerialization. - model/Prep: keyframe layout (rows after the text at the first or last latent frame's time), the conditioning timestep max(t, 0.999), the keyframe noise draws, and a fourth adaLN slot per block. - engine: the conditioning rows in the layout, memory plan, key-range table and DiT inputs; they stay fixed while the target rows step. - h3.h: ABI 3 (h3_prep_inputs first_frame / last_frame / text_tags, h3_prepared and h3_dit_inputs cond_*, h3_window_table cond_rows). The encoder's debug stop, the prep's moments input and model/Keyframe stay h3's: the server puts keyframes on the canvas with Pillow, which Keyframe reproduces. VideoRuntime passes a request's keyframes to h3_prep and its conditioning rows to h3_dit_load. Text-to-video is unchanged: the golden clip's latents and frames stay byte-identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
h3's convert_vae.py (vendored with e6deb5e) converts the encoder with --encoder: vae/vae_enc_f16.bin and manifest_enc.json, from FL2VA's video_vae/source model and config.json, which the installer now downloads. The assembly links them beside the W8 decoder. The converters' identity changes with the converter, so a new entry is keyed. An entry the previous converters made from the same commits holds every image but the encoder's: it is cloned into the new entry (APFS copy on write, so its collection leaves the clones as they are) and only the encoder is converted, instead of the 100 GB conversion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
POST /v1/videos takes OpenAI's input_reference, a PNG or JPEG file of the multipart form, as the clip's first frame, and Splash's last_frame, another, as its last. images.keyframe decodes each with Pillow and puts it on the 1344x768 canvas as MiniMax-H3's MLX port does (first frame stretched, last frame scaled to cover and centre-cropped, LANCZOS); the job's request carries the pixels to the engine and drops them once it finishes. splash generate takes --first-frame and --last-frame. Tests: the stand-in engine logs each keyframe's digest, and the server tests send both through OpenAI's SDK and a multipart form and check the refusals. make test-video-first-frame-real runs the 3 s golden clip from golden frame 0 through /v1/videos: its latents are byte-identical to the h3 engine's of the same inputs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
zaiyang-zhang
marked this pull request as ready for review
October 22, 2026 00:46
zaiyang-zhang
marked this pull request as draft
October 22, 2026 20:01
The weight stream read each block's int8 matrices from stream.bin, a 22 GB copy per variant of what the block files already hold, and compared the two at load. It now reads them from the block files DitWeights keeps open, in the same ring of slots on the same reader thread, through the same uncached sliced reads; h3_request loses stream_image (ABI 4), and the installer no longer writes the image. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Each request read the 50 blocks' adaLN bases, 26 GB of bf16, from FL2VA's transformer shards, so the 62 GB of shards had to stay installed. The tables depend only on the variant's schedule: h3_tabulate_adaln (splash video-adaln WEIGHTS VARIANT TRANSFORMER) runs the same computation once, for the schedule without keyframes and with them, into dit-<variant>/adaln.safetensors (0.3 GB turbo, 1.9 GB default), and h3_prep gathers its steps from it (0.1 / 1.0 GB read). A request of another step count than the variant's is refused. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A video installation converted both DiT variants and linked FL2VA's transformer shards into the assembly for the request preparation: about 100 GB of images beside the 62 GB of shards, which had to stay. It now converts the turbo variant (26 GB with the VAEs and TAEH3), and the default one once asked for, by --variants of install/models.py or by the first `splash generate --variant default` (24 GB more); an entry starts from a clone of the same commits' entry converted before and converts what it lacks, the adaLN tables (splash video-adaln) included. The assembly no longer links the shards, and an installation whose images are converted downloads neither them nor the other conversion sources again, so they can be deleted from the Hugging Face cache, which the installer leaves to the user as it has no step that removes a source. The server refuses a variant its assembly does not hold, and steps other than the variant's. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The engine process drove Splash's copy of the h3 engine through its C ABI
(h3.h, CApi.mm), though both sides are C++ in one process. The VideoRuntime
now composes the request's phases as SGLang Diffusion composes a pipeline:
stages run in order on a batch (SGLang's Req), each reading what the ones
before it left: TextEncodingStage, PrepStage, DenoisingStage (h3's
scheduler and denoising loop), PreviewStage, DecodingStage,
AudioDecodingStage and MuxStage (h3's mp4 writer). The phases' progress,
outputs and cancellation are the stages' events, as before.
The subtree drops its h3 nesting and keeps Splash's layering under the
stages: runtime/video/{pipeline,stages,model,ops,metal}. The operators take
their weights as buffers or as weight views of their own, which model/
loads; the shapes they take are ops/Geometry; the dispatch recorder and the
M5 Pro tuning are the subtree's metal/ layer. check_architecture.py checks
the subtree's layers as it checks Splash's. One Metal backend serves every
stage of the process, and failures are exceptions, as Splash fails.
What only development uses leaves the shipping tree: the synthetic DiT
weights and their fill kernel are the synthetic test's
(dev/tests/engine/VideoSynthetic.*, video_synthetic.metal, linked into a
test metallib), and the golden-clip tests run their own engine
(dev/tools/video_golden.mm), the model's runtime with the text encoding
replaced by saved hidden states and the latents and frames kept, with the
.npz reader and writer (dev/tools/Numpy.*). The engine process takes no
--hidden-states or --keep. h3's prep-directory dump goes with the ABI.
Byte identity: make test-video-real keeps the 3 s golden clip's latents
and frames byte for byte; the synthetic forward keeps h3's hashes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The int8 linears of MiniMax-H3's DiT and video VAE had their format only in h3's converters and in constants spread over the readers and kernels. QuantFormat.h now defines it beside the formats of the MDGG0001 image: QUANT_VIDEO_W8, "h3-w8a8-vdn-v1", dense row-major int8 [N][K] with one fp32 scale per output row (absmax / 127, rounded half to even) after a Hadamard rotation of the inputs in groups of 128 with splitmix64 signs of seed 0x48335138. It is no kQuantFormats row: that table describes tiled planes that every LLM GEMM decodes, where the video GEMMs read the dense rows as an MPP tensor. The kernels' common/quant.h and model/Hadamard.hpp take their constants from it, and the DiT's and the VAE decoder's readers refuse a manifest that names another format or seed. Byte identity: make test-video-real, test-video-http-real and test-video-first-frame-real keep the golden clips byte for byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An installation of MiniMax-H3 downloaded MiniMax's bf16 release, OpenVDN's stages and TAEH3 and converted them on the user's Mac with h3's converters, MLX and the minimax-h3-mlx port. Splash now serves it as it serves a GGUF or MLX repository: from a Hugging Face repository in the engine's own format, whose files it downloads and links as they are. The repository holds the images the assembly linked (dit-turbo/, optional dit-default/, vae/, audio/, preview/), FL2VA's tokenizer, the license, its NOTICE and a model card, and splash-video.json, the marker by which the installer recognizes it: format h3-video, version 1, weights h3-w8a8-vdn-v1 (QUANT_VIDEO_W8), the variants and the sources' commits. The text encoder still comes from its own repository, paired as a draft is, and the license is acknowledged before any weight download. A start without the Hub starts the installation from its pinned snapshots. --model MiniMaxAI/MiniMax-H3 is refused with the way to the engine's format; install/video.py REPOSITORY names the one the documentation, help and tests use, a placeholder. The converters leave the shipped tree for dev/tools/video, and dev/tools/publish_video_repo.py makes the repository: from images converted before, or converted from the release (--convert), as clones of the images but for their manifests, written without the converting machine's paths; dev/tools/vendor_video.sh copies the converters there and keeps the kernels' common/quant.h, Splash's now. prepare_local_model links a directory of the repository's layout, or of h3's converted images, without converting. Conversions an earlier release wrote into .converted/ are collected once nothing links them. Tests: make check; the three golden tests on ~/exp/h3-flat-style images and make test-video-real on an installation of a repository the tool assembled, byte for byte. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on Version 1.4.0
References (details and licenses at the bottom): MiniMax-H3, VDN-H3 (OpenVDN), FastH3, SageAttention 1/2, QuaRot, SGLang Diffusion.
Quantization and sparse attention strategy
turbo= 8-step distilled,default= 50-stepAcceptance: same model in bf16/fp32 as reference, per block against float64; DiT int8 vs bf16 26.9 dB, VAE 57.5 dB.
Tuned for clips ≥ 10 s.
Performance (MacBook Pro M5 Pro 48 GB, macOS 27; 1344×768, 24 fps, with audio)
Disk after install: 26 GB of converted weights (turbo) plus the text encoder; the base model's 62 GB of shards are only needed to convert.
Other MiniMax-H3 implementations on Apple Silicon, as reported by their authors (FreeVideo, FastVideo):
Demo (OpenVDN project-page sample 02, same prompt, different seed): side by side, left OpenVDN reference, right ours · ours, 15 s with audio.
Architecture and UX
UX
Run a video server
First start downloads the sources (after the license acknowledgement), converts once, and serves
/v1/videos. It is a video-only instance; run a secondsplash servefor chat.Generate from a client (OpenAI video API; the OpenAI SDK works)
POST /v1/videoswithprompt,seconds(3 | 10 | 15),seed,variant(turbo|default),preview,audio→ a jobGET /v1/videos/{id}for progress, per phase and per denoising stepGET /v1/videos/{id}/contentfor the mp4;?variant=previewfor a fast preview as soon as denoising endsDELETE /v1/videos/{id}to cancelStart from an image: send
input_reference(PNG/JPEG) to pin the first frame; addlast_frameto end on another image.One clip, no server
Local weights:
prepare_local_modeltakes an H3 source layout or already-converted images, like a GGUF or MLX model.Output: 1344×768, 24 fps, H.264 + AAC stereo, audio generated with the video.
Architecture
Principles:
h3-videomodel kind sits on the inspect → publish → assembly chain), quantization formats, op and dispatch conventions, memory primitives.runtime/video/, kernels inruntime/metal/kernels/video.Video.Detail to be fine-grained.
Limits
Next steps (not in this PR)
References and licenses
Splash ships code only; no weights are in the tree. Full texts in
THIRD_PARTY_NOTICES.🤖 Generated with Claude Code