-
-
Notifications
You must be signed in to change notification settings - Fork 57
feat: add ffmpeg encoding profiles for 'fast' export #48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
4745884
857af4a
d3d3cfb
7bb795a
c980a5a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,6 +24,8 @@ Every Claude Code session then has these tools: | |
| - `fablecut_import_media` — copy a local file into `./media/` and register it. | ||
| - `fablecut_analyze_reference` — turn a reference video into an edit blueprint | ||
| (shots, beats, BPM, energy, drop) + extract its music. See "Remake a reference video". | ||
| - `fablecut_encode_profiles` — list export presets from `encoding-profiles.json` (each is a | ||
| raw ffmpeg args list). Set `project.encodeProfile` via patch to pin a project default. | ||
|
|
||
| ### Token-efficient editing (important for agents) | ||
|
|
||
|
|
@@ -167,6 +169,7 @@ Examples in `library/svg/`: `sparkles.svg` (loop), `lower-third.svg`, | |
| ], | ||
| "disabledTracks": [ "A2" ], | ||
| // ^ optional — track ids (V4 V3 V2 V1 A1 A2 A3) omitted from preview/export when listed | ||
| "encodeProfile": "hq", // optional — fast-export profile id (see encoding-profiles.json) | ||
| "media": [ | ||
| { "id": "m_abc", "name": "intro.mp4", "kind": "video", // video|audio|image|svg | ||
| "src": "/media/intro.mp4", // path under ./media or ./library | ||
|
|
@@ -412,8 +415,12 @@ obvious cuts were missed, raise it if motion is being misread as cuts. | |
| its music into ./media. `GET /api/analyze?src=…` returns the cached blueprint. | ||
| - `GET /api/events` — SSE, emits `change` when project.json, ./media or ./library changes | ||
| - Fast export (used by the UI; browser renders frames, ffmpeg encodes): | ||
| `GET /api/export/ffmpeg` → `{available}` · `POST /api/export/begin` `{fps,name}` → `{id}` | ||
| · `POST /api/export/frame?id=` (JPEG body, in order) · `POST /api/export/audio?id=` (WAV body) | ||
| `GET /api/export/ffmpeg` → `{available}` · `GET /api/export/profiles[?detail=1]` → | ||
| `{default, profiles, issues}` · `POST /api/export/begin` `{fps,name,profile?,hasAudio?}` → | ||
| `{id,profile,label,summary}` (**400** if `profile` is not a defined id, or if ffmpeg | ||
| rejects its args in the dry run) | ||
| · `POST /api/export/frame?id=` (JPEG body, in order) · `POST /api/export/audio?id=` (WAV | ||
| body — must be sent before the first frame; ffmpeg is spawned on frame 1) | ||
| · `POST /api/export/end?id=[&discard=1]` → `{src}` under `/exports/` | ||
|
|
||
| ## Recipes | ||
|
|
@@ -527,8 +534,69 @@ guides (▦) to keep captions out of platform UI zones. | |
|
|
||
| Export is user-driven (Export button → dialog). Two engines: **Fast** (browser | ||
| renders each frame with the normal compositor — including SVG frames, keys and | ||
| AI masks — streams JPEG frames + an offline WAV mix to the server, ffmpeg | ||
| encodes a CRF-18 faststart MP4 into `./exports/`) and **Realtime** | ||
| AI masks — streams JPEG frames + an offline WAV mix to the server, a single ffmpeg | ||
| pass encodes them via an **encoding profile** into `./exports/`) and **Realtime** | ||
| (MediaRecorder fallback). Claude cannot trigger export headlessly — the | ||
| compositor lives in the browser; ask the user to click Export, or render with | ||
| ffmpeg directly from `media/` sources if a file is needed. | ||
|
|
||
| ### Encoding profiles (`encoding-profiles.json`) | ||
|
|
||
| User-editable at the repo root. A profile is a **raw ffmpeg argument list** plus the | ||
| two things that are not ffmpeg arguments: `jpegQuality` (the browser's frame quality) | ||
| and `extension` (which names the file and therefore picks the muxer). Edit the file | ||
| while the server runs — the UI hot-reloads the profile list via SSE. | ||
|
|
||
| ```jsonc | ||
| { | ||
| "default": "delivery", // profile id used when nothing else is set | ||
| "profiles": { | ||
| "draft": { | ||
| "label": "Draft · H.264 fast", | ||
| "description": "Quick preview — smaller file, faster encode.", | ||
| "jpegQuality": 0.85, // browser JPEG frame quality (0.1–1) | ||
| "extension": ".mp4", | ||
| "args": ["-c:v", "libx264", "-preset", "veryfast", "-crf", "23", | ||
| "-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "128k", | ||
| "-movflags", "+faststart", "-shortest"] | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Export is **one ffmpeg pass**. The server owns the input side and the output path; | ||
| `args` is everything in between, verbatim: | ||
|
|
||
| ``` | ||
| ffmpeg -y -f image2pipe -framerate <fps> -i - [-i audio.wav] <args…> exports/<name><extension> | ||
| ``` | ||
|
|
||
| - **There is no allow-list.** Any codec, filter, container or flag your local ffmpeg | ||
| supports works — ProRes, DNxHD, NVENC/QSV/VideoToolbox, VP9/AV1, 10-bit, HDR tags. | ||
| See the shipped `prores422` and `broadcast1080i50` profiles. | ||
| - **Args are validated by ffmpeg itself**, not by a schema: when an export starts the | ||
| server dry-runs the profile against a 0.1 s synthetic input (`lavfi`). A typo or an | ||
| encoder your build lacks is rejected up front with ffmpeg's own message, instead of | ||
| failing after every frame has been rendered. | ||
| - Use the **array form** — each element is passed to `spawn` untouched, so no quoting | ||
| is needed (`["-vf", "drawtext=text='hi there'"]` just works). A plain string is | ||
| accepted and split on whitespace. | ||
| - Nothing is injected for you: `+faststart`, `-shortest`, `-strict -2` for Opus in MP4 | ||
| and pixel-format choices are all yours to write. | ||
| - Frames arrive as **JPEG (4:2:0)**, so `yuv422p`/`yuv444p` cannot recover chroma the | ||
| source never had; raise `jpegQuality` before reaching for a wider pixel format. | ||
| - The audio mix is only present when the timeline has audio; with no audio there is a | ||
| single input, so avoid hardcoded `-map 1:a`. | ||
|
|
||
| **Note:** Fast export pipes **composited JPEG frames** from the browser (`-f image2pipe`), | ||
| not `-i source.mp4`. Filters and codec settings apply to that frame stream. To transcode | ||
| an existing file verbatim, run ffmpeg directly — that is outside the compositor path. | ||
|
|
||
| **Which profile is used (priority):** | ||
| 1. Profile picked in the Export dialog (one-off; saved to browser settings unless overridden) | ||
| 2. `project.encodeProfile` — set via UI reload or `{op:"setProject", set:{encodeProfile:"hq"}}` | ||
| 3. Browser setting `encodeProfile` in localStorage (set when you change the Export dropdown) | ||
| 4. `default` in `encoding-profiles.json` | ||
|
|
||
| **MCP:** `fablecut_encode_profiles` lists profiles; `{detail:true}` includes each `args` | ||
| array; `{profile:"hq"}` returns one profile. `fablecut_status` shows the effective profile. | ||
|
Comment on lines
+601
to
+602
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Do not describe The status implementation only reads 🤖 Prompt for AI Agents |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Label the command fence as shell.
The fence violates MD040. Use
```shfor the ffmpeg command example.🧰 Tools
🪛 markdownlint-cli2 (0.23.1)
[warning] 570-570: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Source: Linters/SAST tools