Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 72 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ Every Claude Code session then has these tools:
- `fablecut_import_media` — copy a local file into `./media/` and register it.
- `fablecut_analyze_reference` — turn a reference video into an edit blueprint
(shots, beats, BPM, energy, drop) + extract its music. See "Remake a reference video".
- `fablecut_encode_profiles` — list export presets from `encoding-profiles.json` (each is a
raw ffmpeg args list). Set `project.encodeProfile` via patch to pin a project default.

### Token-efficient editing (important for agents)

Expand Down Expand Up @@ -167,6 +169,7 @@ Examples in `library/svg/`: `sparkles.svg` (loop), `lower-third.svg`,
],
"disabledTracks": [ "A2" ],
// ^ optional — track ids (V4 V3 V2 V1 A1 A2 A3) omitted from preview/export when listed
"encodeProfile": "hq", // optional — fast-export profile id (see encoding-profiles.json)
"media": [
{ "id": "m_abc", "name": "intro.mp4", "kind": "video", // video|audio|image|svg
"src": "/media/intro.mp4", // path under ./media or ./library
Expand Down Expand Up @@ -412,8 +415,12 @@ obvious cuts were missed, raise it if motion is being misread as cuts.
its music into ./media. `GET /api/analyze?src=…` returns the cached blueprint.
- `GET /api/events` — SSE, emits `change` when project.json, ./media or ./library changes
- Fast export (used by the UI; browser renders frames, ffmpeg encodes):
`GET /api/export/ffmpeg` → `{available}` · `POST /api/export/begin` `{fps,name}` → `{id}`
· `POST /api/export/frame?id=` (JPEG body, in order) · `POST /api/export/audio?id=` (WAV body)
`GET /api/export/ffmpeg` → `{available}` · `GET /api/export/profiles[?detail=1]` →
`{default, profiles, issues}` · `POST /api/export/begin` `{fps,name,profile?,hasAudio?}` →
`{id,profile,label,summary}` (**400** if `profile` is not a defined id, or if ffmpeg
rejects its args in the dry run)
· `POST /api/export/frame?id=` (JPEG body, in order) · `POST /api/export/audio?id=` (WAV
body — must be sent before the first frame; ffmpeg is spawned on frame 1)
· `POST /api/export/end?id=[&discard=1]` → `{src}` under `/exports/`

## Recipes
Expand Down Expand Up @@ -527,8 +534,69 @@ guides (▦) to keep captions out of platform UI zones.

Export is user-driven (Export button → dialog). Two engines: **Fast** (browser
renders each frame with the normal compositor — including SVG frames, keys and
AI masks — streams JPEG frames + an offline WAV mix to the server, ffmpeg
encodes a CRF-18 faststart MP4 into `./exports/`) and **Realtime**
AI masks — streams JPEG frames + an offline WAV mix to the server, a single ffmpeg
pass encodes them via an **encoding profile** into `./exports/`) and **Realtime**
(MediaRecorder fallback). Claude cannot trigger export headlessly — the
compositor lives in the browser; ask the user to click Export, or render with
ffmpeg directly from `media/` sources if a file is needed.

### Encoding profiles (`encoding-profiles.json`)

User-editable at the repo root. A profile is a **raw ffmpeg argument list** plus the
two things that are not ffmpeg arguments: `jpegQuality` (the browser's frame quality)
and `extension` (which names the file and therefore picks the muxer). Edit the file
while the server runs — the UI hot-reloads the profile list via SSE.

```jsonc
{
"default": "delivery", // profile id used when nothing else is set
"profiles": {
"draft": {
"label": "Draft · H.264 fast",
"description": "Quick preview — smaller file, faster encode.",
"jpegQuality": 0.85, // browser JPEG frame quality (0.1–1)
"extension": ".mp4",
"args": ["-c:v", "libx264", "-preset", "veryfast", "-crf", "23",
"-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "128k",
"-movflags", "+faststart", "-shortest"]
}
}
}
```

Export is **one ffmpeg pass**. The server owns the input side and the output path;
`args` is everything in between, verbatim:

```
ffmpeg -y -f image2pipe -framerate <fps> -i - [-i audio.wav] <args…> exports/<name><extension>
```
Comment on lines +570 to +572

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Label the command fence as shell.

The fence violates MD040. Use ```sh for the ffmpeg command example.

🧰 Tools
🪛 markdownlint-cli2 (0.23.1)

[warning] 570-570: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CLAUDE.md` around lines 570 - 572, Label the Markdown code fence containing
the ffmpeg command as shell by changing the opening fence to ```sh, while
leaving the command content unchanged.

Source: Linters/SAST tools


- **There is no allow-list.** Any codec, filter, container or flag your local ffmpeg
supports works — ProRes, DNxHD, NVENC/QSV/VideoToolbox, VP9/AV1, 10-bit, HDR tags.
See the shipped `prores422` and `broadcast1080i50` profiles.
- **Args are validated by ffmpeg itself**, not by a schema: when an export starts the
server dry-runs the profile against a 0.1 s synthetic input (`lavfi`). A typo or an
encoder your build lacks is rejected up front with ffmpeg's own message, instead of
failing after every frame has been rendered.
- Use the **array form** — each element is passed to `spawn` untouched, so no quoting
is needed (`["-vf", "drawtext=text='hi there'"]` just works). A plain string is
accepted and split on whitespace.
- Nothing is injected for you: `+faststart`, `-shortest`, `-strict -2` for Opus in MP4
and pixel-format choices are all yours to write.
- Frames arrive as **JPEG (4:2:0)**, so `yuv422p`/`yuv444p` cannot recover chroma the
source never had; raise `jpegQuality` before reaching for a wider pixel format.
- The audio mix is only present when the timeline has audio; with no audio there is a
single input, so avoid hardcoded `-map 1:a`.

**Note:** Fast export pipes **composited JPEG frames** from the browser (`-f image2pipe`),
not `-i source.mp4`. Filters and codec settings apply to that frame stream. To transcode
an existing file verbatim, run ffmpeg directly — that is outside the compositor path.

**Which profile is used (priority):**
1. Profile picked in the Export dialog (one-off; saved to browser settings unless overridden)
2. `project.encodeProfile` — set via UI reload or `{op:"setProject", set:{encodeProfile:"hq"}}`
3. Browser setting `encodeProfile` in localStorage (set when you change the Export dropdown)
4. `default` in `encoding-profiles.json`

**MCP:** `fablecut_encode_profiles` lists profiles; `{detail:true}` includes each `args`
array; `{profile:"hq"}` returns one profile. `fablecut_status` shows the effective profile.
Comment on lines +601 to +602

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Do not describe fablecut_status as showing the effective profile.

The status implementation only reads project.encodeProfile or the file default; it cannot see the higher-priority browser dialog or localStorage choice documented at lines 596-599. Describe it as the MCP-visible project/default profile instead.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CLAUDE.md` around lines 601 - 602, Update the MCP documentation entry for
fablecut_status to describe it as showing the MCP-visible project/default
profile, not the effective profile. Keep the fablecut_encode_profiles
descriptions unchanged.

134 changes: 124 additions & 10 deletions app.js
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,7 @@ function addLiveAudioTrackBuses(ids) {
const SETTINGS_KEY = "fablecut-settings";
const DEFAULT_SETTINGS = {
linkSelect: false, // timeline ↔ project bin selection sync
encodeProfile: null, // null = server default from encoding-profiles.json
};
let settings = { ...DEFAULT_SETTINGS };
function loadSettings() {
Expand Down Expand Up @@ -235,6 +236,7 @@ const project = {
inPoint: null, // timeline work-area IN (seconds), or null
outPoint: null, // timeline work-area OUT (seconds), or null
disabledTracks: [], // track ids (V4…A3) hidden from preview/export when listed
encodeProfile: null, // optional fast-export profile id (overrides browser setting)
};
const state = {
time: 0, playing: false, pps: 60, snap: true,
Expand Down Expand Up @@ -325,6 +327,9 @@ const els = {
monitorStage: $("monitorStage"), monitorScroll: $("monitorScroll"),
monitorZoomInner: $("monitorZoomInner"), kfGraphs: $("kfGraphs"),
exportSetup: $("exportSetup"), engineFast: $("engineFast"), engineRealtime: $("engineRealtime"),
exportProfileRow: $("exportProfileRow"),
exportProfileSel: $("exportProfileSel"), exportProfileNote: $("exportProfileNote"),
exportProfileHint: $("exportProfileHint"),
};
const ctx2d = els.preview.getContext("2d");

Expand Down Expand Up @@ -436,6 +441,7 @@ async function connectServer() {
listenSSE();
fetch("/api/export/ffmpeg").then((r) => r.json())
.then((j) => { state.ffmpeg = !!j.available; }).catch(() => { });
fetchEncodeProfiles();
} catch {
state.connected = false;
els.projectName.textContent = project.name + " · ⚪ local session";
Expand Down Expand Up @@ -583,6 +589,7 @@ function applyProject(data) {
inPoint: wa.inPoint,
outPoint: wa.outPoint,
disabledTracks,
encodeProfile: data.encodeProfile || null,
});
const folderIds = new Set(project.folders.map((f) => f.id));
for (const m of project.media) {
Expand Down Expand Up @@ -640,24 +647,26 @@ function scheduleSave() {
}, 400);
}
function projectJSON() {
const { name, width, height, fps, background, revision, folders, media, clips, markers, inPoint, outPoint, disabledTracks } = project;
return {
const { name, width, height, fps, background, revision, folders, media, clips, markers, inPoint, outPoint, disabledTracks, encodeProfile } = project;
const out = {
name, width, height, fps, background, revision,
folders: (folders || []).map(({ id, name, parentId, open }) =>
({ id, name, parentId: parentId || null, open: open !== false })),
media: media.filter((m) => !m.transient).map(({ id, name, kind, src, duration, width, height, folderId }) =>
({ id, name, kind, src, duration, width, height, folderId: folderId || null })),
clips: clips.map(({ id, mediaId, kind, track, start, in: inn, duration, name, props, keyframes, transitionIn, transitionOut, linkedId, linkGroup }) => {
const out = { id, mediaId, kind, track, start, in: inn, duration, name, props, keyframes, transitionIn, transitionOut };
if (linkGroup) out.linkGroup = linkGroup;
if (linkedId) out.linkedId = linkedId;
return out;
const clipOut = { id, mediaId, kind, track, start, in: inn, duration, name, props, keyframes, transitionIn, transitionOut };
if (linkGroup) clipOut.linkGroup = linkGroup;
if (linkedId) clipOut.linkedId = linkedId;
return clipOut;
}),
markers: (markers || []).map(({ t, label }) => (label ? { t, label } : { t })),
inPoint: inPoint == null ? null : inPoint,
outPoint: outPoint == null ? null : outPoint,
disabledTracks: normalizeDisabledTracks(disabledTracks),
};
if (encodeProfile) out.encodeProfile = encodeProfile;
return out;
}
function listenSSE() {
const es = new EventSource("/api/events");
Expand All @@ -672,6 +681,7 @@ async function syncFromServer(force) {
runtime.pendingSync = false;
if (state.binTab !== "project") fetchLibrary(state.binTab).then(renderLibrary);
loadLibraryFonts();
fetchEncodeProfiles();
try {
const res = await fetch("/api/project", { cache: "no-store" });
if (!res.ok) return;
Expand Down Expand Up @@ -5198,17 +5208,98 @@ function loop(ts) {
/* Two engines:
– fast: the browser renders every frame with the normal compositor
(frame-accurate, works unfocused) and streams JPEGs + an offline audio
mix to the server, where ffmpeg encodes a real CRF-18 MP4.
mix to the server, where ffmpeg encodes via an encoding profile.
– realtime: the original MediaRecorder capture, kept as the fallback for
local sessions / servers without ffmpeg. */

/* Placeholder until /api/export/profiles answers — the real list (and the real
ffmpeg args) always comes from encoding-profiles.json on the server. */
let encodeProfiles = {
default: "delivery",
profiles: {
draft: {
label: "Draft · H.264 fast",
description: "Quick preview — smaller file, faster encode.",
summary: "-c:v libx264 -preset veryfast -crf 23 -c:a aac -b:a 128k",
jpegQuality: 0.85,
},
delivery: {
label: "Delivery · H.264 balanced",
description: "Default export — good quality and compatibility.",
summary: "-c:v libx264 -preset fast -crf 18 -c:a aac -b:a 192k",
jpegQuality: 0.95,
},
hq: {
label: "High quality · H.264 slow",
description: "Best H.264 quality — slower encode, larger file.",
summary: "-c:v libx264 -preset slow -crf 16 -c:a aac -b:a 256k",
jpegQuality: 0.98,
},
},
};

async function fetchEncodeProfiles() {
if (!state.connected) return;
try {
const r = await fetch("/api/export/profiles", { cache: "no-store" });
if (!r.ok) return;
const data = await r.json();
if (data?.profiles && Object.keys(data.profiles).length) encodeProfiles = data;
} catch { }
}
function effectiveEncodeProfileId() {
return project.encodeProfile || getSetting("encodeProfile") || encodeProfiles.default || "delivery";
}
function exportProfileMeta(id) {
return encodeProfiles.profiles[id] || { label: id, summary: id, jpegQuality: 0.95 };
}
function updateExportProfileNote(id) {
const known = Object.hasOwn(encodeProfiles.profiles, id);
const p = exportProfileMeta(id);
if (els.exportProfileNote) {
els.exportProfileNote.textContent = known
? [p.description, p.summary].filter(Boolean).join(" — ")
: `"${id}" is not defined in encoding-profiles.json — the export will fail until it is added or another profile is picked.`;
}
if (els.exportProfileHint) {
if (project.encodeProfile) {
els.exportProfileHint.textContent =
"Project default (encodeProfile in project.json). Pick another profile here for a one-off export.";
} else if (getSetting("encodeProfile")) {
els.exportProfileHint.textContent = "Browser default — saved when you change this dropdown.";
} else {
els.exportProfileHint.textContent = "Using server default from encoding-profiles.json.";
}
}
}
function populateExportProfileSelect() {
if (!els.exportProfileSel) return;
const ids = Object.keys(encodeProfiles.profiles);
const cur = effectiveEncodeProfileId();
// a project/browser default naming a deleted profile must stay visible rather
// than silently falling through to whichever option happens to be first
if (cur && !ids.includes(cur)) ids.unshift(cur);
els.exportProfileSel.innerHTML = ids.map((id) => {
const p = encodeProfiles.profiles[id];
const sel = id === cur ? " selected" : "";
const label = p ? (p.label || id) : `${id} (not defined on the server)`;
return `<option value="${escapeHtml(id)}"${sel}>${escapeHtml(label)}</option>`;
}).join("");
updateExportProfileNote(els.exportProfileSel.value || cur || "delivery");
}
function syncExportProfileVisibility() {
const show = els.engineFast.checked && !els.engineFast.disabled;
els.exportProfileRow?.classList.toggle("hidden", !show);
}

function openExportSetup() {
if (state.exporting) return;
if (!project.clips.length) { alert("Timeline is empty — add some clips first."); return; }
const fastOk = state.connected && state.ffmpeg;
els.engineFast.disabled = !fastOk;
els.engineFast.checked = fastOk;
els.engineRealtime.checked = !fastOk;
syncExportProfileVisibility();
$("engineFastNote").textContent = fastOk
? "Frame-accurate ffmpeg encode. Keeps rendering if you switch tabs."
: "Needs the server + ffmpeg on PATH.";
Expand All @@ -5226,7 +5317,10 @@ function openExportSetup() {
warn.textContent = "";
warn.classList.add("hidden");
}
els.exportSetup.classList.remove("hidden");
fetchEncodeProfiles().then(() => {
populateExportProfileSelect();
els.exportSetup.classList.remove("hidden");
});
}
function startChosenExport() {
els.exportSetup.classList.add("hidden");
Expand Down Expand Up @@ -5337,8 +5431,18 @@ async function fastExport() {
els.exportTitle.textContent = "Mixing audio…";
const wav = await renderAudioMix(dur);
if (renderCancelled) throw new Error("cancelled");
const profileId = els.exportProfileSel?.value || effectiveEncodeProfileId();
const jpegQ = exportProfileMeta(profileId).jpegQuality ?? 0.95;
const begin = await fetch("/api/export/begin", {
method: "POST", body: JSON.stringify({ fps, name: project.name.replace(/[^\w\- ]+/g, "") || "export" }),
method: "POST",
body: JSON.stringify({
fps,
name: project.name.replace(/[^\w\- ]+/g, "") || "export",
profile: profileId,
// lets the server dry-run the profile with the same input count we
// will actually feed it, so -map based profiles are checked correctly
hasAudio: !!wav,
}),
}).then((r) => r.json());
if (!begin.id) throw new Error(begin.error || "export begin failed");
sessId = begin.id;
Expand All @@ -5354,7 +5458,7 @@ async function fastExport() {
await seekVideosTo(t);
await prepareFrameAssets(t); // exact SVG frames + AI masks
drawFrame(t);
const blob = await new Promise((res) => els.preview.toBlob(res, "image/jpeg", 0.95));
const blob = await new Promise((res) => els.preview.toBlob(res, "image/jpeg", jpegQ));
const r = await fetch("/api/export/frame?id=" + sessId, { method: "POST", body: blob });
if (!r.ok) throw new Error((await r.json()).error || "frame upload failed");
const pct = ((f + 1) / frames) * 100;
Expand Down Expand Up @@ -5453,6 +5557,16 @@ $("btnDelete").addEventListener("click", () => {
});
$("btnExport").addEventListener("click", openExportSetup);
$("btnStartExport").addEventListener("click", startChosenExport);
els.exportSetup?.addEventListener("change", (e) => {
if (e.target.name === "engine") syncExportProfileVisibility();
});
els.exportProfileSel?.addEventListener("change", (e) => {
const id = e.target.value;
if (!project.encodeProfile) {
setSetting("encodeProfile", id === encodeProfiles.default ? null : id);
}
updateExportProfileNote(id);
});
$("btnCancelSetup").addEventListener("click", () => els.exportSetup.classList.add("hidden"));
$("btnCancelExport").addEventListener("click", () => {
if (state.rendering) renderCancelled = true;
Expand Down
Loading
Loading