diff --git a/README.md b/README.md index 3b728d7..f09d0a5 100644 --- a/README.md +++ b/README.md @@ -176,10 +176,13 @@ clip timeline short best-1 clip_ec2f047d 1400 1478 --x 0.4 # words 1400–1478 clip timeline --timeline best-1 frame 3 check.jpg # look at it; move --x if the speaker is off clip timeline --timeline best-1 export best-1.mp4 clip timeline list # main and every short +clip timeline export --all ./shorts # every short, as ./shorts/.mp4 ``` `short` cuts each edge inside the pause the audio shows there, keeping 0.1 s of room tone (where there's no clear pause it follows the transcript and warns), fills the 9:16 frame (`--x` picks the part that shows), sets -14 LUFS and burns in captions. Give several ` ` ranges to join moments, in the order you give them. Any command takes `--timeline ` to work on a short, `clip timeline --timeline best-1 undo` takes the short apart again, and `short best-1 … --replace` rebuilds it. +In `clip ui`, select words in a clip's transcript and press **Make short**: the short opens in a tab of its own above the timeline, next to main. **Export all shorts** renders every one. + --- ## Another example: a product demo from screen recordings diff --git a/SKILL.md b/SKILL.md index ef44f04..8eadb11 100644 --- a/SKILL.md +++ b/SKILL.md @@ -333,7 +333,7 @@ To make a vertical (9:16), square (1:1) or feed (4:5) version, run `timeline ref To even out the volume of the exported mp4, run `timeline loudness -14` (YouTube, Shorts, Reels, TikTok) or `-16` (podcasts); `off` turns it off. Export then measures the finished mp4 and normalizes it to that integrated loudness in a second pass (EBU R128, true peak at most -1.5 dBTP), and reports `loudness` with the measured before and after values. `mode: dynamic` means loudnorm couldn't apply one gain (steady or very short audio, a wide loudness range, or peaks); `skipped` means the audio was silent or too short to measure. It's one setting on the timeline that `timeline undo` reverses, it applies to the mp4 only, and it doesn't even out clips recorded at different levels. -To cut vertical shorts out of a long recording, transcribe it and make the cuts you want everywhere (pauses, retakes) on the main timeline first, then read `timeline transcribe --sentences`, pick each moment as whole sentences and run `timeline short ` with their word indices, where `` is a new name such as `best-1`. Pick whole sentences, 15–60 s in all; you choose the moments, the editor doesn't score them. The short is a timeline of its own, cut from main's clips (main's cuts carry in, main itself doesn't change): it cuts each edge inside the pause the audio shows there, keeping 0.1 s of room tone, so it never keeps the next sentence's first syllable, fills a 9:16 frame, sets -14 LUFS and burns in captions. More ` ` triples join several moments in the order given, so a later line can open the short as its hook. Options: `--x 0..1` to keep the speaker in frame, `--aspect 1:1|4:5`, `--words N` per caption, `--no-captions`, `--loudness `, `--source `, `--dry-run` (prints the pieces), `--replace` to rebuild it. Its `warnings` name a range that starts or ends mid-sentence (widen it to the sentence's edge unless the cut is meant) and an edge with no clear pause in the audio (music under the speech, or no break), which follows the transcript instead: look at that edge or listen to it. Every other command works on a short with `--timeline `: look at it with `timeline --timeline best-1 frame 3 check.jpg`, adjust with `timeline --timeline best-1 reframe 9:16 --x 0.3`, and render it with `timeline --timeline best-1 export best-1.mp4`. `timeline list` shows every timeline, `timeline --timeline best-1 undo` takes the short apart, and `timeline delete best-1` removes it. Over MCP, it's the `make_short` verb in `timeline_edit` with `timeline` set to the new id; every timeline tool takes `timeline`, and `timeline_show` lists them all. +To cut vertical shorts out of a long recording, transcribe it and make the cuts you want everywhere (pauses, retakes) on the main timeline first, then read `timeline transcribe --sentences`, pick each moment as whole sentences and run `timeline short ` with their word indices, where `` is a new name such as `best-1`. Pick whole sentences, 15–60 s in all; you choose the moments, the editor doesn't score them. The short is a timeline of its own, cut from main's clips (main's cuts carry in, main itself doesn't change): it cuts each edge inside the pause the audio shows there, keeping 0.1 s of room tone, so it never keeps the next sentence's first syllable, fills a 9:16 frame, sets -14 LUFS and burns in captions. More ` ` triples join several moments in the order given, so a later line can open the short as its hook. Options: `--x 0..1` to keep the speaker in frame, `--aspect 1:1|4:5`, `--words N` per caption, `--no-captions`, `--loudness `, `--source `, `--dry-run` (prints the pieces), `--replace` to rebuild it. Its `warnings` name a range that starts or ends mid-sentence (widen it to the sentence's edge unless the cut is meant) and an edge with no clear pause in the audio (music under the speech, or no break), which follows the transcript instead: look at that edge or listen to it. Every other command works on a short with `--timeline `: look at it with `timeline --timeline best-1 frame 3 check.jpg`, adjust with `timeline --timeline best-1 reframe 9:16 --x 0.3`, and render it with `timeline --timeline best-1 export best-1.mp4` (`timeline export --all ` renders every short as `/.mp4`). `timeline list` shows every timeline, `timeline --timeline best-1 undo` takes the short apart, and `timeline delete best-1` removes it. Over MCP, it's the `make_short` verb in `timeline_edit` with `timeline` set to the new id; every timeline tool takes `timeline`, and `timeline_show` lists them all. To hand the cut to an editor app, export the timeline as an editor project instead of a render. `timeline export cut.fcpxml` (or `--format fcpxml`) writes FCPXML 1.10 for Final Cut Pro and DaVinci Resolve; `timeline export cut.xml` (or `--format xmeml`) writes Final Cut Pro 7 XML for Premiere Pro, which Resolve reads too. Premiere doesn't import FCPXML, so use `xmeml` for it. Both reference the source files by absolute path (nothing is copied, so keep the sources where they are), place camera files by their start timecode, and keep the mp4's timing: clips, gaps, fades, volume, fade/dissolve transitions (as cross dissolves) and black or colored cards. Text over the picture becomes titles in FCPXML; in the Premiere XML it goes to `.srt` caption files next to the XML, since Premiere drops that format's titles. What the file can't hold — speed, color adjustments, chroma key, a reframe's fill, zoom or position (set the timeline to `--fit contain` and reset clip framing to carry it), rotation, wipe/slide/circle transitions, audio-only tracks, media on two tracks, a frame rate editors don't offer — is refused with the fix to make; what it holds only in part (text styling, caption boxes) is listed in the output's `notes`. diff --git a/docs/backlog/confine-render-media.md b/docs/backlog/confine-render-media.md new file mode 100644 index 0000000..b5ab9c8 --- /dev/null +++ b/docs/backlog/confine-render-media.md @@ -0,0 +1,9 @@ +# Re-check media against the workspace on every render for the UI and MCP + +`POST /api/timelines/export` re-checks each short's files with `confinedMediaMap`, so a registered file swapped for a symlink out of the workspace is refused. Three older render paths still take the registry as ingested (`buildMediaMap`): + +- `POST /api/timeline/export` (mp4) in `src/ui/server.ts`; +- `GET /api/timeline/frame` in `src/ui/server.ts`; +- the mp4 path of MCP `timeline_export` in `src/mcp/server.ts`. + +The transcript and project-file paths already confine. Pass `confinedMediaMap(comp)` on these three, and test each with a symlinked file, as `tests/ui-timelines.test.ts` does for export-all. Found by the Codex review of the UI shorts PR (2026-11-15). diff --git a/docs/backlog/ui-inspector-height.md b/docs/backlog/ui-inspector-height.md new file mode 100644 index 0000000..4efeff5 --- /dev/null +++ b/docs/backlog/ui-inspector-height.md @@ -0,0 +1,5 @@ +# Clip inspector has no room on a short window + +In `clip ui`, the viewer takes up to 44vh plus its bar, and the clip inspector (`.stage-detail`) gets whatever is left of `.main`. At a 1000 px tall window, `.main` is 468 px and the viewer fills all of it, so the inspector is 0 px tall: a selected clip's transcript, framing and Make short can't be reached. It works at 1500 px. Seen on 2026-11-15 while checking the shorts tabs; the layout was the same before them, and the tabs bar takes another ~30 px once a short exists. + +Fix: give the inspector a minimum height (or let the viewer shrink first), and check at 900 px tall. diff --git a/llms.txt b/llms.txt index fcc3577..382efa5 100644 --- a/llms.txt +++ b/llms.txt @@ -21,7 +21,7 @@ - **Captions:** `clip timeline captions` burns in captions timed to the words (short phrases or one word at a time); on a vertical canvas they stay clear of the Shorts, Reels and TikTok overlays, and their width follows the font size - **Reframe:** `clip timeline reframe 9:16` turns a landscape cut vertical (also 1:1, 4:5); `--x` picks the part that shows, `clip timeline framing ` sets one clip's zoom and position - **Loudness:** `clip timeline loudness -14` normalizes the exported mp4 to -14 LUFS (or `-16` for podcasts) in two passes -- **Shorts from a long recording:** `clip timeline short best-1 ` (MCP: the `make_short` verb) turns a range of transcript words into a 9:16 short on a timeline of its own, captioned and at -14 LUFS, keeping the cuts made on the main timeline; `--timeline ` points any command at a short +- **Shorts from a long recording:** `clip timeline short best-1 ` (MCP: the `make_short` verb) turns a range of transcript words into a 9:16 short on a timeline of its own, captioned and at -14 LUFS, keeping the cuts made on the main timeline; `--timeline ` points any command at a short, and `clip timeline export --all ` renders every short. In `clip ui`, Make short works from a transcript selection and each short gets a tab - **Cut check:** `clip timeline check --sheet cuts.jpg` flags cuts that may clip a word and writes a contact sheet - **Export to pro editors:** `clip timeline export --format fcpxml` (Final Cut Pro, DaVinci Resolve) or `--format xmeml` (Premiere Pro; captions as .srt) - Every timeline edit is one undoable step (`clip timeline undo` / `redo`), from the CLI, the MCP server or the browser UI diff --git a/src/cli-timeline.ts b/src/cli-timeline.ts index 0027091..41485d2 100644 --- a/src/cli-timeline.ts +++ b/src/cli-timeline.ts @@ -60,7 +60,7 @@ import { planReframe, } from './timeline/reframe.js'; import { reviewCuts } from './timeline/review.js'; -import { runPlan } from './timeline/run-plan.js'; +import { exportTimelines, runPlan } from './timeline/run-plan.js'; import { MAIN_TIMELINE, TimelineIdSchema } from './timeline/schema.js'; import { checkShortTimelines, planShort, type ShortRange } from './timeline/shorts.js'; import { analyzeSilence, DEFAULT_SENSITIVITY, SensitivitySchema } from './timeline/silence.js'; @@ -109,6 +109,7 @@ Every command works on the main timeline unless --timeline names another clip timeline redo clip timeline log clip timeline export [] [--format mp4|fcpxml|xmeml] + clip timeline export --all (every short, as /.mp4) `; interface Parsed { @@ -117,7 +118,7 @@ interface Parsed { } /** Flags that take no value, so they don't swallow the next argument. */ -const BOOLEAN_FLAGS = new Set(['dry-run', 'reset', 'no-captions', 'replace', 'sentences']); +const BOOLEAN_FLAGS = new Set(['all', 'dry-run', 'reset', 'no-captions', 'replace', 'sentences']); function parseArgs(args: string[]): Parsed { const positional: string[] = []; @@ -868,6 +869,25 @@ export async function runTimeline(args: string[]): Promise { } case 'export': { + if (flags.all === 'true') { + const [dir] = positional; + if (!dir) throw new Error('Usage: clip timeline export --all '); + if (flags.timeline !== undefined || (flags.format && flags.format !== 'mp4')) { + throw new Error( + '--all renders every short as mp4. Export one timeline with --timeline , or a project with --format.', + ); + } + const shorts = (await listTimelines()) + .map((t) => t.id) + .filter((id) => id !== MAIN_TIMELINE); + if (shorts.length === 0) { + throw new Error('No shorts to export. Make one with `clip timeline short`.'); + } + const exported = await exportTimelines(shorts, resolveInput(dir)); + out({ exported }); + if (exported.some((e) => 'error' in e)) process.exitCode = 1; + return; + } const [output] = positional; const format = flags.format || (output && interchangeFormatFor(output)) || 'mp4'; if (format !== 'mp4') { diff --git a/src/index.ts b/src/index.ts index 7206d6b..d8e1a3f 100644 --- a/src/index.ts +++ b/src/index.ts @@ -238,7 +238,13 @@ export { renderCutSheet, reviewCuts, } from './timeline/review.js'; -export { type LoudnessResult, type RunPlanResult, runPlan } from './timeline/run-plan.js'; +export { + exportTimelines, + type LoudnessResult, + type RunPlanResult, + runPlan, + type TimelineExport, +} from './timeline/run-plan.js'; export { AudioStreamSchema, MAIN_TIMELINE, diff --git a/src/timeline/run-plan.ts b/src/timeline/run-plan.ts index 69f64b3..e89b3af 100644 --- a/src/timeline/run-plan.ts +++ b/src/timeline/run-plan.ts @@ -1,5 +1,5 @@ import { randomBytes } from 'node:crypto'; -import { rename, unlink, writeFile } from 'node:fs/promises'; +import { mkdir, rename, unlink, writeFile } from 'node:fs/promises'; import { dirname, extname, join } from 'node:path'; import { buildLoudnormApplyArgs, @@ -7,7 +7,12 @@ import { parseLoudnormJson, } from '../ffmpeg/args/loudnorm.js'; import { filterGraphFileOption, requireDrawtext, runFfmpeg } from '../ffmpeg/run.js'; -import type { FfmpegPlan } from './compile.js'; +import { getWorkspace } from '../workspace.js'; +import { compileTimeline, type FfmpegPlan, type MediaInfo } from './compile.js'; +import type { Composition } from './composition.js'; +import { readComposition } from './document-store.js'; +import { buildMediaMap } from './media-registry.js'; +import type { MediaId } from './schema.js'; export interface LoudnessResult { targetLufs: number; @@ -150,3 +155,56 @@ async function normalizeLoudness( throw err; } } + +/** A timeline `exportTimelines` rendered, or why it couldn't. */ +export type TimelineExport = + | { + timeline: string; + rev: number; + output: string; + durationSec: number; + durationMs: number; + loudness?: LoudnessResult; + } + | { timeline: string; error: string }; + +/** + * Render each timeline to `/.mp4`, one at a time. A timeline that + * can't render is reported with its error, and the rest still render. `media` + * resolves each document's files (default: the registry as ingested); callers + * acting for the UI pass one that re-checks them against the workspace. + */ +export async function exportTimelines( + timelines: readonly string[], + dir: string, + media: (comp: Composition) => Promise> = () => buildMediaMap(), +): Promise { + await mkdir(dir, { recursive: true }); + // Intermediates are named per run, so a concurrent export or frame preview + // in the same workspace can't overwrite them. + const run = randomBytes(4).toString('hex'); + const results: TimelineExport[] = []; + for (const timeline of timelines) { + try { + const comp = await readComposition(timeline); + const plan = compileTimeline(comp, { + media: await media(comp), + dir: getWorkspace(), + output: join(dir, `${timeline}.mp4`), + tag: `${run}-${timeline}`, + }); + const result = await runPlan(plan); + results.push({ + timeline, + rev: comp.rev, + output: result.output, + durationSec: plan.durationSec, + durationMs: result.durationMs, + ...(result.loudness ? { loudness: result.loudness } : {}), + }); + } catch (err) { + results.push({ timeline, error: err instanceof Error ? err.message : String(err) }); + } + } + return results; +} diff --git a/src/ui/server.ts b/src/ui/server.ts index 26b2859..319f47c 100644 --- a/src/ui/server.ts +++ b/src/ui/server.ts @@ -8,7 +8,7 @@ import { fileURLToPath } from 'node:url'; import { serve } from '@hono/node-server'; import { serveStatic } from '@hono/node-server/serve-static'; import getPort from 'get-port'; -import { Hono } from 'hono'; +import { type Context, Hono } from 'hono'; import { csrf } from 'hono/csrf'; import open from 'open'; import { ZodError, z } from 'zod'; @@ -26,9 +26,13 @@ import { import { applyVerbs, CompositionConflictError, + deleteTimeline, + listTimelines, + NoTimelineError, readComposition, readDocOpLog, redoDocOp, + TimelineIdError, undoLastDocOp, } from '../timeline/document-store.js'; import { @@ -40,8 +44,8 @@ import { import { buildMediaMap } from '../timeline/media-registry.js'; import { CompositionOpError } from '../timeline/ops.js'; import { ReframeError } from '../timeline/reframe.js'; -import { runPlan } from '../timeline/run-plan.js'; -import type { MediaId } from '../timeline/schema.js'; +import { exportTimelines, runPlan } from '../timeline/run-plan.js'; +import { MAIN_TIMELINE, type MediaId, TimelineIdSchema } from '../timeline/schema.js'; import { ShortError } from '../timeline/shorts.js'; import { SilenceAnalysisError } from '../timeline/silence.js'; import { TranscriptError, transcribeClip } from '../timeline/transcript.js'; @@ -58,7 +62,12 @@ import { resolveInWorkspace, WorkspaceBoundaryError, } from '../workspace.js'; -import { clipConfinedMedia, confinedMediaRefs, makeVerbContext } from './timeline-tools.js'; +import { + clipConfinedMedia, + confinedMediaMap, + confinedMediaRefs, + makeVerbContext, +} from './timeline-tools.js'; /** Errors about the clip or its setup, not the server: silence removal or * transcription can't work on this clip, whisper.cpp isn't set up, a reframe @@ -75,6 +84,17 @@ function isClipConditionError(err: unknown): boolean { import { isRegisteredTool, TOOL_REGISTRY } from './tool-registry.js'; +/** The timeline a `/api/timeline*` request names with `?timeline=` (default + * `main`); the middleware has already refused a malformed id. */ +function timelineOf(c: Context): string { + return c.req.query('timeline') ?? MAIN_TIMELINE; +} + +/** A timeline that doesn't exist, or a malformed id: `onError` answers 404 or 400. */ +function isTimelineError(err: unknown): boolean { + return err instanceof NoTimelineError || err instanceof TimelineIdError; +} + const HERE = dirname(fileURLToPath(import.meta.url)); // `HERE` resolves to different paths depending on whether we're running // the bundled `dist/cli.js` (everything is flattened so `HERE === dist/`) @@ -161,6 +181,18 @@ export async function startUiServer(options: UiServerOptions = {}): Promise { + if (err instanceof NoTimelineError) return c.json({ error: err.message }, 404); + if (err instanceof TimelineIdError) return c.json({ error: err.message }, 400); + // Otherwise answer as Hono does by default. + if ('getResponse' in err && typeof err.getResponse === 'function') { + const res = (err.getResponse as () => Response)(); + return c.newResponse(res.body, res); + } + console.error(err); + return c.text('Internal Server Error', 500); + }); + app.get('/api/workspace', async (c) => { return c.json({ path: getWorkspace() }); }); @@ -228,9 +260,19 @@ export async function startUiServer(options: UiServerOptions = {}): Promise { + const id = c.req.query('timeline') ?? MAIN_TIMELINE; + if (!TimelineIdSchema.safeParse(id).success) { + return c.json({ error: `Invalid timeline id "${id}".` }, 400); + } + await next(); + }); + + /** GET /api/timeline — the timeline's composition document. */ app.get('/api/timeline', async (c) => { - return c.json({ document: await readComposition() }); + return c.json({ document: await readComposition(timelineOf(c)) }); }); /** @@ -250,9 +292,10 @@ export async function startUiServer(options: UiServerOptions = {}): Promise { try { - const comp = await readComposition(); + const comp = await readComposition(timelineOf(c)); const clipId = c.req.param('clipId'); return c.json(await transcribeClip(comp, clipId, await clipConfinedMedia(comp, clipId))); } catch (err) { + if (isTimelineError(err)) throw err; if (err instanceof WorkspaceBoundaryError) return c.json({ error: err.message }, 403); if (isClipConditionError(err)) return c.json({ error: (err as Error).message }, 422); const message = err instanceof Error ? err.message : String(err); @@ -292,7 +336,7 @@ export async function startUiServer(options: UiServerOptions = {}): Promise { - const comp = await readComposition(); + const comp = await readComposition(timelineOf(c)); const media = await buildMediaMap(); return c.json(checkExportable(comp, media, getWorkspace())); }); @@ -311,7 +355,7 @@ export async function startUiServer(options: UiServerOptions = {}): Promise {}); + if (isTimelineError(err)) throw err; if (err instanceof CompileError) return c.json({ error: err.message }, 422); // Not a document problem — a transient render failure (e.g. an FFmpeg // hiccup). Surface it as retryable instead of leaking a raw 500. @@ -391,12 +436,13 @@ export async function startUiServer(options: UiServerOptions = {}): Promise { - const { undone, label } = await undoLastDocOp(); + const { undone, label } = await undoLastDocOp(timelineOf(c)); return c.json({ undone, label: label ?? null }); }); /** POST /api/timeline/redo — redo the most recently undone document edit. */ app.post('/api/timeline/redo', async (c) => { - const { redone, label } = await redoDocOp(); + const { redone, label } = await redoDocOp(timelineOf(c)); return c.json({ redone, label: label ?? null }); }); /** GET /api/timeline/history — undo/redo availability for the doc op-log. */ app.get('/api/timeline/history', async (c) => { - const log = await readDocOpLog(); + const log = await readDocOpLog(timelineOf(c)); return c.json({ canUndo: log.cursor > 0, canRedo: log.cursor < log.entries.length, }); }); + // ─── The workspace's timelines: main and every short ────────────── + + /** GET /api/timelines — main first, then every short, with its shape. */ + app.get('/api/timelines', async (c) => { + try { + return c.json({ timelines: await listTimelines() }); + } catch (err) { + if (err instanceof WorkspaceBoundaryError) return c.json({ error: err.message }, 403); + throw err; + } + }); + + /** DELETE /api/timelines/:id — remove a short and its undo history. Main can't go. */ + app.delete('/api/timelines/:id', async (c) => { + const id = c.req.param('id'); + try { + await deleteTimeline(id); + return c.json({ deleted: id }); + } catch (err) { + if (err instanceof WorkspaceBoundaryError) return c.json({ error: err.message }, 403); + throw err; + } + }); + + /** + * POST /api/timelines/export — render every short, one at a time, to + * `shorts-/.mp4` in the workspace, and log each to the session so it + * shows in Outputs. A short that can't render is reported with its error. + */ + app.post('/api/timelines/export', async (c) => { + try { + const shorts = (await listTimelines()).map((t) => t.id).filter((id) => id !== MAIN_TIMELINE); + if (shorts.length === 0) { + return c.json({ error: 'No shorts to export yet. Make one from the transcript.' }, 422); + } + const dir = resolve(getWorkspace(), `shorts-${randomBytes(4).toString('hex')}`); + const exported = []; + for (const result of await exportTimelines(shorts, dir, confinedMediaMap)) { + if ('error' in result) { + exported.push(result); + continue; + } + const { timeline, rev, output, durationSec, loudness } = result; + const entry = await appendOp({ + tool: 'timeline_export', + args: { timeline, rev }, + result: { path: output, durationSec, ...(loudness ? { loudness } : {}) }, + }); + exported.push({ ...result, id: entry.id }); + } + return c.json({ dir, exported }); + } catch (err) { + if (err instanceof WorkspaceBoundaryError) return c.json({ error: err.message }, 403); + throw err; + } + }); + /** * Session safety endpoints. These intentionally live outside the * /api/tools/:name registry because snapshot/undo are meta-operations on diff --git a/src/ui/web/src/App.tsx b/src/ui/web/src/App.tsx index 8f57d75..b2ef559 100644 --- a/src/ui/web/src/App.tsx +++ b/src/ui/web/src/App.tsx @@ -1,4 +1,4 @@ -import { useMemo, useState } from 'react'; +import { useEffect, useMemo, useState } from 'react'; import { ClipInspector } from './components/ClipInspector.js'; import { DetailPane } from './components/DetailPane.js'; import { DocTimeline } from './components/DocTimeline.js'; @@ -17,18 +17,32 @@ import { Header } from './components/Header.js'; import { ImportZone } from './components/ImportZone.js'; import { OpList } from './components/OpList.js'; import { Timeline } from './components/Timeline.js'; +import { type TimelineNotice, TimelineTabs } from './components/TimelineTabs.js'; import { ToolPickerModal } from './components/ToolPickerModal.js'; import { Viewer } from './components/Viewer.js'; import { useKeyboardShortcuts } from './hooks/useKeyboardShortcuts.js'; import { useSession } from './hooks/useSession.js'; import { useSessionSafety } from './hooks/useSessionSafety.js'; import { useTimeline } from './hooks/useTimeline.js'; +import { useTimelines } from './hooks/useTimelines.js'; import { findClip, mediaNames, nextClipOnTrack } from './lib/composition.js'; +import { MAIN_TIMELINE, nextShortId, withTimeline } from './lib/timelines.js'; import { applyTimelineVerbs, type Verb } from './lib/verbs.js'; +/** A short `POST /api/timelines/export` rendered, or why it couldn't. */ +type ExportedShort = + | { timeline: string; output: string; loudness?: { outputLufs: number | null } } + | { timeline: string; error: string }; + export function App() { const { session, loading, error, refresh } = useSession(); - const { composition, refresh: refreshTimeline } = useTimeline(); + const { + timelines, + current: timeline, + select: selectTimeline, + refresh: refreshTimelines, + } = useTimelines(); + const { composition, missing, refresh: refreshTimeline } = useTimeline(timeline); const safety = useSessionSafety(); const [selectedId, setSelectedId] = useState(null); const [selectedClipId, setSelectedClipId] = useState(null); @@ -38,11 +52,31 @@ export function App() { const [safetyError, setSafetyError] = useState(null); const [verbBusy, setVerbBusy] = useState(false); const [verbError, setVerbError] = useState(null); + const [exportingAll, setExportingAll] = useState(false); + const [notice, setNotice] = useState(null); const selectedEntry = selectedId === null ? null : (session.entries.find((e) => e.id === selectedId) ?? null); const selected = selectedClipId === null ? null : findClip(composition, selectedClipId); const names = useMemo(() => mediaNames(session.entries), [session.entries]); + const freeShortId = nextShortId(timelines.map((t) => t.id)); + + // Each timeline has its own clips and length, so switching starts clean. + function openTimeline(id: string) { + selectTimeline(id); + setSelectedClipId(null); + setPlayheadSec(0); + setVerbError(null); + } + + // A short deleted elsewhere (the agent, another tab) falls back to main. + useEffect(() => { + if (missing && timeline !== MAIN_TIMELINE) { + selectTimeline(MAIN_TIMELINE); + setSelectedClipId(null); + setPlayheadSec(0); + } + }, [missing, timeline, selectTimeline]); // Selecting an op and selecting a doc-clip are mutually exclusive — each owns // the inspector pane, so picking one clears the other. @@ -63,7 +97,7 @@ export function App() { setVerbBusy(true); setVerbError(null); try { - const { applied } = await applyTimelineVerbs(verbs); + const { applied } = await applyTimelineVerbs(verbs, timeline); await Promise.all([refreshTimeline(), refresh()]); return applied; } catch (err) { @@ -74,10 +108,100 @@ export function App() { } } + /** Make a short on timeline `id` and open it; false when it couldn't be made. */ + async function handleMakeShort(id: string, verb: Verb): Promise { + setVerbBusy(true); + setVerbError(null); + try { + const { warnings } = await applyTimelineVerbs([verb], id); + await Promise.all([refreshTimelines(), refresh()]); + openTimeline(id); + setNotice({ + kind: warnings.length > 0 ? 'warn' : 'info', + lines: [`Made ${id}.`, ...warnings], + }); + return true; + } catch (err) { + setVerbError(err instanceof Error ? err.message : String(err)); + return false; + } finally { + setVerbBusy(false); + } + } + + async function handleDeleteTimeline(id: string): Promise { + if ( + !window.confirm( + `Delete the short "${id}"? Its edits and undo history go with it; files you exported stay.`, + ) + ) { + return; + } + setSafetyError(null); + try { + const res = await fetch(`/api/timelines/${encodeURIComponent(id)}`, { method: 'DELETE' }); + if (!res.ok) { + const json = (await res.json().catch(() => ({}))) as { error?: string }; + throw new Error(json.error ?? `HTTP ${res.status}`); + } + await refreshTimelines(); + if (id === timeline) openTimeline(MAIN_TIMELINE); + } catch (err) { + setSafetyError(`Delete failed: ${err instanceof Error ? err.message : String(err)}`); + } + } + + async function handleExportAll(): Promise { + setExportingAll(true); + setSafetyError(null); + setNotice(null); + try { + const res = await fetch('/api/timelines/export', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: '{}', + }); + const json = (await res.json()) as { + dir?: string; + exported?: ExportedShort[]; + error?: string; + }; + if (!res.ok || !json.exported) throw new Error(json.error ?? `HTTP ${res.status}`); + await refresh(); // each short now shows in Outputs + const saved = json.exported.flatMap((e) => + 'error' in e + ? [] + : [ + e.loudness?.outputLufs != null + ? `${e.timeline} (${e.loudness.outputLufs.toFixed(1)} LUFS)` + : e.timeline, + ], + ); + const failed = json.exported.flatMap((e) => + 'error' in e ? [`${e.timeline} didn't render: ${e.error}`] : [], + ); + setNotice({ + kind: failed.length > 0 ? 'warn' : 'info', + lines: [ + ...(saved.length > 0 + ? [`Saved ${saved.length} in ${json.dir ?? 'the workspace'}: ${saved.join(', ')}.`] + : []), + ...failed, + ], + }); + } catch (err) { + setSafetyError(`Export failed: ${err instanceof Error ? err.message : String(err)}`); + } finally { + setExportingAll(false); + } + } + async function handleTimelineHistory(direction: 'undo' | 'redo'): Promise { setVerbError(null); try { - const res = await fetch(`/api/timeline/${direction}`, { method: 'POST' }); + const res = await fetch(withTimeline(`/api/timeline/${direction}`, timeline), { + method: 'POST', + }); if (!res.ok) throw new Error(`HTTP ${res.status}`); await Promise.all([refreshTimeline(), refresh()]); } catch (err) { @@ -91,7 +215,7 @@ export function App() { setVerbBusy(true); setSafetyError(null); try { - const res = await fetch('/api/timeline/export', { + const res = await fetch(withTimeline('/api/timeline/export', timeline), { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: '{}', @@ -202,7 +326,7 @@ export function App() {
- +
{error ? (
@@ -224,6 +348,7 @@ export function App() { ) : selected ? ( c.id)} busy={verbBusy} error={verbError} + nextShortId={freeShortId} onApply={handleApplyVerbs} + onMakeShort={handleMakeShort} onSeek={setPlayheadSec} onClose={() => setSelectedClipId(null)} /> @@ -244,7 +371,18 @@ export function App() {
+ void handleDeleteTimeline(id)} + onExportAll={() => void handleExportAll()} + onDismissNotice={() => setNotice(null)} + /> ; @@ -64,8 +69,12 @@ export function ClipInspector({ trackMediaIds: string[]; busy: boolean; error: string | null; + /** A name no timeline uses yet, offered for a new short. */ + nextShortId: string; /** Resolves to the number of ops applied, or null if the edit failed. */ onApply: (verbs: Verb[]) => Promise; + /** Make a short on timeline `id` and open it; resolves false when that failed. */ + onMakeShort: (id: string, verb: Verb) => Promise; /** Move the playhead to a timeline time. */ onSeek: (sec: number) => void; onClose: () => void; @@ -194,7 +203,15 @@ export function ClipInspector({ ) : null} {clip.kind === 'media' ? ( - + ) : null}
diff --git a/src/ui/web/src/components/DocTimeline.tsx b/src/ui/web/src/components/DocTimeline.tsx index 44b8158..7d67328 100644 --- a/src/ui/web/src/components/DocTimeline.tsx +++ b/src/ui/web/src/components/DocTimeline.tsx @@ -1,5 +1,6 @@ import { useEffect, useRef, useState } from 'react'; import { clipDuration, clipEndSec, clipLabel, compositionDuration } from '../lib/composition.js'; +import { withTimeline } from '../lib/timelines.js'; import type { Verb } from '../lib/verbs.js'; import type { Clip, Composition, MediaClip } from '../types.js'; import { CaptionsPanel } from './CaptionsPanel.js'; @@ -20,6 +21,7 @@ const MAX_THUMBS = 16; // cap thumbnails per clip to bound requests * playhead is controlled by the parent so the clip inspector can split at it. */ export function DocTimeline({ + timeline, composition, mediaNames, selectedClipId, @@ -33,6 +35,8 @@ export function DocTimeline({ onApply, verbError, }: { + /** The timeline shown: `main` or a short. */ + timeline: string; composition: Composition; /** File name of each media id, for labeling media clips. */ mediaNames: ReadonlyMap; @@ -74,7 +78,7 @@ export function DocTimeline({ }); useEffect(() => { let alive = true; - fetch(`/api/timeline/history?rev=${composition.rev}`) + fetch(withTimeline(`/api/timeline/history?rev=${composition.rev}`, timeline)) .then((r) => r.json() as Promise<{ canUndo: boolean; canRedo: boolean }>) .then((h) => { if (alive) setHistory(h); @@ -83,7 +87,7 @@ export function DocTimeline({ return () => { alive = false; }; - }, [composition.rev]); + }, [timeline, composition.rev]); return (
@@ -119,7 +123,12 @@ export function DocTimeline({ > ↷ - +
{hasClips ? ( diff --git a/src/ui/web/src/components/ExportButton.tsx b/src/ui/web/src/components/ExportButton.tsx index 32d78c4..827aba6 100644 --- a/src/ui/web/src/components/ExportButton.tsx +++ b/src/ui/web/src/components/ExportButton.tsx @@ -1,4 +1,5 @@ import { useEffect, useRef, useState } from 'react'; +import { withTimeline } from '../lib/timelines.js'; type ProjectFormat = 'fcpxml' | 'xmeml'; @@ -15,8 +16,8 @@ const PROJECTS: readonly { format: ProjectFormat; label: string; detail: string /** POST the export route for an editor project; throws with the server's * message (e.g. the 422 naming what the project can't hold). */ -async function exportProject(format: ProjectFormat): Promise { - const res = await fetch('/api/timeline/export', { +async function exportProject(format: ProjectFormat, timeline: string): Promise { + const res = await fetch(withTimeline('/api/timeline/export', timeline), { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ format }), @@ -35,10 +36,12 @@ const fileName = (path: string): string => path.split(/[\\/]/).pop() ?? path; * carries only in part. */ export function ExportButton({ + timeline, busy, disabled, onExportVideo, }: { + timeline: string; busy: boolean; disabled: boolean; onExportVideo: () => void; @@ -72,7 +75,7 @@ export function ExportButton({ setError(null); setCopied(false); try { - setResult(await exportProject(format)); + setResult(await exportProject(format, timeline)); } catch (err) { setError(err instanceof Error ? err.message : String(err)); } finally { diff --git a/src/ui/web/src/components/FramingSection.tsx b/src/ui/web/src/components/FramingSection.tsx index 62d39d2..2576e34 100644 --- a/src/ui/web/src/components/FramingSection.tsx +++ b/src/ui/web/src/components/FramingSection.tsx @@ -110,7 +110,7 @@ export function FramingSection({ ); } -function Slider({ +export function Slider({ label, value, min, diff --git a/src/ui/web/src/components/TimelineTabs.tsx b/src/ui/web/src/components/TimelineTabs.tsx new file mode 100644 index 0000000..0d7057d --- /dev/null +++ b/src/ui/web/src/components/TimelineTabs.tsx @@ -0,0 +1,104 @@ +import { MAIN_TIMELINE, shapeLabel, type TimelineListing } from '../lib/timelines.js'; + +/** What the last short-making or export-all did, shown under the tabs. */ +export interface TimelineNotice { + kind: 'info' | 'warn'; + lines: string[]; +} + +/** + * One tab per timeline in the workspace, main and every short, each with its + * shape. A short's ✕ deletes it (the caller confirms), and "Export all shorts" + * renders each one. Shown once there is a short. + */ +export function TimelineTabs({ + timelines, + current, + exportingAll, + notice, + onSelect, + onDelete, + onExportAll, + onDismissNotice, +}: { + timelines: TimelineListing[]; + current: string; + exportingAll: boolean; + notice: TimelineNotice | null; + onSelect: (id: string) => void; + onDelete: (id: string) => void; + onExportAll: () => void; + onDismissNotice: () => void; +}) { + const shorts = timelines.filter((t) => t.id !== MAIN_TIMELINE); + if (shorts.length === 0 && notice === null) return null; + + return ( +
+
+
+ {timelines.map((t) => { + const isCurrent = t.id === current; + const shape = 'error' in t ? 'unreadable' : shapeLabel(t.width, t.height); + return ( +
+ + {t.id === MAIN_TIMELINE ? null : ( + + )} +
+ ); + })} +
+ +
+ {notice ? ( +
+
+ {notice.lines.map((line) => ( +
{line}
+ ))} +
+ +
+ ) : null} +
+ ); +} diff --git a/src/ui/web/src/components/TranscriptSection.tsx b/src/ui/web/src/components/TranscriptSection.tsx index e3f648e..307d3f9 100644 --- a/src/ui/web/src/components/TranscriptSection.tsx +++ b/src/ui/web/src/components/TranscriptSection.tsx @@ -1,6 +1,8 @@ import { useCallback, useEffect, useRef, useState } from 'react'; +import { isTimelineId, MAIN_TIMELINE, withTimeline } from '../lib/timelines.js'; import type { Verb } from '../lib/verbs.js'; import type { MediaClip } from '../types.js'; +import { Slider } from './FramingSection.js'; /** Mirrors ClipTranscript in src/timeline/transcript.ts. */ interface ClipWord { @@ -19,19 +21,28 @@ interface ClipTranscript { /** * Edit a media clip by its words: transcribe on request (the local whisper.cpp * is optional, so its setup message shows here when it's missing), click a word - * to move the playhead, shift-click to select a range, then cut it. Phrases - * said twice in a row are marked and can all be cut at once. Every cut is one - * undoable edit through the verb layer. + * to move the playhead, shift-click to select a range, then cut it or make a + * short of it. Phrases said twice in a row are marked and can all be cut at + * once. Every cut is one undoable edit through the verb layer. */ export function TranscriptSection({ + timeline, clip, busy, + nextShortId, onApply, + onMakeShort, onSeek, }: { + /** The timeline the clip is on, which a short is cut from. */ + timeline: string; clip: MediaClip; busy: boolean; + /** A name no timeline uses yet, offered for a new short. */ + nextShortId: string; onApply: (verbs: Verb[]) => Promise; + /** Make a short on timeline `id` and open it; resolves false when that failed. */ + onMakeShort: (id: string, verb: Verb) => Promise; onSeek: (sec: number) => void; }) { const [transcript, setTranscript] = useState(null); @@ -39,6 +50,10 @@ export function TranscriptSection({ const [error, setError] = useState(null); const [anchor, setAnchor] = useState(null); const [focus, setFocus] = useState(null); + // null until the name is edited, so the offered name follows the timelines. + const [shortName, setShortName] = useState(null); + const [shortX, setShortX] = useState(0.5); + const [nameError, setNameError] = useState(null); // Set once the transcript has been asked for; `latest` numbers requests so a // response for an older clip window is dropped instead of shown. const wanted = useRef(false); @@ -50,7 +65,9 @@ export function TranscriptSection({ setLoading(true); setError(null); try { - const res = await fetch(`/api/timeline/transcript/${encodeURIComponent(clip.id)}`); + const res = await fetch( + withTimeline(`/api/timeline/transcript/${encodeURIComponent(clip.id)}`, timeline), + ); const json = (await res.json()) as ClipTranscript & { error?: string }; if (request !== latest.current) return; if (!res.ok) throw new Error(json.error ?? `HTTP ${res.status}`); @@ -60,7 +77,7 @@ export function TranscriptSection({ } finally { if (request === latest.current) setLoading(false); } - }, [clip.id]); + }, [clip.id, timeline]); // A cut or trim changes the clip's window: reload a transcript that was asked // for, even one still loading (its response is then dropped as outdated). The @@ -83,6 +100,33 @@ export function TranscriptSection({ from === null || to === null ? 0 : words.filter((w) => w.index >= from && w.index <= to).length; const inRetake = (index: number): boolean => retakes.some((r) => index >= r.fromIndex && index <= r.toIndex); + const name = shortName ?? nextShortId; + + const makeShort = async (): Promise => { + if (from === null || to === null || !transcript) return; + const id = name.trim(); + if (!isTimelineId(id)) { + setNameError('Name it with lowercase letters, digits and dashes, like best-1.'); + return; + } + if (id === MAIN_TIMELINE || id === timeline) { + setNameError(`A short needs a name of its own, not ${id}.`); + return; + } + const made = await onMakeShort(id, { + verb: 'make_short', + ranges: [ + { clipId: clip.id, fromIndex: from, toIndex: to, transcriptId: transcript.transcriptId }, + ], + source: timeline, + x: shortX, + }); + if (made) { + setAnchor(null); + setFocus(null); + setShortName(null); + } + }; return (
@@ -182,6 +226,45 @@ export function TranscriptSection({ ) : null}
+ {selectedCount > 0 ? ( +
+ + + {nameError ?
{nameError}
: null} + +

+ A 9:16 short of these words with captions, at -14 LUFS, its edges in the pauses. It + opens when made; Undo there takes it apart. +

+
+ ) : null}

Click a word to move the playhead; shift-click to select a range. Undo restores.

diff --git a/src/ui/web/src/components/Viewer.tsx b/src/ui/web/src/components/Viewer.tsx index 5782193..ddcbf55 100644 --- a/src/ui/web/src/components/Viewer.tsx +++ b/src/ui/web/src/components/Viewer.tsx @@ -1,4 +1,5 @@ import { useEffect, useRef, useState } from 'react'; +import { withTimeline } from '../lib/timelines.js'; interface Exportable { exportable: boolean; @@ -16,7 +17,7 @@ type FrameState = * from the export. The playhead is debounced so scrubbing doesn't fire an FFmpeg * render per pixel, and a banner surfaces whether the document can render at all. */ -export function Viewer({ atSec, rev }: { atSec: number; rev: number }) { +export function Viewer({ timeline, atSec, rev }: { timeline: string; atSec: number; rev: number }) { const [debouncedAt, setDebouncedAt] = useState(atSec); const [status, setStatus] = useState(null); const [frame, setFrame] = useState({ kind: 'loading' }); @@ -30,7 +31,7 @@ export function Viewer({ atSec, rev }: { atSec: number; rev: number }) { // Re-check exportability whenever the document changes (rev cache-busts). useEffect(() => { let alive = true; - fetch(`/api/timeline/exportable?rev=${rev}`) + fetch(withTimeline(`/api/timeline/exportable?rev=${rev}`, timeline)) .then((r) => r.json() as Promise) .then((s) => { if (alive) setStatus(s); @@ -41,13 +42,13 @@ export function Viewer({ atSec, rev }: { atSec: number; rev: number }) { return () => { alive = false; }; - }, [rev]); + }, [timeline, rev]); // Fetch the composited frame as a blob (not via ) so a superseded // scrub is ABORTED instead of left rendering, and a transient 503 reads as a // clean message rather than a broken image. The previous frame stays on screen // until the next one decodes — no blank flash while FFmpeg works. - const src = `/api/timeline/frame?at=${debouncedAt.toFixed(2)}&rev=${rev}`; + const src = withTimeline(`/api/timeline/frame?at=${debouncedAt.toFixed(2)}&rev=${rev}`, timeline); useEffect(() => { const controller = new AbortController(); fetch(src, { signal: controller.signal }) diff --git a/src/ui/web/src/hooks/useTimeline.ts b/src/ui/web/src/hooks/useTimeline.ts index e815df4..8fe93fa 100644 --- a/src/ui/web/src/hooks/useTimeline.ts +++ b/src/ui/web/src/hooks/useTimeline.ts @@ -1,4 +1,5 @@ -import { useCallback, useEffect, useState } from 'react'; +import { useCallback, useEffect, useRef, useState } from 'react'; +import { withTimeline } from '../lib/timelines.js'; import type { Composition } from '../types.js'; const EMPTY: Composition = { @@ -14,34 +15,49 @@ const EMPTY: Composition = { }; /** - * Polls `/api/timeline` (the CompositionDoc — the source of truth for assembled - * edits) every `intervalMs`. Mirrors `useSession`: returns the latest doc plus - * loading/error state and a `refresh()` for callers that just mutated it (e.g. - * after a verb POST) and want the view to update without waiting for the poll. + * Polls `/api/timeline` for `timeline` (the CompositionDoc — the source of truth + * for assembled edits) every `intervalMs`. Mirrors `useSession`: returns the + * latest doc plus loading/error state and a `refresh()` for callers that just + * mutated it (e.g. after a verb POST) and want the view to update without + * waiting for the poll. `missing` is set when the timeline no longer exists. */ -export function useTimeline(intervalMs = 2000): { +export function useTimeline( + timeline: string, + intervalMs = 2000, +): { composition: Composition; loading: boolean; error: string | null; + missing: boolean; refresh: () => Promise; } { - const [composition, setComposition] = useState(EMPTY); + // Kept with the timeline it belongs to, so a switch shows nothing of the last one. + const [loaded, setLoaded] = useState<{ timeline: string; doc: Composition } | null>(null); const [loading, setLoading] = useState(true); const [error, setError] = useState(null); + // The timeline whose fetch last answered 404, so another one never inherits it. + const [missingTimeline, setMissingTimeline] = useState(null); + // A response for a timeline switched away from is dropped, not shown. + const wanted = useRef(timeline); + wanted.current = timeline; const load = useCallback(async () => { try { - const res = await fetch('/api/timeline'); + const res = await fetch(withTimeline('/api/timeline', timeline)); + if (wanted.current !== timeline) return; + setMissingTimeline(res.status === 404 ? timeline : null); if (!res.ok) throw new Error(`HTTP ${res.status}`); const json = (await res.json()) as { document: Composition }; - setComposition(json.document); + if (wanted.current !== timeline) return; + setLoaded({ timeline, doc: json.document }); setError(null); setLoading(false); } catch (err) { + if (wanted.current !== timeline) return; setError(err instanceof Error ? err.message : String(err)); setLoading(false); } - }, []); + }, [timeline]); useEffect(() => { void load(); @@ -49,5 +65,6 @@ export function useTimeline(intervalMs = 2000): { return () => window.clearInterval(timer); }, [intervalMs, load]); - return { composition, loading, error, refresh: load }; + const composition = loaded?.timeline === timeline ? loaded.doc : EMPTY; + return { composition, loading, error, missing: missingTimeline === timeline, refresh: load }; } diff --git a/src/ui/web/src/hooks/useTimelines.ts b/src/ui/web/src/hooks/useTimelines.ts new file mode 100644 index 0000000..179d8e8 --- /dev/null +++ b/src/ui/web/src/hooks/useTimelines.ts @@ -0,0 +1,48 @@ +import { useCallback, useEffect, useState } from 'react'; +import { isTimelineId, MAIN_TIMELINE, type TimelineListing } from '../lib/timelines.js'; + +/** The timeline the page's `?timeline=` names, so a reload keeps it. */ +function timelineInUrl(): string { + const id = new URLSearchParams(window.location.search).get('timeline'); + return id && isTimelineId(id) ? id : MAIN_TIMELINE; +} + +/** + * The workspace's timelines (main and every short), polled like the document, + * and the one being edited, kept in the page URL. + */ +export function useTimelines(intervalMs = 2000): { + timelines: TimelineListing[]; + current: string; + select: (id: string) => void; + refresh: () => Promise; +} { + const [timelines, setTimelines] = useState([]); + const [current, setCurrent] = useState(timelineInUrl); + + const load = useCallback(async () => { + try { + const res = await fetch('/api/timelines'); + if (!res.ok) return; + setTimelines(((await res.json()) as { timelines: TimelineListing[] }).timelines); + } catch { + // The next poll tries again; the tabs keep the last list meanwhile. + } + }, []); + + useEffect(() => { + void load(); + const timer = window.setInterval(load, intervalMs); + return () => window.clearInterval(timer); + }, [intervalMs, load]); + + const select = useCallback((id: string) => { + setCurrent(id); + const url = new URL(window.location.href); + if (id === MAIN_TIMELINE) url.searchParams.delete('timeline'); + else url.searchParams.set('timeline', id); + window.history.replaceState(null, '', url); + }, []); + + return { timelines, current, select, refresh: load }; +} diff --git a/src/ui/web/src/lib/timelines.ts b/src/ui/web/src/lib/timelines.ts new file mode 100644 index 0000000..683591a --- /dev/null +++ b/src/ui/web/src/lib/timelines.ts @@ -0,0 +1,34 @@ +/** The timeline at `composition.json`; every other timeline is a short. */ +export const MAIN_TIMELINE = 'main'; + +/** Mirrors TimelineIdSchema in src/timeline/schema.ts. */ +const TIMELINE_ID = /^[a-z0-9][a-z0-9-]{0,47}$/; + +/** Mirrors TimelineListing in src/timeline/document-store.ts. */ +export type TimelineListing = + | { id: string; width: number; height: number; durationSec: number; rev: number } + | { id: string; error: string }; + +export function isTimelineId(id: string): boolean { + return TIMELINE_ID.test(id); +} + +/** An API path for the given timeline: every /api/timeline route reads `?timeline=`. */ +export function withTimeline(path: string, timeline: string): string { + return `${path}${path.includes('?') ? '&' : '?'}timeline=${encodeURIComponent(timeline)}`; +} + +/** `16:9`, `9:16`, `1:1`, `4:5`… from the canvas size. */ +export function shapeLabel(width: number, height: number): string { + const gcd = (a: number, b: number): number => (b === 0 ? a : gcd(b, a % b)); + const d = gcd(width, height); + const [w, h] = [width / d, height / d]; + return w <= 32 && h <= 32 ? `${w}:${h}` : `${width}×${height}`; +} + +/** The first `short-N` no timeline uses yet. */ +export function nextShortId(ids: readonly string[]): string { + let n = 1; + while (ids.includes(`short-${n}`)) n++; + return `short-${n}`; +} diff --git a/src/ui/web/src/lib/verbs.ts b/src/ui/web/src/lib/verbs.ts index e528cf5..e3a0ecd 100644 --- a/src/ui/web/src/lib/verbs.ts +++ b/src/ui/web/src/lib/verbs.ts @@ -1,4 +1,5 @@ import type { Composition, Fit } from '../types.js'; +import { withTimeline } from './timelines.js'; /** A timeline editing verb (mirrors CompositionVerb in src/timeline/verbs.ts). * Loosely typed at the field level — the server validates with Zod, which is the @@ -44,26 +45,39 @@ export type Verb = kind?: string; durationSec?: number; track?: string; + } + | { + verb: 'make_short'; + ranges: { clipId: string; fromIndex: number; toIndex: number; transcriptId?: string }[]; + source?: string; + x?: number; }; /** - * POST editing verbs to /api/timeline/verbs — the same op-aware, undoable path - * the agent and CLI use. Throws with the server's message on a 4xx/5xx (e.g. a - * 422 for a bad clip reference or a 403 for a workspace-boundary violation) so - * the caller can surface it. `applied` is the op count; 0 means the verbs changed - * nothing (e.g. remove_silence found no pause). + * POST editing verbs to /api/timeline/verbs for `timeline` — the same op-aware, + * undoable path the agent and CLI use. Throws with the server's message on a + * 4xx/5xx (e.g. a 422 for a bad clip reference or a 403 for a workspace-boundary + * violation) so the caller can surface it. `applied` is the op count; 0 means the + * verbs changed nothing (e.g. remove_silence found no pause). `warnings` name what + * a verb did that may not have been meant (a short cut mid-sentence). */ export async function applyTimelineVerbs( verbs: Verb[], -): Promise<{ document: Composition; applied: number }> { - const res = await fetch('/api/timeline/verbs', { + timeline: string, +): Promise<{ document: Composition; applied: number; warnings: string[] }> { + const res = await fetch(withTimeline('/api/timeline/verbs', timeline), { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ verbs }), }); - const json = (await res.json()) as { document?: Composition; applied?: number; error?: string }; + const json = (await res.json()) as { + document?: Composition; + applied?: number; + warnings?: string[]; + error?: string; + }; if (!res.ok || !json.document) { throw new Error(json.error ?? `HTTP ${res.status}`); } - return { document: json.document, applied: json.applied ?? 0 }; + return { document: json.document, applied: json.applied ?? 0, warnings: json.warnings ?? [] }; } diff --git a/src/ui/web/src/styles.css b/src/ui/web/src/styles.css index 94d3213..4bbb062 100644 --- a/src/ui/web/src/styles.css +++ b/src/ui/web/src/styles.css @@ -918,6 +918,113 @@ body, /* ─── Doc timeline — the CompositionDoc (the editor's centerpiece) ────── */ +/* ─── Timeline tabs: main and every short ─────────────────────────── */ + +.tl-tabs-bar { + flex: 0 0 auto; + background: var(--bg-1); + border-top: 1px solid var(--border-2); +} + +.tl-tabs { + display: flex; + align-items: flex-end; + gap: 0.75rem; + padding: 0.4rem 1.1rem 0; +} + +.tl-tabs-list { + display: flex; + gap: 0.25rem; + min-width: 0; + overflow-x: auto; +} + +.tl-tab { + display: flex; + align-items: center; + border: 1px solid var(--border); + border-bottom: none; + border-radius: var(--r-sm) var(--r-sm) 0 0; + background: var(--bg); + color: var(--text-2); + white-space: nowrap; +} + +.tl-tab.is-current { + background: #181b21; + border-color: var(--border-2); + color: var(--text); +} + +.tl-tab-select { + font: inherit; + font-size: 0.82rem; + display: flex; + align-items: center; + gap: 0.4rem; + padding: 0.35rem 0.4rem 0.35rem 0.7rem; + background: none; + border: none; + color: inherit; + cursor: pointer; +} + +.tl-tab-shape { + font-family: var(--mono); + font-size: 0.68rem; + padding: 0.05rem 0.3rem; + border: 1px solid var(--border-2); + border-radius: 3px; + color: var(--text-2); +} + +.tl-tab.is-current .tl-tab-shape { + border-color: var(--accent); + color: var(--accent-hi); +} + +.tl-tab-delete { + font: inherit; + font-size: 0.72rem; + padding: 0.2rem 0.5rem 0.2rem 0.2rem; + background: none; + border: none; + color: var(--text-3); + cursor: pointer; +} + +.tl-tab-delete:hover { + color: var(--danger); +} + +.tl-export-all { + margin: 0 0 0.3rem auto; + font-size: 0.8rem; + padding: 0.3rem 0.75rem; + white-space: nowrap; +} + +.tl-notice { + display: flex; + align-items: flex-start; + gap: 0.5rem; + padding: 0.4rem 1.1rem; + border-top: 1px solid var(--border); + font-size: 0.8rem; + color: var(--text-2); +} + +.tl-notice.is-warn { + color: var(--warn); +} + +.tl-notice-lines { + flex: 1; + min-width: 0; + overflow-wrap: anywhere; +} + .doc-timeline { background: linear-gradient(180deg, #181b21, #141619); border-top: 1px solid var(--border-2); @@ -1363,6 +1470,24 @@ body, gap: 0.5rem; } +.ci-short { + display: flex; + flex-direction: column; + gap: 0.45rem; + padding: 0.6rem; + border: 1px solid var(--border-2); + border-radius: var(--r-sm); + background: var(--bg-2); +} + +.ci-short-name { + flex: 1; + min-width: 0; + padding: 0.3rem 0.5rem; + font-family: var(--mono); + font-size: 0.82rem; +} + .ci-remove { align-self: flex-start; font: inherit; diff --git a/tests/integration/timeline-export-all.test.ts b/tests/integration/timeline-export-all.test.ts new file mode 100644 index 0000000..15c87fc --- /dev/null +++ b/tests/integration/timeline-export-all.test.ts @@ -0,0 +1,89 @@ +import { mkdtemp, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { runTimeline } from '../../src/cli-timeline.js'; +import { probe } from '../../src/ffmpeg/probe.js'; +import { runFfmpeg } from '../../src/ffmpeg/run.js'; +import type { TimelineExport } from '../../src/timeline/run-plan.js'; + +// `clip timeline export --all` renders every short, each in its own shape and +// at its own loudness, and leaves main out. +const IS_CI = process.env.CI === 'true'; + +let workspace: string; +let saved: string | undefined; + +beforeEach(async () => { + workspace = await mkdtemp(join(tmpdir(), 'mmc-export-all-')); + saved = process.env.MAKEMYCLIP_WORKSPACE; + process.env.MAKEMYCLIP_WORKSPACE = workspace; +}); + +afterEach(async () => { + vi.restoreAllMocks(); + process.exitCode = undefined; + if (saved === undefined) delete process.env.MAKEMYCLIP_WORKSPACE; + else process.env.MAKEMYCLIP_WORKSPACE = saved; + await rm(workspace, { recursive: true, force: true }); +}); + +async function cli>(args: string[]): Promise { + let printed = ''; + const spy = vi.spyOn(process.stdout, 'write').mockImplementation((chunk) => { + printed += String(chunk); + return true; + }); + try { + await runTimeline(args); + } finally { + spy.mockRestore(); + } + return JSON.parse(printed) as T; +} + +describe.skipIf(!IS_CI)('export --all against real FFmpeg', () => { + it('renders each short in its own shape at -14 LUFS, and reports the one that cannot', async () => { + const source = join(workspace, 'talk.mp4'); + await runFfmpeg([ + ...['-hide_banner', '-loglevel', 'error'], + ...['-f', 'lavfi', '-i', 'testsrc=size=1280x720:rate=30:duration=6'], + ...['-f', 'lavfi', '-i', 'sine=frequency=220:sample_rate=48000:duration=6'], + ...['-af', 'volume=0.05', '-c:v', 'libx264', '-preset', 'ultrafast', '-pix_fmt', 'yuv420p'], + ...['-c:a', 'aac', '-shortest', source], + ]); + await cli(['add-media', source]); + for (const [id, width, height, from, to] of [ + ['best-1', '1080', '1920', '0', '2'], + ['best-2', '1080', '1080', '3', '5'], + ] as const) { + await cli(['--timeline', id, 'new', '--width', width, '--height', height, '--fit', 'cover']); + await cli(['--timeline', id, 'add-media', source, from, to]); + await cli(['--timeline', id, 'loudness', '-14']); + } + await cli(['--timeline', 'empty', 'new']); + + const dir = join(workspace, 'shorts'); + const { exported } = await cli<{ exported: TimelineExport[] }>(['export', '--all', dir]); + expect(process.exitCode).toBe(1); + expect(exported.map((e) => e.timeline)).toEqual(['best-1', 'best-2', 'empty']); + expect(exported[2]).toEqual({ + timeline: 'empty', + error: 'Nothing to export: no video track has clips.', + }); + + for (const [i, [width, height]] of [ + [1080, 1920], + [1080, 1080], + ].entries()) { + const result = exported[i]; + if (!result || 'error' in result) throw new Error(`short ${i} failed: ${result?.error}`); + expect(result.output).toBe(join(dir, `best-${i + 1}.mp4`)); + const media = await probe(result.output); + expect([media.video?.width, media.video?.height]).toEqual([width, height]); + expect(media.durationSec).toBeCloseTo(2, 0); + expect(result.loudness?.outputLufs).toBeGreaterThan(-15); + expect(result.loudness?.outputLufs).toBeLessThan(-13); + } + }, 60_000); +}); diff --git a/tests/timeline-export-all.test.ts b/tests/timeline-export-all.test.ts new file mode 100644 index 0000000..38570d2 --- /dev/null +++ b/tests/timeline-export-all.test.ts @@ -0,0 +1,77 @@ +import { mkdtemp, rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { runTimeline } from '../src/cli-timeline.js'; +import { emptyComposition } from '../src/timeline/composition.js'; +import { resetComposition } from '../src/timeline/document-store.js'; +import { exportTimelines } from '../src/timeline/run-plan.js'; + +// `clip timeline export --all` and the `exportTimelines` it shares with the UI, +// on the paths that run no FFmpeg; tests/integration renders real shorts. + +let workspace: string; +let saved: string | undefined; +let printed: string; + +beforeEach(async () => { + workspace = await mkdtemp(join(tmpdir(), 'mmc-export-all-')); + saved = process.env.MAKEMYCLIP_WORKSPACE; + process.env.MAKEMYCLIP_WORKSPACE = workspace; + printed = ''; + vi.spyOn(process.stdout, 'write').mockImplementation((chunk) => { + printed += String(chunk); + return true; + }); +}); + +afterEach(async () => { + vi.restoreAllMocks(); + process.exitCode = undefined; + if (saved === undefined) delete process.env.MAKEMYCLIP_WORKSPACE; + else process.env.MAKEMYCLIP_WORKSPACE = saved; + await rm(workspace, { recursive: true, force: true }); +}); + +describe('exportTimelines', () => { + it('reports a timeline that cannot render and goes on to the next', async () => { + await resetComposition(emptyComposition({ width: 1080, height: 1920 }), 'best-1'); + const results = await exportTimelines(['best-1', 'gone'], join(workspace, 'out')); + expect(results).toEqual([ + { timeline: 'best-1', error: 'Nothing to export: no video track has clips.' }, + { timeline: 'gone', error: expect.stringContaining('No timeline "gone"') }, + ]); + }); +}); + +describe('clip timeline export --all', () => { + it('exports every short but main, and fails the run when one cannot render', async () => { + await resetComposition(emptyComposition({ width: 1080, height: 1920 }), 'best-1'); + await runTimeline(['export', '--all', join(workspace, 'shorts')]); + expect(JSON.parse(printed)).toEqual({ + exported: [{ timeline: 'best-1', error: 'Nothing to export: no video track has clips.' }], + }); + expect(process.exitCode).toBe(1); + }); + + it('needs a folder and at least one short', async () => { + await expect(runTimeline(['export', '--all'])).rejects.toThrow( + 'Usage: clip timeline export --all ', + ); + await expect(runTimeline(['export', '--all', join(workspace, 'shorts')])).rejects.toThrow( + 'No shorts to export.', + ); + }); + + it('refuses --timeline and project formats, which export one timeline', async () => { + await resetComposition(emptyComposition(), 'best-1'); + for (const extra of [ + ['--timeline', 'best-1'], + ['--format', 'fcpxml'], + ]) { + await expect(runTimeline(['export', '--all', 'shorts', ...extra])).rejects.toThrow( + '--all renders every short as mp4.', + ); + } + }); +}); diff --git a/tests/ui-timelines.test.ts b/tests/ui-timelines.test.ts new file mode 100644 index 0000000..be3c613 --- /dev/null +++ b/tests/ui-timelines.test.ts @@ -0,0 +1,215 @@ +import { mkdtemp, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; +import { appendOp } from '../src/session/store.js'; +import { emptyComposition } from '../src/timeline/composition.js'; +import { + listTimelines, + mutateComposition, + readComposition, + resetComposition, +} from '../src/timeline/document-store.js'; +import { colorClip, mediaClip, videoTrack } from '../src/timeline/ops.js'; +import type { MediaId } from '../src/timeline/schema.js'; +import { startUiServer, type UiServer } from '../src/ui/server.js'; + +// The `clip ui` routes for named timelines: every /api/timeline route takes +// `?timeline=`, and /api/timelines lists, deletes and exports shorts. None of +// these cases runs FFmpeg; the integration test renders the shorts. + +let server: UiServer; +let workspace: string; +let savedWorkspace: string | undefined; + +beforeAll(async () => { + savedWorkspace = process.env.MAKEMYCLIP_WORKSPACE; + server = await startUiServer({ openBrowser: false }); +}); + +afterAll(async () => { + await server.stop(); + if (savedWorkspace === undefined) delete process.env.MAKEMYCLIP_WORKSPACE; + else process.env.MAKEMYCLIP_WORKSPACE = savedWorkspace; +}); + +beforeEach(async () => { + if (workspace) await rm(workspace, { recursive: true, force: true }); + workspace = await mkdtemp(join(tmpdir(), 'mmc-ui-timelines-')); + process.env.MAKEMYCLIP_WORKSPACE = workspace; +}); + +async function call( + method: string, + path: string, + body?: unknown, +): Promise<{ status: number; json: Record }> { + const res = await fetch(`${server.url}${path}`, { + method, + headers: { 'Content-Type': 'application/json' }, + ...(body === undefined ? {} : { body: JSON.stringify(body) }), + }); + return { status: res.status, json: (await res.json()) as Record }; +} + +/** A 9:16 short holding one color clip, as make_short would leave it. */ +async function makeShort(id: string): Promise { + await mutateComposition( + [ + { op: 'setCanvas', width: 1080, height: 1920 }, + { op: 'addTrack', track: videoTrack({ id: 'v0' }) }, + { + op: 'addClip', + trackId: 'v0', + clip: colorClip({ id: 'c1', color: 'navy', durationSec: 2, startSec: 0 }), + }, + ], + { timeline: id, create: true }, + ); +} + +describe('?timeline= on the /api/timeline routes', () => { + it('reads main by default and the named short otherwise', async () => { + await makeShort('best-1'); + const main = await call('GET', '/api/timeline'); + const short = await call('GET', '/api/timeline?timeline=best-1'); + expect(main.status).toBe(200); + expect((main.json.document as { width: number }).width).toBe(1920); + expect((short.json.document as { width: number }).width).toBe(1080); + }); + + it('edits, undoes and reports history on the named short only', async () => { + await makeShort('best-1'); + const mainRev = (await readComposition()).rev; + + const edit = await call('POST', '/api/timeline/verbs?timeline=best-1', { + verbs: [{ verb: 'set_loudness', targetLufs: -16 }], + }); + expect(edit.status).toBe(200); + expect(edit.json.applied).toBe(1); + expect(edit.json.warnings).toEqual([]); + expect((await readComposition('best-1')).loudness).toEqual({ targetLufs: -16 }); + expect((await readComposition()).rev).toBe(mainRev); + + expect((await call('GET', '/api/timeline/history')).json).toEqual({ + canUndo: false, + canRedo: false, + }); + expect((await call('GET', '/api/timeline/history?timeline=best-1')).json.canUndo).toBe(true); + + expect((await call('POST', '/api/timeline/undo?timeline=best-1')).json.undone).toBe(true); + expect((await readComposition('best-1')).loudness).not.toEqual({ targetLufs: -16 }); + }); + + it('answers 400 for a malformed id and 404 for a short that does not exist', async () => { + for (const path of [ + '/api/timeline?timeline=Best%201', + '/api/timeline/history?timeline=..', + '/api/timeline/exportable?timeline=a%2Fb', + ]) { + expect((await call('GET', path)).status, path).toBe(400); + } + for (const [method, path] of [ + ['GET', '/api/timeline?timeline=nope'], + ['GET', '/api/timeline/history?timeline=nope'], + ['GET', '/api/timeline/exportable?timeline=nope'], + ['GET', '/api/timeline/frame?at=0&timeline=nope'], + ['GET', '/api/timeline/transcript/c1?timeline=nope'], + ['POST', '/api/timeline/undo?timeline=nope'], + ['POST', '/api/timeline/redo?timeline=nope'], + ['POST', '/api/timeline/export?timeline=nope'], + ] as const) { + const reply = await call(method, path, method === 'POST' ? {} : undefined); + expect(reply.status, path).toBe(404); + expect(reply.json.error, path).toContain('No timeline "nope"'); + } + const edit = await call('POST', '/api/timeline/verbs?timeline=nope', { + verbs: [{ verb: 'set_loudness', targetLufs: -16 }], + }); + expect(edit.status).toBe(404); + expect(await listTimelines()).toHaveLength(1); + }); +}); + +describe('/api/timelines', () => { + it('lists main first, then every short with its shape', async () => { + await makeShort('best-2'); + await makeShort('best-1'); + const { status, json } = await call('GET', '/api/timelines'); + expect(status).toBe(200); + expect(json.timelines).toEqual([ + { id: 'main', width: 1920, height: 1080, durationSec: 0, rev: 0 }, + { id: 'best-1', width: 1080, height: 1920, durationSec: 2, rev: 1 }, + { id: 'best-2', width: 1080, height: 1920, durationSec: 2, rev: 1 }, + ]); + }); + + it('deletes a short, refuses main, and answers 404 for one that is gone', async () => { + await makeShort('best-1'); + expect(await call('DELETE', '/api/timelines/best-1')).toEqual({ + status: 200, + json: { deleted: 'best-1' }, + }); + expect((await listTimelines()).map((t) => t.id)).toEqual(['main']); + expect((await call('DELETE', '/api/timelines/best-1')).status).toBe(404); + + const main = await call('DELETE', '/api/timelines/main'); + expect(main.status).toBe(400); + expect(main.json.error).toContain("main timeline can't be deleted"); + expect((await call('DELETE', '/api/timelines/Not%20an%20id')).status).toBe(400); + }); + + it('refuses to export when there are no shorts', async () => { + const reply = await call('POST', '/api/timelines/export', {}); + expect(reply.status).toBe(422); + expect(reply.json.error).toContain('No shorts to export yet'); + }); + + it('reports a short that cannot render, without logging it as an output', async () => { + await resetComposition(emptyComposition({ width: 1080, height: 1920 }), 'best-1'); + const reply = await call('POST', '/api/timelines/export', {}); + expect(reply.status).toBe(200); + expect(reply.json.dir).toMatch(/shorts-[0-9a-f]{8}$/); + expect(reply.json.exported).toEqual([ + { timeline: 'best-1', error: 'Nothing to export: no video track has clips.' }, + ]); + const session = await call('GET', '/api/session'); + expect(session.json.entries).toEqual([]); + }); + + it('refuses a short whose media was swapped for a link out of the workspace', async () => { + const outside = await mkdtemp(join(tmpdir(), 'mmc-ui-outside-')); + try { + await writeFile(join(outside, 'private.mp4'), 'not for the export'); + const linked = join(workspace, 'talk.mp4'); + await symlink(join(outside, 'private.mp4'), linked); + await appendOp({ + tool: 'ingest', + args: { path: linked }, + result: { mediaId: 'm_0123456789ab', ref: { path: linked, audio: null, video: null } }, + }); + await mutateComposition( + [ + { op: 'addTrack', track: videoTrack({ id: 'v0' }) }, + { + op: 'addClip', + trackId: 'v0', + clip: mediaClip({ + id: 'c1', + mediaId: 'm_0123456789ab' as MediaId, + sourceOutSec: 2, + startSec: 0, + }), + }, + ], + { timeline: 'best-1', create: true }, + ); + const reply = await call('POST', '/api/timelines/export', {}); + expect(reply.json.exported).toEqual([ + { timeline: 'best-1', error: expect.stringContaining('outside the workspace') }, + ]); + } finally { + await rm(outside, { recursive: true, force: true }); + } + }); +});