Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,10 +176,13 @@ clip timeline short best-1 clip_ec2f047d 1400 1478 --x 0.4 # words 1400–1478
clip timeline --timeline best-1 frame 3 check.jpg # look at it; move --x if the speaker is off
clip timeline --timeline best-1 export best-1.mp4
clip timeline list # main and every short
clip timeline export --all ./shorts # every short, as ./shorts/<id>.mp4
```

`short` cuts each edge inside the pause the audio shows there, keeping 0.1 s of room tone (where there's no clear pause it follows the transcript and warns), fills the 9:16 frame (`--x` picks the part that shows), sets -14 LUFS and burns in captions. Give several `<clipId> <from> <to>` ranges to join moments, in the order you give them. Any command takes `--timeline <id>` to work on a short, `clip timeline --timeline best-1 undo` takes the short apart again, and `short best-1 … --replace` rebuilds it.

In `clip ui`, select words in a clip's transcript and press **Make short**: the short opens in a tab of its own above the timeline, next to main. **Export all shorts** renders every one.

---

## Another example: a product demo from screen recordings
Expand Down
2 changes: 1 addition & 1 deletion SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@ To make a vertical (9:16), square (1:1) or feed (4:5) version, run `timeline ref

To even out the volume of the exported mp4, run `timeline loudness -14` (YouTube, Shorts, Reels, TikTok) or `-16` (podcasts); `off` turns it off. Export then measures the finished mp4 and normalizes it to that integrated loudness in a second pass (EBU R128, true peak at most -1.5 dBTP), and reports `loudness` with the measured before and after values. `mode: dynamic` means loudnorm couldn't apply one gain (steady or very short audio, a wide loudness range, or peaks); `skipped` means the audio was silent or too short to measure. It's one setting on the timeline that `timeline undo` reverses, it applies to the mp4 only, and it doesn't even out clips recorded at different levels.

To cut vertical shorts out of a long recording, transcribe it and make the cuts you want everywhere (pauses, retakes) on the main timeline first, then read `timeline transcribe --sentences`, pick each moment as whole sentences and run `timeline short <id> <clipId> <fromIndex> <toIndex>` with their word indices, where `<id>` is a new name such as `best-1`. Pick whole sentences, 15–60 s in all; you choose the moments, the editor doesn't score them. The short is a timeline of its own, cut from main's clips (main's cuts carry in, main itself doesn't change): it cuts each edge inside the pause the audio shows there, keeping 0.1 s of room tone, so it never keeps the next sentence's first syllable, fills a 9:16 frame, sets -14 LUFS and burns in captions. More `<clipId> <from> <to>` triples join several moments in the order given, so a later line can open the short as its hook. Options: `--x 0..1` to keep the speaker in frame, `--aspect 1:1|4:5`, `--words N` per caption, `--no-captions`, `--loudness <LUFS|off>`, `--source <timeline>`, `--dry-run` (prints the pieces), `--replace` to rebuild it. Its `warnings` name a range that starts or ends mid-sentence (widen it to the sentence's edge unless the cut is meant) and an edge with no clear pause in the audio (music under the speech, or no break), which follows the transcript instead: look at that edge or listen to it. Every other command works on a short with `--timeline <id>`: look at it with `timeline --timeline best-1 frame 3 check.jpg`, adjust with `timeline --timeline best-1 reframe 9:16 --x 0.3`, and render it with `timeline --timeline best-1 export best-1.mp4`. `timeline list` shows every timeline, `timeline --timeline best-1 undo` takes the short apart, and `timeline delete best-1` removes it. Over MCP, it's the `make_short` verb in `timeline_edit` with `timeline` set to the new id; every timeline tool takes `timeline`, and `timeline_show` lists them all.
To cut vertical shorts out of a long recording, transcribe it and make the cuts you want everywhere (pauses, retakes) on the main timeline first, then read `timeline transcribe --sentences`, pick each moment as whole sentences and run `timeline short <id> <clipId> <fromIndex> <toIndex>` with their word indices, where `<id>` is a new name such as `best-1`. Pick whole sentences, 15–60 s in all; you choose the moments, the editor doesn't score them. The short is a timeline of its own, cut from main's clips (main's cuts carry in, main itself doesn't change): it cuts each edge inside the pause the audio shows there, keeping 0.1 s of room tone, so it never keeps the next sentence's first syllable, fills a 9:16 frame, sets -14 LUFS and burns in captions. More `<clipId> <from> <to>` triples join several moments in the order given, so a later line can open the short as its hook. Options: `--x 0..1` to keep the speaker in frame, `--aspect 1:1|4:5`, `--words N` per caption, `--no-captions`, `--loudness <LUFS|off>`, `--source <timeline>`, `--dry-run` (prints the pieces), `--replace` to rebuild it. Its `warnings` name a range that starts or ends mid-sentence (widen it to the sentence's edge unless the cut is meant) and an edge with no clear pause in the audio (music under the speech, or no break), which follows the transcript instead: look at that edge or listen to it. Every other command works on a short with `--timeline <id>`: look at it with `timeline --timeline best-1 frame 3 check.jpg`, adjust with `timeline --timeline best-1 reframe 9:16 --x 0.3`, and render it with `timeline --timeline best-1 export best-1.mp4` (`timeline export --all <dir>` renders every short as `<dir>/<id>.mp4`). `timeline list` shows every timeline, `timeline --timeline best-1 undo` takes the short apart, and `timeline delete best-1` removes it. Over MCP, it's the `make_short` verb in `timeline_edit` with `timeline` set to the new id; every timeline tool takes `timeline`, and `timeline_show` lists them all.

To hand the cut to an editor app, export the timeline as an editor project instead of a render. `timeline export cut.fcpxml` (or `--format fcpxml`) writes FCPXML 1.10 for Final Cut Pro and DaVinci Resolve; `timeline export cut.xml` (or `--format xmeml`) writes Final Cut Pro 7 XML for Premiere Pro, which Resolve reads too. Premiere doesn't import FCPXML, so use `xmeml` for it. Both reference the source files by absolute path (nothing is copied, so keep the sources where they are), place camera files by their start timecode, and keep the mp4's timing: clips, gaps, fades, volume, fade/dissolve transitions (as cross dissolves) and black or colored cards. Text over the picture becomes titles in FCPXML; in the Premiere XML it goes to `.srt` caption files next to the XML, since Premiere drops that format's titles. What the file can't hold — speed, color adjustments, chroma key, a reframe's fill, zoom or position (set the timeline to `--fit contain` and reset clip framing to carry it), rotation, wipe/slide/circle transitions, audio-only tracks, media on two tracks, a frame rate editors don't offer — is refused with the fix to make; what it holds only in part (text styling, caption boxes) is listed in the output's `notes`.

Expand Down
9 changes: 9 additions & 0 deletions docs/backlog/confine-render-media.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Re-check media against the workspace on every render for the UI and MCP

`POST /api/timelines/export` re-checks each short's files with `confinedMediaMap`, so a registered file swapped for a symlink out of the workspace is refused. Three older render paths still take the registry as ingested (`buildMediaMap`):

- `POST /api/timeline/export` (mp4) in `src/ui/server.ts`;
- `GET /api/timeline/frame` in `src/ui/server.ts`;
- the mp4 path of MCP `timeline_export` in `src/mcp/server.ts`.

The transcript and project-file paths already confine. Pass `confinedMediaMap(comp)` on these three, and test each with a symlinked file, as `tests/ui-timelines.test.ts` does for export-all. Found by the Codex review of the UI shorts PR (2026-11-15).
5 changes: 5 additions & 0 deletions docs/backlog/ui-inspector-height.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Clip inspector has no room on a short window

In `clip ui`, the viewer takes up to 44vh plus its bar, and the clip inspector (`.stage-detail`) gets whatever is left of `.main`. At a 1000 px tall window, `.main` is 468 px and the viewer fills all of it, so the inspector is 0 px tall: a selected clip's transcript, framing and Make short can't be reached. It works at 1500 px. Seen on 2026-11-15 while checking the shorts tabs; the layout was the same before them, and the tabs bar takes another ~30 px once a short exists.

Fix: give the inspector a minimum height (or let the viewer shrink first), and check at 900 px tall.
2 changes: 1 addition & 1 deletion llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
- **Captions:** `clip timeline captions` burns in captions timed to the words (short phrases or one word at a time); on a vertical canvas they stay clear of the Shorts, Reels and TikTok overlays, and their width follows the font size
- **Reframe:** `clip timeline reframe 9:16` turns a landscape cut vertical (also 1:1, 4:5); `--x` picks the part that shows, `clip timeline framing <clipId>` sets one clip's zoom and position
- **Loudness:** `clip timeline loudness -14` normalizes the exported mp4 to -14 LUFS (or `-16` for podcasts) in two passes
- **Shorts from a long recording:** `clip timeline short best-1 <clipId> <fromIndex> <toIndex>` (MCP: the `make_short` verb) turns a range of transcript words into a 9:16 short on a timeline of its own, captioned and at -14 LUFS, keeping the cuts made on the main timeline; `--timeline <id>` points any command at a short
- **Shorts from a long recording:** `clip timeline short best-1 <clipId> <fromIndex> <toIndex>` (MCP: the `make_short` verb) turns a range of transcript words into a 9:16 short on a timeline of its own, captioned and at -14 LUFS, keeping the cuts made on the main timeline; `--timeline <id>` points any command at a short, and `clip timeline export --all <dir>` renders every short. In `clip ui`, Make short works from a transcript selection and each short gets a tab
- **Cut check:** `clip timeline check --sheet cuts.jpg` flags cuts that may clip a word and writes a contact sheet
- **Export to pro editors:** `clip timeline export --format fcpxml` (Final Cut Pro, DaVinci Resolve) or `--format xmeml` (Premiere Pro; captions as .srt)
- Every timeline edit is one undoable step (`clip timeline undo` / `redo`), from the CLI, the MCP server or the browser UI
Expand Down
24 changes: 22 additions & 2 deletions src/cli-timeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ import {
planReframe,
} from './timeline/reframe.js';
import { reviewCuts } from './timeline/review.js';
import { runPlan } from './timeline/run-plan.js';
import { exportTimelines, runPlan } from './timeline/run-plan.js';
import { MAIN_TIMELINE, TimelineIdSchema } from './timeline/schema.js';
import { checkShortTimelines, planShort, type ShortRange } from './timeline/shorts.js';
import { analyzeSilence, DEFAULT_SENSITIVITY, SensitivitySchema } from './timeline/silence.js';
Expand Down Expand Up @@ -109,6 +109,7 @@ Every command works on the main timeline unless --timeline <id> names another
clip timeline redo
clip timeline log
clip timeline export [<output>] [--format mp4|fcpxml|xmeml]
clip timeline export --all <dir> (every short, as <dir>/<id>.mp4)
`;

interface Parsed {
Expand All @@ -117,7 +118,7 @@ interface Parsed {
}

/** Flags that take no value, so they don't swallow the next argument. */
const BOOLEAN_FLAGS = new Set(['dry-run', 'reset', 'no-captions', 'replace', 'sentences']);
const BOOLEAN_FLAGS = new Set(['all', 'dry-run', 'reset', 'no-captions', 'replace', 'sentences']);

function parseArgs(args: string[]): Parsed {
const positional: string[] = [];
Expand Down Expand Up @@ -868,6 +869,25 @@ export async function runTimeline(args: string[]): Promise<void> {
}

case 'export': {
if (flags.all === 'true') {
const [dir] = positional;
if (!dir) throw new Error('Usage: clip timeline export --all <dir>');
if (flags.timeline !== undefined || (flags.format && flags.format !== 'mp4')) {
throw new Error(
'--all renders every short as mp4. Export one timeline with --timeline <id>, or a project with --format.',
);
}
const shorts = (await listTimelines())
.map((t) => t.id)
.filter((id) => id !== MAIN_TIMELINE);
if (shorts.length === 0) {
throw new Error('No shorts to export. Make one with `clip timeline short`.');
}
const exported = await exportTimelines(shorts, resolveInput(dir));
out({ exported });
if (exported.some((e) => 'error' in e)) process.exitCode = 1;
return;
}
const [output] = positional;
const format = flags.format || (output && interchangeFormatFor(output)) || 'mp4';
if (format !== 'mp4') {
Expand Down
8 changes: 7 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,13 @@ export {
renderCutSheet,
reviewCuts,
} from './timeline/review.js';
export { type LoudnessResult, type RunPlanResult, runPlan } from './timeline/run-plan.js';
export {
exportTimelines,
type LoudnessResult,
type RunPlanResult,
runPlan,
type TimelineExport,
} from './timeline/run-plan.js';
export {
AudioStreamSchema,
MAIN_TIMELINE,
Expand Down
62 changes: 60 additions & 2 deletions src/timeline/run-plan.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
import { randomBytes } from 'node:crypto';
import { rename, unlink, writeFile } from 'node:fs/promises';
import { mkdir, rename, unlink, writeFile } from 'node:fs/promises';
import { dirname, extname, join } from 'node:path';
import {
buildLoudnormApplyArgs,
buildLoudnormMeasureArgs,
parseLoudnormJson,
} from '../ffmpeg/args/loudnorm.js';
import { filterGraphFileOption, requireDrawtext, runFfmpeg } from '../ffmpeg/run.js';
import type { FfmpegPlan } from './compile.js';
import { getWorkspace } from '../workspace.js';
import { compileTimeline, type FfmpegPlan, type MediaInfo } from './compile.js';
import type { Composition } from './composition.js';
import { readComposition } from './document-store.js';
import { buildMediaMap } from './media-registry.js';
import type { MediaId } from './schema.js';

export interface LoudnessResult {
targetLufs: number;
Expand Down Expand Up @@ -150,3 +155,56 @@ async function normalizeLoudness(
throw err;
}
}

/** A timeline `exportTimelines` rendered, or why it couldn't. */
export type TimelineExport =
| {
timeline: string;
rev: number;
output: string;
durationSec: number;
durationMs: number;
loudness?: LoudnessResult;
}
| { timeline: string; error: string };

/**
* Render each timeline to `<dir>/<id>.mp4`, one at a time. A timeline that
* can't render is reported with its error, and the rest still render. `media`
* resolves each document's files (default: the registry as ingested); callers
* acting for the UI pass one that re-checks them against the workspace.
*/
export async function exportTimelines(
timelines: readonly string[],
dir: string,
media: (comp: Composition) => Promise<Map<MediaId, MediaInfo>> = () => buildMediaMap(),
): Promise<TimelineExport[]> {
await mkdir(dir, { recursive: true });
// Intermediates are named per run, so a concurrent export or frame preview
// in the same workspace can't overwrite them.
const run = randomBytes(4).toString('hex');
const results: TimelineExport[] = [];
for (const timeline of timelines) {
try {
const comp = await readComposition(timeline);
const plan = compileTimeline(comp, {
media: await media(comp),
dir: getWorkspace(),
output: join(dir, `${timeline}.mp4`),
tag: `${run}-${timeline}`,
});
const result = await runPlan(plan);
results.push({
timeline,
rev: comp.rev,
output: result.output,
durationSec: plan.durationSec,
durationMs: result.durationMs,
...(result.loudness ? { loudness: result.loudness } : {}),
});
} catch (err) {
results.push({ timeline, error: err instanceof Error ? err.message : String(err) });
}
}
return results;
}
Loading
Loading