Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 63 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,10 @@ render: deterministic mp4 (same input → byte-identical frames) ◀── ove
```

Everything is a pure function of time: `evaluate(scene, t)` — no wall clocks,
no randomness without a seed, scrubbing and distributed rendering for free.
no randomness without a seed, scrubbing and distributed rendering for free. And
it's *enforced*: `reframe lint` compiles the scene twice and flags any IR that
differs (a `Math.random()` or `Date` baked into a prop), so a scene that would
silently render differently each time fails the gate before you ship it.

## Generative choreography at scale

Expand Down Expand Up @@ -270,8 +273,16 @@ your scene.
|---|---|
| `pnpm reframe render <scene.ts\|.json\|.html> [--overlay f]... [-o out]` | deterministic mp4 (mode inferred from extension; output defaults to `out/`) |
| `pnpm reframe batch <scene.ts> <data.json\|csv> [-o dir] [--overlay f]...` | one mp4 per data row (rows = overlays), parallel, with a per-row report |
| `pnpm reframe compile <scene.ts\|.json> [-o out.json] [--json]` | bundle + validate a scene to SceneIR JSON, no render (fast; no ffmpeg/chromium) |
| `pnpm reframe compile <scene.ts\|.json> [-o out.json] [--json]` | bundle + validate a scene to SceneIR JSON, no render (fast; no ffmpeg/chromium); `--json` returns `{ok, kind, issues}` |
| `pnpm reframe frame <scene.ts\|.json> [--t <sec>] [-o out.png]` | render one frame at time `t` to a PNG (chromium only, no mux) — for a render-and-look loop |
| `pnpm reframe manifest <scene.ts\|.json> [--json]` | dump the addressable surface: every node, state, timeline label, and beat with the overlay address that reaches it |
| `pnpm reframe lint <scene.ts\|.json> [--json] [--strict]` | the studio-readiness gate: flag un-addressable motion + verify the scene is a pure function of time; `--strict` exits non-zero |
| `pnpm reframe verify-overlay <base> <overlay>... [--json]` | compose an overlay onto a base and report applied-vs-orphaned, no render — the regen-survival check (non-zero exit on orphans) |
| `pnpm reframe labels <scene.ts\|.json>` | print the compiled event clock (every timeline label → exact seconds) — the timing source for audio cues |
| `pnpm reframe assemble <media...> [-o name]` | probe images/videos (ffprobe) and scaffold an editable montage scene `.ts` wired with `photoMontage` |
| `pnpm reframe player <scene.ts\|.json> [-o out.html]` | bundle a scene into one self-contained HTML that plays the motion live in any browser |
| `pnpm reframe logo <logo.svg\|brand-slug> [--motion <preset>]` | animate a logo (or a simple-icons brand) into a sting |
| `pnpm reframe diff <ref-image> [scene.ts] [--t <sec>] [--mode side\|blend\|diff\|grid]` | compare a render against a reference image |
| `pnpm reframe preview` | scrub/play/edit UI; edits export as overlay JSON |
| `pnpm reframe new <name>` | scaffold a documented starter scene |
| `pnpm reframe motion <mp4\|framesDir>` | calibrated motion profile (speeds, easing, discontinuities) |
Expand All @@ -293,8 +304,55 @@ them with a diagnosis naming the likely rename. The failure hierarchy:
2. Contract broken → loud orphan report.
3. Never: silent edit loss, or a render failure caused by base drift.

### Restructure, don't just re-skin

An overlay isn't limited to patching props and timing — it can change the
**structure** of the cut, keyed by the same stable addresses, and those edits
survive regeneration too: **reorder** a beat (`timeline.<beat>.order`),
**remove** one (`removeTimeline: ["shot-3"]`), or **insert** a whole new unit
(`insertNodes` + `insertTimeline { into: "montage", after: "shot-2" }`). A
montage is built so each shot is the self-contained beat `shot-${i}`, so an
overlay can drop, reorder, or splice in a card without touching the base:

```jsonc
// reorder two cards, drop one, retitle another — all survive an AI regen
{ "removeTimeline": ["shot-3"],
"timeline": { "shot-1": { "order": 2 }, "shot-2": { "order": 1 } },
"nodes": { "shot-0-title": { "content": "REORDERED" } } }
```

Reproduce it with the pure-vector demo (no assets) — the same overlay reorders,
removes, and inserts a card, and every edit is reported applied with zero
orphans:

```bash
pnpm reframe verify-overlay examples/scenes/vector-montage.ts \
examples/overlays/vector-montage-restructure.json # 4 applied, 0 orphaned
pnpm reframe render examples/scenes/vector-montage.ts \
--overlay examples/overlays/vector-montage-insert.json # a new card spliced into the cut
```

![The preview editor: knobs write into a non-destructive overlay](docs/assets/preview-editor.png)

## Embedding reframe (in-process API)

The CLI is a thin shell over an importable, server-side API — load, validate,
and check a scene without spawning a process. From `reframe-video/compile`:

```ts
import { loadScene, loadSceneFromCode, checkDeterminism } from "reframe-video/compile";

const compiled = await loadScene("scene.ts"); // bundle + validate → CompiledScene
// On failure a SceneLoadError carries .kind ("bundle" | "eval" | "validation")
// and .issues: { code, path, message }[] — structured, not prose to parse.
const { deterministic, findings } = await checkDeterminism("scene.ts");
```

`reframe-video/renderer` exposes `renderFrame` / `drawDisplayList` (DisplayList →
Canvas 2D) for live browser playback, and `reframe compile --json` returns the
same structured shape on the CLI: `{ ok: false, kind, issues: [{ code, path, message }] }`
— the feedback loop an agent or a UI reads to point at the exact broken node.

## Documentation

The [`docs/`](docs/) folder is a [Mintlify](https://mintlify.com)-ready site (`docs/docs.json`):
Expand All @@ -303,7 +361,7 @@ The [`docs/`](docs/) folder is a [Mintlify](https://mintlify.com)-ready site (`d
|---|---|
| [Introduction](docs/introduction.mdx) · [Quickstart](docs/quickstart.mdx) · [The loop](docs/the-loop.mdx) | the pitch, install, and the AI-write / human-edit / deterministic-render model |
| [Gallery](docs/gallery.mdx) | a curated visual reel of scenes |
| [Examples](examples/README.md) | all 66 example scenes, by category |
| [Examples](examples/README.md) | all 67 example scenes, by category |
| [Guides](docs/guides/) | the eDSL, directing, HTML/GSAP, and regeneration-contract guides (also `pnpm reframe guide`) |

Curated renders live in [`docs/assets/gallery/`](docs/assets/gallery) and accumulate via `pnpm gallery` (the committed home; `out/` stays scratch).
Expand All @@ -316,7 +374,7 @@ Curated renders live in [`docs/assets/gallery/`](docs/assets/gallery) and accumu
| `packages/renderer-canvas` | DisplayList → Canvas 2D (browser + capture shared) |
| `packages/render-cli` | Playwright capture + ffmpeg encode; also renders arbitrary HTML/GSAP deterministically via a virtual clock |
| `packages/preview` | the Vite editor |
| `examples/` | 66 example scenes (see [`examples/README.md`](examples/README.md)), overlays, compositions, the edit-survival demo |
| `examples/` | 67 example scenes (see [`examples/README.md`](examples/README.md)), overlays, compositions, the edit-survival demo |
| `labs/` | experiments and product probes (live-data → baked scene → render), kept out of `examples/` so it stays purely demonstrative |
| `docs/` | the [Mintlify](https://mintlify.com)-ready docs site + the authoring guides (also `pnpm reframe guide`) |
| `benchmark/` | **measurement artifacts, not product code**: LLM generation benchmark (RESULTS/ANALYSIS.md), regeneration-contract experiment (regen/), calibrated motion profiler (harness/motion/, MOTION.md) |
Expand All @@ -342,5 +400,5 @@ edit survival across AI regeneration, and a five-turn natural-language
iteration loop with zero silent edit loss. Receipts: `benchmark/ANALYSIS.md`,
`benchmark/MOTION.md`, `benchmark/regen/REGEN-ANALYSIS.md`,
`benchmark/nl-loop/NL-LOOP.md`. What "alpha" means honestly: it has not met
strangers yet — surface area is intentionally small (6 node types, one font,
strangers yet — surface area is intentionally small (8 node types, one font,
Canvas 2D) and the IR/overlay schema has no compatibility promise before 1.0.
68 changes: 68 additions & 0 deletions docs/api.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: Programmatic API
description: "Load, validate, and check a scene in-process — the importable surface under the CLI."
---

The CLI is a thin shell over a server-side, importable API. A backend can do NL → eDSL → IR → (preview + diff) without shelling out or re-implementing the evaluator. Three subpath exports of the `reframe-video` npm package:

| import | what |
|---|---|
| `reframe-video` | the core eDSL + IR — `scene`, `compileScene`, `composeScene`, `evaluate`, `sceneManifest`, `lintScene`, `validateScene` |
| `reframe-video/compile` | load + validate + determinism — `loadScene`, `loadSceneFromCode`, `loadModule`, `checkDeterminism`, `SceneLoadError`, `ValidationIssue` |
| `reframe-video/renderer` | `DisplayList` → Canvas 2D — `renderFrame`, `drawDisplayList`, `ImageRegistry`, `VideoRegistry`, `coverRect` |

<Note>
`reframe-video/compile` executes the scene module in-process. Treat untrusted or model-authored source as code: bound it with a timeout and run it where a misbehaving module can't do harm. True sandboxing is a separate concern.
</Note>

## Load & validate a scene

```ts
import { loadScene, loadSceneFromCode, SceneLoadError } from "reframe-video/compile";

try {
const compiled = await loadScene("scene.ts"); // bundle + validate → CompiledScene
// or: await loadSceneFromCode(sourceString) // no file on disk
} catch (err) {
if (err instanceof SceneLoadError) {
err.kind; // "bundle" | "eval" | "validation" — which stage failed
err.issues; // ValidationIssue[] on a validation failure
}
}
```

A `ValidationIssue` is structured, not prose — so a UI can point at the exact broken node:

```ts
interface ValidationIssue {
code: string; // stable category, e.g. "unknown-blend", "duplicate-node-id"
path: string; // locator, e.g. "nodes.box", "timeline.beat(intro)[0]", "camera.zoom"
message: string; // the human-readable line
}
```

The CLI mirrors this: `reframe compile --json` returns `{ ok: false, error, kind, issues }` on failure.

## Check determinism

```ts
import { checkDeterminism } from "reframe-video/compile";

const { deterministic, findings } = await checkDeterminism("scene.ts");
// compiles the source twice and diffs the IR; `findings` pins the first
// differing address when a Math.random() / Date leaked into a prop.
```

This is what `reframe lint` runs for the source-purity half of its gate.

## Render a frame

```ts
import { renderFrame } from "reframe-video/renderer";
import { compileScene, evaluate } from "reframe-video";

const ctx = canvas.getContext("2d")!; // any Canvas 2D context
renderFrame(ctx, compiled, t); // draw the scene at time t
```

`renderFrame` evaluates the compiled scene at `t` and paints the resulting `DisplayList`; pass an `ImageRegistry` / `VideoRegistry` to supply raster sources for `image` / `video` nodes. This is the same path the live browser preview uses.
51 changes: 51 additions & 0 deletions docs/cli-reference.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: CLI reference
description: "Every reframe command — render, inspect, validate, and edit a scene from the terminal."
---

Run any command with `npx reframe-video <command>` (no clone needed) or `pnpm reframe <command>` in a checkout. Scene paths resolve against the directory you invoke from; outputs default to `out/`.

## Render & output

| command | what it does |
|---|---|
| `render <scene.ts\|.json\|.html> [--overlay f]... [-o out]` | deterministic mp4 (mode inferred from extension; composes any `--overlay` first) |
| `batch <scene.ts> <data.json\|csv> [-o dir] [--overlay f]...` | one mp4 per data row (row keys are overlay addresses), in parallel, with a per-row report |
| `frame <scene.ts\|.json> [--t <sec>] [-o out.png]` | render one frame at time `t` to a PNG (chromium only, no mux) — for a render-and-look loop |
| `player <scene.ts\|.json> [-o out.html]` | bundle a scene into one self-contained HTML that plays the motion live in any browser |
| `logo <logo.svg\|brand-slug> [--motion <preset>] [--energy n] [--seed n]` | animate a logo (or a simple-icons brand) into a sting |
| `assemble <media...> [-o name] [--title "…"] [--bgm <synth>]` | probe images/videos (ffprobe) and scaffold an editable montage scene `.ts` |

## Inspect & validate

| command | what it does |
|---|---|
| `compile <scene.ts\|.json> [-o out.json] [--json]` | bundle + validate a scene to SceneIR JSON, no render (fast; no ffmpeg/chromium). `--json` returns `{ ok, kind, issues }` |
| `manifest <scene.ts\|.json> [--json]` | dump the addressable surface — every node, state, timeline label, and beat with the overlay address that reaches it |
| `lint <scene.ts\|.json> [--json] [--strict]` | the studio-readiness gate — flag un-addressable motion + verify the scene is a pure function of time. `--strict` exits non-zero on findings |
| `labels <scene.ts\|.json>` | print the compiled event clock (every timeline label → exact seconds) — the timing source for `audio.cues` |

## Edit & verify

| command | what it does |
|---|---|
| `verify-overlay <base> <overlay>... [--json]` | compose an overlay onto a base and report applied-vs-orphaned, no render. Non-zero exit on orphans — the regen-survival check |
| `preview` | scrub / play / edit UI; edits export as overlay JSON |
| `demo` | render the edit-survival demo (base, base + overlay, AI-regenerated base + the same overlay) |

## Author & measure

| command | what it does |
|---|---|
| `new <name>` | scaffold a documented starter scene `.ts` |
| `diff <ref-image> [scene.ts] [--t <sec>] [--mode side\|blend\|diff\|grid]` | compare a render against a reference image |
| `motion <mp4\|framesDir>` | calibrated motion profile of a clip (speeds, easing, discontinuities) |
| `trace <ref.mp4> [--apply scene.ts]` | extract a video's motion structure; `--apply` re-tells it on your own nodes |
| `guide [--directing\|--regen\|--html]` | print a guide: eDSL syntax (default), the directing workflow, the regeneration contract, or HTML/GSAP scenes |
| `skill [--path]` | print the authoring skill for an agent; `--path` prints the plugin dir to load |

<Tip>
Before patching a scene, run `manifest` to see the real addresses; after a regen, run `verify-overlay` against the new base to prove every edit still applies; gate CI with `lint --strict`.
</Tip>

The same load / validate / determinism surface is importable in-process — see the [programmatic API](/api).
4 changes: 3 additions & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@
"guides/edsl-guide",
"guides/directing-guide",
"guides/html-guide",
"guides/regen-contract"
"guides/regen-contract",
"cli-reference",
"api"
]
},
{
Expand Down
4 changes: 2 additions & 2 deletions docs/examples.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Examples
description: "All 66 example scenes, by category — each a single self-contained file you can render."
description: "All 67 example scenes, by category — each a single self-contained file you can render."
---

Every scene is one `.ts` file in [`examples/scenes/`](https://github.com/kiyeonjeon21/reframe/tree/main/examples/scenes) — self-contained and dependency-free. Render any of them:
Expand Down Expand Up @@ -44,7 +44,7 @@ The [gallery](/gallery) has the curated visual reel; the [repo README](https://g
[camera-demo](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/camera-demo.ts) · [isometric-stack](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/isometric-stack.ts) · [nova-teaser](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/nova-teaser.ts) · [orbit](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/orbit.ts) · [particle-swarm](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/particle-swarm.ts) · [rocket-launch](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/rocket-launch.ts) · [solar-system](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/solar-system.ts) · [spacex-cursor](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/spacex-cursor.ts) · [wavefield](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/wavefield.ts) · [zoom-to-space](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/zoom-to-space.ts)

## Edit-survival / regen
[survive-base](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/survive-base.ts) · [survive-cut](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/survive-cut.ts) · [survive-regen](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/survive-regen.ts)
[survive-base](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/survive-base.ts) · [survive-cut](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/survive-cut.ts) · [survive-regen](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/survive-regen.ts) · [vector-montage](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/vector-montage.ts)

## Other
[glitch-vhs](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/glitch-vhs.ts) · [glyph-reveal](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/glyph-reveal.ts) · [lower-third](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/lower-third.ts) · [photo-montage](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/photo-montage.ts) · [player-sting](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/player-sting.ts) · [video-demo](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/video-demo.ts) · [video-montage](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/video-montage.ts) · [worldcup-glyph](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/worldcup-glyph.ts)
2 changes: 1 addition & 1 deletion docs/gallery.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Gallery
description: "A reel of reframe scenes — each a few-line declaration, each a deterministic render."
---

Every clip below is a scene in [`examples/scenes/`](https://github.com/kiyeonjeon21/reframe/tree/main/examples/scenes). Render any of them yourself with `npx reframe-video render examples/scenes/<name>.ts`. The full list — 66 scenes by category — is on the [Examples](/examples) page.
Every clip below is a scene in [`examples/scenes/`](https://github.com/kiyeonjeon21/reframe/tree/main/examples/scenes). Render any of them yourself with `npx reframe-video render examples/scenes/<name>.ts`. The full list — 67 scenes by category — is on the [Examples](/examples) page.

<Note>
These gifs are the curated showcase. New renders accumulate here via `pnpm gallery` (the committed home, vs the gitignored `out/` scratch).
Expand Down
4 changes: 4 additions & 0 deletions docs/guides/directing-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ Mind the tiers so you're not full-rendering to check small things: `compile`
is the cheap visual look (one PNG, ~1s); `motion`/`trace` below need a finished
render or reference video, so they're end-stage measurement, not the per-edit loop.

- `reframe manifest scene.ts` — every editable address (nodes, states, labels, beats). Read
it before patching an overlay so you target real, stable addresses.
- `reframe lint scene.ts --strict` — flags un-addressable motion and verifies the scene is a
pure function of time (no `Math.random()` / `Date`). A clean lint is the studio-readiness gate.
- `reframe labels scene.ts` — every label → exact seconds. The timing source for audio + a
sanity check that beats land when you think.
- `reframe motion out.mp4` — speeds, static fraction, oscillation rhythm, spikes. A vague
Expand Down
Loading
Loading