feat(audio): scene-fitted Kokoro narration — reframe narrate (0.6.44)

AGENTS.md

@@ -18,6 +18,7 @@ deterministic mp4 render. Human edits survive AI regeneration of the base.
 - `pnpm reframe compile <scene.ts|.json> [-o out.json] [--stdin] [--code "<src>"] [--json]` — bundle + validate eDSL source into SceneIR JSON, NO render (no ffmpeg/chromium; fast). On failure: a concise classified error (`bundle`/`eval`/`validation`), never the base64 bundle; `--json` makes it `{ok:false,error,kind,issues?}` where `issues` is the structured validation problems (each `{code,path,message}` — e.g. `code:"unknown-blend", path:"nodes.box"`). The in-process equivalent is exported as `reframe-video/compile` (`loadScene`/`loadSceneFromCode`/`checkDeterminism`, server-only); a thrown `SceneValidationError` carries `.issues` (and `.problems` for back-compat), and `SceneLoadError.issues` propagates them across the scene bundle. Entry `packages/render-cli/src/compile.ts`; loader `loadScene.ts`.
 - `pnpm reframe frame <scene.ts|.json> [--t <sec>] [-o out.png]` — render ONE frame at time `t` to a PNG (same renderer as `render`, no ffmpeg muxing; chromium only). For an agentic render-and-look loop (feed the frame back to a model). Reuses `renderFrameAt` (`frameLoop.ts`); entry `packages/render-cli/src/frame.ts`.
 - `pnpm reframe assemble <media...> [-o name] [--title "…"] [--bgm <synth>] [--hold s] [--seed N]` — the **files → scene** path: probe each image/video for its real duration (ffprobe) and scaffold an editable montage scene `.ts` wiring `photoMontage` (clip-aware holds, no freeze) + an optional `title` + a music bed. Probed numbers are baked in → the emitted scene is a normal deterministic scene. Probe `packages/render-cli/src/media/probe.ts`; entry `assemble.ts`.
+- `pnpm reframe narrate <scene.ts|.json> [--voice <name>] [--max-speed n] [--script <path>] [--dry-run]` — **scene-fitted Kokoro voiceover**. Reads a sibling `<scene>-vo/script.json` of `{ at, text }` lines (imported into `audio.narration`), computes each line's slot from the compiled label clock, synthesizes it with a Kokoro python sidecar (`narrate.py`), and **auto-fits** its speech rate to the slot (bounded by `--max-speed`, default 1.3; warns if even max overruns). Bakes `file`/`voice`/`speed`/`duration` back into `script.json` (like `assemble` bakes ffprobe numbers); the scene then plays each line as a label-anchored `file` cue (survives retiming/regen) with the bed ducking under the whole utterance. `--dry-run` prints the fit table from a length *estimate* (no synthesis, no Kokoro needed). Kokoro is an **optional dep** (`pip install kokoro` + espeak-ng), preflighted like ffmpeg/chromium; the `.wav` are external assets (same-machine, not golden) — commit `script.json` + wavs together. Entry `packages/render-cli/src/narrate.ts` + sidecar `narrate.py`; the IR field is `AudioIR.narration` (`packages/core/src/ir.ts`), resolved in `resolveAudioPlan` (`audio.ts`). See `examples/scenes/narrated-demo.ts`.
 - `pnpm reframe manifest <scene.ts|.json> [--json]` — dump the scene's **addressable surface**: every node (+ its `editableProps` and `animatedProps`), state, timeline label (+ `patchable` params), beat, and behavior, each with the overlay address that reaches it. The map an AI/human editor reads to patch a scene surgically (vs regenerating). Core `sceneManifest(compiled)` (`packages/core/src/manifest.ts`, exported); entry `packages/render-cli/src/manifest.ts`.
 - `pnpm reframe lint <scene.ts|.json> [--json] [--strict]` — the **studio-readiness gate**: (a) flag un-addressable motion (a tween/to/motionPath with no `label` can't be retimed by an overlay and a regen can silently drop it) + a `motionAddressableRatio` summary, and (b) for a `.ts` source, verify the scene is a **pure function of time** (`non-deterministic-render` finding) — it bundles once and evaluates TWICE, reporting the first IR address that differs (e.g. a `Math.random()`/`Date` baked into a prop), since a non-pure scene silently compiles to a different IR each time. `--strict` exits non-zero on findings (CI gate). Core `lintScene(compiled)` (addressability); `checkDeterminism(path)` (purity, exported via `reframe-video/compile`, `packages/render-cli/src/determinism.ts`); entry `lint.ts`.
 - `pnpm reframe verify-overlay <base.ts|.json> <overlay.json>... [--json]` — compose an overlay onto a base and report applied-vs-orphaned, NO render. The regen-survival check: run vs the original base (all applied), then vs the AI-regenerated base — any orphan is a broken stable address. Non-zero exit on orphans (CI gate). Reuses `composeScene`/`formatComposeReport`; entry `verifyOverlay.ts`.
@@ -290,7 +291,11 @@ addition to the git push, so both channels carry the same skill.
 - Audio: `scene.audio` cues anchor to timeline labels (they survive retiming);
   sfx are procedurally synthesized, CC0 samples live in `assets/sfx/`
   (LICENSE.md records provenance). Determinism contract covers the AudioPlan
-  and WAV bytes, not AAC-encoded mp4 bytes.
+  and WAV bytes, not AAC-encoded mp4 bytes. `audio.narration` lines (spoken VO)
+  resolve to label-anchored `file` cues after `reframe narrate` synthesizes them;
+  the Kokoro `.wav` are **external assets** (same-machine, version-dependent, like
+  images) — NOT part of the golden contract. Synthesis is out-of-band; only the
+  AudioPlan (cue timing + the baked `duration`-sized duck window) is deterministic.
 - Golden snapshots in `packages/core/test/__snapshots__` encode the determinism
   contract; if they change unexpectedly, that's a regression, not noise.
 

CHANGELOG.md

@@ -8,6 +8,27 @@ versions may change them.
 
 ## [Unreleased]
 
+## [0.6.44] - 2026-06-21
+
+### Added
+
+#### Scene-fitted Kokoro narration (`reframe narrate` + `audio.narration`)
+
+- New IR field **`AudioIR.narration`** — spoken voiceover lines (`{ at, text, voice? }`)
+  authored as a sibling `<scene>-vo/script.json` the scene imports. Each line resolves to a
+  **label-anchored `file` cue** (so VO survives retiming/regen), with a baked `duration`
+  sizing the bed's duck window. Additive + golden-safe (no narration → byte-identical plan).
+- New command **`reframe narrate <scene> [--voice] [--max-speed] [--dry-run]`** — reads the
+  compiled **label clock**, synthesizes each line with a Kokoro python sidecar (`narrate.py`),
+  and **auto-fits** its speech rate to the slot between its anchor and the next line (bounded;
+  warns if even max speed overruns). Bakes `file`/`voice`/`speed`/`duration` back into
+  `script.json` (like `assemble` bakes ffprobe numbers). `--dry-run` prints the fit table from
+  a length estimate with no synthesis.
+- Kokoro is an **optional dependency** (`pip install kokoro` + espeak-ng), preflighted like
+  ffmpeg/chromium. The `.wav` are external assets (same-machine, not golden) — the determinism
+  contract still covers the AudioPlan, not the synthesized audio bytes.
+- Example `examples/scenes/narrated-demo.ts` (+ `narrated-demo-vo/script.json`).
+
 ## [0.6.43] - 2026-06-21
 
 ### Added

README.md

@@ -280,6 +280,7 @@ your scene.
 | `pnpm reframe verify-overlay <base> <overlay>... [--json]` | compose an overlay onto a base and report applied-vs-orphaned, no render — the regen-survival check (non-zero exit on orphans) |
 | `pnpm reframe labels <scene.ts\|.json>` | print the compiled event clock (every timeline label → exact seconds) — the timing source for audio cues |
 | `pnpm reframe assemble <media...> [-o name]` | probe images/videos (ffprobe) and scaffold an editable montage scene `.ts` wired with `photoMontage` |
+| `pnpm reframe narrate <scene.ts\|.json> [--voice <name>] [--max-speed n] [--dry-run]` | scene-fitted Kokoro voiceover: synth each `audio.narration` line and auto-fit its rate to the slot (needs python + `kokoro`) |
 | `pnpm reframe player <scene.ts\|.json> [-o out.html]` | bundle a scene into one self-contained HTML that plays the motion live in any browser |
 | `pnpm reframe logo <logo.svg\|brand-slug> [--motion <preset>]` | animate a logo (or a simple-icons brand) into a sting |
 | `pnpm reframe diff <ref-image> [scene.ts] [--t <sec>] [--mode side\|blend\|diff\|grid]` | compare a render against a reference image |
@@ -362,7 +363,7 @@ site. The [`docs/`](docs/) folder is its [Mintlify](https://mintlify.com) source
 |---|---|
 | [Introduction](docs/introduction.mdx) · [Quickstart](docs/quickstart.mdx) · [The loop](docs/the-loop.mdx) | the pitch, install, and the AI-write / human-edit / deterministic-render model |
 | [Gallery](docs/gallery.mdx) | a curated visual reel of scenes |
-| [Examples](examples/README.md) | all 67 example scenes, by category |
+| [Examples](examples/README.md) | all 68 example scenes, by category |
 | [Guides](docs/guides/) | the eDSL, directing, HTML/GSAP, and regeneration-contract guides (also `pnpm reframe guide`) |
 
 Curated renders live in [`docs/assets/gallery/`](docs/assets/gallery) and accumulate via `pnpm gallery` (the committed home; `out/` stays scratch).
@@ -375,7 +376,7 @@ Curated renders live in [`docs/assets/gallery/`](docs/assets/gallery) and accumu
 | `packages/renderer-canvas` | DisplayList → Canvas 2D (browser + capture shared) |
 | `packages/render-cli` | Playwright capture + ffmpeg encode; also renders arbitrary HTML/GSAP deterministically via a virtual clock |
 | `packages/preview` | the Vite editor |
-| `examples/` | 67 example scenes (see [`examples/README.md`](examples/README.md)), overlays, compositions, the edit-survival demo |
+| `examples/` | 68 example scenes (see [`examples/README.md`](examples/README.md)), overlays, compositions, the edit-survival demo |
 | `labs/` | experiments and product probes (live-data → baked scene → render), kept out of `examples/` so it stays purely demonstrative |
 | `docs/` | the [Mintlify](https://mintlify.com)-ready docs site + the authoring guides (also `pnpm reframe guide`) |
 | `benchmark/` | **measurement artifacts, not product code**: LLM generation benchmark (RESULTS/ANALYSIS.md), regeneration-contract experiment (regen/), calibrated motion profiler (harness/motion/, MOTION.md) |

docs/cli-reference.mdx

@@ -15,6 +15,7 @@ Run any command with `npx reframe-video <command>` (no clone needed) or `pnpm re
 | `player <scene.ts\|.json> [-o out.html]` | bundle a scene into one self-contained HTML that plays the motion live in any browser |
 | `logo <logo.svg\|brand-slug> [--motion <preset>] [--energy n] [--seed n]` | animate a logo (or a simple-icons brand) into a sting |
 | `assemble <media...> [-o name] [--title "…"] [--bgm <synth>]` | probe images/videos (ffprobe) and scaffold an editable montage scene `.ts` |
+| `narrate <scene.ts\|.json> [--voice <name>] [--max-speed n] [--dry-run]` | scene-fitted Kokoro voiceover — synth each `audio.narration` line and auto-fit its rate to the slot (needs python + `kokoro`; `--dry-run` estimates without synthesis) |
 
 ## Inspect & validate
 

docs/examples.mdx

@@ -1,6 +1,6 @@
 ---
 title: Examples
-description: "All 67 example scenes, by category — each a single self-contained file you can render."
+description: "All 68 example scenes, by category — each a single self-contained file you can render."
 ---
 
 Every scene is one `.ts` file in [`examples/scenes/`](https://github.com/kiyeonjeon21/reframe/tree/main/examples/scenes) — self-contained and dependency-free. Render any of them:
@@ -35,7 +35,7 @@ The [gallery](/gallery) has the curated visual reel; the [repo README](https://g
 [annual-report](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/annual-report.ts) · [chart-buildup](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/chart-buildup.ts) · [data-explainer](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/data-explainer.ts) · [flow-diagram](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/flow-diagram.ts) · [github-year](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/github-year.ts)
 
 ## Audio
-[audio-visualizer](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/audio-visualizer.ts) · [auto-foley-demo](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/auto-foley-demo.ts) · [sample-showcase](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/sample-showcase.ts) · [sfx-compare](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/sfx-compare.ts) · [sfx-showcase](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/sfx-showcase.ts)
+[audio-visualizer](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/audio-visualizer.ts) · [auto-foley-demo](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/auto-foley-demo.ts) · [narrated-demo](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/narrated-demo.ts) · [sample-showcase](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/sample-showcase.ts) · [sfx-compare](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/sfx-compare.ts) · [sfx-showcase](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/sfx-showcase.ts)
 
 ## Logo stings
 [logo-reveal](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/logo-reveal.ts) · [logo-reveal-regen](https://github.com/kiyeonjeon21/reframe/blob/main/examples/scenes/logo-reveal-regen.ts)

docs/gallery.mdx

@@ -3,7 +3,7 @@ title: Gallery
 description: "A reel of reframe scenes — each a few-line declaration, each a deterministic render."
 ---
 
-Every clip below is a scene in [`examples/scenes/`](https://github.com/kiyeonjeon21/reframe/tree/main/examples/scenes). Render any of them yourself with `npx reframe-video render examples/scenes/<name>.ts`. The full list — 67 scenes by category — is on the [Examples](/examples) page.
+Every clip below is a scene in [`examples/scenes/`](https://github.com/kiyeonjeon21/reframe/tree/main/examples/scenes). Render any of them yourself with `npx reframe-video render examples/scenes/<name>.ts`. The full list — 68 scenes by category — is on the [Examples](/examples) page.
 
 <Note>
   These gifs are the curated showcase. New renders accumulate here via `pnpm gallery` (the committed home, vs the gitignored `out/` scratch).

examples/README.md

@@ -1,6 +1,6 @@
 # Examples
 
-67 curated scenes, one per `.ts` file in [`scenes/`](scenes). Each is a single, self-contained, dependency-free document — render any of them:
+68 curated scenes, one per `.ts` file in [`scenes/`](scenes). Each is a single, self-contained, dependency-free document — render any of them:
 
 ```bash
 pnpm reframe render examples/scenes/<scene>.ts        # in this repo
@@ -90,6 +90,7 @@ Also here: [`overlays/`](overlays) (human-edit layers), [`compositions/`](compos
 |---|---|
 | `audio-visualizer` | "THE DROP": radial spectrum bars, a pulsing core, a particle burst. |
 | `auto-foley-demo` | `autoFoley` scoring motion — whoosh / thud / pop following the tweens. |
+| `narrated-demo` | Scene-fitted Kokoro voiceover: `audio.narration` from a sibling `script.json`, each line auto-fitted to its slot by `reframe narrate`, bed ducking under it. |
 | `sample-showcase` | The CC0 sample library: keypress / footstep / click / confirm / UI sounds. |
 | `sfx-compare` | Synth vs sample A/B for the six original names. |
 | `sfx-showcase` | The procedural SFX palette, per-cue seeded variation as a little melody. |

examples/scenes/narrated-demo-vo/script.json

@@ -0,0 +1,23 @@
+[
+  {
+    "at": "intro",
+    "text": "This is reframe.",
+    "file": "narrated-demo-vo/intro.wav",
+    "voice": "af_heart",
+    "duration": 1.775
+  },
+  {
+    "at": "point",
+    "text": "Anchored to the timeline, it survives.",
+    "file": "narrated-demo-vo/point.wav",
+    "voice": "af_heart",
+    "duration": 2.775
+  },
+  {
+    "at": "close",
+    "text": "Open source.",
+    "file": "narrated-demo-vo/close.wav",
+    "voice": "af_heart",
+    "duration": 1.65
+  }
+]

examples/scenes/narrated-demo.ts

@@ -0,0 +1,45 @@
+// Narrated demo — a pure-vector scene whose voiceover is authored as a sibling
+// `narrated-demo-vo/script.json` (imported into `audio.narration`) and synthesized
+// + fitted to the timeline by `reframe narrate`. Each line anchors to a timeline
+// label, so the VO survives retiming/regen; `narrate` reads the label clock and
+// fits each line's speech rate to its slot.
+//
+//   reframe narrate examples/scenes/narrated-demo.ts --dry-run   # fit table (no synth)
+//   reframe narrate examples/scenes/narrated-demo.ts             # synth + fit (python+kokoro)
+//   reframe render  examples/scenes/narrated-demo.ts             # mp4, bed ducks under the VO
+//
+// The .wav are out-of-band assets (not bundled to npm, not golden) — commit
+// script.json + the generated wavs together. Image/audio file cues don't render
+// in player/artifacts; mp4 only.
+
+import { scene, rect, text, seq, tween, wait, linearGradient } from "@reframe/core";
+import vo from "./narrated-demo-vo/script.json";
+
+const W = 1920, H = 1080;
+
+export default scene({
+  id: "narrated-demo",
+  size: { width: W, height: H },
+  fps: 30,
+  background: "#06070C",
+  nodes: [
+    rect({ id: "bg", x: 0, y: 0, width: W, height: H, fill: linearGradient(["#0A1430", "#06070C"], { angle: 90 }) }),
+    text({ id: "title", x: W / 2, y: H / 2 - 40, anchor: "center", content: "reframe", fontFamily: "Inter", fontSize: 200, fontWeight: 800, fill: "#FFFFFF", opacity: 0 }),
+    text({ id: "sub", x: W / 2, y: H / 2 + 110, anchor: "center", content: "voice that fits the scene", fontFamily: "Inter", fontSize: 46, fontWeight: 500, fill: "#7FB4FF", opacity: 0 }),
+  ],
+  // labels (intro / point / close) are the stable anchors the narration lines bind to
+  timeline: seq(
+    wait(0.4),
+    tween("title", { opacity: 1 }, { duration: 0.6, ease: "easeOutCubic", label: "intro" }),
+    wait(2.4),
+    tween("sub", { opacity: 1 }, { duration: 0.5, ease: "easeOutCubic", label: "point" }),
+    wait(3.2),
+    tween("title", { opacity: 0 }, { duration: 0.6, ease: "easeInCubic", label: "close" }),
+    tween("sub", { opacity: 0 }, { duration: 0.6, ease: "easeInCubic", label: "close-sub" }),
+    wait(0.6),
+  ),
+  audio: {
+    bgm: { synth: "ambient-pad", gain: 0.25, fadeIn: 1, fadeOut: 1.5, duck: { depth: 0.6 } },
+    narration: vo,
+  },
+});

examples/tsconfig.json

@@ -2,7 +2,8 @@
   "extends": "../tsconfig.base.json",
   "compilerOptions": {
     "lib": ["ES2022", "DOM", "DOM.Iterable"],
-    "types": ["node"]
+    "types": ["node"],
+    "resolveJsonModule": true
   },
   "include": ["scenes", "scripts"]
 }