feat(audio): scene-fitted Kokoro narration — reframe narrate (0.6.44)

[kiyeonjeon21]

Make voiceover first-class and fit it to the scene. Today Kokoro VO is generated by a standalone generate.py and the cues are hand-wired, with no awareness of how much time each line has. This adds a narration representation in the scene IR (text + label anchor + voice, authored as a sibling <scene>-vo/script.json the scene imports) and a reframe narrate generator that reads the label clock and auto-fits each line's speech rate to its slot. Being label-anchored, the VO survives retiming/regen.

Core (golden-safe, additive)

• AudioIR.narration (ir.ts) — { at, text, voice?, gain?, offset? } + baked { file, speed, duration }.
• resolveAudioPlan expands narration → label-anchored file cues; the baked duration sizes the bed's duck window; an un-synthesized line warns (not fatal).
• validate.ts narration-* checks. No-narration scenes stay byte-identical (404 tests green, goldens unchanged).

CLI + sidecar

• reframe narrate <scene> [--voice] [--max-speed] [--script] [--dry-run] — slot from compiled.labelTimes, ≤2-pass auto-fit (speed clamped 1.0–maxSpeed, warns on overrun), bakes file/voice/speed/duration into script.json (like assemble bakes ffprobe numbers), prints a fit table. --dry-run estimates with no synthesis.
• narrate.py Kokoro sidecar (stdin JSON → wavs + durations), shipped beside dist/narrate.js. python + kokoro preflighted like ffmpeg/chromium (optional dep).

Determinism

The .wav are external assets (same-machine, not golden), like images — only the AudioPlan is deterministic. Synthesis is out-of-band; commit script.json + wavs together.

Verification

• pnpm test (404) + pnpm typecheck green; goldens unchanged.
• New core tests: narration → file cues with duration-sized duck windows; validation; un-synthesized warning.
• End-to-end (this machine, Kokoro installed): narrate --dry-run fit table → real synth + auto-fit (point sped to fit, then tuned to 1.0×) → render produced an mp4 with 3 VO cues, bed ducking under each line.
• Packaged build ships dist/narrate.js + dist/narrate.py; packaged --dry-run smoke-tested.

Example: examples/scenes/narrated-demo.ts (+ narrated-demo-vo/script.json + wavs). Docs: AGENTS.md, CHANGELOG, README + cli-reference + examples. PATCH bump reframe-video → 0.6.44.

🤖 Generated with Claude Code

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
reframe	🟢 Ready	View Preview	Jun 21, 2026, 11:54 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.