diff --git a/AGENTS.md b/AGENTS.md index ad6bf3c..f4ac69a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,11 +15,11 @@ deterministic mp4 render. Human edits survive AI regeneration of the base. - `pnpm reframe batch ` — one mp4 per row (row keys are overlay addresses like `nodes..`) - `pnpm reframe logo [--motion ] [--energy n] [--seed n]` — animate a logo into a sting (published CLI command; `packages/render-cli/src/logoSting.ts`) - `pnpm reframe labels ` — print the compiled event clock (every timeline label → exact seconds; the timing source for `audio.cues` and beat debugging) -- `pnpm reframe compile [-o out.json] [--stdin] [--code ""] [--json]` — bundle + validate eDSL source into SceneIR JSON, NO render (no ffmpeg/chromium; fast). On failure: a concise classified error (`bundle`/`eval`/`validation`), never the base64 bundle; `--json` makes it `{ok:false,error,kind}`. The in-process equivalent is exported as `reframe-video/compile` (`loadScene`/`loadSceneFromCode`, server-only). Entry `packages/render-cli/src/compile.ts`; loader `loadScene.ts`. +- `pnpm reframe compile [-o out.json] [--stdin] [--code ""] [--json]` — bundle + validate eDSL source into SceneIR JSON, NO render (no ffmpeg/chromium; fast). On failure: a concise classified error (`bundle`/`eval`/`validation`), never the base64 bundle; `--json` makes it `{ok:false,error,kind}`. The in-process equivalent is exported as `reframe-video/compile` (`loadScene`/`loadSceneFromCode`/`checkDeterminism`, server-only). Entry `packages/render-cli/src/compile.ts`; loader `loadScene.ts`. - `pnpm reframe frame [--t ] [-o out.png]` — render ONE frame at time `t` to a PNG (same renderer as `render`, no ffmpeg muxing; chromium only). For an agentic render-and-look loop (feed the frame back to a model). Reuses `renderFrameAt` (`frameLoop.ts`); entry `packages/render-cli/src/frame.ts`. - `pnpm reframe assemble [-o name] [--title "…"] [--bgm ] [--hold s] [--seed N]` — the **files → scene** path: probe each image/video for its real duration (ffprobe) and scaffold an editable montage scene `.ts` wiring `photoMontage` (clip-aware holds, no freeze) + an optional `title` + a music bed. Probed numbers are baked in → the emitted scene is a normal deterministic scene. Probe `packages/render-cli/src/media/probe.ts`; entry `assemble.ts`. - `pnpm reframe manifest [--json]` — dump the scene's **addressable surface**: every node (+ its `editableProps` and `animatedProps`), state, timeline label (+ `patchable` params), beat, and behavior, each with the overlay address that reaches it. The map an AI/human editor reads to patch a scene surgically (vs regenerating). Core `sceneManifest(compiled)` (`packages/core/src/manifest.ts`, exported); entry `packages/render-cli/src/manifest.ts`. -- `pnpm reframe lint [--json] [--strict]` — flag un-addressable motion (a tween/to/motionPath with no `label` can't be retimed by an overlay and a regen can silently drop it) + a `motionAddressableRatio` summary. `--strict` exits non-zero on findings (CI gate). Core `lintScene(compiled)` (same module); entry `lint.ts`. +- `pnpm reframe lint [--json] [--strict]` — the **studio-readiness gate**: (a) flag un-addressable motion (a tween/to/motionPath with no `label` can't be retimed by an overlay and a regen can silently drop it) + a `motionAddressableRatio` summary, and (b) for a `.ts` source, verify the scene is a **pure function of time** (`non-deterministic-render` finding) — it bundles once and evaluates TWICE, reporting the first IR address that differs (e.g. a `Math.random()`/`Date` baked into a prop), since a non-pure scene silently compiles to a different IR each time. `--strict` exits non-zero on findings (CI gate). Core `lintScene(compiled)` (addressability); `checkDeterminism(path)` (purity, exported via `reframe-video/compile`, `packages/render-cli/src/determinism.ts`); entry `lint.ts`. - `pnpm reframe verify-overlay ... [--json]` — compose an overlay onto a base and report applied-vs-orphaned, NO render. The regen-survival check: run vs the original base (all applied), then vs the AI-regenerated base — any orphan is a broken stable address. Non-zero exit on orphans (CI gate). Reuses `composeScene`/`formatComposeReport`; entry `verifyOverlay.ts`. - `pnpm reframe skill [--path]` — print the authoring skill (`plugin/skills/reframe/SKILL.md`) for a programmatic/agent consumer; `--path` prints the plugin root dir. The skill + `.claude-plugin/` ship in the npm package (`files`) so an Agent-SDK consumer can load the plugin from `node_modules/reframe-video` (no repo checkout). Inline in `reframe.ts`. - `pnpm reframe player [-o out.html]` — bundle a scene into ONE self-contained HTML that plays the motion live in any browser (and pastes into a Claude.ai Artifact). esbuild IIFE of core + `renderer-canvas` + the scene on a `` rAF loop, with the Inter fonts inlined; visual-only (no audio / image-node sources). Entry `packages/render-cli/src/player.ts`. diff --git a/CHANGELOG.md b/CHANGELOG.md index c64723b..aa7129c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,26 @@ versions may change them. ## [Unreleased] +## [0.6.42] - 2026-06-21 + +### Added + +#### Determinism guard — `reframe lint` now enforces scene purity + +- Scenes must be pure functions of time (no `Math.random()`/`Date`), but nothing enforced + it: a non-pure scene compiled to a *different IR each time*, silently breaking + reproducibility (the golden tests only cover IR→render, not source→IR). `reframe lint` now + closes that gap for `.ts` sources: it bundles the scene once, **evaluates it twice**, and + emits a `non-deterministic-render` finding pinned to the first IR address that differs + (e.g. `nodes[0].props.x changed 0.42 → 0.97`), with a hint at the culprit construct. Part + of the same `--strict` gate as the addressability checks. +- New in-process API **`checkDeterminism(path)`** (exported via `reframe-video/compile`) → + `{ deterministic, findings }`, so an embedder can verify a freshly generated scene before + trusting it. Additive: pure scenes report clean, and the check only runs on source inputs + (`.json` is the IR itself). Implementation note: the two evals use distinct cache-busting + comments to defeat Node's `data:`-URL ESM module cache (without which the second eval would + return the cached module and miss module-level nondeterminism). + ## [0.6.41] - 2026-06-21 ### Added diff --git a/docs/guides/regen-contract.md b/docs/guides/regen-contract.md index 4c9a1f5..d4b8912 100644 --- a/docs/guides/regen-contract.md +++ b/docs/guides/regen-contract.md @@ -72,7 +72,11 @@ checkable (no render): their editable/animated props, states, timeline labels with patchable params, beats, behaviors). Read it before patching so you target real, stable addresses. - `reframe lint [--strict]` — flag motion with no `label` (timing an - overlay can't reach and a regen can silently drop) + a `motionAddressableRatio`. + overlay can't reach and a regen can silently drop) + a `motionAddressableRatio`, + AND verify the scene is a **pure function of time**: it bundles + evaluates the + source twice and flags any IR that differs (a `Math.random()`/`Date` baked into a + prop). A non-pure scene compiles to a different IR each time, so its render is not + reproducible — use a seeded `wiggle()` or a scene knob instead. - `reframe verify-overlay ...` — compose the overlay onto a base and report applied vs orphaned. Run it against the regenerated base to prove every edit survived; it exits non-zero if any address broke. diff --git a/packages/reframe-video/package.json b/packages/reframe-video/package.json index 1362214..72c9ca0 100644 --- a/packages/reframe-video/package.json +++ b/packages/reframe-video/package.json @@ -1,6 +1,6 @@ { "name": "reframe-video", - "version": "0.6.41", + "version": "0.6.42", "description": "Declarative motion graphics that AI can write and humans can tweak — human edits survive AI regeneration. Deterministic mp4 renders from a plain-data scene format.", "keywords": [ "motion-graphics", diff --git a/packages/render-cli/src/compileApi.ts b/packages/render-cli/src/compileApi.ts index ccb1598..3ea86cc 100644 --- a/packages/render-cli/src/compileApi.ts +++ b/packages/render-cli/src/compileApi.ts @@ -10,3 +10,4 @@ * misbehaving module can't do harm. True sandboxing is a separate concern. */ export { isComposition, loadModule, loadScene, loadSceneFromCode, SceneLoadError } from "./loadScene.js"; +export { checkDeterminism } from "./determinism.js"; diff --git a/packages/render-cli/src/determinism.ts b/packages/render-cli/src/determinism.ts new file mode 100644 index 0000000..70cf196 --- /dev/null +++ b/packages/render-cli/src/determinism.ts @@ -0,0 +1,81 @@ +/** + * Determinism guard — verify a scene is a PURE FUNCTION OF TIME (the contract the + * whole determinism story rests on: same source → same IR → same render). Goldens + * cover IR→render, but the source→IR step is otherwise unguarded: a scene that uses + * `Math.random()`/`Date` compiles and renders fine yet yields a DIFFERENT IR each + * compile, silently breaking reproducibility. + * + * Mechanism: bundle once, evaluate TWICE, deep-compare the two IRs. Any + * nondeterminism makes them differ regardless of where it lives; a pure scene + * yields identical IRs. False-positive-free (seeded wiggle/presets are stable). + */ +import { readFile } from "node:fs/promises"; +import type { LintFinding } from "@reframe/core"; +import { bundle, evalSceneOnce } from "./loadScene.js"; + +let busterN = 0; + +/** First differing JSON path between two values (a `nodes.box.x` style address), or null if deep-equal. */ +export function firstDiff(a: unknown, b: unknown, path = ""): { path: string; a: unknown; b: unknown } | null { + if (Object.is(a, b)) return null; + const oa = typeof a === "object" && a !== null; + const ob = typeof b === "object" && b !== null; + if (!oa || !ob || Array.isArray(a) !== Array.isArray(b)) return { path: path || "(root)", a, b }; + if (Array.isArray(a) && Array.isArray(b)) { + const n = Math.max(a.length, b.length); + for (let i = 0; i < n; i++) { + const d = firstDiff(a[i], b[i], `${path}[${i}]`); + if (d) return d; + } + return null; + } + const ao = a as Record; + const bo = b as Record; + for (const k of new Set([...Object.keys(ao), ...Object.keys(bo)])) { + const d = firstDiff(ao[k], bo[k], path ? `${path}.${k}` : k); + if (d) return d; + } + return null; +} + +const NONDET = /\bMath\.random\b|\bDate\.now\b|\bnew Date\b|\bperformance\.now\b|\bcrypto\.(?:getRandomValues|randomUUID)\b/; + +/** + * Check that a scene compiles to the SAME IR twice. `.json` inputs are the IR + * itself (no eval) → trivially deterministic. Returns structured findings reusing + * the `LintFinding` shape so they merge straight into `reframe lint`. + */ +export async function checkDeterminism(path: string): Promise<{ deterministic: boolean; findings: LintFinding[] }> { + if (path.endsWith(".json")) return { deterministic: true, findings: [] }; + + const code = await bundle({ path }); + // distinct busters defeat Node's data:-URL ESM cache so the module re-evaluates + const a = await evalSceneOnce(code, `${busterN++}`); + const b = await evalSceneOnce(code, `${busterN++}`); + const diff = firstDiff(a, b); + if (!diff) return { deterministic: true, findings: [] }; + + // best-effort hint: name the likely culprit construct from the entry source + let hint = ""; + try { + const m = NONDET.exec(await readFile(path, "utf8")); + if (m) hint = ` (source uses "${m[0]}")`; + } catch { + /* hint is optional */ + } + + return { + deterministic: false, + findings: [ + { + rule: "non-deterministic-render", + severity: "error", + message: + `scene is not a pure function of time — \`${diff.path}\` changed between compiles ` + + `(${JSON.stringify(diff.a)} → ${JSON.stringify(diff.b)})${hint}. ` + + `Avoid Math.random()/Date; use a seeded wiggle() or a scene knob.`, + address: diff.path, + }, + ], + }; +} diff --git a/packages/render-cli/src/lint.ts b/packages/render-cli/src/lint.ts index c04ec13..481e813 100644 --- a/packages/render-cli/src/lint.ts +++ b/packages/render-cli/src/lint.ts @@ -2,11 +2,13 @@ /** * `reframe lint [--json] [--strict]` — flag the surface that * ISN'T overlay-addressable (motion with no label can't be retimed by an edit - * layer, and a base regeneration can silently drop it), plus an addressability - * summary. `--strict` exits non-zero when there are findings (a CI gate). + * layer, and a base regeneration can silently drop it), PLUS verify the scene is + * a pure function of time (deterministic: same source → same IR). Together this is + * the "studio-readiness" gate. `--strict` exits non-zero on findings (a CI gate). */ import { compileScene, lintScene, sceneManifest } from "@reframe/core"; import { loadScene } from "./loadScene.js"; +import { checkDeterminism } from "./determinism.js"; const args = process.argv.slice(2); const json = args.includes("--json"); @@ -19,7 +21,9 @@ if (!path) { async function main() { const compiled = compileScene(await loadScene(path!)); - const findings = lintScene(compiled); + // addressability (IR-level) + determinism (source-level; .json has no source to re-eval) + const det = await checkDeterminism(path!); + const findings = [...det.findings, ...lintScene(compiled)]; const s = sceneManifest(compiled).summary; if (json) { @@ -29,7 +33,7 @@ async function main() { `# ${s.nodeCount} nodes · ${s.labeledSteps} labeled steps · motion addressable ${(s.motionAddressableRatio * 100).toFixed(0)}% (${s.unlabeledMotionSteps} unlabeled)`, ); if (findings.length === 0) { - console.log("✓ no addressability findings"); + console.log("✓ addressable + deterministic — no findings"); } else { for (const f of findings) console.log(` ${f.severity === "error" ? "✗" : "!"} [${f.rule}] ${f.message}`); } diff --git a/packages/render-cli/src/loadScene.ts b/packages/render-cli/src/loadScene.ts index ddf2df4..83380c6 100644 --- a/packages/render-cli/src/loadScene.ts +++ b/packages/render-cli/src/loadScene.ts @@ -47,7 +47,7 @@ const clean = (err: unknown): string => const ALIAS = { "@reframe/core": CORE_ENTRY, "reframe-video": CORE_ENTRY }; /** esbuild a scene to ESM code (from a file entry or inline source) → throws `bundle`. */ -async function bundle(input: { path: string } | { code: string; resolveDir: string }): Promise { +export async function bundle(input: { path: string } | { code: string; resolveDir: string }): Promise { const common: BuildOptions = { bundle: true, format: "esm", @@ -126,6 +126,15 @@ export async function loadSceneFromCode(code: string, resolveDir: string = proce return asScene(await importDefault(await bundle({ code, resolveDir }), ""), ""); } +/** Evaluate ALREADY-BUNDLED scene code into a validated SceneIR. `buster` is + * appended as a comment so the `data:` URL differs per call — Node caches ESM by + * URL, so without it a second eval of identical code returns the cached module + * (and never re-runs module-level code). Used by the determinism check to eval + * the same bundle twice and compare. */ +export async function evalSceneOnce(code: string, buster: string): Promise { + return asScene(await importDefault(`${code}\n//det-${buster}`, ""), ""); +} + /** Load a scene OR composition, validated and discriminated. */ export async function loadModule( path: string, diff --git a/packages/render-cli/test/checkDeterminism.test.ts b/packages/render-cli/test/checkDeterminism.test.ts new file mode 100644 index 0000000..b1896a9 --- /dev/null +++ b/packages/render-cli/test/checkDeterminism.test.ts @@ -0,0 +1,64 @@ +/** + * The source→IR determinism guard (distinct from determinism.test.ts, which covers + * IR→render byte-identity). `checkDeterminism` bundles a scene once, evaluates it + * twice, and flags any difference — catching Math.random()/Date that would make a + * scene compile to a different IR each time. + */ +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterAll, describe, expect, it } from "vitest"; +import { checkDeterminism, firstDiff } from "../src/determinism.js"; + +const dirs: string[] = []; +async function fixture(name: string, src: string): Promise { + const dir = await mkdtemp(join(tmpdir(), "reframe-det-")); + dirs.push(dir); + const path = join(dir, name); + await writeFile(path, src, "utf8"); + return path; +} +afterAll(async () => { + await Promise.all(dirs.map((d) => rm(d, { recursive: true, force: true }))); +}); + +const PURE = `import { scene, rect, beat, tween } from "@reframe/core"; +export default scene({ id: "p", size: { width: 100, height: 100 }, + nodes: [rect({ id: "box", x: 10, y: 10, width: 20, height: 20, fill: "#fff", opacity: 0 })], + timeline: beat("in", {}, [tween("box", { opacity: 1 }, { duration: 0.5, label: "f" })]) });`; + +const IMPURE = `import { scene, rect } from "@reframe/core"; +export default scene({ id: "i", size: { width: 100, height: 100 }, + nodes: [rect({ id: "box", x: Math.random() * 50, y: 10, width: 20, height: 20, fill: "#fff" })] });`; + +describe("checkDeterminism (source → IR purity)", () => { + it("passes a pure scene", async () => { + const r = await checkDeterminism(await fixture("pure.ts", PURE)); + expect(r.deterministic).toBe(true); + expect(r.findings).toEqual([]); + }, 30_000); + + it("catches Math.random — the data:-URL module cache is actually busted", async () => { + const r = await checkDeterminism(await fixture("impure.ts", IMPURE)); + expect(r.deterministic).toBe(false); + expect(r.findings).toHaveLength(1); + const f = r.findings[0]!; + expect(f.rule).toBe("non-deterministic-render"); + expect(f.severity).toBe("error"); + expect(f.address).toContain("props.x"); // pinned to the differing prop + expect(f.message).toContain("Math.random"); // source-scan hint + }, 30_000); + + it(".json input is trivially deterministic (no source to re-evaluate)", async () => { + const json = JSON.stringify({ version: 1, id: "j", size: { width: 1, height: 1 }, nodes: [] }); + const r = await checkDeterminism(await fixture("scene.json", json)); + expect(r.deterministic).toBe(true); + expect(r.findings).toEqual([]); + }); + + it("firstDiff reports the first differing address", () => { + expect(firstDiff({ a: 1, b: { c: 2 } }, { a: 1, b: { c: 3 } })?.path).toBe("b.c"); + expect(firstDiff([1, 2, 3], [1, 9, 3])?.path).toBe("[1]"); + expect(firstDiff({ x: 1 }, { x: 1 })).toBeNull(); + }); +});