Skip to content

feat(cli): determinism guard — reframe lint enforces scene purity (0.6.42)#72

Merged
kiyeonjeon21 merged 1 commit into
mainfrom
feat/determinism-guard
Jun 20, 2026
Merged

feat(cli): determinism guard — reframe lint enforces scene purity (0.6.42)#72
kiyeonjeon21 merged 1 commit into
mainfrom
feat/determinism-guard

Conversation

@kiyeonjeon21

Copy link
Copy Markdown
Owner

What

OSS floor hardening for reframe-studio. Two audits this session (a fundamentals review + an agent-surface audit) converged on one confirmed crack in the determinism floor: scenes must be pure functions of time, but nothing enforced it. A scene using Math.random()/Date compiles and renders fine yet yields a different IR each compile, silently breaking the reproducibility the product's whole pitch rests on. The golden tests only cover IR→render; the source→IR step was unguarded.

How

reframe lint now verifies purity for .ts sources (the studio-readiness gate, alongside addressability):

  • bundle the scene once, evaluate it twice, deep-compare the two IRs
  • emit a non-deterministic-render finding pinned to the first differing address (e.g. nodes[0].props.x changed 0.42 → 0.97) plus a source-scan hint at the culprit construct
  • same --strict exit-code gate

New in-process checkDeterminism(path) exported via reframe-video/compile so an embedder can verify a freshly generated scene before trusting it.

Load-bearing detail: loadScene imports a data:…base64 URL, and Node caches ESM by URL — so a naive double-load returns the same cached module and misses module-level nondeterminism. The two evals append distinct cache-busting comments so the module re-evaluates fresh. The test asserts the impure scene IS caught, guarding this.

Verification

  • 395 tests pass (+4 determinism tests); typecheck clean; reframe-video build bundles checkDeterminism into dist/compile-api.js
  • Goldens byte-identical (this only adds a check; no IR/eval-path change)
  • reframe lint examples/scenes/photo-montage.ts → no determinism finding (pure); a temp scene with Math.random() → flagged with nodes[0].props.x, --strict exits 1

Scope

Focused on the one confirmed floor crack. Ranked follow-ups noted in the plan (not built): addressing-density lint, structured validation errors, renderFrameAt export, timing-walk unification. Version → 0.6.42.

🤖 Generated with Claude Code

…6.42)

Scenes must be pure functions of time, but nothing enforced it: a scene using
Math.random()/Date compiled to a different IR each time, silently breaking
reproducibility (goldens only cover IR→render, not source→IR). This is the floor
reframe-studio's "the agent generates it and it reproduces" promise stands on.

`reframe lint` now verifies purity for .ts sources: bundle once, evaluate twice,
emit a `non-deterministic-render` finding pinned to the first IR address that
differs (e.g. nodes[0].props.x: 0.42 → 0.97) plus a source-scan hint. Same
--strict gate as the addressability checks.

New in-process `checkDeterminism(path)` exported via reframe-video/compile so an
embedder can verify a fresh scene before trusting it. The two evals use distinct
cache-busting comments to defeat Node's data:-URL ESM module cache (without which
the second eval returns the cached module and misses module-level nondeterminism).

Additive — pure scenes report clean; only runs on source inputs (.json is the IR
itself). Goldens byte-identical (adds a check, no IR/eval-path change).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kiyeonjeon21 kiyeonjeon21 merged commit 26aceb9 into main Jun 20, 2026
1 check passed
@kiyeonjeon21 kiyeonjeon21 deleted the feat/determinism-guard branch June 20, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant