feat(editor): add editable cursor motion paths

[YoneRai12]

Summary

• Add editable cursor-motion choreography with straight, arc, wave, loop, and overshoot presets, easing, cycle controls, and per-section speed.
• Turn the recorded cursor trace into independent motion sections at every recorded click and every detected cursor rest (300 ms or longer).
• Preserve rest sections as still holds by default instead of forcing one continuous animated path.
• Let editors auto-split an existing cursor-motion section at its recorded rests/clicks or manually split it at the playhead.
• Add 1x-4x motion-speed controls (0.1 steps and quick presets), with new sections defaulting to 2x while still landing on the recorded click at the original time.
• Add previous/next section navigation, visible speed labels on the timeline, and one-click application of the selected preset/speed/timing to every move section.
• Show distinct rest, click, and manual anchors together with the original recorded trajectory in the on-canvas edit mode.
• Integrate cursor motion with timeline selection, undo/redo, project persistence, MP4 export, and GIF export.

Why

The existing cursor renderer can smooth a recorded trace, but it cannot art-direct the movement between meaningful actions. A single continuous replacement path is also too coarse: pauses and clicks need to remain separate so each move can receive its own preset without making the cursor drift during an intended stop. Once split, those sections also need fast navigation and batch styling rather than repetitive per-item setup.

User impact

Editors place the playhead before recorded cursor activity and use the cursor-motion action. OpenScreen builds separate timeline items for moves and holds through the subsequent recorded clicks. Each item can be selected, styled, and sped up independently. Existing one-piece regions can be auto-split, and any selected item can be split manually at the playhead.

Increasing speed compresses motion into the end of the section, so the cursor waits longer at its source and still reaches the recorded click at exactly the original timestamp. Editors can then copy the chosen preset, speed, turns, and easing to all move sections without copying path-specific control handles.

The motion data uses normalized capture coordinates, so the editor representation is independent of Windows DPI and monitor layout. Preview and export share the same trajectory evaluator. Telemetry gaps (for example, when the cursor leaves the captured display) are not treated as false rests.

Validation

• Full test suite: 429/429 passing on the feature branch.
• Focused cursor-motion and persistence tests: 38/38 passing.
• npx tsc --noEmit.
• npm run build-vite.
• UTF-8 JSON parsing for every shipped locale's changed settings/timeline resources.
• Biome checks on all changed source files.

Notes

• Automatic choreography requires recorded cursor telemetry and at least one later real click.
• The first generated action and every detected move/hold are separate undoable timeline regions; generation itself remains one undo checkpoint.
• Repository-wide lint still reports pre-existing unrelated locale/CRLF drift. The new cursor-motion keys are present in every shipped locale.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: bdd9b33f-24a5-4feb-8286-d72df5915cb2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[EtienneLescot]

main has just been rebased onto the 1.8.0 line — a ~390-commit merge landing the v4 editor, the native Rust compositor and export path, and the Linux capture stack. Every open PR is out of date; sorry for the churn. Here's where this one stands:

Ten of the files this draft touches are gone. src/components/video-editor/ went from 68 files to 8 — only the model layer survives (editorDefaults, projectPersistence, types) — and the web exporter (frameRenderer, gifExporter, videoExporter) was replaced by the native compositor. A rebase won't resolve that.

Suggestion: close this draft and rebuild the cursor-motion editor against the v4 editor in src/components/ai-edition/ and the native export path. The concept is still wanted — the surface it was written against no longer exists.

[EtienneLescot]

Scratch what I said about closing this — I got it wrong.

The code is stale, but the proposal isn't: the interaction model, the presets, the split-at-rests idea all survive the refactor, and no issue captures them anywhere else. Closing would throw the idea out with the diff. Leaving it open.

If you pick it back up, src/lib/ai-edition/timeline/cursor-track.ts and src/lib/cursor/cursorPathSmoothing.ts are the natural hooks in the new tree.

[EtienneLescot]

Follow-up on keeping this open: I've built the half that isn't yours to rebuild. Branch feat/cursor-motion-contract.

It carries your motion model from the scene JSON into the compositor:

• SceneCursor.motion — a list of regions with resolved anchors, control point, preset, cycles, speed, easing. Same fields as your CursorMotionRegion.
• CursorTrack::with_motion in crates/compositor/src/cursor.rs — a direct port of your sampleCursorMotionRegion, presets and all, including the speed rounding (it feeds an exponent, so it isn't cosmetic).
• Applied before smoothing, at both the export and preview call sites. 121 crate tests pass, 7 new ones cover the presets.

The part worth knowing: preview and export both consume the same CursorTrack now — live.rs and the three compositor_*.rs all go through it. Your design goal of "the same pure sampling functions for preview and MP4 export" used to mean keeping two JS paths in sync. It's now free: one Rust implementation, both paths inherit it.

What's left is the feature itself, and it's yours: the editor UI (draggable control points, timeline segments, inspector) and the anchor discovery that produces the regions — rest detection, click splitting, manual splits. The compositor deliberately takes anchors already resolved, so that logic stays in the editor where it belongs.

Hooks in the new tree: src/lib/ai-edition/timeline/cursor-track.ts for the track model, V4Timeline.tsx for the timeline lane, and buildSceneDescription in src/native/sceneDescription.ts is where your regions need to land.

No pressure to pick it up — but the plumbing is out of the way if you do.

[EtienneLescot]

Correction to my last comment: I pointed you at V4Timeline.tsx as the place to rebuild the UI — but #116 has already wired the choreography into exactly those components, and I'd wrongly told you to close that one.

So the split is:

• feat: prototype editable cursor choreography for AI Edition #116 carries the UI against components that survived the merge (V4Timeline, FloatingInspector, Preview, PreviewCanvas, VirtualPreview) plus a CursorPreviewLayer.tsx worth keeping. That's the branch to harvest.
• This PR has the later revision of cursorMotion.ts — 581 lines against feat: prototype editable cursor choreography for AI Edition #116's 561, with the naming settled (CURSOR_MOTION_ANCHOR_KINDS). Worth diffing the two and keeping this one's module.

Its UI, though, is against VideoEditor.tsx / VideoPlayback.tsx / TimelineEditor.tsx / CursorMotionEditorOverlay.tsx, all deleted — that part doesn't survive.

The compositor work I described still stands whichever UI lands: feat/cursor-motion-contract carries the motion regions through the scene contract and samples the presets in Rust, for preview and export alike.