Skip to content

Add media-studio plugin - #3

Merged
a31521424 merged 1 commit into
mainfrom
add-media-studio
Nov 13, 2026
Merged

a31521424 merged 1 commit into
mainfrom
add-media-studio

Conversation

@a31521424

Copy link
Copy Markdown
Owner

Adds plugins/media-studio: review the images, video and audio agents make, for editors who run the whole edit through Paseo agents. Requires Paseo 0.15.3+ and ffmpeg on the daemon host.

  • preview_media agent tool and a card after each turn with the media the turn showed or created
  • Streaming player (fragmented MP4 over plugin RPC) with filmstrip, waveform, in/out, loop and editor keys; long-GOP seeks bridged with a near-lossless encode to the next keyframe (about 1 s)
  • Per-file playback memory across tab switches, reloads and re-renders; timecode entry; A/B flip between versions; safe-area and crop guides; EBU R128 loudness and true peak
  • Timecoded notes with the current frame, sent to busy agents without interrupting them (steer), kept per file and pinned on the timeline, copyable as a list
  • Media panel with resumable uploads and streamed downloads over the Paseo connection (works through direct connections, proxies and tunnels)
  • Files on SSH hosts (alias:/path) processed by that host's ffmpeg

Also adds it to the CI typecheck matrix and both READMEs, including the LLM install prompt.

Tested end to end with headless Chrome against a 0.15.3 daemon over a direct connection (local files, a Cloudflare tunnel, and an SSH GPU host). Not yet tested on real macOS / VideoToolbox hardware.

Review the images, video and audio agents make, for editors who run the
whole edit through Paseo agents:

- preview_media agent tool and a card after each turn with the media it
  showed or created
- streaming player (fragmented MP4 over plugin RPC) with filmstrip,
  waveform, in/out, loop and editor keys; long-GOP seeks bridged with a
  near-lossless encode to the next keyframe
- per-file playback memory across tab switches, reloads and re-renders;
  timecode entry; A/B flip between versions; safe-area and crop guides;
  EBU R128 loudness and true peak
- timecoded notes with the current frame, sent to busy agents without
  interrupting them, kept per file and pinned on the timeline
- Media panel with resumable uploads and streamed downloads over the
  Paseo connection
- files on SSH hosts (alias:/path) processed by that host's ffmpeg
Copilot AI balanced review requested due to automatic review settings November 12, 2026 02:19

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved upload-ID path traversal permits deletion outside the upload directory, while upload races and unbound download reads can silently overwrite or corrupt files.

12 open issues
What changed in this PR

Adds media-studio to Paseo’s plugin collection so users can review agent-generated media and send timecoded feedback without leaving the workspace.

Changes:

  • Adds media preview cards, streaming playback, timelines, playback memory, and review notes.
  • Adds resumable uploads, streamed downloads, and processing through local or SSH-hosted ffmpeg.
  • Documents installation and requirements, provides English and Chinese UI text, and adds CI typechecking.
File Description
README.zh-CN.md Documents features, requirements, and installation in Chinese.
README.md Documents features, requirements, and installation in English.
plugins/​media-studio/​tsconfig.json Configures strict TypeScript checking.
plugins/​media-studio/​shared/​settings.ts Defines host-level plugin settings.
plugins/​media-studio/​shared/​rpc.ts Defines media, transfer, and notes RPC contracts.
plugins/​media-studio/​shared/​media.ts Adds media classification, path, and timecode helpers.
plugins/​media-studio/​shared/​cards.ts Defines preview-card data and tool-input parsing.
plugins/​media-studio/​server/​transfer.ts Implements resumable uploads and chunked downloads.
plugins/​media-studio/​server/​stream.ts Implements streaming, transcoding, and seek bridges.
plugins/​media-studio/​server/​stills.ts Generates frames, filmstrips, waveforms, and image previews.
plugins/​media-studio/​server/​probe.ts Extracts and caches media metadata.
plugins/​media-studio/​server/​notes.ts Persists per-file review notes.
plugins/​media-studio/​server/​loudness.ts Measures audio loudness and true peak.
plugins/​media-studio/​server/​files.ts Scans media folders and selects destination filenames.
plugins/​media-studio/​server/​ffmpeg.ts Adds process execution and concurrency limits.
plugins/​media-studio/​server/​cache.ts Caches generated previews and trims disk usage.
plugins/​media-studio/​server/​agent.ts Registers the preview tool and after-turn cards.
plugins/​media-studio/​README.md Documents installation, architecture, and development.
plugins/​media-studio/​paseo-plugin.json Declares plugin identity and minimum Paseo version.
plugins/​media-studio/​package.json Declares packaging and development dependencies.
plugins/​media-studio/​OVERVIEW.md Explains workflows, controls, setup, and limitations.
plugins/​media-studio/​index.server.ts Wires settings, RPC handlers, and server cleanup.
plugins/​media-studio/​index.client.tsx Registers panels, cards, settings, and commands.
plugins/​media-studio/​client/​ui.tsx Adds shared themed controls and display helpers.
plugins/​media-studio/​client/​transfers.ts Manages transfer retries, controls, and reload handover.
plugins/​media-studio/​client/​timeline.tsx Renders media lanes, markers, and scrubbing controls.
plugins/​media-studio/​client/​state.ts Stores panel requests and transfer state.
plugins/​media-studio/​client/​settings-screen.tsx Provides configuration and host-tool checks.
plugins/​media-studio/​client/​review.tsx Composes and sends timecoded notes with frames.
plugins/​media-studio/​client/​queries.ts Fetches metadata, previews, and file changes.
plugins/​media-studio/​client/​playback.ts Remembers playback state per host and file.
plugins/​media-studio/​client/​notes.tsx Displays, navigates, and deletes review notes.
plugins/​media-studio/​client/​locales/​zh-CN.json Provides Chinese UI translations.
plugins/​media-studio/​client/​locales/​en.json Provides English UI translations.
plugins/​media-studio/​client/​i18n.ts Registers translation resources.
plugins/​media-studio/​client/​file-panel.tsx Integrates media viewing into file tabs.
plugins/​media-studio/​client/​card.tsx Renders inline media cards and thumbnails.
package.json Adds media-studio to the collection description.
.github/​workflows/​typecheck.yml Adds media-studio to the typecheck matrix.

Review effort: Balanced

const stem = name.slice(0, name.length - ext.length);
for (let index = 1; index < 1000; index += 1) {
const candidate = path.posix.join(dir, index === 1 ? name : `${stem} (${index})${ext}`);
if (!(await host.stat(candidate))) return candidate;
}

function recordFile(uploadId: string): string {
return path.join(STATE_DIR, `${uploadId}.json`);
Comment on lines +329 to +330
const data = await download.location.host.readRange(download.location.path, input.offset, length);
if (data.length < length) throw new Error("The file changed while it was being downloaded.");
Comment on lines +91 to +94
/** The frame at `time`, rounded to a tenth of a second so scrubbing reuses requests. */
export function useFrame(info: MediaInfo | undefined, time: number | null, width: number) {
const frame = useRpc(frameRpc);
const rounded = time === null ? null : Math.round(time * 10) / 10;
}

export function clearFinishedTransfers(): void {
transfers.set(transfers.get().filter((transfer) => transfer.status === "running"));
heavy(async () => {
const result = await host.run(
ffmpeg,
["-hide_banner", "-nostdin", "-nostats", "-i", file, "-map", "0:a:0", "-vn", "-af", "ebur128=peak=true:framelog=quiet", "-f", "null", "-"],
{ timeoutMs: 10 * 60_000, maxBytes: 1024 * 1024 },
);
const measured = parseEbur128(result.stderr);
if (result.code !== 0 && measured.integrated === null) throw new Error(result.stderr.trim().split("\n").at(-1) ?? "Loudness measurement failed");
child.on("close", (code) => {
clearTimeout(timer);
if (inBlock > 0) blocks.push(blockPeak);
if (blocks.length === 0) {
Comment on lines +247 to +249
await upload.host.rename(record.partPath, target);
uploads.delete(input.uploadId);
await deleteRecord(input.uploadId);
Comment on lines +83 to +87
const whole = Math.floor(safe + 1e-6);
const hms = `${pad(Math.floor(whole / 3600))}:${pad(Math.floor(whole / 60) % 60)}:${pad(whole % 60)}`;
if (fps && fps > 0) {
const nominal = Math.round(fps);
const frame = Math.min(nominal - 1, Math.floor((safe - whole) * fps + 1e-6));
@a31521424
a31521424 merged commit 955ad80 into main Nov 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants