diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c72597..240ba72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,26 @@ All notable changes to BMad Manticore are documented here. Dates are ISO (YYYY-MM-DD). +## Unreleased + +### Cross-platform: Windows, Linux, and Intel Mac lanes (code-complete, pending real-hardware validation) + +- Transcription is cross-platform: transcribe.py gains an onnx-asr provider running the same parakeet-tdt-0.6b-v3 weights as an ONNX conversion, so verbatim fillers and 80 ms word timestamps carry over on Windows, Linux, and Intel Macs. New default provider `auto` picks parakeet-mlx on macOS Apple Silicon (byte-identical to the 1.0 reference lane) and onnx-asr everywhere else; dependencies select per platform via PEP 508 markers, CUDA machines escalate with `uv run --with "onnx-asr[gpu,hub]" python ` (the `python` command skips the script's cpu-extra dependency so onnxruntime-gpu never co-installs with onnxruntime, and the script warns when an NVIDIA GPU is visible but CUDA is unavailable), long audio is chunked in 20 s windows with 2 s overlap and merged deterministically with a seam-repair pass so boundary words are never duplicated or dropped, and words.json stays byte-compatible across lanes. Honesty rule: when the ONNX runtime exposes no per-token scores, every confidence reads 1.0 (no signal), never a fabricated number. The `[transcription]` default moved from "parakeet-mlx" to "auto"; whisper.cpp and faster-whisper are no longer the documented fallbacks. +- Install-time platform detection: check_deps.py now detects OS, CPU architecture, and GPU vendor (nvidia-smi, /sys/class/drm PCI ids, wmic/PowerShell fallbacks) and emits a platform verdict, in `--json` as a kebab-case `platform` object (os, arch, apple-silicon, gpu, gpu-detail, recommended{stack-file, transcription, torch-index, encoder-ladder, svg-rasterizer, fonts}) and as a table-mode stack block. Three new stack reference files at skills/mc-setup/references/stack-{macos,windows,linux}.md carry the per-OS defaults, Windows notes (short engines-path, LongPathsEnabled, gyan.dev build, OBS over Game Bar), Linux notes (PipeWire/Wayland capture, noto-color-emoji, the free-Resolve H.264/HEVC/AAC codec caveat), vMix and Wirecast alpha notes, and the per-OS DaVinci Resolve Fusion Scripts paths for the free-edition scripted-import lane. mc-setup reads the recommended stack file during the interview. +- Hardware encoder ladders: the final render and the VFR remux pick per-OS hardware encoders validated by a real one-frame test encode (Windows: h264_nvenc, then h264_qsv, then h264_amf; Linux: h264_nvenc, then h264_vaapi wired end to end with hwupload; libx264 fallback everywhere). The preview render, plain and graphics-composited alike, stays libx264 crf 28 veryfast by design on every OS. macOS videotoolbox behavior is byte-for-byte unchanged. +- mc-audio portability: the audio-lab venv interpreter resolves per OS (.venv/bin/python vs .venv\Scripts\python.exe), Windows machines with an NVIDIA GPU install torch from the PyTorch cu126 index (roughly 2.5 to 3 GB extra, surfaced in the consent message and the `--dry-run` torch field), and MusicGen/AudioLDM2 pick cuda, then mps, then cpu. macOS behavior is unchanged. +- Small portability fixes: edl_to_fcpxml.py emits valid Windows file URIs (file:///C:/... and UNC shares) via Path.as_uri() with byte-identical POSIX output; farm_asset.py resolves registered tools with a PATH lookup (Windows npm .cmd/.exe shims launch by bare name), documents POSIX quoting for headless templates on every OS, and refuses to pass arguments containing cmd.exe metacharacters (embedded double quotes, % ^ & | < >) to a .cmd/.bat shim, failing loudly with a re-register hint instead of letting cmd.exe corrupt or expand them; verify_ograf.py prints per-OS manual verification steps and, from a human terminal, serves the package and opens preview.html in the default browser itself. transcribe.py, edl_to_fcpxml.py, and render_final.py read and write their JSON, FCPXML, and concat-list artifacts with explicit UTF-8 so non-ASCII transcripts and paths survive on Windows locale codecs (cp1252). + +### Added + +- Final renders are loudness-normalized by default to -14 LUFS (two-pass ffmpeg loudnorm, TP -1.5, LRA 11; audio-only second pass with the video stream copied), configurable via `[render]` loudness-target and disable-able via `[render]` loudnorm = false or `--no-loudnorm`. Preview renders are never normalized. Silent audio skips the pass with a warning instead of failing. +- mc-package emits uploadable captions and a publishable transcript from the edited timeline: the new stdlib-only captions.py maps each EDL segment's word spans onto output-timeline times (reordered and multi-source edits included) and writes packaging/captions/final.srt, final.vtt, and transcript.md. Caption defaults: 42 chars per line, 2 lines, 1 to 7 s cues, splits at sentence ends, pauses, and cuts. A light filler/stutter cleanup applies to the caption rendition only (`--no-clean` keeps verbatim); transcript/words.json is never modified. +- mc-stream-pack produces and verifies the OBS WebM VP9 alpha deliverable from a ProRes 4444 master in one command (render_verify.py `--transcode-webm`, with `--webm-crf`), with clear errors when ffmpeg lacks libvpx-vp9 or the master has no alpha, plus a pixfmt-failure hint carrying the exact re-encode flags. The stream-pack copy of render_verify.py is now a superset of mc-graphics' copy. + +### Documentation + +- README platform matrix and the user guide rewritten for the cross-platform reality; the Resolve handoff reference gains the free-edition Fusion Scripts install paths, the Linux free-edition codec caveat, and an opt-in pointer to the community samuelgursky/davinci-resolve-mcp server for Studio users (Manticore itself still requires no MCP server). + ## 1.0.1 - 2026-07-07 ### Fixed diff --git a/README.md b/README.md index 7bfab67..c377fd4 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # BMad Manticore -[![Version](https://img.shields.io/badge/version-1.0.0-blue)](.claude-plugin/marketplace.json) +[![Version](https://img.shields.io/badge/version-1.0.1-blue)](.claude-plugin/marketplace.json) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) [![Python Version](https://img.shields.io/badge/python-%3E%3D3.11-blue?logo=python&logoColor=white)](https://www.python.org) [![uv](https://img.shields.io/badge/uv-package%20manager-blueviolet?logo=uv)](https://docs.astral.sh/uv/) @@ -23,7 +23,7 @@ Manticore is an AI video production pipeline for content creators, packaged as a - A render at every step: a fast low-res preview after every cut approval, the same preview re-rendered with graphics composited once the graphics stage has rendered the overlays, and a final-quality render offered at the last gate. The editor timeline export, edl.json, cutplan, and overlays are always written too, so jumping into your editor never loses work. - Motion graphics planned under creativity mandates and your visual density tier, styled by your Production Bible, delivered as brand-themed alpha overlays (ProRes 4444, works in every NLE), anchored to the exact words you speak. - CTAs planned like beats: your configured CTA inventory placed at research-backed seams, approved by you in the same gate as every other graphic. -- Titles, thumbnails (A/B pairs for series), description, and chapters, built around a packaging promise you approved before the script was written. +- Titles, thumbnails (A/B pairs for series), description, chapters, and uploadable SRT/VTT captions with a publishable transcript of the edited timeline, built around a packaging promise you approved before the script was written. - A pipeline that gets smarter every video: your post-publish notes edit the pipeline's own files. ## How a video happens @@ -71,10 +71,13 @@ Full walkthrough: [Configure your own Manticore studio](docs/user-guide.md). | Platform | Status | |---|---| -| macOS on Apple Silicon | The reference platform. Everything runs, including the default transcription lane (parakeet-mlx, free and local). | -| macOS on Intel, Linux, Windows | The pipeline scripts run (uv, ffmpeg, node), but the default transcription lane does not: parakeet-mlx is Apple-Silicon-only. mc-setup's dependency check flags this and points at local whisper.cpp or faster-whisper as fallbacks (they normalize fillers away, so cut quality drops). A supported cross-platform transcription lane is planned; see [TODO.md](TODO.md). | +| macOS on Apple Silicon | The reference platform. Everything runs and is validated here, including the reference transcription lane (parakeet-mlx, free and local) and videotoolbox hardware encode. | +| macOS on Intel | Code-complete: onnx-asr on CPU runs the same parakeet-tdt-0.6b-v3 weights as an ONNX conversion, so verbatim fillers and word timestamps carry over, and videotoolbox encode still applies. Like the Windows and Linux paths, this lane is reviewed and unit-tested but pending validation on real Intel Mac hardware; treat the first run as a shakedown. | +| Windows, Linux | Code-complete. Transcription uses onnx-asr with the same parakeet weights (CPU by default, CUDA via a one-flag escalation); final-render hardware encode probes h264_nvenc, h264_qsv, h264_amf on Windows and h264_nvenc, h264_vaapi on Linux with a real one-frame test encode, libx264 fallback everywhere; the audio farm resolves its venv per OS and installs CUDA torch wheels from the PyTorch index on Windows NVIDIA boxes (with consent). Honest caveat: these paths are reviewed and unit-tested but still pending validation on real Windows and Linux hardware; treat the first run as a shakedown and report what you hit. | -One tool-specific note: Ecamm Live (a planned stream-pack delivery target) is macOS-only. OBS lanes work everywhere OBS does. +mc-setup's dependency check now detects OS, CPU architecture, and GPU vendor and selects a per-platform stack reference (macOS, Windows, Linux) that drives the setup defaults: transcription lane, torch wheel source, encoder ladder, SVG rasterizer, and fonts approach. + +One tool-specific note: Ecamm Live (a planned stream-pack delivery target) is macOS-only, and vMix and Wirecast skip Linux; OBS lanes work everywhere OBS does. ## Works with the tools you already have @@ -83,7 +86,7 @@ Manticore orchestrates tools; it does not replace them. The defaults are local a | Tool | What it provides | Cost model | |---|---|---| | Your editor (DaVinci Resolve, Final Cut Pro, Premiere Pro, Descript, anything) | Where you can finish any time you want to. Resolve/FCP get an FCPXML timeline; Premiere users work from the cut plan, edl.json, and the rendered preview/final until the xmeml lane lands; `timeline-format = "none"` gives you the cut plan, edl.json, and renders to apply in any tool | You already have it | -| parakeet-mlx | Word-level cutting transcripts with verbatim fillers (the "um"s are exactly what gets cut) | Free, runs locally on Apple Silicon, no API key | +| parakeet-mlx / onnx-asr | Word-level cutting transcripts with verbatim fillers (the "um"s are exactly what gets cut); parakeet-mlx on Apple Silicon, onnx-asr running the same parakeet-tdt-0.6b-v3 weights on Windows, Linux, and Intel Mac | Free, runs locally, no API key | | Kokoro-82M (kokoro-onnx) | TTS narration and two-host dialogue for the mc-audio lane (stock voices, no cloning) | Free, local, faster than realtime on CPU | | MusicGen-small + AudioLDM2 | Instrumental music beds and SFX, farmed locally by mc-audio | Free, local, ungated models | | HyperFrames and Remotion | Motion graphics engines for overlay beats, stingers, and karaoke captions | Free (Remotion is free for companies up to 3 people) | @@ -130,7 +133,7 @@ Seven ship by default: talking-head, screen-tutorial (real UI only, generated b- | mc-ograf | Editable broadcast graphics (DaVinci Resolve 21+ and OBS/SPX-GC) | | mc-assets | Farm b-roll stills/clips via your registered CLI tools (metered APIs opt-in), under generative-editing safety rules | | mc-audio | Farm sound, local-first: TTS narration and two-host dialogue (Kokoro-82M), instrumental beds (MusicGen-small), SFX (AudioLDM2); paid lanes opt-in | -| mc-package | Titles, thumbnails (verified at 120px), description, chapters, series A/B pairs, live-event mode | +| mc-package | Titles, thumbnails (verified at 120px), description, chapters, SRT/VTT captions and transcript, series A/B pairs, live-event mode | | mc-stream-pack | A complete branded OBS livestream asset pack | | mc-retro | Your post-publish notes edit the pipeline's own files, plus the post-publish wrap lane | @@ -145,7 +148,7 @@ Taste lives in files (your voice bible, Production Bible, format profiles, brand - Proven in production: the full cut lane (parakeet-mlx word-level transcription validated on real footage, cut candidate detection, edl.json, FCPXML export, preview render with boundary-frame verification), Manny as the front door, setup and dependency checking, config resolution, project scaffolding, the OBS stream pack, and the retro loop. - New in 1.0, implemented and unit-tested, with the least real-project mileage: the render lane (composited preview and the offered final render), the expanded setup interview (render consent, video style, creator-emulation takeaways, headshots, guided voice bible), the Production Bible, creativity mandates and the CTA system, footage-first ingest and the livestream-vod format, series support, graphics render verification, the graphics toolkit (HTML render, snug framing, design-prompting lane), CLI-registry asset farming, and the mc-audio local sound lanes (validated end to end on Apple Silicon 2026-07-07). - The writing lane (braindump, outline, script) is the core promise and is wired end to end with live blacklist linting; it has had the least real-video exercise of the core stages, so treat your first run through it as a shakedown and feed mc-retro afterward. -- Planned: Premiere (xmeml) and CMX3600 EDL export lanes, per-episode stream packs with the Ecamm target (the named 1.0.x fast-follow), multitrack recording support, local-first TTS/SFX/music lanes, and a research/show-prep skill. See [TODO.md](TODO.md) for the full roadmap. +- Planned: Premiere (xmeml) and CMX3600 EDL export lanes, per-episode stream packs with the Ecamm target (the named 1.0.x fast-follow), multitrack recording support, the remaining audio lanes (full songs with vocals, plus the paid opt-in rungs of the audio ladder), and a research/show-prep skill. See [TODO.md](TODO.md) for the full roadmap. ## Part of the BMad ecosystem diff --git a/TODO.md b/TODO.md index 0bdff0f..8ca38e4 100644 --- a/TODO.md +++ b/TODO.md @@ -6,7 +6,7 @@ State as of 2026-07-07, the 1.0.0 release. Read AGENTS.md first (module conventi - Per-episode stream packs and the Ecamm lane (the named 1.0.x fast-follow): mc-stream-pack gains a pre-show per-episode pack lane (topic popups, CTAs, lower thirds mined from the episode plan before the show, delivered as switchable scenes) with the two-tier asset rule (evergreen chrome once into series `common/`, topic graphics per episode). The `[live]` tool key (obs, ecamm, other) already ships and is interviewed at setup; the OBS lane keeps HTML browser sources and WebM stingers; the Ecamm/other lane delivers baked PNG / ProRes 4444 alpha scene stills and loops, a ProRes stinger, a countdown safe-zone spec with a --guides render, and a tool-specific HANDOFF.md. Ecamm Live is macOS-only. Scheduled-livestream packaging (mc-package live-event mode, two-asset thumbnail rule) rides along. - farm_asset.py metered API lane (xAI Imagine REST image ~$0.02 and video ~$0.05/s submit/poll/download; Veo 3.1 via the Gemini API as the escalation lane). Registered CLI tools are the only implemented farming lane in 1.0; the API lane ships opt-in only, never as a default. -- resolve_import.py: push the exported timeline into a running DaVinci Resolve. Requires Resolve Studio (the scripting API is not in the free edition); the mc-cut offer stays gated on the script's implemented status. Native scripting remains the documented path; no MCP dependency. +- resolve_import.py: push the exported timeline into a running DaVinci Resolve. External scripting requires Resolve Studio; free-edition users will run it from inside Resolve via the Fusion Scripts menu (the per-OS install paths are already documented in the mc-setup stack references and mc-ograf's resolve-workflow reference). The mc-cut offer stays gated on the script's implemented status. Native scripting remains the documented path; no MCP dependency. - HyperFrames engine workspace initialization at a pinned version on the first real graphics run (upstream is pre-1.0 and moves fast; v0.7.26 as of 2026-07-03). ## 1.x roadmap @@ -42,10 +42,12 @@ What mc-audio does not cover yet (the shipped ladder, validation record, and lim - xmeml (Premiere Pro) and edl (CMX3600) export lanes alongside the implemented fcpxml exporter; OpenTimelineIO adapters are the likely implementation path. Until then Premiere users work from cutplan.md, edl.json, and the always-rendered preview/final. -### Cross-platform transcription +### Transcription: metered opt-in providers -- A supported non-Apple-Silicon local lane. Today the documented fallbacks are whisper.cpp and faster-whisper (they normalize fillers away, so cut quality drops); parakeet-mlx stays the reference on Apple Silicon. -- Metered API providers behind the same `[transcription]` switch if demand shows up, opt-in only: deepgram-nova3 (keyterm biasing), elevenlabs-scribe (same output shape as the parakeet lane). +The cross-platform local lane landed (onnx-asr running the same parakeet-tdt-0.6b-v3 weights on Windows, Linux, and Intel Mac; see CHANGELOG Unreleased). What remains: + +- Metered API providers behind the same `[transcription]` switch if demand shows up, opt-in only: deepgram-nova3 (keyterm biasing), elevenlabs-scribe (same output shape as the parakeet lane). Also the documented cloud tier for non-European-language creators (Parakeet v3 covers 25 European languages). +- Real-hardware validation of the onnx-asr lane on Windows and Linux (A/B against parakeet-mlx on identical audio comparing word text, starts, AND gap_before/gap_after values, since the onnx lane derives word ends from start-only timestamps and silence-based cutting rides on the gaps; CUDA escalation; chunk-boundary quality). ### Shorts karaoke captions diff --git a/docs/index.html b/docs/index.html index 9b4890f..cd7a1ab 100644 --- a/docs/index.html +++ b/docs/index.html @@ -53,7 +53,7 @@ Manny the Manticore, wings spread, diving head-on over a waterfall at red sunset

BMad Manticore · the guides

-

An AI video production pipeline as skills: brain dump to a rough cut sitting in your editor, in your own words, with approval gates at every taste decision. Manny the Manticore runs the studio; these guides show you around it.

+

An AI video production pipeline as skills: from brain dump to a rendered, graphics-rich video, in your own words, with the editor timeline always written alongside and approval gates at every taste decision. Manny the Manticore runs the studio; these guides show you around it.

▸Working with Manny

diff --git a/docs/manny-under-the-hood.html b/docs/manny-under-the-hood.html index efef0c2..bc63a77 100644 --- a/docs/manny-under-the-hood.html +++ b/docs/manny-under-the-hood.html @@ -265,7 +265,7 @@

The tools and models behind each stage

Transcription

working
-

parakeet-mlx: word-level, keeps the "um"s verbatim, free and local on Apple Silicon. deepgram-nova3 and elevenlabs-scribe sit behind the same switch as opt-in lanes.

+

parakeet-mlx on Apple Silicon: word-level, keeps the "um"s verbatim, free and local. onnx-asr runs the same parakeet weights on Windows, Linux, and Intel Mac; that lane is code-complete but pending real-hardware validation, so treat a first run as a shakedown. deepgram-nova3 and elevenlabs-scribe sit behind the same switch as opt-in lanes.

Image + video gen

opt-in
@@ -277,7 +277,7 @@

The tools and models behind each stage

Rendering

working
-

ffmpeg with hardware encode (videotoolbox on macOS, libx264 fallback), segment-parallel, disk-preflighted.

+

ffmpeg, segment-parallel and disk-preflighted, with videotoolbox hardware encode on macOS; the final render is loudness-normalized to -14 LUFS. The Windows (nvenc, qsv, amf) and Linux (nvenc, vaapi) encoder ladders with libx264 fallback are code-complete, pending real-hardware validation.

Graphics render

working
@@ -508,7 +508,7 @@

One EDL, two renders, every editor

final render GATE 4, OFFERED - delivery res, hardware encode + delivery res, hw encode, -14 LUFS

The FCPXML timeline exports on every cut approval (Resolve and Final Cut import it natively); xmeml (Premiere) and CMX3600 EDL are planned, so those users work from edl.json, cutplan.md, and the renders meanwhile. Same approved EDL, same composite core, so the preview you signed off on is exactly what the final bakes.

diff --git a/docs/user-guide.md b/docs/user-guide.md index 8c0606b..34dd439 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -39,7 +39,7 @@ Say "talk to Manny" and he routes you here, or say "run manticore setup" directl What the interview covers, in order: -- Dependencies and platform: uv (required; runs all pipeline scripts), ffmpeg (with ffprobe), node/npx, git, optionally yt-dlp. It checks, you approve any installs. It also runs a platform gate: the default transcription lane is Apple-Silicon-only, and on other machines it points you at the documented fallbacks (see section 6). +- Dependencies and platform: uv (required; runs all pipeline scripts), ffmpeg (with ffprobe), node/npx, git, optionally yt-dlp. It checks, you approve any installs. It also detects your OS, CPU architecture, and GPU vendor and picks the per-platform stack reference (macOS, Windows, or Linux) that drives the rest of the defaults: transcription lane, torch wheel source, encoder ladder, SVG rasterizer, fonts approach (see section 6 for transcription). - You: name, channel, the links that go in your video descriptions, your speaking rate (the guided voice-bible build later measures it from a real transcript, which beats the generic 145 wpm every time). - Video defaults: record resolution, delivery resolution, fps. Offer it a recent recording and it fills the values from ffprobe instead of guessing. - Live tool: obs, ecamm, or other, which drives the stream-pack lane's deliverable format. Any recurring shows or series you produce get noted for series folders and packaging templates. @@ -57,7 +57,7 @@ Manticore renders by default, and setup asks you to confirm it rather than assum - A fast low-res preview render is produced after every cut-plan approval, so every iteration is watchable. - Once the graphics stage has rendered the overlays, the preview re-renders with them composited, so you iterate on overlays and CTAs visually. -- At the final gate, a final-quality render is offered (delivery resolution and codec per your config). +- At the final gate, a final-quality render is offered (delivery resolution and codec per your config), loudness-normalized to -14 LUFS by default (two-pass ffmpeg loudnorm; the preview is never normalized; `[render]` loudness-target and loudnorm are the knobs) with hardware encode probed per platform (videotoolbox on macOS; nvenc, qsv, amf on Windows; nvenc, vaapi on Linux; libx264 fallback). - The editor timeline export and all assets (edl.json, cutplan.md, overlays) are ALWAYS created alongside, so you can jump into Resolve, Premiere, or any editor at any step without losing work. Decline it and previews and finals become offers the pipeline makes instead of automatic outputs; the timeline export and assets remain always-on either way. @@ -89,15 +89,15 @@ The Production Bible evolves after setup: mc-retro routes every visual-style not ## 6. Transcription for the cut stage -The cut stage needs a word-level transcript with verbatim fillers (the "um"s and restarts are exactly what gets cut). The default provider is parakeet-mlx: free, local, word timestamps, fillers preserved verbatim, no API key. The model downloads once on first run. +The cut stage needs a word-level transcript with verbatim fillers (the "um"s and restarts are exactly what gets cut). The default provider is `auto`: free, local, word timestamps, fillers preserved verbatim, no API key. On macOS Apple Silicon it runs parakeet-mlx (the reference lane); on Windows, Linux, and Intel Mac it runs onnx-asr with the very same parakeet-tdt-0.6b-v3 weights as an ONNX conversion, so filler preservation and 80 ms word timestamps carry over. The model downloads once on first run, and the script's dependencies are selected automatically per platform; no venv setup. -Platform honesty: parakeet-mlx runs only on macOS Apple Silicon. On other machines, setup's dependency check flags it and points at local whisper.cpp or faster-whisper as fallbacks; they normalize fillers away, so cut quality drops, and a supported cross-platform lane is on the roadmap. Metered API providers exist behind the same `[transcription]` switch as explicit opt-in choices only; nothing metered is configured unless you choose it. +Two platform notes: on a CUDA machine the onnx-asr lane escalates to GPU with `uv run --with "onnx-asr[gpu,hub]" python /transcribe.py ...` (the `python` command matters: it skips the script's built-in cpu dependency so the GPU and CPU onnxruntime builds never co-install, a combination ONNX Runtime does not support; the script prints a warning when an NVIDIA GPU is present but CUDA support is missing), and when the ONNX runtime exposes no per-token scores, word confidences read 1.0 (no signal, never a fabricated number). Long recordings on the onnx-asr lane are chunked in 20 second windows with 2 second overlap and merged deterministically. Metered API providers exist behind the same `[transcription]` switch as explicit opt-in choices only; nothing metered is configured unless you choose it. ## 7. Your editor Render-first does not lock you out of your editor; the exit ramp is always built. Tell mc-setup what you finish in: -- DaVinci Resolve or Final Cut Pro: an FCPXML timeline of trimmable clips (implemented), exported on every cut approval. Resolve 21+ users can also set `ograf-editable = true` to receive lower thirds as OGraf packages that stay editable inside Resolve's Inspector. +- DaVinci Resolve or Final Cut Pro: an FCPXML timeline of trimmable clips (implemented), exported on every cut approval. Resolve 21+ users can also set `ograf-editable = true` to receive lower thirds as OGraf packages that stay editable inside Resolve's Inspector. Linux note: the free edition of Resolve cannot decode or encode H.264, H.265, or AAC, so the timeline imports but mp4 media needs transcoding to ProRes or DNxHR first (or Resolve Studio). - Premiere Pro: the xmeml export lane has not landed yet, so Premiere users work from the cut plan, edl.json, and the rendered preview/final, which map 1:1 onto manual cuts. - Descript or anything else: set `timeline-format = "none"`. You get the word-level transcript, cut decisions with reasons, and the renders; you apply the cuts in your tool. @@ -125,7 +125,7 @@ A lane with no good answer stays empty: mc-assets stops and asks at farming time One standing rule regardless of lane: generated footage is for atmosphere and story beats. Anything showing a user interface or text that must read correctly comes from real screen recordings, because AI-generated UI and text render as convincing-at-a-glance gibberish. -Sound follows the same local-first pattern through the mc-audio service skill: TTS narration and two-host dialogue (Kokoro-82M; stock voices, no cloning, so narration in your own voice still means recording it), instrumental music beds (MusicGen-small), and SFX (AudioLDM2) all run free and local. Setup confirms these lanes and offers to build the engine workspace at `manticore/engines/audio-lab` (a several-GB venv, ~340 MB of voice models now, ~5 GB of model cache on the first music or SFX run; nothing downloads without your go-ahead). Full songs with vocals have no validated local lane yet, and paid audio lanes (ElevenLabs, Gemini TTS) are explicit opt-ins. +Sound follows the same local-first pattern through the mc-audio service skill: TTS narration and two-host dialogue (Kokoro-82M; stock voices, no cloning, so narration in your own voice still means recording it), instrumental music beds (MusicGen-small), and SFX (AudioLDM2) all run free and local. The engines pick cuda, then mps, then cpu automatically. Setup confirms these lanes and offers to build the engine workspace at `manticore/engines/audio-lab` (a several-GB venv, ~340 MB of voice models now, ~5 GB of model cache on the first music or SFX run; on Windows with an NVIDIA GPU, torch comes from the PyTorch cu126 index and adds roughly 2.5 to 3 GB more; nothing downloads without your go-ahead). Full songs with vocals have no validated local lane yet, and paid audio lanes (ElevenLabs, Gemini TTS) are explicit opt-ins. ## 9. Your first video @@ -143,7 +143,7 @@ Idea-first, the full pipeline: Footage-first, when the video already exists: 1. Hand Manny the file ("cut this VOD", "make a video from this recording"). mc-new's ingest mode registers the source and writes a post-production stage list that starts at cut. -2. The same gates apply from the cut stage onward: cut plan, beats with CTAs mined from the transcript, graphics, packaging with dual-timeline chapters, the final render offer. +2. The same gates apply from the cut stage onward: cut plan, beats with CTAs mined from the transcript, graphics, packaging with dual-timeline chapters plus SRT/VTT captions and a publishable transcript, the final render offer. ## 10. Formats diff --git a/skills/mc-assets/SKILL.md b/skills/mc-assets/SKILL.md index 7161288..d2f3c60 100644 --- a/skills/mc-assets/SKILL.md +++ b/skills/mc-assets/SKILL.md @@ -10,7 +10,7 @@ description: Source and farm the stills and b-roll the beat table calls for, dri 1. Load the studio config (`uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key modules.manticore`; empty means mc-setup has not run: stop and route the creator there) and this skill's own surface (`uv run {project-root}/_bmad/scripts/resolve_customization.py --skill {skill-root}`; run `{workflow.activation_steps_prepend}` now, `{workflow.activation_steps_append}` after this step, and hold `{workflow.persistent_facts}` as standing context). Resolve `paths` values against `{project-root}`. Read `project.json` (stage `assets`), the beat rows in `beats/beats.md` whose `asset` column names a farmed asset (tolerate 0.x tables without the column: a missing `asset` is `null`, nothing to farm for that row), the format profile, `{brand-path}/production-bible.md` (its image-type policy and sourcing hierarchy govern every choice below), and `{skill-root}/references/generative-editing-rules.md` (hard rules for every generative lane; the checklist mirrors them). If the profile says `generated_broll: banned`, stop and report; something upstream is wrong. 2. For each needed asset, pick the source per the Production Bible's image-type policy and the sourcing hierarchy: real verified imagery first (the creator's own libraries and their locations per the bible, screen recordings, verified photos), generative only for what does not exist, a hand-built text card last. Claim-bearing text accuracy belongs to the SVG/diagrammatic lane (route to mc-graphics; not a farming job). When an asset needs the creator (or any person) in it, pass the approved original photo from `{brand-path}/headshots/` as `--ref` and say it in the prompt: "use the person in this image to {what the asset needs}"; the image models handle the likeness from there. 3. Resolve the generative lane per `[assets]` in the config (`image-provider`, `video-provider`, `escalation-provider`). Each value names either a registered `[[tools]]` CLI (the working default in 1.0) or a metered API lane (`xai-api`, `veo-api`: not implemented in 1.0, planned for 1.0.x). If the lane an asset needs is empty or names nothing registered, STOP and ask the creator which registered tool to use (route to mc-setup step 5 if none exists); never fall back to a metered lane the creator did not explicitly choose. -4. Farm each asset by tool NAME through the farming script, so no session ever has to remember how a tool is driven: first save the resolved config as JSON (`uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key modules.manticore > `), then write the prompt per the generative editing rules (concrete subject, camera and framing, lighting, mood, brand-adjacent palette where it fits; quote exact strings for any short text that must appear; spell out physics for wardrobe and object edits; always list what must NOT change; ask for margins near canvas edges; expression variants from one reference use "use this person but have them {expression}"), then run `uv run {skill-root}/scripts/farm_asset.py --kind image|video --prompt "..." --provider --config --out-dir [--seconds 8] [--ref ]`. The script resolves the provider against `[[tools]]`, surfaces the tool's `notes` (the persistent memory for driving it), substitutes the prompt into its `headless` invocation, runs it in `--out-dir` with the environment passed through, and appends provenance rows to `assets/work/manifest.json`. Escalate to `escalation-provider` only for hero shots where realism must not wobble. Long jobs (video generation, large batches) run in the background with proactive progress reports; never leave the creator staring at a silent stage. +4. Farm each asset by tool NAME through the farming script, so no session ever has to remember how a tool is driven: first save the resolved config as JSON (`uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key modules.manticore > `), then write the prompt per the generative editing rules (concrete subject, camera and framing, lighting, mood, brand-adjacent palette where it fits; quote exact strings for any short text that must appear; spell out physics for wardrobe and object edits; always list what must NOT change; ask for margins near canvas edges; expression variants from one reference use "use this person but have them {expression}"), then run `uv run {skill-root}/scripts/farm_asset.py --kind image|video --prompt "..." --provider --config --out-dir [--seconds 8] [--ref ]`. The script resolves the provider against `[[tools]]`, surfaces the tool's `notes` (the persistent memory for driving it), substitutes the prompt into its `headless` invocation (parsed with POSIX shell quoting on every OS; the tool name is resolved via PATH lookup, so Windows npm shims launch by bare name), runs it in `--out-dir` with the environment passed through, and appends provenance rows to `assets/work/manifest.json`. Escalate to `escalation-provider` only for hero shots where realism must not wobble. Long jobs (video generation, large batches) run in the background with proactive progress reports; never leave the creator staring at a silent stage. 5. Revisions regenerate from the ORIGINAL source assets with every accumulated fix expressed in one prompt; never feed a generated output back in as the base for the next edit (a revision of a revision degrades like a photocopy of a photocopy; re-send all the originals with the improved prompt instead). Small deterministic fixes (a logo swap, one wrong text line, a color correction) are composited programmatically (rsvg, ffmpeg), never regenerated. 6. Self-inspect every output at zoom against the request BEFORE the creator sees it: the gesture points at the right target, the expression matches, no anatomical or rendering artifacts, any text is exactly the requested string. An output that fails inspection is retried, not shown. Standing rule: generated footage never depicts UI or text that must be accurate; real UI comes from screen recordings. 7. Blessed slots: candidates, drafts, and retries stay in `assets/work/`. When the creator picks, copy exactly one blessed file per beat-row slot into `assets/`, named by its asset id, and append its row from the work manifest to `assets/manifest.json` (file, kind, prompt, provider, model, cost, date). Report total spend where a lane reports cost (registered CLI tools draw on the creator's subscription and report none). diff --git a/skills/mc-assets/scripts/farm_asset.py b/skills/mc-assets/scripts/farm_asset.py index cba6388..2046ef1 100644 --- a/skills/mc-assets/scripts/farm_asset.py +++ b/skills/mc-assets/scripts/farm_asset.py @@ -35,6 +35,21 @@ passed through untouched, so the tool's own auth (subscription login, env keys) just works; tool output streams live for progress. New files that appear under --out-dir are the result. + + Template quoting is POSIX shell quoting on EVERY OS (shlex with + posix=True): quote arguments with single or double quotes exactly as + in a POSIX shell, even on Windows; backslash is an escape character, + so prefer forward slashes in any path baked into a template. Before + execution the command's argv[0] is resolved with shutil.which(), so + npm-installed tools registered by bare name launch on Windows too + (which() finds the .cmd/.exe shim via PATH + PATHEXT, which a + shell-free subprocess cannot do by itself). One Windows guard: when + argv[0] resolves to a .cmd/.bat shim, cmd.exe reparses the argument + line with its own quoting and expands metacharacters, so arguments + containing any of " % ^ & | < > (for example a prompt quoting exact + on-image text) would reach the tool corrupted; the script refuses to + run that combination (exit 2) and says to re-register the tool via + its real executable (.exe, or `node `). api (NOT IMPLEMENTED in 1.0) xai-api (xAI Imagine REST) and veo-api (Veo via Gemini REST) are the metered lane, deferred to 1.0.x: see TODO.md build-order item 3. The @@ -75,6 +90,7 @@ import datetime import json import shlex +import shutil import subprocess import sys from pathlib import Path @@ -108,8 +124,11 @@ def plan_invocation(tool, kind, prompt, out_dir, ref=None, seconds=DEFAULT_SECON """Build the argv for a registered tool's headless template. Substitutes , , , , , inside - the shlex-split tokens, so quoting in the template is preserved. Raises - ValueError when the template and the arguments disagree.""" + the shlex-split tokens, so quoting in the template is preserved. The + template is parsed with POSIX shell quoting on every OS (posix=True made + explicit): registered command strings tokenize identically on macOS, + Linux, and Windows. Raises ValueError when the template and the + arguments disagree.""" headless = (tool.get("headless") or "").strip() if not headless: raise ValueError(f"tool {tool.get('name')!r} has no headless invocation; " @@ -134,7 +153,7 @@ def plan_invocation(tool, kind, prompt, out_dir, ref=None, seconds=DEFAULT_SECON "model": models[0] if models else "", } argv = [] - for token in shlex.split(headless): + for token in shlex.split(headless, posix=True): for key, value in subs.items(): placeholder = f"<{key}>" if placeholder in token: @@ -143,6 +162,58 @@ def plan_invocation(tool, kind, prompt, out_dir, ref=None, seconds=DEFAULT_SECON return argv +def resolve_executable(command): + """Return command with argv[0] resolved through shutil.which(). + + subprocess.run without a shell cannot launch Windows npm shims + (.cmd/.bat/.exe) by bare name; which() finds the concrete file via + PATH + PATHEXT (and is a no-op path normalization elsewhere). When + which() finds nothing the command is returned unchanged so the + FileNotFoundError path still reports the registered name.""" + if not command: + return command + resolved = shutil.which(command[0]) + if resolved: + return [resolved, *command[1:]] + return command + + +# Characters cmd.exe rewrites or interprets inside .cmd/.bat argument lines. +CMD_SHIM_UNSAFE = ('"', "%", "^", "&", "|", "<", ">", "\r", "\n") + + +def check_cmd_shim_safety(command): + """Refuse to pass cmd.exe-mangled arguments to a .cmd/.bat shim (pure). + + When argv[0] resolves to a Windows .cmd/.bat file, CreateProcess hands + the command line to cmd.exe, which parses quoting differently from the + MSVCRT rules subprocess used to build it and expands metacharacters + (%VAR%, ^, &, |, <, >): an embedded double quote, e.g. a prompt quoting + exact thumbnail text per generative rule 5, reaches the tool corrupted, + and %WORD% or & can be expanded or executed (BatBadBut-class argument + injection). There is no escaping that is safe for every shim, so this + guard fails loudly instead of generating a wrong or dangerous command. + Raises ValueError naming the offending characters when argv[0] ends in + .cmd/.bat and any later argument contains one; returns the command + unchanged otherwise (all other executables, including .exe, are safe).""" + if not command: + return command + if not str(command[0]).lower().endswith((".cmd", ".bat")): + return command + for arg in command[1:]: + bad = sorted({c for c in CMD_SHIM_UNSAFE if c in arg}) + if bad: + shown = " ".join("newline" if c in "\r\n" else c for c in bad) + raise ValueError( + f"{command[0]} is a cmd.exe batch shim and an argument " + f"contains character(s) cmd.exe would mangle or expand " + f"({shown}); refusing to run with corrupted arguments. " + "Re-register the tool via mc-setup pointing at the real " + "executable (the .exe, or `node ` for npm " + "tools), or remove these characters from the prompt.") + return command + + def manifest_rows(new_files, kind, prompt, provider, model): """One provenance row per new file, in the documented shape.""" today = datetime.date.today().isoformat() @@ -244,13 +315,20 @@ def main(argv=None): return 0 before = snapshot(out_dir) + run_argv = resolve_executable(command) + try: + check_cmd_shim_safety(run_argv) + except ValueError as e: + print(str(e), file=sys.stderr) + return 2 try: # cwd = out_dir so tools that write to their working directory land # here; the environment is inherited untouched (env passthrough); - # output streams live so long generations show progress. - result = subprocess.run(command, cwd=out_dir) + # output streams live so long generations show progress. argv[0] is + # pre-resolved via shutil.which so Windows .cmd/.exe shims launch. + result = subprocess.run(run_argv, cwd=out_dir) except FileNotFoundError: - print(f"tool executable not found: {command[0]} (verify the [[tools]] " + print(f"tool executable not found: {run_argv[0]} (verify the [[tools]] " "registration and PATH)", file=sys.stderr) return 1 if result.returncode != 0: diff --git a/skills/mc-assets/scripts/tests/test-farm_asset.py b/skills/mc-assets/scripts/tests/test-farm_asset.py index d7d64a3..2286895 100644 --- a/skills/mc-assets/scripts/tests/test-farm_asset.py +++ b/skills/mc-assets/scripts/tests/test-farm_asset.py @@ -8,6 +8,7 @@ import importlib.util import json import shlex +import shutil import subprocess import sys import tempfile @@ -110,6 +111,62 @@ def test_flag_ref_without_template_placeholder_raises(self): ref="real.png") self.assertIn("mc-setup", str(ctx.exception)) + def test_template_quoting_is_posix_on_every_os(self): + # Single quotes group a token and double quotes nest inside them, + # exactly as a POSIX shell would, regardless of the host OS. + t = self.tool(headless="grok -p '' --caption 'say \"hi\"'") + argv = mod.plan_invocation(t, "image", "two words", "/out") + self.assertEqual(argv, ["grok", "-p", "two words", + "--caption", 'say "hi"']) + + +class TestResolveExecutable(unittest.TestCase): + def test_argv0_resolves_via_which(self): + resolved = mod.resolve_executable([sys.executable, "-c", "pass"]) + self.assertEqual(resolved, + [shutil.which(sys.executable), "-c", "pass"]) + + def test_unresolvable_argv0_returned_unchanged(self): + argv = ["definitely-not-a-real-tool-xyz", "-p", "x"] + self.assertEqual(mod.resolve_executable(argv), argv) + + def test_empty_command_returned_unchanged(self): + self.assertEqual(mod.resolve_executable([]), []) + + +class TestCmdShimSafety(unittest.TestCase): + # cmd.exe reparses .cmd/.bat argument lines with its own quoting and + # expands metacharacters; passing them through corrupts prompts + # (BatBadBut-class injection), so the guard must fail loudly instead. + + def test_non_shim_commands_pass_through_unchanged(self): + argv = ["/usr/local/bin/grok", "-p", 'say "hi" & del %TEMP%'] + self.assertEqual(mod.check_cmd_shim_safety(argv), argv) + exe = [r"C:\tools\grok.EXE", "-p", 'quote "this" exactly'] + self.assertEqual(mod.check_cmd_shim_safety(exe), exe) + + def test_shim_with_safe_arguments_passes_through(self): + argv = [r"C:\npm\grok.CMD", "-p", "a manticore at red sunset", + "--always-approve"] + self.assertEqual(mod.check_cmd_shim_safety(argv), argv) + + def test_shim_with_embedded_double_quote_is_refused(self): + # generative rule 5 says to quote exact on-image text; through a + # .cmd shim that quote would reach the tool corrupted. + with self.assertRaises(ValueError) as ctx: + mod.check_cmd_shim_safety( + [r"C:\npm\grok.cmd", "-p", 'thumbnail text says "GO"']) + self.assertIn("mc-setup", str(ctx.exception)) + + def test_shim_with_cmd_metacharacters_is_refused(self): + for arg in ("%TEMP%", "a ^ b", "x & del y", "a | b", "a < b", "a > b", + "line\nbreak"): + with self.assertRaises(ValueError): + mod.check_cmd_shim_safety([r"C:\npm\tool.bat", "-p", arg]) + + def test_empty_command_passes_through(self): + self.assertEqual(mod.check_cmd_shim_safety([]), []) + class TestManifest(unittest.TestCase): def test_rows_shape_and_null_cost(self): diff --git a/skills/mc-audio/SKILL.md b/skills/mc-audio/SKILL.md index 1972d85..b73ee2e 100644 --- a/skills/mc-audio/SKILL.md +++ b/skills/mc-audio/SKILL.md @@ -11,7 +11,7 @@ mc-assets farms pictures; this skill farms sound. It is a service skill: it owns 1. Load the studio config (`uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key modules.manticore`; empty means mc-setup has not run: stop and route the creator there) and this skill's own surface (`uv run {project-root}/_bmad/scripts/resolve_customization.py --skill {skill-root}`; run `{workflow.activation_steps_prepend}` now, `{workflow.activation_steps_append}` after this step, and hold `{workflow.persistent_facts}` as standing context). Resolve `paths` values against `{project-root}`. From `[audio]` take the lane values (`tts-provider`, `music-provider`, `sfx-provider`, `song-provider`) and `workspace`; the engine workspace is `{engines-path}/{audio.workspace}`. 2. Resolve the lane for the requested kind. The implemented 1.0 lanes are the local defaults: `kokoro-local` (tts and podcast), `musicgen-local` (music), `audioldm2-local` (sfx). A paid or planned value (`gemini-tts`, `elevenlabs-*`, `stable-audio-open`, `ace-step-local`) means the creator opted into a lane that has not landed: say so plainly and stop; never substitute a paid lane the creator did not choose, and never pretend an unvalidated lane works. An empty `song-provider` is the shipped state: full songs with vocals have no validated local lane yet (ACE-Step is the planned candidate; see the reference). -3. Workspace check: `uv run {skill-root}/scripts/ensure_workspace.py --workspace --check`. Not ready: tell the creator what a bootstrap downloads (venv wheels of several GB, ~340 MB of Kokoro models now, ~5 GB of Hugging Face cache on the first music/sfx run), get their go-ahead, then run it without `--check`. Idempotent: an existing validated workspace (a lab the creator built by hand counts) is used as-is, never rebuilt or duplicated. +3. Workspace check: `uv run {skill-root}/scripts/ensure_workspace.py --workspace --check`. Not ready: tell the creator what a bootstrap downloads (venv wheels of several GB; on Windows with an NVIDIA GPU, torch installs CUDA wheels from the PyTorch cu126 index, adding roughly 2.5 to 3 GB more; ~340 MB of Kokoro models now, ~5 GB of Hugging Face cache on the first music/sfx run), get their go-ahead, then run it without `--check`. The script's `--dry-run` JSON includes a `torch` field stating which wheel source this machine will use; relay it verbatim during consent. Idempotent: an existing validated workspace (a lab the creator built by hand counts) is used as-is, never rebuilt or duplicated. 4. Farm through the entry script, one call per asset: `uv run {skill-root}/scripts/farm_audio.py --kind tts|podcast|music|sfx --provider --workspace --out-dir [--name ]` plus the kind's arguments (`--text/--voice/--speed`, `--script lines.json`, `--prompt/--seconds/--seed`). For podcast dialogue, write the script JSON per the shape in the reference and apply the realism recipe knobs (speed variation, gaps, backchannels) rather than uniform lines. The script appends provenance to `/manifest.json` (same row shape as mc-assets; cost is null on local lanes). 5. Listen before presenting: play or inspect every output (duration matches the request, no silent or truncated file, dialogue lines land in order). Deliver with the honest caveats from the reference where they apply: SFX are 16 kHz (fine under a mix, thin exposed solo), music is instrumental only, TTS voices are stock (no cloning, no "your voice" claims), crosstalk is simulated. 6. First-run model downloads are long: run them in the background with proactive progress reports; never leave the creator staring at a silent stage. Report where every file landed and what was appended to the manifest. diff --git a/skills/mc-audio/scripts/engines/music_musicgen.py b/skills/mc-audio/scripts/engines/music_musicgen.py index fc96b26..8a92db9 100644 --- a/skills/mc-audio/scripts/engines/music_musicgen.py +++ b/skills/mc-audio/scripts/engines/music_musicgen.py @@ -8,7 +8,9 @@ Instrumentals only: beds, stingers, intro themes. No vocals or lyrics (a full-song lane is separate and unvalidated; see references/audio-lanes.md). Validated 2026-07-07 on Apple Silicon MPS: about 10 s of audio in 55 s once -the model is loaded. +the model is loaded. Device ladder: cuda if available, else mps, else cpu +(the CUDA branch is code-complete but awaits a validation run on real +NVIDIA hardware). Args: --prompt, --out , --seconds (default 10, capped at 30). Exit 0 ok, 1 generation failure, 2 usage error. @@ -40,7 +42,8 @@ def main() -> None: from transformers import AutoProcessor, MusicgenForConditionalGeneration - device = "mps" if torch.backends.mps.is_available() else "cpu" + device = ("cuda" if torch.cuda.is_available() + else "mps" if torch.backends.mps.is_available() else "cpu") t0 = time.time() processor = AutoProcessor.from_pretrained("facebook/musicgen-small") model = MusicgenForConditionalGeneration.from_pretrained( diff --git a/skills/mc-audio/scripts/engines/sfx_audioldm2.py b/skills/mc-audio/scripts/engines/sfx_audioldm2.py index 398f002..0b99c8e 100644 --- a/skills/mc-audio/scripts/engines/sfx_audioldm2.py +++ b/skills/mc-audio/scripts/engines/sfx_audioldm2.py @@ -11,7 +11,10 @@ Output is 16 kHz: fine for whooshes and ambience under a mix, thin when exposed solo; upsample/EQ or layer it. Validated 2026-07-07 on Apple -Silicon MPS: 7 to 14 s per 4 s effect once cached. +Silicon MPS: 7 to 14 s per 4 s effect once cached. Device ladder: cuda if +available, else mps, else cpu (the CUDA branch is code-complete but awaits +a validation run on real NVIDIA hardware). The generator stays on cpu for +seed reproducibility. Args: --prompt, --out , --seconds (default 4), --seed (default 7), --steps (default 100), --negative (default quality guard). @@ -45,7 +48,8 @@ def main() -> None: from diffusers import AudioLDM2Pipeline - device = "mps" if torch.backends.mps.is_available() else "cpu" + device = ("cuda" if torch.cuda.is_available() + else "mps" if torch.backends.mps.is_available() else "cpu") t0 = time.time() pipe = AudioLDM2Pipeline.from_pretrained( "cvssp/audioldm2", torch_dtype=torch.float32).to(device) diff --git a/skills/mc-audio/scripts/ensure_workspace.py b/skills/mc-audio/scripts/ensure_workspace.py index a8d3a78..b48edd7 100644 --- a/skills/mc-audio/scripts/ensure_workspace.py +++ b/skills/mc-audio/scripts/ensure_workspace.py @@ -25,6 +25,16 @@ transformers==4.43.4 (validated pair, 2026-07-07). Kokoro and MusicGen work at those versions too, which is why one venv serves all three engines. +The venv interpreter lives at .venv/bin/python on macOS and Linux and at +.venv\\Scripts\\python.exe on Windows; venv_python() resolves the right one. + +Torch wheel source per platform: macOS gets MPS wheels from plain PyPI and +Linux PyPI wheels already bundle CUDA, so both install straight from PyPI. +On Windows with an NVIDIA GPU (nvidia-smi on PATH) torch installs from the +PyTorch cu126 index instead; CUDA wheels add roughly 2.5 to 3 GB on top of +the model downloads, and the consent message the calling skill relays must +say so. Windows without NVIDIA gets plain PyPI CPU wheels. + The CALLING SKILL asks the creator before running this (the downloads are large); the script itself just does the work. @@ -41,6 +51,8 @@ import argparse import json +import os +import shutil import subprocess import sys import urllib.request @@ -55,6 +67,9 @@ "diffusers==0.31.0", "transformers==4.43.4", ] +TORCH_CUDA_INDEX = "https://download.pytorch.org/whl/cu126" +TORCH_CUDA_NOTE = ("torch from the PyTorch cu126 index " + "(Windows + NVIDIA; CUDA wheels add ~2.5-3 GB)") KOKORO_RELEASE = "https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0" KOKORO_FILES = ["kokoro-v1.0.onnx", "voices-v1.0.bin"] VERIFY_SNIPPET = ( @@ -71,9 +86,21 @@ def die(msg: str, code: int = 2) -> None: def venv_python(workspace: Path) -> Path: + """Per-OS venv interpreter path (Windows uses Scripts\\python.exe).""" + if os.name == "nt": + return workspace / ".venv" / "Scripts" / "python.exe" return workspace / ".venv" / "bin" / "python" +def wants_cuda_torch() -> bool: + """True only on Windows with an NVIDIA GPU (nvidia-smi on PATH). + + macOS gets MPS wheels and Linux gets CUDA-bundled wheels from plain + PyPI, so only Windows needs the explicit cu126 index. + """ + return os.name == "nt" and shutil.which("nvidia-smi") is not None + + def verify(workspace: Path) -> list[str]: """Return a list of problems; empty means the workspace is ready.""" problems = [] @@ -91,12 +118,26 @@ def verify(workspace: Path) -> list[str]: return problems -def planned_commands(workspace: Path, python_version: str) -> list[list[str]]: +def planned_commands(workspace: Path, python_version: str, + cuda_torch: bool | None = None) -> list[list[str]]: + """The venv-create command followed by the install command(s). + + With cuda_torch (default: wants_cuda_torch()), torch installs first from + the cu126 index and the remaining pins install from PyPI; the already + satisfied torch is not re-resolved against PyPI. + """ + if cuda_torch is None: + cuda_torch = wants_cuda_torch() py = venv_python(workspace) - return [ - ["uv", "venv", "--python", python_version, str(workspace / ".venv")], - ["uv", "pip", "install", "--python", str(py), *PIN_INSTALL], - ] + cmds = [["uv", "venv", "--python", python_version, str(workspace / ".venv")]] + if cuda_torch: + cmds.append(["uv", "pip", "install", "--python", str(py), + "--index-url", TORCH_CUDA_INDEX, "torch"]) + rest = [p for p in PIN_INSTALL if p != "torch"] + cmds.append(["uv", "pip", "install", "--python", str(py), *rest]) + else: + cmds.append(["uv", "pip", "install", "--python", str(py), *PIN_INSTALL]) + return cmds def download(url: str, dest: Path) -> None: @@ -126,10 +167,13 @@ def main() -> None: print(f"ready: {ws}") return + cuda_torch = wants_cuda_torch() + if args.dry_run: print(json.dumps({ "workspace": str(ws), - "commands": planned_commands(ws, args.python), + "commands": planned_commands(ws, args.python, cuda_torch), + "torch": TORCH_CUDA_NOTE if cuda_torch else "default PyPI wheels", "models": [] if args.skip_models else [f"{KOKORO_RELEASE}/{n}" for n in KOKORO_FILES], }, indent=2)) @@ -138,16 +182,20 @@ def main() -> None: for sub in ("models", "hf-cache", "out"): (ws / sub).mkdir(parents=True, exist_ok=True) + if cuda_torch: + print(f"note: {TORCH_CUDA_NOTE}", file=sys.stderr) + if not venv_python(ws).exists(): - for cmd in planned_commands(ws, args.python): + for cmd in planned_commands(ws, args.python, cuda_torch): r = subprocess.run(cmd) if r.returncode != 0: die(f"error: {' '.join(cmd)} failed", 1) else: # Existing venv: install is idempotent and fixes a broken dep set. - r = subprocess.run(planned_commands(ws, args.python)[1]) - if r.returncode != 0: - die("error: dependency install into existing venv failed", 1) + for cmd in planned_commands(ws, args.python, cuda_torch)[1:]: + r = subprocess.run(cmd) + if r.returncode != 0: + die("error: dependency install into existing venv failed", 1) if not args.skip_models: for name in KOKORO_FILES: diff --git a/skills/mc-audio/scripts/farm_audio.py b/skills/mc-audio/scripts/farm_audio.py index aad7a10..bf5ff16 100644 --- a/skills/mc-audio/scripts/farm_audio.py +++ b/skills/mc-audio/scripts/farm_audio.py @@ -7,8 +7,10 @@ The mc-audio entry point: resolves the requested kind to its engine payload, runs it inside the audio-lab workspace venv, and records provenance. Engine payloads live in engines/ next to this script and run as -/.venv/bin/python (torch-class deps live in that one -persistent venv; ensure_workspace.py builds it). + , where is the workspace venv's +interpreter (.venv/bin/python on macOS and Linux, .venv\\Scripts\\python.exe +on Windows; torch-class deps live in that one persistent venv; +ensure_workspace.py builds it). Kinds and their arguments: tts --text "..." [--voice af_heart] [--speed 1.0] @@ -71,9 +73,19 @@ def die(msg: str, code: int = 2) -> None: sys.exit(code) +def venv_python(workspace: Path) -> Path: + """Per-OS venv interpreter path (Windows uses Scripts\\python.exe). + + Duplicated in ensure_workspace.py; keep the two in sync. + """ + if os.name == "nt": + return workspace / ".venv" / "Scripts" / "python.exe" + return workspace / ".venv" / "bin" / "python" + + def build_command(args: argparse.Namespace, out_file: Path) -> list[str]: payload = Path(__file__).parent / "engines" / ENGINES[args.kind][1] - py = args.workspace / ".venv" / "bin" / "python" + py = venv_python(args.workspace) if args.kind in ("tts", "podcast"): cmd = [str(py), str(payload), "--models-dir", str(args.workspace / "models"), "--out", str(out_file)] @@ -142,7 +154,7 @@ def main() -> None: "HF_HOME": env["HF_HOME"], "model": model}, indent=2)) return - py = args.workspace / ".venv" / "bin" / "python" + py = venv_python(args.workspace) if not py.exists(): die(f"error: workspace not ready ({py} missing); run " "ensure_workspace.py first", 4) diff --git a/skills/mc-audio/scripts/tests/test-ensure_workspace.py b/skills/mc-audio/scripts/tests/test-ensure_workspace.py index 4d55354..cf4d6a9 100644 --- a/skills/mc-audio/scripts/tests/test-ensure_workspace.py +++ b/skills/mc-audio/scripts/tests/test-ensure_workspace.py @@ -3,15 +3,22 @@ # requires-python = ">=3.11" # /// """Tests for ensure_workspace.py: planned commands carry the critical -dependency pin, --check readiness reporting, and verify() against a fake -workspace. No venvs are built, no models downloaded.""" +dependency pin, per-OS venv interpreter paths, the Windows CUDA torch +index plan, --check readiness reporting, and verify() against a fake +workspace. The fake workspace is a real throwaway venv (works on POSIX +and Windows alike) whose site-packages carries stub modules so the +import check passes. No heavy deps are installed, no models downloaded, +no network.""" import importlib.util import json -import stat +import os import subprocess import sys +import sysconfig import tempfile import unittest +import unittest.mock +import venv from pathlib import Path SCRIPT = Path(__file__).resolve().parent.parent / "ensure_workspace.py" @@ -20,16 +27,42 @@ mod = importlib.util.module_from_spec(spec) spec.loader.exec_module(mod) +# Stub modules planted in the fake venv so VERIFY_SNIPPET's imports and +# version asserts pass without installing anything. +VERIFY_STUBS = { + "kokoro_onnx.py": "", + "soundfile.py": "", + "torch.py": "", + "scipy.py": "", + "diffusers.py": "__version__ = '0.31.0'\n", + "transformers.py": "__version__ = '4.43.4'\n", +} + + +def make_venv_shim(ws: Path, stubs: dict[str, str]) -> Path: + """Create a real throwaway venv at ws/.venv and plant stub modules in + its site-packages. Returns the per-OS venv interpreter path. + + Duplicated in test-farm_audio.py; keep the two in sync.""" + venv.create(ws / ".venv", with_pip=False, symlinks=(os.name != "nt")) + site = Path(sysconfig.get_path( + "purelib", scheme="venv", + vars={"base": str(ws / ".venv"), "platbase": str(ws / ".venv")})) + site.mkdir(parents=True, exist_ok=True) + for rel, body in stubs.items(): + dest = site / Path(rel) + dest.parent.mkdir(parents=True, exist_ok=True) + dest.write_text(body, encoding="utf-8") + return mod.venv_python(ws) + def ready_workspace(root: Path) -> Path: - """Fake workspace: shim venv python that passes the import check, plus - empty model files.""" + """Fake workspace: venv shim that passes the import check, plus empty + model files.""" ws = root / "audio-lab" - (ws / ".venv" / "bin").mkdir(parents=True) + ws.mkdir(parents=True) (ws / "models").mkdir() - shim = ws / ".venv" / "bin" / "python" - shim.write_text("#!/bin/sh\necho ok\n") - shim.chmod(shim.stat().st_mode | stat.S_IEXEC) + make_venv_shim(ws, VERIFY_STUBS) for name in mod.KOKORO_FILES: (ws / "models" / name).touch() return ws @@ -46,6 +79,45 @@ def test_verify_snippet_asserts_the_pair(self): self.assertIn("4.43.4", mod.VERIFY_SNIPPET) +class TestVenvPython(unittest.TestCase): + def test_posix_layout(self): + ws = Path("/ws") + with unittest.mock.patch.object(os, "name", "posix"): + py = mod.venv_python(ws) + self.assertEqual(py, ws / ".venv" / "bin" / "python") + + def test_windows_layout(self): + ws = Path("/ws") + with unittest.mock.patch.object(os, "name", "nt"): + py = mod.venv_python(ws) + self.assertEqual(py, ws / ".venv" / "Scripts" / "python.exe") + + +class TestPlannedCommands(unittest.TestCase): + def test_default_lane_installs_all_pins_from_pypi(self): + cmds = mod.planned_commands(Path("/ws"), "3.12", cuda_torch=False) + self.assertEqual(len(cmds), 2) + self.assertEqual(cmds[0][:2], ["uv", "venv"]) + self.assertIn("torch", cmds[1]) + self.assertNotIn("--index-url", cmds[1]) + + def test_cuda_lane_installs_torch_from_cu126_index_first(self): + cmds = mod.planned_commands(Path("/ws"), "3.12", cuda_torch=True) + self.assertEqual(len(cmds), 3) + self.assertIn("--index-url", cmds[1]) + self.assertIn(mod.TORCH_CUDA_INDEX, cmds[1]) + self.assertIn("torch", cmds[1]) + # The PyPI install still carries the critical pin, minus torch. + self.assertIn("diffusers==0.31.0", cmds[2]) + self.assertIn("transformers==4.43.4", cmds[2]) + self.assertNotIn("torch", cmds[2]) + self.assertNotIn("--index-url", cmds[2]) + + def test_cuda_detection_is_windows_only(self): + with unittest.mock.patch.object(os, "name", "posix"): + self.assertFalse(mod.wants_cuda_torch()) + + class TestCli(unittest.TestCase): def run_script(self, *argv): return subprocess.run([sys.executable, str(SCRIPT), *argv], @@ -61,6 +133,7 @@ def test_dry_run_plans_venv_install_and_models(self): self.assertTrue(any("diffusers==0.31.0" in c and "transformers==4.43.4" in c for c in flat)) self.assertEqual(len(plan["models"]), 2) + self.assertIn("torch", plan) def test_dry_run_skip_models_plans_no_downloads(self): with tempfile.TemporaryDirectory() as td: diff --git a/skills/mc-audio/scripts/tests/test-farm_audio.py b/skills/mc-audio/scripts/tests/test-farm_audio.py index 471936a..230d88f 100644 --- a/skills/mc-audio/scripts/tests/test-farm_audio.py +++ b/skills/mc-audio/scripts/tests/test-farm_audio.py @@ -2,17 +2,22 @@ # /// script # requires-python = ">=3.11" # /// -"""Tests for farm_audio.py: command construction per kind, provider gating, -workspace readiness, manifest rows, and the local lane end to end using a -fake venv python (a shell shim). No models, no network, no downloads.""" +"""Tests for farm_audio.py: command construction per kind, per-OS venv +interpreter paths, provider gating, workspace readiness, manifest rows, +and the local lane end to end. The fake workspace is a real throwaway +venv (works on POSIX and Windows alike) whose site-packages carries stub +numpy/scipy/torch/diffusers modules, so the real sfx engine payload runs +without heavy deps. No models, no network, no downloads.""" import importlib.util import json import os -import stat import subprocess import sys +import sysconfig import tempfile import unittest +import unittest.mock +import venv from pathlib import Path SCRIPT = Path(__file__).resolve().parent.parent / "farm_audio.py" @@ -21,6 +26,70 @@ mod = importlib.util.module_from_spec(spec) spec.loader.exec_module(mod) +# Stub modules planted in the fake venv so the real sfx_audioldm2.py +# payload runs end to end on the cpu rung of the device ladder and +# writes the --out file, with no heavy deps. +ENGINE_STUBS = { + "numpy.py": "def abs(x):\n return x\n", + "scipy/__init__.py": "", + "scipy/io/__init__.py": "", + "scipy/io/wavfile.py": ( + "def write(path, sr, data):\n" + " with open(path, 'wb') as f:\n" + " f.write(b'RIFF')\n"), + "torch.py": ( + "float32 = 'float32'\n" + "class _Flag:\n" + " @staticmethod\n" + " def is_available():\n" + " return False\n" + "class _Backends:\n" + " mps = _Flag()\n" + "cuda = _Flag()\n" + "backends = _Backends()\n" + "class Generator:\n" + " def __init__(self, device=None):\n" + " pass\n" + " def manual_seed(self, seed):\n" + " return self\n"), + "diffusers.py": ( + "class _Audio:\n" + " def max(self):\n" + " return 1.0\n" + " def __truediv__(self, other):\n" + " return self\n" + " def __mul__(self, other):\n" + " return self\n" + "class _Out:\n" + " def __init__(self):\n" + " self.audios = [_Audio()]\n" + "class AudioLDM2Pipeline:\n" + " @classmethod\n" + " def from_pretrained(cls, *a, **k):\n" + " return cls()\n" + " def to(self, device):\n" + " return self\n" + " def __call__(self, *a, **k):\n" + " return _Out()\n"), +} + + +def make_venv_shim(ws: Path, stubs: dict[str, str]) -> Path: + """Create a real throwaway venv at ws/.venv and plant stub modules in + its site-packages. Returns the per-OS venv interpreter path. + + Duplicated in test-ensure_workspace.py; keep the two in sync.""" + venv.create(ws / ".venv", with_pip=False, symlinks=(os.name != "nt")) + site = Path(sysconfig.get_path( + "purelib", scheme="venv", + vars={"base": str(ws / ".venv"), "platbase": str(ws / ".venv")})) + site.mkdir(parents=True, exist_ok=True) + for rel, body in stubs.items(): + dest = site / Path(rel) + dest.parent.mkdir(parents=True, exist_ok=True) + dest.write_text(body, encoding="utf-8") + return mod.venv_python(ws) + def make_args(**kw): import argparse @@ -33,22 +102,28 @@ def make_args(**kw): def fake_workspace(root: Path) -> Path: - """A workspace whose venv python is a shim that writes the --out file.""" + """A workspace whose venv runs the real sfx payload against stubs.""" ws = root / "audio-lab" - (ws / ".venv" / "bin").mkdir(parents=True) + ws.mkdir(parents=True) (ws / "models").mkdir() - shim = ws / ".venv" / "bin" / "python" - shim.write_text( - "#!/bin/sh\n" - 'prev=""\n' - 'for a in "$@"; do\n' - ' if [ "$prev" = "--out" ]; then printf RIFF > "$a"; fi\n' - ' prev="$a"\n' - "done\n") - shim.chmod(shim.stat().st_mode | stat.S_IEXEC) + make_venv_shim(ws, ENGINE_STUBS) return ws +class TestVenvPython(unittest.TestCase): + def test_posix_layout(self): + ws = Path("/ws") + with unittest.mock.patch.object(os, "name", "posix"): + py = mod.venv_python(ws) + self.assertEqual(py, ws / ".venv" / "bin" / "python") + + def test_windows_layout(self): + ws = Path("/ws") + with unittest.mock.patch.object(os, "name", "nt"): + py = mod.venv_python(ws) + self.assertEqual(py, ws / ".venv" / "Scripts" / "python.exe") + + class TestBuildCommand(unittest.TestCase): def test_tts_single(self): cmd = mod.build_command( @@ -60,6 +135,11 @@ def test_tts_single(self): self.assertIn("1.1", cmd) self.assertTrue(cmd[1].endswith("tts_kokoro.py")) + def test_command_runs_the_venv_interpreter(self): + cmd = mod.build_command( + make_args(prompt="whoosh"), Path("/out/s.wav")) + self.assertEqual(cmd[0], str(mod.venv_python(Path("/ws")))) + def test_podcast_uses_script_mode(self): cmd = mod.build_command( make_args(kind="podcast", provider="kokoro-local", diff --git a/skills/mc-cut/SKILL.md b/skills/mc-cut/SKILL.md index f91d5b0..a83e49b 100644 --- a/skills/mc-cut/SKILL.md +++ b/skills/mc-cut/SKILL.md @@ -14,7 +14,7 @@ The Descript replacement, render-first. Every approved cut iteration ends in a w - Frame rate: VFR sources are re-encoded to constant frame rate (run the preflight in the background and keep working; transcription waits for it). Record the reported `cfr_master` path in project.json `sources` as the project source of truth; every later step (transcription, EDL times, renders, timeline export) uses the CFR master, never the VFR original. - Disk: free space is checked against a rough estimate (3x source size plus the estimated CFR masters) before any remux write, and the script refuses the remux itself when the estimate does not fit; if `disk.ok` is false in the summary, stop and tell the creator before any render. - Source QC: inspect the extracted first and last frames per source for edge defects (black edges, wrong aspect, letterboxed or cropped content) before any render is built on them. -3. Transcribe each take: `uv run {skill-root}/scripts/transcribe.py raw/ -o transcript/words.json --provider <[transcription] provider from the config>` (suffix the output `.words.json` when the project has multiple sources). Default provider parakeet-mlx runs local and free; the model downloads once on first run. +3. Transcribe each take: `uv run {skill-root}/scripts/transcribe.py raw/ -o transcript/words.json --provider <[transcription] provider from the config>` (suffix the output `.words.json` when the project has multiple sources). Provider values: `auto` (the default) picks parakeet-mlx on macOS Apple Silicon and onnx-asr everywhere else; `parakeet-mlx` and `onnx-asr` force a lane. Both lanes run the same parakeet-tdt-0.6b-v3 weights, so verbatim fillers and 80 ms word timestamps carry over on Windows, Linux, and Intel Macs; on the onnx-asr lane, when the runtime exposes no per-token scores every word's confidence reads 1.0 (no signal, never fabricated). On a CUDA machine escalate with `uv run --with "onnx-asr[gpu,hub]" python {skill-root}/scripts/transcribe.py ...` (PEP 508 markers cannot detect GPUs; the `python` command is required because it skips the script's cpu-extra dependency, so onnxruntime-gpu never co-installs with onnxruntime, and the script warns on stderr when an NVIDIA GPU is visible but CUDA is unavailable). All lanes are local and free; the model downloads once on first run. 4. Candidates: `uv run {skill-root}/scripts/cutplan.py transcript/words.json -o cut/candidates.json` plus any `{workflow.cutplan_flags}` finds silences, filler runs, stutters, and retakes mechanically. On an `interview` source (project.json `sources`), it also flags each spoken interviewer-question read as a `marker` candidate: cut the marker and question, keep the answer. The default marker cue is "question from the interviewer"; projects recorded against the older "question from claude" convention pass `--marker-cues "question from claude"` (via `{workflow.cutplan_flags}`). 5. Make the taste calls: against `script.md` and the Production Bible, pick best takes, order segments, decide keep-or-cut on every candidate in `cut/candidates.json`. Write `cut/cutplan.md` as a short human-readable plan whose spine is the judgment calls, each with a timestamp and the quoted words (the "trailing 'so' at 42:20, keep or cut?" shape). Group the obvious silence trims into one line; itemize only what the creator might disagree with. 6. Write `cut/edl.json`: `{source, source_duration, fade_ms: 30, pad_ms: 60, segments: [...]}` with ordered segments of {source, start, end, beat, quote, reason} obeying the cutting rules below. @@ -23,7 +23,7 @@ The Descript replacement, render-first. Every approved cut iteration ends in a w - Render the preview, always: `uv run {skill-root}/scripts/render_preview.py cut/edl.json -o renders/preview.mp4 --boundary-frames cut/boundaries/` plus any `{workflow.preview_flags}` (720p CRF 28 defaults; pass the `[render]` preview keys from the studio config when set). Inspect the boundary frames per the cutting rules. - Once the graphics stage has rendered overlays into `graphics/`, re-render composited so the creator iterates on overlays and CTAs visually: same command plus `--beats beats/beats.md --graphics-dir graphics/`. Report any `overlays_missing` from the summary. - Export the editor timeline, always, per `[editor] timeline-format` in the config: `fcpxml` via `uv run {skill-root}/scripts/edl_to_fcpxml.py cut/edl.json -o cut/rough.fcpxml` (Resolve and Final Cut import it natively; refuses VFR sources loudly); `xmeml`/`edl` are planned lanes, so Premiere users work from cutplan.md + edl.json + the rendered preview/final until the xmeml lane lands (see TODO); `none` (Descript and manual workflows) skips export, and the deliverables are cutplan.md + edl.json + renders/preview.mp4 as the cut map. - - resolve_import.py (push the timeline into a running Resolve) is currently a stub: do NOT offer it. When its STATUS line says implemented, offer it only if `[mcp] davinci-resolve` is true in the config. + - resolve_import.py (push the timeline into a running Resolve) is currently a stub: do NOT offer it. When its STATUS line says implemented, offer it only if `[mcp] davinci-resolve` is true in the config. Free-edition note for when it lands: Resolve's external scripting API is Studio-only, but the free edition runs scripts launched from inside the app (Workspace > Scripts), so copying the script into Resolve's Fusion Scripts folder unlocks scripted import there; the per-OS folder paths are documented in the setup stack reference for the creator's platform. - Record the ISO date in `approvals.cutplan`, append `cut` to `stages_done`, and set `stage` to the next entry in project.json's `stages` array. ## Composited preview (after graphics) @@ -32,7 +32,7 @@ mc-pipeline routes here as soon as the graphics stage completes (mc-graphics han ## Final render (gate 4) -When the project reaches the final stage, offer the final-quality render from this skill: `uv run {skill-root}/scripts/render_final.py cut/edl.json -o renders/final.mp4 --beats beats/beats.md --graphics-dir graphics/` plus any `{workflow.final_flags}`, with `--codec`, `--crf`, and `--height` per `[render]` in the studio config. It bakes the same EDL the creator approved with graphics composited from the approved beat table, hardware encode when available (videotoolbox on macOS, libx264 fallback), segment-parallel rendering, a disk preflight, progress reporting, and boundary-frame checks. Finishing in the creator's own editor from the always-exported timeline is an equally supported path; either closes gate 4. +When the project reaches the final stage, offer the final-quality render from this skill: `uv run {skill-root}/scripts/render_final.py cut/edl.json -o renders/final.mp4 --beats beats/beats.md --graphics-dir graphics/` plus any `{workflow.final_flags}`, with `--codec`, `--crf`, and `--height` per `[render]` in the studio config, `--loudness-target <[render] loudness-target>`, and `--no-loudnorm` appended when `[render] loudnorm` is false. It bakes the same EDL the creator approved with graphics composited from the approved beat table, hardware encode when available (videotoolbox on macOS; on Windows h264_nvenc, then h264_qsv, then h264_amf; on Linux h264_nvenc then h264_vaapi; each candidate validated by a one-frame test encode, libx264 fallback everywhere), segment-parallel rendering, a disk preflight, progress reporting, and boundary-frame checks. The final render is loudness-normalized to the target LUFS with two-pass ffmpeg loudnorm (the fast preview is never normalized). Finishing in the creator's own editor from the always-exported timeline is an equally supported path; either closes gate 4. ## Dual timecode diff --git a/skills/mc-cut/scripts/composite_core.py b/skills/mc-cut/scripts/composite_core.py index 3ad387f..1b5cad1 100644 --- a/skills/mc-cut/scripts/composite_core.py +++ b/skills/mc-cut/scripts/composite_core.py @@ -18,7 +18,8 @@ - ffmpeg filter_complex and command construction, with optional overlay compositing (ProRes 4444 / WebM / mp4 / PNG over the concat output) - chunk planning for segment-parallel final renders - - encoder selection (videotoolbox hardware on macOS, libx264 fallback) + - encoder selection (videotoolbox on macOS; probed nvenc/qsv/amf ladder + on Windows and nvenc/vaapi ladder on Linux; libx264 fallback) - disk-space estimation and preflight - ffmpeg -progress output parsing - ffprobe wrappers and boundary-frame extraction @@ -204,7 +205,8 @@ def resolve_overlays(beats, graphics_dir): # --- ffmpeg filtergraph and command ----------------------------------------- -def build_filter_complex(edl, source_index, height, overlays=(), overlay_size=None): +def build_filter_complex(edl, source_index, height, overlays=(), overlay_size=None, + hwupload=False): """Build the filter_complex string for the whole timeline. source_index maps each source path to its ffmpeg -i input index. Each @@ -218,6 +220,10 @@ def build_filter_complex(edl, source_index, height, overlays=(), overlay_size=No given, PTS shifted to its timeline start, overlay with eof_action=pass and an enable window). With overlays the chain ends in format=yuv420p so the output stays player-safe. Final labels are always [outv]/[outa]. + + hwupload=True ends the video chain in format=nv12,hwupload instead, for + encoders that only take hardware frames (vaapi); the caller must also set + up the device (encoder_init_flags). """ fade = edl.get("fade_ms", 30) / 1000.0 parts, vlabels, alabels = [], [], [] @@ -252,7 +258,11 @@ def build_filter_complex(edl, source_index, height, overlays=(), overlay_size=No n = len(edl["segments"]) concat_inputs = "".join(v + a for v, a in zip(vlabels, alabels)) if not overlays: - parts.append(f"{concat_inputs}concat=n={n}:v=1:a=1[outv][outa]") + if hwupload: + parts.append(f"{concat_inputs}concat=n={n}:v=1:a=1[basev][outa]") + parts.append("[basev]format=nv12,hwupload[outv]") + else: + parts.append(f"{concat_inputs}concat=n={n}:v=1:a=1[outv][outa]") return ";".join(parts) parts.append(f"{concat_inputs}concat=n={n}:v=1:a=1[basev][outa]") prev = "basev" @@ -270,7 +280,10 @@ def build_filter_complex(edl, source_index, height, overlays=(), overlay_size=No f"enable='between(t,{_fmt(ov['start'])},{_fmt(end_t)})'[{out_lab}]" ) prev = out_lab - parts.append(f"[{prev}]format=yuv420p[outv]") + if hwupload: + parts.append(f"[{prev}]format=nv12,hwupload[outv]") + else: + parts.append(f"[{prev}]format=yuv420p[outv]") return ";".join(parts) @@ -279,7 +292,8 @@ def build_filter_complex(edl, source_index, height, overlays=(), overlay_size=No def build_command(edl, project_dir, output, height, overlays=(), - overlay_size=None, encode=None, extra_output_flags=()): + overlay_size=None, encode=None, extra_output_flags=(), + encoder=None): """Assemble (ffmpeg_argv, source_index) for one render invocation. encode replaces the default preview encode args (libx264 crf 28 veryfast @@ -287,13 +301,18 @@ def build_command(edl, project_dir, output, height, overlays=(), cap (looped/synthetic sources must never run open-ended); video overlay inputs are -t capped to the beat's dur too, so decode stops at the enable window. -movflags +faststart is added for .mp4/.mov outputs. + + encoder (optional) is the encoder name the encode args target; it only + matters for encoders that need device setup and hardware frames (vaapi + gets -init_hw_device flags and an hwupload filtergraph tail). Software + and videotoolbox/nvenc/qsv/amf encoders need nothing here. """ distinct = [] for seg in edl["segments"]: if seg["source"] not in distinct: distinct.append(seg["source"]) source_index = {src: i for i, src in enumerate(distinct)} - argv = ["ffmpeg", "-y"] + argv = ["ffmpeg", "-y", *encoder_init_flags(encoder)] for src in distinct: argv += ["-i", str((project_dir / src).resolve())] ovs = [] @@ -308,7 +327,8 @@ def build_command(edl, project_dir, output, height, overlays=(), ovs.append(entry) argv += [ "-filter_complex", - build_filter_complex(edl, source_index, height, ovs, overlay_size), + build_filter_complex(edl, source_index, height, ovs, overlay_size, + hwupload=encoder_needs_hwupload(encoder)), "-map", "[outv]", "-map", "[outa]", ] argv += list(encode) if encode else list(PREVIEW_ENCODE) @@ -397,17 +417,94 @@ def list_encoders(): return names -def pick_encoder(requested="auto", available=None, system=None): - """Resolve the encoder: hardware videotoolbox on macOS when available, - libx264 otherwise; an explicit request falls back to libx264 when the - local ffmpeg does not list it.""" +# Hardware-encode ladders, probed in order on auto selection. Darwin is not +# in the table: videotoolbox is picked on listing alone (the long-validated +# reference behavior), no test encode. +HW_LADDERS = { + "Windows": ("h264_nvenc", "h264_qsv", "h264_amf"), + "Linux": ("h264_nvenc", "h264_vaapi"), +} + +# Encoders whose rate control is a bitrate from the ladder (no dependable +# CRF mode across drivers). +HW_SUFFIXES = ("_videotoolbox", "_nvenc", "_qsv", "_amf", "_vaapi") + + +def is_hardware_encoder(encoder): + """True for encoders that take the bitrate ladder instead of -crf.""" + return bool(encoder) and encoder.endswith(HW_SUFFIXES) + + +def encoder_needs_hwupload(encoder): + """True for encoders that only accept hardware frames, so the video + chain must end in format=nv12,hwupload (vaapi).""" + return bool(encoder) and encoder.endswith("_vaapi") + + +def encoder_init_flags(encoder): + """Global ffmpeg flags an encoder needs before any input (vaapi device + init and the filter device binding); empty for everything else.""" + if encoder_needs_hwupload(encoder): + return ["-init_hw_device", "vaapi=va", "-filter_hw_device", "va"] + return [] + + +def encoder_probe_command(encoder): + """ffmpeg argv for a one-frame test encode: lavfi color source to the + null muxer. Listing an encoder proves the build has it; only a real + encode proves the driver/hardware behind it works.""" + argv = ["ffmpeg", "-hide_banner", "-v", "error", + *encoder_init_flags(encoder), + "-f", "lavfi", "-i", "color=c=black:size=320x180:rate=30"] + if encoder_needs_hwupload(encoder): + argv += ["-vf", "format=nv12,hwupload"] + argv += ["-frames:v", "1", "-c:v", encoder, "-f", "null", "-"] + return argv + + +_probe_cache = {} + + +def probe_encoder(encoder, cache=None): + """One-frame test encode of `encoder`, cached per process so each + encoder is probed at most once per run. cache=None uses the module + cache; tests pass their own dict.""" + cache = _probe_cache if cache is None else cache + if encoder in cache: + return cache[encoder] + try: + proc = subprocess.run(encoder_probe_command(encoder), + capture_output=True, text=True, timeout=30) + ok = proc.returncode == 0 + except (OSError, subprocess.SubprocessError): + ok = False + cache[encoder] = ok + return ok + + +def pick_encoder(requested="auto", available=None, system=None, probe=None): + """Resolve the encoder for this run. + + Explicit request: returned when the local ffmpeg lists it, libx264 + otherwise (unchanged). Auto on Darwin: h264_videotoolbox when listed, + libx264 otherwise (unchanged, never probed). Auto elsewhere: the first + HW_LADDERS entry for the OS that is both listed by ffmpeg AND passes a + one-frame test encode (probe_encoder, cached per run); libx264 when the + whole ladder fails. probe is injectable for tests.""" system = system or platform.system() if available is None: available = list_encoders() if requested and requested != "auto": return requested if requested in available else "libx264" - if system == "Darwin" and "h264_videotoolbox" in available: - return "h264_videotoolbox" + if system == "Darwin": + if "h264_videotoolbox" in available: + return "h264_videotoolbox" + return "libx264" + if probe is None: + probe = probe_encoder + for enc in HW_LADDERS.get(system, ()): + if enc in available and probe(enc): + return enc return "libx264" @@ -425,15 +522,21 @@ def bitrate_for(height): def encode_args(encoder, crf=18, height=1080): - """Encode argv fragment for the final render. videotoolbox encoders take a - bitrate from the ladder (they have no CRF mode); libx264 takes -crf.""" - if encoder.endswith("_videotoolbox"): - v = ["-c:v", encoder, "-b:v", f"{bitrate_for(height)}k", "-allow_sw", "1"] + """Encode argv fragment for the final render. Hardware encoders take a + bitrate from the ladder (no dependable CRF mode across drivers); libx264 + takes -crf. -pix_fmt is not forced for nvenc/qsv/amf (each negotiates + its own supported format from the yuv420p filtergraph output) nor for + vaapi (it receives hardware frames via the hwupload chain).""" + if is_hardware_encoder(encoder): + v = ["-c:v", encoder, "-b:v", f"{bitrate_for(height)}k"] + if encoder.endswith("_videotoolbox"): + v += ["-allow_sw", "1", "-pix_fmt", "yuv420p"] if encoder == "hevc_videotoolbox": v += ["-tag:v", "hvc1"] else: - v = ["-c:v", encoder, "-crf", str(crf), "-preset", "medium"] - return v + ["-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "192k"] + v = ["-c:v", encoder, "-crf", str(crf), "-preset", "medium", + "-pix_fmt", "yuv420p"] + return v + ["-c:a", "aac", "-b:a", "192k"] def estimate_output_bytes(duration_s, height, encoder="libx264"): diff --git a/skills/mc-cut/scripts/edl_to_fcpxml.py b/skills/mc-cut/scripts/edl_to_fcpxml.py index 8d9dfdd..ee45316 100644 --- a/skills/mc-cut/scripts/edl_to_fcpxml.py +++ b/skills/mc-cut/scripts/edl_to_fcpxml.py @@ -75,7 +75,6 @@ import xml.etree.ElementTree as ET from fractions import Fraction from pathlib import Path -from urllib.parse import quote # --- rational / frame-grid helpers (pure, unit-tested) -------------------- @@ -190,6 +189,17 @@ def _fps_label(num, den): return str(num) if den == 1 else f"{round(num / den, 2)}" +def media_rep_uri(abs_path): + """RFC 8089 file URI for an absolute media path, via Path.as_uri(). + + as_uri() is byte-identical to the previous 'file://' + quote(path) + construction for POSIX paths (same percent-encoding, same safe set), and + unlike it emits valid drive-letter URIs on Windows (file:///C:/...) and + UNC share URIs (file://server/share/...). Takes any absolute PurePath so + the Windows shape is unit-testable from any OS.""" + return abs_path.as_uri() + + def _audio_rate_label(rate): if rate is None: return "48k" @@ -302,7 +312,7 @@ def build_document(edl, sources, project_dir): asset = ET.SubElement(resources, "asset", attrs) ET.SubElement(asset, "media-rep", { "kind": "original-media", - "src": "file://" + quote(str(abs_path)), + "src": media_rep_uri(abs_path), }) # --- library / event / project / sequence / spine --- @@ -390,7 +400,7 @@ def main(argv=None): ) # parse_float=Fraction keeps EDL times exact from the source decimal tokens. - edl = json.loads(edl_path.read_text(), parse_float=Fraction) + edl = json.loads(edl_path.read_text(encoding="utf-8"), parse_float=Fraction) distinct = [] for seg in edl["segments"]: @@ -410,7 +420,10 @@ def main(argv=None): root, meta = build_document(edl, sources, project_dir) text = serialize(root) out_path = Path(args.output) - out_path.write_text(text) + # utf-8 explicitly: the XML declaration says UTF-8, so the bytes must be + # UTF-8 regardless of the locale codec (Windows cp1252 would otherwise + # corrupt or reject non-ASCII clip names). + out_path.write_text(text, encoding="utf-8") # Round-trip validation: re-parse the file we just wrote (the parser skips # the XML declaration and the entity-free DOCTYPE) and check the spine. diff --git a/skills/mc-cut/scripts/preflight.py b/skills/mc-cut/scripts/preflight.py index 7d1f3cb..52bd1ce 100644 --- a/skills/mc-cut/scripts/preflight.py +++ b/skills/mc-cut/scripts/preflight.py @@ -106,16 +106,25 @@ def estimate_master_bytes(duration, height, source_bytes): def remux_command(src, dst, rate, encoder="libx264", crf=18, height=None): """ffmpeg argv re-encoding src to CFR at `rate` (audio copied). - videotoolbox encoders have no CRF mode, so they take the master bitrate - for the source height; libx264 takes -crf.""" - argv = ["ffmpeg", "-y", "-i", str(src), "-vf", f"fps={rate}"] - if encoder.endswith("_videotoolbox"): - argv += ["-c:v", encoder, "-b:v", - f"{master_bitrate_for(height)}k", "-allow_sw", "1"] + Hardware encoders have no dependable CRF mode, so they take the master + bitrate for the source height (videotoolbox additionally -allow_sw; + vaapi additionally device init and an hwupload chain, and no forced + -pix_fmt since the encoder receives hardware frames); libx264 takes + -crf.""" + argv = ["ffmpeg", "-y", *core.encoder_init_flags(encoder), "-i", str(src)] + vf = f"fps={rate}" + if core.encoder_needs_hwupload(encoder): + vf += ",format=nv12,hwupload" + argv += ["-vf", vf] + if core.is_hardware_encoder(encoder): + argv += ["-c:v", encoder, "-b:v", f"{master_bitrate_for(height)}k"] + if encoder.endswith("_videotoolbox"): + argv += ["-allow_sw", "1"] else: argv += ["-c:v", encoder, "-crf", str(crf), "-preset", "fast"] - argv += ["-pix_fmt", "yuv420p", "-c:a", "copy", - "-movflags", "+faststart", str(dst)] + if not core.encoder_needs_hwupload(encoder): + argv += ["-pix_fmt", "yuv420p"] + argv += ["-c:a", "copy", "-movflags", "+faststart", str(dst)] return argv @@ -237,10 +246,12 @@ def main(argv=None): print(json.dumps(build_summary(files, disk), indent=2)) return 1 + # One encoder pick for every remux in this run (pick_encoder's hardware + # probes are cached per process, but there is no reason to ask twice). + encoder = core.pick_encoder("auto") if remux_jobs else None for entry, path, info in remux_jobs: rate = nearest_standard_rate(info["avg_frame_rate"]) dst = path.with_name(path.stem + args.remux_suffix + ".mp4") - encoder = core.pick_encoder("auto") cmd = remux_command(path, dst, rate, encoder, height=info["height"]) print(f"preflight: remuxing VFR {path.name} to CFR {rate} " f"({encoder})...", file=sys.stderr) diff --git a/skills/mc-cut/scripts/render_final.py b/skills/mc-cut/scripts/render_final.py index e2b2971..b4020f3 100644 --- a/skills/mc-cut/scripts/render_final.py +++ b/skills/mc-cut/scripts/render_final.py @@ -9,6 +9,7 @@ uv run {skill-root}/scripts/render_final.py -o renders/final.mp4 \ [--project-dir ] [--beats beats/beats.md --graphics-dir graphics/] \ [--codec auto] [--crf 18] [--height ] [--parallel 2] \ + [--loudness-target -14] [--no-loudnorm] \ [--boundary-frames ] [--skip-disk-check] [--keep-temp] Purpose: @@ -33,13 +34,29 @@ losslessly concatenated (concat demuxer, -c copy, aac_adtstoasc, +faststart). Intermediates are removed unless --keep-temp. - encode --codec auto picks h264_videotoolbox on macOS when this ffmpeg - lists it (hardware; bitrate ladder by output height), libx264 - -crf --crf (default 18, preset medium) otherwise; an explicit - --codec not offered by ffmpeg falls back to libx264. - hevc_videotoolbox gets -tag:v hvc1. Audio aac 192k, video - yuv420p. --height scales (aspect kept, even width); default - keeps the source resolution. + encode --codec auto picks the platform's hardware ladder: on macOS + h264_videotoolbox when this ffmpeg lists it (bitrate ladder by + output height); on Windows the first of h264_nvenc, h264_qsv, + h264_amf that is listed AND passes a one-frame test encode + (lavfi color source to the null muxer, probed once per run); on + Linux h264_nvenc then h264_vaapi the same way (vaapi gets + device init and an hwupload filtergraph tail). libx264 -crf + --crf (default 18, preset medium) is the fallback everywhere; + an explicit --codec not offered by ffmpeg falls back to + libx264. hevc_videotoolbox gets -tag:v hvc1. Audio aac 192k. + --height scales (aspect kept, even width); default keeps the + source resolution. + loudnorm two-pass ffmpeg loudnorm on the finished file, final render + only (the fast preview never normalizes): pass 1 measures + (loudnorm print_format=json over the whole timeline, which is + why it runs after chunk concat, never per chunk), pass 2 + re-encodes the audio with the measured values (linear mode, + TP -1.5, LRA 11, aac 192k 48kHz) while the video stream is + copied, then atomically replaces the output. Target is + --loudness-target in LUFS (default -14, the YouTube reference); + --no-loudnorm skips both passes. Silent/unmeasurable audio + (non-finite measurements) skips pass 2 with a warning instead + of failing. safety disk preflight before any render: estimated output bytes (from the bitrate ladder) times 2 must fit on the output volume, else the render refuses with a clear message (--skip-disk-check @@ -50,7 +67,9 @@ 0.5s, and --boundary-frames extracts before/after stills at every internal cut for the boundary-frame inspection. summary json.dumps on stdout: segments, chunks, encoder, overlays, - overlays_missing, expected/actual duration, output path. + overlays_missing, expected/actual duration, loudnorm (null when + --no-loudnorm; else {target, applied, input_i, output_i, + output_tp}), output path. Exit codes: 0 ok, 1 failure, 2 usage. @@ -60,6 +79,7 @@ import argparse import json +import math import subprocess import sys import threading @@ -70,6 +90,119 @@ sys.path.insert(0, str(Path(__file__).resolve().parent)) import composite_core as core +# loudnorm companions to the integrated target: true peak ceiling and +# loudness range, the common VOD-delivery pairing for a -14 LUFS target. +LOUDNORM_TP = -1.5 +LOUDNORM_LRA = 11.0 + +# The measurement keys pass 2 feeds back to loudnorm; all must be finite. +MEASURED_KEYS = ("input_i", "input_tp", "input_lra", "input_thresh", + "target_offset") + + +def parse_loudnorm_json(stderr_text): + """Extract the JSON stats block loudnorm prints at the end of stderr. + Numeric-looking values (including '-inf') become floats. Returns None + when no parseable block is present.""" + start = stderr_text.rfind("{") + if start == -1: + return None + end = stderr_text.find("}", start) + if end == -1: + return None + try: + raw = json.loads(stderr_text[start:end + 1]) + except json.JSONDecodeError: + return None + stats = {} + for k, v in raw.items(): + try: + stats[k] = float(v) + except (TypeError, ValueError): + stats[k] = v + return stats + + +def loudnorm_spec(target, measured=None): + """The loudnorm filter spec: measurement form (pass 1) without + `measured`, application form (pass 2, linear) with it.""" + spec = f"loudnorm=I={target:g}:TP={LOUDNORM_TP:g}:LRA={LOUDNORM_LRA:g}" + if measured is not None: + spec += (f":measured_I={measured['input_i']:.2f}" + f":measured_TP={measured['input_tp']:.2f}" + f":measured_LRA={measured['input_lra']:.2f}" + f":measured_thresh={measured['input_thresh']:.2f}" + f":offset={measured['target_offset']:.2f}" + ":linear=true") + return spec + ":print_format=json" + + +def measure_loudness(path, target): + """Loudnorm pass 1: decode the audio once, print the measurement JSON. + Returns the stats dict or None on failure (command echoed to stderr).""" + cmd = ["ffmpeg", "-hide_banner", "-nostats", "-i", str(path), + "-vn", "-af", loudnorm_spec(target), "-f", "null", "-"] + proc = subprocess.run(cmd, capture_output=True, text=True) + if proc.returncode != 0: + print("loudnorm measurement failed:", file=sys.stderr) + print(" ".join(cmd), file=sys.stderr) + print(proc.stderr.strip()[-2000:], file=sys.stderr) + return None + stats = parse_loudnorm_json(proc.stderr) + if stats is None: + print("loudnorm measurement produced no stats block", file=sys.stderr) + return stats + + +def apply_loudnorm(path, measured, target): + """Loudnorm pass 2: re-encode the audio with the measured values (video + copied) to a dotfile beside the output, then atomically replace it. + Returns the pass-2 stats dict (carries output_i/output_tp) or None on + failure; the original file is left untouched on failure.""" + tmp = path.with_name(f".{path.stem}-loudnorm{path.suffix}") + cmd = ["ffmpeg", "-y", "-i", str(path), "-c:v", "copy", + "-af", loudnorm_spec(target, measured), + "-c:a", "aac", "-b:a", "192k", "-ar", "48000"] + if path.suffix.lower() in (".mp4", ".mov"): + cmd += ["-movflags", "+faststart"] + cmd.append(str(tmp)) + proc = subprocess.run(cmd, capture_output=True, text=True) + if proc.returncode != 0: + print("loudnorm apply failed:", file=sys.stderr) + print(" ".join(cmd), file=sys.stderr) + print(proc.stderr.strip()[-2000:], file=sys.stderr) + tmp.unlink(missing_ok=True) + return None + stats = parse_loudnorm_json(proc.stderr) or {} + tmp.replace(path) + return stats + + +def run_loudnorm(output, target): + """Both loudnorm passes over the finished render. Returns the summary + dict, or None on a hard failure (caller exits 1).""" + print(f"render_final: loudnorm pass 1 of 2 (measuring, target " + f"{target:g} LUFS)", file=sys.stderr) + measured = measure_loudness(output, target) + if measured is None: + return None + finite = all(isinstance(measured.get(k), float) + and math.isfinite(measured[k]) for k in MEASURED_KEYS) + if not finite: + print("render_final: audio is silent or unmeasurable; skipping " + "loudness normalization", file=sys.stderr) + return {"target": target, "applied": False, + "input_i": measured.get("input_i"), + "output_i": None, "output_tp": None} + print("render_final: loudnorm pass 2 of 2 (applying)", file=sys.stderr) + stats = apply_loudnorm(output, measured, target) + if stats is None: + return None + return {"target": target, "applied": True, + "input_i": measured["input_i"], + "output_i": stats.get("output_i"), + "output_tp": stats.get("output_tp")} + def run_chunks(cmds, durations, total): """Run the chunk ffmpeg commands in parallel, aggregating -progress output @@ -127,7 +260,7 @@ def concat_chunks(chunk_files, output): for p in chunk_files: quoted = str(p.resolve()).replace("'", "'\\''") lines.append(f"file '{quoted}'\n") - list_file.write_text("".join(lines)) + list_file.write_text("".join(lines), encoding="utf-8") cmd = ["ffmpeg", "-y", "-f", "concat", "-safe", "0", "-i", str(list_file), "-c", "copy", "-bsf:a", "aac_adtstoasc", "-movflags", "+faststart", str(output)] @@ -160,6 +293,11 @@ def main(argv=None): help="scale output to this height (default: source native)") parser.add_argument("--parallel", type=int, default=2, help="max parallel render chunks (default 2)") + parser.add_argument("--loudness-target", type=float, default=-14.0, + help="two-pass loudnorm integrated target in LUFS " + "(default -14)") + parser.add_argument("--no-loudnorm", action="store_true", + help="skip loudness normalization entirely") parser.add_argument("--boundary-frames", default=None, help="dir to write per-cut boundary stills into") parser.add_argument("--skip-disk-check", action="store_true") @@ -178,7 +316,7 @@ def main(argv=None): Path(args.project_dir).resolve() if args.project_dir else edl_path.parent.parent ) - edl = json.loads(edl_path.read_text()) + edl = json.loads(edl_path.read_text(encoding="utf-8")) if not edl.get("segments"): print("edl has no segments", file=sys.stderr) return 2 @@ -233,7 +371,8 @@ def main(argv=None): if len(chunks) == 1: cmd, _ = core.build_command(edl, project_dir, output, scale_height, overlays=overlays, overlay_size=overlay_size, - encode=enc, extra_output_flags=progress_flags) + encode=enc, extra_output_flags=progress_flags, + encoder=encoder) rcs, tails = run_chunks([cmd], [total], total) if rcs[0] != 0: print("ffmpeg render failed:", file=sys.stderr) @@ -253,7 +392,8 @@ def main(argv=None): overlays=ch["overlays"], overlay_size=overlay_size, encode=enc, - extra_output_flags=progress_flags) + extra_output_flags=progress_flags, + encoder=encoder) cmds.append(cmd) files.append(f) durs.append(ch["duration"]) @@ -275,6 +415,12 @@ def main(argv=None): if not ok: return 1 + loudnorm = None + if not args.no_loudnorm: + loudnorm = run_loudnorm(output, args.loudness_target) + if loudnorm is None: + return 1 + actual = core.probe_duration(output) boundary_count = 0 @@ -290,6 +436,7 @@ def main(argv=None): "overlays_missing": missing, "expected_duration_seconds": round(total, 3), "actual_duration_seconds": round(actual, 3) if actual is not None else None, + "loudnorm": loudnorm, "boundary_frames": boundary_count, "output": str(output.resolve()), } diff --git a/skills/mc-cut/scripts/tests/test-edl_to_fcpxml.py b/skills/mc-cut/scripts/tests/test-edl_to_fcpxml.py index 6f8a6f8..2243ab3 100644 --- a/skills/mc-cut/scripts/tests/test-edl_to_fcpxml.py +++ b/skills/mc-cut/scripts/tests/test-edl_to_fcpxml.py @@ -9,13 +9,18 @@ The ffprobe path and editor-import sync are exercised by running the script against a real source; they are not unit-tested here.""" import importlib.util +import json +import os +import shutil import subprocess import sys import tempfile import unittest +import warnings import xml.etree.ElementTree as ET from fractions import Fraction -from pathlib import Path +from pathlib import Path, PurePosixPath, PureWindowsPath +from urllib.parse import quote SCRIPT = Path(__file__).resolve().parent.parent / "edl_to_fcpxml.py" @@ -99,6 +104,35 @@ def test_fmt_time_is_exact_multiple_of_frame_duration(self): self.assertEqual((t / fd).denominator, 1) +class TestMediaRepUri(unittest.TestCase): + # PurePath.as_uri() is deprecated (removal 3.19) but only for PURE paths; + # production always passes a concrete resolved Path. Pure paths here let + # the Windows URI shape be asserted from any OS. + + def uri(self, pure_path): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", DeprecationWarning) + return mod.media_rep_uri(pure_path) + + def test_posix_byte_identical_to_previous_quote_form(self): + # The pre-as_uri() construction was 'file://' + quote(str(path)); + # emitted FCPXML must not change on POSIX paths. + for raw in ("/tmp/my project/raw/camera-a.mp4", "/tmp/a:b~c.mp4"): + p = PurePosixPath(raw) + self.assertEqual(self.uri(p), "file://" + quote(str(p))) + self.assertEqual( + self.uri(PurePosixPath("/tmp/my project/raw/camera-a.mp4")), + "file:///tmp/my%20project/raw/camera-a.mp4") + + def test_windows_drive_letter_uri(self): + p = PureWindowsPath(r"C:\media\my take.mp4") + self.assertEqual(self.uri(p), "file:///C:/media/my%20take.mp4") + + def test_windows_unc_share_uri(self): + p = PureWindowsPath(r"\\server\share\take.mp4") + self.assertEqual(self.uri(p), "file://server/share/take.mp4") + + class TestBuildDocument(unittest.TestCase): def build(self, project_dir=Path("/tmp/my project")): edl = canned_edl() @@ -199,5 +233,75 @@ def test_missing_source_exits_1(self): self.assertEqual(r.returncode, 1, r.stderr) +def run_ascii_locale(args): + """Run the script under an ASCII locale codec with UTF-8 mode off. + + Simulates the Windows failure class (locale codec cp1252): any file + read/write that does not pass encoding="utf-8" explicitly corrupts or + crashes on non-ASCII content.""" + env = dict(os.environ, LC_ALL="C", LANG="C", PYTHONCOERCECLOCALE="0") + return subprocess.run( + [sys.executable, "-X", "utf8=0", str(SCRIPT), *args], + capture_output=True, text=True, env=env) + + +class TestUtf8UnderNonUtf8Locale(unittest.TestCase): + def test_reading_a_non_ascii_edl_does_not_depend_on_the_locale(self): + # A UTF-8 edl.json whose beat text is Czech must parse under an + # ASCII locale codec and reach the normal missing-source error, + # never a UnicodeDecodeError. + with tempfile.TemporaryDirectory() as tmp: + edl = Path(tmp) / "cut" / "edl.json" + edl.parent.mkdir() + edl.write_text( + '{"source":"raw/missing.mp4","fade_ms":30,"pad_ms":60,' + '"segments":[{"source":"raw/missing.mp4","start":0,"end":1,' + '"beat":"Čau, uh, světe"}]}', encoding="utf-8") + r = run_ascii_locale([str(edl), "-o", str(Path(tmp) / "o.fcpxml")]) + self.assertNotIn("UnicodeDecodeError", r.stderr) + self.assertEqual(r.returncode, 1, r.stderr) + self.assertIn("source not found", r.stderr) + + @unittest.skipUnless(shutil.which("ffmpeg") and shutil.which("ffprobe"), + "ffmpeg/ffprobe not installed") + def test_non_ascii_clip_names_export_end_to_end(self): + # Full export with a Czech beat name under an ASCII locale codec: + # the FCPXML must be written as the UTF-8 its XML declaration + # promises (the script's own ET.parse round-trip enforces it), so + # the run succeeds and the clip name survives byte-exact. + with tempfile.TemporaryDirectory() as tmp: + proj = Path(tmp) + (proj / "raw").mkdir() + (proj / "cut").mkdir() + r = subprocess.run( + ["ffmpeg", "-y", + "-f", "lavfi", "-t", "2", "-i", + "testsrc2=size=320x180:rate=30", + "-f", "lavfi", "-t", "2", "-i", + "sine=frequency=440:sample_rate=48000", + "-t", "2", "-shortest", + "-c:v", "libx264", "-preset", "ultrafast", "-crf", "30", + "-pix_fmt", "yuv420p", "-c:a", "aac", + str(proj / "raw" / "a.mp4")], + capture_output=True, text=True) + self.assertEqual(r.returncode, 0, r.stderr) + edl = {"source": "raw/a.mp4", "fade_ms": 30, "pad_ms": 60, + "segments": [{"source": "raw/a.mp4", "start": 0.5, + "end": 1.5, "beat": "Čau, uh, světe"}]} + (proj / "cut" / "edl.json").write_text( + json.dumps(edl, ensure_ascii=False), encoding="utf-8") + out = proj / "o.fcpxml" + r = run_ascii_locale([str(proj / "cut" / "edl.json"), + "-o", str(out)]) + self.assertNotIn("UnicodeDecodeError", r.stderr) + self.assertNotIn("UnicodeEncodeError", r.stderr) + self.assertEqual(r.returncode, 0, r.stderr) + text = out.read_text(encoding="utf-8") + self.assertIn("Čau, uh, světe", text) + names = [c.get("name") + for c in ET.parse(out).getroot().iter("asset-clip")] + self.assertIn("Čau, uh, světe", names) + + if __name__ == "__main__": unittest.main() diff --git a/skills/mc-cut/scripts/tests/test-preflight.py b/skills/mc-cut/scripts/tests/test-preflight.py index 65e73d7..7bc0bee 100644 --- a/skills/mc-cut/scripts/tests/test-preflight.py +++ b/skills/mc-cut/scripts/tests/test-preflight.py @@ -3,9 +3,10 @@ # requires-python = ">=3.11" # /// """Tests for preflight.py: the pure parts (rate parsing, VFR detection, -standard-rate selection, remux command construction) plus CLI exit codes and -a probe/QC integration pass over a fixture synthesized with an ffmpeg test -source (skipped when ffmpeg is not installed).""" +standard-rate selection, remux command construction including the hardware +encoder ladder and vaapi wiring) plus CLI exit codes and a probe/QC +integration pass over a fixture synthesized with an ffmpeg test source +(skipped when ffmpeg is not installed). No hardware encoders are probed.""" import importlib.util import json import shutil @@ -87,6 +88,36 @@ def test_hardware_encode_unknown_height_uses_1080_tier(self): "h264_videotoolbox") self.assertIn("-b:v 24000k", " ".join(cmd)) + def test_ladder_hardware_encoders_take_master_bitrate(self): + for enc in ("h264_nvenc", "h264_qsv", "h264_amf"): + cmd = mod.remux_command("in.mov", "out.mp4", "30/1", enc, + height=1080) + joined = " ".join(cmd) + self.assertIn(f"-c:v {enc}", joined) + self.assertIn("-b:v 24000k", joined) + self.assertNotIn("-crf", joined) + self.assertNotIn("-allow_sw", joined) + self.assertNotIn("hwupload", joined) + + def test_vaapi_gets_device_init_and_hwupload(self): + cmd = mod.remux_command("in.mov", "out.mp4", "30/1", "h264_vaapi", + height=1080) + joined = " ".join(cmd) + self.assertIn("-init_hw_device vaapi=va", joined) + self.assertIn("-filter_hw_device va", joined) + self.assertIn("-vf fps=30/1,format=nv12,hwupload", joined) + self.assertIn("-b:v 24000k", joined) + self.assertNotIn("-crf", joined) + # vaapi receives hardware frames; -pix_fmt must not be forced + self.assertNotIn("-pix_fmt", joined) + # device init comes before the input + self.assertLess(cmd.index("-init_hw_device"), cmd.index("-i")) + + def test_software_encoders_keep_pix_fmt(self): + cmd = mod.remux_command("in.mov", "out.mp4", "30/1", "libx264") + self.assertIn("-pix_fmt yuv420p", " ".join(cmd)) + self.assertNotIn("-init_hw_device", cmd) + class TestMasterEstimate(unittest.TestCase): def test_master_bitrate(self): diff --git a/skills/mc-cut/scripts/tests/test-render_final.py b/skills/mc-cut/scripts/tests/test-render_final.py index 1b41dc5..156654d 100644 --- a/skills/mc-cut/scripts/tests/test-render_final.py +++ b/skills/mc-cut/scripts/tests/test-render_final.py @@ -4,12 +4,14 @@ # /// """Tests for render_final.py and the shared composite_core.py: pure logic (timecode parsing, beat-table parsing, overlay resolution, chunk planning, -encoder selection, disk estimation, progress parsing) plus CLI exit codes, -and one end-to-end render over fixtures synthesized with ffmpeg color/test -sources (skipped when ffmpeg is not installed). No model downloads, no real -footage.""" +encoder selection with the per-OS hardware ladders and injected probes, +loudnorm spec/stats parsing, disk estimation, progress parsing) plus CLI +exit codes, and end-to-end renders over fixtures synthesized with ffmpeg +color/test sources including a real two-pass loudnorm run (skipped when +ffmpeg is not installed). No model downloads, no real footage.""" import importlib.util import json +import os import shutil import subprocess import sys @@ -142,6 +144,10 @@ def test_chunks_cover_everything_in_order(self): self.assertAlmostEqual(a["offset"] + a["duration"], b["offset"]) +def never_probe(encoder): + raise AssertionError(f"probe must not be called (got {encoder!r})") + + class TestEncoderSelection(unittest.TestCase): def test_auto_prefers_videotoolbox_on_macos(self): avail = {"libx264", "h264_videotoolbox"} @@ -151,12 +157,68 @@ def test_auto_prefers_videotoolbox_on_macos(self): self.assertEqual(core.pick_encoder("auto", {"libx264"}, "Darwin"), "libx264") + def test_darwin_never_probes(self): + avail = {"libx264", "h264_videotoolbox"} + self.assertEqual(core.pick_encoder("auto", avail, "Darwin", + probe=never_probe), + "h264_videotoolbox") + self.assertEqual(core.pick_encoder("auto", {"libx264"}, "Darwin", + probe=never_probe), "libx264") + def test_explicit_falls_back_when_unavailable(self): self.assertEqual(core.pick_encoder("hevc_videotoolbox", {"libx264"}, "Darwin"), "libx264") self.assertEqual(core.pick_encoder("libx264", {"libx264"}, "Linux"), "libx264") + def test_explicit_request_never_probes(self): + avail = {"libx264", "h264_nvenc"} + self.assertEqual(core.pick_encoder("h264_nvenc", avail, "Windows", + probe=never_probe), "h264_nvenc") + + def test_windows_ladder_order(self): + avail = {"libx264", "h264_nvenc", "h264_qsv", "h264_amf"} + self.assertEqual(core.pick_encoder("auto", avail, "Windows", + probe=lambda e: True), + "h264_nvenc") + self.assertEqual(core.pick_encoder("auto", avail, "Windows", + probe=lambda e: e != "h264_nvenc"), + "h264_qsv") + self.assertEqual(core.pick_encoder( + "auto", avail, "Windows", + probe=lambda e: e == "h264_amf"), "h264_amf") + self.assertEqual(core.pick_encoder("auto", avail, "Windows", + probe=lambda e: False), "libx264") + + def test_linux_ladder_order(self): + avail = {"libx264", "h264_nvenc", "h264_vaapi"} + self.assertEqual(core.pick_encoder("auto", avail, "Linux", + probe=lambda e: True), + "h264_nvenc") + self.assertEqual(core.pick_encoder("auto", avail, "Linux", + probe=lambda e: e == "h264_vaapi"), + "h264_vaapi") + self.assertEqual(core.pick_encoder("auto", avail, "Linux", + probe=lambda e: False), "libx264") + + def test_ladder_skips_unlisted_encoders_without_probing(self): + # amf is not in this build's encoder list, so it must not be probed; + # nvenc/qsv are listed but their probes fail -> libx264. + avail = {"libx264", "h264_nvenc", "h264_qsv"} + probed = [] + + def probe(e): + probed.append(e) + return False + + self.assertEqual(core.pick_encoder("auto", avail, "Windows", + probe=probe), "libx264") + self.assertEqual(probed, ["h264_nvenc", "h264_qsv"]) + + def test_unknown_system_falls_back(self): + self.assertEqual(core.pick_encoder("auto", {"libx264"}, "Haiku", + probe=never_probe), "libx264") + def test_encode_args(self): x264 = core.encode_args("libx264", crf=20, height=1080) self.assertIn("-crf", x264) @@ -170,12 +232,106 @@ def test_encode_args(self): self.assertIn("hvc1", hevc) self.assertIn("40000k", hevc) + def test_encode_args_hardware_ladder_encoders(self): + for enc in ("h264_nvenc", "h264_qsv", "h264_amf"): + args = core.encode_args(enc, crf=20, height=1080) + self.assertIn("-b:v", args) + self.assertIn("12000k", args) + self.assertNotIn("-crf", args) + self.assertNotIn("-allow_sw", args) + # format negotiation is left to ffmpeg for these encoders + self.assertNotIn("-pix_fmt", args) + vaapi = core.encode_args("h264_vaapi", height=720) + self.assertIn("8000k", vaapi) + self.assertNotIn("-crf", vaapi) + self.assertNotIn("-pix_fmt", vaapi) # hardware frames via hwupload + def test_bitrate_ladder(self): self.assertEqual(core.bitrate_for(720), 8000) self.assertEqual(core.bitrate_for(540), 5000) self.assertEqual(core.bitrate_for(1440), 24000) +class TestEncoderProbe(unittest.TestCase): + def test_probe_command_shape(self): + cmd = core.encoder_probe_command("h264_nvenc") + self.assertEqual(cmd[0], "ffmpeg") + self.assertIn("lavfi", cmd) + self.assertEqual(cmd[cmd.index("-frames:v") + 1], "1") + self.assertEqual(cmd[cmd.index("-c:v") + 1], "h264_nvenc") + self.assertEqual(cmd[-3:], ["-f", "null", "-"]) + self.assertNotIn("-init_hw_device", cmd) + self.assertNotIn("hwupload", " ".join(cmd)) + + def test_probe_command_vaapi_gets_device_and_hwupload(self): + cmd = core.encoder_probe_command("h264_vaapi") + self.assertIn("-init_hw_device", cmd) + self.assertIn("vaapi=va", cmd) + self.assertIn("-filter_hw_device", cmd) + self.assertIn("format=nv12,hwupload", cmd) + + def test_probe_cache_short_circuits(self): + # A cached verdict is returned without running anything (a probe of + # this fake encoder name would otherwise fail or invoke ffmpeg). + self.assertTrue(core.probe_encoder("fake_enc", cache={"fake_enc": True})) + self.assertFalse(core.probe_encoder("fake_enc", cache={"fake_enc": False})) + + def test_probe_records_result_once(self): + cache = {} + first = core.probe_encoder("this_encoder_does_not_exist", cache=cache) + self.assertFalse(first) + self.assertEqual(cache, {"this_encoder_does_not_exist": False}) + # poison the cache: a second call must not re-run the probe + cache["this_encoder_does_not_exist"] = True + self.assertTrue(core.probe_encoder("this_encoder_does_not_exist", + cache=cache)) + + @unittest.skipUnless(FFMPEG, "ffmpeg/ffprobe not installed") + def test_real_probe_passes_for_libx264(self): + self.assertTrue(core.probe_encoder("libx264", cache={})) + + +class TestVaapiWiring(unittest.TestCase): + """The vaapi encode path: device init flags on the command and the + hwupload tail on the filtergraph, absent for every other encoder.""" + + def test_init_flags(self): + self.assertEqual(core.encoder_init_flags("h264_vaapi"), + ["-init_hw_device", "vaapi=va", + "-filter_hw_device", "va"]) + for enc in ("libx264", "h264_videotoolbox", "h264_nvenc", None): + self.assertEqual(core.encoder_init_flags(enc), []) + + def test_build_command_vaapi(self): + edl = edl_4x10() + cmd, _ = core.build_command(edl, Path("/proj"), "out.mp4", 1080, + encoder="h264_vaapi") + self.assertEqual(cmd[1:6], ["-y", "-init_hw_device", "vaapi=va", + "-filter_hw_device", "va"]) + fc = cmd[cmd.index("-filter_complex") + 1] + self.assertIn("format=nv12,hwupload[outv]", fc) + self.assertNotIn("format=yuv420p[outv]", fc) + + def test_build_command_software_unchanged(self): + edl = edl_4x10() + cmd, _ = core.build_command(edl, Path("/proj"), "out.mp4", 1080, + encoder="libx264") + self.assertNotIn("-init_hw_device", cmd) + fc = cmd[cmd.index("-filter_complex") + 1] + self.assertNotIn("hwupload", fc) + # default (no encoder given) is also unchanged + cmd2, _ = core.build_command(edl, Path("/proj"), "out.mp4", 1080) + self.assertNotIn("-init_hw_device", cmd2) + + def test_filter_complex_hwupload_with_overlays(self): + edl = edl_4x10() + ovs = [{"index": 1, "start": 5.0, "dur": 2.0, "image": True}] + fc = core.build_filter_complex(edl, {"raw/a.mp4": 0}, 1080, ovs, + (100, 50), hwupload=True) + self.assertIn("format=nv12,hwupload[outv]", fc) + self.assertNotIn("format=yuv420p[outv]", fc) + + class TestDiskPreflight(unittest.TestCase): def test_estimate_scales_with_duration_and_height(self): small = core.estimate_output_bytes(60, 720) @@ -208,6 +364,57 @@ def test_out_time_fallback_and_garbage(self): self.assertEqual(core.parse_progress("out_time_us=N/A"), {}) +LOUDNORM_STDERR = """\ +[Parsed_loudnorm_0 @ 0x600002bb0000] +{ +\t"input_i" : "-27.61", +\t"input_tp" : "-14.46", +\t"input_lra" : "0.00", +\t"input_thresh" : "-37.61", +\t"output_i" : "-14.01", +\t"output_tp" : "-1.52", +\t"output_lra" : "0.00", +\t"output_thresh" : "-24.03", +\t"normalization_type" : "linear", +\t"target_offset" : "0.01" +} +""" + + +class TestLoudnormHelpers(unittest.TestCase): + def test_parse_stats_block(self): + stats = render_final.parse_loudnorm_json( + "noise before\n" + LOUDNORM_STDERR) + self.assertEqual(stats["input_i"], -27.61) + self.assertEqual(stats["output_i"], -14.01) + self.assertEqual(stats["target_offset"], 0.01) + self.assertEqual(stats["normalization_type"], "linear") + + def test_parse_inf_and_garbage(self): + stats = render_final.parse_loudnorm_json( + '{\n"input_i" : "-inf",\n"input_tp" : "-inf"\n}') + self.assertEqual(stats["input_i"], float("-inf")) + self.assertIsNone(render_final.parse_loudnorm_json("no json here")) + self.assertIsNone(render_final.parse_loudnorm_json("{ broken")) + + def test_spec_measurement_form(self): + spec = render_final.loudnorm_spec(-14.0) + self.assertIn("loudnorm=I=-14:TP=-1.5:LRA=11", spec) + self.assertIn("print_format=json", spec) + self.assertNotIn("measured_I", spec) + + def test_spec_application_form(self): + measured = {"input_i": -27.61, "input_tp": -14.46, "input_lra": 0.0, + "input_thresh": -37.61, "target_offset": 0.01} + spec = render_final.loudnorm_spec(-14.0, measured) + self.assertIn("measured_I=-27.61", spec) + self.assertIn("measured_TP=-14.46", spec) + self.assertIn("measured_LRA=0.00", spec) + self.assertIn("measured_thresh=-37.61", spec) + self.assertIn("offset=0.01", spec) + self.assertIn("linear=true", spec) + + def run_cli(args): return subprocess.run([sys.executable, str(SCRIPTS / "render_final.py"), *args], capture_output=True, text=True) @@ -234,6 +441,26 @@ def test_beats_without_graphics_dir_exits_2(self): "--beats", "beats.md"]) self.assertEqual(r.returncode, 2) + def test_non_ascii_edl_reads_under_non_utf8_locale(self): + # A UTF-8 edl.json whose quote text is Czech must parse under an + # ASCII locale codec (the Windows cp1252 failure class, simulated + # with LC_ALL=C and UTF-8 mode off) and reach the normal + # empty-segments error, never a UnicodeDecodeError. + with tempfile.TemporaryDirectory() as tmp: + edl = Path(tmp) / "edl.json" + edl.write_text('{"source":"x","quote":"Čau, uh, světe",' + '"segments":[]}', encoding="utf-8") + env = dict(os.environ, LC_ALL="C", LANG="C", + PYTHONCOERCECLOCALE="0") + r = subprocess.run( + [sys.executable, "-X", "utf8=0", + str(SCRIPTS / "render_final.py"), + str(edl), "-o", str(Path(tmp) / "f.mp4")], + capture_output=True, text=True, env=env) + self.assertNotIn("UnicodeDecodeError", r.stderr) + self.assertEqual(r.returncode, 2, r.stderr) + self.assertIn("no segments", r.stderr) + @unittest.skipUnless(FFMPEG, "ffmpeg/ffprobe not installed") class TestEndToEnd(unittest.TestCase): @@ -299,21 +526,104 @@ def test_parallel_composited_render(self): self.assertAlmostEqual(summary["actual_duration_seconds"], 2.0, delta=0.5) self.assertEqual(summary["boundary_frames"], 2) + # loudnorm ran by default, post-concat, against the default target + self.assertEqual(summary["loudnorm"]["target"], -14.0) + self.assertTrue(summary["loudnorm"]["applied"]) # progress lines reached stderr self.assertIn("render_final:", r.stderr) - # chunk intermediates were cleaned up + # chunk intermediates and loudnorm temp were cleaned up self.assertEqual(list((self.proj / "renders").glob("*.ts")), []) + self.assertEqual(list((self.proj / "renders").glob(".*loudnorm*")), []) def test_single_chunk_plain_render(self): out = self.proj / "renders" / "plain.mp4" r = run_cli([str(self.proj / "cut" / "edl.json"), "-o", str(out), - "--codec", "libx264", "--crf", "30", "--parallel", "1"]) + "--codec", "libx264", "--crf", "30", "--parallel", "1", + "--no-loudnorm"]) self.assertEqual(r.returncode, 0, r.stderr) summary = json.loads(r.stdout) self.assertEqual(summary["chunks"], 1) self.assertEqual(summary["overlays"], 0) + self.assertIsNone(summary["loudnorm"]) # explicit opt-out self.assertTrue(out.is_file()) +@unittest.skipUnless(FFMPEG, "ffmpeg/ffprobe not installed") +class TestLoudnormEndToEnd(unittest.TestCase): + """A real two-pass loudnorm run: a quiet 7s test tone rendered with the + default -14 LUFS target must measure within about 1 LU of it (measured + independently with a fresh loudnorm analysis pass).""" + + @classmethod + def setUpClass(cls): + cls.tmp = tempfile.TemporaryDirectory() + proj = Path(cls.tmp.name) + (proj / "raw").mkdir() + (proj / "cut").mkdir() + # 8s tone at 0.1 amplitude (~-26 LUFS): far from target, headroom + # for the linear gain to land on -14 without hitting the -1.5 TP cap. + r = subprocess.run( + ["ffmpeg", "-y", + "-f", "lavfi", "-t", "8", "-i", "testsrc2=size=320x180:rate=30", + "-f", "lavfi", "-t", "8", "-i", + "sine=frequency=440:sample_rate=48000", + "-t", "8", "-shortest", "-af", "volume=0.1", + "-c:v", "libx264", "-preset", "ultrafast", "-crf", "30", + "-pix_fmt", "yuv420p", "-c:a", "aac", + str(proj / "raw" / "tone.mp4")], + capture_output=True, text=True) + assert r.returncode == 0, r.stderr + edl = {"source": "raw/tone.mp4", "fade_ms": 30, "pad_ms": 60, + "segments": [ + {"source": "raw/tone.mp4", "start": 0.5, "end": 7.5}]} + (proj / "cut" / "edl.json").write_text(json.dumps(edl)) + cls.proj = proj + + @classmethod + def tearDownClass(cls): + cls.tmp.cleanup() + + def measure(self, path): + """Independent loudness measurement of a finished file.""" + proc = subprocess.run( + ["ffmpeg", "-hide_banner", "-nostats", "-i", str(path), "-vn", + "-af", "loudnorm=print_format=json", "-f", "null", "-"], + capture_output=True, text=True) + self.assertEqual(proc.returncode, 0, proc.stderr) + stats = render_final.parse_loudnorm_json(proc.stderr) + self.assertIsNotNone(stats) + return stats + + def test_output_lands_within_one_lu_of_target(self): + out = self.proj / "renders" / "final.mp4" + r = run_cli([str(self.proj / "cut" / "edl.json"), "-o", str(out), + "--codec", "libx264", "--crf", "30", "--parallel", "1"]) + self.assertEqual(r.returncode, 0, r.stderr) + summary = json.loads(r.stdout) + ln = summary["loudnorm"] + self.assertEqual(ln["target"], -14.0) + self.assertTrue(ln["applied"]) + self.assertLess(ln["input_i"], -20.0) # the tone really was quiet + # the script's own report is close to target... + self.assertAlmostEqual(ln["output_i"], -14.0, delta=1.0) + # ...and so is an independent measurement of the file on disk + measured = self.measure(out) + self.assertAlmostEqual(measured["input_i"], -14.0, delta=1.0) + # duration survived the audio re-encode + self.assertAlmostEqual(summary["actual_duration_seconds"], 7.0, + delta=0.5) + + def test_custom_target(self): + out = self.proj / "renders" / "quiet.mp4" + r = run_cli([str(self.proj / "cut" / "edl.json"), "-o", str(out), + "--codec", "libx264", "--crf", "30", "--parallel", "1", + "--loudness-target", "-19"]) + self.assertEqual(r.returncode, 0, r.stderr) + summary = json.loads(r.stdout) + self.assertEqual(summary["loudnorm"]["target"], -19.0) + measured = self.measure(out) + self.assertAlmostEqual(measured["input_i"], -19.0, delta=1.0) + + if __name__ == "__main__": unittest.main() diff --git a/skills/mc-cut/scripts/tests/test-transcribe.py b/skills/mc-cut/scripts/tests/test-transcribe.py index f7156ba..e0f77f3 100644 --- a/skills/mc-cut/scripts/tests/test-transcribe.py +++ b/skills/mc-cut/scripts/tests/test-transcribe.py @@ -3,17 +3,25 @@ # requires-python = ">=3.11" # /// """Tests for transcribe.py: the deterministic, model-free parts: the -token->word mapping (indexing, gap computation, clamping, rounding) and the -provider-switch exit code. +token->word mapping (indexing, gap computation, clamping, rounding), the +platform provider resolution, the onnx-asr token adapter (SentencePiece +boundary markers, end derivation, score-to-confidence mapping), the fixed +window chunk planner, the overlap merge, and the provider-switch exit codes. -The real transcription requires the parakeet-mlx model and a media file; it is -exercised by running the script directly, not unit-tested here. These tests -import only stdlib and the script's pure helpers, so they run without the -parakeet-mlx dependency, no model download, and no network.""" +The real transcription requires the parakeet-mlx or onnx-asr model and a media +file; it is exercised by running the script directly, not unit-tested here. +These tests import only stdlib and the script's pure helpers, so they run +without either model dependency, no model download, and no network.""" +import contextlib import importlib.util +import io +import json +import math import subprocess import sys +import tempfile import unittest +import unittest.mock as mock from pathlib import Path SCRIPT = Path(__file__).resolve().parent.parent / "transcribe.py" @@ -131,6 +139,341 @@ def __init__(self, text, start, end, confidence): self.assertEqual(words[0]["gap_after"], 0.5) +class TestDefaultProvider(unittest.TestCase): + def test_apple_silicon_gets_parakeet_mlx(self): + self.assertEqual( + transcribe.default_provider("darwin", "arm64"), "parakeet-mlx") + + def test_intel_mac_gets_onnx(self): + self.assertEqual( + transcribe.default_provider("darwin", "x86_64"), "onnx-asr") + + def test_windows_gets_onnx(self): + self.assertEqual( + transcribe.default_provider("win32", "AMD64"), "onnx-asr") + + def test_linux_gets_onnx(self): + self.assertEqual( + transcribe.default_provider("linux", "x86_64"), "onnx-asr") + self.assertEqual( + transcribe.default_provider("linux", "aarch64"), "onnx-asr") + + +class TestScoreToConfidence(unittest.TestCase): + def test_none_is_no_signal_one(self): + self.assertEqual(transcribe._score_to_confidence(None), 1.0) + + def test_logprobs_are_exponentiated(self): + self.assertAlmostEqual( + transcribe._score_to_confidence(-0.1), math.exp(-0.1)) + self.assertEqual(transcribe._score_to_confidence(0.0), 1.0) + + def test_probabilities_pass_through_and_clamp(self): + self.assertEqual(transcribe._score_to_confidence(0.97), 0.97) + self.assertEqual(transcribe._score_to_confidence(1.7), 1.0) + + +class TestOnnxTokensToParakeet(unittest.TestCase): + def test_boundary_marker_becomes_leading_space(self): + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁Al", "rig", "ht", ",", "▁I"], + [1.28, 1.52, 1.68, 1.68, 1.76], + ) + self.assertEqual([t["text"] for t in tokens], + [" Al", "rig", "ht", ",", " I"]) + + def test_start_only_timestamps_derive_ends(self): + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁a", "b", "▁c"], [0.0, 0.4, 0.4]) + # end = next start when the next token CONTINUES the same word + self.assertEqual(tokens[0]["end"], 0.4) + # end = start + one 80 ms frame when the next start does not advance + self.assertAlmostEqual(tokens[1]["end"], 0.48) + # last token: start + one frame + self.assertAlmostEqual(tokens[2]["end"], 0.48) + + def test_end_before_a_new_word_is_capped_not_stretched(self): + # The next token opens a NEW word close by: the derived end still + # reaches its onset (the cap only engages past a few frames). + near = transcribe.onnx_tokens_to_parakeet(["▁a", "▁b"], [0.0, 0.1]) + self.assertAlmostEqual(near[0]["end"], 0.1) + # But when the new word's onset sits past the cap, the end must NOT + # be dragged to it: parakeet emits no tokens during silence, and + # stretching the end across the pause would zero the gap data. + far = transcribe.onnx_tokens_to_parakeet(["▁a", "▁b"], [0.0, 3.0]) + self.assertAlmostEqual( + far[0]["end"], + transcribe.WORD_END_CAP_FRAMES * transcribe.FRAME_S) + + def test_multi_second_pause_survives_into_gap_after(self): + # Regression: start-only timestamps around a real 3.1 s pause + # ('point.' then silence then 'Next'). The pause must land in + # gap_after, not be absorbed into the word before it. + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁point", ".", "▁Next"], [9.5, 9.9, 13.0]) + words = transcribe.normalize_words(tokens, duration=15.0) + self.assertEqual([w["word"] for w in words], ["point.", "Next"]) + self.assertLessEqual( + words[0]["end"], + 9.9 + transcribe.WORD_END_CAP_FRAMES * transcribe.FRAME_S) + self.assertGreaterEqual(words[0]["gap_after"], 2.8) + self.assertGreaterEqual(words[1]["gap_before"], 2.8) + + def test_pair_timestamps_are_used_directly(self): + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁hi"], [(0.16, 0.56)]) + self.assertEqual(tokens[0]["start"], 0.16) + self.assertEqual(tokens[0]["end"], 0.56) + + def test_logprobs_map_to_confidence_and_absence_degrades_to_one(self): + with_scores = transcribe.onnx_tokens_to_parakeet( + ["▁a", "b"], [0.0, 0.4], logprobs=[-0.1, 0.0]) + self.assertAlmostEqual(with_scores[0]["confidence"], math.exp(-0.1)) + self.assertEqual(with_scores[1]["confidence"], 1.0) + without = transcribe.onnx_tokens_to_parakeet( + ["▁a", "b"], [0.0, 0.4]) + self.assertEqual([t["confidence"] for t in without], [1.0, 1.0]) + + def test_bare_marker_token_carries_boundary_to_next_token(self): + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁a", "▁", "b"], [0.0, 0.4, 0.48]) + self.assertEqual([t["text"] for t in tokens], [" a", " b"]) + self.assertEqual(tokens[1]["start"], 0.48) + + def test_clamp_caps_end_times(self): + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁a"], [19.96], clamp=20.0) + self.assertEqual(tokens[0]["end"], 20.0) + + def test_count_mismatch_raises(self): + with self.assertRaises(ValueError): + transcribe.onnx_tokens_to_parakeet(["▁a", "b"], [0.0]) + + def test_parity_with_parakeet_lane_words(self): + # The same subwords the parakeet fixture uses, as SentencePiece + # tokens with (start, end) pairs, must normalize to the identical + # pinned word schema (byte-compatible words.json across lanes). + tokens = transcribe.onnx_tokens_to_parakeet( + ["▁Al", "rig", "ht", ",", "▁I", "'", "m", "▁doing"], + [(1.28, 1.52), (1.52, 1.68), (1.68, 1.68), (1.68, 1.76), + (1.76, 1.84), (1.84, 1.92), (1.92, 2.0), (2.0, 2.24)], + ) + words = transcribe.normalize_words(tokens, duration=10.0) + self.assertEqual(words[0], { + "word": "Alright,", "start": 1.28, "end": 1.76, "confidence": 1.0, + "i": 0, "gap_before": 1.28, "gap_after": 0.0, + }) + self.assertEqual([w["word"] for w in words], ["Alright,", "I'm", "doing"]) + + +class TestPlanChunks(unittest.TestCase): + def test_short_audio_is_a_single_chunk(self): + self.assertEqual(transcribe.plan_chunks(12.5, window=20.0, overlap=2.0), + [(0.0, 12.5)]) + self.assertEqual(transcribe.plan_chunks(20.0, window=20.0, overlap=2.0), + [(0.0, 20.0)]) + + def test_long_audio_steps_by_window_minus_overlap(self): + chunks = transcribe.plan_chunks(40.0, window=20.0, overlap=2.0) + self.assertEqual(chunks, [(0.0, 20.0), (18.0, 20.0), (36.0, 4.0)]) + # full coverage: last chunk reaches the duration + self.assertEqual(chunks[-1][0] + chunks[-1][1], 40.0) + # adjacent chunks share exactly the overlap + self.assertEqual(chunks[0][0] + chunks[0][1] - chunks[1][0], 2.0) + + def test_zero_duration_is_a_single_empty_chunk(self): + self.assertEqual(transcribe.plan_chunks(0.0), [(0.0, 0.0)]) + + def test_bad_window_or_overlap_raises(self): + with self.assertRaises(ValueError): + transcribe.plan_chunks(10.0, window=0.0) + with self.assertRaises(ValueError): + transcribe.plan_chunks(10.0, window=20.0, overlap=20.0) + + +class TestOffsetTokens(unittest.TestCase): + def test_offsets_start_and_end_only(self): + out = transcribe.offset_tokens( + [{"text": " a", "start": 1.0, "end": 1.5, "confidence": 0.9}], 18.0) + self.assertEqual(out, [{"text": " a", "start": 19.0, "end": 19.5, + "confidence": 0.9}]) + + +class TestMergeChunkTokens(unittest.TestCase): + def _tok(self, text, start, end): + return {"text": text, "start": start, "end": end, "confidence": 1.0} + + def test_single_chunk_passes_through(self): + tokens = [self._tok(" a", 0.0, 0.5), self._tok(" b", 1.0, 1.5)] + self.assertEqual( + transcribe.merge_chunk_tokens([(0.0, 20.0, tokens)]), tokens) + + def test_overlap_cuts_at_midpoint_without_duplication(self): + # overlap region 18..20, cut at 19: the word at 18.5 belongs to the + # first chunk, the shared boundary word (both chunks heard it, with + # slightly different times) belongs to the second. + first = [ + self._tok(" early", 5.0, 5.5), + self._tok(" late", 18.5, 18.9), + self._tok(" edge", 19.5, 19.9), # first chunk's take, dropped + ] + second = [ + self._tok(" edge", 19.45, 19.9), # second chunk's take, kept + self._tok(" after", 21.0, 21.5), + ] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], + [" early", " late", " edge", " after"]) + self.assertEqual(merged[2]["start"], 19.45) + + def test_word_starting_exactly_at_cut_goes_to_the_later_chunk(self): + first = [self._tok(" at", 19.0, 19.4)] + second = [self._tok(" at", 19.0, 19.4)] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], [" at"]) + + def test_multi_token_words_stay_whole(self): + # A word's subword continuations travel with its first token even + # when the continuations fall past the cut. + first = [ + self._tok(" spl", 18.8, 19.0), + self._tok("it", 19.0, 19.2), + ] + second = [self._tok(" next", 20.5, 21.0)] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], [" spl", "it", " next"]) + + def test_straddling_estimates_do_not_duplicate(self): + # The two chunks time the same boundary word independently; when the + # estimates straddle the cut (18.9 < 19 <= 19.05) the base rule keeps + # BOTH takes. The seam pass must keep exactly one. + first = [ + self._tok(" before", 17.0, 17.4), + self._tok(" edge", 18.9, 19.3), # < cut: kept by chunk 1 + ] + second = [ + self._tok(" edge", 19.05, 19.45), # >= cut: kept by chunk 2 + self._tok(" after", 21.0, 21.5), + ] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], + [" before", " edge", " after"]) + # the takes' mean midpoint (19.175) is past the cut, nearer the + # second chunk's window center, so its take wins + self.assertEqual(merged[1]["start"], 19.05) + + def test_crosswise_estimates_do_not_drop(self): + # Crosswise takes: chunk 1 heard the word at 19.1 (>= cut, dropped), + # chunk 2 heard it at 18.9 (< cut, dropped). The base rule loses the + # word entirely; the seam pass must restore exactly one take. + first = [ + self._tok(" before", 17.0, 17.4), + self._tok(" edge", 19.1, 19.5), # >= cut: dropped by chunk 1 + ] + second = [ + self._tok(" edge", 18.9, 19.3), # < cut: dropped by chunk 2 + self._tok(" after", 21.0, 21.5), + ] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], + [" before", " edge", " after"]) + # mean midpoint of the takes (19.2) is past the cut, nearer the + # second chunk's center, so chunk 2's take is restored + self.assertEqual(merged[1]["start"], 18.9) + + def test_seam_repair_survives_multi_token_words(self): + # A duplicated word made of several subword tokens deduplicates as a + # whole run, never token by token. + first = [self._tok(" spl", 18.8, 19.0), self._tok("it", 19.0, 19.2)] + second = [self._tok(" spl", 19.0, 19.2), self._tok("it", 19.2, 19.4)] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], [" spl", "it"]) + + def test_repeated_words_far_apart_are_not_deduplicated(self): + # Identical text alone never merges words; the takes' time spans must + # overlap. Two real occurrences of "the" survive. + first = [self._tok(" the", 18.2, 18.5)] + second = [self._tok(" the", 19.4, 19.7)] + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], [" the", " the"]) + + def test_words_come_out_sorted_by_start(self): + first = [self._tok(" b", 10.0, 10.5)] + second = [self._tok(" a", 19.0, 19.5)] + # second chunk's word starts inside the first chunk's exclusive zone + # is impossible by construction; this asserts the stable sort on the + # kept words. + merged = transcribe.merge_chunk_tokens( + [(0.0, 20.0, first), (18.0, 38.0, second)]) + self.assertEqual([t["text"] for t in merged], [" b", " a"]) + self.assertLess(merged[0]["start"], merged[1]["start"]) + + +class TestOutputEncoding(unittest.TestCase): + def test_words_json_is_written_utf8_never_the_locale_codec(self): + # The onnx-asr lane exists for the 25 European parakeet languages; + # a Czech transcript must land as UTF-8 bytes even where the locale + # codec is cp1252 (Windows), so the write must pass encoding="utf-8" + # explicitly and downstream utf-8 readers must get the text back. + tokens = [ + {"text": " Čau,", "start": 0.0, "end": 0.4, "confidence": 1.0}, + {"text": " světe", "start": 0.5, "end": 0.9, "confidence": 1.0}, + ] + with tempfile.TemporaryDirectory() as tmp: + media = Path(tmp) / "take.mp4" + media.write_bytes(b"\x00") + out = Path(tmp) / "words.json" + with mock.patch.object( + transcribe, "transcribe_parakeet", + return_value=("Čau, světe", tokens, 1.0)), \ + mock.patch.object(Path, "write_text", autospec=True, + side_effect=Path.write_text) as writes, \ + contextlib.redirect_stdout(io.StringIO()): + rc = transcribe.main([str(media), "-o", str(out), + "--provider", "parakeet-mlx"]) + self.assertEqual(rc, 0) + encodings = [c.kwargs.get("encoding") + for c in writes.call_args_list] + self.assertEqual(encodings, ["utf-8"]) + payload = json.loads(out.read_bytes().decode("utf-8")) + self.assertEqual(payload["words"][0]["word"], "Čau,") + self.assertEqual(payload["text"], "Čau, světe") + + +class TestCudaFallbackWarning(unittest.TestCase): + def test_nvidia_box_without_cuda_ep_warns_loudly(self): + msg = transcribe.cuda_fallback_warning( + ["CPUExecutionProvider"], nvidia_smi_present=True) + self.assertIsNotNone(msg) + self.assertIn("CUDAExecutionProvider", msg) + self.assertIn('onnx-asr[gpu,hub]', msg) + # the escalation must go through the python command so the script's + # cpu extra is skipped (onnxruntime + onnxruntime-gpu never co-install) + self.assertIn("python", msg) + + def test_nvidia_box_with_cuda_ep_is_silent(self): + self.assertIsNone(transcribe.cuda_fallback_warning( + ["CUDAExecutionProvider", "CPUExecutionProvider"], + nvidia_smi_present=True)) + + def test_no_nvidia_gpu_is_silent(self): + self.assertIsNone(transcribe.cuda_fallback_warning( + ["CPUExecutionProvider"], nvidia_smi_present=False)) + self.assertIsNone(transcribe.cuda_fallback_warning( + None, nvidia_smi_present=False)) + + def test_no_provider_list_on_an_nvidia_box_warns(self): + self.assertIsNotNone(transcribe.cuda_fallback_warning( + None, nvidia_smi_present=True)) + + class TestProviderSwitch(unittest.TestCase): def test_unimplemented_provider_exits_3(self): r = run(["some.mp4", "-o", "out.json", "--provider", "deepgram-nova3"]) @@ -141,6 +484,23 @@ def test_missing_output_arg_is_usage_error(self): r = run(["some.mp4"]) self.assertEqual(r.returncode, 2) + def test_onnx_asr_is_an_implemented_provider(self): + # provider accepted; the missing media is caught first (exit 2), + # so no model dependency is touched. + r = run(["missing.mp4", "-o", "out.json", "--provider", "onnx-asr"]) + self.assertEqual(r.returncode, 2) + self.assertIn("media not found", r.stderr) + + def test_auto_resolves_and_reports_a_platform_lane(self): + r = run(["missing.mp4", "-o", "out.json", "--provider", "auto"]) + self.assertEqual(r.returncode, 2) + self.assertIn("provider resolved:", r.stderr) + + def test_help_exits_zero(self): + r = run(["--help"]) + self.assertEqual(r.returncode, 0) + self.assertIn("--provider", r.stdout) + if __name__ == "__main__": unittest.main() diff --git a/skills/mc-cut/scripts/transcribe.py b/skills/mc-cut/scripts/transcribe.py index 321ad44..7fd2783 100644 --- a/skills/mc-cut/scripts/transcribe.py +++ b/skills/mc-cut/scripts/transcribe.py @@ -1,14 +1,17 @@ #!/usr/bin/env python3 # /// script # requires-python = ">=3.11" -# dependencies = ["parakeet-mlx"] +# dependencies = [ +# "parakeet-mlx; sys_platform == 'darwin' and platform_machine == 'arm64'", +# "onnx-asr[cpu,hub]; sys_platform != 'darwin' or platform_machine != 'arm64'", +# ] # /// """Word-level transcription for the cut stage (build-order item 1). Usage: uv run {skill-root}/scripts/transcribe.py {projects-path}//raw/ \ -o {projects-path}//transcript/words.json \ - [--provider parakeet-mlx] [--model mlx-community/parakeet-tdt-0.6b-v3] + [--provider auto|parakeet-mlx|onnx-asr] [--model ] Contract: input a media file (audio or video); the transcriber extracts audio via @@ -31,16 +34,62 @@ times round to 2 decimals, confidence to 4. "i" is the word index. gap_before = start - previous word's end (first word: its start). gap_after = next word's start - end (last word: duration - end). - gaps never go negative (clamped to 0.0). + gaps never go negative (clamped to 0.0). The shape is identical + across providers; downstream consumers (cutplan.py) never need to + know which lane produced the file. provider from the studio config [transcription] table; this script is the - switch. parakeet-mlx is the reference and the default: free, local - on Apple Silicon, native word timestamps, and empirically it keeps - verbatim fillers ("uh", "um", "Hmm") that cutting depends on. - Metered API providers (for example elevenlabs-scribe or - deepgram-nova3) are possible future OPT-IN lanes behind - --provider and the [transcription] switch; none is implemented - here, nothing defaults to them, and no API key name ships in - any default. + switch. "auto" (the default) picks per platform: + macOS Apple Silicon -> parakeet-mlx (the reference lane) + everything else -> onnx-asr (same weights, ONNX runtime) + parakeet-mlx is the reference: free, local on Apple Silicon, + native word timestamps, and empirically it keeps verbatim fillers + ("uh", "um", "Hmm") that cutting depends on. onnx-asr runs the + SAME parakeet-tdt-0.6b-v3 weights (the istupakov ONNX conversion, + onnx-asr model name "nemo-parakeet-tdt-0.6b-v3"), so filler + preservation and 80 ms frame-granular timestamps carry over on + Windows, Linux, and Intel Macs. Metered API providers (for + example elevenlabs-scribe or deepgram-nova3) are possible future + OPT-IN lanes behind --provider and the [transcription] switch; + none is implemented here, nothing defaults to them, and no API + key name ships in any default. + gpu PEP 508 markers cannot detect GPUs, so the onnx-asr dependency + ships with the cpu extra. On a CUDA box, escalate at the call + site with a uv override: + uv run --with "onnx-asr[gpu,hub]" \ + python {skill-root}/scripts/transcribe.py ... + The `python` command is load-bearing: it makes uv skip this + script's inline metadata, so ONLY the gpu extra installs. + Running the script path directly with --with would merge the + extras and co-install onnxruntime (cpu) with onnxruntime-gpu, + which ship the same import package; ONNX Runtime does not + support that, and the collision routinely leaves + CUDAExecutionProvider unavailable. When onnxruntime reports + CUDAExecutionProvider available, this script prefers it + automatically; otherwise it runs on CPU, and if nvidia-smi is + on PATH it prints a loud warning that the GPU escalation is + needed (or failed) instead of silently running slow. + chunking parakeet-mlx chunks long audio internally. The onnx-asr lane + caps around 20-30 s per call, so this script extracts fixed + 20 s windows with 2 s overlap via ffmpeg (16 kHz mono wav), + offsets each chunk's timestamps by its window start, and merges + at the overlap midpoint on word boundaries with a seam-repair + pass (the two chunks time boundary words independently, so a + take kept by both sides deduplicates and a take kept by + neither restores from the nearer chunk): no word is split, + duplicated, or dropped across chunks. + confidence parakeet-mlx reports per-token confidence natively. The + onnx-asr lane maps per-token scores when the runtime exposes + them (logprobs are exponentiated into probabilities, values + already in 0..1 pass through, everything clamps to 0..1). When + the runtime exposes no per-token scores, confidence degrades to + 1.0 for every word: downstream consumers see "no signal", never + a fabricated number. + +Exit codes: + 0 transcript written + 1 transcription failed (model, ffmpeg, or provider runtime error) + 2 usage error (bad arguments, media not found) + 3 provider not implemented (future opt-in lanes) Why parakeet-mlx over generic Whisper: cutting needs verbatim fillers ("um", "uh", restarts) plus word gap data; Whisper normalizes exactly those away. @@ -48,12 +97,56 @@ import argparse import json +import math +import platform +import shutil import subprocess import sys +import tempfile from pathlib import Path -DEFAULT_PROVIDER = "parakeet-mlx" -DEFAULT_MODEL = "mlx-community/parakeet-tdt-0.6b-v3" +PROVIDER_AUTO = "auto" +PROVIDER_MLX = "parakeet-mlx" +PROVIDER_ONNX = "onnx-asr" +IMPLEMENTED_PROVIDERS = (PROVIDER_MLX, PROVIDER_ONNX) + +DEFAULT_PROVIDER = PROVIDER_AUTO +DEFAULT_MODELS = { + PROVIDER_MLX: "mlx-community/parakeet-tdt-0.6b-v3", + # onnx-asr registered name; resolves to the HF hub conversion + # istupakov/parakeet-tdt-0.6b-v3-onnx (same weights as the mlx lane). + PROVIDER_ONNX: "nemo-parakeet-tdt-0.6b-v3", +} +# Back-compat alias: the historical single-lane default model id. +DEFAULT_MODEL = DEFAULT_MODELS[PROVIDER_MLX] + +# Parakeet timestamps are 80 ms frame-granular on every runtime. +FRAME_S = 0.08 +# onnx-asr reports start-only token timestamps. When the NEXT token opens a +# new word, its onset can sit on the far side of a pause (parakeet emits no +# tokens during silence), so the current token's derived end is capped at +# this many frames past its start; otherwise every pause would be absorbed +# into the preceding word and the gap data cutting depends on would read 0. +WORD_END_CAP_FRAMES = 3 +# onnx-asr caps most models around 20-30 s per call; fixed windows + overlap. +CHUNK_WINDOW_S = 20.0 +CHUNK_OVERLAP_S = 2.0 + +# SentencePiece word-boundary marker used by the ONNX tokenizer. +SP_MARK = "▁" + + +def default_provider(platform_name=None, machine=None): + """Resolve the automatic provider for a platform (pure). + + macOS Apple Silicon gets the parakeet-mlx reference lane; every other + platform (Windows, Linux, Intel Mac) gets onnx-asr with the same weights. + """ + platform_name = platform_name if platform_name is not None else sys.platform + machine = machine if machine is not None else platform.machine() + if platform_name == "darwin" and machine == "arm64": + return PROVIDER_MLX + return PROVIDER_ONNX def _get(t, key): @@ -152,21 +245,392 @@ def transcribe_parakeet(media, model_id): return result.text, result.tokens, duration +# --- onnx-asr lane ----------------------------------------------------------- + + +def _score_to_confidence(value): + """Map a per-token score into the 0..1 confidence shape (pure). + + None (runtime exposes no scores) -> 1.0, the documented no-signal value. + Values <= 0 are treated as logprobs and exponentiated. Positive values are + treated as probabilities. Everything clamps into 0..1. + """ + if value is None: + return 1.0 + v = float(value) + if v <= 0.0: + return math.exp(v) + return min(1.0, v) + + +def onnx_tokens_to_parakeet(tokens, timestamps, logprobs=None, frame=FRAME_S, + clamp=None): + """Convert onnx-asr token/timestamp arrays into parakeet-shaped dicts (pure). + + tokens are SentencePiece pieces where the word-boundary marker (U+2581) + starts a new word; the marker becomes the leading space the parakeet + grouping keys on, so normalize_words consumes both lanes identically. + timestamps entries are either start seconds or (start, end) pairs. With + start-only entries the end is derived: the next token's start when that + token CONTINUES the same word; capped at start + WORD_END_CAP_FRAMES + frames when the next token opens a NEW word (its onset may sit past a + pause, and dragging the end across the silence would zero the gap data + downstream cutting keys on); start + one 80 ms frame for the last token. + logprobs, when provided, map per token through + _score_to_confidence; when absent every token gets confidence 1.0. + clamp, when given, caps end times (chunk length). A bare marker token + carries its word boundary onto the next token instead of emitting an + empty word. Raises ValueError when tokens and timestamps disagree in + length. + """ + if len(tokens) != len(timestamps): + raise ValueError( + f"token/timestamp count mismatch: {len(tokens)} tokens, " + f"{len(timestamps)} timestamps" + ) + + def _start_of(entry): + if isinstance(entry, (list, tuple)): + return float(entry[0]) + return float(entry) + + def _starts_word(tok): + # A raw token opens a new word when it begins with the SentencePiece + # boundary marker (a bare marker token counts: it carries the + # boundary onto the next token). + return str(tok).replace(SP_MARK, " ").startswith(" ") + + out = [] + pending_space = False + n = len(tokens) + for i, tok in enumerate(tokens): + text = str(tok).replace(SP_MARK, " ") + if text.strip() == "": + # bare boundary marker (or empty token): carry the word boundary + # onto the next token rather than emitting an empty word. + if text.startswith(" "): + pending_space = True + continue + if pending_space: + if not text.startswith(" "): + text = " " + text + pending_space = False + + entry = timestamps[i] + if isinstance(entry, (list, tuple)) and len(entry) >= 2: + start = float(entry[0]) + end = float(entry[1]) + else: + start = _start_of(entry) + if i + 1 < n: + nxt = _start_of(timestamps[i + 1]) + if nxt <= start: + end = start + frame + elif _starts_word(tokens[i + 1]): + # The next token opens a new word; parakeet emits no + # tokens during silence, so its onset may sit across a + # pause. Cap the derived end near this token instead of + # dragging it over the silence. + end = min(nxt, start + WORD_END_CAP_FRAMES * frame) + else: + end = nxt + else: + end = start + frame + if clamp is not None: + end = min(end, float(clamp)) + if end < start: + end = start + + score = None + if logprobs is not None and i < len(logprobs): + score = logprobs[i] + out.append({ + "text": text, + "start": start, + "end": end, + "confidence": _score_to_confidence(score), + }) + return out + + +def offset_tokens(tokens, offset): + """Shift token dicts by a chunk's window start (pure).""" + return [ + {**t, "start": t["start"] + offset, "end": t["end"] + offset} + for t in tokens + ] + + +def plan_chunks(duration, window=CHUNK_WINDOW_S, overlap=CHUNK_OVERLAP_S): + """Fixed transcription windows with overlap (pure). + + Returns [(start, length), ...] covering the full duration. A duration + within one window yields a single chunk. Later chunks start one window + minus one overlap after the previous, so adjacent chunks share `overlap` + seconds; the merge cuts at the overlap midpoint. + """ + duration = float(duration) + if window <= 0: + raise ValueError("window must be positive") + if overlap < 0 or overlap >= window: + raise ValueError("overlap must be >= 0 and smaller than window") + if duration <= 0: + return [(0.0, 0.0)] + if duration <= window: + return [(0.0, round(duration, 3))] + chunks = [] + step = window - overlap + start = 0.0 + while True: + length = min(window, duration - start) + chunks.append((round(start, 3), round(length, 3))) + if start + window >= duration: + break + start += step + return chunks + + +def _word_runs(tokens): + """Group token dicts into word runs by the leading-space boundary (pure).""" + runs = [] + for t in tokens: + if not runs or t["text"][:1] == " ": + runs.append([t]) + else: + runs[-1].append(t) + return runs + + +def _run_text(run): + """Normalized word text of a subword run, for cross-chunk matching (pure).""" + return "".join(t["text"] for t in run).strip().lower() + + +def _runs_overlap(a, b): + """True when two runs' time spans intersect (pure).""" + return a[0]["start"] < b[-1]["end"] and b[0]["start"] < a[-1]["end"] + + +def merge_chunk_tokens(chunks): + """Merge per-chunk token lists across overlaps (pure). + + chunks is [(chunk_start, chunk_end, tokens_abs), ...] in order, where + tokens_abs already carry absolute (offset) times. For each overlap the cut + point is the midpoint; a word (run of subword tokens) belongs to the chunk + whose side of the cut its own start falls on. Both chunks transcribe the + boundary words independently (different acoustic context, 80 ms frame + granularity), so the two takes' start estimates can disagree about the + cut; a seam pass over each overlap then repairs the two failure shapes: + a word kept by BOTH chunks (time spans overlap, identical normalized + text) keeps only the take from the chunk whose window center is nearer, + and a word kept by NEITHER (the takes landed crosswise around the cut) + is restored from the nearer chunk's take. No word is split, duplicated, + or dropped across a seam. Words are emitted sorted by start time. + """ + n = len(chunks) + kept = [] # per chunk: runs on their own side of the cuts + dropped = [] # per chunk: runs a cut excluded + for i, (cstart, cend, tokens) in enumerate(chunks): + left = (chunks[i - 1][1] + cstart) / 2 if i > 0 else float("-inf") + right = (cend + chunks[i + 1][0]) / 2 if i < n - 1 else float("inf") + k, d = [], [] + for run in _word_runs(tokens): + (k if left <= run[0]["start"] < right else d).append(run) + kept.append(k) + dropped.append(d) + + # Seam pass over each overlap window. + for i in range(n - 1): + cut = (chunks[i][1] + chunks[i + 1][0]) / 2 + center_i = (chunks[i][0] + chunks[i][1]) / 2 + center_j = (chunks[i + 1][0] + chunks[i + 1][1]) / 2 + + def _first_is_nearer(a, b): + # midpoint of the two takes; ties go to the later chunk (matching + # the base rule that a start exactly at the cut is the later + # chunk's). + mid = (a[0]["start"] + a[-1]["end"] + + b[0]["start"] + b[-1]["end"]) / 4 + return abs(mid - center_i) < abs(mid - center_j) + + # Duplicates: the same word kept by both chunks (chunk i's estimate + # fell before the cut, chunk i+1's on or after it). Keep one take. + overlap_start = chunks[i + 1][0] + for a in list(kept[i]): + if a[-1]["end"] <= overlap_start: + continue # entirely before the overlap window, no seam risk + for b in list(kept[i + 1]): + if _runs_overlap(a, b) and _run_text(a) == _run_text(b): + if _first_is_nearer(a, b): + kept[i + 1].remove(b) + else: + kept[i].remove(a) + break + + # Lost words: both takes landed crosswise (chunk i's estimate on or + # after the cut, chunk i+1's before it), so neither side kept the + # word. Restore the nearer chunk's take. + lost_left = [b for b in dropped[i + 1] if b[0]["start"] < cut] + for a in dropped[i]: + if a[0]["start"] < cut: + continue # dropped at chunk i's left seam, not this one + for b in list(lost_left): + if _runs_overlap(a, b) and _run_text(a) == _run_text(b): + if _first_is_nearer(a, b): + kept[i].append(a) + else: + kept[i + 1].append(b) + lost_left.remove(b) + break + + ordered = sorted( + (run for chunk_runs in kept for run in chunk_runs), + key=lambda run: run[0]["start"], + ) + merged = [] + for run in ordered: + merged.extend(run) + return merged + + +def extract_chunk(media, start, length, out_wav): + """Extract one 16 kHz mono wav window via ffmpeg. Raises on failure.""" + subprocess.run( + [ + "ffmpeg", "-hide_banner", "-loglevel", "error", "-y", + "-ss", f"{start:.3f}", "-t", f"{length:.3f}", + "-i", str(media), + "-vn", "-ac", "1", "-ar", "16000", "-c:a", "pcm_s16le", + str(out_wav), + ], + capture_output=True, text=True, check=True, + ) + + +def _preferred_onnx_providers(): + """Prefer CUDA when the installed onnxruntime exposes it, else CPU.""" + try: + import onnxruntime + available = list(onnxruntime.get_available_providers()) + except Exception: + return None + ordered = [ + p for p in ("CUDAExecutionProvider", "CPUExecutionProvider") + if p in available + ] + return ordered or None + + +def cuda_fallback_warning(available_providers, nvidia_smi_present): + """Warning text when a CUDA GPU is visible but unusable, else None (pure). + + An NVIDIA box (nvidia-smi on PATH) whose installed onnxruntime lacks + CUDAExecutionProvider means the GPU escalation was skipped or failed + (typically onnxruntime and onnxruntime-gpu co-installed and colliding); + transcription would silently run on CPU at multi-x realtime. This makes + that state loud instead of silent.""" + if not nvidia_smi_present: + return None + if "CUDAExecutionProvider" in (available_providers or []): + return None + return ( + "WARNING: nvidia-smi is on PATH but the installed onnxruntime has no " + "CUDAExecutionProvider; transcription will run on CPU. For GPU, " + 'escalate with: uv run --with "onnx-asr[gpu,hub]" python ' + "/scripts/transcribe.py ... (the `python` command skips " + "this script's cpu-extra dependency so only the GPU build installs; " + "onnxruntime and onnxruntime-gpu must never be co-installed)." + ) + + +def _load_onnx_model(model_id): + """Load the onnx-asr model, preferring CUDA when available.""" + import onnx_asr + + providers = _preferred_onnx_providers() + warning = cuda_fallback_warning( + providers, shutil.which("nvidia-smi") is not None) + if warning: + print(warning, file=sys.stderr) + if providers: + try: + return onnx_asr.load_model(model_id, providers=providers) + except TypeError: + pass # older onnx-asr without a providers kwarg + return onnx_asr.load_model(model_id) + + +def _result_scores(result): + """Best-effort per-token scores off an onnx-asr result, else None.""" + for attr in ("logprobs", "scores", "confidences"): + scores = getattr(result, attr, None) + if scores is not None: + return list(scores) + return None + + +def transcribe_onnx(media, model_id, window=CHUNK_WINDOW_S, + overlap=CHUNK_OVERLAP_S): + """Run onnx-asr over fixed windows and return (full_text, tokens, duration). + + Every window is extracted to a 16 kHz mono wav via ffmpeg (so video + containers work exactly like the mlx lane), recognized with timestamps, + converted to parakeet-shaped token dicts, offset by the window start, and + merged at overlap midpoints. Imports are local so the pure helpers stay + importable without the onnx-asr dependency. + """ + duration = probe_duration(media) + chunks = plan_chunks(duration, window=window, overlap=overlap) + model = _load_onnx_model(model_id) + + per_chunk = [] + with tempfile.TemporaryDirectory(prefix="mc-transcribe-") as tmp: + for i, (start, length) in enumerate(chunks): + print( + f"chunk {i + 1}/{len(chunks)}: {start:.2f}s +{length:.2f}s", + file=sys.stderr, + ) + wav = Path(tmp) / f"chunk{i:04d}.wav" + extract_chunk(media, start, length, wav) + result = model.with_timestamps().recognize(str(wav)) + tokens = onnx_tokens_to_parakeet( + list(result.tokens), + list(result.timestamps), + logprobs=_result_scores(result), + clamp=length, + ) + per_chunk.append((start, start + length, offset_tokens(tokens, start))) + + merged = merge_chunk_tokens(per_chunk) + text = "".join(t["text"] for t in merged).strip() + return text, merged, duration + + def main(argv=None): parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) parser.add_argument("media", help="path to the media file (audio or video)") parser.add_argument("-o", "--output", required=True, help="path to write the words.json transcript") parser.add_argument("--provider", default=DEFAULT_PROVIDER, - help=f"transcription provider (default: {DEFAULT_PROVIDER})") - parser.add_argument("--model", default=DEFAULT_MODEL, - help=f"model id (default: {DEFAULT_MODEL})") + help="transcription provider: auto (default; " + "parakeet-mlx on macOS Apple Silicon, onnx-asr " + "elsewhere), parakeet-mlx, or onnx-asr") + parser.add_argument("--model", default=None, + help="model id (default per provider: " + f"{DEFAULT_MODELS[PROVIDER_MLX]} for parakeet-mlx, " + f"{DEFAULT_MODELS[PROVIDER_ONNX]} for onnx-asr)") args = parser.parse_args(argv) - if args.provider != DEFAULT_PROVIDER: + provider = args.provider + if provider == PROVIDER_AUTO: + provider = default_provider() + print(f"provider resolved: {provider} (auto)", file=sys.stderr) + + if provider not in IMPLEMENTED_PROVIDERS: print( - f"provider {args.provider} not implemented; parakeet-mlx is the " - "default. See the [transcription] switch in the studio config.", + f"provider {provider} not implemented; implemented lanes are " + f"{', '.join(IMPLEMENTED_PROVIDERS)} (auto picks per platform). " + "See the [transcription] switch in the studio config.", file=sys.stderr, ) return 3 @@ -177,17 +641,29 @@ def main(argv=None): return 2 output = Path(args.output) + model_id = args.model or DEFAULT_MODELS[provider] try: - text, tokens, duration = transcribe_parakeet(media, args.model) + if provider == PROVIDER_MLX: + text, tokens, duration = transcribe_parakeet(media, model_id) + else: + text, tokens, duration = transcribe_onnx(media, model_id) + except ImportError as exc: + print( + f"error: provider {provider} dependencies unavailable on this " + f"platform: {exc}. Use --provider auto to pick the platform " + "default lane.", + file=sys.stderr, + ) + return 1 except Exception as exc: # transcription failure print(f"error: transcription failed: {exc}", file=sys.stderr) return 1 words = normalize_words(tokens, duration) payload = { - "provider": DEFAULT_PROVIDER, - "model": args.model, + "provider": provider, + "model": model_id, "media": args.media, "duration": round(float(duration), 2), "text": text.strip(), @@ -195,11 +671,12 @@ def main(argv=None): } output.parent.mkdir(parents=True, exist_ok=True) - output.write_text(json.dumps(payload, indent=2, ensure_ascii=False)) + output.write_text(json.dumps(payload, indent=2, ensure_ascii=False), + encoding="utf-8") print(json.dumps({ - "provider": DEFAULT_PROVIDER, - "model": args.model, + "provider": provider, + "model": model_id, "media": args.media, "duration": payload["duration"], "words": len(words), diff --git a/skills/mc-ograf/SKILL.md b/skills/mc-ograf/SKILL.md index 8d12e9f..40b0c67 100644 --- a/skills/mc-ograf/SKILL.md +++ b/skills/mc-ograf/SKILL.md @@ -30,7 +30,7 @@ Generate with `uv run {skill-root}/scripts/scaffold_ograf.py` rather than hand-w ## Verify before handoff -Run `uv run {skill-root}/scripts/verify_ograf.py `, passing the same `--width`, `--height`, and `--duration` used at scaffold time (defaults 1920x1080, 10000ms): it serves the folder, simulates exactly what Resolve does (register class once, instantiate, `load({renderType:"nonrealtime"})`, `goToTime` across the timeline), saves a transparent screenshot for eyeball review, and fails on any console error or empty DOM render. A skipped run is not a verified package. Fix and re-run until clean. At handoff, point the user at `{skill-root}/references/resolve-workflow.md` for import steps, and remind them: serve `preview.html` over HTTP, never open it from disk. +Run `uv run {skill-root}/scripts/verify_ograf.py `, passing the same `--width`, `--height`, and `--duration` used at scaffold time (defaults 1920x1080, 10000ms): it serves the folder, simulates exactly what Resolve does (register class once, instantiate, `load({renderType:"nonrealtime"})`, `goToTime` across the timeline), saves a transparent screenshot for eyeball review, and fails on any console error or empty DOM render. A skipped run is not a verified package. Fix and re-run until clean. The manual verification steps it prints are per-OS (open on macOS, start on Windows, xdg-open on Linux, `uv run python -m http.server 8771`), and when run from a human terminal with a `preview.html` present it serves the package and opens the preview in the default browser itself (tty-gated; agent runs are unaffected). At handoff, point the user at `{skill-root}/references/resolve-workflow.md` for import steps, and remind them: serve `preview.html` over HTTP, never open it from disk. ## Non-negotiable standards diff --git a/skills/mc-ograf/references/resolve-workflow.md b/skills/mc-ograf/references/resolve-workflow.md index 3595344..7bc194b 100644 --- a/skills/mc-ograf/references/resolve-workflow.md +++ b/skills/mc-ograf/references/resolve-workflow.md @@ -28,4 +28,22 @@ Run `uv run scripts/verify_ograf.py ` first — it reproduces #2 an ## Previewing locally before Resolve -Serve the folder and open `preview.html` over HTTP — `python3 -m http.server` in the package folder, then open `localhost:/preview.html`. **Never double-click `preview.html` (`file://`)**: the browser blocks its ES-module import and inlined data-URL assets, so the graphic silently fails while the controls/checkerboard still show ("Some content has been disabled"). This is a browser preview limit only; Resolve's renderer is unaffected. +Serve the folder and open `preview.html` over HTTP: `uv run python -m http.server 8771` in the package folder, then open `localhost:/preview.html` (verify_ograf.py prints per-OS steps and, from a human terminal, serves and opens the preview itself). **Never double-click `preview.html` (`file://`)**: the browser blocks its ES-module import and inlined data-URL assets, so the graphic silently fails while the controls/checkerboard still show ("Some content has been disabled"). This is a browser preview limit only; Resolve's renderer is unaffected. + +## Scripted import on free Resolve (Fusion Scripts menu) + +Resolve's external scripting API is Studio-only through Resolve 21; the free edition only executes scripts launched from inside the app (Console or Workspace > Scripts). To run the pipeline's scripted timeline import (resolve_import.py, once implemented) on free Resolve, copy the script into the Fusion Scripts folder and launch it from Workspace > Scripts: + +- macOS: `~/Library/Application Support/Blackmagic Design/DaVinci Resolve/Fusion/Scripts/Utility/` +- Windows: `%APPDATA%\Blackmagic Design\DaVinci Resolve\Support\Fusion\Scripts\Utility\` +- Linux: `~/.local/share/DaVinciResolve/Fusion/Scripts/` + +This upgrades the free lane from manual FCPXML import to native scripted import with zero dependencies. + +## Linux free-edition codec caveat + +The Linux free edition cannot decode or encode H.264 or H.265 and has no AAC at all. The FCPXML timeline imports fine, but mp4/AAC media is undecodable there; transcode sources to ProRes or DNxHR first, or use Resolve Studio. + +## Studio power lane: Resolve MCP (opt-in) + +Studio users who want conversational post-import work (timeline surgery, Text+ titles, markers, render-queue automation) can opt into the community [samuelgursky/davinci-resolve-mcp](https://github.com/samuelgursky/davinci-resolve-mcp) server (MIT, macOS/Windows/Linux; Studio-only, because it uses the external scripting API). Manticore keeps doing media import and timeline construction itself through the FCPXML lane and never requires an MCP server; record an already-running server via mc-setup's `[mcp]` step if you want skills to use it. diff --git a/skills/mc-ograf/scripts/tests/test-verify_ograf.py b/skills/mc-ograf/scripts/tests/test-verify_ograf.py index 5d95acc..aafa15e 100644 --- a/skills/mc-ograf/scripts/tests/test-verify_ograf.py +++ b/skills/mc-ograf/scripts/tests/test-verify_ograf.py @@ -3,11 +3,16 @@ # requires-python = ">=3.11" # /// """Tests for verify_ograf.py — the deterministic, browser-free parts: manifest -discovery, usage/structure errors, and the graceful no-headless fallback. +discovery, usage/structure errors, the per-OS manual-step strings, and the +graceful no-headless fallback. The full headless render check requires Playwright + a browser and is exercised -by running the script directly on a package; it is not unit-tested here.""" +by running the script directly on a package; it is not unit-tested here. The +interactive serve-and-open branch is tty-gated and preview.html-gated, so it +never fires under the test harness; only its gating is asserted.""" +import contextlib import importlib.util +import io import json import subprocess import sys @@ -64,6 +69,51 @@ def test_find_manifest_exits_when_absent(self): verify.find_manifest(empty) +class TestManualSteps(unittest.TestCase): + def test_steps_per_os(self): + pkg = Path("/x/pkg") + expected_open = {"Darwin": "open http://", + "Windows": "start http://", + "Linux": "xdg-open http://"} + for system, needle in expected_open.items(): + steps = verify.manual_verify_steps(pkg, system=system) + joined = "\n".join(steps) + self.assertIn(needle, joined, system) + # No shell chaining and no bare python3: both are POSIX-shaped. + self.assertNotIn("&&", joined, system) + self.assertNotIn("python3", joined, system) + self.assertIn("uv run python -m http.server 8771", joined, system) + self.assertIn(str(pkg), joined, system) + + def test_windows_cd_crosses_drives(self): + steps = verify.manual_verify_steps(Path("/x/pkg"), system="Windows") + self.assertIn("cd /d", "\n".join(steps)) + + def test_default_system_is_this_machine(self): + a = verify.manual_verify_steps(Path("/x/pkg")) + b = verify.manual_verify_steps(Path("/x/pkg"), + system=verify.platform.system()) + self.assertEqual(a, b) + + def test_manual_steps_prints_json_and_never_blocks_without_tty(self): + with tempfile.TemporaryDirectory() as tmp: + pkg = make_pkg(tmp) # no preview.html and no tty: both gates hold + buf = io.StringIO() + with contextlib.redirect_stdout(buf): + verify.manual_steps(pkg, "Playwright not installed") + payload = json.loads(buf.getvalue()) + self.assertEqual(payload["status"], "skipped-no-headless") + self.assertEqual(payload["reason"], "Playwright not installed") + self.assertEqual(payload["manual_verify"], + verify.manual_verify_steps(pkg)) + + def test_open_preview_noop_without_preview_html(self): + with tempfile.TemporaryDirectory() as tmp: + pkg = make_pkg(tmp) + # Must return immediately (no server, no browser, no input()). + self.assertIsNone(verify.open_preview_if_interactive(pkg)) + + class TestVerifyCli(unittest.TestCase): def test_missing_main_fails_usage(self): with tempfile.TemporaryDirectory() as tmp: diff --git a/skills/mc-ograf/scripts/verify_ograf.py b/skills/mc-ograf/scripts/verify_ograf.py index c048da3..580c346 100644 --- a/skills/mc-ograf/scripts/verify_ograf.py +++ b/skills/mc-ograf/scripts/verify_ograf.py @@ -13,7 +13,12 @@ 2 bad usage / package not found 3 could not run headless (Playwright missing or no browser) -> manual steps printed -Playwright is an OPTIONAL dependency. Without it, this prints how to verify by hand. +Playwright is an OPTIONAL dependency. Without it, this prints how to verify by +hand: per-OS instructions (open/start/xdg-open, no shell chaining), and when a +human terminal is attached AND the package has a preview.html, it also serves +the package in-process and opens the preview in the default browser +(webbrowser.open) so the manual check starts already running. Agent/CI runs +(no tty) never block. Stdlib only except for the optional Playwright import. """ # /// script @@ -26,11 +31,13 @@ import argparse import http.server import json +import platform import socket import socketserver import sys import tempfile import threading +import webbrowser from pathlib import Path VERIFY_HTML = "_ograf_verify.html" @@ -67,20 +74,70 @@ def log_message(self, *a): return httpd, port +def manual_verify_steps(pkg: Path, system: str | None = None) -> list[str]: + """Per-OS manual verification steps (no shell chaining, no bare python3). + + system defaults to this machine (platform.system()); tests pass it + explicitly to assert every OS variant from any OS.""" + system = system or platform.system() + url = "http://localhost:8771/preview.html" + if system == "Windows": + cd_cmd = f'cd /d "{pkg}"' + open_cmd = f"start {url}" + elif system == "Darwin": + cd_cmd = f'cd "{pkg}"' + open_cmd = f"open {url}" + else: + cd_cmd = f'cd "{pkg}"' + open_cmd = f"xdg-open {url}" + return [ + f"In a terminal, change into the package: {cd_cmd}", + "Serve it: uv run python -m http.server 8771", + f"Open the preview in a browser: {open_cmd}", + "Scrub the slider end-to-end; the graphic must animate in, hold, and out.", + "Open the browser console — there must be ZERO errors.", + "The checkerboard must show through (transparency).", + ] + + +def open_preview_if_interactive(pkg: Path) -> None: + """When a human terminal is attached, serve the package in-process and + open preview.html in the default browser (webbrowser.open picks the + right opener on every OS), then hold the server until Enter. + + A no-op when there is no tty (agent/CI runs must never block), when the + package has no preview.html, or on any failure.""" + if not (pkg / "preview.html").is_file(): + return + try: + if not (sys.stdin.isatty() and sys.stderr.isatty()): + return + httpd, port = serve(pkg) + url = f"http://127.0.0.1:{port}/preview.html" + try: + if not webbrowser.open(url): + return + print(f"preview served at {url}; press Enter to stop the server " + "when done...", file=sys.stderr) + try: + input() + except EOFError: + pass + finally: + httpd.shutdown() + except Exception: + return + + def manual_steps(pkg: Path, msg: str): print(json.dumps({ "ok": None, "status": "skipped-no-headless", "reason": msg, - "manual_verify": [ - f"cd {pkg} && python3 -m http.server 8771", - "open http://localhost:8771/preview.html", - "Scrub the slider end-to-end; the graphic must animate in, hold, and out.", - "Open the browser console — there must be ZERO errors.", - "The checkerboard must show through (transparency).", - ], + "manual_verify": manual_verify_steps(pkg), "enable_headless": "install the 'playwright' package, then run: playwright install chromium", }, indent=2)) + open_preview_if_interactive(pkg) def main(): diff --git a/skills/mc-package/SKILL.md b/skills/mc-package/SKILL.md index af92a62..e1a8f0c 100644 --- a/skills/mc-package/SKILL.md +++ b/skills/mc-package/SKILL.md @@ -5,7 +5,7 @@ description: Produce title+thumbnail packages, description, CTA metadata, and ch # mc-package -Packaging pays off the promise approved at gate 1; it is not invented fresh here. Read `references/cta-placement.md` in full before writing the description, the pinned comment, or the end-screen guidance (the file is duplicated from mc-beats; keep both copies identical). Two flows: the VOD flow (steps 1 to 9) and the live-event flow for scheduled broadcasts (see Live-event mode below). +Packaging pays off the promise approved at gate 1; it is not invented fresh here. Read `references/cta-placement.md` in full before writing the description, the pinned comment, or the end-screen guidance (the file is duplicated from mc-beats; keep both copies identical). Two flows: the VOD flow (steps 1 to 10) and the live-event flow for scheduled broadcasts (see Live-event mode below). ## Steps @@ -27,7 +27,8 @@ Packaging pays off the promise approved at gate 1; it is not invented fresh here 6. Description: the first 2 lines carry the hook and the search terms (they show before the fold), and when the video has a conversion CTA its link goes there too (description-top is half of its click surface, per the reference). Then the CTA lines drawn from `[cta]` items in priority order (imperative plus benefit, 7 words or fewer of ask copy per item); then the creator's `[owner]` `links`, in order; then the chapters block. Copy matches the lane: never live framing on a VOD ("enjoying the stream?", "link in chat" are wrong on a replay; use "comment below", "link in the description", schedule-tied subscribe framing), and livestream-vod projects get replay framing throughout. 7. Pinned comment and end screen, to `packaging/cta.md`: a paste-ready pinned-comment suggestion pointing at the same next step as the description-top link (identical URL; end screen, cards, pinned comment, and description-top all point at one next step), plus end-screen guidance for upload: the final 10 to 20 seconds are the outro runway, a 2-element layout (one watch-next plus one subscribe) beats cluttered screens, the watch-next target must be topically continuous, and the narration must verbally bridge to it. Check the script or transcript for that verbal bridge and flag loudly when it is missing. 8. Chapters: from the edited transcript's beat boundaries; first chapter 0:00, honest labels, no keyword stuffing. Dual-timeline rule: whenever `cut/edl.json` exists, chapters are a dual-timeline deliverable. `packaging/chapters.md` opens with the paste-ready block in edited (published) timecodes, followed by a clearly labeled table adding the original-source timecode per chapter (for finding the moment in the raw footage or VOD). The original column comes from this skill's own remap utility (a duplicate of the cut stage's, per the script-duplication convention), run against `cut/edl.json` (a project file): write the edited-timecode chapter list to `packaging/work/chapters-edited.md`, run `uv run {skill-root}/scripts/remap_timecode.py cut/edl.json --direction clean-to-orig --chapters packaging/work/chapters-edited.md -o packaging/work/chapters-orig.md`, and pair the two files line by line into the table. On a multi-source EDL, add a source column: use the script's `--events` mode instead (it records `source` on each remapped entry). If the cut does not exist yet (early run), chapters are pending: skip this step and the description's chapter block, and tell the creator to re-run mc-package after the cut to finish them. -9. Write `packaging/titles.md`, `packaging/description.md`, `packaging/cta.md`, and `packaging/chapters.md` (only when chapters were produced); update `artifacts` in `project.json`. If the project's stage is `package` and chapters are done, append `package` to `stages_done` and set `stage` to the next stage in the project's `stages`; on an early run, leave `stage` and `stages_done` untouched. +9. Captions and transcript (only when the cut exists, the same gate as chapters: `cut/edl.json` and `transcript/words.json` present). Run `uv run {skill-root}/scripts/captions.py cut/edl.json --words transcript/words.json --out-dir packaging/captions/`. Multi-source projects pass one `--words` per source words file (`transcript/.words.json`), binding explicitly when media fields differ: `--words raw/=transcript/.words.json`. The script emits `packaging/captions/final.srt`, `final.vtt`, and `transcript.md` for the EDITED timeline; a light filler/stutter cleanup runs by default on this derived rendition only (`transcript/words.json` is never modified). Offer the creator `--no-clean` if they want verbatim captions. Present `transcript.md` for a skim before calling the deliverable done. If the cut does not exist yet, skip like chapters and finish captions on the re-run. +10. Write `packaging/titles.md`, `packaging/description.md`, `packaging/cta.md`, `packaging/chapters.md`, and `packaging/captions/` (the last two only when the cut existed to produce them); update `artifacts` in `project.json`. If the project's stage is `package` and chapters are done, append `package` to `stages_done` and set `stage` to the next stage in the project's `stages`; on an early run, leave `stage` and `stages_done` untouched. ## The pick and blessed slots @@ -63,6 +64,7 @@ Locked anchors are non-negotiable within an episode; changing them is a series-l - The description's first 2 lines carry the hook, search terms, and the conversion link when one exists; CTA copy matches the lane (no live framing on a VOD). - Pinned comment, description-top, and end-screen guidance all point at the same next step. - Chapters are dual-timeline whenever an EDL exists; the original column came from this skill's remap_timecode.py run against `cut/edl.json`. +- Captions and transcript (`packaging/captions/final.srt`, `final.vtt`, `transcript.md`) were emitted whenever the cut exists; the cleanup pass touched only the caption rendition, never `transcript/words.json`. - A scheduled-broadcast thumbnail is never a plain brand card (the two-asset rule). - After a pick: exactly one blessed asset per slot in `packaging/final/`, recorded in `project.json` `artifacts`. - Run `uv run {skill-root}/scripts/lint_script.py --blacklist {brand-path}/blacklist.md` on titles.md, description.md, and cta.md. diff --git a/skills/mc-package/scripts/captions.py b/skills/mc-package/scripts/captions.py new file mode 100644 index 0000000..b1bd795 --- /dev/null +++ b/skills/mc-package/scripts/captions.py @@ -0,0 +1,406 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# /// +"""Emit SRT/VTT captions and a readable transcript for the EDITED timeline +from transcript/words.json and cut/edl.json. + +Usage: + uv run {skill-root}/scripts/captions.py \ + --words transcript/words.json --out-dir packaging/captions/ + uv run {skill-root}/scripts/captions.py \ + --words raw/a.mp4=transcript/a.words.json \ + --words raw/b.mp4=transcript/b.words.json \ + --out-dir packaging/captions/ [--no-clean] [--basename final] + +Contract: + edl.json is the cut stage's neutral edit description ({source, + source_duration, segments: [{source, start, end, ...}]}); segment start + and end are seconds against that segment's own source. Each --words file + is a transcribe.py words.json (verbatim word timestamps against ONE + source). --words may repeat; the form = binds a file to an + EDL segment source explicitly, a bare binds via the file's own + "media" field (exact match, then basename match, then the trivial match + when the EDL uses a single source and a single file was given). Every + source named by a segment must resolve to a words file. + + Each segment's words (word midpoint inside [start, end)) are remapped to + output-timeline times (segment offset + word time - segment start, span + clamped to the segment), so reordered and multi-source edits caption + correctly. Words are grouped into cues: new cue at segment boundaries, + after sentence-ending punctuation, at speech pauses (--pause-split, 0.6 s + default), and whenever text would no longer wrap into --max-lines (2) + lines of --max-line-chars (42), or the cue would exceed --max-cue-seconds + (7.0). Cues shorter than --min-cue-seconds (1.0) are extended toward the + next cue. + + A light cleanup pass runs by default on this derived rendition only + (words.json is never modified): standalone filler tokens (um, uh, ...) + are dropped and stutter repeats (repeated word, or a "th-" fragment + completed by the next word) are collapsed. --no-clean keeps the captions + verbatim. + +Output: + Writes /.srt, /.vtt, and + /transcript.md (timecoded paragraphs of the same rendition), + creating if needed, then prints a JSON summary to stdout. + +Exit codes: 0 ok, 1 failure (unreadable input, unmatched source, no words +on the timeline), 2 usage. + +STATUS: implemented (covered by scripts/tests/test-captions.py). +""" + +import argparse +import json +import re +import sys +from pathlib import Path + +FILLERS = frozenset({ + "um", "umm", "uh", "uhh", "uhm", "erm", "er", "hmm", "hm", "mm", "mhm", +}) +PUNCT_STRIP = re.compile(r"[^\w']+", re.UNICODE) +SENTENCE_END = re.compile(r"[.!?][\"')\]]*$") + + +def normalize_token(word): + """Lowercased word with punctuation stripped ('Alright,' -> 'alright').""" + return PUNCT_STRIP.sub("", str(word)).lower() + + +def clean_words(words, enabled=True): + """Filler drop + stutter collapse over word dicts (needs 'word' key). + + Returns (new_words, stats) with stats {'fillers_dropped', + 'repeats_collapsed'}. Collapsing merges the repeat into one word: text of + the later word, start of the earlier, end of the later. Never mutates + the input dicts.""" + stats = {"fillers_dropped": 0, "repeats_collapsed": 0} + if not enabled: + return list(words), stats + out = [] + for w in words: + norm = normalize_token(w["word"]) + if norm in FILLERS: + stats["fillers_dropped"] += 1 + continue + if out: + prev = out[-1] + prev_norm = normalize_token(prev["word"]) + same_source = (prev.get("_source") == w.get("_source") + and prev.get("seg") == w.get("seg")) + repeat = norm != "" and norm == prev_norm + fragment = (str(prev["word"]).rstrip().endswith(("-", "—")) + and prev_norm != "" and norm.startswith(prev_norm) + and norm != prev_norm) + if same_source and (repeat or fragment): + merged = dict(w) + merged["start"] = prev["start"] + out[-1] = merged + stats["repeats_collapsed"] += 1 + continue + out.append(dict(w)) + return out, stats + + +def match_words_files(edl, entries): + """Resolve each EDL segment source to a parsed words payload. + + entries: [(explicit_source_or_None, payload_dict)]. Match order per + source: explicit binding, exact media string, media basename, then the + single-file single-source fallback. Returns {source: [word, ...]}; + raises ValueError when a source stays unmatched.""" + sources = [] + for seg in edl.get("segments", []): + if seg["source"] not in sources: + sources.append(seg["source"]) + resolved = {} + for src in sources: + payload = None + for explicit, data in entries: + if explicit == src: + payload = data + break + if payload is None: + for explicit, data in entries: + if explicit is None and str(data.get("media", "")) == src: + payload = data + break + if payload is None: + for explicit, data in entries: + if explicit is None and Path( + str(data.get("media", ""))).name == Path(src).name: + payload = data + break + if payload is None and len(sources) == 1 and len(entries) == 1: + payload = entries[0][1] + if payload is None: + raise ValueError( + f"no words file matches EDL source {src!r}; pass " + f"--words {src}=") + resolved[src] = payload.get("words", []) + return resolved + + +def assign_output_times(edl, words_by_source): + """Project each kept word onto the edited timeline. + + A word belongs to a segment when its midpoint falls in [start, end); + its span is clamped to the segment and offset onto the output timeline. + Returns timeline-ordered dicts {word, start, end, confidence, seg, + _source}.""" + out = [] + offset = 0.0 + for seg_index, seg in enumerate(edl["segments"]): + s, e = float(seg["start"]), float(seg["end"]) + for w in words_by_source.get(seg["source"], []): + mid = (float(w["start"]) + float(w["end"])) / 2.0 + if not (s <= mid < e): + continue + out.append({ + "word": str(w["word"]), + "start": round(offset + max(float(w["start"]), s) - s, 3), + "end": round(offset + min(float(w["end"]), e) - s, 3), + "confidence": w.get("confidence"), + "seg": seg_index, + "_source": seg["source"], + }) + offset += e - s + return out + + +def wrap_lines(text, max_chars): + """Greedy word wrap; a single word longer than max_chars overflows its + own line rather than being split.""" + lines = [] + current = "" + for token in text.split(): + candidate = token if not current else current + " " + token + if current and len(candidate) > max_chars: + lines.append(current) + current = token + else: + current = candidate + if current: + lines.append(current) + return lines + + +def group_cues(timed_words, max_line_chars=42, max_lines=2, + min_cue_seconds=1.0, max_cue_seconds=7.0, pause_split=0.6): + """Group timeline-ordered words into caption cues. + + Cue boundaries: segment change, sentence-ending punctuation, a pause of + pause_split seconds or more before the next word, text overflowing + max_lines lines of max_line_chars, or duration passing max_cue_seconds. + Short cues are stretched toward the next cue up to min_cue_seconds. + Returns [{start, end, lines, text, words}].""" + cues = [] + bucket = [] + + def flush(): + if not bucket: + return + text = " ".join(w["word"] for w in bucket).strip() + if text: + cues.append({ + "start": bucket[0]["start"], + "end": bucket[-1]["end"], + "lines": wrap_lines(text, max_line_chars), + "text": text, + "words": list(bucket), + }) + bucket.clear() + + for w in timed_words: + if bucket: + candidate = " ".join(x["word"] for x in bucket) + " " + w["word"] + overflow = len(wrap_lines(candidate, max_line_chars)) > max_lines + too_long = w["end"] - bucket[0]["start"] > max_cue_seconds + new_seg = w["seg"] != bucket[-1]["seg"] + paused = w["start"] - bucket[-1]["end"] >= pause_split + if overflow or too_long or new_seg or paused: + flush() + bucket.append(w) + if SENTENCE_END.search(w["word"].strip()): + flush() + flush() + + for i, cue in enumerate(cues): + if cue["end"] - cue["start"] < min_cue_seconds: + target = cue["start"] + min_cue_seconds + if i + 1 < len(cues): + target = min(target, cues[i + 1]["start"]) + cue["end"] = round(max(cue["end"], target), 3) + return cues + + +def format_srt_time(seconds): + """Seconds -> '00:00:01,280' (SRT comma milliseconds).""" + ms = max(0, round(seconds * 1000)) + h, rem = divmod(ms, 3_600_000) + m, rem = divmod(rem, 60_000) + s, ms = divmod(rem, 1000) + return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}" + + +def format_vtt_time(seconds): + """Seconds -> '00:00:01.280' (WebVTT dot milliseconds).""" + return format_srt_time(seconds).replace(",", ".") + + +def render_srt(cues): + blocks = [] + for i, cue in enumerate(cues, 1): + blocks.append(f"{i}\n{format_srt_time(cue['start'])} --> " + f"{format_srt_time(cue['end'])}\n" + + "\n".join(cue["lines"])) + return "\n\n".join(blocks) + "\n" + + +def render_vtt(cues): + blocks = ["WEBVTT"] + for cue in cues: + blocks.append(f"{format_vtt_time(cue['start'])} --> " + f"{format_vtt_time(cue['end'])}\n" + + "\n".join(cue["lines"])) + return "\n\n".join(blocks) + "\n" + + +def transcript_timecode(seconds): + """Seconds -> '[m:ss]' or '[h:mm:ss]' paragraph marker.""" + total = int(seconds) + h, rem = divmod(total, 3600) + m, s = divmod(rem, 60) + if h: + return f"[{h}:{m:02d}:{s:02d}]" + return f"[{m}:{s:02d}]" + + +def render_transcript(timed_words, paragraph_pause=1.5): + """Timecoded paragraphs of the edited timeline. New paragraph at a + segment change or a pause of paragraph_pause seconds or more.""" + paragraphs = [] + bucket = [] + for w in timed_words: + if bucket and (w["seg"] != bucket[-1]["seg"] + or w["start"] - bucket[-1]["end"] >= paragraph_pause): + paragraphs.append(bucket) + bucket = [] + bucket.append(w) + if bucket: + paragraphs.append(bucket) + out = ["# Transcript", ""] + for para in paragraphs: + text = " ".join(w["word"] for w in para).strip() + if not text: + continue + out.append(f"{transcript_timecode(para[0]['start'])} {text}") + out.append("") + return "\n".join(out).rstrip("\n") + "\n" + + +def parse_words_arg(value): + """'--words path' or '--words source=path' -> (source_or_None, path). + + Splits on the first '=' only when the left side names an EDL source + (contains no path separator ambiguity worth guessing beyond '=').""" + if "=" in value: + source, path = value.split("=", 1) + if source and path: + return source, path + return None, value + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("edl", help="path to cut/edl.json") + parser.add_argument("--words", action="append", required=True, + metavar="[SOURCE=]PATH", + help="words.json path, optionally bound to an EDL " + "segment source; repeatable") + parser.add_argument("--out-dir", required=True, + help="directory for the caption deliverables") + parser.add_argument("--basename", default="final", + help="basename for the .srt/.vtt pair (default final)") + parser.add_argument("--no-clean", action="store_true", + help="skip the filler/stutter cleanup pass") + parser.add_argument("--max-line-chars", type=int, default=42) + parser.add_argument("--max-lines", type=int, default=2) + parser.add_argument("--min-cue-seconds", type=float, default=1.0) + parser.add_argument("--max-cue-seconds", type=float, default=7.0) + parser.add_argument("--pause-split", type=float, default=0.6) + args = parser.parse_args(argv) + + try: + edl = json.loads(Path(args.edl).read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as e: + print(f"captions: cannot read {args.edl}: {e}", file=sys.stderr) + return 1 + if not edl.get("segments"): + print("captions: edl has no segments", file=sys.stderr) + return 1 + + entries = [] + for raw in args.words: + source, path = parse_words_arg(raw) + try: + data = json.loads(Path(path).read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError) as e: + print(f"captions: cannot read {path}: {e}", file=sys.stderr) + return 1 + entries.append((source, data)) + + try: + words_by_source = match_words_files(edl, entries) + except ValueError as e: + print(f"captions: {e}", file=sys.stderr) + return 1 + + timed = assign_output_times(edl, words_by_source) + if not timed: + print("captions: no words fall inside any EDL segment", + file=sys.stderr) + return 1 + cleaned, clean_stats = clean_words(timed, enabled=not args.no_clean) + cues = group_cues(cleaned, + max_line_chars=args.max_line_chars, + max_lines=args.max_lines, + min_cue_seconds=args.min_cue_seconds, + max_cue_seconds=args.max_cue_seconds, + pause_split=args.pause_split) + if not cues: + print("captions: cleanup removed every word; nothing to caption", + file=sys.stderr) + return 1 + + out_dir = Path(args.out_dir) + try: + out_dir.mkdir(parents=True, exist_ok=True) + srt_path = out_dir / f"{args.basename}.srt" + vtt_path = out_dir / f"{args.basename}.vtt" + md_path = out_dir / "transcript.md" + srt_path.write_text(render_srt(cues), encoding="utf-8") + vtt_path.write_text(render_vtt(cues), encoding="utf-8") + md_path.write_text(render_transcript(cleaned), encoding="utf-8") + except OSError as e: + print(f"captions: cannot write outputs: {e}", file=sys.stderr) + return 1 + + print(json.dumps({ + "ok": True, + "cues": len(cues), + "words": len(cleaned), + "clean": not args.no_clean, + **clean_stats, + "duration": round(cues[-1]["end"], 3), + "srt": str(srt_path), + "vtt": str(vtt_path), + "transcript": str(md_path), + })) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/mc-package/scripts/tests/test-captions.py b/skills/mc-package/scripts/tests/test-captions.py new file mode 100644 index 0000000..410d911 --- /dev/null +++ b/skills/mc-package/scripts/tests/test-captions.py @@ -0,0 +1,389 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# /// +"""Tests for captions.py: word-to-output-timeline mapping over reordered and +multi-source EDLs, the filler/stutter cleanup pass, cue grouping and wrapping +rules, SRT/VTT/transcript rendering, and the CLI end to end over synthetic +words.json + edl.json fixtures. Pure logic, no media, no network.""" +import importlib.util +import json +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +SCRIPT = Path(__file__).resolve().parent.parent / "captions.py" + +spec = importlib.util.spec_from_file_location("captions", SCRIPT) +mod = importlib.util.module_from_spec(spec) +spec.loader.exec_module(mod) + + +def word(text, start, end, confidence=0.9): + return {"word": text, "start": start, "end": end, + "confidence": confidence} + + +def words_payload(media, words): + return {"provider": "parakeet-mlx", "model": "m", "media": media, + "duration": 60.0, "text": " ".join(w["word"] for w in words), + "words": words} + + +def canned_words_a(): + # Source raw/a.mp4: intro sentence at 1-3 s, filler + point at 10-13 s. + return [ + word("Alright,", 1.0, 1.4), + word("welcome", 1.5, 1.9), + word("back.", 2.0, 2.4), + word("Um,", 10.0, 10.2), + word("the", 10.3, 10.5), + word("the", 10.6, 10.8), + word("point", 10.9, 11.3), + word("is", 11.4, 11.6), + word("simple.", 11.7, 12.3), + ] + + +def canned_words_b(): + return [ + word("Demo", 5.0, 5.4), + word("time.", 5.5, 5.9), + ] + + +def multi_source_edl(): + # Timeline: B[4.5,6.5) at 0-2, then A[9.5,13) at 2-5.5 (reordered join). + return { + "source": "raw/a.mp4", + "source_duration": 60.0, + "fade_ms": 30, + "pad_ms": 60, + "segments": [ + {"source": "raw/b.mp4", "start": 4.5, "end": 6.5}, + {"source": "raw/a.mp4", "start": 9.5, "end": 13.0}, + ], + } + + +def single_source_edl(): + # Reorder within one source: A[9.5,13) first, then A[0.5,3) after it. + return { + "source": "raw/a.mp4", + "source_duration": 60.0, + "segments": [ + {"source": "raw/a.mp4", "start": 9.5, "end": 13.0}, + {"source": "raw/a.mp4", "start": 0.5, "end": 3.0}, + ], + } + + +class TestCleanWords(unittest.TestCase): + def test_fillers_dropped_and_repeats_collapsed(self): + words = [ + {"word": "Um,", "start": 0.0, "end": 0.2}, + {"word": "the", "start": 0.3, "end": 0.5}, + {"word": "the", "start": 0.6, "end": 0.8}, + {"word": "point", "start": 0.9, "end": 1.2}, + {"word": "uh", "start": 1.3, "end": 1.4}, + {"word": "stands.", "start": 1.5, "end": 2.0}, + ] + out, stats = mod.clean_words(words) + self.assertEqual([w["word"] for w in out], + ["the", "point", "stands."]) + self.assertEqual(stats, {"fillers_dropped": 2, + "repeats_collapsed": 1}) + # Collapsed word keeps the first start and the last end. + self.assertEqual(out[0]["start"], 0.3) + self.assertEqual(out[0]["end"], 0.8) + # Input untouched. + self.assertEqual(words[1]["start"], 0.3) + self.assertEqual(len(words), 6) + + def test_hyphen_fragment_collapses_into_completion(self): + words = [ + {"word": "th-", "start": 0.0, "end": 0.2}, + {"word": "the", "start": 0.3, "end": 0.5}, + {"word": "plan", "start": 0.6, "end": 1.0}, + ] + out, stats = mod.clean_words(words) + self.assertEqual([w["word"] for w in out], ["the", "plan"]) + self.assertEqual(out[0]["start"], 0.0) + self.assertEqual(stats["repeats_collapsed"], 1) + + def test_disabled_keeps_everything(self): + words = [ + {"word": "um", "start": 0.0, "end": 0.2}, + {"word": "the", "start": 0.3, "end": 0.5}, + {"word": "the", "start": 0.6, "end": 0.8}, + ] + out, stats = mod.clean_words(words, enabled=False) + self.assertEqual([w["word"] for w in out], ["um", "the", "the"]) + self.assertEqual(stats, {"fillers_dropped": 0, + "repeats_collapsed": 0}) + + def test_no_collapse_across_segments(self): + words = [ + {"word": "go", "start": 0.9, "end": 1.0, "seg": 0}, + {"word": "Go", "start": 1.0, "end": 1.2, "seg": 1}, + ] + out, stats = mod.clean_words(words) + self.assertEqual(len(out), 2) + self.assertEqual(stats["repeats_collapsed"], 0) + + +class TestAssignOutputTimes(unittest.TestCase): + def test_multi_source_join_maps_by_segment_source(self): + timed = mod.assign_output_times(multi_source_edl(), { + "raw/a.mp4": canned_words_a(), + "raw/b.mp4": canned_words_b(), + }) + texts = [w["word"] for w in timed] + self.assertEqual(texts, ["Demo", "time.", "Um,", "the", "the", + "point", "is", "simple."]) + # B[4.5,6.5) at offset 0: "Demo" 5.0-5.4 -> 0.5-0.9. + self.assertEqual((timed[0]["start"], timed[0]["end"]), (0.5, 0.9)) + self.assertEqual(timed[0]["_source"], "raw/b.mp4") + self.assertEqual(timed[0]["seg"], 0) + # A[9.5,13) at offset 2.0: "point" 10.9-11.3 -> 3.4-3.8. + self.assertEqual((timed[5]["start"], timed[5]["end"]), (3.4, 3.8)) + self.assertEqual(timed[5]["seg"], 1) + + def test_reorder_within_one_source(self): + timed = mod.assign_output_times(single_source_edl(), + {"raw/a.mp4": canned_words_a()}) + texts = [w["word"] for w in timed] + # Later material plays first, intro follows. + self.assertEqual(texts, ["Um,", "the", "the", "point", "is", + "simple.", "Alright,", "welcome", "back."]) + # "Alright," 1.0-1.4 in A[0.5,3) at offset 3.5 -> 4.0-4.4. + self.assertEqual((timed[6]["start"], timed[6]["end"]), (4.0, 4.4)) + + def test_midpoint_selection_and_span_clamp(self): + edl = {"segments": [{"source": "s", "start": 1.0, "end": 2.0}]} + timed = mod.assign_output_times(edl, {"s": [ + word("out", 0.0, 0.9), # midpoint 0.45, excluded + word("edge", 0.8, 1.4), # midpoint 1.1, kept, start clamps + word("gone", 1.9, 2.5), # midpoint 2.2, excluded + ]}) + self.assertEqual([w["word"] for w in timed], ["edge"]) + self.assertEqual((timed[0]["start"], timed[0]["end"]), (0.0, 0.4)) + + +class TestMatchWordsFiles(unittest.TestCase): + def test_explicit_media_and_basename_matching(self): + edl = multi_source_edl() + a = words_payload("raw/a.mp4", canned_words_a()) + b = words_payload("/abs/elsewhere/b.mp4", canned_words_b()) + resolved = mod.match_words_files(edl, [(None, a), (None, b)]) + self.assertEqual(resolved["raw/a.mp4"], a["words"]) # exact media + self.assertEqual(resolved["raw/b.mp4"], b["words"]) # basename + explicit = mod.match_words_files(edl, [("raw/a.mp4", b), + ("raw/b.mp4", a)]) + self.assertEqual(explicit["raw/a.mp4"], b["words"]) # explicit wins + + def test_single_file_single_source_fallback(self): + edl = {"segments": [{"source": "raw/take.mov", + "start": 0.0, "end": 5.0}]} + payload = words_payload("something-else.wav", canned_words_a()) + resolved = mod.match_words_files(edl, [(None, payload)]) + self.assertEqual(resolved["raw/take.mov"], payload["words"]) + + def test_unmatched_source_raises(self): + with self.assertRaises(ValueError): + mod.match_words_files( + multi_source_edl(), + [(None, words_payload("raw/a.mp4", canned_words_a()))]) + + +class TestGroupCues(unittest.TestCase): + def tw(self, text, start, end, seg=0): + return {"word": text, "start": start, "end": end, "seg": seg} + + def test_sentence_end_and_pause_split(self): + words = [ + self.tw("Hello.", 0.0, 0.5), + self.tw("Next", 0.6, 0.9), + self.tw("bit", 2.0, 2.4), # 1.1 s pause before this word + ] + cues = mod.group_cues(words, pause_split=0.6) + self.assertEqual([c["text"] for c in cues], + ["Hello.", "Next", "bit"]) + + def test_segment_boundary_splits(self): + words = [self.tw("one", 0.0, 0.4, seg=0), + self.tw("two", 0.4, 0.8, seg=1)] + cues = mod.group_cues(words) + self.assertEqual(len(cues), 2) + + def test_line_capacity_split_and_wrap(self): + words = [self.tw("x" * 20, i * 0.4, i * 0.4 + 0.3) + for i in range(5)] + cues = mod.group_cues(words, max_line_chars=42, max_lines=2) + # 5 x 20 chars cannot wrap into 2 lines of 42; first cue holds 4. + self.assertEqual(len(cues), 2) + self.assertEqual(len(cues[0]["words"]), 4) + self.assertEqual(cues[0]["lines"], ["x" * 20 + " " + "x" * 20] * 2) + self.assertTrue(all(len(line) <= 42 for line in cues[0]["lines"])) + + def test_max_duration_split(self): + words = [self.tw("a", 0.0, 0.2), self.tw("b", 7.5, 7.8)] + cues = mod.group_cues(words, max_cue_seconds=7.0, pause_split=99) + self.assertEqual(len(cues), 2) + + def test_min_duration_extends_but_not_past_next_cue(self): + words = [self.tw("Hi.", 0.0, 0.3), self.tw("There.", 0.8, 1.1)] + cues = mod.group_cues(words, min_cue_seconds=1.0) + self.assertEqual(cues[0]["end"], 0.8) # clamped to next cue start + self.assertEqual(cues[1]["end"], 1.8) # free to extend + + def test_wrap_lines_keeps_long_word_whole(self): + self.assertEqual(mod.wrap_lines("a bb ccc", 4), ["a bb", "ccc"]) + self.assertEqual(mod.wrap_lines("supercalifragilistic", 5), + ["supercalifragilistic"]) + + +class TestRendering(unittest.TestCase): + def test_time_formats(self): + self.assertEqual(mod.format_srt_time(1.28), "00:00:01,280") + self.assertEqual(mod.format_srt_time(3661.5), "01:01:01,500") + self.assertEqual(mod.format_vtt_time(1.28), "00:00:01.280") + self.assertEqual(mod.format_srt_time(-0.5), "00:00:00,000") + + def test_srt_and_vtt_blocks(self): + cues = [{"start": 0.0, "end": 1.5, "lines": ["Hello there."], + "text": "Hello there.", "words": []}, + {"start": 2.0, "end": 4.0, "lines": ["Line one", "line two"], + "text": "Line one line two", "words": []}] + srt = mod.render_srt(cues) + self.assertIn("1\n00:00:00,000 --> 00:00:01,500\nHello there.", srt) + self.assertIn("2\n00:00:02,000 --> 00:00:04,000\n" + "Line one\nline two", srt) + vtt = mod.render_vtt(cues) + self.assertTrue(vtt.startswith("WEBVTT\n\n")) + self.assertIn("00:00:02.000 --> 00:00:04.000\nLine one\nline two", + vtt) + self.assertNotIn(",", vtt.splitlines()[2]) + + def test_transcript_paragraphs(self): + words = [ + {"word": "First", "start": 0.0, "end": 0.4, "seg": 0}, + {"word": "part.", "start": 0.5, "end": 0.9, "seg": 0}, + {"word": "Second", "start": 65.0, "end": 65.4, "seg": 1}, + {"word": "part.", "start": 65.5, "end": 65.9, "seg": 1}, + ] + md = mod.render_transcript(words) + self.assertIn("# Transcript", md) + self.assertIn("[0:00] First part.", md) + self.assertIn("[1:05] Second part.", md) + + +def run_cli(args): + return subprocess.run([sys.executable, str(SCRIPT), *args], + capture_output=True, text=True) + + +class TestCli(unittest.TestCase): + def write_fixtures(self, tmp): + tmp = Path(tmp) + edl = tmp / "edl.json" + edl.write_text(json.dumps(multi_source_edl())) + wa = tmp / "a.words.json" + wa.write_text(json.dumps(words_payload("raw/a.mp4", + canned_words_a()))) + wb = tmp / "b.words.json" + wb.write_text(json.dumps(words_payload("raw/b.mp4", + canned_words_b()))) + return edl, wa, wb + + def test_end_to_end_multi_source_with_cleanup(self): + with tempfile.TemporaryDirectory() as tmp: + edl, wa, wb = self.write_fixtures(tmp) + out_dir = Path(tmp) / "captions" + r = run_cli([str(edl), "--words", str(wa), "--words", str(wb), + "--out-dir", str(out_dir)]) + self.assertEqual(r.returncode, 0, r.stderr) + summary = json.loads(r.stdout) + self.assertTrue(summary["ok"]) + self.assertTrue(summary["clean"]) + self.assertEqual(summary["fillers_dropped"], 1) + self.assertEqual(summary["repeats_collapsed"], 1) + srt = (out_dir / "final.srt").read_text() + vtt = (out_dir / "final.vtt").read_text() + md = (out_dir / "transcript.md").read_text() + # B material captions first, cleaned A material follows. + self.assertIn("Demo time.", srt) + self.assertIn("the point is simple.", srt) + self.assertNotIn("Um,", srt) + self.assertNotIn("the the", srt) + self.assertLess(srt.index("Demo time."), + srt.index("the point is simple.")) + # "Demo" starts at 0.5 s on the edited timeline. + self.assertIn("00:00:00,500 -->", srt) + self.assertTrue(vtt.startswith("WEBVTT")) + self.assertIn("00:00:00.500 -->", vtt) + self.assertIn("[0:00] Demo time.", md) + + def test_no_clean_keeps_fillers(self): + with tempfile.TemporaryDirectory() as tmp: + edl, wa, wb = self.write_fixtures(tmp) + out_dir = Path(tmp) / "captions" + r = run_cli([str(edl), "--words", str(wa), "--words", str(wb), + "--out-dir", str(out_dir), "--no-clean"]) + self.assertEqual(r.returncode, 0, r.stderr) + summary = json.loads(r.stdout) + self.assertFalse(summary["clean"]) + self.assertEqual(summary["fillers_dropped"], 0) + srt = (out_dir / "final.srt").read_text() + self.assertIn("Um,", srt) + self.assertIn("the the", srt) + + def test_explicit_source_binding_and_basename(self): + with tempfile.TemporaryDirectory() as tmp: + edl, wa, wb = self.write_fixtures(tmp) + out_dir = Path(tmp) / "captions" + r = run_cli([str(edl), "--words", f"raw/a.mp4={wa}", + "--words", f"raw/b.mp4={wb}", + "--out-dir", str(out_dir), "--basename", "edited"]) + self.assertEqual(r.returncode, 0, r.stderr) + self.assertTrue((out_dir / "edited.srt").exists()) + self.assertTrue((out_dir / "edited.vtt").exists()) + self.assertTrue((out_dir / "transcript.md").exists()) + + def test_unmatched_source_exits_1(self): + with tempfile.TemporaryDirectory() as tmp: + edl, wa, _wb = self.write_fixtures(tmp) + r = run_cli([str(edl), "--words", str(wa), + "--out-dir", str(Path(tmp) / "captions")]) + self.assertEqual(r.returncode, 1) + self.assertIn("raw/b.mp4", r.stderr) + + def test_missing_required_args_exit_2(self): + r = run_cli(["edl.json"]) + self.assertEqual(r.returncode, 2) + r = run_cli(["edl.json", "--words", "w.json"]) + self.assertEqual(r.returncode, 2) + + def test_missing_edl_exits_1(self): + with tempfile.TemporaryDirectory() as tmp: + r = run_cli(["/nonexistent/edl.json", "--words", + str(Path(tmp) / "w.json"), "--out-dir", tmp]) + self.assertEqual(r.returncode, 1) + + def test_no_words_on_timeline_exits_1(self): + with tempfile.TemporaryDirectory() as tmp: + edl = Path(tmp) / "edl.json" + edl.write_text(json.dumps({"segments": [ + {"source": "raw/a.mp4", "start": 50.0, "end": 55.0}]})) + wa = Path(tmp) / "a.words.json" + wa.write_text(json.dumps(words_payload("raw/a.mp4", + canned_words_a()))) + r = run_cli([str(edl), "--words", str(wa), "--out-dir", tmp]) + self.assertEqual(r.returncode, 1) + self.assertIn("no words", r.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/skills/mc-pipeline/PIPELINE.md b/skills/mc-pipeline/PIPELINE.md index 2807f3c..6dc25a3 100644 --- a/skills/mc-pipeline/PIPELINE.md +++ b/skills/mc-pipeline/PIPELINE.md @@ -24,8 +24,8 @@ Format profiles select a subset of these stages (see the `stages:` frontmatter o | 7 | beats | mc-beats | gate 3: beats | `beats/beats.md` (the beat table), `beats/STORYBOARD.md` | | 8 | graphics | mc-graphics | | `graphics/` alpha MOVs + `graphics/HANDOFF.md`; on completion the router routes through mc-cut to re-render `renders/preview.mp4` with the overlays composited | | 9 | assets | mc-assets | | `assets/` + `assets/manifest.json` | -| 10 | package | mc-package | | `packaging/titles.md`, `packaging/thumbs/`, `packaging/description.md`, `packaging/chapters.md` | -| 11 | final | the creator, with an offered pipeline render | gate 4: final | `renders/final.mp4` (the offered final-quality render: same EDL, graphics composited from the beat table, delivery resolution and codec per `[render]`), or the creator's own editor render into `renders/` | +| 10 | package | mc-package | | `packaging/titles.md`, `packaging/thumbs/`, `packaging/description.md`, `packaging/chapters.md`, `packaging/captions/` (final.srt, final.vtt, transcript.md, when the cut exists) | +| 11 | final | the creator, with an offered pipeline render | gate 4: final | `renders/final.mp4` (the offered final-quality render: same EDL, graphics composited from the beat table, delivery resolution and codec per `[render]`, loudness-normalized to the `[render]` loudness-target unless loudnorm is off), or the creator's own editor render into `renders/` | | 12 | retro | mc-retro | | edits to `{formats-path}/.md` learnings + offending skill files | Stages 8 and 9 may run in parallel once gate 3 is approved. Stage 10 may start any time after gate 1 (the packaging promise exists from the outline). diff --git a/skills/mc-setup/SKILL.md b/skills/mc-setup/SKILL.md index 503f23a..3426880 100644 --- a/skills/mc-setup/SKILL.md +++ b/skills/mc-setup/SKILL.md @@ -45,7 +45,7 @@ Finish with step 8 as usual so the migrated config is verified and the pending g Bootstrap first: check `uv --version`. If uv is missing, offer to install it; otherwise the official installer from docs.astral.sh/uv, and wait for the creator's confirmation; every pipeline script runs through uv, so nothing works without it. -Then run `uv run {skill-root}/scripts/check_deps.py`. Report what is missing with the exact install command (brew/apt/winget as fits the platform). Install nothing without the creator confirming each item. The report includes a platform gate: the default transcription lane (parakeet-mlx) runs only on Apple Silicon. If this machine is not Apple Silicon, relay the script's fallback pointer (local whisper.cpp or faster-whisper; they normalize fillers away, so cut quality drops, and a supported cross-platform lane is planned) and carry that honesty into step 3's transcription question. +Then run `uv run {skill-root}/scripts/check_deps.py`. Report what is missing with the exact install command (brew/apt/winget as fits the platform). Install nothing without the creator confirming each item. The report ends with a platform verdict: detected OS, CPU architecture, and GPU vendor, plus the recommended stack file (`{skill-root}/references/stack-macos.md`, `stack-windows.md`, or `stack-linux.md`) and the platform-specific defaults it implies (transcription lane, torch index, encoder ladder, SVG rasterizer, fonts approach). Read the named stack file now and hold it as context for the rest of setup. If this machine is not Apple Silicon, relay the script's pointer honestly: the parakeet-mlx reference lane will not run here, and the recommended lane is onnx-asr with the same parakeet-tdt-0.6b-v3 weights (implemented; verbatim fillers and word timestamps carry over, CPU or CUDA per the verdict). Carry that honesty into step 3's transcription question. ### 3. The basics interview @@ -59,7 +59,7 @@ Ask, offering current values (or the `[defaults]` from customize.toml) as defaul - Live tool (`[live] tool`): obs, ecamm, or other. Drives the stream-pack lane's deliverable format. - Recurring shows or series they produce (names, cadence). Note them for format-profile choices and future series folders. - Editor (`[editor]`): which NLE they finish in. Set `timeline-format` accordingly: fcpxml for DaVinci Resolve or Final Cut Pro; xmeml/edl for Premiere; none for Descript or manual workflows (they get the cut plan, edl.json, and preview instead of a timeline file). Set `ograf-editable = true` ONLY for DaVinci Resolve 21+. -- Transcription (`[transcription]`): default parakeet-mlx (free, local, verbatim fillers preserved, no API key; Apple Silicon only). If step 2's platform gate flagged this machine, say plainly that the default lane will not run here and point at the documented local fallbacks. Metered API lanes exist behind the same switch as explicit opt-in choices; if, and only if, the creator picks one, set `provider` and `api-key-env` now and handle key sourcing in step 7. +- Transcription (`[transcription]`): default `auto` (free, local, verbatim fillers preserved, no API key): parakeet-mlx on Apple Silicon, onnx-asr with the same parakeet-tdt-0.6b-v3 weights everywhere else. If this machine is not Apple Silicon, present the stack file's recommended lane (onnx-asr, cpu or gpu extra per the step 2 verdict) as what this machine will use, relaying implemented/planned status from customize.toml; whisper-based fallbacks are no longer recommended (they normalize fillers away). Metered API lanes exist behind the same switch as explicit opt-in choices; if, and only if, the creator picks one, set `provider` and `api-key-env` now and handle key sourcing in step 7. - Interview marker cue: the spoken phrase that marks each question read aloud during interview-recording capture (mc-braindump's camera-rolling mode), so the cut stage can segment the recording mechanically. Default "question from the interviewer"; keep it unless the creator wants their own phrasing or has footage recorded against the older "question from claude" convention. Record a non-default cue as `cutplan_flags = '--marker-cues ""'` in `{project-root}/_bmad/custom/mc-cut.toml` (mc-cut's team override file, resolved by resolve_customization.py; edit it surgically, preserving any existing keys); mc-cut passes those flags straight to cutplan.py. The default needs no entry. Be honest about lane status: the comments in this skill's `customize.toml` under `[defaults.editor]` and `[defaults.transcription]` mark each timeline and transcription lane as implemented or planned; relay that, and never promise a planned lane as working. @@ -68,7 +68,7 @@ Be honest about lane status: the comments in this skill's `customize.toml` under Present and confirm the render-first default before writing `[render]`: Manticore renders a fast low-res preview for every cut and beats iteration, and offers a final-quality render at gate 4; the editor timeline export and all assets (edl.json, cutplan.md, overlays) are ALWAYS created alongside, so the creator can jump into Resolve, Premiere, or any editor at any step. -- Accepted (the default): `self-render = true`. Offer the quality knobs (preview-height, preview-crf, final-codec, final-crf) only if they ask; the defaults are sane. +- Accepted (the default): `self-render = true`. Offer the quality knobs (preview-height, preview-crf, final-codec, final-crf, loudnorm, loudness-target) only if they ask; the defaults are sane. Mention that the final render is loudness-normalized to -14 LUFS by default (the YouTube reference; the preview never is), alongside the final-codec and final-crf defaults. On a non-Mac machine, also confirm the stack file's platform expectations here: the encoder ladder the final render will probe, and on Windows with NVIDIA the torch cu126 index consent note (CUDA wheels add roughly 2.5 to 3 GB) for step 3d's audio workspace. - Declined: `self-render = false`. Previews and finals become offers the pipeline makes instead of automatic outputs; the timeline export and assets remain always-on. Record the answer explicitly; this consent is required, not assumed. @@ -131,7 +131,7 @@ CLI-tool-first: a registered CLI backed by a subscription the creator already pa Ask what they use for image generation, video generation, and offloaded research: any agentic or generation CLI they run. For each tool: -- name, capabilities (image/video/research/...), the exact headless invocation, preferred models. +- name, capabilities (image/video/research/...), the exact headless invocation, preferred models. The `headless` template is parsed with POSIX shell quoting on every OS (prefer forward slashes in template paths; backslash is an escape), and the tool name is resolved with a PATH lookup at run time, so npm-installed tools register by bare name on Windows too (.cmd/.exe shims are found via PATH+PATHEXT). - The `notes` field: everything future sessions must remember about driving it (quirks, output behavior, what it is bad at). Write it down now; this is the memory that stops every session from rediscovering the tool. - Verify end to end, with permission: first the version/help command (exists on PATH), then one small real invocation per registered capability (for example a tiny test image into a scratch folder), confirming the output file actually exists. Record the result in `notes` as verified end-to-end with the ISO date, or unverified. diff --git a/skills/mc-setup/assets/formats/livestream-pack.md b/skills/mc-setup/assets/formats/livestream-pack.md index d7dacd0..5068fd3 100644 --- a/skills/mc-setup/assets/formats/livestream-pack.md +++ b/skills/mc-setup/assets/formats/livestream-pack.md @@ -25,7 +25,8 @@ Not a video. One run of mc-stream-pack producing a complete OBS asset pack from - Static scenes as self-contained local HTML (starting-soon with countdown, BRB, ending, full overlay). OBS browser sources render local HTML transparent by default; no server. - Scenes are reactive via the `window.obsstudio` JS API (countdown resets on scene activation, lower thirds re-trigger entrance on visibility) with a plain-browser fallback. -- Stinger transition: one Remotion comp rendered twice (VP9 yuva420p WebM for OBS, ProRes 4444 MOV for the editor lane), 1 to 2 seconds. +- Stinger transition: one Remotion comp rendered twice (VP9 yuva420p WebM for OBS, ProRes 4444 MOV for the editor lane), 1 to 2 seconds. Baked alpha scene and lower-third deliverables list WebM VP9 alpha (libvpx-vp9 yuva420p) for OBS on any platform alongside the ProRes 4444 MOV; render_verify.py can transcode and verify the WebM from the ProRes master in one step. +- vMix note: vMix rejects MP4 stingers and prefers PNG sequences; when the live tool is vMix, deliver a PNG sequence or the ProRes 4444 MOV instead of WebM. Wirecast takes the ProRes 4444 MOV directly. - Lower thirds and topic cards as OGraf (via the mc-ograf skill), standalone-capable and SPX-GC-compatible for click-to-trigger later. ## Verification, not vibes diff --git a/skills/mc-setup/customize.toml b/skills/mc-setup/customize.toml index 3043e5f..94b402c 100644 --- a/skills/mc-setup/customize.toml +++ b/skills/mc-setup/customize.toml @@ -67,6 +67,12 @@ preview-crf = 28 # Final render: delivery quality, produced only on the gate-4 offer. final-codec = "h264" final-crf = 18 +# Two-pass ffmpeg loudnorm on the final render only; the fast preview is +# never normalized. Set false to skip normalization entirely. +loudnorm = true +# Integrated LUFS target for the final render; -14 is the YouTube reference. +# TP -1.5 / LRA 11 are fixed in the script. +loudness-target = -14 [defaults.style] @@ -115,15 +121,23 @@ ograf-editable = false [defaults.transcription] # Word-level transcription for the cut stage. Options: -# "parakeet-mlx" the default: free + local, Apple Silicon ONLY, word +# "auto" the default: picks parakeet-mlx on macOS Apple Silicon +# and onnx-asr everywhere else. Both lanes are free + +# local and run the same parakeet-tdt-0.6b-v3 weights. +# "parakeet-mlx" the reference lane: Apple Silicon ONLY, word # timestamps, verbatim fillers preserved (validated on # real footage 2026-07-05; model parakeet-tdt-0.6b-v3) -# whisper.cpp / faster-whisper local fallbacks for non-Apple-Silicon -# machines (planned lane; they normalize fillers away, -# so cut quality drops; see the README platform matrix) -# "deepgram-nova3" metered API lane with keyterm biasing (planned; opt-in -# only, chosen explicitly during setup) -provider = "parakeet-mlx" +# "onnx-asr" the cross-platform lane (implemented): Windows, Linux, +# and Intel Mac, same parakeet-tdt-0.6b-v3 weights as an +# ONNX conversion, so verbatim fillers and word +# timestamps carry over. CPU by default; CUDA machines +# escalate with uv run --with "onnx-asr[gpu,hub]" +# python (the python command is required: +# it skips the script's cpu-extra dependency so +# onnxruntime-gpu never co-installs with onnxruntime). +# "elevenlabs-scribe" / "deepgram-nova3" metered API lanes (planned; +# opt-in only, chosen explicitly during setup) +provider = "auto" # Env var holding the key for a metered provider. Leave blank for local lanes; # set only when choosing a metered provider. api-key-env = "" @@ -195,6 +209,12 @@ davinci-resolve = false # mc-setup, one block per agentic/generation CLI. The `notes` field is the # persistent memory: everything a skill needs to drive the tool correctly. # +# The `headless` template is parsed with POSIX shell quoting on every OS +# (single/double quotes as in a POSIX shell, backslash escapes; prefer +# forward slashes in template paths). The tool name (argv[0]) is resolved +# with a PATH lookup at run time, so npm-installed tools may be registered +# by bare name on Windows too (.cmd/.exe shims are found via PATH+PATHEXT). +# # [[modules.manticore.tools]] # name = "grok" # capabilities = ["image", "video", "x"] # Imagine image + video gen, X/Twitter access diff --git a/skills/mc-setup/references/stack-linux.md b/skills/mc-setup/references/stack-linux.md new file mode 100644 index 0000000..50d671c --- /dev/null +++ b/skills/mc-setup/references/stack-linux.md @@ -0,0 +1,38 @@ +# Linux default stack + +Selected when check_deps.py reports os Linux. The GPU verdict (nvidia, amd, intel, none, unknown) splits the transcription extra and the encoder ladder. Research basis: platform and capabilities audit, 2026-07-21. + +## Default stack + +| Concern | Default | +|---|---| +| Transcription | onnx-asr, gpu,hub extras if CUDA is detected, else cpu,hub; parakeet-tdt-0.6b-v3 ONNX weights. NeMo is the documented advanced alternative (heavy, not a uv-run default); ROCm is best-effort via onnxruntime-rocm | +| TTS | kokoro-onnx (identical everywhere; manylinux_2_28 wheels, glibc 2.28 or newer) | +| Music / SFX | MusicGen / AudioLDM2, CUDA if present else CPU; plain PyPI torch already bundles CUDA on Linux | +| Final render hardware encode | probe h264_nvenc, then h264_vaapi, then libx264; distro ffmpeg builds vary, always probe | +| Alpha master | ProRes 4444 (prores_ks); probe with `ffmpeg -encoders`, never assume | +| Alpha live deliverable | WebM VP9 alpha for OBS (identical everywhere) | +| Live tool | OBS (vMix, Wirecast, and Ecamm all skip Linux) | +| SVG rasterizer | rsvg-convert with the FONTCONFIG_FILE shim | +| Fonts and emoji | fontconfig works natively; install the noto-color-emoji package for emoji graphics | +| Screen recording | OBS with the PipeWire screen-capture source (Wayland portals); X11-style grabs black-screen on Wayland | + +## Torch index + +Plain PyPI. Linux torch wheels bundle CUDA already; no custom index. ROCm builds are Linux-only and opt-in. + +## Linux notes + +- Playwright needs system libraries; `playwright install-deps` automates this on Ubuntu and Debian only. On other distros install the equivalent packages manually. +- Minimal or headless machines need the noto-color-emoji package or Chromium renders emoji as tofu. +- Screen capture on Wayland must go through the PipeWire portal source in OBS; legacy X11 grabs produce black frames. + +## DaVinci Resolve notes + +The Linux free edition cannot decode or encode H.264 or H.265 and has no AAC at all. The FCPXML timeline imports fine, but mp4/AAC media is undecodable there; either transcode sources to ProRes or DNxHR first, or use Resolve Studio. + +The free edition only executes scripts launched from inside the app (Console or Workspace > Scripts); the external scripting API is Studio-only through Resolve 21. To run the scripted import on free Resolve, copy the pipeline's resolve_import.py into the Fusion Scripts folder and launch it from Workspace > Scripts: + +`~/.local/share/DaVinciResolve/Fusion/Scripts/` + +Studio users who want conversational post-import timeline surgery can opt into a community Resolve MCP server; Manticore itself keeps doing media import and timeline construction through the FCPXML lane and never requires an MCP server. diff --git a/skills/mc-setup/references/stack-macos.md b/skills/mc-setup/references/stack-macos.md new file mode 100644 index 0000000..07e37ef --- /dev/null +++ b/skills/mc-setup/references/stack-macos.md @@ -0,0 +1,34 @@ +# macOS default stack + +Selected when check_deps.py reports os Darwin. Apple Silicon is the module's reference platform; everything here is the shipped default behavior, confirmed with the creator during the mc-setup interview. Research basis: platform and capabilities audit, 2026-07-21. + +## Default stack (Apple Silicon) + +| Concern | Default | +|---|---| +| Transcription | parakeet-mlx (parakeet-tdt-0.6b-v3), free, local, verbatim fillers, word timestamps | +| TTS | kokoro-onnx (CPU realtime) | +| Music / SFX | MusicGen / AudioLDM2 on MPS (plain PyPI torch ships MPS wheels) | +| Final render hardware encode | h264_videotoolbox / hevc_videotoolbox, libx264 fallback | +| Alpha master | ProRes 4444 (prores_ks); probe with `ffmpeg -encoders`, never assume | +| Alpha live deliverable | WebM VP9 alpha for OBS browser and stinger use | +| Live tool | OBS default; Ecamm Live is the interviewed macOS-only option | +| SVG rasterizer | rsvg-convert or html_to_png.py (Chromium) | +| Fonts | system fonts via CoreText; data-URI @font-face inlining also works | +| Screen recording | Cmd-Shift-5 or OBS | + +## Intel Mac variant + +parakeet-mlx requires Apple Silicon (MLX runs on Metal only). On an Intel Mac the transcription lane is onnx-asr with the cpu,hub extras, running the same parakeet-tdt-0.6b-v3 weights as an ONNX conversion, so verbatim fillers and word timestamps carry over. Everything else above still applies, including videotoolbox encode. + +## Torch index + +Plain PyPI. macOS wheels include MPS support; no custom index is ever needed. + +## DaVinci Resolve notes + +The free edition only executes scripts launched from inside the app (Console or Workspace > Scripts); the external scripting API is Studio-only through Resolve 21. To run the scripted import on free Resolve, copy the pipeline's resolve_import.py into the Fusion Scripts folder and launch it from Workspace > Scripts: + +`~/Library/Application Support/Blackmagic Design/DaVinci Resolve/Fusion/Scripts/Utility/` + +Studio users who want conversational post-import timeline surgery can opt into a community Resolve MCP server; Manticore itself keeps doing media import and timeline construction through the FCPXML lane and never requires an MCP server. diff --git a/skills/mc-setup/references/stack-windows.md b/skills/mc-setup/references/stack-windows.md new file mode 100644 index 0000000..eed3611 --- /dev/null +++ b/skills/mc-setup/references/stack-windows.md @@ -0,0 +1,48 @@ +# Windows default stack + +Selected when check_deps.py reports os Windows. The GPU verdict (nvidia, intel, amd, none, unknown) splits the stack below. Research basis: platform and capabilities audit, 2026-07-21. + +## Default stack (NVIDIA GPU) + +| Concern | Default | +|---|---| +| Transcription | onnx-asr with the gpu,hub extras (onnxruntime CUDA execution provider), parakeet-tdt-0.6b-v3 ONNX weights; no WSL2 required | +| TTS | kokoro-onnx (identical everywhere; espeakng-loader wheels bundle espeak-ng.dll and data) | +| Music / SFX | MusicGen / AudioLDM2 on CUDA; torch from `--index-url https://download.pytorch.org/whl/cu126` (CUDA wheels add roughly 2.5 to 3 GB; the workspace consent message must say so) | +| Final render hardware encode | probe h264_nvenc, then h264_qsv, then h264_amf, then libx264; each probe is a 1-frame test encode, not just an encoder listing | +| Alpha master | ProRes 4444 (prores_ks); probe with `ffmpeg -encoders`, never assume | +| Alpha live deliverable | WebM VP9 alpha for OBS; see the vMix and Wirecast notes below | +| Live tool | OBS default; vMix and Wirecast are opt-in | +| SVG rasterizer | html_to_png.py (Chromium) or resvg; not rsvg-convert (awkward on Windows, never renders color emoji) | +| Fonts | data-URI @font-face inlining; Chromium uses DirectWrite on Windows, so the fontconfig shim is a no-op | +| Screen recording | OBS; Game Bar captures a single app only, never the Desktop or Explorer | + +## CPU-only variant + +Same as above except: + +- Transcription: onnx-asr with the cpu,hub extras at fp32. The 0.6B model is CPU-viable; never default to int8, which has a reported missing-words defect. +- Torch: plain PyPI (CPU wheels); no custom index. +- Music is usable but slow on CPU; AudioLDM2 at 100 steps is painful on CPU. Set expectations before farming. +- Render encode: libx264, with h264_qsv probed on Intel iGPUs and h264_amf on AMD. + +## Alpha assets for vMix and Wirecast + +- vMix rejects MP4 stingers; it wants MOV or, preferably, PNG sequences for stinger transitions. Deliver baked alpha as a PNG sequence or ProRes 4444 MOV when the live tool is vMix. +- Wirecast accepts ProRes 4444 MOV directly; the standard alpha master already satisfies it. +- OBS takes the WebM VP9 alpha deliverable and HTML browser sources; that stays the default. + +## Windows notes + +- Keep `{engines-path}` short. Deep Node engine workspaces (HyperFrames, Remotion) can trip MAX_PATH; also enable LongPathsEnabled in the registry. +- Install the gyan.dev full ffmpeg build; it carries libx264, prores_ks, libvpx-vp9, and drawtext with freetype/fontconfig. +- Screen recording goes through OBS, not Game Bar (single-app only, no Desktop or Explorer capture). +- Registered CLI tools installed via npm run through `.cmd` shims; the pipeline resolves them with PATH lookup before launching. Headless templates use POSIX quoting. + +## DaVinci Resolve notes + +The free edition only executes scripts launched from inside the app (Console or Workspace > Scripts); the external scripting API is Studio-only through Resolve 21. To run the scripted import on free Resolve, copy the pipeline's resolve_import.py into the Fusion Scripts folder and launch it from Workspace > Scripts: + +`%APPDATA%\Blackmagic Design\DaVinci Resolve\Support\Fusion\Scripts\Utility\` + +Studio users who want conversational post-import timeline surgery can opt into a community Resolve MCP server; Manticore itself keeps doing media import and timeline construction through the FCPXML lane and never requires an MCP server. diff --git a/skills/mc-setup/scripts/check_deps.py b/skills/mc-setup/scripts/check_deps.py index b8da23c..9ab200b 100644 --- a/skills/mc-setup/scripts/check_deps.py +++ b/skills/mc-setup/scripts/check_deps.py @@ -7,11 +7,16 @@ Usage: uv run check_deps.py [--json] -Checks presence on PATH (and Python version), plus a platform gate: the -default transcription lane (parakeet-mlx) runs only on Apple Silicon. On -other machines the report points at the documented local fallbacks -(whisper.cpp or faster-whisper); the gate never fails the check. Prints a -table (or JSON) and exits 0 if all required deps are present, 1 otherwise. +Checks presence on PATH (and Python version), detects the platform (OS, +CPU architecture, GPU vendor via cheap best-effort probes), and reports +the recommended per-platform stack: which stack reference file applies +(references/stack-macos.md, stack-windows.md, or stack-linux.md) and the +platform-specific defaults it implies (transcription lane, torch index, +hardware-encoder ladder, SVG rasterizer, fonts approach). + +The platform rows and the stack verdict are informational only; they +never fail the check. Prints a table (or JSON: {"results", "platform", +"ok"}) and exits 0 if all required deps are present, 1 otherwise. Installs nothing. """ @@ -19,7 +24,9 @@ import json import platform import shutil +import subprocess import sys +from pathlib import Path DEPS = [ # (command, required, why) @@ -32,11 +39,164 @@ ("yt-dlp", False, "pulling your published transcripts for the voice bible"), ] +# PCI vendor ids seen in /sys/class/drm on Linux. +PCI_VENDORS = { + "0x10de": "NVIDIA", + "0x1002": "AMD", + "0x1022": "AMD", + "0x8086": "Intel", +} + +TORCH_CUDA_INDEX = "https://download.pytorch.org/whl/cu126" + def is_apple_silicon() -> bool: return sys.platform == "darwin" and platform.machine() == "arm64" +def classify_gpu(names: list[str]) -> str: + """Map a list of adapter/vendor names to a vendor verdict. + + Empty list means the probe ran and found nothing ("none"); names that + match no known vendor yield "unknown". + """ + if not names: + return "none" + joined = " ".join(names).lower() + if "nvidia" in joined: + return "nvidia" + if "amd" in joined or "radeon" in joined or "advanced micro devices" in joined: + return "amd" + if "intel" in joined: + return "intel" + return "unknown" + + +def gpu_names_linux() -> list[str] | None: + """Best-effort GPU vendor names from sysfs; None when the probe fails.""" + drm = Path("/sys/class/drm") + if not drm.is_dir(): + return None + names: list[str] = [] + for vendor_file in sorted(drm.glob("card[0-9]*/device/vendor")): + try: + vid = vendor_file.read_text().strip().lower() + except OSError: + continue + names.append(PCI_VENDORS.get(vid, vid)) + return names + + +def gpu_names_windows() -> list[str] | None: + """Best-effort adapter names via PowerShell CIM or legacy wmic; None on failure. + + PowerShell first: wmic is disabled by default on Windows 11 23H2+ and is + being removed, so it is only a fallback for older installs.""" + commands = [ + ["powershell", "-NoProfile", "-Command", + "Get-CimInstance Win32_VideoController | Select-Object -ExpandProperty Name"], + ["wmic", "path", "win32_VideoController", "get", "name"], + ] + for cmd in commands: + if not shutil.which(cmd[0]): + continue + try: + proc = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=15) + except (OSError, subprocess.SubprocessError): + continue + if proc.returncode != 0: + continue + lines = [ln.strip() for ln in proc.stdout.splitlines() if ln.strip()] + names = [ln for ln in lines if ln.lower() != "name"] # wmic header row + if names: + return names + return None + + +def detect_gpu(os_name: str, arch: str) -> tuple[str, str]: + """Return (vendor, detail). Vendor: apple/nvidia/amd/intel/none/unknown.""" + if os_name == "Darwin": + if arch == "arm64": + return "apple", "Apple Silicon GPU (Metal/MPS)" + return "unknown", "Intel Mac; GPU vendor not probed (CPU lanes apply)" + if shutil.which("nvidia-smi"): + return "nvidia", "nvidia-smi found on PATH" + if os_name == "Windows": + names = gpu_names_windows() + elif os_name == "Linux": + names = gpu_names_linux() + else: + names = None + if names is None: + return "unknown", "GPU probe unavailable or failed on this platform" + if not names: + return "none", "no GPU adapters detected (best-effort probe)" + return classify_gpu(names), "; ".join(names) + + +def recommend_stack(os_name: str, arch: str, gpu: str) -> dict: + """Pure decision tree: platform facts -> stack file + per-platform defaults. + + Keys are kebab-case. Non-Darwin, non-Windows systems get the Linux stack + (the closest POSIX shape). + """ + if os_name == "Darwin": + return { + "stack-file": "references/stack-macos.md", + "transcription": ("parakeet-mlx" if arch == "arm64" + else "onnx-asr[cpu,hub] (parakeet-tdt-0.6b-v3 ONNX weights)"), + "torch-index": "default PyPI (MPS wheels on macOS)", + "encoder-ladder": ["h264_videotoolbox", "libx264"], + "svg-rasterizer": "rsvg-convert or html_to_png.py (Chromium)", + "fonts": "system fonts via CoreText; data-URI @font-face inlining also works", + } + if os_name == "Windows": + if gpu == "nvidia": + ladder = ["h264_nvenc", "h264_qsv", "h264_amf", "libx264"] + elif gpu == "intel": + ladder = ["h264_qsv", "libx264"] + elif gpu == "amd": + ladder = ["h264_amf", "libx264"] + else: + ladder = ["libx264"] + return { + "stack-file": "references/stack-windows.md", + "transcription": ("onnx-asr[gpu,hub] (onnxruntime CUDA EP)" if gpu == "nvidia" + else "onnx-asr[cpu,hub] at fp32 (never int8)"), + "torch-index": (TORCH_CUDA_INDEX if gpu == "nvidia" + else "default PyPI (CPU-only wheels on Windows)"), + "encoder-ladder": ladder, + "svg-rasterizer": "html_to_png.py (Chromium) or resvg; not rsvg-convert", + "fonts": "data-URI @font-face inlining (DirectWrite; fontconfig shim is a no-op)", + } + # Linux and any other POSIX. + ladder = (["h264_nvenc", "h264_vaapi", "libx264"] if gpu == "nvidia" + else ["h264_vaapi", "libx264"]) + return { + "stack-file": "references/stack-linux.md", + "transcription": ("onnx-asr[gpu,hub] (onnxruntime CUDA EP)" if gpu == "nvidia" + else "onnx-asr[cpu,hub]"), + "torch-index": "default PyPI (CUDA-bundled wheels on Linux)", + "encoder-ladder": ladder, + "svg-rasterizer": "rsvg-convert with the FONTCONFIG_FILE shim", + "fonts": "fontconfig; install noto-color-emoji for emoji graphics", + } + + +def platform_report() -> dict: + os_name = platform.system() + arch = platform.machine() + gpu, gpu_detail = detect_gpu(os_name, arch) + return { + "os": os_name, + "arch": arch, + "apple-silicon": is_apple_silicon(), + "gpu": gpu, + "gpu-detail": gpu_detail, + "recommended": recommend_stack(os_name, arch, gpu), + } + + def main() -> None: ap = argparse.ArgumentParser(description=__doc__) ap.add_argument("--json", action="store_true") @@ -58,10 +218,11 @@ def main() -> None: "detail": ( "default transcription lane (parakeet-mlx) is supported on this machine" if apple_silicon - else "default transcription lane (parakeet-mlx) is Apple-Silicon-only; use a " - "local whisper.cpp or faster-whisper fallback on this machine (word " - "timestamps, but fillers get normalized, so cut quality drops; a supported " - "cross-platform lane is planned). See the README platform matrix." + else "default transcription lane (parakeet-mlx) is Apple-Silicon-only; the " + "recommended lane on this machine is onnx-asr with the same " + "parakeet-tdt-0.6b-v3 weights (verbatim fillers and word timestamps " + "carry over; CPU or CUDA). See the recommended stack below and the " + "README platform matrix." ), }) for cmd, required, why in DEPS: @@ -69,19 +230,28 @@ def main() -> None: results.append({"dep": cmd, "required": required, "found": bool(path), "detail": path or why}) missing_required = [r for r in results if r["required"] and not r["found"]] + plat = platform_report() if args.json: - print(json.dumps({"results": results, "ok": not missing_required}, indent=2)) + print(json.dumps({"results": results, "platform": plat, "ok": not missing_required}, indent=2)) else: for r in results: mark = "ok " if r["found"] else ("MISSING " if r["required"] else "missing (optional) ") print(f"{mark:22} {r['dep']:14} {r['detail']}") + rec = plat["recommended"] + print(f"\nPlatform: {plat['os']} {plat['arch']}, gpu {plat['gpu']} ({plat['gpu-detail']})") + print(f"Recommended stack file: {rec['stack-file']}") + print(f" transcription: {rec['transcription']}") + print(f" torch-index: {rec['torch-index']}") + print(f" encoder-ladder: {' -> '.join(rec['encoder-ladder'])}") + print(f" svg-rasterizer: {rec['svg-rasterizer']}") + print(f" fonts: {rec['fonts']}") if not apple_silicon: print( "\nNOTE: this machine cannot run the default transcription lane " - "(parakeet-mlx is Apple-Silicon-only). Until the cross-platform lane " - "ships, run whisper.cpp or faster-whisper locally instead; see the " - "README platform matrix." + "(parakeet-mlx is Apple-Silicon-only). The recommended lane here is " + "onnx-asr with the same parakeet-tdt-0.6b-v3 weights; see the " + "recommended stack file above and the README platform matrix." ) if missing_required: print(f"\n{len(missing_required)} required dependency(ies) missing.") diff --git a/skills/mc-setup/scripts/tests/test-check_deps.py b/skills/mc-setup/scripts/tests/test-check_deps.py index 2b936c6..6acecfd 100644 --- a/skills/mc-setup/scripts/tests/test-check_deps.py +++ b/skills/mc-setup/scripts/tests/test-check_deps.py @@ -2,10 +2,15 @@ # /// script # requires-python = ">=3.11" # /// -"""Tests for check_deps.py: --json output shape and exit-code contract. +"""Tests for check_deps.py: --json output shape, exit-code contract, and the +platform/stack decision tree. PATH-dependent results are not asserted per-dep; the tests pin the shape and -the ok/exit-code relationship, which hold on any machine.""" +the ok/exit-code relationship, which hold on any machine. The recommend_stack +and classify_gpu decision tables are pure functions tested for every OS/GPU +combination without probing real hardware; no GPU probes are executed beyond +what the script itself does on this machine.""" +import importlib.util import json import subprocess import sys @@ -14,6 +19,15 @@ SCRIPT = Path(__file__).resolve().parent.parent / "check_deps.py" +spec = importlib.util.spec_from_file_location("check_deps", SCRIPT) +check_deps = importlib.util.module_from_spec(spec) +spec.loader.exec_module(check_deps) + +RECOMMENDED_KEYS = { + "stack-file", "transcription", "torch-index", + "encoder-ladder", "svg-rasterizer", "fonts", +} + def run(args): return subprocess.run([sys.executable, str(SCRIPT), *args], capture_output=True, text=True) @@ -39,6 +53,7 @@ def test_exit_code_matches_ok(self): def test_table_output_runs(self): proc = run([]) self.assertIn("uv", proc.stdout) + self.assertIn("Recommended stack file:", proc.stdout) self.assertIn(proc.returncode, (0, 1)) def test_platform_gate_row(self): @@ -50,7 +65,106 @@ def test_platform_gate_row(self): self.assertFalse(row["required"]) # informational; never fails the check self.assertIn("parakeet-mlx", row["detail"]) if not row["found"]: - self.assertIn("whisper", row["detail"]) + self.assertIn("onnx-asr", row["detail"]) + + def test_json_platform_shape(self): + proc = run(["--json"]) + data = json.loads(proc.stdout) + plat = data["platform"] + self.assertEqual( + {"os", "arch", "apple-silicon", "gpu", "gpu-detail", "recommended"}, + set(plat), + ) + self.assertIn(plat["gpu"], ("apple", "nvidia", "amd", "intel", "none", "unknown")) + rec = plat["recommended"] + self.assertEqual(RECOMMENDED_KEYS, set(rec)) + self.assertRegex(rec["stack-file"], r"^references/stack-(macos|windows|linux)\.md$") + self.assertIsInstance(rec["encoder-ladder"], list) + self.assertEqual(rec["encoder-ladder"][-1], "libx264") + + +class TestClassifyGpu(unittest.TestCase): + def test_empty_is_none(self): + self.assertEqual(check_deps.classify_gpu([]), "none") + + def test_vendors(self): + self.assertEqual(check_deps.classify_gpu(["NVIDIA GeForce RTX 4090"]), "nvidia") + self.assertEqual(check_deps.classify_gpu(["AMD Radeon RX 7900"]), "amd") + self.assertEqual(check_deps.classify_gpu(["Radeon Pro W6800"]), "amd") + self.assertEqual(check_deps.classify_gpu(["Intel(R) UHD Graphics 770"]), "intel") + + def test_nvidia_wins_over_igpu(self): + names = ["Intel(R) UHD Graphics 770", "NVIDIA GeForce RTX 3060"] + self.assertEqual(check_deps.classify_gpu(names), "nvidia") + + def test_unrecognized_is_unknown(self): + self.assertEqual(check_deps.classify_gpu(["0x1234"]), "unknown") + + +class TestRecommendStack(unittest.TestCase): + def test_macos_apple_silicon(self): + rec = check_deps.recommend_stack("Darwin", "arm64", "apple") + self.assertEqual(rec["stack-file"], "references/stack-macos.md") + self.assertEqual(rec["transcription"], "parakeet-mlx") + self.assertEqual(rec["encoder-ladder"], ["h264_videotoolbox", "libx264"]) + self.assertIn("MPS", rec["torch-index"]) + + def test_intel_mac_gets_onnx_asr_cpu(self): + rec = check_deps.recommend_stack("Darwin", "x86_64", "unknown") + self.assertEqual(rec["stack-file"], "references/stack-macos.md") + self.assertIn("onnx-asr[cpu,hub]", rec["transcription"]) + + def test_windows_nvidia(self): + rec = check_deps.recommend_stack("Windows", "AMD64", "nvidia") + self.assertEqual(rec["stack-file"], "references/stack-windows.md") + self.assertIn("onnx-asr[gpu,hub]", rec["transcription"]) + self.assertEqual(rec["torch-index"], check_deps.TORCH_CUDA_INDEX) + self.assertEqual(rec["encoder-ladder"], + ["h264_nvenc", "h264_qsv", "h264_amf", "libx264"]) + + def test_windows_cpu_only(self): + rec = check_deps.recommend_stack("Windows", "AMD64", "none") + self.assertIn("onnx-asr[cpu,hub]", rec["transcription"]) + self.assertIn("never int8", rec["transcription"]) + self.assertNotIn("download.pytorch.org", rec["torch-index"]) + self.assertEqual(rec["encoder-ladder"], ["libx264"]) + + def test_windows_intel_igpu_probes_qsv(self): + rec = check_deps.recommend_stack("Windows", "AMD64", "intel") + self.assertEqual(rec["encoder-ladder"], ["h264_qsv", "libx264"]) + + def test_windows_amd_probes_amf(self): + rec = check_deps.recommend_stack("Windows", "AMD64", "amd") + self.assertEqual(rec["encoder-ladder"], ["h264_amf", "libx264"]) + + def test_linux_nvidia(self): + rec = check_deps.recommend_stack("Linux", "x86_64", "nvidia") + self.assertEqual(rec["stack-file"], "references/stack-linux.md") + self.assertIn("onnx-asr[gpu,hub]", rec["transcription"]) + self.assertEqual(rec["encoder-ladder"], + ["h264_nvenc", "h264_vaapi", "libx264"]) + self.assertNotIn("download.pytorch.org", rec["torch-index"]) + + def test_linux_no_gpu(self): + rec = check_deps.recommend_stack("Linux", "x86_64", "none") + self.assertIn("onnx-asr[cpu,hub]", rec["transcription"]) + self.assertEqual(rec["encoder-ladder"], ["h264_vaapi", "libx264"]) + self.assertIn("noto-color-emoji", rec["fonts"]) + + def test_unknown_posix_falls_to_linux_stack(self): + rec = check_deps.recommend_stack("FreeBSD", "amd64", "unknown") + self.assertEqual(rec["stack-file"], "references/stack-linux.md") + + +class TestDetectGpu(unittest.TestCase): + def test_darwin_arm64_is_apple(self): + vendor, detail = check_deps.detect_gpu("Darwin", "arm64") + self.assertEqual(vendor, "apple") + self.assertIn("MPS", detail) + + def test_darwin_intel_is_unknown(self): + vendor, _ = check_deps.detect_gpu("Darwin", "x86_64") + self.assertEqual(vendor, "unknown") if __name__ == "__main__": diff --git a/skills/mc-stream-pack/SKILL.md b/skills/mc-stream-pack/SKILL.md index fcbc5d9..4b59fb3 100644 --- a/skills/mc-stream-pack/SKILL.md +++ b/skills/mc-stream-pack/SKILL.md @@ -10,8 +10,8 @@ Brand tokens in, complete pack out. Spec lives in the `livestream-pack` format p ## Steps 1. Load the studio config (`uv run {project-root}/_bmad/scripts/resolve_config.py --project-root {project-root} --key modules.manticore`; empty means mc-setup has not run: stop and route the creator there) and this skill's own surface (`uv run {project-root}/_bmad/scripts/resolve_customization.py --skill {skill-root}`; run `{workflow.activation_steps_prepend}` now, `{workflow.activation_steps_append}` after this step, and hold `{workflow.persistent_facts}` as standing context). Resolve `paths` values against `{project-root}`. Read `project.json` (stage `stream-pack`), the `livestream-pack` format profile, `{brand-path}/tokens.json`, and `{brand-path}/production-bible.md` when it exists (the styling contract beyond tokens: overlay and popup aesthetic for scenes and lower thirds, per-series template sections, and the CTA section). The OGraf standards apply to the lower thirds; the mc-ograf skill enforces them in step 2. -2. Build the pack the profile specifies (scene list, reactivity, render formats, and durations come from the profile, not from here): static scenes as self-contained local HTML in `graphics/scenes/`, all styling from tokens.json; the stinger as one Remotion comp in `{engines-path}/remotion/`, rendered to both formats the profile names; lower thirds and topic cards via the mc-ograf skill (never reach into its folder). Sound for the pack (the stinger whoosh, a Starting Soon music bed) routes through the mc-audio service skill the same way; deliver the wavs alongside the scenes with OBS wiring noted in HANDOFF.md. This live lane does not require `[editor] ograf-editable`; OBS/SPX-GC is editor-independent. -3. Verify, not vibes: run the profile's verification section. Scene screenshots land in `graphics/_verify/` and every one is visually checked; stinger checks run via `uv run {skill-root}/scripts/render_verify.py`. +2. Build the pack the profile specifies (scene list, reactivity, render formats, and durations come from the profile, not from here): static scenes as self-contained local HTML in `graphics/scenes/`, all styling from tokens.json; the stinger as one Remotion comp in `{engines-path}/remotion/`, rendered to both formats the profile names; lower thirds and topic cards via the mc-ograf skill (never reach into its folder). Baked alpha deliverables headed for OBS browser or stinger use on any platform get a WebM VP9 alpha variant produced and verified in one step: `uv run {skill-root}/scripts/render_verify.py graphics/.mov --transcode-webm graphics/.webm` (checks default to yuva420p; add `--expect-res`/`--expect-fps`/`--expect-dur` from the profile). When `[live] tool` is vmix or other, know the targets: vMix rejects MP4 stingers and prefers PNG sequences, so deliver a PNG sequence (`ffmpeg -i .mov -pix_fmt rgba graphics/-png/%04d.png`) or the ProRes 4444 MOV instead of WebM; Wirecast takes the ProRes 4444 MOV directly. Sound for the pack (the stinger whoosh, a Starting Soon music bed) routes through the mc-audio service skill the same way; deliver the wavs alongside the scenes with OBS wiring noted in HANDOFF.md. This live lane does not require `[editor] ograf-editable`; OBS/SPX-GC is editor-independent. +3. Verify, not vibes: run the profile's verification section. Scene screenshots land in `graphics/_verify/` and every one is visually checked; stinger checks run via `uv run {skill-root}/scripts/render_verify.py`. Stinger and baked-asset WebM variants are verified with `--pixfmt yuva420p`, or produced and verified in one step via `--transcode-webm` as in step 2. 4. Write `graphics/HANDOFF.md`: OBS setup steps per asset (browser source URLs/sizes, stinger transition settings). Update project.json artifacts and advance stage per the profile's stages list (next after `stream-pack`, normally `final`): the creator loads the pack in OBS and approves the look live. ## Checklist diff --git a/skills/mc-stream-pack/scripts/render_verify.py b/skills/mc-stream-pack/scripts/render_verify.py index 94acc34..ce5a98f 100644 --- a/skills/mc-stream-pack/scripts/render_verify.py +++ b/skills/mc-stream-pack/scripts/render_verify.py @@ -9,6 +9,7 @@ [--meta PATH] [--pixfmt prores4444|yuva420p|] [--expect-dur SECONDS] [--expect-fps FPS] [--expect-res WxH] [--dur-tol SECONDS] [--frames 5] [--checker] [--out-dir PATH] + [--transcode-webm OUT.webm] [--webm-crf 30] Contract: input a rendered MOV/WebM/mp4; every expectation arrives explicitly from @@ -30,10 +31,20 @@ checkerboard when the file carries alpha, or when --checker is passed) into a _verify/ folder next to the input (or --out-dir) for visual inspection by the calling skill + webm --transcode-webm OUT.webm first transcodes the input (normally the + ProRes 4444 alpha master) to a libvpx-vp9 yuva420p WebM at OUT (the + OBS browser-source / stinger deliverable), then runs every check + and frame extraction against OUT instead of the input; the pixfmt + expectation defaults to yuva420p in this mode when none is given; + hard errors (exit 2) when ffmpeg lacks the libvpx-vp9 encoder or + the input carries no alpha; --webm-crf tunes quality (default 30) output structured JSON to stdout: probe summary, per-check expected/actual/pass, extracted frame paths; exit 0 when every check passes, exit 1 when any check fails, exit 2 on hard errors - (missing input, no video stream, missing ffprobe/ffmpeg) + (missing input, no video stream, missing ffprobe/ffmpeg, + failed or impossible WebM transcode); a failed yuva420p pixfmt + check on an alpha-less file carries a "hint" with the exact + re-render flags rule a render is NOT done until frames have been extracted and visually checked (the self-QA loop: edit, lint, preview, draft render CRF 28, single-frame verify, final render) @@ -117,6 +128,48 @@ def check_pixfmt(expected: str, stream: dict) -> bool: return pix_fmt == expected +def pixfmt_hint(expected: str, stream: dict) -> str | None: + """A human-readable pointer for a failed pixfmt check, or None.""" + if expected not in ("yuva420p", "prores4444") or has_alpha(stream): + return None + return ( + "no alpha channel in this file; for a VP9 alpha WebM re-encode with " + "-c:v libvpx-vp9 -pix_fmt yuva420p -auto-alt-ref 0 from an alpha " + "master (ProRes 4444), or rerun this script with --transcode-webm" + ) + + +def encoder_available(name: str) -> bool: + r = subprocess.run(["ffmpeg", "-hide_banner", "-encoders"], + capture_output=True, text=True) + if r.returncode != 0: + return False + return any(line.split()[1:2] == [name] for line in r.stdout.splitlines() + if line.strip()) + + +def transcode_webm(src: Path, out: Path, stream: dict, crf: int) -> None: + """Transcode an alpha master to a libvpx-vp9 yuva420p WebM at out.""" + if shutil.which("ffmpeg") is None: + die("ffmpeg not found on PATH") + if not has_alpha(stream): + die(f"input has no alpha channel ({stream.get('codec_name', '?')}/" + f"{stream.get('pix_fmt', '?')}); a VP9 alpha WebM must be " + "transcoded from an alpha master such as ProRes 4444") + if not encoder_available("libvpx-vp9"): + die("this ffmpeg build has no libvpx-vp9 encoder; install a full " + "ffmpeg build (brew/apt/winget builds include it) to produce the " + "VP9 alpha WebM deliverable") + out.parent.mkdir(parents=True, exist_ok=True) + cmd = ["ffmpeg", "-y", "-v", "error", "-i", str(src), + "-c:v", "libvpx-vp9", "-pix_fmt", "yuva420p", + "-auto-alt-ref", "0", "-b:v", "0", "-crf", str(crf), + "-row-mt", "1", "-c:a", "libopus", str(out)] + r = subprocess.run(cmd, capture_output=True, text=True) + if r.returncode != 0 or not out.is_file(): + die(f"VP9 WebM transcode failed: {r.stderr.strip()}") + + def extract_frames(path: Path, out_dir: Path, count: int, duration: float | None, width: int, height: int, checker: bool) -> list[str]: if shutil.which("ffmpeg") is None: @@ -177,6 +230,11 @@ def main() -> None: p.add_argument("--checker", action="store_true", help="force checkerboard compositing (automatic for alpha files)") p.add_argument("--out-dir", help="frame output folder (default: _verify/ next to the input)") + p.add_argument("--transcode-webm", metavar="OUT.webm", + help="transcode the input (alpha master) to a libvpx-vp9 " + "yuva420p WebM at OUT, then verify OUT instead") + p.add_argument("--webm-crf", type=int, default=30, + help="CRF for the VP9 WebM transcode (default 30)") args = p.parse_args() path = Path(args.input) @@ -211,6 +269,20 @@ def main() -> None: stream = streams[0] fmt = data.get("format", {}) + transcoded = None + if args.transcode_webm: + webm_out = Path(args.transcode_webm) + transcode_webm(path, webm_out, stream, args.webm_crf) + transcoded = str(webm_out) + path = webm_out + data = probe(path) + streams = [s for s in data.get("streams", []) if s.get("codec_type") == "video"] + if not streams: + die(f"no video stream in transcoded {path}") + stream = streams[0] + fmt = data.get("format", {}) + expect.setdefault("pixfmt", "yuva420p") + width = int(stream.get("width", 0)) height = int(stream.get("height", 0)) fps = parse_fps(stream) @@ -219,11 +291,16 @@ def main() -> None: checks: dict = {} if "pixfmt" in expect: + pf_pass = check_pixfmt(str(expect["pixfmt"]), stream) checks["pixfmt"] = { "expected": str(expect["pixfmt"]), "actual": f"{stream.get('codec_name', '?')}/{stream.get('pix_fmt', '?')}", - "pass": check_pixfmt(str(expect["pixfmt"]), stream), + "pass": pf_pass, } + if not pf_pass: + hint = pixfmt_hint(str(expect["pixfmt"]), stream) + if hint: + checks["pixfmt"]["hint"] = hint if "res" in expect: try: ew, eh = (int(v) for v in str(expect["res"]).lower().split("x")) @@ -249,6 +326,7 @@ def main() -> None: print(json.dumps({ "ok": ok, "input": str(path), + "transcoded": transcoded, "probe": {"codec": stream.get("codec_name"), "pix_fmt": stream.get("pix_fmt"), "width": width, "height": height, "fps": fps, "duration": duration, "alpha": alpha}, diff --git a/skills/mc-stream-pack/scripts/tests/test-render_verify.py b/skills/mc-stream-pack/scripts/tests/test-render_verify.py new file mode 100644 index 0000000..12cf5f3 --- /dev/null +++ b/skills/mc-stream-pack/scripts/tests/test-render_verify.py @@ -0,0 +1,245 @@ +#!/usr/bin/env python3 +# /// script +# requires-python = ">=3.11" +# /// +"""Tests for render_verify.py against synthetic ffmpeg fixtures: the verifier +must probe correctly, honor explicit and meta.json expectations, extract +frames (checkerboarded for alpha), fail loudly on mismatches, and (stream-pack +extension) transcode an alpha master to a libvpx-vp9 yuva420p WebM via +--transcode-webm and verify the result. ffmpeg-gated; no models, no network, +no downloads.""" +import json +import shutil +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +SCRIPT = Path(__file__).resolve().parent.parent / "render_verify.py" +HAVE_FFMPEG = shutil.which("ffmpeg") is not None and shutil.which("ffprobe") is not None + + +def have_encoder(name): + if not HAVE_FFMPEG: + return False + r = subprocess.run(["ffmpeg", "-hide_banner", "-encoders"], + capture_output=True, text=True) + return r.returncode == 0 and any( + line.split()[1:2] == [name] for line in r.stdout.splitlines() if line.strip()) + + +HAVE_VP9 = have_encoder("libvpx-vp9") + + +def run(args): + r = subprocess.run([sys.executable, str(SCRIPT), *args], capture_output=True, text=True) + out = None + if r.stdout.strip(): + out = json.loads(r.stdout) + return r, out + + +def make_fixture(out: Path, *, codec_args, color="red@0.5", size="320x180", rate=30, dur=2, + pixel="rgba"): + cmd = ["ffmpeg", "-y", "-v", "error", "-f", "lavfi", + "-i", f"color=c={color}:s={size}:r={rate}:d={dur},format={pixel}", + *codec_args, str(out)] + subprocess.run(cmd, check=True, capture_output=True) + + +@unittest.skipUnless(HAVE_FFMPEG, "ffmpeg/ffprobe not on PATH") +class TestRenderVerify(unittest.TestCase): + tmp: tempfile.TemporaryDirectory + + @classmethod + def setUpClass(cls): + cls.tmp = tempfile.TemporaryDirectory() + root = Path(cls.tmp.name) + cls.prores = root / "prores.mov" + make_fixture(cls.prores, codec_args=["-c:v", "prores_ks", "-profile:v", "4444", + "-pix_fmt", "yuva444p10le"]) + cls.vp9 = root / "vp9.webm" + make_fixture(cls.vp9, color="lime@0.5", dur=1, + codec_args=["-c:v", "libvpx-vp9", "-pix_fmt", "yuva420p", + "-b:v", "100k", "-deadline", "realtime", "-cpu-used", "8"]) + cls.opaque = root / "opaque.mp4" + make_fixture(cls.opaque, color="blue", rate=25, pixel="yuv420p", + codec_args=["-c:v", "libx264", "-pix_fmt", "yuv420p"]) + + @classmethod + def tearDownClass(cls): + cls.tmp.cleanup() + + def _out_dir(self): + d = tempfile.mkdtemp(dir=self.tmp.name) + return Path(d) / "_verify" + + def test_prores_all_checks_pass(self): + r, out = run([str(self.prores), "--pixfmt", "prores4444", + "--expect-res", "320x180", "--expect-fps", "30", + "--expect-dur", "2", "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + self.assertTrue(out["ok"]) + self.assertTrue(all(c["pass"] for c in out["checks"].values())) + self.assertTrue(out["probe"]["alpha"]) + self.assertTrue(out["checkerboard"]) + + def test_checkerboard_frames_contain_the_graphic(self): + # The fixture is half-transparent red over the full frame, so every + # extracted checkerboard frame must be visibly red, not a bare board + # (regression: seek+overlay in one command emitted board-only frames). + d = self._out_dir() + r, out = run([str(self.prores), "--frames", "2", "--out-dir", str(d)]) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + for f in out["frames"]: + px = subprocess.run( + ["ffmpeg", "-v", "error", "-i", f, "-vf", "crop=1:1:160:90", + "-frames:v", "1", "-f", "rawvideo", "-pix_fmt", "rgb24", "-"], + capture_output=True, check=True).stdout + rr, gg, bb = px[0], px[1], px[2] + self.assertGreater(rr, gg + 60, + f"{f}: expected red-tinted composite, got rgb({rr},{gg},{bb})") + + def test_extracts_requested_frame_count(self): + d = self._out_dir() + r, out = run([str(self.prores), "--frames", "3", "--out-dir", str(d)]) + self.assertEqual(r.returncode, 0) + self.assertEqual(len(out["frames"]), 3) + for f in out["frames"]: + self.assertTrue(Path(f).is_file(), f"missing extracted frame {f}") + + def test_resolution_mismatch_fails(self): + r, out = run([str(self.prores), "--expect-res", "1920x1080", + "--out-dir", str(self._out_dir())]) + self.assertNotEqual(r.returncode, 0) + self.assertFalse(out["ok"]) + self.assertFalse(out["checks"]["res"]["pass"]) + + def test_duration_mismatch_fails(self): + r, out = run([str(self.prores), "--expect-dur", "5", + "--out-dir", str(self._out_dir())]) + self.assertNotEqual(r.returncode, 0) + self.assertFalse(out["checks"]["dur"]["pass"]) + + def test_pixfmt_mismatch_fails(self): + r, out = run([str(self.opaque), "--pixfmt", "prores4444", + "--out-dir", str(self._out_dir())]) + self.assertNotEqual(r.returncode, 0) + self.assertFalse(out["checks"]["pixfmt"]["pass"]) + + def test_vp9_webm_alpha_mode_counts_as_yuva420p(self): + # libvpx-vp9 alpha in WebM probes as yuv420p + alpha_mode tag + r, out = run([str(self.vp9), "--pixfmt", "yuva420p", "--expect-dur", "1", + "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + self.assertTrue(out["probe"]["alpha"]) + self.assertTrue(out["checks"]["pixfmt"]["pass"]) + + def test_opaque_file_skips_checkerboard(self): + r, out = run([str(self.opaque), "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 0) + self.assertFalse(out["probe"]["alpha"]) + self.assertFalse(out["checkerboard"]) + + def test_checker_flag_forces_checkerboard(self): + r, out = run([str(self.opaque), "--checker", "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 0) + self.assertTrue(out["checkerboard"]) + + def test_meta_json_contract(self): + meta = Path(self.tmp.name) / "meta.json" + meta.write_text(json.dumps({"pixfmt": "prores4444", "res": "320x180", + "fps": 30, "dur": 2})) + r, out = run([str(self.prores), "--meta", str(meta), + "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + self.assertEqual(set(out["checks"]), {"pixfmt", "res", "fps", "dur"}) + self.assertTrue(out["ok"]) + + def test_explicit_flag_overrides_meta(self): + meta = Path(self.tmp.name) / "meta-wrong.json" + meta.write_text(json.dumps({"res": "1920x1080"})) + r, out = run([str(self.prores), "--meta", str(meta), + "--expect-res", "320x180", "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + self.assertTrue(out["checks"]["res"]["pass"]) + + def test_missing_input_is_hard_error(self): + r, out = run(["/nonexistent/render.mov"]) + self.assertEqual(r.returncode, 2) + self.assertFalse(out["ok"]) + self.assertIn("error", out) + + def test_json_output_parses_on_failure(self): + r, out = run([str(self.prores), "--expect-fps", "60", + "--out-dir", str(self._out_dir())]) + self.assertNotEqual(r.returncode, 0) + self.assertIsInstance(out, dict) # json.loads already succeeded in run() + + def test_failed_yuva420p_check_on_opaque_file_carries_hint(self): + r, out = run([str(self.opaque), "--pixfmt", "yuva420p", + "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 1) + self.assertFalse(out["checks"]["pixfmt"]["pass"]) + self.assertIn("hint", out["checks"]["pixfmt"]) + self.assertIn("libvpx-vp9", out["checks"]["pixfmt"]["hint"]) + + def test_transcode_webm_refuses_opaque_input(self): + webm = Path(self.tmp.name) / "refused" / "out.webm" + r, out = run([str(self.opaque), "--transcode-webm", str(webm), + "--out-dir", str(self._out_dir())]) + self.assertEqual(r.returncode, 2) + self.assertFalse(out["ok"]) + self.assertIn("no alpha", out["error"]) + self.assertFalse(webm.exists()) + + +@unittest.skipUnless(HAVE_VP9, "ffmpeg libvpx-vp9 encoder not available") +class TestWebmTranscode(unittest.TestCase): + tmp: tempfile.TemporaryDirectory + + @classmethod + def setUpClass(cls): + cls.tmp = tempfile.TemporaryDirectory() + root = Path(cls.tmp.name) + cls.prores = root / "master.mov" + make_fixture(cls.prores, codec_args=["-c:v", "prores_ks", "-profile:v", "4444", + "-pix_fmt", "yuva444p10le"], dur=1) + + @classmethod + def tearDownClass(cls): + cls.tmp.cleanup() + + def test_transcode_produces_verified_vp9_alpha_webm(self): + root = Path(self.tmp.name) + webm = root / "deliver" / "stinger.webm" + r, out = run([str(self.prores), "--transcode-webm", str(webm), + "--expect-res", "320x180", "--frames", "2", + "--out-dir", str(root / "_verify")]) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + self.assertTrue(out["ok"]) + self.assertTrue(webm.is_file()) + self.assertEqual(out["transcoded"], str(webm)) + self.assertEqual(out["input"], str(webm)) # checks ran on the WebM + # pixfmt expectation defaulted to yuva420p and passed + self.assertEqual(out["checks"]["pixfmt"]["expected"], "yuva420p") + self.assertTrue(out["checks"]["pixfmt"]["pass"]) + self.assertEqual(out["probe"]["codec"], "vp9") + self.assertTrue(out["probe"]["alpha"]) + self.assertTrue(out["checkerboard"]) + self.assertEqual(len(out["frames"]), 2) + + def test_explicit_pixfmt_flag_overrides_webm_default(self): + root = Path(self.tmp.name) + webm = root / "deliver" / "override.webm" + r, out = run([str(self.prores), "--transcode-webm", str(webm), + "--pixfmt", "prores4444", + "--out-dir", str(root / "_verify2")]) + self.assertEqual(r.returncode, 1) # a vp9 webm is not prores4444 + self.assertEqual(out["checks"]["pixfmt"]["expected"], "prores4444") + self.assertFalse(out["checks"]["pixfmt"]["pass"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/skills/module-help.csv b/skills/module-help.csv index 604048e..6582dde 100644 --- a/skills/module-help.csv +++ b/skills/module-help.csv @@ -13,6 +13,6 @@ BMad Manticore,mc-graphics,Build Graphics,GX,"Execute the approved beat table in BMad Manticore,mc-assets,Farm Assets,FA,"Source and farm the stills and b-roll the beat table calls for through registered CLI tools (metered APIs opt-in), real verified imagery first.",,,3-graphics,mc-beats,mc-package,false,projects-path,*/assets/manifest.json BMad Manticore,mc-audio,Farm Sound,AU,"Service skill, no stage or gate: local-first TTS narration and two-host dialogue (Kokoro-82M), instrumental beds (MusicGen-small), SFX (AudioLDM2). Called from graphics, stream packs, and voiceover narration, or directly.",,,anytime,,,false,,*/manifest.json BMad Manticore,mc-ograf,OGraf Graphics,OG,"Service skill: editable broadcast graphics where the target supports them (DaVinci Resolve 21+ editor lane, OBS/SPX-GC live lane). Everyone else gets baked alpha.",,,anytime,,,false,, -BMad Manticore,mc-package,Package,PK,"Titles, thumbnails verified at 120px, description, CTAs, dual-timeline chapters, series A/B pairs, live-event mode. May start any time after gate 1; offer it during dead time between stages.",,,4-package,mc-outline,mc-retro,false,projects-path,*/packaging/* +BMad Manticore,mc-package,Package,PK,"Titles, thumbnails verified at 120px, description, CTAs, dual-timeline chapters, SRT/VTT captions and a publishable transcript from the edited timeline, series A/B pairs, live-event mode. May start any time after gate 1; offer it during dead time between stages.",,,4-package,mc-outline,mc-retro,false,projects-path,*/packaging/* BMad Manticore,mc-stream-pack,Stream Pack,LS,"A complete branded livestream asset pack for OBS (scenes, stinger, lower thirds) from brand tokens; the livestream-pack format lane.",,,anytime,,,false,projects-path,*/graphics/scenes/* BMad Manticore,mc-retro,Retro,RT,"After publishing: one round of notes edits the format profile, the bibles, and the brand files so the next video starts smarter, then the post-publish wrap.",,,5-wrap,mc-package,,false,brand-path,production-bible.md