diff --git a/.claude/commands/fk-change-provider.md b/.claude/commands/fk-change-provider.md index 9639d96ad..38ac02551 100644 --- a/.claude/commands/fk-change-provider.md +++ b/.claude/commands/fk-change-provider.md @@ -1,4 +1,4 @@ -fk-change-provider — View & Switch AI CLI Provider (Video Review) +fk-change-provider — View & Switch the Reviewer for a Role Read and follow the instructions in skills/fk-change-provider.md diff --git a/.claude/commands/fk-dashboard.md b/.claude/commands/fk-dashboard.md index a2ec9c86f..abc8cf2d2 100644 --- a/.claude/commands/fk-dashboard.md +++ b/.claude/commands/fk-dashboard.md @@ -1,30 +1,5 @@ Show live GLA status in Claude Code statusline. + +Read and follow the instructions in skills/fk-dashboard.md -Usage: `/fk-dashboard` - -The GLA statusline shows real-time project status at the bottom of Claude Code: - -``` -Opus 4.6 (1M ctx) ctx:14% rl:18%/5h 67%/7d | GLA: ✓ext Operation Hormu 40sc img:40 vid:40 4K:26 ▶0/5 -``` - -## What it shows - -- **OMC info**: model, context usage, rate limits (5h & 7d) -- **GLA info**: extension status, project name, scene count, image/video/4K progress, worker slots - -## Setup - -Statusline is auto-configured by `bash setup.sh`. To manually configure: - -```bash -# In .claude/settings.local.json, add: -{ - "statusLine": { - "type": "command", - "command": "/scripts/statusline.sh" - } -} -``` - -Requires `jq` and `curl` installed. +Arguments: $ARGUMENTS diff --git a/.claude/commands/fk-doctor.md b/.claude/commands/fk-doctor.md index 813338ea9..e968e50d8 100644 --- a/.claude/commands/fk-doctor.md +++ b/.claude/commands/fk-doctor.md @@ -1,4 +1,4 @@ -Diagnose any FlowKit error and prescribe a fix. Knows the full error taxonomy across Flow, extension, FastAPI, worker, and YouTube. +Diagnose any FlowKit error and prescribe a fix. Knows the full error taxonomy across Google Flow, the Chrome extension, the FastAPI layer, the worker, and the YouTube upload pipeline. Read and follow the instructions in skills/fk-doctor.md diff --git a/.claude/commands/fk-gen-chain-videos.md b/.claude/commands/fk-gen-chain-videos.md index 9cf64ef06..0a763cc2d 100644 --- a/.claude/commands/fk-gen-chain-videos.md +++ b/.claude/commands/fk-gen-chain-videos.md @@ -1,113 +1,5 @@ - Generate videos with automatic scene chaining (start+end frame transitions). + +Read and follow the instructions in skills/fk-gen-chain-videos.md -Usage: `/gen-chain-videos ` - -This creates smooth transitions between scenes in a chain by using the **NEXT scene's image as the endImage** of the current scene's video, so the last frame of scene N matches the first frame of scene N+1 → seamless concat. - -## How chaining works - -For any scene that has a CHILD in the chain (i.e. some other scene with `parent_scene_id == this.id`): - -``` -Scene N (chain head or middle): startImage = sceneN.image, endImage = sceneN+1.image - → video plays FROM sceneN.image TO sceneN+1.image (the next scene's image) - -Scene N+1 (next in chain): startImage = sceneN+1.image, endImage = sceneN+2.image (if N+2 exists) - → continues the chain - -Last scene in chain (no child): startImage = lastScene.image, NO endImage - → plain frame_2_video, no chained transition -``` - -**Key invariant**: at the cut between two consecutive chain scenes, `sceneN.video.last_frame == sceneN+1.image == sceneN+1.video.first_frame` → **concat is seamless**. - -The `endImage` is the **CHILD scene's image** (the next scene's image_media_id), NOT the parent's. Chain heads (ROOT scenes that have a child) DO need an endImage; only chain tails (last scene in chain) and standalone ROOT scenes leave `endImage` null. - -## Step 1: Pre-check - -```bash -# All scene images must be ready with UUID media_ids -curl -s "http://127.0.0.1:8100/api/scenes?video_id=" -``` - -ABORT if any scene is missing `${ori}_image_media_id` (UUID). - -## Step 2: Set up end_scene_media_ids for chaining - -For each scene that has a CHILD in the chain (i.e. some other scene's `parent_scene_id == this.id`), set its `${ori}_end_scene_media_id` to that **child** scene's `${ori}_image_media_id`: - -```bash -curl -X PATCH http://127.0.0.1:8100/api/scenes/ \ - -H "Content-Type: application/json" \ - -d '{"${ori}_end_scene_media_id": ""}' -``` - -Logic: -1. Sort scenes by `display_order` -2. Build a child map: `parent_id -> child_scene` (scan all scenes, key by `parent_scene_id`) -3. For each scene `S`: - - If `S` has a child in the map → set `${ori}_end_scene_media_id = child.${ori}_image_media_id` - - Else (no child = chain tail or standalone ROOT) → leave `${ori}_end_scene_media_id` null - -**Affected scenes** (those that need an `end_scene_media_id`): chain heads (ROOT-with-child) AND chain middles (CONTINUATION-with-child). NOT chain tails, NOT standalone ROOTs. - -**Common mistake**: setting `end_scene_media_id` to the PARENT's image. That makes scene N's video end at scene N-1's frame — when concatenated forward (N then N+1), the cut between N's last frame and N+1's first frame is hard, defeating the chain. The correct frame at the end of scene N's video is `sceneN+1.image`. - -## Step 3: Submit ALL video requests at once - -The server handles throttling automatically (max 5 concurrent, 10s cooldown). The worker reads `${ori}_end_scene_media_id` from each scene (set in Step 2) and passes it as `endImage` to the API. This triggers `start_end_frame_2_video` (i2v_fl) instead of plain `frame_2_video` (i2v). - -```bash -curl -X POST http://127.0.0.1:8100/api/requests/batch \ - -H "Content-Type: application/json" \ - -d '{ - "requests": [ - {"type": "GENERATE_VIDEO", "scene_id": "", "project_id": "", "video_id": "", "orientation": "${ORI}"}, - {"type": "GENERATE_VIDEO", "scene_id": "", "project_id": "", "video_id": "", "orientation": "${ORI}"} - ] - }' -``` - -Build the `requests` array from ALL scenes in display_order. Do NOT manually batch or loop. - -Poll aggregate status every 30s until done: - -```bash -curl -s "http://127.0.0.1:8100/api/requests/batch-status?video_id=&type=GENERATE_VIDEO" -# Wait for: "done": true -# If "all_succeeded": false → some failed, check individual failures -``` - -## Known Limitation: Concat Gap - -**Problem:** When concatenating chained videos, the endImage frames of scene N overlap with the startImage frames of scene N+1 — both are the same image. This produces **10-16 static/duplicate frames (~0.4-0.7s)** at each cut point where nothing moves. - -``` -Scene 1 video: [action...] [endImage = scene2 still] ← 10-16 frames -Scene 2 video: [startImage = scene2 still] [action...] ← 10-16 frames -Concat result: ...action → [~0.8-1.4s static gap] → action... -``` - -**Mitigations:** -- **Trim overlap** — use `trim_start` / `trim_end` on scenes to cut the static frames before concat. Typically trim 0.4-0.7s from the end of scene N and/or the start of scene N+1. -- **Don't overuse chaining** — only chain scenes that truly need smooth visual continuity (same location, continuous action). For scene changes (new location, time jump), use ROOT without endImage — a hard cut is more natural. -- **Mix techniques** — alternate between chained (CONTINUATION) and unchained (ROOT) scenes. Hard cuts between different locations feel intentional; gaps between continuous action feel broken. - -**When to chain vs not:** - -| Situation | Recommendation | -|-----------|---------------| -| Same location, continuous action | CONTINUATION (chain) | -| Location change, time jump | ROOT (hard cut — no gap) | -| Dramatic moment, reaction shot | INSERT (hard cut — intentional) | -| Dream/flashback transition | ROOT or R2V (stylistic break) | - -## Step 4: Output - -Print table: -| Scene | Order | Chain | endImage from | video_status | Duration | -|-------|-------|-------|---------------|-------------|----------| - -Print: "Chained videos ready. Run /fk-concat to merge." -Remind: "Check concat gaps — trim 0.4-0.7s overlap at chain boundaries if needed." +Arguments: $ARGUMENTS diff --git a/.claude/commands/fk-gen-music.md b/.claude/commands/fk-gen-music.md index 6c9b524d5..6a1a8559c 100644 --- a/.claude/commands/fk-gen-music.md +++ b/.claude/commands/fk-gen-music.md @@ -1,244 +1,5 @@ -# fk-gen-music — Generate Music via Suno +fk-gen-music — Generate Music via Suno + +Read and follow the instructions in skills/fk-gen-music.md -Generate background music or songs for video projects using the Suno API (sunoapi.org). - -## Prerequisites - -- GLA server running: `curl http://127.0.0.1:8100/health` -- Suno API key configured: `export SUNO_API_KEY=your-key` -- Get API key at https://sunoapi.org/api-key - -## Workflow - -### Step 1: Choose a Template (Optional) - -Browse available song templates to find the right style: - -```bash -# List all templates -curl -s http://127.0.0.1:8100/api/music/templates | python3 -m json.tool - -# Get a specific template (see style, tags, example lyrics) -curl -s http://127.0.0.1:8100/api/music/templates/cinematic_epic -``` - -Available categories: Children & Family, Love & Romance, Pop, Rock, Hip-Hop, Electronic, Country & Folk, Cinematic, Motivational. - -### Step 2: Generate Music - -**Three modes:** - -#### Mode A: Template-based (recommended for video projects) - -Use a template to auto-fill style tags. Provide custom lyrics or let the template's example lyrics run: - -```bash -# With custom lyrics + template style -curl -X POST http://127.0.0.1:8100/api/music/generate \ - -H "Content-Type: application/json" \ - -d '{ - "template_id": "cinematic_epic", - "prompt": "[Verse]\nFrom the shadows they emerge\nAcross the ancient sea\n[Chorus]\nRise, rise to glory\nThe world will hear our story", - "title": "Rise to Glory", - "poll": true - }' - -# Template defaults (uses example lyrics + suno_tags) -curl -X POST http://127.0.0.1:8100/api/music/generate \ - -H "Content-Type: application/json" \ - -d '{ - "template_id": "lullaby_gentle", - "title": "Goodnight Luna", - "poll": true - }' -``` - -#### Mode B: Custom (full control) - -Provide your own lyrics and style tags: - -```bash -curl -X POST http://127.0.0.1:8100/api/music/generate \ - -H "Content-Type: application/json" \ - -d '{ - "prompt": "[Verse]\nWalking through the rain\nSearching for the light\n[Chorus]\nWe will find our way\nThrough the darkest night", - "style": "lo-fi hip hop, chill, piano, rainy, 90 BPM", - "title": "Rainy Night", - "poll": true - }' -``` - -#### Mode C: Description (AI writes everything) - -Just describe what you want in natural language: - -```bash -curl -X POST http://127.0.0.1:8100/api/music/generate \ - -H "Content-Type: application/json" \ - -d '{ - "prompt": "an epic orchestral track for a military documentary about naval battles, dramatic and heroic", - "instrumental": true, - "custom_mode": false, - "poll": true - }' -``` - -### Step 3: Check Results - -Each generation produces **2 clips** (variations). When `poll: true`, the response waits for completion. - -```bash -# If poll was false, check status manually: -curl -s http://127.0.0.1:8100/api/music/tasks/ - -# Poll until complete: -curl -X POST http://127.0.0.1:8100/api/music/tasks//poll -``` - -**Task statuses:** `PENDING` → `GENERATING` → `SUCCESS` or `FAILED` - -### Step 4: Download - -```bash -curl -X POST http://127.0.0.1:8100/api/music/tasks//download -# Returns: {"task_id": "...", "downloaded": [{"clip_id": "...", "path": "output/_shared/music/title_abcd1234.mp3", ...}]} -``` - -### Step 5: Use in Video (Optional) - -Add the downloaded music as background for your concat video: - -```bash -# Get project output directory -PROJ_OUT=$(curl -s http://127.0.0.1:8100/api/projects//output-dir) -OUTDIR=$(echo "$PROJ_OUT" | python3 -c "import sys,json; print(json.load(sys.stdin)['path'])") -SLUG=$(echo "$PROJ_OUT" | python3 -c "import sys,json; print(json.load(sys.stdin)['slug'])") - -# Mix music with concat video using ffmpeg -ffmpeg -y -i "${OUTDIR}/${SLUG}_final.mp4" -i output/_shared/music/track.mp3 \ - -filter_complex "[1:a]volume=0.3[bg]; [0:a][bg]amix=inputs=2:duration=first[out]" \ - -map 0:v -map "[out]" -c:v copy -c:a aac "${OUTDIR}/${SLUG}_with_music.mp4" -``` - -## Extend a Track - -Continue or extend an existing clip from a previous generation: - -```bash -curl -X POST http://127.0.0.1:8100/api/music/extend \ - -H "Content-Type: application/json" \ - -d '{ - "audio_id": "", - "prompt": "[Chorus]\nKeep the fire burning bright", - "continue_at": 120, - "poll": true - }' -``` - -## Vocal Removal (Stem Separation) - -Separate vocals from instrumental: - -```bash -curl -X POST http://127.0.0.1:8100/api/music/vocal-removal \ - -H "Content-Type: application/json" \ - -d '{ - "task_id": "", - "audio_id": "", - "poll": true - }' -# Returns: instrumental_url + vocal_url -``` - -## Convert to WAV - -Get lossless WAV from a generated clip: - -```bash -curl -X POST http://127.0.0.1:8100/api/music/convert-to-wav \ - -H "Content-Type: application/json" \ - -d '{ - "task_id": "", - "audio_id": "", - "poll": true - }' -``` - -## Generate Lyrics Only - -Ask Suno's AI to write lyrics from a description, optionally guided by a template: - -```bash -curl -X POST http://127.0.0.1:8100/api/music/generate-lyrics \ - -H "Content-Type: application/json" \ - -d '{ - "prompt": "a song about a cat astronaut exploring a candy planet", - "template_id": "children_adventure", - "poll": true - }' -``` - -## Check Credits - -```bash -curl -s http://127.0.0.1:8100/api/music/credits -``` - -## API Reference - -| Endpoint | Method | Description | -|----------|--------|-------------| -| `/api/music/templates` | GET | List all song templates | -| `/api/music/templates/{id}` | GET | Get template details | -| `/api/music/generate` | POST | Generate music (returns taskId, 2 clips) | -| `/api/music/tasks/{id}` | GET | Get task status + clips | -| `/api/music/tasks/{id}/poll` | POST | Poll task until complete | -| `/api/music/tasks/{id}/download` | POST | Download all clips to local | -| `/api/music/generate-lyrics` | POST | Generate lyrics from prompt | -| `/api/music/extend` | POST | Extend/continue existing track | -| `/api/music/vocal-removal` | POST | Separate vocals from instrumental | -| `/api/music/convert-to-wav` | POST | Convert clip to WAV format | -| `/api/music/credits` | GET | Check Suno credits/quota | - -## Models - -| Model | Max Duration | Notes | -|-------|-------------|-------| -| V4 | 4 min | Highest audio quality | -| V4_5 | 8 min | Superior genre blending | -| V4_5PLUS | 8 min | Richer sound | -| V4_5ALL | 8 min | Better song structure | -| V5 | 8 min | Faster generation | -| V5_5 | 8 min | Custom model tailoring | - -Default: `V4` (set via `SUNO_MODEL` env var). - -## Template → Suno Mapping - -| Template Field | Suno API Field | Purpose | -|---------------|---------------|---------| -| `suno_tags` | `style` | Musical style descriptors | -| `example_lyrics` | `prompt` | Lyrics with section markers | -| `vocal_style` | (embedded in style) | Voice character | -| `bpm_range` | (embedded in style) | Tempo | - -## Tips - -- Each generation costs ~10 credits (5 per clip) and produces 2 variations -- Use `instrumental: true` for background music (no vocals) -- `poll: true` waits for completion (~30-120s) — use for scripted workflows -- `poll: false` returns immediately with taskId — use for interactive/async workflows -- For video background music, `cinematic_epic` or `cinematic_emotional` templates work well -- For children's content, use `lullaby_gentle`, `children_adventure`, or `nursery_rhyme` -- Generated files are stored by Suno for 15 days — download promptly - -## Important Notes - -- Suno generates ~2-8 min tracks (depends on model). For shorter clips, trim with ffmpeg -- Each generation creates 2 clip variations — listen and pick the best one -- Credits are per-account. Check `/api/music/credits` before batch generation -- The `[Verse]`, `[Chorus]`, `[Bridge]`, `[Outro]` markers in lyrics control song structure -- Use `[Instrumental]`, `[Soft]`, `[Powerful]`, `[Whispered]` tags for dynamics -- Style field: max 200 chars (V4) or 1000 chars (V4.5+) -- Title field: max 80 chars (V4/V4_5ALL) or 100 chars (others) -- Prompt field: max 3000 chars (V4) or 5000 chars (V4.5+) +Arguments: $ARGUMENTS diff --git a/.claude/commands/fk-gen-text-overlays.md b/.claude/commands/fk-gen-text-overlays.md index 498669005..1e0eeab33 100644 --- a/.claude/commands/fk-gen-text-overlays.md +++ b/.claude/commands/fk-gen-text-overlays.md @@ -1,144 +1,5 @@ -# fk-gen-text-overlays — Generate Text Overlays from Narrator Text +fk-gen-text-overlays — Generate Text Overlays from Narrator Text + +Read and follow the instructions in skills/fk-gen-text-overlays.md -Analyze narrator text for each scene and extract key data points (dates, locations, statistics, milestones, costs) to create `text_overlays.json` for `/fk-concat-fit-narrator`. - -Usage: `/fk-gen-text-overlays [--language vi]` - -- `video_id` — the video to generate overlays for -- `--language` — target language code (default: auto-detect from narrator text). **All overlay text MUST be in this language with proper diacritics/characters.** - -## Step 1: Load project, video, scenes - -```bash -curl -s http://127.0.0.1:8100/api/videos/ -# Get project_id -curl -s http://127.0.0.1:8100/api/projects/ -curl -s "http://127.0.0.1:8100/api/scenes?video_id=" -curl -s http://127.0.0.1:8100/api/projects//output-dir -``` - -Sort scenes by `display_order`. Note `OUTDIR` from output-dir response. - -## Step 2: Detect target language - -If `--language` not provided, auto-detect from the first scene's `narrator_text`: -- Vietnamese diacritics (ắ, ề, ổ, ử, ơ, etc.) → `vi` -- Spanish characters (ñ, á, é, etc.) → `es` -- Default → `en` - -**CRITICAL RULE: All overlay text MUST be written in the detected language with full diacritics and proper characters. NEVER strip diacritics or romanize.** - -Examples: -- Vietnamese: "Chiến dịch Tapalpa" NOT "Chien dich Tapalpa" -- Vietnamese: "Tiền thưởng" NOT "Tien thuong" -- Spanish: "Operación Tapalpa" NOT "Operacion Tapalpa" - -## Step 3: Analyze each scene's narrator text - -For each scene with `narrator_text`, extract data points that fit these overlay categories: - -### Overlay styles - -| Style | What to extract | Color | -|-------|----------------|-------| -| `date` | Dates, times, timestamps, time references | Cyan | -| `name` | Locations, operation names, people, organizations | White | -| `stat` | Numbers, quantities, distances, technology milestones | White | -| `cost` | Casualties, monetary values, damage, human cost | Gold | - -### Extraction rules - -1. **NOT every scene needs overlays.** Only add overlays when there's a clear, impactful data point. -2. **Target ~40-50% of scenes** having overlays (e.g., 25-30 out of 60 scenes). -3. **Max 2 text items per scene** — one primary, one secondary. -4. **Keep text SHORT** — max 40 characters per line. Abbreviate if needed. -5. **Prioritize hard data** — numbers, dates, names, costs over descriptive text. -6. **Avoid repeating** the same information across adjacent scenes. - -### What makes a good overlay - -Good: -- "22 tháng 2, 2026" (specific date) -- "Tiền thưởng: $15,000,000" (hard number) -- "Guadalajara, Jalisco" (specific location) -- "25 lính hy sinh — 70+ người chết" (casualty stats) -- "Net-Centric Warfare" (tech milestone — keep original English terms) - -Bad: -- "Mọi thứ thay đổi" (too vague) -- "Tình hình rất căng thẳng" (no data point) -- "Chiến dịch tiếp tục" (no specific info) - -### Special handling - -- **Technical terms** (Net-Centric Warfare, RPG, DEA, CJNG, FIFA): keep in original language/acronym. -- **People names**: keep original spelling (El Mencho, El Pelón, Nemesio Oseguera Cervantes). -- **Currency**: use `$X,XXX,XXX` format with dollar sign. -- **Casualty numbers**: pair with context (e.g., "25 lính hy sinh" not just "25"). - -## Step 4: Generate text_overlays.json - -Build JSON object where keys are scene `display_order` (as string), values are arrays of overlay items: - -```json -{ - "0": [ - {"text": "22 tháng 2, 2026", "style": "date"}, - {"text": "Tapalpa, Jalisco, Mexico", "style": "name"} - ], - "8": [ - {"text": "Mật danh: Chiến dịch Tapalpa", "style": "name"}, - {"text": "6 trực thăng — hàng trăm đặc nhiệm", "style": "stat"} - ] -} -``` - -## Step 5: Validate and save - -Before saving, validate: -1. All text is in the correct target language with proper diacritics -2. No text exceeds 40 characters -3. No scene has more than 2 overlay items -4. Scene indices are valid (within 0 to scene_count-1) -5. All styles are one of: `date`, `name`, `stat`, `cost` - -Print validation table: - -``` -Scene | Style | Text (40 char max) -------|-------|------------------- - 000 | date | 22 tháng 2, 2026 - 000 | name | Tapalpa, Jalisco, Mexico - 008 | name | Mật danh: Chiến dịch Tapalpa - ... -Total: X overlays across Y scenes (Z% coverage) -``` - -Save to `${OUTDIR}/text_overlays.json`. - -```bash -# Verify JSON is valid -python3 -c "import json; d=json.load(open('${OUTDIR}/text_overlays.json')); print(f'{len(d)} scenes with overlays, {sum(len(v) for v in d.values())} total items')" -``` - -## Step 6: Report - -``` -Text overlays generated: - Language: - Scenes with overlays: X/Y (Z%) - Total overlay items: N - Styles: date=A, name=B, stat=C, cost=D - Output: ${OUTDIR}/text_overlays.json - -Ready for /fk-concat-fit-narrator -``` - -## Common Issues - -| Issue | Cause | Fix | -|-------|-------|-----| -| Text without diacritics | Language not detected | Pass `--language vi` explicitly | -| Too many overlays | Over-extraction | Aim for 40-50% scene coverage | -| Text too long | Not abbreviated | Keep under 40 chars, use short forms | -| Overlays feel repetitive | Adjacent scenes have similar data | Space out overlays, skip adjacent scenes | +Arguments: $ARGUMENTS diff --git a/.claude/commands/fk-import-voice.md b/.claude/commands/fk-import-voice.md index 6772e0fa2..c1b479838 100644 --- a/.claude/commands/fk-import-voice.md +++ b/.claude/commands/fk-import-voice.md @@ -1,115 +1,5 @@ -# fk-import-voice — Import Existing Voice as Template +fk-import-voice — Import Existing Voice as Template + +Read and follow the instructions in skills/fk-import-voice.md -Register an existing WAV file as a reusable voice template for narration. Auto-transcribes the audio and registers it in the template system. - -## When to Use - -- You have a real voice recording (WAV) you want to use for narration -- You want to clone a specific voice rather than designing one with `instruct` -- Complementary to `/fk-gen-tts-template` which creates synthetic voices - -## Prerequisites - -- GLA server running: `curl http://127.0.0.1:8100/health` -- `faster-whisper` installed in `/opt/homebrew/bin/python3.10` -- WAV file placed in `output/_shared/tts_templates/` - -## Workflow - -### Step 1: Locate the WAV file - -```bash -ls -la output/_shared/tts_templates/*.wav -``` - -The user should have already placed their WAV file here. File should be: -- Format: WAV (any encoding — pcm_s16le, pcm_f32le, etc.) -- Duration: 3-10s ideal for voice cloning -- Content: Clear speech, minimal background noise - -### Step 2: Transcribe with faster-whisper - -Use `large-v3` model for accurate transcription. **Always use `/opt/homebrew/bin/python3.10`** — it has torch + faster-whisper installed. - -```bash -/opt/homebrew/bin/python3.10 -c " -from faster_whisper import WhisperModel -model = WhisperModel('large-v3', device='cpu') -segments, info = model.transcribe('', beam_size=5) -print(f'Language: {info.language} ({info.language_probability:.0%})') -for seg in segments: - print(seg.text.strip()) -" 2>&1 | grep -v '\[ctranslate2\]' -``` - -**IMPORTANT:** This runs on CPU and takes ~60s for large-v3 model loading + transcription. Be patient. - -Show the transcript to the user and ask them to confirm or correct it. Accurate `ref_text` is critical for voice cloning quality. - -### Step 3: Register template in templates.json - -After user confirms transcript: - -```bash -python3 -c " -import json -from pathlib import Path - -templates_dir = Path('output/_shared/tts_templates') -meta_file = templates_dir / 'templates.json' -wav_path = str(templates_dir / '.wav') - -meta = json.loads(meta_file.read_text()) if meta_file.exists() else {} -meta[''] = { - 'name': '', - 'audio_path': wav_path, - 'text': '', - 'instruct': '', - 'duration': , -} -meta_file.write_text(json.dumps(meta, indent=2, ensure_ascii=False)) -print('Template registered') -" -``` - -Template name: derive from filename (e.g., `vi_male_narrator.wav` → `vi_male_narrator`). - -### Step 4: Verify via API - -```bash -curl -s http://127.0.0.1:8100/api/tts/templates -# Should list the new template -``` - -### Step 5: Test voice clone - -Generate a short test sentence using the imported voice: - -```bash -curl -s -X POST http://127.0.0.1:8100/api/tts/generate \ - -H "Content-Type: application/json" \ - -d '{ - "text": "", - "instruct": "", - "ref_audio": "", - "ref_text": "" - }' -``` - -Play the output for user to verify voice quality matches the original. - -## Naming Convention - -| Language | Template name pattern | Example | -|----------|----------------------|---------| -| Vietnamese | `vi_{gender}_narrator` | `vi_male_narrator` | -| English | `en_{gender}_narrator` | `en_female_narrator` | -| Japanese | `ja_{gender}_narrator` | `ja_male_narrator` | -| Multi-lang | `{primary_lang}_{gender}_narrator` | `vi_male_narrator` | - -## Notes - -- **Transcript accuracy matters** — `ref_text` must match the audio closely for good voice cloning. Always confirm with the user. -- **CPU only** — whisper large-v3 and OmniVoice both run on CPU. MPS produces artifacts. -- **3-10s audio** — shorter clips lack enough voice characteristics; longer clips slow down cloning. -- **One template per voice** — don't register multiple templates for the same voice. Update instead. +Arguments: $ARGUMENTS diff --git a/.claude/commands/fk-monitor.md b/.claude/commands/fk-monitor.md index 36a245873..4de6a17fe 100644 --- a/.claude/commands/fk-monitor.md +++ b/.claude/commands/fk-monitor.md @@ -1,415 +1,5 @@ -# fk-monitor — Full Pipeline Monitor +fk-monitor — Full Pipeline Monitor + +Read and follow the instructions in skills/fk-monitor.md -Poll project pipeline status every N seconds, detect state changes across all stages, send Telegram notifications, and optionally auto-download completed upscales. - -Usage: `/fk-monitor [project_id] [orientation] [--download] [--interval N]` - -- `project_id` — omit to use most recently active project -- `orientation` — `HORIZONTAL` or `VERTICAL` (auto-detected from video.orientation if omitted) -- `--download` — auto-download new upscales to `output/{slug}/4k/` -- `--interval N` — poll interval in seconds (default: 30) - -## When to Use - -- Long-running batch: submitted image/video/upscale jobs and want to be notified when stages complete -- Unattended generation: step away and get Telegram pings at milestones -- Debugging: spot stalled workers or failed requests early -- Auto-download: pull 4K upscales as they finish without manual polling - -## Prerequisites - -```bash -curl -s http://127.0.0.1:8100/health -# extension_connected must be true -``` - -Telegram notifications require the `mcp__telegram__reply` tool to be available in this session. - ---- - -## Step 1: Resolve project and slug - -If no `project_id` supplied, fetch the most recent project: - -```bash -curl -s http://127.0.0.1:8100/api/projects -# Use the last item in the array → id, name -``` - -Derive slug from project name: - -```python -import unicodedata, re - -def slugify(name): - # Strip diacritics - normalized = unicodedata.normalize('NFD', name) - ascii_only = normalized.encode('ascii', 'ignore').decode() - # Lowercase, replace non-alphanum with underscore - slug = re.sub(r'[^a-z0-9]+', '_', ascii_only.lower()) - # Collapse and trim - slug = re.sub(r'_+', '_', slug).strip('_') - return slug -``` - -Examples: -- "My Project - Edition" → `my_project_edition` -- "Chiến dịch giải cứu F-15E" → `chien_dich_giai_cuu_f_15e` - -Get output dir (also creates directory structure): - -```bash -curl -s http://127.0.0.1:8100/api/projects//output-dir -# Returns: {"slug": "...", "path": "output/...", "meta": {...}} -``` - ---- - -## Step 2: Resolve video_id and scene count - -```bash -curl -s "http://127.0.0.1:8100/api/videos?project_id=" -# Use first video → id, get scene count - -curl -s "http://127.0.0.1:8100/api/scenes?video_id=" -# Total scene count = N -``` - ---- - -## Step 3: Get Telegram chat_id - -The monitor sends all notifications to one chat. Ask the user for their Telegram chat_id if not already known: - -``` -To send Telegram notifications, I need your Telegram chat_id. -You can find it by messaging @userinfobot on Telegram. -``` - -Store as `CHAT_ID` for all `mcp__telegram__reply` calls. - ---- - -## Step 4: Send start notification - -``` -mcp__telegram__reply( - chat_id=CHAT_ID, - text="🔍 Monitor started: \n" - "Orientation: | Interval: s\n" - "Stages: ref images, scene images, videos, 4K upscale, downloads, TTS" -) -``` - ---- - -## Step 5: Poll loop - -Run the poll cycle in a Python inline script. Each cycle: - -### 5a. Fetch all data - -```bash -# Fetch in parallel (run as background + wait) -curl -s http://127.0.0.1:8100/api/projects//characters > /tmp/fk_chars.json & -curl -s "http://127.0.0.1:8100/api/scenes?video_id=" > /tmp/fk_scenes.json & -curl -s "http://127.0.0.1:8100/api/requests?status=PENDING" > /tmp/fk_pending.json & -curl -s "http://127.0.0.1:8100/api/requests?status=PROCESSING" > /tmp/fk_processing.json & -curl -s "http://127.0.0.1:8100/api/requests?status=FAILED" > /tmp/fk_failed.json & -wait -``` - -### 5b. Compute snapshot - -Detect orientation: if not passed as argument, read from `GET /api/videos/{vid}` → `orientation` field. Fall back to `HORIZONTAL` only if API returns null. Set `PREFIX` = detected orientation lowercase. - -```python -import json, os, glob, time - -chars = json.load(open('/tmp/fk_chars.json')) -scenes = json.load(open('/tmp/fk_scenes.json')) -pending = json.load(open('/tmp/fk_pending.json')) -processing = json.load(open('/tmp/fk_processing.json')) -failed = json.load(open('/tmp/fk_failed.json')) - -PREFIX = video_orientation.lower() # from GET /api/videos/{vid} → orientation field; fall back to 'horizontal' -OUTDIR = 'output/' -TOTAL = len(scenes) -TOTAL_CHARS = len(chars) - -snapshot = { - # Stage 1: ref images - 'refs_done': sum(1 for c in chars if c.get('media_id')), - 'refs_total': TOTAL_CHARS, - - # Stage 2: scene images - 'images_done': sum(1 for s in scenes if s.get(f'{PREFIX}_image_status') == 'COMPLETED'), - 'images_total': TOTAL, - - # Stage 3: scene videos - 'videos_done': sum(1 for s in scenes if s.get(f'{PREFIX}_video_status') == 'COMPLETED'), - 'videos_total': TOTAL, - - # Stage 4: 4K upscale - 'upscales_done': sum(1 for s in scenes if s.get(f'{PREFIX}_upscale_status') == 'COMPLETED'), - 'upscales_total': TOTAL, - - # Stage 5: 4K downloads - 'downloads_done': len(glob.glob(f'{OUTDIR}/4k/scene_*.mp4')), - 'downloads_total': TOTAL, - - # Stage 6: TTS narration - 'tts_done': len(glob.glob(f'{OUTDIR}/tts/scene_*.wav')), - 'tts_total': TOTAL, - - # Worker queue - 'pending_count': len(pending) if isinstance(pending, list) else pending.get('total', 0), - 'processing_count': len(processing) if isinstance(processing, list) else processing.get('total', 0), - - # Failures - 'failed_count': len(failed) if isinstance(failed, list) else failed.get('total', 0), - 'failed_types': list({r.get('type','unknown') for r in (failed if isinstance(failed, list) else [])}), -} -``` - -### 5c. Diff with previous snapshot and emit notifications - -```python -# Milestone thresholds -MILESTONE_STEP = 10 - -def milestone_crossed(prev, curr, total): - """Returns milestone value if a multiple of MILESTONE_STEP was crossed.""" - if total == 0: - return None - prev_m = (prev // MILESTONE_STEP) * MILESTONE_STEP - curr_m = (curr // MILESTONE_STEP) * MILESTONE_STEP - if curr_m > prev_m and curr < total: - return curr_m - return None - -notifications = [] - -STAGES = [ - ('refs', 'Ref images', snapshot['refs_done'], snapshot['refs_total']), - ('images', 'Scene images', snapshot['images_done'], snapshot['images_total']), - ('videos', 'Scene videos', snapshot['videos_done'], snapshot['videos_total']), - ('upscales', '4K upscale', snapshot['upscales_done'], snapshot['upscales_total']), - ('downloads', '4K downloads', snapshot['downloads_done'], snapshot['downloads_total']), - ('tts', 'TTS narration', snapshot['tts_done'], snapshot['tts_total']), -] - -for key, label, curr, total in STAGES: - prev = prev_snapshot.get(f'{key}_done', 0) if prev_snapshot else 0 - - if total == 0: - continue - - # 100% completion - if curr == total and prev < total: - notifications.append(f'✅ {label} complete: {curr}/{total}') - - # Milestone every 10 - elif (m := milestone_crossed(prev, curr, total)): - notifications.append(f'🔄 {label}: {curr}/{total}') - -# Worker stall detection -if (snapshot['pending_count'] > 0 - and snapshot['processing_count'] == 0 - and time.time() - last_change_ts > 120): - notifications.append( - f'⚠️ Worker stalled! {snapshot["pending_count"]} requests pending but 0 processing.\n' - f'Check server logs — may need to restart the GLA server.' - ) - -# New failures -prev_failed = prev_snapshot.get('failed_count', 0) if prev_snapshot else 0 -if snapshot['failed_count'] > prev_failed: - new_n = snapshot['failed_count'] - prev_failed - types_str = ', '.join(snapshot['failed_types']) or 'unknown' - notifications.append(f'❌ {new_n} new failure(s) detected: {types_str}') - -# All complete check -all_done = all( - snapshot[f'{k}_done'] >= snapshot[f'{k}_total'] - for k in ['refs', 'images', 'videos'] - if snapshot[f'{k}_total'] > 0 -) -if all_done and not prev_all_done: - notifications.append( - '🎉 Project complete! All pipeline stages done.\n' - 'Ready for /fk-concat' - ) -``` - -### 5d. Send notifications - -For each notification string: - -``` -mcp__telegram__reply( - chat_id=CHAT_ID, - text="[]\n" + notification_text -) -``` - -Send a combined status summary every 5 cycles regardless of changes: - -``` -mcp__telegram__reply( - chat_id=CHAT_ID, - text=( - f"[] Status update\n" - f"Refs: {refs_done}/{refs_total}\n" - f"Images: {images_done}/{images_total}\n" - f"Videos: {videos_done}/{videos_total}\n" - f"4K upscale:{upscales_done}/{upscales_total}\n" - f"Downloads: {downloads_done}/{downloads_total}\n" - f"TTS: {tts_done}/{tts_total}\n" - f"Queue: {pending_count} pending / {processing_count} processing" - ) -) -``` - -### 5e. Auto-download (when --download) - -When `upscales_done` increases from previous cycle, find newly completed scenes and download: - -```python -# Get scenes with completed upscale not yet downloaded locally -newly_completed = [ - s for s in scenes - if s.get(f'{PREFIX}_upscale_status') == 'COMPLETED' - and s.get(f'{PREFIX}_upscale_url') - and not os.path.exists(f"{OUTDIR}/4k/scene_{s['display_order']:03d}_{s['id']}.mp4") -] -``` - -```bash -# For each newly completed scene: -mkdir -p output//4k -curl -s "" \ - -o "output//4k/scene__.mp4" -echo "Downloaded scene → output//4k/scene__.mp4" -``` - -After each download, verify with: - -```bash -ffprobe -v quiet -show_entries format=duration \ - -of csv=p=0 "output//4k/scene__.mp4" -# Should return a positive number (duration in seconds) -``` - ---- - -## Step 6: Stop conditions - -Stop the loop when ANY of these are true: - -1. **All upscaled + all downloaded** (if `--download`): `upscales_done == upscales_total AND downloads_done == upscales_total` -2. **All videos done** (if no `--download`): `videos_done == videos_total` -3. **User interrupts**: Ctrl+C or task cancelled -4. **Max iterations reached**: if `--max-iter N` was specified - -On stop, send final summary: - -``` -mcp__telegram__reply( - chat_id=CHAT_ID, - text=( - f"[] Monitor stopped\n" - f"Final state:\n" - f" Refs: {refs_done}/{refs_total}\n" - f" Images: {images_done}/{images_total}\n" - f" Videos: {videos_done}/{videos_total}\n" - f" 4K upscale:{upscales_done}/{upscales_total}\n" - f" Downloads: {downloads_done}/{downloads_total}\n" - f" TTS: {tts_done}/{tts_total}\n" - f" Failed: {failed_count}" - ) -) -``` - ---- - -## Full Poll Loop Structure - -```python -import time - -prev_snapshot = None -prev_all_done = False -last_change_ts = time.time() -cycle = 0 -INTERVAL = 30 # override with --interval N - -while True: - cycle += 1 - - # 1. Fetch all data (via curl to /tmp files) - # 2. Compute snapshot - snapshot = compute_snapshot() - - # 3. Detect change for stall tracking - if snapshot != prev_snapshot: - last_change_ts = time.time() - - # 4. Diff + emit notifications - notifications = diff_snapshots(prev_snapshot, snapshot, last_change_ts) - for n in notifications: - send_telegram(CHAT_ID, f'[{project_name}]\n{n}') - - # 5. Periodic summary every 5 cycles - if cycle % 5 == 0: - send_telegram(CHAT_ID, build_summary(snapshot)) - - # 6. Auto-download if --download - if download_flag: - download_new_upscales(snapshot, scenes) - - # 7. Check stop conditions - if should_stop(snapshot, download_flag): - send_final_summary(CHAT_ID, snapshot) - break - - prev_snapshot = snapshot - prev_all_done = all_done_check(snapshot) - time.sleep(INTERVAL) -``` - ---- - -## Console Output (each cycle) - -Print a compact status line to console each cycle so progress is visible: - -``` -[cycle 12 | 14:32:01] - Refs: 5/5 ✓ | Images: 23/40 | Videos: 18/40 | 4K: 10/40 - Queue: 3 pending / 2 processing | Failed: 0 - Next poll in 30s... -``` - ---- - -## Common Issues - -| Issue | Cause | Fix | -|-------|-------|-----| -| Stall notification fires immediately | Server just restarted, processing=0 briefly | Stall threshold is 120s — brief gaps won't trigger | -| Downloads fail with 403 | GCS signed URL expired (~8h TTL) | Regenerate via `REGENERATE_IMAGE` then re-upscale | -| `media_id` missing on refs | Ref image generation failed | Check `/api/requests?status=FAILED`, resubmit `GENERATE_CHARACTER_IMAGE` | -| Scene count shows 0 | Wrong `video_id` | Verify `GET /api/videos?project_id=` returns correct video | -| Telegram notifications not sent | `mcp__telegram__reply` not available | Ensure Telegram MCP is connected in this session | - ---- - -## Next Steps After Monitor Completes - -| Scenario | Next Skill | -|----------|------------| -| All videos done, no 4K | `/fk-concat` | -| All 4K downloaded | `/fk-concat --4k` | -| Need narration first | `/fk-gen-narrator` → `/fk-concat --with-tts` | -| Upload to YouTube | `/fk-youtube-seo` → `/fk-youtube-upload` | +Arguments: $ARGUMENTS diff --git a/.claude/commands/fk-pipeline.md b/.claude/commands/fk-pipeline.md index b09d1373e..a52df9784 100644 --- a/.claude/commands/fk-pipeline.md +++ b/.claude/commands/fk-pipeline.md @@ -1,489 +1,5 @@ -# fk-pipeline — Smart Full-Pipeline Orchestrator +fk-pipeline — Smart Full-Pipeline Orchestrator + +Read and follow the instructions in skills/fk-pipeline.md -Auto-detect project state and run the correct stages (continuation or full run). Handles batching, parallelism, retries, downloads, TTS, and Telegram notifications. - -Usage: `/fk-pipeline [project_id] [orientation] [options]` - -Options: -- `--upscale` — include 4K upscale stage (TIER_TWO only) -- `--tts` — include TTS narration stage (parallel with upscale) -- `--download` — auto-download 4K files as upscales complete -- `--concat` — run concat after all stages done -- `--notify` — send Telegram notifications at milestones -- `--interval N` — poll interval in seconds (default: 15) -- `--orientation H|V` — HORIZONTAL or VERTICAL (auto-detected from video.orientation if omitted) - -Examples: -- `/fk-pipeline` — detect state and continue most recent project -- `/fk-pipeline --upscale --tts --download --notify` — full run with all options -- `/fk-pipeline HORIZONTAL --upscale --download` — explicit project + orientation - ---- - -## When to Use - -- Starting a new project from scratch (after scenes are created) -- Resuming a partially-complete project -- Batch-running all remaining stages hands-free -- Want TTS + upscale to run simultaneously -- Want 4K downloads to roll in as each upscale completes - ---- - -## Step 1: Resolve Project and State - -### 1a. Fetch project - -```bash -# Most recent project -curl -s http://127.0.0.1:8100/api/projects -# Use last item → {id, name} - -# Get video + scenes -VID=$(curl -s "http://127.0.0.1:8100/api/videos?project_id=" | python3 -c "import sys,json; print(json.load(sys.stdin)[0]['id'])") -curl -s "http://127.0.0.1:8100/api/scenes?video_id=$VID" > /tmp/fk_scenes.json -curl -s "http://127.0.0.1:8100/api/projects//characters" > /tmp/fk_chars.json -curl -s "http://127.0.0.1:8100/api/projects//output-dir" > /tmp/fk_outdir.json -``` - -### 1b. Derive slug and output dir - -```python -import json - -outdir_meta = json.load(open('/tmp/fk_outdir.json')) -SLUG = outdir_meta['slug'] -OUTDIR = outdir_meta['path'] -``` - -### 1c. State detection - -```python -import json, glob as globmod, os - -scenes = json.load(open('/tmp/fk_scenes.json')) -chars = json.load(open('/tmp/fk_chars.json')) -PREFIX = video_orientation.lower() # from GET /api/videos/{vid} → orientation field; 'horizontal' or 'vertical' -N = len(scenes) - -def pct(done, total): - return f"{done}/{total}" if total else "0/0" - -state = { - # Stage 0: ref images - 'refs_total': len(chars), - 'refs_done': sum(1 for c in chars if c.get('media_id')), - 'refs_missing': [c['id'] for c in chars if not c.get('media_id')], - - # Stage 1: scene images - 'images_total': N, - 'images_done': sum(1 for s in scenes if s.get(f'{PREFIX}_image_status') == 'COMPLETED'), - 'images_pending': [s['id'] for s in scenes if s.get(f'{PREFIX}_image_status') != 'COMPLETED'], - - # Stage 2: videos - 'videos_total': N, - 'videos_done': sum(1 for s in scenes if s.get(f'{PREFIX}_video_status') == 'COMPLETED'), - 'videos_pending': [s['id'] for s in scenes if s.get(f'{PREFIX}_video_status') != 'COMPLETED'], - - # Stage 2.5: review - 'review_done': False, # set True after review passes or max cycles - - # Stage 3: upscale - 'upscale_total': N, - 'upscale_done': sum(1 for s in scenes if s.get(f'{PREFIX}_upscale_status') == 'COMPLETED'), - 'upscale_pending': [s['id'] for s in scenes - if s.get(f'{PREFIX}_upscale_status') not in ('COMPLETED', 'FAILED')], - 'upscale_failed': [s['id'] for s in scenes if s.get(f'{PREFIX}_upscale_status') == 'FAILED'], - - # Stage 4: TTS - 'tts_total': N, - 'tts_done': len(globmod.glob(f'{OUTDIR}/tts/scene_*.wav')), - 'has_narrator': any(s.get('narrator_text') for s in scenes), - - # Stage 5: downloads - 'dl_total': N, - 'dl_done': len(globmod.glob(f'{OUTDIR}/4k/scene_*.mp4')), -} -``` - -### 1d. Print detected state and plan - -``` -Detected state for [HORIZONTAL]: - Refs: 5/5 ✓ - Images: 50/50 ✓ - Videos: 50/50 ✓ - Review: passed (48/50 good, 2 fixed after regen) - Upscale: 39/50 ← in progress - TTS: 50/50 ✓ - Downloads: 39/50 ← behind upscale - -Plan: - [ACTIVE] Continue upscale (11 remaining) - [ACTIVE] Download as upscales complete - [SKIP] TTS (already done) - [QUEUED] Concat after upscale+download complete -``` - ---- - -## Step 2: Stage Routing - -Use this decision tree to determine which stages to run and in what order. - -```python -stages_to_run = [] - -# Sequential gates — each stage blocks the next -if state['refs_done'] < state['refs_total']: - stages_to_run.append('REFS') -elif state['images_done'] < state['images_total']: - stages_to_run.append('IMAGES') -elif state['videos_done'] < state['videos_total']: - stages_to_run.append('VIDEOS') - -# Review (after videos done, before upscale) -if state['videos_done'] == state['videos_total'] and not state['review_done']: - stages_to_run.append('REVIEW') - -# Upscale (after review passes) -if upscale_flag and state['videos_done'] == state['videos_total'] and state['review_done']: - if state['upscale_done'] < state['upscale_total']: - stages_to_run.append('UPSCALE') - -# TTS — parallel with upscale/videos (independent) -if tts_flag and state['has_narrator'] and state['tts_done'] < state['tts_total']: - stages_to_run.append('TTS') # runs concurrently - -# Downloads — rolling alongside upscale -if download_flag and state['dl_done'] < state['dl_total']: - stages_to_run.append('DOWNLOAD') # runs concurrently with UPSCALE - -# Concat — after all downloads + TTS done -if concat_flag: - stages_to_run.append('CONCAT') # runs last -``` - -**Parallelism rules:** -- `REFS` → `IMAGES` → `VIDEOS` → `REVIEW` are **sequential** (each requires previous) -- `UPSCALE` + `TTS` can run **simultaneously** (spawn 2 agents) -- `DOWNLOAD` runs **rolling** alongside UPSCALE (check each poll cycle) -- `CONCAT` runs **after** UPSCALE + DOWNLOAD + TTS all complete - ---- - -## Step 3: Run Each Stage - -### Stage 0 — Ref Images - -Only run if any entity is missing `media_id`. - -```bash -# For each entity missing media_id, submit GENERATE_CHARACTER_IMAGE -# Batch 5 at a time -for CID in : - curl -X POST http://127.0.0.1:8100/api/requests \ - -H "Content-Type: application/json" \ - -d '{"type":"GENERATE_CHARACTER_IMAGE","character_id":"","project_id":""}' -``` - -**Poll until all entities have `media_id`** before moving to Stage 1. - -**Retry rule:** If a ref gen fails, resubmit once. If fails again, warn user and skip. - ---- - -### Stage 1 — Scene Images - -Only run after all refs have `media_id`. - -```bash -# For each scene with image_status != COMPLETED: -curl -X POST http://127.0.0.1:8100/api/requests \ - -H "Content-Type: application/json" \ - -d '{"type":"GENERATE_IMAGE","scene_id":"","project_id":"","video_id":"","orientation":""}' -``` - -Batch 5 at a time. Poll every 15s. Submit next batch when current batch completes. - -**Failed images:** Resubmit once with `REGENERATE_IMAGE` (forces re-run). - ---- - -### Stage 2 — Scene Videos - -Only run after all scene images COMPLETED. - -```bash -curl -X POST http://127.0.0.1:8100/api/requests \ - -H "Content-Type: application/json" \ - -d '{"type":"GENERATE_VIDEO","scene_id":"","project_id":"","video_id":"","orientation":""}' -``` - -Batch 5. Poll 15s. Each video takes 2-5 min. - ---- - -### Stage 2.5 — Review Videos - -Only run after all videos COMPLETED. Uses `/fk-review-video` to catch AI generation errors before upscaling. - -```bash -# Run light review on all completed videos -curl -X POST "http://127.0.0.1:8100/api/videos//review?project_id=&mode=light&orientation=" -# Poll until complete -``` - -**Interpret results:** -- Scenes scoring **7.5+** (good/excellent) → pass, move to upscale -- Scenes scoring **4.0–7.4** (acceptable/poor) → update `video_prompt` based on `fix_guide` + `errors`, then regen video -- Scenes scoring **0–3.9** (unusable) → update `video_prompt` based on errors, regen image first (`REGENERATE_IMAGE`), then regen video - -**Fix-and-regen loop (max 2 cycles per scene):** - -```python -for cycle in range(2): - review = run_review(VID, mode='light') - bad_scenes = [s for s in review if s['total_score'] < 7.5] - if not bad_scenes: - break # all pass - - for scene in bad_scenes: - # Update video_prompt based on review errors + fix_guide - # e.g. add "static camera" for camera drift, "no brand logos" for logo errors - new_prompt = improve_prompt(scene['video_prompt'], scene['errors'], scene['fix_guide']) - curl_patch(f"/api/scenes/{scene['scene_id']}", {"video_prompt": new_prompt}) - - if scene['total_score'] < 4.0: - # Unusable — regen image first (cascades video) - submit_request("REGENERATE_IMAGE", scene['scene_id']) - else: - # Poor/acceptable — regen video only - submit_request("GENERATE_VIDEO", scene['scene_id']) - - # Poll until all regens complete, then re-review -``` - -**After review passes (or max cycles exhausted):** -- Log scenes that still fail with their scores and errors -- Proceed to upscale with all scenes that scored 7.5+ -- Report skipped scenes at end - ---- - -### Stage 3 — Upscale (4K) - -Only run after review passes (or max review cycles exhausted). TIER_TWO only. - -```bash -curl -X POST http://127.0.0.1:8100/api/requests \ - -H "Content-Type: application/json" \ - -d '{"type":"UPSCALE_VIDEO","scene_id":"","project_id":"","video_id":"","orientation":""}' -``` - -Batch 5. **Resubmit failed upscales** once automatically. - ---- - -### Stage 4 — TTS Narration (parallel) - -Runs in parallel with Stage 2 or 3. Requires `narrator_text` on scenes and a voice template. - -```bash -# Check templates -curl -s http://127.0.0.1:8100/api/tts/templates -# Pick template name (e.g. vi_male_narrator) - -# Trigger narration for video -curl -X POST http://127.0.0.1:8100/api/videos//narrate \ - -H "Content-Type: application/json" \ - -d '{"template":""}' -``` - -Poll `output//tts/` for WAV files as they appear. - -**If no template exists:** Pause and instruct user to run `/fk-gen-tts-template` or `/fk-import-voice` first. - ---- - -### Stage 5 — Rolling Downloads (parallel with upscale) - -Each poll cycle, check for newly completed upscales not yet downloaded: - -```python -newly_completed = [ - s for s in scenes - if s.get(f'{PREFIX}_upscale_status') == 'COMPLETED' - and s.get(f'{PREFIX}_upscale_url') - and not os.path.exists(f"{OUTDIR}/4k/scene_{s['display_order']:03d}_{s['id']}.mp4") -] -``` - -For each, write URL to temp file and download via curl: - -```bash -URL=$(cat /tmp/url_scene_.txt) -curl -s "$URL" -o "output//4k/scene__.mp4" -``` - -**Important:** Write URL to temp file before passing to curl — avoids shell encoding mangling GCS signatures. - -Verify each download: -```bash -ffprobe -v quiet -show_entries format=duration -of csv=p=0 "" -# Must be > 0 seconds -``` - ---- - -### Stage 6 — Concat - -Run after UPSCALE + DOWNLOAD + TTS all complete. Delegates to `/fk-concat`. - -Invoke: `/fk-concat --4k --with-tts` (or appropriate flags based on what was run). - ---- - -## Step 4: Poll Loop - -```python -import time - -INTERVAL = 15 -cycle = 0 - -while stages_to_run: - cycle += 1 - print(f"[cycle {cycle}] Polling...") - - # Refresh scene data - scenes = fetch_scenes(VID) - state = compute_state(scenes, chars, OUTDIR, PREFIX) - - # Submit next batch for active sequential stage - active_stage = stages_to_run[0] # HEAD of sequential chain - if active_stage in ('REFS','IMAGES','VIDEOS','UPSCALE'): - submit_next_batch(active_stage, state, PID, VID, ORIENTATION) - - # Check if active stage completed - if stage_complete(active_stage, state): - stages_to_run.pop(0) - print(f"✅ {active_stage} complete") - if notify_flag: - send_telegram(f"✅ {active_stage} complete for {project_name}") - - # Parallel: Rolling downloads (every cycle when upscaling) - if 'DOWNLOAD' in stages_to_run: - download_newly_completed(scenes, state, OUTDIR, PREFIX, PID=PID, VID=VID) - if state['dl_done'] >= state['upscale_done']: - pass # downloads caught up — keep watching - - # Check if all done - remaining_sequential = [s for s in stages_to_run if s not in ('TTS','DOWNLOAD','CONCAT')] - if not remaining_sequential and state['dl_done'] == state['upscale_done']: - if 'CONCAT' in stages_to_run: - run_concat() - break - - time.sleep(INTERVAL) -``` - ---- - -## Step 5: Failure Handling - -| Failure type | Detection | Action | -|---|---|---| -| Ref image FAILED | `media_id` missing after request COMPLETED | Resubmit `GENERATE_CHARACTER_IMAGE` once | -| Scene image FAILED | `horizontal_image_status == FAILED` | Resubmit `REGENERATE_IMAGE` once | -| Video FAILED | `horizontal_video_status == FAILED` | Resubmit `GENERATE_VIDEO` once | -| Review FAILED (score < 7.5) | `total_score < 7.5` in review results | Update `video_prompt` from `fix_guide` + `errors`, regen video (max 2 cycles) | -| Review UNUSABLE (score < 4.0) | `total_score < 4.0` in review results | Update `video_prompt`, regen image first (`REGENERATE_IMAGE`), then video | -| Upscale FAILED | `horizontal_upscale_status == FAILED` | Resubmit `UPSCALE_VIDEO` once | -| Download 4KB (XML error) | `ffprobe` returns 0s or non-numeric | Re-download (URL still valid for ~8h) | -| Worker stalled | pending > 0, processing = 0 for 2+ min | Print warning; suggest server restart | -| TTS no template | `GET /api/tts/templates` returns empty | Pause; prompt user to create template | - -**Max retries:** 2 per scene per stage. After 2 failures, log and skip — report at end. - ---- - -## Step 6: Final Summary - -``` -Pipeline complete for - Refs: 5/5 - Images: 50/50 - Videos: 50/50 - Review: passed (48 good, 2 fixed after regen) - Upscale: 50/50 (1 retry) - Downloads: 50/50 - TTS: 50/50 - Failed: 0 - -Ready for: /fk-concat --4k --with-tts -``` - -Send via Telegram if `--notify`. - ---- - -## Concurrent TTS + Upscale Pattern - -When both `--tts` and `--upscale` are requested and videos are done, spawn two parallel tracks: - -``` -Track A (main agent): UPSCALE → batch 5 → poll → batch 5 → ... - └─ DOWNLOAD: rolling on each poll cycle - -Track B (sub-agent): TTS → POST /api/videos//narrate → poll tts/ dir -``` - -Use the Agent tool with `oh-my-claudecode:executor` for Track B: - -``` -Agent( - subagent_type="oh-my-claudecode:executor", - prompt="Run TTS narration for video using template