feat(library): clips-to-library export pipeline + streaming storage

docs/superpowers/specs/2026-09-16-clips-to-library-design.md

@@ -0,0 +1,115 @@
+# Clips to Library: export, store, preview, download
+
+**Date:** 2026-09-16
+**Status:** approved in chat (brainstorming), pending written review
+**Sub-project:** 1 of 3 in the Crayo pipeline rebuild (2: VPS media worker replacing Daytona fetch; 3: verify remaining Crayo commands)
+
+## Problem
+
+`/autoclip` never yields a video. When Crayo finishes an AutoClip, both callers
+(`runAutoclip` in `src/lib/server/crayo-tools.server.ts` and the background job's
+final phase in `src/lib/server/media-fetch-job.server.ts`) ingest only each clip's
+`thumbnail_url`. The clips exist as Crayo `project_id`s that are never exported, so the
+Library receives JPGs, not mp4s. There is also no way to download finished media:
+`/api/library/file` forces `content-disposition: inline` and only caption sidecars have a
+Download button.
+
+## Decisions already made
+
+- Heavy fetching (YouTube etc.) will move to an always-on worker on the VPS (sub-project 2).
+  This sub-project does not touch fetching or Daytona.
+- Finished clips are stored in Supabase Storage (bucket `clippy-library`, already the default
+  backend in `src/lib/server/library-storage.server.ts`), shown in the Library with the existing
+  player, and downloadable from there. No automatic copy to the operator's computer.
+
+## Architecture and data flow
+
+1. AutoClip reports done with N clips, each a Crayo `project_id`.
+2. For each clip the app starts a Crayo export (`POST /projects/{id}/export`) and polls
+   `GET /exports/{id}` in bounded steps until it returns an mp4 URL.
+3. The mp4 is streamed from Crayo's CDN into the library bucket through a new streaming write
+   path, so no clip is buffered in a Vercel function's memory. The Library asset row records the
+   clip title, duration, source URL, run id, segment index, and Crayo project id. The thumbnail is
+   attached to the same asset (no separate JPG asset).
+4. The run's results list the library asset ids; the Agent results panel and Library cards render
+   the clip with the existing `<video>` player plus a Download button.
+
+One new server module owns "clip project → stored library asset" so both callers share it.
+The background job gains a phase `exporting` between `autoclipping` and `done`; the tick loop
+drives exports the same bounded, resumable way it drives uploads.
+
+## Components and interfaces
+
+### `src/lib/server/clip-export.server.ts` (new)
+
+- `exportClipToLibrary(input)`: `{ projectId, title, source: { runId, clientId, sourceUrl, segmentIndex }, actorId }`
+  → `{ assetId, bytes, durationSec }`. Starts the export, polls, streams to storage, attaches the
+  thumbnail. Idempotent: the asset row is keyed on `projectId`; a retry finds the existing asset.
+- `exportClipStep(state)`: resumable single step for the job. Advances one clip one transition
+  (start export → poll → store) and returns the updated per-clip state.
+- Both take the Crayo client as a parameter (default: the real one) so tests inject a fake.
+
+### `src/lib/server/library-storage.server.ts`
+
+- New `writeLibraryStream(key, contentType, sizeHint, body: Readable)` next to the bytes writer.
+  Same backend chain and order (Supabase Storage → S3-compatible → local disk). The local spool copy
+  is skipped for streamed writes. The asset row records which backend holds the file.
+
+### `src/lib/server/media-fetch-job.server.ts`
+
+- `MediaFetchJobState.phase` gains `"exporting"`. Each segment gains
+  `clips: { projectId, title, thumbnailUrl, exportId?, exportStatus: "pending" | "exporting" | "stored" | "failed", assetId?, error? }[]`.
+- The tick enters `exporting` once every segment's AutoClip is done, runs at most two clip exports
+  concurrently, and moves to `done` when every clip is `stored` or `failed`.
+- The thumbnail-only ingest is removed, not kept as a fallback.
+
+### `src/lib/server/crayo-tools.server.ts`
+
+- `runAutoclip` (synchronous path, direct mp4 links) calls `exportClipToLibrary` per clip after
+  its existing AutoClip poll. `/export` (`crayo.export_project`) reuses the same function so a
+  manual export also lands in the Library.
+
+### UI
+
+- `src/routes/api/library.file.ts`: `download=1` query → `content-disposition: attachment`,
+  filename = clip title (sanitised) + extension.
+- `src/components/library/asset-card.tsx`, `asset-drawer.tsx`, `src/components/agent/results.tsx`:
+  a Download button linking to the signed URL with `download=1`. Agent results also get a Retry
+  button on a failed clip (calls `exportClipToLibrary` again).
+- Cards say which backend holds the file (replaces the misleading "Filebase library" copy where it
+  appears on these surfaces).
+
+## Error handling and spend control
+
+- **Spend guard:** before exporting, read export credits (same call as `/account`). If credits <
+  clips, stop with a message naming both numbers; export nothing; keep the AutoClip result
+  viewable. Hard ceiling of 10 exports per run.
+- **Export failures:** a failed or timed-out export marks only that clip failed with Crayo's
+  message; the run continues. The summary lists succeeded and failed clips by title.
+- **Storage failures:** on a mid-stream failure the partial object is deleted and the clip is
+  marked failed with the storage error. Backend fallback chain applies.
+- **Size:** check Crayo's reported export size against the bucket limit (512 MB) before streaming.
+- **Visibility:** every step writes a progress line to the run ("Exporting clip 2 of 3",
+  "Stored clip 2 of 3 (34 MB)"). Transient polling errors surface via the PR #50 mechanism.
+
+## Testing
+
+- `src/lib/server/clip-export.test.ts` (node:test): spend guard; `exportClipStep` transitions
+  against a fake Crayo client (pending → done → stored; failed export; storage error cleans up;
+  retry with existing export id is a no-op); idempotent lookup by project id.
+- `src/lib/media-fetch.test.ts`: job-state parser accepts the `exporting` phase and clip entries.
+- First tests for `crayo.server.ts`: export start and export poll response shaping from fixtures.
+- `writeLibraryStream`: 20 MB generated stream to the local-disk backend; size check; cleanup on
+  abort.
+- Integration on local dev (headless Firefox): `/autoclip` on a direct mp4 link with 2 clips →
+  playable clip in Library → Download returns an attachment with the right filename. Credit cost
+  stated in the PR.
+
+## Rollout
+
+One PR, no feature flag. Old thumbnail-only ingest removed.
+
+## Out of scope
+
+YouTube/page fetching and Daytona (sub-project 2); `/short`, `/voice`, `/image`, `/import`,
+`/ingest` verification (sub-project 3); any copy to the operator's machine.

migrations/0032_media_assets_external_ref.sql

@@ -0,0 +1,4 @@
+-- Idempotency key and clip thumbnail pointer for Crayo → library export.
+alter table media_assets add column if not exists external_ref text;
+alter table media_assets add column if not exists thumbnail_version_id text;
+create index if not exists media_assets_external_ref_idx on media_assets (external_ref);

src/lib/clip-export.test.ts

@@ -0,0 +1,80 @@
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import {
+  clipExternalRef,
+  MAX_EXPORTS_PER_RUN,
+  planExportBudget,
+  readAutoclipClips,
+  readExportPayload,
+} from "./clip-export.ts";
+
+test("clipExternalRef is stable and namespaced", () => {
+  assert.equal(clipExternalRef("proj_123"), "crayo:project:proj_123");
+  assert.equal(clipExternalRef(" proj_123 "), "crayo:project:proj_123");
+});
+
+test("planExportBudget refuses when credits are short and caps at the ceiling", () => {
+  assert.deepEqual(planExportBudget({ exportCredits: 1, clipCount: 3 }), {
+    ok: false,
+    reason: "3 clips need 3 export credits; the Crayo account has 1. Nothing was exported.",
+  });
+  assert.deepEqual(planExportBudget({ exportCredits: 50, clipCount: 3 }), { ok: true, count: 3 });
+  assert.deepEqual(planExportBudget({ exportCredits: 50, clipCount: 25 }), {
+    ok: true,
+    count: MAX_EXPORTS_PER_RUN,
+  });
+  // unknown credits (account call failed) → proceed, Crayo enforces
+  assert.deepEqual(planExportBudget({ exportCredits: null, clipCount: 2 }), { ok: true, count: 2 });
+});
+
+test("readExportPayload handles wrapped and flat shapes", () => {
+  assert.deepEqual(readExportPayload({ export: { id: "exp_1", status: "processing" } }), {
+    status: "pending",
+    url: null,
+    bytes: null,
+    exportId: "exp_1",
+  });
+  assert.deepEqual(
+    readExportPayload({
+      id: "exp_2",
+      status: "completed",
+      video_url: "https://cdn-crayo.com/x.mp4",
+      file_size: 123,
+    }),
+    {
+      status: "done",
+      url: "https://cdn-crayo.com/x.mp4",
+      bytes: 123,
+      exportId: "exp_2",
+    },
+  );
+  assert.deepEqual(
+    readExportPayload({ export: { id: "exp_3", status: "failed", error: "boom" } }),
+    {
+      status: "failed",
+      url: null,
+      bytes: null,
+      exportId: "exp_3",
+    },
+  );
+  assert.equal(readExportPayload({ status: "completed", url: "http://insecure/x.mp4" }).url, null);
+});
+
+test("readAutoclipClips keeps only clips with a project id", () => {
+  const rows = readAutoclipClips({
+    autoclip: {
+      status: "completed",
+      clips: [
+        { title: "A", project_id: "p1", thumbnail_url: "https://cdn-crayo.com/a.jpg" },
+        { title: "", project_id: "p2" },
+        { title: "no project" },
+      ],
+    },
+  });
+  assert.deepEqual(rows, [
+    { title: "A", projectId: "p1", thumbnailUrl: "https://cdn-crayo.com/a.jpg" },
+    { title: "AutoClip", projectId: "p2", thumbnailUrl: null },
+  ]);
+  // flat shape used by the sync path
+  assert.equal(readAutoclipClips({ clips: [{ title: "B", id: "p3" }] })[0]?.projectId, "p3");
+});

src/lib/clip-export.ts

@@ -0,0 +1,82 @@
+/** Client-safe helpers for the Crayo clip → Library export flow. No secrets, no I/O. */
+
+export function clipExternalRef(projectId: string): string {
+  return `crayo:project:${projectId.trim()}`;
+}
+
+export const MAX_EXPORTS_PER_RUN = 10;
+export const EXPORT_CONCURRENCY = 2;
+export const BUCKET_OBJECT_LIMIT_BYTES = 512 * 1024 * 1024;
+
+export function planExportBudget(input: {
+  exportCredits: number | null;
+  clipCount: number;
+  ceiling?: number;
+}): { ok: true; count: number } | { ok: false; reason: string } {
+  const ceiling = input.ceiling ?? MAX_EXPORTS_PER_RUN;
+  const count = Math.max(0, Math.min(ceiling, Math.floor(input.clipCount)));
+  if (count === 0) return { ok: false, reason: "AutoClip returned no clips to export." };
+  if (input.exportCredits != null && input.exportCredits < count) {
+    return {
+      ok: false,
+      reason: `${count} clips need ${count} export credits; the Crayo account has ${input.exportCredits}. Nothing was exported.`,
+    };
+  }
+  return { ok: true, count };
+}
+
+type Rec = Record<string, unknown>;
+const rec = (v: unknown): Rec =>
+  v && typeof v === "object" && !Array.isArray(v) ? (v as Rec) : {};
+const str = (v: unknown): string => (typeof v === "string" ? v.trim() : "");
+const num = (v: unknown): number | null => (typeof v === "number" && Number.isFinite(v) ? v : null);
+
+export function readExportPayload(payload: unknown): {
+  status: "pending" | "done" | "failed";
+  url: string | null;
+  bytes: number | null;
+  exportId: string | null;
+} {
+  const outer = rec(payload);
+  const inner = "export" in outer ? rec(outer.export) : outer;
+  const raw = str(inner.status).toLowerCase();
+  const status =
+    raw === "completed" || raw === "complete" || raw === "succeeded"
+      ? "done"
+      : raw === "failed" || raw === "error"
+        ? "failed"
+        : "pending";
+  const candidates = [
+    inner.video_url,
+    inner.videoUrl,
+    inner.download_url,
+    inner.downloadUrl,
+    inner.url,
+    rec(inner.output).url,
+    rec(inner.result).url,
+  ];
+  const url = candidates.map(str).find((u) => u.startsWith("https://")) ?? null;
+  const bytes = num(inner.file_size) ?? num(inner.bytes) ?? num(inner.size) ?? null;
+  return { status, url: status === "done" ? url : null, bytes, exportId: str(inner.id) || null };
+}
+
+export function readAutoclipClips(
+  payload: unknown,
+): { title: string; projectId: string; thumbnailUrl: string | null }[] {
+  const outer = rec(payload);
+  const inner = "autoclip" in outer ? rec(outer.autoclip) : outer;
+  const list = Array.isArray(inner.clips) ? inner.clips : [];
+  const out: { title: string; projectId: string; thumbnailUrl: string | null }[] = [];
+  for (const item of list.slice(0, 20)) {
+    const clip = rec(item);
+    const projectId = str(clip.project_id) || str(clip.projectId) || str(clip.id);
+    if (!projectId) continue;
+    const thumb = str(clip.thumbnail_url) || str(clip.thumbnailUrl);
+    out.push({
+      title: str(clip.title) || "AutoClip",
+      projectId,
+      thumbnailUrl: thumb.startsWith("https://") ? thumb : null,
+    });
+  }
+  return out;
+}

src/lib/library.ts

@@ -72,7 +72,10 @@ export const DEFAULT_RENDER_OPTIONS: RenderOptions = {
   format: "mp4",
 };
 
-export const PRESET_SIZE: Record<Exclude<RenderPreset, "CUSTOM">, { width: number; height: number }> = {
+export const PRESET_SIZE: Record<
+  Exclude<RenderPreset, "CUSTOM">,
+  { width: number; height: number }
+> = {
   REELS_9x16: { width: 1080, height: 1920 },
   SQUARE_1x1: { width: 1080, height: 1080 },
   LANDSCAPE_16x9: { width: 1920, height: 1080 },
@@ -97,6 +100,9 @@ export type LibraryAsset = {
   checksum: string | null;
   currentVersionId: string | null;
   parentAssetId: string | null;
+  externalRef: string | null;
+  thumbnailVersionId: string | null;
+  thumbnailUrl: string | null;
   tags: string[];
   previewUrl: string | null;
   createdAt: string;
@@ -296,7 +302,8 @@ export function parseTimecode(raw: string): number | null {
     const minutes = Number(match[2]);
     const seconds = Number(match[3]);
     const ms = (match[4] ?? "").padEnd(3, "0").slice(0, 3);
-    if (![hours, minutes, seconds].every(Number.isFinite) || minutes > 59 || seconds > 59) return null;
+    if (![hours, minutes, seconds].every(Number.isFinite) || minutes > 59 || seconds > 59)
+      return null;
     return hours * 3600000 + minutes * 60000 + seconds * 1000 + Number(ms);
   }
   const asNumber = Number(trimmed);
@@ -327,7 +334,10 @@ export function cuesToSrt(cues: CaptionCue[]): string {
 
 export function cuesToVtt(cues: CaptionCue[]): string {
   const body = cues
-    .map((cue) => `${cueStamp(cue.startMs, ".")} --> ${cueStamp(cue.endMs, ".")}\n${cue.text.trim()}\n`)
+    .map(
+      (cue) =>
+        `${cueStamp(cue.startMs, ".")} --> ${cueStamp(cue.endMs, ".")}\n${cue.text.trim()}\n`,
+    )
     .join("\n");
   return `WEBVTT\n\n${body}`;
 }

src/lib/server/library-storage.server.test.ts

@@ -0,0 +1,31 @@
+import assert from "node:assert/strict";
+import { test } from "node:test";
+import { mkdtemp, stat, writeFile, readFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+
+// Point the local backend at a scratch dir before the module reads it.
+process.env.AGENCY_LIBRARY_ROOT = await mkdtemp(join(tmpdir(), "clippy-lib-"));
+delete process.env.SUPABASE_SERVICE_ROLE_KEY;
+delete process.env.SUPABASE_SECRET_KEY;
+
+const { writeLibraryFile, backendFromStorageKey, storagePath } =
+  await import("./library-storage.server.ts");
+
+test("backendFromStorageKey reads the prefix", () => {
+  assert.equal(backendFromStorageKey("supabase:a/b.mp4"), "supabase");
+  assert.equal(backendFromStorageKey("s3:a/b.mp4"), "s3");
+  assert.equal(backendFromStorageKey("/tmp/agency-library/a/b.mp4"), "local");
+});
+
+test("writeLibraryFile streams a 20 MB file into the local backend without loading it", async () => {
+  const src = join(process.env.AGENCY_LIBRARY_ROOT!, "src.bin");
+  await writeFile(src, Buffer.alloc(20 * 1024 * 1024, 7));
+  const before = process.memoryUsage().heapUsed;
+  const result = await writeLibraryFile("asset-1/v1.mp4", src, "video/mp4");
+  const after = process.memoryUsage().heapUsed;
+  assert.equal(result, storagePath("asset-1/v1.mp4"));
+  assert.equal((await stat(result)).size, 20 * 1024 * 1024);
+  assert.ok(after - before < 15 * 1024 * 1024, "file was buffered into memory");
+  assert.equal((await readFile(result)).subarray(0, 4).toString("hex"), "07070707");
+});