Skip to content

Build the SA3 Song Builder: structured/pattern song generation + wired controls #89

Description

@brxs

Decided in ADR-0033 (song = arrangement of reusable parts) and ADR-0034 (backend continuation-conditioning render), merged in #88.

Problem

SA3 song generation is one-shot: one prompt → one WAV up to 6:20, authored in the Generate tab of frontend/src/media/MediaExplorer.tsx, sent as {prompt, seconds, kind}. It can't produce real structure (a chorus that returns identically), it renders the full length every time, and it isn't rerollable per section. Issue #54 (PR #87) already threaded SA3's full model surface — init_audio/audio-to-audio, inpaint, negative_prompt, cfg/apg, seed — through /api/generate, but API-only: none of it is reachable from the UI. We have powerful primitives no user can touch, and no way to compose a song from parts.

Proposed approach

Per ADR-0033/0034: a standalone Song Builder window (ADR-0032 pattern) with One-shot and Structured modes. Structured = a song base prompt, parts as prompt extensions (+ per-part SA3 controls + length), an arrangement letter string with fresh/repeat/variation slots, and shared coherence (seed/BPM/key). A backend orchestration renders distinct parts with neighbour-tail continuation conditioning, reuses clips for repeats, renders variations from a parent (audio-to-audio or inpaint), and stitches beat-aligned parts with crossfades (Python-side) into one GeneratedTrack WAV — cancellable, per-part progress. The #54 controls get wired into the UI. Detailed plan is for /berlitz-engineering:fix-issue.

Acceptance criteria

  • A standalone Song Builder window toggles open/hide per ADR-0032; its rendered output is a normal GeneratedTrack that loads to a deck via the existing save/preview/load path.
  • One-shot mode reproduces today's generation exactly (same request/argv for a plain prompt+length).
  • Structured mode: set a base prompt, define parts (prompt extension + optional SA3 controls + length), and an arrangement (ABABCD, AB A' B). Repeat reuses the same clip; fresh generates; variation (A') renders from its parent.
  • Distinct parts render with continuation conditioning on the neighbour's tail; parts stitch on bar/beat boundaries with crossfades into one WAV.
  • A song render is cancellable and reports per-part progress.
  • The serialized song config is saved under generated_songs/configs/ and referenced by relative path from the registry row (resolved via scoped_path; the rendered .wav stays the registry identity).
  • The Song Builder window lists saved configs and reloads one back into the editor for re-rolling/adapting.
  • The Thread Stable Audio 3's full generation surface (audio-to-audio, inpainting, CFG, seed) through the pipeline #54 controls (seed, cfg/apg, negative prompt, init audio) are reachable and functional from the UI.
  • just check passes; a human checklist covers the listen tests unit tests can't hear (identical repeats, coherent seams, variation similarity).

Scope & non-goals

In scope: the Song Builder window; one-shot + structured modes; the part/arrangement/variation model; the config artifact + configs/ library; the backend continuation-conditioning render + Python-side stitch; wiring the #54 controls.

Non-goals: changing the Magenta /api/render engine (SA3 only); auto-detecting structure from an existing track; a full drag-drop arrangement DAW timeline beyond the letter-string + per-part settings; bar-accurate lengths depend on issue #77 (target BPM) — if #77 isn't ready, parts fall back to seconds; deck-source→init-audio conversion beyond what #54 accepts natively.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions