From 1a4ee622537697d636e6528fe28ee1d8d619d5fa Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Fri, 4 Sep 2026 09:11:42 +0900 Subject: [PATCH 01/14] docs(adr): define playable stem delivery boundary --- ...001-playable-stem-delivery-and-audition.md | 203 +++++++++++++ docs/adr/README.md | 18 ++ .../2026-09-04-playable-stem-audition.md | 266 ++++++++++++++++++ 3 files changed, 487 insertions(+) create mode 100644 docs/adr/0001-playable-stem-delivery-and-audition.md create mode 100644 docs/adr/README.md create mode 100644 docs/superpowers/plans/2026-09-04-playable-stem-audition.md diff --git a/docs/adr/0001-playable-stem-delivery-and-audition.md b/docs/adr/0001-playable-stem-delivery-and-audition.md new file mode 100644 index 000000000..ac78a8d9f --- /dev/null +++ b/docs/adr/0001-playable-stem-delivery-and-audition.md @@ -0,0 +1,203 @@ +# ADR-0001: Deliver playable stems through revocable native authority + +- **Status:** Proposed +- **Date:** 2026-09-04 +- **Decision owners:** Source Separation, Active Player, Desktop Security +- **Related work:** Issue #961, PR #971, Issue #770, PR #828, Issue #781, PR #866 +- **Supersedes:** None + +## Context + +BandScope already separates admitted local audio into the canonical Demucs sources `vocals`, `bass`, `drums`, and `other`. The analysis pipeline persists NumPy feature arrays for internal reuse, but those arrays are not a renderer-safe media contract. The canonical Active Player in PR #971 can play the admitted full mix through the revocable `bandscope-playback` protocol, yet its stem actions remain unavailable because no real stem source reaches that protocol. + +The buyer-visible gap is therefore not another source-separation model. It is a missing product boundary: + +```text +successful local separation +→ durable-enough session artifact +→ native validation and authority registration +→ opaque renderer handle +→ the existing Active Player transport +→ audible, truthful stem audition +``` + +The first implementation must not imply that `other` is a separately identified guitar or keyboard. It must not claim perceptual separation quality merely because an artifact can be played. It must also avoid creating a second transport store beside PR #971. + +## Decision + +### 1. Materialize a versioned playable-stem artifact set + +After successful separation, the analysis engine will materialize the four aligned mono sources as PCM16 WAV files under the current project's app-owned temporary workspace. A single shared gain is applied to the whole set when needed to prevent clipping while preserving inter-stem level relationships. + +The internal native manifest is versioned and contains, at minimum: + +```text +artifact_set_id +format_version +sample_rate +channel_count +sample_count +duration_seconds +applied_gain +stem_artifacts[] + artifact_id + stem_kind + native_file_path + file_size_bytes + content_hash_sha256 + media_type +``` + +The engine may disclose the native path only to the trusted Tauri process over the existing bounded JSONL subprocess boundary. It must never enter renderer state, UI copy, logs, exported handoffs, or persisted project JSON. + +### 2. Extend the existing revocable playback authority + +The Tauri layer will validate each artifact against the current project's app-owned root before registering it. Registration requires: + +- one artifact for each canonical stem and no unknown stem; +- a regular, non-symlink WAV file under the expected project artifact directory; +- bounded nonzero size; +- manifest/file identity agreement; +- common sample rate, sample count, channel count, and duration; +- no duplicate artifact identifier or stem kind. + +The renderer receives only a strict opaque authority such as: + +```text +bandscope-project://project-…/stem-vocals +``` + +The existing `bandscope-playback` protocol remains the only media-serving boundary. Source replacement or project replacement revokes both the full mix and every stem authority together. + +### 3. Ship single-source audition before multitrack mixing + +The first buyer-visible increment adds a source selector to the existing Active Player: + +```text +Full mix | Vocals | Bass | Drums | Other instruments +``` + +Only sources present in a validated artifact set are enabled. Choosing a source re-arms the current loop on the same transport state machine. The selected source can then use the existing play, pause, stop, seek, count-in, playback-rate, loop-boundary, cue-navigation, and keyboard behavior. + +This is genuine isolated-source audition: selecting one stem makes that one generated source audible. It is not yet a multitrack gain mixer. The UI will not show per-stem volume sliders, simultaneous mute combinations, or a synchronization claim until the later multitrack acceptance work is complete. + +### 4. Add a synchronized mixer only after measurable timing evidence + +A later successor may use one `AudioContext`, one authoritative clock, and per-source gain nodes to support simultaneous stem mixing. It must first prove: + +- common decoded duration and sample alignment; +- bounded start and loop-restart skew; +- bounded accumulated drift over a long run; +- deterministic solo, mute, and gain transitions; +- safe device loss, suspension, cancellation, and project close; +- no unbounded decode or full-track duplication beyond the registered resource budget. + +Multiple independently controlled `HTMLAudioElement` instances are not an acceptable synchronization architecture. + +### 5. Keep capability evidence separate from quality evidence + +Artifact creation proves that an output exists and can be played. It does not prove perceptual quality, instrument identity beyond the canonical model outputs, or genre-wide accuracy. Source-separation accuracy and rights-cleared listening evidence remain owned by Issue #770 and PR #828. Resource admission and decode bounds remain owned by Issue #781 and its canonical PR #866. + +## Domain ownership + +| Responsibility | Owner | +| --- | --- | +| Decode and separation computation | `services/analysis-engine` Source Separation | +| PCM artifact materialization | `services/analysis-engine` Playback Artifact adapter | +| Native path, identity, revocation, and byte-range serving | `apps/desktop/src-tauri` Playback Authority | +| Public status and opaque source contracts | `packages/shared-types` | +| Loop, count-in, seek, rate, and source selection | `apps/desktop` Active Player | +| MIR accuracy and claim boundaries | Issue #770 / PR #828 | +| Resource admission and decode policy | Issue #781 / PR #866 | +| Reopened-project persistence and cleanup recovery | Issue #962 | + +The playback artifact adapter does not become a second source-separation owner, and the renderer does not gain filesystem authority. + +## Security and privacy + +### Attack surface + +- model-produced floating-point arrays; +- app-owned cache and temporary directories; +- Python-to-Rust manifest fields; +- custom-protocol paths and byte-range requests; +- renderer source-selection events. + +### Trust boundaries + +- analysis arrays are untrusted numeric output until shape, finiteness, cardinality, and alignment validation succeeds; +- native paths are untrusted subprocess output until Tauri validates app-owned containment and file identity; +- renderer handles are identifiers only and never paths; +- selecting a source grants no new filesystem, network, export, or generic execution capability. + +### Safe failure + +Malformed, incomplete, stale, replaced, oversized, unaligned, or non-finite stem sets produce no renderer authority. The full mix remains usable when it is still valid; the UI states that stems are unavailable rather than fabricating successful separation. + +### Data handling + +Separated audio is sensitive derived media. It remains local, is excluded from logs and ordinary exports, and is scoped to an app-owned project workspace. Session revocation is mandatory in this slice. Durable cleanup, recovery after abnormal termination, and reopened-project retention policy remain explicit Issue #962 follow-up work rather than an implied guarantee. + +## Alternatives considered + +### A. Put NumPy arrays directly into the renderer + +Rejected. `.npz` is an analysis cache format, not a bounded browser media contract. This would expand IPC payloads, expose derived audio to renderer memory, and duplicate decode logic. + +### B. Create four independent media elements and keep them aligned with timers + +Rejected. Independent media clocks and asynchronous seeks create avoidable drift and race conditions. It would also create a second transport authority beside PR #971. + +### C. Wait until the complete multitrack mixer is finished + +Rejected. A single-source audition slice closes a real buyer gap, reuses the current transport, and establishes the security contract required by the later mixer without advertising unfinished gain controls. + +### D. Export stem files and require users to open another player + +Rejected. It breaks the rehearsal loop, exposes filesystem concerns to the user, and does not connect isolated listening to BandScope's sections, cues, and role guidance. + +### E. Label `other` as guitar or keys based on the selected role + +Rejected. The current canonical model does not establish that identity. The shipped label is `Other instruments` until independently validated model capability says otherwise. + +## Consequences + +### Positive + +- Real separation becomes an audible product capability instead of an internal analysis detail. +- The renderer continues to receive opaque, revocable authority rather than native paths. +- The existing Active Player remains the single transport state machine. +- The design creates a measurable path to a later synchronized mixer without prematurely shipping fake controls. +- Cache hits and fresh separation can converge on the same versioned artifact contract. + +### Costs and constraints + +- Derived WAV files consume temporary disk space and require explicit lifecycle management. +- The Python/Rust status boundary needs a trusted-internal manifest and a sanitized renderer projection. +- The first slice auditions one source at a time; it does not satisfy the complete multitrack mixing expectation. +- Changes touch Python, Rust, TypeScript, Tauri permissions, CSP/protocol tests, UI, and documentation, so cross-platform exact-head evidence is mandatory. + +## Acceptance criteria for `Accepted` + +This ADR remains `Proposed` until one unchanged exact head proves all of the following: + +1. successful local separation produces exactly four aligned, finite, playable WAV artifacts; +2. a feature-cache hit can recreate or reuse the same artifact contract without rerunning the model; +3. Tauri rejects paths outside the current app-owned project root, symlinks, replacements, stale projects, unknown stems, duplicates, and metadata mismatch; +4. renderer-visible status contains no native path; +5. the source selector exposes only `Full mix`, `Vocals`, `Bass`, `Drums`, and `Other instruments` when actually available; +6. selecting a stem produces observable audio playback through the same section loop and transport controls as the full mix; +7. source replacement, app project replacement, playback error, and project close fail safely; +8. keyboard-only and screen-reader journeys can identify and select the active source; +9. normal, loading, unavailable, error, and partial/fail-closed states have executable component evidence; +10. macOS and Windows production desktop tests use rights-cleared audio and confirm the audible source change; +11. repository-owned production statement and branch coverage and public documentation remain 100%; +12. current-head CI, security, SAST, dependency, SBOM, package, release, review-thread, and independent-approval gates pass without bypass. + +## References + +Défossez, A. (2021). Hybrid spectrogram and waveform source separation. *Proceedings of the ISMIR 2021 Workshop on Music Source Separation*. + +WHATWG. (2026). *HTML Standard: Media elements*. https://html.spec.whatwg.org/multipage/media.html + +World Wide Web Consortium. (2024). *Web Audio API 1.1* (First Public Working Draft). https://www.w3.org/TR/webaudio-1.1/ diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 000000000..8e1f642b9 --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,18 @@ +# Architecture decision records + +BandScope records durable architecture decisions in this directory. + +## Status lifecycle + +- `Proposed`: the decision is under implementation or still lacks exact-head production evidence. +- `Accepted`: the implementation, contracts, tests, security review, and production evidence all match the decision. +- `Superseded`: a later ADR replaces the decision without rewriting its history. +- `Rejected`: the decision was considered and deliberately not adopted. + +A pull request, prototype, Storybook story, or passing unit test does not by itself make an ADR `Accepted`. The owning PR must satisfy the repository's protected current-head merge gate and preserve the stated decision in the shipped product. + +## Index + +| ADR | Status | Decision | +| --- | --- | --- | +| [0001](0001-playable-stem-delivery-and-audition.md) | Proposed | Deliver locally generated stems through revocable native authority and add single-source audition before a synchronized multitrack mixer. | diff --git a/docs/superpowers/plans/2026-09-04-playable-stem-audition.md b/docs/superpowers/plans/2026-09-04-playable-stem-audition.md new file mode 100644 index 000000000..4bb3b1ab1 --- /dev/null +++ b/docs/superpowers/plans/2026-09-04-playable-stem-audition.md @@ -0,0 +1,266 @@ +# Playable stem audition implementation plan + +> **Execution rule:** Apply test-driven development. Keep this work stacked on canonical Active Player PR #971. Do not create a second transport store, duplicate Issue #781 resource policy, or claim MIR quality from transport tests. + +**Goal:** Turn BandScope's already-generated four source-separation outputs into revocable, renderer-safe media sources that the existing Active Player can actually audition one at a time. + +**Architecture:** The analysis engine materializes an aligned PCM16 WAV artifact set. Tauri validates and registers the native files under the current project, rewrites the internal manifest to opaque `bandscope-project://…/stem-*` authorities, and emits only that sanitized projection. The React Active Player keeps one media element and one transport state machine while switching between the full mix and one validated stem. A synchronized Web Audio gain mixer is a later successor after timing evidence. + +**Stack:** Python 3.12+ (`numpy`, standard-library `wave`), Rust 1.97.1, Tauri 2, React 19, TypeScript, Vitest, pytest, cargo test. + +**Base identity at plan creation:** `feat/rehearsal-player-first-section-loop@803019e092376383500c0a4f8c2f52c68eb7232f`. + +--- + +## Task 1: Establish the decision and traceability baseline + +**Files** + +- Create: `docs/adr/README.md` +- Create: `docs/adr/0001-playable-stem-delivery-and-audition.md` +- Create: `docs/superpowers/plans/2026-09-04-playable-stem-audition.md` + +**Steps** + +1. Record ADR-0001 as `Proposed`. +2. Bind the work to #961/#971 while preserving #770/#828, #781/#866, and #962 ownership. +3. Record the rejected independent-media-clock design and the later synchronized mixer gate. +4. Commit the documentation before implementation so reviewers can reconstruct the intended boundary. + +## Task 2: Materialize aligned native WAV artifacts + +**Files** + +- Create: `services/analysis-engine/src/bandscope_analysis/separation/playback_artifacts.py` +- Create: `services/analysis-engine/tests/test_playback_artifacts.py` +- Modify later: `services/analysis-engine/src/bandscope_analysis/separation/__init__.py` + +**RED tests** + +- exact canonical stem set required; +- nonempty, one-dimensional, finite arrays required; +- every stem has the same sample count; +- sample rate is a bounded positive integer and not Boolean; +- invalid artifact-set identifiers fail closed; +- symlink or non-directory artifact roots fail closed; +- one shared gain prevents clipping without per-stem normalization; +- generated files are mono PCM16 WAV with exact sample count/rate; +- hash and byte-size metadata match on-disk bytes; +- rerunning the same artifact set is deterministic and leaves no temporary files; +- failed publication removes partial temporary files. + +**GREEN implementation** + +1. Validate all arrays before creating output. +2. Derive one shared gain from the set-wide peak. +3. Write each file to a same-directory temporary file. +4. flush/fsync, atomically replace the deterministic target, then hash the published bytes. +5. Return a JSON-serializable internal native manifest. +6. Run focused pytest with branch coverage, Ruff, and mypy. + +## Task 3: Attach artifact production to fresh and cached analysis + +**Files** + +- Modify: `services/analysis-engine/src/bandscope_analysis/api.py` +- Modify: `services/analysis-engine/tests/test_api.py` +- Modify: `services/analysis-engine/tests/test_branch_coverage_contract.py` + +**RED tests** + +- successful separation emits a native artifact set in the terminal status; +- cached features emit the same artifact contract without invoking Demucs; +- a final-result cache hit attempts bounded feature reuse and omits stems honestly when no feature set exists; +- fallback or separation failure never emits a success-shaped stem set; +- native paths do not enter cached rehearsal-song JSON; +- artifact generation failure keeps the analysis result usable but reports stems unavailable. + +**GREEN implementation** + +1. Derive a deterministic artifact-set identifier from the versioned source/analysis identity. +2. Materialize artifacts after fresh separation and after feature-cache load. +3. Add an internal optional native artifact field to the analysis JSONL status. +4. Keep the final `RehearsalSong` contract free of filesystem metadata. +5. Preserve #781 resource bounds; do not add a second decoder. + +## Task 4: Define and validate native/public contracts + +**Files** + +- Modify: `apps/desktop/core/src/lib.rs` +- Modify: `apps/desktop/core/tests/analysis_job_status.rs` or the current owning status-contract test +- Modify: `packages/shared-types/src/index.ts` +- Modify: `packages/shared-types/test/index.test.ts` + +**RED tests** + +- native manifest denies unknown fields and malformed values; +- public artifact source accepts only the four canonical stem kinds and opaque BandScope authorities; +- native path is rejected by the public parser; +- unknown stems, duplicates, misalignment, nonfinite duration, zero size, bad hashes, and invalid media type fail closed; +- optional absence remains compatible with old results. + +**GREEN implementation** + +1. Add a trusted-internal native status shape for subprocess ingestion. +2. Add a separate renderer-safe projection shape. +3. Use semantic multiword names and translate only at the IPC boundary. +4. Keep all existing analysis-job fields backward compatible. + +## Task 5: Register stems with revocable Tauri playback authority + +**Files** + +- Modify: `apps/desktop/src-tauri/src/playback_protocol.rs` +- Modify: `apps/desktop/src-tauri/src/main.rs` +- Modify: `apps/desktop/src-tauri/build.rs` only if a command surface changes +- Modify: `apps/desktop/src-tauri/tauri.conf.json` only if CSP needs an explicit path update +- Modify: existing Rust protocol and native integration tests + +**RED tests** + +- mix compatibility URI remains valid; +- exact stem URIs resolve only for the active project; +- old project and old artifact-set handles are revoked; +- only GET/HEAD and one bounded byte range are accepted; +- native artifact path must be under the current project temp artifact directory; +- symlinks, file replacement, wrong size/hash/type, duplicate kind, unknown kind, and cross-project registration fail; +- a status emitted to the renderer contains opaque authorities and no native path; +- a malformed internal manifest cannot partially register a set. + +**GREEN implementation** + +1. Refactor the authority from one source into one current project containing a full mix plus an atomic stem-source map. +2. Preserve `/project-id` for the full mix and add strict `/project-id/stem-*` paths. +3. Register the complete validated set in one operation. +4. Rewrite or remove native fields before status storage and event emission. +5. Hold revocation protection through each bounded serve operation. + +## Task 6: Connect the sanitized status to the mounted workspace + +**Files** + +- Modify: `apps/desktop/src/lib/analysis.ts` +- Modify: `apps/desktop/src/App.tsx` +- Modify: `apps/desktop/src/App.test.tsx` +- Modify: `apps/desktop/src/features/workspace/Workspace.tsx` +- Modify: `apps/desktop/src/features/workspace/Workspace.test.tsx` + +**RED tests** + +- current job and current project identity must match before stems become available; +- stale job completion cannot attach stems to a replacement song; +- no stem set leaves the existing unavailable controls enabled; +- `other` is presented as `Other instruments`, not inferred guitar or keys; +- renderer state and error copy contain no path. + +**GREEN implementation** + +1. Parse the public artifact set on every bridge/event boundary. +2. Store it beside the current successful job, not inside the song. +3. Clear it on source replacement, job replacement, failed analysis, and project load without matching authority. +4. Pass the sanitized set into the Active Player. + +## Task 7: Add real single-source audition to Active Player + +**Files** + +- Modify: `apps/desktop/src/features/workspace/RehearsalPlayer.tsx` +- Modify: `apps/desktop/src/features/workspace/RehearsalPlayer.test.tsx` +- Create if useful: `apps/desktop/src/features/workspace/rehearsalPlaybackSource.ts` +- Create if useful: `apps/desktop/src/features/workspace/rehearsalPlaybackSource.test.ts` + +**RED tests** + +- `Full mix` remains the default source; +- only validated available stems render as source choices; +- choosing a stem changes the actual `