Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
1a4ee62
docs(adr): define playable stem delivery boundary
seonghobae Sep 4, 2026
b743f49
test(separation): require playable stem artifact materialization
seonghobae Sep 4, 2026
16b525f
feat(separation): materialize aligned playable stem artifacts
seonghobae Sep 4, 2026
5d4a2f5
test(separation): cover playable artifact failure boundaries
seonghobae Sep 4, 2026
319cdd9
test(separation): require playable artifact package exports
seonghobae Sep 4, 2026
17b1746
feat(separation): export playable artifact contract
seonghobae Sep 4, 2026
46629bf
test(separation): require path-free stem artifact status projection
seonghobae Sep 4, 2026
f873959
feat(separation): project path-free stem artifact references
seonghobae Sep 4, 2026
edfe167
feat(separation): export path-free artifact reference contract
seonghobae Sep 4, 2026
f6a0f13
docs(adr): narrow playable stem process authority
seonghobae Sep 4, 2026
6c29f64
test(api): require path-free playable stem status evidence
seonghobae Sep 4, 2026
5da57df
chore(repair): verify playable stem status integration
seonghobae Sep 4, 2026
8bfe72b
chore(ci): remove self-modifying playable stem status repair
seonghobae Sep 4, 2026
460cb9f
fix(separation): wire playable stem status artifacts
seonghobae Sep 4, 2026
dbafdd7
chore(stems): restack materialization on current Active Player
seonghobae Sep 4, 2026
22a9f18
chore(stems): restack materialization on protected Active Player
seonghobae Sep 4, 2026
c27f378
chore(stack): adopt current Active Player base
seonghobae Sep 4, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
224 changes: 224 additions & 0 deletions docs/adr/0001-playable-stem-delivery-and-audition.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
# ADR-0001: Deliver playable stems through revocable native authority

- **Status:** Proposed
- **Date:** 2026-09-04
- **Decision owners:** Source Separation, Active Player, Desktop Security
- **Related work:** Issue #961, PR #971, Issue #770, PR #828, Issue #781, PR #866
- **Supersedes:** None

## Context

BandScope already separates admitted local audio into the canonical Demucs sources `vocals`, `bass`, `drums`, and `other`. The analysis pipeline persists NumPy feature arrays for internal reuse, but those arrays are not a renderer-safe media contract. The canonical Active Player in PR #971 can play the admitted full mix through the revocable `bandscope-playback` protocol, yet its stem actions remain unavailable because no real stem source reaches that protocol.

The buyer-visible gap is therefore not another source-separation model. It is a missing product boundary:

```text
successful local separation
→ durable-enough session artifact
→ native validation and authority registration
→ opaque renderer handle
→ the existing Active Player transport
→ audible, truthful stem audition
```

The first implementation must not imply that `other` is a separately identified guitar or keyboard. It must not claim perceptual separation quality merely because an artifact can be played. It must also avoid creating a second transport store beside PR #971.

## Decision

### 1. Materialize a versioned playable-stem artifact set

After successful separation, the analysis engine will materialize the four aligned mono sources as PCM16 WAV files under the current project's app-owned temporary workspace. A single shared gain is applied to the whole set when needed to prevent clipping while preserving inter-stem level relationships.

The Python playback-artifact adapter may use an in-process native manifest while it publishes and verifies the files:

```text
artifact_set_id
format_version
sample_rate
channel_count
sample_count
duration_seconds
applied_gain
stem_artifacts[]
artifact_id
stem_kind
native_file_path
file_size_bytes
content_hash_sha256
media_type
```

The native path does **not** cross the analysis JSONL boundary. Before status emission, the adapter creates a detached path-free reference containing only artifact identity, canonical stem kind, media metadata, size, and content hash. Tauri already owns the project temporary root and reconstructs the only permitted location from the fixed suffix:

```text
{project_temp_root}/playable-stems-v1/{artifact_set_id}/{stem_kind}.wav
```

This avoids granting a Python-returned path filesystem authority. Native paths must never enter renderer state, UI copy, logs, exported handoffs, or persisted project JSON.

### 2. Extend the existing revocable playback authority

The Tauri layer will validate each artifact against the current project's app-owned root before registering it. Registration requires:

- one artifact for each canonical stem and no unknown stem;
- a regular, non-symlink WAV file under the expected project artifact directory;
- bounded nonzero size;
- path-free reference and file identity agreement;
- common sample rate, sample count, channel count, and duration;
- no duplicate artifact identifier or stem kind.

The renderer receives only a strict opaque authority such as:

```text
bandscope-project://project-…/stem-vocals
```

The existing `bandscope-playback` protocol remains the only media-serving boundary. Source replacement or project replacement revokes both the full mix and every stem authority together.

### 3. Ship single-source audition before multitrack mixing

The first buyer-visible increment adds a source selector to the existing Active Player:

```text
Full mix | Vocals | Bass | Drums | Other instruments
```

Only sources present in a validated artifact set are enabled. Choosing a source re-arms the current loop on the same transport state machine. The selected source can then use the existing play, pause, stop, seek, count-in, playback-rate, loop-boundary, cue-navigation, and keyboard behavior.

This is genuine isolated-source audition: selecting one stem makes that one generated source audible. It is not yet a multitrack gain mixer. The UI will not show per-stem volume sliders, simultaneous mute combinations, or a synchronization claim until the later multitrack acceptance work is complete.

### 4. Add a synchronized mixer only after measurable timing evidence

A later successor may use one `AudioContext`, one authoritative clock, and per-source gain nodes to support simultaneous stem mixing. It must first prove:

- common decoded duration and sample alignment;
- bounded start and loop-restart skew;
- bounded accumulated drift over a long run;
- deterministic solo, mute, and gain transitions;
- safe device loss, suspension, cancellation, and project close;
- no unbounded decode or full-track duplication beyond the registered resource budget.

Multiple independently controlled `HTMLAudioElement` instances are not an acceptable synchronization architecture.

### 5. Keep capability evidence separate from quality evidence

Artifact creation proves that an output exists and can be played. It does not prove perceptual quality, instrument identity beyond the canonical model outputs, or genre-wide accuracy. Source-separation accuracy and rights-cleared listening evidence remain owned by Issue #770 and PR #828. Resource admission and decode bounds remain owned by Issue #781 and its canonical PR #866.

### 6. Keep the Python implementation outside the numerical hot path

The first artifact adapter is implemented in Python because Demucs currently yields NumPy arrays inside the existing analysis process, and the adapter performs bounded validation, PCM16 encoding, hashing, and atomic publication without introducing another array copy across a language boundary. It does not estimate, separate, resample, align, or score audio.

This is an explicit adapter exception rather than a precedent for Python-owned production DSP. The source-separation and timing hot paths remain subject to the repository's Rust-first policy. Move PCM encoding or hashing into a Rust-owned native service if profiling shows this adapter materially affects latency, memory, concurrency, or attack-surface goals; preserve the same versioned artifact contract and tests when doing so.

## Domain ownership

| Responsibility | Owner |
| --- | --- |
| Decode and separation computation | `services/analysis-engine` Source Separation |
| PCM artifact materialization | `services/analysis-engine` Playback Artifact adapter |
| Path-free artifact reference | `services/analysis-engine` Playback Artifact adapter |
| Native location derivation, identity, revocation, and byte-range serving | `apps/desktop/src-tauri` Playback Authority |
| Public status and opaque source contracts | `packages/shared-types` |
| Loop, count-in, seek, rate, and source selection | `apps/desktop` Active Player |
| MIR accuracy and claim boundaries | Issue #770 / PR #828 |
| Resource admission and decode policy | Issue #781 / PR #866 |
| Reopened-project persistence and cleanup recovery | Issue #962 |

The playback artifact adapter does not become a second source-separation owner, and the renderer does not gain filesystem authority.

## Security and privacy

### Attack surface

- model-produced floating-point arrays;
- app-owned cache and temporary directories;
- path-free Python-to-Rust artifact-reference fields;
- custom-protocol paths and byte-range requests;
- renderer source-selection events.

### Trust boundaries

- analysis arrays are untrusted numeric output until shape, finiteness, cardinality, and alignment validation succeeds;
- the analysis process emits no native path as authority;
- Tauri derives the fixed artifact location from its own project root and a strict lowercase SHA-256 artifact-set identifier;
- every derived file is revalidated by native identity, size, hash, media structure, and alignment before registration;
- renderer handles are identifiers only and never paths;
- selecting a source grants no new filesystem, network, export, or generic execution capability.

### Safe failure

Malformed, incomplete, stale, replaced, oversized, unaligned, or non-finite stem sets produce no renderer authority. The full mix remains usable when it is still valid; the UI states that stems are unavailable rather than fabricating successful separation.

### Data handling

Separated audio is sensitive derived media. It remains local, is excluded from logs and ordinary exports, and is scoped to an app-owned project workspace. Session revocation is mandatory in this slice. Durable cleanup, recovery after abnormal termination, and reopened-project retention policy remain explicit Issue #962 follow-up work rather than an implied guarantee.

## Alternatives considered

### A. Put NumPy arrays directly into the renderer

Rejected. `.npz` is an analysis cache format, not a bounded browser media contract. This would expand IPC payloads, expose derived audio to renderer memory, and duplicate decode logic.

### B. Create four independent media elements and keep them aligned with timers

Rejected. Independent media clocks and asynchronous seeks create avoidable drift and race conditions. It would also create a second transport authority beside PR #971.

### C. Wait until the complete multitrack mixer is finished

Rejected. A single-source audition slice closes a real buyer gap, reuses the current transport, and establishes the security contract required by the later mixer without advertising unfinished gain controls.

### D. Export stem files and require users to open another player

Rejected. It breaks the rehearsal loop, exposes filesystem concerns to the user, and does not connect isolated listening to BandScope's sections, cues, and role guidance.

### E. Label `other` as guitar or keys based on the selected role

Rejected. The current canonical model does not establish that identity. The shipped label is `Other instruments` until independently validated model capability says otherwise.

### F. Send native artifact paths to Tauri in the analysis status

Rejected. Although Tauri is trusted, accepting a subprocess-returned path would unnecessarily widen the authority parser. A path-free artifact reference plus a fixed location derived from the already-authorized project root provides a narrower and more reviewable boundary.

## Consequences

### Positive

- Real separation becomes an audible product capability instead of an internal analysis detail.
- No native artifact path crosses the process or renderer status contract.
- The renderer continues to receive opaque, revocable authority rather than native paths.
- The existing Active Player remains the single transport state machine.
- The design creates a measurable path to a later synchronized mixer without prematurely shipping fake controls.
- Cache hits and fresh separation can converge on the same versioned artifact contract.

### Costs and constraints

- Derived WAV files consume temporary disk space and require explicit lifecycle management.
- Python and Rust must implement the same versioned path-free reference validation and location derivation contract.
- The first slice auditions one source at a time; it does not satisfy the complete multitrack mixing expectation.
- Changes touch Python, Rust, TypeScript, Tauri permissions, CSP/protocol tests, UI, and documentation, so cross-platform exact-head evidence is mandatory.

## Acceptance criteria for `Accepted`

This ADR remains `Proposed` until one unchanged exact head proves all of the following:

1. successful local separation produces exactly four aligned, finite, playable WAV artifacts;
2. a feature-cache hit can recreate or reuse the same artifact contract without rerunning the model;
3. the emitted analysis reference contains no native path and is detached from the in-process native manifest;
4. Tauri rejects artifact references or files with paths outside the current app-owned project root, symlinks, replacements, stale projects, unknown stems, duplicates, or metadata mismatch;
5. renderer-visible status contains no native path, file hash, or internal storage location;
6. the source selector exposes only `Full mix`, `Vocals`, `Bass`, `Drums`, and `Other instruments` when actually available;
7. selecting a stem produces observable audio playback through the same section loop and transport controls as the full mix;
8. source replacement, app project replacement, playback error, and project close fail safely;
9. keyboard-only and screen-reader journeys can identify and select the active source;
10. normal, loading, unavailable, error, and partial/fail-closed states have executable component evidence;
11. macOS and Windows production desktop tests use rights-cleared audio and confirm the audible source change;
12. repository-owned production statement and branch coverage and public documentation remain 100%;
13. current-head CI, security, SAST, dependency, SBOM, package, release, review-thread, and independent-approval gates pass without bypass.

## References

Défossez, A. (2021). Hybrid spectrogram and waveform source separation. *Proceedings of the ISMIR 2021 Workshop on Music Source Separation*.

WHATWG. (2026). *HTML Standard: Media elements*. https://html.spec.whatwg.org/multipage/media.html

World Wide Web Consortium. (2024). *Web Audio API 1.1* (First Public Working Draft). https://www.w3.org/TR/webaudio-1.1/
18 changes: 18 additions & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Architecture decision records

BandScope records durable architecture decisions in this directory.

## Status lifecycle

- `Proposed`: the decision is under implementation or still lacks exact-head production evidence.
- `Accepted`: the implementation, contracts, tests, security review, and production evidence all match the decision.
- `Superseded`: a later ADR replaces the decision without rewriting its history.
- `Rejected`: the decision was considered and deliberately not adopted.

A pull request, prototype, Storybook story, or passing unit test does not by itself make an ADR `Accepted`. The owning PR must satisfy the repository's protected current-head merge gate and preserve the stated decision in the shipped product.

## Index

| ADR | Status | Decision |
| --- | --- | --- |
| [0001](0001-playable-stem-delivery-and-audition.md) | Proposed | Deliver locally generated stems through revocable native authority and add single-source audition before a synchronized multitrack mixer. |
Loading