Skip to content

current-truth.md

Latest commit

 

History

History
254 lines (214 loc) · 15 KB

current-truth.md

File metadata and controls

254 lines (214 loc) · 15 KB
Raw

Current Truth

This document is the operator-facing summary of what is canonical in the repo today.

Product Boundary

AutoVoice currently targets a reliable single-user, local-first workflow:

  • create and manage target-user voice profiles
  • ingest source-artist material from uploaded songs
  • ingest YouTube source material through a review-gated workflow that downloads audio, splits vocals/instrumental, diarizes vocals, suggests profile matches, and only creates or assigns source profiles after operator confirmation
  • train target-user profiles
  • run offline conversion jobs
  • run live karaoke sessions, including a second client joining an existing session and mid-session voice switching by voice model id
  • record browser sing-along takes for target-user profiles from the computer using the web UI, with browser-side input/output selection and local take quality checks before attach
  • browse and curate the speaker library (extraction, clusters with rename/merge/split, identification, track metadata) from the Diarization page
  • monitor live conversion quality, adapter health, and degradation from the /quality page, with analyze/compare operator tools (server-side paths)
  • manage the full profile lifecycle from the UI: export, purge, duplicate check, retrain check, and LoRA retrain
  • receive server-dispatched webhook notifications for training/conversion completion and job failures (fire-and-forget, /api/v1/notifications/webhooks)
  • audit product actions from the operator console (/api/v1/audit/events)
  • persist local product state under DATA_DIR, including per-profile checkpoint records created automatically on successful training completion

Readiness Vocabulary And Current Status

Use these terms consistently. Avoid the unqualified phrase "production-ready" in release notes or operator docs because it hides the support boundary.

Tier Current status Boundary
Local-only single-user Supported MVP target Trusted operator machine, local DATA_DIR, no external users, no public ingress.
Private/operator-controlled hosted Conditional Requires API auth, explicit CORS origins, media-consent gates, current-head health/readiness/preflight proof, and operator-owned media.
Public multi-user Not ready Requires account auth, per-user isolation, quotas, abuse review, retention/export/deletion policy, and current-head full hardware evidence.
Commercial launch Not ready Requires the public multi-user controls plus legal/policy approval for voice, likeness, copyright, biometric privacy, and platform terms.

The latest pushed local-ready baseline is tag local-ready-2026-04-30 at source SHA 25d484880f803a551d457e6f893daf48b34506eb. Its no-Docker evidence lives at reports/completion/local-final-20260430T032639Z-25d484880f80/completion_matrix.json and passed with ok: true for the supported local/hardware lanes that were in scope. A readiness claim for any later commit must still cite artifacts whose embedded git SHA matches that candidate commit; rerun the evidence commands after the final release commit is selected.

That local-ready pass ran in autovoice-thor. It passed GitNexus refresh, skip-audit policy, backend contract smoke tests, compose-config validation without Docker deployment, benchmark dashboard build and validation, experimental evidence validation, local hosted preflight, frontend lint/typecheck/build/browser smoke, Jetson CUDA/TensorRT validation, TensorRT engine-suite validation, and TensorRT checkpoint parity. Real compose and release-candidate compose remain explicit skipped lanes for local-only work. Deterministic real-audio E2E can now be run without Docker by passing --real-audio; the network-backed YouTube lane remains opt-in via --live-youtube and AUTOVOICE_LIVE_YOUTUBE_URL. MeanVC performance remains experimental and is no longer a default local-only readiness gate; run it explicitly with AUTOVOICE_MEANVC_FULL=1 and the prepared MeanVC assets when promoting that lane.

As of 2026-07-02, reports/completion/latest/completion_matrix.json references f7e851de634cd1e8bcd7fb6ba487f1dc32dcea80 (branch review-fixes-2026-07) and passed ok: true with --real-audio: zero failed lanes; skips are the documented local-mode lanes (real compose, release-candidate compose, live-youtube, Jetson CUDA/TensorRT, TensorRT engine suite/parity, benchmark-publish, and the matrix's own frontend lane — the frontend gate was run separately at that commit: typecheck, lint, build, and 18/18 Playwright smoke specs green). The same commit's full pytest run (reports/full_pytest_cov_20260702.log) measured 88% overall / 92% inference coverage with all supported lanes passing. Do not treat a latest pointer as authoritative for any other commit; a readiness claim must cite artifacts whose embedded git SHA matches the candidate commit. Release decisions remain scoped by the local/no-Docker support boundary unless hardware/deployment lanes are explicitly enabled.

The latest closeout and post-release quality plan are not contradictory when read with this vocabulary: AutoVoice has meaningful local/private deployment proof and post-release quality work, but public/commercial production release remains blocked until the release-candidate matrix, benchmark evidence, full supported pytest lanes, and hardware/model lanes are current-head green or explicitly gated.

Canonical Runtime

  • backend entrypoint: autovoice serve
  • canonical swarm entrypoint: autovoice swarm
  • frontend entrypoint: frontend/ Vite app
  • canonical REST base: /api/v1
  • canonical durable app-state store for training jobs, presets, and conversion history: AppStateStore
  • canonical profile routes: /api/v1/voice/profiles/*
  • compatibility helper routes: /api/v1/profiles/*
  • notification webhook routes: /api/v1/notifications/webhooks* (server-side fire-and-forget dispatch on training/conversion completion and job failure)
  • canonical non-karaoke Socket.IO namespace: /
  • dedicated live namespace: /karaoke
  • canonical offline pipeline: quality_seedvc
  • canonical fast/live pipeline: realtime
  • experimental pipelines: quality, quality_shortcut, realtime_meanvc
  • supported local train/serve contract: trained artifacts are served by every conversion path. Full-model training saves {profile}_full_model.pt and LoRA training saves both the deltas-only {profile}_adapter.pt (AdapterManager consumers) and a self-contained {profile}_adapter_model.pt (base + LoRA). The shared artifact loader (build_voice_model_from_checkpoint) detects the artifact family from the state dict and loads it into the class that produced it; deltas-only payloads are rejected loudly. Consumers: the offline quality pipeline (ModelManager), offline realtime jobs (trained decoder + the 80-mel universal HiFiGAN loaded by default), live karaoke sessions (full-model and adapter-model profiles route through RealtimeVoiceConversionPipeline; the streaming SOTA backend receives the profile store, serves self-contained artifacts with an atomic decoder/vocoder swap, and skips per-chunk vocal separation for mic input). The canonical quality_seedvc offline path remains reference-audio driven. Known limitation: the on-disk bigvgan_generator.pt is the official bigvgan_v2 export and is incompatible with the in-repo BigVGAN module, so the SOTA base 100-mel vocoder runs untrained; trained profiles are served through the 80-mel HiFiGAN instead.
  • canonical training feature contract: ContentVec content embeddings are 768 dims, RMVPE/PitchEncoder pitch embeddings are 768 dims, and speaker embeddings are 256 dims. CoMoSVC training jobs and regression tests must use that contract.
  • experimental quality upgrades remain behind the evidence gate defined in config/experimental_evidence.json and validated by python scripts/validate_experimental_evidence.py
  • benchmark/release promotion evidence is defined by config/benchmark_suites.json and validated by python scripts/validate_benchmark_dashboard.py
  • production completion evidence is written by python scripts/run_completion_matrix.py under reports/completion/latest/; use --real-audio for deterministic local real-audio E2E without Docker, and use --live-youtube only with AUTOVOICE_LIVE_YOUTUBE_URL set to operator-owned media
  • local HTTPS browser-capture testing is started with scripts/local_https_dev.sh; the helper generates a self-signed certificate, serves the backend with autovoice serve --ssl-cert --ssl-key, and configures the Vite frontend to proxy to the HTTPS backend. Same-machine HTTPS can be validated locally; true browser-device validation from another LAN computer remains a separate manual LAN acceptance step.

Vendor Model Repos

The nested model directories are canonical vendor repos tracked as gitlinks/submodules:

  • models/hq-svc
  • models/meanvc
  • models/seed-vc

Initialize or refresh them with:

git submodule update --init --recursive

Audit their parent-repo contract and local hygiene with:

PYTHONNOUSERSITE=1 PYTHONPATH=src /home/kp/anaconda3/envs/autovoice-thor/bin/python \
  scripts/audit_vendor_repos.py

Use --require-clean only when you explicitly want local dirt in those vendor repos to fail the audit. Local experiments or runtime artifacts inside nested vendor repos are not parent-repo product changes until they are committed in the vendor repo and the parent gitlink is updated intentionally.

Canonical Training SSIM

There is one supported SSIM implementation on the current training path:

  • src/auto_voice/models/so_vits_svc.py::_ssim_loss

There is no separate canonical src/auto_voice/training/ssim_loss.py module.

There is no separate /training Socket.IO namespace in the current backend.

Canonical Docs

Read these first:

Governance And Swarm State

  • bd is the canonical task and planning source of truth.
  • GitNexus is the canonical code-intelligence layer for repo exploration and impact analysis.
  • config/swarm_config.yaml, config/agent_contexts.yaml, config/swarm_manifests/*.yaml, and autovoice swarm ... define the canonical DAG-based swarm runner.
  • docs/swarm-operator-contract.md is the canonical operator contract for run taxonomy, required GitNexus inputs, required MemKraft writes, artifact paths, and lane completion rules.
  • scripts/launch_swarms.sh and scripts/swarm_orchestrator.py are compatibility wrappers around the repo-native swarm runner, not separate orchestration systems.
  • MemKraft is now the preferred durable swarm memory backend for autovoice swarm runs when the Python package is installed. Run ledgers under DATA_DIR/swarm_runs/ remain the canonical execution record, and DATA_DIR/swarm_memory/ stores the MemKraft channel/task/agent context.
  • autovoice swarm status reports live ledger state before completion, and autovoice swarm cancel, autovoice swarm resume, and autovoice swarm retry are the canonical run-control commands.
  • GitNexus remains the required code-context input; do not treat swarm memory as a replacement for fresh code-graph inspection.

Historical Artifacts

These are useful for archaeology but are not the current product spec:

  • conductor/
  • ORCHESTRATOR*.md
  • date-stamped coverage and readiness reports under docs/
  • generated runtime reports under reports/, output/, and output/reports/
  • older swarm/claude-flow helper scripts and notes

Use those only after validating against the canonical docs and live code paths.

Artifact And Fixture Policy

See repo-hygiene.md for the full policy. The short version:

  • canonical source lives in src/, frontend/, config/, scripts/, tests/, docs/, and explicitly tracked fixtures
  • canonical evidence is generated under ignored reports/ paths and is valid for release claims only when its embedded git SHA matches the candidate commit
  • generated media/model outputs under output/ are not tracked; do not add new generated media/model artifacts without documenting the owner, purpose, and retention rule as an explicit fixture exception
  • voice-profile export includes profile metadata, training samples, profile-scoped app-state, registered owned asset references, and recent audit events; purge removes profile-linked app-state plus registered owned asset references while retaining durable audit records
  • local runtime state, scratch outputs, and swarm run artifacts belong in ignored report/data paths, not in the root repository

Current Verification Entry Points

  • backend contract slice: targeted pytest suites in tests/, run with PYTHONNOUSERSITE=1, PYTHONPATH=src, and PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
  • canonical local test entrypoint: ./run_tests.sh
  • generated API docs: /api/v1/openapi.json, /api/v1/openapi.yaml, /docs
  • frontend build: cd frontend && npm run build
  • release-candidate validation: python scripts/validate_release_candidate.py --base-url http://127.0.0.1:10001 --wait-seconds 180 This validates /api/v1/health, /ready, and /api/v1/metrics, and verifies benchmark evidence schema/provenance against HEAD or GITHUB_SHA.
  • production completion matrix: python scripts/run_completion_matrix.py Use --real-audio in autovoice-thor to run deterministic local real-audio E2E without Docker. Use --live-youtube with AUTOVOICE_LIVE_YOUTUBE_URL=<operator-owned-url> for the opt-in network-backed smoke lane. Use --full only on a capable Jetson/compose/hosted runner; local smoke mode records unavailable frontend, compose, and hardware lanes as explicit skipped lanes.
  • full hardware RC evidence preflight: python scripts/preflight_full_hardware_rc.py --output reports/release_candidates/AV-j4cd/preflight.json --benchmark-report <current-head-benchmark-report.json>
  • full hardware RC evidence bundle: python scripts/run_full_hardware_rc.py --benchmark-report <current-head-benchmark-report.json>
  • <current-head-benchmark-report.json> must be the raw comprehensive_report.json; the RC runner derives release_evidence.json.
  • local Jetson release decision without Docker deployment: python scripts/run_full_hardware_rc.py --bead-id local-jetson --deployment-base-url http://127.0.0.1:10600 --local-base-url http://127.0.0.1:10600 --no-require-hosted-probes --no-require-production-smoke-stems --no-run-real-compose --no-run-full-hosted-preflight --no-require-docker --require-clean-head --benchmark-report <current-head-benchmark-report.json>
  • benchmark dashboard contract validation: python scripts/validate_benchmark_dashboard.py
  • hosted deployment preflight: python scripts/validate_hosted_deployment.py --hostname autovoice.giggahost.com
  • experimental evidence validation: python scripts/validate_experimental_evidence.py
  • Jetson/TensorRT validation: bash scripts/validate_cuda_stack.sh --pipeline all --output-dir reports/platform