Skip to content

feat: add Intel XPU inference support - #35

Merged
baicai-1145 merged 4 commits into
pymss-project:mainfrom
undownding:feat/intel-xpu-support
Nov 3, 2026
Merged

baicai-1145 merged 4 commits into
pymss-project:mainfrom
undownding:feat/intel-xpu-support

Conversation

@undownding

Copy link
Copy Markdown
Contributor

Summary

  • Add explicit and automatic Intel XPU device selection with device ID validation and cache cleanup.
  • Enable XPU STFT/ISTFT, overlap finalization, and the FP32 chunked inference fast path.
  • Document PyTorch XPU installation and add runtime and CLI coverage.

Validation

  • XPU, NPU, device-selection, and overlap tests: 116 passed, 13 skipped.
  • Real Intel Arc A770M smoke test: automatic selection, STFT/ISTFT round trip, chunked demix, and cache cleanup passed.
  • uv build passed.
  • Full suite: 730 passed, 23 skipped. The WebUI failure also reproduces on origin/main; this PR does not change server code. The loudnorm test passes from the standard checkout layout (its nested-worktree error was caused by the sibling-plugin fixture path).

@undownding

undownding commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor Author

Smoke tests passed on Intel Arc 770M, Ubuntu 26.04

@undownding
undownding marked this pull request as ready for review October 28, 2026 01:06
@baicai-1145

Copy link
Copy Markdown
Member

Thanks — the device-selection refactor is clean and sharing _validate_device_ids across backends is a real improvement. I can't merge this yet: there's one blocking bug, and I'd like the change split by risk.

Blocker — pymss serve --device xpu is rejected. serve_parser (pymss/cli.py:985) still has the old choices list, so the HTTP server / WebUI path cannot use XPU at all. Verified on 979295d: pymss serve --device xpu → error: argument --device: invalid choice: 'xpu' (choose from 'auto', 'cpu', 'cuda', 'rocm', 'mps', 'npu', 'mlx'). test_cli_accepts_xpu only covers infer/workflow/comfy, which is why CI missed it. Please also update the --device-id help at pymss/cli.py:991.

Evidence. The PR body and your comment say smoke tests passed on an Arc A770M, but there are no commands or output. Could you paste the commands used plus torch.xpu.is_available() / torch.xpu.get_device_name(0) output, RTF for at least one model on XPU vs the same machine's CPU, and si_sdr(XPU output, CPU output) for one model — the repo already ships pymss.kit.sdr for that.

The FP32 decision is implicit. "XPU currently uses the FP32 inference path" is enforced only because _autocast (pymss/utils.py:308) and vr_separator.py:336 happen to exclude xpu. That's fragile and undocumented. Please either make it explicit (shared constant + comment + a test asserting _autocast("xpu", True) returns nullcontext), or support bf16 properly. Note XPU AMP is fully supported and bf16 — not fp16 — is the right dtype there, so that should be its own PR with accuracy numbers.

Tests and docs. test_auto_prefers_xpu_over_npu (test/test_xpu_runtime.py:32) patches NPU to unavailable, so it doesn't test the ordering it is named after. There is no coverage for the xpu branch in del_cache. _xpu_runtime_available and _validate_device_ids have no docstrings while the NPU counterparts do. docs/msseparator.md:98 and the CN version still say "CUDA device IDs" while the docstring now says "Accelerator device IDs".

Suggested split. (1) device selection / validation / CLI / docs — mergeable once the serve bug is fixed; (2) hardware paths (STFT/ISTFT, overlap, cache) — mergeable with the evidence above plus an "experimental" note in the docs; (3) bf16 AMP and per-model RTF work — separate PR, and I'd want a benchmark harness first.

@undownding

Copy link
Copy Markdown
Contributor Author

Implemented the review fixes in 4f4698a, with the reproducible benchmark and hardware evidence in ec35a6c. I checked the items below against the pushed head, ec35a6c0de77e2c2a1c9a7e3683c9fd3fd0e6c60.

Implemented review items

  • Server CLI blocker: serve --device xpu is now accepted. The --device-id help says accelerator device ID and explains that XPU/NPU use the first ID. test_cli_accepts_xpu now includes serve --device xpu --device-id 1, alongside infer/workflow/comfy. CLI, regression tests.
  • Explicit FP32 policy: FP32_INFERENCE_DEVICE_TYPES = frozenset({"xpu"}) and its explanatory comment make the decision explicit. _autocast returns nullcontext for XPU, and VR now calls that same helper. Tests cover xpu and xpu:0, with AMP requested both on and off, and assert that torch.amp.autocast is never invoked. This implements the FP32 option requested in the review; it does not add bf16 AMP. Shared policy, VR integration.
  • Device priority: test_auto_prefers_xpu_over_npu now makes both XPU and NPU available, with CUDA unavailable, and verifies selection of xpu:0.
  • Cache coverage: added tests verifying XPU cache clearing for an XPU device, no XPU cache clearing for CPU, and garbage collection in both cases.
  • Docstrings and device IDs: added docstrings to _xpu_runtime_available and _validate_device_ids; both English and Chinese API docs now describe accelerator IDs and the first-ID-only behavior for XPU/NPU.
  • Experimental status: both READMEs explicitly mark XPU hardware inference as experimental, explain the FP32 policy even when use_amp=True, and state that bf16 requires separate accuracy/performance validation.

Hardware evidence and reproduction

Added test/manual/xpu_benchmark.py, raw results, and documented commands and limitations.

Environment: Ubuntu 26.04.1, Intel Core i7-12700H, Intel Arc A770M, PyTorch 2.10.0+xpu. Commands run from the repository:

.venv/bin/python -c 'import torch; print(torch.xpu.is_available()); print(torch.xpu.get_device_name(0))'
# True
# Intel(R) Arc(TM) A770M Graphics

.venv/bin/python -m test.manual.xpu_benchmark --input .tmp/input.wav

Model: locally cached bs_roformer_voc_hyperacev2. Identical input on both devices: first 10 seconds, stereo, 44.1 kHz. Identical inference settings: FP32, batch size 1, chunk size 132300, overlap size 22050, use_amp=False. CPU used 4 Torch threads. One warmup and one timed separation per device; loading and file I/O are excluded, and XPU is synchronized around timing. RTF = elapsed seconds / input duration.

Device Elapsed RTF
CPU 25.8937 s 2.58937
XPU 1.34794 s 0.13479

Using the shipped pymss.kit.sdr, with CPU output as reference and XPU output as estimate:

Stem SI-SDR per channel (dB) Mean SI-SDR (dB)
Vocals 13.5963, 13.1912 13.3938
Instrumental 94.0761, 94.9531 94.5146

Accuracy limitation: the vocal output is nearly silent (CPU RMS 8.48e-7, XPU RMS 8.51e-7). These measurements establish output agreement for this sample, not general vocal separation accuracy. More representative vocal material is still needed. The input is a local clip and is not distributed; its decoded PCM hash is included in the results, and the harness accepts another input file to reproduce the procedure.

The same run selected xpu:0 automatically, completed real XPU STFT/ISTFT with maximum absolute round-trip error 7.15e-7, ran chunked model inference, and completed cache cleanup.

Verification and scope

Re-ran the relevant tests on a clean export of the exact pushed commit (without the unrelated local server changes):

python -m pytest -q test/test_xpu_runtime.py test/test_npu_runtime.py \
  test/test_separator_device_defaults.py test/test_overlap_finalize.py test/test_use_amp.py
# 127 passed, 13 skipped

The suggested split into separate device-selection and hardware-path PRs has not been implemented: both remain in this PR. The fixes and benchmark evidence are separate follow-up commits, not separate PRs. Hardware paths are now documented as experimental. No bf16 AMP or broad per-model performance work is included; the benchmark harness provides a starting point for that future work. I am therefore not claiming that the PR-splitting suggestion or comprehensive accuracy validation is complete.

@baicai-1145
baicai-1145 merged commit 4b1e65d into pymss-project:main Nov 3, 2026
baicai-1145 added a commit to baicai-1145/pymss that referenced this pull request Nov 5, 2026
Resolve conflicts by keeping the profile-driven device layer while
preserving main's XPU work (PR pymss-project#35):

- pymss/cli.py: --device choices come from SUPPORTED_DEVICES for every
  subcommand, so xpu and the vendor aliases cannot drift apart again.
- pymss/separator.py: keep both _VENDOR_ALIASES and _validate_device_ids;
  auto order is cuda -> xpu -> npu -> gcu/mlu/supa -> mps -> cpu; explicit
  --device xpu keeps the "XPU-enabled PyTorch build" guard.
- pymss/utils.py: _autocast stays profile-driven. FP32_INFERENCE_DEVICE_TYPES
  is dropped because DeviceProfile.amp is now the only source of truth
  (XPU reports amp=False). _finalize_overlap and the chunked fast path use
  device_profile().is_accelerator, which covers xpu.
- pymss/modules/vocal_remover/vr_separator.py: one AMP policy via
  pymss.utils._autocast instead of a duplicate autocast call site.
- spec_utils.py: keep the profile-based accelerator test.
- pymss/plugins/manifest.py: keep math plus the tomllib/tomli fallback.
- pyproject.toml: 2.4.0, pymss-core>=0.1.15, packaging>=24.0 from main.

Note: test_graph_auto_node_uses_the_first_id_with_an_explicit_gpu_runtime
fails identically on clean origin/main (pymss.graph.nodes imports
register_node("input_audio") outside the builtin bootstrap), so it is a
pre-existing main-branch defect, not a merge regression.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants