Repository navigation
feat(device): support --device tpu through torch_tpu - #43
Open
baicai-1145 wants to merge 1 commit into
Open
baicai-1145 wants to merge 1 commit into
baicai-1145 wants to merge 1 commit into
Conversation
torch_tpu is the official PyTorch TPU backend (not torch_xla) and needs Python 3.13+ with torch 2.11-2.15. It renames PrivateUse1 to "tpu", and both torch.tpu.is_available/device_count and torch.tpu.set_device are rejected as "internal to TorchTPU", so availability and index selection go through torch.accelerator instead. TPU is probed last in the auto path: importing torch_tpu starts the libtpu runtime, so every cheaper backend gets to answer first. docs/tpu.md records the whole measurement set -- operator parity, the istft and fold numerics fixes, group_norm contiguity, matmul precision, per-family throughput and the torch.compile numbers.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
--device tpusupport for Google TPU viatorch_tpu(official PyTorch TPU backend, nottorch_xla). Needs Python 3.13+ and torch 2.11–2.15.Companion PR: pymss-project/pymss-core#14 — the device profile, the numerics fixes and the
torch.compileunblocking all live there.Changes
separator.py:tpuadded to the vendor aliases and the public-device whitelist._accelerator_available(): availability goes throughtorch.accelerator, becausetorch.tpu.is_available/device_countare rejected as "internal to TorchTPU" (torch.tpu.set_devicelikewise)._select_private_deviceusestorch.accelerator.device_countfor the same reason.torch_tpustarts the libtpu runtime, so every cheaper backend gets to answer first.docs/tpu.md: the full measurement set (operator parity, istft/fold numerics, group_norm contiguity, matmul precision, per-family throughput, torch.compile numbers).Highlights from docs/tpu.md
TPU has no operator gaps — complex STFT, RNN, grouped/dilated conv, fold/unfold and SDPA are all native. The adaptation is about numerics:
torch.istftistft_npu(3.8e-05, 205 ms)F.fold/col2imbf16 accumulatescatter_add_shim, 4.2 ms (24x faster)F.group_normnon-contiguousview():— SCNet and htdemucs both abort →.contiguous()matmul_precision="high"gives 1.4e-05 for freeAfter the fixes:
legacy_hdemucsend-to-end deviation 1.60e-05, and repeated identical input gives 0.0.Performance
Single forward, 5 s stereo audio, TPU v5e1:
legacy_demucslegacy_hdemucslegacy_v3_demucsapollo/scnet/mdx23cbandit/bandit_v2bs_roformer_hyperacebs_conformerbs_roformermel_band_conformermel_band_roformerTwo things worth knowing before tuning
use_amp:scnet2.56x andbandit1.78x at bs=8, butbs_roformergains just 7%.Utilisation is 3.8–14.5% of the 50.2 TFLOPS fp32 peak and dispatch is only 20–46% of runtime, so the wall is per-op granularity (45.5 µs per dispatched op vs ~4 µs on CUDA), not FLOPs. A 4-minute song:
bs_roformer34 s fp32 / 18 s bf16.