Repository navigation
hrx: keep contiguous F32 duplicates on the device - #164
Merged
AaronStGeorge merged 2 commits intoOct 30, 2026
Merged
AaronStGeorge merged 2 commits into
AaronStGeorge merged 2 commits into
Conversation
Reuse the copy dispatch for DUP and constrain placement to the supported contiguous F32 case. Cover the active copy in the existing CPU reference test. Assisted-by: Codex (cherry picked from commit 73f4940)
AaronStGeorge
deleted the
users/astgeorg/hrx-contiguous-f32-dup-20261030
branch
October 30, 2026 20:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep contiguous F32
DUPoperations on HRX by reusing the existing copy dispatch. The mixed-embedding input duplicate otherwise runs on CPU and introduces a backend boundary. Advertise support only for nonempty contiguous F32 tensors with matching shapes, a distinct contiguous output, and at most 2^30 elements. Extend the existing CPU-reference test to execute and compare the duplicate.This is one commit on
hrx-graph-develop-v2at2120f7434eeb8d958cd0c0fab3be581e12a8f29b(after #162 and #163), cherry-picked from73f49408113923d48bcb7df7602cf43075dc2900.Performance against the current base
Local before/after measurements used the Radeon RX 7900 XTX (
gfx1100), Qwen3-0.6B Q4_0, context 4096, one slot, 16 CPU threads, and a 17-token prompt. Both builds usedsources/build.pyand the same HRX revision,82214dfb029975205d1c3a59164b678e2e21fc41. The only source difference for this comparison is the DUP patch. Each server handled three 20-token requests followed by three warm 256-token requests.The before-DUP source tree at
760fe39afis identical to the current target branch at2120f7434. The after-DUP tree is identical to this PR.This is 5.12% higher throughput, or 4.87% less generation time, for these warm requests. Generated text was byte-identical before and after. Scheduler diagnostics showed three partitions (HRX / CPU / HRX) becoming one HRX partition, removing the CPU F32 DUP (4 KiB for a single-token input).
These are three sequential warm requests from one server session per version. The result is a limited local measurement, without repeated alternating sessions or a confidence interval. The separate compiler optimizations are not included; their previously reported 2.71x gain does not quantify this change.
Validation
build.pybuild passed on pinned gfx1100.test-backend-opsF32 DUP comparison passed against CPU; unsupported types and noncontiguous inputs remained rejected.dynamic_generic_ops.cpu_referencetest now checks HRX placement, copy-kernel selection, and exact numerical agreement for the active DUP.Both CI runs used staging
c6985eaf0ad6f254f973ed122d46e6faec610eb6, llama.cpp73f49408113923d48bcb7df7602cf43075dc2900, and unchanged HRX82214dfb029975205d1c3a59164b678e2e21fc41. This PR head6945a7fe669a84c68b842a894e7759f1773848e0and the tested llama.cpp revision have exactly the same Git source tree,a1356018805a971e51fa8de997987117e083ad94; only commit history differs. The existing single-model run therefore covers the proposed source state, and no duplicate run was dispatched. CI timings are not used to claim an isolated DUP speedup.Assisted-by: Codex