Repository navigation
ggml-hrx: restore correctness after upstream interface rebase - #163
Merged
AaronStGeorge merged 4 commits intoOct 30, 2026
Merged
AaronStGeorge merged 4 commits into
AaronStGeorge merged 4 commits into
Conversation
Port the focused fix from 602f120 onto the upstream-rebase branch. Keep initializer accesses for uploads and hazards, but copy back only writes made by main commands. Add the replay regression for deferred input replacement, outputs, in-place writes, and prepared-cache reuse. Assisted-by: Codex (cherry picked from commit a5bf921)
Recognize the rebased GDN Q/K normalization graph and preserve its additive epsilon semantics while retaining L2 clamp support. Cover CPU-reference numerics, snapshot/projection entry points, and fusion boundaries. Bead: lemonade-ws-7mt.1 Assisted-by: Codex (cherry picked from commit 48263e2)
Assisted-by: Codex (cherry picked from commit 9791711)
Skip inactive branch storage and scheduling, invalidate cached programs when branch selection changes, and preserve compute masks in graph snapshots. Add host and device regressions for inactive SET_ROWS and stable-UID selection changes. Assisted-by: Codex (cherry picked from commit 760fe39)
This was referenced Oct 30, 2026
Base automatically changed from
users/astgeorg/hrx-host-input-ownership-20261030
to
hrx-graph-develop-v2
October 30, 2026 18:10
AaronStGeorge
marked this pull request as ready for review
October 30, 2026 18:11
AaronStGeorge
deleted the
users/astgeorg/hrx-rebase-interface-20261030
branch
October 30, 2026 18:11
AaronStGeorge
added a commit
that referenced
this pull request
Oct 30, 2026
## Summary Keep contiguous F32 `DUP` operations on HRX by reusing the existing copy dispatch. The mixed-embedding input duplicate otherwise runs on CPU and introduces a backend boundary. Advertise support only for nonempty contiguous F32 tensors with matching shapes, a distinct contiguous output, and at most 2^30 elements. Extend the existing CPU-reference test to execute and compare the duplicate. This is one commit on `hrx-graph-develop-v2` at `2120f7434eeb8d958cd0c0fab3be581e12a8f29b` (after #162 and #163), cherry-picked from `73f49408113923d48bcb7df7602cf43075dc2900`. ## Performance against the current base Local before/after measurements used the Radeon RX 7900 XTX (`gfx1100`), Qwen3-0.6B Q4_0, context 4096, one slot, 16 CPU threads, and a 17-token prompt. Both builds used `sources/build.py` and the same HRX revision, `82214dfb029975205d1c3a59164b678e2e21fc41`. The only source difference for this comparison is the DUP patch. Each server handled three 20-token requests followed by three warm 256-token requests. The before-DUP source tree at `760fe39af` is identical to the current target branch at `2120f7434`. The after-DUP tree is identical to this PR. | 256-token generation | Base, ms | Base + DUP, ms | | --- | ---: | ---: | | Warm request 1 | 715.798 | 669.302 | | Warm request 2 | 622.058 | 596.952 | | Warm request 3 | 608.892 | 585.599 | | Mean | 648.916 | 617.284 | This is **5.12% higher throughput**, or 4.87% less generation time, for these warm requests. Generated text was byte-identical before and after. Scheduler diagnostics showed three partitions (HRX / CPU / HRX) becoming one HRX partition, removing the CPU F32 DUP (4 KiB for a single-token input). These are three sequential warm requests from one server session per version. The result is a limited local measurement, without repeated alternating sessions or a confidence interval. The separate compiler optimizations are not included; their previously reported 2.71x gain does not quantify this change. ## Validation - Local `build.py` build passed on pinned gfx1100. - Local `test-backend-ops` F32 DUP comparison passed against CPU; unsupported types and noncontiguous inputs remained rejected. - The existing `dynamic_generic_ops.cpu_reference` test now checks HRX placement, copy-kernel selection, and exact numerical agreement for the active DUP. - **Completed single-model CI:** [41059144662](https://github.com/ROCm/ggml-staging-automation/actions/runs/41059144662), SUCCESS on gfx1151. Qwen3-0.6B chat checks, prefill/decode perplexity comparisons, and all four packaged HRX suites passed. Perplexity was HRX 20.4920 / Vulkan 20.4695 for prefill and HRX 19.2819 / Vulkan 19.2796 for decode. - **Completed full-tier CI:** [41061540987](https://github.com/ROCm/ggml-staging-automation/actions/runs/41061540987), SUCCESS: all 31 HRX chat cases and 30 perplexity models passed their underlying checks, plus all four packaged HRX suites. Both CI runs used staging `c6985eaf0ad6f254f973ed122d46e6faec610eb6`, llama.cpp `73f49408113923d48bcb7df7602cf43075dc2900`, and unchanged HRX `82214dfb029975205d1c3a59164b678e2e21fc41`. This PR head `6945a7fe669a84c68b842a894e7759f1773848e0` and the tested llama.cpp revision have exactly the same Git source tree, `a1356018805a971e51fa8de997987117e083ad94`; only commit history differs. The existing single-model run therefore covers the proposed source state, and no duplicate run was dispatched. CI timings are not used to claim an isolated DUP speedup. Assisted-by: Codex
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adapt HRX correctness to the rebased ggml graph and backend interfaces after rebase:
Validation