HRX: restore consumer-driven activation publication (dropped whole in f2099e9b) #449

Activity

1. bong-water-water-bong commented on Oct 18, 2026

   

   bong-water-water-bong

   on Oct 18, 2026

   CollaboratorAuthor

   More actions

   The accepted regression now has a guard that cannot be quietly accepted: 1bit-MONSTER/llama.cpp#105 (added to the open #105 branch, base 8dafc4a0).

   • hrx-backend-test runner has a KNOWN-FAIL status: a tagged case must fail, prints [KNOWN-FAIL #449] and is counted apart from real failures; if it ever passes it prints [UNEXPECTED-PASS #449] and the run exits non-zero, so the marker has to be retired on purpose.
   • Guards restored with their original pre-HRX: pin llama.cpp 96f6b89, fixing #95 (GLM-4.7-Flash, Qwen3-Coder at full context) #104 expectations (adapted only to compile: activation_publication_diagnostics() is alternate_values() now): generic_swiglu_k16_publish, tiled_matmul_alternate_publish, rmsnorm_binary_f16_output, qwen_decode_publication_consumer_qualification, qwen_weighted_reduce_next_rmsnorm_q8_publication (re-added - HRX: pin llama.cpp 96f6b89, fixing #95 (GLM-4.7-Flash, Qwen3-Coder at full context) #104 had deleted them) and packed_f16_producer_consumer, rmsnorm_binary_k16_output, gdn_rmsnorm_gate_dispatch (put back from HRX: pin llama.cpp 96f6b89, fixing #95 (GLM-4.7-Flash, Qwen3-Coder at full context) #104's rewritten-to-pass versions).

   Host suite on CPU: 66 cases - 44 passed, 13 failed, 9 known-fail #449/#450, 0 unexpected-pass. All eight #449 rows report KNOWN-FAIL, so the lost publication paths stay visible until the kernels and registrations come back or the loss is ruled accepted, at which point the tag is removed deliberately.
2. bong-water-water-bong commented on Oct 18, 2026

   

   bong-water-water-bong

   on Oct 18, 2026

   CollaboratorAuthor

   More actions

   Per-family evidence for the 8 #449 known-fail guards, gathered on the current line: fork
   1bit/hrx-vulkan-patched @ 7227a77, hrx-system 2ab3114966a4, hrx-backend-test built Release
   against HRX_SOURCE_DIR, host suite on CPU (ryzen, no GPU). Baseline on this pin is exactly what the
   guard PR reported: 44 passed, 13 failed, 9 known-fail, 0 unexpected-pass, 66 total. Each guard was run
   with a temporary plan dump (kernel names, bindings, alternates, generated resources; reverted).

   Correction to this issue's kernel table

   There is no *publish*-named kernel in the corpus on this pin at all. Counting kernel.def export(...) names across the four manifests: 124 exports, 0 containing publish. In particular
   ggml_rmsnorm_gate_f32_publish is corpus=N, not corpus=Y: the AMD-core re-head kept the
   rmsnorm_gate_f32.publish_body template and split it into two exports,
   ggml_rmsnorm_gate_f32_f16 and ggml_rmsnorm_gate_f32_q8_1_x4, with the publication format as a
   compile parameter. So the "cheap one, only the registration was removed" premise in the task does not
   hold; whatever is restored here has to be written for the current corpus, not re-registered.

   What each guard actually fails on, and what that means

   Guard	Failing requirement	Plan on this pin	Verdict
   gdn_rmsnorm_gate_dispatch	counts ggml_rmsnorm_gate_f32_publish	llm_gated_delta_net_f32_wmma_head128, ggml_copy_f32, ggml_rmsnorm_gate_f32_f16 (5 bindings, publishes a 262144 B F16K16Major generated resource), ggml_mul_mat_q4_k_f16_wmma_prefill_wave32 (f16_input_layout=1), ggml_scale_bias_f32	name only. Publication is present and consumed. Fixed in llama.cpp#106 (expectation + marker retired).
   qwen_decode_publication_consumer_qualification	(q8 != nullptr) == test.q8 on the UnsupportedWeight case	qwen3_moe_rmsnorm_f32_quantize_q8_1_x4 publishes a 2304 B q8_1 x4 alternate for a BF16 consumer (and for a 63-output consumer); Dense and AliasedDense are fine	real. Publication is no longer consumer-driven: has_decode_q8_consumer() in dispatch-qwen-rmsnorm.cpp accepts any MUL_MAT/MUL_MAT_ID consumer. A qualified-consumer check restores it (Q4_K/Q6_K weights with the right geometry; the q8-consuming consumer kernels exist: qwen3_moe_dense_linear_q4k_q8_1_x4_next_q8, ..._routed_gate_up_swiglu_q4k_q8). The RoutedRoute case additionally needs the routed chain to be qualified as a whole, not just the first consumer.
   qwen_weighted_reduce_next_rmsnorm_q8_publication	producer kernel name	match at the weighted node is qwen3_moe_routed_down_weighted_reduce_f16_f32 (4 bindings, no alternate); the fused ..._next_rmsnorm_f32 path exists (5 bindings) but is not matched in this graph	real, cheap. The q8 kernel is in the corpus (qwen3_moe_routed_down_weighted_reduce_next_rmsnorm_q8_1_x4, qwen_moe manifest, config decls identical to the f32 variant) and registered nowhere. Needs the registration plus the consumer qualification, i.e. the same work as the row above.
   rmsnorm_binary_k16_output	producer is ggml_rmsnorm_binary_f32_k16	ggml_rmsnorm_binary_f32, ggml_copy_f16_k16_major, ggml_mul_mat_q4_k_f16_wmma_prefill_wave32	real, partially restorable. The k16 kernel is in the corpus and registered, but has_qualified_rmsnorm_k16_consumer() allows only tokens=512 x hidden {4096,5120,6144,8192} and 1024 x 5120. The kernel itself needs hidden a multiple of 1024 in [1024,8192], so the guard's 3072 case cannot be served by it at all; 2048 can. Either restore a 128-wide k16 publisher or split the row.
   rmsnorm_binary_f16_output	producer is ggml_rmsnorm_binary_f32_f16	ggml_rmsnorm_binary_f32, ggml_copy_f32_f16, ggml_mul_mat_q4_k_f16_wmma_prefill_wave32 (f16 alternate 7340032 B present)	real. No _f32_f16 export in the corpus; the f16 alternate exists but costs a separate copy dispatch per layer.
   tiled_matmul_alternate_publish	producer is ggml_mul_mat_tiled_input_f32_publish_f32_f16_alternate	ggml_mul_mat_f32_f32_wmma, ggml_copy_f32_f16, ggml_mul_mat_q6_k_f16_wmma_prefill_wave32 (f16 alternate present)	real. Alternate exists, the fused publish producer does not: one extra dispatch per layer. Kernel-side work in mul_mat_f32_f32_wmma.
   generic_swiglu_k16_publish	producer is ggml_mul_mat_tiled_pair_input_f32_binary_publish_f32_k16	ggml_mul_mat_swiglu_f32_f32_wmma, ggml_copy_f16_k16_major, ggml_mul_mat_q6_k_f16_wmma_prefill_wave32	real. Same shape as above; the swiglu producer emits f32 and the pack is a separate dispatch.
   packed_f16_producer_consumer	(alternate != nullptr) == test.alternate on the first sub-case that expects the f16 alternate	ggml_copy_f16_k16_major, ggml_mul_mat_swiglu_q4_k_f16_wmma_prefill_wave32 (f16_output_layout=1), q6_k projection; two F16K16Major generated resources, no f16 alternate	real. The packed path is fused, the unpacked f16 alternate is not published by the producer any more.

   Shape of the remaining work, cheapest first

   1. qwen_weighted_reduce_next_rmsnorm_q8_publication: register the q8 kernel that is already in the
      corpus. Needs a device check (numbers + logits against the f32 path).
   2. qwen_decode_publication_consumer_qualification: make publication consumer-driven again in
      dispatch-qwen-rmsnorm.cpp; same device check, plus the negative cases are plan-level and cheap.
   3. rmsnorm_binary_k16_output / rmsnorm_binary_f16_output: backend kernel work (k16 for 128-wide
      hiddens, _f32_f16 export for the plain f16 alternate), then the shape gate.
   4. tiled_matmul_alternate_publish, generic_swiglu_k16_publish, packed_f16_producer_consumer: one
      family, the f16/k16 alternate emitted from inside the producer; biggest kernel-side job.

   Items 1 and 2 change what runs on the device, so each gets test-backend-ops -b HRX0 plus a Qwen3 MoE
   and a dense model tg/pp A/B against this pin (dispatch count per token, tok/s) and the needle 7341
   @4732 before a marker is retired; items 3 and 4 additionally need pp/tg on the affected prefill shapes.
   Everything above is plan-level evidence only; no device run was made for this comment.

   Command for reproducing the plan dumps: build hrx-backend-test with
   -DGGML_HRX=ON -DGGML_HRX_KERNEL_CORPUS_SOURCE_FORMAT=binary -DHRX_SOURCE_DIR=<hrx-system at 2ab3114966a4>
   and -DGGML_HRX_HIP_TARGETS=gfx1151, then run it on CPU (the device cases skip without a GPU).
3. bong-water-water-bong commented on Oct 18, 2026

   

   bong-water-water-bong

   on Oct 18, 2026

   CollaboratorAuthor

   More actions

   Hygiene pass (2026-10-18): the fork side moved, the issue is not fixed end to end, so this stays open.

   Correction to this issue's own list. The original comment named gdn_rmsnorm_gate_dispatch as one of the rows this issue files against. That row has since been ruled the other way and retired: fork #106 (953b6397, "tests: retire the #449 guard for gdn rmsnorm-gate publication", engine #449). The reason given is that loom_libs:ggml_rmsnorm_gate_f32_publish was not a kernel that shipped and died - ggml_rmsnorm_gate_f32_f16 and _q8_1_x4 run the same publish_body and publish the packed alternate from inside their own dispatch, selected by f16_output_row_width, so the row's "publication unreachable" claim did not hold on this pin. If that reasoning is right it belongs here too, because it says the set is smaller than 8; worth a ruling rather than another guard.

   What is still red on purpose, at branch tip 953b6397 (tests/hrx-backend-test.cpp, tagged "#449"): packed_f16_producer_consumer, rmsnorm_binary_k16_output, generic_swiglu_k16_publish, tiled_matmul_alternate_publish, rmsnorm_binary_f16_output, qwen_decode_publication_consumer_qualification, qwen_weighted_reduce_next_rmsnorm_q8_publication - 7 rows, plus #450's symmetric_i4_consumer_qualification. Totals after #106: 45 passed, 13 failed, 8 known-fail, 0 unexpected-pass (66), from that commit's own message.

   Not done here. No suite run and no build: an HRX build on this box is the engine's ExternalProject (TheRock toolchain, GPU lock) and the box was busy with hw_gate. The per-family evidence already in this thread stands; what is missing is the ruling on the remaining 7 rows, not a measurement.

   Closing this needs either the kernels back (or the registration fixed) so the 7 rows pass, or a per-row ruling like #106's that retires them with the reason. Until then the tags are correct and the runner keeps them from being quietly accepted.
4. bong-water-water-bong commented on Oct 18, 2026

   

   bong-water-water-bong

   on Oct 18, 2026

   CollaboratorAuthor

   More actions

   Device verification of the #449 guards, from the T422 GPU slot (strixhalo, gfx1151, build d9fac8aa):

   • On the device run of hrx-backend-test, all eight known_fail("#449") rows report KNOWN-FAIL, none UNEXPECTED-PASS: packed_f16_producer_consumer, rmsnorm_binary_k16_output, gdn_rmsnorm_gate_dispatch, generic_swiglu_k16_publish, tiled_matmul_alternate_publish, rmsnorm_binary_f16_output, qwen_decode_publication_consumer_qualification, qwen_weighted_reduce_next_rmsnorm_q8_publication.
   • Device suite totals: 11 cases, 10 passed, 1 failed (zero_output_device_support, a supports_op capability claim - reported on hrx-backend-test: 49 host cases red after the AMD-core re-head — classification (stale expectation vs needs a ruling) #445, unrelated to publication). Host part unchanged at 44/13/9.

   So the publication losses this issue records are still visible on hardware, and the guards cannot rot silently: the day the kernels and registrations come back (or the loss is ruled accepted), the row turns UNEXPECTED-PASS and the run fails until the marker is retired on purpose.
5. bong-water-water-bong commented on Oct 18, 2026

   

   bong-water-water-bong

   on Oct 18, 2026

   CollaboratorAuthor

   More actions

   Items 1 and 2 are implemented and past the device gate. Both PRs stacked, markers retired in the same PRs.

   What changed. (1) llama.cpp#107 registers qwen3_moe:qwen3_moe_routed_down_weighted_reduce_next_rmsnorm_q8_1_x4, which was in the corpus and registered nowhere: the fused reduce+next-RMSNorm publishes the Q8_1 x4 next-projection input in place of the f32 row (same five bindings as the f32 form, so it removes the f32 row and the separate quantize). (2) llama.cpp#108 makes dispatch-qwen-rmsnorm.cpp publication consumer-driven again: has_decode_q8_consumer() accepted any MUL_MAT/MUL_MAT_ID, so it quantized a value for BF16 and 63-output consumers too. Both use one shared qualification header (qualified dense q8 projection, directly or through a full-range alias, or the fused routed-MoE chain).

   Device gate, HRX0 on strixhalo (Thermal-run wrapper, one process at a time, Tctl <= 65 C):

   • test-backend-ops -b HRX0: Testing 2 devices, Backend 1/2: HRX0, 1141/1141 tests passed.
   • llama-bench -p 512 -n 128 -r 2 -dev HRX0 -ngl 99 -t 4, every row devices=HRX0, gpu_info=AMD Radeon 8060S Graphics (gfx1151):

   model	arm A 953b639	arm B (1+2)
   Qwen3-Coder-30B-A3B pp512	2041 +-22 t/s	2036 +-18 t/s
   Qwen3-Coder-30B-A3B tg128	91.317 +-0.013 t/s	91.864 +-0.069 t/s
   Qwen3-8B pp512	2267 +-39 t/s	2342 +-4 t/s
   Qwen3-8B tg128	42.647 +-0.027 t/s	42.703 +-0.008 t/s

   • needle 7341 @4732 (Qwen3-8B, arm B): 3/3 OK, answer 7341, ptok 4732, same sha each rep.
   • Host suite (CPU, ryzen): 48 passed, 12 failed, 6 known-fail, 0 unexpected-pass, 66 total. Both HRX: restore consumer-driven activation publication (dropped whole in f2099e9b) #449 markers retired; the 12 red rows are the unchanged class A/B rows plus qwen_router_top8_dispatch.

   Box note, affects other slots. HRX registers no device unless IREE_HAL_AMDGPU_LIBHSA_PATH points at TheRock's libhsa-runtime64.so.1 (~/therock100/lib/python3.14/site-packages/_rocm_sdk_core/lib/…): the distro's answers the PM4_EMULATION probe wrongly on gfx1151, which is what cmake/hrx.cmake documents. Without it llama-bench --list-devices prints (none), test-backend-ops -b HRX0 silently runs CPU-only ('Skipping'), and llama-server says 'no usable GPU found'; backends=HRX in a llama-bench CSV only means the registry loaded, not that a device was used. Worth re-checking any earlier HRX device results that did not pass this variable.

   Remaining for #449: items 3-4 (rmsnorm_binary k16/f16 fused publisher, tiled-matmul and swiglu f16/k16 producers) are the four fused-producer rows; per the orchestrator they wait until these two land.
6. bong-water-water-bong

   added a commit that references this issue on Oct 19, 2026

   Move the llama.cpp pin to 00f2a68de: qwen3moe q8 next-RMSNorm writes …

   ...

   41ff9df
7. bong-water-water-bong commented on Oct 23, 2026

   

   bong-water-water-bong

   on Oct 23, 2026

   CollaboratorAuthor

   More actions

   Ruled out / landed at the current pin, PR: 1bit-MONSTER/llama.cpp#126 (closes this issue).

   Short version: the corpus on this pin has 99 kernels and none named publish — publication is done in the consumer (common_prepare_f16_input() emits ggml_copy_f32_f16 / common.mul_mat.f16 and records the alternate; common_prepare_k16_major_f16_input() the same for the F16K16Major pack), so what f2099e9b removed is the fusion into the producing dispatch, not the publication. The five remaining known_fail("#449") guards now assert that publication and all five pass; markers retired.

   Measured cost of the whole F16/K16 path on/off (arm B disables common.mul_mat.f16, …k16_major_f16, …swiglu.k16_major_f16, …rmsnorm_binary.k16_major_f16, …rmsnorm_gate.k16_major_f16), strixhalo gfx1151, llama-bench -p 512 -n 128 -r 2 -dev HRX0 -ngl 99 -t 4, 90 W cap, PCM pad (23 Oct): Qwen3-8B pp512 2070.84 ± 5.62 vs 2115.34 ± 24.50 t/s, tg128 42.45 ± 0.05 vs 42.53 ± 0.03 t/s — publishing costs nothing measurable, which is why the two shapes where nothing is published (rmsnorm_binary hidden 7168, the two-token silu graph) are ruled out rather than restored from history.

   Host suite at the pin: 58 passed, 0 failed, 8 known-fail (#533 x2, #534, #445 x5), 0 unexpected-pass, exit 0. Device half and the 30B-A3B arm were cut short by the release-candidate holds and are re-queued.
8. bong-water-water-bong commented on Oct 23, 2026

   

   bong-water-water-bong

   on Oct 23, 2026

   CollaboratorAuthor

   More actions

   Closed by 1bit-MONSTER/llama.cpp#126 (merged 9b846c17, pi issues-b): on the current pin the corpus has no publish kernels because consumer-driven publication happens in the consumer (common_prepare_f16_input / common_prepare_k16_major_f16_input record the alternate; the gated producers that fuse the pack are still registered). What f2099e9b removed was the fusion of the pack into the producing dispatch, not the publication; hrx-backend-test now asserts the publication each line makes, and the eight known_fail("#449") rows are retired. Cross-repo 'Closes' does not fire from the fork, hence this manual close.
9. bong-water-water-bong

   closed this as completedon Oct 23, 2026