Repository navigation
cuda: emit banked HC MMQ activation images - #152
Draft
GenerelSchwerz wants to merge 27 commits into
Draft
GenerelSchwerz wants to merge 27 commits into
GenerelSchwerz wants to merge 27 commits into
Conversation
Carry the reviewed kernel sources from PR125, PR129 and PR132 as the matching control for the separate tensor-affine HC follow-up. Assisted-by: Codex
Preserve gate intermediates, original rounding and live HC outputs. Retain strict input alias checks and standard allocation dependencies. Assisted-by: Codex
Use the exact tested control sources from published PR130, PR129 and PR132. Assisted-by: Codex
Reuse the allocated affine match to find the planned HC_POST and retain prepared emission priority. Reject expanded normalization cuts whose terminal is not written by this kernel. Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Assisted-by: Codex
Adapt the published PR133 and PR130 bundle helpers to the PR152 parent. Keep the original banked single-image API and writer unchanged. Assisted-by: Codex
Reuse the original normalization and quantization primitives to prepare D4, DS4 and D2S6 images for eligible banked consumers in one producer. Keep existing single-image kernels and conservative dispatch contracts. Assisted-by: Codex
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
HC_POST normalization and eligible affine gates can emit banked MMQ activation images directly, retaining the original F32 outputs and producing F16/BF16 images when an existing reader needs them. Readers requiring different D4, DS4 and D2S6 layouts can now share the same producer instead of running another activation converter for each layout. These layouts serve multiple weight quantizations; this is not limited to FP4 weights.
The multiple-layout extension adds 120 lines and removes 12 across three existing CUDA files. Its separately committed prerequisite adapts the published #133 (
77fb0d934e2f10e4d5b4e00f87e89f675a018d32) bundle helpers, including their #130 prerequisite, to this branch: 279 additions and 20 deletions. The original banked implementation depends on #149 (15b12ec519711b001b6485c317e39478b5109be9) and its prerequisites. These are explicit dependencies, not additive speedups. The PR base remainsreference/upstream-kernels-20261001at official upstreamdcd387a412ca54e172a8d60eb71ef6753850c8ca.Coverage and limits
Matched component measurements
RTX 5070 Ti. The exact prior PR152 plus the adapted published bundle prerequisite is the control; only the new banked leaf differs. Three rotated AB/BA pairs per graph, with the full captured producer/reader graph timed:
The complete utility gate contains 24 graphs and 72 pairs. All 15 eligible cases were faster in 3/3 pairs; two N256 cases were near flat at 0.09% and 0.31%. Nine unchanged controls stayed within 0.047%. These are component measurements, not serving, native-implementation parity, Flash Next, MTP, older-GPU, universal or additive speedup claims.
Validation
5e-4unchanged.-j18with two heavy build slots; verified hard 12 GiB host memory, zero swap, process/time limits and complete owned-process teardown. Existing validated objects were reused. No model server was launched.Failed fixture checks and their diagnoses are preserved. The first typed probe correctly found that MMF readers did not require typed images; a later standalone-kernel name classification error was corrected in the probe. Completed controls were adopted rather than repeated. Neither required production changes or relaxed tolerances.
Exact commands, source/object/library hashes, original outputs, CPU references, image bytes and teardown proofs are retained in
results/upstream-kernel-series-20261001/HC-BANKED-MULTILAYOUT-PRODUCER-20261011/normal-source4/isolation-v1/FINAL-QUALIFIED-V1.jsonandFINAL-RESULTS-V1.md. Earlier single-image evidence remains underHYPERCONNECTION/HC-BANKED-MMQ-EMISSIONin the same benchmark workspace.Requirements