[Kernel][SM70] Integrate repaired E4M3 FP32 small-query attention (#524) - #550
Merged
yangzhuxinyzx merged 14 commits intoSep 7, 2026
Merged
Conversation
Preserve explicit per-query causal lengths and zero graph padding for the single-request small-Q route. Keep legacy E5M2 and sparse defaults unchanged. Assisted-by: OpenAI Codex Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Assisted-by: OpenAI Codex Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Compensate QK and probability residuals, keep PV accumulation tile-local, and reject stale precision binaries. Preserve default-off routing and record the failed same-process 128K token gate; this is not model promotion. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
…integration-20260906-050258 Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com> # Conflicts: # docs/design/sm70_v100_migration_control.md
Retain current-main migration history and distinguish verified small-Q replay from incomplete full-model and planner integration admission. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Retain the q8 E4M3 capability and add padded-stride graph regressions. Runtime/model acceptance remains pending; no default promotion. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Keep model admission pending and distinguish pinned core dependencies from the rebuilt Flash-V100 extension. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Preserve max/sum and unnormalized PV, version the workspace contract, and cover exact FP16 midpoint rounding. Keep the route experimental pending integration and model gates. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Retain the non-monotonic per-input results and the aborted startup provenance; model admission remains pending. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Preserve the completed 128K pass and boundary-256K failure without transferring operator admission to model acceptance. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Clone only exceptional unaligned contiguous Q on the active stream. Preserve aligned arithmetic and graph behavior; retain failed model counterfactuals in the admission log. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Record the completed FP32 SSM/head brackets and late GDN build provenance; do not approve the failing boundary route. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Merge #524 onto latest main, preserve v37 prefill, validate same-KV arithmetic and enable compatible precision-revision-3 route by default. Strict greedy identity is not the maintainer acceptance criterion. Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
Co-authored-by: OpenAI Codex <noreply@openai.com> Signed-off-by: yangzhuxinyzx <153831768+yangzhuxinyzx@users.noreply.github.com>
yangzhuxinyzx
marked this pull request as ready for review
September 7, 2026 10:37
This was referenced Sep 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-assisted maintainer-requested integration of #524, retaining original ancestry and latest main (#548 and #549). No model-identity restrictions and no greedy-identity acceptance gate.
Purpose: precision-revision-3 compensated attention with FP32 partial state, explicit row lengths, safe aligned/strided loads. Compatible SM70 E4M3 small-Q route defaults on; stale binary, other dtype/device/layout and explicit rollback retain existing fallback. The separate v37 prefill bridge remains unchanged.
Test Result: fresh CUDA 12.8 Flash-V100 build, 246 focused kernel/routing tests passed on owned V100. Includes FP64 same-KV references through 256K and changing-state CUDA Graph tests. Alternating paired operator graphs improve vs current scalar fallback from 204.256 to 63.853 us (Q2/8K), 405.286 to 69.882 us (Q5/8K), 14626.714 to 1303.667 us (Q5/256K). Not model-throughput claims. Historical model runs retain coherent alternative text at near-ties and rounding-floor attention error; strict-token failures are preserved in the audit document. No E2E suite rerun. Local scoped pre-commit passed; final default-policy rerun and hosted gates pending.