Skip to content

[Build][SM70] Copy FlashQLA sources into the csrc wheel stage - #411

Closed
ga-it wants to merge 1 commit into
1CatAI:mainfrom
ga-it:review/sm70-flash-qla-build-input
Closed

[Build][SM70] Copy FlashQLA sources into the csrc wheel stage#411
ga-it wants to merge 1 commit into
1CatAI:mainfrom
ga-it:review/sm70-flash-qla-build-input

Conversation

@ga-it

@ga-it ga-it commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

[Build][SM70] Copy FlashQLA sources into the csrc wheel stage

AI-assisted draft. AI assistance was used for upstream comparison,
implementation, validation, live build profiling, and this write-up. The
human submitter must review the changed line and run the relevant tests before
submission, then add their review confirmation while retaining this
disclosure.

Purpose

setup.py bundles the native flash_qla_sm70_gdn_strided extension from the
tracked flash_qla/.../sm70/csrc/gdn_forward.cu source, but the Docker csrc
wheel-build stage copies csrc/ and not flash_qla/. A clean Docker build can
therefore reach the setup hook without its required source tree.

This PR adds the missing build input:

COPY flash_qla flash_qla/

This is specifically an SM70/V100 build repair. It does not change Blackwell
kernels or runtime selection.

Base SHA: 62ad1e02693f4c857f3b7547cef1860ee54e8053.

Duplicate-work audit

The full open PR list and relevant build issues were refreshed on 2026-08-28.
No current PR supplies flash_qla to this builder. Closed
PR 43 was a broad, stale V100
build proposal and does not contain this current-main one-line fix.
Issue 193 documents SM70
wheel/runtime compiler requirements but does not fix the missing Docker build
context.

Test Plan

Apply the one-line patch to a clean checkout at the recorded base, confirm the
tracked FlashQLA source consumed by setup.py is present in the csrc-builder
context, run patch hygiene checks, and build the SM70 CUDA image. Smoke the
resulting TP4 V100 service through its health, model, metrics, and completion
endpoints.

Test Result

Test-Path flash_qla/ops/gated_delta_rule/chunk/sm70/csrc/gdn_forward.cu
True

static check: Dockerfile contains COPY flash_qla flash_qla/
True

static check: setup.py requires the same gdn_forward.cu source
True

git diff --check
exit 0

git apply --check 0002-upstream-docker-copy-flash-qla.patch
exit 0 against 62ad1e0

The patch was also applied to a disposable Linux worktree on gazasrv16.
git diff --check passed, the tracked flash_qla package and its ops tree were
present, and the builder-stage COPY was confirmed at line 347. No production
container or GPU was touched.

The integrated source image completed a full CUDA 12.8.1 SM70 build and
produced a 23,369,579,173-byte image. Imports succeeded for vLLM core, MoE,
the native SM70 sampler, FlashAttention-V100, and FlashQLA. The retained image
digest is
sha256:d0fdeefbea5b61a12caa75e57543e06a01b2dadf3ea4eb74d0dfa0a90a48b95a.

The local preparation host's Docker Desktop daemon was not running, so a second
local docker buildx build --check result is not claimed. The full live image
build is the end-to-end build evidence.

V100 runtime smoke evidence

These figures establish that the resulting TP4 V100 image starts and serves;
the one-line copy change is not claimed to cause the runtime speedups.

Request TTFT Decode MTP4 acceptance Total
6,316 prompt / 512 output 2.9538 s 87.0357 tok/s 73.2824% 8.8249 s
564,577 cold / 512 output 690.9660 s 31.2186 tok/s 79.7131% 707.3345 s
564,577 identical replay / 512 output 9.2287 s 31.1965 tok/s 79.7131% 25.6088 s

The service exposed 2,557,299 FP8 E5M2 KV-cache tokens, reached 99.6087%
prefix-cache reuse on the identical replay, returned HTTP 200 for /health,
/v1/models, and /metrics, and remained healthy with zero restarts.

Risk and rollback

The change only enlarges the csrc builder's input set with an already tracked
source directory that setup.py conditionally consumes for SM70. Rollback is a
one-line revert.


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, including the related current and historical build work that was checked.
  • The test plan, including source-presence, Dockerfile, patch-hygiene, SM70 image-build, import, and V100 runtime checks.
  • The test results, including the completed CUDA 12.8.1 SM70 build, image size/digest, imports, service health, decode performance, MTP acceptance, KV-cache capacity, prefix-cache reuse, and restart status.
  • (Optional) The necessary documentation update was considered. This only restores a tracked build input already consumed by setup.py; it does not change a public API or supported-model declaration.

@github-actions

Copy link
Copy Markdown

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

Closing as superseded by merged #421. Current public main already contains the identical COPY flash_qla flash_qla/ build-stage input in docker/Dockerfile, introduced by commit 8ab1bf6. Re-merging this stale one-line branch would add no source change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants