Repository navigation
[CI] Reject new VLLM_* environment reads that bypass envs.py - #711
Merged
Merged
Conversation
envs.compile_factors() hashes every variable registered in vllm/envs.py into the torch.compile cache key. A switch read straight from os.environ is invisible to it, so flipping such a switch can load an AOT artifact compiled for the other setting. Add a pre-commit hook that flags os.getenv / os.environ.get / os.environ.setdefault / os.environ[...] reads of VLLM_* names under vllm/ that are not keys of envs.environment_variables. The 137 direct reads that exist today are listed as a baseline, so the tree stays green; the list is meant to shrink as switches get registered (a test fails once a baseline name is registered but still listed). Co-authored-by: Claude <noreply@anthropic.com> Signed-off-by: Peuqui <peuqui@github.com>
This was referenced Oct 1, 2026
Peuqui
added a commit
to Peuqui/1Cat-vLLM
that referenced
this pull request
Oct 2, 2026
Brings in 1CatAI#763 (Qwen3 XML tool-call stream boundaries) and 1CatAI#764 (SM70 compressed-tensors NVFP4 linear kernel selection). Auto-merged without conflicts; the files both sides touched (.pre-commit-config.yaml, config/kernel.py, config/vllm.py, kernels/linear/__init__.py) keep main's additions next to those of our open PRs (1CatAI#711, 1CatAI#742, 1CatAI#750, 1CatAI#646/1CatAI#717). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Peuqui <43776522+Peuqui@users.noreply.github.com>
Contributor
|
The regex scanner matches examples in comments/strings and misses imported aliases and keyword keys. Follow-up #775 uses AST parsing while preserving the legacy baseline and registration policy. The three new regressions fail with the original scanner; all eight scanner tests pass after the fix. No runtime variable or default is added. |
Peuqui
added a commit
to Peuqui/1Cat-vLLM
that referenced
this pull request
Oct 2, 2026
Brings in 1CatAI#773 (our 1CatAI#710, 1CatAI#743, 1CatAI#749 and 1CatAI#726 plus 1CatAI#705, with the PP token check extended to the whole payload), 1CatAI#711 and 1CatAI#766 (Flash-Next SM70 FP16 projections admitted by operator requirements instead of TP4). Conflicts: - gpu_model_runner.py, test_pp_token_check.py: main's version, which contains our 1CatAI#749 and checks the complete payload. - nvidia/ple_layer.py: our cascade structure (1CatAI#646/1CatAI#717) with main's e9cd8ed carried over: the hybrid-PLE override applies to the derived host budget before the host cap, and the cap counts min(TP, local world size) x local DP ranks. Git had auto-merged the override into _device_spill_bytes, where it assigned an unused name. Tests adapted to the fork: test_ple_auto_hybrid_budget.py drives _plan_placement (as in the rebased 1CatAI#646), one cascade test sets the vLLM config around materialize_tables(), and the PP2xTP4 QPN8 tests set use_qpn8 on the Fp8LinearMethod they build (1CatAI#750's 7a4c665). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Peuqui <43776522+Peuqui@users.noreply.github.com>
This was referenced Oct 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
envs.compile_factors() hashes every variable registered in vllm/envs.py into
the torch.compile cache key. A switch that is read straight from os.environ is
invisible to it: with the AOT compile cache on (the default since torch 2.10),
flipping such a switch can load an artifact compiled for the other setting.
We ran into this with our own route switches on a V100 + RTX 8000 rig: the
second boot after changing an unregistered switch loaded the previous graph
and failed; registering the switches in envs.py fixed it.
main currently reads 137 VLLM_* names directly that are not keys of
envs.environment_variables (831 registered). Many of them are dump, trace and
profiling switches that never change compiled code; those belong in
ignored_factors once registered, so they do not invalidate the cache. Others
select kernel routes (for example the FlashQLA prefill switches, the QSA
indexer switches or the SM70 indexer cuBLAS switches).
This PR does not register them. It adds a pre-commit hook so the list stops
growing:
os.environ.get, os.environ.setdefault and os.environ[...] reads of VLLM_*
names under vllm/ that are not keys of envs.environment_variables (read
with ast, so lambda, env_with_choices and named functions all count).
Writes (os.environ[...] = ...) are not reads and pass.
asks to register a new switch, and to add it to ignored_factors if it never
changes compiled code.
baseline only shrinks.
Why this is not a duplicate. No open PR or issue covers environment
registration or the compile cache key (gh pr list / gh search issues for
"envs.py", "compile_factors", "environment variable", "compile cache").
Related: #621 (compile cache opt-out) and #622/#672 (process-local addresses
in AOT artifacts) are about other ways an artifact can go stale.
AI assistance was used for this change. Every line was reviewed and the tests
below were run by the submitter.
Test Plan
Test Result
5 passed. The hook passes on the whole tree (exit 0), rejects a new
os.getenv("VLLM_BRAND_NEW_SWITCH") and accepts a registered read
(VLLM_PORT) and an os.environ write. pre-commit and mypy-3.10 clean.
🤖 Generated with Claude Code