Skip to content

[CI] Reject new VLLM_* environment reads that bypass envs.py - #711

Merged
yangzhuxinyzx merged 1 commit into
1CatAI:mainfrom
Peuqui:check-env-registration
Oct 2, 2026
Merged

yangzhuxinyzx merged 1 commit into
1CatAI:mainfrom
Peuqui:check-env-registration

Conversation

@Peuqui

@Peuqui Peuqui commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Purpose

envs.compile_factors() hashes every variable registered in vllm/envs.py into
the torch.compile cache key. A switch that is read straight from os.environ is
invisible to it: with the AOT compile cache on (the default since torch 2.10),
flipping such a switch can load an artifact compiled for the other setting.
We ran into this with our own route switches on a V100 + RTX 8000 rig: the
second boot after changing an unregistered switch loaded the previous graph
and failed; registering the switches in envs.py fixed it.

main currently reads 137 VLLM_* names directly that are not keys of
envs.environment_variables (831 registered). Many of them are dump, trace and
profiling switches that never change compiled code; those belong in
ignored_factors once registered, so they do not invalidate the cache. Others
select kernel routes (for example the FlashQLA prefill switches, the QSA
indexer switches or the SM70 indexer cuBLAS switches).

This PR does not register them. It adds a pre-commit hook so the list stops
growing:

  • tools/pre_commit/check_env_registration.py flags os.getenv,
    os.environ.get, os.environ.setdefault and os.environ[...] reads of VLLM_*
    names under vllm/ that are not keys of envs.environment_variables (read
    with ast, so lambda, env_with_choices and named functions all count).
    Writes (os.environ[...] = ...) are not reads and pass.
  • The 137 existing direct reads are listed as a baseline. The error message
    asks to register a new switch, and to add it to ignored_factors if it never
    changes compiled code.
  • A test fails once a baseline name is registered but still listed, so the
    baseline only shrinks.

Why this is not a duplicate. No open PR or issue covers environment
registration or the compile cache key (gh pr list / gh search issues for
"envs.py", "compile_factors", "environment variable", "compile cache").
Related: #621 (compile cache opt-out) and #622/#672 (process-local addresses
in AOT artifacts) are about other ways an artifact can go stale.

AI assistance was used for this change. Every line was reviewed and the tests
below were run by the submitter.

Test Plan

pytest tests/tools/test_check_env_registration.py
pre-commit run check-env-registration --all-files

Test Result

5 passed. The hook passes on the whole tree (exit 0), rejects a new
os.getenv("VLLM_BRAND_NEW_SWITCH") and accepts a registered read
(VLLM_PORT) and an os.environ write. pre-commit and mypy-3.10 clean.

🤖 Generated with Claude Code

envs.compile_factors() hashes every variable registered in vllm/envs.py
into the torch.compile cache key. A switch read straight from os.environ
is invisible to it, so flipping such a switch can load an AOT artifact
compiled for the other setting.

Add a pre-commit hook that flags os.getenv / os.environ.get /
os.environ.setdefault / os.environ[...] reads of VLLM_* names under
vllm/ that are not keys of envs.environment_variables. The 137 direct
reads that exist today are listed as a baseline, so the tree stays
green; the list is meant to shrink as switches get registered (a test
fails once a baseline name is registered but still listed).

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Peuqui <peuqui@github.com>
Peuqui added a commit to Peuqui/1Cat-vLLM that referenced this pull request Oct 2, 2026
Brings in 1CatAI#763 (Qwen3 XML tool-call stream boundaries) and 1CatAI#764 (SM70
compressed-tensors NVFP4 linear kernel selection). Auto-merged without
conflicts; the files both sides touched (.pre-commit-config.yaml,
config/kernel.py, config/vllm.py, kernels/linear/__init__.py) keep main's
additions next to those of our open PRs (1CatAI#711, 1CatAI#742, 1CatAI#750, 1CatAI#646/1CatAI#717).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Peuqui <43776522+Peuqui@users.noreply.github.com>
@yangzhuxinyzx
yangzhuxinyzx merged commit e7d21fe into 1CatAI:main Oct 2, 2026
4 checks passed
@yangzhuxinyzx

Copy link
Copy Markdown
Contributor

The regex scanner matches examples in comments/strings and misses imported aliases and keyword keys. Follow-up #775 uses AST parsing while preserving the legacy baseline and registration policy. The three new regressions fail with the original scanner; all eight scanner tests pass after the fix. No runtime variable or default is added.

Peuqui added a commit to Peuqui/1Cat-vLLM that referenced this pull request Oct 2, 2026
Brings in 1CatAI#773 (our 1CatAI#710, 1CatAI#743, 1CatAI#749 and 1CatAI#726 plus 1CatAI#705, with the PP
token check extended to the whole payload), 1CatAI#711 and 1CatAI#766 (Flash-Next
SM70 FP16 projections admitted by operator requirements instead of TP4).

Conflicts:
- gpu_model_runner.py, test_pp_token_check.py: main's version, which
  contains our 1CatAI#749 and checks the complete payload.
- nvidia/ple_layer.py: our cascade structure (1CatAI#646/1CatAI#717) with main's
  e9cd8ed carried over: the hybrid-PLE override applies to the derived
  host budget before the host cap, and the cap counts
  min(TP, local world size) x local DP ranks. Git had auto-merged the
  override into _device_spill_bytes, where it assigned an unused name.

Tests adapted to the fork: test_ple_auto_hybrid_budget.py drives
_plan_placement (as in the rebased 1CatAI#646), one cascade test sets the vLLM
config around materialize_tables(), and the PP2xTP4 QPN8 tests set
use_qpn8 on the Fp8LinearMethod they build (1CatAI#750's 7a4c665).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Peuqui <43776522+Peuqui@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants