Skip to content

env: add isolated CUDA 13 / vLLM 0.25.1 / LMCache 0.5.2 profile - #150

Closed
rickisba wants to merge 4 commits into
mainfrom
env/cu130-vllm0251-lmcache052
Closed

env: add isolated CUDA 13 / vLLM 0.25.1 / LMCache 0.5.2 profile#150
rickisba wants to merge 4 commits into
mainfrom
env/cu130-vllm0251-lmcache052

Conversation

@rickisba

Copy link
Copy Markdown
Collaborator

Summary

Add an isolated, additive development-image profile for the target serving baseline in #148:

  • CUDA 13.0.0 on Ubuntu 22.04;
  • Python 3.12 in /opt/venv;
  • PyTorch 2.11.0+cu130;
  • torchvision / torchaudio 0.26.0 / 2.11.0;
  • vLLM 0.25.1;
  • LMCache 0.5.2;
  • compatible CacheRoute application/development dependency ranges.

Changes

  • Add env/docker/cu130/Dockerfile.
  • Add env/docker/cu130/constraints.txt for the exact serving stack and shared compatibility ranges.
  • Add env/docker/cu130/requirements-dev.txt for the new profile without modifying the legacy root requirements.txt.
  • Add env/docker/cu130/README.md with build, container, editable-install, runtime, and validation instructions.

Compatibility and non-regression

This PR is intentionally additive:

  • it does not modify env/docker/Dockerfile;
  • it does not modify the legacy root requirements.txt;
  • it does not modify pyproject.toml;
  • it does not change Scheduler, KDN, Proxy, Instance, routing, queueing, or storage behavior;
  • the Docker build uses env/docker/cu130 as its context, avoiding large model/KDN/log directories without adding a repository-wide .dockerignore that could affect existing builds.

The new profile uses /opt/venv to isolate the serving stack from Ubuntu system Python packages. It installs FFmpeg for TorchCodec and excludes the Python-2-era Booktype==1.5, which can overwrite the modern redis module with invalid Python 2 source.

Validation performed

The image was built successfully and validated with:

  • Python 3.12.13;
  • PyTorch 2.11.0+cu130 and CUDA runtime 13.0;
  • vLLM 0.25.1;
  • LMCache 0.5.2;
  • Transformers 5.12.1;
  • Redis Python client 8.0.1;
  • successful python3 -m pip check;
  • successful imports of Torch, TorchCodec, vLLM, LMCache, and Redis;
  • successful startup of the existing CacheRoute Scheduler, KDN, Proxy, and Instance workflow.

Scope boundary

This PR provides the new installation and image-build interface, but does not claim that #148 is complete. The KDN raw Redis dump/inject contract still requires runtime migration and validation against LMCache MP RESP L2 behavior. The README records that limitation explicitly.

Partially addresses #148.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66a9a26e27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +67 to +71
/usr/bin/python3 \
python3 \
"/usr/bin/python${PYTHON_VERSION}" \
1 && \
python3 -m venv "${VIRTUAL_ENV}"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep Ubuntu's system Python on 3.10

When this development image later runs Ubuntu Python-based administration tools such as add-apt-repository, replacing /usr/bin/python3 with Python 3.12 makes those tools load the wrong interpreter; Ubuntu 22.04 installs extensions such as apt_pkg for its system Python 3.10, so the commands can fail with import errors. The checked update-alternatives --help confirms that --install <link> <name> <path> turns the supplied link into an alternatives-managed symlink. Create the virtual environment with /usr/bin/python${PYTHON_VERSION} -m venv directly and leave /usr/bin/python3 unchanged; the existing PATH already selects /opt/venv/bin/python3 for application commands.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

The isolated CUDA 13 / vLLM 0.25.1 / LMCache 0.5.2 environment profile from this PR has now been incorporated into draft PR #149 (v1/runtime-compat).

#149 contains the same additive environment boundary plus the runtime compatibility layer and KDN migration work. The imported profile is under env/docker/cu130/, and the dedicated image now defaults to CACHEROUTE_RUNTIME_PROFILE=v1.

No changes from this PR were applied to the legacy Dockerfile, root requirements.txt, or pyproject.toml. After verifying #149, this PR can be closed as superseded.

Copy link
Copy Markdown
Collaborator Author

This environment profile has been superseded. Its CUDA 13 / vLLM 0.25.1 / LMCache 0.5.2 files were consolidated and merged through PR #149, the homepage/quick-start entry was merged through PR #151, and the migration closeout plus reproducible validation pipeline is now tracked in PR #152. Closing #150 avoids accidentally merging the older standalone branch a second time.

@rickisba rickisba closed this Jul 28, 2026
@rickisba
rickisba deleted the env/cu130-vllm0251-lmcache052 branch July 28, 2026 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant