Skip to content

virtual-environments.md

Latest commit

 

History

History
220 lines (166 loc) · 8.44 KB

virtual-environments.md

File metadata and controls

220 lines (166 loc) · 8.44 KB
Raw

Python Virtual Environments

Overview

A virtual environment is a self-contained directory holding a reference to a base interpreter, a private site-packages directory, and its own executable scripts. Packages installed into it are visible only to that environment, so projects can depend on different versions of the same library without conflict. The venv module ships with the standard library and creates one.

Isolation is not a stylistic preference. Without it, every project shares one package set, a single upgrade can break unrelated tooling, resolution reflects the history of the workstation rather than the project's declaration, and externally managed interpreters refuse installation.

A virtual environment is inexpensive and expendable: build output rather than source. Never commit it, never repair it by hand, and recreate it whenever the base interpreter changes or the installed state becomes uncertain.

Create an Environment

Create a virtual environment in the current project. The directory name .venv is a widely recognized convention understood by most editors and tools:

python3 -m venv .venv

The interpreter used to run this command becomes the base interpreter of the environment. To target a specific release, invoke it explicitly, for example python3.13 -m venv .venv.

Option Effect
--prompt <name> Sets the name shown in the shell prompt
--system-site-packages Grants access to the base interpreter packages
--clear Deletes existing contents before creating
--upgrade Upgrades the environment to the running interpreter
--upgrade-deps Upgrades pip in the new environment

Avoid --system-site-packages for application projects: it reintroduces the base interpreter's packages into the resolution path and defeats isolation. Since Python 3.12, a new environment contains pip but not setuptools, which projects requiring it must declare.

Activation and Deactivation

Activation prepends the environment's bin directory to PATH and adjusts the shell prompt. It is a convenience, not a requirement. Activate in zsh or bash, then leave the environment when work is complete; deactivate is a shell function defined by the activation script that restores the previous PATH:

source .venv/bin/activate
deactivate

Activation applies only to the current shell session and does not persist to new terminals, to subshells started before activation, or to scheduled processes. Do not place an activation command in .zshrc: it forces one project's environment onto every shell.

Invoking the environment's interpreter by path produces identical behavior and is preferable in scripts, editors, and continuous integration jobs, where shell state is not guaranteed:

.venv/bin/python -m pip install -r requirements.txt

Inspect the Active Environment

VIRTUAL_ENV

The activation script sets VIRTUAL_ENV to the absolute path of the environment root, and deactivate unsets it. Many tools read this variable to detect an active environment, so it is the most direct signal of activation state. An empty value means no environment is active, and setting the variable by hand activates nothing: it only misleads tools that trust it. Setting VIRTUAL_ENV_DISABLE_PROMPT before activation suppresses the prompt change.

Confirm the Environment

Verify the recorded root, the interpreter, the release, and the installer binding together. All must reference the same environment, and the two prefixes below differ only while an environment is active:

echo "$VIRTUAL_ENV"
command -v python
python --version
python -m pip --version
python -c "import sys; print(sys.prefix, sys.base_prefix, sep='\n')"

pyvenv.cfg

Each environment records its origin in a pyvenv.cfg file at its root:

home = /opt/homebrew/opt/python@3.13/bin
include-system-site-packages = false
version = 3.13.5

The home value identifies the base interpreter the environment depends on. When that path no longer exists, the environment is unusable and must be recreated, not edited.

Recreate an Environment After an Interpreter Upgrade

A virtual environment binds to its base interpreter by path. A minor release upgrade, a cleanup, or a change of version manager can remove that path and leave the environment broken. Recreation is the supported repair.

Confirm first that every dependency is declared in a requirements file or in pyproject.toml; where that record is incomplete, capture the current state with python -m pip freeze before changing anything. Then recreate:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt

Do not delete an environment until the required dependencies are declared and the target path has been verified. Prefer python3 -m venv --clear .venv when recreating in place, because it names the directory explicitly instead of relying on a separate removal command.

Exclude the Environment from Version Control

Add the environment directory to the project's .gitignore. Never commit a virtual environment: it holds absolute paths, platform-specific binaries, and files derivable from the declaration.

.venv/

Commit the dependency declaration and any lock artifact instead: those files, not the environment, are the reproducible record.

Comparison with Container-Based Isolation

Dimension Virtual environment Container
Isolation scope Python packages only Full userspace filesystem
System libraries Shared with the host Declared in the image
Build toolchain Must exist on the host Declared in the image
Startup cost Negligible Image build and runtime overhead
Production parity Partial High when the image is deployed

A virtual environment isolates Python packages but not compilers, system libraries, locales, or operating system releases. A container isolates the entire userspace and is deployed as the same artifact that was tested, which is the stronger reproducibility guarantee.

The two are complementary. Use a virtual environment for local development speed, and a container when parity with the deployment target, non-Python system dependencies, or reproducible builds are required.

Security and Reliability Practices

  • Isolate every project in a virtual environment or container.
  • Create the environment from a managed interpreter, never from a system one.
  • Keep the environment directory out of version control.
  • Reinstall from the declaration rather than repairing by hand, and recreate environments after any interpreter upgrade or cleanup. Compiled extension modules must be reinstalled after a minor release change.
  • Prefer explicit interpreter paths over activation in automated contexts.
  • Verify VIRTUAL_ENV and sys.executable before installing packages.
  • Do not grant access to system site packages without a documented reason.

Troubleshooting

Activation Appears to Succeed but the Wrong Python Runs

Inspect the resolution order and the recorded environment root. A shell that cached a path before activation may still resolve the previous interpreter, which the last command corrects:

type -a python
echo "$VIRTUAL_ENV"
hash -r

deactivate Is Not Found

The function exists only in a shell where the activation script was sourced. Running the script in a subshell leaves the parent shell unmodified, so source it again in the current shell.

The Environment Fails to Start After a System Change

Inspect the recorded base interpreter and confirm that it still exists. If the path is missing, recreate the environment; editing pyvenv.cfg is not a repair:

python -c "import sys; print(sys.base_prefix)"

Installations Land Outside the Environment

Confirm the installer binding with python -m pip --version. The reported path must be inside the environment directory; if it is not, the environment is not active and the installation targeted another interpreter.

Related Documentation