Skip to content

dependency-management.md

Latest commit

 

History

History
220 lines (166 loc) · 7.79 KB

dependency-management.md

File metadata and controls

220 lines (166 loc) · 7.79 KB
Raw

Python Dependency Management

Overview

pip is Python's standard package installer. It resolves and installs packages from the Python Package Index or another configured source into one interpreter's site-packages directory. Most reported pip problems are the result of an unintended interpreter binding.

Dependency management covers more than installation: knowing what is installed and why, upgrading deliberately, recording the resolved set so another machine can reproduce it, verifying artifact integrity, and auditing for known vulnerabilities. Updates should be controlled and tested, because an upgrade applied in bulk removes the ability to attribute a regression to one package.

Bind Package Operations to an Interpreter

Prefer python -m pip over a standalone pip or pip3 command: the module form runs the installer belonging to the interpreter invoked. Confirm the binding, then upgrade pip itself:

python -c "import sys; print(sys.executable)"
python -m pip --version
python -m pip install --upgrade pip

Outside a virtual environment, managed installations may prevent global package modification. Do not bypass an externally managed environment with force flags unless the interpreter's ownership is understood.

Inspect Installed Packages

List every installed package, restrict the listing to packages that nothing else depends on (roughly the direct set), report which have newer releases, then inspect a single package:

python -m pip list
python -m pip list --not-required
python -m pip list --outdated
python -m pip show <package>
python -m pip show --files <package>
python -m pip check

pip show reports the version, location, requirements, and packages that require the named one; --files lists installed files. pip check reports unsatisfied or conflicting requirements.

Upgrade Packages

Update One Package

Upgrade one package, then run the project's tests and static analysis:

python -m pip install --upgrade <package>

By default, pip upgrades a dependency of the named package only when the installed version does not satisfy the new requirement. Adding --upgrade-strategy eager upgrades every dependency as well, widening the change set; --dry-run previews either resolution without modifying anything.

Avoid Blind Bulk Updates

The following pattern upgrades every outdated package:

python -m pip list --outdated --format=freeze \
  | cut -d= -f1 \
  | xargs -n1 python -m pip install --upgrade

Treat this command as a diagnostic or disposable-environment technique, not as a default maintenance workflow. It can introduce incompatible transitive versions, obscure which update caused a regression, and bypass declared constraints. Its legitimate use is answering a question, such as whether the project still passes against the newest releases, in a disposable environment.

For maintained projects:

  1. Update declared direct dependencies intentionally.
  2. Review release notes and compatibility requirements.
  3. Regenerate the lock or resolved dependency set.
  4. Run tests, linting, type checks, and application smoke tests.
  5. Commit the dependency declaration and lock changes together.

Requirements and Constraints Files

Install a requirements file into the active environment. An accompanying constraints file limits the version chosen for a package without installing it:

python -m pip install -r requirements.txt -c constraints.txt

Use exact pins when reproducibility is required:

fastapi==0.116.1
uvicorn==0.35.0

Do not use pip freeze from a shared or global environment to create project requirements: it may include unrelated packages.

Lock-Capable Workflows

A requirements file with pinned direct dependencies still allows transitive versions to float. A lock artifact records the resolved graph, so a later install reproduces the same versions everywhere.

Tool Declaration Lock artifact
pip-tools requirements.in Compiled requirements.txt
uv pyproject.toml uv.lock
Poetry pyproject.toml poetry.lock
PDM pyproject.toml pdm.lock
Pipenv Pipfile Pipfile.lock

Every such tool separates resolution from installation. With pip-tools, the first command below resolves and the second installs exactly that resolution; uv lock and uv sync are the equivalent pair:

pip-compile --generate-hashes requirements.in
pip-sync requirements.txt

PEP 751 defines pylock.toml as a standardized lock file format.

Hash-Checking Mode

Hash-checking mode requires every downloaded artifact to match a recorded digest, protecting against a compromised index or a replaced release file. Record hashes alongside the pinned versions:

<package>==<version> \
    --hash=sha256:<digest>

Compute a local distribution file's digest, then require hashes:

python -m pip hash <path-to-distribution-file>
python -m pip install --require-hashes -r requirements.txt

The mode imposes two constraints that are features rather than obstacles: every requirement must be pinned exactly, and every transitive dependency must appear explicitly. Generate the file with pip-compile --generate-hashes.

Audit Dependencies

pip-audit is maintained by the Python Packaging Authority. Install it, audit the environment, then audit a declared set without installing it:

python -m pip install pip-audit
pip-audit
pip-audit -r requirements.txt

An audit reports only published advisories: it is not proof of trustworthiness.

Security and Reliability Practices

  • Isolate every project in a virtual environment or container, and avoid project-specific packages in global or system-managed Python installs.
  • Bind package commands to an interpreter with python -m pip.
  • Declare direct dependencies and supported Python versions; remove unused dependencies.
  • Pin or lock dependencies according to reproducibility needs, and enable hash-checking mode for deployments and continuous integration.
  • Review dependency changes instead of applying bulk upgrades, and run automated tests and security checks after every update.
  • Install packages only from trusted indexes. Prefer a single --index-url over adding --extra-index-url, because resolution across several indexes allows an untrusted index to supply a package under an expected name.
  • Never place index credentials in commands, source code, or committed configuration; use a credential helper such as keyring.
  • Treat installation scripts and build backends as executable code, since installing a source distribution executes the project's build backend.

Troubleshooting

Confirm the Environment and Its Conflicts

The first two commands must reference the intended environment; the next two report conflicting requirements and identify which packages require a dependency. Unexpected index URLs usually come from configuration files, which the last command reports:

python -c "import sys; print(sys.executable)"
python -m pip --version
python -m pip check
python -m pip show <package>
python -m pip config list

Rebuild an Environment Cleanly

When installed state is uncertain, rebuild from the declaration rather than repair incrementally; deleting lock files or caches first discards reproducibility information before the cause is understood.

Related Documentation