Skip to content

Move from Poetry to uv and from black to ruff format - #583

Open
meandmytram wants to merge 5 commits into
mainfrom
uv-ruff-migration
Open

meandmytram wants to merge 5 commits into
mainfrom
uv-ruff-migration

Conversation

@meandmytram

@meandmytram meandmytram commented Sep 15, 2026

Copy link
Copy Markdown
Member

Part of #437. Ruff takes over formatting only. pylint still does the linting, so #437 stays open.

Summary

  • Poetry to uv. Dependency groups move to PEP 735 [dependency-groups], the package builds with uv_build, and uv.lock replaces poetry.lock.
  • black to ruff format. The pre-commit hook and the docs switch over. The reformat itself is a separate, mechanical commit.
  • For contributors, poetry install --with dev,test becomes uv sync --group dev --group test, and poetry run ... becomes uv run ....

Commits

  1. Move packaging from Poetry to uv and formatting from black to ruff. All tooling, lock, docs and script changes.
  2. Reformat Python sources with ruff format. 24 files, each exactly ruff 0.16.7's output of its main-branch version, with an identical syntax tree. Two more reformatted files (dem_rerun.py, compare_qubit_order_bb.py) are in commit 1 because they also carry text edits.
  3. Document the OpenBLAS wheels for uv environments. See the Apple silicon section.
  4. Allow the diskcache advisory in dependency review. See the Dependency review section.
  5. Request uv groups and tools explicitly in campaign instructions. quantum_surface.sh runs pyinstrument with uv run --group dev. The DEM campaign README and dem_rerun.py install the optional beliefmatching decoder with uv pip install --no-deps beliefmatching, because uv environments have no pip and its numpy<=2.2.6 pin keeps it out of the lock. In a scratch uv environment, beliefmatching 0.2.0 decoded a distance-3 surface-code memory experiment normally on NumPy 2.4.6. uv run and uv sync --inexact keep it, and a plain uv sync removes it.

Details

pyproject.toml

  • The dev, test and docs groups keep their members and bounds. Poetry carets are written out as ranges, and black is replaced by ruff>=0.16.7,<0.17.
  • [tool.uv] default-groups = [] keeps Poetry's optional-group behaviour: a plain uv sync installs only the runtime dependencies.
  • The build backend is uv_build>=0.12.13,<0.13, with module-root = "" for the flat layout.
  • [tool.ruff] keeps black's line length of 88 and targets py311. Notebooks and Markdown are excluded because black never formatted them; ruff would otherwise reflow 14 notebooks and the README's code blocks. force-exclude keeps that true when pre-commit passes file names explicitly.
  • [project] is unchanged.

Locks

  • On main, dependabot had bumped requirements.txt without touching poetry.lock, so the two disagreed. uv.lock pins each package at the newer of the two, so every version in it already appears in one of those files, apart from the new ruff and tomli entries.
  • 22 packages differ from poetry.lock (table below). black, pathspec and pytokens leave with black. cloudpickle, highspy, qdldl and sparsediffpy are dependencies of the newer joblib and cvxpy. tomli comes from coverage[toml] and only applies on Python 3.11.0.
  • requirements.txt, which Read the Docs installs, is now uv export output: runtime dependencies only, with hashes. Three pins rise to match the lock and pyproject.toml's own lower bounds: ldpc 2.3.6 to 2.4.1, qldpc 0.1.3 to 0.1.4, numpy 2.2.6 to 2.4.6. exceptiongroup and tomli (Python below 3.11 only) drop out. pygments comes in because ldpc depends on pytest. The uv-export hook keeps the file in step with the lock from now on.
Version differences between main's poetry.lock and uv.lock
package poetry.lock uv.lock
cffi 1.17.1 2.1.1
clarabel 0.11.0 0.11.1
cvxpy 1.6.6 1.9.2
fonttools 4.55.3 4.65.0
iniconfig 2.0.0 2.3.0
joblib 1.5.1 1.6.0
kiwisolver 1.4.8 1.5.1
llvmlite 0.48.0 0.49.0
markupsafe 3.0.2 3.0.3
more-itertools 10.5.0 10.8.0
networkx 3.4.2 3.6.1
numba 0.66.0 0.67.0
osqp 1.0.4 1.1.3
packaging 24.2 26.3
pillow 11.1.0 12.3.0
platformdirs 4.3.6 4.11.7
pluggy 1.5.0 1.6.0
pyparsing 3.2.1 3.3.2
scipy 1.15.3 1.17.1
scs 3.2.7.post2 3.3.1
sinter 1.15.0 1.16.0
tzdata 2025.2 2026.3

CI

  • tests, lint, mypy and notebooks use astral-sh/setup-uv@v10.1.0, then uv sync --locked --group <group> and uv run. setup-uv does not publish a floating v10 tag, so the pin is the full release tag. --locked fails the job when uv.lock is stale. The notebooks path filter now watches uv.lock instead of poetry.lock.
  • ci.yml and cd.yml are unchanged. pip and python -m build fetch uv_build through PEP 517 like any other backend, and both paths were checked locally.
  • dependabot moves from the pip ecosystem to uv and keeps the mpmath ignore.

pre-commit

  • ruff-format (ruff-pre-commit v0.16.7) replaces black.
  • uv-lock replaces poetry-check, and uv-export regenerates requirements.txt. Both succeed offline with an empty cache, which matters because pre-commit.ci runs hooks without network access.
  • check-added-large-files skips uv.lock, which is 558 KB against the 500 KB limit.

Devcontainer, docs and scripts

  • The devcontainer installs uv instead of Poetry, drops the Poetry feature, and runs uv sync --group dev --group test --group docs.
  • Updated text: the README and its docs/source/README.rst copy, getting_started.rst, testing.rst, contributing.rst, .github/CONTRIBUTING.md, the five campaign shell scripts, the DEM campaign README, a comment in dem_rerun.py, and the usage line of compare_qubit_order_bb.py, whose path was also out of date.

Apple silicon

uv, like pip, installs the Accelerate-linked macosx_14_0_arm64 NumPy and SciPy wheels. The README's pip route to the OpenBLAS wheels does not survive in a uv environment: the next uv run reinstalls the locked Accelerate wheels. uv sync --python-platform aarch64-apple-darwin targets macOS 13, so it picks the OpenBLAS wheels, and later uv run and uv sync calls keep them. Tested in a scratch copy with all three groups: nothing builds from source and the backend tests pass. The README note and the warning's docstring now describe this.

Build artifacts compared with Poetry's build of main

Poetry, main uv_build, this branch
wheel files 63 the same 63
sdist files 67 the same 67, plus pyproject.toml.orig
  • Wheel contents differ only in the reformatted sources, whose syntax trees are unchanged, and in the edited docstrings.
  • uv_build writes a TOML 1.0 rendering of pyproject.toml into every sdist and keeps the original as pyproject.toml.orig, so that older frontends can parse it. This is documented, stable uv behaviour.
  • METADATA is identical except that Author-email and Maintainer-email now include the name, Requires-Python gains a space after the comma, and the long description carries this PR's README edits.
  • twine check passes on the output of both uv build and python -m build.

Verification

Local runs on macOS arm64 with Python 3.11.16, using the migrated workflow commands:

check result
uv sync --locked --group test, then the tests.yml pytest command 245 passed, 2 skipped
uv sync --locked --group dev, then pylint mdopt --fail-under=9 9.28/10
mypy mdopt no issues in 17 source files
uv sync --locked with the docs group, and with all three groups resolve and import
pip install -e . in a plain venv, as ci.yml does works
uv build, python -m build, twine check pass
pre-commit run --all-files, and the hooks on each commit every hook passes

Dependency review

Dependency review flagged diskcache 5.6.3 (GHSA-w8v5-vhqr-4h9v, moderate: unsafe pickle deserialization for an attacker who can write to the cache directory). Main already locked the same version, and Dependabot alerts #70 and #71 flagged it there in February. The check caught it here only because GitHub reads uv.lock as a new manifest, so every locked package counts as added.

The package can't be dropped. No fixed release exists, since 5.6.3 from 2023 is the latest. Every qldpc release, including the latest 0.3.3, requires it, and from qldpc.codes import BBCode imports it through qldpc.cache. The fourth commit therefore allows exactly this advisory in dependency-review.yml, with a comment saying when to remove it. Any other vulnerable dependency still fails the check.

Not verified locally: the devcontainer image, because Docker isn't running here. The Read the Docs and notebooks builds run on this PR.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud

meandmytram and others added 3 commits September 14, 2026 23:01
pyproject.toml: the Poetry groups become PEP 735 dependency groups (dev,
test, docs) with the same bounds, black is swapped for ruff in dev, and
the package builds with uv_build. default-groups is empty so that, as
with Poetry's optional groups, a plain `uv sync` installs only the
runtime dependencies. [tool.ruff] keeps black's line length and, like
black, leaves notebooks and Markdown alone.

uv.lock replaces poetry.lock. Each package is pinned at the newer of its
poetry.lock and requirements.txt versions, which had drifted apart on
main. requirements.txt, still read by Read the Docs, is now exported
from the lock.

The tests, lint, mypy and notebooks workflows install uv with
astral-sh/setup-uv and run `uv sync --locked --group ...` and `uv run`.
pre-commit runs ruff-format, uv-lock and uv-export in place of black and
poetry-check, and the large-file check skips the generated uv.lock.
dependabot tracks the uv ecosystem. The devcontainer, docs, campaign
scripts and two comments now point at uv. dem_rerun.py and
compare_qubit_order_bb.py are reformatted here because they also carry
text edits; the remaining ruff reformat follows in its own commit.

Part of #437.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud
Mechanical: each of these 24 files is exactly ruff 0.16.7's output of
its previous version, under the [tool.ruff] settings from the previous
commit, and parses to the same syntax tree as before.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud
A NumPy or SciPy wheel swapped in by hand, as the README's pip commands
do, lasts only until the next `uv run`, which reinstalls the locked
Accelerate build. `uv sync --python-platform aarch64-apple-darwin`
targets macOS 13, selects the OpenBLAS wheels, and keeps them across
later `uv run` and `uv sync` calls. Checked in a scratch copy of the
project with all dependency groups: nothing builds from source and the
backend tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud
@codecov

codecov Bot commented Sep 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.86%. Comparing base (7bd950a) to head (91a0097).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #583   +/-   ##
=======================================
  Coverage   96.86%   96.86%           
=======================================
  Files          28       28           
  Lines        5574     5574           
=======================================
  Hits         5399     5399           
  Misses        175      175           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

diskcache 5.6.3 (GHSA-w8v5-vhqr-4h9v) has no fixed release, and qldpc,
which mdopt's BB-code helpers import, loads it at module level, so the
package cannot be dropped. Main already locked this version; the check
only flagged it here because uv.lock is a new manifest. Allow exactly
this advisory so that any other vulnerable dependency still fails.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The profiler script omits its required dev group, and the campaign guide relies on pip in an unseeded uv environment.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Migrates mdopt’s packaging and formatting workflows from Poetry/Black to uv/Ruff while preserving existing project behavior.

Changes:

  • Adds uv-based dependency locking, builds, CI, development environments, and documentation.
  • Replaces Black with Ruff format and reformats Python sources.
  • Documents Apple silicon wheel handling and the diskcache advisory exception.
File summaries
File Description
tests/optimiser/test_utils.py Ruff reformat.
tests/mps/test_utils.py Ruff reformat.
tests/mps/test_explicit.py Ruff reformat.
tests/mps/test_canonical.py Ruff reformat.
tests/decoding/test_surface_ler_regression.py Ruff reformat.
tests/decoding/test_dem.py Ruff reformat.
tests/decoding/test_decoders.py Ruff reformat.
requirements.txt Regenerates runtime pins with uv.
README.md Documents uv, Ruff, and Apple silicon setup.
pyproject.toml Configures dependency groups, uv builds, and Ruff.
mdopt/mps/canonical.py Ruff reformat.
mdopt/examples/decoding/quantum_surface.py Ruff reformat.
mdopt/examples/decoding/quantum_hypergraph_product.py Ruff reformat.
mdopt/examples/decoding/quantum_csp.py Ruff reformat.
mdopt/examples/decoding/quantum_bivariate_bicycle.py Ruff reformat.
mdopt/examples/decoding/plotting/gen_vert_horiz_surface.py Ruff reformat.
mdopt/examples/decoding/plotting/gen_vert_horiz_surface_highchi.py Ruff reformat.
mdopt/examples/decoding/plotting/gen_qubit_order_bb.py Ruff reformat.
mdopt/examples/decoding/plotting/compare_qubit_order_bb.py Updates usage and formatting.
mdopt/examples/decoding/plotting/augment_bp.py Ruff reformat.
mdopt/examples/decoding/plotting/aggregate_vert_horiz_surface.py Ruff reformat.
mdopt/examples/decoding/plotting/aggregate_vert_horiz_surface_variants.py Ruff reformat.
mdopt/examples/decoding/plotting/aggregate_vert_horiz_bb.py Ruff reformat.
mdopt/examples/decoding/plotting/aggregate_qubit_order_bb.py Ruff reformat.
mdopt/examples/decoding/data_handling.py Ruff reformat.
mdopt/examples/decoding/classical_ldpc.py Ruff reformat.
mdopt/examples/decoding/classical_ldpc_bp.py Ruff reformat.
mdopt/backend/array.py Documents uv’s OpenBLAS setup.
examples/decoding/quantum_surface.sh Migrates execution to uv.
examples/decoding/quantum_hypergraph_product.sh Migrates execution to uv.
examples/decoding/quantum_csp.sh Migrates execution to uv.
examples/decoding/quantum_csp_chi_scan.sh Migrates execution to uv.
examples/decoding/quantum_bivariate_bicycle.sh Migrates execution to uv.
examples/decoding/dem_campaign/README.md Updates campaign environment instructions.
examples/decoding/dem_campaign/nishimori.py Ruff reformat.
examples/decoding/dem_campaign/dem_rerun.py Updates dependency note and formatting.
docs/source/testing.rst Updates test commands for uv.
docs/source/README.rst Updates installation and formatting guidance.
docs/source/getting_started.rst Replaces Poetry setup with uv.
docs/source/contributing.rst Replaces Black instructions with Ruff.
.pre-commit-config.yaml Adds Ruff and uv hooks.
.github/workflows/tests.yml Migrates test CI to uv.
.github/workflows/notebooks.yml Migrates notebook CI to uv.
.github/workflows/mypy.yml Migrates type-checking CI to uv.
.github/workflows/lint.yml Migrates lint CI to uv.
.github/workflows/dependency-review.yml Allows the documented diskcache advisory.
.github/dependabot.yml Changes Dependabot to the uv ecosystem.
.github/CONTRIBUTING.md Updates formatter guidance.
.devcontainer/Dockerfile Installs uv instead of Poetry.
.devcontainer/devcontainer.json Syncs dependency groups with uv.
.devcontainer/devcontainer-lock.json Removes the Poetry feature lock.
uv.lock Adds the uv dependency lock.
poetry.lock Removes the Poetry lock.
Review details
  • Files reviewed: 36/53 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread examples/decoding/quantum_surface.sh Outdated
Comment thread examples/decoding/dem_campaign/README.md
quantum_surface.sh profiles with pyinstrument, which lives in the dev
group, and default-groups is empty, so the script now runs
`uv run --group dev`.

uv environments are not seeded with pip, so the DEM campaign README and
dem_rerun.py now install the optional beliefmatching decoder with
`uv pip install --no-deps beliefmatching`. Its numpy<=2.2.6 pin is what
keeps it out of the lock; in a scratch uv environment 0.2.0 decoded a
distance-3 surface-code memory experiment normally on numpy 2.4.6. `uv
run` and `uv sync --inexact` keep it, and a plain `uv sync` removes it,
which the README now says.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants