Profile and optimise the decoding hot paths

[meandmytram]

What this is

A profiling-driven optimisation pass over the MPS/decoding hot paths, with a
benchmark suite (benchmarks/bench_suite.py) whose deterministic workloads
double as a correctness contract: exact fingerprints (energies, verdicts,
constrained-state overlaps and spectra) that must not move, and
chi-truncated posteriors that may move within truncation noise only for a
change that alters the gauge trajectory. Baselines are written from an
unoptimised main checkout; every change below matches them.

Changes kept (in measured-impact order)

1. _to_numpy no longer imports cupy per call (the failing module search
   ran 66k times per decode, ~18% of runtime).
2. The zip-up's two fixed einsums are issued as direct tensordots and the
   remaining opt_einsum expressions are cached per (subscripts, path).
3. Orthogonality-centre moves factor the centre alone whenever the right
   neighbour is an isometry (checked per site) and nothing would be
   truncated: the centre's spectrum is the two-site tensor's, so the cut and
   the chi_max count are identical. Covers plain and renormalised moves and
   returned spectra; a non-isometric neighbour falls back to the two-site
   SVD.
4. One private copy of the chain per constraint sweep (inplace moves
   and zip-ups), the cheaper isometry gate, the reverse() centre-at-0 fix,
   the product-state centre from site norms.
5. Benchmark suite with seven workloads (surface, RCM surface, Shor,
   classical LDPC, DMRG, circuit-level DEM d=3 and d=5) and
   --qubit_order_strategy for the four CSS campaign CLIs.

Not delivered: the QR/LQ pre-reduction of strongly rectangular SVDs

It is in the code behind SVD_QR_PREREDUCTION (opt-in through
MDOPT_SVD_QR_PREREDUCTION=1, backend-aware helpers for the QR, the small
SVD and the back-multiplication) but off by default, so the numbers
below do not include it. With it, the [[72,12,6]] bivariate-bicycle decode
of a single-Z error at chi_max=400 in the natural order failed all 24 shots
it was given (flat or wrongly peaked posterior) while main, the
thesis-era code and this branch without it decode every one of them, and
every unit test and benchmark fingerprint passed throughout. The cause is
on the NumPy/Accelerate side (macOS arm64 wheels, numpy 2.2.6): NumPy's
linalg.qr dies with SIGBUS on a 636x304 centre tensor from that decode
when called in a loop with allocations in between, and a 12-second replay
of one contraction gives a heap-layout-dependent result with any variant of
the pre-reduction (NumPy or SciPy QR, SVD and matmul) while the plain
decomposition is deterministic and exact. Details in
this comment.
Enable the flag only where tests/decoding/test_convergence.py passes on
the target build.

Measured (idle Apple M5, single run of the suite, this branch vs main)

workload	main	this branch	speed-up
dem_d5 (circuit-level d=5, chi=32)	28.3 s	19.0 s	1.49x
surface_bitflip (5x5, chi=64, natural order)	23.6 s	18.3 s	1.29x
dem_d3 (circuit-level d=3, chi=32)	7.9 s	4.8 s	1.63x
classical_ldpc (3 codes, chi=64, DMRG readout)	3.8 s	2.8 s	1.38x
css_optimised (RCM order)	1.45 s	1.02 s	1.42x
shor_depolarising	0.40 s	0.12 s	3.3x
dmrg_ground_state	0.21 s	0.17 s	1.2x

The earlier campaign reports (1,
2) quote
numbers measured with the pre-reduction on (dem_d5 17.8 s, surface 16.4 s);
those are superseded by the table above.

Tests

• tests/decoding/test_convergence.py (opt-in, MDOPT_RUN_SLOW=1, 30-60
  min): the offending BB instance at chi_max 128 and 400 must decode to a
  delta posterior on the identity; fails deterministically with the
  pre-reduction on this machine.
• tests/decoding/test_surface_ler_regression.py (35 s, in CI): forty
  fixed-seed shots on the L=5 surface code at chi_max=16; no failure at
  p=0.05, at most the pinned four at p=0.08, verdict pattern pinned.
• tests/utils/test_utils.py: svd under allocator churn on graded
  rank-deficient matrices, for both values of the flag (a guard, not a
  certain detector).
• Exact-enumeration references, the one-site-move-vs-two-site comparison,
  and the full suite (243 tests) pass.

Findings that matter beyond the code

• qubit_order_strategy="Optimised" (reverse Cuthill-McKee) is 5x faster than
  the natural order on the 5x5 surface code at chi=64 and changes the
  convergence of the BB codes; the BB campaign scripts still run the natural
  order at chi=400.
• Every experiment result computed on this branch with the pre-reduction on
  (the BB ordering and DEM mass-loss experiments) is being regenerated.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud

[codecov]

Codecov Report

❌ Patch coverage is 93.71069% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.94%. Comparing base (519b7a8) to head (e384d91).

Files with missing lines	Patch %	Lines
tests/decoding/test_convergence.py	35.29%	11 Missing ⚠️
mdopt/contractor/contractor.py	89.18%	4 Missing ⚠️
mdopt/utils/utils.py	89.18%	4 Missing ⚠️
mdopt/mps/utils.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #543      +/-   ##
==========================================
- Coverage   97.29%   96.94%   -0.36%     
==========================================
  Files          26       28       +2     
  Lines        5259     5537     +278     
==========================================
+ Hits         5117     5368     +251     
- Misses        142      169      +27     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟡 Changes recommended

The DMRG fingerprint cannot detect incorrect states, and the new SVD branches lack deterministic coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Optimizes decoding and MPS hot paths while adding profiling benchmarks.

Changes:

• Caches CuPy detection and accelerates rectangular SVDs using QR/LQ reduction.
• Extracts dense-posterior scoring for deterministic diagnostics testing.
• Adds four profiling workloads with fingerprints.

File summaries

File	Description
mdopt/utils/utils.py	Optimizes conversion and SVD paths.
mdopt/examples/decoding/decoding.py	Extracts posterior scoring logic.
tests/decoding/test_decoders.py	Makes warning tests deterministic.
benchmarks/bench_suite.py	Adds profiling workloads.
benchmarks/.gitignore	Ignores benchmark results.

Review details

• Files reviewed: 5/5 changed files
• Comments generated: 2
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The finiteness check disables GPU SVD for CuPy arrays, and several correctness checks are too weak to enforce the stated tolerance.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

tests/utils/test_utils.py:673

• The truncated-spectrum checks also inherit NumPy's default rtol=1e-5, allowing materially larger drift than the stated tolerances. Set rtol explicitly so this branch actually verifies the optimization's numerical contract.

            assert np.allclose(s_t, s_ref[:chi], atol=1e-12), label
            assert np.isclose(
                err, float(np.linalg.norm(s_ref[chi:]) ** 2), atol=1e-12
            ), label

benchmarks/bench_suite.py:76

• As in the surface workload, recording only the 0/1 decoding verdict cannot detect numerical drift in the dense posterior unless it crosses a decision boundary. Include the posterior itself so this workload can enforce the PR's claimed 1e-10 behavior-preservation contract.

        outputs.append(float(success))

• Files reviewed: 6/6 changed files
• Comments generated: 3
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Benchmark timing is order-dependent, direct SVDs incur unnecessary scans, and one numerical assertion remains too permissive.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/utils/test_utils.py:676

• This truncation-error check still inherits np.isclose's default rtol=1e-5. Because the discarded-spectrum norm can be much larger than one, the assertion can accept errors orders of magnitude above the stated 1e-12 tolerance; set rtol=0.0 here as in the other new equivalence checks.

• Files reviewed: 6/6 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The truncation-error test retains a loose default relative tolerance that weakens the numerical correctness contract.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/utils/test_utils.py:676

• This truncation-error comparison still uses NumPy's default rtol=1e-5. Since the expected squared residual can be order 100, the test can accept errors around 1e-3, so it does not enforce the stated numerical-equivalence contract. Set rtol=0.0 as in the other new SVD comparisons.

• Files reviewed: 6/6 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The truncation-error test retains a loose default relative tolerance and does not enforce the stated numerical bound.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The classical LDPC fingerprint does not validate the constrained MPS, allowing contraction regressions that preserve the decoded codeword to pass.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 16/16 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The bond-1 shortcut can assign an invalid orthogonality centre to partially biased product states.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

mdopt/mps/canonical.py:156

• The existing test_canonical_reverse always uses a centre at num_sites - 2, and the newly added move tests only reverse centres greater than zero, so the corrected site-0 branch is still untested. Add a case that reverses an MPS with orth_centre == 0 and asserts the result's centre is num_sites - 1.

• Files reviewed: 17/17 changed files
• Comments generated: 1
• Review effort level: Balanced

[meandmytram]

@Copilot investigate how to use QRs so that they do not break the decoding of BB codes

[meandmytram]

Correctness finding: the QR pre-reduction of svd is now off by default (0ba5f0b)

Symptom. On the [[72,12,6]] bivariate-bicycle code at chi_max=400 in the natural qubit order (depolarising bias 0.01), this branch failed every one of 24 non-trivial shots it was given (flat or wrongly peaked posterior), while main, the thesis-era code (40be370) and this branch with the pre-reduction disabled decode the same shots to a clean identity verdict. At chi_max 64/128/256 the branch was fine, and all 7 benchmark fingerprints matched main throughout.

What it is not. Not a tie-breaking or truncation-policy effect: replaying a checkpointed contraction, every SVD call of the pre-reduced path agrees with a reference decomposition to 1e-15; a degeneracy-aware cut and a norm-relative cut do not rescue it; main under a different LAPACK driver (gesvd) or 1e-14 input noise still decodes correctly.

What it is. A memory-safety problem in the pre-reduced path on NumPy wheels linked against Accelerate (macOS arm64, numpy 2.2.6):

• numpy.linalg.qr dies with SIGBUS (exit 138, 3 of 3 runs) on a 636x304 centre tensor from this decode when called 200 times with allocations in between, and in other heap states returns a factorisation whose product is not the input (whole columns off by 1.0; 9 of 30 calls in one process). SciPy's qr and every SVD driver reconstruct the same matrix to 1e-15 every time.
• The contraction that first diverges from main (string 55 of 96) is a 12-second reproducer: with any variant of the pre-reduction (NumPy or SciPy QR, NumPy or SciPy small SVD, NumPy contiguous matmul or SciPy gemm for the back-multiplication) its result depends on the heap layout of the process — four harness scripts, four different norms (0.00015, 0.068, 0.117, 0.248) against the correct 0.7033 — while the plain decomposition returns 0.703325986518838 in every script, identical to main.

Change. The pre-reduction sits behind mdopt.utils.utils.SVD_QR_PREREDUCTION (default False) with backend-aware helpers (_qr_reduced, _svd_small, _back_multiply: SciPy's LAPACK/BLAS on the NumPy backend, the device's own on CuPy), to be enabled only where the slow test below passes. Cost on a quiet M5, single run of the suite, this branch vs main: dem_d5 19.0 vs 28.3 s, surface_bitflip 18.3 vs 23.6 s, dem_d3 4.8 vs 7.9 s, classical_ldpc 2.8 vs 3.8 s (the campaign report's numbers with the pre-reduction were 17.8 / 16.4 / 4.6 / 2.7 s).

Tests. tests/decoding/test_convergence.py (skipped unless MDOPT_RUN_SLOW=1, ~30–60 min): the offending instance at chi_max 128 and 400 must decode to the identity with a delta posterior — it fails deterministically on the previous code. tests/utils/test_utils.py::test_svd_reconstructs_graded_rank_deficient_matrices_under_allocator_churn: a fast guard (the fault is heap-state dependent, so it is not a certain detector).

Consequences. Every result computed on this branch with the pre-reduction on this machine (the BB ordering experiment, the DEM mass-loss experiment, the partial bias-vs-chi sweep) is being regenerated. Related upstream reports: numpy#26791 (Accelerate matmul precision), scipy#21862 (Accelerate failures on macOS 15.1); the SIGBUS reproducer (qr_segv_min.py + the 636x304 matrix) is kept locally for an upstream issue.

[Copilot]

🟡 Changes recommended

The advertised rectangular-SVD optimization is disabled, and its reduced branches are not exercised by the new tests.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (2)

tests/utils/test_utils.py:688

• The global pre-reduction flag is False, so this test currently feeds NaNs directly to the backend SVD and never verifies the documented reduced-QR failure/fallback chain. Enable the flag locally before calling svd.

def test_svd_nonfinite_input_takes_the_fallback_chain():
    """A non-finite input must make the whole call raise.

    There is no finiteness pre-scan: the reduced path's QR of a NaN matrix
    yields a NaN factor whose SVD raises LinAlgError (LAPACK gesdd on NaN
    input), which sends the call through the fallback chain, and the
    jitter attempt cannot rescue a NaN either."""

tests/mps/test_canonical.py:960

• This test name still calls the optimized branch a QR path, but move_orth_centre now performs a direct one-site SVD and contains no QR factorization. Rename it to describe the one-site path so failures identify the implementation actually under test.

def test_move_orth_centre_qr_path_matches_svd_path_on_full_rank_states():

• Files reviewed: 19/19 changed files
• Comments generated: 3
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The reduced-SVD non-finite fallback is not exercised under the default test configuration.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

tests/mps/test_canonical.py:960

• The optimized branch now performs a one-site SVD, not QR, so this test name is stale and obscures which path it covers. Rename it to identify the one-site path versus the two-site reference.

def test_move_orth_centre_qr_path_matches_svd_path_on_full_rank_states():

• Files reviewed: 19/19 changed files
• Comments generated: 1
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

The broad numerical and backend-sensitive optimizations warrant final human validation despite extensive regression coverage.

Review details

Suppressed comments (1)

tests/mps/test_canonical.py:960

• This test name is stale: the optimized branch now factors the centre with svd, not QR. Rename it to identify the one-site path so future failures are attributed to the implementation actually under test.

def test_move_orth_centre_qr_path_matches_svd_path_on_full_rank_states():

• Files reviewed: 19/19 changed files
• Comments generated: 0 new
• Review effort level: Balanced