Correct figures in the comments and docs added by #543 - #581
Closed
meandmytram wants to merge 1 commit into
Closed
meandmytram wants to merge 1 commit into
meandmytram wants to merge 1 commit into
Conversation
An audit of the claims #543 added, against the result files and fresh measurements, found five wrong figures: - svd() comment and the convergence test's docstring: the chi_max=400 [[72,12,6]] failure was 21 of 22 shots with a non-trivial error, not "24 of 24" (that count mixed repeated runs of one shot with a second). - svd() comment: the pre-reduction saved at most 7% and nothing measurable on most workloads, rather than "0-7%". - _to_numpy docstring: the per-call cupy import cost about 11,000 failed imports per decode, between roughly a tenth and a third of a decode (surface_bitflip 4-13%, dem_d3 about a third, timed on pre-#543 main), not a flat "~18%". - _contract_cached docstring: a shape-keyed cache misses 1-2% of zip-up calls on the DEM workloads and 23% on the RCM surface workload, not "7-14% on large codes". - test_convergence docstring: the test takes about 20 minutes on an idle laptop, not 30-60 minutes per decode. The benchmark README's classical_ldpc row now says three LDPC codes. Claims checked and left as they are: Apple M5 10 cores, the ~1700-site DEM chain (1677 mechanisms), the DEM workload parameters, the opt_einsum overhead at small chi (42% of a zip step at chi=32), and agreement of pre- and post-#543 decoders at chi=1e5 (1.6e-14 on 22 posteriors). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #581 +/- ##
==========================================
+ Coverage 96.89% 96.91% +0.01%
==========================================
Files 28 28
Lines 5509 5509
==========================================
+ Hits 5338 5339 +1
+ Misses 171 170 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The documentation-only corrections are consistent with the benchmark implementation and supplied measurements.
Pull request overview
Corrects inaccurate benchmark and regression figures introduced by #543 without changing runtime behavior.
Changes:
- Corrects SVD failure rates and test duration.
- Clarifies measured optimization and cache impacts.
- Documents that the LDPC benchmark uses three codes.
File summaries
| File | Description |
|---|---|
tests/decoding/test_convergence.py |
Corrects failure count and runtime. |
mdopt/utils/utils.py |
Corrects performance and SVD measurements. |
mdopt/contractor/contractor.py |
Corrects cache-miss measurements. |
benchmarks/README.md |
Corrects the LDPC workload description. |
Review details
- Files reviewed: 4/4 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This was referenced Sep 14, 2026
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An audit of every factual claim #543 added (code comments, docstrings, the benchmark README), checked against the experiment result files and fresh measurements, found five wrong figures. This PR corrects them; no code behaviour changes.
svd()comment,test_convergence.pydocstringsvd()comment_to_numpydocstring_contract_cacheddocstringtest_convergence.pydocstringbenchmarks/README.mdChecked and correct as written: Apple M5 (10 cores), the ~1700-site DEM chain (1677 mechanisms), the DEM workload parameters in
bench_suite.py, the opt_einsum expression overhead on small tensors (42% of a zip step at chi=32, none at chi=64), the Frobenius-versus-maximum bound in the isometry gate comment, and the suite header's agreement of old and new decoders at chi=1e5 (pre- and post-#543 main agree to 1.6e-14 on 22 converged posteriors).#543's description is corrected in place; the errors in its older comments are listed in a correction comment on #543.
🤖 Generated with Claude Code
https://claude.ai/code/session_01S6mbxc9eJDQMVx7tjvYwud