SQLite snapshot storage, docs site, and release workflow - #6
Merged
Conversation
…ning vendored benchmarks
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
<snapshot_dir>/snapshots.db) instead of thousands of per-test.pkl/.pkl.gzfiles. Payloads are pickled, gzipped, and content-addressed by sha256 in a refcountedblobstable, so benchmarks producing identical outputs share one blob on disk. Per-test metadata lives in asnapshotstable.<module>/<class.benchmark>/<param-hash>.jsonmetadata sidecars and the flatbaseline.jsonare still written exactly where they were, since downstream tooling consumes them.store_snapshotnow returns the sidecar path..pkl/.pkl.gzread path. Existing pickle snapshot directories are not migrated — callers re-capture. Version bumped0.1.0 -> 0.2.0to signal the format break.docs/(+mkdocs.yml) and aCLAUDE.mddescribing the codebase..github/workflows/release.ymlpublishes to PyPI via OIDC trusted publishing on GitHub Release (no secrets).Storage shape (shapely roundtrip, local smoke)
Test plan
pytestunit suite (excludingtests/test_repos/,tests/test_cli_roundtrip.py, and the two pre-existing-hang discovery tests): 153 passed, 8 skippedtests/test_storage_comprehensive.py: 32 passed (incl. new dedup + DB-compression tests)tests/test_integration.py: 22 passed (fixedstore_failed_captureto coerce non-stringfailure_reason)list -> capture -> baseline -> verify→ 57 pass / 1 skip, correct transition matrixtests.yml(py3.8–3.13) +test-astropy/pandas/shapelyroundtripsNotes
main(failing on every recent commit, mostlyruffrules vs. the py3.8Optional[...]convention and vendoredtests/test_repos/sources). Not introduced here; my changed files areruff format-clean. The meaningful gates are the test workflows.tests/test_class_capture.py/test_class_instance.pyhang onmainindependently of this change — theyBenchmarkDiscoveryintotests/test_repos/(~1479 benchmarks). Out of scope here.Update: CI repair folded in (per maintainer request)
CI was 100% pre-existing red on
main(Lint failed in ~9s; Tests ran 2–6h then failed via an unbounded discovery hang). This PR now also makes CI genuinely green:pyproject.toml): exclude vendoredtests/test_repos/(third-party sources), ignore the pyupgrade rules that conflict with the deliberate py3.8Optional[...]convention, andper-file-ignoresfor the intentional__init__.pyimport-after-logging pattern and test-scaffolding rules.comparator.py, set-comprehension intransitions.py, repo-wideruff format.test_class_capture.py::test_class_instance_capturewalked all oftests/(incl. ~1500 vendored benchmarks) and ran each with no timeout. Scoped it to itstest_class_instancefixture module + added a 30s runner timeout.Local verification of the exact CI commands:
ruff format --check src/ tests/+ruff check src/ tests/→ All checks passedpytest -v --ignore=tests/test_repos/ --ignore=tests/test_cli_roundtrip.py→ 155 passed, 8 skipped in ~2.6s (was failing after hours)