Skip to content

SQLite snapshot storage, docs site, and release workflow - #6

Merged
atharvas merged 9 commits into
mainfrom
sqlite-storage-and-docs
May 15, 2026
Merged

SQLite snapshot storage, docs site, and release workflow#6
atharvas merged 9 commits into
mainfrom
sqlite-storage-and-docs

Conversation

@atharvas

@atharvas atharvas commented May 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Storage backend rewrite: snapshots now live in a single SQLite database (<snapshot_dir>/snapshots.db) instead of thousands of per-test .pkl/.pkl.gz files. Payloads are pickled, gzipped, and content-addressed by sha256 in a refcounted blobs table, so benchmarks producing identical outputs share one blob on disk. Per-test metadata lives in a snapshots table.
  • JSON sidecars unchanged: the per-test <module>/<class.benchmark>/<param-hash>.json metadata sidecars and the flat baseline.json are still written exactly where they were, since downstream tooling consumes them. store_snapshot now returns the sidecar path.
  • Clean break: there is no .pkl/.pkl.gz read path. Existing pickle snapshot directories are not migrated — callers re-capture. Version bumped 0.1.0 -> 0.2.0 to signal the format break.
  • Docs: new MkDocs Material site under docs/ (+ mkdocs.yml) and a CLAUDE.md describing the codebase.
  • Release infra: .github/workflows/release.yml publishes to PyPI via OIDC trusted publishing on GitHub Release (no secrets).

Storage shape (shapely roundtrip, local smoke)

58 snapshots -> 45 unique blobs   (~22% deduplicated)
raw payload  111 MB  ->  gzipped in SQLite  84 MB

Test plan

  • pytest unit suite (excluding tests/test_repos/, tests/test_cli_roundtrip.py, and the two pre-existing-hang discovery tests): 153 passed, 8 skipped
  • tests/test_storage_comprehensive.py: 32 passed (incl. new dedup + DB-compression tests)
  • tests/test_integration.py: 22 passed (fixed store_failed_capture to coerce non-string failure_reason)
  • Local end-to-end shapely roundtrip: list -> capture -> baseline -> verify → 57 pass / 1 skip, correct transition matrix
  • CI: tests.yml (py3.8–3.13) + test-astropy/pandas/shapely roundtrips

Notes

  • The Lint and Format job is pre-existing red on main (failing on every recent commit, mostly ruff rules vs. the py3.8 Optional[...] convention and vendored tests/test_repos/ sources). Not introduced here; my changed files are ruff format-clean. The meaningful gates are the test workflows.
  • tests/test_class_capture.py / test_class_instance.py hang on main independently of this change — they BenchmarkDiscovery into tests/test_repos/ (~1479 benchmarks). Out of scope here.

Update: CI repair folded in (per maintainer request)

CI was 100% pre-existing red on main (Lint failed in ~9s; Tests ran 2–6h then failed via an unbounded discovery hang). This PR now also makes CI genuinely green:

  • ruff config (pyproject.toml): exclude vendored tests/test_repos/ (third-party sources), ignore the pyupgrade rules that conflict with the deliberate py3.8 Optional[...] convention, and per-file-ignores for the intentional __init__.py import-after-logging pattern and test-scaffolding rules.
  • real lint fixes: removed dead locals + tightened a strict type check in comparator.py, set-comprehension in transitions.py, repo-wide ruff format.
  • discovery hang: test_class_capture.py::test_class_instance_capture walked all of tests/ (incl. ~1500 vendored benchmarks) and ran each with no timeout. Scoped it to its test_class_instance fixture module + added a 30s runner timeout.

Local verification of the exact CI commands:

  • ruff format --check src/ tests/ + ruff check src/ tests/All checks passed
  • pytest -v --ignore=tests/test_repos/ --ignore=tests/test_cli_roundtrip.py155 passed, 8 skipped in ~2.6s (was failing after hours)

@atharvas
atharvas merged commit af973d2 into main May 15, 2026
7 checks passed
@atharvas
atharvas deleted the sqlite-storage-and-docs branch May 15, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant