From 98268b872528ca9978c58d04a4dc74c244c4d5c1 Mon Sep 17 00:00:00 2001 From: TomHortons Date: Thu, 9 Jul 2026 09:28:30 +0000 Subject: [PATCH 1/5] Point the DOI badge and CITATION.cff at the Zenodo concept DOI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 10.5281/zenodo.21097248 is the v1.0.0 version DOI, so the README badge and citation metadata sent readers to the v1.0.0 record permanently. The concept DOI is 10.5281/zenodo.21097247 — verified to resolve to the latest archived release (currently v1.3.0, 10.5281/zenodo.21276606) — which is what both files always claimed to carry. Co-Authored-By: Claude Fable 5 --- CITATION.cff | 2 +- README.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CITATION.cff b/CITATION.cff index e37483a..ee909de 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -17,7 +17,7 @@ authors: orcid: "https://orcid.org/0009-0007-0920-536X" version: "1.3.0" date-released: "2026-07-09" -doi: "10.5281/zenodo.21097248" +doi: "10.5281/zenodo.21097247" repository-code: "https://github.com/tim-nish/QuantScenarioBench" license: MIT abstract: >- diff --git a/README.md b/README.md index def43ae..5e968b7 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # QuantScenarioBench [![CI](https://github.com/tim-nish/QuantScenarioBench/actions/workflows/ci.yml/badge.svg)](https://github.com/tim-nish/QuantScenarioBench/actions/workflows/ci.yml) -[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21097248.svg)](https://doi.org/10.5281/zenodo.21097248) +[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21097247.svg)](https://doi.org/10.5281/zenodo.21097247) A JAX-native Python framework for generating reproducible stochastic market scenarios and benchmarking portfolio strategies against them, with built-in export to Parquet and the Hugging Face Hub. @@ -542,7 +542,7 @@ If you use QuantScenarioBench in your research, please cite it. Citation metadata lives in [`CITATION.cff`](CITATION.cff) — use GitHub's **"Cite this repository"** button for ready-made BibTeX/APA, or cite the archived release via the Zenodo DOI badge at the top of this README -(the concept DOI `10.5281/zenodo.21097248` always resolves to the latest +(the concept DOI `10.5281/zenodo.21097247` always resolves to the latest release). --- From b6eb282324820f6e4dbbe0642b354548666628fe Mon Sep 17 00:00:00 2001 From: TomHortons Date: Thu, 9 Jul 2026 09:29:29 +0000 Subject: [PATCH 2/5] Reflect first PyPI publication in docs quantscenariobench 1.3.0 is live on PyPI (wheel + sdist, verified by a clean-venv install and smoke test). Adds the PyPI version badge to the README, removes the release checklist's "not on PyPI" warning (and notes the ==X.Y.Z pin option for the Space), and updates the v1.3.0 CHANGELOG note written before the upload happened. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 8 ++++---- README.md | 1 + docs/RELEASE_CHECKLIST.md | 7 ++----- 3 files changed, 7 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5450692..a777b71 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,10 +67,10 @@ in each `Scenario`'s metadata). remains banned from the installable package's runtime dependencies and every non-dev extra — a plain `pip install quantscenariobench` still never pulls it. -- This is intended to be the first release published to **PyPI**; until it - is uploaded, the README's `pip install quantscenariobench` instruction - refers to the upcoming publication (see `docs/RELEASE_CHECKLIST.md`, - "Package publish"). +- This is the first release published to **PyPI** + ([pypi.org/project/quantscenariobench](https://pypi.org/project/quantscenariobench/)) — + the README's `pip install quantscenariobench` instruction works as of + this release. ## [1.2.1] - 2026-07-04 diff --git a/README.md b/README.md index 5e968b7..a85dce1 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,7 @@ # QuantScenarioBench [![CI](https://github.com/tim-nish/QuantScenarioBench/actions/workflows/ci.yml/badge.svg)](https://github.com/tim-nish/QuantScenarioBench/actions/workflows/ci.yml) +[![PyPI](https://img.shields.io/pypi/v/quantscenariobench.svg)](https://pypi.org/project/quantscenariobench/) [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21097247.svg)](https://doi.org/10.5281/zenodo.21097247) A JAX-native Python framework for generating reproducible stochastic market scenarios and benchmarking portfolio strategies against them, with built-in export to Parquet and the Hugging Face Hub. diff --git a/docs/RELEASE_CHECKLIST.md b/docs/RELEASE_CHECKLIST.md index fcbf5e1..3478cc4 100644 --- a/docs/RELEASE_CHECKLIST.md +++ b/docs/RELEASE_CHECKLIST.md @@ -29,7 +29,8 @@ branch and merged via PR, then tagged (`v1.0.0`–`v1.2.1` all followed this). - [ ] Update the README: Roadmap table statuses, docs for new capabilities, and any version numbers quoted in examples. - [ ] Bump the pinned library ref in `spaces/leaderboard/requirements.txt` - (`quantscenariobench @ git+...@vX.Y.Z` — or the PyPI pin once published there). + (`quantscenariobench @ git+...@vX.Y.Z`, or a `quantscenariobench==X.Y.Z` + PyPI pin — the package is on PyPI since v1.3.0). - [ ] Verify `spaces/leaderboard/README.md` front-matter (`sdk_version`) still matches what the Space actually runs. - [ ] Update `CITATION.cff` (version + release date; DOI after step 3 if versioned DOIs are cited). @@ -44,10 +45,6 @@ branch and merged via PR, then tagged (`v1.0.0`–`v1.2.1` all followed this). ## 4. Package publish (PyPI) -> ⚠️ As of v1.2.1 the package is **not** on PyPI even though the README says -> `pip install quantscenariobench`. Until first PyPI publication, either publish -> or fix the README install instructions — don't release with a broken install path. - - [ ] Build from the tag: `python -m build` (hatchling backend); `twine check dist/*`. - [ ] Upload (`twine upload` — or GitHub Actions trusted publishing once configured). - [ ] Verify in a clean venv: `pip install quantscenariobench==X.Y.Z`, then import From dd2f39ed134b1203db2e54caa4578196ccd1c048 Mon Sep 17 00:00:00 2001 From: TomHortons Date: Thu, 9 Jul 2026 09:45:50 +0000 Subject: [PATCH 3/5] Ignore q_a/ session scratch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit q_a/ holds local Q&A working files (never team-shared; q_a/1 and q_a/2 have always been untracked). Ignoring it keeps session scratch out of git status. Intentionally leaves _bmad-output/, specs/, and benchmark_playground.py untracked-but-visible — those are handled together in the planned BMAD cleanup PR. Co-Authored-By: Claude Fable 5 --- .gitignore | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.gitignore b/.gitignore index 245a639..93b2302 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,6 @@ htmlcov/ # Virtual environments .venv/ venv/ + +# Session scratch — local Q&A working files, not team-shared +q_a/ From d4e025d3cbb7db57d0d607ac48420d98fb6d1b75 Mon Sep 17 00:00:00 2001 From: TomHortons Date: Thu, 9 Jul 2026 09:54:28 +0000 Subject: [PATCH 4/5] Pin the Leaderboard Space to the PyPI package, not the git tag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit quantscenariobench is on PyPI since v1.3.0 (verified installable), so the Space's requirements.txt now uses the exact PyPI pin `quantscenariobench==1.3.0` instead of the `git+...@v1.3.0` GitHub-tag install — a cleaner, faster Space build that consumes the same published artifact as everyone else. Updates the deployment guard test accordingly: its old premise ("not on PyPI, so the pin must be git+/@") no longer holds. It now enforces the real invariant — quantscenariobench must be pinned to a specific, installable version — accepting either an exact PyPI pin (preferred) or a git+tag reference, while still rejecting unpinned/range specs. Co-Authored-By: Claude Fable 5 --- spaces/leaderboard/requirements.txt | 2 +- tests/test_leaderboard_space_deployment.py | 33 +++++++++++++--------- 2 files changed, 21 insertions(+), 14 deletions(-) diff --git a/spaces/leaderboard/requirements.txt b/spaces/leaderboard/requirements.txt index 9671fff..a659a7a 100644 --- a/spaces/leaderboard/requirements.txt +++ b/spaces/leaderboard/requirements.txt @@ -1,3 +1,3 @@ gradio>=6.19 pandas>=2.0 -quantscenariobench @ git+https://github.com/tim-nish/QuantScenarioBench.git@v1.3.0 +quantscenariobench==1.3.0 diff --git a/tests/test_leaderboard_space_deployment.py b/tests/test_leaderboard_space_deployment.py index 7538b13..770967d 100644 --- a/tests/test_leaderboard_space_deployment.py +++ b/tests/test_leaderboard_space_deployment.py @@ -130,23 +130,30 @@ def test_readme_documents_no_hardware_upgrade_should_be_requested(): # --------------------------------------------------------------------------- -# Deploying QuantScenarioBench/qsb-leaderboard hit a real BUILD_ERROR: -# quantscenariobench is not published on PyPI ("ERROR: Could not find a -# version that satisfies the requirement quantscenariobench>=1.1.0 (from -# versions: none)"). requirements.txt must install it from a source pip -# can actually resolve (the project's own git repo, pinned to a tag), -# not a bare version specifier that silently assumes PyPI availability. +# The Space must pin quantscenariobench to a *specific, installable* version so +# its build is reproducible — never a floating spec that can drift under it. +# History: an early deploy hit a real BUILD_ERROR because the package was not +# yet on PyPI ("Could not find a version that satisfies the requirement +# quantscenariobench>=1.1.0 ... from versions: none"), so the pin had to be a +# git+tag reference. Since v1.3.0 the package IS on PyPI, so an exact PyPI pin +# (quantscenariobench==X.Y.Z) is the preferred installable form; a git+...@tag +# reference is still accepted. What stays banned is an unpinned or range spec. # --------------------------------------------------------------------------- -def test_requirements_does_not_assume_quantscenariobench_is_on_pypi(): +def test_requirements_pins_quantscenariobench_to_an_installable_version(): + import re + text = (_space_dir() / "requirements.txt").read_text() for line in text.splitlines(): - if line.strip().lower().startswith("quantscenariobench"): - assert "git+" in line or "@" in line, ( - "quantscenariobench is not published on PyPI — a bare " - f"version specifier ('{line.strip()}') fails to build on " - "the Hub. Pin it to an installable source, e.g. " - "'quantscenariobench @ git+https://github.com/.../QuantScenarioBench.git@vX.Y.Z'" + stripped = line.strip() + if stripped.lower().startswith("quantscenariobench"): + exact_pypi_pin = re.search(r"==\s*\d+\.\d+\.\d+", stripped) is not None + git_tag_pin = "git+" in stripped and "@" in stripped + assert exact_pypi_pin or git_tag_pin, ( + f"quantscenariobench must be pinned to an exact, installable " + f"version ('{stripped}' is not): either a PyPI pin " + "'quantscenariobench==X.Y.Z' (preferred, on PyPI since v1.3.0) " + "or a 'git+https://.../QuantScenarioBench.git@vX.Y.Z' reference." ) break else: From 0ada693fd22dccb0ca90509957020d3b522bc11d Mon Sep 17 00:00:00 2001 From: TomHortons Date: Thu, 9 Jul 2026 09:54:38 +0000 Subject: [PATCH 5/5] Add HF Space release automation to the roadmap (planned, deferred) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit v1.3.0 exposed that redeploying the Leaderboard Space is a manual, easy-to-forget step with no CI safety net. Documents the plan to automate it via GitHub Actions and hardens the manual step until then: - docs/todo-hf-space-release-automation.md — concrete issue draft: goal, the three blockers (undecided Space repo ID, required HF_TOKEN secret, PyPI-publish ordering), required secrets, workflow trigger, target files, an implementation sketch, acceptance criteria, and a rollback plan. - README roadmap: new "Planned" row linking the draft. - RELEASE_CHECKLIST step 5: the Space redeploy is now a prominent manual reminder ("do not forget — no CI safety net today"), notes the PyPI-must-be-live ordering, and links the automation draft; step 2 now prefers the PyPI pin. - spaces/leaderboard/README.md: its "deferred CI-triggered push" note now points at the tracked draft. Automation itself is intentionally not implemented: it is blocked on the Space repo ID and a maintainer-created HF_TOKEN, and pushes to a live production Space (not safely testable from a PR). Co-Authored-By: Claude Fable 5 --- README.md | 1 + docs/RELEASE_CHECKLIST.md | 16 +++- docs/todo-hf-space-release-automation.md | 113 +++++++++++++++++++++++ spaces/leaderboard/README.md | 2 +- 4 files changed, 126 insertions(+), 6 deletions(-) create mode 100644 docs/todo-hf-space-release-automation.md diff --git a/README.md b/README.md index a85dce1..9523084 100644 --- a/README.md +++ b/README.md @@ -531,6 +531,7 @@ The same suite runs in CI on every push and pull request (see the badge at the t | Correlated multi-asset scenario generation (`simulate_correlated_basket`) | Shipped — v1.3 | | Hierarchical Risk Parity baseline strategy | Shipped — v1.3 | | Scenario realism diagnostics — stylized-facts validation (`realism_report`) | Shipped — v1.3 | +| Hugging Face Space release automation — GitHub Actions repins the Space to the released PyPI version and rebuilds it on release | Planned — see [`docs/todo-hf-space-release-automation.md`](docs/todo-hf-space-release-automation.md) | | Advanced analytics, visualizations, historical/trend tracking, strategy-comparison tooling | Not planned | **v1.1 shipped the data; v1.2 ships the dashboard:** `aggregate_evaluation_results()` still returns a plain `list[dict]` you can put in a `pandas.DataFrame`, a notebook, or your own app — but as of v1.2 you can also browse the same ranked table live on the hosted [Leaderboard Space](#leaderboard-space), with sorting and filtering built in. That Space was the explicit v1.2 goal referenced in earlier releases' roadmaps; it is deliberately scoped to the ranked table alone — see the Leaderboard Space section's "Out of scope" note for what's intentionally not included. diff --git a/docs/RELEASE_CHECKLIST.md b/docs/RELEASE_CHECKLIST.md index 3478cc4..4122087 100644 --- a/docs/RELEASE_CHECKLIST.md +++ b/docs/RELEASE_CHECKLIST.md @@ -28,9 +28,9 @@ branch and merged via PR, then tagged (`v1.0.0`–`v1.2.1` all followed this). (Keep a Changelog format: Added / Changed / Fixed / Notes). - [ ] Update the README: Roadmap table statuses, docs for new capabilities, and any version numbers quoted in examples. -- [ ] Bump the pinned library ref in `spaces/leaderboard/requirements.txt` - (`quantscenariobench @ git+...@vX.Y.Z`, or a `quantscenariobench==X.Y.Z` - PyPI pin — the package is on PyPI since v1.3.0). +- [ ] Bump the pinned library ref in `spaces/leaderboard/requirements.txt` to + the **PyPI** pin `quantscenariobench==X.Y.Z` (the package is on PyPI since + v1.3.0; prefer this over the old `git+...@vX.Y.Z` GitHub-tag install). - [ ] Verify `spaces/leaderboard/README.md` front-matter (`sdk_version`) still matches what the Space actually runs. - [ ] Update `CITATION.cff` (version + release date; DOI after step 3 if versioned DOIs are cited). @@ -59,8 +59,14 @@ branch and merged via PR, then tagged (`v1.0.0`–`v1.2.1` all followed this). release notes so downstream benchmark users can pin them. - [ ] Confirm the Evaluation Results repo (`QSB_EVAL_RESULTS_REPO`) is reachable and its documented default matches reality. -- [ ] Redeploy the Leaderboard Space: push `spaces/leaderboard/` contents to the - Space's git repo; watch the build; confirm the table renders. +- [ ] **Redeploy the Leaderboard Space (manual until automated).** Confirm + `spaces/leaderboard/requirements.txt` pins `quantscenariobench==X.Y.Z` + (the PyPI version, done in step 2) — this must happen only *after* PyPI is + live (step 4), or the Space's `pip install` will fail — then push + `spaces/leaderboard/` contents to the Space's git repo, watch the build, + and confirm the table renders. This step is slated to be automated via + GitHub Actions; see `docs/todo-hf-space-release-automation.md`. **Do not + forget it** — it is the one release step with no CI safety net today. - [ ] End-to-end smoke test: run a baseline strategy on a published dataset with the released version, `publish_evaluation_results()`, and confirm the run appears on the live Space. diff --git a/docs/todo-hf-space-release-automation.md b/docs/todo-hf-space-release-automation.md new file mode 100644 index 0000000..0af1cda --- /dev/null +++ b/docs/todo-hf-space-release-automation.md @@ -0,0 +1,113 @@ +# TODO / Issue draft: Automate Hugging Face Leaderboard Space updates on release + +**Status:** Planned — deferred (blocked, see below). Draft for a future GitHub issue. + +## Goal + +When a QSB release is published, automatically update the hosted Leaderboard +Space to consume the newly released **PyPI** package and rebuild it — removing +the manual `spaces/leaderboard/requirements.txt` edit and manual push that is +currently step 5 of `docs/RELEASE_CHECKLIST.md` and easy to forget. + +On release the automation should: + +1. Rewrite the Space's `requirements.txt` to pin `quantscenariobench==X.Y.Z` + (PyPI), matching the just-released version. +2. Commit/push that to the Space's Hugging Face git repo, which triggers an + automatic rebuild. +3. Verify (or, at minimum, document how to verify) that the Space finishes + building and reaches the `RUNNING` stage. + +## Why this is deferred (blockers — resolve before implementing) + +1. **The target Space repo ID is not finalized.** `spaces/leaderboard/README.md` + and the PRD (Open Questions 18, 22) note the Hugging Face namespace is still + undecided. The workflow needs a concrete `spaces//` to push to. + **Decide this first.** +2. **Requires a Hugging Face write token as a GitHub secret** — only the + maintainer can create it (see Required secrets). +3. **Publish ordering.** The Space rebuild runs `pip install + quantscenariobench==X.Y.Z`, so it must run only after the PyPI upload is + live and installable. PyPI publish is currently a manual `twine upload` + (checklist step 4), so the workflow must either wait for PyPI or be chained + after an automated publish job. + +## Required secrets + +- **`HF_TOKEN`** — a Hugging Face **write** access token, ideally a + fine-grained token scoped to only the Leaderboard Space repo. Stored as a + GitHub Actions repository secret. Used to authenticate the commit/push (or + the `huggingface_hub` upload) to the Space. Must never be echoed in logs. + +## Workflow trigger + +Recommended: `on: release: types: [published]` — fires when the GitHub Release +is published (which the checklist does right after tagging). Preferred over +`push: tags` because a published Release is an explicit human action and won't +fire on arbitrary tags, and it naturally lands after the tag exists. + +Because PyPI publish is currently manual, the workflow **must first confirm the +target version is installable from PyPI** before touching the Space — e.g. poll +`https://pypi.org/pypi/quantscenariobench/json` for the release version with a +bounded retry, and fail fast if it never appears. If/when PyPI publishing is +automated via a trusted-publishing `release.yml`, chain this Space job after it +with `needs:`/`workflow_run` instead of polling. + +Also provide a `workflow_dispatch` entry point so the automation can be tested +against a scratch Space before it is trusted on real releases. + +## Target files + +- **New:** `.github/workflows/deploy-space.yml` (the automation). +- **Edited at runtime in the _Space's_ repo:** `requirements.txt` (the PyPI pin). +- **In this repo:** `spaces/leaderboard/requirements.txt` stays the source of + truth for the Space's contents and is already on the PyPI pin + (`quantscenariobench==X.Y.Z`). The workflow derives the version from the + release tag and pushes the updated file to the Space repo. + +## Implementation sketch (for when unblocked) + +Two viable mechanisms: + +1. **`huggingface_hub` (recommended):** `HfApi().upload_file(path_in_repo= + "requirements.txt", repo_id="/", repo_type="space", + token=HF_TOKEN, commit_message="Pin quantscenariobench==X.Y.Z")`. The commit + triggers an automatic rebuild. +2. **Git push:** clone `https://user:${HF_TOKEN}@huggingface.co/spaces//`, + rewrite `requirements.txt`, commit, push. + +Verify by polling `HfApi().get_space_runtime(repo_id).stage` (or `GET +https://huggingface.co/api/spaces//runtime`) until it reaches `RUNNING`, +failing on `BUILD_ERROR`/`RUNTIME_ERROR` or a timeout. Optionally also HTTP-GET +the Space URL and assert a 200. + +## Acceptance criteria + +- Publishing a GitHub Release `vX.Y.Z` runs the workflow automatically. +- It refuses to proceed (fails visibly) until `quantscenariobench==X.Y.Z` is + installable from PyPI. +- The Space's `requirements.txt` ends up pinned to `quantscenariobench==X.Y.Z` + (PyPI, not the GitHub tag). +- The Space rebuilds and reaches `RUNNING`; the workflow reports the final stage. +- On build failure or timeout, the workflow fails with a log pointing at the + Space's build logs. +- No secret is printed in logs. +- A `workflow_dispatch` dry-run path exists for testing against a scratch Space. + +## Rollback plan + +- The workflow only edits the Space's `requirements.txt`. To roll back, revert + that one commit in the Space repo (the Space keeps its own git history) and it + rebuilds against the previous pin. +- If the automation misbehaves, disable it by removing/renaming + `.github/workflows/deploy-space.yml` or disabling the workflow in the Actions + tab; the manual checklist step remains the unchanged fallback. +- The Space and the library are separate artifacts, so a bad Space deploy never + affects the published PyPI package, the GitHub Release, or the Zenodo archive. + +## Until this exists + +Follow `docs/RELEASE_CHECKLIST.md` step 5 manually: after PyPI is live, ensure +`spaces/leaderboard/requirements.txt` pins `quantscenariobench==X.Y.Z` and push +`spaces/leaderboard/` to the Space repo, then confirm the Space rebuilds and the +table renders. diff --git a/spaces/leaderboard/README.md b/spaces/leaderboard/README.md index c206c50..846731c 100644 --- a/spaces/leaderboard/README.md +++ b/spaces/leaderboard/README.md @@ -25,4 +25,4 @@ A hosted, browsable Leaderboard for [QuantScenarioBench](https://github.com/tim- ## Deployment (AD-30) -Standard Hugging Face Space git-push-to-deploy, mirroring how Benchmark Datasets (Feature 4.4) and the Evaluation Results repo (Feature 4.9) already publish to the Hub: push the contents of this `spaces/leaderboard/` directory to the Space's own git repo. `requirements.txt` is self-sufficient for a clean build — no additional manual configuration is required beyond this README's `sdk`/`app_file` declarations. A CI-triggered push on release/tag is a deferred refinement, not part of this story. +Standard Hugging Face Space git-push-to-deploy, mirroring how Benchmark Datasets (Feature 4.4) and the Evaluation Results repo (Feature 4.9) already publish to the Hub: push the contents of this `spaces/leaderboard/` directory to the Space's own git repo. `requirements.txt` is self-sufficient for a clean build — no additional manual configuration is required beyond this README's `sdk`/`app_file` declarations. A CI-triggered push on release/tag is a deferred refinement, not part of this story — it is now tracked as a planned task in [`docs/todo-hf-space-release-automation.md`](../../docs/todo-hf-space-release-automation.md).