Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# CI — full test suite on every push to main and every pull request.
#
# Design notes:
# - Matrix covers the supported floor (3.11, per requires-python) and the
# development version (3.12). 3.13 is deliberately omitted until the JAX/
# diffrax stack is verified against it — widen the matrix in its own PR.
# - JAX runs CPU-only on hosted runners; every test is written for that
# (the full suite takes ~8 minutes locally on CPU).
# - No HF_TOKEN or network secrets: Hub-publishing tests run against mocks.
name: CI

on:
push:
branches: [main]
pull_request:

concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12"]
steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: pip

- name: Install package with dev dependencies
run: pip install -e ".[dev]"

- name: Run test suite
run: pytest -q
34 changes: 34 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Citation metadata (Citation File Format) — renders GitHub's
# "Cite this repository" button.
#
# Maintenance notes:
# - `doi` is the Zenodo *concept* DOI (same as the README badge): it always
# resolves to the latest release, so it does not need updating per release.
# - `version` / `date-released` DO change per release — bumping them is a
# step in docs/RELEASE_CHECKLIST.md ("Release-prep PR").
cff-version: 1.2.0
message: "If you use QuantScenarioBench in your research, please cite it as below."
title: "QuantScenarioBench"
type: software
authors:
- family-names: "Imanishi"
given-names: "Tomoya"
alias: "tim-nish"
orcid: "https://orcid.org/0009-0007-0920-536X"
version: "1.2.1"
date-released: "2026-07-04"
doi: "10.5281/zenodo.21097248"
repository-code: "https://github.com/tim-nish/QuantScenarioBench"
license: MIT
abstract: >-
A JAX-native Python framework for generating reproducible stochastic
market scenarios and benchmarking portfolio strategies against them,
with built-in export to Parquet and the Hugging Face Hub.
keywords:
- quantitative finance
- benchmark
- stochastic volatility
- scenario generation
- portfolio optimization
- reproducibility
- JAX
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Tomoya Imanishi

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# QuantScenarioBench

[![CI](https://github.com/tim-nish/QuantScenarioBench/actions/workflows/ci.yml/badge.svg)](https://github.com/tim-nish/QuantScenarioBench/actions/workflows/ci.yml)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21097248.svg)](https://doi.org/10.5281/zenodo.21097248)

A JAX-native Python framework for generating reproducible stochastic market scenarios and benchmarking portfolio strategies against them, with built-in export to Parquet and the Hugging Face Hub.
Expand Down Expand Up @@ -509,6 +510,8 @@ pytest

The test suite covers closed-form price validation (Gil-Pélaez inversion for Heston; Black-Scholes formula for GBM), statistical properties (skew monotonicity in H for rBergomi), Parquet round-trips, and dataset card conformance — plus the Benchmark Core (metrics and baselines validated against hand-derived reference values, Portfolio Optimizer conformance suite), the Evaluation Results pipeline (`BenchmarkResult` → `EvaluationResult` transform, local storage, and Leaderboard aggregation), and the Leaderboard Space (`spaces/leaderboard/`: data loading/rendering, sorting, filtering, and deployment configuration).

The same suite runs in CI on every push and pull request (see the badge at the top of this README). Releases follow [`docs/RELEASE_CHECKLIST.md`](docs/RELEASE_CHECKLIST.md).

---

## Roadmap
Expand All @@ -527,6 +530,17 @@ The test suite covers closed-form price validation (Gil-Pélaez inversion for He

---

## Citing

If you use QuantScenarioBench in your research, please cite it. Citation
metadata lives in [`CITATION.cff`](CITATION.cff) — use GitHub's
**"Cite this repository"** button for ready-made BibTeX/APA, or cite the
archived release via the Zenodo DOI badge at the top of this README
(the concept DOI `10.5281/zenodo.21097248` always resolves to the latest
release).

---

## License

MIT
MIT — see [LICENSE](LICENSE) for the full text.
76 changes: 76 additions & 0 deletions docs/RELEASE_CHECKLIST.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Release Checklist

Covers a full QuantScenarioBench release: the Python library (GitHub tag → PyPI),
the Zenodo DOI, and the Hugging Face assets (benchmark datasets, Evaluation
Results repo, Leaderboard Space). The library follows SemVer; `dataset_version`
is versioned independently (see `CHANGELOG.md` header and `export/_publish.py`).

Release history convention: each release is prepared on a `docs/vX.Y.Z-release-notes`
branch and merged via PR, then tagged (`v1.0.0`–`v1.2.1` all followed this).

## 1. Pre-flight

- [ ] All feature branches for this release are merged to `main`; working tree clean.
- [ ] CI is green on `main` (`.github/workflows/ci.yml`).
- [ ] Fresh-environment test run passes: `pip install -e ".[dev]" && pytest`.
- [ ] Model and strategy conformance suites pass (`tests/test_conformance.py`,
`tests/test_benchmark_conformance.py`).
- [ ] Golden fixtures (`tests/fixtures/golden_benchmark_results.json`) unchanged —
or every intentional metric/behavior change is called out in the CHANGELOG.
- [ ] Decide the version bump (SemVer: breaking → major, feature → minor, fix → patch).
- [ ] Triage open bugs: anything release-blocking (e.g. serialization bugs that
corrupt published EvaluationResults) is fixed or explicitly deferred in the notes.

## 2. Release-prep PR (branch: `docs/vX.Y.Z-release-notes`)

- [ ] Bump `version` in `pyproject.toml`.
- [ ] Add the `## [X.Y.Z] - YYYY-MM-DD` section to `CHANGELOG.md`
(Keep a Changelog format: Added / Changed / Fixed / Notes).
- [ ] Update the README: Roadmap table statuses, docs for new capabilities,
and any version numbers quoted in examples.
- [ ] Bump the pinned library ref in `spaces/leaderboard/requirements.txt`
(`quantscenariobench @ git+...@vX.Y.Z` — or the PyPI pin once published there).
- [ ] Verify `spaces/leaderboard/README.md` front-matter (`sdk_version`) still
matches what the Space actually runs.
- [ ] Update `CITATION.cff` (version + release date; DOI after step 3 if versioned DOIs are cited).
- [ ] Open PR, review, merge to `main`.

## 3. Tag, GitHub Release, DOI

- [ ] `git tag vX.Y.Z && git push origin vX.Y.Z`.
- [ ] Create the GitHub Release with notes (copy from the CHANGELOG section).
- [ ] Verify Zenodo archived the release and minted the version DOI; confirm the
concept-DOI badge in the README still resolves.

## 4. Package publish (PyPI)

> ⚠️ As of v1.2.1 the package is **not** on PyPI even though the README says
> `pip install quantscenariobench`. Until first PyPI publication, either publish
> or fix the README install instructions — don't release with a broken install path.

- [ ] Build from the tag: `python -m build` (hatchling backend); `twine check dist/*`.
- [ ] Upload (`twine upload` — or GitHub Actions trusted publishing once configured).
- [ ] Verify in a clean venv: `pip install quantscenariobench==X.Y.Z`, then import
and run a minimal `simulate()` + `run_benchmark()` smoke test.

## 5. Hugging Face assets

- [ ] If generation code, model parameters, or the Parquet schema changed:
regenerate benchmark sample datasets, bump `dataset_version`, publish with
`publish_to_hub()`, and confirm the auto-generated dataset cards are correct.
- [ ] Record the Hub dataset **revision hashes** for the canonical datasets in the
release notes so downstream benchmark users can pin them.
- [ ] Confirm the Evaluation Results repo (`QSB_EVAL_RESULTS_REPO`) is reachable
and its documented default matches reality.
- [ ] Redeploy the Leaderboard Space: push `spaces/leaderboard/` contents to the
Space's git repo; watch the build; confirm the table renders.
- [ ] End-to-end smoke test: run a baseline strategy on a published dataset with
the released version, `publish_evaluation_results()`, and confirm the run
appears on the live Space.

## 6. Post-release

- [ ] Delete the merged `docs/vX.Y.Z-release-notes` branch (local + remote).
- [ ] Close the milestone / shipped issues.
- [ ] Announce (HF org page, relevant communities).
- [ ] File follow-up issues for anything deferred during the release.
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ build-backend = "hatchling.build"
name = "quantscenariobench"
version = "1.2.1"
description = "JAX-native framework for generating reproducible market scenarios, benchmarking portfolio strategies against them, and publishing results as versioned datasets with a hosted Leaderboard"
readme = "README.md"
license = {text = "MIT"}
requires-python = ">=3.11"
dependencies = [
"jax>=0.4.38",
Expand All @@ -22,6 +24,7 @@ dev = [
"pytest-cov>=5.0",
"pandas>=2.0",
"datasets>=2.0",
"gradio>=6.19",
]

[tool.hatch.build.targets.wheel]
Expand Down
23 changes: 21 additions & 2 deletions tests/test_leaderboard_space_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,8 +105,27 @@ def test_space_requirements_pin_gradio_and_quantscenariobench():


def test_gradio_absent_from_installable_package_dependencies():
pyproject_text = (_repo_root() / "pyproject.toml").read_text()
assert "gradio" not in pyproject_text.lower()
# AD-27's guard, narrowed (PR #100): gradio must never be a dependency
# of the installable package — neither runtime nor any extra except
# [dev]. The dev extra is the one sanctioned exception: this test suite
# itself imports spaces/leaderboard/app.py (which imports gradio), so
# the documented contributor flow — pip install -e ".[dev]" && pytest —
# must bring gradio with it (the repository's first CI run failed 23
# Space tests without it). The previous whole-file string scan could
# not distinguish the package's own dependency list from the dev
# tooling's, so it is replaced with a check of the actual lists.
import tomllib

pyproject = tomllib.loads((_repo_root() / "pyproject.toml").read_text())
runtime_deps = "\n".join(pyproject["project"]["dependencies"])
assert "gradio" not in runtime_deps.lower()
extras = pyproject["project"].get("optional-dependencies", {})
for extra_name, extra_deps in extras.items():
if extra_name == "dev":
continue
assert "gradio" not in "\n".join(extra_deps).lower(), (
f"gradio leaked into the {extra_name!r} extra"
)


# ---------------------------------------------------------------------------
Expand Down
Loading