A plugin-based ML framework for Polymarket prediction markets. Build, backtest and paper-trade models for any market family (weather, sports, politics, crypto) with a backtest that cannot look ahead and a gate that no failing model gets past.
You write a small plugin that says how to find a family's markets, build its features and resolve its outcomes. pmlab does the rest: edge and cost arithmetic, walk-forward backtesting, a hard GO / NO_GO gate before a model is promoted, paper trading, settlement, drift monitoring and a cron-friendly watchdog.
Not financial advice. pmlab is research software; trading prediction markets can lose all the money you put in. No
pmlabcommand places an order. Only aLiveBrokeryou build yourself, with your own Polymarket credentials, does, anddry_run=Trueis how you try it first.
These are enforced in code and checked by the test suite, not left to convention.
- No lookahead.
rolling_origin_evaltrains only on rows whose label was already known when the fold decided: a decision for tomorrow's temperature is not a label until tomorrow is over.build_panelrefuses a price observed at or after the decision it informs, andcheck_no_leakageraises when a feature gives the outcome away. How the checks work - A failed gate cannot ship.
ChampionManifest.publishraises unless the gate says GO, and that is not configurable. Paper trading reads its allowed segments fromchampion.json, the record made at promotion, and a trade is never settled before the plugin says the truth is final. - Reproducible. Every backtest records its seeds, library versions, resolved config and a
fingerprint of the data;
pmlab reproducere-runs it and verifies the metrics come back. Reproducibility - Stable. From 1.0 pmlab follows semantic versioning. The public API (the
names, the
pmlabcommand and its options, the file formats) is frozen and compared with a checked-in snapshot on every change; nothing public is removed before 2.0 without a deprecation warning first. What is public
pip install pmlab # Python 3.12+; or: uv add pmlab
pip install "pmlab[tune]" # + Optuna, for TunedForecaster
pip install "pmlab[track]" # + MLflow, for MLflowTrackerThis runs offline on a small synthetic sample bundled in the package: 750 weather-temperature markets, no network and no account. The numbers say nothing about real markets.
from pmlab import (ChampionManifest, HoldoutGateResult, LGBMForecaster, TradePolicy,
build_panel, rolling_origin_eval)
from pmlab.datasets import load_sample
sample = load_sample() # 750 synthetic weather markets, checked against a checksum
resolver = sample.plugin() # the real WeatherTmaxPlugin, wired to the bundled file
found = [(day, spec) for day in sample.days for spec in sample.plugin(day).discover_markets()]
# One row per (market, bin), each at its own price. build_panel refuses a price seen at or after
# the decision, and check_leakage=True raises if a feature gives the outcome away.
panel = build_panel(
[resolver.build_training_rows(s, "previous_evening", decision_time=f"{d}T12:00:00+00:00")
for d, s in found],
check_leakage=True,
)
# The forecast alone cannot tell a market's bins apart: add how far each bin sits from it.
bounds = {(s.market_id, b.label): [x for x in (b.lower, b.upper) if x is not None]
for _, s in found for b in s.outcome_bins}
centre = [sum(bounds[k]) / len(bounds[k]) for k in zip(panel["market_id"], panel["outcome_label"])]
panel["feature_bin_dist"] = centre - panel["feature_fcst_tmax"]
# Walk-forward backtest: every fit sees only rows from before the day it predicts.
model = LGBMForecaster(random_state=7, n_jobs=1)
result = rolling_origin_eval(
panel, model, stride=10, min_train_rows=2700,
policy=TradePolicy.live(min_edge=0.05), # what `pmlab record-trades` trades: best bin and side
)
gate = HoldoutGateResult.evaluate(result.trades)
print(len(result.trades), "trades | PnL", round(result.trades["realized_pnl"].sum(), 2), "|", gate.decision)
panel.to_parquet("panel.parquet") # the command line below reads this file
# A gate that fails cannot publish a champion.
failed = HoldoutGateResult.evaluate(result.trades, required_segments=["Atlantis"])
try:
ChampionManifest.publish(model, failed, "champion", plugin_family="weather_tmax")
except ValueError as err:
print("refused:", err)It prints the number of trades, the total PnL and the gate's decision, then the refusal. The same panel runs from the command line:
pmlab backtest --plugin weather_tmax --panel panel.parquet \
--stride 10 --min-train-rows 2700 --min-edge 0.05 --output metrics.json
pmlab reproduce metrics.json # re-runs the backtest and verifies the same metrics come backThe tutorial continues from here: tuning, a per-city gate, a published champion, paper trading, settlement and an HTML report, in one notebook that CI executes.
| Command | What it does |
|---|---|
pmlab backtest |
Walk-forward backtest of a panel, recorded so it can be reproduced |
pmlab reproduce |
Re-run a recorded backtest and verify the metrics |
pmlab promote-champion |
Publish a model as champion, only if its gate is GO |
pmlab scan-markets |
List a family's open markets from the Gamma API |
pmlab record-trades |
Paper-trade the scan's signals that clear --min-edge |
pmlab status · report · resettle |
Inspect paper trades, render an HTML report, repair old settlements |
pmlab monitor |
The watchdog: drift, live calibration and live PnL; silent when healthy |
pmlab settle-trades |
A placeholder that exits 1: it settles nothing and prints the SettlementEngine call that does |
pmlab version |
Print the version |
pmlab --help and pmlab <command> --help list every option. Command line reference
A plugin is one class with four methods. Everything else is shared.
from pmlab import MarketPlugin
class MyPlugin(MarketPlugin):
family = "my_markets"
def discover_markets(self, **kwargs): ...
def fetch_features(self, spec, horizon, **kwargs): ...
def fetch_truth(self, spec, **kwargs): ...
def build_training_row(self, spec, horizon, **kwargs): ...Everything else lives on the documentation site.
| Start here | Tutorial · Architecture · Command line · Plugin authoring |
| Build models | Data layer · Features · Modeling · Tuning · Costs · Position sizing |
| Run it | Live trading · Drift monitoring · Watchdog · Champion registry |
| Reference | API reference · Public API and stability · Deprecation policy |
| Project | Changelog · Roadmap · Contributing · Security |
git clone https://github.com/ArtBreguez/polymarket-lab && cd polymarket-lab
uv sync --extra dev
uv run pytest # the suite; the coverage gate is in pyproject.tomlCONTRIBUTING.md has the rest: the test-first rule, the end-to-end, performance and notebook suites, and how the public API is changed.
MIT. See LICENSE.