Add Open-Meteo historical-archive integration for retrospective analysis

[ducroq]

Context

The Open-Meteo Historical Forecast / Archive API (`https://archive-api.open-meteo.com/v1/archive\` and `https://historical-forecast-api.open-meteo.com/v1/forecast\`) provides:

• Hourly historical weather from 1940 onwards (ECMWF ERA5 reanalysis)
• Historical forecast archive (what the forecast said at the time, with vintage) — useful for honest backtests
• Same field set as the live forecast endpoint
• Free, no API key

Today augur uses only the current-forecast Open-Meteo data shipped daily by energydatahub. Historical context is limited to whatever was collected and committed in HAN's data directory (started Sep 2025 roughly).

Why this matters

Three distinct potential uses, in increasing ambition:

1. Longer training-window experiments for LightGBM — currently rolling 56 days; with historical weather we could backtest against 5+ years of vintage-correct exogenous data
2. Honest backtest with point-in-time forecasts — the historical-forecast endpoint returns the forecast as it stood on a given past date, eliminating hindsight bias that plagues backtest comparisons. Directly addresses the "forecaster's dilemma" cautioned in M4 metric-redesign work
3. FyE B1 student exercises — retrospective buurt-level energy modeling against years of historical weather (e.g. winter 2018 cold snap demand impact at Elsweide)

Scope of this issue

Tracker only — concrete work scoped as follow-ons. Plausible first experiment:

• Build `scripts/fetch_openmeteo_archive.py` that takes (lat, lon, start, end) and produces a parquet matching the live forecast schema
• One-time historical pull for the 6 strategic locations × 5 years
• Append to `ml/data/training_history.parquet` as an extended training corpus
• Run a multi-window robustness experiment (28/56/112/365-day windows) on the extended corpus — does the M4-EXP-014 winning recipe hold with more training data?

Dependencies

• Soft prerequisite: weather-pipeline-unification work (in progress 2026-06-05) so the field schema is locked in before we go retrofit historical data to match it

Out of scope

• Building a real-time archive collector in energydatahub — historical data doesn't need a daily cron
• Replacing the live-forecast pipeline — augment, not replace
• FyE-specific historical exercises — that's FyE B1 student work, not augur infrastructure

🤖 Generated with Claude Code