[WIP] release(4.0.0): OpenSTEF 4.0.0 tracking branch.#637
Open
egordm wants to merge 132 commits into
Open
Conversation
Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(STEF-2297): Initial commit. Cleared OpenSTEF 3.0 code. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Updated readme to exclude 3.0 specific parts. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Created directory structure for the 4.0. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Set python version to 3.12 Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(STEF-2297): Initial commit. Cleared OpenSTEF 3.0 code. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Updated readme to exclude 3.0 specific parts. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Created directory structure for the 4.0. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Configured Reuse tool. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Created reuse fix tool. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Added initial ruff config. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Configured ruff, poe, pytest and pyright. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Configured pyproject-fmt. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Added example scripts for ci testing. Added example integration test. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Added support for doctests. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(STEF-2297): Initial commit. Cleared OpenSTEF 3.0 code. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Updated readme to exclude 3.0 specific parts. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2297): Created directory structure for the 4.0. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Configured Reuse tool. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Created reuse fix tool. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Added initial ruff config. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Configured ruff, poe, pytest and pyright. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Configured pyproject-fmt. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Added example scripts for ci testing. Added example integration test. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2298): Added support for doctests. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added added github action for REUSE compliance check. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added added github quality checks. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Fixed reuse compliance. in gha. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Changed triggers for gha checks. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Changed triggers for gha checks. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Narrowed check gha permissions. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Updated github action. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added coverage checking. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added coverage checking. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added release / dev build job. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Updated checks workflow. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Updated checks workflow. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Updated checks workflow. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Improved version pinning. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added extra and default dependencies for openstef root package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Added extra and default dependencies for openstef root package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Enabled dev release publishing. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Disabled dev release publishing until mvp is ready. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2299): Style fixes. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(STEF-2300): Changed python dev version. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added docs setup. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Fixed styles. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added improved docs and api documentation. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added v4 docs publishing. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added new docs. Added contributing guide. Updated readme. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Added new docs. Added contributing guide. Updated readme. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Updated issue template links and slack channel mentions. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(STEF-2300): Updated docs gha trigger to remove test branch. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Collaborator
Author
Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
…added documentation on how to do logging. (#643) * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#629): Added logging configuration and documentation on how to use logging in this package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#629): Added license gitignore exclusion for vscode stuff. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-compatibility/README.md Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * change: Updated docs. Removed vscode example config. * change: Added all modules to api reference. Removed context lib import from init. * feature: Sign commit. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com>
…red code. (#645) * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Improved reference docs. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(#628): Migrated openstef beam backtesting. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Migrated openstef beam backtesting. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Migrated metrics to openstef beam. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Migrated metrics and metric provider. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Migrated evaluation code. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Migrated analysis module to openstef beam. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Migrated benchmarking module to openstef beam. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added module level docs. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added docs for backtesting, evaluation and core. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added docs for analysis module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Documented openstef beam benchmark. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added docs for benchmarking module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated docstrings to avoid references to model interface and be more general regarding forecast targets. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
…rsioned parts to avoid O(n^2) problem when doing cross dataset operations. (#657) * feature(#628): Started on versioned timeseries dataset. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Started on versioned timeseries dataset. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Started on versioned timeseries dataset. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated rest of the code to use new versioned dataset type. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated rest of the code to use new versioned dataset type. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Fixed doctests. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Improved docs. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated docsting in mixin for select function and inherited it from the dataset part. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated tests and removed unnecessary docstrings. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated tests and removed unnecessary docstrings. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* Update CITATION.cff (#642) * Update CITATION.cff order citation according to DOI on zenodo Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> * Update CITATION.cff Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> * fix: yaml formatting Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> --------- Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> Co-authored-by: Bart Pleiter <bart.pleiter@alliander.com> * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * add cylic features Signed-off-by: lschilders <lars.schilders@alliander.com> * update cylic features Signed-off-by: lschilders <lars.schilders@alliander.com> * add cylic features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add timeOfDay cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * configurable cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * configurable cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add datetime features and tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add feature clipper and tests Signed-off-by: lschilders <lars.schilders@alliander.com> * added scaling transform Signed-off-by: lschilders <lars.schilders@alliander.com> * added scaling transform tests Signed-off-by: lschilders <lars.schilders@alliander.com> * fixed all poe tasks Signed-off-by: lschilders <lars.schilders@alliander.com> * add daylight features Signed-off-by: lschilders <lars.schilders@alliander.com> * import or skip pvlib Signed-off-by: lschilders <lars.schilders@alliander.com> * remove icalendar dependency Signed-off-by: lschilders <lars.schilders@alliander.com> * fix test formatting * install all packages in workflow * fix test formatting * feature(#626): Missing value imputation (#649) * Initial implementation for missing values handling feature engineering Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add to pyproject Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Added logic to remove rows with trailing nan for a feature. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add tests. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Restore pyproject.toml and uv.lock from origin/feature/626-feature-engineering Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting, formatting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * More linting and formatting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix typing Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Review comments Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Pyright Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Changes based on review comments Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix tests and warnings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix warnings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix fixtures. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Clean up Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add warning assert to test Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix attribute type Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Reset uv.lock Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Config as part of init Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Remove type ignores Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Extend transform from baseconfig Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Validate for fill value when constant strategy. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Use default value for missing_value Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Move imputer to private attribute. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix warning Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * initial commit weather transforms Signed-off-by: lschilders <lars.schilders@alliander.com> * fix type errors * fix most type and formatting errors * fix all errors except for weather transforms * fix radiation_derived_features tests * vectorize feature clipper computation * rename scaler to scaler_transform * rename rolling aggregate features * formatting Signed-off-by: lschilders <lars.schilders@alliander.com> * feature(#628): Update feature naming. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated forecasting transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Refactored temporal transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Refactored temporal transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Fixed tests and doctests. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated docstring to include the right unit. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated doctests to include output examples. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Signed-off-by: lschilders <lars.schilders@alliander.com> Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> Co-authored-by: Frank Kreuwel <frank.kreuwel@alliander.com> Co-authored-by: Bart Pleiter <bart.pleiter@alliander.com> Co-authored-by: Egor Dmitriev <egor.dmitriev@alliander.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Marnix van Lieshout <47454346+MvLieshout@users.noreply.github.com>
* Update CITATION.cff (#642) * Update CITATION.cff order citation according to DOI on zenodo Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> * Update CITATION.cff Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> * fix: yaml formatting Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> --------- Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> Co-authored-by: Bart Pleiter <bart.pleiter@alliander.com> * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * add cylic features Signed-off-by: lschilders <lars.schilders@alliander.com> * update cylic features Signed-off-by: lschilders <lars.schilders@alliander.com> * add cylic features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add timeOfDay cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * configurable cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * configurable cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add datetime features and tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add feature clipper and tests Signed-off-by: lschilders <lars.schilders@alliander.com> * added scaling transform Signed-off-by: lschilders <lars.schilders@alliander.com> * added scaling transform tests Signed-off-by: lschilders <lars.schilders@alliander.com> * fixed all poe tasks Signed-off-by: lschilders <lars.schilders@alliander.com> * add daylight features Signed-off-by: lschilders <lars.schilders@alliander.com> * import or skip pvlib Signed-off-by: lschilders <lars.schilders@alliander.com> * remove icalendar dependency Signed-off-by: lschilders <lars.schilders@alliander.com> * fix test formatting * install all packages in workflow * fix test formatting * feature(#626): Missing value imputation (#649) * Initial implementation for missing values handling feature engineering Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add to pyproject Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Added logic to remove rows with trailing nan for a feature. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add tests. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Restore pyproject.toml and uv.lock from origin/feature/626-feature-engineering Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting, formatting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * More linting and formatting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix typing Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Review comments Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Pyright Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Changes based on review comments Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix tests and warnings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix warnings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix fixtures. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Clean up Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add warning assert to test Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix attribute type Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Reset uv.lock Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Config as part of init Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Remove type ignores Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Extend transform from baseconfig Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Validate for fill value when constant strategy. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Use default value for missing_value Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Move imputer to private attribute. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix warning Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * initial commit weather transforms Signed-off-by: lschilders <lars.schilders@alliander.com> * fix type errors * fix most type and formatting errors * fix all errors except for weather transforms * fix radiation_derived_features tests * Init setup Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * vectorize feature clipper computation * rename scaler to scaler_transform * rename rolling aggregate features * formatting Signed-off-by: lschilders <lars.schilders@alliander.com> * Added sufficient complete check. Added tests. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Remove comment Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * feature(#628): Update feature naming. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated forecasting transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Fix docs Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * More testcases Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Raise error and optionally run on transform Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add override Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Raise error or return original dataframe Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix based on review Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Minor fixes Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Remove state from completeness check Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Signed-off-by: lschilders <lars.schilders@alliander.com> Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> Co-authored-by: Frank Kreuwel <frank.kreuwel@alliander.com> Co-authored-by: Bart Pleiter <bart.pleiter@alliander.com> Co-authored-by: Egor Dmitriev <egor.dmitriev@alliander.com> Co-authored-by: lschilders <lars.schilders@alliander.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update CITATION.cff (#642) * Update CITATION.cff order citation according to DOI on zenodo Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> * Update CITATION.cff Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> * fix: yaml formatting Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> --------- Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> Co-authored-by: Bart Pleiter <bart.pleiter@alliander.com> * feature(#629): Added openstef compatibility package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added openstef core package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added interfaces and implementations for TimeSeries datasets and their accessors. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added tests for versioned timeseries. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added moved transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added documentation to datasets module. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-core/tests/unit/datasets/test_versioned_timeseries_accessors.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * add cylic features Signed-off-by: lschilders <lars.schilders@alliander.com> * update cylic features Signed-off-by: lschilders <lars.schilders@alliander.com> * add cylic features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add timeOfDay cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * configurable cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * configurable cyclic features Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add rolling aggregate features tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add datetime features and tests Signed-off-by: lschilders <lars.schilders@alliander.com> * add feature clipper and tests Signed-off-by: lschilders <lars.schilders@alliander.com> * added scaling transform Signed-off-by: lschilders <lars.schilders@alliander.com> * added scaling transform tests Signed-off-by: lschilders <lars.schilders@alliander.com> * fixed all poe tasks Signed-off-by: lschilders <lars.schilders@alliander.com> * add daylight features Signed-off-by: lschilders <lars.schilders@alliander.com> * import or skip pvlib Signed-off-by: lschilders <lars.schilders@alliander.com> * remove icalendar dependency Signed-off-by: lschilders <lars.schilders@alliander.com> * fix test formatting * install all packages in workflow * fix test formatting * feature(#626): Missing value imputation (#649) * Initial implementation for missing values handling feature engineering Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add to pyproject Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Added logic to remove rows with trailing nan for a feature. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add tests. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Restore pyproject.toml and uv.lock from origin/feature/626-feature-engineering Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting, formatting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * More linting and formatting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix typing Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Review comments Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Pyright Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Changes based on review comments Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix tests and warnings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix warnings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix fixtures. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Clean up Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add warning assert to test Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix attribute type Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Reset uv.lock Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Config as part of init Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Remove type ignores Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Extend transform from baseconfig Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix linting Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Validate for fill value when constant strategy. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Use default value for missing_value Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Move imputer to private attribute. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix warning Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * initial commit weather transforms Signed-off-by: lschilders <lars.schilders@alliander.com> * fix type errors * Add flatliner check transform Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * fix most type and formatting errors * fix all errors except for weather transforms * fix radiation_derived_features tests * Remove redundant comment Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix import statement Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * vectorize feature clipper computation * rename scaler to scaler_transform * rename rolling aggregate features * formatting Signed-off-by: lschilders <lars.schilders@alliander.com> * Add absolute tolerance arg. Update docstrings and tests. Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * feature(#628): Update feature naming. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Improve docstring Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * feature(#628): Updated forecasting transforms. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Added flatliner detection to transform Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add option to give custom load column name Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix order of exceptions Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add overrides Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Frank Kreuwel <frank.kreuwel@alliander.com> Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Signed-off-by: lschilders <lars.schilders@alliander.com> Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> Co-authored-by: Frank Kreuwel <frank.kreuwel@alliander.com> Co-authored-by: Bart Pleiter <bart.pleiter@alliander.com> Co-authored-by: Egor Dmitriev <egor.dmitriev@alliander.com> Co-authored-by: lschilders <lars.schilders@alliander.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Moved transforms to the openstef-models package. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Updated package deps. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(#628): Added feature transform pipeline. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Improved doc example. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-models/tests/unit/feature_engineering/validation_transforms/test_flatliner_check.py Co-authored-by: Marnix van Lieshout <47454346+MvLieshout@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * feature(#628): Fixed doctest for FeaturPipeline. Added repr to leadtime. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Improved test cases for feature pipeline. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Improved test cases for horizon splitting. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Co-authored-by: Marnix van Lieshout <47454346+MvLieshout@users.noreply.github.com>
…t part can have duplicate timestamps in index. (#665) * feature(#628): Added feature transform pipeline. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Improved doc example. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Added lag transform. Fixed bug where timeseries dataset part can have duplicate timestamps in index. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#628): Clarified test doc. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* Add file Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add calculations with references Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Fix quality checks Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Make stateless, fix doctest Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Rename to atmosphere derived features Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Rename test file Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add hardcoded values Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add more info and DOI Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Remove redundant comment Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Add more info to docstring Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
* Init commit Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Clean up Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Improve docstrings Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> * Improve naming Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
…by fixing github links to docs and disabling source button. (#667) * docs(#656): Updated code style guide regarding types. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * docs(#661): Fixed links to edit files on github. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(#627): Initial model interface implementation. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#627): Documented model interface mixins and updated documentation guidelines with new Invariant section. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#627): Added tests for combine_horizon_forecasts. Updated docstrings / and tests for mixins. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * fix(STEF-2391): Added tests for constant median forecaster and documentation to validated datasets. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#627): Updated code style guide. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * fix(STEF-2391): Added docs for forecaster implementations. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * fix(STEF-2391): Added docs for forecaster implementations. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * fix(STEF-2391): Fixed tests so fitting is more adaptable for multi horizon adapter. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#627): Minor fixes after feedback. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
…torage, callbacks. Refactor by reorganizing transforms. (#674) * refactor(#633): Moved feature engineering into transforms. Favoring a more flat structure for different operations on datasets. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added storage, callback and workflow interfaces. Added an example and implemented example storage. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added tests for new components. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added docstings to the new pipelines. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added docstings to the storage. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Renamed postprocessing pipeline to ForecastTransformPipeline. Refactored holiday features transform to avoid patching since it often breaks tests due to minor changes. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-models/src/openstef_models/models/forecasting_model.py Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * feature(#633): Updated dependency groups. Implemented is_fitted. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com>
* add rIQD and relative pinball loss metrics Signed-off-by: MentReeze <ment.reeze@alliander.com> * docstring fix Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com> Signed-off-by: MentReeze <ment.reeze@alliander.com> * add unit test for symmetric quantile logic of rIQD provider Signed-off-by: MentReeze <ment.reeze@alliander.com> --------- Signed-off-by: MentReeze <ment.reeze@alliander.com> Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com>
…n data (#676) * Add the option to plot timeseries without showing interpolated lines where gaps in data Signed-off-by: MentReeze <ment.reeze@alliander.com> * Fix ruff formatting issues Signed-off-by: MentReeze <ment.reeze@alliander.com> --------- Signed-off-by: MentReeze <ment.reeze@alliander.com>
…ded split component pipeline. (#677) * feature(#633): Moved model mixins into a single namespace inside models. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Updated transform interfaces. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Refactored feature pipeline and transform base. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added component splitting. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Refactored transforms, predictors, and statefulness. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Improved doctests and docstrings. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Improved docstrings. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added tests voor multi horizon transform adapter. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Added support for validation set during training. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#633): Changed dataset to data naming for consistency in forecasting model. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
* feature(#687): Implemented xgboost model. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#687): Added tests for loss objectives. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#687): Added docstrings for xgboost implementation. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#687): Improved tests. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * Update packages/openstef-models/src/openstef_models/models/forecasting/xgboost_forecaster.py Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * feature(#687): Documented model state. Moved input_data filtering into forecast input dataset. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com>
* feature: add sample_weight_column to ForecastInputDataset * add doctest example * add code and config for gblinear * feature(#687): Implemented xgboost model. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#687): Added tests for loss objectives. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#687): Added docstrings for xgboost implementation. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * feature(#687): Improved tests. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * refactor gblinear and tests * Update packages/openstef-models/src/openstef_models/models/forecasting/xgboost_forecaster.py Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> * feature(#687): Documented model state. Moved input_data filtering into forecast input dataset. Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> * align GBLinear implementation with XGBoost and add booster type checking * add GBLinear implementation * revert changes to types * revert changes to pyproject * revert changes to pyproject * add doctest SKIP --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Egor Dmitriev <egordmitriev2@gmail.com> Co-authored-by: Egor Dmitriev <egor.dmitriev@alliander.com>
## Changes - Add `predict_nonzero_flatliner` field to `EnsembleForecastingWorkflowConfig` (was missing, already present in single-model config) - Enable `error_on_flatliner=True` in both single-model and ensemble presets so `FlatlinerDetectedError` is actually raised and the flatliner fallback path is reachable ## Context Previously `error_on_flatliner=False` meant the `FlatlinerDetectedError` was never raised, making the flatliner fallback dead code. This activates it for all models. --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
## Summary `FlatlinerForecaster.feature_importances` built a DataFrame with a single scalar value but N quantile column labels, raising: ``` ValueError: Shape of passed values is (1, 1), indices imply (1, N) ``` whenever `len(quantiles) > 1`. This crashes the MLflow callback's feature-importance plotting during flatliner-fallback training, which means the flatliner model is **never persisted**. Downstream the prediction service then falls back to serving the previous (non-flatliner) model, so the flatliner fallback is silently broken for any model with more than one quantile. ## Reproduction (TDD) Added regression test `test_feature_importances_shape_matches_quantiles` that: - Constructs a `FlatlinerForecaster` with 5 quantiles - Asserts `.feature_importances` has columns matching the configured quantiles The test fails on `release/v4.0.0` and passes after the fix. ## Fix Build the row as a 2D list of zeros sized to the quantile count. ## Verification - `uv run poe lint` ✅ - `uv run poe format` ✅ - `uv run poe type` ✅ (0 errors, 0 warnings) - `uv run --package openstef-models pytest packages/openstef-models/tests/unit/models/forecasting/test_flatliner_forecaster.py` ✅ (4 passed) ## Ticket STEF-3036 --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com>
…866) ## Summary Fixes compatibility with XGBoost 3.2, which changed the shape of arrays passed to custom objectives via the sklearn interface. With `multi_strategy="one_output_per_tree"`, `y_true` and `y_pred` are now passed as 2D `(n_samples, n_quantiles)` arrays instead of 1D flattened arrays of length `n_samples * n_quantiles`. Closes #865 ## Root Cause The old reshape logic in both objective functions assumed 1D input: ```python n_items = len(y_true) n_quantiles = len(quantiles) n_rows = n_items // n_quantiles # ← breaks with 2D input y_pred = np.reshape(y_pred, (n_rows, n_quantiles)) y_true = np.reshape(y_true, (n_rows, n_quantiles)) sample_weight = np.reshape(sample_weight, (n_rows, -1)) if sample_weight is not None else None ``` With 2D input, `len(y_true)` returns `n_samples` (not `n_samples * n_quantiles`), so `n_rows = n_samples // n_quantiles` is wrong, causing the reshape to fail or produce incorrect results. The below image shows running a minimal script using XGBoost 3.1: <img width="1222" height="284" alt="Screenshot 2026-05-04 at 21 19 23" src="https://github.com/user-attachments/assets/ef4e031c-9cd0-426c-a3cf-16ac7ef8b7c8" /> The below image shows running the same minimal script using XGBoost 3.2: <img width="1208" height="475" alt="Screenshot 2026-05-04 at 21 22 12" src="https://github.com/user-attachments/assets/8a4a6a42-6554-452c-9906-742cd8152c2c" /> ## Fix Replaced with shape-agnostic normalisation in `arctan_loss_multi_objective` and `pinball_loss_multi_objective`: ```python n_quantiles = len(quantiles) y_pred = np.reshape(y_pred, (-1, n_quantiles)) y_true = np.reshape(y_true, (-1, n_quantiles)) sample_weight = sample_weight[:, np.newaxis] if sample_weight is not None else None ``` `np.reshape(arr, (-1, n_quantiles))` is a no-op when the array is already `(n_samples, n_quantiles)` and correctly reshapes a 1D flat array, maintaining backward compatibility with XGBoost <3.2. ## Investigation Notes - **`mean_pinball_loss`**: already used `np.reshape(y_pred, [-1, n_quantiles])` which handles both shapes correctly — no change needed - **`metrics_probabilistic.py`**: no changes needed, `sample_weight_eval_set` verified working correctly with XGBoost 3.2 - **`xgboost_forecaster.py`**: no changes needed - **`pyproject.toml`**: already at `>=3,<4` on this branch — no change needed
Signed-off-by: Bart Pleiter <bart.pleiter@alliander.com>
## Summary Adds regression tests verifying that `arctan_loss_multi_objective` and `pinball_loss_multi_objective` accept **2D `(n_samples, n_quantiles)` input arrays** as passed by XGBoost 3.2+, and produce results identical to the flat 1D arrays used in older versions. ## Context XGBoost 3.2 changed the calling convention for custom objectives with multi-output trees: arrays are now passed as 2D `(n_samples, n_outputs)` instead of the flat 1D `(n_samples * n_outputs,)` used previously. This was tracked in #865 and fixed in #866. ## Relationship to #866 - These tests **intentionally FAIL** on the unfixed code (this branch's base, `release/v4.0.0` before #866). - They will **pass** once #866 is merged. - This PR should be merged **after** #866 to ensure CI stays green. ## Tests added - `test_loss_fn__2d_input_matches_1d_input[pinball]` — pinball objective with 2D input - `test_loss_fn__2d_input_matches_1d_input[arctan]` — arctan objective with 2D input - `test_loss_fn__2d_input_with_sample_weights_matches_1d[pinball]` — pinball with 2D input + sample weights - `test_loss_fn__2d_input_with_sample_weights_matches_1d[arctan]` — arctan with 2D input + sample weights Closes #865 (when merged after #866) --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
This pull request introduces a new configuration option for controlling the maximum number of days to look back for day-based lags in forecasting workflows. ### Forecasting Workflow Configuration * Added a new `max_day_lags` parameter (default: 14, minimum: 1) to both `EnsembleForecastingWorkflowConfig` and `ForecastingWorkflowConfig`, allowing users to control the maximum lookback window for day-based lags. This parameter is now passed to relevant feature adders and forecaster builders. [[1]](diffhunk://#diff-621668fe8b344788245394f15bdf3af1284ad28a07966acfe1bc790654fe0842R236-R241) [[2]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R230-R235) [[3]](diffhunk://#diff-621668fe8b344788245394f15bdf3af1284ad28a07966acfe1bc790654fe0842R339) [[4]](diffhunk://#diff-621668fe8b344788245394f15bdf3af1284ad28a07966acfe1bc790654fe0842R424) [[5]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R345) --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
## Problem `restore_target` unconditionally restored all target values from the original dataset, undoing any NaN introduced by the `OutlierHandler`. This meant that even when `nan_on_outlier_features` included the target column, outlier rows were never excluded from training. ## Solution: Sentinel Columns The `OutlierHandler` now adds boolean sentinel columns (`__outlier_nan_<feature>__`) when using `outlier_action="nan"`, explicitly marking which values were NaN'd as outliers. `restore_target` checks for the sentinel column of the target feature: - If `__outlier_nan_load__` exists and is `True` → preserve the NaN (don't restore original value) - After restoring, all sentinel columns are removed from the dataset (they are not model features) This is explicit — no inference about which NaN values to preserve. Only NaN values explicitly flagged by the OutlierHandler are kept. ## Testing - 4 new unit tests (OutlierHandler sentinel column behavior + restore_target sentinel handling) - All 369 openstef-models tests pass --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
Fix: When calling a forecaster's `predict(input_data)` method, the target column from `input_data` was not carried over to the output `ForecastDataset`, so that the target column of the method's output is always the default "load" value. This led to errors in the downstream postprocessing. Should fix #871. Signed-off-by: Alexander Geiseler <don-alejandrino@outlook.com> Co-authored-by: Alexander Geiseler <don-alejandrino@outlook.com>
## Problem The OutlierHandler emits sentinel columns (e.g. `__outlier_nan_load_lag_P7D__`) when it NaN's values. These columns were included in `TimeSeriesDataset.feature_names`, causing feature-aware transforms (Scaler) to fit on them during training. At predict time, when no outliers are detected, the sentinel columns are absent → sklearn's feature-name validation crashes: ``` ValueError: The feature names should match those that were passed during fit. Feature names seen at fit time, yet now missing: - __outlier_nan_load_lag_P7D__ ``` ## Fix `TimeSeriesDataset` now treats any column whose name starts with `__` as an internal column (same mechanism as `horizon`/`available_at` columns). These are: - Kept in `data` so transforms can pass them through the pipeline - Excluded from `feature_names` so feature-aware transforms ignore them This makes the sentinel column approach work correctly: they flow through the pipeline untouched, are consumed by `restore_target` at the end, and never interfere with the Scaler or model. ## Changes - **openstef-core/timeseries_dataset.py**: Initialize `_internal_columns` with any `__`-prefixed columns; also apply in the non-versioned branch - **test_outlier_handler.py**: Assert sentinel columns are excluded from `feature_names` --------- Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com>
…851) ## New feature: Bayesian hyperparameter tuning with Optuna This PR introduces a first-class hyperparameter tuning API to OpenSTEF, powered by [Optuna](https://optuna.readthedocs.io). The design goal is to keep search space declarations **co-located with the model definition** — no separate config files, no extra boilerplate. --- ### How it works **Step 1 — declare the search space inline** Pass a `FloatRange`, `IntRange`, or `CategoricalRange` directly in the `HyperParams` constructor to mark fields for tuning. Everything else keeps its default value. ```python from openstef_core.mixins.param_ranges import FloatRange, IntRange from openstef_models.models.forecasting.xgboost_forecaster import XGBoostHyperParams config = ForecastingWorkflowConfig( model_id="my_forecast", model="xgboost", xgboost_hyperparams=XGBoostHyperParams( learning_rate=FloatRange(0.01, 0.3, log=True, tune=True), n_estimators=IntRange(50, 500, tune=True), max_depth=IntRange(3, 10, tune=True), ), ) ``` **Step 2 — run the tuner** ```python from openstef_models.integrations.optuna import HyperparameterTuner tuner = HyperparameterTuner( config=config, train_dataset=dataset, create_workflow=create_forecasting_workflow, target_quantile=Q(0.5), metric_name="R2", n_trials=50, ) result = tuner.fit_with_tuning() ``` **Step 3 — use the result** ```python result.best_config # ForecastingWorkflowConfig with tuned values applied result.workflow # Fitted workflow, ready to generate forecasts result.study # Full optuna.Study for analysis and visualisation ``` --- ### What's included | Component | Package | Description | |-----------|---------|-------------| | `FloatRange`, `IntRange`, `CategoricalRange` | `openstef-core` | Frozen dataclasses that encode tuning search spaces as field annotations | | `HyperParams.get_search_space()` | `openstef-core` | Resolves instance overrides against class-level defaults and returns only `tune=True` fields | | `get_tuning_range(field_info)` | `openstef-core` | Public helper that extracts a `TuningRange` from a Pydantic `FieldInfo`'s metadata | | `HyperparameterTuner` | `openstef-models` | Optuna-backed tuner: TPE sampling, median pruning, configurable direction/parallelism | | `TuningResult` | `openstef-models` | Typed result container: `best_config`, `study`, and fitted `workflow` | | `XGBoostHyperParams`, `GBLinearHyperParams` | `openstef-models` | All fields annotated with class-level default ranges; tuning opt-in per instantiation | The `HyperparameterTuner` is extensible: override `_evaluate_trial`, `_create_study`, or `suggest_value` to customise sampling, cross-validation, or pruning behaviour. --- ### Installation Optuna is an optional dependency. Install via: ```bash pip install openstef-models[tuning] # or with uv: uv add openstef-models[tuning] ``` --- ### Tutorial `examples/tutorials/hyperparameter_tuning_with_optuna.ipynb` walks through the full workflow: - Downloading the Liander benchmark dataset - Declaring an inline search space - Running a 20-trial study - Inspecting trial history and hyperparameter importances - Training the final model with the best config --- ### Testing ```bash # All tests (includes 36 targeted tuning tests) uv run poe tests # Targeted subset uv run pytest packages/openstef-models/tests/unit/integrations/optuna/ uv run pytest packages/openstef-core/tests/unit/mixins/ ``` --------- Signed-off-by: Fleur Petit <fleur.petit@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Signed-off-by: Lars van Someren <lvsom1@gmail.com> Signed-off-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> Signed-off-by: lschilders <lars.schilders@alliander.com> Signed-off-by: MentReeze <ment.reeze@alliander.com> Co-authored-by: Fleur Petit <fleur.petit@alliander.com> Co-authored-by: Lars Schilders <123180911+lschilders@users.noreply.github.com> Co-authored-by: Jan Maarten van Doorn <52956303+JanMaartenvanDoorn@users.noreply.github.com> Co-authored-by: Lars van Someren <93349011+Lars800@users.noreply.github.com> Co-authored-by: floriangoethals <floriangoehtals@gmail.com> Co-authored-by: Marnix van Lieshout <marnix.van.lieshout@alliander.com> Co-authored-by: Marnix van Lieshout <47454346+MvLieshout@users.noreply.github.com> Co-authored-by: MentReeze <ment.reeze@alliander.com>
…Error (#875) This pull request introduces a new callback for evaluating model performance in forecasting workflows, ensuring that models meet defined performance criteria before proceeding. It also adds a custom exception for underperforming models and comprehensive unit tests for the new logic. **Model performance evaluation and error handling:** * Added a new `ModelPerformanceCallback` class in `model_performance_callback.py` that checks model performance against a specified metric and threshold at the end of fitting. If the model does not meet the criteria, a `ModelUnderperfomingError` is raised, allowing workflows to stop early or use fallback models. * Introduced the `ModelUnderperfomingError` exception in `exceptions.py`, providing a descriptive error message when a model fails to meet performance requirements. **Testing:** * Added unit tests in `test_model_performance_callback.py` to verify correct behavior of the callback, including scenarios where metrics are missing, metric values are above/below thresholds, and both "higher_is_better" and "lower_is_better" directions are handled. --------- Signed-off-by: lschilders <lars.schilders@alliander.com>
Signed-off-by: lschilders <lars.schilders@alliander.com>
|
…#878) When adding standard (trivial) lags to a multi-horizon dataset, currently the longest horizon determines the smallest available lag. As an example, assume we have two horizons, 1 hour and 1 day. Using this logic, no lag smaller than 1 day is added to the dataset. This means that for the forecasting model, the data for BOTH horizons looks exactly like the data for the 1-day horizon. Instead, the shortest horizon should determine the smallest available lag. This means that in the above example, the following behavior should be observed: - All lags smaller than 1 hour are never available and therefore not added to the dataset. - All lags larger than 1 hour but smaller than 1 day are available for the 1-hour horizon, but not for the 1-day horizon. Therefore, the corresponding lag columns are filled partially, depending on the horizon. - All lags larger than 1 day are always available, so the corresponding columns are filled completely (except of course any transient behavior at the beginning). This PR proposes to basically just exchange `max_horizon` by `min_horizon` as a simple fix. I also added a test to verify the expected behavior end-to-end. --------- Signed-off-by: Alexander Geiseler <don-alejandrino@outlook.com> Co-authored-by: Alexander Geiseler <don-alejandrino@outlook.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| # Fall back to DummyClassifier when one forecaster dominates — sklearn classifiers need ≥2 classes | ||
| if len(labels.unique()) == 1: | ||
| logger.warning("Quantile %s has only 1 class — switching to DummyClassifier.", quantile.format()) | ||
| self._models[quantile] = DummyClassifier(strategy="most_frequent") |
| @pytest.fixture | ||
| def forecast_dataset_factory() -> Callable[[], ForecastDataset]: | ||
| def _make_forecast_dataset() -> ForecastDataset: | ||
| rng = np.random.default_rng() |
| @pytest.fixture | ||
| def forecast_dataset_factory() -> Callable[[], ForecastDataset]: | ||
| def _make() -> ForecastDataset: | ||
| rng = np.random.default_rng() |
| def dataset() -> tuple[pd.DataFrame, pd.Series]: | ||
| n_samples = 100 | ||
| n_features = 5 | ||
| rng = default_rng() |
…er into workflows (#879) This pull request introduces enhancements to the forecasting workflow configuration by adding a new model type and enabling model performance evaluation via a callback. The changes allow users to select a constant quantile forecaster and to configure a callback that checks model performance at the end of training against a configurable metric and threshold. **New model support:** * Added `ConstantQuantileForecaster` as a selectable model type in the `ForecastingWorkflowConfig`, allowing users to choose a model that predicts constant quantiles. [[1]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R33) [[2]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6L119-R121) [[3]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R508-R514) **Model performance evaluation:** * Introduced `ModelPerformanceCallback` and made it available for use in both standard and ensemble forecasting workflows. This callback can be enabled via the configuration and will evaluate model performance at the end of fitting, based on a user-specified metric, direction, quantile, and threshold. [[1]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R69) [[2]](diffhunk://#diff-c6a1961b093c5d165b3636b52517db96ccbf38d1a8061eb30ea9c04f64e5d75dR8-R13) [[3]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R292-R302) [[4]](diffhunk://#diff-621668fe8b344788245394f15bdf3af1284ad28a07966acfe1bc790654fe0842R75) [[5]](diffhunk://#diff-621668fe8b344788245394f15bdf3af1284ad28a07966acfe1bc790654fe0842R304-R314) [[6]](diffhunk://#diff-64a1e38462aac9581ce0cd24dd45e6c0de405f6a1ab401f02ac29d72886c2ee6R553-R563) [[7]](diffhunk://#diff-621668fe8b344788245394f15bdf3af1284ad28a07966acfe1bc790654fe0842R609-R619) These changes improve the flexibility and robustness of the forecasting workflow by allowing more model selection options and providing automated performance checks. --------- Signed-off-by: lschilders <lars.schilders@alliander.com>
## Summary Integrates [myst-nb](https://myst-nb.readthedocs.io/) into the Sphinx docs build so tutorial notebooks in `examples/tutorials/` render as notebook pages with code cells and narrative prose. This is **Phase 1** of the documentation infrastructure plan from #884. ## Changes ### Docs build - Replace `myst-parser` with `myst-nb` (superset that adds notebook execution) - Add `jupyter-cache` and `jupytext` as docs dependencies - Configure `nb_execution_mode="off"` (execution enabled in follow-up #886) - Exclude `.ipynb` from Sphinx source tree (use `.py` via `nb_custom_formats`) - Fix `rstjinja` handler to skip non-RST sources (prevent Jinja errors on `{}` in notebooks) - Symlink `docs/source/tutorials` → `examples/tutorials` ### Notebooks - Create `.py` jupytext pair for backtesting tutorial - Tag SPDX cells with `remove-cell` to hide license boilerplate from rendered output ### Tooling - Add `poe notebooks-clear` task (strip outputs from `.ipynb`) - Extend `poe notebooks-check` with output validation via `tools/check_notebook_outputs.py` ### Configuration - Add `E501`/`PTH` lint ignores for tutorials (educational code, not production) - Exclude `examples/tutorials/` from pyright (notebooks have cell-scoped execution) - Add `.jupyter_cache/` to `.gitignore` - Add tutorial `.py` files to `REUSE.toml` ### Documentation - Wire 3 tutorials into Examples toctree - Add "Working with tutorial notebooks" section to Contributing guide ## Verification - `poe all` ✅ (1020 tests pass, lint clean, types clean, notebooks synced) - `poe docs` ✅ (build succeeds, tutorials render without SPDX leakage) - No SPDX text appears in rendered HTML - Examples page lists all 3 tutorials with proper navigation ## Follow-up - #886: Optimize notebooks for CI execution, enable `nb_execution_mode="cache"`, switch to PNG plotting backend ## Related - Part of: #884 --------- Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com>
Add ContributionsPlotter with heatmap, waterfall and bar chart methods for visualizing per-sample feature contributions (SHAP values). ✅ Acceptance criteria - [x] Interesting plots from https://shap.readthedocs.io/en/latest/example_notebooks/overviews/An%20introduction%20to%20explainable%20AI%20with%20Shapley%20values.html added in a general way. Extended existing `ContributionsMixin` to include plotting method using ContibutionsPlotter. One of 3 types of plots can be selected: - A heatmap with contributions for each timestamp - Waterfall for deep-dive contributions of feature contributions for a single timestamp - Bar plot with mean contributions --------- Signed-off-by: Fleur Petit <fleur.petit@alliander.com> Signed-off-by: Egor Dmitriev <egor.dmitriev@alliander.com> Co-authored-by: Egor Dmitriev <egor.dmitriev@alliander.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.