Add forecast classes by peanutfun · Pull Request #1115 · CLIMADA-project/climada_python

peanutfun · 2025-11-21T17:26:10Z

Changes proposed in this PR:

Add forecast variants for Hazard and Impact.
Adapt ImpactCalc to return ImpactForecast when needed.
Add tests.
Add tutorial for forecast data.

This PR resolves a major portion of #1118. Note that sub-tasks have been marked as completed when merged into this branch.

PR Author Checklist

PR Reviewer Checklist

* define base class * add init test * Fix #1127 --------- Co-authored-by: Valentin Gebhart <valentin.gebhart@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add ImpactForecast * Add unit tests

* Create hazardForecast base class * Update datetime to timedelta for lead_time * Add tests --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* change leadtime to timedelta * adapt impactforecast docstrings and tests * Remove stray comment --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add HazardForecast.select test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add test for hazardForecast.concat * Skip HazardForecast.concat test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add test for ImpactForecast.select * minor test modification --------- Co-authored-by: Evelyn Mühlhofer <evelyn.muehlhofer@meteoswiss.ch>

* Add test stub for ImpactForecast.concat * Use correct command for skipping a test * Fix merge issue

* Add data shape check for HazardForecast, ImpactForecast

Adapt Hazard from_hdf5 and write_hdf5 to forecast attributes

* Add extended select impact forecast test * Add extended hazard forecast test --------- Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Return impactForecast object in _return_impact * Return impactForecast object in _return_empty * Add full impactcalc test for impactForecast * Raise value error when computing impact with impact forecast without saving impact matrix * Cosmetics: Improve error message, move test to own class * Write nans for eai_exp and aai_agg when forecast is used --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add idx boolean selection for member and leadtime * Add test for weird data types --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add ImpactForecast.select --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* add HazardForecast.select and extent test --------- Co-authored-by: Eliane Kobler <ekobler@student.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add HazardForecast.concat

Add min/mean/max methods --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Make Impact.concat support ImpactForecast * Add tests

* implement changes in Impact IO and ImpactForecast IO * Update tests * Fix small issue when reading H5 for ImpactForecast --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

* Add quantile and median for hazard and impact forecast * Review --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

…ython into forecast-class

* Add impact forecast tutorial * Add 'dim' argument to reductions * Update tests --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com> Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Evelyn Mühlhofer <evelyn.muehlhofer@meteoswiss.ch> Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com>

…ython into forecast-class

peanutfun · 2026-02-04T09:00:15Z

@Evelyn-M How was the fcst_icon_seamless_2024-06-18_12_00.nc test file created? I see that it's in a different CRS than the "latest" example files downloaded in the tutorial.

Evelyn-M · 2026-02-04T09:53:39Z

@Evelyn-M How was the fcst_icon_seamless_2024-06-18_12_00.nc test file created? I see that it's in a different CRS than the "latest" example files downloaded in the tutorial.

I downloaded it from the meteoswiss-internal data storage, since the API only permits downloads until -24 hrs.

peanutfun · 2026-02-05T12:39:36Z

So it had been reprojected? Does MCH store the data differently than OGD? I am just wondering because it might be important to highlight the caveats when importing datasets with varying CRS and coordinate specifications.

Evelyn-M · 2026-02-05T12:54:38Z

I suppose so, yes. One is the MCH grid commonly used internally for national gridded forecast products, and the OGD data has a different grid and a much larger rectangular extent around Switzerland.

chahank · 2026-02-16T15:29:05Z

Can you please let me know once this is final for review?

ValentinGebhart

Looks very good to me, all comments are minor I think.

I have not yet reviewed the tutorial, will do a separate review for that when i have time.

ValentinGebhart · 2026-02-06T17:11:46Z

+import numpy as np
+
+
+class Forecast:


It seems that there are people recommending to add "Mixin" to the class name for mixin classes. E.g. "ForecastMixin"

ValentinGebhart · 2026-02-06T19:27:15Z

+    """
+    concat_kws = {} if concat_kws is None else concat_kws
+    return forecast.concat(
+        [


I found this a bit hard to read, to understand what is happening. I see that the code is doing what it should, but maybe the variable names could be changed to make things clearer?
E.g.:
values --> attr_values
select --> attr_name
reduce_attr --> reduce_method

Renamed and reworked a bit

ValentinGebhart · 2026-02-10T14:50:33Z

+    npt.assert_array_equal(idx, np.array([True, False, False, False]), strict=True)
+
+    idx = forecast.idx_lead_time(None)
+    npt.assert_array_equal(idx, np.array([False, False, False, False]), strict=True)


no tests for
_reduce_attrs
reduce_unique_selection
_unique_or_default

but the working of these might be tested in the functions calling them

Private methods cannot be tested separately (because they are private?). reduce_unique_selection is tested via the reduction methods of the final classes. It's correct that a unit test should check the standalone function

ValentinGebhart · 2026-02-10T14:52:36Z

+        return reduced_attrs
+
+
+def reduce_unique_selection(


on first reading I found the back on forth confusing. e.g. HazardForecast.min() calls reduce_unqiue_slection with again calls HazardForecast.min().

I see why this works, just to comment on readability

ValentinGebhart · 2026-02-25T09:53:26Z

+        expected_n_events = (
+            forecast_netcdf_file["n_eps"] * forecast_netcdf_file["n_lead_time"]
+        )
+        npt.assert_array_equal(haz_fc.date, np.zeros(expected_n_events, dtype=int))


what about setting the date to the date of the reference time? Maybe it makes sense not to, just thinking

See comment above

ValentinGebhart · 2026-02-25T17:47:03Z

+
+    @at_event.setter
+    def at_event(self, value):
+        """Set the total impact for each member/lead_time combination."""


was this getter/setter set to allow for some future type checking or so? otherwise maybe not necessary?

Getter/setter methods exist to emit a warning when used

ValentinGebhart · 2026-02-25T18:00:31Z

+        select_mask = np.array([2, 1])
+        ordered_select_mask = np.array([1, 2])
+        if var == "date":
+            # Date needs to be a valid delta


same as in test_hazard_forecast.py

ValentinGebhart · 2026-02-25T18:06:28Z

+        )
+
+        num_centroids = len(impact_kwargs["coord_exp"])
+        imp_fc_select = impact_forecast.select(event_names=["aaaaa", "foo"])


Is it on purpose that in hazard_forecast.select() we raise an error when there is an emtpy selection and here not?

This follows the respective base class implementations 🤷

ValentinGebhart · 2026-02-25T18:16:46Z

+    @pytest.fixture
+    def reduction_results_dim(self):
+        return {
+            "lead_time": {


I think at_event should always return the sum of the impact matrix over the locations (i.e., axis=1), independent of if the eventdimension corresponds to lead_time or member.

In the test, this is tested for "member" but not for "lead_time". The values look like they would give the same sum(axis=1) and sum(axis=0). Maybe we could adapt a value such that we know the right axis is summed? If too compilated, maybe the test for "member" is enough.

I don't understand. at_event is always exactly np.sum(imp_mat, axis=1), both for member and for lead_time.

I think at_event should always return the sum of the impact matrix over the locations (i.e., axis=1), ...

I agree! The test checks this explicitly, in my perception?

sorry, my comment was not clear. You are right, at_event should always be exactly np.sum(imp_mat, axis=1).

My point was that in the test of "lead_time", the test values seem to be chosen such that the impact matrix is always symmetric (off diagonals are the same), which is why np.sum(imp_mat, axis=1) is the same as np.sum(imp_mat, axis=0). So we actually do not test if the correct axis is summed here. However, we test this in the test of "member", so that should suffice. For me we can leave as is.

peanutfun · 2026-03-03T20:51:11Z

@chahank This is final for review 🙂

chahank · 2026-03-06T10:08:03Z

Excellent! I will try to do it swiftly.

chahank · 2026-03-06T12:54:58Z

+        "aai_agg": 14.4,
+        "unit": "USD",
+        "frequency_unit": "1/month",
+        "imp_mat": sparse.csr_matrix(


Is it worth to test for negative impact values too?

ValentinGebhart · 2026-03-11T14:59:19Z

After downloading the data, we do

dict_ens_data["TOT_PREC"] = time_ops.delta( dict_ens_data["TOT_PREC"], np.timedelta64(1, "h") )

This looks like we change from total values to differences. I assume we do this because how TOT_PREC is defined?

We should for sure add an explanation and warning because if I just copy the code and use it for wind speed (VMAX_10M) I do NOT want to compute the difference.

Indeed, I added a comment

chahank · 2026-03-26T16:35:03Z

+        "haz_type": "TC",
+        "pool": None,
+        "intensity": sparse.csr_matrix(
+            [[0.2, 0.3, 0.4], [0.1, 0.1, 0.01], [4.3, 2.1, 1.0], [5.3, 0.2, 0.0]]


Please include negative value intensity.

The change to this file only wraps the previous dummy_hazard function into a fixture. I think changing the intensity values is out of scope, because changes here might have effects on all other hazard tests. We can pick this up in a discussion about the conftest.py fixtures, see #1278

chahank · 2026-03-26T16:40:51Z

Is there a plan already to work on the comments by @ValentinGebhart?

chahank · 2026-03-26T16:41:18Z

The tutorial is now titled only impact forecast, but it is mostly about hazard forecast. This is quite confusing. Can this maybe be split into two tutorials? Or give it a different name?

chahank · 2026-03-26T16:50:54Z

A few comments on the tutorial:

Can the tutorial of the impact forecast please add a section a the top that shows the whole pipeline within one cell for a fast introduction?
I think the cells with ?ImpactForecast and ?HazardForecast do not serve a clear purpose.
OVerall, the tutorial could need a bit of improvements in clarity. Some of the cells are just there without explanations.
There are a lot of of Logging that indicate something should be improved in the code I think. It is unclear why imp_fct_all_mem_1lt = impact_forecast.max(dim="lead_time") triggers many lines of info and warning logging. Or in some cell there is several repetitions of 2026-01-08 14:55:43,321 - climada.engine.impact - INFO - The eai_exp and aai_agg are computed for the selected subset of events WITHOUT modification of the frequencies..

chahank

Excellent addition to the CLIMADA platform!

Please find a few detailled comments by @ValentinGebhart to address, and some user friendliness requests for the tutorial.

Co-authored-by: Valentin Gebhart <60438839+ValentinGebhart@users.noreply.github.com>

…ython into forecast-class

peanutfun and others added 21 commits November 21, 2025 18:23

Forecast templates

ad819f2

Clean slate for Forecast

abafba4

1127 - Forecast base class init (#1167)

1688641

* define base class * add init test * Fix #1127 --------- Co-authored-by: Valentin Gebhart <valentin.gebhart@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Impact forecast class (#1168)

d66c8dd

* Add ImpactForecast * Add unit tests

Hazard forecast class (#1171)

35f9517

* Create hazardForecast base class * Update datetime to timedelta for lead_time * Add tests --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Change leadtime from datetime to timedelta (#1172)

54bb592

* change leadtime to timedelta * adapt impactforecast docstrings and tests * Remove stray comment --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Hazard forecast test select (#1175)

4aaad4e

* Add HazardForecast.select test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Hazard forecast concat test (#1178)

d04fecf

* Add test for hazardForecast.concat * Skip HazardForecast.concat test --------- Co-authored-by: luseverin <luca.severino@usys.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Check if Impact.select works on ImpactForecast (#1170)

d3cc83a

* Add test for ImpactForecast.select * minor test modification --------- Co-authored-by: Evelyn Mühlhofer <evelyn.muehlhofer@meteoswiss.ch>

Check if Impact.concat works on ImpactForecast (#1174)

a386d22

* Add test stub for ImpactForecast.concat * Use correct command for skipping a test * Fix merge issue

adapt from_hdf5 and write_hdf5

34612c5

Check sizes of forecast data vs. Hazard and Impact (#1180)

8a63b43

* Add data shape check for HazardForecast, ImpactForecast

implement code review

e123b7b

add comment to test

9ce7f73

Merge pull request #1181 from CLIMADA-project/adapt_hdf5_write_from

48b6d40

Adapt Hazard from_hdf5 and write_hdf5 to forecast attributes

Forecast/select extended tests (#1182)

64da484

* Add extended select impact forecast test * Add extended hazard forecast test --------- Co-authored-by: Chahan Kropf <chahan.kropf@posteo.com> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

add idx boolean selection for member and leadtime (#1183)

9d6fef9

* add idx boolean selection for member and leadtime * Add test for weird data types --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Add select to ImpactForecast (#1188)

b17bcc9

* Add ImpactForecast.select --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Add select method to HazardForecast (#1185)

302be63

* add HazardForecast.select and extent test --------- Co-authored-by: Eliane Kobler <ekobler@student.ethz.ch> Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Make Impact.concat support ImpactForecast

85b92ef

peanutfun force-pushed the forecast-class branch from 85b92ef to 302be63 Compare December 10, 2025 13:29

peanutfun and others added 8 commits December 10, 2025 14:32

Add HazardForecast.concat (#1184)

52edc45

* Add HazardForecast.concat

Support ImpactForecast.concat (#1192)

bb99794

* Make Impact.concat support ImpactForecast * Add tests

Add hdf5 IO for ImpactForecast (#1190)

0e7857f

* implement changes in Impact IO and ImpactForecast IO * Update tests * Fix small issue when reading H5 for ImpactForecast --------- Co-authored-by: Lukas Riedel <34276446+peanutfun@users.noreply.github.com>

Merge branch 'forecast-class' of github.com:CLIMADA-project/climada_p…

d84db1e

…ython into forecast-class

Merge branch 'forecast-class' of github.com:CLIMADA-project/climada_p…

99e82a1

…ython into forecast-class

ValentinGebhart reviewed Feb 25, 2026

View reviewed changes

chahank reviewed Mar 6, 2026

View reviewed changes

ValentinGebhart reviewed Mar 11, 2026

View reviewed changes

chahank reviewed Mar 26, 2026

View reviewed changes

chahank requested changes Mar 26, 2026

View reviewed changes

peanutfun and others added 15 commits March 27, 2026 17:04

Merge branch 'develop' into forecast-class

d487c27

Rename 'Forecast' to 'ForecastMixin'

4c07a37

Rework parameters to 'reduce_unique_selection'

380f0dd

Fix inconsistent override of Impact.from_hdf5

9137976

Update docstrings

0f06d36

Fix overrides in HazardForecast

8487eb9

Fix more docstrings

98b8bcb

Update climada/engine/test/test_impact_forecast.py

006559b

Co-authored-by: Valentin Gebhart <60438839+ValentinGebhart@users.noreply.github.com>

Avoid warnings emitted from ImpactForecast.select

ab37a8d

Update tests

259396f

Merge branch 'forecast-class' of github.com:CLIMADA-project/climada_p…

9e3cf17

…ython into forecast-class

Try suppressing more warnings

45150ed

Update tutorial

98e799d

Update tutorial

aaf80c1

Fix impact_calc fixture

6cc4eb5

Conversation

peanutfun commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Author Checklist

PR Reviewer Checklist

Uh oh!

peanutfun commented Feb 4, 2026

Uh oh!

Evelyn-M commented Feb 4, 2026

Uh oh!

peanutfun commented Feb 5, 2026

Uh oh!

Evelyn-M commented Feb 5, 2026

Uh oh!

chahank commented Feb 16, 2026

Uh oh!

ValentinGebhart left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

peanutfun commented Mar 3, 2026

Uh oh!

chahank commented Mar 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chahank commented Mar 26, 2026

Uh oh!

chahank commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chahank commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chahank left a comment

peanutfun commented Nov 21, 2025 •

edited

Loading

ValentinGebhart left a comment •

edited

Loading

chahank commented Mar 26, 2026 •

edited

Loading

chahank commented Mar 26, 2026 •

edited

Loading