Skip to content

chore: remove plugins now covered by OCS core, and fix the broken ERA5-Land import - #31

Merged
karimbahgat merged 4 commits into
mainfrom
chore/remove-core-duplicates
Nov 13, 2026
Merged

karimbahgat merged 4 commits into
mainfrom
chore/remove-core-duplicates

Conversation

@karimbahgat

@karimbahgat karimbahgat commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

OCS core now ships several data sources that this repo also had, and core's ERA5-Land refactor (open-climate-service #505) broke an import here.

Removed, now in core

  • CLMS NDVI: core has clms_ndvi_v3_dekadal (#507). The tests of the shared _clms.py moved to the productivity tests.
  • Copernicus DEM 90 m: core defines copdem90_elevation_static, the same id (#503).
  • ERA5-Land daily minimum and maximum temperature, and dewpoint: core has era5land_temperature_daily_min, era5land_temperature_daily_max and era5land_dewpoint_daily (#506). Note that the min/max temperature in ocs-plugin was local timezone aware, while in core they run in UTC+0 (local timezone support will be added later).

Fixed

  • ERA5-Land soil moisture and wind speed imported helpers from core's era5_land.py, which no longer exists. The five helpers they use are now copied into rasters/_era5_land.py rather than imported from core's private code.

Checks

  • Ruff and the full test suite pass. The lockfile tracks current core main.
  • No dataset id here collides with one in core.

Not changed: credentials work as before, and should be standalone/separate from builtin credentials.

Note: tests/test_era5_land_daily_stats_plugin.py was already empty, so the two remaining ERA5-Land plugins have no tests.

Core now has clms_ndvi_v3_dekadal (open-climate-service #507, CLIM-1153), the same
CLMS 300 m V3 product. The asset path and S3 key tests of the shared _clms reader
move to the productivity tests.
Core defines copdem90_elevation_static in copdem.yaml (open-climate-service #503),
the same id as this plugin's.
…OCS core

Core has era5land_temperature_daily_min/_max and era5land_dewpoint_daily
(open-climate-service #506, CLIM-1564). Soil moisture and wind speed stay.

Core removed era5_land.py in #505 (CLIM-1418), which broke this plugin's import.
The five helpers it used are copied into rasters/_era5_land.py rather than taken
from core's remaining private helpers.
@karimbahgat
karimbahgat requested a review from turban November 12, 2026 13:05

@turban turban left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is a good cleanup, and copying the helpers is better than reaching into core's private modules. Approving. A few notes, none blocking:

  1. The ERA5-Land replacements count days differently. The removed min/max/dewpoint templates asked CDS for days in the instance's own time zone. Core's era5land_temperature_daily_min/_max/era5land_dewpoint_daily work from the hourly ARCO store and their days run in UTC (their descriptions say so). For an instance like Nepal (+5:45), a "day" is shifted by almost six hours. That's probably fine, but worth a line in the PR body so nobody expects the same numbers. The ids also changed (era5land_temperature_min_daily → era5land_temperature_daily_min, clms_ndvi_dekadal → clms_ndvi_v3_dekadal), so instances that used the old ids need to re-ingest under the new ones. One leftover: the usage example in nepal-climate-service/scripts/animate.py still says clms_ndvi_dekadal.

  2. Possible follow-up: soil moisture could become a YAML-only template using core's ERA5LandCDSDailyFromHourlyPlugin (variable: swvl1, temporal_aggregation: mean), since core's ARCO store mapping already includes swvl1. That would also give UTC days. Wind speed still needs its own plugin because of the hypot, so the helpers would stay for now. Not for this PR.

  3. Tests: _era5_land.py is now maintained here, and the remaining ERA5-Land plugins have no tests. A couple of small unit tests for cds_fallback_area (the clamping at the poles and at ±180) and collapse_expver would be cheap and would catch drift.

@karimbahgat

Copy link
Copy Markdown
Collaborator Author

Thanks @turban . 1) True, added a brief note about it in the PR description. 2) Yes, and planning to include these in core as well, so waiting until then to make any improvements. 3) This was also raised while implementing this PR, but I don't think it's worth improving in this quick proof-of-concept repo, and will anyway disappear once soil moisture and wind speed is added to core.

@karimbahgat
karimbahgat merged commit c86dbcd into main Nov 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants