Repository navigation
feat(processes): land cover groups with remap_classes and an ESA CCI workflow - #30
Merged
Merged
Conversation
…workflow remap_classes merges a categorical raster's classes into named groups and records the groups as the cube's CF flags, so OCS core's aggregate_spatial_classes reports a share, total or majority per group, labelled. esacci_landcover_groups applies the ten IPCC-style groups of the fractions dataset to the core ESA CCI class dataset.
There was a problem hiding this comment.
🟡 Changes recommended
Unbounded lookup allocation can exhaust worker memory, and malformed class codes can silently produce incorrect mappings.
2 open issues
What changed in this PR
Adds categorical land-cover grouping before spatial aggregation, allowing OCS core to report group shares and correctly determine the majority group without precomputed fractions.
Changes:
- Adds
remap_classesto remap raster codes and replace their CF classification flags. - Adds an ESA CCI workflow defining ten land-cover groups.
- Documents usage and adds tests for remapping, aggregation, validation, discovery, and class coverage.
| File | Description |
|---|---|
tests/test_remap_classes.py |
Tests remapping, aggregation, validation, and workflow integration. |
README.md |
Documents grouping and its placement before aggregation. |
ocs_plugins/workflows/esacci_landcover_groups.json |
Defines the ten-group ESA CCI workflow. |
ocs_plugins/processes/remap_classes.py |
Implements class remapping and replacement classification metadata. |
ocs_plugins/__init__.py |
Updates the package docstring to mention workflows. |
Review effort: Balanced
…s, and sizes its table by the codes given A table indexed by class code took memory in proportion to the largest code, so one typo could ask for gigabytes; sorted codes with searchsorted take memory in proportion to the mapping. A code of 10.9 or true was quietly read as 10 or 1, and is now refused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


First step of CLIM-1575: land cover groups per district, computed from the class dataset rather than precomputed at ingest.
remap_classes(data, groups): merges a categorical raster's classes into named groups, numbered in the order given, and records the groups as the cube's CF flags. OCS core'saggregate_spatial_classesreads its classes from those flags, so it reports one share, total or majority per group, labelled with the group names, unchanged. A remap through the standardapplyandarray_elementkeeps the original flags, so the aggregation would still look for the original classes.esacci_landcover_groupsworkflow: OCS core'sesacci_landcover_class_yearlyin the ten groups of the fractions dataset (the IPCC land categories, with shrubland, sparse vegetation and bare land kept apart).majorityis otherwise wrong.The fractions dataset it replaces is removed in #29. Sending a per-district group share to DHIS2 still needs a change in OCS core's export (step 2 of CLIM-1575).
Tests: codes to groups and flags, a group's share equal to the sum of its classes' shares,
majoritypicking the group that covers most when no single class does, malformed groups refused, discovery through the entry point, and the workflow's groups covering every class of the core dataset exactly once.Run live against the Nepal test instance's 2022 ESA CCI store: the workflow followed by
aggregate_spatial_classesgave ten labelled shares per district, adding up to 1, in under two seconds for five districts.