Feat: scheduler support for device groups#1934
Conversation
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
…ontext Signed-off-by: F.N. Claessen <felix@seita.nl>
…formulation Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: F.N. Claessen <felix@seita.nl>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…and map them to the respective group id Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Documentation build overview
4 files changed± changelog.html± genindex.html± _autosummary/flexmeasures.data.models.planning.html± api/v3_0.html |
There was a problem hiding this comment.
Great draft! Let's start collecting todos in the PR description. We can decide what is in scope and what is out. One idea would be to split it up by feature:
Feature: multi-commodity
- spec how we would encode the commodity per device in the flex-model
- extend flex-model schema, limiting the commodity field to electricity and gas
- exclude gas-fired devices from all current EMS commitments
- add gas-price field to flex-context
- set up gas commitment using gas-powered devices and gas-price
- extend scheduling.rst
- extend tutorial
Feature: beyond electricity and gas
- add commodity field to commitment schema
- document how to set up other commodities
Feature: multi-feed stocks
- spec how we would encode the shared storage per device in the flex-model
- extend flex-model schema
- extend scheduling.rst
- extend tutorial
Feature: multi-feed flows
- spec how we would encode the shared capacity per device in the flex-model
- extend flex-model schema
- extend scheduling.rst
- extend tutorial
Alternatively, we do one PR first to create support for all 3 features in the device_scheduler only, and touch the StorageScheduler just enough so we can safely experiment with the first two features in simulation. Then, once we're experienced we tackle the flex-model schema, and split up the documentation work for the 3 features.
NB for the schema speccing, we already have some results from a previous session regarding feature 3, in terms of defining a grid topology. I believe the design that came out of that should also fit the other two features, so we end up with a similar UX for what would at first glance appear to be three very different features.
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>
…n-gas-and-electricity # Conflicts: # flexmeasures/data/models/planning/tests/test_commitments.py
…n-gas-and-electricity
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…mmitment does the optimizer treat their stocks as a combined resource Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
… preference for full soc sooner Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
The two changes together are the backwards-compatible default that was missing: when no device is specified (device=None), to_frame() must emit NaN in the device_group column (not crash), so the optimizer routes the commitment through ems_flow_commitment_equalities exactly as it did before device_group was introduced. Signed-off-by: F.N. Claessen <claessen@seita.nl>
…the pre-existing behaviour — each device is its own group Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
| expected_ev_costs = 2.2375 | ||
| expected_battery_costs = -5.515 | ||
| expected_total_cost = -3.2775 | ||
| expected_ev_costs = 2.3125 |
There was a problem hiding this comment.
This is the one thing I'm still unsure about. The total costs stayed the same, which is good, but the distribution between EV and battery shifted in the battery's favour. I don't have an explanation for it. I do know that there is currently no tie-breaking rule that would favour one over the other, which makes me willing to accept this.
…n-gas-and-electricity # Conflicts: # documentation/changelog.rst
| if isinstance(v, (list, tuple, set)): | ||
| groups = {device_group_map[d] for d in v} | ||
| if len(groups) != 1: | ||
| raise ValueError(f"Devices {v} map to multiple device groups: {groups}") |
There was a problem hiding this comment.
This will likely become problematic once we need to model a device that belongs to both a shared power constraint and a shared storage, but let's deal with it once the problem actually presents itself.
Add a new `group` field to the storage flex-model, letting device entries reference a power sensor that represents a group of devices (e.g. a hybrid inverter shared by a battery and PV). The group sensor's own flex-model entry constrains the group's aggregate power: - power-capacity: hard constraint (both directions) - consumption-capacity / production-capacity: soft constraints, enforced via breach commitments with default breach prices (10000 <currency>/kW) The group's scheduled aggregate power is saved to the group sensor. Nested groups are supported (cycles rejected); groups require multi-device flex-models and members must share a commodity. Implemented on top of the existing device-group solver machinery (ems_constraint_groups from the multi-commodity work, and grouped FlowCommitments from PR #1934); no changes to the optimizer itself. Also fixes a latent device-index misalignment in MetaStorageScheduler._prepare, where per-device lists were built from the full flex-model list (including stock-only entries) instead of the filtered device models. Closes #2092 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX Signed-off-by: F.N. Claessen <claessen@seita.nl>
…el group field) (#2276) * feat: support intermediate power constraints on groups of devices Add a new `group` field to the storage flex-model, letting device entries reference a power sensor that represents a group of devices (e.g. a hybrid inverter shared by a battery and PV). The group sensor's own flex-model entry constrains the group's aggregate power: - power-capacity: hard constraint (both directions) - consumption-capacity / production-capacity: soft constraints, enforced via breach commitments with default breach prices (10000 <currency>/kW) The group's scheduled aggregate power is saved to the group sensor. Nested groups are supported (cycles rejected); groups require multi-device flex-models and members must share a commodity. Implemented on top of the existing device-group solver machinery (ems_constraint_groups from the multi-commodity work, and grouped FlowCommitments from PR #1934); no changes to the optimizer itself. Also fixes a latent device-index misalignment in MetaStorageScheduler._prepare, where per-device lists were built from the full flex-model list (including stock-only entries) instead of the filtered device models. Closes #2092 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX * docs: cite PR #2276 in changelog entry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX * feat: support intermediate power constraints on groups of devices Add a new `group` field to the storage flex-model, letting device entries reference a power sensor that represents a group of devices (e.g. a hybrid inverter shared by a battery and PV). The group sensor's own flex-model entry constrains the group's aggregate power: - power-capacity: hard constraint (both directions) - consumption-capacity / production-capacity: soft constraints, enforced via breach commitments with default breach prices (10000 <currency>/kW) The group's scheduled aggregate power is saved to the group sensor. Nested groups are supported (cycles rejected); groups require multi-device flex-models and members must share a commodity. Implemented on top of the existing device-group solver machinery (ems_constraint_groups from the multi-commodity work, and grouped FlowCommitments from PR #1934); no changes to the optimizer itself. Also fixes a latent device-index misalignment in MetaStorageScheduler._prepare, where per-device lists were built from the full flex-model list (including stock-only entries) instead of the filtered device models. Closes #2092 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs: cite PR #2276 in changelog entry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: add group field to UI flex-model schema The UI-DB flex-model schema parity test requires every DBStorageFlexModelSchema field to have UI support. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: define intermediate power constraints via DB flex-models on the asset tree (#2277) Extend the storage flex-model's group field to also accept {"asset": <id>}, so a group can be identified by an asset-keyed flex-model entry - the form that DB-stored flex-models naturally produce. This makes intermediate power constraints fully definable via an asset tree with stored flex-models, with an empty flex-model in the scheduling trigger. - Asset-referenced group entries define no power sensor of their own; the group's scheduled aggregate power is saved via the entry's consumption and/or production output sensors, following the usual output-sensor conventions (full profile on a single sensor; clip-split when both given). - Sensor-referenced groups keep saving the aggregate to the group power sensor, and may now also define output sensors. - The UI asset flex-model editor recommends the parent asset when editing a child's group field (one-click suggestion), and hints in both the parent's and children's editors when the parent's flex-model defines power-capacity. - New tutorial: toy example for intermediate power constraints, driven entirely by DB-stored flex-models. Bug fixes uncovered along the way: - AssetTriggerSchema.check_flex_model_sensors raised KeyError on asset-only flex-model entries. - The freeze_server_now test fixture leaked its server_now monkeypatch into all subsequently run tests in the same process, causing order-dependent duplicate-key failures when scheduling jobs saved beliefs with identical (frozen) belief times. Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX Signed-off-by: F.N. Claessen <claessen@seita.nl> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> * docs: keep the group-constraints tutorial audience-appropriate Remove references to code modules, tests and GitHub issues from the tutorial; tutorials address FlexMeasures users, not developers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TAad46Ayg86DHpL6nY54sX Signed-off-by: F.N. Claessen <claessen@seita.nl> * fix: address review on the group field (XSS, error path, schema base) - asset_properties.html: HTML-escape the parent asset name before it is inserted into innerHTML in the group hint (it is user-controlled -> XSS). - storage.py: attach the "group must reference a power sensor" ValidationError to the `group` field, not `sensor`. - GroupReferenceSchema now inherits from SharedSensorReferenceSchema instead of SensorReferenceSchema, so it accepts only `sensor`/`asset` (a group is a device-group identifier, not a belief-query reference) and no longer exposes the source-* filter fields; regenerate openapi-specs.json accordingly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B Signed-off-by: F.N. Claessen <claessen@seita.nl> * docs: the model computes group schedules, it does not save them Reword _add_group_schedules' docstring: the scheduling model assembles and returns each group's aggregate power schedule; persistence is the data service's responsibility. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016MLCUiSdXDqDBmg8GbYp1B Signed-off-by: F.N. Claessen <claessen@seita.nl> --------- Signed-off-by: F.N. Claessen <claessen@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This PR extends the Commitment model and device scheduler to support grouped devices within a single commitment. This work is the basis for quite the list of follow-up features:
Why
Until now, a commitment could apply to:
This was insufficient for real‑world setups such as:
What changed
Commitments now support custom device groups, allowing a single
FlowCommitment or StockCommitment to aggregate over multiple devices.
The scheduler aggregates flows or stocks per device group and applies deviation penalties once per group per timestep.
Commitments may reference:
New Pyomo sets (cg, cjg) and constraints enforce grouped commitment
equalities consistently for both flow and stock commitments.
Cost reporting correctly aggregates costs per logical commitment instead of
duplicating them per device.
Result
This enables correct modelling of shared buffers and feeder‑level soft
constraints without artificial cost inflation or duplicated baselines.
How to test
1️⃣ Run the new unit tests
These tests verify:
2️⃣ Sanity‑check behaviour (what to look for)