From 07b068d892674292f9cf5275986149820720d2ff Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 17:12:50 +0000 Subject: [PATCH 01/17] Record the design for a Sienna investments portfolio Writes the agreed design before the work that implements it: what makes a PLEXOS object a candidate, which half of the cost PyPSA works out for itself, why one candidate becomes one technology, and why the portfolio is written beside the base system it expands. Three things in the SiennaSchemas investments namespace differ from what the work was scoped against, and each one decides part of the design, so the spec states them: there is no Node or Zone, the existing-fleet records are supplemental attributes rather than components, and TechnologyFinancialData has no field for a weighted average cost of capital. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- ...-09-08-plexos-sienna-investments-design.md | 121 ++++++++++++++++++ 1 file changed, 121 insertions(+) create mode 100644 docs/specs/2026-09-08-plexos-sienna-investments-design.md diff --git a/docs/specs/2026-09-08-plexos-sienna-investments-design.md b/docs/specs/2026-09-08-plexos-sienna-investments-design.md new file mode 100644 index 0000000..cb58cb9 --- /dev/null +++ b/docs/specs/2026-09-08-plexos-sienna-investments-design.md @@ -0,0 +1,121 @@ +# A PLEXOS expansion plan becomes a Sienna investments portfolio + +## The problem + +Every translation this repository makes gives an operations model. An operations model holds +a fixed fleet, so it has no place for a build cost, a build limit or a build year. Sienna +keeps investments in a separate schema with separate types, so this is a new destination, not +a flag on the dispatch path. + +Four things stop the expansion data today: + +- `PlexosProperty` names `Units` and `Units Out` and nothing else about building. +- The PLEXOS leg fixes every capacity on purpose, in five places. The note beside each one + says so: "v1 translates a dispatch model, so capacity is fixed". +- A run reads the plan at one moment. `stage_plexos_xml` takes a `horizon_year` and + `plexos_dated_properties.apply_window` narrows every dated property to the value in force, + so a generator that is not yet built reads `Units = 0` and is dropped as retired. +- A PLEXOS `Constraint` reaches nothing. `map_constraints` reads every one and reports it as + not carried. + +The route is PLEXOS to PyPSA to Sienna. A PyPSA network that states its own expansion reaches +the same destination through the second leg alone. + +## What the destination is, as it stands + +`Sienna-Platform/SiennaSchemas` is the authority. Its `Investments/` namespace holds 21 types. +Four things about it decide the design, and three of them differ from what the work was +scoped against: + +- There is **no `Node` and no `Zone`** under `Investments/`. `PortfolioDocument.aggregation` + names a region type the consumer resolves, so a PLEXOS `Region` becomes a region of the base + system, not an investments component. +- There is no `ExistingCapacity`. There is `ExistingDevices`, and it is a **supplemental + attribute** holding names of devices in the base system, as are `RetirementPotential` and + `TopologyMapping`. `PortfolioDocument` carries them in a flat `supplemental_attributes` + array beside a `supplemental_attribute_associations` table. +- `TechnologyFinancialData` has **no `interest_rate`**. It requires `capital_recovery_period`, + `technology_base_year`, `debt_fraction`, `debt_rate`, `return_on_equity` and `tax_rate`. +- `CarbonCaps` carries `target_year`, `max_mtons` and `max_tons_mwh`, and **no member list and + no region**. + +## The design + +### `Max Units Built` is what makes an object a candidate + +An object stating a count above zero may be built; an object stating none may not, and its +capacity stays fixed. The rule is the same for a `Generator`, a `Battery` and a +pumped-storage turbine, so one derivation serves all three. + +A candidate becomes an extendable PyPSA component. `p_nom_min` is the capacity the object +already has, which a build cannot take away; `p_nom_max` is that capacity plus `Max Units +Built` units of it. + +The dispatch pipelines keep writing what they write now. Every new column is null for an +object with no `Max Units Built`, and the sink omits a null column so PyPSA applies its own +default. + +### PyPSA works out the annuity, so the translator does not + +PyPSA 1.2 states that `overnight_cost` "takes precedence over `capital_cost`" and that PyPSA +"calculates annuity using `discount_rate` and `lifetime`". So the translator writes +`overnight_cost` from `Build Cost`, `discount_rate` from `WACC`, `lifetime` from `Economic +Life` and `fom_cost` from `FO&M Charge`, and never assembles a capital cost of its own. + +`Economic Life` is the capital recovery period, which is the period PyPSA annuitises across. +`Technical Life` is how long the plant runs, and PyPSA has one lifetime field which the +recovery period already claims. So the technical life travels in the extensions sidecar, and +the capacity of one unit travels beside it because PyPSA sizes a candidate by `p_nom_max` +alone. + +### One candidate becomes one technology + +Each PLEXOS candidate object becomes its own `SupplyTechnology` or `StorageTechnology`, named +after the object. Aggregating a category into one technology would need a rule for joining +unequal build costs, discount rates and lifetimes, and no PLEXOS field states that rule. The +mappings file keeps the shape it has now; an expansion run adds rows for the candidate +categories a dispatch run leaves out. + +### One run writes the portfolio and the base system it expands + +`ExistingDevices.existing_devices`, `RetirementPotential.eligible_generators` and +`TopologyMapping.buses` are all lists of names in the base system. Without a base system they +point at nothing, so `pypsa-to-sienna-investments` runs the operations steps and the new +investments steps over one network and writes both documents. A component whose +`p_nom_extendable` is true is not a base system component, and the operations steps report +each one as skipped on that leg. + +### `WACC` is written as all-equity financing + +`TechnologyFinancialData` requires all six of its fields and has none for a weighted average +cost of capital. The portfolio states `debt_fraction: 0`, `debt_rate: 0`, `tax_rate: 0` and +`return_on_equity` equal to the model's `WACC`. With no debt the weighted average cost of +capital equals the return on equity, so the number the model states is the number the solver +uses. It is a derivation, not a default. + +### A scoped `Constraint` is left out + +`CarbonCaps` names no members and no region, so a PLEXOS `Constraint` over a named subset of +generators cannot be written as one. Only a constraint whose members cover the whole model +becomes a `CarbonCaps`. Any other is left out and named in `decisions.md`, rather than written +as a cap that silently applies to more than the model meant. + +### The build year rides the hub; the retirement year rides the sidecar + +PyPSA carries `build_year` on Generator, StorageUnit, Line and Link, so a build year survives +the netCDF round trip as a real field. PyPSA has no retirement year, so a planned retirement +travels in the extensions sidecar, as `ExtensionKind.RESERVE` already travels for a concept +PyPSA does not hold. + +Reading either one needs the whole dated `Units` series, which the `horizon_year` narrowing +throws away today, so the source stages every date band beside the value in force. + +## What this does not do + +- It does not translate the investment periods, the representative days or their weights. + Those live on the model template rather than in the portfolio file, so they belong on a + solve request beside the network model and the solver options. +- It does not write the PowerSystemsInvestments.jl JSON envelope. The product of a + translation stays the SiennaSchemas document. +- It does not aggregate candidates by category. +- It does not translate an extendable Line or Link into a transport technology. From b16da00270eb261cf130a8b2862385b82ce39898 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 17:13:00 +0000 Subject: [PATCH 02/17] Read a PLEXOS expansion plan and write an extendable component for it The PLEXOS reader named Units and Units Out and nothing else about building, and every capacity mapping wrote p_nom_extendable False on purpose. So a model stating what it may build lost all of it on the first leg. PlexosProperty now names Build Cost, Max Units Built, WACC, Economic Life, Technical Life and FO&M Charge, and Build Cost and FO&M Charge convert from $/kW as they stage. Max Units Built above zero is what makes an object a candidate, and the rule is the same for a Generator, a Battery and a pumped-storage turbine, so one derivation serves all three. PyPSA takes overnight_cost in preference to capital_cost and works the annuity out from the discount rate and the lifetime, so this writes those four fields and assembles no cost of its own. Economic Life is the recovery period PyPSA annuitises across; Technical Life and the size of one unit have no PyPSA field and travel in the extensions sidecar instead. A candidate that has nothing yet takes the capacity it may build as its p_nom. PyPSA reads p_nom only for a component whose capacity is fixed, and it is what every per-unit field on the component is read against, so against nothing each of those would come out at zero. A candidate that states no Build Cost is left out: nothing prices building it, so an expansion would take it for free. Each one is recorded, and one warning names a few of them. Every new column is null for an object the model cannot build, and the sink omits a null column, so the dispatch pipelines write what they wrote before. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 64 ++++++- interop/core/extensions.py | 12 ++ interop/plugins/shared/constants.py | 3 + interop/plugins/shared/plexos_constants.py | 7 + .../plexos_pypsa_translations/_batteries.py | 14 +- .../_generator_decisions.py | 131 ++++++++++++- .../_generator_derivation.py | 90 ++++++++- .../_generator_lookups.py | 4 + .../plexos_pypsa_translations/_generators.py | 91 +++++++-- .../plexos_pypsa_translations/_shared.py | 13 +- .../_storage_hydro.py | 14 +- .../_storage_shared.py | 175 +++++++++++++++++- .../_storage_units.py | 29 +++ .../plexos_pypsa_translations/constants.py | 12 +- .../plexos_pypsa_translations/decisions.py | 14 +- interop/plugins/shared/plexos_units.py | 24 ++- interop/plugins/shared/pypsa_constants.py | 16 ++ interop/plugins/sinks/emit_pypsa_network.py | 19 +- .../plexos_to_pypsa/generators.feature | 57 ++++++ .../plexos_to_pypsa/storage_units.feature | 31 ++++ 20 files changed, 775 insertions(+), 45 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index d0d8dae..5b2ebcd 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -3,10 +3,11 @@ This document tells you what each part of your PLEXOS model becomes in the PyPSA network. It gives the source of each field. -> **Scope:** the translator accepts electricity-only models, and it translates them for -> dispatch. It does not translate capacity expansion, custom constraints or hydro cascades. -> It carries the reserves to a sidecar file, but it does not apply them. Refer to -> [Reserves](#reserve--extensions-sidecar) and [Not translated](#not-translated). The +> **Scope:** the translator accepts electricity-only models. It does not translate custom +> constraints or hydro cascades. It carries the reserves to a sidecar file, but it does not +> apply them. Refer to [Reserves](#reserve--extensions-sidecar) and +> [Not translated](#not-translated). It reads what your model allows to be built and writes +> an extendable component for it; refer to [What a candidate is](#what-a-candidate-is). The > `plexos-to-pypsa-monte-carlo-reliability` pipeline also adds a load shedding generator at > each bus. Refer to [Load shedding](#load-shedding). @@ -269,16 +270,24 @@ is `committable` when it is thermal, or when its `p_min_pu` is more than `0`. | `start_up_cost` | $ | Refer to [What a start costs](#what-a-start-costs) | `derived` | | `shut_down_cost` | $ | `0.0` | `default` | | `up_time_before` | snapshots | `0` | `default` | - -**The translator does not translate four cases.** It records each one as a skipped +| `p_nom_extendable` | | `True` where `Max Units Built` is above zero. If not, `False`. | `derived` / `default` | +| `p_nom_min` | MW | `Max Capacity × Units`, for a candidate only | `derived` | +| `p_nom_max` | MW | `Max Capacity × (Units + Max Units Built)`, for a candidate only | `derived` | +| `overnight_cost` | $/MW | `Build Cost`, for a candidate only | `direct` | +| `discount_rate` | | `WACC`, for a candidate only | `derived` | +| `lifetime` | yr | `Economic Life`, for a candidate only | `direct` | +| `fom_cost` | $/MW/yr | `FO&M Charge`, for a candidate only | `direct` | + +**The translator does not translate five cases.** It records each one as a skipped component: | Case | Cause | | --- | --- | | The generator has no `Node` | There is no bus to connect the generator to. | -| The generator has no [`Units`](#which-entry-applies-when) at any time in the horizon | The unit is retired. | +| The generator has no [`Units`](#which-entry-applies-when) at any time in the horizon, and no `Max Units Built` | The unit is retired. | | `Max Capacity` comes from a data file | There is no single `p_nom`. Thus the translator cannot set the size of the generator, and it cannot calculate the availability per unit. | | `p_nom` is 0 | The generator can never dispatch. | +| The generator is a candidate and gives no `Build Cost` | Nothing prices building it, so an expansion would take it for free. | The repair rates, the unit commitment solver options and the energy budgets are `dropped`. @@ -349,6 +358,7 @@ has more than one fuel uses its primary fuel. | `state_of_charge_initial` | MWh | `Initial SoC % × Capacity` | `derived` | | `cyclic_state_of_charge` | | `True` if `End Effects Method` is `RECYCLE`, or if the model gives no `Initial SoC` | `derived` | | `marginal_cost` | $/MWh | `0.0` | `default` | +| `p_nom_extendable`, `p_nom_min`, `p_nom_max`, `overnight_cost`, `discount_rate`, `lifetime`, `fom_cost` | | Refer to [What a candidate is](#what-a-candidate-is) | `derived` / `default` | The energy capacity of a battery is its `Capacity`. If the model gives a duration in place of a capacity, the energy capacity is `Duration × Max Power`. `max_hours` and @@ -381,6 +391,7 @@ name of the turbine is the name of that `StorageUnit`. | `cyclic_state_of_charge` | | `True` if `End Effects Method` is `RECYCLE` | `derived` | | `marginal_cost` | $/MWh | `VO&M Charge`. If there is none, `0.0`. | `derived` | | `inflow` | MW | The `Natural Inflow` of the head reservoir, if the model gives it in a unit that converts to MW. A `Natural Inflow` that reads a data file becomes a time series. | `derived` | +| `p_nom_extendable`, `p_nom_min`, `p_nom_max`, `overnight_cost`, `discount_rate`, `lifetime`, `fom_cost` | | Refer to [What a candidate is](#what-a-candidate-is) | `derived` / `default` | The turbine finds its reservoirs through its `Head Storage` membership and its `Tail Storage` membership. The translator never compares the names of the reservoirs. The @@ -686,6 +697,45 @@ the file. `Max Capacity`, `Min Stable Level`, `Rating`, `Rating Factor`, `Units Out` and a Region `Load` can all come to the translation in this form. +### What a candidate is + +`Max Units Built` is what makes a PLEXOS object a candidate. An object stating a count above +zero may be built; an object stating none may not, and its capacity is fixed. The rule is the +same for a `Generator`, a `Battery` and a pumped-storage turbine. + +A candidate becomes an extendable PyPSA component. These are the fields it carries, and the +PLEXOS property each one comes from: + +| PyPSA field | Unit | From | +| --- | --- | --- | +| `p_nom_extendable` | | `Max Units Built` above zero | +| `p_nom_min` | MW | The capacity the object already has, which a build cannot take away | +| `p_nom_max` | MW | That capacity plus `Max Units Built` units of it | +| `overnight_cost` | $/MW | `Build Cost` | +| `discount_rate` | | `WACC`, read as a fraction where the model states a percentage | +| `lifetime` | yr | `Economic Life` | +| `fom_cost` | $/MW/yr | `FO&M Charge` | + +PyPSA works out what a year of new capacity costs from `overnight_cost`, `discount_rate` and +`lifetime` together, and `overnight_cost` takes precedence over `capital_cost`. So the +translator writes those three and never assembles an annuity of its own. + +`Economic Life` is the period the capital is recovered over, which is the period PyPSA +annuitises across. `Technical Life` is how long the plant runs, and PyPSA has one lifetime +field, which the recovery period already claims. So the technical life travels in the +extensions sidecar as `technical_life_years`. The capacity of one unit travels beside it as +`unit_size_mw`, because PyPSA sizes a candidate by `p_nom_max` alone. + +A candidate that has nothing yet takes the capacity it may build as its `p_nom`. PyPSA +optimises `p_nom_opt` between `p_nom_min` and `p_nom_max` and reads `p_nom` only for a +component whose capacity is fixed, so this does not bind the dispatch. It is what every +per-unit field on the component is read against — `p_min_pu`, a ramp limit, an availability +profile stated in MW — and against nothing each of those would come out at zero. + +A candidate that states no `Build Cost` is left out. Nothing prices building it, so an +expansion would take it for free. The translator records a `COMPONENT_SKIPPED` event naming +the object, and warns once naming a few of them. + ### Which entry applies when A property can have a base value and more entries. Each of the other entries has a scope, diff --git a/interop/core/extensions.py b/interop/core/extensions.py index 8ddad15..f0bd711 100644 --- a/interop/core/extensions.py +++ b/interop/core/extensions.py @@ -107,6 +107,12 @@ class GeneratorExtension(ExtensionRecord): p_nom_extendable: bool | None = None # PLEXOS only: the generator's category, a grouping string the user chooses. category: str | None = None + # MW. PLEXOS Max Capacity: what one unit of a candidate is. PyPSA sizes a candidate by + # p_nom_max alone, so the size of a single unit has no field there. + unit_size_mw: float | None = None + # yr. PLEXOS Technical Life: how long the plant runs. PyPSA has one lifetime and the + # capital recovery period claims it, so this has no field there. + technical_life_years: float | None = None class LoadExtension(ExtensionRecord): @@ -145,6 +151,12 @@ class ControllableLineExtension(ExtensionRecord): class StorageExtension(ExtensionRecord): p_nom_extendable: bool | None = None # PyPSA only + # MW. PLEXOS Max Power: what one unit of a candidate is. PyPSA sizes a candidate by + # p_nom_max alone, so the size of a single unit has no field there. + unit_size_mw: float | None = None + # yr. PLEXOS Technical Life: how long the plant runs. PyPSA has one lifetime and the + # capital recovery period claims it, so this has no field there. + technical_life_years: float | None = None class ReserveExtension(ExtensionRecord): diff --git a/interop/plugins/shared/constants.py b/interop/plugins/shared/constants.py index 5ea33ff..5ec953b 100644 --- a/interop/plugins/shared/constants.py +++ b/interop/plugins/shared/constants.py @@ -53,6 +53,9 @@ class StagedTimeSeriesCol: UNIT_SNAPSHOTS = "snapshots" UNIT_DOLLARS = "$" UNIT_DOLLARS_PER_MWH = "$/MWh" +UNIT_DOLLARS_PER_MW = "$/MW" +UNIT_DOLLARS_PER_MW_YEAR = "$/MW/yr" +UNIT_YEARS = "yr" UNIT_DOLLARS_PER_GJ = "$/GJ" UNIT_DOLLARS_PER_TONNE = "$/tonne" UNIT_GJ = "GJ" diff --git a/interop/plugins/shared/plexos_constants.py b/interop/plugins/shared/plexos_constants.py index 32ddbed..e6a40f2 100644 --- a/interop/plugins/shared/plexos_constants.py +++ b/interop/plugins/shared/plexos_constants.py @@ -162,6 +162,13 @@ class PlexosProperty(StrEnum): MIN_UP_TIME = "Min Up Time" MIN_DOWN_TIME = "Min Down Time" PUMP_EFFICIENCY = "Pump Efficiency" + # Expansion, which a Generator and a Battery state alike + BUILD_COST = "Build Cost" + MAX_UNITS_BUILT = "Max Units Built" + WACC = "WACC" + ECONOMIC_LIFE = "Economic Life" + TECHNICAL_LIFE = "Technical Life" + FOM_CHARGE = "FO&M Charge" # Battery CAPACITY = "Capacity" MAX_POWER = "Max Power" diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index 714862f..a4e0883 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -24,7 +24,6 @@ CARRIER_NOTE, CHARGE_NOTE, EFFICIENCY_NOTE, - EXTENDABLE_NOTE, FULL_DISCHARGE_NOTE, NO_RESERVOIR_INFLOW_NOTE, MappedOrSkipped, @@ -35,6 +34,7 @@ StorageLookups, StorageUnitMapping, derive_bus, + derive_expansion, derive_max_hours, derive_round_trip_efficiency, derive_state_of_charge_initial, @@ -50,7 +50,6 @@ STORAGE_FULL_CHARGE_PU, STORAGE_FULL_DISCHARGE_PU, STORAGE_MARGINAL_COST, - STORAGE_P_NOM_EXTENDABLE, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( Decision, @@ -116,6 +115,7 @@ def _battery_p_nom(staged: StagedObject) -> Decision: def _derive_battery(rated: RatedObject) -> StorageUnitMapping: capacity = _battery_energy_capacity(rated) max_hours = derive_max_hours(capacity, rated.p_nom.value) + expansion = derive_expansion(PlexosClass.BATTERY, rated) return StorageUnitMapping( name=rated.name, bus=derive_bus(PlexosClass.BATTERY, rated.name, rated.node), @@ -131,8 +131,16 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: ), inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE), cyclic=_battery_cyclic(rated), - p_nom_extendable=Decision.default(STORAGE_P_NOM_EXTENDABLE, EXTENDABLE_NOTE), + p_nom_extendable=expansion.p_nom_extendable, + p_nom_min=expansion.p_nom_min, + p_nom_max=expansion.p_nom_max, + overnight_cost=expansion.overnight_cost, + discount_rate=expansion.discount_rate, + lifetime=expansion.lifetime, + fom_cost=expansion.fom_cost, units=rated.properties.get(PlexosProperty.UNITS), + unit_size=expansion.unit_size, + technical_life=expansion.technical_life, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py index b122376..8677a05 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py @@ -13,6 +13,8 @@ from interop.plugins.shared.constants import ( UNIT_DOLLARS, UNIT_DOLLARS_PER_GJ, + UNIT_DOLLARS_PER_MW, + UNIT_DOLLARS_PER_MW_YEAR, UNIT_DOLLARS_PER_MWH, UNIT_DOLLARS_PER_TONNE, UNIT_GJ, @@ -23,6 +25,7 @@ UNIT_MW_PER_MINUTE, UNIT_PER_UNIT_PER_HOUR, UNIT_SNAPSHOTS, + UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import ( PlexosClass, @@ -39,6 +42,7 @@ UnitCommitment, ) from interop.plugins.shared.plexos_pypsa_translations.constants import ( + DIRECT_DERIVATION, FULL_AVAILABILITY, MARGINAL_COST_CARBON_TERM, START_UP_COST_FUEL_TERM, @@ -97,8 +101,18 @@ _SHUT_DOWN_NOTE = "PLEXOS prices only starts, so shutting down is free" _EFFICIENCY_DERIVATION = "p_nom / (Heat Rate Base + Heat Rate Incr x p_nom) x 3.6" _DISCARDED_FUEL_NOTE = "a multi-fuel generator keeps its first fuel; this one is discarded" -_EXTENDABLE_NOTE = "v1 translates a dispatch model, so capacity is fixed" +_EXTENDABLE_DERIVATION = "Max Units Built above zero is what makes a generator a candidate" +_NOT_EXTENDABLE_NOTE = "the generator states no Max Units Built, so its capacity is fixed" _NOT_EXTENDABLE = False +_P_NOM_MIN_DERIVATION = ( + "Max Capacity x Units, the capacity the generator already has and a build cannot remove" +) +_P_NOM_MAX_DERIVATION = "Max Capacity x (Units + Max Units Built)" +_P_NOM_CANDIDATE_DERIVATION = ( + "the generator has no units yet, so its nominal power is the capacity it may build: " + "Max Capacity x Max Units Built" +) +_DISCOUNT_RATE_DERIVATION = "WACC, read as a fraction where the model states a percentage" # The value a p_max_pu event carries when the ceiling varies over the horizon. _PROFILE = "profile" @@ -140,6 +154,12 @@ class GeneratorDecisions: start_up_cost: Decision = maps_to(PyPSAGeneratorCol.START_UP_COST, unit=UNIT_DOLLARS) shut_down_cost: Decision = maps_to(PyPSAGeneratorCol.SHUT_DOWN_COST, unit=UNIT_DOLLARS) p_nom_extendable: Decision = maps_to(PyPSAGeneratorCol.P_NOM_EXTENDABLE) + p_nom_min: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MIN, unit=UNIT_MW) + p_nom_max: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MAX, unit=UNIT_MW) + overnight_cost: Decision = maps_to(PyPSAGeneratorCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW) + discount_rate: Decision = maps_to(PyPSAGeneratorCol.DISCOUNT_RATE) + lifetime: Decision = maps_to(PyPSAGeneratorCol.LIFETIME, unit=UNIT_YEARS) + fom_cost: Decision = maps_to(PyPSAGeneratorCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR) def decide_generator(mapping: GeneratorMapping) -> GeneratorDecisions: @@ -168,7 +188,13 @@ def decide_generator(mapping: GeneratorMapping) -> GeneratorDecisions: up_time_before=commitment.up_time_before, start_up_cost=commitment.start_up_cost, shut_down_cost=commitment.shut_down_cost, - p_nom_extendable=Decision.default(_NOT_EXTENDABLE, _EXTENDABLE_NOTE), + p_nom_extendable=_p_nom_extendable(mapping), + p_nom_min=_p_nom_min(mapping), + p_nom_max=_p_nom_max(mapping), + overnight_cost=_overnight_cost(mapping), + discount_rate=_discount_rate(mapping), + lifetime=_economic_life(mapping), + fom_cost=_fom_cost(mapping), ) @@ -215,10 +241,10 @@ def _carrier(mapping: GeneratorMapping) -> Decision: def _p_nom(mapping: GeneratorMapping) -> Decision: - nameplate = [ - _source(mapping.name, PlexosProperty.MAX_CAPACITY, mapping.max_capacity, UNIT_MW), - _source(mapping.name, PlexosProperty.UNITS, mapping.units), - ] + nameplate = _capacity_sources(mapping) + if not mapping.units and mapping.expansion.is_candidate: + sources = [nameplate[0], _units_built_source(mapping)] + return Decision.derived(mapping.p_nom, sources, _P_NOM_CANDIDATE_DERIVATION) capacity = mapping.rating_as_capacity if capacity is None: return Decision.derived(mapping.p_nom, nameplate, _P_NOM_DERIVATION) @@ -226,6 +252,99 @@ def _p_nom(mapping: GeneratorMapping) -> Decision: return Decision.derived(mapping.p_nom, [rating, *nameplate], _P_NOM_RATING_DERIVATION) +def _capacity_sources(mapping: GeneratorMapping) -> list[SourceValue]: + """The two properties every capacity bound of a generator is built from.""" + return [ + _source(mapping.name, PlexosProperty.MAX_CAPACITY, mapping.max_capacity, UNIT_MW), + _source(mapping.name, PlexosProperty.UNITS, mapping.units), + ] + + +def _units_built_source(mapping: GeneratorMapping) -> SourceValue: + return _source(mapping.name, PlexosProperty.MAX_UNITS_BUILT, mapping.expansion.max_units_built) + + +def _p_nom_extendable(mapping: GeneratorMapping) -> Decision: + if not mapping.expansion.is_candidate: + return Decision.default(_NOT_EXTENDABLE, _NOT_EXTENDABLE_NOTE) + return Decision.derived(True, [_units_built_source(mapping)], _EXTENDABLE_DERIVATION) + + +def _p_nom_min(mapping: GeneratorMapping) -> Decision: + """A build adds to what the generator has, so what it has is the floor of what it keeps.""" + p_nom_min = mapping.expansion.p_nom_min + if p_nom_min is None: + return Decision.unreported(None) + return Decision.derived(p_nom_min, _capacity_sources(mapping), _P_NOM_MIN_DERIVATION) + + +def _p_nom_max(mapping: GeneratorMapping) -> Decision: + p_nom_max = mapping.expansion.p_nom_max + if p_nom_max is None: + return Decision.unreported(None) + sources = [*_capacity_sources(mapping), _units_built_source(mapping)] + return Decision.derived(p_nom_max, sources, _P_NOM_MAX_DERIVATION) + + +def _overnight_cost(mapping: GeneratorMapping) -> Decision: + cost = mapping.expansion.overnight_cost + return _from_property( + mapping.name, + cost, + cost, + PlexosProperty.BUILD_COST, + UNIT_DOLLARS_PER_MW, + DIRECT_DERIVATION, + ) + + +def _discount_rate(mapping: GeneratorMapping) -> Decision: + expansion = mapping.expansion + return _from_property( + mapping.name, + expansion.discount_rate, + expansion.wacc, + PlexosProperty.WACC, + None, + _DISCOUNT_RATE_DERIVATION, + ) + + +def _economic_life(mapping: GeneratorMapping) -> Decision: + """PyPSA reads one lifetime and spreads the overnight cost across it, which is the + period PLEXOS states as the Economic Life rather than the Technical Life.""" + life = mapping.expansion.economic_life + return _from_property( + mapping.name, life, life, PlexosProperty.ECONOMIC_LIFE, UNIT_YEARS, DIRECT_DERIVATION + ) + + +def _fom_cost(mapping: GeneratorMapping) -> Decision: + cost = mapping.expansion.fom_cost + return _from_property( + mapping.name, + cost, + cost, + PlexosProperty.FOM_CHARGE, + UNIT_DOLLARS_PER_MW_YEAR, + DIRECT_DERIVATION, + ) + + +def _from_property( + name: str, + value: float | None, + stated: float | None, + plexos_property: str, + unit: str | None, + derivation: str, +) -> Decision: + """One expansion value and the property it came from, unreported where it states none.""" + if value is None: + return Decision.unreported(None) + return Decision.derived(value, [_source(name, plexos_property, stated, unit)], derivation) + + def _p_min_pu(mapping: GeneratorMapping) -> Decision: minimum = mapping.minimum if minimum.source_property is None or minimum.source_value is None: diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py index 3771bf5..6164e9f 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py @@ -18,6 +18,7 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._generator_lookups import Lookups +from interop.plugins.shared.plexos_pypsa_translations._shared import as_rate from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_P_MIN_PU, DEFAULT_SHUT_DOWN_COST, @@ -26,6 +27,7 @@ FULL_AVAILABILITY, MAX_RAMP_LIMIT_PU, NEGLIGIBLE_P_MIN_PU, + NOTHING_TO_BUILD, PERCENT, ) @@ -85,23 +87,59 @@ class SourceGenerator: category: str units: float props: dict[str, float] + stated_units: dict[str, str | None] max_capacity: float + @property + def max_units_built(self) -> float: + """How many units the model allows to be built, which is 0 for a fixed generator.""" + return _value(self.props, PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) + + @property + def is_candidate(self) -> bool: + """Whether the model allows this generator to be built, which is what Max Units + Built decides. A generator that is only a candidate states no Units at all.""" + return self.max_units_built > NOTHING_TO_BUILD + + @property + def buildable(self) -> float: + """MW the model allows to be built, on top of whatever the generator already has.""" + return self.max_units_built * self.max_capacity + @property def nameplate(self) -> float: return self.max_capacity * self.units @property def rating_as_capacity(self) -> float | None: - """A Rating above the nameplate replaces Max Capacity rather than derating it.""" + """A Rating above the nameplate replaces Max Capacity rather than derating it. + + A candidate with no units has no nameplate for a Rating to replace, so its Rating + derates the capacity it may build like any other generator's. + """ + if not self.units: + return None rating = _optional(self.props, PlexosProperty.RATING) return rating if rating is not None and rating > self.nameplate else None @property - def p_nom(self) -> float: + def existing(self) -> float: + """The capacity the generator already has, before anything is built.""" capacity = self.rating_as_capacity return self.nameplate if capacity is None else capacity + @property + def p_nom(self) -> float: + """What the generator has, or what it may build where it has nothing yet. + + PyPSA optimises ``p_nom_opt`` between ``p_nom_min`` and ``p_nom_max`` and reads + ``p_nom`` only for a generator whose capacity is fixed, so what stands here for a + candidate does not bind its dispatch. It is what every per-unit field on the + component is read against -- ``p_min_pu``, a ramp limit, an availability profile + stated in MW -- and against nothing each of those would come out at zero. + """ + return self.existing or self.buildable + def read_source(generator: dict[str, Any], name: str, lookups: Lookups) -> SourceGenerator: props = lookups.gen_props.get(name, {}) @@ -110,6 +148,7 @@ def read_source(generator: dict[str, Any], name: str, lookups: Lookups) -> Sourc category=generator.get(PlexosObjectCol.CATEGORY) or "", units=_value(props, PlexosProperty.UNITS, DEFAULT_UNITS), props=props, + stated_units=lookups.gen_units.get(name, {}), max_capacity=_rated_capacity(name, props, lookups), ) @@ -127,6 +166,51 @@ def _rated_capacity(name: str, props: dict[str, float], lookups: Lookups) -> flo return lookups.profile_peaks[PlexosProperty.RATING].get(name, static) +@dataclass(frozen=True) +class Expansion: + """What a PLEXOS object may build, and what building it costs. + + Every field is null for an object the model does not allow to be built. PyPSA works out + what a year of new capacity costs from the overnight cost, the discount rate and the + lifetime together, so nothing here assembles that number itself. + """ + + max_units_built: float | None = None + unit_size: float | None = None + p_nom_min: float | None = None + p_nom_max: float | None = None + overnight_cost: float | None = None + wacc: float | None = None + discount_rate: float | None = None + economic_life: float | None = None + fom_cost: float | None = None + technical_life: float | None = None + + @property + def is_candidate(self) -> bool: + return self.max_units_built is not None + + +def derive_expansion(source: SourceGenerator) -> Expansion: + """What one candidate may build; every field is null for a generator that may not.""" + if not source.is_candidate: + return Expansion() + wacc = _optional(source.props, PlexosProperty.WACC) + return Expansion( + max_units_built=source.max_units_built, + unit_size=source.max_capacity, + # A generator already running cannot be un-built, so what it has is its own floor. + p_nom_min=source.existing, + p_nom_max=source.existing + source.buildable, + overnight_cost=_optional(source.props, PlexosProperty.BUILD_COST), + wacc=wacc, + discount_rate=as_rate(wacc, source.stated_units.get(PlexosProperty.WACC)), + economic_life=_optional(source.props, PlexosProperty.ECONOMIC_LIFE), + fom_cost=_optional(source.props, PlexosProperty.FOM_CHARGE), + technical_life=_optional(source.props, PlexosProperty.TECHNICAL_LIFE), + ) + + @dataclass(frozen=True) class GeneratorMapping: """Values derived from one PLEXOS Generator, before events and the output row. @@ -152,6 +236,7 @@ class GeneratorMapping: cost: Cost efficiency: float | None unit_commitment: UnitCommitment | None + expansion: Expansion @property def is_committable(self) -> bool: @@ -195,6 +280,7 @@ def derive_generator(source: SourceGenerator, node: str, lookups: Lookups) -> Ge unit_commitment=_derive_unit_commitment(source, _start_fuel(source, fuel, lookups), lookups) if _commits(fuel, minimum) else None, + expansion=derive_expansion(source), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py index 2628033..6185fd8 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py @@ -21,9 +21,11 @@ from interop.plugins.shared.plexos_pypsa_translations._shared import ( MultiValueRule, ObjectProperties, + ObjectUnits, built_bus_names, collapse_membership_properties, collapse_properties_by_object, + collapse_units_by_object, read_file_backed_properties, relate_child, relate_children, @@ -48,6 +50,7 @@ class Lookups: """Per-class property values and membership resolutions the generator loop reads.""" gen_props: ObjectProperties + gen_units: ObjectUnits fuel_props: ObjectProperties emission_props: ObjectProperties bus_names: set[str] @@ -72,6 +75,7 @@ def build_lookups(state: State) -> Lookups: gen_props=collapse_properties_by_object( properties, PlexosClass.GENERATOR, _GENERATOR_RULES ), + gen_units=collapse_units_by_object(properties, PlexosClass.GENERATOR), fuel_props=collapse_properties_by_object(properties, PlexosClass.FUEL), emission_props=collapse_properties_by_object(properties, PlexosClass.EMISSION), bus_names=built_bus_names(state), diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index f41a3b2..2e59eca 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -9,13 +9,13 @@ from __future__ import annotations import logging -from dataclasses import dataclass +from dataclasses import dataclass, field from typing import Any from interop.core.extensions import ExtensionKind, GeneratorExtension, append_extensions from interop.core.pipeline import State from interop.core.reporting import ScopedRecorder -from interop.plugins.shared.constants import UNIT_MW +from interop.plugins.shared.constants import UNIT_DOLLARS_PER_MW, UNIT_MW, UNIT_YEARS from interop.plugins.shared.plexos_constants import ( PlexosClass, PlexosCollection, @@ -43,6 +43,8 @@ storage_turbine_names, ) from interop.plugins.shared.plexos_pypsa_translations.constants import ( + EXT_TECHNICAL_LIFE_FIELD, + EXT_UNIT_SIZE_FIELD, GENERATOR_EXT_CATEGORY_FIELD, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( @@ -75,7 +77,17 @@ "Max Capacity comes from a data file rather than a value, so the generator " "has no p_nom to size it or to per-unitise its availability against" ) -_RETIRED_NOTE = "Units = 0 marks a retired generator" +_RETIRED_NOTE = "Units = 0 and no Max Units Built marks a retired generator" +_NO_BUILD_COST_NOTE = ( + "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" +) +_UNIT_SIZE_DERIVATION = ( + "PyPSA sizes a candidate by p_nom_max alone, so what one unit of it is travels beside it" +) +_TECHNICAL_LIFE_DERIVATION = ( + "PyPSA's one lifetime holds the capital recovery period, so the technology lifetime " + "travels beside it" +) _CATEGORY_DERIVATION = "a PyPSA generator carries one carrier, so the category travels beside it" _PROFILE_NOT_STAGED_NOTE = ( "the source staged no series for this profile, so p_max_pu keeps the static " @@ -87,6 +99,8 @@ _PROFILE = "profile" _CATEGORY_COLUMN = MappedColumns((GENERATOR_EXT_CATEGORY_FIELD,)) +_UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) +_TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) def map_generators(state: State, recorder: ScopedRecorder) -> None: @@ -97,12 +111,14 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: lookups = build_lookups(state) reporter = ComponentReporter(recorder, PyPSAComponent.GENERATOR) storage_turbines = storage_turbine_names(state) + drops = _Drops() translated = [ one for generator in generators.collect().iter_rows(named=True) if generator[PlexosObjectCol.NAME] not in storage_turbines - if (one := _map_one(generator, lookups, reporter)) is not None + if (one := _map_one(generator, lookups, reporter, drops)) is not None ] + _warn_unpriced_candidates(drops.candidates_without_build_cost) if not translated: return append_destination_rows( @@ -115,10 +131,27 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: GENERATORS_DESTINATION_SCHEMA, ) mappings = [one.mapping for one in translated] - _carry_categories_to_extensions(state, mappings, reporter) + _carry_to_extensions(state, mappings, reporter) _record_availability_time_series(state, mappings, reporter) +@dataclass +class _Drops: + """What one walk left out, so a warning names a few of them rather than each on its own.""" + + candidates_without_build_cost: list[str] = field(default_factory=list) + + +def _warn_unpriced_candidates(names: list[str]) -> None: + if not names: + return + log.warning( + "plexos: %d candidate generator(s) state no Build Cost, so each is left out. Each one: %s", + len(names), + name_a_few(sorted(names)), + ) + + @dataclass(frozen=True) class _TranslatedGenerator: """One generator's derived values and the destination decisions they justify.""" @@ -127,28 +160,57 @@ class _TranslatedGenerator: decisions: GeneratorDecisions -def _carry_categories_to_extensions( +def _carry_to_extensions( state: State, mappings: list[GeneratorMapping], reporter: ComponentReporter ) -> None: - """Put every generator's PLEXOS category in the sidecar, since only one of it and the - fuel could become the carrier. The one that did not is what the report names. + """Put what the network file cannot hold in the sidecar: the PLEXOS category, since only + one of it and the fuel could become the carrier, and the two expansion values PyPSA has + no field for. """ for mapping in mappings: if mapping.carrier != mapping.category: reporter.record(mapping.name, _CATEGORY_COLUMN, _category_decision(mapping)) + _record_expansion_extensions(mapping, reporter) records = [ - GeneratorExtension(name=mapping.name, category=mapping.category) for mapping in mappings + GeneratorExtension( + name=mapping.name, + category=mapping.category, + unit_size_mw=mapping.expansion.unit_size, + technical_life_years=mapping.expansion.technical_life, + ) + for mapping in mappings ] append_extensions(state.destination_extensions, ExtensionKind.GENERATOR, records) +def _record_expansion_extensions(mapping: GeneratorMapping, reporter: ComponentReporter) -> None: + """The two candidate values the network file has no column for, each where it is stated.""" + expansion = mapping.expansion + if expansion.unit_size is not None: + source = _source(mapping.name, PlexosProperty.MAX_CAPACITY, expansion.unit_size, UNIT_MW) + reporter.record( + mapping.name, + _UNIT_SIZE_COLUMN, + Decision.derived(expansion.unit_size, [source], _UNIT_SIZE_DERIVATION), + ) + if expansion.technical_life is not None: + source = _source( + mapping.name, PlexosProperty.TECHNICAL_LIFE, expansion.technical_life, UNIT_YEARS + ) + reporter.record( + mapping.name, + _TECHNICAL_LIFE_COLUMN, + Decision.derived(expansion.technical_life, [source], _TECHNICAL_LIFE_DERIVATION), + ) + + def _category_decision(mapping: GeneratorMapping) -> Decision: source = _source(mapping.name, PlexosObjectCol.CATEGORY, mapping.category) return Decision.derived(mapping.category, [source], _CATEGORY_DERIVATION) def _map_one( - generator: dict[str, Any], lookups: Lookups, reporter: ComponentReporter + generator: dict[str, Any], lookups: Lookups, reporter: ComponentReporter, drops: _Drops ) -> _TranslatedGenerator | None: name = generator[PlexosObjectCol.NAME] node = lookups.gen_to_node.get(name) @@ -161,7 +223,7 @@ def _map_one( ) return None source = read_source(generator, name, lookups) - if source.units == 0.0: + if source.units == 0.0 and not source.is_candidate: reporter.record_skipped(_source(name, PlexosProperty.UNITS, source.units), _RETIRED_NOTE) return None if source.p_nom <= 0.0: @@ -170,6 +232,13 @@ def _map_one( f"generator dropped: p_nom is {source.p_nom} MW, so it can never dispatch", ) return None + if source.is_candidate and PlexosProperty.BUILD_COST not in source.props: + reporter.record_skipped( + _source(name, PlexosProperty.BUILD_COST, None, UNIT_DOLLARS_PER_MW), + _NO_BUILD_COST_NOTE, + ) + drops.candidates_without_build_cost.append(name) + return None mapping = derive_generator(source, node, lookups) if has_infeasible_dispatch_range(mapping): _report_infeasible_dispatch_range(mapping, reporter) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_shared.py index a00f67e..c3d3307 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_shared.py @@ -21,7 +21,11 @@ PlexosProperty, PlexosPropertyCol, ) -from interop.plugins.shared.plexos_pypsa_translations.constants import FULL_AVAILABILITY +from interop.plugins.shared.plexos_pypsa_translations.constants import ( + FULL_AVAILABILITY, + PERCENT, +) +from interop.plugins.shared.plexos_units import is_percent from interop.plugins.shared.pypsa_constants import ( PyPSABusCol, PyPSADestinationTable, @@ -157,6 +161,13 @@ def _read_membership_property_rows( return frame.iter_rows() +def as_rate(stated: float | None, stated_unit: str | None) -> float | None: + """A rate the model writes as a percentage, read as the fraction the destination wants.""" + if stated is None: + return None + return stated / PERCENT if is_percent(stated_unit) else stated + + def collapse_units_by_object(properties: pl.LazyFrame, plexos_class: PlexosClass) -> ObjectUnits: """The unit each object states each of its properties in, first band winning. diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index a8aeda8..c59bb90 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -26,7 +26,6 @@ CARRIER_NOTE, CHARGE_NOTE, EFFICIENCY_NOTE, - EXTENDABLE_NOTE, FULL_DISCHARGE_NOTE, MAX_HOURS_NOTE, NO_RESERVOIR_INFLOW_NOTE, @@ -39,6 +38,7 @@ StorageLookups, StorageUnitMapping, derive_bus, + derive_expansion, derive_max_hours, derive_round_trip_efficiency, derive_state_of_charge_initial, @@ -55,7 +55,6 @@ STORAGE_FULL_DISCHARGE_PU, STORAGE_GENERATE_ONLY_PU, STORAGE_MARGINAL_COST, - STORAGE_P_NOM_EXTENDABLE, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( Decision, @@ -226,6 +225,7 @@ def _derive_turbine( rated.p_nom.value, _VOLUME_NOT_ENERGY_NOTE if unreadable else MAX_HOURS_NOTE, ) + expansion = derive_expansion(PlexosClass.GENERATOR, rated) return StorageUnitMapping( name=rated.name, bus=derive_bus(PlexosClass.GENERATOR, rated.name, rated.node), @@ -241,8 +241,16 @@ def _derive_turbine( ), inflow=_reservoir_inflow(head), cyclic=Decision.default(variant.cyclic, variant.cyclic_note), - p_nom_extendable=Decision.default(STORAGE_P_NOM_EXTENDABLE, EXTENDABLE_NOTE), + p_nom_extendable=expansion.p_nom_extendable, + p_nom_min=expansion.p_nom_min, + p_nom_max=expansion.p_nom_max, + overnight_cost=expansion.overnight_cost, + discount_rate=expansion.discount_rate, + lifetime=expansion.lifetime, + fom_cost=expansion.fom_cost, inflow_storage=_inflow_storage(head), + unit_size=expansion.unit_size, + technical_life=expansion.technical_life, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index 1759e50..30cabfa 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -15,10 +15,13 @@ from interop.core.pipeline import State from interop.plugins.shared.constants import ( + UNIT_DOLLARS_PER_MW, + UNIT_DOLLARS_PER_MW_YEAR, UNIT_DOLLARS_PER_MWH, UNIT_HOURS, UNIT_MW, UNIT_MWH, + UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import ( PlexosClass, @@ -30,6 +33,7 @@ from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, ObjectUnits, + as_rate, collapse_properties_by_object, collapse_units_by_object, read_file_backed_properties, @@ -38,6 +42,9 @@ from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_STATE_OF_CHARGE_INITIAL, DEFAULT_STORAGE_MAX_HOURS, + DEFAULT_UNITS, + DIRECT_DERIVATION, + NOTHING_TO_BUILD, PERCENT, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( @@ -61,7 +68,23 @@ FULL_DISCHARGE_NOTE = "full rated power available for discharge" -EXTENDABLE_NOTE = "v1 translates a dispatch model; capacity is fixed" +EXTENDABLE_NOTE = "the object states no Max Units Built, so its capacity is fixed" + +EXTENDABLE_DERIVATION = "Max Units Built above zero is what makes an object a candidate" + +# A storage path that reads no expansion writes null and says nothing about it. +NOT_A_CANDIDATE = Decision.unreported(None) + +_P_NOM_MIN_DERIVATION = "the rated power the object already has, which a build cannot take away" +_P_NOM_MAX_DERIVATION = "rated power x (Units + Max Units Built)" +_DISCOUNT_RATE_DERIVATION = "WACC, read as a fraction where the model states a percentage" +_UNIT_SIZE_DERIVATION = ( + "PyPSA sizes a candidate by p_nom_max alone, so what one unit of it is travels beside it" +) +_TECHNICAL_LIFE_DERIVATION = ( + "PyPSA's one lifetime holds the capital recovery period, so the technology lifetime " + "travels beside it" +) EFFICIENCY_NOTE = "PLEXOS states no round-trip efficiency; storage is modelled lossless" @@ -110,8 +133,27 @@ class StorageUnitMapping: inflow: Decision = maps_to(PyPSAStorageUnitCol.INFLOW, unit=UNIT_MW) cyclic: Decision = maps_to(PyPSAStorageUnitCol.CYCLIC_STATE_OF_CHARGE) p_nom_extendable: Decision = maps_to(PyPSAStorageUnitCol.P_NOM_EXTENDABLE) + p_nom_min: Decision = maps_to( + PyPSAStorageUnitCol.P_NOM_MIN, unit=UNIT_MW, default=NOT_A_CANDIDATE + ) + p_nom_max: Decision = maps_to( + PyPSAStorageUnitCol.P_NOM_MAX, unit=UNIT_MW, default=NOT_A_CANDIDATE + ) + overnight_cost: Decision = maps_to( + PyPSAStorageUnitCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW, default=NOT_A_CANDIDATE + ) + discount_rate: Decision = maps_to(PyPSAStorageUnitCol.DISCOUNT_RATE, default=NOT_A_CANDIDATE) + lifetime: Decision = maps_to( + PyPSAStorageUnitCol.LIFETIME, unit=UNIT_YEARS, default=NOT_A_CANDIDATE + ) + fom_cost: Decision = maps_to( + PyPSAStorageUnitCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR, default=NOT_A_CANDIDATE + ) # Units is a Battery-only reading; a units-out trace derates against it. units: float | None = None + # What the sidecar carries because the network file has no column for it. + unit_size: Decision | None = None + technical_life: Decision | None = None # The head Storage whose Natural Inflow this unit reads, where that inflow is power. An # inflow profile is keyed by the Storage's name, not by this unit's. inflow_storage: str | None = None @@ -179,6 +221,7 @@ class StagedObject: name: str properties: dict[str, float] + stated_units: dict[str, str | None] node: str | None file_backed: list[str] @@ -196,6 +239,8 @@ class StorageLookups: tail_by_generator: dict[str, str] file_backed_by_battery: dict[str, list[str]] file_backed_by_generator: dict[str, list[str]] + battery_units: ObjectUnits + generator_units: ObjectUnits storage_units: ObjectUnits storages_with_inflow_profile: set[str] @@ -203,6 +248,7 @@ def battery(self, name: str) -> StagedObject: return StagedObject( name=name, properties=self.battery_properties.get(name, {}), + stated_units=self.battery_units.get(name, {}), node=self.node_by_battery.get(name), file_backed=self.file_backed_by_battery.get(name, []), ) @@ -211,6 +257,7 @@ def generator(self, name: str) -> StagedObject: return StagedObject( name=name, properties=self.generator_properties.get(name, {}), + stated_units=self.generator_units.get(name, {}), node=self.node_by_generator.get(name), file_backed=self.file_backed_by_generator.get(name, []), ) @@ -299,6 +346,8 @@ def build_lookups(state: State) -> StorageLookups: ), file_backed_by_battery=read_file_backed_properties(properties, PlexosClass.BATTERY), file_backed_by_generator=read_file_backed_properties(properties, PlexosClass.GENERATOR), + battery_units=collapse_units_by_object(properties, PlexosClass.BATTERY), + generator_units=collapse_units_by_object(properties, PlexosClass.GENERATOR), storage_units=collapse_units_by_object(properties, PlexosClass.STORAGE), storages_with_inflow_profile=_storages_with_inflow_profile(state), ) @@ -337,6 +386,7 @@ class RatedObject: name: str properties: dict[str, float] + stated_units: dict[str, str | None] node: str p_nom: Decision @@ -352,7 +402,7 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp p_nom = rating.derive(staged) if p_nom.value <= 0.0: return _skipped_zero_p_nom(rating, staged.name, p_nom.value) - return RatedObject(staged.name, staged.properties, staged.node, p_nom) + return RatedObject(staged.name, staged.properties, staged.stated_units, staged.node, p_nom) def skip_object( @@ -408,6 +458,127 @@ def derive_state_of_charge_initial(level: Decision | None, usable_mwh: float) -> return Decision.derived(held, level.sources, level.explanation + _CLAMPED_DERIVATION) +@dataclass(frozen=True) +class StorageExpansion: + """What a candidate storage object may build, as the columns and the sidecar read it.""" + + p_nom_extendable: Decision + p_nom_min: Decision + p_nom_max: Decision + overnight_cost: Decision + discount_rate: Decision + lifetime: Decision + fom_cost: Decision + unit_size: Decision | None + technical_life: Decision | None + + +def derive_expansion(plexos_class: PlexosClass, rated: RatedObject) -> StorageExpansion: + """What one storage object may build; a fixed object states nothing about any of it. + + PyPSA works out what a year of new capacity costs from the overnight cost, the discount + rate and the lifetime together, so nothing here assembles that number itself. + """ + props = rated.properties + max_units_built = props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) + if max_units_built <= NOTHING_TO_BUILD: + return _fixed_capacity() + name = rated.name + units = props.get(PlexosProperty.UNITS, DEFAULT_UNITS) + rated_power = rated.p_nom.value + built = SourceValue(plexos_class, name, PlexosProperty.MAX_UNITS_BUILT, max_units_built) + wacc = props.get(PlexosProperty.WACC) + return StorageExpansion( + p_nom_extendable=Decision.derived(True, [built], EXTENDABLE_DERIVATION), + p_nom_min=Decision.derived( + rated_power if units else 0.0, [*rated.p_nom.sources], _P_NOM_MIN_DERIVATION + ), + p_nom_max=Decision.derived( + rated_power * (units + max_units_built), + [*rated.p_nom.sources, built], + _P_NOM_MAX_DERIVATION, + ), + overnight_cost=_from_property( + plexos_class, + name, + props.get(PlexosProperty.BUILD_COST), + PlexosProperty.BUILD_COST, + UNIT_DOLLARS_PER_MW, + DIRECT_DERIVATION, + ), + discount_rate=_discount_rate(plexos_class, name, wacc, rated.stated_units), + lifetime=_from_property( + plexos_class, + name, + props.get(PlexosProperty.ECONOMIC_LIFE), + PlexosProperty.ECONOMIC_LIFE, + UNIT_YEARS, + DIRECT_DERIVATION, + ), + fom_cost=_from_property( + plexos_class, + name, + props.get(PlexosProperty.FOM_CHARGE), + PlexosProperty.FOM_CHARGE, + UNIT_DOLLARS_PER_MW_YEAR, + DIRECT_DERIVATION, + ), + unit_size=Decision.derived(rated_power, [*rated.p_nom.sources], _UNIT_SIZE_DERIVATION), + technical_life=_from_property( + plexos_class, + name, + props.get(PlexosProperty.TECHNICAL_LIFE), + PlexosProperty.TECHNICAL_LIFE, + UNIT_YEARS, + _TECHNICAL_LIFE_DERIVATION, + ), + ) + + +def _fixed_capacity() -> StorageExpansion: + return StorageExpansion( + p_nom_extendable=Decision.default(False, EXTENDABLE_NOTE), # noqa: FBT003 + p_nom_min=NOT_A_CANDIDATE, + p_nom_max=NOT_A_CANDIDATE, + overnight_cost=NOT_A_CANDIDATE, + discount_rate=NOT_A_CANDIDATE, + lifetime=NOT_A_CANDIDATE, + fom_cost=NOT_A_CANDIDATE, + unit_size=None, + technical_life=None, + ) + + +def _discount_rate( + plexos_class: PlexosClass, name: str, wacc: float | None, stated_units: dict[str, str | None] +) -> Decision: + rate = as_rate(wacc, stated_units.get(PlexosProperty.WACC)) + if rate is None: + return NOT_A_CANDIDATE + source = SourceValue(plexos_class, name, PlexosProperty.WACC, wacc) + return Decision.derived(rate, [source], _DISCOUNT_RATE_DERIVATION) + + +def _from_property( + plexos_class: PlexosClass, + name: str, + value: float | None, + plexos_property: str, + unit: str | None, + derivation: str, +) -> Decision: + """One expansion value and the property it came from, unreported where none is stated.""" + if value is None: + return NOT_A_CANDIDATE + source = SourceValue(plexos_class, name, plexos_property, value, unit) + return Decision.derived(value, [source], derivation) + + +def sidecar_value(decision: Decision | None) -> float | None: + """What a sidecar record carries for a value only a candidate states.""" + return None if decision is None or decision.value is None else float(decision.value) + + def derive_bus(plexos_class: PlexosClass, name: str, node: str) -> Decision: source = SourceValue(plexos_class, name, PlexosCollection.NODES, node) return Decision.derived(node, [source], _BUS_DERIVATION) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index 7e3960c..97e5f40 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -11,12 +11,14 @@ from dataclasses import dataclass from dataclasses import field as dataclass_field +from interop.core.extensions import ExtensionKind, StorageExtension, append_extensions from interop.core.pipeline import State from interop.core.reporting import ScopedRecorder from interop.plugins.shared.constants import ( UNIT_MW, UNIT_MWH, UNIT_PERCENT, + UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import ( PlexosClass, @@ -41,13 +43,19 @@ build_lookups, orphan_storage_skips, read_object_names, + sidecar_value, warn_about_skipped, ) from interop.plugins.shared.plexos_pypsa_translations._storage_turbines import ( storage_turbine_names, ) +from interop.plugins.shared.plexos_pypsa_translations.constants import ( + EXT_TECHNICAL_LIFE_FIELD, + EXT_UNIT_SIZE_FIELD, +) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( ComponentReporter, + MappedColumns, SourceValue, destination_row, ) @@ -61,6 +69,9 @@ log = logging.getLogger(__name__) +_UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) +_TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) + def write_storage_units(state: State, mappings: list[StorageUnitMapping]) -> None: append_destination_rows( @@ -213,6 +224,7 @@ def map_storage_units(state: State, recorder: ScopedRecorder) -> None: storage_units = derive_storage_units(state) _record(storage_units, recorder) write_storage_units(state, storage_units.mappings) + _carry_to_extensions(state, storage_units.mappings) record_battery_outages(state, storage_units.mappings) record_reservoir_inflows(state, storage_units.mappings) @@ -221,8 +233,25 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No reporter = ComponentReporter(recorder, PyPSAComponent.STORAGE_UNIT) for mapping in storage_units.mappings: reporter.record_mapping(mapping.name, mapping) + if mapping.unit_size is not None: + reporter.record(mapping.name, _UNIT_SIZE_COLUMN, mapping.unit_size) + if mapping.technical_life is not None: + reporter.record(mapping.name, _TECHNICAL_LIFE_COLUMN, mapping.technical_life) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) warn_about_skipped(skipped) for dropped in storage_units.dropped: reporter.record_dropped(dropped.source, dropped.note) + + +def _carry_to_extensions(state: State, mappings: list[StorageUnitMapping]) -> None: + """Put the two candidate values in the sidecar, which the network file cannot hold.""" + records = [ + StorageExtension( + name=mapping.name, + unit_size_mw=sidecar_value(mapping.unit_size), + technical_life_years=sidecar_value(mapping.technical_life), + ) + for mapping in mappings + ] + append_extensions(state.destination_extensions, ExtensionKind.STORAGE, records) diff --git a/interop/plugins/shared/plexos_pypsa_translations/constants.py b/interop/plugins/shared/plexos_pypsa_translations/constants.py index 054b302..31535fc 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/constants.py +++ b/interop/plugins/shared/plexos_pypsa_translations/constants.py @@ -17,6 +17,12 @@ DEFAULT_UNITS: float = 1.0 """A generator with no ``Units`` property is a single unit.""" +EXT_UNIT_SIZE_FIELD: str = "extensions.unit_size_mw" +"""Names that sidecar key in the audit trail; the network file itself has no such column.""" + +EXT_TECHNICAL_LIFE_FIELD: str = "extensions.technical_life_years" +"""Names that sidecar key in the audit trail; the network file itself has no such column.""" + # --- generators --------------------------------------------------------------- DEFAULT_UP_TIME_BEFORE: float = 0.0 @@ -38,6 +44,9 @@ FULL_AVAILABILITY: float = 1.0 """A dispatchable generator can run at full output; an outage or profile derates this.""" +NOTHING_TO_BUILD: float = 0.0 +"""``Max Units Built`` for an object the model does not allow to be built.""" + MAX_RAMP_LIMIT_PU: float = 1.0 """A ramp of the whole of ``p_nom`` in one snapshot, which is as far as PyPSA reads.""" @@ -88,9 +97,6 @@ HYDRO_CYCLIC: bool = False """Reservoir hydro follows its inflow, so its level need not close the loop.""" -STORAGE_P_NOM_EXTENDABLE: bool = False -"""v1 translates a dispatch model; storage capacity is fixed.""" - # --- load shedding ------------------------------------------------------------ # Naming a shedding generator after its bus keeps it traceable without a lookup table. diff --git a/interop/plugins/shared/plexos_pypsa_translations/decisions.py b/interop/plugins/shared/plexos_pypsa_translations/decisions.py index 26bd1ea..a3fcc25 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/decisions.py @@ -106,18 +106,22 @@ def value_for(self, column: str, value: Any) -> Any: return value.by_column[column] if isinstance(value, PerColumn) else value -def maps_to(*columns: str, unit: str | None = None) -> Any: +def maps_to(*columns: str, unit: str | None = None, default: Decision | None = None) -> Any: """Declare which destination columns a mapping field fills. The return type is ``Any`` so the field keeps its ``Decision`` annotation; ``field`` - itself is what the dataclass machinery reads. + itself is what the dataclass machinery reads. ``default`` is the decision a mapping + that has nothing to say about this column carries, for a dataclass several paths build. """ - return declares(MappedColumns(columns, unit)) + return declares(MappedColumns(columns, unit), default) -def declares(mapped: MappedColumns) -> Any: +def declares(mapped: MappedColumns, default: Decision | None = None) -> Any: """``maps_to`` for columns already named as a constant, so an extra event can reuse them.""" - return field(metadata={_MAPPED_COLUMNS: mapped}) + metadata = {_MAPPED_COLUMNS: mapped} + if default is None: + return field(metadata=metadata) + return field(metadata=metadata, default=default) def mapped_fields(mapping: Any) -> Iterator[tuple[MappedColumns, Decision]]: diff --git a/interop/plugins/shared/plexos_units.py b/interop/plugins/shared/plexos_units.py index d01a41a..2d1d944 100644 --- a/interop/plugins/shared/plexos_units.py +++ b/interop/plugins/shared/plexos_units.py @@ -15,6 +15,8 @@ from interop.plugins.shared.constants import ( UNIT_DOLLARS, UNIT_DOLLARS_PER_GJ, + UNIT_DOLLARS_PER_MW, + UNIT_DOLLARS_PER_MW_YEAR, UNIT_DOLLARS_PER_MWH, UNIT_DOLLARS_PER_TONNE, UNIT_GJ, @@ -28,6 +30,7 @@ UNIT_MW_PER_MINUTE, UNIT_MWH, UNIT_PERCENT, + UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import PlexosCollection, PlexosProperty from interop.plugins.shared.warning_text import name_a_few @@ -53,7 +56,7 @@ _KM_PER_MILE = 1.609344 # One unit written two ways, so the conversion table states each of them once. -_SPELLINGS = {"MMBtu": "MMBTU", "hr": "h"} +_SPELLINGS = {"MMBtu": "MMBTU", "hr": "h", "year": "yr", "years": "yr", "yrs": "yr"} # The unit each property is converted into as it stages, which is the one every mapping # reading it already assumes. A property absent here stages as the model wrote it. @@ -75,6 +78,10 @@ (PlexosCollection.GENERATORS, PlexosProperty.MIN_PUMP_LOAD): UNIT_MW, (PlexosCollection.GENERATORS, PlexosProperty.LOAD_POINT): UNIT_MW, (PlexosCollection.GENERATORS, PlexosProperty.START_COST): UNIT_DOLLARS, + (PlexosCollection.GENERATORS, PlexosProperty.BUILD_COST): UNIT_DOLLARS_PER_MW, + (PlexosCollection.GENERATORS, PlexosProperty.FOM_CHARGE): UNIT_DOLLARS_PER_MW_YEAR, + (PlexosCollection.GENERATORS, PlexosProperty.ECONOMIC_LIFE): UNIT_YEARS, + (PlexosCollection.GENERATORS, PlexosProperty.TECHNICAL_LIFE): UNIT_YEARS, (PlexosCollection.START_FUELS, PlexosProperty.OFFTAKE_AT_START): UNIT_GJ, (PlexosCollection.LINES, PlexosProperty.MAX_FLOW): UNIT_MW, (PlexosCollection.LINES, PlexosProperty.MIN_FLOW): UNIT_MW, @@ -87,6 +94,10 @@ (PlexosCollection.BATTERIES, PlexosProperty.MAX_POWER): UNIT_MW, (PlexosCollection.BATTERIES, PlexosProperty.CAPACITY): UNIT_MWH, (PlexosCollection.BATTERIES, PlexosProperty.DURATION): UNIT_HOURS, + (PlexosCollection.BATTERIES, PlexosProperty.BUILD_COST): UNIT_DOLLARS_PER_MW, + (PlexosCollection.BATTERIES, PlexosProperty.FOM_CHARGE): UNIT_DOLLARS_PER_MW_YEAR, + (PlexosCollection.BATTERIES, PlexosProperty.ECONOMIC_LIFE): UNIT_YEARS, + (PlexosCollection.BATTERIES, PlexosProperty.TECHNICAL_LIFE): UNIT_YEARS, (PlexosCollection.STORAGES, PlexosProperty.MAX_VOLUME): UNIT_MWH, (PlexosCollection.STORAGES, PlexosProperty.INITIAL_VOLUME): UNIT_MWH, (PlexosCollection.STORAGES, PlexosProperty.NATURAL_INFLOW): UNIT_MW, @@ -157,6 +168,17 @@ UNIT_KV: {UNIT_KV: 1.0, "V": 1.0 / _THOUSAND, "MV": _THOUSAND}, UNIT_KM: {UNIT_KM: 1.0, "m": 1.0 / _THOUSAND, "mile": _KM_PER_MILE}, UNIT_DOLLARS: {UNIT_DOLLARS: 1.0, "$000": _THOUSAND}, + UNIT_DOLLARS_PER_MW: { + UNIT_DOLLARS_PER_MW: 1.0, + "$/kW": _THOUSAND, + "$/GW": 1.0 / _THOUSAND, + }, + UNIT_DOLLARS_PER_MW_YEAR: { + UNIT_DOLLARS_PER_MW_YEAR: 1.0, + "$/kW/yr": _THOUSAND, + "$/GW/yr": 1.0 / _THOUSAND, + }, + UNIT_YEARS: {UNIT_YEARS: 1.0}, } # A currency symbol, optionally prefixed by up to two letters as in A$. Cents and named diff --git a/interop/plugins/shared/pypsa_constants.py b/interop/plugins/shared/pypsa_constants.py index 0ef00fd..a96bd83 100644 --- a/interop/plugins/shared/pypsa_constants.py +++ b/interop/plugins/shared/pypsa_constants.py @@ -479,6 +479,14 @@ class ReverseTimeSeriesMetadataCol: PyPSAGeneratorCol.START_UP_COST: pl.Float64, PyPSAGeneratorCol.SHUT_DOWN_COST: pl.Float64, PyPSAGeneratorCol.P_NOM_EXTENDABLE: pl.Boolean, + # Expansion fields, null for a component whose capacity the source fixes. The sink + # omits a null column so PyPSA applies its own default. + PyPSAGeneratorCol.P_NOM_MIN: pl.Float64, + PyPSAGeneratorCol.P_NOM_MAX: pl.Float64, + PyPSAGeneratorCol.OVERNIGHT_COST: pl.Float64, + PyPSAGeneratorCol.DISCOUNT_RATE: pl.Float64, + PyPSAGeneratorCol.LIFETIME: pl.Float64, + PyPSAGeneratorCol.FOM_COST: pl.Float64, } # Unit-commitment columns a non-committable generator leaves unset; the sink omits null @@ -508,6 +516,14 @@ class ReverseTimeSeriesMetadataCol: PyPSAStorageUnitCol.CYCLIC_STATE_OF_CHARGE: pl.Boolean, PyPSAStorageUnitCol.P_NOM_EXTENDABLE: pl.Boolean, PyPSAStorageUnitCol.INFLOW: pl.Float64, + # Expansion fields, null for a component whose capacity the source fixes. The sink + # omits a null column so PyPSA applies its own default. + PyPSAStorageUnitCol.P_NOM_MIN: pl.Float64, + PyPSAStorageUnitCol.P_NOM_MAX: pl.Float64, + PyPSAStorageUnitCol.OVERNIGHT_COST: pl.Float64, + PyPSAStorageUnitCol.DISCOUNT_RATE: pl.Float64, + PyPSAStorageUnitCol.LIFETIME: pl.Float64, + PyPSAStorageUnitCol.FOM_COST: pl.Float64, } LOADS_DESTINATION_SCHEMA: dict[str, pl.DataType | type[pl.DataType]] = { diff --git a/interop/plugins/sinks/emit_pypsa_network.py b/interop/plugins/sinks/emit_pypsa_network.py index 5c18511..baf67f9 100644 --- a/interop/plugins/sinks/emit_pypsa_network.py +++ b/interop/plugins/sinks/emit_pypsa_network.py @@ -284,7 +284,8 @@ def _add_generators(network: pypsa.Network, generators: pl.DataFrame | None) -> PyPSAGeneratorCol.P_NOM_EXTENDABLE, ), # Efficiency and the unit-commitment fields are null for renewables and unset - # thermals; let PyPSA default them. + # thermals, and the expansion fields are null for a generator the source fixes; + # let PyPSA default them. optional=( PyPSAGeneratorCol.EFFICIENCY, PyPSAGeneratorCol.RAMP_LIMIT_UP, @@ -294,6 +295,12 @@ def _add_generators(network: pypsa.Network, generators: pl.DataFrame | None) -> PyPSAGeneratorCol.UP_TIME_BEFORE, PyPSAGeneratorCol.START_UP_COST, PyPSAGeneratorCol.SHUT_DOWN_COST, + PyPSAGeneratorCol.P_NOM_MIN, + PyPSAGeneratorCol.P_NOM_MAX, + PyPSAGeneratorCol.OVERNIGHT_COST, + PyPSAGeneratorCol.DISCOUNT_RATE, + PyPSAGeneratorCol.LIFETIME, + PyPSAGeneratorCol.FOM_COST, ), ) @@ -321,6 +328,16 @@ def _add_storage_units(network: pypsa.Network, storage_units: pl.DataFrame | Non PyPSAStorageUnitCol.P_NOM_EXTENDABLE, PyPSAStorageUnitCol.INFLOW, ), + # The expansion fields are null for a storage unit the source fixes; let PyPSA + # default them. + optional=( + PyPSAStorageUnitCol.P_NOM_MIN, + PyPSAStorageUnitCol.P_NOM_MAX, + PyPSAStorageUnitCol.OVERNIGHT_COST, + PyPSAStorageUnitCol.DISCOUNT_RATE, + PyPSAStorageUnitCol.LIFETIME, + PyPSAStorageUnitCol.FOM_COST, + ), ) diff --git a/tests/features/plexos_to_pypsa/generators.feature b/tests/features/plexos_to_pypsa/generators.feature index bfa0171..7b00b49 100644 --- a/tests/features/plexos_to_pypsa/generators.feature +++ b/tests/features/plexos_to_pypsa/generators.feature @@ -493,3 +493,60 @@ Feature: Translate PLEXOS generators into a PyPSA network Then the PyPSA generator "DualStart" in "outputs/network.nc" has "start_up_cost" equal to 800 # A generator burning no fuel has no heat rate to prefer one by, so the largest start wins. And the PyPSA generator "NeitherStart" in "outputs/network.nc" has "start_up_cost" equal to 45000 + + Scenario: a candidate generator becomes extendable and carries what building it costs + Given a Plexos model + And the model states "Build Cost" in "$/kW" + And the model states "FO&M Charge" in "$/kW/yr" + And the model states "WACC" in "%" + And the model contains generator "REZ_Solar" with "node=Grid_Node, category=Solar, Max Capacity=100, Units=0, Max Units Built=5, Build Cost=1200, WACC=7, Economic Life=25, Technical Life=30, FO&M Charge=15" + And the model is saved as "inputs/candidate.xml" + When I run translate against "inputs/candidate.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "REZ_Solar" is extendable + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "p_nom_min" equal to 0 + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "p_nom_max" equal to 500 + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "overnight_cost" equal to 1200000 + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "discount_rate" equal to 0.07 + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "lifetime" equal to 25 + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "fom_cost" equal to 15000 + # Nothing is built yet, so the capacity it may build is what its per-unit fields read against. + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "p_nom" equal to 500 + # PyPSA works the annuity out itself from the three fields above, so nothing writes one. + And the PyPSA generator "REZ_Solar" in "outputs/network.nc" has "capital_cost" equal to 0 + # The unit size and the technical life have no PyPSA column, so they travel beside it. + And the file "outputs/extensions.json" parses as JSON generator extension record for "REZ_Solar" having "unit_size_mw" set to 100.0 + And the file "outputs/extensions.json" parses as JSON generator extension record for "REZ_Solar" having "technical_life_years" set to 30.0 + And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.Max Units Built` = 5.0 | `pypsa.Generator.REZ_Solar.p_nom_extendable` = True | Max Units Built above zero is what makes a generator a candidate |" + And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.Build Cost` = 1200000.0 $/MW | `pypsa.Generator.REZ_Solar.overnight_cost` = 1200000.0 $/MW | direct |" + And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.WACC` = 7.0 | `pypsa.Generator.REZ_Solar.discount_rate` = 0.07 | WACC, read as a fraction where the model states a percentage |" + + Scenario: a plant that may also expand keeps the capacity it has as its floor + Given a Plexos model + And the model contains generator "CoalUnit" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=2, Max Units Built=1, Build Cost=2000000" + And the model is saved as "inputs/expanding_plant.xml" + When I run translate against "inputs/expanding_plant.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "CoalUnit" is extendable + And the PyPSA generator "CoalUnit" in "outputs/network.nc" has "p_nom" equal to 200 + And the PyPSA generator "CoalUnit" in "outputs/network.nc" has "p_nom_min" equal to 200 + And the PyPSA generator "CoalUnit" in "outputs/network.nc" has "p_nom_max" equal to 300 + + Scenario: a candidate that prices building nothing is left out and named + Given a Plexos model + And the model contains generator "Unpriced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4" + And the model contains generator "Priced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000" + And the model is saved as "inputs/unpriced_candidate.xml" + When I run translate against "inputs/unpriced_candidate.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" has no generator "Unpriced_REZ" + And the PyPSA network "outputs/network.nc" generator "Priced_REZ" is extendable + And the file "decisions.md" contains "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" + And the log contains "1 candidate generator(s) state no Build Cost, so each is left out" + + Scenario: a generator the model cannot build states nothing about expansion + Given a Plexos model + And the model contains generator "FixedPlant" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=1" + And the model is saved as "inputs/fixed_plant.xml" + When I run translate against "inputs/fixed_plant.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "FixedPlant" is not extendable + And the PyPSA generator "FixedPlant" in "outputs/network.nc" has no "overnight_cost" + And the PyPSA generator "FixedPlant" in "outputs/network.nc" has no "discount_rate" + And the file "decisions.md" contains "the generator states no Max Units Built, so its capacity is fixed" diff --git a/tests/features/plexos_to_pypsa/storage_units.feature b/tests/features/plexos_to_pypsa/storage_units.feature index c67d9cd..cb49d32 100644 --- a/tests/features/plexos_to_pypsa/storage_units.feature +++ b/tests/features/plexos_to_pypsa/storage_units.feature @@ -504,3 +504,34 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And the model is saved as "inputs/orphan_inflow.xml" When I run translate against "inputs/orphan_inflow.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the file "decisions.md" contains "| `plexos.Storage.orphan_head.Natural Inflow` = 40.0 MW | | | no turbine draws from this reservoir, so its Natural Inflow is dropped | plexos-to-pypsa | plexos_to_pypsa_map_storage_units |" + + Scenario: a candidate battery becomes an extendable storage unit priced by its build cost + Given a Plexos model + And the model states "Build Cost" in "$/kW" + And the model contains region "East" + And the model contains node "node_c" in region "East" + And the model contains battery "new_bat" on node "node_c" with max_power 50 capacity 200 charge_efficiency 90 initial_soc 50 + And battery "new_bat" has property "Units" 0 + And battery "new_bat" has property "Max Units Built" 6 + And battery "new_bat" has property "Build Cost" 800 + And battery "new_bat" has property "Economic Life" 20 + And battery "new_bat" has property "Technical Life" 15 + And the model is saved as "inputs/candidate_battery.xml" + When I run translate against "inputs/candidate_battery.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" storage unit "new_bat" is extendable + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "p_nom_min" is 0 + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "p_nom_max" is 300 + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "overnight_cost" is 800000 + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "lifetime" is 20 + And the file "outputs/extensions.json" parses as JSON storage extension record for "new_bat" having "unit_size_mw" set to 50.0 + And the file "outputs/extensions.json" parses as JSON storage extension record for "new_bat" having "technical_life_years" set to 15.0 + + Scenario: a battery the model cannot build stays fixed + Given a Plexos model + And the model contains region "East" + And the model contains node "node_f" in region "East" + And the model contains battery "old_bat" on node "node_f" with max_power 50 capacity 200 charge_efficiency 90 initial_soc 50 + And the model is saved as "inputs/fixed_battery.xml" + When I run translate against "inputs/fixed_battery.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" storage unit "old_bat" is not extendable + And the file "decisions.md" contains "the object states no Max Units Built, so its capacity is fixed" From 55cfa81d5e8ca38f972bff1adf4434d9e079f38d Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 17:53:27 +0000 Subject: [PATCH 03/17] Give one expansion rule the size of one unit to build with The generator path and the storage path each wrote the expansion mapping, and the storage copy inferred the size of one unit from the rated power. That inference was wrong for two of the three classes it served. A pumped-storage turbine's rated power is Max Capacity times Units, so a turbine with two 100 MW units and one more to build got a ceiling of 600 MW where the model offers 300; a battery's rated power is its Max Power alone, so the same expression read one value as two different things and let a solver remove capacity the model states as built. The turbine also never reached the rule at all when it stated Units = 0, because its rated power worked out to zero and the object was dropped as unable to dispatch, which is exactly the candidate the documentation says the translator carries. The rated power now arrives as a pair: what the object already has, and what one unit of it is. Each class states both for itself, so a Generator and a turbine read Max Capacity as one unit and multiply by Units, and a Battery reads Max Power. One module, _expansion.py, holds the rule the three share: the extendable flag, the floor, the ceiling, the build cost, the discount rate, the lifetime, the fixed cost, and the two values that travel in the sidecar. The floor is the rated power the object has, and the ceiling adds Max Units Built units of one unit's rated power to it. A candidate with nothing built takes the capacity it may build as its p_nom, on the storage path as it already did on the generator path. A mapping dataclass holds one ExpansionDecisions field and adopts its column declarations, so the reported events and the destination row still come from one walk. A candidate that states a Build Cost but no WACC is now left out with the Build Cost case. PyPSA's cost consistency check raises whenever an overnight_cost is set and the discount rate is missing, whatever the strict setting, and Network.optimize runs that check, so such a network cannot be solved at all. On the PyPSA to Sienna leg, base_power and the active power limits took p_nom_opt from any extendable component. An unsolved network states zero there, so every PLEXOS candidate this branch now writes would have translated at zero capacity. The generator, renewable, storage and hydro mappings now guard that choice with p_nom_opt above zero, as the link mapping already did and as the mapping document already described. Two generator scenarios asserted derivation text that named a generator; the shared rule names an object, and they now assert that. Two scenarios stating a Build Cost and no WACC gained one, since the translator no longer writes a network PyPSA refuses. New scenarios cover a pumped-storage turbine that runs two units and may build a third, and one that has yet to build anything. Addresses r1-01, r1-02, r1-03, r1-07, r1-08, r1-09, r1-10, r1-11, r1-15, r1-16, r1-17, r1-18, r1-19, r1-20, r1-21, r1-22, r1-23, r1-24, r1-25, r1-26, r1-27, r1-29, r1-30 and r1-31. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 18 +- .../translation-from-pypsa-to-sienna.md | 14 +- .../plexos_pypsa_translations/_batteries.py | 42 +-- .../plexos_pypsa_translations/_expansion.py | 241 ++++++++++++++++++ .../_generator_decisions.py | 142 +---------- .../_generator_derivation.py | 95 ++++--- .../plexos_pypsa_translations/_generators.py | 98 +++---- .../_storage_hydro.py | 45 ++-- .../_storage_shared.py | 196 ++------------ .../_storage_units.py | 24 +- .../plexos_pypsa_translations/decisions.py | 23 +- .../pypsa_sienna_translations/_generators.py | 7 +- .../pypsa_sienna_translations/_hydro.py | 7 +- .../pypsa_sienna_translations/_renewables.py | 6 +- .../pypsa_sienna_translations/_storage.py | 7 +- .../plexos_to_pypsa/generators.feature | 13 +- .../plexos_to_pypsa/storage_units.feature | 36 +++ .../pypsa_to_sienna/renewables.feature | 2 +- .../thermal_generators.feature | 2 +- 19 files changed, 500 insertions(+), 518 deletions(-) create mode 100644 interop/plugins/shared/plexos_pypsa_translations/_expansion.py diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index 5b2ebcd..4899e59 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -271,8 +271,8 @@ is `committable` when it is thermal, or when its `p_min_pu` is more than `0`. | `shut_down_cost` | $ | `0.0` | `default` | | `up_time_before` | snapshots | `0` | `default` | | `p_nom_extendable` | | `True` where `Max Units Built` is above zero. If not, `False`. | `derived` / `default` | -| `p_nom_min` | MW | `Max Capacity × Units`, for a candidate only | `derived` | -| `p_nom_max` | MW | `Max Capacity × (Units + Max Units Built)`, for a candidate only | `derived` | +| `p_nom_min` | MW | The capacity the generator already has, for a candidate only | `derived` | +| `p_nom_max` | MW | That capacity plus `Max Capacity × Max Units Built`, for a candidate only | `derived` | | `overnight_cost` | $/MW | `Build Cost`, for a candidate only | `direct` | | `discount_rate` | | `WACC`, for a candidate only | `derived` | | `lifetime` | yr | `Economic Life`, for a candidate only | `direct` | @@ -288,6 +288,7 @@ component: | `Max Capacity` comes from a data file | There is no single `p_nom`. Thus the translator cannot set the size of the generator, and it cannot calculate the availability per unit. | | `p_nom` is 0 | The generator can never dispatch. | | The generator is a candidate and gives no `Build Cost` | Nothing prices building it, so an expansion would take it for free. | +| The generator is a candidate and gives no `WACC` | PyPSA annuitises a build cost with a discount rate, and refuses a network that states one without the other. | The repair rates, the unit commitment solver options and the energy budgets are `dropped`. @@ -710,7 +711,7 @@ PLEXOS property each one comes from: | --- | --- | --- | | `p_nom_extendable` | | `Max Units Built` above zero | | `p_nom_min` | MW | The capacity the object already has, which a build cannot take away | -| `p_nom_max` | MW | That capacity plus `Max Units Built` units of it | +| `p_nom_max` | MW | That capacity plus `Max Units Built` units, each one unit's rated power | | `overnight_cost` | $/MW | `Build Cost` | | `discount_rate` | | `WACC`, read as a fraction where the model states a percentage | | `lifetime` | yr | `Economic Life` | @@ -732,9 +733,14 @@ component whose capacity is fixed, so this does not bind the dispatch. It is wha per-unit field on the component is read against — `p_min_pu`, a ramp limit, an availability profile stated in MW — and against nothing each of those would come out at zero. -A candidate that states no `Build Cost` is left out. Nothing prices building it, so an -expansion would take it for free. The translator records a `COMPONENT_SKIPPED` event naming -the object, and warns once naming a few of them. +The rated power of one unit is the `Max Capacity` of a `Generator` or a pumped-storage +turbine, and the `Max Power` of a `Battery`. An object that states `Units` above one already +holds that many, so a build adds units to what it has rather than multiplying it. + +A candidate that states no `Build Cost`, or no `WACC`, is left out. Without a build cost +nothing prices building it, so an expansion would take it for free; without a discount rate +PyPSA cannot annuitise the build cost, and refuses the network. The translator records a +`COMPONENT_SKIPPED` event naming the object, and warns once naming a few of them. ### Which entry applies when diff --git a/docs/translation_mappings/translation-from-pypsa-to-sienna.md b/docs/translation_mappings/translation-from-pypsa-to-sienna.md index 28688a8..39b9223 100644 --- a/docs/translation_mappings/translation-from-pypsa-to-sienna.md +++ b/docs/translation_mappings/translation-from-pypsa-to-sienna.md @@ -81,7 +81,7 @@ This document targets the SiennaSchemas **Operations** namespace (the dispatchab **Implication for this translation.** PyPSA's capacity-expansion fields (`p_nom_extendable`, `capital_cost`, `p_nom_min`/`p_nom_max`, `build_year`, `lifetime`) belong to **Investments**, which is a separate translation target (PyPSA expansion → SiennaSchemas Investments), not extra fields on an Operations component. It is **out of scope for v1**. For v1 we translate the **solved** fleet: take `p_nom_opt` (post-solve capacities) as fixed Operations capacity and drop the expansion parameters (or keep round-trip crumbs in the `ext` sidecar). A genuine expansion problem is the future Investments path. -Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True`, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`; for an extendable component in a solved network `effective_p_nom = p_nom_opt > 0`. An extendable component in an *unsolved* network has `p_nom = p_nom_opt = 0`, so `effective_p_nom = 0` — such a component has no meaningful capacity to translate and lies outside the v1 scope. +Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True` and that `p_nom_opt` is above zero, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`; for an extendable component in a solved network `effective_p_nom = p_nom_opt > 0`. An extendable component whose `p_nom_opt` is 0 has not been solved, so `effective_p_nom` falls back to `p_nom`. **Why Operations and not Investments.** This is a deliberate scope choice tied to the deliverable, not a claim that Operations is the more natural fit: @@ -263,7 +263,7 @@ The following carrier appears in PyPSA networks alongside Generator carriers but | `reactive_power_limits` | `Union{Nothing,MinMax}` (MVAR) | — | `nothing` | defaulted | PyPSA does not carry Q limits for most generators | | `ramp_limits` | `Union{Nothing,UpDown}` (MW/min) | `n.generators.ramp_limit_up`, `ramp_limit_down` | `effective_p_nom × ramp_limit_up ÷ (dt_h × 60)` | derived | PyPSA ramp is pu/snapshot → MW/min: divide by snapshot duration (hours) and by 60. If `NaN` → `nothing`. See snapshot note in cross-cutting section | | `operation_cost` | `ThermalGenerationCost` | `n.generators.marginal_cost`, `start_up_cost`, `shut_down_cost` | See cost pattern above | derived | `marginal_cost` ($/MWh) → `CostCurve(LinearCurve(marginal_cost))`. See cost notes below | -| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` when `p_nom_extendable` else `p_nom`) | derived | See convention discussion above | +| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | See convention discussion above | | `time_limits` | `Union{Nothing,UpDown}` (hours) | `n.generators.min_up_time`, `min_down_time` | `min_up_time × dt_h` | derived | PyPSA counts snapshots; multiply by snapshot duration in hours. If both `0` → `nothing` | | `must_run` | `Bool` | — | `false` | defaulted | No PyPSA equivalent; could be inferred from `p_min_pu ≈ p_max_pu` for must-run units | | `prime_mover_type` | `PrimeMovers` | (carrier classification) | from carrier mapping table | derived | Set per carrier: e.g. `CC` for CCGT, `ST` for coal/nuclear | @@ -348,7 +348,7 @@ The reverse direction is documented in [Translation from Sienna to PyPSA](./tran | `reactive_power_limits` | `Union{Nothing,MinMax}` (MVAR) | — | `nothing` | defaulted | | | `power_factor` | `Float64` ([0,1]) | — | `1.0` | defaulted | No scalar power factor in PyPSA generator schema | | `operation_cost` | `RenewableGenerationCost` | `n.generators.marginal_cost` | `CostCurve(LinearCurve(marginal_cost))` | derived | Usually `0.0` for wind/solar; non-zero for curtailment-penalised networks | -| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` when `p_nom_extendable` else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | | `dynamic_injector` | `Union{Nothing,DynamicInjection}` | — | `nothing` | defaulted | | **Minimal valid JSON example (`RenewableDispatch`):** @@ -416,7 +416,7 @@ The reverse direction is documented in [Translation from Sienna to PyPSA](./tran | `rating` | `Float64` (pu) | `n.generators.p_max_pu` | `p_max_pu` | derived | Typically `1.0` | | `prime_mover_type` | `PrimeMovers` | (carrier classification) | `PVe` | derived | | | `power_factor` | `Float64` ([0,1]) | — | `1.0` | defaulted | | -| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` when `p_nom_extendable` else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | | `dynamic_injector` | `Union{Nothing,DynamicInjection}` | — | `nothing` | defaulted | | **Minimal valid JSON example (`RenewableNonDispatch`):** @@ -465,7 +465,7 @@ Same mechanism as `RenewableDispatch`: emit a `TimeSeriesAssociation` on `max_ac | `rating` | `Float64` (pu) | `n.storage_units.p_max_pu` | `p_max_pu` | derived | | | `prime_mover_type` | `PrimeMovers` | (carrier classification) | `HY` | derived | | | `active_power_limits` | `MinMax` (MW) | `p_nom`, `p_min_pu`, `p_max_pu` | `(effective_p_nom×p_min_pu, effective_p_nom×p_max_pu)` | derived | | -| `base_power` | `Float64` (MVA) | `n.storage_units.p_nom` | `effective_p_nom` (`p_nom_opt` when `p_nom_extendable` else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `n.storage_units.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | | `operation_cost` | `HydroGenerationCost` | `n.storage_units.marginal_cost` | `CostCurve(LinearCurve(marginal_cost))` | derived | Water value if any; default `0.0` | **Minimal valid JSON example (`HydroDispatch`):** @@ -539,7 +539,7 @@ The translator emits `HydroDispatch` from PyPSA `StorageUnit` rows with carrier | `rating` | `Float64` (pu) | `p_max_pu` | `p_max_pu` | derived | | | `active_power_limits` | `MinMax` (MW) | `p_nom`, `p_min_pu`, `p_max_pu` | `(effective_p_nom×p_min_pu, effective_p_nom×p_max_pu)` | derived | | | `reactive_power_limits` | `Union{Nothing,MinMax}` | — | `nothing` | defaulted | | -| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` when `p_nom_extendable` else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | | `operation_cost` | `HydroGenerationCost` | `marginal_cost` | `CostCurve(LinearCurve(marginal_cost))` | derived | | | `powerhouse_elevation` | `Float64` (m) | — | `0.0` | defaulted | Not in PyPSA; default to sea level | | `ramp_limits` | `Union{Nothing,UpDown}` (MW/min) | `ramp_limit_up`, `ramp_limit_down` | `effective_p_nom × ramp_limit_up ÷ (dt_h × 60)` | derived | | @@ -612,7 +612,7 @@ The translator emits each PHS unit as a single `EnergyReservoirStorage` componen | `efficiency` | `NamedTuple{(:in, :out)}` ([0, 1] each) | `efficiency_store`, `efficiency_dispatch` | `(in=efficiency_store, out=efficiency_dispatch)` | direct | Direct mapping from StorageUnit | | `reactive_power` | `Float64` (MVAR) | — | `0.0` | defaulted | | | `reactive_power_limits` | `Union{Nothing, MinMax}` | — | `nothing` | defaulted | | -| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` when `p_nom_extendable` else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | | `operation_cost` | `Union{StorageCost, MarketBidCost}` | `marginal_cost`, cyclic flag | `StorageCost(charge_variable_cost=0, discharge_variable_cost=marginal_cost, energy_shortage_cost=energy_surplus_cost=1e6 if cyclic else 0)` | derived | Discharge cost takes the PyPSA `marginal_cost`. Symmetric shortage / surplus penalty makes `storage_target` a hard constraint when cyclic | | `conversion_factor` | `Float64` | — | `1.0` | defaulted | No unit conversion between `storage_capacity` and energy variable | | `storage_target` | `Float64` (pu-hours of base_power) | `state_of_charge_initial`, `cyclic_state_of_charge` | `initial_storage_capacity_level * max_hours` if cyclic else `0.0` | derived | End-of-horizon energy target on the same internal scale as the energy variable (despite the schema docstring's "ratio" wording, `StorageDispatchWithReserves` reads it literally in `StateofChargeTargetConstraint`). Translator raises if `cyclic_state_of_charge` is mixed across PHS units, since the formulation attribute is per device-model | diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index a4e0883..1b74554 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -19,6 +19,11 @@ PlexosClass, PlexosProperty, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + NOTHING_BUILT_DERIVATION, + RatedCapacity, + derive_expansion, +) from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( CARRIER_NOTE, @@ -34,7 +39,6 @@ StorageLookups, StorageUnitMapping, derive_bus, - derive_expansion, derive_max_hours, derive_round_trip_efficiency, derive_state_of_charge_initial, @@ -44,6 +48,7 @@ BATTERY_CYCLIC, DEFAULT_INFLOW, DEFAULT_ROUND_TRIP_EFFICIENCY, + DEFAULT_UNITS, DIRECT_DERIVATION, END_EFFECTS_RECYCLE, PERCENT, @@ -101,21 +106,28 @@ def map_battery(name: str, lookups: StorageLookups) -> MappedOrSkipped: return _derive_battery(rated) -def _battery_p_nom(staged: StagedObject) -> Decision: +def _battery_rating(staged: StagedObject) -> RatedCapacity: + """A PLEXOS Battery states the power of one of its units as its Max Power.""" max_power = staged.properties[PlexosProperty.MAX_POWER] - source = SourceValue( + power = SourceValue( PlexosClass.BATTERY, staged.name, PlexosProperty.MAX_POWER, max_power, UNIT_MW ) - return Decision.derived(max_power, [source], DIRECT_DERIVATION) + unit_size = Decision.derived(max_power, [power], DIRECT_DERIVATION) + units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS) + if units: + return RatedCapacity(existing=unit_size, unit_size=unit_size) + counted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units) + nothing_built = Decision.derived(0.0, [power, counted], NOTHING_BUILT_DERIVATION) + return RatedCapacity(existing=nothing_built, unit_size=unit_size) -_BATTERY_POWER = RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER, _battery_p_nom) +_BATTERY_POWER = RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER, _battery_rating) def _derive_battery(rated: RatedObject) -> StorageUnitMapping: capacity = _battery_energy_capacity(rated) max_hours = derive_max_hours(capacity, rated.p_nom.value) - expansion = derive_expansion(PlexosClass.BATTERY, rated) + expansion = derive_expansion(rated.candidate) return StorageUnitMapping( name=rated.name, bus=derive_bus(PlexosClass.BATTERY, rated.name, rated.node), @@ -131,16 +143,8 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: ), inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE), cyclic=_battery_cyclic(rated), - p_nom_extendable=expansion.p_nom_extendable, - p_nom_min=expansion.p_nom_min, - p_nom_max=expansion.p_nom_max, - overnight_cost=expansion.overnight_cost, - discount_rate=expansion.discount_rate, - lifetime=expansion.lifetime, - fom_cost=expansion.fom_cost, + expansion=expansion, units=rated.properties.get(PlexosProperty.UNITS), - unit_size=expansion.unit_size, - technical_life=expansion.technical_life, ) @@ -174,15 +178,15 @@ def _battery_energy_capacity(rated: RatedObject) -> Decision | None: duration = rated.properties.get(PlexosProperty.DURATION) if duration is None: return None - return _capacity_from_duration(rated.name, duration, rated.p_nom.value) + return _capacity_from_duration(rated.name, duration, rated.p_nom) -def _capacity_from_duration(name: str, duration: float, max_power: float) -> Decision: +def _capacity_from_duration(name: str, duration: float, p_nom: Decision) -> Decision: return Decision.derived( - duration * max_power, + duration * p_nom.value, [ SourceValue(PlexosClass.BATTERY, name, PlexosProperty.DURATION, duration, UNIT_HOURS), - SourceValue(PlexosClass.BATTERY, name, PlexosProperty.MAX_POWER, max_power, UNIT_MW), + *p_nom.sources, ], _CAPACITY_FROM_DURATION_DERIVATION, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py new file mode 100644 index 0000000..c120270 --- /dev/null +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -0,0 +1,241 @@ +"""What a PLEXOS object may build, and what building it costs. + +A Generator, a Battery and a pumped-storage turbine all state a build the same way, so each +path hands this module the rated power of one unit beside the rated power the object already +has, and reads the destination decisions back. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from interop.plugins.shared.constants import ( + UNIT_DOLLARS_PER_MW, + UNIT_DOLLARS_PER_MW_YEAR, + UNIT_MW, + UNIT_YEARS, +) +from interop.plugins.shared.plexos_constants import PlexosClass, PlexosProperty +from interop.plugins.shared.plexos_pypsa_translations._shared import as_rate +from interop.plugins.shared.plexos_pypsa_translations.constants import ( + DIRECT_DERIVATION, + EXT_TECHNICAL_LIFE_FIELD, + EXT_UNIT_SIZE_FIELD, + NOTHING_TO_BUILD, +) +from interop.plugins.shared.plexos_pypsa_translations.decisions import ( + ComponentReporter, + Decision, + MappedColumns, + SourceValue, + maps_to, +) +from interop.plugins.shared.pypsa_constants import PyPSAGeneratorCol + +NOT_A_CANDIDATE = Decision.unreported(None) + +NOTHING_BUILT_DERIVATION = "Units is zero, so none of the rated power is built yet" + +P_NOM_CANDIDATE_DERIVATION = ( + "the object has no units yet, so its nominal power is the capacity it may build: " + "one unit's rated power x Max Units Built" +) + +_EXTENDABLE_DERIVATION = "Max Units Built above zero is what makes an object a candidate" +_NOT_EXTENDABLE_NOTE = "the object states no Max Units Built, so its capacity is fixed" +_P_NOM_MIN_DERIVATION = "the rated power the object already has, which a build cannot take away" +_P_NOM_MAX_DERIVATION = "the rated power it has + one unit's rated power x Max Units Built" +_DISCOUNT_RATE_DERIVATION = "WACC, read as a fraction where the model states a percentage" +_UNIT_SIZE_DERIVATION = ( + "PyPSA sizes a candidate by p_nom_max alone, so what one unit of it is travels beside it" +) +_TECHNICAL_LIFE_DERIVATION = ( + "PyPSA's one lifetime holds the capital recovery period, so the technology lifetime " + "travels beside it" +) +_NO_BUILD_COST_NOTE = ( + "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" +) +_NO_WACC_NOTE = ( + "a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost over, " + "so PyPSA refuses the network" +) + +UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) +TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) + + +@dataclass(frozen=True) +class RatedCapacity: + """The rated power an object already has, beside the rated power of one unit of it. + + The two differ wherever a model states more than one unit, and wherever a candidate + states no unit at all. + """ + + existing: Decision + unit_size: Decision + + +@dataclass(frozen=True) +class CandidateSource: + """One staged PLEXOS object as the expansion rule reads it.""" + + plexos_class: PlexosClass + name: str + props: dict[str, float] + stated_units: dict[str, str | None] + rated: RatedCapacity + + @property + def max_units_built(self) -> float: + return self.props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) + + @property + def is_candidate(self) -> bool: + return self.max_units_built > NOTHING_TO_BUILD + + def name_units_built(self) -> SourceValue: + return SourceValue( + self.plexos_class, self.name, PlexosProperty.MAX_UNITS_BUILT, self.max_units_built + ) + + +@dataclass(frozen=True) +class ExpansionDecisions: + p_nom_extendable: Decision = maps_to(PyPSAGeneratorCol.P_NOM_EXTENDABLE) + p_nom_min: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MIN, unit=UNIT_MW) + p_nom_max: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MAX, unit=UNIT_MW) + overnight_cost: Decision = maps_to(PyPSAGeneratorCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW) + discount_rate: Decision = maps_to(PyPSAGeneratorCol.DISCOUNT_RATE) + lifetime: Decision = maps_to(PyPSAGeneratorCol.LIFETIME, unit=UNIT_YEARS) + fom_cost: Decision = maps_to(PyPSAGeneratorCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR) + # What the sidecar carries because the network file has no column for it. + unit_size: Decision = NOT_A_CANDIDATE + technical_life: Decision = NOT_A_CANDIDATE + + @property + def is_candidate(self) -> bool: + return bool(self.p_nom_extendable.value) + + +FIXED_CAPACITY = ExpansionDecisions( + p_nom_extendable=Decision.default(False, _NOT_EXTENDABLE_NOTE), # noqa: FBT003 + p_nom_min=NOT_A_CANDIDATE, + p_nom_max=NOT_A_CANDIDATE, + overnight_cost=NOT_A_CANDIDATE, + discount_rate=NOT_A_CANDIDATE, + lifetime=NOT_A_CANDIDATE, + fom_cost=NOT_A_CANDIDATE, +) + + +def derive_expansion(source: CandidateSource) -> ExpansionDecisions: + if not source.is_candidate: + return FIXED_CAPACITY + rated = source.rated + built = source.name_units_built() + return ExpansionDecisions( + p_nom_extendable=Decision.derived(True, [built], _EXTENDABLE_DERIVATION), + p_nom_min=Decision.derived( + rated.existing.value, rated.existing.sources, _P_NOM_MIN_DERIVATION + ), + p_nom_max=Decision.derived( + rated.existing.value + rated.unit_size.value * source.max_units_built, + gather_sources(rated.existing.sources, rated.unit_size.sources, [built]), + _P_NOM_MAX_DERIVATION, + ), + overnight_cost=_from_property( + source, PlexosProperty.BUILD_COST, UNIT_DOLLARS_PER_MW, DIRECT_DERIVATION + ), + discount_rate=_discount_rate(source), + lifetime=_from_property( + source, PlexosProperty.ECONOMIC_LIFE, UNIT_YEARS, DIRECT_DERIVATION + ), + fom_cost=_from_property( + source, PlexosProperty.FOM_CHARGE, UNIT_DOLLARS_PER_MW_YEAR, DIRECT_DERIVATION + ), + unit_size=Decision.derived( + rated.unit_size.value, rated.unit_size.sources, _UNIT_SIZE_DERIVATION + ), + technical_life=_from_property( + source, PlexosProperty.TECHNICAL_LIFE, UNIT_YEARS, _TECHNICAL_LIFE_DERIVATION + ), + ) + + +def derive_buildable(source: CandidateSource) -> Decision: + """The capacity a candidate may build, which stands as its p_nom while it has none.""" + return Decision.derived( + source.rated.unit_size.value * source.max_units_built, + [*source.rated.unit_size.sources, source.name_units_built()], + P_NOM_CANDIDATE_DERIVATION, + ) + + +def derive_p_nom(source: CandidateSource) -> Decision: + """What the object has, or what it may build where it has nothing yet.""" + if source.rated.existing.value or not source.is_candidate: + return source.rated.existing + return derive_buildable(source) + + +@dataclass(frozen=True) +class UnpricedBuild: + """A property a candidate has to state before PyPSA can price building it.""" + + plexos_property: str + unit: str | None + note: str + + +_PRICES_A_BUILD = ( + UnpricedBuild(PlexosProperty.BUILD_COST, UNIT_DOLLARS_PER_MW, _NO_BUILD_COST_NOTE), + UnpricedBuild(PlexosProperty.WACC, None, _NO_WACC_NOTE), +) + + +def find_unpriced_build(props: dict[str, float]) -> UnpricedBuild | None: + """The first property a candidate leaves out that stops PyPSA pricing its build.""" + if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD: + return None + return next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None) + + +def record_expansion_extensions( + name: str, expansion: ExpansionDecisions, reporter: ComponentReporter +) -> None: + reporter.record(name, UNIT_SIZE_COLUMN, expansion.unit_size) + reporter.record(name, TECHNICAL_LIFE_COLUMN, expansion.technical_life) + + +def read_sidecar_value(decision: Decision) -> float | None: + return None if decision.value is None else float(decision.value) + + +def gather_sources(*groups: tuple[SourceValue, ...] | list[SourceValue]) -> list[SourceValue]: + """Every source of several values in order, naming a source shared by two of them once.""" + gathered: dict[SourceValue, None] = {} + for group in groups: + for source in group: + gathered[source] = None + return list(gathered) + + +def _discount_rate(source: CandidateSource) -> Decision: + wacc = source.props.get(PlexosProperty.WACC) + rate = as_rate(wacc, source.stated_units.get(PlexosProperty.WACC)) + if rate is None: + return NOT_A_CANDIDATE + stated = SourceValue(source.plexos_class, source.name, PlexosProperty.WACC, wacc) + return Decision.derived(rate, [stated], _DISCOUNT_RATE_DERIVATION) + + +def _from_property( + source: CandidateSource, plexos_property: str, unit: str | None, derivation: str +) -> Decision: + value = source.props.get(plexos_property) + if value is None: + return NOT_A_CANDIDATE + stated = SourceValue(source.plexos_class, source.name, plexos_property, value, unit) + return Decision.derived(value, [stated], derivation) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py index 8677a05..35d5618 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py @@ -13,8 +13,6 @@ from interop.plugins.shared.constants import ( UNIT_DOLLARS, UNIT_DOLLARS_PER_GJ, - UNIT_DOLLARS_PER_MW, - UNIT_DOLLARS_PER_MW_YEAR, UNIT_DOLLARS_PER_MWH, UNIT_DOLLARS_PER_TONNE, UNIT_GJ, @@ -25,7 +23,6 @@ UNIT_MW_PER_MINUTE, UNIT_PER_UNIT_PER_HOUR, UNIT_SNAPSHOTS, - UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import ( PlexosClass, @@ -33,6 +30,10 @@ PlexosObjectCol, PlexosProperty, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + ExpansionDecisions, + derive_p_nom, +) from interop.plugins.shared.plexos_pypsa_translations._generator_derivation import ( CarbonTerm, GeneratorMapping, @@ -42,7 +43,6 @@ UnitCommitment, ) from interop.plugins.shared.plexos_pypsa_translations.constants import ( - DIRECT_DERIVATION, FULL_AVAILABILITY, MARGINAL_COST_CARBON_TERM, START_UP_COST_FUEL_TERM, @@ -53,13 +53,12 @@ MappedColumns, SourceValue, declares, + holds, maps_to, ) from interop.plugins.shared.pypsa_constants import PyPSAComponent, PyPSAGeneratorCol _BUS_DERIVATION = "the Nodes membership names the bus" -_P_NOM_DERIVATION = "Max Capacity x Units" -_P_NOM_RATING_DERIVATION = "Rating above Max Capacity x Units, so the Rating is the capacity" _THERMAL_CARRIER_DERIVATION = "the fuel names the carrier" _CATEGORY_CARRIER_DERIVATION = "the category names the carrier" _P_MIN_PU_DERIVATION = "the first available minimum-generation property, converted to per unit" @@ -101,18 +100,6 @@ _SHUT_DOWN_NOTE = "PLEXOS prices only starts, so shutting down is free" _EFFICIENCY_DERIVATION = "p_nom / (Heat Rate Base + Heat Rate Incr x p_nom) x 3.6" _DISCARDED_FUEL_NOTE = "a multi-fuel generator keeps its first fuel; this one is discarded" -_EXTENDABLE_DERIVATION = "Max Units Built above zero is what makes a generator a candidate" -_NOT_EXTENDABLE_NOTE = "the generator states no Max Units Built, so its capacity is fixed" -_NOT_EXTENDABLE = False -_P_NOM_MIN_DERIVATION = ( - "Max Capacity x Units, the capacity the generator already has and a build cannot remove" -) -_P_NOM_MAX_DERIVATION = "Max Capacity x (Units + Max Units Built)" -_P_NOM_CANDIDATE_DERIVATION = ( - "the generator has no units yet, so its nominal power is the capacity it may build: " - "Max Capacity x Max Units Built" -) -_DISCOUNT_RATE_DERIVATION = "WACC, read as a fraction where the model states a percentage" # The value a p_max_pu event carries when the ceiling varies over the horizon. _PROFILE = "profile" @@ -153,13 +140,7 @@ class GeneratorDecisions: up_time_before: Decision = maps_to(PyPSAGeneratorCol.UP_TIME_BEFORE, unit=UNIT_SNAPSHOTS) start_up_cost: Decision = maps_to(PyPSAGeneratorCol.START_UP_COST, unit=UNIT_DOLLARS) shut_down_cost: Decision = maps_to(PyPSAGeneratorCol.SHUT_DOWN_COST, unit=UNIT_DOLLARS) - p_nom_extendable: Decision = maps_to(PyPSAGeneratorCol.P_NOM_EXTENDABLE) - p_nom_min: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MIN, unit=UNIT_MW) - p_nom_max: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MAX, unit=UNIT_MW) - overnight_cost: Decision = maps_to(PyPSAGeneratorCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW) - discount_rate: Decision = maps_to(PyPSAGeneratorCol.DISCOUNT_RATE) - lifetime: Decision = maps_to(PyPSAGeneratorCol.LIFETIME, unit=UNIT_YEARS) - fom_cost: Decision = maps_to(PyPSAGeneratorCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR) + expansion: ExpansionDecisions = holds() def decide_generator(mapping: GeneratorMapping) -> GeneratorDecisions: @@ -188,13 +169,7 @@ def decide_generator(mapping: GeneratorMapping) -> GeneratorDecisions: up_time_before=commitment.up_time_before, start_up_cost=commitment.start_up_cost, shut_down_cost=commitment.shut_down_cost, - p_nom_extendable=_p_nom_extendable(mapping), - p_nom_min=_p_nom_min(mapping), - p_nom_max=_p_nom_max(mapping), - overnight_cost=_overnight_cost(mapping), - discount_rate=_discount_rate(mapping), - lifetime=_economic_life(mapping), - fom_cost=_fom_cost(mapping), + expansion=mapping.expansion, ) @@ -241,108 +216,7 @@ def _carrier(mapping: GeneratorMapping) -> Decision: def _p_nom(mapping: GeneratorMapping) -> Decision: - nameplate = _capacity_sources(mapping) - if not mapping.units and mapping.expansion.is_candidate: - sources = [nameplate[0], _units_built_source(mapping)] - return Decision.derived(mapping.p_nom, sources, _P_NOM_CANDIDATE_DERIVATION) - capacity = mapping.rating_as_capacity - if capacity is None: - return Decision.derived(mapping.p_nom, nameplate, _P_NOM_DERIVATION) - rating = _source(mapping.name, PlexosProperty.RATING, capacity, UNIT_MW) - return Decision.derived(mapping.p_nom, [rating, *nameplate], _P_NOM_RATING_DERIVATION) - - -def _capacity_sources(mapping: GeneratorMapping) -> list[SourceValue]: - """The two properties every capacity bound of a generator is built from.""" - return [ - _source(mapping.name, PlexosProperty.MAX_CAPACITY, mapping.max_capacity, UNIT_MW), - _source(mapping.name, PlexosProperty.UNITS, mapping.units), - ] - - -def _units_built_source(mapping: GeneratorMapping) -> SourceValue: - return _source(mapping.name, PlexosProperty.MAX_UNITS_BUILT, mapping.expansion.max_units_built) - - -def _p_nom_extendable(mapping: GeneratorMapping) -> Decision: - if not mapping.expansion.is_candidate: - return Decision.default(_NOT_EXTENDABLE, _NOT_EXTENDABLE_NOTE) - return Decision.derived(True, [_units_built_source(mapping)], _EXTENDABLE_DERIVATION) - - -def _p_nom_min(mapping: GeneratorMapping) -> Decision: - """A build adds to what the generator has, so what it has is the floor of what it keeps.""" - p_nom_min = mapping.expansion.p_nom_min - if p_nom_min is None: - return Decision.unreported(None) - return Decision.derived(p_nom_min, _capacity_sources(mapping), _P_NOM_MIN_DERIVATION) - - -def _p_nom_max(mapping: GeneratorMapping) -> Decision: - p_nom_max = mapping.expansion.p_nom_max - if p_nom_max is None: - return Decision.unreported(None) - sources = [*_capacity_sources(mapping), _units_built_source(mapping)] - return Decision.derived(p_nom_max, sources, _P_NOM_MAX_DERIVATION) - - -def _overnight_cost(mapping: GeneratorMapping) -> Decision: - cost = mapping.expansion.overnight_cost - return _from_property( - mapping.name, - cost, - cost, - PlexosProperty.BUILD_COST, - UNIT_DOLLARS_PER_MW, - DIRECT_DERIVATION, - ) - - -def _discount_rate(mapping: GeneratorMapping) -> Decision: - expansion = mapping.expansion - return _from_property( - mapping.name, - expansion.discount_rate, - expansion.wacc, - PlexosProperty.WACC, - None, - _DISCOUNT_RATE_DERIVATION, - ) - - -def _economic_life(mapping: GeneratorMapping) -> Decision: - """PyPSA reads one lifetime and spreads the overnight cost across it, which is the - period PLEXOS states as the Economic Life rather than the Technical Life.""" - life = mapping.expansion.economic_life - return _from_property( - mapping.name, life, life, PlexosProperty.ECONOMIC_LIFE, UNIT_YEARS, DIRECT_DERIVATION - ) - - -def _fom_cost(mapping: GeneratorMapping) -> Decision: - cost = mapping.expansion.fom_cost - return _from_property( - mapping.name, - cost, - cost, - PlexosProperty.FOM_CHARGE, - UNIT_DOLLARS_PER_MW_YEAR, - DIRECT_DERIVATION, - ) - - -def _from_property( - name: str, - value: float | None, - stated: float | None, - plexos_property: str, - unit: str | None, - derivation: str, -) -> Decision: - """One expansion value and the property it came from, unreported where it states none.""" - if value is None: - return Decision.unreported(None) - return Decision.derived(value, [_source(name, plexos_property, stated, unit)], derivation) + return derive_p_nom(mapping.candidate) def _p_min_pu(mapping: GeneratorMapping) -> Decision: diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py index 6164e9f..cdbedde 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py @@ -13,23 +13,32 @@ from enum import Enum, auto from typing import Any +from interop.plugins.shared.constants import UNIT_MW from interop.plugins.shared.plexos_constants import ( + PlexosClass, PlexosObjectCol, PlexosProperty, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + CandidateSource, + ExpansionDecisions, + RatedCapacity, + derive_expansion, +) from interop.plugins.shared.plexos_pypsa_translations._generator_lookups import Lookups -from interop.plugins.shared.plexos_pypsa_translations._shared import as_rate from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_P_MIN_PU, DEFAULT_SHUT_DOWN_COST, DEFAULT_UNITS, DEFAULT_UP_TIME_BEFORE, + DIRECT_DERIVATION, FULL_AVAILABILITY, MAX_RAMP_LIMIT_PU, NEGLIGIBLE_P_MIN_PU, NOTHING_TO_BUILD, PERCENT, ) +from interop.plugins.shared.plexos_pypsa_translations.decisions import Decision, SourceValue @dataclass(frozen=True) @@ -78,6 +87,9 @@ def cost_without_fuel(self) -> float: # percentage; the other two are MW. _MIN_STABLE_MEGAWATT_PROPERTIES = (PlexosProperty.MIN_STABLE_LEVEL, PlexosProperty.MIN_PUMP_LOAD) +P_NOM_DERIVATION = "Max Capacity x Units" +P_NOM_RATING_DERIVATION = "Rating above Max Capacity x Units, so the Rating is the capacity" + @dataclass(frozen=True) class SourceGenerator: @@ -92,13 +104,10 @@ class SourceGenerator: @property def max_units_built(self) -> float: - """How many units the model allows to be built, which is 0 for a fixed generator.""" return _value(self.props, PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) @property def is_candidate(self) -> bool: - """Whether the model allows this generator to be built, which is what Max Units - Built decides. A generator that is only a candidate states no Units at all.""" return self.max_units_built > NOTHING_TO_BUILD @property @@ -166,49 +175,34 @@ def _rated_capacity(name: str, props: dict[str, float], lookups: Lookups) -> flo return lookups.profile_peaks[PlexosProperty.RATING].get(name, static) -@dataclass(frozen=True) -class Expansion: - """What a PLEXOS object may build, and what building it costs. - - Every field is null for an object the model does not allow to be built. PyPSA works out - what a year of new capacity costs from the overnight cost, the discount rate and the - lifetime together, so nothing here assembles that number itself. - """ +def read_candidate(source: SourceGenerator) -> CandidateSource: + """One staged Generator as the shared expansion rule reads it.""" + capacity = SourceValue( + PlexosClass.GENERATOR, + source.name, + PlexosProperty.MAX_CAPACITY, + source.max_capacity, + UNIT_MW, + ) + counted = SourceValue(PlexosClass.GENERATOR, source.name, PlexosProperty.UNITS, source.units) + return CandidateSource( + PlexosClass.GENERATOR, + source.name, + source.props, + source.stated_units, + RatedCapacity( + existing=_existing(source, capacity, counted), + unit_size=Decision.derived(source.max_capacity, [capacity], DIRECT_DERIVATION), + ), + ) - max_units_built: float | None = None - unit_size: float | None = None - p_nom_min: float | None = None - p_nom_max: float | None = None - overnight_cost: float | None = None - wacc: float | None = None - discount_rate: float | None = None - economic_life: float | None = None - fom_cost: float | None = None - technical_life: float | None = None - @property - def is_candidate(self) -> bool: - return self.max_units_built is not None - - -def derive_expansion(source: SourceGenerator) -> Expansion: - """What one candidate may build; every field is null for a generator that may not.""" - if not source.is_candidate: - return Expansion() - wacc = _optional(source.props, PlexosProperty.WACC) - return Expansion( - max_units_built=source.max_units_built, - unit_size=source.max_capacity, - # A generator already running cannot be un-built, so what it has is its own floor. - p_nom_min=source.existing, - p_nom_max=source.existing + source.buildable, - overnight_cost=_optional(source.props, PlexosProperty.BUILD_COST), - wacc=wacc, - discount_rate=as_rate(wacc, source.stated_units.get(PlexosProperty.WACC)), - economic_life=_optional(source.props, PlexosProperty.ECONOMIC_LIFE), - fom_cost=_optional(source.props, PlexosProperty.FOM_CHARGE), - technical_life=_optional(source.props, PlexosProperty.TECHNICAL_LIFE), - ) +def _existing(source: SourceGenerator, capacity: SourceValue, counted: SourceValue) -> Decision: + rating = source.rating_as_capacity + if rating is None: + return Decision.derived(source.nameplate, [capacity, counted], P_NOM_DERIVATION) + stated = SourceValue(PlexosClass.GENERATOR, source.name, PlexosProperty.RATING, rating, UNIT_MW) + return Decision.derived(rating, [stated, capacity, counted], P_NOM_RATING_DERIVATION) @dataclass(frozen=True) @@ -227,16 +221,15 @@ class GeneratorMapping: carrier: str fuel: FuelUse | None discarded_fuels: tuple[str, ...] - max_capacity: float units: float p_nom: float - rating_as_capacity: float | None minimum: MinimumGeneration availability: Availability cost: Cost efficiency: float | None unit_commitment: UnitCommitment | None - expansion: Expansion + candidate: CandidateSource + expansion: ExpansionDecisions @property def is_committable(self) -> bool: @@ -256,6 +249,7 @@ def has_infeasible_dispatch_range(mapping: GeneratorMapping) -> bool: def derive_generator(source: SourceGenerator, node: str, lookups: Lookups) -> GeneratorMapping: + candidate = read_candidate(source) fuels = lookups.gen_fuels.get(source.name, []) fuel = _fuel_use(source, fuels[0] if fuels else None, lookups) availability = _availability(source, lookups.availability_profiles.get(source.name)) @@ -269,10 +263,8 @@ def derive_generator(source: SourceGenerator, node: str, lookups: Lookups) -> Ge carrier=_classify(fuel, source), fuel=fuel, discarded_fuels=tuple(fuels[1:]), - max_capacity=source.max_capacity, units=source.units, p_nom=source.p_nom, - rating_as_capacity=source.rating_as_capacity, minimum=minimum, availability=availability, cost=_assemble_cost(source.props, fuel), @@ -280,7 +272,8 @@ def derive_generator(source: SourceGenerator, node: str, lookups: Lookups) -> Ge unit_commitment=_derive_unit_commitment(source, _start_fuel(source, fuel, lookups), lookups) if _commits(fuel, minimum) else None, - expansion=derive_expansion(source), + candidate=candidate, + expansion=derive_expansion(candidate), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index 2e59eca..92bb2d9 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -9,19 +9,24 @@ from __future__ import annotations import logging -from dataclasses import dataclass, field +from dataclasses import dataclass from typing import Any from interop.core.extensions import ExtensionKind, GeneratorExtension, append_extensions from interop.core.pipeline import State from interop.core.reporting import ScopedRecorder -from interop.plugins.shared.constants import UNIT_DOLLARS_PER_MW, UNIT_MW, UNIT_YEARS +from interop.plugins.shared.constants import UNIT_MW from interop.plugins.shared.plexos_constants import ( PlexosClass, PlexosCollection, PlexosObjectCol, PlexosProperty, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + find_unpriced_build, + read_sidecar_value, + record_expansion_extensions, +) from interop.plugins.shared.plexos_pypsa_translations._generator_decisions import ( GeneratorDecisions, decide_generator, @@ -43,8 +48,6 @@ storage_turbine_names, ) from interop.plugins.shared.plexos_pypsa_translations.constants import ( - EXT_TECHNICAL_LIFE_FIELD, - EXT_UNIT_SIZE_FIELD, GENERATOR_EXT_CATEGORY_FIELD, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( @@ -78,16 +81,6 @@ "has no p_nom to size it or to per-unitise its availability against" ) _RETIRED_NOTE = "Units = 0 and no Max Units Built marks a retired generator" -_NO_BUILD_COST_NOTE = ( - "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" -) -_UNIT_SIZE_DERIVATION = ( - "PyPSA sizes a candidate by p_nom_max alone, so what one unit of it is travels beside it" -) -_TECHNICAL_LIFE_DERIVATION = ( - "PyPSA's one lifetime holds the capital recovery period, so the technology lifetime " - "travels beside it" -) _CATEGORY_DERIVATION = "a PyPSA generator carries one carrier, so the category travels beside it" _PROFILE_NOT_STAGED_NOTE = ( "the source staged no series for this profile, so p_max_pu keeps the static " @@ -99,8 +92,6 @@ _PROFILE = "profile" _CATEGORY_COLUMN = MappedColumns((GENERATOR_EXT_CATEGORY_FIELD,)) -_UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) -_TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) def map_generators(state: State, recorder: ScopedRecorder) -> None: @@ -111,14 +102,14 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: lookups = build_lookups(state) reporter = ComponentReporter(recorder, PyPSAComponent.GENERATOR) storage_turbines = storage_turbine_names(state) - drops = _Drops() + unpriced: dict[str, list[str]] = {} translated = [ one for generator in generators.collect().iter_rows(named=True) if generator[PlexosObjectCol.NAME] not in storage_turbines - if (one := _map_one(generator, lookups, reporter, drops)) is not None + if (one := _map_one(generator, lookups, reporter, unpriced)) is not None ] - _warn_unpriced_candidates(drops.candidates_without_build_cost) + _warn_unpriced_candidates(unpriced) if not translated: return append_destination_rows( @@ -135,21 +126,14 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: _record_availability_time_series(state, mappings, reporter) -@dataclass -class _Drops: - """What one walk left out, so a warning names a few of them rather than each on its own.""" - - candidates_without_build_cost: list[str] = field(default_factory=list) - - -def _warn_unpriced_candidates(names: list[str]) -> None: - if not names: - return - log.warning( - "plexos: %d candidate generator(s) state no Build Cost, so each is left out. Each one: %s", - len(names), - name_a_few(sorted(names)), - ) +def _warn_unpriced_candidates(unpriced: dict[str, list[str]]) -> None: + for plexos_property, names in unpriced.items(): + log.warning( + "plexos: %d candidate generator(s) state no %s, so each is left out. Each one: %s", + len(names), + plexos_property, + name_a_few(sorted(names)), + ) @dataclass(frozen=True) @@ -163,54 +147,35 @@ class _TranslatedGenerator: def _carry_to_extensions( state: State, mappings: list[GeneratorMapping], reporter: ComponentReporter ) -> None: - """Put what the network file cannot hold in the sidecar: the PLEXOS category, since only - one of it and the fuel could become the carrier, and the two expansion values PyPSA has - no field for. + """Put what the network file cannot hold in the sidecar, starting with the PLEXOS + category, since only one of it and the fuel could become the carrier. """ for mapping in mappings: if mapping.carrier != mapping.category: reporter.record(mapping.name, _CATEGORY_COLUMN, _category_decision(mapping)) - _record_expansion_extensions(mapping, reporter) + record_expansion_extensions(mapping.name, mapping.expansion, reporter) records = [ GeneratorExtension( name=mapping.name, category=mapping.category, - unit_size_mw=mapping.expansion.unit_size, - technical_life_years=mapping.expansion.technical_life, + unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), + technical_life_years=read_sidecar_value(mapping.expansion.technical_life), ) for mapping in mappings ] append_extensions(state.destination_extensions, ExtensionKind.GENERATOR, records) -def _record_expansion_extensions(mapping: GeneratorMapping, reporter: ComponentReporter) -> None: - """The two candidate values the network file has no column for, each where it is stated.""" - expansion = mapping.expansion - if expansion.unit_size is not None: - source = _source(mapping.name, PlexosProperty.MAX_CAPACITY, expansion.unit_size, UNIT_MW) - reporter.record( - mapping.name, - _UNIT_SIZE_COLUMN, - Decision.derived(expansion.unit_size, [source], _UNIT_SIZE_DERIVATION), - ) - if expansion.technical_life is not None: - source = _source( - mapping.name, PlexosProperty.TECHNICAL_LIFE, expansion.technical_life, UNIT_YEARS - ) - reporter.record( - mapping.name, - _TECHNICAL_LIFE_COLUMN, - Decision.derived(expansion.technical_life, [source], _TECHNICAL_LIFE_DERIVATION), - ) - - def _category_decision(mapping: GeneratorMapping) -> Decision: source = _source(mapping.name, PlexosObjectCol.CATEGORY, mapping.category) return Decision.derived(mapping.category, [source], _CATEGORY_DERIVATION) def _map_one( - generator: dict[str, Any], lookups: Lookups, reporter: ComponentReporter, drops: _Drops + generator: dict[str, Any], + lookups: Lookups, + reporter: ComponentReporter, + unpriced: dict[str, list[str]], ) -> _TranslatedGenerator | None: name = generator[PlexosObjectCol.NAME] node = lookups.gen_to_node.get(name) @@ -232,12 +197,13 @@ def _map_one( f"generator dropped: p_nom is {source.p_nom} MW, so it can never dispatch", ) return None - if source.is_candidate and PlexosProperty.BUILD_COST not in source.props: + unpriced_build = find_unpriced_build(source.props) + if unpriced_build is not None: reporter.record_skipped( - _source(name, PlexosProperty.BUILD_COST, None, UNIT_DOLLARS_PER_MW), - _NO_BUILD_COST_NOTE, + _source(name, unpriced_build.plexos_property, None, unpriced_build.unit), + unpriced_build.note, ) - drops.candidates_without_build_cost.append(name) + unpriced.setdefault(unpriced_build.plexos_property, []).append(name) return None mapping = derive_generator(source, node, lookups) if has_infeasible_dispatch_range(mapping): diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index c59bb90..211fd26 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -22,6 +22,10 @@ PlexosCollection, PlexosProperty, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + RatedCapacity, + derive_expansion, +) from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( CARRIER_NOTE, CHARGE_NOTE, @@ -38,7 +42,6 @@ StorageLookups, StorageUnitMapping, derive_bus, - derive_expansion, derive_max_hours, derive_round_trip_efficiency, derive_state_of_charge_initial, @@ -49,6 +52,7 @@ DEFAULT_INFLOW, DEFAULT_ROUND_TRIP_EFFICIENCY, DEFAULT_UNITS, + DIRECT_DERIVATION, HYDRO_CYCLIC, PUMPED_STORAGE_CYCLIC, STORAGE_FULL_CHARGE_PU, @@ -193,27 +197,24 @@ def _classify_turbine( return None -def _generator_p_nom(staged: StagedObject) -> Decision: +def _turbine_rating(staged: StagedObject) -> RatedCapacity: + """A PLEXOS Generator states the power of one of its units as its Max Capacity.""" max_capacity = staged.properties[PlexosProperty.MAX_CAPACITY] stated_units = staged.properties.get(PlexosProperty.UNITS) units = DEFAULT_UNITS if stated_units is None else stated_units - return Decision.derived( - max_capacity * units, - [ - SourceValue( - PlexosClass.GENERATOR, - staged.name, - PlexosProperty.MAX_CAPACITY, - max_capacity, - UNIT_MW, - ), - SourceValue(PlexosClass.GENERATOR, staged.name, PlexosProperty.UNITS, units), - ], - _P_NOM_FROM_UNITS_DERIVATION, + capacity = SourceValue( + PlexosClass.GENERATOR, staged.name, PlexosProperty.MAX_CAPACITY, max_capacity, UNIT_MW + ) + counted = SourceValue(PlexosClass.GENERATOR, staged.name, PlexosProperty.UNITS, units) + return RatedCapacity( + existing=Decision.derived( + max_capacity * units, [capacity, counted], _P_NOM_FROM_UNITS_DERIVATION + ), + unit_size=Decision.derived(max_capacity, [capacity], DIRECT_DERIVATION), ) -_GENERATOR_POWER = RatedPower(PlexosClass.GENERATOR, PlexosProperty.MAX_CAPACITY, _generator_p_nom) +_GENERATOR_POWER = RatedPower(PlexosClass.GENERATOR, PlexosProperty.MAX_CAPACITY, _turbine_rating) def _derive_turbine( @@ -225,7 +226,7 @@ def _derive_turbine( rated.p_nom.value, _VOLUME_NOT_ENERGY_NOTE if unreadable else MAX_HOURS_NOTE, ) - expansion = derive_expansion(PlexosClass.GENERATOR, rated) + expansion = derive_expansion(rated.candidate) return StorageUnitMapping( name=rated.name, bus=derive_bus(PlexosClass.GENERATOR, rated.name, rated.node), @@ -241,16 +242,8 @@ def _derive_turbine( ), inflow=_reservoir_inflow(head), cyclic=Decision.default(variant.cyclic, variant.cyclic_note), - p_nom_extendable=expansion.p_nom_extendable, - p_nom_min=expansion.p_nom_min, - p_nom_max=expansion.p_nom_max, - overnight_cost=expansion.overnight_cost, - discount_rate=expansion.discount_rate, - lifetime=expansion.lifetime, - fom_cost=expansion.fom_cost, + expansion=expansion, inflow_storage=_inflow_storage(head), - unit_size=expansion.unit_size, - technical_life=expansion.technical_life, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index 30cabfa..d1be9a7 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -15,13 +15,10 @@ from interop.core.pipeline import State from interop.plugins.shared.constants import ( - UNIT_DOLLARS_PER_MW, - UNIT_DOLLARS_PER_MW_YEAR, UNIT_DOLLARS_PER_MWH, UNIT_HOURS, UNIT_MW, UNIT_MWH, - UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import ( PlexosClass, @@ -30,10 +27,16 @@ PlexosProperty, PlexosResolvedTable, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + CandidateSource, + ExpansionDecisions, + RatedCapacity, + derive_p_nom, + find_unpriced_build, +) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, ObjectUnits, - as_rate, collapse_properties_by_object, collapse_units_by_object, read_file_backed_properties, @@ -42,14 +45,12 @@ from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_STATE_OF_CHARGE_INITIAL, DEFAULT_STORAGE_MAX_HOURS, - DEFAULT_UNITS, - DIRECT_DERIVATION, - NOTHING_TO_BUILD, PERCENT, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( Decision, SourceValue, + holds, maps_to, ) from interop.plugins.shared.plexos_units import conversion_factor @@ -68,24 +69,6 @@ FULL_DISCHARGE_NOTE = "full rated power available for discharge" -EXTENDABLE_NOTE = "the object states no Max Units Built, so its capacity is fixed" - -EXTENDABLE_DERIVATION = "Max Units Built above zero is what makes an object a candidate" - -# A storage path that reads no expansion writes null and says nothing about it. -NOT_A_CANDIDATE = Decision.unreported(None) - -_P_NOM_MIN_DERIVATION = "the rated power the object already has, which a build cannot take away" -_P_NOM_MAX_DERIVATION = "rated power x (Units + Max Units Built)" -_DISCOUNT_RATE_DERIVATION = "WACC, read as a fraction where the model states a percentage" -_UNIT_SIZE_DERIVATION = ( - "PyPSA sizes a candidate by p_nom_max alone, so what one unit of it is travels beside it" -) -_TECHNICAL_LIFE_DERIVATION = ( - "PyPSA's one lifetime holds the capital recovery period, so the technology lifetime " - "travels beside it" -) - EFFICIENCY_NOTE = "PLEXOS states no round-trip efficiency; storage is modelled lossless" CHARGE_NOTE = "full rated power available for charging (negative dispatch is charging)" @@ -132,28 +115,9 @@ class StorageUnitMapping: ) inflow: Decision = maps_to(PyPSAStorageUnitCol.INFLOW, unit=UNIT_MW) cyclic: Decision = maps_to(PyPSAStorageUnitCol.CYCLIC_STATE_OF_CHARGE) - p_nom_extendable: Decision = maps_to(PyPSAStorageUnitCol.P_NOM_EXTENDABLE) - p_nom_min: Decision = maps_to( - PyPSAStorageUnitCol.P_NOM_MIN, unit=UNIT_MW, default=NOT_A_CANDIDATE - ) - p_nom_max: Decision = maps_to( - PyPSAStorageUnitCol.P_NOM_MAX, unit=UNIT_MW, default=NOT_A_CANDIDATE - ) - overnight_cost: Decision = maps_to( - PyPSAStorageUnitCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW, default=NOT_A_CANDIDATE - ) - discount_rate: Decision = maps_to(PyPSAStorageUnitCol.DISCOUNT_RATE, default=NOT_A_CANDIDATE) - lifetime: Decision = maps_to( - PyPSAStorageUnitCol.LIFETIME, unit=UNIT_YEARS, default=NOT_A_CANDIDATE - ) - fom_cost: Decision = maps_to( - PyPSAStorageUnitCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR, default=NOT_A_CANDIDATE - ) + expansion: ExpansionDecisions = holds() # Units is a Battery-only reading; a units-out trace derates against it. units: float | None = None - # What the sidecar carries because the network file has no column for it. - unit_size: Decision | None = None - technical_life: Decision | None = None # The head Storage whose Natural Inflow this unit reads, where that inflow is power. An # inflow profile is keyed by the Storage's name, not by this unit's. inflow_storage: str | None = None @@ -377,7 +341,7 @@ class RatedPower: plexos_class: PlexosClass capacity_property: PlexosProperty - derive: Callable[[StagedObject], Decision] + derive: Callable[[StagedObject], RatedCapacity] @dataclass(frozen=True) @@ -389,6 +353,7 @@ class RatedObject: stated_units: dict[str, str | None] node: str p_nom: Decision + candidate: CandidateSource def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | SkippedComponent: @@ -399,10 +364,24 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp return _skipped_file_backed(rating, staged.name) if rating.capacity_property not in staged.properties: return _skipped_without_capacity(rating, staged.name) - p_nom = rating.derive(staged) + unpriced = find_unpriced_build(staged.properties) + if unpriced is not None: + return skip_object( + rating.plexos_class, staged.name, unpriced.plexos_property, unpriced.note + ) + candidate = CandidateSource( + rating.plexos_class, + staged.name, + staged.properties, + staged.stated_units, + rating.derive(staged), + ) + p_nom = derive_p_nom(candidate) if p_nom.value <= 0.0: return _skipped_zero_p_nom(rating, staged.name, p_nom.value) - return RatedObject(staged.name, staged.properties, staged.stated_units, staged.node, p_nom) + return RatedObject( + staged.name, staged.properties, staged.stated_units, staged.node, p_nom, candidate + ) def skip_object( @@ -458,127 +437,6 @@ def derive_state_of_charge_initial(level: Decision | None, usable_mwh: float) -> return Decision.derived(held, level.sources, level.explanation + _CLAMPED_DERIVATION) -@dataclass(frozen=True) -class StorageExpansion: - """What a candidate storage object may build, as the columns and the sidecar read it.""" - - p_nom_extendable: Decision - p_nom_min: Decision - p_nom_max: Decision - overnight_cost: Decision - discount_rate: Decision - lifetime: Decision - fom_cost: Decision - unit_size: Decision | None - technical_life: Decision | None - - -def derive_expansion(plexos_class: PlexosClass, rated: RatedObject) -> StorageExpansion: - """What one storage object may build; a fixed object states nothing about any of it. - - PyPSA works out what a year of new capacity costs from the overnight cost, the discount - rate and the lifetime together, so nothing here assembles that number itself. - """ - props = rated.properties - max_units_built = props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) - if max_units_built <= NOTHING_TO_BUILD: - return _fixed_capacity() - name = rated.name - units = props.get(PlexosProperty.UNITS, DEFAULT_UNITS) - rated_power = rated.p_nom.value - built = SourceValue(plexos_class, name, PlexosProperty.MAX_UNITS_BUILT, max_units_built) - wacc = props.get(PlexosProperty.WACC) - return StorageExpansion( - p_nom_extendable=Decision.derived(True, [built], EXTENDABLE_DERIVATION), - p_nom_min=Decision.derived( - rated_power if units else 0.0, [*rated.p_nom.sources], _P_NOM_MIN_DERIVATION - ), - p_nom_max=Decision.derived( - rated_power * (units + max_units_built), - [*rated.p_nom.sources, built], - _P_NOM_MAX_DERIVATION, - ), - overnight_cost=_from_property( - plexos_class, - name, - props.get(PlexosProperty.BUILD_COST), - PlexosProperty.BUILD_COST, - UNIT_DOLLARS_PER_MW, - DIRECT_DERIVATION, - ), - discount_rate=_discount_rate(plexos_class, name, wacc, rated.stated_units), - lifetime=_from_property( - plexos_class, - name, - props.get(PlexosProperty.ECONOMIC_LIFE), - PlexosProperty.ECONOMIC_LIFE, - UNIT_YEARS, - DIRECT_DERIVATION, - ), - fom_cost=_from_property( - plexos_class, - name, - props.get(PlexosProperty.FOM_CHARGE), - PlexosProperty.FOM_CHARGE, - UNIT_DOLLARS_PER_MW_YEAR, - DIRECT_DERIVATION, - ), - unit_size=Decision.derived(rated_power, [*rated.p_nom.sources], _UNIT_SIZE_DERIVATION), - technical_life=_from_property( - plexos_class, - name, - props.get(PlexosProperty.TECHNICAL_LIFE), - PlexosProperty.TECHNICAL_LIFE, - UNIT_YEARS, - _TECHNICAL_LIFE_DERIVATION, - ), - ) - - -def _fixed_capacity() -> StorageExpansion: - return StorageExpansion( - p_nom_extendable=Decision.default(False, EXTENDABLE_NOTE), # noqa: FBT003 - p_nom_min=NOT_A_CANDIDATE, - p_nom_max=NOT_A_CANDIDATE, - overnight_cost=NOT_A_CANDIDATE, - discount_rate=NOT_A_CANDIDATE, - lifetime=NOT_A_CANDIDATE, - fom_cost=NOT_A_CANDIDATE, - unit_size=None, - technical_life=None, - ) - - -def _discount_rate( - plexos_class: PlexosClass, name: str, wacc: float | None, stated_units: dict[str, str | None] -) -> Decision: - rate = as_rate(wacc, stated_units.get(PlexosProperty.WACC)) - if rate is None: - return NOT_A_CANDIDATE - source = SourceValue(plexos_class, name, PlexosProperty.WACC, wacc) - return Decision.derived(rate, [source], _DISCOUNT_RATE_DERIVATION) - - -def _from_property( - plexos_class: PlexosClass, - name: str, - value: float | None, - plexos_property: str, - unit: str | None, - derivation: str, -) -> Decision: - """One expansion value and the property it came from, unreported where none is stated.""" - if value is None: - return NOT_A_CANDIDATE - source = SourceValue(plexos_class, name, plexos_property, value, unit) - return Decision.derived(value, [source], derivation) - - -def sidecar_value(decision: Decision | None) -> float | None: - """What a sidecar record carries for a value only a candidate states.""" - return None if decision is None or decision.value is None else float(decision.value) - - def derive_bus(plexos_class: PlexosClass, name: str, node: str) -> Decision: source = SourceValue(plexos_class, name, PlexosCollection.NODES, node) return Decision.derived(node, [source], _BUS_DERIVATION) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index 97e5f40..b3012bb 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -18,7 +18,6 @@ UNIT_MW, UNIT_MWH, UNIT_PERCENT, - UNIT_YEARS, ) from interop.plugins.shared.plexos_constants import ( PlexosClass, @@ -28,6 +27,10 @@ map_battery, record_battery_outages, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + read_sidecar_value, + record_expansion_extensions, +) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, ) @@ -43,19 +46,13 @@ build_lookups, orphan_storage_skips, read_object_names, - sidecar_value, warn_about_skipped, ) from interop.plugins.shared.plexos_pypsa_translations._storage_turbines import ( storage_turbine_names, ) -from interop.plugins.shared.plexos_pypsa_translations.constants import ( - EXT_TECHNICAL_LIFE_FIELD, - EXT_UNIT_SIZE_FIELD, -) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( ComponentReporter, - MappedColumns, SourceValue, destination_row, ) @@ -69,9 +66,6 @@ log = logging.getLogger(__name__) -_UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) -_TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) - def write_storage_units(state: State, mappings: list[StorageUnitMapping]) -> None: append_destination_rows( @@ -233,10 +227,7 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No reporter = ComponentReporter(recorder, PyPSAComponent.STORAGE_UNIT) for mapping in storage_units.mappings: reporter.record_mapping(mapping.name, mapping) - if mapping.unit_size is not None: - reporter.record(mapping.name, _UNIT_SIZE_COLUMN, mapping.unit_size) - if mapping.technical_life is not None: - reporter.record(mapping.name, _TECHNICAL_LIFE_COLUMN, mapping.technical_life) + record_expansion_extensions(mapping.name, mapping.expansion, reporter) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) warn_about_skipped(skipped) @@ -245,12 +236,11 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No def _carry_to_extensions(state: State, mappings: list[StorageUnitMapping]) -> None: - """Put the two candidate values in the sidecar, which the network file cannot hold.""" records = [ StorageExtension( name=mapping.name, - unit_size_mw=sidecar_value(mapping.unit_size), - technical_life_years=sidecar_value(mapping.technical_life), + unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), + technical_life_years=read_sidecar_value(mapping.expansion.technical_life), ) for mapping in mappings ] diff --git a/interop/plugins/shared/plexos_pypsa_translations/decisions.py b/interop/plugins/shared/plexos_pypsa_translations/decisions.py index a3fcc25..b92188a 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/decisions.py @@ -23,6 +23,7 @@ from interop.ports.outbound.reporting import SourceField _MAPPED_COLUMNS = "mapped_columns" +_NESTED_MAPPING = "nested_mapping" @dataclass(frozen=True) @@ -106,22 +107,23 @@ def value_for(self, column: str, value: Any) -> Any: return value.by_column[column] if isinstance(value, PerColumn) else value -def maps_to(*columns: str, unit: str | None = None, default: Decision | None = None) -> Any: +def maps_to(*columns: str, unit: str | None = None) -> Any: """Declare which destination columns a mapping field fills. The return type is ``Any`` so the field keeps its ``Decision`` annotation; ``field`` - itself is what the dataclass machinery reads. ``default`` is the decision a mapping - that has nothing to say about this column carries, for a dataclass several paths build. + itself is what the dataclass machinery reads. """ - return declares(MappedColumns(columns, unit), default) + return declares(MappedColumns(columns, unit)) -def declares(mapped: MappedColumns, default: Decision | None = None) -> Any: +def declares(mapped: MappedColumns) -> Any: """``maps_to`` for columns already named as a constant, so an extra event can reuse them.""" - metadata = {_MAPPED_COLUMNS: mapped} - if default is None: - return field(metadata=metadata) - return field(metadata=metadata, default=default) + return field(metadata={_MAPPED_COLUMNS: mapped}) + + +def holds() -> Any: + """Declare a field carrying a mapping of its own, whose declarations this one adopts.""" + return field(metadata={_NESTED_MAPPING: True}) def mapped_fields(mapping: Any) -> Iterator[tuple[MappedColumns, Decision]]: @@ -131,6 +133,9 @@ def mapped_fields(mapping: Any) -> Iterator[tuple[MappedColumns, Decision]]: not decisions in their own right. """ for mapping_field in fields(mapping): + if mapping_field.metadata.get(_NESTED_MAPPING): + yield from mapped_fields(getattr(mapping, mapping_field.name)) + continue mapped = mapping_field.metadata.get(_MAPPED_COLUMNS) if isinstance(mapped, MappedColumns): yield mapped, getattr(mapping, mapping_field.name) diff --git a/interop/plugins/shared/pypsa_sienna_translations/_generators.py b/interop/plugins/shared/pypsa_sienna_translations/_generators.py index 18b5de2..a8c72ff 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_generators.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_generators.py @@ -97,7 +97,9 @@ def fill_generator_defaults(table: pl.DataFrame) -> pl.DataFrame: ], ) return table.with_columns( - pl.when(pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE)) + pl.when( + pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE) & (pl.col(PyPSAGeneratorCol.P_NOM_OPT) > 0) + ) .then(pl.col(PyPSAGeneratorCol.P_NOM_OPT)) .otherwise(pl.col(PyPSAGeneratorCol.P_NOM)) .alias(_EFFECTIVE_P_NOM) @@ -360,7 +362,7 @@ def build_generator_extensions( dest_col=T.BASE_POWER, expr=pl.col(_EFFECTIVE_P_NOM), unit=UNIT_MW, - derivation="p_nom_opt when p_nom_extendable else p_nom", + derivation="p_nom_opt where an extendable component has one, else p_nom", ) GENERATOR_ACTIVE_POWER = _direct( @@ -406,6 +408,7 @@ def build_generator_extensions( attribute=( PyPSAGeneratorCol.P_NOM_OPT if old[PyPSAGeneratorCol.P_NOM_EXTENDABLE] + and old[PyPSAGeneratorCol.P_NOM_OPT] > 0 else PyPSAGeneratorCol.P_NOM ), value=old[_EFFECTIVE_P_NOM], diff --git a/interop/plugins/shared/pypsa_sienna_translations/_hydro.py b/interop/plugins/shared/pypsa_sienna_translations/_hydro.py index 13933f9..2f3bc45 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_hydro.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_hydro.py @@ -107,7 +107,10 @@ def fill_hydro_defaults(table: pl.DataFrame) -> pl.DataFrame: [(PyPSAStorageUnitCol.P_NOM_EXTENDABLE, False)], ) return table.with_columns( - pl.when(pl.col(PyPSAStorageUnitCol.P_NOM_EXTENDABLE)) + pl.when( + pl.col(PyPSAStorageUnitCol.P_NOM_EXTENDABLE) + & (pl.col(PyPSAStorageUnitCol.P_NOM_OPT) > 0) + ) .then(pl.col(PyPSAStorageUnitCol.P_NOM_OPT)) .otherwise(pl.col(PyPSAStorageUnitCol.P_NOM)) .alias(_EFFECTIVE_P_NOM) @@ -272,7 +275,7 @@ def build_hydro_ts_associations( dest_col=H.BASE_POWER, expr=pl.col(_EFFECTIVE_P_NOM), unit=UNIT_MW, - derivation="p_nom_opt when p_nom_extendable else p_nom", + derivation="p_nom_opt where an extendable component has one, else p_nom", ) HYDRO_ACTIVE_POWER = _direct( diff --git a/interop/plugins/shared/pypsa_sienna_translations/_renewables.py b/interop/plugins/shared/pypsa_sienna_translations/_renewables.py index 15db166..7f963a4 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_renewables.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_renewables.py @@ -89,7 +89,9 @@ def fill_renewable_defaults(table: pl.DataFrame) -> pl.DataFrame: [(PyPSAGeneratorCol.P_NOM_EXTENDABLE, False)], ) return table.with_columns( - pl.when(pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE)) + pl.when( + pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE) & (pl.col(PyPSAGeneratorCol.P_NOM_OPT) > 0) + ) .then(pl.col(PyPSAGeneratorCol.P_NOM_OPT)) .otherwise(pl.col(PyPSAGeneratorCol.P_NOM)) .alias(_EFFECTIVE_P_NOM) @@ -263,7 +265,7 @@ def _renewable_translations( dest_col=R.BASE_POWER, expr=pl.col(_EFFECTIVE_P_NOM), unit=UNIT_MW, - derivation="p_nom_opt when p_nom_extendable else p_nom", + derivation="p_nom_opt where an extendable component has one, else p_nom", ), direct( source_col=PyPSAGeneratorCol.P_NOM, diff --git a/interop/plugins/shared/pypsa_sienna_translations/_storage.py b/interop/plugins/shared/pypsa_sienna_translations/_storage.py index 33404ad..a76608d 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_storage.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_storage.py @@ -95,7 +95,10 @@ def fill_storage_defaults(table: pl.DataFrame) -> pl.DataFrame: ], ) return table.with_columns( - pl.when(pl.col(PyPSAStorageUnitCol.P_NOM_EXTENDABLE)) + pl.when( + pl.col(PyPSAStorageUnitCol.P_NOM_EXTENDABLE) + & (pl.col(PyPSAStorageUnitCol.P_NOM_OPT) > 0) + ) .then(pl.col(PyPSAStorageUnitCol.P_NOM_OPT)) .otherwise(pl.col(PyPSAStorageUnitCol.P_NOM)) .alias(_EFFECTIVE_P_NOM) @@ -316,7 +319,7 @@ def _min_max(max_expr: pl.Expr) -> pl.Expr: source_col=PyPSAStorageUnitCol.P_NOM, dest_col=S.BASE_POWER, expr=pl.col(_EFFECTIVE_P_NOM), - derivation="p_nom_opt when p_nom_extendable else p_nom", + derivation="p_nom_opt where an extendable component has one, else p_nom", ) STORAGE_COST = Translation( diff --git a/tests/features/plexos_to_pypsa/generators.feature b/tests/features/plexos_to_pypsa/generators.feature index 7b00b49..3bf3eee 100644 --- a/tests/features/plexos_to_pypsa/generators.feature +++ b/tests/features/plexos_to_pypsa/generators.feature @@ -516,13 +516,13 @@ Feature: Translate PLEXOS generators into a PyPSA network # The unit size and the technical life have no PyPSA column, so they travel beside it. And the file "outputs/extensions.json" parses as JSON generator extension record for "REZ_Solar" having "unit_size_mw" set to 100.0 And the file "outputs/extensions.json" parses as JSON generator extension record for "REZ_Solar" having "technical_life_years" set to 30.0 - And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.Max Units Built` = 5.0 | `pypsa.Generator.REZ_Solar.p_nom_extendable` = True | Max Units Built above zero is what makes a generator a candidate |" + And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.Max Units Built` = 5.0 | `pypsa.Generator.REZ_Solar.p_nom_extendable` = True | Max Units Built above zero is what makes an object a candidate |" And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.Build Cost` = 1200000.0 $/MW | `pypsa.Generator.REZ_Solar.overnight_cost` = 1200000.0 $/MW | direct |" And the file "decisions.md" contains "`plexos.Generator.REZ_Solar.WACC` = 7.0 | `pypsa.Generator.REZ_Solar.discount_rate` = 0.07 | WACC, read as a fraction where the model states a percentage |" Scenario: a plant that may also expand keeps the capacity it has as its floor Given a Plexos model - And the model contains generator "CoalUnit" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=2, Max Units Built=1, Build Cost=2000000" + And the model contains generator "CoalUnit" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=2, Max Units Built=1, Build Cost=2000000, WACC=0.07" And the model is saved as "inputs/expanding_plant.xml" When I run translate against "inputs/expanding_plant.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" generator "CoalUnit" is extendable @@ -533,13 +533,18 @@ Feature: Translate PLEXOS generators into a PyPSA network Scenario: a candidate that prices building nothing is left out and named Given a Plexos model And the model contains generator "Unpriced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4" - And the model contains generator "Priced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000" + And the model contains generator "Undiscounted_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000" + And the model contains generator "Priced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000, WACC=0.07" And the model is saved as "inputs/unpriced_candidate.xml" When I run translate against "inputs/unpriced_candidate.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" has no generator "Unpriced_REZ" + # PyPSA annuitises an overnight cost with the discount rate, and refuses a network stating one without the other. + And the PyPSA network "outputs/network.nc" has no generator "Undiscounted_REZ" And the PyPSA network "outputs/network.nc" generator "Priced_REZ" is extendable And the file "decisions.md" contains "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" + And the file "decisions.md" contains "a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost over" And the log contains "1 candidate generator(s) state no Build Cost, so each is left out" + And the log contains "1 candidate generator(s) state no WACC, so each is left out" Scenario: a generator the model cannot build states nothing about expansion Given a Plexos model @@ -549,4 +554,4 @@ Feature: Translate PLEXOS generators into a PyPSA network Then the PyPSA network "outputs/network.nc" generator "FixedPlant" is not extendable And the PyPSA generator "FixedPlant" in "outputs/network.nc" has no "overnight_cost" And the PyPSA generator "FixedPlant" in "outputs/network.nc" has no "discount_rate" - And the file "decisions.md" contains "the generator states no Max Units Built, so its capacity is fixed" + And the file "decisions.md" contains "the object states no Max Units Built, so its capacity is fixed" diff --git a/tests/features/plexos_to_pypsa/storage_units.feature b/tests/features/plexos_to_pypsa/storage_units.feature index cb49d32..4b8e0e0 100644 --- a/tests/features/plexos_to_pypsa/storage_units.feature +++ b/tests/features/plexos_to_pypsa/storage_units.feature @@ -514,6 +514,7 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And battery "new_bat" has property "Units" 0 And battery "new_bat" has property "Max Units Built" 6 And battery "new_bat" has property "Build Cost" 800 + And battery "new_bat" has property "WACC" 0.07 And battery "new_bat" has property "Economic Life" 20 And battery "new_bat" has property "Technical Life" 15 And the model is saved as "inputs/candidate_battery.xml" @@ -523,6 +524,7 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "p_nom_max" is 300 And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "overnight_cost" is 800000 And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "lifetime" is 20 + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "discount_rate" is 0.07 And the file "outputs/extensions.json" parses as JSON storage extension record for "new_bat" having "unit_size_mw" set to 50.0 And the file "outputs/extensions.json" parses as JSON storage extension record for "new_bat" having "technical_life_years" set to 15.0 @@ -535,3 +537,37 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr When I run translate against "inputs/fixed_battery.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" storage unit "old_bat" is not extendable And the file "decisions.md" contains "the object states no Max Units Built, so its capacity is fixed" + + Scenario: a pumped-storage turbine that may build more keeps the units it runs as its floor + Given a Plexos model + And the model contains region "East" + And the model contains node "node_p" in region "East" + And the model contains pumped storage "phs_grow" on node "node_p" with max_capacity 100 pump_efficiency 64 head "grow_head" tail "grow_tail" max_volume 3000 initial_volume 1500 + And generator "phs_grow" has property "Units" 2 + And generator "phs_grow" has property "Max Units Built" 1 + And generator "phs_grow" has property "Build Cost" 1500000 + And generator "phs_grow" has property "WACC" 0.07 + And the model is saved as "inputs/growing_phs.xml" + When I run translate against "inputs/growing_phs.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" storage unit "phs_grow" is extendable + And the PyPSA network "outputs/network.nc" storage unit "phs_grow" attribute "p_nom" is 200 + And the PyPSA network "outputs/network.nc" storage unit "phs_grow" attribute "p_nom_min" is 200 + # One more unit of 100 MW, not one more of everything the two units come to. + And the PyPSA network "outputs/network.nc" storage unit "phs_grow" attribute "p_nom_max" is 300 + + Scenario: a pumped-storage turbine the model has yet to build becomes a candidate + Given a Plexos model + And the model contains region "East" + And the model contains node "node_n" in region "East" + And the model contains pumped storage "phs_new" on node "node_n" with max_capacity 100 pump_efficiency 64 head "new_head" tail "new_tail" max_volume 3000 initial_volume 1500 + And generator "phs_new" has property "Units" 0 + And generator "phs_new" has property "Max Units Built" 3 + And generator "phs_new" has property "Build Cost" 1500000 + And generator "phs_new" has property "WACC" 0.07 + And the model is saved as "inputs/new_phs.xml" + When I run translate against "inputs/new_phs.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" storage unit "phs_new" is extendable + And the PyPSA network "outputs/network.nc" storage unit "phs_new" attribute "p_nom_min" is 0 + And the PyPSA network "outputs/network.nc" storage unit "phs_new" attribute "p_nom_max" is 300 + # Nothing is built yet, so the capacity it may build is what its per-unit fields read against. + And the PyPSA network "outputs/network.nc" storage unit "phs_new" attribute "p_nom" is 300 diff --git a/tests/features/pypsa_to_sienna/renewables.feature b/tests/features/pypsa_to_sienna/renewables.feature index ce2e8a6..4fdeb0e 100644 --- a/tests/features/pypsa_to_sienna/renewables.feature +++ b/tests/features/pypsa_to_sienna/renewables.feature @@ -27,7 +27,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn And the file "decisions.md" contains "| `pypsa.Generator.solar_1.bus` = bus_1 | `sienna.RenewableDispatch.solar_1.bus_name` = bus_1 | direct | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.solar_1.carrier` = solar | `sienna.RenewableDispatch.solar_1.type` = RenewableDispatch | renewable carrier -> RenewableDispatch | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.solar_1.carrier` = solar | `sienna.RenewableDispatch.solar_1.prime_mover_type` = PVe | carrier -> PrimeMovers via user defined mapping | | pypsa-to-sienna | pypsa_to_sienna_map_components |" - And the file "decisions.md" contains "| `pypsa.Generator.solar_1.p_nom` = 200.0 MW | `sienna.RenewableDispatch.solar_1.base_power` = 200.0 MW | p_nom_opt when p_nom_extendable else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" + And the file "decisions.md" contains "| `pypsa.Generator.solar_1.p_nom` = 200.0 MW | `sienna.RenewableDispatch.solar_1.base_power` = 200.0 MW | p_nom_opt where an extendable component has one, else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.solar_1.p_max_pu` = 1.0 | `sienna.RenewableDispatch.solar_1.rating` = 1.0 | p_max_pu (per-unit nameplate rating; typically 1.0) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" # TRANSLATOR_DEFAULT_APPLIED decisions And the file "decisions.md" contains "| | `sienna.RenewableDispatch.solar_1.id` = 1 | | assigned by 1-based row position in renewable generators DataFrame | pypsa-to-sienna | pypsa_to_sienna_map_components |" diff --git a/tests/features/pypsa_to_sienna/thermal_generators.feature b/tests/features/pypsa_to_sienna/thermal_generators.feature index fddf42f..05af39a 100644 --- a/tests/features/pypsa_to_sienna/thermal_generators.feature +++ b/tests/features/pypsa_to_sienna/thermal_generators.feature @@ -25,7 +25,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn # VALUE_DERIVED decisions And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.name` = ccgt_1 | `sienna.ThermalStandard.ccgt_1.name` = ccgt_1 | direct | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.bus` = bus_1 | `sienna.ThermalStandard.ccgt_1.bus_name` = bus_1 | direct | | pypsa-to-sienna | pypsa_to_sienna_map_components |" - And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.base_power` = 500.0 MW | p_nom_opt when p_nom_extendable else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" + And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.base_power` = 500.0 MW | p_nom_opt where an extendable component has one, else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.active_power` = 100.0 MW | effective_p_nom * p_min_pu (initial dispatch = min operating point) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_max_pu` = 1.0 | `sienna.ThermalStandard.ccgt_1.rating` = 1.0 | p_max_pu (per-unit nameplate rating; typically 1.0) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.active_power_limits` = {'min': 100.0, 'max': 500.0} MW | min=effective_p_nom*p_min_pu, max=effective_p_nom*p_max_pu (static) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" From 767dbc27573ea0e7fdb5add22e975c937ca31f74 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 17:57:21 +0000 Subject: [PATCH 04/17] Track what the review cycle has learned about this repository code-review/ is committed with the code it judges, so a teammate and a fresh clone start with the rules this repository has earned rather than with none. The README says what each file in it is for. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- code-review/README.md | 37 +++++++++++++++++++++++++++++++++++++ code-review/ledger.jsonl | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100644 code-review/README.md diff --git a/code-review/README.md b/code-review/README.md new file mode 100644 index 0000000..5ee15e1 --- /dev/null +++ b/code-review/README.md @@ -0,0 +1,37 @@ +# code-review + +This directory holds what the review-and-fix cycle learned about reviewing this +repository. Git tracks it, so the whole team gets it, and a new clone starts +with it. + +| File | What it holds | +|---|---| +| `always-flag.md` | Rules a reviewer must check for. Each one comes from a comment the reader wrote themselves. | +| `never-flag.md` | Rules a reviewer must not raise a comment against. Each one comes from a comment the reader rejected. | +| `ledger.jsonl` | One line for each comment that has not earned a rule yet. | +| `README.md` | This file. | + +## How a rule arrives + +1. A reviewer raises a comment. The reader accepts it, rejects it or defers it. +2. The verdict becomes a line in `ledger.jsonl`. +3. When one category collects three records across three pull requests, the + cycle drafts a rule and asks the reader to approve the wording. +4. The approved rule goes into `always-flag.md` or `never-flag.md`, and the + records that earned it leave `ledger.jsonl`. + +A rule carries the reasons that earned it on its `because:` line, so it explains +itself after those records are gone. + +## No reviewer reads this directory + +The cycle takes every file under `code-review/` out of the diff it gives a +reviewer, and it drops any comment about a file here. + +A rule quotes the code it forbids. A reviewer that read the quote asked for the +quoted code to be deleted. + +## You own these files + +Edit a rule, delete one, or edit a line of the ledger by hand at any time. The +cycle reads what it finds. diff --git a/code-review/ledger.jsonl b/code-review/ledger.jsonl index 3e037fe..c3e3761 100644 --- a/code-review/ledger.jsonl +++ b/code-review/ledger.jsonl @@ -27,3 +27,35 @@ {"id": "r2-08", "pr": "305", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "docs/translation_mappings/plexos-to-sienna-gap-analysis.md", "line": 349, "severity": "minor", "label": "question", "claim": "The number 392 counts time series associations, not units.", "suggested_fix": "```markdown\nit. On the CAISO 2026 Summer Assessment this covered 4 generators of the 411 the ensemble\nholds.\n```", "rule_created": false, "ts": "2026-08-28T16:08:02Z", "source": "reviewer", "verdict": "accepted", "reason": "", "category": null, "promote_now": false, "suppressed_by": null} {"id": "r2-09", "pr": "305", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_ensemble.py", "line": 39, "severity": "minor", "label": "suggestion", "claim": "choose_ensemble_samples reads the sample column of every staged series two times.", "suggested_fix": "```python\n# interop/plugins/shared/staged_samples.py\ndef ensemble_samples(per_series: list[set[str]]) -> list[str]:\n \"\"\"The replications every sampled series carries, in ascending numeric order.\"\"\"\n return sorted(set.intersection(*per_series), key=int) if per_series else []\n\n\ndef staged_ensemble_samples(state: State) -> list[str]:\n return ensemble_samples(staged_sample_sets(state))\n\n\n# interop/plugins/shared/plexos_pypsa_translations/_ensemble.py\n per_profile = staged_sample_sets(state)\n if not per_profile:\n log.warning(\"plexos: no sampled profile is staged; %s\", _NO_NETWORKS_NOTE)\n return\n shared = ensemble_samples(per_profile)\n```", "rule_created": false, "ts": "2026-08-28T16:08:02Z", "source": "reviewer", "verdict": "accepted", "reason": "", "category": null, "promote_now": false, "suppressed_by": null} {"id": "r2-10", "pr": "305", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "libs/interop-testing/src/interop_testing/steps/sienna_system.py", "line": 512, "severity": "minor", "label": "suggestion", "claim": "Both new steps pass when neither system states the key, because .get answers None on each side.", "suggested_fix": "```python\n@then(parsers.parse('the Sienna systems \"{first}\" and \"{second}\" state the same components'))\ndef assert_same_components(first: str, second: str) -> None:\n \"\"\"An ensemble states one set of components, so two replications must not disagree.\"\"\"\n expected = read_json(first)[\"components\"]\n assert expected, f\"expected {first} to state components\"\n assert read_json(second)[\"components\"] == expected, (\n f\"expected {first} and {second} to state the same components\"\n )\n\n\n@then(\n parsers.parse(\n 'the Sienna systems \"{first}\" and \"{second}\" state the same time series associations'\n )\n)\ndef assert_same_time_series_associations(first: str, second: str) -> None:\n \"\"\"One association table serves every companion, so the rows must be identical.\"\"\"\n key = \"time_series_associations\"\n expected = read_json(first)[key]\n assert expected, f\"expected {first} to state {key}\"\n assert read_json(second)[key] == expected, (\n f\"expected {first} and {second} to state the same {key}\"\n )\n```", "rule_created": false, "ts": "2026-08-28T16:08:02Z", "source": "reviewer", "verdict": "accepted", "reason": "", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 493, "severity": "critical", "label": "issue", "claim": "`derive_expansion` multiplies the rated power by `Units`, but the turbine path already put `Units` into that rated power.", "suggested_fix": "The code now:\n\n```python\n units = props.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n rated_power = rated.p_nom.value\n...\n p_nom_min=Decision.derived(\n rated_power if units else 0.0, [*rated.p_nom.sources], _P_NOM_MIN_DERIVATION\n ),\n p_nom_max=Decision.derived(\n rated_power * (units + max_units_built),\n [*rated.p_nom.sources, built],\n _P_NOM_MAX_DERIVATION,\n ),\n```\n\nThe code to use. Each `RatedPower` states the rated power of one unit, and `RatedObject` carries it:\n\n```python\n existing = rated_power if units else 0.0\n...\n p_nom_min=Decision.derived(\n existing, [*rated.p_nom.sources], _P_NOM_MIN_DERIVATION\n ),\n p_nom_max=Decision.derived(\n existing + rated.unit_size * max_units_built,\n [*rated.p_nom.sources, built],\n _P_NOM_MAX_DERIVATION,\n ),\n unit_size=Decision.derived(\n rated.unit_size, [*rated.p_nom.sources], _UNIT_SIZE_DERIVATION\n ),\n```\n\nChange `_P_NOM_MAX_DERIVATION` to \"the rated power it has + one unit's rated power x Max Units Built\".", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 403, "severity": "major", "label": "issue", "claim": "`rate_object` drops an object with no rated power, so a turbine candidate that states `Units = 0` never reaches `derive_expansion`.", "suggested_fix": "The code now:\n\n```python\n p_nom = rating.derive(staged)\n if p_nom.value <= 0.0:\n return _skipped_zero_p_nom(rating, staged.name, p_nom.value)\n return RatedObject(staged.name, staged.properties, staged.stated_units, staged.node, p_nom)\n```\n\nThe code to use. Give the guard the fallback `SourceGenerator.p_nom` already has, so a candidate reads the capacity it may build:\n\n```python\n p_nom = rating.derive(staged)\n if p_nom.value <= 0.0:\n p_nom = rating.buildable(staged) # unit size x Max Units Built, or the same zero\n if p_nom.value <= 0.0:\n return _skipped_zero_p_nom(rating, staged.name, p_nom.value)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-03", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 476, "severity": "major", "label": "suggestion", "claim": "The change writes the expansion translation two times, one time for a generator and one time for a storage object.", "suggested_fix": "There is a code judo move here. A code judo move is a restructure that deletes complexity; it does not move the complexity to another place. Put one expansion mapping in one module and let both paths hold it:\n\n```python\n# interop/plugins/shared/plexos_pypsa_translations/_expansion.py\n@dataclass(frozen=True)\nclass ExpansionDecisions:\n p_nom_extendable: Decision = maps_to(PyPSAGeneratorCol.P_NOM_EXTENDABLE)\n p_nom_min: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MIN, unit=UNIT_MW)\n p_nom_max: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MAX, unit=UNIT_MW)\n overnight_cost: Decision = maps_to(PyPSAGeneratorCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW)\n discount_rate: Decision = maps_to(PyPSAGeneratorCol.DISCOUNT_RATE)\n lifetime: Decision = maps_to(PyPSAGeneratorCol.LIFETIME, unit=UNIT_YEARS)\n fom_cost: Decision = maps_to(PyPSAGeneratorCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR)\n\ndef derive_expansion(\n plexos_class: PlexosClass, name: str, props: dict[str, float],\n stated_units: dict[str, str | None], existing: float, unit_size: float,\n) -> ExpansionDecisions: ...\n```\n\n`GeneratorDecisions` and `StorageUnitMapping` each hold one `expansion: ExpansionDecisions` field, and `mapped_fields` walks into it. The six column strings are equal on both enums, for example `PyPSAGeneratorCol.P_NOM_MIN` and `PyPSAStorageUnitCol.P_NOM_MIN` are both `\"p_nom_min\"`.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-04", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/decisions.py", "line": 109, "severity": "minor", "label": "issue", "claim": "`maps_to` takes a new optional `default`, but every `StorageUnitMapping` construction gives each expansion field a value.", "suggested_fix": "Delete the parameter and the field defaults:\n\n```python\ndef maps_to(*columns: str, unit: str | None = None) -> Any:\n return declares(MappedColumns(columns, unit))\n\n\ndef declares(mapped: MappedColumns) -> Any:\n return field(metadata={_MAPPED_COLUMNS: mapped})\n```\n\nIn `_storage_shared.py`, drop `default=NOT_A_CANDIDATE` from the six field declarations. Keep `NOT_A_CANDIDATE`, because `_fixed_capacity` and `_from_property` return it.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-05", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 273, "severity": "minor", "label": "issue", "claim": "`_p_nom_min` states the derivation `Max Capacity x Units`, but the value is the `Rating` where a `Rating` is above the nameplate.", "suggested_fix": "The code now:\n\n```python\ndef _p_nom_min(mapping: GeneratorMapping) -> Decision:\n \"\"\"A build adds to what the generator has, so what it has is the floor of what it keeps.\"\"\"\n p_nom_min = mapping.expansion.p_nom_min\n if p_nom_min is None:\n return Decision.unreported(None)\n return Decision.derived(p_nom_min, _capacity_sources(mapping), _P_NOM_MIN_DERIVATION)\n```\n\nThe code to use. Build the existing capacity one time, and let all three bounds read its sources, as the storage path reads `rated.p_nom.sources`:\n\n```python\ndef _existing_capacity(mapping: GeneratorMapping) -> Decision:\n \"\"\"The capacity the generator already has, with the properties that state it.\"\"\"\n capacity = mapping.rating_as_capacity\n if capacity is None:\n return Decision.derived(mapping.existing, _capacity_sources(mapping), _P_NOM_DERIVATION)\n rating = _source(mapping.name, PlexosProperty.RATING, capacity, UNIT_MW)\n return Decision.derived(mapping.existing, [rating], _P_NOM_RATING_DERIVATION)\n\n\ndef _p_nom_min(mapping: GeneratorMapping) -> Decision:\n if mapping.expansion.p_nom_min is None:\n return Decision.unreported(None)\n return _existing_capacity(mapping)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-06", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 138, "severity": "minor", "label": "nitpick", "claim": "`_Drops` holds one list and travels through `_map_one` as a mutable argument.", "suggested_fix": "The code now:\n\n```python\n@dataclass\nclass _Drops:\n \"\"\"What one walk left out, so a warning names a few of them rather than each on its own.\"\"\"\n\n candidates_without_build_cost: list[str] = field(default_factory=list)\n```\n\nThe code to use. Pass the list itself and delete the type:\n\n```python\n unpriced_candidates: list[str] = []\n translated = [\n one\n for generator in generators.collect().iter_rows(named=True)\n if generator[PlexosObjectCol.NAME] not in storage_turbines\n if (one := _map_one(generator, lookups, reporter, unpriced_candidates)) is not None\n ]\n _warn_unpriced_candidates(unpriced_candidates)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-07", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 235, "severity": "major", "label": "issue", "claim": "The translator writes `overnight_cost` for a candidate that states no `WACC`, and PyPSA then rejects the network.", "suggested_fix": "```python\n_NO_WACC_NOTE = (\n \"a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost with\"\n)\n\n if source.is_candidate and PlexosProperty.WACC not in source.props:\n reporter.record_skipped(_source(name, PlexosProperty.WACC, None), _NO_WACC_NOTE)\n drops.candidates_without_discount_rate.append(name)\n return None\n```\nApply the same rule on the storage path, or leave `overnight_cost` out where the model states no `WACC`.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-08", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 496, "severity": "major", "label": "issue", "claim": "`p_nom_max` multiplies `rated_power` by `units`, but `rated_power` already holds `Max Capacity x Units` for a turbine.", "suggested_fix": "```python\n existing = rated_power if units else 0.0\n unit_power = rated_power / units if units else rated_power\n ...\n p_nom_min=Decision.derived(existing, [*rated.p_nom.sources], _P_NOM_MIN_DERIVATION),\n p_nom_max=Decision.derived(\n existing + unit_power * max_units_built,\n [*rated.p_nom.sources, built],\n _P_NOM_MAX_DERIVATION,\n ),\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-09", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 501, "severity": "major", "label": "issue", "claim": "The storage path keeps a candidate that states no `Build Cost`, so an expansion builds that storage for free.", "suggested_fix": "```python\n# in rate_object, which both the Battery and the turbine path call\n if (\n staged.properties.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) > NOTHING_TO_BUILD\n and PlexosProperty.BUILD_COST not in staged.properties\n ):\n return skip_object(\n rating.plexos_class, staged.name, PlexosProperty.BUILD_COST, _NO_BUILD_COST_NOTE\n )\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-10", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 267, "severity": "major", "label": "issue", "claim": "The `plexos-to-sienna` chain reads `p_nom_opt` for an extendable generator, and an unsolved network states zero.", "suggested_fix": "```python\n# interop/plugins/shared/pypsa_sienna_translations/_generators.py, and the same in\n# _renewables.py, _storage.py and _hydro.py\n return table.with_columns(\n pl.when(\n pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE) & (pl.col(PyPSAGeneratorCol.P_NOM_OPT) > 0)\n )\n .then(pl.col(PyPSAGeneratorCol.P_NOM_OPT))\n .otherwise(pl.col(PyPSAGeneratorCol.P_NOM))\n .alias(_EFFECTIVE_P_NOM)\n )\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-11", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 476, "severity": "major", "label": "issue", "claim": "`rate_object` drops a turbine that states `Units = 0`, so a pumped-storage candidate never reaches `derive_expansion`.", "suggested_fix": "```python\n# in rate_object, before the zero test, mirroring SourceGenerator.p_nom\n p_nom = rating.derive(staged)\n if p_nom.value <= 0.0:\n buildable = rating.buildable(staged) # Max Capacity x Max Units Built\n if buildable <= 0.0:\n return _skipped_zero_p_nom(rating, staged.name, p_nom.value)\n p_nom = rating.derive_from_units_built(staged)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-12", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 120, "severity": "minor", "label": "question", "claim": "A candidate with no `Units` divides its static `Rating` by the whole buildable capacity, which makes `p_max_pu` small.", "suggested_fix": "```python\n# _static_rating_derate, for a candidate that has no units yet\n rating = _optional(source.props, PlexosProperty.RATING)\n if rating is not None and not source.units and source.is_candidate:\n return rating / source.max_capacity\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-13", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 100, "severity": "nit", "label": "nitpick", "claim": "The docstring says a candidate states no `Units`, but a generator with no `Units` property takes `DEFAULT_UNITS`, which is one.", "suggested_fix": "```python\n \"\"\"Whether the model allows this generator to be built, which is what Max Units\n Built decides. A generator that is only a candidate states Units = 0.\"\"\"\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-14", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_units.py", "line": 249, "severity": "minor", "label": "suggestion", "claim": "`_carry_to_extensions` writes a sidecar record for every storage unit, including one that states no candidate value.", "suggested_fix": "```python\n records = [\n StorageExtension(\n name=mapping.name,\n unit_size_mw=sidecar_value(mapping.unit_size),\n technical_life_years=sidecar_value(mapping.technical_life),\n )\n for mapping in mappings\n if mapping.unit_size is not None or mapping.technical_life is not None\n ]\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-15", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 95, "severity": "minor", "label": "suggestion", "claim": "The docstrings on `max_units_built` and `is_candidate` say again what the two names say.", "suggested_fix": "Delete both docstrings. The neighbouring `nameplate` property carries none.\n\nThe code now:\n\n```python\n @property\n def max_units_built(self) -> float:\n \"\"\"How many units the model allows to be built, which is 0 for a fixed generator.\"\"\"\n return _value(self.props, PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD)\n\n @property\n def is_candidate(self) -> bool:\n \"\"\"Whether the model allows this generator to be built, which is what Max Units\n Built decides. A generator that is only a candidate states no Units at all.\"\"\"\n return self.max_units_built > NOTHING_TO_BUILD\n```\n\nThe code to use:\n\n```python\n @property\n def max_units_built(self) -> float:\n return _value(self.props, PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD)\n\n @property\n def is_candidate(self) -> bool:\n return self.max_units_built > NOTHING_TO_BUILD\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-16", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 173, "severity": "minor", "label": "suggestion", "claim": "The second paragraph of the `Expansion` docstring records what the class does not do.", "suggested_fix": "Delete the second paragraph and keep the summary line.\n\nThe code now:\n\n```python\n@dataclass(frozen=True)\nclass Expansion:\n \"\"\"What a PLEXOS object may build, and what building it costs.\n\n Every field is null for an object the model does not allow to be built. PyPSA works out\n what a year of new capacity costs from the overnight cost, the discount rate and the\n lifetime together, so nothing here assembles that number itself.\n \"\"\"\n```\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass Expansion:\n \"\"\"What a PLEXOS object may build, and what building it costs.\"\"\"\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-17", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 195, "severity": "nit", "label": "suggestion", "claim": "The `derive_expansion` docstring repeats the function name and the guard below it.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\ndef derive_expansion(source: SourceGenerator) -> Expansion:\n \"\"\"What one candidate may build; every field is null for a generator that may not.\"\"\"\n if not source.is_candidate:\n return Expansion()\n```\n\nThe code to use:\n\n```python\ndef derive_expansion(source: SourceGenerator) -> Expansion:\n if not source.is_candidate:\n return Expansion()\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-18", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 202, "severity": "nit", "label": "suggestion", "claim": "This comment repeats the derivation label the audit trail already carries.", "suggested_fix": "Delete the comment.\n\nThe code now:\n\n```python\n unit_size=source.max_capacity,\n # A generator already running cannot be un-built, so what it has is its own floor.\n p_nom_min=source.existing,\n p_nom_max=source.existing + source.buildable,\n```\n\nThe code to use:\n\n```python\n unit_size=source.max_capacity,\n p_nom_min=source.existing,\n p_nom_max=source.existing + source.buildable,\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-19", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 256, "severity": "nit", "label": "suggestion", "claim": "The `_capacity_sources` docstring says again what the function name says.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\ndef _capacity_sources(mapping: GeneratorMapping) -> list[SourceValue]:\n \"\"\"The two properties every capacity bound of a generator is built from.\"\"\"\n return [\n _source(mapping.name, PlexosProperty.MAX_CAPACITY, mapping.max_capacity, UNIT_MW),\n _source(mapping.name, PlexosProperty.UNITS, mapping.units),\n ]\n```\n\nThe code to use:\n\n```python\ndef _capacity_sources(mapping: GeneratorMapping) -> list[SourceValue]:\n return [\n _source(mapping.name, PlexosProperty.MAX_CAPACITY, mapping.max_capacity, UNIT_MW),\n _source(mapping.name, PlexosProperty.UNITS, mapping.units),\n ]\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-20", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 274, "severity": "minor", "label": "suggestion", "claim": "The `_p_nom_min` docstring repeats the derivation label the same file declares.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\ndef _p_nom_min(mapping: GeneratorMapping) -> Decision:\n \"\"\"A build adds to what the generator has, so what it has is the floor of what it keeps.\"\"\"\n p_nom_min = mapping.expansion.p_nom_min\n```\n\nThe code to use:\n\n```python\ndef _p_nom_min(mapping: GeneratorMapping) -> Decision:\n p_nom_min = mapping.expansion.p_nom_min\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-21", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 342, "severity": "nit", "label": "suggestion", "claim": "The `_from_property` docstring describes the three lines of code below it.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\n) -> Decision:\n \"\"\"One expansion value and the property it came from, unreported where it states none.\"\"\"\n if value is None:\n return Decision.unreported(None)\n return Decision.derived(value, [_source(name, plexos_property, stated, unit)], derivation)\n```\n\nThe code to use:\n\n```python\n) -> Decision:\n if value is None:\n return Decision.unreported(None)\n return Decision.derived(value, [_source(name, plexos_property, stated, unit)], derivation)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-22", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 140, "severity": "minor", "label": "suggestion", "claim": "The `_Drops` docstring argues for collecting the names instead of warning about each one.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\n@dataclass\nclass _Drops:\n \"\"\"What one walk left out, so a warning names a few of them rather than each on its own.\"\"\"\n\n candidates_without_build_cost: list[str] = field(default_factory=list)\n```\n\nThe code to use:\n\n```python\n@dataclass\nclass _Drops:\n candidates_without_build_cost: list[str] = field(default_factory=list)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-23", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 187, "severity": "nit", "label": "suggestion", "claim": "The `_record_expansion_extensions` docstring repeats the name and the two guards below it.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\ndef _record_expansion_extensions(mapping: GeneratorMapping, reporter: ComponentReporter) -> None:\n \"\"\"The two candidate values the network file has no column for, each where it is stated.\"\"\"\n expansion = mapping.expansion\n if expansion.unit_size is not None:\n```\n\nThe code to use:\n\n```python\ndef _record_expansion_extensions(mapping: GeneratorMapping, reporter: ComponentReporter) -> None:\n expansion = mapping.expansion\n if expansion.unit_size is not None:\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-24", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 463, "severity": "nit", "label": "suggestion", "claim": "The `StorageExpansion` docstring says again what the class name and its fields say.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\n@dataclass(frozen=True)\nclass StorageExpansion:\n \"\"\"What a candidate storage object may build, as the columns and the sidecar read it.\"\"\"\n\n p_nom_extendable: Decision\n```\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass StorageExpansion:\n p_nom_extendable: Decision\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-25", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 479, "severity": "minor", "label": "suggestion", "claim": "This paragraph repeats the `Expansion` docstring in `_generator_derivation.py` word for word.", "suggested_fix": "Delete the paragraph and keep the summary line.\n\nThe code now:\n\n```python\ndef derive_expansion(plexos_class: PlexosClass, rated: RatedObject) -> StorageExpansion:\n \"\"\"What one storage object may build; a fixed object states nothing about any of it.\n\n PyPSA works out what a year of new capacity costs from the overnight cost, the discount\n rate and the lifetime together, so nothing here assembles that number itself.\n \"\"\"\n```\n\nThe code to use:\n\n```python\ndef derive_expansion(plexos_class: PlexosClass, rated: RatedObject) -> StorageExpansion:\n \"\"\"What one storage object may build; a fixed object states nothing about any of it.\"\"\"\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-26", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 570, "severity": "nit", "label": "suggestion", "claim": "The docstrings on `_from_property` and `sidecar_value` describe the lines of code below them.", "suggested_fix": "Delete both.\n\nThe code now:\n\n```python\n) -> Decision:\n \"\"\"One expansion value and the property it came from, unreported where none is stated.\"\"\"\n if value is None:\n return NOT_A_CANDIDATE\n source = SourceValue(plexos_class, name, plexos_property, value, unit)\n return Decision.derived(value, [source], derivation)\n\n\ndef sidecar_value(decision: Decision | None) -> float | None:\n \"\"\"What a sidecar record carries for a value only a candidate states.\"\"\"\n return None if decision is None or decision.value is None else float(decision.value)\n```\n\nThe code to use:\n\n```python\n) -> Decision:\n if value is None:\n return NOT_A_CANDIDATE\n source = SourceValue(plexos_class, name, plexos_property, value, unit)\n return Decision.derived(value, [source], derivation)\n\n\ndef sidecar_value(decision: Decision | None) -> float | None:\n return None if decision is None or decision.value is None else float(decision.value)\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-27", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_units.py", "line": 248, "severity": "nit", "label": "suggestion", "claim": "The `_carry_to_extensions` docstring repeats the function name and records an absence.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\ndef _carry_to_extensions(state: State, mappings: list[StorageUnitMapping]) -> None:\n \"\"\"Put the two candidate values in the sidecar, which the network file cannot hold.\"\"\"\n records = [\n```\n\nThe code to use:\n\n```python\ndef _carry_to_extensions(state: State, mappings: list[StorageUnitMapping]) -> None:\n records = [\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-28", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/decisions.py", "line": 113, "severity": "nit", "label": "suggestion", "claim": "The clause \"for a dataclass several paths build\" is shorthand from the author's head.", "suggested_fix": "Cut the trailing clause and keep the contract of the parameter.\n\nThe code now:\n\n```python\n The return type is ``Any`` so the field keeps its ``Decision`` annotation; ``field``\n itself is what the dataclass machinery reads. ``default`` is the decision a mapping\n that has nothing to say about this column carries, for a dataclass several paths build.\n \"\"\"\n```\n\nThe code to use:\n\n```python\n The return type is ``Any`` so the field keeps its ``Decision`` annotation; ``field``\n itself is what the dataclass machinery reads. ``default`` is the decision a mapping\n that has nothing to say about this column carries.\n \"\"\"\n```", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-29", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/decisions.py", "line": 109, "severity": "minor", "label": "suggestion", "claim": "The new `default` parameter on `maps_to` and `declares` gives six fields a fallback that no caller ever uses.", "suggested_fix": "Delete the `default` parameter from `maps_to` and `declares`, restore `declares` to the single `return field(metadata=...)` line, remove the docstring sentence about it, and drop `default=NOT_A_CANDIDATE` from the six `StorageUnitMapping` fields so each fits one line again. Keep `NOT_A_CANDIDATE`, which `_fixed_capacity`, `_from_property` and `_discount_rate` still use. Net: -15 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-30", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 139, "severity": "minor", "label": "suggestion", "claim": "The `_Drops` dataclass holds one list, and one call site builds it.", "suggested_fix": "Delete the `_Drops` class, pass `unpriced: list[str]` to `_map_one` in its place, append the name to that list, and give `_warn_unpriced_candidates` the list directly. Drop `field` from the `dataclasses` import, which then has no other user in the module. Net: -7 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-31", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 273, "severity": "major", "label": "issue", "claim": "`_P_NOM_MIN_DERIVATION` and `_P_NOM_MAX_DERIVATION` say `Max Capacity x Units`, but the value they explain can be the `Rating` instead.", "suggested_fix": "Make `_p_nom_min` and `_p_nom_max` follow `_p_nom`: where `mapping.rating_as_capacity` is set, cite the `Rating` source as well and use a derivation text that names it. Net: +8 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-32", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 71, "severity": "nit", "label": "nitpick", "claim": "`EXTENDABLE_NOTE`, `EXTENDABLE_DERIVATION` and `NOT_A_CANDIDATE` have no leading underscore, but no other module reads them.", "suggested_fix": "Rename the three to `_EXTENDABLE_NOTE`, `_EXTENDABLE_DERIVATION` and `_NOT_A_CANDIDATE`, matching `_P_NOM_MIN_DERIVATION` and `_SOC_NOTE` beside them. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} From f6eec133e5e777c5feb42a0a0d307062eb050ec4 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 18:30:34 +0000 Subject: [PATCH 05/17] Read a solved zero as a rejected build, and give the expansion rule one owner The effective_p_nom rule stood in seven places across the PyPSA leg, and each copy read a p_nom_opt of zero as an unsolved network. A solve that builds none of an extendable component writes p_nom_opt 0; only a network no solve has touched leaves the column out. Reading that 0 as unsolved carried the whole candidate capacity into the Sienna operations system, adding a plant the expansion refused. The rule now lives once in the pypsa_sienna_translations _shared module, keyed on whether the network states a p_nom_opt at all, and the generator, renewable, hydro, storage and link modules call it. The four fill_defaults calls leave p_nom_opt null rather than filling it to zero, so an unsolved network is still distinguishable. A candidate battery divided its stated Capacity by the capacity it may build, so its max_hours came out Max Units Built times too small. A Battery states its Capacity beside one unit's Max Power, so both now per-unitise against that one power, and the initial state of charge reads its percentage against p_nom times max_hours rather than against one unit's energy. A candidate that states no Economic Life is left out beside one that states no Build Cost or no WACC. PyPSA's lifetime default is infinity, not NaN, so nothing refuses the network and the build is annuitised as a perpetuity; a candidate stating WACC 0 with no Economic Life would be built for free. The rest is duplication and dead code. SourceGenerator no longer restates the candidate rule in plain floats: it reads its p_nom off the Decision _expansion builds, so the reported event and the emitted row cannot disagree. The warning about an unpriced candidate moves beside that rule, so the generator path and the storage path both cap it at a few names instead of the storage path warning once per object. NOT_A_CANDIDATE becomes NOTHING_TO_REPORT, which is what a genuine candidate stating no FO&M Charge also gets. ExpansionDecisions.is_candidate and RatedObject.stated_units had no reader, and three docstrings restated their own signature or a rule the code does not have. The scenarios that reached the expansion path with a Build Cost and a WACC alone now state an Economic Life, and one more candidate stands beside them to show what leaving it out does. Two new assertions pin the numbers the fixes change: a candidate battery's max_hours and initial level, and the base_power of an extendable generator a solve built none of. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 23 +++-- .../translation-from-pypsa-to-sienna.md | 2 +- .../plexos_pypsa_translations/_batteries.py | 29 +++--- .../plexos_pypsa_translations/_expansion.py | 94 ++++++++++++++----- .../_generator_derivation.py | 30 ++---- .../plexos_pypsa_translations/_generators.py | 39 +++----- .../_storage_shared.py | 33 ++++--- .../_storage_units.py | 7 +- .../pypsa_sienna_translations/_generators.py | 42 +++++---- .../pypsa_sienna_translations/_hydro.py | 31 +++--- .../pypsa_sienna_translations/_links.py | 20 ++-- .../pypsa_sienna_translations/_renewables.py | 26 ++--- .../pypsa_sienna_translations/_shared.py | 29 ++++++ .../pypsa_sienna_translations/_storage.py | 27 +++--- .../plexos_to_pypsa/generators.feature | 13 ++- .../plexos_to_pypsa/storage_units.feature | 6 ++ .../thermal_generators.feature | 14 +++ 17 files changed, 282 insertions(+), 183 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index 4899e59..4015428 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -278,7 +278,7 @@ is `committable` when it is thermal, or when its `p_min_pu` is more than `0`. | `lifetime` | yr | `Economic Life`, for a candidate only | `direct` | | `fom_cost` | $/MW/yr | `FO&M Charge`, for a candidate only | `direct` | -**The translator does not translate five cases.** It records each one as a skipped +**The translator does not translate six cases.** It records each one as a skipped component: | Case | Cause | @@ -289,6 +289,7 @@ component: | `p_nom` is 0 | The generator can never dispatch. | | The generator is a candidate and gives no `Build Cost` | Nothing prices building it, so an expansion would take it for free. | | The generator is a candidate and gives no `WACC` | PyPSA annuitises a build cost with a discount rate, and refuses a network that states one without the other. | +| The generator is a candidate and gives no `Economic Life` | PyPSA annuitises a build cost across a lifetime. The PyPSA default is infinity, which prices the build as a perpetuity. | The repair rates, the unit commitment solver options and the energy budgets are `dropped`. @@ -361,10 +362,12 @@ has more than one fuel uses its primary fuel. | `marginal_cost` | $/MWh | `0.0` | `default` | | `p_nom_extendable`, `p_nom_min`, `p_nom_max`, `overnight_cost`, `discount_rate`, `lifetime`, `fom_cost` | | Refer to [What a candidate is](#what-a-candidate-is) | `derived` / `default` | -The energy capacity of a battery is its `Capacity`. If the model gives a duration in place -of a capacity, the energy capacity is `Duration × Max Power`. `max_hours` and -`state_of_charge_initial` both read that one value. Thus they cannot disagree about the -energy of the battery. +The energy one unit of a battery holds is its `Capacity`. If the model gives a duration in +place of a capacity, that energy is `Duration × Max Power`. `max_hours` is that energy +divided by `Max Power`, which is the power of the same one unit, so `p_nom × max_hours` is +the energy of however many units the `p_nom` stands for. `state_of_charge_initial` reads +`Initial SoC` against that same energy. Thus they cannot disagree about the energy of the +battery. `Charge Efficiency` is a round trip value. The translator divides it equally between the charge and the discharge. Thus the round trip value does not change. `Min SoC` and `Max @@ -737,10 +740,12 @@ The rated power of one unit is the `Max Capacity` of a `Generator` or a pumped-s turbine, and the `Max Power` of a `Battery`. An object that states `Units` above one already holds that many, so a build adds units to what it has rather than multiplying it. -A candidate that states no `Build Cost`, or no `WACC`, is left out. Without a build cost -nothing prices building it, so an expansion would take it for free; without a discount rate -PyPSA cannot annuitise the build cost, and refuses the network. The translator records a -`COMPONENT_SKIPPED` event naming the object, and warns once naming a few of them. +A candidate that states no `Build Cost`, no `WACC` or no `Economic Life` is left out. +Without a build cost nothing prices building it, so an expansion would take it for free. +Without a discount rate PyPSA cannot annuitise the build cost, and refuses the network. +Without an economic life PyPSA annuitises across its own default lifetime of infinity, which +prices the build as a perpetuity. The translator records a `COMPONENT_SKIPPED` event naming +the object, and warns once naming a few of them. ### Which entry applies when diff --git a/docs/translation_mappings/translation-from-pypsa-to-sienna.md b/docs/translation_mappings/translation-from-pypsa-to-sienna.md index 39b9223..a1db7a4 100644 --- a/docs/translation_mappings/translation-from-pypsa-to-sienna.md +++ b/docs/translation_mappings/translation-from-pypsa-to-sienna.md @@ -81,7 +81,7 @@ This document targets the SiennaSchemas **Operations** namespace (the dispatchab **Implication for this translation.** PyPSA's capacity-expansion fields (`p_nom_extendable`, `capital_cost`, `p_nom_min`/`p_nom_max`, `build_year`, `lifetime`) belong to **Investments**, which is a separate translation target (PyPSA expansion → SiennaSchemas Investments), not extra fields on an Operations component. It is **out of scope for v1**. For v1 we translate the **solved** fleet: take `p_nom_opt` (post-solve capacities) as fixed Operations capacity and drop the expansion parameters (or keep round-trip crumbs in the `ext` sidecar). A genuine expansion problem is the future Investments path. -Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True` and that `p_nom_opt` is above zero, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`; for an extendable component in a solved network `effective_p_nom = p_nom_opt > 0`. An extendable component whose `p_nom_opt` is 0 has not been solved, so `effective_p_nom` falls back to `p_nom`. +Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True` and the network states a `p_nom_opt`, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`. A solve that builds none of an extendable component writes `p_nom_opt` 0, and that 0 is the capacity to translate: the plan rejected the build. Only a network no solve has touched leaves `p_nom_opt` out of the file altogether, and there `effective_p_nom` falls back to `p_nom`. **Why Operations and not Investments.** This is a deliberate scope choice tied to the deliverable, not a claim that Operations is the more natural fit: diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index 1b74554..f1d198a 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -95,6 +95,7 @@ def record_battery_outages(state: State, mappings: list[StorageUnitMapping]) -> _CAPACITY_DERIVATION = "Capacity" _CAPACITY_FROM_DURATION_DERIVATION = "Duration * Max Power" _SOC_FROM_PERCENT_DERIVATION = "Initial SoC / 100 * energy capacity" +_PER_MAX_POWER_DERIVATION = " / Max Power" # --- batteries --------------------------------------------------------------- @@ -125,8 +126,10 @@ def _battery_rating(staged: StagedObject) -> RatedCapacity: def _derive_battery(rated: RatedObject) -> StorageUnitMapping: - capacity = _battery_energy_capacity(rated) - max_hours = derive_max_hours(capacity, rated.p_nom.value) + unit_size = rated.candidate.rated.unit_size + capacity = _battery_energy_capacity(rated, unit_size) + max_hours = derive_max_hours(capacity, unit_size.value, per=_PER_MAX_POWER_DERIVATION) + stored_energy = rated.p_nom.value * max_hours.value expansion = derive_expansion(rated.candidate) return StorageUnitMapping( name=rated.name, @@ -139,7 +142,7 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: efficiency=_battery_efficiency(rated), marginal_cost=Decision.default(STORAGE_MARGINAL_COST, _BATTERY_MARGINAL_COST_NOTE), state_of_charge_initial=derive_state_of_charge_initial( - _battery_initial_level(rated, capacity), rated.p_nom.value * max_hours.value + _battery_initial_level(rated, capacity, stored_energy), stored_energy ), inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE), cyclic=_battery_cyclic(rated), @@ -167,8 +170,8 @@ def _battery_cyclic(rated: RatedObject) -> Decision: return Decision.default(BATTERY_CYCLIC, _BATTERY_CYCLIC_NOTE) -def _battery_energy_capacity(rated: RatedObject) -> Decision | None: - """The MWh a battery holds when full, from Capacity or from Duration * Max Power.""" +def _battery_energy_capacity(rated: RatedObject, unit_size: Decision) -> Decision | None: + """The MWh one unit holds when full, from Capacity or from Duration * Max Power.""" capacity = rated.properties.get(PlexosProperty.CAPACITY) if capacity is not None: source = SourceValue( @@ -178,22 +181,24 @@ def _battery_energy_capacity(rated: RatedObject) -> Decision | None: duration = rated.properties.get(PlexosProperty.DURATION) if duration is None: return None - return _capacity_from_duration(rated.name, duration, rated.p_nom) + return _capacity_from_duration(rated.name, duration, unit_size) -def _capacity_from_duration(name: str, duration: float, p_nom: Decision) -> Decision: +def _capacity_from_duration(name: str, duration: float, unit_size: Decision) -> Decision: return Decision.derived( - duration * p_nom.value, + duration * unit_size.value, [ SourceValue(PlexosClass.BATTERY, name, PlexosProperty.DURATION, duration, UNIT_HOURS), - *p_nom.sources, + *unit_size.sources, ], _CAPACITY_FROM_DURATION_DERIVATION, ) -def _battery_initial_level(rated: RatedObject, capacity: Decision | None) -> Decision | None: - """Where the battery starts, as MWh, from Initial SoC against its energy capacity.""" +def _battery_initial_level( + rated: RatedObject, capacity: Decision | None, stored_energy: float +) -> Decision | None: + """Where the battery starts, as MWh, from Initial SoC against the energy it holds.""" initial_soc = rated.properties.get(PlexosProperty.INITIAL_SOC) if initial_soc is None or capacity is None: return None @@ -201,7 +206,7 @@ def _battery_initial_level(rated: RatedObject, capacity: Decision | None) -> Dec PlexosClass.BATTERY, rated.name, PlexosProperty.INITIAL_SOC, initial_soc, UNIT_PERCENT ) return Decision.derived( - initial_soc / PERCENT * capacity.value, + initial_soc / PERCENT * stored_energy, [source, *capacity.sources], _SOC_FROM_PERCENT_DERIVATION, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py index c120270..b6dd181 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -7,6 +7,8 @@ from __future__ import annotations +import logging +from collections.abc import Sequence from dataclasses import dataclass from interop.plugins.shared.constants import ( @@ -31,8 +33,11 @@ maps_to, ) from interop.plugins.shared.pypsa_constants import PyPSAGeneratorCol +from interop.plugins.shared.warning_text import name_a_few -NOT_A_CANDIDATE = Decision.unreported(None) +log = logging.getLogger(__name__) + +NOTHING_TO_REPORT = Decision.unreported(None) NOTHING_BUILT_DERIVATION = "Units is zero, so none of the rated power is built yet" @@ -60,6 +65,10 @@ "a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost over, " "so PyPSA refuses the network" ) +_NO_ECONOMIC_LIFE_NOTE = ( + "a candidate with no Economic Life gives PyPSA no period to annuitise its Build Cost over, " + "so PyPSA prices the build as a perpetuity" +) UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) @@ -67,11 +76,7 @@ @dataclass(frozen=True) class RatedCapacity: - """The rated power an object already has, beside the rated power of one unit of it. - - The two differ wherever a model states more than one unit, and wherever a candidate - states no unit at all. - """ + """The rated power an object already has, beside the rated power of one unit of it.""" existing: Decision unit_size: Decision @@ -111,22 +116,18 @@ class ExpansionDecisions: lifetime: Decision = maps_to(PyPSAGeneratorCol.LIFETIME, unit=UNIT_YEARS) fom_cost: Decision = maps_to(PyPSAGeneratorCol.FOM_COST, unit=UNIT_DOLLARS_PER_MW_YEAR) # What the sidecar carries because the network file has no column for it. - unit_size: Decision = NOT_A_CANDIDATE - technical_life: Decision = NOT_A_CANDIDATE - - @property - def is_candidate(self) -> bool: - return bool(self.p_nom_extendable.value) + unit_size: Decision = NOTHING_TO_REPORT + technical_life: Decision = NOTHING_TO_REPORT FIXED_CAPACITY = ExpansionDecisions( p_nom_extendable=Decision.default(False, _NOT_EXTENDABLE_NOTE), # noqa: FBT003 - p_nom_min=NOT_A_CANDIDATE, - p_nom_max=NOT_A_CANDIDATE, - overnight_cost=NOT_A_CANDIDATE, - discount_rate=NOT_A_CANDIDATE, - lifetime=NOT_A_CANDIDATE, - fom_cost=NOT_A_CANDIDATE, + p_nom_min=NOTHING_TO_REPORT, + p_nom_max=NOTHING_TO_REPORT, + overnight_cost=NOTHING_TO_REPORT, + discount_rate=NOTHING_TO_REPORT, + lifetime=NOTHING_TO_REPORT, + fom_cost=NOTHING_TO_REPORT, ) @@ -165,7 +166,14 @@ def derive_expansion(source: CandidateSource) -> ExpansionDecisions: def derive_buildable(source: CandidateSource) -> Decision: - """The capacity a candidate may build, which stands as its p_nom while it has none.""" + """The capacity a candidate may build, which stands as its p_nom while it has none. + + PyPSA optimises ``p_nom_opt`` between ``p_nom_min`` and ``p_nom_max`` and reads ``p_nom`` + only for an object whose capacity is fixed, so what stands here does not bind the + dispatch. It is what every per-unit field on the component is read against -- + ``p_min_pu``, a ramp limit, an availability profile stated in MW -- and against nothing + each of those would come out at zero. + """ return Decision.derived( source.rated.unit_size.value * source.max_units_built, [*source.rated.unit_size.sources, source.name_units_built()], @@ -174,7 +182,6 @@ def derive_buildable(source: CandidateSource) -> Decision: def derive_p_nom(source: CandidateSource) -> Decision: - """What the object has, or what it may build where it has nothing yet.""" if source.rated.existing.value or not source.is_candidate: return source.rated.existing return derive_buildable(source) @@ -192,14 +199,53 @@ class UnpricedBuild: _PRICES_A_BUILD = ( UnpricedBuild(PlexosProperty.BUILD_COST, UNIT_DOLLARS_PER_MW, _NO_BUILD_COST_NOTE), UnpricedBuild(PlexosProperty.WACC, None, _NO_WACC_NOTE), + UnpricedBuild(PlexosProperty.ECONOMIC_LIFE, UNIT_YEARS, _NO_ECONOMIC_LIFE_NOTE), ) -def find_unpriced_build(props: dict[str, float]) -> UnpricedBuild | None: +@dataclass(frozen=True) +class UnpricedCandidate: + """One candidate left out, and the property it left out that priced its build.""" + + plexos_class: PlexosClass + name: str + unpriced: UnpricedBuild + + @property + def source(self) -> SourceValue: + return SourceValue( + self.plexos_class, self.name, self.unpriced.plexos_property, None, self.unpriced.unit + ) + + @property + def note(self) -> str: + return self.unpriced.note + + +def find_unpriced_build( + plexos_class: PlexosClass, name: str, props: dict[str, float] +) -> UnpricedCandidate | None: """The first property a candidate leaves out that stops PyPSA pricing its build.""" if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD: return None - return next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None) + unpriced = next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None) + return None if unpriced is None else UnpricedCandidate(plexos_class, name, unpriced) + + +def warn_unpriced_builds(candidates: Sequence[UnpricedCandidate]) -> None: + """One warning per class and property a candidate left out, naming a few of the objects.""" + by_property: dict[tuple[str, str], list[str]] = {} + for one in candidates: + key = (one.plexos_class, one.unpriced.plexos_property) + by_property.setdefault(key, []).append(one.name) + for (plexos_class, plexos_property), names in sorted(by_property.items()): + log.warning( + "plexos: %d candidate %s(s) state no %s, so each is left out. Each one: %s", + len(names), + plexos_class, + plexos_property, + name_a_few(sorted(names)), + ) def record_expansion_extensions( @@ -226,7 +272,7 @@ def _discount_rate(source: CandidateSource) -> Decision: wacc = source.props.get(PlexosProperty.WACC) rate = as_rate(wacc, source.stated_units.get(PlexosProperty.WACC)) if rate is None: - return NOT_A_CANDIDATE + return NOTHING_TO_REPORT stated = SourceValue(source.plexos_class, source.name, PlexosProperty.WACC, wacc) return Decision.derived(rate, [stated], _DISCOUNT_RATE_DERIVATION) @@ -236,6 +282,6 @@ def _from_property( ) -> Decision: value = source.props.get(plexos_property) if value is None: - return NOT_A_CANDIDATE + return NOTHING_TO_REPORT stated = SourceValue(source.plexos_class, source.name, plexos_property, value, unit) return Decision.derived(value, [stated], derivation) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py index cdbedde..cee7c87 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py @@ -11,6 +11,7 @@ from dataclasses import dataclass from enum import Enum, auto +from functools import cached_property from typing import Any from interop.plugins.shared.constants import UNIT_MW @@ -24,6 +25,7 @@ ExpansionDecisions, RatedCapacity, derive_expansion, + derive_p_nom, ) from interop.plugins.shared.plexos_pypsa_translations._generator_lookups import Lookups from interop.plugins.shared.plexos_pypsa_translations.constants import ( @@ -35,7 +37,6 @@ FULL_AVAILABILITY, MAX_RAMP_LIMIT_PU, NEGLIGIBLE_P_MIN_PU, - NOTHING_TO_BUILD, PERCENT, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import Decision, SourceValue @@ -102,18 +103,13 @@ class SourceGenerator: stated_units: dict[str, str | None] max_capacity: float - @property - def max_units_built(self) -> float: - return _value(self.props, PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) + @cached_property + def candidate(self) -> CandidateSource: + return read_candidate(self) @property def is_candidate(self) -> bool: - return self.max_units_built > NOTHING_TO_BUILD - - @property - def buildable(self) -> float: - """MW the model allows to be built, on top of whatever the generator already has.""" - return self.max_units_built * self.max_capacity + return self.candidate.is_candidate @property def nameplate(self) -> float: @@ -139,15 +135,8 @@ def existing(self) -> float: @property def p_nom(self) -> float: - """What the generator has, or what it may build where it has nothing yet. - - PyPSA optimises ``p_nom_opt`` between ``p_nom_min`` and ``p_nom_max`` and reads - ``p_nom`` only for a generator whose capacity is fixed, so what stands here for a - candidate does not bind its dispatch. It is what every per-unit field on the - component is read against -- ``p_min_pu``, a ramp limit, an availability profile - stated in MW -- and against nothing each of those would come out at zero. - """ - return self.existing or self.buildable + """What the generator has, or what it may build where it has nothing yet.""" + return float(derive_p_nom(self.candidate).value) def read_source(generator: dict[str, Any], name: str, lookups: Lookups) -> SourceGenerator: @@ -176,7 +165,6 @@ def _rated_capacity(name: str, props: dict[str, float], lookups: Lookups) -> flo def read_candidate(source: SourceGenerator) -> CandidateSource: - """One staged Generator as the shared expansion rule reads it.""" capacity = SourceValue( PlexosClass.GENERATOR, source.name, @@ -249,7 +237,7 @@ def has_infeasible_dispatch_range(mapping: GeneratorMapping) -> bool: def derive_generator(source: SourceGenerator, node: str, lookups: Lookups) -> GeneratorMapping: - candidate = read_candidate(source) + candidate = source.candidate fuels = lookups.gen_fuels.get(source.name, []) fuel = _fuel_use(source, fuels[0] if fuels else None, lookups) availability = _availability(source, lookups.availability_profiles.get(source.name)) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index 92bb2d9..2bb449e 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -23,9 +23,11 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + UnpricedCandidate, find_unpriced_build, read_sidecar_value, record_expansion_extensions, + warn_unpriced_builds, ) from interop.plugins.shared.plexos_pypsa_translations._generator_decisions import ( GeneratorDecisions, @@ -102,14 +104,14 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: lookups = build_lookups(state) reporter = ComponentReporter(recorder, PyPSAComponent.GENERATOR) storage_turbines = storage_turbine_names(state) - unpriced: dict[str, list[str]] = {} - translated = [ + outcomes = [ one for generator in generators.collect().iter_rows(named=True) if generator[PlexosObjectCol.NAME] not in storage_turbines - if (one := _map_one(generator, lookups, reporter, unpriced)) is not None + if (one := _map_one(generator, lookups, reporter)) is not None ] - _warn_unpriced_candidates(unpriced) + warn_unpriced_builds([one for one in outcomes if isinstance(one, UnpricedCandidate)]) + translated = [one for one in outcomes if isinstance(one, _TranslatedGenerator)] if not translated: return append_destination_rows( @@ -126,16 +128,6 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: _record_availability_time_series(state, mappings, reporter) -def _warn_unpriced_candidates(unpriced: dict[str, list[str]]) -> None: - for plexos_property, names in unpriced.items(): - log.warning( - "plexos: %d candidate generator(s) state no %s, so each is left out. Each one: %s", - len(names), - plexos_property, - name_a_few(sorted(names)), - ) - - @dataclass(frozen=True) class _TranslatedGenerator: """One generator's derived values and the destination decisions they justify.""" @@ -172,11 +164,8 @@ def _category_decision(mapping: GeneratorMapping) -> Decision: def _map_one( - generator: dict[str, Any], - lookups: Lookups, - reporter: ComponentReporter, - unpriced: dict[str, list[str]], -) -> _TranslatedGenerator | None: + generator: dict[str, Any], lookups: Lookups, reporter: ComponentReporter +) -> _TranslatedGenerator | UnpricedCandidate | None: name = generator[PlexosObjectCol.NAME] node = lookups.gen_to_node.get(name) if node is None: @@ -197,14 +186,10 @@ def _map_one( f"generator dropped: p_nom is {source.p_nom} MW, so it can never dispatch", ) return None - unpriced_build = find_unpriced_build(source.props) - if unpriced_build is not None: - reporter.record_skipped( - _source(name, unpriced_build.plexos_property, None, unpriced_build.unit), - unpriced_build.note, - ) - unpriced.setdefault(unpriced_build.plexos_property, []).append(name) - return None + unpriced = find_unpriced_build(PlexosClass.GENERATOR, name, source.props) + if unpriced is not None: + reporter.record_skipped(unpriced.source, unpriced.note) + return unpriced mapping = derive_generator(source, node, lookups) if has_infeasible_dispatch_range(mapping): _report_infeasible_dispatch_range(mapping, reporter) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index d1be9a7..a0aa1f0 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -31,6 +31,7 @@ CandidateSource, ExpansionDecisions, RatedCapacity, + UnpricedCandidate, derive_p_nom, find_unpriced_build, ) @@ -135,10 +136,15 @@ def warn_about_skipped(skipped: SkippedComponent) -> None: @dataclass(frozen=True) class SkippedComponent: - """A PLEXOS object the mapping deliberately did not translate, and why.""" + """A PLEXOS object the mapping deliberately did not translate, and why. + + ``unpriced`` is set where the expansion rule left the object out, since that rule warns + about all of them together rather than one line each. + """ source: SourceValue note: str + unpriced: UnpricedCandidate | None = None # Every storage object ends as one or the other: a unit to write, or a recorded reason not to. @@ -350,7 +356,6 @@ class RatedObject: name: str properties: dict[str, float] - stated_units: dict[str, str | None] node: str p_nom: Decision candidate: CandidateSource @@ -364,11 +369,9 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp return _skipped_file_backed(rating, staged.name) if rating.capacity_property not in staged.properties: return _skipped_without_capacity(rating, staged.name) - unpriced = find_unpriced_build(staged.properties) + unpriced = find_unpriced_build(rating.plexos_class, staged.name, staged.properties) if unpriced is not None: - return skip_object( - rating.plexos_class, staged.name, unpriced.plexos_property, unpriced.note - ) + return SkippedComponent(unpriced.source, unpriced.note, unpriced) candidate = CandidateSource( rating.plexos_class, staged.name, @@ -379,9 +382,7 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp p_nom = derive_p_nom(candidate) if p_nom.value <= 0.0: return _skipped_zero_p_nom(rating, staged.name, p_nom.value) - return RatedObject( - staged.name, staged.properties, staged.stated_units, staged.node, p_nom, candidate - ) + return RatedObject(staged.name, staged.properties, staged.node, p_nom, candidate) def skip_object( @@ -417,13 +418,21 @@ def _skipped_zero_p_nom(rating: RatedPower, name: str, p_nom: float) -> SkippedC def derive_max_hours( - capacity: Decision | None, p_nom: float, absent_note: str = MAX_HOURS_NOTE + capacity: Decision | None, + rated_power: float, + absent_note: str = MAX_HOURS_NOTE, + per: str = _PER_P_NOM_DERIVATION, ) -> Decision: - """Hours of storage at rated power, from whichever property stated the energy capacity.""" + """Hours of storage at rated power, from whichever property stated the energy capacity. + + ``capacity`` and ``rated_power`` have to describe the same thing: a reservoir shared by + every unit divides by the power of all of them, and an energy stated beside one unit's + power divides by that one power. + """ if capacity is None: return Decision.default(DEFAULT_STORAGE_MAX_HOURS, absent_note) return Decision.derived( - capacity.value / p_nom, capacity.sources, capacity.explanation + _PER_P_NOM_DERIVATION + capacity.value / rated_power, capacity.sources, capacity.explanation + per ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index b3012bb..a74d08b 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -30,6 +30,7 @@ from interop.plugins.shared.plexos_pypsa_translations._expansion import ( read_sidecar_value, record_expansion_extensions, + warn_unpriced_builds, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, @@ -230,7 +231,11 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No record_expansion_extensions(mapping.name, mapping.expansion, reporter) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) - warn_about_skipped(skipped) + if skipped.unpriced is None: + warn_about_skipped(skipped) + warn_unpriced_builds( + [skipped.unpriced for skipped in storage_units.skipped if skipped.unpriced is not None] + ) for dropped in storage_units.dropped: reporter.record_dropped(dropped.source, dropped.note) diff --git a/interop/plugins/shared/pypsa_sienna_translations/_generators.py b/interop/plugins/shared/pypsa_sienna_translations/_generators.py index a8c72ff..6a92b2c 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_generators.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_generators.py @@ -27,11 +27,15 @@ ExtensionSpec, ) from interop.plugins.shared.pypsa_sienna_translations._shared import ( + EFFECTIVE_P_NOM, + EFFECTIVE_P_NOM_DERIVATION, + holds_solved_capacity, pypsa_skip_report, pypsa_source_field, sienna_dest_field, ts_association_row, variable_cost_curve, + with_effective_p_nom, ) from interop.plugins.shared.pypsa_sienna_translations._ts_info import TimeSeriesInfo from interop.plugins.shared.pypsa_sienna_user_mappings import CarrierMappings @@ -70,7 +74,6 @@ _PRIME_MOVER_COL = "_prime_mover_raw" _TS_PEAK_PMAX = "_ts_peak_pmax" _DT_MINUTES_COL = "_dt_minutes" -_EFFECTIVE_P_NOM = "_effective_p_nom" def fill_generator_defaults(table: pl.DataFrame) -> pl.DataFrame: @@ -79,7 +82,7 @@ def fill_generator_defaults(table: pl.DataFrame) -> pl.DataFrame: table, [ (PyPSAGeneratorCol.P_NOM, 0.0), - (PyPSAGeneratorCol.P_NOM_OPT, 0.0), + (PyPSAGeneratorCol.P_NOM_OPT, None), (PyPSAGeneratorCol.P_MIN_PU, 0.0), (PyPSAGeneratorCol.P_MAX_PU, 1.0), (PyPSAGeneratorCol.MARGINAL_COST, 0.0), @@ -96,13 +99,11 @@ def fill_generator_defaults(table: pl.DataFrame) -> pl.DataFrame: (PyPSAGeneratorCol.P_NOM_EXTENDABLE, False), ], ) - return table.with_columns( - pl.when( - pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE) & (pl.col(PyPSAGeneratorCol.P_NOM_OPT) > 0) - ) - .then(pl.col(PyPSAGeneratorCol.P_NOM_OPT)) - .otherwise(pl.col(PyPSAGeneratorCol.P_NOM)) - .alias(_EFFECTIVE_P_NOM) + return with_effective_p_nom( + table, + PyPSAGeneratorCol.P_NOM_EXTENDABLE, + PyPSAGeneratorCol.P_NOM_OPT, + PyPSAGeneratorCol.P_NOM, ) @@ -360,15 +361,15 @@ def build_generator_extensions( GENERATOR_BASE_POWER = _direct( source_col=PyPSAGeneratorCol.P_NOM, dest_col=T.BASE_POWER, - expr=pl.col(_EFFECTIVE_P_NOM), + expr=pl.col(EFFECTIVE_P_NOM), unit=UNIT_MW, - derivation="p_nom_opt where an extendable component has one, else p_nom", + derivation=EFFECTIVE_P_NOM_DERIVATION, ) GENERATOR_ACTIVE_POWER = _direct( source_col=PyPSAGeneratorCol.P_NOM, dest_col=T.ACTIVE_POWER, - expr=pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MIN_PU), + expr=pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MIN_PU), unit=UNIT_MW, derivation="effective_p_nom * p_min_pu (initial dispatch = min operating point)", ) @@ -388,10 +389,10 @@ def build_generator_extensions( GENERATOR_APL = Translation( exprs=[ pl.struct( - min=(pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MIN_PU)).cast(pl.Float64), + min=(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MIN_PU)).cast(pl.Float64), max=pl.when(pl.col(_TS_PEAK_PMAX).is_not_null()) - .then(pl.col(_EFFECTIVE_P_NOM) * pl.col(_TS_PEAK_PMAX)) - .otherwise(pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MAX_PU)) + .then(pl.col(EFFECTIVE_P_NOM) * pl.col(_TS_PEAK_PMAX)) + .otherwise(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MAX_PU)) .cast(pl.Float64), ) .cast(ACTIVE_POWER_LIMITS_DTYPE) @@ -407,11 +408,12 @@ def build_generator_extensions( name=old[PyPSAGeneratorCol.NAME], attribute=( PyPSAGeneratorCol.P_NOM_OPT - if old[PyPSAGeneratorCol.P_NOM_EXTENDABLE] - and old[PyPSAGeneratorCol.P_NOM_OPT] > 0 + if holds_solved_capacity( + old, PyPSAGeneratorCol.P_NOM_EXTENDABLE, PyPSAGeneratorCol.P_NOM_OPT + ) else PyPSAGeneratorCol.P_NOM ), - value=old[_EFFECTIVE_P_NOM], + value=old[EFFECTIVE_P_NOM], unit=UNIT_MW, ) ], @@ -443,12 +445,12 @@ def build_generator_extensions( .then( pl.struct( up=( - pl.col(_EFFECTIVE_P_NOM) + pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.RAMP_LIMIT_UP) / pl.col(_DT_MINUTES_COL) ).cast(pl.Float64), down=( - pl.col(_EFFECTIVE_P_NOM) + pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.RAMP_LIMIT_DOWN) / pl.col(_DT_MINUTES_COL) ).cast(pl.Float64), diff --git a/interop/plugins/shared/pypsa_sienna_translations/_hydro.py b/interop/plugins/shared/pypsa_sienna_translations/_hydro.py index 2f3bc45..20b11d7 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_hydro.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_hydro.py @@ -33,11 +33,14 @@ ) from interop.plugins.shared.pypsa_sienna_translations._prime_mover import enrich_prime_mover from interop.plugins.shared.pypsa_sienna_translations._shared import ( + EFFECTIVE_P_NOM, + EFFECTIVE_P_NOM_DERIVATION, pypsa_skip_report, pypsa_source_field, sienna_dest_field, ts_association_row, variable_cost_curve, + with_effective_p_nom, ) from interop.plugins.shared.pypsa_sienna_translations._ts_info import TimeSeriesInfo from interop.plugins.shared.pypsa_sienna_user_mappings import CarrierMappings @@ -73,7 +76,6 @@ HYDRO_MAX_ACTIVE_POWER_ATTR = "_hydro_max_active_power" _PRIME_MOVER_COL = "_prime_mover_raw" -_EFFECTIVE_P_NOM = "_effective_p_nom" _source = partial(pypsa_source_field, PyPSAComponent.STORAGE_UNIT) @@ -98,7 +100,7 @@ def fill_hydro_defaults(table: pl.DataFrame) -> pl.DataFrame: table, [ (PyPSAStorageUnitCol.P_NOM, 0.0), - (PyPSAStorageUnitCol.P_NOM_OPT, 0.0), + (PyPSAStorageUnitCol.P_NOM_OPT, None), (PyPSAStorageUnitCol.P_MIN_PU, 0.0), (PyPSAStorageUnitCol.P_MAX_PU, 1.0), (PyPSAStorageUnitCol.MARGINAL_COST, 0.0), @@ -106,14 +108,11 @@ def fill_hydro_defaults(table: pl.DataFrame) -> pl.DataFrame: ], [(PyPSAStorageUnitCol.P_NOM_EXTENDABLE, False)], ) - return table.with_columns( - pl.when( - pl.col(PyPSAStorageUnitCol.P_NOM_EXTENDABLE) - & (pl.col(PyPSAStorageUnitCol.P_NOM_OPT) > 0) - ) - .then(pl.col(PyPSAStorageUnitCol.P_NOM_OPT)) - .otherwise(pl.col(PyPSAStorageUnitCol.P_NOM)) - .alias(_EFFECTIVE_P_NOM) + return with_effective_p_nom( + table, + PyPSAStorageUnitCol.P_NOM_EXTENDABLE, + PyPSAStorageUnitCol.P_NOM_OPT, + PyPSAStorageUnitCol.P_NOM, ) @@ -208,7 +207,7 @@ def build_hydro_ts_associations( source_table=PyPSATable.STORAGE_UNITS, source_attribute=PyPSAStorageUnitCol.INFLOW, scaling_factor=( - src_row[_EFFECTIVE_P_NOM] / src_row[PyPSAStorageUnitCol.EFFICIENCY_DISPATCH] + src_row[EFFECTIVE_P_NOM] / src_row[PyPSAStorageUnitCol.EFFICIENCY_DISPATCH] ), ) ) @@ -273,15 +272,15 @@ def build_hydro_ts_associations( HYDRO_BASE_POWER = _direct( source_col=PyPSAStorageUnitCol.P_NOM, dest_col=H.BASE_POWER, - expr=pl.col(_EFFECTIVE_P_NOM), + expr=pl.col(EFFECTIVE_P_NOM), unit=UNIT_MW, - derivation="p_nom_opt where an extendable component has one, else p_nom", + derivation=EFFECTIVE_P_NOM_DERIVATION, ) HYDRO_ACTIVE_POWER = _direct( source_col=PyPSAStorageUnitCol.P_NOM, dest_col=H.ACTIVE_POWER, - expr=pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.P_MIN_PU), + expr=pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.P_MIN_PU), unit=UNIT_MW, derivation="effective_p_nom * p_min_pu (initial dispatch = min operating point)", ) @@ -302,8 +301,8 @@ def build_hydro_ts_associations( source_col=PyPSAStorageUnitCol.P_NOM, dest_col=H.ACTIVE_POWER_LIMITS, expr=pl.struct( - min=(pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.P_MIN_PU)).cast(pl.Float64), - max=(pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.P_MAX_PU)).cast(pl.Float64), + min=(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.P_MIN_PU)).cast(pl.Float64), + max=(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.P_MAX_PU)).cast(pl.Float64), ).cast(ACTIVE_POWER_LIMITS_DTYPE), unit=UNIT_MW, derivation="min=effective_p_nom*p_min_pu, max=effective_p_nom*p_max_pu", diff --git a/interop/plugins/shared/pypsa_sienna_translations/_links.py b/interop/plugins/shared/pypsa_sienna_translations/_links.py index 16edc1d..24eea50 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_links.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_links.py @@ -23,7 +23,11 @@ PyPSATable, PyPSATimeSeriesCol, ) -from interop.plugins.shared.pypsa_sienna_translations._shared import pypsa_skip_report +from interop.plugins.shared.pypsa_sienna_translations._shared import ( + effective_p_nom, + holds_solved_capacity, + pypsa_skip_report, +) from interop.plugins.shared.sienna_constants import ( IO_CURVE_DTYPE, MIN_MAX_DTYPE, @@ -82,9 +86,9 @@ def _dest( def fill_link_defaults(table: pl.DataFrame) -> pl.DataFrame: """Add optional PyPSA link columns absent when all links share the PyPSA default.""" - float_defaults: list[tuple[str, float]] = [ + float_defaults: list[tuple[str, float | None]] = [ (PyPSALinkCol.P_NOM, 0.0), - (PyPSALinkCol.P_NOM_OPT, 0.0), + (PyPSALinkCol.P_NOM_OPT, None), (PyPSALinkCol.P_MIN_PU, 0.0), (PyPSALinkCol.P_MAX_PU, 1.0), (PyPSALinkCol.EFFICIENCY, 1.0), @@ -104,7 +108,7 @@ def fill_link_defaults(table: pl.DataFrame) -> pl.DataFrame: return table.with_columns( [ pl.col(PyPSALinkCol.P_NOM).fill_nan(0.0).fill_null(0.0), - pl.col(PyPSALinkCol.P_NOM_OPT).fill_nan(0.0).fill_null(0.0), + pl.col(PyPSALinkCol.P_NOM_OPT).fill_nan(None), pl.col(PyPSALinkCol.P_MIN_PU).fill_nan(0.0).fill_null(0.0), pl.col(PyPSALinkCol.P_MAX_PU).fill_nan(1.0).fill_null(1.0), pl.col(PyPSALinkCol.EFFICIENCY).fill_nan(1.0).fill_null(1.0), @@ -208,10 +212,8 @@ def _time_varying_flags(name: str, time_varying_owners: dict[str, set[str]]) -> # --- Limit / loss expressions --- -_effective_p_nom = ( - pl.when(pl.col(PyPSALinkCol.P_NOM_EXTENDABLE) & (pl.col(PyPSALinkCol.P_NOM_OPT) > 0)) - .then(pl.col(PyPSALinkCol.P_NOM_OPT)) - .otherwise(pl.col(PyPSALinkCol.P_NOM)) +_effective_p_nom = effective_p_nom( + PyPSALinkCol.P_NOM_EXTENDABLE, PyPSALinkCol.P_NOM_OPT, PyPSALinkCol.P_NOM ) _is_bidirectional = pl.col(PyPSALinkCol.P_MIN_PU) < 0 _from_min = ( @@ -228,7 +230,7 @@ def _time_varying_flags(name: str, time_varying_owners: dict[str, set[str]]) -> def _capacity_source(old: dict[str, Any]) -> tuple[str, float]: """The PyPSA capacity attribute actually used for the power limits, and its value.""" - if bool(old[PyPSALinkCol.P_NOM_EXTENDABLE]) and old[PyPSALinkCol.P_NOM_OPT] > 0: + if holds_solved_capacity(old, PyPSALinkCol.P_NOM_EXTENDABLE, PyPSALinkCol.P_NOM_OPT): return PyPSALinkCol.P_NOM_OPT, old[PyPSALinkCol.P_NOM_OPT] return PyPSALinkCol.P_NOM, old[PyPSALinkCol.P_NOM] diff --git a/interop/plugins/shared/pypsa_sienna_translations/_renewables.py b/interop/plugins/shared/pypsa_sienna_translations/_renewables.py index 7f963a4..ae6cfe9 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_renewables.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_renewables.py @@ -29,10 +29,13 @@ ) from interop.plugins.shared.pypsa_sienna_translations._prime_mover import enrich_prime_mover from interop.plugins.shared.pypsa_sienna_translations._shared import ( + EFFECTIVE_P_NOM, + EFFECTIVE_P_NOM_DERIVATION, pypsa_source_field, sienna_dest_field, ts_association_row, variable_cost_curve, + with_effective_p_nom, ) from interop.plugins.shared.pypsa_sienna_translations._ts_info import TimeSeriesInfo from interop.plugins.shared.pypsa_sienna_user_mappings import CarrierMappings @@ -67,7 +70,6 @@ # Enrichment column names added before translations; never written to destination table. _PRIME_MOVER_COL = "_prime_mover_raw" _TS_PTP = "_ts_ptp_pmax" -_EFFECTIVE_P_NOM = "_effective_p_nom" _source = partial(pypsa_source_field, PyPSAComponent.GENERATOR) @@ -81,20 +83,18 @@ def fill_renewable_defaults(table: pl.DataFrame) -> pl.DataFrame: table, [ (PyPSAGeneratorCol.P_NOM, 0.0), - (PyPSAGeneratorCol.P_NOM_OPT, 0.0), + (PyPSAGeneratorCol.P_NOM_OPT, None), (PyPSAGeneratorCol.P_MIN_PU, 0.0), (PyPSAGeneratorCol.P_MAX_PU, 1.0), (PyPSAGeneratorCol.MARGINAL_COST, 0.0), ], [(PyPSAGeneratorCol.P_NOM_EXTENDABLE, False)], ) - return table.with_columns( - pl.when( - pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE) & (pl.col(PyPSAGeneratorCol.P_NOM_OPT) > 0) - ) - .then(pl.col(PyPSAGeneratorCol.P_NOM_OPT)) - .otherwise(pl.col(PyPSAGeneratorCol.P_NOM)) - .alias(_EFFECTIVE_P_NOM) + return with_effective_p_nom( + table, + PyPSAGeneratorCol.P_NOM_EXTENDABLE, + PyPSAGeneratorCol.P_NOM_OPT, + PyPSAGeneratorCol.P_NOM, ) @@ -263,9 +263,9 @@ def _renewable_translations( direct( source_col=PyPSAGeneratorCol.P_NOM, dest_col=R.BASE_POWER, - expr=pl.col(_EFFECTIVE_P_NOM), + expr=pl.col(EFFECTIVE_P_NOM), unit=UNIT_MW, - derivation="p_nom_opt where an extendable component has one, else p_nom", + derivation=EFFECTIVE_P_NOM_DERIVATION, ), direct( source_col=PyPSAGeneratorCol.P_NOM, @@ -370,7 +370,7 @@ def _renewable_cost_loss(dest: DestinationFieldFactory) -> Translation: sienna_component=SiennaComponent.RENEWABLE_DISPATCH, id_note="assigned by 1-based row position in renewable generators DataFrame", sienna_type_derivation="renewable carrier -> RenewableDispatch", - active_power_expr=pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MIN_PU), + active_power_expr=pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MIN_PU), active_power_derivation="effective_p_nom * p_min_pu (initial dispatch = min operating point)", tail=[_renewable_reactive_power_limits(_dest), _renewable_cost(_dest)], ) @@ -380,7 +380,7 @@ def _renewable_cost_loss(dest: DestinationFieldFactory) -> Translation: sienna_component=SiennaComponent.RENEWABLE_NON_DISPATCH, id_note="assigned by 1-based row position in renewable non-dispatch generators DataFrame", sienna_type_derivation="renewable carrier -> RenewableNonDispatch", - active_power_expr=pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MAX_PU), + active_power_expr=pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAGeneratorCol.P_MAX_PU), active_power_derivation=( "effective_p_nom * p_max_pu (must-take initial dispatch = peak available)" ), diff --git a/interop/plugins/shared/pypsa_sienna_translations/_shared.py b/interop/plugins/shared/pypsa_sienna_translations/_shared.py index 8249828..9d5c41d 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_shared.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_shared.py @@ -34,6 +34,35 @@ PYPSA_TO_SIENNA = "pypsa-to-sienna" +# An enrichment column on the source table, which finalise() drops. +EFFECTIVE_P_NOM = "_effective_p_nom" + +EFFECTIVE_P_NOM_DERIVATION = "p_nom_opt where an extendable component has one, else p_nom" + + +def has_solved_capacity(extendable: str, opt: str) -> pl.Expr: + """True where a solve wrote a capacity for an extendable component. + + A solve that builds none of a component writes p_nom_opt 0; only a network no solve has + touched leaves the column out, which stages as null. + """ + return pl.col(extendable) & pl.col(opt).is_not_null() + + +def holds_solved_capacity(row: dict[str, Any], extendable: str, opt: str) -> bool: + """``has_solved_capacity`` for one row, so an event names the attribute the value came from.""" + return bool(row[extendable]) and row[opt] is not None + + +def effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr: + """The capacity to translate: the solved one where there is one, else the stated one.""" + return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom)) + + +def with_effective_p_nom(table: pl.DataFrame, extendable: str, opt: str, nom: str) -> pl.DataFrame: + return table.with_columns(effective_p_nom(extendable, opt, nom).alias(EFFECTIVE_P_NOM)) + + UNNAMED_CARRIER_NOTE = "the user mappings file names no such carrier" # Loads, generators, hydro units and storage units all leave a non-AC bus the same way. diff --git a/interop/plugins/shared/pypsa_sienna_translations/_storage.py b/interop/plugins/shared/pypsa_sienna_translations/_storage.py index a76608d..3c432dc 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_storage.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_storage.py @@ -29,10 +29,13 @@ ) from interop.plugins.shared.pypsa_sienna_translations._prime_mover import enrich_prime_mover from interop.plugins.shared.pypsa_sienna_translations._shared import ( + EFFECTIVE_P_NOM, + EFFECTIVE_P_NOM_DERIVATION, pypsa_skip_report, pypsa_source_field, sienna_dest_field, variable_cost_curve, + with_effective_p_nom, ) from interop.plugins.shared.pypsa_sienna_translations._ts_info import TimeSeriesInfo from interop.plugins.shared.pypsa_sienna_user_mappings import CarrierMappings @@ -67,7 +70,6 @@ ) _PRIME_MOVER_COL = "_prime_mover_raw" -_EFFECTIVE_P_NOM = "_effective_p_nom" _source = partial(pypsa_source_field, PyPSAComponent.STORAGE_UNIT) @@ -80,7 +82,7 @@ def fill_storage_defaults(table: pl.DataFrame) -> pl.DataFrame: table, [ (PyPSAStorageUnitCol.P_NOM, 0.0), - (PyPSAStorageUnitCol.P_NOM_OPT, 0.0), + (PyPSAStorageUnitCol.P_NOM_OPT, None), (PyPSAStorageUnitCol.P_MIN_PU, -1.0), (PyPSAStorageUnitCol.P_MAX_PU, 1.0), (PyPSAStorageUnitCol.MARGINAL_COST, 0.0), @@ -94,14 +96,11 @@ def fill_storage_defaults(table: pl.DataFrame) -> pl.DataFrame: (PyPSAStorageUnitCol.P_NOM_EXTENDABLE, False), ], ) - return table.with_columns( - pl.when( - pl.col(PyPSAStorageUnitCol.P_NOM_EXTENDABLE) - & (pl.col(PyPSAStorageUnitCol.P_NOM_OPT) > 0) - ) - .then(pl.col(PyPSAStorageUnitCol.P_NOM_OPT)) - .otherwise(pl.col(PyPSAStorageUnitCol.P_NOM)) - .alias(_EFFECTIVE_P_NOM) + return with_effective_p_nom( + table, + PyPSAStorageUnitCol.P_NOM_EXTENDABLE, + PyPSAStorageUnitCol.P_NOM_OPT, + PyPSAStorageUnitCol.P_NOM, ) @@ -156,10 +155,10 @@ def _skip_without_energy(_series: pl.LazyFrame | None) -> SkipRule: # Defaults to 0.0 when capacity is zero (e.g. unsolved extendable unit) to # avoid a division-by-zero that would clip to 1.0 (incorrectly "fully charged"). _initial_level = ( - pl.when(pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0) + pl.when(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0) .then( pl.col(PyPSAStorageUnitCol.STATE_OF_CHARGE_INITIAL) - / (pl.col(_EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS)) + / (pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS)) ) .otherwise(0.0) .clip(0.0, 1.0) @@ -318,8 +317,8 @@ def _min_max(max_expr: pl.Expr) -> pl.Expr: STORAGE_BASE_POWER = _direct( source_col=PyPSAStorageUnitCol.P_NOM, dest_col=S.BASE_POWER, - expr=pl.col(_EFFECTIVE_P_NOM), - derivation="p_nom_opt where an extendable component has one, else p_nom", + expr=pl.col(EFFECTIVE_P_NOM), + derivation=EFFECTIVE_P_NOM_DERIVATION, ) STORAGE_COST = Translation( diff --git a/tests/features/plexos_to_pypsa/generators.feature b/tests/features/plexos_to_pypsa/generators.feature index 3bf3eee..ef0f2ab 100644 --- a/tests/features/plexos_to_pypsa/generators.feature +++ b/tests/features/plexos_to_pypsa/generators.feature @@ -522,7 +522,7 @@ Feature: Translate PLEXOS generators into a PyPSA network Scenario: a plant that may also expand keeps the capacity it has as its floor Given a Plexos model - And the model contains generator "CoalUnit" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=2, Max Units Built=1, Build Cost=2000000, WACC=0.07" + And the model contains generator "CoalUnit" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=2, Max Units Built=1, Build Cost=2000000, WACC=0.07, Economic Life=30" And the model is saved as "inputs/expanding_plant.xml" When I run translate against "inputs/expanding_plant.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" generator "CoalUnit" is extendable @@ -534,17 +534,22 @@ Feature: Translate PLEXOS generators into a PyPSA network Given a Plexos model And the model contains generator "Unpriced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4" And the model contains generator "Undiscounted_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000" - And the model contains generator "Priced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000, WACC=0.07" + And the model contains generator "Everlasting_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000, WACC=0.07" + And the model contains generator "Priced_REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000, WACC=0.07, Economic Life=25" And the model is saved as "inputs/unpriced_candidate.xml" When I run translate against "inputs/unpriced_candidate.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" has no generator "Unpriced_REZ" # PyPSA annuitises an overnight cost with the discount rate, and refuses a network stating one without the other. And the PyPSA network "outputs/network.nc" has no generator "Undiscounted_REZ" + # PyPSA annuitises over the lifetime, and a lifetime of infinity prices the build as a perpetuity. + And the PyPSA network "outputs/network.nc" has no generator "Everlasting_REZ" And the PyPSA network "outputs/network.nc" generator "Priced_REZ" is extendable And the file "decisions.md" contains "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" And the file "decisions.md" contains "a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost over" - And the log contains "1 candidate generator(s) state no Build Cost, so each is left out" - And the log contains "1 candidate generator(s) state no WACC, so each is left out" + And the file "decisions.md" contains "a candidate with no Economic Life gives PyPSA no period to annuitise its Build Cost over" + And the log contains "1 candidate Generator(s) state no Build Cost, so each is left out" + And the log contains "1 candidate Generator(s) state no WACC, so each is left out" + And the log contains "1 candidate Generator(s) state no Economic Life, so each is left out" Scenario: a generator the model cannot build states nothing about expansion Given a Plexos model diff --git a/tests/features/plexos_to_pypsa/storage_units.feature b/tests/features/plexos_to_pypsa/storage_units.feature index 4b8e0e0..7885d48 100644 --- a/tests/features/plexos_to_pypsa/storage_units.feature +++ b/tests/features/plexos_to_pypsa/storage_units.feature @@ -522,6 +522,10 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr Then the PyPSA network "outputs/network.nc" storage unit "new_bat" is extendable And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "p_nom_min" is 0 And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "p_nom_max" is 300 + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "p_nom" is 300 + # Capacity is the energy beside one unit's Max Power, so the hours are one unit's 200 / 50. + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "max_hours" is 4 + And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "state_of_charge_initial" is 600 And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "overnight_cost" is 800000 And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "lifetime" is 20 And the PyPSA network "outputs/network.nc" storage unit "new_bat" attribute "discount_rate" is 0.07 @@ -547,6 +551,7 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And generator "phs_grow" has property "Max Units Built" 1 And generator "phs_grow" has property "Build Cost" 1500000 And generator "phs_grow" has property "WACC" 0.07 + And generator "phs_grow" has property "Economic Life" 40 And the model is saved as "inputs/growing_phs.xml" When I run translate against "inputs/growing_phs.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" storage unit "phs_grow" is extendable @@ -564,6 +569,7 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And generator "phs_new" has property "Max Units Built" 3 And generator "phs_new" has property "Build Cost" 1500000 And generator "phs_new" has property "WACC" 0.07 + And generator "phs_new" has property "Economic Life" 40 And the model is saved as "inputs/new_phs.xml" When I run translate against "inputs/new_phs.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" storage unit "phs_new" is extendable diff --git a/tests/features/pypsa_to_sienna/thermal_generators.feature b/tests/features/pypsa_to_sienna/thermal_generators.feature index 05af39a..d62fc80 100644 --- a/tests/features/pypsa_to_sienna/thermal_generators.feature +++ b/tests/features/pypsa_to_sienna/thermal_generators.feature @@ -187,3 +187,17 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn And a user mappings file with all standard carriers When I run translate against "inputs/gen_not_extendable.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" Then the file "outputs/extensions.json" parses as JSON generator extension record for "nuclear_1" having "p_nom_extendable" set to false + + Scenario: a solve that built none of an extendable generator rates it at nothing, not at p_nom + Given a PyPSA network + And the network contains bus "bus_1" carrier "AC" v_nom 380.0 + And the network contains generator "built_ccgt" on "bus_1" carrier "CCGT" p_nom 100.0 p_nom_extendable True + And generator "built_ccgt" has p_nom_opt 400 + And the network contains generator "rejected_ccgt" on "bus_1" carrier "CCGT" p_nom 500.0 p_nom_extendable True + And generator "rejected_ccgt" has p_nom_opt 0 + And the network is saved as "inputs/solved_expansion.nc" + And a user mappings file with all standard carriers + When I run translate against "inputs/solved_expansion.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" + Then the file "outputs/system.json" parses as JSON with component "ThermalStandard" named "built_ccgt" having "base_power" set to 400.0 + # The plan refused this build, so its p_nom is not capacity the operations system may dispatch. + And the file "outputs/system.json" parses as JSON with component "ThermalStandard" named "rejected_ccgt" having "base_power" set to 0.0 From bba62e4d8b5f286bd4449bf998d1f76cc6b4b438 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 19:17:00 +0000 Subject: [PATCH 06/17] Rate a Battery by every unit it runs, and keep a candidate out of a dispatch system PLEXOS states a Battery's Max Power per unit, as the mapping document's own account of what a candidate is already said, so a Battery running two units now reaches PyPSA at twice that power rather than once. The units-out trace derates p_max_pu by units_out / units, so p_nom has to stand for every unit before that derate means anything. A Battery stating Units 0 and no Max Units Built now works out to no rated power and is left out, with the reason recorded and warned about, as a turbine with no units already was. (r3-01, r3-08) The pypsa-to-sienna leg leaves out an extendable component no solve has sized. PyPSA ignores the p_nom of an extendable component, and the network the PLEXOS leg writes puts the capacity a candidate may build there, so reading it back gave a Sienna operations system a plant nobody has built. A component a solve has sized still translates at its p_nom_opt. (r3-06) One carrier now reports every PLEXOS skip. SkippedComponent moves to decisions.py and carries the group a warning speaks for, so the unpriced-candidate rule needs neither a return type of its own nor a field of its own, and every generator skip warns as a storage skip already did. (r3-02) Deletes SourceGenerator.existing, which nothing read (r3-11), and two docstrings that restated the line below them (r3-09, r3-10). The mapping documents now say which reading of Max Power is correct, that a turbine's p_nom is Max Capacity times Units and nothing else, that a candidate stating Units 0 is written rather than skipped, and that a candidate does not reach a Sienna operations system. (r3-13, r3-14) The scenarios follow the behaviour. The battery p_nom event cites Units beside Max Power. The three scenarios asserting that p_nom_extendable round-trips into extensions.json give their component a p_nom_opt, since a network no solve has touched no longer translates its candidates. The infeasible-dispatch warning is now the note the skip records, which says "sits above" where the warning it replaces said "is above". Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 9 ++- .../translation-from-plexos-to-sienna.md | 6 +- .../translation-from-pypsa-to-sienna.md | 2 +- .../plexos_pypsa_translations/_batteries.py | 17 ++--- .../plexos_pypsa_translations/_expansion.py | 62 +++++-------------- .../_generator_derivation.py | 6 -- .../plexos_pypsa_translations/_generators.py | 43 +++++-------- .../_storage_hydro.py | 2 +- .../_storage_shared.py | 30 +-------- .../_storage_units.py | 11 +--- .../plexos_pypsa_translations/decisions.py | 52 +++++++++++++++- .../pypsa_sienna_translations/__init__.py | 3 +- .../pypsa_sienna_translations/_shared.py | 47 +++++++++++--- .../steps/pypsa_to_sienna_map_components.py | 15 +++++ .../plexos_to_pypsa/generators.feature | 2 +- .../plexos_to_pypsa/storage_units.feature | 28 ++++++++- .../pypsa_to_sienna/renewables.feature | 1 + .../features/pypsa_to_sienna/storage.feature | 1 + .../thermal_generators.feature | 12 ++++ 19 files changed, 208 insertions(+), 141 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index 4015428..d69d875 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -353,7 +353,7 @@ has more than one fuel uses its primary fuel. | `name` | | `Battery.name` | `direct` | | `bus` | | The `Node` of the battery | `direct` | | `carrier` | | `battery` | `default` | -| `p_nom` | MW | `Max Power` | `direct` | +| `p_nom` | MW | `Max Power × Units` | `derived` | | `max_hours` | h | The energy capacity divided by `p_nom` | `derived` | | `p_max_pu` / `p_min_pu` | | `1.0` / `-1.0` | `default` | | `efficiency_store` / `efficiency_dispatch` | | `√(Charge Efficiency)` for each | `derived` | @@ -375,7 +375,10 @@ SoC` are `dropped`, and the full energy capacity is available. A `Battery` or a turbine can have a rated power of zero. For example, `Units 0` puts a unit into storage. Such a unit cannot dispatch. The translator does not write it, and it makes a -`COMPONENT_SKIPPED` event that gives the name of the unit. +`COMPONENT_SKIPPED` event that gives the name of the unit. An object that states `Units 0` +beside a `Max Units Built` is a candidate rather than a unit in storage: it takes the +capacity it may build as its `p_nom`, and the translator writes it. Refer to +[What a candidate is](#what-a-candidate-is). ## Pumped storage → `StorageUnit` @@ -387,7 +390,7 @@ name of the turbine is the name of that `StorageUnit`. | `name` | | The `Generator.name` of the turbine | `direct` | | `bus` | | The `Node` of the turbine | `direct` | | `carrier` | | `PHS` | `default` | -| `p_nom` | MW | `Max Capacity × Units`, or the static `Rating` where that is higher, or the peak of the profile that supplies the capacity | `derived` | +| `p_nom` | MW | `Max Capacity × Units` | `derived` | | `max_hours` | h | The `Max Volume` of the head reservoir divided by `p_nom`, if the model gives that volume in MWh | `derived` | | `p_max_pu` / `p_min_pu` | | `1.0` / `-1.0` | `default` | | `efficiency_store` / `efficiency_dispatch` | | `√(Pump Efficiency)` for each | `derived` | diff --git a/docs/translation_mappings/translation-from-plexos-to-sienna.md b/docs/translation_mappings/translation-from-plexos-to-sienna.md index 25dbe85..8a13338 100644 --- a/docs/translation_mappings/translation-from-plexos-to-sienna.md +++ b/docs/translation_mappings/translation-from-plexos-to-sienna.md @@ -55,6 +55,9 @@ property of the PLEXOS to Sienna mapping. [The carrier mappings file](#the-carrier-mappings-file). - **A component whose carrier your file does not name is left out.** The run completes and `decisions.md` names each one. +- **A candidate is not a plant.** A `Generator`, a `Battery` or a turbine that states + `Max Units Built` is a build the plan has yet to decide, and a dispatch system has no + capacity for one, so it is left out and `decisions.md` names it. - **One scenario only.** The Model you select applies its own Scenario overlays. The translator reads no other scenario. - **One calendar year at a time.** The year you give narrows the Horizon of the Model. Every @@ -263,13 +266,14 @@ An availability that changes with time becomes a `TimeSeriesAssociation` on `max_active_power`. That is the name PowerSimulations reads an availability forecast under. The stored shape peaks at 1.0 and scales back to MW through `active_power_limits.max`. -**The translator does not translate four cases.** It records each one as a skipped +**The translator does not translate five cases.** It records each one as a skipped component: | Case | Cause | | --- | --- | | The generator has no `Node` | There is no bus to connect it to. | | The generator has no `Units` at any time in the horizon | The unit is retired. | +| The generator states `Max Units Built` | It is a candidate to build, not capacity to dispatch. | | `Max Capacity` comes from a data file | There is no single capacity to divide the per-unit fields by. | | The capacity is 0 | The generator can never dispatch. | diff --git a/docs/translation_mappings/translation-from-pypsa-to-sienna.md b/docs/translation_mappings/translation-from-pypsa-to-sienna.md index a1db7a4..b731c11 100644 --- a/docs/translation_mappings/translation-from-pypsa-to-sienna.md +++ b/docs/translation_mappings/translation-from-pypsa-to-sienna.md @@ -81,7 +81,7 @@ This document targets the SiennaSchemas **Operations** namespace (the dispatchab **Implication for this translation.** PyPSA's capacity-expansion fields (`p_nom_extendable`, `capital_cost`, `p_nom_min`/`p_nom_max`, `build_year`, `lifetime`) belong to **Investments**, which is a separate translation target (PyPSA expansion → SiennaSchemas Investments), not extra fields on an Operations component. It is **out of scope for v1**. For v1 we translate the **solved** fleet: take `p_nom_opt` (post-solve capacities) as fixed Operations capacity and drop the expansion parameters (or keep round-trip crumbs in the `ext` sidecar). A genuine expansion problem is the future Investments path. -Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True` and the network states a `p_nom_opt`, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`. A solve that builds none of an extendable component writes `p_nom_opt` 0, and that 0 is the capacity to translate: the plan rejected the build. Only a network no solve has touched leaves `p_nom_opt` out of the file altogether, and there `effective_p_nom` falls back to `p_nom`. +Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True` and the network states a `p_nom_opt`, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`. A solve that builds none of an extendable component writes `p_nom_opt` 0, and that 0 is the capacity to translate: the plan rejected the build. Only a network no solve has touched leaves `p_nom_opt` out of the file altogether, and an extendable component there is a candidate rather than a plant: the generator, renewable, hydro and storage mappings leave it out and record it as skipped. **Why Operations and not Investments.** This is a deliberate scope choice tied to the deliverable, not a claim that Operations is the more natural fit: diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index f1d198a..bf3219e 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -20,7 +20,6 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( - NOTHING_BUILT_DERIVATION, RatedCapacity, derive_expansion, ) @@ -34,7 +33,6 @@ MappedOrSkipped, RatedObject, RatedPower, - SkippedComponent, StagedObject, StorageLookups, StorageUnitMapping, @@ -58,6 +56,7 @@ ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( Decision, + SkippedComponent, SourceValue, ) from interop.plugins.shared.pypsa_constants import ( @@ -94,6 +93,7 @@ def record_battery_outages(state: State, mappings: list[StorageUnitMapping]) -> _BATTERY_RECYCLE_DERIVATION = "End Effects Method recycles the level" _CAPACITY_DERIVATION = "Capacity" _CAPACITY_FROM_DURATION_DERIVATION = "Duration * Max Power" +_P_NOM_FROM_UNITS_DERIVATION = "Max Power * Units" _SOC_FROM_PERCENT_DERIVATION = "Initial SoC / 100 * energy capacity" _PER_MAX_POWER_DERIVATION = " / Max Power" @@ -110,16 +110,17 @@ def map_battery(name: str, lookups: StorageLookups) -> MappedOrSkipped: def _battery_rating(staged: StagedObject) -> RatedCapacity: """A PLEXOS Battery states the power of one of its units as its Max Power.""" max_power = staged.properties[PlexosProperty.MAX_POWER] + units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS) power = SourceValue( PlexosClass.BATTERY, staged.name, PlexosProperty.MAX_POWER, max_power, UNIT_MW ) - unit_size = Decision.derived(max_power, [power], DIRECT_DERIVATION) - units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS) - if units: - return RatedCapacity(existing=unit_size, unit_size=unit_size) counted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units) - nothing_built = Decision.derived(0.0, [power, counted], NOTHING_BUILT_DERIVATION) - return RatedCapacity(existing=nothing_built, unit_size=unit_size) + return RatedCapacity( + existing=Decision.derived( + max_power * units, [power, counted], _P_NOM_FROM_UNITS_DERIVATION + ), + unit_size=Decision.derived(max_power, [power], DIRECT_DERIVATION), + ) _BATTERY_POWER = RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER, _battery_rating) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py index b6dd181..f145d36 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -7,8 +7,6 @@ from __future__ import annotations -import logging -from collections.abc import Sequence from dataclasses import dataclass from interop.plugins.shared.constants import ( @@ -29,18 +27,15 @@ ComponentReporter, Decision, MappedColumns, + SkipGroup, + SkippedComponent, SourceValue, maps_to, ) from interop.plugins.shared.pypsa_constants import PyPSAGeneratorCol -from interop.plugins.shared.warning_text import name_a_few - -log = logging.getLogger(__name__) NOTHING_TO_REPORT = Decision.unreported(None) -NOTHING_BUILT_DERIVATION = "Units is zero, so none of the rated power is built yet" - P_NOM_CANDIDATE_DERIVATION = ( "the object has no units yet, so its nominal power is the capacity it may build: " "one unit's rated power x Max Units Built" @@ -203,49 +198,26 @@ class UnpricedBuild: ) -@dataclass(frozen=True) -class UnpricedCandidate: - """One candidate left out, and the property it left out that priced its build.""" - - plexos_class: PlexosClass - name: str - unpriced: UnpricedBuild - - @property - def source(self) -> SourceValue: - return SourceValue( - self.plexos_class, self.name, self.unpriced.plexos_property, None, self.unpriced.unit - ) - - @property - def note(self) -> str: - return self.unpriced.note - - def find_unpriced_build( plexos_class: PlexosClass, name: str, props: dict[str, float] -) -> UnpricedCandidate | None: - """The first property a candidate leaves out that stops PyPSA pricing its build.""" +) -> SkippedComponent | None: + """The first property a candidate leaves out that stops PyPSA pricing its build. + + Every candidate leaving out the same property is named in one warning rather than a + line each, because a model that omits one of these omits it wholesale. + """ if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD: return None unpriced = next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None) - return None if unpriced is None else UnpricedCandidate(plexos_class, name, unpriced) - - -def warn_unpriced_builds(candidates: Sequence[UnpricedCandidate]) -> None: - """One warning per class and property a candidate left out, naming a few of the objects.""" - by_property: dict[tuple[str, str], list[str]] = {} - for one in candidates: - key = (one.plexos_class, one.unpriced.plexos_property) - by_property.setdefault(key, []).append(one.name) - for (plexos_class, plexos_property), names in sorted(by_property.items()): - log.warning( - "plexos: %d candidate %s(s) state no %s, so each is left out. Each one: %s", - len(names), - plexos_class, - plexos_property, - name_a_few(sorted(names)), - ) + if unpriced is None: + return None + return SkippedComponent( + source=SourceValue(plexos_class, name, unpriced.plexos_property, None, unpriced.unit), + note=unpriced.note, + warn_with=SkipGroup( + counted=f"candidate {plexos_class}(s)", reason=f"state no {unpriced.plexos_property}" + ), + ) def record_expansion_extensions( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py index cee7c87..05cda77 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py @@ -127,12 +127,6 @@ def rating_as_capacity(self) -> float | None: rating = _optional(self.props, PlexosProperty.RATING) return rating if rating is not None and rating > self.nameplate else None - @property - def existing(self) -> float: - """The capacity the generator already has, before anything is built.""" - capacity = self.rating_as_capacity - return self.nameplate if capacity is None else capacity - @property def p_nom(self) -> float: """What the generator has, or what it may build where it has nothing yet.""" diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index 2bb449e..1c7b574 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -23,11 +23,9 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( - UnpricedCandidate, find_unpriced_build, read_sidecar_value, record_expansion_extensions, - warn_unpriced_builds, ) from interop.plugins.shared.plexos_pypsa_translations._generator_decisions import ( GeneratorDecisions, @@ -56,8 +54,10 @@ ComponentReporter, Decision, MappedColumns, + SkippedComponent, SourceValue, destination_row, + warn_about_skips, ) from interop.plugins.shared.pypsa_constants import ( GENERATORS_DESTINATION_SCHEMA, @@ -105,12 +105,14 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: reporter = ComponentReporter(recorder, PyPSAComponent.GENERATOR) storage_turbines = storage_turbine_names(state) outcomes = [ - one + _map_one(generator, lookups, reporter) for generator in generators.collect().iter_rows(named=True) if generator[PlexosObjectCol.NAME] not in storage_turbines - if (one := _map_one(generator, lookups, reporter)) is not None ] - warn_unpriced_builds([one for one in outcomes if isinstance(one, UnpricedCandidate)]) + skipped = [one for one in outcomes if isinstance(one, SkippedComponent)] + for one in skipped: + reporter.record_skipped(one.source, one.note) + warn_about_skips(skipped) translated = [one for one in outcomes if isinstance(one, _TranslatedGenerator)] if not translated: return @@ -165,57 +167,42 @@ def _category_decision(mapping: GeneratorMapping) -> Decision: def _map_one( generator: dict[str, Any], lookups: Lookups, reporter: ComponentReporter -) -> _TranslatedGenerator | UnpricedCandidate | None: +) -> _TranslatedGenerator | SkippedComponent: name = generator[PlexosObjectCol.NAME] node = lookups.gen_to_node.get(name) if node is None: - reporter.record_skipped(_source(name, PlexosCollection.NODES, None), _NO_BUS_NOTE) - return None + return SkippedComponent(_source(name, PlexosCollection.NODES, None), _NO_BUS_NOTE) if PlexosProperty.MAX_CAPACITY in lookups.file_backed_properties.get(name, []): - reporter.record_skipped( + return SkippedComponent( _source(name, PlexosProperty.MAX_CAPACITY, _DATA_FILE, UNIT_MW), _FILE_BACKED_NOTE ) - return None source = read_source(generator, name, lookups) if source.units == 0.0 and not source.is_candidate: - reporter.record_skipped(_source(name, PlexosProperty.UNITS, source.units), _RETIRED_NOTE) - return None + return SkippedComponent(_source(name, PlexosProperty.UNITS, source.units), _RETIRED_NOTE) if source.p_nom <= 0.0: - reporter.record_skipped( + return SkippedComponent( _source(name, PlexosProperty.MAX_CAPACITY, None, UNIT_MW), f"generator dropped: p_nom is {source.p_nom} MW, so it can never dispatch", ) - return None unpriced = find_unpriced_build(PlexosClass.GENERATOR, name, source.props) if unpriced is not None: - reporter.record_skipped(unpriced.source, unpriced.note) return unpriced mapping = derive_generator(source, node, lookups) if has_infeasible_dispatch_range(mapping): - _report_infeasible_dispatch_range(mapping, reporter) - return None + return _infeasible_dispatch_range(mapping) decisions = decide_generator(mapping) record_generator(reporter, decisions) return _TranslatedGenerator(mapping, decisions) -def _report_infeasible_dispatch_range( - mapping: GeneratorMapping, reporter: ComponentReporter -) -> None: +def _infeasible_dispatch_range(mapping: GeneratorMapping) -> SkippedComponent: p_min_pu = mapping.minimum.p_min_pu p_max_pu = mapping.availability.static_p_max_pu - reporter.record_skipped( + return SkippedComponent( _source(mapping.name, mapping.minimum.source_property, mapping.minimum.source_value), f"p_min_pu {p_min_pu} sits above p_max_pu {p_max_pu}, which PyPSA cannot dispatch, " "so the generator is dropped", ) - log.warning( - "plexos: dropping Generator %r: p_min_pu %s is above p_max_pu %s, which PyPSA " - "cannot dispatch", - mapping.name, - p_min_pu, - p_max_pu, - ) def _source( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index 211fd26..d11c92b 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -37,7 +37,6 @@ MappedOrSkipped, RatedObject, RatedPower, - SkippedComponent, StagedObject, StorageLookups, StorageUnitMapping, @@ -62,6 +61,7 @@ ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( Decision, + SkippedComponent, SourceValue, ) from interop.plugins.shared.pypsa_constants import ( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index a0aa1f0..368c2f2 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -8,7 +8,6 @@ from __future__ import annotations -import logging from collections.abc import Callable from dataclasses import dataclass from math import sqrt @@ -31,7 +30,6 @@ CandidateSource, ExpansionDecisions, RatedCapacity, - UnpricedCandidate, derive_p_nom, find_unpriced_build, ) @@ -50,6 +48,7 @@ ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( Decision, + SkippedComponent, SourceValue, holds, maps_to, @@ -62,8 +61,6 @@ series_components, ) -log = logging.getLogger(__name__) - NO_RESERVOIR_INFLOW_NOTE = "this unit has no reservoir, so nothing flows into it" MAX_HOURS_NOTE = "PLEXOS states no reservoir capacity; max_hours uses the PyPSA default" @@ -124,29 +121,6 @@ class StorageUnitMapping: inflow_storage: str | None = None -def warn_about_skipped(skipped: SkippedComponent) -> None: - """A skip is recorded per component, and warned about, so neither view alone hides it.""" - log.warning( - "plexos: dropping %s %r: %s", - skipped.source.component, - skipped.source.name, - skipped.note, - ) - - -@dataclass(frozen=True) -class SkippedComponent: - """A PLEXOS object the mapping deliberately did not translate, and why. - - ``unpriced`` is set where the expansion rule left the object out, since that rule warns - about all of them together rather than one line each. - """ - - source: SourceValue - note: str - unpriced: UnpricedCandidate | None = None - - # Every storage object ends as one or the other: a unit to write, or a recorded reason not to. MappedOrSkipped = StorageUnitMapping | SkippedComponent @@ -371,7 +345,7 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp return _skipped_without_capacity(rating, staged.name) unpriced = find_unpriced_build(rating.plexos_class, staged.name, staged.properties) if unpriced is not None: - return SkippedComponent(unpriced.source, unpriced.note, unpriced) + return unpriced candidate = CandidateSource( rating.plexos_class, staged.name, diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index a74d08b..1ab2412 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -30,7 +30,6 @@ from interop.plugins.shared.plexos_pypsa_translations._expansion import ( read_sidecar_value, record_expansion_extensions, - warn_unpriced_builds, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, @@ -41,21 +40,21 @@ ) from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( MappedOrSkipped, - SkippedComponent, StorageLookups, StorageUnitMapping, build_lookups, orphan_storage_skips, read_object_names, - warn_about_skipped, ) from interop.plugins.shared.plexos_pypsa_translations._storage_turbines import ( storage_turbine_names, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( ComponentReporter, + SkippedComponent, SourceValue, destination_row, + warn_about_skips, ) from interop.plugins.shared.pypsa_constants import ( STORAGE_UNITS_DESTINATION_SCHEMA, @@ -231,11 +230,7 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No record_expansion_extensions(mapping.name, mapping.expansion, reporter) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) - if skipped.unpriced is None: - warn_about_skipped(skipped) - warn_unpriced_builds( - [skipped.unpriced for skipped in storage_units.skipped if skipped.unpriced is not None] - ) + warn_about_skips(storage_units.skipped) for dropped in storage_units.dropped: reporter.record_dropped(dropped.source, dropped.note) diff --git a/interop/plugins/shared/plexos_pypsa_translations/decisions.py b/interop/plugins/shared/plexos_pypsa_translations/decisions.py index b92188a..52f8419 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/decisions.py @@ -12,16 +12,20 @@ from __future__ import annotations +import logging from collections.abc import Iterator, Sequence from dataclasses import dataclass, field, fields from enum import Enum, auto -from typing import Any +from typing import Any, NamedTuple from interop.core.reporting import ScopedRecorder from interop.plugins.shared.constants import Framework from interop.plugins.shared.framework_reporting import DestinationReporter +from interop.plugins.shared.warning_text import name_a_few from interop.ports.outbound.reporting import SourceField +log = logging.getLogger(__name__) + _MAPPED_COLUMNS = "mapped_columns" _NESTED_MAPPING = "nested_mapping" @@ -56,6 +60,52 @@ def derived_earlier( return cls(component, name, attribute, value, unit, framework=Framework.PYPSA) +class SkipGroup(NamedTuple): + """The words one warning speaks with for every skip that shares a reading.""" + + counted: str + """What stands where "Generator(s)" does, already plural.""" + + reason: str + """Completes "N , so each is left out".""" + + +@dataclass(frozen=True) +class SkippedComponent: + """A PLEXOS object the mapping deliberately did not translate, and why. + + ``warn_with`` is set where one warning speaks for every object a reading left out, and + left unset where the object gets a line of its own. + """ + + source: SourceValue + note: str + warn_with: SkipGroup | None = None + + +def warn_about_skips(skipped: Sequence[SkippedComponent]) -> None: + """One line for each object that stands alone, and one line for each group. + + A skip is recorded per component and warned about, so neither view alone hides it. + """ + grouped: dict[SkipGroup, list[str]] = {} + for one in skipped: + if one.warn_with is None: + log.warning( + "plexos: dropping %s %r: %s", one.source.component, one.source.name, one.note + ) + else: + grouped.setdefault(one.warn_with, []).append(one.source.name) + for group, names in sorted(grouped.items()): + log.warning( + "plexos: %d %s %s, so each is left out. Each one: %s", + len(names), + group.counted, + group.reason, + name_a_few(sorted(names)), + ) + + class DecisionKind(Enum): """What a destination value is: read from the source, worked out, or the translator's.""" diff --git a/interop/plugins/shared/pypsa_sienna_translations/__init__.py b/interop/plugins/shared/pypsa_sienna_translations/__init__.py index d87fdf1..f97edf2 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/__init__.py +++ b/interop/plugins/shared/pypsa_sienna_translations/__init__.py @@ -57,7 +57,7 @@ enrich_renewable_ts_stats, fill_renewable_defaults, ) -from ._shared import ScopeSkips, carrier_scope_skips +from ._shared import ScopeSkips, carrier_scope_skips, unbuilt_candidate_skip from ._storage import ( ENERGY_RESERVOIR_STORAGE_TRANSLATIONS, PHS_STORAGE_MAPPING, @@ -72,6 +72,7 @@ "choose_ensemble_samples", "ScopeSkips", "carrier_scope_skips", + "unbuilt_candidate_skip", "THERMAL_MAPPING", "RENEWABLE_DISPATCH_MAPPING", "RENEWABLE_NON_DISPATCH_MAPPING", diff --git a/interop/plugins/shared/pypsa_sienna_translations/_shared.py b/interop/plugins/shared/pypsa_sienna_translations/_shared.py index 9d5c41d..666cfa1 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_shared.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_shared.py @@ -26,7 +26,7 @@ SiennaVariableCostType, time_series_uuid, ) -from interop.plugins.shared.translation_runner import SkippedNames, SkipReport +from interop.plugins.shared.translation_runner import SkippedNames, SkipReport, SkipRule from interop.ports.outbound.reporting import ( DestinationField, SourceField, @@ -34,17 +34,16 @@ PYPSA_TO_SIENNA = "pypsa-to-sienna" -# An enrichment column on the source table, which finalise() drops. +# Enrichment columns on the source table, which finalise() drops. EFFECTIVE_P_NOM = "_effective_p_nom" +STATES_BUILT_CAPACITY = "_states_built_capacity" EFFECTIVE_P_NOM_DERIVATION = "p_nom_opt where an extendable component has one, else p_nom" def has_solved_capacity(extendable: str, opt: str) -> pl.Expr: - """True where a solve wrote a capacity for an extendable component. - - A solve that builds none of a component writes p_nom_opt 0; only a network no solve has - touched leaves the column out, which stages as null. + """A solve that builds none of a component writes p_nom_opt 0; only a network no solve + has touched leaves the column out, which stages as null. """ return pl.col(extendable) & pl.col(opt).is_not_null() @@ -55,12 +54,23 @@ def holds_solved_capacity(row: dict[str, Any], extendable: str, opt: str) -> boo def effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr: - """The capacity to translate: the solved one where there is one, else the stated one.""" return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom)) +def states_built_capacity(extendable: str, opt: str) -> pl.Expr: + """Whether the component has capacity an operations model may dispatch. + + An extendable component no solve has sized states its capacity as a build the plan has + yet to decide, and PyPSA ignores the p_nom of an extendable component altogether. + """ + return ~pl.col(extendable) | pl.col(opt).is_not_null() + + def with_effective_p_nom(table: pl.DataFrame, extendable: str, opt: str, nom: str) -> pl.DataFrame: - return table.with_columns(effective_p_nom(extendable, opt, nom).alias(EFFECTIVE_P_NOM)) + return table.with_columns( + effective_p_nom(extendable, opt, nom).alias(EFFECTIVE_P_NOM), + states_built_capacity(extendable, opt).alias(STATES_BUILT_CAPACITY), + ) UNNAMED_CARRIER_NOTE = "the user mappings file names no such carrier" @@ -117,6 +127,27 @@ def carrier_scope_skips(naming: PyPSAComponentNaming) -> ScopeSkips: ) +UNBUILT_CANDIDATE_REASON = "are extendable and no solve has sized them" +UNBUILT_CANDIDATE_NOTE = ( + "p_nom_extendable is true and the network states no p_nom_opt, so this is capacity the " + "plan may build rather than capacity an operations model may dispatch" +) + + +def unbuilt_candidate_skip(naming: PyPSAComponentNaming) -> SkipRule: + """The drop for a candidate an operations system must not read as a built plant.""" + return SkipRule( + keep=pl.col(STATES_BUILT_CAPACITY), + report=pypsa_skip_report( + component=naming.display, + name_col=PyPSAComponentCol.NAME, + counted_noun=naming.plural, + reason=UNBUILT_CANDIDATE_REASON, + note=UNBUILT_CANDIDATE_NOTE, + ), + ) + + def pypsa_source_field( component: str, name: str, diff --git a/interop/plugins/steps/pypsa_to_sienna_map_components.py b/interop/plugins/steps/pypsa_to_sienna_map_components.py index 52a5053..1cd8152 100644 --- a/interop/plugins/steps/pypsa_to_sienna_map_components.py +++ b/interop/plugins/steps/pypsa_to_sienna_map_components.py @@ -66,6 +66,7 @@ link_time_varying_owners, load_in_scope, load_is_interruptible, + unbuilt_candidate_skip, ) from interop.plugins.shared.pypsa_sienna_user_mappings import CarrierMappings from interop.plugins.shared.sienna_constants import ( @@ -307,6 +308,7 @@ def _prepare_source( carriers = self._carrier_mappings.get_carriers(mapping.sienna_component) table = mapping.fill_defaults(source) table = table.filter(pl.col(mapping.carrier_col).is_in(list(carriers))) + table = self._drop_unbuilt_candidates(table, mapping) series = self._source_time_series(state, mapping) if mapping.skip is not None: rule = mapping.skip(series) @@ -318,6 +320,19 @@ def _prepare_source( ts_info=ts_info, ) + def _drop_unbuilt_candidates( + self, table: pl.DataFrame, mapping: ComponentMapping + ) -> pl.DataFrame: + """Leave out the candidates, which a Sienna operations system has no capacity for. + + A component a solve has sized carries the capacity it built, and one whose capacity is + fixed carries what it has. Anything else is a build the plan has yet to decide, and + writing it here would give the operations system a plant nobody has built. + """ + rule = unbuilt_candidate_skip(PYPSA_COMPONENT_NAMING[mapping.source_table]) + kept, _ = filter_component(table, rule.keep, rule.report, self._recorder) + return kept + def _add_derived_series( self, state: State, mapping: ComponentMapping, prepared: _PreparedSource ) -> None: diff --git a/tests/features/plexos_to_pypsa/generators.feature b/tests/features/plexos_to_pypsa/generators.feature index ef0f2ab..47b65c4 100644 --- a/tests/features/plexos_to_pypsa/generators.feature +++ b/tests/features/plexos_to_pypsa/generators.feature @@ -135,7 +135,7 @@ Feature: Translate PLEXOS generators into a PyPSA network When I run translate against "inputs/infeasible.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" has no generator "InfeasiblePlant" And the log contains "dropping Generator 'InfeasiblePlant'" - And the log contains "p_min_pu 0.5 is above p_max_pu 0.4" + And the log contains "p_min_pu 0.5 sits above p_max_pu 0.4" And the file "decisions.md" contains "| `plexos.Generator.InfeasiblePlant.Min Stable Factor` = 50.0 | | | p_min_pu 0.5 sits above p_max_pu 0.4, which PyPSA cannot dispatch, so the generator is dropped |" Scenario: a non-fuel dispatchable generator gets a flat cost from its category diff --git a/tests/features/plexos_to_pypsa/storage_units.feature b/tests/features/plexos_to_pypsa/storage_units.feature index 7885d48..cfaeb30 100644 --- a/tests/features/plexos_to_pypsa/storage_units.feature +++ b/tests/features/plexos_to_pypsa/storage_units.feature @@ -25,9 +25,35 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And the PyPSA network "outputs/network.nc" storage unit "bat_1" attribute "state_of_charge_initial" is 100.0 And the PyPSA network "outputs/network.nc" storage unit "bat_1" attribute "marginal_cost" is 0.0 And the PyPSA network "outputs/network.nc" storage unit "bat_1" is not cyclic - And the file "decisions.md" contains "| `plexos.Battery.bat_1.Max Power` = 100.0 MW | `pypsa.StorageUnit.bat_1.p_nom` = 100.0 MW | direct | | plexos-to-pypsa | plexos_to_pypsa_map_storage_units |" + And the file "decisions.md" contains "| `plexos.Battery.bat_1.Max Power` = 100.0 MW
`plexos.Battery.bat_1.Units` = 1.0 | `pypsa.StorageUnit.bat_1.p_nom` = 100.0 MW | Max Power * Units | | plexos-to-pypsa | plexos_to_pypsa_map_storage_units |" And the file "decisions.md" contains "| `plexos.Battery.bat_1.Charge Efficiency` = 81.0 % | `pypsa.StorageUnit.bat_1.efficiency_store` = 0.9 | sqrt(round-trip / 100), split symmetrically | | plexos-to-pypsa | plexos_to_pypsa_map_storage_units |" + Scenario: a Battery of several units is rated at the power of all of them + Given a Plexos model + And the model contains region "West" + And the model contains node "node_u" in region "West" + And the model contains battery "bat_pair" on node "node_u" with max_power 50 capacity 200 charge_efficiency 81 initial_soc 50 + And battery "bat_pair" has property "Units" 2 + And the model is saved as "inputs/battery_units.xml" + When I run translate against "inputs/battery_units.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" storage unit "bat_pair" attribute "p_nom" is 100.0 + # Max Power and Capacity both describe one unit, so the hours are one unit's 200 / 50. + And the PyPSA network "outputs/network.nc" storage unit "bat_pair" attribute "max_hours" is 4.0 + And the PyPSA network "outputs/network.nc" storage unit "bat_pair" attribute "state_of_charge_initial" is 200.0 + And the file "decisions.md" contains "| `plexos.Battery.bat_pair.Max Power` = 50.0 MW
`plexos.Battery.bat_pair.Units` = 2.0 | `pypsa.StorageUnit.bat_pair.p_nom` = 100.0 MW | Max Power * Units | | plexos-to-pypsa | plexos_to_pypsa_map_storage_units |" + + Scenario: a Battery with no units in service cannot dispatch and is left out + Given a Plexos model + And the model contains region "West" + And the model contains node "node_z" in region "West" + And the model contains battery "bat_stored" on node "node_z" with max_power 50 capacity 200 charge_efficiency 81 initial_soc 50 + And battery "bat_stored" has property "Units" 0 + And the model is saved as "inputs/battery_stored.xml" + When I run translate against "inputs/battery_stored.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" has 0 storage units + And the log contains "dropping Battery 'bat_stored'" + And the file "decisions.md" contains "rated power works out to 0.0 MW, so this unit cannot dispatch" + Scenario: a pumped-storage plant becomes a cyclic PyPSA PHS StorageUnit Given a Plexos model And the model contains region "West" diff --git a/tests/features/pypsa_to_sienna/renewables.feature b/tests/features/pypsa_to_sienna/renewables.feature index 4fdeb0e..b15b191 100644 --- a/tests/features/pypsa_to_sienna/renewables.feature +++ b/tests/features/pypsa_to_sienna/renewables.feature @@ -108,6 +108,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn Given a PyPSA network And the network contains bus "bus_1" carrier "AC" v_nom 380.0 And the network contains generator "solar_1" on "bus_1" carrier "solar" p_nom 200.0 p_nom_extendable True + And generator "solar_1" has p_nom_opt 200 And the network is saved as "inputs/solar_extendable.nc" When I run translate against "inputs/solar_extendable.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" Then the file "outputs/extensions.json" parses as JSON generator extension record for "solar_1" having "p_nom_extendable" set to true diff --git a/tests/features/pypsa_to_sienna/storage.feature b/tests/features/pypsa_to_sienna/storage.feature index 8331d35..3bdeba1 100644 --- a/tests/features/pypsa_to_sienna/storage.feature +++ b/tests/features/pypsa_to_sienna/storage.feature @@ -80,6 +80,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA StorageUnit PHS to Sien Given a PyPSA network And the network contains bus "bus_1" carrier "AC" v_nom 380.0 And the network contains storage unit "phs_1" on "bus_1" carrier "PHS" p_nom 1000.0 max_hours 6.0 p_nom_extendable True + And storage unit "phs_1" has p_nom_opt 1000 And the network is saved as "inputs/phs_extendable.nc" When I run translate against "inputs/phs_extendable.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" Then the file "outputs/extensions.json" parses as JSON storage extension record for "phs_1" having "p_nom_extendable" set to true diff --git a/tests/features/pypsa_to_sienna/thermal_generators.feature b/tests/features/pypsa_to_sienna/thermal_generators.feature index d62fc80..bc13d66 100644 --- a/tests/features/pypsa_to_sienna/thermal_generators.feature +++ b/tests/features/pypsa_to_sienna/thermal_generators.feature @@ -174,11 +174,23 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn Given a PyPSA network And the network contains bus "bus_1" carrier "AC" v_nom 380.0 And the network contains generator "coal_1" on "bus_1" carrier "coal" p_nom 500.0 p_nom_extendable True + And generator "coal_1" has p_nom_opt 500 And the network is saved as "inputs/gen_extendable.nc" And a user mappings file with all standard carriers When I run translate against "inputs/gen_extendable.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" Then the file "outputs/extensions.json" parses as JSON generator extension record for "coal_1" having "p_nom_extendable" set to true + Scenario: an extendable generator no solve has sized is a candidate, so it is left out + Given a PyPSA network + And the network contains bus "bus_1" carrier "AC" v_nom 380.0 + And the network contains generator "planned_ccgt" on "bus_1" carrier "CCGT" p_nom 500.0 p_nom_extendable True + And the network is saved as "inputs/gen_candidate.nc" + And a user mappings file with all standard carriers + When I run translate against "inputs/gen_candidate.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" + Then the file "outputs/system.json" parses as JSON with 0 components of type "ThermalStandard" + And the log contains "1 Generator(s) are extendable and no solve has sized them, so each is left out" + And the file "decisions.md" contains "p_nom_extendable is true and the network states no p_nom_opt, so this is capacity the plan may build rather than capacity an operations model may dispatch" + Scenario: generator without p_nom_extendable records the flag false in extensions Given a PyPSA network And the network contains bus "bus_1" carrier "AC" v_nom 380.0 From e7aef9ba8b59141dc23bbd100ccf22345b45d4c2 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 19:42:50 +0000 Subject: [PATCH 07/17] Scale a candidate's dated capacity by the units it may build A dated Max Capacity becomes a p_max_pu series, scaled so the band in force reads as a share of p_nom. The scale was the unit count the object runs, which a candidate states as zero, so every snapshot of the band held the candidate at zero and a solve could build none of it. The unit count p_nom stands for is what the object runs, or what a candidate running none of them may build, which is the reading derive_p_nom already takes, so CandidateSource states it once and the dated rows scale by it. A scenario covers a candidate whose capacity rises during the year: it fails against the old scale and passes against the new one. Addresses r4-01 and r4-04. _drop_unbuilt_candidates had one caller and two working lines, so they move into _prepare_source beside the SkipRule it already applies (r4-09). The reason the deleted docstring gave stands in unbuilt_candidate_skip, so no comment replaces it (r4-06). Drops the paragraph defending the grouped warning in find_unpriced_build (r4-07), and the two feature comments repeating the decisions.md text the same scenario asserts (r4-08). Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../plexos_pypsa_translations/_expansion.py | 18 +++++++++++++----- .../plexos_pypsa_translations/_generators.py | 2 +- .../steps/pypsa_to_sienna_map_components.py | 16 ++-------------- .../plexos_to_pypsa/dated_properties.feature | 13 +++++++++++++ .../plexos_to_pypsa/generators.feature | 2 -- 5 files changed, 29 insertions(+), 22 deletions(-) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py index f145d36..06e9c40 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -18,6 +18,7 @@ from interop.plugins.shared.plexos_constants import PlexosClass, PlexosProperty from interop.plugins.shared.plexos_pypsa_translations._shared import as_rate from interop.plugins.shared.plexos_pypsa_translations.constants import ( + DEFAULT_UNITS, DIRECT_DERIVATION, EXT_TECHNICAL_LIFE_FIELD, EXT_UNIT_SIZE_FIELD, @@ -95,6 +96,17 @@ def max_units_built(self) -> float: def is_candidate(self) -> bool: return self.max_units_built > NOTHING_TO_BUILD + @property + def rated_unit_count(self) -> float: + """How many units the nominal power stands for. + + It is what the object runs, or what a candidate running none of them may build, which + is the same reading ``derive_p_nom`` takes. + """ + if self.rated.existing.value: + return self.props.get(PlexosProperty.UNITS, DEFAULT_UNITS) + return self.max_units_built + def name_units_built(self) -> SourceValue: return SourceValue( self.plexos_class, self.name, PlexosProperty.MAX_UNITS_BUILT, self.max_units_built @@ -201,11 +213,7 @@ class UnpricedBuild: def find_unpriced_build( plexos_class: PlexosClass, name: str, props: dict[str, float] ) -> SkippedComponent | None: - """The first property a candidate leaves out that stops PyPSA pricing its build. - - Every candidate leaving out the same property is named in one warning rather than a - line each, because a model that omits one of these omits it wholesale. - """ + """The first property a candidate leaves out that stops PyPSA pricing its build.""" if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD: return None unpriced = next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index 1c7b574..aabf91e 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -325,7 +325,7 @@ def _dated_capacity_rows(state: State, mappings: list[GeneratorMapping]) -> list attribute=PyPSAGeneratorCol.P_MAX_PU, source_owner_type=PlexosClass.GENERATOR, source_series_name=PlexosProperty.MAX_CAPACITY, - scaling_factor=mapping.units / mapping.p_nom, + scaling_factor=mapping.candidate.rated_unit_count / mapping.p_nom, timing=timing, ) for mapping in mappings diff --git a/interop/plugins/steps/pypsa_to_sienna_map_components.py b/interop/plugins/steps/pypsa_to_sienna_map_components.py index 1cd8152..69d67ed 100644 --- a/interop/plugins/steps/pypsa_to_sienna_map_components.py +++ b/interop/plugins/steps/pypsa_to_sienna_map_components.py @@ -308,7 +308,8 @@ def _prepare_source( carriers = self._carrier_mappings.get_carriers(mapping.sienna_component) table = mapping.fill_defaults(source) table = table.filter(pl.col(mapping.carrier_col).is_in(list(carriers))) - table = self._drop_unbuilt_candidates(table, mapping) + candidates = unbuilt_candidate_skip(PYPSA_COMPONENT_NAMING[mapping.source_table]) + table, _ = filter_component(table, candidates.keep, candidates.report, self._recorder) series = self._source_time_series(state, mapping) if mapping.skip is not None: rule = mapping.skip(series) @@ -320,19 +321,6 @@ def _prepare_source( ts_info=ts_info, ) - def _drop_unbuilt_candidates( - self, table: pl.DataFrame, mapping: ComponentMapping - ) -> pl.DataFrame: - """Leave out the candidates, which a Sienna operations system has no capacity for. - - A component a solve has sized carries the capacity it built, and one whose capacity is - fixed carries what it has. Anything else is a build the plan has yet to decide, and - writing it here would give the operations system a plant nobody has built. - """ - rule = unbuilt_candidate_skip(PYPSA_COMPONENT_NAMING[mapping.source_table]) - kept, _ = filter_component(table, rule.keep, rule.report, self._recorder) - return kept - def _add_derived_series( self, state: State, mapping: ComponentMapping, prepared: _PreparedSource ) -> None: diff --git a/tests/features/plexos_to_pypsa/dated_properties.feature b/tests/features/plexos_to_pypsa/dated_properties.feature index 563e824..b0c9b47 100644 --- a/tests/features/plexos_to_pypsa/dated_properties.feature +++ b/tests/features/plexos_to_pypsa/dated_properties.feature @@ -57,6 +57,19 @@ Feature: a PLEXOS property dated to a period is read for the year being translat Then the PyPSA network "outputs/network.nc" generator "Farm" attribute "p_nom" is 100 And the PyPSA network "outputs/network.nc" generator "Farm" has a p_max_pu time series 0.5 1.0 + Scenario: a candidate's dated capacity is a share of the capacity it may build + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generator "REZ" with "node=Grid_Node, category=Wind, Max Capacity=50, Units=0, Max Units Built=4, Build Cost=900000, WACC=0.07, Economic Life=25" + And generator "REZ" states "Max Capacity" of 100 from "2026-01-02" + And the model contains model "Plan" + And the model contains horizon "H1" on model "Plan" starting "2026-01-01" spanning 2 days at 1 periods per day + And the model is saved as "inputs/model.xml" + When I run translate against "inputs/model.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "REZ" attribute "p_nom" is 400 + And the PyPSA network "outputs/network.nc" generator "REZ" has a p_max_pu time series 0.5 1.0 + Scenario: an outage dated between two days ends when the second one does Given a Plexos model And the model contains region "Grid" diff --git a/tests/features/plexos_to_pypsa/generators.feature b/tests/features/plexos_to_pypsa/generators.feature index 47b65c4..115b348 100644 --- a/tests/features/plexos_to_pypsa/generators.feature +++ b/tests/features/plexos_to_pypsa/generators.feature @@ -539,9 +539,7 @@ Feature: Translate PLEXOS generators into a PyPSA network And the model is saved as "inputs/unpriced_candidate.xml" When I run translate against "inputs/unpriced_candidate.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" has no generator "Unpriced_REZ" - # PyPSA annuitises an overnight cost with the discount rate, and refuses a network stating one without the other. And the PyPSA network "outputs/network.nc" has no generator "Undiscounted_REZ" - # PyPSA annuitises over the lifetime, and a lifetime of infinity prices the build as a perpetuity. And the PyPSA network "outputs/network.nc" has no generator "Everlasting_REZ" And the PyPSA network "outputs/network.nc" generator "Priced_REZ" is extendable And the file "decisions.md" contains "a candidate with no Build Cost prices building nothing, so an expansion would take it for free" From 12a03c2f94b94837a7575cdaeac31e0e1ffab390 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 20:11:40 +0000 Subject: [PATCH 08/17] Keep a running plant when the model prices none of its expansion A Generator, a Battery or a turbine that states a Max Units Built the model prices nothing for was left out whole, so a plant already generating vanished from the network because no expansion of it was priced. An object with units in service now keeps the capacity it runs, with that capacity fixed, and only the build is left out: decisions.md carries a NOT_MAPPED row naming the property the model left out, and one warning names a few of the objects it happened to. An object with no units yet is the build and nothing else, so it is still skipped. (r5-04) The battery's state_of_charge_initial named Capacity and Initial SoC while its value came from the energy of every unit. The energy the battery holds is now a Decision of its own, so p_nom and max_hours travel with the level and the derivation states what the code does. (r5-01, r5-09) Name a case the zero guard in the Sienna storage initial level can still meet, since an extendable unit no solve has sized is now dropped before it. (r5-10) Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 19 ++-- .../plexos_pypsa_translations/_batteries.py | 18 ++-- .../plexos_pypsa_translations/_expansion.py | 95 ++++++++++++++++--- .../plexos_pypsa_translations/_generators.py | 10 +- .../_storage_shared.py | 8 +- .../_storage_units.py | 6 +- .../pypsa_sienna_translations/_storage.py | 6 +- .../plexos_to_pypsa/generators.feature | 12 +++ .../plexos_to_pypsa/storage_units.feature | 16 +++- 9 files changed, 151 insertions(+), 39 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index d69d875..ea27526 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -287,9 +287,9 @@ component: | The generator has no [`Units`](#which-entry-applies-when) at any time in the horizon, and no `Max Units Built` | The unit is retired. | | `Max Capacity` comes from a data file | There is no single `p_nom`. Thus the translator cannot set the size of the generator, and it cannot calculate the availability per unit. | | `p_nom` is 0 | The generator can never dispatch. | -| The generator is a candidate and gives no `Build Cost` | Nothing prices building it, so an expansion would take it for free. | -| The generator is a candidate and gives no `WACC` | PyPSA annuitises a build cost with a discount rate, and refuses a network that states one without the other. | -| The generator is a candidate and gives no `Economic Life` | PyPSA annuitises a build cost across a lifetime. The PyPSA default is infinity, which prices the build as a perpetuity. | +| The generator has no units yet and gives no `Build Cost` | Nothing prices building it, so an expansion would take it for free. | +| The generator has no units yet and gives no `WACC` | PyPSA annuitises a build cost with a discount rate, and refuses a network that states one without the other. | +| The generator has no units yet and gives no `Economic Life` | PyPSA annuitises a build cost across a lifetime. The PyPSA default is infinity, which prices the build as a perpetuity. | The repair rates, the unit commitment solver options and the energy budgets are `dropped`. @@ -357,7 +357,7 @@ has more than one fuel uses its primary fuel. | `max_hours` | h | The energy capacity divided by `p_nom` | `derived` | | `p_max_pu` / `p_min_pu` | | `1.0` / `-1.0` | `default` | | `efficiency_store` / `efficiency_dispatch` | | `√(Charge Efficiency)` for each | `derived` | -| `state_of_charge_initial` | MWh | `Initial SoC % × Capacity` | `derived` | +| `state_of_charge_initial` | MWh | `Initial SoC % × p_nom × max_hours` | `derived` | | `cyclic_state_of_charge` | | `True` if `End Effects Method` is `RECYCLE`, or if the model gives no `Initial SoC` | `derived` | | `marginal_cost` | $/MWh | `0.0` | `default` | | `p_nom_extendable`, `p_nom_min`, `p_nom_max`, `overnight_cost`, `discount_rate`, `lifetime`, `fom_cost` | | Refer to [What a candidate is](#what-a-candidate-is) | `derived` / `default` | @@ -743,12 +743,17 @@ The rated power of one unit is the `Max Capacity` of a `Generator` or a pumped-s turbine, and the `Max Power` of a `Battery`. An object that states `Units` above one already holds that many, so a build adds units to what it has rather than multiplying it. -A candidate that states no `Build Cost`, no `WACC` or no `Economic Life` is left out. +A candidate that states no `Build Cost`, no `WACC` or no `Economic Life` prices no build. Without a build cost nothing prices building it, so an expansion would take it for free. Without a discount rate PyPSA cannot annuitise the build cost, and refuses the network. Without an economic life PyPSA annuitises across its own default lifetime of infinity, which -prices the build as a perpetuity. The translator records a `COMPONENT_SKIPPED` event naming -the object, and warns once naming a few of them. +prices the build as a perpetuity. + +What happens next depends on whether the object already runs. An object with units in +service keeps the capacity it runs: the translator writes it with that capacity fixed, and +records a `NOT_MAPPED` event naming the property the model left out. An object with no units +yet is the build and nothing else, so nothing is left to write: the translator records a +`COMPONENT_SKIPPED` event naming the object, and warns once naming a few of them. ### Which entry applies when diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index bf3219e..bc164d7 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -22,6 +22,7 @@ from interop.plugins.shared.plexos_pypsa_translations._expansion import ( RatedCapacity, derive_expansion, + gather_sources, ) from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( @@ -94,7 +95,8 @@ def record_battery_outages(state: State, mappings: list[StorageUnitMapping]) -> _CAPACITY_DERIVATION = "Capacity" _CAPACITY_FROM_DURATION_DERIVATION = "Duration * Max Power" _P_NOM_FROM_UNITS_DERIVATION = "Max Power * Units" -_SOC_FROM_PERCENT_DERIVATION = "Initial SoC / 100 * energy capacity" +_STORED_ENERGY_DERIVATION = "p_nom * max_hours" +_SOC_FROM_PERCENT_DERIVATION = "Initial SoC / 100 * p_nom * max_hours" _PER_MAX_POWER_DERIVATION = " / Max Power" # --- batteries --------------------------------------------------------------- @@ -130,7 +132,11 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: unit_size = rated.candidate.rated.unit_size capacity = _battery_energy_capacity(rated, unit_size) max_hours = derive_max_hours(capacity, unit_size.value, per=_PER_MAX_POWER_DERIVATION) - stored_energy = rated.p_nom.value * max_hours.value + stored = Decision.derived( + rated.p_nom.value * max_hours.value, + gather_sources(rated.p_nom.sources, max_hours.sources), + _STORED_ENERGY_DERIVATION, + ) expansion = derive_expansion(rated.candidate) return StorageUnitMapping( name=rated.name, @@ -143,7 +149,7 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: efficiency=_battery_efficiency(rated), marginal_cost=Decision.default(STORAGE_MARGINAL_COST, _BATTERY_MARGINAL_COST_NOTE), state_of_charge_initial=derive_state_of_charge_initial( - _battery_initial_level(rated, capacity, stored_energy), stored_energy + _battery_initial_level(rated, capacity, stored), stored.value ), inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE), cyclic=_battery_cyclic(rated), @@ -197,7 +203,7 @@ def _capacity_from_duration(name: str, duration: float, unit_size: Decision) -> def _battery_initial_level( - rated: RatedObject, capacity: Decision | None, stored_energy: float + rated: RatedObject, capacity: Decision | None, stored: Decision ) -> Decision | None: """Where the battery starts, as MWh, from Initial SoC against the energy it holds.""" initial_soc = rated.properties.get(PlexosProperty.INITIAL_SOC) @@ -207,8 +213,8 @@ def _battery_initial_level( PlexosClass.BATTERY, rated.name, PlexosProperty.INITIAL_SOC, initial_soc, UNIT_PERCENT ) return Decision.derived( - initial_soc / PERCENT * stored_energy, - [source, *capacity.sources], + initial_soc / PERCENT * stored.value, + [source, *stored.sources], _SOC_FROM_PERCENT_DERIVATION, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py index 06e9c40..7d71054 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -7,7 +7,9 @@ from __future__ import annotations -from dataclasses import dataclass +import logging +from collections.abc import Iterable +from dataclasses import dataclass, replace from interop.plugins.shared.constants import ( UNIT_DOLLARS_PER_MW, @@ -34,6 +36,9 @@ maps_to, ) from interop.plugins.shared.pypsa_constants import PyPSAGeneratorCol +from interop.plugins.shared.warning_text import name_a_few + +log = logging.getLogger(__name__) NOTHING_TO_REPORT = Decision.unreported(None) @@ -65,6 +70,11 @@ "a candidate with no Economic Life gives PyPSA no period to annuitise its Build Cost over, " "so PyPSA prices the build as a perpetuity" ) +_BUILD_LEFT_OUT_NOTE = "; the object keeps the capacity it runs and only its build is left out" +_UNPRICED_BUILD_DERIVATION = ( + "the model prices no build for this object, so it keeps the capacity it runs and that " + "capacity is fixed" +) UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) @@ -125,6 +135,8 @@ class ExpansionDecisions: # What the sidecar carries because the network file has no column for it. unit_size: Decision = NOTHING_TO_REPORT technical_life: Decision = NOTHING_TO_REPORT + # The build of an object that runs already and prices no expansion of itself. + dropped_build: DroppedBuild | None = None FIXED_CAPACITY = ExpansionDecisions( @@ -141,6 +153,9 @@ class ExpansionDecisions: def derive_expansion(source: CandidateSource) -> ExpansionDecisions: if not source.is_candidate: return FIXED_CAPACITY + unpriced = _find_unpriced_build(source) + if unpriced is not None: + return _fixed_at_what_it_runs(source, unpriced) rated = source.rated built = source.name_units_built() return ExpansionDecisions( @@ -210,29 +225,59 @@ class UnpricedBuild: ) -def find_unpriced_build( - plexos_class: PlexosClass, name: str, props: dict[str, float] -) -> SkippedComponent | None: - """The first property a candidate leaves out that stops PyPSA pricing its build.""" - if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD: +@dataclass(frozen=True) +class DroppedBuild: + """A build the model prices nothing for, on an object that keeps the capacity it runs.""" + + source: SourceValue + note: str + + +def find_unpriced_candidate(source: CandidateSource) -> SkippedComponent | None: + """A candidate with nothing running yet whose build the model prices nothing for. + + Its whole capacity is the build, so there is nothing to write once the build goes. An + object that already runs states capacity a dispatch model needs, so it stays. + """ + if source.rated.existing.value or not source.is_candidate: return None - unpriced = next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None) + unpriced = _find_unpriced_build(source) if unpriced is None: return None return SkippedComponent( - source=SourceValue(plexos_class, name, unpriced.plexos_property, None, unpriced.unit), + source=_names_absent(source, unpriced), note=unpriced.note, warn_with=SkipGroup( - counted=f"candidate {plexos_class}(s)", reason=f"state no {unpriced.plexos_property}" + counted=f"candidate {source.plexos_class}(s)", + reason=f"state no {unpriced.plexos_property}", ), ) -def record_expansion_extensions( - name: str, expansion: ExpansionDecisions, reporter: ComponentReporter -) -> None: +def record_expansion(name: str, expansion: ExpansionDecisions, reporter: ComponentReporter) -> None: + """What travels beside the destination row: the sidecar fields, and a build left out.""" reporter.record(name, UNIT_SIZE_COLUMN, expansion.unit_size) reporter.record(name, TECHNICAL_LIFE_COLUMN, expansion.technical_life) + if expansion.dropped_build is not None: + reporter.record_dropped(expansion.dropped_build.source, expansion.dropped_build.note) + + +def warn_about_dropped_builds(expansions: Iterable[ExpansionDecisions]) -> None: + """One line for each property that left a running object's build unpriced.""" + dropped = [one.dropped_build for one in expansions if one.dropped_build is not None] + grouped: dict[tuple[str, str], list[str]] = {} + for one in dropped: + key = (one.source.component, str(one.source.attribute)) + grouped.setdefault(key, []).append(one.source.name) + for (component, plexos_property), names in sorted(grouped.items()): + log.warning( + "plexos: %d %s(s) that already run state no %s, so each keeps the capacity it " + "runs and none of the build it may make. Each one: %s", + len(names), + component, + plexos_property, + name_a_few(sorted(names)), + ) def read_sidecar_value(decision: Decision) -> float | None: @@ -248,6 +293,32 @@ def gather_sources(*groups: tuple[SourceValue, ...] | list[SourceValue]) -> list return list(gathered) +def _find_unpriced_build(source: CandidateSource) -> UnpricedBuild | None: + """The first property a candidate leaves out that stops PyPSA pricing its build.""" + return next((one for one in _PRICES_A_BUILD if one.plexos_property not in source.props), None) + + +def _fixed_at_what_it_runs(source: CandidateSource, unpriced: UnpricedBuild) -> ExpansionDecisions: + return replace( + FIXED_CAPACITY, + p_nom_extendable=Decision.derived( + False, # noqa: FBT003 + [source.name_units_built()], + _UNPRICED_BUILD_DERIVATION, + ), + dropped_build=DroppedBuild( + source=_names_absent(source, unpriced), + note=unpriced.note + _BUILD_LEFT_OUT_NOTE, + ), + ) + + +def _names_absent(source: CandidateSource, unpriced: UnpricedBuild) -> SourceValue: + return SourceValue( + source.plexos_class, source.name, unpriced.plexos_property, None, unpriced.unit + ) + + def _discount_rate(source: CandidateSource) -> Decision: wacc = source.props.get(PlexosProperty.WACC) rate = as_rate(wacc, source.stated_units.get(PlexosProperty.WACC)) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index aabf91e..f291f4f 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -23,9 +23,10 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( - find_unpriced_build, + find_unpriced_candidate, read_sidecar_value, - record_expansion_extensions, + record_expansion, + warn_about_dropped_builds, ) from interop.plugins.shared.plexos_pypsa_translations._generator_decisions import ( GeneratorDecisions, @@ -147,7 +148,7 @@ def _carry_to_extensions( for mapping in mappings: if mapping.carrier != mapping.category: reporter.record(mapping.name, _CATEGORY_COLUMN, _category_decision(mapping)) - record_expansion_extensions(mapping.name, mapping.expansion, reporter) + record_expansion(mapping.name, mapping.expansion, reporter) records = [ GeneratorExtension( name=mapping.name, @@ -157,6 +158,7 @@ def _carry_to_extensions( ) for mapping in mappings ] + warn_about_dropped_builds(mapping.expansion for mapping in mappings) append_extensions(state.destination_extensions, ExtensionKind.GENERATOR, records) @@ -184,7 +186,7 @@ def _map_one( _source(name, PlexosProperty.MAX_CAPACITY, None, UNIT_MW), f"generator dropped: p_nom is {source.p_nom} MW, so it can never dispatch", ) - unpriced = find_unpriced_build(PlexosClass.GENERATOR, name, source.props) + unpriced = find_unpriced_candidate(source.candidate) if unpriced is not None: return unpriced mapping = derive_generator(source, node, lookups) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index 368c2f2..caee1e6 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -31,7 +31,7 @@ ExpansionDecisions, RatedCapacity, derive_p_nom, - find_unpriced_build, + find_unpriced_candidate, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, @@ -343,9 +343,6 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp return _skipped_file_backed(rating, staged.name) if rating.capacity_property not in staged.properties: return _skipped_without_capacity(rating, staged.name) - unpriced = find_unpriced_build(rating.plexos_class, staged.name, staged.properties) - if unpriced is not None: - return unpriced candidate = CandidateSource( rating.plexos_class, staged.name, @@ -353,6 +350,9 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp staged.stated_units, rating.derive(staged), ) + unpriced = find_unpriced_candidate(candidate) + if unpriced is not None: + return unpriced p_nom = derive_p_nom(candidate) if p_nom.value <= 0.0: return _skipped_zero_p_nom(rating, staged.name, p_nom.value) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index 1ab2412..d9e9bee 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -29,7 +29,8 @@ ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( read_sidecar_value, - record_expansion_extensions, + record_expansion, + warn_about_dropped_builds, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, @@ -227,7 +228,8 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No reporter = ComponentReporter(recorder, PyPSAComponent.STORAGE_UNIT) for mapping in storage_units.mappings: reporter.record_mapping(mapping.name, mapping) - record_expansion_extensions(mapping.name, mapping.expansion, reporter) + record_expansion(mapping.name, mapping.expansion, reporter) + warn_about_dropped_builds(mapping.expansion for mapping in storage_units.mappings) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) warn_about_skips(storage_units.skipped) diff --git a/interop/plugins/shared/pypsa_sienna_translations/_storage.py b/interop/plugins/shared/pypsa_sienna_translations/_storage.py index 3c432dc..59308e0 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_storage.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_storage.py @@ -151,9 +151,9 @@ def _skip_without_energy(_series: pl.LazyFrame | None) -> SkipRule: # --- Reusable expressions --- -# state_of_charge_initial / (effective_p_nom * max_hours), clamped to [0, 1]. -# Defaults to 0.0 when capacity is zero (e.g. unsolved extendable unit) to -# avoid a division-by-zero that would clip to 1.0 (incorrectly "fully charged"). +# state_of_charge_initial / (effective_p_nom * max_hours), clamped to [0, 1]. A capacity of +# zero, from a solve that built none of an extendable unit or from a p_nom of zero, gives 0.0 +# rather than a division by zero that would clip to 1.0 and read as fully charged. _initial_level = ( pl.when(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0) .then( diff --git a/tests/features/plexos_to_pypsa/generators.feature b/tests/features/plexos_to_pypsa/generators.feature index 115b348..40f5e04 100644 --- a/tests/features/plexos_to_pypsa/generators.feature +++ b/tests/features/plexos_to_pypsa/generators.feature @@ -558,3 +558,15 @@ Feature: Translate PLEXOS generators into a PyPSA network And the PyPSA generator "FixedPlant" in "outputs/network.nc" has no "overnight_cost" And the PyPSA generator "FixedPlant" in "outputs/network.nc" has no "discount_rate" And the file "decisions.md" contains "the object states no Max Units Built, so its capacity is fixed" + + Scenario: a plant that already runs keeps its capacity when its build is unpriced + Given a Plexos model + And the model contains generator "CoalUnit" with "node=Grid_Node, category=Coal, Max Capacity=100, Units=2, Max Units Built=1" + And the model is saved as "inputs/unpriced_plant.xml" + When I run translate against "inputs/unpriced_plant.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA generator "CoalUnit" in "outputs/network.nc" has "p_nom" equal to 200 + And the PyPSA network "outputs/network.nc" generator "CoalUnit" is not extendable + And the PyPSA generator "CoalUnit" in "outputs/network.nc" has no "overnight_cost" + And the file "decisions.md" contains "a candidate with no Build Cost prices building nothing, so an expansion would take it for free; the object keeps the capacity it runs and only its build is left out" + And the file "decisions.md" contains "the model prices no build for this object, so it keeps the capacity it runs and that capacity is fixed" + And the log contains "1 Generator(s) that already run state no Build Cost, so each keeps the capacity it runs and none of the build it may make" diff --git a/tests/features/plexos_to_pypsa/storage_units.feature b/tests/features/plexos_to_pypsa/storage_units.feature index cfaeb30..cc08829 100644 --- a/tests/features/plexos_to_pypsa/storage_units.feature +++ b/tests/features/plexos_to_pypsa/storage_units.feature @@ -275,7 +275,7 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr When I run translate against "inputs/bat_duration.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" storage unit "bat_duration" attribute "max_hours" is 2.0 And the PyPSA network "outputs/network.nc" storage unit "bat_duration" attribute "state_of_charge_initial" is 100.0 - And the file "decisions.md" contains "`pypsa.StorageUnit.bat_duration.state_of_charge_initial` = 100.0 MWh | Initial SoC / 100 * energy capacity |" + And the file "decisions.md" contains "`pypsa.StorageUnit.bat_duration.state_of_charge_initial` = 100.0 MWh | Initial SoC / 100 * p_nom * max_hours |" Scenario: a mothballed turbine has no rated power and is skipped Given a Plexos model @@ -603,3 +603,17 @@ Feature: PLEXOS to PyPSA Pipeline translates batteries, pumped storage, and hydr And the PyPSA network "outputs/network.nc" storage unit "phs_new" attribute "p_nom_max" is 300 # Nothing is built yet, so the capacity it may build is what its per-unit fields read against. And the PyPSA network "outputs/network.nc" storage unit "phs_new" attribute "p_nom" is 300 + + Scenario: a battery that already runs keeps its capacity when its build is unpriced + Given a Plexos model + And the model contains region "East" + And the model contains node "node_k" in region "East" + And the model contains battery "bat_running" on node "node_k" with max_power 50 capacity 200 charge_efficiency 90 initial_soc 50 + And battery "bat_running" has property "Units" 2 + And battery "bat_running" has property "Max Units Built" 3 + And the model is saved as "inputs/unpriced_battery.xml" + When I run translate against "inputs/unpriced_battery.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" storage unit "bat_running" attribute "p_nom" is 100.0 + And the PyPSA network "outputs/network.nc" storage unit "bat_running" is not extendable + And the file "decisions.md" contains "a candidate with no Build Cost prices building nothing, so an expansion would take it for free; the object keeps the capacity it runs and only its build is left out" + And the log contains "1 Battery(s) that already run state no Build Cost, so each keeps the capacity it runs and none of the build it may make" From 1c2a36f2629c7dc40d34a39abb77f3c2467118fe Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 20:34:49 +0000 Subject: [PATCH 09/17] Rate a storage object once, from the class and property it already names RatedPower carries the PLEXOS class and the capacity property, which is everything the two rating functions differed by, so the rating becomes a method on it and the callback field goes. The derivation string still reads "Max Power * Units" and "Max Capacity * Units", built from the property the constant names (r6-01). Drops two docstrings that repeat what the name beside them already says (r6-08, r6-09), and counts the skipped generator cases the mappings table actually lists (r6-10). Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 2 +- .../plexos_pypsa_translations/_batteries.py | 24 +------------------ .../plexos_pypsa_translations/_expansion.py | 2 -- .../_storage_hydro.py | 23 +----------------- .../_storage_shared.py | 21 +++++++++++++--- .../pypsa_sienna_translations/_shared.py | 1 - 6 files changed, 21 insertions(+), 52 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index ea27526..3a25f2f 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -278,7 +278,7 @@ is `committable` when it is thermal, or when its `p_min_pu` is more than `0`. | `lifetime` | yr | `Economic Life`, for a candidate only | `direct` | | `fom_cost` | $/MW/yr | `FO&M Charge`, for a candidate only | `direct` | -**The translator does not translate six cases.** It records each one as a skipped +**The translator does not translate seven cases.** It records each one as a skipped component: | Case | Cause | diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index bc164d7..66f25b4 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -11,7 +11,6 @@ from interop.core.pipeline import State from interop.plugins.shared.constants import ( UNIT_HOURS, - UNIT_MW, UNIT_MWH, UNIT_PERCENT, ) @@ -20,7 +19,6 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( - RatedCapacity, derive_expansion, gather_sources, ) @@ -34,7 +32,6 @@ MappedOrSkipped, RatedObject, RatedPower, - StagedObject, StorageLookups, StorageUnitMapping, derive_bus, @@ -47,8 +44,6 @@ BATTERY_CYCLIC, DEFAULT_INFLOW, DEFAULT_ROUND_TRIP_EFFICIENCY, - DEFAULT_UNITS, - DIRECT_DERIVATION, END_EFFECTS_RECYCLE, PERCENT, STORAGE_FULL_CHARGE_PU, @@ -94,7 +89,6 @@ def record_battery_outages(state: State, mappings: list[StorageUnitMapping]) -> _BATTERY_RECYCLE_DERIVATION = "End Effects Method recycles the level" _CAPACITY_DERIVATION = "Capacity" _CAPACITY_FROM_DURATION_DERIVATION = "Duration * Max Power" -_P_NOM_FROM_UNITS_DERIVATION = "Max Power * Units" _STORED_ENERGY_DERIVATION = "p_nom * max_hours" _SOC_FROM_PERCENT_DERIVATION = "Initial SoC / 100 * p_nom * max_hours" _PER_MAX_POWER_DERIVATION = " / Max Power" @@ -109,23 +103,7 @@ def map_battery(name: str, lookups: StorageLookups) -> MappedOrSkipped: return _derive_battery(rated) -def _battery_rating(staged: StagedObject) -> RatedCapacity: - """A PLEXOS Battery states the power of one of its units as its Max Power.""" - max_power = staged.properties[PlexosProperty.MAX_POWER] - units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS) - power = SourceValue( - PlexosClass.BATTERY, staged.name, PlexosProperty.MAX_POWER, max_power, UNIT_MW - ) - counted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units) - return RatedCapacity( - existing=Decision.derived( - max_power * units, [power, counted], _P_NOM_FROM_UNITS_DERIVATION - ), - unit_size=Decision.derived(max_power, [power], DIRECT_DERIVATION), - ) - - -_BATTERY_POWER = RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER, _battery_rating) +_BATTERY_POWER = RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER) def _derive_battery(rated: RatedObject) -> StorageUnitMapping: diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py index 7d71054..f5b9780 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -90,8 +90,6 @@ class RatedCapacity: @dataclass(frozen=True) class CandidateSource: - """One staged PLEXOS object as the expansion rule reads it.""" - plexos_class: PlexosClass name: str props: dict[str, float] diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index d11c92b..db6a789 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -23,7 +23,6 @@ PlexosProperty, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( - RatedCapacity, derive_expansion, ) from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( @@ -50,8 +49,6 @@ from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_INFLOW, DEFAULT_ROUND_TRIP_EFFICIENCY, - DEFAULT_UNITS, - DIRECT_DERIVATION, HYDRO_CYCLIC, PUMPED_STORAGE_CYCLIC, STORAGE_FULL_CHARGE_PU, @@ -130,7 +127,6 @@ def record_reservoir_inflows(state: State, mappings: list[StorageUnitMapping]) - _NO_INFLOW_STATED_NOTE = "the head Storage states no Natural Inflow; the reservoir does not refill" _NO_VOM_NOTE = "the PLEXOS Generator states no VO&M Charge" _PUMPED_STORAGE_CYCLIC_NOTE = "a pumped-storage plant returns to its starting level" -_P_NOM_FROM_UNITS_DERIVATION = "Max Capacity * Units" _SOC_FROM_VOLUME_DERIVATION = "head Storage.Initial Volume" _VOLUME_NOT_ENERGY_NOTE = ( "the head Storage names a volume unit that is not megawatt-hours, so how much energy " @@ -197,24 +193,7 @@ def _classify_turbine( return None -def _turbine_rating(staged: StagedObject) -> RatedCapacity: - """A PLEXOS Generator states the power of one of its units as its Max Capacity.""" - max_capacity = staged.properties[PlexosProperty.MAX_CAPACITY] - stated_units = staged.properties.get(PlexosProperty.UNITS) - units = DEFAULT_UNITS if stated_units is None else stated_units - capacity = SourceValue( - PlexosClass.GENERATOR, staged.name, PlexosProperty.MAX_CAPACITY, max_capacity, UNIT_MW - ) - counted = SourceValue(PlexosClass.GENERATOR, staged.name, PlexosProperty.UNITS, units) - return RatedCapacity( - existing=Decision.derived( - max_capacity * units, [capacity, counted], _P_NOM_FROM_UNITS_DERIVATION - ), - unit_size=Decision.derived(max_capacity, [capacity], DIRECT_DERIVATION), - ) - - -_GENERATOR_POWER = RatedPower(PlexosClass.GENERATOR, PlexosProperty.MAX_CAPACITY, _turbine_rating) +_GENERATOR_POWER = RatedPower(PlexosClass.GENERATOR, PlexosProperty.MAX_CAPACITY) def _derive_turbine( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index caee1e6..a5e8a63 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -8,7 +8,6 @@ from __future__ import annotations -from collections.abc import Callable from dataclasses import dataclass from math import sqrt @@ -44,6 +43,8 @@ from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_STATE_OF_CHARGE_INITIAL, DEFAULT_STORAGE_MAX_HOURS, + DEFAULT_UNITS, + DIRECT_DERIVATION, PERCENT, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( @@ -85,6 +86,7 @@ # The Storage properties whose stated unit decides whether they are energy at all. _VOLUME_PROPERTIES = (PlexosProperty.MAX_VOLUME, PlexosProperty.INITIAL_VOLUME) +_TIMES_UNITS_DERIVATION = " * Units" _PER_P_NOM_DERIVATION = " / p_nom" _CLAMPED_DERIVATION = ", clamped to 0..p_nom * max_hours" @@ -321,7 +323,20 @@ class RatedPower: plexos_class: PlexosClass capacity_property: PlexosProperty - derive: Callable[[StagedObject], RatedCapacity] + + def rate(self, staged: StagedObject) -> RatedCapacity: + power = staged.properties[self.capacity_property] + units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS) + stated = SourceValue(self.plexos_class, staged.name, self.capacity_property, power, UNIT_MW) + counted = SourceValue(self.plexos_class, staged.name, PlexosProperty.UNITS, units) + return RatedCapacity( + existing=Decision.derived( + power * units, + [stated, counted], + f"{self.capacity_property}{_TIMES_UNITS_DERIVATION}", + ), + unit_size=Decision.derived(power, [stated], DIRECT_DERIVATION), + ) @dataclass(frozen=True) @@ -348,7 +363,7 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp staged.name, staged.properties, staged.stated_units, - rating.derive(staged), + rating.rate(staged), ) unpriced = find_unpriced_candidate(candidate) if unpriced is not None: diff --git a/interop/plugins/shared/pypsa_sienna_translations/_shared.py b/interop/plugins/shared/pypsa_sienna_translations/_shared.py index 666cfa1..d06069f 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_shared.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_shared.py @@ -135,7 +135,6 @@ def carrier_scope_skips(naming: PyPSAComponentNaming) -> ScopeSkips: def unbuilt_candidate_skip(naming: PyPSAComponentNaming) -> SkipRule: - """The drop for a candidate an operations system must not read as a built plant.""" return SkipRule( keep=pl.col(STATES_BUILT_CAPACITY), report=pypsa_skip_report( From 1a76b82668488f3d5fa2c19e1ae95711281c5918 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 21:00:49 +0000 Subject: [PATCH 10/17] Keep an extendable component that already runs in the operations system The pypsa-to-sienna scope rule read p_nom_extendable and p_nom_opt and nothing else, so it left out every extendable component no solve had touched. A plant that already generates and merely states what it may build states that running capacity as p_nom_min, and PyPSA ignores the p_nom of an extendable component, so the rule deleted real capacity from the Sienna system. The PLEXOS leg writes exactly that shape for a plant with a Max Units Built. A component belongs in the operations system when it holds capacity an operations model may dispatch: any fixed component, an extendable one a solve has sized, and now an extendable one whose p_nom_min is a capacity a build cannot take away. effective_p_nom rates it at that floor, and the events name p_nom_min as the attribute the rating came from. Only an extendable component with neither a p_nom_opt nor a p_nom_min is a build the plan has yet to decide, and that one is still left out and named in decisions.md. (r7-01) Links read the same rating, so an extendable link with a floor now rates at it rather than at a p_nom PyPSA never reads. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-sienna.md | 9 +-- .../translation-from-pypsa-to-sienna.md | 14 ++--- .../pypsa_sienna_translations/_generators.py | 16 ++--- .../pypsa_sienna_translations/_hydro.py | 2 + .../pypsa_sienna_translations/_links.py | 20 ++++-- .../pypsa_sienna_translations/_renewables.py | 2 + .../pypsa_sienna_translations/_shared.py | 63 +++++++++++++------ .../pypsa_sienna_translations/_storage.py | 2 + .../pypsa_to_sienna/renewables.feature | 2 +- .../thermal_generators.feature | 20 ++++-- 10 files changed, 104 insertions(+), 46 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-sienna.md b/docs/translation_mappings/translation-from-plexos-to-sienna.md index 8a13338..5f93949 100644 --- a/docs/translation_mappings/translation-from-plexos-to-sienna.md +++ b/docs/translation_mappings/translation-from-plexos-to-sienna.md @@ -55,9 +55,10 @@ property of the PLEXOS to Sienna mapping. [The carrier mappings file](#the-carrier-mappings-file). - **A component whose carrier your file does not name is left out.** The run completes and `decisions.md` names each one. -- **A candidate is not a plant.** A `Generator`, a `Battery` or a turbine that states - `Max Units Built` is a build the plan has yet to decide, and a dispatch system has no - capacity for one, so it is left out and `decisions.md` names it. +- **A build the plan has yet to decide is not a plant.** A `Generator`, a `Battery` or a + turbine that states `Max Units Built` and runs no units yet is a build and nothing else, and + a dispatch system has no capacity for one, so it is left out and `decisions.md` names it. One + that already runs keeps the capacity it runs, and only the build it may add is left out. - **One scenario only.** The Model you select applies its own Scenario overlays. The translator reads no other scenario. - **One calendar year at a time.** The year you give narrows the Horizon of the Model. Every @@ -273,7 +274,7 @@ component: | --- | --- | | The generator has no `Node` | There is no bus to connect it to. | | The generator has no `Units` at any time in the horizon | The unit is retired. | -| The generator states `Max Units Built` | It is a candidate to build, not capacity to dispatch. | +| The generator states `Max Units Built` and runs no units yet | It is a build to decide, not capacity to dispatch. | | `Max Capacity` comes from a data file | There is no single capacity to divide the per-unit fields by. | | The capacity is 0 | The generator can never dispatch. | diff --git a/docs/translation_mappings/translation-from-pypsa-to-sienna.md b/docs/translation_mappings/translation-from-pypsa-to-sienna.md index b731c11..b759cc5 100644 --- a/docs/translation_mappings/translation-from-pypsa-to-sienna.md +++ b/docs/translation_mappings/translation-from-pypsa-to-sienna.md @@ -81,7 +81,7 @@ This document targets the SiennaSchemas **Operations** namespace (the dispatchab **Implication for this translation.** PyPSA's capacity-expansion fields (`p_nom_extendable`, `capital_cost`, `p_nom_min`/`p_nom_max`, `build_year`, `lifetime`) belong to **Investments**, which is a separate translation target (PyPSA expansion → SiennaSchemas Investments), not extra fields on an Operations component. It is **out of scope for v1**. For v1 we translate the **solved** fleet: take `p_nom_opt` (post-solve capacities) as fixed Operations capacity and drop the expansion parameters (or keep round-trip crumbs in the `ext` sidecar). A genuine expansion problem is the future Investments path. -Throughout this document, **`effective_p_nom`** denotes `p_nom_opt` when `p_nom_extendable` is `True` and the network states a `p_nom_opt`, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`. A solve that builds none of an extendable component writes `p_nom_opt` 0, and that 0 is the capacity to translate: the plan rejected the build. Only a network no solve has touched leaves `p_nom_opt` out of the file altogether, and an extendable component there is a candidate rather than a plant: the generator, renewable, hydro and storage mappings leave it out and record it as skipped. +Throughout this document, **`effective_p_nom`** denotes the capacity an operations model may dispatch: `p_nom_opt` where an extendable component states one, `p_nom_min` where it states a capacity a build cannot take away, and `p_nom` otherwise. All capacity-derived fields — `base_power`, `active_power`, `active_power_limits`, `ramp_limits`, and the hydro energy budget scaling factor — use `effective_p_nom`. For a non-extendable component `effective_p_nom = p_nom`. A solve that builds none of an extendable component writes `p_nom_opt` 0, and that 0 is the capacity to translate: the plan rejected the build. A network no solve has touched leaves `p_nom_opt` out of the file altogether, and an extendable component there is rated at the `p_nom_min` it already runs. An extendable component with neither a `p_nom_opt` nor a `p_nom_min` is a candidate rather than a plant: the generator, renewable, hydro and storage mappings leave it out and record it as skipped. **Why Operations and not Investments.** This is a deliberate scope choice tied to the deliverable, not a claim that Operations is the more natural fit: @@ -263,7 +263,7 @@ The following carrier appears in PyPSA networks alongside Generator carriers but | `reactive_power_limits` | `Union{Nothing,MinMax}` (MVAR) | — | `nothing` | defaulted | PyPSA does not carry Q limits for most generators | | `ramp_limits` | `Union{Nothing,UpDown}` (MW/min) | `n.generators.ramp_limit_up`, `ramp_limit_down` | `effective_p_nom × ramp_limit_up ÷ (dt_h × 60)` | derived | PyPSA ramp is pu/snapshot → MW/min: divide by snapshot duration (hours) and by 60. If `NaN` → `nothing`. See snapshot note in cross-cutting section | | `operation_cost` | `ThermalGenerationCost` | `n.generators.marginal_cost`, `start_up_cost`, `shut_down_cost` | See cost pattern above | derived | `marginal_cost` ($/MWh) → `CostCurve(LinearCurve(marginal_cost))`. See cost notes below | -| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | See convention discussion above | +| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, `p_nom_min` where it states a capacity a build cannot take away, else `p_nom`) | derived | See convention discussion above | | `time_limits` | `Union{Nothing,UpDown}` (hours) | `n.generators.min_up_time`, `min_down_time` | `min_up_time × dt_h` | derived | PyPSA counts snapshots; multiply by snapshot duration in hours. If both `0` → `nothing` | | `must_run` | `Bool` | — | `false` | defaulted | No PyPSA equivalent; could be inferred from `p_min_pu ≈ p_max_pu` for must-run units | | `prime_mover_type` | `PrimeMovers` | (carrier classification) | from carrier mapping table | derived | Set per carrier: e.g. `CC` for CCGT, `ST` for coal/nuclear | @@ -348,7 +348,7 @@ The reverse direction is documented in [Translation from Sienna to PyPSA](./tran | `reactive_power_limits` | `Union{Nothing,MinMax}` (MVAR) | — | `nothing` | defaulted | | | `power_factor` | `Float64` ([0,1]) | — | `1.0` | defaulted | No scalar power factor in PyPSA generator schema | | `operation_cost` | `RenewableGenerationCost` | `n.generators.marginal_cost` | `CostCurve(LinearCurve(marginal_cost))` | derived | Usually `0.0` for wind/solar; non-zero for curtailment-penalised networks | -| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, `p_nom_min` where it states a capacity a build cannot take away, else `p_nom`) | derived | | | `dynamic_injector` | `Union{Nothing,DynamicInjection}` | — | `nothing` | defaulted | | **Minimal valid JSON example (`RenewableDispatch`):** @@ -416,7 +416,7 @@ The reverse direction is documented in [Translation from Sienna to PyPSA](./tran | `rating` | `Float64` (pu) | `n.generators.p_max_pu` | `p_max_pu` | derived | Typically `1.0` | | `prime_mover_type` | `PrimeMovers` | (carrier classification) | `PVe` | derived | | | `power_factor` | `Float64` ([0,1]) | — | `1.0` | defaulted | | -| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `n.generators.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, `p_nom_min` where it states a capacity a build cannot take away, else `p_nom`) | derived | | | `dynamic_injector` | `Union{Nothing,DynamicInjection}` | — | `nothing` | defaulted | | **Minimal valid JSON example (`RenewableNonDispatch`):** @@ -465,7 +465,7 @@ Same mechanism as `RenewableDispatch`: emit a `TimeSeriesAssociation` on `max_ac | `rating` | `Float64` (pu) | `n.storage_units.p_max_pu` | `p_max_pu` | derived | | | `prime_mover_type` | `PrimeMovers` | (carrier classification) | `HY` | derived | | | `active_power_limits` | `MinMax` (MW) | `p_nom`, `p_min_pu`, `p_max_pu` | `(effective_p_nom×p_min_pu, effective_p_nom×p_max_pu)` | derived | | -| `base_power` | `Float64` (MVA) | `n.storage_units.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `n.storage_units.p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, `p_nom_min` where it states a capacity a build cannot take away, else `p_nom`) | derived | | | `operation_cost` | `HydroGenerationCost` | `n.storage_units.marginal_cost` | `CostCurve(LinearCurve(marginal_cost))` | derived | Water value if any; default `0.0` | **Minimal valid JSON example (`HydroDispatch`):** @@ -539,7 +539,7 @@ The translator emits `HydroDispatch` from PyPSA `StorageUnit` rows with carrier | `rating` | `Float64` (pu) | `p_max_pu` | `p_max_pu` | derived | | | `active_power_limits` | `MinMax` (MW) | `p_nom`, `p_min_pu`, `p_max_pu` | `(effective_p_nom×p_min_pu, effective_p_nom×p_max_pu)` | derived | | | `reactive_power_limits` | `Union{Nothing,MinMax}` | — | `nothing` | defaulted | | -| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, `p_nom_min` where it states a capacity a build cannot take away, else `p_nom`) | derived | | | `operation_cost` | `HydroGenerationCost` | `marginal_cost` | `CostCurve(LinearCurve(marginal_cost))` | derived | | | `powerhouse_elevation` | `Float64` (m) | — | `0.0` | defaulted | Not in PyPSA; default to sea level | | `ramp_limits` | `Union{Nothing,UpDown}` (MW/min) | `ramp_limit_up`, `ramp_limit_down` | `effective_p_nom × ramp_limit_up ÷ (dt_h × 60)` | derived | | @@ -612,7 +612,7 @@ The translator emits each PHS unit as a single `EnergyReservoirStorage` componen | `efficiency` | `NamedTuple{(:in, :out)}` ([0, 1] each) | `efficiency_store`, `efficiency_dispatch` | `(in=efficiency_store, out=efficiency_dispatch)` | direct | Direct mapping from StorageUnit | | `reactive_power` | `Float64` (MVAR) | — | `0.0` | defaulted | | | `reactive_power_limits` | `Union{Nothing, MinMax}` | — | `nothing` | defaulted | | -| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, else `p_nom`) | derived | | +| `base_power` | `Float64` (MVA) | `p_nom` | `effective_p_nom` (`p_nom_opt` where an extendable component has one, `p_nom_min` where it states a capacity a build cannot take away, else `p_nom`) | derived | | | `operation_cost` | `Union{StorageCost, MarketBidCost}` | `marginal_cost`, cyclic flag | `StorageCost(charge_variable_cost=0, discharge_variable_cost=marginal_cost, energy_shortage_cost=energy_surplus_cost=1e6 if cyclic else 0)` | derived | Discharge cost takes the PyPSA `marginal_cost`. Symmetric shortage / surplus penalty makes `storage_target` a hard constraint when cyclic | | `conversion_factor` | `Float64` | — | `1.0` | defaulted | No unit conversion between `storage_capacity` and energy variable | | `storage_target` | `Float64` (pu-hours of base_power) | `state_of_charge_initial`, `cyclic_state_of_charge` | `initial_storage_capacity_level * max_hours` if cyclic else `0.0` | derived | End-of-horizon energy target on the same internal scale as the energy variable (despite the schema docstring's "ratio" wording, `StorageDispatchWithReserves` reads it literally in `StateofChargeTargetConstraint`). Translator raises if `cyclic_state_of_charge` is mixed across PHS units, since the formulation attribute is per device-model | diff --git a/interop/plugins/shared/pypsa_sienna_translations/_generators.py b/interop/plugins/shared/pypsa_sienna_translations/_generators.py index 6a92b2c..dd77fe9 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_generators.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_generators.py @@ -29,7 +29,7 @@ from interop.plugins.shared.pypsa_sienna_translations._shared import ( EFFECTIVE_P_NOM, EFFECTIVE_P_NOM_DERIVATION, - holds_solved_capacity, + choose_capacity_attribute, pypsa_skip_report, pypsa_source_field, sienna_dest_field, @@ -83,6 +83,7 @@ def fill_generator_defaults(table: pl.DataFrame) -> pl.DataFrame: [ (PyPSAGeneratorCol.P_NOM, 0.0), (PyPSAGeneratorCol.P_NOM_OPT, None), + (PyPSAGeneratorCol.P_NOM_MIN, 0.0), (PyPSAGeneratorCol.P_MIN_PU, 0.0), (PyPSAGeneratorCol.P_MAX_PU, 1.0), (PyPSAGeneratorCol.MARGINAL_COST, 0.0), @@ -104,6 +105,7 @@ def fill_generator_defaults(table: pl.DataFrame) -> pl.DataFrame: PyPSAGeneratorCol.P_NOM_EXTENDABLE, PyPSAGeneratorCol.P_NOM_OPT, PyPSAGeneratorCol.P_NOM, + PyPSAGeneratorCol.P_NOM_MIN, ) @@ -406,12 +408,12 @@ def build_generator_extensions( framework=Framework.PYPSA, component=PyPSAComponent.GENERATOR, name=old[PyPSAGeneratorCol.NAME], - attribute=( - PyPSAGeneratorCol.P_NOM_OPT - if holds_solved_capacity( - old, PyPSAGeneratorCol.P_NOM_EXTENDABLE, PyPSAGeneratorCol.P_NOM_OPT - ) - else PyPSAGeneratorCol.P_NOM + attribute=choose_capacity_attribute( + old, + PyPSAGeneratorCol.P_NOM_EXTENDABLE, + PyPSAGeneratorCol.P_NOM_OPT, + PyPSAGeneratorCol.P_NOM, + PyPSAGeneratorCol.P_NOM_MIN, ), value=old[EFFECTIVE_P_NOM], unit=UNIT_MW, diff --git a/interop/plugins/shared/pypsa_sienna_translations/_hydro.py b/interop/plugins/shared/pypsa_sienna_translations/_hydro.py index 20b11d7..e88c6ea 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_hydro.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_hydro.py @@ -101,6 +101,7 @@ def fill_hydro_defaults(table: pl.DataFrame) -> pl.DataFrame: [ (PyPSAStorageUnitCol.P_NOM, 0.0), (PyPSAStorageUnitCol.P_NOM_OPT, None), + (PyPSAStorageUnitCol.P_NOM_MIN, 0.0), (PyPSAStorageUnitCol.P_MIN_PU, 0.0), (PyPSAStorageUnitCol.P_MAX_PU, 1.0), (PyPSAStorageUnitCol.MARGINAL_COST, 0.0), @@ -113,6 +114,7 @@ def fill_hydro_defaults(table: pl.DataFrame) -> pl.DataFrame: PyPSAStorageUnitCol.P_NOM_EXTENDABLE, PyPSAStorageUnitCol.P_NOM_OPT, PyPSAStorageUnitCol.P_NOM, + PyPSAStorageUnitCol.P_NOM_MIN, ) diff --git a/interop/plugins/shared/pypsa_sienna_translations/_links.py b/interop/plugins/shared/pypsa_sienna_translations/_links.py index 24eea50..ab63044 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_links.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_links.py @@ -24,8 +24,8 @@ PyPSATimeSeriesCol, ) from interop.plugins.shared.pypsa_sienna_translations._shared import ( + choose_capacity_attribute, effective_p_nom, - holds_solved_capacity, pypsa_skip_report, ) from interop.plugins.shared.sienna_constants import ( @@ -89,6 +89,7 @@ def fill_link_defaults(table: pl.DataFrame) -> pl.DataFrame: float_defaults: list[tuple[str, float | None]] = [ (PyPSALinkCol.P_NOM, 0.0), (PyPSALinkCol.P_NOM_OPT, None), + (PyPSALinkCol.P_NOM_MIN, 0.0), (PyPSALinkCol.P_MIN_PU, 0.0), (PyPSALinkCol.P_MAX_PU, 1.0), (PyPSALinkCol.EFFICIENCY, 1.0), @@ -109,6 +110,7 @@ def fill_link_defaults(table: pl.DataFrame) -> pl.DataFrame: [ pl.col(PyPSALinkCol.P_NOM).fill_nan(0.0).fill_null(0.0), pl.col(PyPSALinkCol.P_NOM_OPT).fill_nan(None), + pl.col(PyPSALinkCol.P_NOM_MIN).fill_nan(0.0).fill_null(0.0), pl.col(PyPSALinkCol.P_MIN_PU).fill_nan(0.0).fill_null(0.0), pl.col(PyPSALinkCol.P_MAX_PU).fill_nan(1.0).fill_null(1.0), pl.col(PyPSALinkCol.EFFICIENCY).fill_nan(1.0).fill_null(1.0), @@ -213,7 +215,10 @@ def _time_varying_flags(name: str, time_varying_owners: dict[str, set[str]]) -> # --- Limit / loss expressions --- _effective_p_nom = effective_p_nom( - PyPSALinkCol.P_NOM_EXTENDABLE, PyPSALinkCol.P_NOM_OPT, PyPSALinkCol.P_NOM + PyPSALinkCol.P_NOM_EXTENDABLE, + PyPSALinkCol.P_NOM_OPT, + PyPSALinkCol.P_NOM, + PyPSALinkCol.P_NOM_MIN, ) _is_bidirectional = pl.col(PyPSALinkCol.P_MIN_PU) < 0 _from_min = ( @@ -230,9 +235,14 @@ def _time_varying_flags(name: str, time_varying_owners: dict[str, set[str]]) -> def _capacity_source(old: dict[str, Any]) -> tuple[str, float]: """The PyPSA capacity attribute actually used for the power limits, and its value.""" - if holds_solved_capacity(old, PyPSALinkCol.P_NOM_EXTENDABLE, PyPSALinkCol.P_NOM_OPT): - return PyPSALinkCol.P_NOM_OPT, old[PyPSALinkCol.P_NOM_OPT] - return PyPSALinkCol.P_NOM, old[PyPSALinkCol.P_NOM] + attribute = choose_capacity_attribute( + old, + PyPSALinkCol.P_NOM_EXTENDABLE, + PyPSALinkCol.P_NOM_OPT, + PyPSALinkCol.P_NOM, + PyPSALinkCol.P_NOM_MIN, + ) + return attribute, old[attribute] # --- Translation constants --- diff --git a/interop/plugins/shared/pypsa_sienna_translations/_renewables.py b/interop/plugins/shared/pypsa_sienna_translations/_renewables.py index ae6cfe9..88f6973 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_renewables.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_renewables.py @@ -84,6 +84,7 @@ def fill_renewable_defaults(table: pl.DataFrame) -> pl.DataFrame: [ (PyPSAGeneratorCol.P_NOM, 0.0), (PyPSAGeneratorCol.P_NOM_OPT, None), + (PyPSAGeneratorCol.P_NOM_MIN, 0.0), (PyPSAGeneratorCol.P_MIN_PU, 0.0), (PyPSAGeneratorCol.P_MAX_PU, 1.0), (PyPSAGeneratorCol.MARGINAL_COST, 0.0), @@ -95,6 +96,7 @@ def fill_renewable_defaults(table: pl.DataFrame) -> pl.DataFrame: PyPSAGeneratorCol.P_NOM_EXTENDABLE, PyPSAGeneratorCol.P_NOM_OPT, PyPSAGeneratorCol.P_NOM, + PyPSAGeneratorCol.P_NOM_MIN, ) diff --git a/interop/plugins/shared/pypsa_sienna_translations/_shared.py b/interop/plugins/shared/pypsa_sienna_translations/_shared.py index d06069f..5fbf0ca 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_shared.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_shared.py @@ -38,7 +38,10 @@ EFFECTIVE_P_NOM = "_effective_p_nom" STATES_BUILT_CAPACITY = "_states_built_capacity" -EFFECTIVE_P_NOM_DERIVATION = "p_nom_opt where an extendable component has one, else p_nom" +EFFECTIVE_P_NOM_DERIVATION = ( + "p_nom_opt where an extendable component has one, p_nom_min where it states a capacity a " + "build cannot take away, else p_nom" +) def has_solved_capacity(extendable: str, opt: str) -> pl.Expr: @@ -48,28 +51,52 @@ def has_solved_capacity(extendable: str, opt: str) -> pl.Expr: return pl.col(extendable) & pl.col(opt).is_not_null() -def holds_solved_capacity(row: dict[str, Any], extendable: str, opt: str) -> bool: - """``has_solved_capacity`` for one row, so an event names the attribute the value came from.""" - return bool(row[extendable]) and row[opt] is not None - - -def effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr: - return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom)) +def has_capacity_floor(extendable: str, nom_min: str) -> pl.Expr: + """An extendable component's p_nom_min is capacity it already has, which a build cannot + take away, so an operations model may dispatch it whether or not a solve has run. + """ + return pl.col(extendable) & (pl.col(nom_min) > 0) + + +def choose_capacity_attribute( + row: dict[str, Any], extendable: str, opt: str, nom: str, nom_min: str +) -> str: + """The PyPSA attribute ``effective_p_nom`` read for one row, so an event names it.""" + if row[extendable]: + if row[opt] is not None: + return opt + if row[nom_min] > 0: + return nom_min + return nom + + +def effective_p_nom(extendable: str, opt: str, nom: str, nom_min: str) -> pl.Expr: + return ( + pl.when(has_solved_capacity(extendable, opt)) + .then(pl.col(opt)) + .when(has_capacity_floor(extendable, nom_min)) + .then(pl.col(nom_min)) + .otherwise(pl.col(nom)) + ) -def states_built_capacity(extendable: str, opt: str) -> pl.Expr: +def states_built_capacity(extendable: str, opt: str, nom_min: str) -> pl.Expr: """Whether the component has capacity an operations model may dispatch. - An extendable component no solve has sized states its capacity as a build the plan has - yet to decide, and PyPSA ignores the p_nom of an extendable component altogether. + A component that is not extendable is rated at its p_nom. An extendable one is rated at + the capacity a solve gave it, or at the p_nom_min a build cannot take away. An extendable + component with neither states its capacity as a build the plan has yet to decide, and + PyPSA ignores the p_nom of an extendable component altogether. """ - return ~pl.col(extendable) | pl.col(opt).is_not_null() + return ~pl.col(extendable) | pl.col(opt).is_not_null() | (pl.col(nom_min) > 0) -def with_effective_p_nom(table: pl.DataFrame, extendable: str, opt: str, nom: str) -> pl.DataFrame: +def with_effective_p_nom( + table: pl.DataFrame, extendable: str, opt: str, nom: str, nom_min: str +) -> pl.DataFrame: return table.with_columns( - effective_p_nom(extendable, opt, nom).alias(EFFECTIVE_P_NOM), - states_built_capacity(extendable, opt).alias(STATES_BUILT_CAPACITY), + effective_p_nom(extendable, opt, nom, nom_min).alias(EFFECTIVE_P_NOM), + states_built_capacity(extendable, opt, nom_min).alias(STATES_BUILT_CAPACITY), ) @@ -127,10 +154,10 @@ def carrier_scope_skips(naming: PyPSAComponentNaming) -> ScopeSkips: ) -UNBUILT_CANDIDATE_REASON = "are extendable and no solve has sized them" +UNBUILT_CANDIDATE_REASON = "are extendable and state no capacity they already hold" UNBUILT_CANDIDATE_NOTE = ( - "p_nom_extendable is true and the network states no p_nom_opt, so this is capacity the " - "plan may build rather than capacity an operations model may dispatch" + "p_nom_extendable is true, the network states no p_nom_opt and p_nom_min is 0, so this is " + "capacity the plan may build rather than capacity an operations model may dispatch" ) diff --git a/interop/plugins/shared/pypsa_sienna_translations/_storage.py b/interop/plugins/shared/pypsa_sienna_translations/_storage.py index 59308e0..1acdacc 100644 --- a/interop/plugins/shared/pypsa_sienna_translations/_storage.py +++ b/interop/plugins/shared/pypsa_sienna_translations/_storage.py @@ -83,6 +83,7 @@ def fill_storage_defaults(table: pl.DataFrame) -> pl.DataFrame: [ (PyPSAStorageUnitCol.P_NOM, 0.0), (PyPSAStorageUnitCol.P_NOM_OPT, None), + (PyPSAStorageUnitCol.P_NOM_MIN, 0.0), (PyPSAStorageUnitCol.P_MIN_PU, -1.0), (PyPSAStorageUnitCol.P_MAX_PU, 1.0), (PyPSAStorageUnitCol.MARGINAL_COST, 0.0), @@ -101,6 +102,7 @@ def fill_storage_defaults(table: pl.DataFrame) -> pl.DataFrame: PyPSAStorageUnitCol.P_NOM_EXTENDABLE, PyPSAStorageUnitCol.P_NOM_OPT, PyPSAStorageUnitCol.P_NOM, + PyPSAStorageUnitCol.P_NOM_MIN, ) diff --git a/tests/features/pypsa_to_sienna/renewables.feature b/tests/features/pypsa_to_sienna/renewables.feature index b15b191..57e74ac 100644 --- a/tests/features/pypsa_to_sienna/renewables.feature +++ b/tests/features/pypsa_to_sienna/renewables.feature @@ -27,7 +27,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn And the file "decisions.md" contains "| `pypsa.Generator.solar_1.bus` = bus_1 | `sienna.RenewableDispatch.solar_1.bus_name` = bus_1 | direct | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.solar_1.carrier` = solar | `sienna.RenewableDispatch.solar_1.type` = RenewableDispatch | renewable carrier -> RenewableDispatch | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.solar_1.carrier` = solar | `sienna.RenewableDispatch.solar_1.prime_mover_type` = PVe | carrier -> PrimeMovers via user defined mapping | | pypsa-to-sienna | pypsa_to_sienna_map_components |" - And the file "decisions.md" contains "| `pypsa.Generator.solar_1.p_nom` = 200.0 MW | `sienna.RenewableDispatch.solar_1.base_power` = 200.0 MW | p_nom_opt where an extendable component has one, else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" + And the file "decisions.md" contains "| `pypsa.Generator.solar_1.p_nom` = 200.0 MW | `sienna.RenewableDispatch.solar_1.base_power` = 200.0 MW | p_nom_opt where an extendable component has one, p_nom_min where it states a capacity a build cannot take away, else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.solar_1.p_max_pu` = 1.0 | `sienna.RenewableDispatch.solar_1.rating` = 1.0 | p_max_pu (per-unit nameplate rating; typically 1.0) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" # TRANSLATOR_DEFAULT_APPLIED decisions And the file "decisions.md" contains "| | `sienna.RenewableDispatch.solar_1.id` = 1 | | assigned by 1-based row position in renewable generators DataFrame | pypsa-to-sienna | pypsa_to_sienna_map_components |" diff --git a/tests/features/pypsa_to_sienna/thermal_generators.feature b/tests/features/pypsa_to_sienna/thermal_generators.feature index bc13d66..6c0129b 100644 --- a/tests/features/pypsa_to_sienna/thermal_generators.feature +++ b/tests/features/pypsa_to_sienna/thermal_generators.feature @@ -25,7 +25,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn # VALUE_DERIVED decisions And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.name` = ccgt_1 | `sienna.ThermalStandard.ccgt_1.name` = ccgt_1 | direct | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.bus` = bus_1 | `sienna.ThermalStandard.ccgt_1.bus_name` = bus_1 | direct | | pypsa-to-sienna | pypsa_to_sienna_map_components |" - And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.base_power` = 500.0 MW | p_nom_opt where an extendable component has one, else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" + And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.base_power` = 500.0 MW | p_nom_opt where an extendable component has one, p_nom_min where it states a capacity a build cannot take away, else p_nom | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.active_power` = 100.0 MW | effective_p_nom * p_min_pu (initial dispatch = min operating point) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_max_pu` = 1.0 | `sienna.ThermalStandard.ccgt_1.rating` = 1.0 | p_max_pu (per-unit nameplate rating; typically 1.0) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" And the file "decisions.md" contains "| `pypsa.Generator.ccgt_1.p_nom` = 500.0 MW | `sienna.ThermalStandard.ccgt_1.active_power_limits` = {'min': 100.0, 'max': 500.0} MW | min=effective_p_nom*p_min_pu, max=effective_p_nom*p_max_pu (static) | | pypsa-to-sienna | pypsa_to_sienna_map_components |" @@ -180,7 +180,7 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn When I run translate against "inputs/gen_extendable.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" Then the file "outputs/extensions.json" parses as JSON generator extension record for "coal_1" having "p_nom_extendable" set to true - Scenario: an extendable generator no solve has sized is a candidate, so it is left out + Scenario: an extendable generator with nothing built yet is a candidate, so it is left out Given a PyPSA network And the network contains bus "bus_1" carrier "AC" v_nom 380.0 And the network contains generator "planned_ccgt" on "bus_1" carrier "CCGT" p_nom 500.0 p_nom_extendable True @@ -188,8 +188,20 @@ Feature: pypsa_to_sienna_map_components translates PyPSA Generator rows to Sienn And a user mappings file with all standard carriers When I run translate against "inputs/gen_candidate.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" Then the file "outputs/system.json" parses as JSON with 0 components of type "ThermalStandard" - And the log contains "1 Generator(s) are extendable and no solve has sized them, so each is left out" - And the file "decisions.md" contains "p_nom_extendable is true and the network states no p_nom_opt, so this is capacity the plan may build rather than capacity an operations model may dispatch" + And the log contains "1 Generator(s) are extendable and state no capacity they already hold, so each is left out" + And the file "decisions.md" contains "p_nom_extendable is true, the network states no p_nom_opt and p_nom_min is 0, so this is capacity the plan may build rather than capacity an operations model may dispatch" + + Scenario: an extendable generator no solve has sized keeps the capacity a build cannot take away + Given a PyPSA network + And the network contains bus "bus_1" carrier "AC" v_nom 380.0 + And the network contains generator "running_ccgt" on "bus_1" carrier "CCGT" p_nom 500.0 p_nom_extendable True + And generator "running_ccgt" has p_nom_min 200 + And the network is saved as "inputs/gen_floor.nc" + And a user mappings file with all standard carriers + When I run translate against "inputs/gen_floor.nc" pipeline "pypsa-to-sienna" sink output "outputs/system.json" + Then the file "outputs/system.json" parses as JSON with component "ThermalStandard" named "running_ccgt" having "base_power" set to 200.0 + And the file "outputs/system.json" parses as JSON with component "ThermalStandard" named "running_ccgt" having "active_power_limits.max" set to 200.0 + And the file "decisions.md" contains "`pypsa.Generator.running_ccgt.p_nom_min` = 200.0 MW | `sienna.ThermalStandard.running_ccgt.active_power_limits`" Scenario: generator without p_nom_extendable records the flag false in extensions Given a PyPSA network From 6d3826c15aa17d86199bae6341c7dc0f41de558a Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 21:15:21 +0000 Subject: [PATCH 11/17] Record what eight review rounds found on this branch The ledger is committed with the code it judges, so a rule earned here is a rule a teammate and a fresh clone start with. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- code-review/ledger.jsonl | 68 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/code-review/ledger.jsonl b/code-review/ledger.jsonl index c3e3761..3f2a204 100644 --- a/code-review/ledger.jsonl +++ b/code-review/ledger.jsonl @@ -59,3 +59,71 @@ {"id": "r1-30", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 139, "severity": "minor", "label": "suggestion", "claim": "The `_Drops` dataclass holds one list, and one call site builds it.", "suggested_fix": "Delete the `_Drops` class, pass `unpriced: list[str]` to `_map_one` in its place, append the name to that list, and give `_warn_unpriced_candidates` the list directly. Drop `field` from the `dataclasses` import, which then has no other user in the module. Net: -7 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} {"id": "r1-31", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py", "line": 273, "severity": "major", "label": "issue", "claim": "`_P_NOM_MIN_DERIVATION` and `_P_NOM_MAX_DERIVATION` say `Max Capacity x Units`, but the value they explain can be the `Rating` instead.", "suggested_fix": "Make `_p_nom_min` and `_p_nom_max` follow `_p_nom`: where `mapping.rating_as_capacity` is set, cite the `Rating` source as well and use a derivation text that names it. Net: +8 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} {"id": "r1-32", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 71, "severity": "nit", "label": "nitpick", "claim": "`EXTENDABLE_NOTE`, `EXTENDABLE_DERIVATION` and `NOT_A_CANDIDATE` have no leading underscore, but no other module reads them.", "suggested_fix": "Rename the three to `_EXTENDABLE_NOTE`, `_EXTENDABLE_DERIVATION` and `_NOT_A_CANDIDATE`, matching `_P_NOM_MIN_DERIVATION` and `_SOC_NOTE` beside them. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T17:30:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/pypsa_sienna_translations/_generators.py", "line": 100, "severity": "major", "label": "issue", "claim": "The diff copies the effective_p_nom rule into four more modules, so one rule now stands in seven places.", "suggested_fix": "The code now, repeated in `_generators.py:100`, `_renewables.py:92`, `_hydro.py:110`, `_storage.py:98` and (before the diff) `_links.py:212`, with the Python twin at `_generators.py:408` and `_links.py:231`:\n\n```python\npl.when(\n pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE) & (pl.col(PyPSAGeneratorCol.P_NOM_OPT) > 0)\n)\n.then(pl.col(PyPSAGeneratorCol.P_NOM_OPT))\n.otherwise(pl.col(PyPSAGeneratorCol.P_NOM))\n.alias(_EFFECTIVE_P_NOM)\n```\n\nThe code to use, in `pypsa_sienna_translations/_shared.py`, which already exists to delete this kind of repetition:\n\n```python\nEFFECTIVE_P_NOM = \"_effective_p_nom\"\nEFFECTIVE_P_NOM_DERIVATION = \"p_nom_opt where an extendable component has one, else p_nom\"\n\n\ndef has_solved_capacity(extendable: str, opt: str) -> pl.Expr:\n \"\"\"True where an extendable component holds a solved capacity.\"\"\"\n return pl.col(extendable) & (pl.col(opt) > 0)\n\n\ndef effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr:\n return (\n pl.when(has_solved_capacity(extendable, opt))\n .then(pl.col(opt))\n .otherwise(pl.col(nom))\n .alias(EFFECTIVE_P_NOM)\n )\n```\n\nEach module then calls it, and the Python twin calls `has_solved_capacity`'s row-wise partner:\n\n```python\nreturn table.with_columns(\n effective_p_nom(\n PyPSAGeneratorCol.P_NOM_EXTENDABLE,\n PyPSAGeneratorCol.P_NOM_OPT,\n PyPSAGeneratorCol.P_NOM,\n )\n)\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 106, "severity": "major", "label": "issue", "claim": "`SourceGenerator` states the candidate rule a second time in plain floats, and `_expansion.py` already states it in `Decision` values.", "suggested_fix": "The code now holds the whole rule twice. In `_generator_derivation.py`:\n\n```python\n@property\ndef max_units_built(self) -> float:\n return _value(self.props, PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD)\n\n@property\ndef is_candidate(self) -> bool:\n return self.max_units_built > NOTHING_TO_BUILD\n\n@property\ndef buildable(self) -> float:\n return self.max_units_built * self.max_capacity\n\n@property\ndef p_nom(self) -> float:\n return self.existing or self.buildable\n```\n\nAnd in `_expansion.py`, the same three rules again:\n\n```python\ndef derive_p_nom(source: CandidateSource) -> Decision:\n if source.rated.existing.value or not source.is_candidate:\n return source.rated.existing\n return derive_buildable(source)\n```\n\nThe code to use keeps one owner, `_expansion.py`, and reads the value off its `Decision`:\n\n```python\n@cached_property\ndef candidate(self) -> CandidateSource:\n return read_candidate(self)\n\n@property\ndef is_candidate(self) -> bool:\n return self.candidate.is_candidate\n\n@property\ndef p_nom(self) -> float:\n \"\"\"What the generator has, or what it may build where it has nothing yet.\"\"\"\n return derive_p_nom(self.candidate).value\n```\n\nDelete `max_units_built` and `buildable`, and let `derive_generator` read `source.candidate` in place of its own `read_candidate(source)` call.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-03", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 105, "severity": "major", "label": "issue", "claim": "`map_generators` passes a mutable dict into `_map_one` to collect the names one warning lists, and the storage path warns about the same rule a different way.", "suggested_fix": "The code now threads an accumulator through a per-generator mapper:\n\n```python\nunpriced: dict[str, list[str]] = {}\ntranslated = [\n one\n for generator in generators.collect().iter_rows(named=True)\n if generator[PlexosObjectCol.NAME] not in storage_turbines\n if (one := _map_one(generator, lookups, reporter, unpriced)) is not None\n]\n_warn_unpriced_candidates(unpriced)\n```\n\nThe code to use puts the warning beside the rule in `_expansion.py`, so both paths call it:\n\n```python\ndef warn_unpriced_builds(plexos_class: PlexosClass, skipped: Sequence[UnpricedCandidate]) -> None:\n \"\"\"One warning per property a candidate left out, naming a few of the objects.\"\"\"\n by_property: dict[str, list[str]] = {}\n for one in skipped:\n by_property.setdefault(one.unpriced.plexos_property, []).append(one.name)\n for plexos_property, names in by_property.items():\n log.warning(\n \"plexos: %d candidate %s(s) state no %s, so each is left out. Each one: %s\",\n len(names), plexos_class, plexos_property, name_a_few(sorted(names)),\n )\n```\n\n`_map_one` then returns `_TranslatedGenerator | UnpricedCandidate | None`, and `map_generators` splits the list once. `_storage_units.derive_storage_units` collects the same `UnpricedCandidate` values and calls the same function.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-04", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 106, "severity": "minor", "label": "issue", "claim": "`ExpansionDecisions` names `PyPSAGeneratorCol` columns, and `StorageUnitMapping` builds its storage row and its events from them.", "suggested_fix": "The code now:\n\n```python\nclass ExpansionDecisions:\n p_nom_min: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MIN, unit=UNIT_MW)\n overnight_cost: Decision = maps_to(PyPSAGeneratorCol.OVERNIGHT_COST, unit=UNIT_DOLLARS_PER_MW)\n```\n\n`_storage_units.write_storage_units` then keys a `STORAGE_UNITS_DESTINATION_SCHEMA` row from those generator constants.\n\nThe code to use names the shared columns once, in `pypsa_constants.py`:\n\n```python\nclass PyPSAExpansionCol(StrEnum):\n \"\"\"The expansion columns a Generator and a StorageUnit spell the same way.\"\"\"\n\n P_NOM_MIN = \"p_nom_min\"\n P_NOM_MAX = \"p_nom_max\"\n OVERNIGHT_COST = \"overnight_cost\"\n DISCOUNT_RATE = \"discount_rate\"\n LIFETIME = \"lifetime\"\n FOM_COST = \"fom_cost\"\n```\n\n`ExpansionDecisions` and both destination schemas then read `PyPSAExpansionCol`.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-05", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_units.py", "line": 239, "severity": "minor", "label": "suggestion", "claim": "`_carry_to_extensions` writes a name-only `StorageExtension` for every storage unit the model cannot build.", "suggested_fix": "The code now writes one record per unit, and the sidecar drops the null fields, so a fixed unit leaves `{\"name\": \"old_bat\"}`:\n\n```python\nrecords = [\n StorageExtension(\n name=mapping.name,\n unit_size_mw=read_sidecar_value(mapping.expansion.unit_size),\n technical_life_years=read_sidecar_value(mapping.expansion.technical_life),\n )\n for mapping in mappings\n]\n```\n\nThe code to use writes a record only where there is something to carry, with the `is_candidate` property `ExpansionDecisions` already declares:\n\n```python\nrecords = [\n StorageExtension(\n name=mapping.name,\n unit_size_mw=read_sidecar_value(mapping.expansion.unit_size),\n technical_life_years=read_sidecar_value(mapping.expansion.technical_life),\n )\n for mapping in mappings\n if mapping.expansion.is_candidate\n]\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-06", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 129, "severity": "major", "label": "issue", "claim": "A candidate battery divides its energy capacity by the buildable power, so its max_hours becomes Max Units Built times too small.", "suggested_fix": "```python\ndef _derive_battery(rated: RatedObject) -> StorageUnitMapping:\n capacity = _battery_energy_capacity(rated)\n # The stated Capacity is the energy beside one unit's Max Power, so per-unitise it\n # against that power rather than against the capacity the object may build.\n max_hours = derive_max_hours(capacity, rated.candidate.rated.unit_size.value)\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-07", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 192, "severity": "major", "label": "issue", "claim": "A candidate that states no Economic Life keeps PyPSA's default lifetime of infinity, so PyPSA annuitises the Build Cost as a perpetuity.", "suggested_fix": "```python\n_NO_ECONOMIC_LIFE_NOTE = (\n \"a candidate with no Economic Life gives PyPSA no period to annuitise its Build Cost \"\n \"over, so PyPSA prices the build as a perpetuity\"\n)\n\n_PRICES_A_BUILD = (\n UnpricedBuild(PlexosProperty.BUILD_COST, UNIT_DOLLARS_PER_MW, _NO_BUILD_COST_NOTE),\n UnpricedBuild(PlexosProperty.WACC, None, _NO_WACC_NOTE),\n UnpricedBuild(PlexosProperty.ECONOMIC_LIFE, UNIT_YEARS, _NO_ECONOMIC_LIFE_NOTE),\n)\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-08", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/pypsa_sienna_translations/_generators.py", "line": 101, "severity": "major", "label": "issue", "claim": "An extendable component that a solve builds none of has p_nom_opt 0, so the new rule reads its p_nom as installed capacity.", "suggested_fix": "```python\n # A solve that builds nothing writes p_nom_opt 0; only an unsolved network\n # leaves the column out, so keep that difference rather than reading 0 as unsolved.\n (PyPSAGeneratorCol.P_NOM_OPT, None),\n...\n pl.when(\n pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE)\n & pl.col(PyPSAGeneratorCol.P_NOM_OPT).is_not_null()\n )\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-09", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 118, "severity": "minor", "label": "question", "claim": "The docstring reads Max Power as one unit's power, but the code gives the same value to a battery that states Units above one.", "suggested_fix": "```python\n if units:\n counted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units)\n existing = Decision.derived(\n max_power * units, [power, counted], \"Max Power x Units\"\n )\n return RatedCapacity(existing=existing, unit_size=unit_size)\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-10", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 72, "severity": "minor", "label": "issue", "claim": "This paragraph states a rule the battery path breaks, so it misleads the reader.", "suggested_fix": "Delete the paragraph. `_battery_rating` in `_batteries.py` returns `existing=unit_size` for every battery that states any units, so the two fields stay equal where the model states more than one unit.\n\nThe code now:\n\n```python\n@dataclass(frozen=True)\nclass RatedCapacity:\n \"\"\"The rated power an object already has, beside the rated power of one unit of it.\n\n The two differ wherever a model states more than one unit, and wherever a candidate\n states no unit at all.\n \"\"\"\n\n existing: Decision\n unit_size: Decision\n```\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass RatedCapacity:\n \"\"\"The rated power an object already has, beside the rated power of one unit of it.\"\"\"\n\n existing: Decision\n unit_size: Decision\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-11", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 179, "severity": "nit", "label": "nitpick", "claim": "This docstring paraphrases the name and repeats the CandidateSource class docstring.", "suggested_fix": "Delete it. The name, the parameter and the return type say the same thing, and `CandidateSource` already carries the sentence \"One staged PLEXOS object as the expansion rule reads it.\"\n\nThe code now:\n\n```python\ndef read_candidate(source: SourceGenerator) -> CandidateSource:\n \"\"\"One staged Generator as the shared expansion rule reads it.\"\"\"\n capacity = SourceValue(\n```\n\nThe code to use:\n\n```python\ndef read_candidate(source: SourceGenerator) -> CandidateSource:\n capacity = SourceValue(\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-12", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 177, "severity": "nit", "label": "nitpick", "claim": "This docstring narrates the two lines below it and repeats the SourceGenerator.p_nom summary.", "suggested_fix": "Delete it. The body reads the same way, and `SourceGenerator.p_nom` in `_generator_derivation.py` opens with the same sentence.\n\nThe code now:\n\n```python\ndef derive_p_nom(source: CandidateSource) -> Decision:\n \"\"\"What the object has, or what it may build where it has nothing yet.\"\"\"\n if source.rated.existing.value or not source.is_candidate:\n return source.rated.existing\n return derive_buildable(source)\n```\n\nThe code to use:\n\n```python\ndef derive_p_nom(source: CandidateSource) -> Decision:\n if source.rated.existing.value or not source.is_candidate:\n return source.rated.existing\n return derive_buildable(source)\n```", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-13", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 117, "severity": "minor", "label": "suggestion", "claim": "The property `ExpansionDecisions.is_candidate` has no caller in production code or tests.", "suggested_fix": "Delete the `is_candidate` property from `ExpansionDecisions`, together with its `@property` line and the blank line above it. Net: -5 lines.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-14", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 353, "severity": "minor", "label": "suggestion", "claim": "The field `RatedObject.stated_units` is written at construction and never read.", "suggested_fix": "Delete the `stated_units` field from `RatedObject` and drop the `staged.stated_units` argument from the `RatedObject(...)` call at the end of `rate_object`, which then fits on one line. Net: -3 lines.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-15", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 35, "severity": "major", "label": "issue", "claim": "The constant name `NOT_A_CANDIDATE` does not describe what `_from_property` and `_discount_rate` return it for, because a genuine candidate that states no Economic Life, no FO&M Charge or no Technical Life also gets it.", "suggested_fix": "Rename the constant to what it means, for example `NO_EVENT` or `NOTHING_TO_REPORT`, and update its ten uses in _expansion.py. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-16", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 70, "severity": "minor", "label": "issue", "claim": "The `RatedCapacity` docstring says the two values differ wherever a model states more than one unit, which is untrue on the Battery path.", "suggested_fix": "Say which readings make the two differ: a candidate that has built nothing, and a Generator or turbine that states more than one unit. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-17", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 90, "severity": "nit", "label": "chore", "claim": "`P_NOM_DERIVATION` and `P_NOM_RATING_DERIVATION` lost their leading underscore, but no other module imports them.", "suggested_fix": "Restore the leading underscore on both constants and on their two uses. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T18:10:17Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 110, "severity": "major", "label": "issue", "claim": "For a Battery with more than one unit, `_battery_rating` gives the power of one unit as the power the object already has.", "suggested_fix": "The code now:\n\n```python\nunit_size = Decision.derived(max_power, [power], DIRECT_DERIVATION)\nunits = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS)\nif units:\n return RatedCapacity(existing=unit_size, unit_size=unit_size)\ncounted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units)\nnothing_built = Decision.derived(0.0, [power, counted], NOTHING_BUILT_DERIVATION)\nreturn RatedCapacity(existing=nothing_built, unit_size=unit_size)\n```\n\nThe code to use, which reads `Units` as `_turbine_rating` reads it:\n\n```python\nunits = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS)\ncounted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units)\nreturn RatedCapacity(\n existing=Decision.derived(\n max_power * units, [power, counted], _EXISTING_FROM_UNITS_DERIVATION\n ),\n unit_size=Decision.derived(max_power, [power], DIRECT_DERIVATION),\n)\n```\n\nAdd a scenario for a Battery with `Units` 2, and state in the mapping document which reading of `Max Power` is correct.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 113, "severity": "major", "label": "suggestion", "claim": "An unpriced candidate reaches its warning through two different channels: a wider return type here, and an optional field on `SkippedComponent`.", "suggested_fix": "The code now, in two files:\n\n```python\n# _generators.py: _map_one returns _TranslatedGenerator | UnpricedCandidate | None\nwarn_unpriced_builds([one for one in outcomes if isinstance(one, UnpricedCandidate)])\ntranslated = [one for one in outcomes if isinstance(one, _TranslatedGenerator)]\n\n# _storage_units.py\nfor skipped in storage_units.skipped:\n reporter.record_skipped(skipped.source, skipped.note)\n if skipped.unpriced is None:\n warn_about_skipped(skipped)\nwarn_unpriced_builds([s.unpriced for s in storage_units.skipped if s.unpriced])\n```\n\nThe code to use, with one carrier for every skip:\n\n```python\n@dataclass(frozen=True)\nclass SkippedComponent:\n source: SourceValue\n note: str\n # The class and the property one grouped warning names, where a skip groups.\n warn_with: tuple[str, str] | None = None\n\n\ndef warn_about_skips(skipped: Sequence[SkippedComponent]) -> None:\n \"\"\"One line for each object that stands alone, and one line for each group.\"\"\"\n```\n\nLet `_map_one` return a `SkippedComponent` for every skip it makes, and let `map_generators` record the skips and call `warn_about_skips` once.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-03", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 225, "severity": "minor", "label": "suggestion", "claim": "`find_unpriced_build` reads the candidate rule from a raw property dictionary, so the module states that rule twice.", "suggested_fix": "The code now:\n\n```python\ndef find_unpriced_build(\n plexos_class: PlexosClass, name: str, props: dict[str, float]\n) -> UnpricedCandidate | None:\n if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD:\n return None\n```\n\nThe code to use:\n\n```python\ndef find_unpriced_build(source: CandidateSource) -> UnpricedCandidate | None:\n if not source.is_candidate:\n return None\n unpriced = next(\n (one for one in _PRICES_A_BUILD if one.plexos_property not in source.props), None\n )\n if unpriced is None:\n return None\n return UnpricedCandidate(source.plexos_class, source.name, unpriced)\n```\n\n`_map_one` already holds `source.candidate`. In `rate_object`, build the `CandidateSource` before the call and pass it.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-04", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 111, "severity": "minor", "label": "suggestion", "claim": "`ExpansionDecisions` names the StorageUnit destination columns with the `PyPSAGeneratorCol` enum.", "suggested_fix": "The code now:\n\n```python\nclass ExpansionDecisions:\n p_nom_extendable: Decision = maps_to(PyPSAGeneratorCol.P_NOM_EXTENDABLE)\n p_nom_min: Decision = maps_to(PyPSAGeneratorCol.P_NOM_MIN, unit=UNIT_MW)\n```\n\nThe code to use, with one enum for the columns both components share:\n\n```python\n# pypsa_constants.py\nclass PyPSAExpansionCol(StrEnum):\n \"\"\"The expansion columns a Generator and a StorageUnit spell the same way.\"\"\"\n\n P_NOM_EXTENDABLE = \"p_nom_extendable\"\n P_NOM_MIN = \"p_nom_min\"\n\n\n# _expansion.py\np_nom_min: Decision = maps_to(PyPSAExpansionCol.P_NOM_MIN, unit=UNIT_MW)\n```", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-05", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/pypsa_sienna_translations/_links.py", "line": 89, "severity": "minor", "label": "suggestion", "claim": "`fill_link_defaults` repeats by hand what the canonical `fill_defaults` helper already does.", "suggested_fix": "The code to use, which keeps the same behaviour:\n\n```python\ndef fill_link_defaults(table: pl.DataFrame) -> pl.DataFrame:\n \"\"\"Add optional PyPSA link columns absent when all links share the PyPSA default.\"\"\"\n return fill_defaults(\n table,\n [\n (PyPSALinkCol.P_NOM, 0.0),\n (PyPSALinkCol.P_NOM_OPT, None),\n (PyPSALinkCol.P_MIN_PU, 0.0),\n (PyPSALinkCol.P_MAX_PU, 1.0),\n (PyPSALinkCol.EFFICIENCY, 1.0),\n ],\n [(PyPSALinkCol.P_NOM_EXTENDABLE, False), (PyPSALinkCol.ACTIVE, True)],\n [(PyPSALinkCol.CARRIER, \"\"), (PyPSALinkCol.BUS2, \"\"), (PyPSALinkCol.BUS3, \"\")],\n )\n```", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-06", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 180, "severity": "major", "label": "issue", "claim": "A PLEXOS candidate generator with Units 0 now reaches the Sienna operations system at the full capacity it may build.", "suggested_fix": "Drop an extendable component on the operations leg, as the design spec states. For example, in `interop/plugins/shared/pypsa_sienna_translations/_generators.py`:\n\n```python\nGENERATOR_EXTENDABLE_SKIP = pypsa_skip_report(\n component=PyPSAComponent.GENERATOR,\n reason=\"is extendable, so it is a candidate rather than built capacity\",\n counted_noun=\"Generator(s)\",\n note=\"p_nom_extendable is true, so this generator belongs to the investments portfolio\",\n)\n# apply beside the existing skip rules, keeping only rows where\n# `~pl.col(PyPSAGeneratorCol.P_NOM_EXTENDABLE)`\n```\n\nApply the same rule to storage units.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-07", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/pypsa_sienna_translations/_links.py", "line": 215, "severity": "minor", "label": "issue", "claim": "The Line mapping keeps the old rule, so a rejected line build reads `s_nom` while a rejected generator build reads 0.", "suggested_fix": "Give `_lines.py` the same reading:\n\n```python\n# interop/plugins/shared/pypsa_sienna_translations/_lines.py\n_effective_s_nom = effective_p_nom(\n PyPSALineCol.S_NOM_EXTENDABLE, PyPSALineCol.S_NOM_OPT, PyPSALineCol.S_NOM\n)\n\n\ndef _rating_uses_opt(old: dict[str, Any]) -> bool:\n return holds_solved_capacity(\n old, PyPSALineCol.S_NOM_EXTENDABLE, PyPSALineCol.S_NOM_OPT\n )\n```\n\nAlso change the `S_NOM_OPT` default in `fill_line_defaults` from `0.0` to `None`.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-08", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 119, "severity": "minor", "label": "issue", "claim": "`_battery_rating` gives a Battery that states Units above one the power of a single unit as its existing capacity.", "suggested_fix": "Count the units the object holds, as the Generator path does:\n\n```python\n counted = SourceValue(PlexosClass.BATTERY, staged.name, PlexosProperty.UNITS, units)\n if units:\n return RatedCapacity(\n existing=Decision.derived(\n max_power * units, [power, counted], _POWER_FROM_UNITS_DERIVATION\n ),\n unit_size=unit_size,\n )\n```\n\nIf `Max Power` is instead the power of every unit together, divide it by `units` to get `unit_size`.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-09", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/pypsa_sienna_translations/_shared.py", "line": 58, "severity": "nit", "label": "nitpick", "claim": "This docstring paraphrases the function name and duplicates the EFFECTIVE_P_NOM_DERIVATION constant above it.", "suggested_fix": "Delete the docstring.\n\nThe code now:\n\n```python\nEFFECTIVE_P_NOM_DERIVATION = \"p_nom_opt where an extendable component has one, else p_nom\"\n\n...\n\ndef effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr:\n \"\"\"The capacity to translate: the solved one where there is one, else the stated one.\"\"\"\n return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom))\n```\n\nThe code to use:\n\n```python\ndef effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr:\n return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom))\n```", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-10", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/pypsa_sienna_translations/_shared.py", "line": 44, "severity": "nit", "label": "nitpick", "claim": "The first docstring line paraphrases the name has_solved_capacity, so it is filler.", "suggested_fix": "Delete the summary line and keep the paragraph, which names the trap.\n\nThe code now:\n\n```python\ndef has_solved_capacity(extendable: str, opt: str) -> pl.Expr:\n \"\"\"True where a solve wrote a capacity for an extendable component.\n\n A solve that builds none of a component writes p_nom_opt 0; only a network no solve has\n touched leaves the column out, which stages as null.\n \"\"\"\n return pl.col(extendable) & pl.col(opt).is_not_null()\n```\n\nThe code to use:\n\n```python\ndef has_solved_capacity(extendable: str, opt: str) -> pl.Expr:\n \"\"\"A solve that builds none of a component writes p_nom_opt 0; only a network no solve\n has touched leaves the column out, which stages as null.\n \"\"\"\n return pl.col(extendable) & pl.col(opt).is_not_null()\n```", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-11", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 130, "severity": "minor", "label": "issue", "claim": "The new property `SourceGenerator.existing` has no caller anywhere in the repository.", "suggested_fix": "Delete the `existing` property and its blank separator line. The module-level `_existing()` already builds the same reading as a `Decision`, and that is what `read_candidate` uses. Net: -6 lines.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-12", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 91, "severity": "nit", "label": "nitpick", "claim": "`P_NOM_DERIVATION` and `P_NOM_RATING_DERIVATION` lost the leading underscore they had on main, but no sibling module reads them.", "suggested_fix": "Rename both back to `_P_NOM_DERIVATION` and `_P_NOM_RATING_DERIVATION` and update the two uses at lines 191 and 193. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-13", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "docs/translation_mappings/translation-from-plexos-to-pypsa.md", "line": 376, "severity": "major", "label": "issue", "claim": "This paragraph says a Battery or a turbine with `Units 0` is not written, which the branch's own candidate rule no longer does.", "suggested_fix": "Qualify the paragraph: a rated power of zero skips the object only where it states no `Max Units Built`; a candidate with `Units 0` takes the capacity it may build as its `p_nom` and is written. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-14", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "docs/translation_mappings/translation-from-plexos-to-pypsa.md", "line": 356, "severity": "minor", "label": "issue", "claim": "The Battery `p_nom` and `max_hours` rows still say `Max Power` and `divided by p_nom`, which contradicts the prose the branch rewrote three lines below.", "suggested_fix": "Say `Max Power`, or `Max Power × Max Units Built` for a candidate with nothing built, on the `p_nom` row, and say the energy of one unit divided by `Max Power` on the `max_hours` row, matching the paragraph at lines 365-371. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T18:51:50Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 180, "severity": "major", "label": "issue", "claim": "A candidate generator states `Units` 0, so `_dated_capacity_rows` scales its dated `Max Capacity` series by zero.", "suggested_fix": "This guard now lets a generator with `Units` 0 through, and `_dated_capacity_rows` still divides the unit count by `p_nom`.\n\nThe code now (`_generators.py:328`):\n\n```python\n source_series_name=PlexosProperty.MAX_CAPACITY,\n scaling_factor=mapping.units / mapping.p_nom,\n timing=timing,\n```\n\nThe code to use:\n\n```python\n source_series_name=PlexosProperty.MAX_CAPACITY,\n # p_nom stands for the units the object has, or the units a candidate may build.\n scaling_factor=(mapping.units or mapping.candidate.max_units_built) / mapping.p_nom,\n timing=timing,\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 201, "severity": "minor", "label": "suggestion", "claim": "`find_unpriced_build` reads `Max Units Built` from a raw property dictionary, so the candidate rule lives in two places.", "suggested_fix": "The code now (`_expansion.py:201`, and the call at `_storage_shared.py:346`):\n\n```python\ndef find_unpriced_build(\n plexos_class: PlexosClass, name: str, props: dict[str, float]\n) -> SkippedComponent | None:\n if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD:\n return None\n\n unpriced = find_unpriced_build(rating.plexos_class, staged.name, staged.properties)\n if unpriced is not None:\n return unpriced\n candidate = CandidateSource(...)\n```\n\nThe code to use:\n\n```python\ndef find_unpriced_build(source: CandidateSource) -> SkippedComponent | None:\n if not source.is_candidate:\n return None\n unpriced = next(\n (one for one in _PRICES_A_BUILD if one.plexos_property not in source.props), None\n )\n\n candidate = CandidateSource(...)\n unpriced = find_unpriced_build(candidate)\n```\n\n`_generators.py:187` then reads `find_unpriced_build(source.candidate)`.", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-03", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/decisions.py", "line": 100, "severity": "minor", "label": "suggestion", "claim": "`warn_about_skips` writes the group warning that `_warn_dropped` in `translation_runner.py` writes already.", "suggested_fix": "The code now (`decisions.py:100` beside `translation_runner.py:292`):\n\n```python\n log.warning(\n \"plexos: %d %s %s, so each is left out. Each one: %s\",\n len(names), group.counted, group.reason, name_a_few(sorted(names)),\n )\n\n log.warning(\n \"%s: %d %s %s, so each is left out. %s: %s\",\n report.pipeline, skipped.height, report.counted_noun, report.reason,\n listing.label, name_a_few(named),\n )\n```\n\nThe code to use, in `translation_runner.py`, called by both:\n\n```python\ndef warn_left_out(\n pipeline: str, counted_noun: str, reason: str, names: Sequence[str],\n label: str = DROPPED_NAMES_LABEL,\n) -> None:\n log.warning(\n \"%s: %d %s %s, so each is left out. %s: %s\",\n pipeline, len(names), counted_noun, reason, label, name_a_few(sorted(names)),\n )\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-04", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 180, "severity": "major", "label": "issue", "claim": "A candidate with no units now reaches `_dated_capacity_rows`, which scales its dated Max Capacity by units 0.", "suggested_fix": "```python\ndef _capacity_unit_count(mapping: GeneratorMapping) -> float:\n \"\"\"The unit count p_nom stands for: what runs, or what a fresh candidate may build.\"\"\"\n return mapping.units or mapping.candidate.max_units_built\n\n\n# in _dated_capacity_rows\n scaling_factor=_capacity_unit_count(mapping) / mapping.p_nom,\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-05", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py", "line": 229, "severity": "minor", "label": "issue", "claim": "A candidate turbine keeps `max_hours` against the power it has, so a build scales a head reservoir that does not grow.", "suggested_fix": "```python\n expansion = derive_expansion(rated.candidate)\n # A head reservoir does not grow with the turbines on it, so a candidate reads its\n # hours against the most a build may reach.\n rated_power = expansion.p_nom_max.value or rated.p_nom.value\n max_hours = derive_max_hours(\n _reservoir_energy_capacity(head),\n rated_power,\n _VOLUME_NOT_ENERGY_NOTE if unreadable else MAX_HOURS_NOTE,\n )\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-06", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/steps/pypsa_to_sienna_map_components.py", "line": 328, "severity": "minor", "label": "suggestion", "claim": "This paragraph repeats UNBUILT_CANDIDATE_NOTE and the states_built_capacity docstring, so it duplicates another artifact.", "suggested_fix": "Delete the second paragraph and keep the one-line docstring, which matches the neighbouring methods.\n\nThe code now:\n\n```python\n def _drop_unbuilt_candidates(\n self, table: pl.DataFrame, mapping: ComponentMapping\n ) -> pl.DataFrame:\n \"\"\"Leave out the candidates, which a Sienna operations system has no capacity for.\n\n A component a solve has sized carries the capacity it built, and one whose capacity is\n fixed carries what it has. Anything else is a build the plan has yet to decide, and\n writing it here would give the operations system a plant nobody has built.\n \"\"\"\n rule = unbuilt_candidate_skip(PYPSA_COMPONENT_NAMING[mapping.source_table])\n```\n\nThe code to use:\n\n```python\n def _drop_unbuilt_candidates(\n self, table: pl.DataFrame, mapping: ComponentMapping\n ) -> pl.DataFrame:\n \"\"\"Leave out the candidates, which a Sienna operations system has no capacity for.\"\"\"\n rule = unbuilt_candidate_skip(PYPSA_COMPONENT_NAMING[mapping.source_table])\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-07", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 206, "severity": "minor", "label": "suggestion", "claim": "This paragraph argues for the grouped warning, so it is reviewer justification.", "suggested_fix": "Delete the second paragraph. Keep the first line.\n\nThe code now:\n\n```python\ndef find_unpriced_build(\n plexos_class: PlexosClass, name: str, props: dict[str, float]\n) -> SkippedComponent | None:\n \"\"\"The first property a candidate leaves out that stops PyPSA pricing its build.\n\n Every candidate leaving out the same property is named in one warning rather than a\n line each, because a model that omits one of these omits it wholesale.\n \"\"\"\n if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD:\n```\n\nThe code to use:\n\n```python\ndef find_unpriced_build(\n plexos_class: PlexosClass, name: str, props: dict[str, float]\n) -> SkippedComponent | None:\n \"\"\"The first property a candidate leaves out that stops PyPSA pricing its build.\"\"\"\n if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD:\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-08", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "tests/features/plexos_to_pypsa/generators.feature", "line": 542, "severity": "nit", "label": "nitpick", "claim": "Each comment repeats the decisions.md text the same scenario asserts below, so both duplicate another artifact.", "suggested_fix": "Delete both comments. The assertions five lines below state the same reason.\n\nThe code now:\n\n```gherkin\n Then the PyPSA network \"outputs/network.nc\" has no generator \"Unpriced_REZ\"\n # PyPSA annuitises an overnight cost with the discount rate, and refuses a network stating one without the other.\n And the PyPSA network \"outputs/network.nc\" has no generator \"Undiscounted_REZ\"\n # PyPSA annuitises over the lifetime, and a lifetime of infinity prices the build as a perpetuity.\n And the PyPSA network \"outputs/network.nc\" has no generator \"Everlasting_REZ\"\n And the PyPSA network \"outputs/network.nc\" generator \"Priced_REZ\" is extendable\n And the file \"decisions.md\" contains \"a candidate with no Build Cost prices building nothing, so an expansion would take it for free\"\n And the file \"decisions.md\" contains \"a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost over\"\n And the file \"decisions.md\" contains \"a candidate with no Economic Life gives PyPSA no period to annuitise its Build Cost over\"\n```\n\nThe code to use:\n\n```gherkin\n Then the PyPSA network \"outputs/network.nc\" has no generator \"Unpriced_REZ\"\n And the PyPSA network \"outputs/network.nc\" has no generator \"Undiscounted_REZ\"\n And the PyPSA network \"outputs/network.nc\" has no generator \"Everlasting_REZ\"\n And the PyPSA network \"outputs/network.nc\" generator \"Priced_REZ\" is extendable\n And the file \"decisions.md\" contains \"a candidate with no Build Cost prices building nothing, so an expansion would take it for free\"\n And the file \"decisions.md\" contains \"a candidate with no WACC gives PyPSA no discount rate to annuitise its Build Cost over\"\n And the file \"decisions.md\" contains \"a candidate with no Economic Life gives PyPSA no period to annuitise its Build Cost over\"\n```", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-09", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/steps/pypsa_to_sienna_map_components.py", "line": 323, "severity": "minor", "label": "suggestion", "claim": "The method `_drop_unbuilt_candidates` has one caller and a body of three lines.", "suggested_fix": "Move the two working lines into `_prepare_source`, beside the `mapping.skip` block that already applies a `SkipRule` the same way, keep the reason as a one-line comment, and delete the method. Net: -10 lines.", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-10", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/core/extensions.py", "line": 154, "severity": "minor", "label": "issue", "claim": "The comment on `StorageExtension.unit_size_mw` names only `Max Power`, but a pumped-storage turbine writes its `Max Capacity` into that field.", "suggested_fix": "Name both source properties in the comment, for example the Battery `Max Power` and the pumped-storage turbine `Max Capacity`. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-11", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/pypsa_sienna_translations/_storage.py", "line": 155, "severity": "minor", "label": "issue", "claim": "The comment gives an unsolved extendable unit as the example of zero capacity, but the step now drops such a unit before this expression runs.", "suggested_fix": "Replace the example with a solve that built none of an extendable unit, which still reaches this expression with `p_nom_opt` 0. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r4-12", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 4, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 91, "severity": "nit", "label": "nitpick", "claim": "`P_NOM_DERIVATION` and `P_NOM_RATING_DERIVATION` lost the leading underscore they had, but no other module reads them.", "suggested_fix": "Restore the `_` prefix on both names and on their two uses in this file. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T19:35:42Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 210, "severity": "major", "label": "issue", "claim": "The state_of_charge_initial event names Capacity and Initial SoC, but the value now comes from the energy of every unit.", "suggested_fix": "Make the stored energy a `Decision`, so its sources travel with the level.\n\nThe code now:\n\n```python\n stored_energy = rated.p_nom.value * max_hours.value\n ...\n return Decision.derived(\n initial_soc / PERCENT * stored_energy,\n [source, *capacity.sources],\n _SOC_FROM_PERCENT_DERIVATION,\n )\n```\n\nThe code to use:\n\n```python\n stored = Decision.derived(\n rated.p_nom.value * max_hours.value,\n gather_sources(rated.p_nom.sources, max_hours.sources),\n _STORED_ENERGY_DERIVATION, # \"p_nom x max_hours\"\n )\n ...\n return Decision.derived(\n initial_soc / PERCENT * stored.value,\n [source, *stored.sources],\n _SOC_FROM_PERCENT_DERIVATION, # \"Initial SoC / 100 * p_nom * max_hours\"\n )\n```", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 213, "severity": "minor", "label": "suggestion", "claim": "find_unpriced_build writes the candidate rule a second time, over a raw property dictionary.", "suggested_fix": "Give the function the `CandidateSource` and read the rule off it.\n\nThe code now:\n\n```python\ndef find_unpriced_build(\n plexos_class: PlexosClass, name: str, props: dict[str, float]\n) -> SkippedComponent | None:\n if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD:\n return None\n unpriced = next((one for one in _PRICES_A_BUILD if one.plexos_property not in props), None)\n```\n\nThe code to use:\n\n```python\ndef find_unpriced_build(source: CandidateSource) -> SkippedComponent | None:\n if not source.is_candidate:\n return None\n unpriced = next(\n (one for one in _PRICES_A_BUILD if one.plexos_property not in source.props), None\n )\n```\n\nBuild the `CandidateSource` before the call in `rate_object`, and pass `source.candidate` in `_generators._map_one`.", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-03", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/pypsa_sienna_translations/_links.py", "line": 91, "severity": "minor", "label": "suggestion", "claim": "fill_link_defaults states each default twice, and the canonical fill_defaults states each default once.", "suggested_fix": "Call the canonical helper the other four tables call.\n\nThe code to use:\n\n```python\ndef fill_link_defaults(table: pl.DataFrame) -> pl.DataFrame:\n \"\"\"Add optional PyPSA link columns absent when all links share the PyPSA default.\"\"\"\n return fill_defaults(\n table,\n [\n (PyPSALinkCol.P_NOM, 0.0),\n (PyPSALinkCol.P_NOM_OPT, None),\n (PyPSALinkCol.P_MIN_PU, 0.0),\n (PyPSALinkCol.P_MAX_PU, 1.0),\n (PyPSALinkCol.EFFICIENCY, 1.0),\n ],\n [(PyPSALinkCol.P_NOM_EXTENDABLE, False), (PyPSALinkCol.ACTIVE, True)],\n [(col, \"\") for col in (PyPSALinkCol.CARRIER, PyPSALinkCol.BUS2, PyPSALinkCol.BUS3)],\n )\n```\n\nThe helper also fills a null `p_nom_extendable` with `False`, which this function leaves in place.", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-04", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 213, "severity": "major", "label": "issue", "claim": "find_unpriced_build drops a plant that already runs, when that plant states Max Units Built but no Build Cost.", "suggested_fix": "```python\ndef find_unpriced_build(\n plexos_class: PlexosClass, name: str, props: dict[str, float], rated: RatedCapacity\n) -> SkippedComponent | None:\n \"\"\"The first property a candidate leaves out that stops PyPSA pricing its build.\"\"\"\n if props.get(PlexosProperty.MAX_UNITS_BUILT, NOTHING_TO_BUILD) <= NOTHING_TO_BUILD:\n return None\n if rated.existing.value:\n # The object already runs, so the plant stays and only its build is left out.\n return None\n ...\n```\nThen let `derive_expansion` return `FIXED_CAPACITY` for such an object, and record the unpriced build as a dropped value on the component.", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-05", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 209, "severity": "minor", "label": "issue", "claim": "The state_of_charge_initial event names Initial SoC and Capacity only, but the value now also multiplies by Units.", "suggested_fix": "```python\n return Decision.derived(\n initial_soc / PERCENT * stored_energy,\n gather_sources([source], capacity.sources, rated.p_nom.sources),\n _SOC_FROM_PERCENT_DERIVATION,\n )\n```", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-06", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/pypsa_sienna_translations/_shared.py", "line": 60, "severity": "minor", "label": "suggestion", "claim": "The Line mapping keeps the old rule, so a line the solve did not build still takes its s_nom as the rating.", "suggested_fix": "```python\n# interop/plugins/shared/pypsa_sienna_translations/_lines.py\n(PyPSALineCol.S_NOM_OPT, None),\n...\npl.col(PyPSALineCol.S_NOM_OPT).fill_nan(None),\n...\n_effective_s_nom = effective_p_nom(\n PyPSALineCol.S_NOM_EXTENDABLE, PyPSALineCol.S_NOM_OPT, PyPSALineCol.S_NOM\n)\n\ndef _rating_uses_opt(old: dict[str, Any]) -> bool:\n return holds_solved_capacity(\n old, PyPSALineCol.S_NOM_EXTENDABLE, PyPSALineCol.S_NOM_OPT\n )\n```", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-07", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/pypsa_sienna_translations/_storage.py", "line": 154, "severity": "minor", "label": "issue", "claim": "The comment is a trap, but it names an unsolved extendable unit, a case the new candidate filter drops before this expression runs.", "suggested_fix": "Replace the stale example with the case that still arrives here. `unbuilt_candidate_skip` now removes every extendable row with a null `p_nom_opt` in `_prepare_source`, so the only zero capacity left is a solve that built none of an extendable unit, or a unit with `p_nom` 0.\n\nThe code now:\n\n```python\n# state_of_charge_initial / (effective_p_nom * max_hours), clamped to [0, 1].\n# Defaults to 0.0 when capacity is zero (e.g. unsolved extendable unit) to\n# avoid a division-by-zero that would clip to 1.0 (incorrectly \"fully charged\").\n_initial_level = (\n pl.when(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0)\n```\n\nThe code to use:\n\n```python\n# state_of_charge_initial / (effective_p_nom * max_hours), clamped to [0, 1].\n# Defaults to 0.0 when capacity is zero, which a solve that built none of an\n# extendable unit writes, to avoid a division-by-zero that would clip to 1.0\n# (incorrectly \"fully charged\").\n_initial_level = (\n pl.when(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0)\n```", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-08", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 101, "severity": "nit", "label": "nitpick", "claim": "The first half of the second sentence narrates the two branches below it; only the reference to derive_p_nom earns its place.", "suggested_fix": "Cut the narration and keep the constraint.\n\nThe code now:\n\n```python\n @property\n def rated_unit_count(self) -> float:\n \"\"\"How many units the nominal power stands for.\n\n It is what the object runs, or what a candidate running none of them may build, which\n is the same reading ``derive_p_nom`` takes.\n \"\"\"\n if self.rated.existing.value:\n return self.props.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n return self.max_units_built\n```\n\nThe code to use:\n\n```python\n @property\n def rated_unit_count(self) -> float:\n \"\"\"How many units the nominal power stands for.\n\n Keep this in step with ``derive_p_nom``: the two have to read the same object the\n same way.\n \"\"\"\n if self.rated.existing.value:\n return self.props.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n return self.max_units_built\n```", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-09", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_batteries.py", "line": 210, "severity": "major", "label": "issue", "claim": "The derivation text and the source list of `state_of_charge_initial` still describe one unit's energy capacity, but the body now multiplies the share by the energy of every unit.", "suggested_fix": "Give the decision the sources of `rated.p_nom` beside the capacity sources, and reword `_SOC_FROM_PERCENT_DERIVATION` to say the level is a share of the energy of every unit the p_nom stands for. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-10", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/pypsa_sienna_translations/_storage.py", "line": 155, "severity": "minor", "label": "nitpick", "claim": "The comment names an unsolved extendable unit as the case the zero guard catches, but such a unit is now dropped before this expression runs.", "suggested_fix": "Name a case that can still arrive, such as a solve that built none of an extendable unit or a `p_nom` of zero, and fit the comment on two lines. Net: -1 lines.", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r5-11", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 5, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 91, "severity": "nit", "label": "nitpick", "claim": "`P_NOM_DERIVATION` and `P_NOM_RATING_DERIVATION` are public names that only their own module reads.", "suggested_fix": "Restore the leading underscore on both constants, as every other module-level constant in the file has. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T19:59:07Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py", "line": 200, "severity": "major", "label": "suggestion", "claim": "The functions `_turbine_rating` and `_battery_rating` hold the same code, and `RatedPower` already holds the two names that differ.", "suggested_fix": "The code now, in two modules:\n\n```python\n# _storage_hydro.py\ndef _turbine_rating(staged: StagedObject) -> RatedCapacity:\n max_capacity = staged.properties[PlexosProperty.MAX_CAPACITY]\n stated_units = staged.properties.get(PlexosProperty.UNITS)\n units = DEFAULT_UNITS if stated_units is None else stated_units\n ...\n_GENERATOR_POWER = RatedPower(PlexosClass.GENERATOR, PlexosProperty.MAX_CAPACITY, _turbine_rating)\n\n# _batteries.py\ndef _battery_rating(staged: StagedObject) -> RatedCapacity:\n max_power = staged.properties[PlexosProperty.MAX_POWER]\n units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n ...\n_BATTERY_POWER = RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER, _battery_rating)\n```\n\nThe code to use, in `_storage_shared.py` alone:\n\n```python\n@dataclass(frozen=True)\nclass RatedPower:\n plexos_class: PlexosClass\n capacity_property: PlexosProperty\n\n def rate(self, staged: StagedObject) -> RatedCapacity:\n \"\"\"The power of one unit, beside the power of every unit the object runs.\"\"\"\n power = staged.properties[self.capacity_property]\n units = staged.properties.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n stated = SourceValue(self.plexos_class, staged.name, self.capacity_property, power, UNIT_MW)\n counted = SourceValue(self.plexos_class, staged.name, PlexosProperty.UNITS, units)\n return RatedCapacity(\n existing=Decision.derived(\n power * units, [stated, counted], f\"{self.capacity_property} * Units\"\n ),\n unit_size=Decision.derived(power, [stated], DIRECT_DERIVATION),\n )\n```\n\nThen `rate_object` calls `rating.rate(staged)`, the `derive` field goes, and each constant becomes `RatedPower(PlexosClass.BATTERY, PlexosProperty.MAX_POWER)`. The derivation string stays `\"Max Power * Units\"` and `\"Max Capacity * Units\"`, so the feature files do not change.", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/plexos_pypsa_translations/_shared.py", "line": 164, "severity": "major", "label": "suggestion", "claim": "`as_rate` converts a WACC percentage at map time, but `CANONICAL_UNIT` already converts `Build Cost`, `FO&M Charge` and `Economic Life` at stage time.", "suggested_fix": "The code now:\n\n```python\n# _shared.py\ndef as_rate(stated: float | None, stated_unit: str | None) -> float | None:\n if stated is None:\n return None\n return stated / PERCENT if is_percent(stated_unit) else stated\n\n# _expansion.py\ndef _discount_rate(source: CandidateSource) -> Decision:\n wacc = source.props.get(PlexosProperty.WACC)\n rate = as_rate(wacc, source.stated_units.get(PlexosProperty.WACC))\n```\n\nThe code to use:\n\n```python\n# plexos_units.py, beside the other four expansion properties\n(PlexosCollection.GENERATORS, PlexosProperty.WACC): UNIT_FRACTION,\n(PlexosCollection.BATTERIES, PlexosProperty.WACC): UNIT_FRACTION,\n...\nUNIT_FRACTION: {UNIT_FRACTION: 1.0, UNIT_PERCENT: 1.0 / _HUNDRED},\n\n# _expansion.py\ndiscount_rate=_from_property(source, PlexosProperty.WACC, None, DIRECT_DERIVATION),\n```\n\nThis deletes `as_rate`, the `stated_units` field on `CandidateSource`, `StagedObject` and `SourceGenerator`, and the `gen_units`, `battery_units` and `generator_units` lookups.", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-03", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 206, "severity": "minor", "label": "suggestion", "claim": "Three members of `_expansion.py` repeat the test for a candidate that runs no units, so this concept has no name.", "suggested_fix": "The code now:\n\n```python\n@property\ndef rated_unit_count(self) -> float:\n if self.rated.existing.value:\n return self.props.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n return self.max_units_built\n\ndef derive_p_nom(source: CandidateSource) -> Decision:\n if source.rated.existing.value or not source.is_candidate:\n return source.rated.existing\n return derive_buildable(source)\n\ndef find_unpriced_candidate(source: CandidateSource) -> SkippedComponent | None:\n if source.rated.existing.value or not source.is_candidate:\n return None\n```\n\nThe code to use:\n\n```python\n@property\ndef is_unbuilt_candidate(self) -> bool:\n \"\"\"A candidate that runs no units, so its whole capacity is the build.\"\"\"\n return self.is_candidate and not self.rated.existing.value\n\n@property\ndef rated_unit_count(self) -> float:\n if self.is_unbuilt_candidate:\n return self.max_units_built\n return self.props.get(PlexosProperty.UNITS, DEFAULT_UNITS)\n\ndef derive_p_nom(source: CandidateSource) -> Decision:\n return derive_buildable(source) if source.is_unbuilt_candidate else source.rated.existing\n\ndef find_unpriced_candidate(source: CandidateSource) -> SkippedComponent | None:\n if not source.is_unbuilt_candidate:\n return None\n```", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-04", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "docs/translation_mappings/translation-from-plexos-to-pypsa.md", "line": 281, "severity": "minor", "label": "issue", "claim": "The text says six cases, but the table below it lists seven cases.", "suggested_fix": "```markdown\n**The translator does not translate seven cases.** It records each one as a skipped\ncomponent:\n```", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-05", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "docs/translation_mappings/translation-from-plexos-to-sienna.md", "line": 276, "severity": "minor", "label": "issue", "claim": "The table says a generator that states Max Units Built is left out, but a running plant with an unpriced build stays.", "suggested_fix": "```markdown\n| The generator states `Max Units Built`, and the model prices the build | It is a candidate to build, not capacity to dispatch. A generator that already runs and prices no build keeps the capacity it runs. |\n```\nThe \"A candidate is not a plant\" bullet near the top of the file needs the same qualification.", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-06", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/plexos_pypsa_translations/_generators.py", "line": 116, "severity": "minor", "label": "issue", "claim": "The translator writes one warning line for each skipped generator, so a model with hundreds of retired generators writes hundreds of lines.", "suggested_fix": "Give each shared reading a `SkipGroup`, as the unpriced-candidate skip already does:\n```python\n_RETIRED_GROUP = SkipGroup(\n counted=\"Generator(s)\", reason=\"state no Units and no Max Units Built\"\n)\n\n if source.units == 0.0 and not source.is_candidate:\n return SkippedComponent(\n _source(name, PlexosProperty.UNITS, source.units),\n _RETIRED_NOTE,\n warn_with=_RETIRED_GROUP,\n )\n```\nThe missing-Node and zero-p_nom skips want the same treatment.", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-07", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/pypsa_sienna_translations/_storage.py", "line": 154, "severity": "minor", "label": "suggestion", "claim": "The first sentence duplicates the derivation string on line 272 and restates the expression below it.", "suggested_fix": "Delete the first sentence. Keep the trap in the second.\n\nThe code now:\n\n```python\n# state_of_charge_initial / (effective_p_nom * max_hours), clamped to [0, 1]. A capacity of\n# zero, from a solve that built none of an extendable unit or from a p_nom of zero, gives 0.0\n# rather than a division by zero that would clip to 1.0 and read as fully charged.\n_initial_level = (\n pl.when(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0)\n```\n\nThe code to use:\n\n```python\n# A capacity of zero, from a solve that built none of an extendable unit or from a p_nom of\n# zero, gives 0.0 rather than a division by zero that would clip to 1.0 and read as fully\n# charged.\n_initial_level = (\n pl.when(pl.col(EFFECTIVE_P_NOM) * pl.col(PyPSAStorageUnitCol.MAX_HOURS) > 0)\n```", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-08", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/pypsa_sienna_translations/_shared.py", "line": 138, "severity": "minor", "label": "suggestion", "claim": "This docstring paraphrases the function name and repeats UNBUILT_CANDIDATE_NOTE, which sits three lines above.", "suggested_fix": "Delete the docstring.\n\nThe code now:\n\n```python\ndef unbuilt_candidate_skip(naming: PyPSAComponentNaming) -> SkipRule:\n \"\"\"The drop for a candidate an operations system must not read as a built plant.\"\"\"\n return SkipRule(\n keep=pl.col(STATES_BUILT_CAPACITY),\n```\n\nThe code to use:\n\n```python\ndef unbuilt_candidate_skip(naming: PyPSAComponentNaming) -> SkipRule:\n return SkipRule(\n keep=pl.col(STATES_BUILT_CAPACITY),\n```", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-09", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/plexos_pypsa_translations/_expansion.py", "line": 93, "severity": "nit", "label": "nitpick", "claim": "This docstring paraphrases the class name and states no rule the fields below do not show.", "suggested_fix": "Delete the docstring.\n\nThe code now:\n\n```python\n@dataclass(frozen=True)\nclass CandidateSource:\n \"\"\"One staged PLEXOS object as the expansion rule reads it.\"\"\"\n\n plexos_class: PlexosClass\n name: str\n```\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass CandidateSource:\n plexos_class: PlexosClass\n name: str\n```", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-10", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "docs/translation_mappings/translation-from-plexos-to-pypsa.md", "line": 281, "severity": "major", "label": "issue", "claim": "The sentence says six cases, but the table below it lists seven.", "suggested_fix": "Change \"six cases\" to \"seven cases\" on line 281. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r6-11", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 6, "file": "interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py", "line": 91, "severity": "nit", "label": "nitpick", "claim": "`P_NOM_DERIVATION` and `P_NOM_RATING_DERIVATION` lost the leading underscore they had, but no other module reads them.", "suggested_fix": "Rename both back to `_P_NOM_DERIVATION` and `_P_NOM_RATING_DERIVATION`, matching the other derivation constants in the module. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T20:28:06Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r7-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 7, "file": "interop/plugins/shared/pypsa_sienna_translations/_shared.py", "line": 60, "severity": "major", "label": "issue", "claim": "The rule drops an extendable component that already runs, because it reads p_nom_extendable and p_nom_opt but never p_nom_min.", "suggested_fix": "Read the floor a build cannot take away, and rate the component at it.\n\nThe code now:\n\n```python\ndef states_built_capacity(extendable: str, opt: str) -> pl.Expr:\n return ~pl.col(extendable) | pl.col(opt).is_not_null()\n\n\ndef effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr:\n return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom))\n```\n\nThe code to use:\n\n```python\ndef states_built_capacity(extendable: str, opt: str, nom_min: str) -> pl.Expr:\n \"\"\"Capacity an operations model may dispatch: fixed, solved, or the floor a build keeps.\"\"\"\n return ~pl.col(extendable) | pl.col(opt).is_not_null() | (pl.col(nom_min) > 0)\n\n\ndef effective_p_nom(extendable: str, opt: str, nom: str, nom_min: str) -> pl.Expr:\n return (\n pl.when(has_solved_capacity(extendable, opt))\n .then(pl.col(opt))\n .when(pl.col(extendable))\n .then(pl.col(nom_min))\n .otherwise(pl.col(nom))\n )\n```\n\nAdd `(P_NOM_MIN, 0.0)` to each `fill_*_defaults` float default list, and pass the column through `with_effective_p_nom`.", "rule_created": false, "ts": "2026-09-08T20:51:10Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r7-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 7, "file": "interop/core/extensions.py", "line": 154, "severity": "minor", "label": "issue", "claim": "The comment names one source property, but a StorageExtension also holds a pumped-storage turbine, which states Max Capacity.", "suggested_fix": "Name both properties.\n\nThe code now:\n\n```python\nclass StorageExtension(ExtensionRecord):\n p_nom_extendable: bool | None = None # PyPSA only\n # MW. PLEXOS Max Power: what one unit of a candidate is. PyPSA sizes a candidate by\n # p_nom_max alone, so the size of a single unit has no field there.\n unit_size_mw: float | None = None\n```\n\nThe code to use:\n\n```python\nclass StorageExtension(ExtensionRecord):\n p_nom_extendable: bool | None = None # PyPSA only\n # MW. PLEXOS Max Power on a Battery, Max Capacity on a pumped-storage turbine: what one\n # unit of a candidate is. PyPSA sizes a candidate by p_nom_max alone, so the size of a\n # single unit has no field there.\n unit_size_mw: float | None = None\n```", "rule_created": false, "ts": "2026-09-08T20:51:10Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r8-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 8, "file": "interop/core/extensions.py", "line": 154, "severity": "minor", "label": "issue", "claim": "The comment on `StorageExtension.unit_size_mw` names only `Max Power`, but a pumped-storage turbine fills the same field from `Max Capacity`.", "suggested_fix": "Change the comment to name both source properties, for example \"MW. PLEXOS Max Power for a Battery, Max Capacity for a pumped-storage turbine: what one unit of a candidate is.\" Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T21:15:13Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} From be79e245f058dcbcb248ffa6f254837c6e381f7f Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 21:45:03 +0000 Subject: [PATCH 12/17] Keep the years a PLEXOS plan states, and carry its constraints A PLEXOS expansion plan writes a schedule of dated Units: an object runs none of itself until the year it is built, and its units fall back to zero in the year it retires. The staging layer narrowed every dated property to the value in force when the window opened and discarded every band edge outside it, so a generator built in 2032 read as no units at all in a 2026 window and the year itself was gone. The source now stages every resolved row a third time, unclipped and beside the dates it applies between. The generator and battery mappings read the Units bands off that table: the first year the units rise above zero becomes the build_year PyPSA carries on the component, and the first year after that in which they fall back to zero becomes the retirement_year in the extensions sidecar, since PyPSA has no field for the other end of a plant's life. A Constraint was read in full -- its sense, its right-hand sides, the objects it names and the coefficients weighting them -- and then reported as carried nowhere. PyPSA's GlobalConstraint limits one carrier over the whole horizon and cannot name a set of components, which is the ground on which a reserve already travels in the sidecar, so a constraint now travels there too: its sense as the inequality it holds in, each right-hand side beside the span it applies over, each member with the class its name belongs to and the coefficient weighting it, and whether the expansion plan has to meet it. One stating no sense, or no right-hand side at all, states no inequality to carry, so it is still left out and reported. The operations leg relays the records on unchanged, so nothing reports them as dropped on the way to Sienna. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 44 ++-- .../translation-from-plexos-to-sienna.md | 4 +- interop/core/extensions.py | 88 ++++++++ interop/plugins/shared/plexos_constants.py | 17 +- interop/plugins/shared/plexos_dates.py | 32 +++ .../plexos_pypsa_translations/_batteries.py | 6 + .../plexos_pypsa_translations/_constraints.py | 128 +++++++++--- .../_generator_decisions.py | 3 + .../_generator_derivation.py | 5 + .../_generator_lookups.py | 10 +- .../plexos_pypsa_translations/_generators.py | 10 + .../plexos_pypsa_translations/_lifespan.py | 191 ++++++++++++++++++ .../_storage_hydro.py | 6 + .../_storage_shared.py | 22 +- .../_storage_units.py | 6 + .../plexos_pypsa_translations/constants.py | 3 + interop/plugins/shared/pypsa_constants.py | 8 + interop/plugins/sinks/emit_pypsa_network.py | 2 + .../sources/plexos_dated_properties.py | 46 +++-- interop/plugins/sources/stage_plexos_xml.py | 14 +- .../steps/plexos_to_pypsa/map_constraints.py | 4 +- .../steps/pypsa_to_sienna_map_components.py | 14 ++ .../plexos_to_pypsa/constraints.feature | 60 +++++- .../plexos_to_pypsa/dated_properties.feature | 42 ++++ 24 files changed, 692 insertions(+), 73 deletions(-) create mode 100644 interop/plugins/shared/plexos_dates.py create mode 100644 interop/plugins/shared/plexos_pypsa_translations/_lifespan.py diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index cbc7c51..ec5edb6 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -31,7 +31,8 @@ It gives the source of each field. | [`Market`](#market--generator) | An import `Generator` | | [`Reserve`](#reserve--extensions-sidecar) | No component. The translator carries it to the reserves sidecar, but nothing applies it. | | [Region `VoLL`](#load-shedding) | No component in the two faithful pipelines. In `plexos-to-pypsa-monte-carlo-reliability`, a load shedding `Generator` at each bus. | -| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) | +| [`Constraint`](#constraint--extensions-sidecar) | No component. The translator carries it to the sidecar, but nothing applies it. | +| `Zone`, `Interface`, `Transformer`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) | | `Transmission`, `ST`/`MT Schedule`, `PASA`, `Production`, `Performance`, `Stochastic`, `Report`, `Diagnostic`, `System`, `List` | Not translated. These are solver settings, not model data. | ## Reading the tables @@ -277,6 +278,7 @@ is `committable` when it is thermal, or when its `p_min_pu` is more than `0`. | `discount_rate` | | `WACC`, for a candidate only | `derived` | | `lifetime` | yr | `Economic Life`, for a candidate only | `direct` | | `fom_cost` | $/MW/yr | `FO&M Charge`, for a candidate only | `direct` | +| `build_year` | yr | The first year the dated [`Units`](#which-entry-applies-when) rise above zero. The field is absent where the model dates no `Units`, and PyPSA reads `0`. | `derived` | **The translator does not translate seven cases.** It records each one as a skipped component: @@ -361,6 +363,7 @@ has more than one fuel uses its primary fuel. | `cyclic_state_of_charge` | | `True` if `End Effects Method` is `RECYCLE`, or if the model gives no `Initial SoC` | `derived` | | `marginal_cost` | $/MWh | `0.0` | `default` | | `p_nom_extendable`, `p_nom_min`, `p_nom_max`, `overnight_cost`, `discount_rate`, `lifetime`, `fom_cost` | | Refer to [What a candidate is](#what-a-candidate-is) | `derived` / `default` | +| `build_year` | yr | The first year the dated [`Units`](#which-entry-applies-when) rise above zero. The field is absent where the model dates no `Units`, and PyPSA reads `0`. | `derived` | The energy one unit of a battery holds is its `Capacity`. If the model gives a duration in place of a capacity, that energy is `Duration × Max Power`. `max_hours` is that energy @@ -771,7 +774,7 @@ Two properties use dated entries as a schedule. They do not use them as correcti | Property | Meaning | | --- | --- | -| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. | +| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. The translator reads both years off the entries as the model dates them, whatever year it translates: the first year the value rises above zero is the `build_year` of the component it writes, and the first year after that in which the value falls back to zero is the `retirement_year` in the extensions sidecar. A component running no units in the year translated is still left out, since it can dispatch nothing that year. | | `Max Capacity` | A capacity expansion schedule. The translator applies the entry that is in force at the snapshot. Where it needs one value, it uses the entry that is in force at the start of the model. | ### Timeslice patterns @@ -994,7 +997,7 @@ reservoir and no head reservoir. | `Interface` | Nothing applies the group flow limits. Thus the dispatch can be more than a transfer limit that your PLEXOS model obeys. | | `Transformer` | The translator does not carry it. | | `Reserve` requirements | Nothing applies them. The generators that contribute can operate at full output. The translator does carry the reserves. Refer to [`Reserve`](#reserve--extensions-sidecar). | -| `Constraint` | Nothing applies the custom constraints. This includes the energy budgets, the running hour limits, the RPS targets and the emission targets. The translator reports every one. Refer to [`Constraint`](#constraint). | +| `Constraint` | Nothing applies the custom constraints. This includes the energy budgets, the running hour limits, the RPS targets and the emission targets. The translator reports every one, and carries each one it can read to the extensions sidecar. Refer to [`Constraint`](#constraint--extensions-sidecar). | | `Waterway` | The cascade route between reservoirs is lost. Each reservoir is independent. | | `Decision Variable` | The translator does not carry it. | | Emission caps | Nothing applies them. Only the carbon price goes into the cost. | @@ -1003,7 +1006,7 @@ reservoir and no head reservoir. | Ancillary service and demand response pseudo-generators | The translator skips nothing. Refer to [`Generator`](#generator--generator). | | Gas, heat and water networks | The translator accepts electricity only. | -## `Constraint` +## `Constraint` → extensions sidecar A PLEXOS `Constraint` holds a weighted sum over the objects it names to a right-hand side. It weights each object by a coefficient on the membership — `Generation Coefficient`, @@ -1013,14 +1016,31 @@ group of hydro units and an annual running hour cap on a group of peakers are bo this way. A PyPSA `GlobalConstraint` limits one **carrier** over the whole horizon, and it has no way -to name a set of components. Thus no shape of `Constraint` fits it, and the translator -carries none of them. - -It does report all of them. Every right-hand side a `Constraint` states becomes a not -mapped entry against the object that states it, giving the value, the sense, and each term -of the weighted sum: the object, the class it belongs to, and the coefficient weighting it. -A `Constraint` stating no right-hand side is reported against the object itself. One -warning names a few of them and counts the rest. +to name a set of components. Thus no shape of `Constraint` fits it, and the network file +holds none of them. + +Each one the translator can read travels in the `extensions.json` file adjacent to the +network instead, as a `constraint` record in framework-neutral terms. The contract is in +`interop/core/extensions.py`. Nothing applies the limit, in the network or in the solve; +the sidecar carries it for a program that decides to use it, and for a later hop into a +framework that can express it. + +| Sidecar field | From | +| --- | --- | +| `name` | `Constraint.name` | +| `sense` | `Sense`, as the inequality it holds in: `<=`, `==` or `>=` | +| `limits` | One entry per right-hand side the `Constraint` states, each giving the value, the span it applies over (`horizon`, `hour`, `day`, `week`, `month` or `year`) and the unit the model stated it in | +| `members` | One entry per object the sum names, each giving the name, the PLEXOS class it belongs to, the coefficient and the property that states the coefficient | +| `applies_to_expansion_plan` | `Include in LT Plan` | + +A `Constraint` that states no sense, or no right-hand side at all, states no inequality to +carry. The translator leaves that one out and reports it. + +The report carries all of them either way. Every right-hand side a `Constraint` states +becomes a not mapped entry against the object that states it, giving the value, the sense, +and each term of the weighted sum: the object, the class it belongs to, and the coefficient +weighting it. A `Constraint` stating no right-hand side is reported against the object +itself. One warning names a few of them and counts the rest. **Read that section of the report before you trust the dispatch.** A model that caps hydro energy or peaker running hours with a `Constraint` gives a translated network in which diff --git a/docs/translation_mappings/translation-from-plexos-to-sienna.md b/docs/translation_mappings/translation-from-plexos-to-sienna.md index 5f93949..30aa452 100644 --- a/docs/translation_mappings/translation-from-plexos-to-sienna.md +++ b/docs/translation_mappings/translation-from-plexos-to-sienna.md @@ -35,7 +35,7 @@ property of the PLEXOS to Sienna mapping. | [`Market`](#market--thermalstandard) | An import `ThermalStandard` | | `Reserve` | No component. The record reaches `extensions.json`. Refer to [Not translated](#not-translated). | | [Region `VoLL`](#region-load--interruptiblepowerload) | The `operation_cost` of an `InterruptiblePowerLoad`, on a reliability run only. | -| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) | +| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated). A `Constraint` reaches the sidecar, but nothing applies it. | | `Transmission`, `ST`/`MT Schedule`, `PASA`, `Production`, `Performance`, `Stochastic`, `Report`, `Diagnostic`, `System`, `List` | Not translated. These are solver settings, not model data. | ## Reading the tables @@ -504,7 +504,7 @@ energy than your model gives it. | `Zone` | The zonal group is lost. The regional group still becomes an `Area`. | | `Interface` | Nothing applies the group flow limits, so a transfer can go above a limit your model obeys. | | `Transformer` | The translator does not carry it. | -| `Constraint` | Nothing applies the custom constraints, which include the RPS targets and the emission targets. | +| `Constraint` | The record reaches `extensions.json`, but nothing applies the custom constraints, which include the RPS targets and the emission targets. | | `Waterway` | The cascade route between reservoirs is lost. Each reservoir is independent. | | `Decision Variable` | The translator does not carry it. | | Emission caps | Nothing applies them. Only the carbon price reaches the cost. | diff --git a/interop/core/extensions.py b/interop/core/extensions.py index f0bd711..eab5c17 100644 --- a/interop/core/extensions.py +++ b/interop/core/extensions.py @@ -47,6 +47,10 @@ class ExtensionKind(StrEnum): # PLEXOS Reserve; Sienna VariableReserve and ConstantReserve. PyPSA has none, which is # why the concept needs the sidecar to survive a hop through it. RESERVE = "reserve" + # PLEXOS Constraint. PyPSA's GlobalConstraint limits one carrier over the whole horizon + # and cannot name a set of components, which is why the concept needs the sidecar to + # survive a hop through it. Sienna has no equivalent either. + CONSTRAINT = "constraint" NETWORK = "network" # PyPSA network-level attributes. No Sienna or PLEXOS equivalent. @@ -80,6 +84,29 @@ class ReserveKind(StrEnum): UNKNOWN = "unknown" +class ConstraintSense(StrEnum): + """Which way a constraint holds its weighted sum against the right-hand side.""" + + AT_MOST = "<=" # PLEXOS Sense -1 + EXACTLY = "==" # PLEXOS Sense 0 + AT_LEAST = ">=" # PLEXOS Sense 1 + + +class ConstraintPeriod(StrEnum): + """The span one right-hand side applies over. + + PLEXOS states a limit per hour, day, week, month or year, or over the whole horizon, + and spells each as a right-hand side property of its own. + """ + + HORIZON = "horizon" # PLEXOS RHS + HOUR = "hour" # PLEXOS RHS Hour + DAY = "day" # PLEXOS RHS Day + WEEK = "week" # PLEXOS RHS Week + MONTH = "month" # PLEXOS RHS Month + YEAR = "year" # PLEXOS RHS Year + + class ExtensionRecord(BaseModel): """One component's record. ``name`` is the identifier in every framework.""" @@ -113,6 +140,9 @@ class GeneratorExtension(ExtensionRecord): # yr. PLEXOS Technical Life: how long the plant runs. PyPSA has one lifetime and the # capital recovery period claims it, so this has no field there. technical_life_years: float | None = None + # The year the plant leaves service, which PLEXOS states as a dated Units of zero. PyPSA + # carries build_year on the component and nothing for the other end of its life. + retirement_year: int | None = None class LoadExtension(ExtensionRecord): @@ -157,6 +187,9 @@ class StorageExtension(ExtensionRecord): # yr. PLEXOS Technical Life: how long the plant runs. PyPSA has one lifetime and the # capital recovery period claims it, so this has no field there. technical_life_years: float | None = None + # The year the unit leaves service, which PLEXOS states as a dated Units of zero. PyPSA + # carries build_year on the component and nothing for the other end of its life. + retirement_year: int | None = None class ReserveExtension(ExtensionRecord): @@ -191,6 +224,51 @@ class ReserveExtension(ExtensionRecord): is_mutually_exclusive: bool | None = None +class ConstraintMember(BaseModel): + """One object a constraint weights, and the coefficient it is weighted by. + + Two classes can hold an object of the same name and a constraint can weight a generator + and an emission alike, so the member carries the class its name belongs to. The + coefficient is absent where the constraint names the object without weighting it. + """ + + model_config = ConfigDict(extra="forbid") + + name: str + member_class: str # PLEXOS Generator, Line, Emission and the rest + coefficient: float | None = None + # PLEXOS names the coefficient per class: Generation Coefficient, Flow Coefficient, and + # so on. The name says which quantity of the member the coefficient weights. + coefficient_property: str | None = None + + +class ConstraintLimit(BaseModel): + """One right-hand side the weighted sum is held to, and the span it applies over.""" + + model_config = ConfigDict(extra="forbid") + + period: ConstraintPeriod + value: float + unit: str | None = None # the unit the source stated the limit in + + +class ConstraintExtension(ExtensionRecord): + """A weighted sum over named objects, held to one or more right-hand sides. + + PLEXOS has the concept and PyPSA has none: a GlobalConstraint limits one carrier over + the whole horizon and cannot name a set of components. A constraint therefore travels + here rather than in the network, so a later hop into a framework that can express it + still has the limit. + """ + + sense: ConstraintSense + limits: list[ConstraintLimit] = [] + members: list[ConstraintMember] = [] + # PLEXOS Include in LT Plan: whether the expansion plan has to meet the constraint as + # well as the dispatch. + applies_to_expansion_plan: bool | None = None + + class NetworkExtension(ExtensionRecord): """The model file's own attributes. PyPSA only: neither Sienna nor PLEXOS has these.""" @@ -211,6 +289,7 @@ class Extensions(BaseModel): controllable_line: list[ControllableLineExtension] = [] storage: list[StorageExtension] = [] reserve: list[ReserveExtension] = [] + constraint: list[ConstraintExtension] = [] network: list[NetworkExtension] = [] @@ -222,6 +301,7 @@ class Extensions(BaseModel): ExtensionKind.CONTROLLABLE_LINE: ControllableLineExtension, ExtensionKind.STORAGE: StorageExtension, ExtensionKind.RESERVE: ReserveExtension, + ExtensionKind.CONSTRAINT: ConstraintExtension, ExtensionKind.NETWORK: NetworkExtension, } @@ -402,6 +482,10 @@ def record_for( staged: StagedExtensions, kind: Literal[ExtensionKind.RESERVE], name: str ) -> ReserveExtension | None: ... @overload +def record_for( + staged: StagedExtensions, kind: Literal[ExtensionKind.CONSTRAINT], name: str +) -> ConstraintExtension | None: ... +@overload def record_for( staged: StagedExtensions, kind: Literal[ExtensionKind.NETWORK], name: str ) -> NetworkExtension | None: ... @@ -466,6 +550,10 @@ def read(self, kind: Literal[ExtensionKind.STORAGE]) -> ExtensionLookup[StorageE @overload def read(self, kind: Literal[ExtensionKind.RESERVE]) -> ExtensionLookup[ReserveExtension]: ... @overload + def read( + self, kind: Literal[ExtensionKind.CONSTRAINT] + ) -> ExtensionLookup[ConstraintExtension]: ... + @overload def read(self, kind: Literal[ExtensionKind.NETWORK]) -> ExtensionLookup[NetworkExtension]: ... def read(self, kind: ExtensionKind) -> ExtensionLookup[Any]: diff --git a/interop/plugins/shared/plexos_constants.py b/interop/plugins/shared/plexos_constants.py index e6a40f2..d078fa5 100644 --- a/interop/plugins/shared/plexos_constants.py +++ b/interop/plugins/shared/plexos_constants.py @@ -40,10 +40,11 @@ class PlexosClass(StrEnum): class PlexosResolvedTable: - """Keys of the two long tables ``stage_plexos_xml`` resolves from the raw ``t_*`` tables.""" + """Keys of the long tables ``stage_plexos_xml`` resolves from the raw ``t_*`` tables.""" MEMBERSHIPS = "memberships" PROPERTIES = "properties" + DATED_PROPERTIES = "dated_properties" class PlexosObjectCol: @@ -95,6 +96,19 @@ class PlexosPropertyCol: SCALING = "scaling" +class PlexosDatedPropertyCol(PlexosPropertyCol): + """Columns of the resolved ``dated_properties`` table. + + The ``properties`` columns, plus the dates the value on the row applies between. Every + resolved row is here, whether or not the window being translated covers its band, so a + mapping reading a schedule sees the years the model states rather than one value. + ``date_to`` names a whole day, so the band runs to the end of it. + """ + + DATE_FROM = "date_from" + DATE_TO = "date_to" + + class PlexosCollection(StrEnum): """PLEXOS collection names, the relationship kind on a ``memberships`` row. @@ -211,6 +225,7 @@ class PlexosProperty(StrEnum): RHS_WEEK = "RHS Week" RHS_MONTH = "RHS Month" RHS_YEAR = "RHS Year" + INCLUDE_IN_LT_PLAN = "Include in LT Plan" def is_plexos_true(value: float) -> bool: diff --git a/interop/plugins/shared/plexos_dates.py b/interop/plugins/shared/plexos_dates.py new file mode 100644 index 0000000..0ced89a --- /dev/null +++ b/interop/plugins/shared/plexos_dates.py @@ -0,0 +1,32 @@ +"""When a dated PLEXOS value applies. + +PLEXOS stamps a ``t_data`` row with the dates it applies between. The source narrows those +bands to the window being translated, and a mapping reading a schedule out of them -- the +year a unit arrives, the year it goes -- reads the same bands as the model states them, so +both sides share one reading of what a band covers. +""" + +from __future__ import annotations + +from datetime import datetime, timedelta +from typing import NamedTuple + + +class DateBand(NamedTuple): + """When a ``t_data`` value applies. An open end runs from, or until, forever.""" + + date_from: datetime | None + date_to: datetime | None + + @property + def ends(self) -> datetime | None: + """A ``date_to`` names a whole day, so the band runs to the end of it.""" + return None if self.date_to is None else self.date_to + timedelta(days=1) + + def covers(self, moment: datetime) -> bool: + return (self.date_from is None or self.date_from <= moment) and ( + self.ends is None or moment < self.ends + ) + + +UNDATED = DateBand(None, None) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index 66f25b4..fdfc248 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -22,6 +22,10 @@ derive_expansion, gather_sources, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + derive_build_year, + derive_retirement_year, +) from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( CARRIER_NOTE, @@ -131,7 +135,9 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: ), inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE), cyclic=_battery_cyclic(rated), + build_year=derive_build_year(PlexosClass.BATTERY, rated.name, rated.lifespan), expansion=expansion, + retirement_year=derive_retirement_year(PlexosClass.BATTERY, rated.name, rated.lifespan), units=rated.properties.get(PlexosProperty.UNITS), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py index 1f7f287..feaa8d5 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py @@ -1,11 +1,15 @@ -"""PLEXOS Constraint -> the translation report. +"""PLEXOS Constraint -> extensions sidecar. A PLEXOS Constraint holds a weighted sum over the objects it names to a right-hand side. It weights each object by a coefficient stated on the membership, and it may state the right-hand side over an hour, a day, a week, a month, a year, or the whole horizon. PyPSA's GlobalConstraint limits one carrier over the whole horizon and has no way to name -a set of components, so no shape of Constraint has a home in the network file. +a set of components, so no shape of Constraint has a home in the network file. Each one is +carried into the extensions sidecar instead, in framework-neutral terms (see +``interop/core/extensions.py``), so a later hop into a framework that can express it still +has the limit. A Constraint stating no sense, or no right-hand side at all, says too little +to carry, so it is left out and reported. """ from __future__ import annotations @@ -13,6 +17,15 @@ import logging from dataclasses import dataclass +from interop.core.extensions import ( + ConstraintExtension, + ConstraintLimit, + ConstraintMember, + ConstraintPeriod, + ConstraintSense, + ExtensionKind, + append_extensions, +) from interop.core.pipeline import State from interop.core.reporting import ScopedRecorder from interop.plugins.shared.plexos_constants import ( @@ -20,6 +33,7 @@ PlexosObjectCol, PlexosProperty, PlexosResolvedTable, + is_plexos_true, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ClassMember, @@ -39,23 +53,33 @@ log = logging.getLogger(__name__) +_CARRIED_NOTE = ( + "constraint carried to the extensions sidecar; PyPSA's GlobalConstraint cannot hold a " + "weighted sum over the objects a Constraint names, so the network file itself does not " + "limit them" +) _NOT_CARRIED_NOTE = ( "a Constraint holds a weighted sum over the objects it names to its right-hand side, " "which PyPSA's GlobalConstraint cannot express, so the limit is not carried" ) # PLEXOS states which way a Constraint binds as one integer. -_SENSES: dict[float, str] = {-1.0: "<=", 0.0: "==", 1.0: ">="} +_SENSES: dict[float, ConstraintSense] = { + -1.0: ConstraintSense.AT_MOST, + 0.0: ConstraintSense.EXACTLY, + 1.0: ConstraintSense.AT_LEAST, +} _UNSTATED_SENSE = "unstated" -_RIGHT_HAND_SIDES = ( - PlexosProperty.RHS, - PlexosProperty.RHS_HOUR, - PlexosProperty.RHS_DAY, - PlexosProperty.RHS_WEEK, - PlexosProperty.RHS_MONTH, - PlexosProperty.RHS_YEAR, -) +# The right-hand side properties, each with the span it holds the sum over. +_PERIODS: dict[str, ConstraintPeriod] = { + PlexosProperty.RHS: ConstraintPeriod.HORIZON, + PlexosProperty.RHS_HOUR: ConstraintPeriod.HOUR, + PlexosProperty.RHS_DAY: ConstraintPeriod.DAY, + PlexosProperty.RHS_WEEK: ConstraintPeriod.WEEK, + PlexosProperty.RHS_MONTH: ConstraintPeriod.MONTH, + PlexosProperty.RHS_YEAR: ConstraintPeriod.YEAR, +} _NO_COEFFICIENT = "no coefficient" @@ -76,19 +100,34 @@ class _Term: @dataclass(frozen=True) class _Constraint: name: str - sense: str + sense: ConstraintSense | None terms: tuple[_Term, ...] right_hand_sides: dict[str, float] units: dict[str, str | None] + applies_to_expansion_plan: bool | None + + +@dataclass(frozen=True) +class _Outcome: + """One Constraint and the record it travels as, which is None where it states too little.""" + + constraint: _Constraint + record: ConstraintExtension | None def map_constraints(state: State, recorder: ScopedRecorder) -> None: constraints = _read_constraints(state) if not constraints: return + outcomes = [_Outcome(constraint, _carry(constraint)) for constraint in constraints] reporter = SourceReporter(recorder) - for constraint in constraints: - _record(reporter, constraint) + for outcome in outcomes: + _record(reporter, outcome) + append_extensions( + state.destination_extensions, + ExtensionKind.CONSTRAINT, + [outcome.record for outcome in outcomes if outcome.record is not None], + ) _warn(constraints) @@ -123,10 +162,11 @@ def _read_one( terms=_build_terms(members, coefficients), right_hand_sides={ property_name: stated[property_name] - for property_name in _RIGHT_HAND_SIDES + for property_name in _PERIODS if property_name in stated }, units=units.get(name, {}), + applies_to_expansion_plan=_read_plan_flag(stated), ) @@ -159,13 +199,53 @@ def _constraint_names(state: State) -> list[str]: return names -def _read_sense(stated: dict[str, float]) -> str: +def _read_sense(stated: dict[str, float]) -> ConstraintSense | None: code = stated.get(PlexosProperty.SENSE) - return _UNSTATED_SENSE if code is None else _SENSES.get(code, _UNSTATED_SENSE) + return None if code is None else _SENSES.get(code) + + +def _read_plan_flag(stated: dict[str, float]) -> bool | None: + flag = stated.get(PlexosProperty.INCLUDE_IN_LT_PLAN) + return None if flag is None else is_plexos_true(flag) + + +def _carry(constraint: _Constraint) -> ConstraintExtension | None: + """The sidecar record, or None where the constraint states too little to hold anything to. + + A limit needs both a sense and a right-hand side: without either there is no inequality + to state, whatever the objects it names. + """ + if constraint.sense is None or not constraint.right_hand_sides: + return None + return ConstraintExtension( + name=constraint.name, + sense=constraint.sense, + limits=[ + ConstraintLimit( + period=_PERIODS[property_name], + value=value, + unit=constraint.units.get(property_name), + ) + for property_name, value in constraint.right_hand_sides.items() + ], + members=[_member(term) for term in constraint.terms], + applies_to_expansion_plan=constraint.applies_to_expansion_plan, + ) + + +def _member(term: _Term) -> ConstraintMember: + return ConstraintMember( + name=term.member.name, + member_class=term.member.member_class, + coefficient=term.coefficient, + coefficient_property=term.coefficient_property, + ) -def _record(reporter: SourceReporter, constraint: _Constraint) -> None: - note = f"{_NOT_CARRIED_NOTE}. {_describe(constraint)}" +def _record(reporter: SourceReporter, outcome: _Outcome) -> None: + constraint = outcome.constraint + carried = _CARRIED_NOTE if outcome.record is not None else _NOT_CARRIED_NOTE + note = f"{carried}. {_describe(constraint)}" if not constraint.right_hand_sides: reporter.record_dropped(_source(constraint.name, None, None), note) return @@ -182,10 +262,11 @@ def _source( def _describe(constraint: _Constraint) -> str: """The sense the Constraint binds in, and the weighted sum it binds.""" + sense = constraint.sense or _UNSTATED_SENSE if not constraint.terms: - return f"Sense {constraint.sense}, over no objects" + return f"Sense {sense}, over no objects" return ( - f"Sense {constraint.sense} over {len(constraint.terms)} term(s): " + f"Sense {sense} over {len(constraint.terms)} term(s): " f"{name_a_few(_describe_term(term) for term in constraint.terms)}" ) @@ -201,8 +282,9 @@ def _describe_term(term: _Term) -> str: def _warn(constraints: list[_Constraint]) -> None: log.warning( - "plexos: %s Constraint(s) limit what the model may dispatch, and PyPSA has no home " - "for any of them, so none is enforced: %s", + "plexos: %s Constraint(s) limit what the model may dispatch and the network file " + "enforces none of them; each one the translator can read travels in the extensions " + "sidecar: %s", len(constraints), name_a_few(sorted(constraint.name for constraint in constraints)), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py index 2cfadf3..2c74b7a 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py @@ -42,6 +42,7 @@ ThermalCostTerms, UnitCommitment, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import derive_build_year from interop.plugins.shared.plexos_pypsa_translations.constants import ( FULL_AVAILABILITY, MARGINAL_COST_CARBON_TERM, @@ -150,6 +151,7 @@ class GeneratorDecisions: up_time_before: Decision = maps_to(PyPSAGeneratorCol.UP_TIME_BEFORE, unit=UNIT_SNAPSHOTS) start_up_cost: Decision = maps_to(PyPSAGeneratorCol.START_UP_COST, unit=UNIT_DOLLARS) shut_down_cost: Decision = maps_to(PyPSAGeneratorCol.SHUT_DOWN_COST, unit=UNIT_DOLLARS) + build_year: Decision = maps_to(PyPSAGeneratorCol.BUILD_YEAR) expansion: ExpansionDecisions = holds() @@ -180,6 +182,7 @@ def decide_generator(mapping: GeneratorMapping) -> GeneratorDecisions: up_time_before=commitment.up_time_before, start_up_cost=commitment.start_up_cost, shut_down_cost=commitment.shut_down_cost, + build_year=derive_build_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan), expansion=mapping.expansion, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py index 192aef5..6c57f14 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_derivation.py @@ -28,6 +28,7 @@ derive_p_nom, ) from interop.plugins.shared.plexos_pypsa_translations._generator_lookups import Lookups +from interop.plugins.shared.plexos_pypsa_translations._lifespan import NO_LIFESPAN, Lifespan from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_P_MIN_PU, DEFAULT_SHUT_DOWN_COST, @@ -102,6 +103,7 @@ class SourceGenerator: props: dict[str, float] stated_units: dict[str, str | None] max_capacity: float + lifespan: Lifespan @cached_property def candidate(self) -> CandidateSource: @@ -142,6 +144,7 @@ def read_source(generator: dict[str, Any], name: str, lookups: Lookups) -> Sourc props=props, stated_units=lookups.gen_units.get(name, {}), max_capacity=_rated_capacity(name, props, lookups), + lifespan=lookups.lifespans.get(name, NO_LIFESPAN), ) @@ -212,6 +215,7 @@ class GeneratorMapping: unit_commitment: UnitCommitment | None candidate: CandidateSource expansion: ExpansionDecisions + lifespan: Lifespan @property def is_committable(self) -> bool: @@ -258,6 +262,7 @@ def derive_generator(source: SourceGenerator, node: str, lookups: Lookups) -> Ge else None, candidate=candidate, expansion=derive_expansion(candidate), + lifespan=source.lifespan, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py index 73ba10d..8ddb0a3 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_lookups.py @@ -1,6 +1,6 @@ """The per-class lookups the generator mapping reads while it walks the Generator class. -Each is built once from the two long staged tables, so mapping a generator is dictionary +Each is built once from the long staged tables, so mapping a generator is dictionary reads rather than a scan per generator. """ @@ -18,6 +18,10 @@ PlexosProperty, PlexosResolvedTable, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + Lifespan, + read_lifespans, +) from interop.plugins.shared.plexos_pypsa_translations._shared import ( MultiValueRule, ObjectProperties, @@ -65,6 +69,7 @@ class Lookups: profile_peaks: dict[str, dict[str, float]] capacity_peaks: dict[str, float] dated_fuel_prices: dict[str, float] + lifespans: dict[str, Lifespan] minutes_per_snapshot: float @@ -90,6 +95,9 @@ def build_lookups(state: State) -> Lookups: profile_peaks={prop: _series_peaks(state, prop) for prop in _PROFILE_PROPERTIES}, capacity_peaks=_series_peaks(state, PlexosProperty.MAX_CAPACITY), dated_fuel_prices=_mean_fuel_prices(state), + lifespans=read_lifespans( + state.source_topology[PlexosResolvedTable.DATED_PROPERTIES], PlexosClass.GENERATOR + ), # Every staged series shares the network's snapshots, so any of them fixes the # resolution the hour-based generator properties convert against. minutes_per_snapshot=resolution_minutes( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index f291f4f..a2daaf9 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -44,6 +44,10 @@ Lookups, build_lookups, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + RETIREMENT_YEAR_COLUMN, + derive_retirement_year, +) from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_turbines import ( storage_turbine_names, @@ -149,12 +153,18 @@ def _carry_to_extensions( if mapping.carrier != mapping.category: reporter.record(mapping.name, _CATEGORY_COLUMN, _category_decision(mapping)) record_expansion(mapping.name, mapping.expansion, reporter) + reporter.record( + mapping.name, + RETIREMENT_YEAR_COLUMN, + derive_retirement_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan), + ) records = [ GeneratorExtension( name=mapping.name, category=mapping.category, unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), technical_life_years=read_sidecar_value(mapping.expansion.technical_life), + retirement_year=mapping.lifespan.retirement_year, ) for mapping in mappings ] diff --git a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py new file mode 100644 index 0000000..69e543b --- /dev/null +++ b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py @@ -0,0 +1,191 @@ +"""When a PLEXOS object comes into service, and when it goes out of it. + +A model writes an expansion plan as a schedule of dated ``Units``: an object running none +of itself until a later year arrives in that year, and one whose units fall back to zero +leaves in the year they do. The window being translated holds one value per property, so +these years are read from the dated bands the source stages beside it, unclipped. +""" + +from __future__ import annotations + +from datetime import datetime +from typing import NamedTuple + +import polars as pl + +from interop.plugins.shared.plexos_constants import ( + PlexosClass, + PlexosDatedPropertyCol, + PlexosProperty, +) +from interop.plugins.shared.plexos_dates import UNDATED, DateBand +from interop.plugins.shared.plexos_pypsa_translations.constants import ( + EXT_RETIREMENT_YEAR_FIELD, +) +from interop.plugins.shared.plexos_pypsa_translations.decisions import ( + Decision, + MappedColumns, + SourceValue, +) + +# What a schedule runs while no band it states covers the moment. +_NOT_IN_SERVICE = 0.0 + +RETIREMENT_YEAR_COLUMN = MappedColumns((EXT_RETIREMENT_YEAR_FIELD,)) + +_BUILD_DERIVATION = "the first year the dated Units rise above zero" +_RETIREMENT_DERIVATION = "the first year the dated Units fall back to zero" + + +class Milestone(NamedTuple): + """A year a schedule changes what an object runs, and what it runs from then on.""" + + year: int + units: float + + +class Lifespan(NamedTuple): + """When a dated ``Units`` schedule brings an object in, and when it takes it out. + + Either end can be absent: a schedule may only build, only retire, or say neither. + """ + + build: Milestone | None + retirement: Milestone | None + + @property + def build_year(self) -> int | None: + return None if self.build is None else self.build.year + + @property + def retirement_year(self) -> int | None: + return None if self.retirement is None else self.retirement.year + + +NO_LIFESPAN = Lifespan(None, None) + + +class _UnitsBand(NamedTuple): + """How many units one dated ``Units`` row runs, and the dates it runs them between.""" + + dates: DateBand + units: float + + +class _Change(NamedTuple): + """A moment a schedule changes what it runs, and what it ran until then.""" + + at: datetime + units: float + was: float + + +def read_lifespans(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, Lifespan]: + """What the dated ``Units`` say about each object of a class that dates them.""" + return {name: _read_lifespan(bands) for name, bands in _read_bands(dated, plexos_class).items()} + + +def derive_build_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision: + """The year the schedule brings the object into service.""" + return _derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION) + + +def derive_retirement_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision: + """The year the schedule takes the object out of service, which the sidecar carries.""" + return _derive_year(plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION) + + +def _derive_year( + plexos_class: PlexosClass, name: str, milestone: Milestone | None, derivation: str +) -> Decision: + if milestone is None: + return Decision.unreported(None) + source = SourceValue(plexos_class, name, PlexosProperty.UNITS, milestone.units) + return Decision.derived(milestone.year, [source], derivation) + + +def read_year(decision: Decision) -> int | None: + """The year a lifespan decision states, for the sidecar field that carries it.""" + return None if decision.value is None else int(decision.value) + + +def _read_bands(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, list[_UnitsBand]]: + """Each object's ``Units`` rows, earliest band first, for the objects that date any. + + One row per object per band, so the frame is component-scale and safe to collect. + """ + if PlexosDatedPropertyCol.CHILD_CLASS not in dated.collect_schema().names(): + return {} + frame = ( + dated.filter( + (pl.col(PlexosDatedPropertyCol.CHILD_CLASS) == plexos_class) + & (pl.col(PlexosDatedPropertyCol.PROPERTY) == PlexosProperty.UNITS) + & pl.col(PlexosDatedPropertyCol.VALUE).is_not_null() + ) + .select( + PlexosDatedPropertyCol.CHILD_OBJECT, + PlexosDatedPropertyCol.DATE_FROM, + PlexosDatedPropertyCol.DATE_TO, + PlexosDatedPropertyCol.VALUE, + ) + .collect() + ) + bands: dict[str, list[_UnitsBand]] = {} + for name, date_from, date_to, units in frame.iter_rows(): + bands.setdefault(name, []).append(_UnitsBand(DateBand(date_from, date_to), units)) + return {name: sorted(rows, key=_opens) for name, rows in bands.items() if _states_a_date(rows)} + + +def _states_a_date(bands: list[_UnitsBand]) -> bool: + """An object whose ``Units`` are one value for all time keeps that value for all time.""" + return any(band.dates != UNDATED for band in bands) + + +def _opens(band: _UnitsBand) -> datetime: + """An undated band sorts first, being in force before any dated one begins.""" + return band.dates.date_from or datetime.min + + +def _read_lifespan(bands: list[_UnitsBand]) -> Lifespan: + changes = _changes(bands) + build = next((one for one in changes if not _runs(one.was) and _runs(one.units)), None) + retirement = next( + ( + one + for one in changes + if _runs(one.was) and not _runs(one.units) and (build is None or one.at > build.at) + ), + None, + ) + return Lifespan(_milestone(build), _milestone(retirement)) + + +def _milestone(change: _Change | None) -> Milestone | None: + return None if change is None else Milestone(change.at.year, change.units) + + +def _runs(units: float) -> bool: + return units > _NOT_IN_SERVICE + + +def _changes(bands: list[_UnitsBand]) -> list[_Change]: + """What the schedule runs from each of its edges, beside what it ran before that edge.""" + changes: list[_Change] = [] + running = _units_at(bands, datetime.min) + for moment in _edges(bands): + units = _units_at(bands, moment) + changes.append(_Change(moment, units, running)) + running = units + return changes + + +def _units_at(bands: list[_UnitsBand], moment: datetime) -> float: + """The latest band covering the moment, else nothing: the model runs none of the object.""" + covering = [band for band in bands if band.dates.covers(moment)] + return covering[-1].units if covering else _NOT_IN_SERVICE + + +def _edges(bands: list[_UnitsBand]) -> list[datetime]: + """Every moment a band of the schedule opens or closes, earliest first.""" + moments = {edge for band in bands for edge in (band.dates.date_from, band.dates.ends)} + return sorted(moment for moment in moments if moment is not None) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index db6a789..4790f7d 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -25,6 +25,10 @@ from interop.plugins.shared.plexos_pypsa_translations._expansion import ( derive_expansion, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + derive_build_year, + derive_retirement_year, +) from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( CARRIER_NOTE, CHARGE_NOTE, @@ -221,7 +225,9 @@ def _derive_turbine( ), inflow=_reservoir_inflow(head), cyclic=Decision.default(variant.cyclic, variant.cyclic_note), + build_year=derive_build_year(PlexosClass.GENERATOR, rated.name, rated.lifespan), expansion=expansion, + retirement_year=derive_retirement_year(PlexosClass.GENERATOR, rated.name, rated.lifespan), inflow_storage=_inflow_storage(head), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index a5e8a63..439f4b4 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -26,12 +26,18 @@ PlexosResolvedTable, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( + NOTHING_TO_REPORT, CandidateSource, ExpansionDecisions, RatedCapacity, derive_p_nom, find_unpriced_candidate, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + NO_LIFESPAN, + Lifespan, + read_lifespans, +) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, ObjectUnits, @@ -115,7 +121,10 @@ class StorageUnitMapping: ) inflow: Decision = maps_to(PyPSAStorageUnitCol.INFLOW, unit=UNIT_MW) cyclic: Decision = maps_to(PyPSAStorageUnitCol.CYCLIC_STATE_OF_CHARGE) + build_year: Decision = maps_to(PyPSAStorageUnitCol.BUILD_YEAR) expansion: ExpansionDecisions = holds() + # The sidecar carries the retirement year, since PyPSA has no column for it. + retirement_year: Decision = NOTHING_TO_REPORT # Units is a Battery-only reading; a units-out trace derates against it. units: float | None = None # The head Storage whose Natural Inflow this unit reads, where that inflow is power. An @@ -170,6 +179,7 @@ class StagedObject: stated_units: dict[str, str | None] node: str | None file_backed: list[str] + lifespan: Lifespan @dataclass(frozen=True) @@ -189,6 +199,8 @@ class StorageLookups: generator_units: ObjectUnits storage_units: ObjectUnits storages_with_inflow_profile: set[str] + lifespan_by_battery: dict[str, Lifespan] + lifespan_by_generator: dict[str, Lifespan] def battery(self, name: str) -> StagedObject: return StagedObject( @@ -197,6 +209,7 @@ def battery(self, name: str) -> StagedObject: stated_units=self.battery_units.get(name, {}), node=self.node_by_battery.get(name), file_backed=self.file_backed_by_battery.get(name, []), + lifespan=self.lifespan_by_battery.get(name, NO_LIFESPAN), ) def generator(self, name: str) -> StagedObject: @@ -206,6 +219,7 @@ def generator(self, name: str) -> StagedObject: stated_units=self.generator_units.get(name, {}), node=self.node_by_generator.get(name), file_backed=self.file_backed_by_generator.get(name, []), + lifespan=self.lifespan_by_generator.get(name, NO_LIFESPAN), ) def has_head_and_tail(self, generator: str) -> bool: @@ -278,6 +292,7 @@ def _states_inflow_in_other_units(volumes: dict[str, float], units: dict[str, st def build_lookups(state: State) -> StorageLookups: properties = state.source_topology[PlexosResolvedTable.PROPERTIES] memberships = state.source_topology[PlexosResolvedTable.MEMBERSHIPS] + dated = state.source_topology[PlexosResolvedTable.DATED_PROPERTIES] return StorageLookups( battery_properties=collapse_properties_by_object(properties, PlexosClass.BATTERY), generator_properties=collapse_properties_by_object(properties, PlexosClass.GENERATOR), @@ -296,6 +311,8 @@ def build_lookups(state: State) -> StorageLookups: generator_units=collapse_units_by_object(properties, PlexosClass.GENERATOR), storage_units=collapse_units_by_object(properties, PlexosClass.STORAGE), storages_with_inflow_profile=_storages_with_inflow_profile(state), + lifespan_by_battery=read_lifespans(dated, PlexosClass.BATTERY), + lifespan_by_generator=read_lifespans(dated, PlexosClass.GENERATOR), ) @@ -348,6 +365,7 @@ class RatedObject: node: str p_nom: Decision candidate: CandidateSource + lifespan: Lifespan def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | SkippedComponent: @@ -371,7 +389,9 @@ def rate_object(staged: StagedObject, rating: RatedPower) -> RatedObject | Skipp p_nom = derive_p_nom(candidate) if p_nom.value <= 0.0: return _skipped_zero_p_nom(rating, staged.name, p_nom.value) - return RatedObject(staged.name, staged.properties, staged.node, p_nom, candidate) + return RatedObject( + staged.name, staged.properties, staged.node, p_nom, candidate, staged.lifespan + ) def skip_object( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index d9e9bee..c565824 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -32,6 +32,10 @@ record_expansion, warn_about_dropped_builds, ) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + RETIREMENT_YEAR_COLUMN, + read_year, +) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, ) @@ -229,6 +233,7 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No for mapping in storage_units.mappings: reporter.record_mapping(mapping.name, mapping) record_expansion(mapping.name, mapping.expansion, reporter) + reporter.record(mapping.name, RETIREMENT_YEAR_COLUMN, mapping.retirement_year) warn_about_dropped_builds(mapping.expansion for mapping in storage_units.mappings) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) @@ -243,6 +248,7 @@ def _carry_to_extensions(state: State, mappings: list[StorageUnitMapping]) -> No name=mapping.name, unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), technical_life_years=read_sidecar_value(mapping.expansion.technical_life), + retirement_year=read_year(mapping.retirement_year), ) for mapping in mappings ] diff --git a/interop/plugins/shared/plexos_pypsa_translations/constants.py b/interop/plugins/shared/plexos_pypsa_translations/constants.py index 31535fc..7f5ab79 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/constants.py +++ b/interop/plugins/shared/plexos_pypsa_translations/constants.py @@ -23,6 +23,9 @@ EXT_TECHNICAL_LIFE_FIELD: str = "extensions.technical_life_years" """Names that sidecar key in the audit trail; the network file itself has no such column.""" +EXT_RETIREMENT_YEAR_FIELD: str = "extensions.retirement_year" +"""Names that sidecar key in the audit trail; the network file itself has no such column.""" + # --- generators --------------------------------------------------------------- DEFAULT_UP_TIME_BEFORE: float = 0.0 diff --git a/interop/plugins/shared/pypsa_constants.py b/interop/plugins/shared/pypsa_constants.py index a96bd83..627b037 100644 --- a/interop/plugins/shared/pypsa_constants.py +++ b/interop/plugins/shared/pypsa_constants.py @@ -195,6 +195,7 @@ class PyPSALineCol: LENGTH = "length" NUM_PARALLEL = "num_parallel" TERRAIN_FACTOR = "terrain_factor" + BUILD_YEAR = "build_year" S_NOM_MIN = "s_nom_min" S_NOM_MAX = "s_nom_max" OVERNIGHT_COST = "overnight_cost" @@ -222,6 +223,7 @@ class PyPSALinkCol: ACTIVE = "active" CARRIER = "carrier" TERRAIN_FACTOR = "terrain_factor" + BUILD_YEAR = "build_year" P_NOM_MIN = "p_nom_min" P_NOM_MAX = "p_nom_max" OVERNIGHT_COST = "overnight_cost" @@ -256,6 +258,7 @@ class PyPSAGeneratorCol(PyPSAComponentCol): CAPITAL_COST = "capital_cost" LIFETIME = "lifetime" FOM_COST = "fom_cost" + BUILD_YEAR = "build_year" class PyPSAStorageUnitCol(PyPSAComponentCol): @@ -282,6 +285,7 @@ class PyPSAStorageUnitCol(PyPSAComponentCol): CAPITAL_COST = "capital_cost" LIFETIME = "lifetime" FOM_COST = "fom_cost" + BUILD_YEAR = "build_year" class PyPSAStoreCol(PyPSAComponentCol): @@ -487,6 +491,8 @@ class ReverseTimeSeriesMetadataCol: PyPSAGeneratorCol.DISCOUNT_RATE: pl.Float64, PyPSAGeneratorCol.LIFETIME: pl.Float64, PyPSAGeneratorCol.FOM_COST: pl.Float64, + # The year the source brings the generator into service, null where it states none. + PyPSAGeneratorCol.BUILD_YEAR: pl.Int64, } # Unit-commitment columns a non-committable generator leaves unset; the sink omits null @@ -524,6 +530,8 @@ class ReverseTimeSeriesMetadataCol: PyPSAStorageUnitCol.DISCOUNT_RATE: pl.Float64, PyPSAStorageUnitCol.LIFETIME: pl.Float64, PyPSAStorageUnitCol.FOM_COST: pl.Float64, + # The year the source brings the storage unit into service, null where it states none. + PyPSAStorageUnitCol.BUILD_YEAR: pl.Int64, } LOADS_DESTINATION_SCHEMA: dict[str, pl.DataType | type[pl.DataType]] = { diff --git a/interop/plugins/sinks/emit_pypsa_network.py b/interop/plugins/sinks/emit_pypsa_network.py index 88e9a45..def9261 100644 --- a/interop/plugins/sinks/emit_pypsa_network.py +++ b/interop/plugins/sinks/emit_pypsa_network.py @@ -301,6 +301,7 @@ def _add_generators(network: pypsa.Network, generators: pl.DataFrame | None) -> PyPSAGeneratorCol.DISCOUNT_RATE, PyPSAGeneratorCol.LIFETIME, PyPSAGeneratorCol.FOM_COST, + PyPSAGeneratorCol.BUILD_YEAR, ), ) @@ -337,6 +338,7 @@ def _add_storage_units(network: pypsa.Network, storage_units: pl.DataFrame | Non PyPSAStorageUnitCol.DISCOUNT_RATE, PyPSAStorageUnitCol.LIFETIME, PyPSAStorageUnitCol.FOM_COST, + PyPSAStorageUnitCol.BUILD_YEAR, ), ) diff --git a/interop/plugins/sources/plexos_dated_properties.py b/interop/plugins/sources/plexos_dated_properties.py index a4e492b..f4fc03b 100644 --- a/interop/plugins/sources/plexos_dated_properties.py +++ b/interop/plugins/sources/plexos_dated_properties.py @@ -10,13 +10,18 @@ from __future__ import annotations import logging -from datetime import datetime, timedelta +from datetime import datetime from typing import Any, NamedTuple import polars as pl from interop.plugins.shared.constants import StagedTimeSeriesCol -from interop.plugins.shared.plexos_constants import PlexosMembershipCol, PlexosPropertyCol +from interop.plugins.shared.plexos_constants import ( + PlexosDatedPropertyCol, + PlexosMembershipCol, + PlexosPropertyCol, +) +from interop.plugins.shared.plexos_dates import UNDATED, DateBand from interop.plugins.sources.plexos_horizon import Window from interop.plugins.sources.plexos_tables import Rows, RowsByTable @@ -39,26 +44,6 @@ _NOT_IN_EFFECT = 0.0 -class DateBand(NamedTuple): - """When a ``t_data`` value applies. An open end runs from, or until, forever.""" - - date_from: datetime | None - date_to: datetime | None - - @property - def ends(self) -> datetime | None: - """A ``date_to`` names a whole day, so the band runs to the end of it.""" - return None if self.date_to is None else self.date_to + timedelta(days=1) - - def covers(self, moment: datetime) -> bool: - return (self.date_from is None or self.date_from <= moment) and ( - self.ends is None or moment < self.ends - ) - - -UNDATED = DateBand(None, None) - - class DatedRow(NamedTuple): """One resolved property row, and when the value on it applies.""" @@ -92,6 +77,23 @@ def apply_window(resolved: list[DatedRow], window: Window) -> tuple[Rows, Rows]: return in_force, stepped +def dated_rows(resolved: list[DatedRow]) -> Rows: + """Every resolved property row with the dates it applies between, as the model states them. + + ``apply_window`` reads one value per property for the window being translated, which + loses the years the bands outside it name. A schedule stated as a series of bands is + read from these rows instead. + """ + return [ + { + **dated.row, + PlexosDatedPropertyCol.DATE_FROM: dated.dates.date_from, + PlexosDatedPropertyCol.DATE_TO: dated.dates.date_to, + } + for dated in resolved + ] + + def _property_identity(row: dict[str, Any]) -> tuple[Any, ...]: """What makes a property one property: its membership, its name, and its band.""" return ( diff --git a/interop/plugins/sources/stage_plexos_xml.py b/interop/plugins/sources/stage_plexos_xml.py index 5876cf8..7f2a41a 100644 --- a/interop/plugins/sources/stage_plexos_xml.py +++ b/interop/plugins/sources/stage_plexos_xml.py @@ -10,6 +10,8 @@ ``t_data`` value under the selected Model's Scenario overlays (highest Read Order wins, base value otherwise), with a ``data_file`` path where the value is file-backed; +- ``topology/dated_properties.parquet``, the same resolved values before they are + narrowed to the window, each beside the dates it applies between; - one ``source_time_series`` frame per (owner class, property) whose value comes from an external CSV, streamed in as ``(snapshot, component, sample, value)`` rows. ``sample`` is null except where the CSV carries one column per Monte Carlo @@ -46,6 +48,7 @@ PlexosPropertyCol, PlexosResolvedTable, ) +from interop.plugins.shared.plexos_dates import UNDATED, DateBand from interop.plugins.shared.plexos_units import ( StatedValue, UnitConversions, @@ -62,11 +65,10 @@ warn_unstageable_layout, ) from interop.plugins.sources.plexos_dated_properties import ( - UNDATED, - DateBand, DatedRow, apply_window, date_bands, + dated_rows, stepped_series_parts, ) from interop.plugins.sources.plexos_horizon import Chronology, Window, reindex_onto @@ -255,8 +257,10 @@ def _stage_topology( topology = self._stage(resolved.objects_by_class, staging_dir) memberships = PlexosResolvedTable.MEMBERSHIPS properties = PlexosResolvedTable.PROPERTIES + dated = PlexosResolvedTable.DATED_PROPERTIES topology[memberships] = self._stage_table(memberships, resolved.memberships, staging_dir) topology[properties] = self._stage_table(properties, resolved.properties, staging_dir) + topology[dated] = self._stage_table(dated, resolved.dated_properties, staging_dir) return topology def _stage(self, objects_by_class: _RowsByClass, staging_dir: Path) -> dict[str, pl.LazyFrame]: @@ -376,17 +380,20 @@ class StagePlexosXmlEnsemble(StagePlexosXml): @dataclass(frozen=True) class _ResolvedDataset: - """The three staged topology tables, resolved from the raw ``t_*`` tables. + """The staged topology tables, resolved from the raw ``t_*`` tables. ``properties`` holds one row per property, the value in force when the window opens. ``stepped_properties`` holds every value a property takes within the window, and only for the properties that take more than one, so a consumer can read the shape. + ``dated_properties`` holds every resolved row with the dates it applies between, + narrowed to no window at all. """ objects_by_class: _RowsByClass memberships: _Rows properties: _Rows stepped_properties: _Rows + dated_properties: _Rows def _resolve_dataset( @@ -400,6 +407,7 @@ def _resolve_dataset( memberships=memberships, properties=in_force, stepped_properties=stepped, + dated_properties=dated_rows(resolved), ) diff --git a/interop/plugins/steps/plexos_to_pypsa/map_constraints.py b/interop/plugins/steps/plexos_to_pypsa/map_constraints.py index 9d5d260..f18a8c5 100644 --- a/interop/plugins/steps/plexos_to_pypsa/map_constraints.py +++ b/interop/plugins/steps/plexos_to_pypsa/map_constraints.py @@ -1,4 +1,4 @@ -"""PLEXOS Constraint -> the translation report, as a sub-step of the composite mapping step.""" +"""PLEXOS Constraint -> extensions sidecar, as a sub-step of the composite mapping step.""" from __future__ import annotations @@ -12,7 +12,7 @@ class PlexosToPypsaMapConstraints(TranslationStep): - """Reports every PLEXOS Constraint, none of which PyPSA has a home for.""" + """Carries every PLEXOS Constraint to the extensions sidecar; PyPSA enforces none.""" name: ClassVar[str] = "plexos_to_pypsa_map_constraints" params_schema: ClassVar[type[BaseModel] | None] = None diff --git a/interop/plugins/steps/pypsa_to_sienna_map_components.py b/interop/plugins/steps/pypsa_to_sienna_map_components.py index 69d67ed..7b5983d 100644 --- a/interop/plugins/steps/pypsa_to_sienna_map_components.py +++ b/interop/plugins/steps/pypsa_to_sienna_map_components.py @@ -216,6 +216,7 @@ def run(self, state: State, params: BaseModel | None) -> State: state = self._map_lines(state) state = self._map_links(state) _relay_reserves(state, reader) + _relay_constraints(state, reader) choose_ensemble_samples(state, self._recorder) # A record no mapping here read is dropped and reported, rather than relayed into a # sidecar this hop's reader cannot say anything about. @@ -534,6 +535,19 @@ def _map_links(self, state: State) -> State: return state +def _relay_constraints(state: State, reader: ExtensionReader) -> None: + """Carry each constraint the hop before set aside into this hop's own sidecar. + + Neither PyPSA nor Sienna holds a weighted sum over a named set of components, so a + constraint survives the chain only in the sidecar and travels on as it stands. + """ + append_extensions( + state.destination_extensions, + ExtensionKind.CONSTRAINT, + reader.relay(ExtensionKind.CONSTRAINT), + ) + + def _relay_reserves(state: State, reader: ExtensionReader) -> None: """Carry each reserve the hop before set aside into this hop's own sidecar. diff --git a/tests/features/plexos_to_pypsa/constraints.feature b/tests/features/plexos_to_pypsa/constraints.feature index fbae8bb..683bab6 100644 --- a/tests/features/plexos_to_pypsa/constraints.feature +++ b/tests/features/plexos_to_pypsa/constraints.feature @@ -1,14 +1,16 @@ @slow @fork_unsafe -Feature: PLEXOS Constraint objects are reported rather than silently dropped +Feature: PLEXOS Constraint objects travel in the extensions sidecar A PLEXOS Constraint limits a weighted sum over a named set of objects, and states its right-hand side per day, per hour, per week, per month, per year, or over the whole horizon. PyPSA's GlobalConstraint limits one carrier over the whole horizon and cannot name a set of components, so none of those shapes has a home in the network file. - The translation therefore carries no Constraint, and says so: every right-hand side a - Constraint states is reported against the object stating it, so a reader can see which - limits the solved network is not holding to. + Each Constraint therefore travels in the extensions sidecar instead, and the network's + silence about it is reported: every right-hand side a Constraint states is recorded + against the object stating it, so a reader can see which limits the solved network is not + holding to. A Constraint stating no sense, or no right-hand side at all, states no + inequality to carry, so it is left out and reported as such. Scenario: a daily energy limit over named generators is reported, naming what it binds Given a Plexos model @@ -28,9 +30,9 @@ Feature: PLEXOS Constraint objects are reported rather than silently dropped When I run translate against "inputs/constraint.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the file "outputs/network.nc" exists And the file "decisions.md" contains "`plexos.Constraint.RiverSystem.RHS Day` = 1.708" - And the file "decisions.md" contains "a Constraint holds a weighted sum over the objects it names to its right-hand side, which PyPSA's GlobalConstraint cannot express, so the limit is not carried" + And the file "decisions.md" contains "constraint carried to the extensions sidecar; PyPSA's GlobalConstraint cannot hold a weighted sum over the objects a Constraint names, so the network file itself does not limit them" And the file "decisions.md" contains "Sense <= over 2 term(s): 1.0 x Generator AA1 (Generation Coefficient), 1.0 x Generator AA2 (Generation Coefficient)" - And the log contains "plexos: 1 Constraint(s) limit what the model may dispatch, and PyPSA has no home for any of them, so none is enforced: RiverSystem" + And the log contains "plexos: 1 Constraint(s) limit what the model may dispatch and the network file enforces none of them; each one the translator can read travels in the extensions sidecar: RiverSystem" Scenario: every right-hand side a Constraint states is reported, one for each Given a Plexos model @@ -80,6 +82,52 @@ Feature: PLEXOS Constraint objects are reported rather than silently dropped And the model is saved as "inputs/unbounded.xml" When I run translate against "inputs/unbounded.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the file "decisions.md" contains "`plexos.Constraint.Unbounded`" + And the file "decisions.md" contains "a Constraint holds a weighted sum over the objects it names to its right-hand side, which PyPSA's GlobalConstraint cannot express, so the limit is not carried" + And the file "outputs/extensions.json" does not contain "Unbounded" + + Scenario: a Constraint stating no sense states no inequality, so it is left out + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generator "Peaker" with "node=Grid_Node, category=Gas, Max Capacity=100" + And the model contains constraint "Senseless" over: + | class | name | coefficient property | coefficient | + | Generator | Peaker | Generation Coefficient | 1 | + And constraint "Senseless" states "RHS Day" of 400 + And the model is saved as "inputs/senseless.xml" + When I run translate against "inputs/senseless.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the file "decisions.md" contains "`plexos.Constraint.Senseless.RHS Day` = 400.0" + And the file "decisions.md" contains "so the limit is not carried" + And the file "outputs/extensions.json" does not contain "Senseless" + + Scenario: a readable Constraint reaches the sidecar with its sense, its limit and its members + The sidecar states the limit in its own vocabulary: the sense as the inequality it holds + in, the right-hand side beside the span it applies over, and each object the sum weights + with the class it belongs to. + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generators: + | name | node | category | Max Capacity | + | AA1 | Grid_Node | Hydro | 21 | + | AA2 | Grid_Node | Hydro | 23 | + And the model contains constraint "RiverSystem" over: + | class | name | coefficient property | coefficient | + | Generator | AA1 | Generation Coefficient | 1 | + | Generator | AA2 | Generation Coefficient | 2.5 | + And constraint "RiverSystem" states "Sense" of -1 + And constraint "RiverSystem" states "RHS Day" of 1.708 + And constraint "RiverSystem" states "Include in LT Plan" of 1 + And the model is saved as "inputs/carried.xml" + When I run translate against "inputs/carried.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the file "outputs/extensions.json" parses as JSON with "constraint.0.name" set to "RiverSystem" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.sense" set to "<=" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.limits.0.period" set to "day" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.limits.0.value" set to 1.708 + And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.0.name" set to "AA1" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.0.member_class" set to "Generator" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.1.coefficient" set to 2.5 + And the file "outputs/extensions.json" parses as JSON with "constraint.0.applies_to_expansion_plan" set to true Scenario: a model with no Constraint objects warns about nothing Given a Plexos model diff --git a/tests/features/plexos_to_pypsa/dated_properties.feature b/tests/features/plexos_to_pypsa/dated_properties.feature index 9f4420f..a388e8c 100644 --- a/tests/features/plexos_to_pypsa/dated_properties.feature +++ b/tests/features/plexos_to_pypsa/dated_properties.feature @@ -123,3 +123,45 @@ Feature: a PLEXOS property dated to a period is read for the year being translat When I run translate against "inputs/model.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" Then the PyPSA generator "GasPlant" in "outputs/network.nc" has "marginal_cost" equal to 45 And the file "decisions.md" contains "the mean of the fuel's own dated price series" + + Scenario: a generator whose units start at zero and rise in a later year is built in it + The plan commits this candidate for 2032, which the 2026 network still records, so a + reader knows the year the model brings it into service. + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generator "REZ" with "node=Grid_Node, category=Wind, Max Capacity=100, Units=0, Max Units Built=1, Build Cost=900000, WACC=0.07, Economic Life=25" + And generator "REZ" states "Units" of 1 from "2032-01-01" + And the model contains model "Plan" + And the model contains horizon "H1" on model "Plan" starting "2026-01-01" spanning 2 days at 24 periods per day + And the model is saved as "inputs/built_later.xml" + When I run translate against "inputs/built_later.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "REZ" attribute "build_year" is 2032 + And the file "decisions.md" contains "the first year the dated Units rise above zero" + + Scenario: a generator whose units fall to zero in a later year retires in it + PyPSA has no retirement year, so the sidecar carries it. + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generator "OldCoal" with "node=Grid_Node, category=Coal, Max Capacity=500, Units=2" + And generator "OldCoal" states "Units" of 0 from "2035-01-01" + And the model contains model "Plan" + And the model contains horizon "H1" on model "Plan" starting "2026-01-01" spanning 2 days at 24 periods per day + And the model is saved as "inputs/retires_later.xml" + When I run translate against "inputs/retires_later.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "OldCoal" attribute "p_nom" is 1000 + And the file "outputs/extensions.json" parses as JSON with "generator.0.retirement_year" set to 2035 + And the file "decisions.md" contains "the first year the dated Units fall back to zero" + + Scenario: a generator that dates no units states neither year + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generator "Steady" with "node=Grid_Node, category=Coal, Max Capacity=500, Units=1" + And the model contains model "Plan" + And the model contains horizon "H1" on model "Plan" starting "2026-01-01" spanning 2 days at 24 periods per day + And the model is saved as "inputs/undated_units.xml" + When I run translate against "inputs/undated_units.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "Steady" attribute "build_year" is 0 + And the file "outputs/extensions.json" does not contain "retirement_year" From 09726e2bb972cdb7668e64bfa1caad6a32a58099 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 22:14:40 +0000 Subject: [PATCH 13/17] Give one owner to the band reading and to the retirement year r1-01: `plexos_dates` now holds `latest_covering`, `band_edges` and `opens_at`, and both the source's window narrowing and the lifespan reading call them. The rule about what a band covers, and the rule that an undated band sorts first, each stand in one place. r1-02, r1-14, r1-16: the generator path derives the retirement year once and reads the sidecar value off that decision, as the storage path and the expansion fields already do. `Lifespan.build_year` and `Lifespan.retirement_year` go with the second route. The reviewer's suggestion of moving the derivation into the storage report does not work: that loop carries both batteries and hydro turbines, and the two name different PLEXOS classes. r1-03: `dated_rows` now stages only the rows of a property the model dates somewhere. Staging every band of every property costs a multiple of the `properties` table on a model with monthly bands, and an undated property states the same value for all time, which `properties` already carries. Keeping every dated property rather than only `Units` leaves the table usable by the next reader. r1-05: `ConstraintExtension.sense` takes a default, since `ExtensionLookup.get` builds an empty record for a name the sidecar never mentioned and promises to answer with a record rather than raise. r1-17: the note against a Constraint that is left out now says why it is left out. It stated that PyPSA's GlobalConstraint cannot express the limit, which is no longer the reason anything is dropped, since a readable Constraint reaches the sidecar. r1-08, r1-09, r1-10, r1-11, r1-12, r1-13: delete docstrings that repeat a name, argue for a module, narrate the line below, or say a third time what the constants above already said. r1-15: delete the unread `BUILD_YEAR` from `PyPSALineCol` and `PyPSALinkCol`. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- interop/core/extensions.py | 10 +--- interop/plugins/shared/plexos_constants.py | 6 +-- interop/plugins/shared/plexos_dates.py | 30 ++++++++--- .../plexos_pypsa_translations/_constraints.py | 10 ++-- .../plexos_pypsa_translations/_generators.py | 13 ++--- .../plexos_pypsa_translations/_lifespan.py | 43 +++++----------- .../plexos_pypsa_translations/constants.py | 6 +-- interop/plugins/shared/pypsa_constants.py | 2 - .../sources/plexos_dated_properties.py | 51 ++++++++++++------- interop/plugins/sources/stage_plexos_xml.py | 8 +-- .../plexos_to_pypsa/constraints.feature | 4 +- 11 files changed, 90 insertions(+), 93 deletions(-) diff --git a/interop/core/extensions.py b/interop/core/extensions.py index eab5c17..8a829ad 100644 --- a/interop/core/extensions.py +++ b/interop/core/extensions.py @@ -253,15 +253,9 @@ class ConstraintLimit(BaseModel): class ConstraintExtension(ExtensionRecord): - """A weighted sum over named objects, held to one or more right-hand sides. + """A weighted sum over named objects, held to one or more right-hand sides.""" - PLEXOS has the concept and PyPSA has none: a GlobalConstraint limits one carrier over - the whole horizon and cannot name a set of components. A constraint therefore travels - here rather than in the network, so a later hop into a framework that can express it - still has the limit. - """ - - sense: ConstraintSense + sense: ConstraintSense | None = None limits: list[ConstraintLimit] = [] members: list[ConstraintMember] = [] # PLEXOS Include in LT Plan: whether the expansion plan has to meet the constraint as diff --git a/interop/plugins/shared/plexos_constants.py b/interop/plugins/shared/plexos_constants.py index d078fa5..67797ed 100644 --- a/interop/plugins/shared/plexos_constants.py +++ b/interop/plugins/shared/plexos_constants.py @@ -100,9 +100,9 @@ class PlexosDatedPropertyCol(PlexosPropertyCol): """Columns of the resolved ``dated_properties`` table. The ``properties`` columns, plus the dates the value on the row applies between. Every - resolved row is here, whether or not the window being translated covers its band, so a - mapping reading a schedule sees the years the model states rather than one value. - ``date_to`` names a whole day, so the band runs to the end of it. + row of a property the model dates is here, whether or not the window being translated + covers its band, so a mapping reading a schedule sees the years the model states rather + than one value. """ DATE_FROM = "date_from" diff --git a/interop/plugins/shared/plexos_dates.py b/interop/plugins/shared/plexos_dates.py index 0ced89a..0455e60 100644 --- a/interop/plugins/shared/plexos_dates.py +++ b/interop/plugins/shared/plexos_dates.py @@ -1,15 +1,10 @@ -"""When a dated PLEXOS value applies. - -PLEXOS stamps a ``t_data`` row with the dates it applies between. The source narrows those -bands to the window being translated, and a mapping reading a schedule out of them -- the -year a unit arrives, the year it goes -- reads the same bands as the model states them, so -both sides share one reading of what a band covers. -""" +"""When a dated PLEXOS value applies.""" from __future__ import annotations +from collections.abc import Iterable, Sequence from datetime import datetime, timedelta -from typing import NamedTuple +from typing import NamedTuple, TypeVar class DateBand(NamedTuple): @@ -30,3 +25,22 @@ def covers(self, moment: datetime) -> bool: UNDATED = DateBand(None, None) + +T = TypeVar("T") + + +def latest_covering(bands: Sequence[tuple[DateBand, T]], moment: datetime) -> T | None: + """What the last band covering the moment states, else None.""" + covering = [value for band, value in bands if band.covers(moment)] + return covering[-1] if covering else None + + +def band_edges(bands: Iterable[tuple[DateBand, object]]) -> list[datetime]: + """Every moment a band opens or closes, earliest first.""" + moments = {edge for band, _ in bands for edge in (band.date_from, band.ends)} + return sorted(moment for moment in moments if moment is not None) + + +def opens_at(band: tuple[DateBand, object]) -> datetime: + """A sort key putting an undated band first, since it stands before any dated one begins.""" + return band[0].date_from or datetime.min diff --git a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py index feaa8d5..e6fc425 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py @@ -59,8 +59,8 @@ "limit them" ) _NOT_CARRIED_NOTE = ( - "a Constraint holds a weighted sum over the objects it names to its right-hand side, " - "which PyPSA's GlobalConstraint cannot express, so the limit is not carried" + "this Constraint states no sense, or no right-hand side, so it holds no inequality to " + "carry to the extensions sidecar" ) # PLEXOS states which way a Constraint binds as one integer. @@ -210,11 +210,7 @@ def _read_plan_flag(stated: dict[str, float]) -> bool | None: def _carry(constraint: _Constraint) -> ConstraintExtension | None: - """The sidecar record, or None where the constraint states too little to hold anything to. - - A limit needs both a sense and a right-hand side: without either there is no inequality - to state, whatever the objects it names. - """ + """The sidecar record, or None where the constraint states too little to hold anything to.""" if constraint.sense is None or not constraint.right_hand_sides: return None return ConstraintExtension( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index a2daaf9..5b4d673 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -47,6 +47,7 @@ from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( RETIREMENT_YEAR_COLUMN, derive_retirement_year, + read_year, ) from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_turbines import ( @@ -149,22 +150,22 @@ def _carry_to_extensions( """Put what the network file cannot hold in the sidecar, starting with the PLEXOS category, since only one of it and the fuel could become the carrier. """ + retirements = { + mapping.name: derive_retirement_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan) + for mapping in mappings + } for mapping in mappings: if mapping.carrier != mapping.category: reporter.record(mapping.name, _CATEGORY_COLUMN, _category_decision(mapping)) record_expansion(mapping.name, mapping.expansion, reporter) - reporter.record( - mapping.name, - RETIREMENT_YEAR_COLUMN, - derive_retirement_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan), - ) + reporter.record(mapping.name, RETIREMENT_YEAR_COLUMN, retirements[mapping.name]) records = [ GeneratorExtension( name=mapping.name, category=mapping.category, unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), technical_life_years=read_sidecar_value(mapping.expansion.technical_life), - retirement_year=mapping.lifespan.retirement_year, + retirement_year=read_year(retirements[mapping.name]), ) for mapping in mappings ] diff --git a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py index 69e543b..381adf0 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py @@ -18,7 +18,13 @@ PlexosDatedPropertyCol, PlexosProperty, ) -from interop.plugins.shared.plexos_dates import UNDATED, DateBand +from interop.plugins.shared.plexos_dates import ( + UNDATED, + DateBand, + band_edges, + latest_covering, + opens_at, +) from interop.plugins.shared.plexos_pypsa_translations.constants import ( EXT_RETIREMENT_YEAR_FIELD, ) @@ -53,14 +59,6 @@ class Lifespan(NamedTuple): build: Milestone | None retirement: Milestone | None - @property - def build_year(self) -> int | None: - return None if self.build is None else self.build.year - - @property - def retirement_year(self) -> int | None: - return None if self.retirement is None else self.retirement.year - NO_LIFESPAN = Lifespan(None, None) @@ -81,17 +79,14 @@ class _Change(NamedTuple): def read_lifespans(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, Lifespan]: - """What the dated ``Units`` say about each object of a class that dates them.""" return {name: _read_lifespan(bands) for name, bands in _read_bands(dated, plexos_class).items()} def derive_build_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision: - """The year the schedule brings the object into service.""" return _derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION) def derive_retirement_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision: - """The year the schedule takes the object out of service, which the sidecar carries.""" return _derive_year(plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION) @@ -105,7 +100,6 @@ def _derive_year( def read_year(decision: Decision) -> int | None: - """The year a lifespan decision states, for the sidecar field that carries it.""" return None if decision.value is None else int(decision.value) @@ -133,7 +127,9 @@ def _read_bands(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, lis bands: dict[str, list[_UnitsBand]] = {} for name, date_from, date_to, units in frame.iter_rows(): bands.setdefault(name, []).append(_UnitsBand(DateBand(date_from, date_to), units)) - return {name: sorted(rows, key=_opens) for name, rows in bands.items() if _states_a_date(rows)} + return { + name: sorted(rows, key=opens_at) for name, rows in bands.items() if _states_a_date(rows) + } def _states_a_date(bands: list[_UnitsBand]) -> bool: @@ -141,11 +137,6 @@ def _states_a_date(bands: list[_UnitsBand]) -> bool: return any(band.dates != UNDATED for band in bands) -def _opens(band: _UnitsBand) -> datetime: - """An undated band sorts first, being in force before any dated one begins.""" - return band.dates.date_from or datetime.min - - def _read_lifespan(bands: list[_UnitsBand]) -> Lifespan: changes = _changes(bands) build = next((one for one in changes if not _runs(one.was) and _runs(one.units)), None) @@ -172,7 +163,7 @@ def _changes(bands: list[_UnitsBand]) -> list[_Change]: """What the schedule runs from each of its edges, beside what it ran before that edge.""" changes: list[_Change] = [] running = _units_at(bands, datetime.min) - for moment in _edges(bands): + for moment in band_edges(bands): units = _units_at(bands, moment) changes.append(_Change(moment, units, running)) running = units @@ -180,12 +171,6 @@ def _changes(bands: list[_UnitsBand]) -> list[_Change]: def _units_at(bands: list[_UnitsBand], moment: datetime) -> float: - """The latest band covering the moment, else nothing: the model runs none of the object.""" - covering = [band for band in bands if band.dates.covers(moment)] - return covering[-1].units if covering else _NOT_IN_SERVICE - - -def _edges(bands: list[_UnitsBand]) -> list[datetime]: - """Every moment a band of the schedule opens or closes, earliest first.""" - moments = {edge for band in bands for edge in (band.dates.date_from, band.dates.ends)} - return sorted(moment for moment in moments if moment is not None) + """What the schedule runs at a moment; no band covering it means none of the object.""" + units = latest_covering(bands, moment) + return _NOT_IN_SERVICE if units is None else units diff --git a/interop/plugins/shared/plexos_pypsa_translations/constants.py b/interop/plugins/shared/plexos_pypsa_translations/constants.py index 7f5ab79..f29e587 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/constants.py +++ b/interop/plugins/shared/plexos_pypsa_translations/constants.py @@ -17,14 +17,10 @@ DEFAULT_UNITS: float = 1.0 """A generator with no ``Units`` property is a single unit.""" +# Each names a sidecar key in the audit trail; the network file itself has no such column. EXT_UNIT_SIZE_FIELD: str = "extensions.unit_size_mw" -"""Names that sidecar key in the audit trail; the network file itself has no such column.""" - EXT_TECHNICAL_LIFE_FIELD: str = "extensions.technical_life_years" -"""Names that sidecar key in the audit trail; the network file itself has no such column.""" - EXT_RETIREMENT_YEAR_FIELD: str = "extensions.retirement_year" -"""Names that sidecar key in the audit trail; the network file itself has no such column.""" # --- generators --------------------------------------------------------------- diff --git a/interop/plugins/shared/pypsa_constants.py b/interop/plugins/shared/pypsa_constants.py index 627b037..15f64d3 100644 --- a/interop/plugins/shared/pypsa_constants.py +++ b/interop/plugins/shared/pypsa_constants.py @@ -195,7 +195,6 @@ class PyPSALineCol: LENGTH = "length" NUM_PARALLEL = "num_parallel" TERRAIN_FACTOR = "terrain_factor" - BUILD_YEAR = "build_year" S_NOM_MIN = "s_nom_min" S_NOM_MAX = "s_nom_max" OVERNIGHT_COST = "overnight_cost" @@ -223,7 +222,6 @@ class PyPSALinkCol: ACTIVE = "active" CARRIER = "carrier" TERRAIN_FACTOR = "terrain_factor" - BUILD_YEAR = "build_year" P_NOM_MIN = "p_nom_min" P_NOM_MAX = "p_nom_max" OVERNIGHT_COST = "overnight_cost" diff --git a/interop/plugins/sources/plexos_dated_properties.py b/interop/plugins/sources/plexos_dated_properties.py index f4fc03b..86294f6 100644 --- a/interop/plugins/sources/plexos_dated_properties.py +++ b/interop/plugins/sources/plexos_dated_properties.py @@ -21,7 +21,13 @@ PlexosMembershipCol, PlexosPropertyCol, ) -from interop.plugins.shared.plexos_dates import UNDATED, DateBand +from interop.plugins.shared.plexos_dates import ( + UNDATED, + DateBand, + band_edges, + latest_covering, + opens_at, +) from interop.plugins.sources.plexos_horizon import Window from interop.plugins.sources.plexos_tables import Rows, RowsByTable @@ -70,7 +76,7 @@ def apply_window(resolved: list[DatedRow], window: Window) -> tuple[Rows, Rows]: in_force: Rows = [] stepped: Rows = [] for dated_rows in by_property.values(): - steps = _steps_within(sorted(dated_rows, key=_band_order), window) + steps = _steps_within(sorted(dated_rows, key=opens_at), window) in_force.append(steps[0].row) if len(steps) > 1: stepped.extend({**step.row, StagedTimeSeriesCol.SNAPSHOT: step.at} for step in steps) @@ -78,34 +84,39 @@ def apply_window(resolved: list[DatedRow], window: Window) -> tuple[Rows, Rows]: def dated_rows(resolved: list[DatedRow]) -> Rows: - """Every resolved property row with the dates it applies between, as the model states them. + """Every row of a property the model dates, beside the dates it applies between. ``apply_window`` reads one value per property for the window being translated, which loses the years the bands outside it name. A schedule stated as a series of bands is - read from these rows instead. + read from these rows instead. A property the model dates nowhere states the same value + for all time, which ``properties`` already carries. """ + by_property: dict[tuple[Any, ...], list[DatedRow]] = {} + for dated in resolved: + by_property.setdefault(_property_name(dated.row), []).append(dated) return [ { **dated.row, PlexosDatedPropertyCol.DATE_FROM: dated.dates.date_from, PlexosDatedPropertyCol.DATE_TO: dated.dates.date_to, } - for dated in resolved + for group in by_property.values() + if any(dated.dates != UNDATED for dated in group) + for dated in group ] -def _property_identity(row: dict[str, Any]) -> tuple[Any, ...]: - """What makes a property one property: its membership, its name, and its band.""" +def _property_name(row: dict[str, Any]) -> tuple[Any, ...]: + """What names one property across its bands: its membership and its property name.""" return ( *(row[column] for column in _MEMBERSHIP_NAME_COLUMNS), row[PlexosPropertyCol.PROPERTY], - row[PlexosPropertyCol.BAND], ) -def _band_order(dated: DatedRow) -> datetime: - """Undated values sort first, being in force before any dated band begins.""" - return dated.dates.date_from or datetime.min +def _property_identity(row: dict[str, Any]) -> tuple[Any, ...]: + """What makes a property one property: its membership, its name, and its band.""" + return (*_property_name(row), row[PlexosPropertyCol.BAND]) def _steps_within(ordered: list[DatedRow], window: Window) -> list[_Step]: @@ -118,29 +129,31 @@ def _steps_within(ordered: list[DatedRow], window: Window) -> list[_Step]: ] -def _stated_for_no_date(ordered: list[DatedRow]) -> DatedRow | None: - return next((dated for dated in ordered if dated.dates == UNDATED), None) +def _stated_for_no_date(ordered: list[DatedRow]) -> dict[str, Any] | None: + return next((dated.row for dated in ordered if dated.dates == UNDATED), None) -def _value_at(ordered: list[DatedRow], outside: DatedRow | None, moment: datetime) -> float | None: +def _value_at( + ordered: list[DatedRow], outside: dict[str, Any] | None, moment: datetime +) -> float | None: """The latest band covering the moment, else the value stated for no date, else none. A property stated only for a period is not in effect outside one, and a property with no value in effect is a property the model is not applying: it reads as zero. """ - covering = [dated for dated in ordered if dated.dates.covers(moment)] - stating = covering[-1] if covering else outside + stating = latest_covering(ordered, moment) + if stating is None: + stating = outside if stating is None: return _NOT_IN_EFFECT - value: float | None = stating.row[PlexosPropertyCol.VALUE] + value: float | None = stating[PlexosPropertyCol.VALUE] return value def _change_moments(ordered: list[DatedRow], window: Window) -> list[datetime]: """When the window opens, and every band edge inside it.""" moments = {window.start} - edges = (edge for dated in ordered for edge in (dated.dates.date_from, dated.dates.ends)) - moments.update(edge for edge in edges if edge is not None and window.start < edge < window.end) + moments.update(edge for edge in band_edges(ordered) if window.start < edge < window.end) return sorted(moments) diff --git a/interop/plugins/sources/stage_plexos_xml.py b/interop/plugins/sources/stage_plexos_xml.py index 7f2a41a..4182ffb 100644 --- a/interop/plugins/sources/stage_plexos_xml.py +++ b/interop/plugins/sources/stage_plexos_xml.py @@ -10,8 +10,8 @@ ``t_data`` value under the selected Model's Scenario overlays (highest Read Order wins, base value otherwise), with a ``data_file`` path where the value is file-backed; -- ``topology/dated_properties.parquet``, the same resolved values before they are - narrowed to the window, each beside the dates it applies between; +- ``topology/dated_properties.parquet``, every row of a property the model dates + before it is narrowed to the window, each beside the dates it applies between; - one ``source_time_series`` frame per (owner class, property) whose value comes from an external CSV, streamed in as ``(snapshot, component, sample, value)`` rows. ``sample`` is null except where the CSV carries one column per Monte Carlo @@ -385,8 +385,8 @@ class _ResolvedDataset: ``properties`` holds one row per property, the value in force when the window opens. ``stepped_properties`` holds every value a property takes within the window, and only for the properties that take more than one, so a consumer can read the shape. - ``dated_properties`` holds every resolved row with the dates it applies between, - narrowed to no window at all. + ``dated_properties`` holds every row of a property the model dates, with the dates it + applies between, narrowed to no window at all. """ objects_by_class: _RowsByClass diff --git a/tests/features/plexos_to_pypsa/constraints.feature b/tests/features/plexos_to_pypsa/constraints.feature index 683bab6..feea62e 100644 --- a/tests/features/plexos_to_pypsa/constraints.feature +++ b/tests/features/plexos_to_pypsa/constraints.feature @@ -82,7 +82,7 @@ Feature: PLEXOS Constraint objects travel in the extensions sidecar And the model is saved as "inputs/unbounded.xml" When I run translate against "inputs/unbounded.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the file "decisions.md" contains "`plexos.Constraint.Unbounded`" - And the file "decisions.md" contains "a Constraint holds a weighted sum over the objects it names to its right-hand side, which PyPSA's GlobalConstraint cannot express, so the limit is not carried" + And the file "decisions.md" contains "this Constraint states no sense, or no right-hand side, so it holds no inequality to carry to the extensions sidecar" And the file "outputs/extensions.json" does not contain "Unbounded" Scenario: a Constraint stating no sense states no inequality, so it is left out @@ -97,7 +97,7 @@ Feature: PLEXOS Constraint objects travel in the extensions sidecar And the model is saved as "inputs/senseless.xml" When I run translate against "inputs/senseless.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the file "decisions.md" contains "`plexos.Constraint.Senseless.RHS Day` = 400.0" - And the file "decisions.md" contains "so the limit is not carried" + And the file "decisions.md" contains "this Constraint states no sense, or no right-hand side, so it holds no inequality to carry to the extensions sidecar" And the file "outputs/extensions.json" does not contain "Senseless" Scenario: a readable Constraint reaches the sidecar with its sense, its limit and its members From d46d56ecc492866b4f7957fdbba626999a897e20 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 22:41:31 +0000 Subject: [PATCH 14/17] Give both ends of a lifespan one owner, and name the Sense nothing reads The build year and the retirement year are two ends of one concept, so they now travel as one LifespanDecisions bundle beside the ExpansionDecisions bundle: derive_lifespan states both ends, record_lifespan records the end the sidecar carries, and each mapping declares one field. The generator path loses the local dict of retirement years it built inside _carry_to_extensions and reads the ends off the decisions it already derived (r2-01). An undated band covers every moment, so latest_covering always finds one where the property states any, and the fallback to the undated row could never run. It goes, together with the parameter and the helper feeding it (r2-02). A Sense that is not -1, 0 or 1 read as no Sense at all, so the report told the reader the model stated nothing where it stated something the translator cannot read. The Constraint keeps the code the model stated, the note says the code is unreadable, and the description names it. A scenario covers it. The map_constraints docstring said the step carries every Constraint, which a Constraint stating no sense or no right-hand side does not (r2-11), and the two sidecar scenarios in constraints.feature built the same model twice, so the assertions of the second now stand on the first (r2-12). Comments and docstrings that restated the code they sit on are gone: the ConstraintPeriod paragraph, the two build_year schema comments, the _states_a_date and latest_covering docstrings, and the _PERIODS heading (r2-06, r2-07, r2-08, r2-09, r2-10). Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 5 +- interop/core/extensions.py | 6 +- interop/plugins/shared/plexos_dates.py | 1 - .../plexos_pypsa_translations/_batteries.py | 8 +-- .../plexos_pypsa_translations/_constraints.py | 25 +++++++- .../_generator_decisions.py | 9 ++- .../plexos_pypsa_translations/_generators.py | 33 +++++----- .../plexos_pypsa_translations/_lifespan.py | 26 ++++++-- .../_storage_hydro.py | 6 +- .../_storage_shared.py | 6 +- .../_storage_units.py | 6 +- interop/plugins/shared/pypsa_constants.py | 2 - .../sources/plexos_dated_properties.py | 15 +---- .../steps/plexos_to_pypsa/map_constraints.py | 2 +- .../plexos_to_pypsa/constraints.feature | 60 +++++++++---------- 15 files changed, 110 insertions(+), 100 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index ec5edb6..b36051d 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -1033,8 +1033,9 @@ framework that can express it. | `members` | One entry per object the sum names, each giving the name, the PLEXOS class it belongs to, the coefficient and the property that states the coefficient | | `applies_to_expansion_plan` | `Include in LT Plan` | -A `Constraint` that states no sense, or no right-hand side at all, states no inequality to -carry. The translator leaves that one out and reports it. +A `Constraint` that states no sense, a `Sense` other than `-1`, `0` or `1`, or no +right-hand side at all, states no inequality to carry. The translator leaves that one out +and reports it, naming the `Sense` it could not read where that is the reason. The report carries all of them either way. Every right-hand side a `Constraint` states becomes a not mapped entry against the object that states it, giving the value, the sense, diff --git a/interop/core/extensions.py b/interop/core/extensions.py index 8a829ad..5a6fc0d 100644 --- a/interop/core/extensions.py +++ b/interop/core/extensions.py @@ -93,11 +93,7 @@ class ConstraintSense(StrEnum): class ConstraintPeriod(StrEnum): - """The span one right-hand side applies over. - - PLEXOS states a limit per hour, day, week, month or year, or over the whole horizon, - and spells each as a right-hand side property of its own. - """ + """The span one right-hand side applies over.""" HORIZON = "horizon" # PLEXOS RHS HOUR = "hour" # PLEXOS RHS Hour diff --git a/interop/plugins/shared/plexos_dates.py b/interop/plugins/shared/plexos_dates.py index 0455e60..dcb6aa1 100644 --- a/interop/plugins/shared/plexos_dates.py +++ b/interop/plugins/shared/plexos_dates.py @@ -30,7 +30,6 @@ def covers(self, moment: datetime) -> bool: def latest_covering(bands: Sequence[tuple[DateBand, T]], moment: datetime) -> T | None: - """What the last band covering the moment states, else None.""" covering = [value for band, value in bands if band.covers(moment)] return covering[-1] if covering else None diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index fdfc248..96ce8ef 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -22,10 +22,7 @@ derive_expansion, gather_sources, ) -from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( - derive_build_year, - derive_retirement_year, -) +from interop.plugins.shared.plexos_pypsa_translations._lifespan import derive_lifespan from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( CARRIER_NOTE, @@ -135,9 +132,8 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping: ), inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE), cyclic=_battery_cyclic(rated), - build_year=derive_build_year(PlexosClass.BATTERY, rated.name, rated.lifespan), + lifespan=derive_lifespan(PlexosClass.BATTERY, rated.name, rated.lifespan), expansion=expansion, - retirement_year=derive_retirement_year(PlexosClass.BATTERY, rated.name, rated.lifespan), units=rated.properties.get(PlexosProperty.UNITS), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py index e6fc425..e1ef704 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py @@ -62,6 +62,10 @@ "this Constraint states no sense, or no right-hand side, so it holds no inequality to " "carry to the extensions sidecar" ) +_UNREADABLE_SENSE_NOTE = ( + "this Constraint states a Sense that is not -1, 0 or 1, so the translator cannot read the " + "inequality it holds in and carries nothing to the extensions sidecar" +) # PLEXOS states which way a Constraint binds as one integer. _SENSES: dict[float, ConstraintSense] = { @@ -71,7 +75,6 @@ } _UNSTATED_SENSE = "unstated" -# The right-hand side properties, each with the span it holds the sum over. _PERIODS: dict[str, ConstraintPeriod] = { PlexosProperty.RHS: ConstraintPeriod.HORIZON, PlexosProperty.RHS_HOUR: ConstraintPeriod.HOUR, @@ -101,6 +104,8 @@ class _Term: class _Constraint: name: str sense: ConstraintSense | None + # What the model states, kept so a code the translator cannot read reports as itself. + stated_sense: float | None terms: tuple[_Term, ...] right_hand_sides: dict[str, float] units: dict[str, str | None] @@ -159,6 +164,7 @@ def _read_one( return _Constraint( name=name, sense=_read_sense(stated), + stated_sense=stated.get(PlexosProperty.SENSE), terms=_build_terms(members, coefficients), right_hand_sides={ property_name: stated[property_name] @@ -240,7 +246,7 @@ def _member(term: _Term) -> ConstraintMember: def _record(reporter: SourceReporter, outcome: _Outcome) -> None: constraint = outcome.constraint - carried = _CARRIED_NOTE if outcome.record is not None else _NOT_CARRIED_NOTE + carried = _CARRIED_NOTE if outcome.record is not None else _not_carried_note(constraint) note = f"{carried}. {_describe(constraint)}" if not constraint.right_hand_sides: reporter.record_dropped(_source(constraint.name, None, None), note) @@ -256,9 +262,16 @@ def _source( return SourceValue(PlexosClass.CONSTRAINT, name, attribute, value, unit) +def _not_carried_note(constraint: _Constraint) -> str: + """Why the Constraint holds no inequality: a sense it lacks, or one nothing reads.""" + if constraint.sense is None and constraint.stated_sense is not None: + return _UNREADABLE_SENSE_NOTE + return _NOT_CARRIED_NOTE + + def _describe(constraint: _Constraint) -> str: """The sense the Constraint binds in, and the weighted sum it binds.""" - sense = constraint.sense or _UNSTATED_SENSE + sense = _sense_text(constraint) if not constraint.terms: return f"Sense {sense}, over no objects" return ( @@ -267,6 +280,12 @@ def _describe(constraint: _Constraint) -> str: ) +def _sense_text(constraint: _Constraint) -> str: + if constraint.sense is not None: + return constraint.sense + return _UNSTATED_SENSE if constraint.stated_sense is None else str(constraint.stated_sense) + + def _describe_term(term: _Term) -> str: if term.coefficient is None: return f"{term.member.member_class} {term.member.name} with {_NO_COEFFICIENT}" diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py index 2c74b7a..a83ba06 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generator_decisions.py @@ -42,7 +42,10 @@ ThermalCostTerms, UnitCommitment, ) -from interop.plugins.shared.plexos_pypsa_translations._lifespan import derive_build_year +from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( + LifespanDecisions, + derive_lifespan, +) from interop.plugins.shared.plexos_pypsa_translations.constants import ( FULL_AVAILABILITY, MARGINAL_COST_CARBON_TERM, @@ -151,7 +154,7 @@ class GeneratorDecisions: up_time_before: Decision = maps_to(PyPSAGeneratorCol.UP_TIME_BEFORE, unit=UNIT_SNAPSHOTS) start_up_cost: Decision = maps_to(PyPSAGeneratorCol.START_UP_COST, unit=UNIT_DOLLARS) shut_down_cost: Decision = maps_to(PyPSAGeneratorCol.SHUT_DOWN_COST, unit=UNIT_DOLLARS) - build_year: Decision = maps_to(PyPSAGeneratorCol.BUILD_YEAR) + lifespan: LifespanDecisions = holds() expansion: ExpansionDecisions = holds() @@ -182,7 +185,7 @@ def decide_generator(mapping: GeneratorMapping) -> GeneratorDecisions: up_time_before=commitment.up_time_before, start_up_cost=commitment.start_up_cost, shut_down_cost=commitment.shut_down_cost, - build_year=derive_build_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan), + lifespan=derive_lifespan(PlexosClass.GENERATOR, mapping.name, mapping.lifespan), expansion=mapping.expansion, ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_generators.py b/interop/plugins/shared/plexos_pypsa_translations/_generators.py index 5b4d673..8580548 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_generators.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_generators.py @@ -45,9 +45,8 @@ build_lookups, ) from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( - RETIREMENT_YEAR_COLUMN, - derive_retirement_year, read_year, + record_lifespan, ) from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series from interop.plugins.shared.plexos_pypsa_translations._storage_turbines import ( @@ -131,9 +130,8 @@ def map_generators(state: State, recorder: ScopedRecorder) -> None: ], GENERATORS_DESTINATION_SCHEMA, ) - mappings = [one.mapping for one in translated] - _carry_to_extensions(state, mappings, reporter) - _record_availability_time_series(state, mappings, reporter) + _carry_to_extensions(state, translated, reporter) + _record_availability_time_series(state, [one.mapping for one in translated], reporter) @dataclass(frozen=True) @@ -145,31 +143,28 @@ class _TranslatedGenerator: def _carry_to_extensions( - state: State, mappings: list[GeneratorMapping], reporter: ComponentReporter + state: State, translated: list[_TranslatedGenerator], reporter: ComponentReporter ) -> None: """Put what the network file cannot hold in the sidecar, starting with the PLEXOS category, since only one of it and the fuel could become the carrier. """ - retirements = { - mapping.name: derive_retirement_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan) - for mapping in mappings - } - for mapping in mappings: + for one in translated: + mapping = one.mapping if mapping.carrier != mapping.category: reporter.record(mapping.name, _CATEGORY_COLUMN, _category_decision(mapping)) record_expansion(mapping.name, mapping.expansion, reporter) - reporter.record(mapping.name, RETIREMENT_YEAR_COLUMN, retirements[mapping.name]) + record_lifespan(mapping.name, one.decisions.lifespan, reporter) records = [ GeneratorExtension( - name=mapping.name, - category=mapping.category, - unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), - technical_life_years=read_sidecar_value(mapping.expansion.technical_life), - retirement_year=read_year(retirements[mapping.name]), + name=one.mapping.name, + category=one.mapping.category, + unit_size_mw=read_sidecar_value(one.mapping.expansion.unit_size), + technical_life_years=read_sidecar_value(one.mapping.expansion.technical_life), + retirement_year=read_year(one.decisions.lifespan.retirement_year), ) - for mapping in mappings + for one in translated ] - warn_about_dropped_builds(mapping.expansion for mapping in mappings) + warn_about_dropped_builds(one.mapping.expansion for one in translated) append_extensions(state.destination_extensions, ExtensionKind.GENERATOR, records) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py index 381adf0..09bb170 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py @@ -8,6 +8,7 @@ from __future__ import annotations +from dataclasses import dataclass from datetime import datetime from typing import NamedTuple @@ -25,14 +26,18 @@ latest_covering, opens_at, ) +from interop.plugins.shared.plexos_pypsa_translations._expansion import NOTHING_TO_REPORT from interop.plugins.shared.plexos_pypsa_translations.constants import ( EXT_RETIREMENT_YEAR_FIELD, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( + ComponentReporter, Decision, MappedColumns, SourceValue, + maps_to, ) +from interop.plugins.shared.pypsa_constants import PyPSAGeneratorCol # What a schedule runs while no band it states covers the moment. _NOT_IN_SERVICE = 0.0 @@ -63,6 +68,13 @@ class Lifespan(NamedTuple): NO_LIFESPAN = Lifespan(None, None) +@dataclass(frozen=True) +class LifespanDecisions: + build_year: Decision = maps_to(PyPSAGeneratorCol.BUILD_YEAR) + # The sidecar carries the retirement year, since PyPSA has no column for it. + retirement_year: Decision = NOTHING_TO_REPORT + + class _UnitsBand(NamedTuple): """How many units one dated ``Units`` row runs, and the dates it runs them between.""" @@ -82,12 +94,17 @@ def read_lifespans(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, return {name: _read_lifespan(bands) for name, bands in _read_bands(dated, plexos_class).items()} -def derive_build_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision: - return _derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION) +def derive_lifespan(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> LifespanDecisions: + return LifespanDecisions( + build_year=_derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION), + retirement_year=_derive_year( + plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION + ), + ) -def derive_retirement_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision: - return _derive_year(plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION) +def record_lifespan(name: str, decisions: LifespanDecisions, reporter: ComponentReporter) -> None: + reporter.record(name, RETIREMENT_YEAR_COLUMN, decisions.retirement_year) def _derive_year( @@ -133,7 +150,6 @@ def _read_bands(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, lis def _states_a_date(bands: list[_UnitsBand]) -> bool: - """An object whose ``Units`` are one value for all time keeps that value for all time.""" return any(band.dates != UNDATED for band in bands) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index 4790f7d..8ab1eae 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -26,8 +26,7 @@ derive_expansion, ) from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( - derive_build_year, - derive_retirement_year, + derive_lifespan, ) from interop.plugins.shared.plexos_pypsa_translations._storage_shared import ( CARRIER_NOTE, @@ -225,9 +224,8 @@ def _derive_turbine( ), inflow=_reservoir_inflow(head), cyclic=Decision.default(variant.cyclic, variant.cyclic_note), - build_year=derive_build_year(PlexosClass.GENERATOR, rated.name, rated.lifespan), + lifespan=derive_lifespan(PlexosClass.GENERATOR, rated.name, rated.lifespan), expansion=expansion, - retirement_year=derive_retirement_year(PlexosClass.GENERATOR, rated.name, rated.lifespan), inflow_storage=_inflow_storage(head), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index 439f4b4..3383ed4 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -26,7 +26,6 @@ PlexosResolvedTable, ) from interop.plugins.shared.plexos_pypsa_translations._expansion import ( - NOTHING_TO_REPORT, CandidateSource, ExpansionDecisions, RatedCapacity, @@ -36,6 +35,7 @@ from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( NO_LIFESPAN, Lifespan, + LifespanDecisions, read_lifespans, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( @@ -121,10 +121,8 @@ class StorageUnitMapping: ) inflow: Decision = maps_to(PyPSAStorageUnitCol.INFLOW, unit=UNIT_MW) cyclic: Decision = maps_to(PyPSAStorageUnitCol.CYCLIC_STATE_OF_CHARGE) - build_year: Decision = maps_to(PyPSAStorageUnitCol.BUILD_YEAR) + lifespan: LifespanDecisions = holds() expansion: ExpansionDecisions = holds() - # The sidecar carries the retirement year, since PyPSA has no column for it. - retirement_year: Decision = NOTHING_TO_REPORT # Units is a Battery-only reading; a units-out trace derates against it. units: float | None = None # The head Storage whose Natural Inflow this unit reads, where that inflow is power. An diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index c565824..978e5fb 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -33,8 +33,8 @@ warn_about_dropped_builds, ) from interop.plugins.shared.plexos_pypsa_translations._lifespan import ( - RETIREMENT_YEAR_COLUMN, read_year, + record_lifespan, ) from interop.plugins.shared.plexos_pypsa_translations._shared import ( ObjectProperties, @@ -233,7 +233,7 @@ def _record(storage_units: _DerivedStorageUnits, recorder: ScopedRecorder) -> No for mapping in storage_units.mappings: reporter.record_mapping(mapping.name, mapping) record_expansion(mapping.name, mapping.expansion, reporter) - reporter.record(mapping.name, RETIREMENT_YEAR_COLUMN, mapping.retirement_year) + record_lifespan(mapping.name, mapping.lifespan, reporter) warn_about_dropped_builds(mapping.expansion for mapping in storage_units.mappings) for skipped in storage_units.skipped: reporter.record_skipped(skipped.source, skipped.note) @@ -248,7 +248,7 @@ def _carry_to_extensions(state: State, mappings: list[StorageUnitMapping]) -> No name=mapping.name, unit_size_mw=read_sidecar_value(mapping.expansion.unit_size), technical_life_years=read_sidecar_value(mapping.expansion.technical_life), - retirement_year=read_year(mapping.retirement_year), + retirement_year=read_year(mapping.lifespan.retirement_year), ) for mapping in mappings ] diff --git a/interop/plugins/shared/pypsa_constants.py b/interop/plugins/shared/pypsa_constants.py index 15f64d3..edf08af 100644 --- a/interop/plugins/shared/pypsa_constants.py +++ b/interop/plugins/shared/pypsa_constants.py @@ -489,7 +489,6 @@ class ReverseTimeSeriesMetadataCol: PyPSAGeneratorCol.DISCOUNT_RATE: pl.Float64, PyPSAGeneratorCol.LIFETIME: pl.Float64, PyPSAGeneratorCol.FOM_COST: pl.Float64, - # The year the source brings the generator into service, null where it states none. PyPSAGeneratorCol.BUILD_YEAR: pl.Int64, } @@ -528,7 +527,6 @@ class ReverseTimeSeriesMetadataCol: PyPSAStorageUnitCol.DISCOUNT_RATE: pl.Float64, PyPSAStorageUnitCol.LIFETIME: pl.Float64, PyPSAStorageUnitCol.FOM_COST: pl.Float64, - # The year the source brings the storage unit into service, null where it states none. PyPSAStorageUnitCol.BUILD_YEAR: pl.Int64, } diff --git a/interop/plugins/sources/plexos_dated_properties.py b/interop/plugins/sources/plexos_dated_properties.py index 86294f6..21d089b 100644 --- a/interop/plugins/sources/plexos_dated_properties.py +++ b/interop/plugins/sources/plexos_dated_properties.py @@ -122,28 +122,19 @@ def _property_identity(row: dict[str, Any]) -> tuple[Any, ...]: def _steps_within(ordered: list[DatedRow], window: Window) -> list[_Step]: """One step per moment the property's value changes inside the window.""" template = ordered[0].row - outside = _stated_for_no_date(ordered) return [ - _Step(moment, {**template, PlexosPropertyCol.VALUE: _value_at(ordered, outside, moment)}) + _Step(moment, {**template, PlexosPropertyCol.VALUE: _value_at(ordered, moment)}) for moment in _change_moments(ordered, window) ] -def _stated_for_no_date(ordered: list[DatedRow]) -> dict[str, Any] | None: - return next((dated.row for dated in ordered if dated.dates == UNDATED), None) - - -def _value_at( - ordered: list[DatedRow], outside: dict[str, Any] | None, moment: datetime -) -> float | None: - """The latest band covering the moment, else the value stated for no date, else none. +def _value_at(ordered: list[DatedRow], moment: datetime) -> float | None: + """The latest band covering the moment, else none. A property stated only for a period is not in effect outside one, and a property with no value in effect is a property the model is not applying: it reads as zero. """ stating = latest_covering(ordered, moment) - if stating is None: - stating = outside if stating is None: return _NOT_IN_EFFECT value: float | None = stating[PlexosPropertyCol.VALUE] diff --git a/interop/plugins/steps/plexos_to_pypsa/map_constraints.py b/interop/plugins/steps/plexos_to_pypsa/map_constraints.py index f18a8c5..cab0eba 100644 --- a/interop/plugins/steps/plexos_to_pypsa/map_constraints.py +++ b/interop/plugins/steps/plexos_to_pypsa/map_constraints.py @@ -12,7 +12,7 @@ class PlexosToPypsaMapConstraints(TranslationStep): - """Carries every PLEXOS Constraint to the extensions sidecar; PyPSA enforces none.""" + """Carries each PLEXOS Constraint it can read to the extensions sidecar; PyPSA enforces none.""" name: ClassVar[str] = "plexos_to_pypsa_map_constraints" params_schema: ClassVar[type[BaseModel] | None] = None diff --git a/tests/features/plexos_to_pypsa/constraints.feature b/tests/features/plexos_to_pypsa/constraints.feature index feea62e..d639c24 100644 --- a/tests/features/plexos_to_pypsa/constraints.feature +++ b/tests/features/plexos_to_pypsa/constraints.feature @@ -9,10 +9,14 @@ Feature: PLEXOS Constraint objects travel in the extensions sidecar Each Constraint therefore travels in the extensions sidecar instead, and the network's silence about it is reported: every right-hand side a Constraint states is recorded against the object stating it, so a reader can see which limits the solved network is not - holding to. A Constraint stating no sense, or no right-hand side at all, states no - inequality to carry, so it is left out and reported as such. + holding to. A Constraint stating no sense, no sense the translator can read, or no + right-hand side at all, states no inequality to carry, so it is left out and reported as + such. - Scenario: a daily energy limit over named generators is reported, naming what it binds + Scenario: a daily energy limit over named generators reaches the sidecar and the report + The sidecar states the limit in its own vocabulary: the sense as the inequality it holds + in, the right-hand side beside the span it applies over, and each object the sum weights + with the class it belongs to. Given a Plexos model And the model contains region "Grid" And the model contains node "Grid_Node" in region "Grid" @@ -23,16 +27,25 @@ Feature: PLEXOS Constraint objects travel in the extensions sidecar And the model contains constraint "RiverSystem" over: | class | name | coefficient property | coefficient | | Generator | AA1 | Generation Coefficient | 1 | - | Generator | AA2 | Generation Coefficient | 1 | + | Generator | AA2 | Generation Coefficient | 2.5 | And constraint "RiverSystem" states "Sense" of -1 And constraint "RiverSystem" states "RHS Day" of 1.708 + And constraint "RiverSystem" states "Include in LT Plan" of 1 And the model is saved as "inputs/constraint.xml" When I run translate against "inputs/constraint.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" Then the file "outputs/network.nc" exists And the file "decisions.md" contains "`plexos.Constraint.RiverSystem.RHS Day` = 1.708" And the file "decisions.md" contains "constraint carried to the extensions sidecar; PyPSA's GlobalConstraint cannot hold a weighted sum over the objects a Constraint names, so the network file itself does not limit them" - And the file "decisions.md" contains "Sense <= over 2 term(s): 1.0 x Generator AA1 (Generation Coefficient), 1.0 x Generator AA2 (Generation Coefficient)" + And the file "decisions.md" contains "Sense <= over 2 term(s): 1.0 x Generator AA1 (Generation Coefficient), 2.5 x Generator AA2 (Generation Coefficient)" And the log contains "plexos: 1 Constraint(s) limit what the model may dispatch and the network file enforces none of them; each one the translator can read travels in the extensions sidecar: RiverSystem" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.name" set to "RiverSystem" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.sense" set to "<=" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.limits.0.period" set to "day" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.limits.0.value" set to 1.708 + And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.0.name" set to "AA1" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.0.member_class" set to "Generator" + And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.1.coefficient" set to 2.5 + And the file "outputs/extensions.json" parses as JSON with "constraint.0.applies_to_expansion_plan" set to true Scenario: every right-hand side a Constraint states is reported, one for each Given a Plexos model @@ -100,34 +113,21 @@ Feature: PLEXOS Constraint objects travel in the extensions sidecar And the file "decisions.md" contains "this Constraint states no sense, or no right-hand side, so it holds no inequality to carry to the extensions sidecar" And the file "outputs/extensions.json" does not contain "Senseless" - Scenario: a readable Constraint reaches the sidecar with its sense, its limit and its members - The sidecar states the limit in its own vocabulary: the sense as the inequality it holds - in, the right-hand side beside the span it applies over, and each object the sum weights - with the class it belongs to. + Scenario: a Constraint whose Sense the translator cannot read names the code it stated Given a Plexos model And the model contains region "Grid" And the model contains node "Grid_Node" in region "Grid" - And the model contains generators: - | name | node | category | Max Capacity | - | AA1 | Grid_Node | Hydro | 21 | - | AA2 | Grid_Node | Hydro | 23 | - And the model contains constraint "RiverSystem" over: - | class | name | coefficient property | coefficient | - | Generator | AA1 | Generation Coefficient | 1 | - | Generator | AA2 | Generation Coefficient | 2.5 | - And constraint "RiverSystem" states "Sense" of -1 - And constraint "RiverSystem" states "RHS Day" of 1.708 - And constraint "RiverSystem" states "Include in LT Plan" of 1 - And the model is saved as "inputs/carried.xml" - When I run translate against "inputs/carried.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" - Then the file "outputs/extensions.json" parses as JSON with "constraint.0.name" set to "RiverSystem" - And the file "outputs/extensions.json" parses as JSON with "constraint.0.sense" set to "<=" - And the file "outputs/extensions.json" parses as JSON with "constraint.0.limits.0.period" set to "day" - And the file "outputs/extensions.json" parses as JSON with "constraint.0.limits.0.value" set to 1.708 - And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.0.name" set to "AA1" - And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.0.member_class" set to "Generator" - And the file "outputs/extensions.json" parses as JSON with "constraint.0.members.1.coefficient" set to 2.5 - And the file "outputs/extensions.json" parses as JSON with "constraint.0.applies_to_expansion_plan" set to true + And the model contains generator "Peaker" with "node=Grid_Node, category=Gas, Max Capacity=100" + And the model contains constraint "Unreadable" over: + | class | name | coefficient property | coefficient | + | Generator | Peaker | Generation Coefficient | 1 | + And constraint "Unreadable" states "Sense" of 7 + And constraint "Unreadable" states "RHS Day" of 400 + And the model is saved as "inputs/unreadable_sense.xml" + When I run translate against "inputs/unreadable_sense.xml" pipeline "plexos-to-pypsa" sink output "outputs/network.nc" + Then the file "decisions.md" contains "this Constraint states a Sense that is not -1, 0 or 1, so the translator cannot read the inequality it holds in and carries nothing to the extensions sidecar" + And the file "decisions.md" contains "Sense 7.0 over 1 term(s)" + And the file "outputs/extensions.json" does not contain "Unreadable" Scenario: a model with no Constraint objects warns about nothing Given a Plexos model From 313fa7b7fb9587155b192ab69631f519aaa7ff92 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 23:03:00 +0000 Subject: [PATCH 15/17] Read one PLEXOS class per storage lookup, and drop what the code does not do StorageLookups stated the same five readings twice, once for a Battery and once for a Generator. One ClassLookups per PLEXOS class holds them instead, and staged(plexos_class, name) builds the StagedObject from whichever class the caller names, so a new per-object reading costs one field rather than two. The Units row of the PLEXOS to PyPSA mapping claimed a component running no units in the year translated is left out. It is not: derive_p_nom gives a candidate its buildable capacity where its existing rating is zero, so the component is written with the build_year the schedule states. The skipped-component table already gives the real rule, so the sentence goes. _states_a_date guarded nothing. A band list that is all undated yields no band edge, so it already read as NO_LIFESPAN, which is what both callers fall back to for a name the map does not hold. Three docstrings that only paraphrased what they sat above go with them. Addresses r3-01, r3-03, r3-04, r3-05, r3-06, r3-07 and r3-08. Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- .../translation-from-plexos-to-pypsa.md | 2 +- .../plexos_pypsa_translations/_batteries.py | 2 +- .../plexos_pypsa_translations/_constraints.py | 2 - .../plexos_pypsa_translations/_lifespan.py | 11 +-- .../_storage_hydro.py | 2 +- .../_storage_shared.py | 80 ++++++++++--------- .../_storage_units.py | 2 +- .../sources/plexos_dated_properties.py | 6 +- 8 files changed, 49 insertions(+), 58 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index b36051d..9c1b6ea 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -774,7 +774,7 @@ Two properties use dated entries as a schedule. They do not use them as correcti | Property | Meaning | | --- | --- | -| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. The translator reads both years off the entries as the model dates them, whatever year it translates: the first year the value rises above zero is the `build_year` of the component it writes, and the first year after that in which the value falls back to zero is the `retirement_year` in the extensions sidecar. A component running no units in the year translated is still left out, since it can dispatch nothing that year. | +| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. The translator reads both years off the entries as the model dates them, whatever year it translates: the first year the value rises above zero is the `build_year` of the component it writes, and the first year after that in which the value falls back to zero is the `retirement_year` in the extensions sidecar. | | `Max Capacity` | A capacity expansion schedule. The translator applies the entry that is in force at the snapshot. Where it needs one value, it uses the entry that is in force at the start of the model. | ### Timeslice patterns diff --git a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py index 96ce8ef..5075d2a 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_batteries.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_batteries.py @@ -98,7 +98,7 @@ def record_battery_outages(state: State, mappings: list[StorageUnitMapping]) -> def map_battery(name: str, lookups: StorageLookups) -> MappedOrSkipped: - rated = rate_object(lookups.battery(name), _BATTERY_POWER) + rated = rate_object(lookups.staged(PlexosClass.BATTERY, name), _BATTERY_POWER) if isinstance(rated, SkippedComponent): return rated return _derive_battery(rated) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py index e1ef704..c02929b 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py @@ -114,8 +114,6 @@ class _Constraint: @dataclass(frozen=True) class _Outcome: - """One Constraint and the record it travels as, which is None where it states too little.""" - constraint: _Constraint record: ConstraintExtension | None diff --git a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py index 09bb170..19e2146 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py @@ -20,7 +20,6 @@ PlexosProperty, ) from interop.plugins.shared.plexos_dates import ( - UNDATED, DateBand, band_edges, latest_covering, @@ -76,8 +75,6 @@ class LifespanDecisions: class _UnitsBand(NamedTuple): - """How many units one dated ``Units`` row runs, and the dates it runs them between.""" - dates: DateBand units: float @@ -144,13 +141,7 @@ def _read_bands(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, lis bands: dict[str, list[_UnitsBand]] = {} for name, date_from, date_to, units in frame.iter_rows(): bands.setdefault(name, []).append(_UnitsBand(DateBand(date_from, date_to), units)) - return { - name: sorted(rows, key=opens_at) for name, rows in bands.items() if _states_a_date(rows) - } - - -def _states_a_date(bands: list[_UnitsBand]) -> bool: - return any(band.dates != UNDATED for band in bands) + return {name: sorted(rows, key=opens_at) for name, rows in bands.items()} def _read_lifespan(bands: list[_UnitsBand]) -> Lifespan: diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py index 8ab1eae..9f91c86 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_hydro.py @@ -172,7 +172,7 @@ class _GeneratorStorageVariant: def map_turbine(name: str, lookups: StorageLookups) -> MappedOrSkipped: - staged = lookups.generator(name) + staged = lookups.staged(PlexosClass.GENERATOR, name) variant = _classify_turbine(staged, lookups) if variant is None: return skip_object( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index 3383ed4..c0b7316 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -11,6 +11,8 @@ from dataclasses import dataclass from math import sqrt +import polars as pl + from interop.core.pipeline import State from interop.plugins.shared.constants import ( UNIT_DOLLARS_PER_MWH, @@ -180,44 +182,37 @@ class StagedObject: lifespan: Lifespan +@dataclass(frozen=True) +class ClassLookups: + """What one PLEXOS class states about each of its objects.""" + + properties: ObjectProperties + stated_units: ObjectUnits + nodes: dict[str, str] + file_backed: dict[str, list[str]] + lifespans: dict[str, Lifespan] + + @dataclass(frozen=True) class StorageLookups: """The per-object properties and memberships the three storage paths read.""" - battery_properties: ObjectProperties - generator_properties: ObjectProperties + by_class: dict[PlexosClass, ClassLookups] storage_properties: ObjectProperties - node_by_battery: dict[str, str] - node_by_generator: dict[str, str] + storage_units: ObjectUnits head_by_generator: dict[str, str] tail_by_generator: dict[str, str] - file_backed_by_battery: dict[str, list[str]] - file_backed_by_generator: dict[str, list[str]] - battery_units: ObjectUnits - generator_units: ObjectUnits - storage_units: ObjectUnits storages_with_inflow_profile: set[str] - lifespan_by_battery: dict[str, Lifespan] - lifespan_by_generator: dict[str, Lifespan] - - def battery(self, name: str) -> StagedObject: - return StagedObject( - name=name, - properties=self.battery_properties.get(name, {}), - stated_units=self.battery_units.get(name, {}), - node=self.node_by_battery.get(name), - file_backed=self.file_backed_by_battery.get(name, []), - lifespan=self.lifespan_by_battery.get(name, NO_LIFESPAN), - ) - def generator(self, name: str) -> StagedObject: + def staged(self, plexos_class: PlexosClass, name: str) -> StagedObject: + one = self.by_class[plexos_class] return StagedObject( name=name, - properties=self.generator_properties.get(name, {}), - stated_units=self.generator_units.get(name, {}), - node=self.node_by_generator.get(name), - file_backed=self.file_backed_by_generator.get(name, []), - lifespan=self.lifespan_by_generator.get(name, NO_LIFESPAN), + properties=one.properties.get(name, {}), + stated_units=one.stated_units.get(name, {}), + node=one.nodes.get(name), + file_backed=one.file_backed.get(name, []), + lifespan=one.lifespans.get(name, NO_LIFESPAN), ) def has_head_and_tail(self, generator: str) -> bool: @@ -292,25 +287,34 @@ def build_lookups(state: State) -> StorageLookups: memberships = state.source_topology[PlexosResolvedTable.MEMBERSHIPS] dated = state.source_topology[PlexosResolvedTable.DATED_PROPERTIES] return StorageLookups( - battery_properties=collapse_properties_by_object(properties, PlexosClass.BATTERY), - generator_properties=collapse_properties_by_object(properties, PlexosClass.GENERATOR), + by_class={ + plexos_class: _read_class(properties, memberships, dated, plexos_class) + for plexos_class in (PlexosClass.BATTERY, PlexosClass.GENERATOR) + }, storage_properties=collapse_properties_by_object(properties, PlexosClass.STORAGE), - node_by_battery=relate_child(memberships, PlexosClass.BATTERY, PlexosCollection.NODES), - node_by_generator=relate_child(memberships, PlexosClass.GENERATOR, PlexosCollection.NODES), + storage_units=collapse_units_by_object(properties, PlexosClass.STORAGE), head_by_generator=relate_child( memberships, PlexosClass.GENERATOR, PlexosCollection.HEAD_STORAGE ), tail_by_generator=relate_child( memberships, PlexosClass.GENERATOR, PlexosCollection.TAIL_STORAGE ), - file_backed_by_battery=read_file_backed_properties(properties, PlexosClass.BATTERY), - file_backed_by_generator=read_file_backed_properties(properties, PlexosClass.GENERATOR), - battery_units=collapse_units_by_object(properties, PlexosClass.BATTERY), - generator_units=collapse_units_by_object(properties, PlexosClass.GENERATOR), - storage_units=collapse_units_by_object(properties, PlexosClass.STORAGE), storages_with_inflow_profile=_storages_with_inflow_profile(state), - lifespan_by_battery=read_lifespans(dated, PlexosClass.BATTERY), - lifespan_by_generator=read_lifespans(dated, PlexosClass.GENERATOR), + ) + + +def _read_class( + properties: pl.LazyFrame, + memberships: pl.LazyFrame, + dated: pl.LazyFrame, + plexos_class: PlexosClass, +) -> ClassLookups: + return ClassLookups( + properties=collapse_properties_by_object(properties, plexos_class), + stated_units=collapse_units_by_object(properties, plexos_class), + nodes=relate_child(memberships, plexos_class, PlexosCollection.NODES), + file_backed=read_file_backed_properties(properties, plexos_class), + lifespans=read_lifespans(dated, plexos_class), ) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index 978e5fb..e93cb5b 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -187,7 +187,7 @@ def _dropped_values(state: State, lookups: StorageLookups) -> list[_DroppedValue *_dropped_from( read_object_names(state, PlexosClass.BATTERY), PlexosClass.BATTERY, - lookups.battery_properties, + lookups.by_class[PlexosClass.BATTERY].properties, _BATTERY_DROPPED, ), *_dropped_from(storages, PlexosClass.STORAGE, lookups.storage_properties, _STORAGE_DROPPED), diff --git a/interop/plugins/sources/plexos_dated_properties.py b/interop/plugins/sources/plexos_dated_properties.py index 21d089b..f2c6359 100644 --- a/interop/plugins/sources/plexos_dated_properties.py +++ b/interop/plugins/sources/plexos_dated_properties.py @@ -129,10 +129,8 @@ def _steps_within(ordered: list[DatedRow], window: Window) -> list[_Step]: def _value_at(ordered: list[DatedRow], moment: datetime) -> float | None: - """The latest band covering the moment, else none. - - A property stated only for a period is not in effect outside one, and a property with - no value in effect is a property the model is not applying: it reads as zero. + """A property stated only for a period is not in effect outside one, and a property + with no value in effect is a property the model is not applying: it reads as zero. """ stating = latest_covering(ordered, moment) if stating is None: From 3056a8ee01cbbcfcb6ef9755544f0e4cd06f63c6 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Tue, 8 Sep 2026 23:13:02 +0000 Subject: [PATCH 16/17] Record what the review rounds found on the dated-units branch Co-Authored-By: Claude Opus 5 (1M context) Signed-off-by: Thomas C. King --- code-review/ledger.jsonl | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/code-review/ledger.jsonl b/code-review/ledger.jsonl index 3f2a204..7f1567d 100644 --- a/code-review/ledger.jsonl +++ b/code-review/ledger.jsonl @@ -127,3 +127,41 @@ {"id": "r7-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 7, "file": "interop/plugins/shared/pypsa_sienna_translations/_shared.py", "line": 60, "severity": "major", "label": "issue", "claim": "The rule drops an extendable component that already runs, because it reads p_nom_extendable and p_nom_opt but never p_nom_min.", "suggested_fix": "Read the floor a build cannot take away, and rate the component at it.\n\nThe code now:\n\n```python\ndef states_built_capacity(extendable: str, opt: str) -> pl.Expr:\n return ~pl.col(extendable) | pl.col(opt).is_not_null()\n\n\ndef effective_p_nom(extendable: str, opt: str, nom: str) -> pl.Expr:\n return pl.when(has_solved_capacity(extendable, opt)).then(pl.col(opt)).otherwise(pl.col(nom))\n```\n\nThe code to use:\n\n```python\ndef states_built_capacity(extendable: str, opt: str, nom_min: str) -> pl.Expr:\n \"\"\"Capacity an operations model may dispatch: fixed, solved, or the floor a build keeps.\"\"\"\n return ~pl.col(extendable) | pl.col(opt).is_not_null() | (pl.col(nom_min) > 0)\n\n\ndef effective_p_nom(extendable: str, opt: str, nom: str, nom_min: str) -> pl.Expr:\n return (\n pl.when(has_solved_capacity(extendable, opt))\n .then(pl.col(opt))\n .when(pl.col(extendable))\n .then(pl.col(nom_min))\n .otherwise(pl.col(nom))\n )\n```\n\nAdd `(P_NOM_MIN, 0.0)` to each `fill_*_defaults` float default list, and pass the column through `with_effective_p_nom`.", "rule_created": false, "ts": "2026-09-08T20:51:10Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} {"id": "r7-02", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 7, "file": "interop/core/extensions.py", "line": 154, "severity": "minor", "label": "issue", "claim": "The comment names one source property, but a StorageExtension also holds a pumped-storage turbine, which states Max Capacity.", "suggested_fix": "Name both properties.\n\nThe code now:\n\n```python\nclass StorageExtension(ExtensionRecord):\n p_nom_extendable: bool | None = None # PyPSA only\n # MW. PLEXOS Max Power: what one unit of a candidate is. PyPSA sizes a candidate by\n # p_nom_max alone, so the size of a single unit has no field there.\n unit_size_mw: float | None = None\n```\n\nThe code to use:\n\n```python\nclass StorageExtension(ExtensionRecord):\n p_nom_extendable: bool | None = None # PyPSA only\n # MW. PLEXOS Max Power on a Battery, Max Capacity on a pumped-storage turbine: what one\n # unit of a candidate is. PyPSA sizes a candidate by p_nom_max alone, so the size of a\n # single unit has no field there.\n unit_size_mw: float | None = None\n```", "rule_created": false, "ts": "2026-09-08T20:51:10Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} {"id": "r8-01", "pr": "9", "repo": "transition-zero/tz-oss-interop", "round": 8, "file": "interop/core/extensions.py", "line": 154, "severity": "minor", "label": "issue", "claim": "The comment on `StorageExtension.unit_size_mw` names only `Max Power`, but a pumped-storage turbine fills the same field from `Max Capacity`.", "suggested_fix": "Change the comment to name both source properties, for example \"MW. PLEXOS Max Power for a Battery, Max Capacity for a pumped-storage turbine: what one unit of a candidate is.\" Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T21:15:13Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-01", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 182, "severity": "major", "label": "suggestion", "claim": "The new `_lifespan` module repeats the band logic that `plexos_dated_properties` already holds.", "suggested_fix": "Move the reading of a set of bands into `plexos_dates.py`, which both modules already import. A code judo move is a restructure that deletes complexity; it does not move the complexity to another place. Three private helpers become one shared pair.\n\nThe code now, in two modules:\n\n```python\n# _lifespan.py\ndef _units_at(bands: list[_UnitsBand], moment: datetime) -> float:\n covering = [band for band in bands if band.dates.covers(moment)]\n return covering[-1].units if covering else _NOT_IN_SERVICE\n\ndef _edges(bands: list[_UnitsBand]) -> list[datetime]:\n moments = {edge for band in bands for edge in (band.dates.date_from, band.dates.ends)}\n return sorted(moment for moment in moments if moment is not None)\n\n# plexos_dated_properties.py\ndef _value_at(ordered: list[DatedRow], outside: DatedRow | None, moment: datetime) -> float | None:\n covering = [dated for dated in ordered if dated.dates.covers(moment)]\n stating = covering[-1] if covering else outside\n```\n\nThe code to use, in `interop/plugins/shared/plexos_dates.py`:\n\n```python\nT = TypeVar(\"T\")\n\ndef latest_covering(bands: Sequence[tuple[DateBand, T]], moment: datetime) -> T | None:\n \"\"\"What the last band covering the moment states, else None.\"\"\"\n covering = [value for band, value in bands if band.covers(moment)]\n return covering[-1] if covering else None\n\ndef band_edges(bands: Iterable[tuple[DateBand, object]]) -> list[datetime]:\n \"\"\"Every moment a band opens or closes, earliest first.\"\"\"\n moments = {edge for band, _ in bands for edge in (band.date_from, band.ends)}\n return sorted(moment for moment in moments if moment is not None)\n```\n\nThen `_units_at`, `_edges`, `_opens`, `_value_at`, `_change_moments` and `_band_order` call the two shared functions, and each caller applies its own zero default.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-02", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 107, "severity": "major", "label": "suggestion", "claim": "Two different paths put the same retirement year into the sidecar.", "suggested_fix": "Give `StorageUnitMapping` the `Lifespan` that `RatedObject` already carries, then read both years off it in both paths.\n\nThe code now:\n\n```python\n# _lifespan.py\ndef read_year(decision: Decision) -> int | None:\n return None if decision.value is None else int(decision.value)\n\n# _generators.py:167\n retirement_year=mapping.lifespan.retirement_year,\n\n# _storage_units.py:251\n retirement_year=read_year(mapping.retirement_year),\n```\n\nThe code to use:\n\n```python\n# _storage_shared.py, in StorageUnitMapping\n lifespan: Lifespan = NO_LIFESPAN # in place of retirement_year: Decision\n\n# _storage_units.py, beside the generator path\n retirement_year=mapping.lifespan.retirement_year,\n# and the report derives the Decision where it records it:\n reporter.record(\n mapping.name,\n RETIREMENT_YEAR_COLUMN,\n derive_retirement_year(PlexosClass.BATTERY, mapping.name, mapping.lifespan),\n )\n```\n\nDelete `read_year` with the last caller. Delete the `Lifespan.build_year` property too, because no code reads it.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-03", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/sources/plexos_dated_properties.py", "line": 80, "severity": "major", "label": "suggestion", "claim": "The source stages every dated band of every property, but only a dated property holds a schedule.", "suggested_fix": "Keep the rows of a property that states at least one date, and drop the rest.\n\nThe code now:\n\n```python\ndef dated_rows(resolved: list[DatedRow]) -> Rows:\n return [\n {\n **dated.row,\n PlexosDatedPropertyCol.DATE_FROM: dated.dates.date_from,\n PlexosDatedPropertyCol.DATE_TO: dated.dates.date_to,\n }\n for dated in resolved\n ]\n```\n\nThe code to use:\n\n```python\ndef dated_rows(resolved: list[DatedRow]) -> Rows:\n \"\"\"Every row of a property the model dates, beside the dates it applies between.\"\"\"\n by_property: dict[tuple[Any, ...], list[DatedRow]] = {}\n for dated in resolved:\n key = (\n *(dated.row[column] for column in _MEMBERSHIP_NAME_COLUMNS),\n dated.row[PlexosPropertyCol.PROPERTY],\n )\n by_property.setdefault(key, []).append(dated)\n return [\n {\n **dated.row,\n PlexosDatedPropertyCol.DATE_FROM: dated.dates.date_from,\n PlexosDatedPropertyCol.DATE_TO: dated.dates.date_to,\n }\n for group in by_property.values()\n if any(dated.dates != UNDATED for dated in group)\n for dated in group\n ]\n```\n\nThe group holds the undated row beside the dated ones, so `_read_bands` still reads what an object runs before its first band opens.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-04", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/pypsa_constants.py", "line": 198, "severity": "minor", "label": "chore", "claim": "`PyPSALineCol.BUILD_YEAR` and `PyPSALinkCol.BUILD_YEAR` are new, but no code reads them.", "suggested_fix": "Delete both constants until a mapping writes the column.\n\n```python\nclass PyPSALineCol:\n ...\n TERRAIN_FACTOR = \"terrain_factor\"\n BUILD_YEAR = \"build_year\" # delete: no schema and no mapping name it\n S_NOM_MIN = \"s_nom_min\"\n```\n\nThe generator and the storage unit keep theirs, because `GENERATORS_DESTINATION_SCHEMA`, `STORAGE_UNITS_DESTINATION_SCHEMA` and `emit_pypsa_network` name them.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-05", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/core/extensions.py", "line": 264, "severity": "minor", "label": "issue", "claim": "The `sense` field has no default, so `ExtensionLookup.get` raises a ValidationError for a constraint name the sidecar does not hold.", "suggested_fix": "```python\nclass ConstraintExtension(ExtensionRecord):\n ...\n sense: ConstraintSense | None = None\n```\n`_carry` already refuses to build a record without a sense, so no writer emits `None`, and `read(ExtensionKind.CONSTRAINT).get(name)` keeps answering with a record.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-06", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 152, "severity": "minor", "label": "issue", "claim": "The code discards a retirement that comes before the build, so a unit that stops and restarts gets a wrong `build_year` and no `retirement_year`.", "suggested_fix": "```python\ndef _read_lifespan(bands: list[_UnitsBand]) -> Lifespan:\n changes = _changes(bands)\n retirement = next(\n (one for one in changes if _runs(one.was) and not _runs(one.units)), None\n )\n build = next(\n (\n one\n for one in changes\n if not _runs(one.was)\n and _runs(one.units)\n and (retirement is None or one.at < retirement.at)\n ),\n None,\n )\n return Lifespan(_milestone(build), _milestone(retirement))\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-07", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/pypsa_constants.py", "line": 198, "severity": "nit", "label": "chore", "claim": "The change adds `BUILD_YEAR` to `PyPSALineCol` and to `PyPSALinkCol`, but no module reads either constant.", "suggested_fix": "```python\nclass PyPSALineCol:\n ...\n TERRAIN_FACTOR = \"terrain_factor\"\n S_NOM_MIN = \"s_nom_min\"\n```\nRemove the same line from `PyPSALinkCol` (line 226). Only `PyPSAGeneratorCol.BUILD_YEAR` and `PyPSAStorageUnitCol.BUILD_YEAR` reach a destination schema and the sink.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-08", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 84, "severity": "minor", "label": "suggestion", "claim": "Four docstrings here only paraphrase the function name. The standard deletes a docstring that repeats the name.", "suggested_fix": "Delete all four. The names already say it, and the derivation strings above say the rest.\n\nThe code now:\n\n```python\ndef read_lifespans(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, Lifespan]:\n \"\"\"What the dated ``Units`` say about each object of a class that dates them.\"\"\"\n return {name: _read_lifespan(bands) for name, bands in _read_bands(dated, plexos_class).items()}\n\n\ndef derive_build_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision:\n \"\"\"The year the schedule brings the object into service.\"\"\"\n return _derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION)\n\n\ndef derive_retirement_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision:\n \"\"\"The year the schedule takes the object out of service, which the sidecar carries.\"\"\"\n return _derive_year(plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION)\n\n\ndef _derive_year(\n plexos_class: PlexosClass, name: str, milestone: Milestone | None, derivation: str\n) -> Decision:\n if milestone is None:\n return Decision.unreported(None)\n source = SourceValue(plexos_class, name, PlexosProperty.UNITS, milestone.units)\n return Decision.derived(milestone.year, [source], derivation)\n\n\ndef read_year(decision: Decision) -> int | None:\n \"\"\"The year a lifespan decision states, for the sidecar field that carries it.\"\"\"\n return None if decision.value is None else int(decision.value)\n```\n\nThe code to use:\n\n```python\ndef read_lifespans(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, Lifespan]:\n return {name: _read_lifespan(bands) for name, bands in _read_bands(dated, plexos_class).items()}\n\n\ndef derive_build_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision:\n return _derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION)\n\n\ndef derive_retirement_year(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> Decision:\n return _derive_year(plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION)\n\n\ndef _derive_year(\n plexos_class: PlexosClass, name: str, milestone: Milestone | None, derivation: str\n) -> Decision:\n if milestone is None:\n return Decision.unreported(None)\n source = SourceValue(plexos_class, name, PlexosProperty.UNITS, milestone.units)\n return Decision.derived(milestone.year, [source], derivation)\n\n\ndef read_year(decision: Decision) -> int | None:\n return None if decision.value is None else int(decision.value)\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-09", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_dates.py", "line": 3, "severity": "minor", "label": "suggestion", "claim": "This paragraph argues why the module exists. The standard deletes a reviewer justification.", "suggested_fix": "Delete the paragraph and keep the summary line. Its first sentence also stands word for word in the ``plexos_dated_properties`` module docstring.\n\nThe code now:\n\n```python\n\"\"\"When a dated PLEXOS value applies.\n\nPLEXOS stamps a ``t_data`` row with the dates it applies between. The source narrows those\nbands to the window being translated, and a mapping reading a schedule out of them -- the\nyear a unit arrives, the year it goes -- reads the same bands as the model states them, so\nboth sides share one reading of what a band covers.\n\"\"\"\n```\n\nThe code to use:\n\n```python\n\"\"\"When a dated PLEXOS value applies.\"\"\"\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-10", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 215, "severity": "minor", "label": "suggestion", "claim": "This paragraph narrates the test on the next line. The standard deletes a comment that explains the code below it.", "suggested_fix": "Delete the second paragraph and keep the summary line, which states the contract of the return value.\n\nThe code now:\n\n```python\ndef _carry(constraint: _Constraint) -> ConstraintExtension | None:\n \"\"\"The sidecar record, or None where the constraint states too little to hold anything to.\n\n A limit needs both a sense and a right-hand side: without either there is no inequality\n to state, whatever the objects it names.\n \"\"\"\n if constraint.sense is None or not constraint.right_hand_sides:\n return None\n```\n\nThe code to use:\n\n```python\ndef _carry(constraint: _Constraint) -> ConstraintExtension | None:\n \"\"\"The sidecar record, or None where the constraint states too little to hold anything to.\"\"\"\n if constraint.sense is None or not constraint.right_hand_sides:\n return None\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-11", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/core/extensions.py", "line": 258, "severity": "minor", "label": "suggestion", "claim": "This paragraph repeats the ExtensionKind.CONSTRAINT comment in the same file. The standard deletes a duplicate.", "suggested_fix": "Delete the paragraph and keep the summary line. The same argument now stands in four new places: this docstring, the ``ExtensionKind.CONSTRAINT`` comment at line 50, the ``_constraints`` module docstring and ``_relay_constraints``.\n\nThe code now:\n\n```python\nclass ConstraintExtension(ExtensionRecord):\n \"\"\"A weighted sum over named objects, held to one or more right-hand sides.\n\n PLEXOS has the concept and PyPSA has none: a GlobalConstraint limits one carrier over\n the whole horizon and cannot name a set of components. A constraint therefore travels\n here rather than in the network, so a later hop into a framework that can express it\n still has the limit.\n \"\"\"\n```\n\nThe code to use:\n\n```python\nclass ConstraintExtension(ExtensionRecord):\n \"\"\"A weighted sum over named objects, held to one or more right-hand sides.\"\"\"\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-12", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_constants.py", "line": 105, "severity": "minor", "label": "nitpick", "claim": "This sentence copies the DateBand.ends docstring in plexos_dates.py. The standard deletes a duplicate that will drift.", "suggested_fix": "Delete the sentence. ``DateBand.ends`` states the same rule where the code acts on it.\n\nThe code now:\n\n```python\nclass PlexosDatedPropertyCol(PlexosPropertyCol):\n \"\"\"Columns of the resolved ``dated_properties`` table.\n\n The ``properties`` columns, plus the dates the value on the row applies between. Every\n resolved row is here, whether or not the window being translated covers its band, so a\n mapping reading a schedule sees the years the model states rather than one value.\n ``date_to`` names a whole day, so the band runs to the end of it.\n \"\"\"\n```\n\nThe code to use:\n\n```python\nclass PlexosDatedPropertyCol(PlexosPropertyCol):\n \"\"\"Columns of the resolved ``dated_properties`` table.\n\n The ``properties`` columns, plus the dates the value on the row applies between. Every\n resolved row is here, whether or not the window being translated covers its band, so a\n mapping reading a schedule sees the years the model states rather than one value.\n \"\"\"\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-13", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/constants.py", "line": 27, "severity": "nit", "label": "nitpick", "claim": "This docstring is the third identical copy in eight lines. The standard deletes a duplicate.", "suggested_fix": "Give the three sidecar-key constants one comment above the group and drop the repeated docstrings.\n\nThe code now:\n\n```python\nEXT_UNIT_SIZE_FIELD: str = \"extensions.unit_size_mw\"\n\"\"\"Names that sidecar key in the audit trail; the network file itself has no such column.\"\"\"\n\nEXT_TECHNICAL_LIFE_FIELD: str = \"extensions.technical_life_years\"\n\"\"\"Names that sidecar key in the audit trail; the network file itself has no such column.\"\"\"\n\nEXT_RETIREMENT_YEAR_FIELD: str = \"extensions.retirement_year\"\n\"\"\"Names that sidecar key in the audit trail; the network file itself has no such column.\"\"\"\n```\n\nThe code to use:\n\n```python\n# Each names a sidecar key in the audit trail; the network file itself has no such column.\nEXT_UNIT_SIZE_FIELD: str = \"extensions.unit_size_mw\"\nEXT_TECHNICAL_LIFE_FIELD: str = \"extensions.technical_life_years\"\nEXT_RETIREMENT_YEAR_FIELD: str = \"extensions.retirement_year\"\n```", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-14", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 56, "severity": "minor", "label": "chore", "claim": "The property `Lifespan.build_year` has no caller.", "suggested_fix": "Delete the `build_year` property from `Lifespan`. The build year reaches the network through `derive_build_year`, which reads `lifespan.build` itself. Net: -4 lines.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-15", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/pypsa_constants.py", "line": 198, "severity": "nit", "label": "chore", "claim": "The branch adds `BUILD_YEAR` to `PyPSALineCol` and `PyPSALinkCol`, and no code reads either name.", "suggested_fix": "Delete `BUILD_YEAR` from `PyPSALineCol` (line 198) and from `PyPSALinkCol` (line 226), and add each one back with the mapping that first writes a build year onto a line or a link. Net: -2 lines.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-16", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 107, "severity": "nit", "label": "suggestion", "claim": "The one-line helper `read_year` has one caller, and the other sidecar field takes the same year by a second route.", "suggested_fix": "Use one route for both. In `_generators._carry_to_extensions`, hold the `derive_retirement_year` decision it already builds for the report, give it to `read_year` for `GeneratorExtension.retirement_year`, and delete the `Lifespan.retirement_year` property. Net: -4 lines.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r1-17", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 1, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 247, "severity": "minor", "label": "issue", "claim": "`_NOT_CARRIED_NOTE` states that PyPSA cannot express the limit, but the code now applies it to a Constraint that is left out because the source states no sense or no right-hand side.", "suggested_fix": "Add a note that names the missing part, and select it where `_carry` returns None because `constraint.sense` is None or `constraint.right_hand_sides` is empty. Net: +6 lines.", "rule_created": false, "ts": "2026-09-08T22:05:22Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-01", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 85, "severity": "major", "label": "suggestion", "claim": "The build year and the retirement year are two ends of one concept, but four call sites derive them one end at a time.", "suggested_fix": "Bundle the two ends as `_expansion.py` bundles the expansion fields, and let each mapping hold the bundle.\n\nThe code now:\n\n```python\n# _generator_decisions.py\nbuild_year=derive_build_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan),\n# _generators.py\nretirements = {\n mapping.name: derive_retirement_year(PlexosClass.GENERATOR, mapping.name, mapping.lifespan)\n for mapping in mappings\n}\n# _batteries.py\nbuild_year=derive_build_year(PlexosClass.BATTERY, rated.name, rated.lifespan),\nretirement_year=derive_retirement_year(PlexosClass.BATTERY, rated.name, rated.lifespan),\n```\n\nThe code to use:\n\n```python\n# _lifespan.py\n@dataclass(frozen=True)\nclass LifespanDecisions:\n build_year: Decision = maps_to(PyPSAGeneratorCol.BUILD_YEAR)\n # The sidecar carries the retirement year, since PyPSA has no column for it.\n retirement_year: Decision = NOTHING_TO_REPORT\n\n\ndef derive_lifespan(plexos_class: PlexosClass, name: str, lifespan: Lifespan) -> LifespanDecisions:\n return LifespanDecisions(\n build_year=_derive_year(plexos_class, name, lifespan.build, _BUILD_DERIVATION),\n retirement_year=_derive_year(plexos_class, name, lifespan.retirement, _RETIREMENT_DERIVATION),\n )\n\n\ndef record_lifespan(name: str, decisions: LifespanDecisions, reporter: ComponentReporter) -> None:\n reporter.record(name, RETIREMENT_YEAR_COLUMN, decisions.retirement_year)\n\n# each mapping then declares one field\nlifespan: LifespanDecisions = holds()\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-02", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/sources/plexos_dated_properties.py", "line": 132, "severity": "major", "label": "issue", "claim": "The `outside` fallback can never run, because an undated band covers every moment and `latest_covering` therefore always finds it.", "suggested_fix": "Delete `_stated_for_no_date` and the `outside` parameter.\n\nThe code now:\n\n```python\ndef _stated_for_no_date(ordered: list[DatedRow]) -> dict[str, Any] | None:\n return next((dated.row for dated in ordered if dated.dates == UNDATED), None)\n\n\ndef _value_at(\n ordered: list[DatedRow], outside: dict[str, Any] | None, moment: datetime\n) -> float | None:\n stating = latest_covering(ordered, moment)\n if stating is None:\n stating = outside\n if stating is None:\n return _NOT_IN_EFFECT\n value: float | None = stating[PlexosPropertyCol.VALUE]\n return value\n```\n\nThe code to use:\n\n```python\ndef _value_at(ordered: list[DatedRow], moment: datetime) -> float | None:\n \"\"\"The latest band covering the moment, else none.\n\n A property stated only for a period is not in effect outside one, and a property with\n no value in effect is a property the model is not applying: it reads as zero.\n \"\"\"\n stating = latest_covering(ordered, moment)\n if stating is None:\n return _NOT_IN_EFFECT\n value: float | None = stating[PlexosPropertyCol.VALUE]\n return value\n```\n\n`_steps_within` then drops its `outside` local and calls `_value_at(ordered, moment)`.", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-03", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 110, "severity": "minor", "label": "suggestion", "claim": "The `_Outcome` dataclass and the optional return of `_carry` both state one predicate, which the `_Constraint` itself can answer.", "suggested_fix": "Ask the constraint whether it holds an inequality, and let `_carry` return a record.\n\nThe code now:\n\n```python\n@dataclass(frozen=True)\nclass _Outcome:\n constraint: _Constraint\n record: ConstraintExtension | None\n\n\ndef map_constraints(state: State, recorder: ScopedRecorder) -> None:\n constraints = _read_constraints(state)\n if not constraints:\n return\n outcomes = [_Outcome(constraint, _carry(constraint)) for constraint in constraints]\n reporter = SourceReporter(recorder)\n for outcome in outcomes:\n _record(reporter, outcome)\n append_extensions(\n state.destination_extensions,\n ExtensionKind.CONSTRAINT,\n [outcome.record for outcome in outcomes if outcome.record is not None],\n )\n _warn(constraints)\n```\n\nThe code to use:\n\n```python\n# on _Constraint\n@property\ndef holds_an_inequality(self) -> bool:\n return self.sense is not None and bool(self.right_hand_sides)\n\n\ndef map_constraints(state: State, recorder: ScopedRecorder) -> None:\n constraints = _read_constraints(state)\n if not constraints:\n return\n reporter = SourceReporter(recorder)\n for constraint in constraints:\n _record(reporter, constraint)\n append_extensions(\n state.destination_extensions,\n ExtensionKind.CONSTRAINT,\n [_carry(one) for one in constraints if one.holds_an_inequality],\n )\n _warn(constraints)\n```\n\n`_record` then reads `constraint.holds_an_inequality` to choose between `_CARRIED_NOTE` and `_NOT_CARRIED_NOTE`.", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-04", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 141, "severity": "minor", "label": "question", "claim": "The end of a dated Units band reads as a retirement, so a model that states no zero still gets a retirement_year.", "suggested_fix": "```python\ndef _read_lifespan(bands: list[_UnitsBand]) -> Lifespan:\n changes = _changes(bands)\n build = next((one for one in changes if not _runs(one.was) and _runs(one.units)), None)\n retirement = next(\n (\n one\n for one in changes\n if _runs(one.was)\n and not _runs(one.units)\n # A band must state the zero; a schedule that simply stops states no retirement.\n and latest_covering(bands, one.at) is not None\n and (build is None or one.at > build.at)\n ),\n None,\n )\n return Lifespan(_milestone(build), _milestone(retirement))\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-05", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 204, "severity": "minor", "label": "issue", "claim": "_read_sense turns a Sense code it cannot map into None, so the report tells the user the Constraint states no sense.", "suggested_fix": "```python\n_UNREADABLE_SENSE_NOTE = (\n \"this Constraint states a Sense that is not -1, 0 or 1, so the translator cannot read \"\n \"the inequality it holds in and carries nothing to the extensions sidecar\"\n)\n\n\ndef _not_carried_note(constraint: _Constraint) -> str:\n if constraint.sense is None and constraint.stated_sense is not None:\n return _UNREADABLE_SENSE_NOTE\n return _NOT_CARRIED_NOTE\n```\nKeep the stated code on `_Constraint` (`stated_sense=stated.get(PlexosProperty.SENSE)`) and pick the note in `_record`.", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-06", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/core/extensions.py", "line": 98, "severity": "minor", "label": "suggestion", "claim": "This paragraph repeats the PLEXOS property that each enum member below already names in its own comment.", "suggested_fix": "Delete the second paragraph. The per-member comments carry the same mapping.\n\nThe code now:\n\n```python\nclass ConstraintPeriod(StrEnum):\n \"\"\"The span one right-hand side applies over.\n\n PLEXOS states a limit per hour, day, week, month or year, or over the whole horizon,\n and spells each as a right-hand side property of its own.\n \"\"\"\n\n HORIZON = \"horizon\" # PLEXOS RHS\n HOUR = \"hour\" # PLEXOS RHS Hour\n DAY = \"day\" # PLEXOS RHS Day\n WEEK = \"week\" # PLEXOS RHS Week\n MONTH = \"month\" # PLEXOS RHS Month\n YEAR = \"year\" # PLEXOS RHS Year\n```\n\nThe code to use:\n\n```python\nclass ConstraintPeriod(StrEnum):\n \"\"\"The span one right-hand side applies over.\"\"\"\n\n HORIZON = \"horizon\" # PLEXOS RHS\n HOUR = \"hour\" # PLEXOS RHS Hour\n DAY = \"day\" # PLEXOS RHS Day\n WEEK = \"week\" # PLEXOS RHS Week\n MONTH = \"month\" # PLEXOS RHS Month\n YEAR = \"year\" # PLEXOS RHS Year\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-07", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/pypsa_constants.py", "line": 492, "severity": "nit", "label": "nitpick", "claim": "This comment paraphrases the column name build_year, and the storage unit schema repeats it.", "suggested_fix": "Delete both comments. Neighbouring comments in these dicts earn their place because they state what the sink does with a null column; these two only restate the name.\n\nThe code now:\n\n```python\n PyPSAGeneratorCol.FOM_COST: pl.Float64,\n # The year the source brings the generator into service, null where it states none.\n PyPSAGeneratorCol.BUILD_YEAR: pl.Int64,\n}\n```\n\nand at line 531:\n\n```python\n PyPSAStorageUnitCol.FOM_COST: pl.Float64,\n # The year the source brings the storage unit into service, null where it states none.\n PyPSAStorageUnitCol.BUILD_YEAR: pl.Int64,\n}\n```\n\nThe code to use:\n\n```python\n PyPSAGeneratorCol.FOM_COST: pl.Float64,\n PyPSAGeneratorCol.BUILD_YEAR: pl.Int64,\n}\n```\n\nand:\n\n```python\n PyPSAStorageUnitCol.FOM_COST: pl.Float64,\n PyPSAStorageUnitCol.BUILD_YEAR: pl.Int64,\n}\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-08", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 136, "severity": "nit", "label": "nitpick", "claim": "This docstring states a circular sentence and does not say what the function returns.", "suggested_fix": "Delete it. The name and the one-line body already say that the function looks for a dated band.\n\nThe code now:\n\n```python\ndef _states_a_date(bands: list[_UnitsBand]) -> bool:\n \"\"\"An object whose ``Units`` are one value for all time keeps that value for all time.\"\"\"\n return any(band.dates != UNDATED for band in bands)\n```\n\nThe code to use:\n\n```python\ndef _states_a_date(bands: list[_UnitsBand]) -> bool:\n return any(band.dates != UNDATED for band in bands)\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-09", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_dates.py", "line": 33, "severity": "nit", "label": "nitpick", "claim": "This docstring paraphrases the function name and the return type.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\ndef latest_covering(bands: Sequence[tuple[DateBand, T]], moment: datetime) -> T | None:\n \"\"\"What the last band covering the moment states, else None.\"\"\"\n covering = [value for band, value in bands if band.covers(moment)]\n return covering[-1] if covering else None\n```\n\nThe code to use:\n\n```python\ndef latest_covering(bands: Sequence[tuple[DateBand, T]], moment: datetime) -> T | None:\n covering = [value for band, value in bands if band.covers(moment)]\n return covering[-1] if covering else None\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-10", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 74, "severity": "nit", "label": "nitpick", "claim": "This comment paraphrases the dictionary below it, which pairs each right-hand side property with a span.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\n# The right-hand side properties, each with the span it holds the sum over.\n_PERIODS: dict[str, ConstraintPeriod] = {\n PlexosProperty.RHS: ConstraintPeriod.HORIZON,\n PlexosProperty.RHS_HOUR: ConstraintPeriod.HOUR,\n```\n\nThe code to use:\n\n```python\n_PERIODS: dict[str, ConstraintPeriod] = {\n PlexosProperty.RHS: ConstraintPeriod.HORIZON,\n PlexosProperty.RHS_HOUR: ConstraintPeriod.HOUR,\n```", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-11", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "interop/plugins/steps/plexos_to_pypsa/map_constraints.py", "line": 15, "severity": "major", "label": "issue", "claim": "The class docstring says the step carries every PLEXOS Constraint to the sidecar, but a Constraint that states no sense, or no right-hand side, is left out.", "suggested_fix": "Change the docstring to \"Carries each PLEXOS Constraint the translator can read to the extensions sidecar; PyPSA enforces none.\", which is what the log warning already says. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r2-12", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 2, "file": "tests/features/plexos_to_pypsa/constraints.feature", "line": 103, "severity": "minor", "label": "suggestion", "claim": "The new scenario \"a readable Constraint reaches the sidecar...\" repeats the whole model of the first scenario, down to the constraint name, the sense and the right-hand side.", "suggested_fix": "Delete the new scenario, add `And constraint \"RiverSystem\" states \"Include in LT Plan\" of 1` and the AA2 coefficient of 2.5 to the first scenario, and move the eight `extensions.json` assertions onto it. No assertion is lost, and one full translate run goes with it. Net: -20 lines.", "rule_created": false, "ts": "2026-09-08T22:30:23Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-01", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py", "line": 200, "severity": "major", "label": "suggestion", "claim": "The class `StorageLookups` states five readings twice, once for a Battery and once for a Generator, and this change adds the fifth pair.", "suggested_fix": "The code now:\n\n```python\n lifespan_by_battery: dict[str, Lifespan]\n lifespan_by_generator: dict[str, Lifespan]\n\n def battery(self, name: str) -> StagedObject:\n return StagedObject(\n name=name,\n properties=self.battery_properties.get(name, {}),\n stated_units=self.battery_units.get(name, {}),\n node=self.node_by_battery.get(name),\n file_backed=self.file_backed_by_battery.get(name, []),\n lifespan=self.lifespan_by_battery.get(name, NO_LIFESPAN),\n )\n```\n\n`generator` holds the same body, over the five `_by_generator` fields.\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass _ClassLookups:\n \"\"\"What one PLEXOS class states about each of its objects.\"\"\"\n\n properties: ObjectProperties\n stated_units: ObjectUnits\n nodes: dict[str, str]\n file_backed: dict[str, list[str]]\n lifespans: dict[str, Lifespan]\n\n\n@dataclass(frozen=True)\nclass StorageLookups:\n by_class: dict[PlexosClass, _ClassLookups]\n\n def staged(self, plexos_class: PlexosClass, name: str) -> StagedObject:\n one = self.by_class[plexos_class]\n return StagedObject(\n name=name,\n properties=one.properties.get(name, {}),\n stated_units=one.stated_units.get(name, {}),\n node=one.nodes.get(name),\n file_backed=one.file_backed.get(name, []),\n lifespan=one.lifespans.get(name, NO_LIFESPAN),\n )\n```\n\nThe two call sites become `lookups.staged(PlexosClass.BATTERY, name)` in `_batteries.py`\nand `lookups.staged(PlexosClass.GENERATOR, name)` in `_storage_hydro.py`.", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-02", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 116, "severity": "minor", "label": "suggestion", "claim": "The class `_Outcome` and the optional return of `_carry` both state one fact, which the `_Constraint` can state itself.", "suggested_fix": "The code now:\n\n```python\n@dataclass(frozen=True)\nclass _Outcome:\n \"\"\"One Constraint and the record it travels as, which is None where it states too little.\"\"\"\n\n constraint: _Constraint\n record: ConstraintExtension | None\n\n\ndef map_constraints(state: State, recorder: ScopedRecorder) -> None:\n constraints = _read_constraints(state)\n if not constraints:\n return\n outcomes = [_Outcome(constraint, _carry(constraint)) for constraint in constraints]\n reporter = SourceReporter(recorder)\n for outcome in outcomes:\n _record(reporter, outcome)\n append_extensions(\n state.destination_extensions,\n ExtensionKind.CONSTRAINT,\n [outcome.record for outcome in outcomes if outcome.record is not None],\n )\n _warn(constraints)\n```\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass _Constraint:\n # the fields stay as they are\n\n @property\n def is_carried(self) -> bool:\n return self.sense is not None and bool(self.right_hand_sides)\n\n\ndef map_constraints(state: State, recorder: ScopedRecorder) -> None:\n constraints = _read_constraints(state)\n if not constraints:\n return\n reporter = SourceReporter(recorder)\n for constraint in constraints:\n _record(reporter, constraint)\n append_extensions(\n state.destination_extensions,\n ExtensionKind.CONSTRAINT,\n [_carry(one) for one in constraints if one.is_carried],\n )\n _warn(constraints)\n```\n\n`_carry` then returns a `ConstraintExtension`, and `_record` takes a `_Constraint` and\nreads `constraint.is_carried`.", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-03", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "docs/translation_mappings/translation-from-plexos-to-pypsa.md", "line": 777, "severity": "minor", "label": "issue", "claim": "The document says the translator leaves out a component that runs no units, but the translator keeps a candidate with zero units.", "suggested_fix": "```markdown\n| `Units` | ... A component that runs no units in the year translated is still left out, since it can dispatch nothing that year, unless it is a [candidate](#what-a-candidate-is), which the translator writes as extendable. |\n```", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-04", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/sources/plexos_dated_properties.py", "line": 132, "severity": "minor", "label": "suggestion", "claim": "This first line paraphrases the name, and it is wrong: no covering band gives zero, not none.", "suggested_fix": "Delete the first line. The paragraph below it already gives the reader the rule, and gives it correctly.\n\nThe code now:\n\n```python\ndef _value_at(ordered: list[DatedRow], moment: datetime) -> float | None:\n \"\"\"The latest band covering the moment, else none.\n\n A property stated only for a period is not in effect outside one, and a property with\n no value in effect is a property the model is not applying: it reads as zero.\n \"\"\"\n stating = latest_covering(ordered, moment)\n```\n\nThe code to use:\n\n```python\ndef _value_at(ordered: list[DatedRow], moment: datetime) -> float | None:\n \"\"\"A property stated only for a period is not in effect outside one, and a property\n with no value in effect is a property the model is not applying: it reads as zero.\n \"\"\"\n stating = latest_covering(ordered, moment)\n```", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-05", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 79, "severity": "nit", "label": "suggestion", "claim": "This docstring paraphrases the two field names and adds nothing to them.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\nclass _UnitsBand(NamedTuple):\n \"\"\"How many units one dated ``Units`` row runs, and the dates it runs them between.\"\"\"\n\n dates: DateBand\n units: float\n```\n\nThe code to use:\n\n```python\nclass _UnitsBand(NamedTuple):\n dates: DateBand\n units: float\n```", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-06", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_constraints.py", "line": 117, "severity": "minor", "label": "suggestion", "claim": "This docstring repeats what the _carry docstring says, and it restates the None in the type.", "suggested_fix": "Delete it.\n\nThe code now:\n\n```python\n@dataclass(frozen=True)\nclass _Outcome:\n \"\"\"One Constraint and the record it travels as, which is None where it states too little.\"\"\"\n\n constraint: _Constraint\n record: ConstraintExtension | None\n```\n\nThe code to use:\n\n```python\n@dataclass(frozen=True)\nclass _Outcome:\n constraint: _Constraint\n record: ConstraintExtension | None\n```", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-07", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "docs/translation_mappings/translation-from-plexos-to-pypsa.md", "line": 777, "severity": "major", "label": "issue", "claim": "The new sentence says a component that runs no units in the translated year is always left out, but a candidate that runs no units is written with a `build_year`.", "suggested_fix": "Delete the sentence \"A component running no units in the year translated is still left out, since it can dispatch nothing that year.\" from the `Units` row. The skipped-component table at line 289 already states the real rule, which is that the generator goes only when it has no `Units` at any time in the horizon **and** no `Max Units Built`. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "accepted", "reason": "it blocks the merge", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-08", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/shared/plexos_pypsa_translations/_lifespan.py", "line": 152, "severity": "minor", "label": "suggestion", "claim": "The guard `_states_a_date` cannot change any result, because a set of bands that are all undated already gives no milestone.", "suggested_fix": "Delete `_states_a_date`, make the return of `_read_bands` `return {name: sorted(rows, key=opens_at) for name, rows in bands.items()}`, and drop the now unused `UNDATED` import. Net: -7 lines.", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "accepted", "reason": "it takes lines away", "category": null, "promote_now": false, "suppressed_by": null} +{"id": "r3-09", "pr": "10", "repo": "transition-zero/tz-oss-interop", "round": 3, "file": "interop/plugins/sources/plexos_dated_properties.py", "line": 86, "severity": "nit", "label": "nitpick", "claim": "The new function `dated_rows` has the same name as a local variable in `apply_window` in the same module.", "suggested_fix": "Rename the loop variable at lines 78-79 of `apply_window` to `rows`, so the module has one meaning for `dated_rows`. Net: 0 lines.", "rule_created": false, "ts": "2026-09-08T22:56:48Z", "source": "reviewer", "verdict": "deferred", "reason": "an automated run acts only on a blocking comment or one that removes lines", "category": null, "promote_now": false, "suppressed_by": null} From f982f34b8a77fca2bce03a9bfb871c0d4c82bdb0 Mon Sep 17 00:00:00 2001 From: "Thomas C. King" Date: Thu, 10 Sep 2026 19:42:21 +0000 Subject: [PATCH 17/17] Read a mothball as a mothball, and drop the filter the merge duplicated Round 1 of the automatic review cycle. Addresses r1-01, r1-02, r1-03, r1-04, r1-05, r1-06, r1-08 and r1-10. r1-01: the merge with the parent branch left the unbuilt-candidate rule in two places. _rows_in_scope already applies it to the same rows, so the copy in _prepare_source dropped nothing and cost one Polars pass per mapping. r1-02: a schedule whose Units return above zero after a dated zero band read as a build in the return year. A generator running two units with a dated zero from 2035 to 2040 reported build_year 2041, although it runs in the year translated. An object that runs before its first dated change now states no build year, and one that runs again after a zero band states no retirement year. A new scenario covers it, and the retirement label says "the last year" because the rule now takes the last fall to zero. r1-03: the Units row of the mapping document named a shape the translator drops. A generator that runs no units and states no Max Units Built is left out as retired, so no build_year reaches the network for it. r1-04: GeneratorExtension and StorageExtension declared unit_size_mw and technical_life_years again, which they inherit from ExpansionExtension. retirement_year moves to the base class, where both records need it. r1-05: _expansion.py holds the sidecar keys as constants now, rather than as literals beside the constants that spell the same three strings. r1-06: StorageLookups answers properties_of(plexos_class), so a caller stops reaching through by_class. The docstring says that by_class holds the Battery and the Generator alone. r1-08 and r1-10: two docstrings that paraphrase the name they sit on go. Co-Authored-By: Claude Opus 5 (1M context) --- .../translation-from-plexos-to-pypsa.md | 2 +- interop/core/extensions.py | 21 ++---------- .../plexos_pypsa_translations/_constraints.py | 1 - .../plexos_pypsa_translations/_expansion.py | 9 ++++-- .../plexos_pypsa_translations/_lifespan.py | 32 +++++++++++-------- .../_storage_shared.py | 9 +++++- .../_storage_units.py | 2 +- .../plexos_pypsa_translations/constants.py | 1 + .../steps/pypsa_to_sienna_map_components.py | 2 -- .../plexos_to_pypsa/dated_properties.feature | 18 ++++++++++- 10 files changed, 55 insertions(+), 42 deletions(-) diff --git a/docs/translation_mappings/translation-from-plexos-to-pypsa.md b/docs/translation_mappings/translation-from-plexos-to-pypsa.md index a4ae5b8..6fe8cfc 100644 --- a/docs/translation_mappings/translation-from-plexos-to-pypsa.md +++ b/docs/translation_mappings/translation-from-plexos-to-pypsa.md @@ -779,7 +779,7 @@ Two properties use dated entries as a schedule. They do not use them as correcti | Property | Meaning | | --- | --- | -| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. The translator reads both years off the entries as the model dates them, whatever year it translates: the first year the value rises above zero is the `build_year` of the component it writes, and the first year after that in which the value falls back to zero is the `retirement_year` in the extensions sidecar. | +| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. The translator reads both years off the entries as the model dates them, whatever year it translates: the first year the value rises above zero is the `build_year` of the component it writes, and the last year in which it falls back to zero is the `retirement_year` in the extensions sidecar. An object that already runs before its first dated entry states no `build_year`, and one that runs again after an entry of zero states no `retirement_year`, because that entry is a mothball rather than a retirement. A generator that runs no units in the year being translated and states no `Max Units Built` is left out as retired, so no `build_year` reaches the network for it. | | `Max Capacity` | A capacity expansion schedule. The translator applies the entry that is in force at the snapshot. Where it needs one value, it uses the entry that is in force at the start of the model. | ### Timeslice patterns diff --git a/interop/core/extensions.py b/interop/core/extensions.py index 063e47f..077a0c9 100644 --- a/interop/core/extensions.py +++ b/interop/core/extensions.py @@ -132,6 +132,9 @@ class ExpansionExtension(ExtensionRecord): # $/MW/yr. PyPSA's fom_cost is a charge for the whole modelled horizon, not a yearly # one, so a yearly charge has no field there. fom_charge_per_mw_year: float | None = None + # The year the object leaves service, which PLEXOS states as a dated Units of zero. + # PyPSA carries build_year on the component and nothing for the other end of its life. + retirement_year: int | None = None class GeneratorExtension(ExpansionExtension): @@ -144,15 +147,6 @@ class GeneratorExtension(ExpansionExtension): p_nom_extendable: bool | None = None # PLEXOS only: the generator's category, a grouping string the user chooses. category: str | None = None - # MW. PLEXOS Max Capacity: what one unit of a candidate is. PyPSA sizes a candidate by - # p_nom_max alone, so the size of a single unit has no field there. - unit_size_mw: float | None = None - # yr. PLEXOS Technical Life: how long the plant runs. PyPSA has one lifetime and the - # capital recovery period claims it, so this has no field there. - technical_life_years: float | None = None - # The year the plant leaves service, which PLEXOS states as a dated Units of zero. PyPSA - # carries build_year on the component and nothing for the other end of its life. - retirement_year: int | None = None class LoadExtension(ExtensionRecord): @@ -191,15 +185,6 @@ class ControllableLineExtension(ExtensionRecord): class StorageExtension(ExpansionExtension): p_nom_extendable: bool | None = None # PyPSA only - # MW. PLEXOS Max Power: what one unit of a candidate is. PyPSA sizes a candidate by - # p_nom_max alone, so the size of a single unit has no field there. - unit_size_mw: float | None = None - # yr. PLEXOS Technical Life: how long the plant runs. PyPSA has one lifetime and the - # capital recovery period claims it, so this has no field there. - technical_life_years: float | None = None - # The year the unit leaves service, which PLEXOS states as a dated Units of zero. PyPSA - # carries build_year on the component and nothing for the other end of its life. - retirement_year: int | None = None class ReserveExtension(ExtensionRecord): diff --git a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py index c02929b..55947e4 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_constraints.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_constraints.py @@ -261,7 +261,6 @@ def _source( def _not_carried_note(constraint: _Constraint) -> str: - """Why the Constraint holds no inequality: a sense it lacks, or one nothing reads.""" if constraint.sense is None and constraint.stated_sense is not None: return _UNREADABLE_SENSE_NOTE return _NOT_CARRIED_NOTE diff --git a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py index 3bf986a..b978032 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_expansion.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_expansion.py @@ -16,6 +16,9 @@ from interop.plugins.shared.plexos_pypsa_translations.constants import ( DEFAULT_UNITS, DIRECT_DERIVATION, + EXT_FOM_CHARGE_FIELD, + EXT_TECHNICAL_LIFE_FIELD, + EXT_UNIT_SIZE_FIELD, NOTHING_TO_BUILD, ) from interop.plugins.shared.plexos_pypsa_translations.decisions import ( @@ -71,9 +74,9 @@ "capacity is fixed" ) -UNIT_SIZE_COLUMN = MappedColumns(("extensions.unit_size_mw",), UNIT_MW) -TECHNICAL_LIFE_COLUMN = MappedColumns(("extensions.technical_life_years",), UNIT_YEARS) -FOM_CHARGE_COLUMN = MappedColumns(("extensions.fom_charge_per_mw_year",), UNIT_DOLLARS_PER_MW_YEAR) +UNIT_SIZE_COLUMN = MappedColumns((EXT_UNIT_SIZE_FIELD,), UNIT_MW) +TECHNICAL_LIFE_COLUMN = MappedColumns((EXT_TECHNICAL_LIFE_FIELD,), UNIT_YEARS) +FOM_CHARGE_COLUMN = MappedColumns((EXT_FOM_CHARGE_FIELD,), UNIT_DOLLARS_PER_MW_YEAR) @dataclass(frozen=True) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py index 19e2146..0a68d98 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_lifespan.py @@ -44,7 +44,7 @@ RETIREMENT_YEAR_COLUMN = MappedColumns((EXT_RETIREMENT_YEAR_FIELD,)) _BUILD_DERIVATION = "the first year the dated Units rise above zero" -_RETIREMENT_DERIVATION = "the first year the dated Units fall back to zero" +_RETIREMENT_DERIVATION = "the last year the dated Units fall back to zero" class Milestone(NamedTuple): @@ -55,10 +55,7 @@ class Milestone(NamedTuple): class Lifespan(NamedTuple): - """When a dated ``Units`` schedule brings an object in, and when it takes it out. - - Either end can be absent: a schedule may only build, only retire, or say neither. - """ + """When a dated ``Units`` schedule brings an object in, and when it takes it out.""" build: Milestone | None retirement: Milestone | None @@ -145,19 +142,26 @@ def _read_bands(dated: pl.LazyFrame, plexos_class: PlexosClass) -> dict[str, lis def _read_lifespan(bands: list[_UnitsBand]) -> Lifespan: + """An object running before its first dated change was built before the model starts, + and one running again after a zero band was mothballed rather than retired, so a + schedule that runs at both ends of itself states neither year. + """ changes = _changes(bands) - build = next((one for one in changes if not _runs(one.was) and _runs(one.units)), None) - retirement = next( - ( - one - for one in changes - if _runs(one.was) and not _runs(one.units) and (build is None or one.at > build.at) - ), - None, - ) + if not changes: + return NO_LIFESPAN + build = _first_start(changes) if not _runs(changes[0].was) else None + retirement = _last_stop(changes) if not _runs(changes[-1].units) else None return Lifespan(_milestone(build), _milestone(retirement)) +def _first_start(changes: list[_Change]) -> _Change | None: + return next((one for one in changes if _runs(one.units)), None) + + +def _last_stop(changes: list[_Change]) -> _Change | None: + return next((one for one in reversed(changes) if _runs(one.was) and not _runs(one.units)), None) + + def _milestone(change: _Change | None) -> Milestone | None: return None if change is None else Milestone(change.at.year, change.units) diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py index c6b577d..6690170 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_shared.py @@ -195,7 +195,11 @@ class ClassLookups: @dataclass(frozen=True) class StorageLookups: - """The per-object properties and memberships the three storage paths read.""" + """The per-object properties and memberships the three storage paths read. + + ``by_class`` holds the Battery and the Generator only, because a Storage states no + capacity of its own and its properties stand beside it in ``storage_properties``. + """ by_class: dict[PlexosClass, ClassLookups] storage_properties: ObjectProperties @@ -204,6 +208,9 @@ class StorageLookups: tail_by_generator: dict[str, str] storages_with_inflow_profile: set[str] + def properties_of(self, plexos_class: PlexosClass) -> ObjectProperties: + return self.by_class[plexos_class].properties + def staged(self, plexos_class: PlexosClass, name: str) -> StagedObject: one = self.by_class[plexos_class] return StagedObject( diff --git a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py index 4e4d773..5f8e503 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py +++ b/interop/plugins/shared/plexos_pypsa_translations/_storage_units.py @@ -184,7 +184,7 @@ def _dropped_values(state: State, lookups: StorageLookups) -> list[_DroppedValue *_dropped_from( read_object_names(state, PlexosClass.BATTERY), PlexosClass.BATTERY, - lookups.by_class[PlexosClass.BATTERY].properties, + lookups.properties_of(PlexosClass.BATTERY), _BATTERY_DROPPED, ), *_dropped_from(storages, PlexosClass.STORAGE, lookups.storage_properties, _STORAGE_DROPPED), diff --git a/interop/plugins/shared/plexos_pypsa_translations/constants.py b/interop/plugins/shared/plexos_pypsa_translations/constants.py index f29e587..ac3201e 100644 --- a/interop/plugins/shared/plexos_pypsa_translations/constants.py +++ b/interop/plugins/shared/plexos_pypsa_translations/constants.py @@ -21,6 +21,7 @@ EXT_UNIT_SIZE_FIELD: str = "extensions.unit_size_mw" EXT_TECHNICAL_LIFE_FIELD: str = "extensions.technical_life_years" EXT_RETIREMENT_YEAR_FIELD: str = "extensions.retirement_year" +EXT_FOM_CHARGE_FIELD: str = "extensions.fom_charge_per_mw_year" # --- generators --------------------------------------------------------------- diff --git a/interop/plugins/steps/pypsa_to_sienna_map_components.py b/interop/plugins/steps/pypsa_to_sienna_map_components.py index 5b4fd1d..3196001 100644 --- a/interop/plugins/steps/pypsa_to_sienna_map_components.py +++ b/interop/plugins/steps/pypsa_to_sienna_map_components.py @@ -308,8 +308,6 @@ def _prepare_source( carriers = self._carrier_mappings.get_carriers(mapping.sienna_component) table = mapping.fill_defaults(source) table = table.filter(pl.col(mapping.carrier_col).is_in(list(carriers))) - candidates = unbuilt_candidate_skip(PYPSA_COMPONENT_NAMING[mapping.source_table]) - table, _ = filter_component(table, candidates.keep, candidates.report, self._recorder) series = self._source_time_series(state, mapping) for rule in mapping.skips(series) if mapping.skips is not None else (): table, _ = filter_component(table, rule.keep, rule.report, self._recorder) diff --git a/tests/features/plexos_to_pypsa/dated_properties.feature b/tests/features/plexos_to_pypsa/dated_properties.feature index a388e8c..e43e913 100644 --- a/tests/features/plexos_to_pypsa/dated_properties.feature +++ b/tests/features/plexos_to_pypsa/dated_properties.feature @@ -152,7 +152,23 @@ Feature: a PLEXOS property dated to a period is read for the year being translat When I run translate against "inputs/retires_later.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" Then the PyPSA network "outputs/network.nc" generator "OldCoal" attribute "p_nom" is 1000 And the file "outputs/extensions.json" parses as JSON with "generator.0.retirement_year" set to 2035 - And the file "decisions.md" contains "the first year the dated Units fall back to zero" + And the file "decisions.md" contains "the last year the dated Units fall back to zero" + + Scenario: a generator whose units return after a dated zero is mothballed, not retired + A zero band that gives way again to the units the generator already runs takes the + plant out for those years alone, so neither end of a life is stated. + Given a Plexos model + And the model contains region "Grid" + And the model contains node "Grid_Node" in region "Grid" + And the model contains generator "Mothball" with "node=Grid_Node, category=Coal, Max Capacity=500, Units=2" + And generator "Mothball" states "Units" of 0 from "2035-01-01" to "2040-12-31" + And the model contains model "Plan" + And the model contains horizon "H1" on model "Plan" starting "2026-01-01" spanning 2 days at 24 periods per day + And the model is saved as "inputs/mothballed.xml" + When I run translate against "inputs/mothballed.xml" pipeline "plexos-to-pypsa" for model "Plan" year 2026 sink output "outputs/network.nc" + Then the PyPSA network "outputs/network.nc" generator "Mothball" attribute "p_nom" is 1000 + And the PyPSA network "outputs/network.nc" generator "Mothball" attribute "build_year" is 0 + And the file "outputs/extensions.json" does not contain "retirement_year" Scenario: a generator that dates no units states neither year Given a Plexos model