Keep the years a PLEXOS plan states, and carry its constraints

docs/translation_mappings/translation-from-plexos-to-pypsa.md

@@ -31,7 +31,8 @@ It gives the source of each field.
 | [`Market`](#market--generator) | An import `Generator` |
 | [`Reserve`](#reserve--extensions-sidecar) | No component. The translator carries it to the reserves sidecar, but nothing applies it. |
 | [Region `VoLL`](#load-shedding) | No component in the two faithful pipelines. In `plexos-to-pypsa-monte-carlo-reliability`, a load shedding `Generator` at each bus. |
-| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) |
+| [`Constraint`](#constraint--extensions-sidecar) | No component. The translator carries it to the sidecar, but nothing applies it. |
+| `Zone`, `Interface`, `Transformer`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) |
 | `Transmission`, `ST`/`MT Schedule`, `PASA`, `Production`, `Performance`, `Stochastic`, `Report`, `Diagnostic`, `System`, `List` | Not translated. These are solver settings, not model data. |
 
 ## Reading the tables
@@ -277,6 +278,7 @@ is `committable` when it is thermal, or when its `p_min_pu` is more than `0`.
 | `discount_rate` | | `WACC`, for a candidate whose build the model prices | `derived` |
 | `lifetime` | yr | `Economic Life`, for a candidate whose build the model prices | `direct` |
 | `extensions.fom_charge_per_mw_year` | $/MW/yr | `FO&M Charge`, for a candidate whose build the model prices | `direct` |
+| `build_year` | yr | The first year the dated [`Units`](#which-entry-applies-when) rise above zero. The field is absent where the model dates no `Units`, and PyPSA reads `0`. | `derived` |
 
 **The translator does not translate seven cases.** It records each one as a skipped
 component:
@@ -361,6 +363,7 @@ has more than one fuel uses its primary fuel.
 | `cyclic_state_of_charge` | | `True` if `End Effects Method` is `RECYCLE`, or if the model gives no `Initial SoC` | `derived` |
 | `marginal_cost` | $/MWh | `0.0` | `default` |
 | `p_nom_extendable`, `p_nom_min`, `p_nom_max`, `overnight_cost`, `discount_rate`, `lifetime` | | Refer to [What a candidate is](#what-a-candidate-is) | `derived` / `default` |
+| `build_year` | yr | The first year the dated [`Units`](#which-entry-applies-when) rise above zero. The field is absent where the model dates no `Units`, and PyPSA reads `0`. | `derived` |
 
 The energy one unit of a battery holds is its `Capacity`. If the model gives a duration in
 place of a capacity, that energy is `Duration × Max Power`. `max_hours` is that energy
@@ -776,7 +779,7 @@ Two properties use dated entries as a schedule. They do not use them as correcti
 
 | Property | Meaning |
 | --- | --- |
-| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. |
+| `Units` | A capacity that starts, retires or partly derates. A static value above zero with a later entry of zero is a **retirement**. A static zero, or no value, with a later entry above zero is a **new build**. The translator reads both years off the entries as the model dates them, whatever year it translates: the first year the value rises above zero is the `build_year` of the component it writes, and the last year in which it falls back to zero is the `retirement_year` in the extensions sidecar. An object that already runs before its first dated entry states no `build_year`, and one that runs again after an entry of zero states no `retirement_year`, because that entry is a mothball rather than a retirement. A generator that runs no units in the year being translated and states no `Max Units Built` is left out as retired, so no `build_year` reaches the network for it. |
 | `Max Capacity` | A capacity expansion schedule. The translator applies the entry that is in force at the snapshot. Where it needs one value, it uses the entry that is in force at the start of the model. |
 
 ### Timeslice patterns
@@ -1001,7 +1004,7 @@ reservoir and no head reservoir.
 | `Interface` | Nothing applies the group flow limits. Thus the dispatch can be more than a transfer limit that your PLEXOS model obeys. |
 | `Transformer` | The translator does not carry it. |
 | `Reserve` requirements | Nothing applies them. The generators that contribute can operate at full output. The translator does carry the reserves. Refer to [`Reserve`](#reserve--extensions-sidecar). |
-| `Constraint` | Nothing applies the custom constraints. This includes the energy budgets, the running hour limits, the RPS targets and the emission targets. The translator reports every one. Refer to [`Constraint`](#constraint). |
+| `Constraint` | Nothing applies the custom constraints. This includes the energy budgets, the running hour limits, the RPS targets and the emission targets. The translator reports every one, and carries each one it can read to the extensions sidecar. Refer to [`Constraint`](#constraint--extensions-sidecar). |
 | `Waterway` | The cascade route between reservoirs is lost. Each reservoir is independent. |
 | `Decision Variable` | The translator does not carry it. |
 | Emission caps | Nothing applies them. Only the carbon price goes into the cost. |
@@ -1010,7 +1013,7 @@ reservoir and no head reservoir.
 | Ancillary service and demand response pseudo-generators | The translator skips nothing. Refer to [`Generator`](#generator--generator). |
 | Gas, heat and water networks | The translator accepts electricity only. |
 
-## `Constraint`
+## `Constraint` → extensions sidecar
 
 A PLEXOS `Constraint` holds a weighted sum over the objects it names to a right-hand side.
 It weights each object by a coefficient on the membership — `Generation Coefficient`,
@@ -1020,14 +1023,32 @@ group of hydro units and an annual running hour cap on a group of peakers are bo
 this way.
 
 A PyPSA `GlobalConstraint` limits one **carrier** over the whole horizon, and it has no way
-to name a set of components. Thus no shape of `Constraint` fits it, and the translator
-carries none of them.
-
-It does report all of them. Every right-hand side a `Constraint` states becomes a not
-mapped entry against the object that states it, giving the value, the sense, and each term
-of the weighted sum: the object, the class it belongs to, and the coefficient weighting it.
-A `Constraint` stating no right-hand side is reported against the object itself. One
-warning names a few of them and counts the rest.
+to name a set of components. Thus no shape of `Constraint` fits it, and the network file
+holds none of them.
+
+Each one the translator can read travels in the `extensions.json` file adjacent to the
+network instead, as a `constraint` record in framework-neutral terms. The contract is in
+`interop/core/extensions.py`. Nothing applies the limit, in the network or in the solve;
+the sidecar carries it for a program that decides to use it, and for a later hop into a
+framework that can express it.
+
+| Sidecar field | From |
+| --- | --- |
+| `name` | `Constraint.name` |
+| `sense` | `Sense`, as the inequality it holds in: `<=`, `==` or `>=` |
+| `limits` | One entry per right-hand side the `Constraint` states, each giving the value, the span it applies over (`horizon`, `hour`, `day`, `week`, `month` or `year`) and the unit the model stated it in |
+| `members` | One entry per object the sum names, each giving the name, the PLEXOS class it belongs to, the coefficient and the property that states the coefficient |
+| `applies_to_expansion_plan` | `Include in LT Plan` |
+
+A `Constraint` that states no sense, a `Sense` other than `-1`, `0` or `1`, or no
+right-hand side at all, states no inequality to carry. The translator leaves that one out
+and reports it, naming the `Sense` it could not read where that is the reason.
+
+The report carries all of them either way. Every right-hand side a `Constraint` states
+becomes a not mapped entry against the object that states it, giving the value, the sense,
+and each term of the weighted sum: the object, the class it belongs to, and the coefficient
+weighting it. A `Constraint` stating no right-hand side is reported against the object
+itself. One warning names a few of them and counts the rest.
 
 **Read that section of the report before you trust the dispatch.** A model that caps hydro
 energy or peaker running hours with a `Constraint` gives a translated network in which

docs/translation_mappings/translation-from-plexos-to-sienna.md

@@ -35,7 +35,7 @@ property of the PLEXOS to Sienna mapping.
 | [`Market`](#market--thermalstandard) | An import `ThermalStandard` |
 | `Reserve` | No component. The record reaches `extensions.json`. Refer to [Not translated](#not-translated). |
 | [Region `VoLL`](#region-load--interruptiblepowerload) | The `operation_cost` of an `InterruptiblePowerLoad`, on a reliability run only. |
-| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) |
+| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated). A `Constraint` reaches the sidecar, but nothing applies it. |
 | `Transmission`, `ST`/`MT Schedule`, `PASA`, `Production`, `Performance`, `Stochastic`, `Report`, `Diagnostic`, `System`, `List` | Not translated. These are solver settings, not model data. |
 
 ## Reading the tables
@@ -504,7 +504,7 @@ energy than your model gives it.
 | `Zone` | The zonal group is lost. The regional group still becomes an `Area`. |
 | `Interface` | Nothing applies the group flow limits, so a transfer can go above a limit your model obeys. |
 | `Transformer` | The translator does not carry it. |
-| `Constraint` | Nothing applies the custom constraints, which include the RPS targets and the emission targets. |
+| `Constraint` | The record reaches `extensions.json`, but nothing applies the custom constraints, which include the RPS targets and the emission targets. |
 | `Waterway` | The cascade route between reservoirs is lost. Each reservoir is independent. |
 | `Decision Variable` | The translator does not carry it. |
 | Emission caps | Nothing applies them. Only the carbon price reaches the cost. |

interop/core/extensions.py

@@ -47,6 +47,10 @@ class ExtensionKind(StrEnum):
     # PLEXOS Reserve; Sienna VariableReserve and ConstantReserve. PyPSA has none, which is
     # why the concept needs the sidecar to survive a hop through it.
     RESERVE = "reserve"
+    # PLEXOS Constraint. PyPSA's GlobalConstraint limits one carrier over the whole horizon
+    # and cannot name a set of components, which is why the concept needs the sidecar to
+    # survive a hop through it. Sienna has no equivalent either.
+    CONSTRAINT = "constraint"
     NETWORK = "network"  # PyPSA network-level attributes. No Sienna or PLEXOS equivalent.
 
 
@@ -80,6 +84,25 @@ class ReserveKind(StrEnum):
     UNKNOWN = "unknown"
 
 
+class ConstraintSense(StrEnum):
+    """Which way a constraint holds its weighted sum against the right-hand side."""
+
+    AT_MOST = "<="  # PLEXOS Sense -1
+    EXACTLY = "=="  # PLEXOS Sense 0
+    AT_LEAST = ">="  # PLEXOS Sense 1
+
+
+class ConstraintPeriod(StrEnum):
+    """The span one right-hand side applies over."""
+
+    HORIZON = "horizon"  # PLEXOS RHS
+    HOUR = "hour"  # PLEXOS RHS Hour
+    DAY = "day"  # PLEXOS RHS Day
+    WEEK = "week"  # PLEXOS RHS Week
+    MONTH = "month"  # PLEXOS RHS Month
+    YEAR = "year"  # PLEXOS RHS Year
+
+
 class ExtensionRecord(BaseModel):
     """One component's record. ``name`` is the identifier in every framework."""
 
@@ -109,6 +132,9 @@ class ExpansionExtension(ExtensionRecord):
     # $/MW/yr. PyPSA's fom_cost is a charge for the whole modelled horizon, not a yearly
     # one, so a yearly charge has no field there.
     fom_charge_per_mw_year: float | None = None
+    # The year the object leaves service, which PLEXOS states as a dated Units of zero.
+    # PyPSA carries build_year on the component and nothing for the other end of its life.
+    retirement_year: int | None = None
 
 
 class GeneratorExtension(ExpansionExtension):
@@ -193,6 +219,45 @@ class ReserveExtension(ExtensionRecord):
     is_mutually_exclusive: bool | None = None
 
 
+class ConstraintMember(BaseModel):
+    """One object a constraint weights, and the coefficient it is weighted by.
+
+    Two classes can hold an object of the same name and a constraint can weight a generator
+    and an emission alike, so the member carries the class its name belongs to. The
+    coefficient is absent where the constraint names the object without weighting it.
+    """
+
+    model_config = ConfigDict(extra="forbid")
+
+    name: str
+    member_class: str  # PLEXOS Generator, Line, Emission and the rest
+    coefficient: float | None = None
+    # PLEXOS names the coefficient per class: Generation Coefficient, Flow Coefficient, and
+    # so on. The name says which quantity of the member the coefficient weights.
+    coefficient_property: str | None = None
+
+
+class ConstraintLimit(BaseModel):
+    """One right-hand side the weighted sum is held to, and the span it applies over."""
+
+    model_config = ConfigDict(extra="forbid")
+
+    period: ConstraintPeriod
+    value: float
+    unit: str | None = None  # the unit the source stated the limit in
+
+
+class ConstraintExtension(ExtensionRecord):
+    """A weighted sum over named objects, held to one or more right-hand sides."""
+
+    sense: ConstraintSense | None = None
+    limits: list[ConstraintLimit] = []
+    members: list[ConstraintMember] = []
+    # PLEXOS Include in LT Plan: whether the expansion plan has to meet the constraint as
+    # well as the dispatch.
+    applies_to_expansion_plan: bool | None = None
+
+
 class NetworkExtension(ExtensionRecord):
     """The model file's own attributes. PyPSA only: neither Sienna nor PLEXOS has these."""
 
@@ -213,6 +278,7 @@ class Extensions(BaseModel):
     controllable_line: list[ControllableLineExtension] = []
     storage: list[StorageExtension] = []
     reserve: list[ReserveExtension] = []
+    constraint: list[ConstraintExtension] = []
     network: list[NetworkExtension] = []
 
 
@@ -224,6 +290,7 @@ class Extensions(BaseModel):
     ExtensionKind.CONTROLLABLE_LINE: ControllableLineExtension,
     ExtensionKind.STORAGE: StorageExtension,
     ExtensionKind.RESERVE: ReserveExtension,
+    ExtensionKind.CONSTRAINT: ConstraintExtension,
     ExtensionKind.NETWORK: NetworkExtension,
 }
 
@@ -404,6 +471,10 @@ def record_for(
     staged: StagedExtensions, kind: Literal[ExtensionKind.RESERVE], name: str
 ) -> ReserveExtension | None: ...
 @overload
+def record_for(
+    staged: StagedExtensions, kind: Literal[ExtensionKind.CONSTRAINT], name: str
+) -> ConstraintExtension | None: ...
+@overload
 def record_for(
     staged: StagedExtensions, kind: Literal[ExtensionKind.NETWORK], name: str
 ) -> NetworkExtension | None: ...
@@ -468,6 +539,10 @@ def read(self, kind: Literal[ExtensionKind.STORAGE]) -> ExtensionLookup[StorageE
     @overload
     def read(self, kind: Literal[ExtensionKind.RESERVE]) -> ExtensionLookup[ReserveExtension]: ...
     @overload
+    def read(
+        self, kind: Literal[ExtensionKind.CONSTRAINT]
+    ) -> ExtensionLookup[ConstraintExtension]: ...
+    @overload
     def read(self, kind: Literal[ExtensionKind.NETWORK]) -> ExtensionLookup[NetworkExtension]: ...
 
     def read(self, kind: ExtensionKind) -> ExtensionLookup[Any]:

interop/plugins/shared/plexos_constants.py

@@ -40,10 +40,11 @@ class PlexosClass(StrEnum):
 
 
 class PlexosResolvedTable:
-    """Keys of the two long tables ``stage_plexos_xml`` resolves from the raw ``t_*`` tables."""
+    """Keys of the long tables ``stage_plexos_xml`` resolves from the raw ``t_*`` tables."""
 
     MEMBERSHIPS = "memberships"
     PROPERTIES = "properties"
+    DATED_PROPERTIES = "dated_properties"
 
 
 class PlexosObjectCol:
@@ -95,6 +96,19 @@ class PlexosPropertyCol:
     SCALING = "scaling"
 
 
+class PlexosDatedPropertyCol(PlexosPropertyCol):
+    """Columns of the resolved ``dated_properties`` table.
+
+    The ``properties`` columns, plus the dates the value on the row applies between. Every
+    row of a property the model dates is here, whether or not the window being translated
+    covers its band, so a mapping reading a schedule sees the years the model states rather
+    than one value.
+    """
+
+    DATE_FROM = "date_from"
+    DATE_TO = "date_to"
+
+
 class PlexosCollection(StrEnum):
     """PLEXOS collection names, the relationship kind on a ``memberships`` row.
 
@@ -211,6 +225,7 @@ class PlexosProperty(StrEnum):
     RHS_WEEK = "RHS Week"
     RHS_MONTH = "RHS Month"
     RHS_YEAR = "RHS Year"
+    INCLUDE_IN_LT_PLAN = "Include in LT Plan"
 
 
 def is_plexos_true(value: float) -> bool:

interop/plugins/shared/plexos_dates.py

@@ -0,0 +1,45 @@
+"""When a dated PLEXOS value applies."""
+
+from __future__ import annotations
+
+from collections.abc import Iterable, Sequence
+from datetime import datetime, timedelta
+from typing import NamedTuple, TypeVar
+
+
+class DateBand(NamedTuple):
+    """When a ``t_data`` value applies. An open end runs from, or until, forever."""
+
+    date_from: datetime | None
+    date_to: datetime | None
+
+    @property
+    def ends(self) -> datetime | None:
+        """A ``date_to`` names a whole day, so the band runs to the end of it."""
+        return None if self.date_to is None else self.date_to + timedelta(days=1)
+
+    def covers(self, moment: datetime) -> bool:
+        return (self.date_from is None or self.date_from <= moment) and (
+            self.ends is None or moment < self.ends
+        )
+
+
+UNDATED = DateBand(None, None)
+
+T = TypeVar("T")
+
+
+def latest_covering(bands: Sequence[tuple[DateBand, T]], moment: datetime) -> T | None:
+    covering = [value for band, value in bands if band.covers(moment)]
+    return covering[-1] if covering else None
+
+
+def band_edges(bands: Iterable[tuple[DateBand, object]]) -> list[datetime]:
+    """Every moment a band opens or closes, earliest first."""
+    moments = {edge for band, _ in bands for edge in (band.date_from, band.ends)}
+    return sorted(moment for moment in moments if moment is not None)
+
+
+def opens_at(band: tuple[DateBand, object]) -> datetime:
+    """A sort key putting an undated band first, since it stands before any dated one begins."""
+    return band[0].date_from or datetime.min

interop/plugins/shared/plexos_pypsa_translations/_batteries.py

@@ -20,6 +20,7 @@
     derive_expansion,
     gather_sources,
 )
+from interop.plugins.shared.plexos_pypsa_translations._lifespan import derive_lifespan
 from interop.plugins.shared.plexos_pypsa_translations._shared import outage_time_series
 from interop.plugins.shared.plexos_pypsa_translations._storage_shared import (
     CARRIER_NOTE,
@@ -93,7 +94,7 @@ def record_battery_outages(state: State, mappings: list[StorageUnitMapping]) ->
 
 
 def map_battery(name: str, lookups: StorageLookups) -> MappedOrSkipped:
-    rated = rate_object(lookups.battery(name), _BATTERY_POWER)
+    rated = rate_object(lookups.staged(PlexosClass.BATTERY, name), _BATTERY_POWER)
     if isinstance(rated, SkippedComponent):
         return rated
     return _derive_battery(rated)
@@ -128,6 +129,7 @@ def _derive_battery(rated: RatedObject) -> StorageUnitMapping:
         ),
         inflow=Decision.default(DEFAULT_INFLOW, NO_RESERVOIR_INFLOW_NOTE),
         cyclic=_battery_cyclic(rated),
+        lifespan=derive_lifespan(PlexosClass.BATTERY, rated.name, rated.lifespan),
         expansion=expansion,
         units=rated.properties.get(PlexosProperty.UNITS),
     )