diff --git a/README.md b/README.md index 52e5bd3..6dd2158 100644 --- a/README.md +++ b/README.md @@ -304,6 +304,7 @@ Use cases live under `interop/core/use_cases/`, each implementing a Protocol por - `docs/translation_mappings/translation-from-pypsa-to-sienna.md` is the authoritative mapping reference for PyPSA / Sienna field translation. - `docs/translation_mappings/translation-from-plexos-to-pypsa.md` states what each part of a PLEXOS model becomes in PyPSA. - `docs/translation_mappings/translation-from-plexos-to-sienna.md` states the same for Sienna, and `plexos-to-sienna-gap-analysis.md` beside it states what that translation loses and what each loss does to a dispatch. +- `docs/translation_mappings/translation-from-plexos-to-sienna-investments.md` states what a PLEXOS expansion plan becomes in a Sienna investments portfolio, which `plexos-to-sienna-investments` writes beside the base system, and `plexos-to-sienna-gap-analysis.md` states what the portfolio leaves out and what each loss does to an expansion. - `docs/developer_documentation/mutation-testing.md` covers the mutmut workflow. - `tests/features/README.md` explains the in-process REPL-driven BDD harness and when subprocess is needed. diff --git a/docs/case_studies/aemo-isp-2024.md b/docs/case_studies/aemo-isp-2024.md index 38fcd53..25b3379 100644 --- a/docs/case_studies/aemo-isp-2024.md +++ b/docs/case_studies/aemo-isp-2024.md @@ -121,11 +121,11 @@ Select `translate`. Then give these answers: | Source framework | `plexos` | | Destination framework | `sienna` | | Pipeline | `plexos-to-sienna` | -| User mappings file | `inputs/plexos_user_mappings.yaml` | | the PLEXOS `` input XML | `case_study_inputs/aemo-isp-2024/2024 ISP Step Change Model.xml` | | which PLEXOS Model to translate | `Step Change` | | a four-digit year such as 2026 | `2025` | | the SiennaSchemas system.json | `outputs/system.json` | +| User mappings file | `inputs/plexos_user_mappings.yaml` | Give the year `2025` here, unlike the PyPSA run above, which leaves the year empty. The chronology of the Model is 2024-07-01 to 2025-06-30, so the year 2025 narrows it to @@ -158,6 +158,67 @@ Then select `solve`. Give the model type `sienna` and the system `linearised`, and the HiGHS defaults. Leave the time limit empty. For more data about these prompts, refer to [the solve tutorial](../tutorials/solve.md#sienna-path). +### The expansion path + +The same model also translates to a Sienna investments portfolio, which is what a partner +running an expansion in PowerSystemsInvestments.jl needs. That run writes the expansion +problem: the technologies the plan may build, the demand they meet, and the caps they run +under. It is a translation only. interop runs no expansion solve, so this path stops at the +files. This section covers the Step Change scenario. The other two follow the same steps. + +Write a second mappings file, `inputs/plexos_expansion_mappings.yaml`. It is the dispatch +file plus one row for each carrier a candidate takes. A candidate whose carrier the file does +not name is left out of the portfolio, and `decisions.md` names each one. + +The dispatch run above leaves seven named categories, and their siblings, out of its file. +The expansion run names the ones that hold candidate plants. A generator takes the name of its +`Fuel` where it burns one at a heat rate, and its category where it does not, so add one +`category` row for each of these: + +- `2023 REZ NSW`, and the sibling category of each other state: a wind or a solar candidate, + so `RenewableDispatch` with the prime mover `WT` or `PVe`; +- `New Entrants NSW`, and the sibling category of each other state: a wind, a solar or a gas + candidate, so the prime mover `WT`, `PVe`, `CC` or `CT`; +- `LTESA Projects`, `Policy Projects` and `VRET Projects`. + +The translator reads no meaning from a category name, so give each row the +`sienna_component_type` and the `sienna_prime_mover_type` of the plant that category holds in +your copy of the model. +[Across all components](../translation_mappings/translation-from-plexos-to-sienna-investments.md#across-all-components) +states the types a row may name, and what happens to a row that names another kind's type. + +Leave `REZ Augmentation` and `Group REZ Augmentation` out of this file as well. They are +transmission augmentations written as generators, and the portfolio holds no transport +technology, so a row for one states a plant your model does not mean. + +Select `translate`. Then give these answers: + +| Prompt | Answer | +| --- | --- | +| Source framework | `plexos` | +| Destination framework | `sienna` | +| Pipeline | `plexos-to-sienna-investments` | +| the PLEXOS `` input XML | `case_study_inputs/aemo-isp-2024/2024 ISP Step Change Model.xml` | +| which PLEXOS Model to translate | `Step Change` | +| a four-digit year such as 2026 | `2025` | +| the SiennaSchemas system.json | `outputs/system.json` | +| the SiennaSchemas portfolio document to write | `outputs/portfolio.json` | +| User mappings file | `inputs/plexos_expansion_mappings.yaml` | + +Keep the default at every other prompt. The mappings prompt comes last, after the file +prompts of both sinks. + +That run writes four files: the three the Sienna path writes, and `outputs/portfolio.json` +beside them. The portfolio names `system.json` in its `base_system_file`, so the two are read +together. + +Every cost in a portfolio is quoted in a base year, and no PLEXOS field states one. This run +takes the default base year of 2020. To state another one, refer to +[The base year](../translation_mappings/translation-from-plexos-to-sienna-investments.md#the-base-year). + +[The gap analysis](../translation_mappings/plexos-to-sienna-gap-analysis.md) states what the +portfolio leaves out and what each loss does to an expansion. + ## The headline number **What you can check by yourself.** All three scenarios translate. All three solves give diff --git a/docs/case_studies/caiso-sa26.md b/docs/case_studies/caiso-sa26.md index 9ad8378..3a66221 100644 --- a/docs/case_studies/caiso-sa26.md +++ b/docs/case_studies/caiso-sa26.md @@ -178,12 +178,12 @@ Select `translate`. Then give these answers: | Source framework | `plexos` | | Destination framework | `sienna` | | Pipeline | `plexos-to-sienna-monte-carlo` | -| User mappings file | `inputs/plexos_user_mappings.yaml` | | the PLEXOS `` input XML | `case_study_inputs/caiso-sa26/CAISOSA26 20260429.xml` | | which PLEXOS Model to translate | `M09Y2026 SA26` | | a four-digit year such as 2026 | Leave empty, as for the PyPSA run above. | | directory to hold the ensemble | `outputs/caiso-m09-sienna` | | names each replication's directory | Keep the default, `{sample}` | +| User mappings file | `inputs/plexos_user_mappings.yaml` | That run writes one directory per replication, `1`, `2` and `3`, each holding four files: `system.json`, its HDF5 companion `system_time_series_storage.h5`, `extensions.json`, and the diff --git a/docs/case_studies/sem-2024-2032.md b/docs/case_studies/sem-2024-2032.md index fed7d88..af72352 100644 --- a/docs/case_studies/sem-2024-2032.md +++ b/docs/case_studies/sem-2024-2032.md @@ -94,11 +94,11 @@ Select `translate`. Then give these answers: | Source framework | `plexos` | | Destination framework | `sienna` | | Pipeline | `plexos-to-sienna` | -| User mappings file | `inputs/plexos_user_mappings.yaml` | | the PLEXOS `` input XML | `case_study_inputs/sem-2024-2032/PUBLIC Validation 2024-2032 Model 2025-03-14.xml` | | which PLEXOS Model to translate | `Opt A 24-32 (Avail, Uplift, Wheeling)--MIP` | | a four-digit year such as 2026 | `2026` | | the SiennaSchemas system.json | `outputs/system.json` | +| User mappings file | `inputs/plexos_user_mappings.yaml` | That run writes three files: `outputs/system.json`, its HDF5 companion `outputs/system_time_series_storage.h5`, and `outputs/extensions.json`. Those three files diff --git a/docs/translation_mappings/plexos-to-sienna-gap-analysis.md b/docs/translation_mappings/plexos-to-sienna-gap-analysis.md index bb13c66..37f831d 100644 --- a/docs/translation_mappings/plexos-to-sienna-gap-analysis.md +++ b/docs/translation_mappings/plexos-to-sienna-gap-analysis.md @@ -1,11 +1,11 @@ # What a PLEXOS to Sienna translation loses -This document lists each thing a `plexos-to-sienna` run loses, and what that loss does to a -dispatch. It covers the translation and the validation run that proves the system solves in -PowerSimulations.jl. +This document lists each thing a PLEXOS to Sienna run loses. For what the translation keeps, refer to -[Translation from PLEXOS to Sienna](translation-from-plexos-to-sienna.md). +[Translation from PLEXOS to Sienna](translation-from-plexos-to-sienna.md), and for the +portfolio to +[Translation from PLEXOS to a Sienna investments portfolio](translation-from-plexos-to-sienna-investments.md). Each entry gives four things: @@ -15,10 +15,16 @@ Each entry gives four things: | What happens to it | Where it goes, or that it goes nowhere | | The cause | Why | | The effect on the dispatch | What the solve then does differently from your PLEXOS model | +| The effect on the expansion | What a plan built from the portfolio then does differently from your PLEXOS model | --- -## Reserve requirements +## What `plexos-to-sienna` loses from a dispatch + +These entries cover the translation and the validation run that proves the system solves in +PowerSimulations.jl. + +### Reserve requirements **The PLEXOS data.** Each `Reserve` object, its type, its requirement in MW or as a share of a profile, and the generators that can provide it. @@ -37,7 +43,7 @@ model, and a scarcity price your model shows does not appear. --- -## Load shedding, on a plain run +### Load shedding, on a plain run **The PLEXOS data.** Each Region states a `VoLL`, the value of lost load. @@ -55,7 +61,7 @@ than a shortfall in MWh. Run the reliability chain to get the shortfall in MWh i --- -## Unit commitment, relaxed +### Unit commitment, relaxed **The PLEXOS data.** A start cost, a minimum up time and a minimum down time on each thermal generator. @@ -78,7 +84,7 @@ two paths do not compare under that answer. --- -## Thermal availability where the model states none +### Thermal availability where the model states none **The PLEXOS data.** An `Outage Factor`, an `Outage Rating`, a `Rating` profile or a `Units Out` profile on some of the thermal fleet, and nothing on the rest. @@ -98,7 +104,7 @@ number. --- -## A hydro unit whose inflow is water, not power +### A hydro unit whose inflow is water, not power **The PLEXOS data.** A reservoir hydro turbine with a `Natural Inflow` stated in cumec or in m³/day, or with no `Natural Inflow` at all. @@ -117,7 +123,7 @@ The AEMO 2024 ISP states every inflow in cumec, so its whole reservoir fleet is --- -## A storage unit that states no energy +### A storage unit that states no energy **The PLEXOS data.** A `Battery` with no `Capacity` and no `Duration`, or a pumped storage head reservoir whose `Max Volume` is in water. @@ -133,7 +139,7 @@ across the hours it would have covered. --- -## A generator that is not a power plant +### A generator that is not a power plant **The PLEXOS data.** A generator whose category names a transmission augmentation, a policy project or another pseudo-object rather than a technology. @@ -150,7 +156,7 @@ each run. --- -## Heat rate bands +### Heat rate bands **The PLEXOS data.** A `Heat Rate` stated as several bands, so the efficiency changes with output. @@ -168,7 +174,7 @@ differently. --- -## A generator that burns more than one fuel +### A generator that burns more than one fuel **The PLEXOS data.** Several `Fuels` memberships on one generator. @@ -183,7 +189,7 @@ price. A dual-fuel unit that your model switches to a cheaper fuel does not swit --- -## A Fuel and a generator category of one name +### A Fuel and a generator category of one name **The PLEXOS data.** A `Fuel` and a generator category that share a name, for example a fuel `HVO` and a category `HVO`. @@ -201,7 +207,7 @@ groups of generators then take that type. --- -## Zones, interfaces and custom constraints +### Zones, interfaces and custom constraints **The PLEXOS data.** `Zone` objects, `Interface` flow limits and `Constraint` objects, which include energy budgets, running hour limits, RPS targets and emission caps. @@ -224,7 +230,7 @@ the dispatch. --- -## Hydro cascades and volumes in water +### Hydro cascades and volumes in water **The PLEXOS data.** `Waterway` objects joining reservoirs, and `Max Volume` and `Initial Volume` stated in 1000 m³ or a `Natural Inflow` stated in cumec. @@ -243,7 +249,7 @@ number that depends on hydro. --- -## One solve, one window +### One solve, one window **The PLEXOS data.** A Horizon, which can be many years long. @@ -261,7 +267,7 @@ shorter year. --- -## Region Price of Dump Energy +### Region Price of Dump Energy **The PLEXOS data.** A Region `Price of Dump Energy`, the price of energy the system spills. @@ -276,7 +282,7 @@ objective. The Region `VoLL` beside it does reach a reliability run: refer to --- -## Sienna holds no Monte Carlo forecast a solve reads +### Sienna holds no Monte Carlo forecast a solve reads **The PLEXOS data.** A pre-sampled model states many values for one property at one snapshot, one per replication, and a run over it draws a distribution of outcomes. @@ -294,7 +300,7 @@ counts how many of them lose load. Solve the replications and count the outcomes --- -## A Sienna objective and a PyPSA objective do not compare +### A Sienna objective and a PyPSA objective do not compare **The PLEXOS data.** A Region `VoLL`, in a reliability run on both sides of the PyPSA hub. @@ -313,7 +319,7 @@ not the same quantity and must not be compared or subtracted. --- -## A profile that reaches only some replications +### A profile that reaches only some replications **The PLEXOS data.** An outage draw that takes a unit out in one replication and leaves it available through the whole window in another. @@ -334,7 +340,7 @@ holds. --- -## A reliability solve reports its unserved energy in the results files +### A reliability solve reports its unserved energy in the results files **The PLEXOS data.** The energy a window cannot serve, which a reliability run prices at the Region `VoLL`. @@ -350,3 +356,139 @@ does not read yet. **The effect on the dispatch.** None. The number is in the solve output; no report collects it for you. + +--- + +## What `plexos-to-sienna-investments` loses from an expansion + +### A `Constraint` a cap cannot carry + +**The PLEXOS data.** A `Constraint` holding a weighted sum over the objects it names to a +right-hand side: an emission cap over a group of plants, a target over one region, a budget +over one technology. + +**What happens to it.** The portfolio writes a `CarbonCaps` only for a constraint that meets +all five of these conditions: + +- it names every generator and storage object the base system and the portfolio hold; +- it holds its weighted sum to `<=`; +- it states a right-hand side for a year or for the whole horizon; +- that right-hand side is a finite number; +- its `Include in LT Plan` is not false, and a constraint that states nothing stays in. + +Every other constraint is left out, `decisions.md` names it, and the log warns. Every +constraint still reaches the `extensions.json` sidecar, whether or not it became a cap. + +**The cause.** `CarbonCaps` names no members and no region: a cap in a portfolio holds the +whole portfolio. A cap also states one ceiling over the whole run, so a constraint held to +`>=` or to `==`, and a constraint whose only right-hand side bounds a repeating window inside +the run, give it nothing to carry. A cap states a number of million tonnes, and neither NaN +nor Infinity is one, so a right-hand side that is not a finite number gives it nothing either. +A constraint the expansion plan does not have to meet would bound a problem your model leaves +free. + +**The effect on the expansion.** Nothing bounds what the constraint names, so the plan may +build and run those objects up to their own limits. A model whose targets are all regional, +technology-scoped, or written as a floor reaches the portfolio with no cap at all. Read the +constraints in the sidecar before you trust what the plan builds. + +--- + +### A candidate whose build nothing prices + +**The PLEXOS data.** A `Generator`, a `Battery` or a pumped-storage turbine stating `Max Units +Built` but not all of `Build Cost`, `WACC` and `Economic Life`. + +**What happens to it.** The PLEXOS leg leaves the build out. An object with nothing running +yet is its build and nothing else, so the whole object goes and `decisions.md` names it. An +object that already runs keeps the capacity it runs, and only the build it may add is +dropped. The second leg drops a candidate that reaches it with no finite upper bound on +capacity, a capacity floor above that upper bound, no finite lifetime, a lifetime below one +year, no overnight cost or no discount rate. It drops a storage candidate that holds no +energy, or that puts no upper bound on the energy a build may add. It names each one the same +way. + +**The cause.** PyPSA annuitises an overnight cost with a discount rate over a lifetime. It +refuses a network that states an overnight cost and no discount rate, it prices a build with +no lifetime as a perpetuity, and it builds for free a candidate that states no overnight cost +at all. A Sienna technology states its own price and its own financing, and +`TechnologyFinancialData` requires both a return on equity and a capital recovery period. + +**The effect on the expansion.** That candidate is not in the portfolio, so no plan built +from it can build that technology. + +--- + +### Transmission a plan may build + +**The PLEXOS data.** A `Line` or a transformer the plan may expand, and the `Line.Type` that +says which technology LT Plan expands it with. + +**What happens to it.** Neither leg writes an expandable branch. The base system holds each +line and each link at the rating it already has, and the portfolio holds no technology for a +corridor. + +**The cause.** SiennaSchemas states transport technologies of its own, and this translation +writes none of them. The PLEXOS leg fixes the capacity of every line and link, so nothing +reaches the hub for a second leg to read either. + +**The effect on the expansion.** The network is fixed. A plan can put new capacity only where +the corridors that already exist can carry it, so it builds nearer to the demand than your +model would, and an expansion your model meets by reinforcing a corridor is met by generation +or not at all. + +--- + +### The year a cap applies in, and a cap on carbon intensity + +**The PLEXOS data.** A `Constraint` right-hand side, stated for a year or over the whole +horizon, and the span it applies over. + +**What happens to it.** The yearly right-hand side becomes `max_mtons`, the cap's limit in +million tonnes, and the horizon-wide one is read where the constraint states no yearly limit. +`CarbonCaps.target_year` and `CarbonCaps.max_tons_mwh` are left unmapped and are absent from +the document; `decisions.md` records both. + +**The cause.** PLEXOS states the span a right-hand side applies over, not the year it applies +in, and it has no rate-based right-hand side for `max_tons_mwh` to carry. + +**The effect on the expansion.** The cap holds the whole run, whichever year your model +stated it for, so a yearly right-hand side becomes a budget for the whole run. With +`max_tons_mwh` absent, nothing limits the carbon intensity of what it builds. + +--- + +### A storage build that prices only its discharge + +**The PLEXOS data.** The `Build Cost` of a `Battery`, or of a pumped-storage turbine, which +prices the unit by its power. + +**What happens to it.** It becomes the technology's `capital_costs.discharge_capital_cost`. +The `charge_capital_cost` and the `energy_capital_cost` beside it are written as zero curves. + +**The cause.** A Sienna storage technology adds charge power, discharge power and energy +independently and prices each of the three. PLEXOS prices the object by its power alone, and +PyPSA carries one overnight cost for a storage unit, so neither states the other two prices. + +**The effect on the expansion.** A consumer that sizes the three parts separately takes the +largest energy and the largest charging power its limits allow for nothing, because nothing +prices either. The whole price of a storage build sits on its discharge capacity, which is +the right number only if you read the source's build cost as the price of a whole unit. + +--- + +### The years a plan steps through + +**The PLEXOS data.** The investment periods of an LT Plan, the years it steps through, and +the representative days and weights it samples each year with. + +**What happens to it.** Neither reaches the portfolio. The portfolio states one expansion +problem, with no schedule of periods and no representative-day weighting. + +**The cause.** A portfolio document holds the technologies, the requirements and the regions +of an expansion problem. The periods and the representative days are terms of the solve. + +**The effect on the expansion.** Whoever solves the portfolio chooses the periods and the +sampling. A plan built over a different set of years, or against a different set of +representative days, from the ones your PLEXOS model uses builds a different fleet, so its +result and your model's LT Plan result are not the same quantity. diff --git a/docs/translation_mappings/translation-from-plexos-to-sienna.md b/docs/translation_mappings/translation-from-plexos-to-sienna.md index 25dbe85..942cf5b 100644 --- a/docs/translation_mappings/translation-from-plexos-to-sienna.md +++ b/docs/translation_mappings/translation-from-plexos-to-sienna.md @@ -3,12 +3,14 @@ This document tells you what each part of your PLEXOS model becomes in the Sienna system. It gives the source of each field. -> **Scope:** the translator accepts electricity-only models, and it translates them for -> dispatch. It does not translate capacity expansion, custom constraints or hydro cascades. -> It does not carry the reserves to a file you keep. Refer to -> [Not translated](#not-translated) and to +> **Scope:** the translator accepts electricity-only models, and the `plexos-to-sienna` +> pipeline translates them for dispatch. It writes no capacity expansion: what your plan may +> build becomes a Sienna investments portfolio instead, which +> [the `plexos-to-sienna-investments` pipeline](translation-from-plexos-to-sienna-investments.md) +> writes. This pipeline does not translate custom constraints or hydro cascades, and it does +> not carry the reserves to a file you keep. Refer to [Not translated](#not-translated) and to > [the gap analysis](plexos-to-sienna-gap-analysis.md), which states what each loss does to a -> dispatch. +> dispatch and what the portfolio leaves out of an expansion. The `plexos-to-sienna` pipeline runs through a PyPSA network on the way. This document does not describe that network. It states the mapping as one step, because that is what you give @@ -35,7 +37,7 @@ property of the PLEXOS to Sienna mapping. | [`Market`](#market--thermalstandard) | An import `ThermalStandard` | | `Reserve` | No component. The record reaches `extensions.json`. Refer to [Not translated](#not-translated). | | [Region `VoLL`](#region-load--interruptiblepowerload) | The `operation_cost` of an `InterruptiblePowerLoad`, on a reliability run only. | -| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated) | +| `Zone`, `Interface`, `Transformer`, `Constraint`, `Waterway`, `Decision Variable` | [Not translated](#not-translated). A `Constraint` reaches the sidecar, but nothing applies it. | | `Transmission`, `ST`/`MT Schedule`, `PASA`, `Production`, `Performance`, `Stochastic`, `Report`, `Diagnostic`, `System`, `List` | Not translated. These are solver settings, not model data. | ## Reading the tables @@ -55,6 +57,10 @@ property of the PLEXOS to Sienna mapping. [The carrier mappings file](#the-carrier-mappings-file). - **A component whose carrier your file does not name is left out.** The run completes and `decisions.md` names each one. +- **A build the plan has yet to decide is not a plant.** A `Generator`, a `Battery` or a + turbine that states `Max Units Built` and runs no units yet is a build and nothing else, and + a dispatch system has no capacity for one, so it is left out and `decisions.md` names it. One + that already runs keeps the capacity it runs, and only the build it may add is left out. - **One scenario only.** The Model you select applies its own Scenario overlays. The translator reads no other scenario. - **One calendar year at a time.** The year you give narrows the Horizon of the Model. Every @@ -263,13 +269,14 @@ An availability that changes with time becomes a `TimeSeriesAssociation` on `max_active_power`. That is the name PowerSimulations reads an availability forecast under. The stored shape peaks at 1.0 and scales back to MW through `active_power_limits.max`. -**The translator does not translate four cases.** It records each one as a skipped +**The translator does not translate five cases.** It records each one as a skipped component: | Case | Cause | | --- | --- | | The generator has no `Node` | There is no bus to connect it to. | | The generator has no `Units` at any time in the horizon | The unit is retired. | +| The generator states `Max Units Built` and runs no units yet | It is a build to decide, not capacity to dispatch. | | `Max Capacity` comes from a data file | There is no single capacity to divide the per-unit fields by. | | The capacity is 0 | The generator can never dispatch. | @@ -438,7 +445,7 @@ value that changes during that year becomes a time series over the snapshots. | Property | Meaning | | --- | --- | -| `Units` | The quantity of units in service. `0` at every date is a retired unit. | +| `Units` | The quantity of units in service. `0` at every date is a retired unit, unless the object also states `Max Units Built`, which makes it a candidate. | | `Max Capacity` | The capacity of one unit. | ### Availability and outages @@ -499,7 +506,7 @@ energy than your model gives it. | `Zone` | The zonal group is lost. The regional group still becomes an `Area`. | | `Interface` | Nothing applies the group flow limits, so a transfer can go above a limit your model obeys. | | `Transformer` | The translator does not carry it. | -| `Constraint` | Nothing applies the custom constraints, which include the RPS targets and the emission targets. | +| `Constraint` | The record reaches `extensions.json`, but nothing applies the custom constraints, which include the RPS targets and the emission targets. | | `Waterway` | The cascade route between reservoirs is lost. Each reservoir is independent. | | `Decision Variable` | The translator does not carry it. | | Emission caps | Nothing applies them. Only the carbon price reaches the cost. |