Optimise construction of capacity_annual_constraint for flatline demands

docs/source/mathematical_formulation.rst

@@ -1044,7 +1044,7 @@ v_flow_out
 The most fundamental variable in the Temoa formulation is the
 :code:`v_flow_out` variable. It describes the commodity flow out of a
 process in a given time slice. To balance input and output flows in the
-:code:`CommodityBalance_constraint`, the commodity flow into a given
+:code:`commodity_balance_constraint`, the commodity flow into a given
 process can be calculated as
 :math:`\sum_{T, V, O} \textbf{FO}_{p, s, d, c, t, v, o}
 /EFF_{c,t,v,o}`.
@@ -1078,7 +1078,7 @@ the :code:`tech_flex` set allows for the overproduction of propane and
 kerosene, allowing the model to fulfill the endogenous demand
 for gasoline. This flexible technology designation activates a slack
 variable (:math:`\textbf{FLX}_{r, p, s, d, i, t, v, c}`)representing
-the excess production in the :code:`CommodityBalanceAnnual_constraint`.
+the excess production in the :code:`annual_commodity_balance_constraint`.
 
 
 v_flex_annual
@@ -1220,23 +1220,88 @@ Equations
 ---------
 
 There are four main equations that govern the flow of energy through the model
-network.  The :code:`Demand_Constrant` :eq:`Demand` ensures that the supply meets
-demand in every time slice.  For each process, the :code:`Capacity_constraint` :eq:`Capacity`
+network.  The :code:`demand_constraint` :eq:`Demand` ensures that the supply meets
+demand in every time slice.  For each process, the :code:`capacity_constraint` :eq:`Capacity`
 ensures that there is sufficient capacity to meet the optimal commodity flows across all
-time slices. Between processes, the :code:`CommodityBalance_constraint` :eq:`CommodityBalance`
+time slices. Between processes, the :code:`commodity_balance_constraint` :eq:`CommodityBalance`
 ensures that global commodity production across the energy system is sufficient to meet the
 endogenous demands for that commodity. Finally, the objective function :eq:`obj_invest` drives
 the model to minimize the system-wide cost of energy supply by optimizing the deployment and
 utilization of energy technologies across the system.
 
-One additional point regarding the model formulation. Technologies that
-produce constant annual output can be placed in the :code:`tech_annual` set.
-While not required, doing so improves computational performance by eliminating the
-season and time of day :code:`(s,d)` indices associated with these technologies.
-In order to ensure the model functions correctly with these simplified technologies,
-slightly different formulations of the capacity and commodity balance constraints
-are required. See the :code:`AnnualCommodityBalance_constraint` and
-:code:`CapacityAnnual_constraint` :eq:`CapacityAnnual` below for details.
+Two modelling features deserve additional explanation before the constraints are
+presented, as they significantly affect model size and must be used with care.
+
+Annual commodities (:math:`\text{C}^a`)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Commodities flagged :code:`a` (or :code:`wa` for waste variants) belong to
+:code:`commodity_annual` and are balanced only at the *period* level by the
+:code:`annual_commodity_balance_constraint`, rather than at each individual
+timeslice. This can produce very large reductions in model size: every
+technology that consumes or produces an annual commodity loses its
+:code:`(s,d)` balance constraints for that flow.
+
+The trade-off is that annual commodities *decouple timeslice synchronisation*
+between the technologies upstream and downstream of that commodity. This is
+often intentional — for example, connecting an annual tech to a non-annual tech
+via an annual intermediate commodity lets the two sides of the network operate
+at different temporal resolutions without explicit time-aggregation constraints.
+However, the same decoupling can silently break physical consistency when
+synchronisation matters. A clear example is electricity: modelling an
+electricity carrier as annual would allow a generator and a load to be
+"balanced" over the year even if the generator only runs in summer and the load
+is in winter, which is physically meaningless. Annual commodities should
+therefore be reserved for carriers where within-year timing genuinely does not
+matter (e.g., fuel stocks, annual hydrogen allocations), or where the explicit
+goal is to abstract away timeslice detail for a portion of the network.
+
+Annual technologies (:code:`tech_annual`)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Technologies that produce constant annual output can be placed in the
+:code:`tech_annual` set. While not required, doing so improves computational
+performance by eliminating the season and time of day :code:`(s,d)` indices
+associated with these technologies. Slightly different formulations of the
+capacity and commodity balance constraints are used for these technologies; see
+:code:`annual_commodity_balance_constraint` and :code:`capacity_annual_constraint`
+:eq:`CapacityAnnual` below for details.
+
+Annualised demand technologies
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Technologies in :code:`tech_annual` that serve demand commodities are a common and
+important case. The DSD already pins the share of annual output delivered to each
+timeslice, and the :code:`demand_activity_constraint` locks all processes serving
+the same demand to the same proportional dispatch — so per-:code:`(s,d)` flow
+variables carry no additional information and can be dropped, saving a large number
+of variables and constraints. The *physical* output to each timeslice is still
+implicitly shaped by the DSD:
+even though the solver sees only a single annual flow variable, the
+:code:`demand_constraint` scales it by the DSD fraction to determine what must be
+delivered in each :code:`(s,d)`. The installed capacity must therefore be sufficient
+to meet the peak-DSD timeslice, not merely the annual average. For technologies with
+a non-flat DSD this binding timeslice constraint is preserved via the full
+:code:`capacity_constraint` :eq:`Capacity`.
+
+When an annual process's output demands all have a *flatline* DSD — that is, demand is
+distributed uniformly across all timeslices — the timeslice-level capacity constraint
+reduces exactly to the annual constraint: every timeslice imposes the same limit, so
+only one representative constraint is needed. Temoa exploits this: processes in
+:code:`tech_annual` whose outputs are all either non-demand commodities or demand
+commodities with a flatline DSD are assigned only the single
+:code:`capacity_annual_constraint`, omitting the full set of :code:`(s,d)` constraints.
+For large models this can eliminate millions of redundant constraints, substantially
+reducing solver pre-processing time and memory use. Processes that feed any commodity
+present in :code:`commodity_dsd` (the set of demand commodities with a defined,
+non-trivial distribution) retain the timeslice-level constraints to preserve
+feasibility. The same applies when a process has a defined
+:code:`capacity_factor_tech` or :code:`capacity_factor_process`: even if the DSD is
+flat, those capacity factors can vary by season or time-of-day, and the resulting
+hourly capacity limits may still be structurally meaningful for the model.
+
+Constraints
+~~~~~~~~~~~
 
 The rest of this section defines each model constraint, with a rationale for
 existence.  We use the implementation-specific names for the constraints to
@@ -1255,7 +1320,7 @@ constraint is only defined for the exact indices that the modeler specified.
 
 Capacity-Defining Constraints
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-We begin with the :code:`Capacity_constraint` and :code:`CapacityAnnual_constraint`,
+We begin with the :code:`capacity_constraint` and :code:`capacity_annual_constraint`,
 which are particularly important because they define the relationship between
 installed capacity and allowable commodity flow.
 

temoa/components/capacity.py

@@ -204,10 +204,20 @@ def regional_exchange_capacity_constraint_indices(
 def capacity_annual_constraint_indices(
     model: TemoaModel,
 ) -> set[tuple[Region, Period, Technology, Vintage]]:
+    # Optimisation: We use the capacity annual constraint if the tech is annual
+    # so we only have one capacity constraint per process per period.
+    # Exceptions:
+    #   - Has capacity factor tech/process constraints (which could be time-varying)
+    #   - Feeds a defined DSD (so could have variable output due to DSD shape)
+    cft_rt = {(r, t) for r, _s, _d, t in model.capacity_factor_tech.sparse_keys()}
+    cfp_rtv = {(r, t, v) for r, _s, _d, t, v in model.capacity_factor_process.sparse_keys()}
     return {
         (r, p, t, v)
         for r, p, t, v in model.active_capacity_rptv
-        if t in model.tech_annual and t not in model.tech_demand
+        if t in model.tech_annual
+        and (r, t) not in cft_rt
+        and (r, t, v) not in cfp_rtv
+        and not any(o in model.commodity_dsd for o in model.process_outputs[r, p, t, v])
     }
 
 
@@ -217,9 +227,9 @@ def capacity_constraint_indices(
     return {
         (r, p, s, d, t, v)
         for r, p, t, v in model.active_capacity_rptv
+        if (r, p, t, v) not in model.capacity_annual_constraint_rptv
         for s in model.time_season
         for d in model.time_of_day
-        if t not in model.tech_annual or t in model.tech_demand
         if t not in model.tech_storage
     }
 

temoa/components/commodities.py

@@ -224,7 +224,6 @@ def demand_constraint(model: TemoaModel, r: Region, p: Period, dem: Commodity) -
     return expr
 
 
-# devnote: no longer needed
 def demand_activity_constraint(
     model: TemoaModel,
     r: Region,
@@ -760,6 +759,9 @@ def create_demands(model: TemoaModel) -> None:
             # Remaining checks would be caught by the validation of segment_fraction so skip
             continue
 
+        # Time-varying demands
+        model.commodity_dsd.add(dem)
+
         # If any subset of timeslices missing, inform the user. Not technically a problem
         # (will just default to zero) but likely not intended behaviour.
         if len(keys) != expected_key_length:

temoa/core/model.py

@@ -297,7 +297,7 @@ def __init__(
         #        process, which is currently incapable of reducing initializations on retirements.
         # Note2: I think this has been fixed but I can't tell what the problem was. Suspect
         #        it was the old storage_init constraint
-        self.tech_retirement = Set(within=self.tech_with_capacity)  # - M.tech_storage)
+        self.tech_retirement = Set(within=self.tech_with_capacity)
 
         self.validate_techs = BuildAction(rule=validate_tech_sets)
 
@@ -315,6 +315,7 @@ def __init__(
             initialize=self.commodity_carrier | self.commodity_emissions,
             validate=no_slash_or_pipe,
         )
+        self.commodity_dsd = Set()  # time-varying demands (not flatlined)
 
         ################################################
         #              Model Parameters                #
@@ -551,6 +552,9 @@ def __init__(
             default=capacity.get_default_capacity_factor,
         )
 
+        self.capacity_annual_constraint_rptv = Set(
+            dimen=4, initialize=capacity.capacity_annual_constraint_indices
+        )
         self.capacity_constraint_rpsdtv = Set(
             dimen=6, initialize=capacity.capacity_constraint_indices
         )
@@ -847,10 +851,6 @@ def __init__(
         self.capacity_constraint = Constraint(
             self.capacity_constraint_rpsdtv, rule=capacity.capacity_constraint
         )
-
-        self.capacity_annual_constraint_rptv = Set(
-            dimen=4, initialize=capacity.capacity_annual_constraint_indices
-        )
         self.capacity_annual_constraint = Constraint(
             self.capacity_annual_constraint_rptv, rule=capacity.capacity_annual_constraint
         )
@@ -886,7 +886,6 @@ def __init__(
             self.demand_constraint_rpc, rule=commodities.demand_constraint
         )
 
-        # devnote: testing a workaround
         self.demand_activity_constraint_rpsdtv_dem = Set(
             dimen=7, initialize=commodities.demand_activity_constraint_indices
         )
@@ -909,10 +908,6 @@ def __init__(
             rule=commodities.annual_commodity_balance_constraint,
         )
 
-        # M.ResourceExtractionConstraint = Constraint(
-        #     M.ResourceConstraint_rpr, rule=ResourceExtraction_constraint
-        # )
-
         self.baseload_diurnal_constraint_rpsdtv = Set(
             dimen=6, initialize=operations.baseload_diurnal_constraint_indices
         )

tests/testing_data/mediumville_sets.json

@@ -12,6 +12,7 @@
   "commodity_balance_constraint_rpsdc": "a1d06a815047ba07b42ced6dc65df3511301af2f30791b3094fb7f25ccdd4217",
   "commodity_carrier": "1b95135b3de272d1ecab3761bdb976f1e48c044dcb2b5b6d320bc19dae0038cd",
   "commodity_demand": "f0ecafabcdc5015dab09f4b23f3c742a27b536f1c5634057a902a9b9c4a741b8",
+  "commodity_dsd": "f0ecafabcdc5015dab09f4b23f3c742a27b536f1c5634057a902a9b9c4a741b8",
   "commodity_emissions": "849d0ab948e289950369de3a31be985d420635fa93525416da6d3ee9149019fc",
   "commodity_flex": "bfbe6f995c3a9fd19c92c7b6ffc526035a773a24d302c7eae9951b7327aef904",
   "commodity_physical": "7dd03855822916eb73171efbb05042535e0f89c26614139ed357a002b8a80fd1",

tests/testing_data/test_system_sets.json

@@ -12,6 +12,7 @@
   "commodity_balance_constraint_rpsdc": "4a46a0de6ec903639858a35c63fc46223f82ab1e057b20f4a1624eacb6652229",
   "commodity_carrier": "3508a787012b20265e9c97acebe2c39d6648b972e7bb4a6d42083e7e167b5c92",
   "commodity_demand": "6b657255c2baf8b6d4fc96f21cb21263bae7378eb11a9b9cf25a0d8aa83c6aca",
+  "commodity_dsd": "da13897924d72489d932aa77e6bd9c51b8e8d0e162adaeaf22e257367cde29a3",
   "commodity_emissions": "6aa406c61418fb1a0cc0d9f505fafcbf94e65a66cb5c5f1dc0afa8117b762979",
   "commodity_flex": "(empty)",
   "commodity_physical": "555367a723a957f166d9ec6c580ea46e958bd3a3f684aceee050e26ba2d423c6",

tests/testing_data/utopia_sets.json

@@ -12,6 +12,7 @@
   "commodity_balance_constraint_rpsdc": "79a5589e5d9b834ff2132fd4ce354a16e3a17529415245da9b9d8b13814ad382",
   "commodity_carrier": "23deb826164f35ff8adf73b9c64a4edb3efeeb51b683e14ea40f0771eba42eb8",
   "commodity_demand": "8d35b293b5cef0a6d0c6552029082eaec587db0bc5813783f24ab1aafe7491c8",
+  "commodity_dsd": "f0ecafabcdc5015dab09f4b23f3c742a27b536f1c5634057a902a9b9c4a741b8",
   "commodity_emissions": "cc2dd294fd87cfc0324fcd1d62eb647ff83bd89d7b5abd5dd5374debea96d91b",
   "commodity_flex": "(empty)",
   "commodity_physical": "3aab3aa4622d1545fe0f3cb8bd1a01fb9918632cff7186cfd49fa8575b73b2f3",