Repository navigation
Caching/memoization of fields #113
Description
@hellemo was working on improving model construction. A major issue is related to the extraction of input and output resources, especially of links. As outlined in #110, we access the field every single time period in model construction. this is especially an issue with respect to the Link resources as they are calculated through the connected nodes. Three solutions were proposed:
- explicitly compute the resources once, store them in a dictionary, and access them from said dictionary (Compute node and link resources once when building flows and couplings #110),
- use the package
Memoization.jlto precalculate the values when creating the model (Memoize link resources and node links with Memoization.jl #111), and - create a similar routine as in
Memoization.jlto reduce dependencies (Cache link resources and node links while a model is built #112).
I personally think it would be beneficial to discuss here which approach we want to utilize.
Compute node and link resources once when building flows and couplings (#110)
No pros or cons mentioned in the PR.
Memoize link resources and node links with Memoization.jl (#111)
Pros:
- No change to the formulation
- Benefits all extension packages
Cons:
- An extra dependency
- Global state
Cache link resources and node links while a model is built (#112)
Pros:
- keeps the formulation and should benefit extension packages as well
- no extra dependency
Cons:
- a bit more code/complexity to maintain in this package
Activity
JulStraus commentedon Oct 19, 2026
Personally speaking, I think it is something which is quite relevant for each subtype of
AbstractElementor comparable (i.e.,TransmissionMode). In all situations, it boils down to provide a fast approach for the functionsinputsandoutputsalthough, due to the design ofDirect, this is redirected tolink_res.I personally think that we can work with the approach outlined in #110. I do not necessarily think that we need memoization or caching although it could potentially speed it up slightly later. The only places where we need these functions with a lot of calculations are
variables_flowandconstraints_couple. For both functions, new methods have to be defined when users create new subtypes ofAbstractElement. What are your thoughts?Important
The approach with
link_resis internal withinEnergyModelsBaseas a fall back if no resources are specified. AllLinks in packages within this group have specific methods, e.g., (DhPipeorCapacityCostLink. This implies in my opinion that we should not focus specifically on this method, but instead oninputsandoutputs.hellemo commentedon Oct 21, 2026
For #110 it think the main would be:
pro:
con:
I think I lean towards #112 as it keeps the native formulation and gets the performance benefit without taking on additional dependencies, but any of three should be ok.
JulStraus commentedon Oct 21, 2026
I agree with respect to leaning towards #112. However, #112 does not yet include caching/memoizing for inputs and outputs of nodes. The question is whether any caching/memoizing of these parameters do make sense? Is accessing dictionaries faster than the field access function? Have you tested it out?
hellemo commentedon Oct 21, 2026
I think the main win is from avoiding the repeated calculation of the intersection, so probably less potential for gains on plain inputs/outputs, but can have a look.
EDIT: A quick test suggests dictionary lookups would not help for inputs/outputs, rather make it slower. It's indeed the intersection that is expensive.
espenbodal commentedon Oct 28, 2026
#112 looks like a good solution to me. I don't have much more to add than what already has been mentioned.
JulStraus commentedon Oct 28, 2026
@hellemo It seems that we align with respect to #112. I will do a short review on it now That implies we can close #110 and #111.