Repository navigation
Conversation
| + ( | ||
| (pd.Timedelta('5D') if sw.GSw_HourlyType == 'wek' else pd.Timedelta('1D')) | ||
| - pd.Timedelta('1h') | ||
| - pd.Timedelta('1H') |
There was a problem hiding this comment.
The Pacific run test on the github actions failed with this error (see log output in the "run ReEDS model" step here):
solve.py | 2026-08-27 22:52:34 | ERROR | ValueError: Invalid frequency: 1H. Failed to parse with error message: ValueError("Invalid frequency: H. Failed to parse with error message: KeyError('H'). Did you mean h?")
Given that I think we should revert these to '1h' to '1H' changes. Was this change working for your runs? If it was, were you testing with a clean reeds environment?
There was a problem hiding this comment.
yes I had the same issue when testing, fixed and pushed the change.
|
|
||
| ### Check all stress criteria; for regions that fail, add new stress periods |
There was a problem hiding this comment.
Can you revert this one too?
| ### Check all stress criteria; for regions that fail, add new stress periods | |
| ### Check all stress criteria; for regions that fail, add new stress periods |
| write_surplus=False, | ||
| write_energy=False, | ||
| write_shortfall_samples=False, | ||
| write_shortfall_samples_totals=True, |
There was a problem hiding this comment.
It sounds like PRAS.ShortfallSamples() increases memory use within PRAS, but NatLabRockies/PRAS#109 fixes it. Can we check the PRAS memory use on a full-US run on this branch and on main before merging (I think Kodi has done this before)? If it goes up a lot, we should probably wait to merge this PR until we can switch to a new PRAS release that includes this PRAS PR.
There was a problem hiding this comment.
It seems like that PRAS PR has been merged, but I'd still be interested in a check on the memory usage before merging.
bsergi
left a comment
bsergi
left a comment
There was a problem hiding this comment.
Just the two last items to address (the 1H --> 1h fix and checking PRAS memory use) but otherwise looks good to me
| write_surplus=False, | ||
| write_energy=False, | ||
| write_shortfall_samples=False, | ||
| write_shortfall_samples_totals=True, |
There was a problem hiding this comment.
It seems like that PRAS PR has been merged, but I'd still be interested in a check on the memory usage before merging.
Co-authored-by: Patrick Brown <25125211+patrickbrown4@users.noreply.github.com>
This PR continues the work and review discussion from #160. The updated implementation was moved to an upstream ReEDS branch so the GitHub Actions tests can run.
Most of the review comments from #160 were addressed in
bs/cvar. This branch is based on that work and adds the remaining dependency changes: it moves PRAS to 0.9 and reads the per-sample shortfall totals from the regularShortfallresult (see below). It is up to date with main.Comparison with main
I ran
USA_defaultson this branch and on the matching main commit (eb75a6bc). Both runs used the same Julia environment (PRAS 0.9).The two runs produced identical model results in all years: capacity, generation, new capacity, retirements, curtailment, transmission, electricity prices, present value of system cost, and NEUE by region. The difference plots are zero everywhere. The additional CVAR/NCVAR reporting does not change the ReEDS solution.
seff)Runtime is the same within normal run-to-run variation.
The full comparison can be found here:
PRAS 0.9 and memory
Earlier versions of this branch requested
PRAS.ShortfallSamples()to get per-sample shortfall totals for CVAR. That result keeps a region × hour × sample array in memory, which made full runs with many PRAS samples very memory-heavy.PRAS 0.9 adds per-sample totals to the regular
Shortfallresult (shortfall_samples,shortfall_region_samples). This branch now reads the CVAR inputs fromShortfall, and only requestsShortfallSampleswhen the hourly sample output is written (GSw_PRM_UpdateMethod > 1). The-shortfall_totals_by_sample.h5file has the same format as before, so the Python CVAR calculation is unchanged.Effect on full runs with 500 PRAS samples: peak memory was about 510–560 GB with the previous implementation and about 110–120 GB with this branch.
Illustrative CVAR/NCVAR diagnostics
The figures below illustrate the new CVAR/NCVAR reporting using the
USA_defaultstest case. The run used 100 PRAS samples andGSw_PRM_CVARalpha = 0.95, so each CVAR estimate is the mean of the five largest sample-total shortfalls. With only five samples in the tail, these estimates are noisy. The figures are meant to check the calculation, not to support conclusions about tail risk; that needs many more samples.Each PRAS sample covers the full 15-weather-year resource adequacy period. CVAR is calculated on the 15-year sample totals and then divided by 15 to report it per year. This is not the same as the CVAR of annual shortfall: a bad weather year is averaged with the other years in the same sample.
The national results show large variation in tail shortfall across solve years. In the regional results, a small number of regions have most of the tail shortfall, with ND_MISO having the largest regional CVAR in several of the years shown. Regional CVARs should not be added up: the worst samples can differ by region, so the national CVAR can be smaller than the sum of the regional CVARs.
National CVAR by solve year. CVAR95 of the 15-year sample-total shortfall, divided by 15. With 100 samples, the α = 0.95 tail contains five samples.
Regional CVAR by solve year. Bubble size shows CVAR95 (per year) at the ReEDS region level. Only positive region–year values are shown.
USA shortfall exceedance curve for 2029. The dashed vertical line marks the 5% tail boundary. VaR95 is the shortfall at that boundary; CVAR95 is the mean of the five samples in the upper tail.