Consume PowerIO directly; drop the ParsedCase layer#60
Merged
Conversation
PowerIO.to_powerdata already returns normalized, per-unit, filtered, slack-inferred, cost-rescaled network data, which parser.jl reimplemented as ParsedCase. Build DCNetwork/ACNetwork straight from a PowerIO.Network instead, keeping only PowerDiff's OPF modeling: polynomial cost interpretation, a finite flow-limit fallback when rate_a is 0, default angle-difference bounds, and rejection of storage/HVDC records PowerDiff does not model. - parser.jl: parse_file/parse_matpower return a PowerIO.Network; _network_data builds the network tables; remove ParsedCase/ParsedBus/... and the old normalization helpers. - DCNetwork/ACNetwork/DCOPFProblem/ACOPFProblem/calc_demand_vector take a PowerIO.Network or the network-tables NamedTuple; drop the ParsedCase methods. - IDMapping no longer tracks per-load/shunt ids (loads and shunts are aggregated per bus by to_powerdata). - Read generator costs from PowerIO's raw records: to_powerdata mangles costs declared with ncost>3 (e.g. MATPOWER case14, a quadratic padded to ncost=5). - Finalize PowerIO as a registered dependency: drop the [sources] git pin and set [compat] PowerIO = "0.1". - Migrate the test suite and the IPP experiment off ParsedCase; examples already used the parse_file -> network constructor path. Co-authored-by: Cameron Khanpour <99142483+cameronkhanpour@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PowerIO is registered and public, so CI no longer needs the private-repo probe. - CI.yml / Documentation.yml / Benchmark.yml: drop the POWERIO_JL_TOKEN env, the PowerIO access probe/skip/configure steps, the availability gates, and JULIA_PKG_USE_CLI_GIT. The test and build (docs) job names are preserved. - docs: drop the removed ParsedCase/Parsed* @docs entries from the API reference and rewrite the PowerIO integration page for the direct to_powerdata path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Member
Author
|
Filed the three |
get_path resolves PowerDiff's bundled PGLib artifact — a data-library concern, not parsing. Pulling it (and the LazyArtifacts dependency it needs for `artifact"..."`) into src/artifacts.jl leaves parser.jl as just the PowerIO entry points and the network-data adapter. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PowerIO 0.1.3 makes to_powerdata a complete data layer: source bus ids on bus_i, an inferred reference (type == 3), and correct polynomial costs (ncost > 3 no longer mangled, so a quadratic padded to ncost=5 keeps its linear term). Bump compat to 0.1.3 and remove the three workarounds _network_data carried for the old gaps: - read gen cost straight from to_powerdata's rows (model_poly/n/c, already per-unit and leading-zero collapsed) instead of re-reading raw costs from PowerIO.generators and rescaling; drop the _cost_tuple helper - drop the biggest-pmax reference promotion; the reference now comes from to_powerdata (type == 3) - drop the try/catch around to_powerdata, which now throws ArgumentError on malformed input itself Kept as consumer-side solver prep (PowerIO leaves these to the caller): the rate_a == 0 finite-limit fallback, the +/-60 deg default angle bounds, rejection of storage/HVDC and PWL/higher-than-quadratic costs, and the dense gen.bus/f_bus/t_bus -> source id mapping. Rename src/parser.jl to src/network_data.jl (it builds network tables, it is not a parser) and move `using PowerIO` to the top of PowerDiff.jl. DCNetwork/ACNetwork field values are unchanged: a before/after field dump over pglib case5/14/30 and a non-basic-id case (ids 1,2,3,4,10) is identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The MATPOWER parse wrappers (parse_file/parse_matpower), the _network_data adapter, and its solver-prep helpers (_poly_cost, _fallback_rate_a, _normalize_angle_bounds) now live in types/dc_network.jl rather than a separate file. dc_network.jl is included before ac_network.jl, so ACNetwork and the OPF problem constructors reuse the shared _network_data. This removes the standalone src/network_data.jl, which was just the old parser.jl renamed. Pure relocation: DCNetwork/ACNetwork field values are unchanged (before/after field dump over pglib case5/14/30 and a non-basic-id case is identical), the test suite passes, and docs build with parse_* docstrings resolving from dc_network.jl. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Correctness:
- _poly_cost: accept generators with no gencost row (gencost is optional in
MATPOWER). PowerIO returns model_poly=false, n=0 for them; treat as cost-free
instead of throwing, which had broken even power-flow-only construction. PWL
(model_poly=false, n>0) is still rejected, and to_powerdata rejects
higher-than-quadratic itself.
Consistency:
- reject storage from the raw network (PowerIO.storage(net)) like HVDC, so both
guards see out-of-service records; to_powerdata's filtered pd.storage dropped
them, silently accepting a file that declared disabled storage.
Efficiency / clarity:
- _poly_cost reads to_powerdata's right-aligned (cq, cl, cc) directly instead of
collect/slice/popfirst per generator.
- build the branch table with a concrete-eltype comprehension + _branch_row
helper instead of an abstract Vector{NamedTuple} + push!.
- drop the duplicate per-bus vmax array; _branch_row indexes the buses table.
- parse_matpower_struct no longer advertises kwargs... it cannot forward.
Docs:
- fix stale claims that parse_file returns "PowerDiff's typed representation"
(it returns a PowerIO.Network) in README, getting-started, index, advanced.
- powerio-integration.md: costs come from to_powerdata, not raw records, and the
reference bus comes from to_powerdata, not a largest-generator promotion.
No behavior change for valid inputs: DCNetwork/ACNetwork field values are
identical (before/after field dump over pglib case5/14/30 and a non-basic-id
case), and the test suite passes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dropping the ParsedCase layer folded shunts into per-bus gs/bs (which the network constructors consume) but removed the separate data.shunt records. Add a `shunt` field back to the _network_data tables: one (; index, shunt_bus, gs, bs) record per bus with a nonzero shunt admittance, derived from the per-bus values to_powerdata already aggregates (no raw re-read). DCNetwork/ACNetwork are unchanged (field dump byte-identical); the inline parser test asserts the restored shunt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
samtalki
added a commit
that referenced
this pull request
Jun 15, 2026
* init (cherry picked from commit df6c11d) * restore * Finish Exa backend construction and parity coverage * Adapt Exa backend to the new ExaModels API * Complete typed MATPOWER backends * Added ExaModels.jl Dependency Item to README * Address PR 48 review feedback * Quality fixes: solve-status warnings, demand indexing, dead code, docs - Gate the AC/Exa `:acceptable` solve warning on `silence()` and state the sensitivity consequence; mirror the DC backend's status messages and add an `:unbounded` branch. - Fix `calc_demand_vector(::ParsedCase)` to index by the sorted `IDMapping` instead of file order, so loads align when bus IDs are unsorted; reuse the caller's `id_map` and add a regression test. - Remove the unused `ACNetwork.i_max` field and the dead `_branch_data_dict`. - Refresh stale PowerModels docstrings (KKT flow equations, `branch_data`, `ACNetwork(Y)`) and document `DCOPFSolution.eta_ref`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Use PowerIO as the parser layer * Handle private PowerIO.jl CI dependency * Add PowerIO compat bound and list it in README dependencies PowerIO = "0.0.1" rides alongside the [sources] pin so the bound is already right when the pin is dropped after registration. README dependency list now names the parser layer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Track PowerIO.jl main and widen compat to 0.1 The release-prep branch merged; main now carries PowerIO.jl 0.1.x with the powerio v0.2.1 binary artifact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Consume PowerIO directly; drop the ParsedCase layer (#60) * Consume PowerIO directly; drop the ParsedCase layer PowerIO.to_powerdata already returns normalized, per-unit, filtered, slack-inferred, cost-rescaled network data, which parser.jl reimplemented as ParsedCase. Build DCNetwork/ACNetwork straight from a PowerIO.Network instead, keeping only PowerDiff's OPF modeling: polynomial cost interpretation, a finite flow-limit fallback when rate_a is 0, default angle-difference bounds, and rejection of storage/HVDC records PowerDiff does not model. - parser.jl: parse_file/parse_matpower return a PowerIO.Network; _network_data builds the network tables; remove ParsedCase/ParsedBus/... and the old normalization helpers. - DCNetwork/ACNetwork/DCOPFProblem/ACOPFProblem/calc_demand_vector take a PowerIO.Network or the network-tables NamedTuple; drop the ParsedCase methods. - IDMapping no longer tracks per-load/shunt ids (loads and shunts are aggregated per bus by to_powerdata). - Read generator costs from PowerIO's raw records: to_powerdata mangles costs declared with ncost>3 (e.g. MATPOWER case14, a quadratic padded to ncost=5). - Finalize PowerIO as a registered dependency: drop the [sources] git pin and set [compat] PowerIO = "0.1". - Migrate the test suite and the IPP experiment off ParsedCase; examples already used the parse_file -> network constructor path. Co-authored-by: Cameron Khanpour <99142483+cameronkhanpour@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Resolve PowerIO from the registry in CI; refresh docs PowerIO is registered and public, so CI no longer needs the private-repo probe. - CI.yml / Documentation.yml / Benchmark.yml: drop the POWERIO_JL_TOKEN env, the PowerIO access probe/skip/configure steps, the availability gates, and JULIA_PKG_USE_CLI_GIT. The test and build (docs) job names are preserved. - docs: drop the removed ParsedCase/Parsed* @docs entries from the API reference and rewrite the PowerIO integration page for the direct to_powerdata path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Move get_path (and LazyArtifacts) out of parser.jl get_path resolves PowerDiff's bundled PGLib artifact — a data-library concern, not parsing. Pulling it (and the LazyArtifacts dependency it needs for `artifact"..."`) into src/artifacts.jl leaves parser.jl as just the PowerIO entry points and the network-data adapter. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Drop PowerIO workarounds; consume to_powerdata directly on 0.1.3 PowerIO 0.1.3 makes to_powerdata a complete data layer: source bus ids on bus_i, an inferred reference (type == 3), and correct polynomial costs (ncost > 3 no longer mangled, so a quadratic padded to ncost=5 keeps its linear term). Bump compat to 0.1.3 and remove the three workarounds _network_data carried for the old gaps: - read gen cost straight from to_powerdata's rows (model_poly/n/c, already per-unit and leading-zero collapsed) instead of re-reading raw costs from PowerIO.generators and rescaling; drop the _cost_tuple helper - drop the biggest-pmax reference promotion; the reference now comes from to_powerdata (type == 3) - drop the try/catch around to_powerdata, which now throws ArgumentError on malformed input itself Kept as consumer-side solver prep (PowerIO leaves these to the caller): the rate_a == 0 finite-limit fallback, the +/-60 deg default angle bounds, rejection of storage/HVDC and PWL/higher-than-quadratic costs, and the dense gen.bus/f_bus/t_bus -> source id mapping. Rename src/parser.jl to src/network_data.jl (it builds network tables, it is not a parser) and move `using PowerIO` to the top of PowerDiff.jl. DCNetwork/ACNetwork field values are unchanged: a before/after field dump over pglib case5/14/30 and a non-basic-id case (ids 1,2,3,4,10) is identical. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Fold network_data.jl into the DCNetwork construction path The MATPOWER parse wrappers (parse_file/parse_matpower), the _network_data adapter, and its solver-prep helpers (_poly_cost, _fallback_rate_a, _normalize_angle_bounds) now live in types/dc_network.jl rather than a separate file. dc_network.jl is included before ac_network.jl, so ACNetwork and the OPF problem constructors reuse the shared _network_data. This removes the standalone src/network_data.jl, which was just the old parser.jl renamed. Pure relocation: DCNetwork/ACNetwork field values are unchanged (before/after field dump over pglib case5/14/30 and a non-basic-id case is identical), the test suite passes, and docs build with parse_* docstrings resolving from dc_network.jl. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Address code-review findings on the PowerIO adapter Correctness: - _poly_cost: accept generators with no gencost row (gencost is optional in MATPOWER). PowerIO returns model_poly=false, n=0 for them; treat as cost-free instead of throwing, which had broken even power-flow-only construction. PWL (model_poly=false, n>0) is still rejected, and to_powerdata rejects higher-than-quadratic itself. Consistency: - reject storage from the raw network (PowerIO.storage(net)) like HVDC, so both guards see out-of-service records; to_powerdata's filtered pd.storage dropped them, silently accepting a file that declared disabled storage. Efficiency / clarity: - _poly_cost reads to_powerdata's right-aligned (cq, cl, cc) directly instead of collect/slice/popfirst per generator. - build the branch table with a concrete-eltype comprehension + _branch_row helper instead of an abstract Vector{NamedTuple} + push!. - drop the duplicate per-bus vmax array; _branch_row indexes the buses table. - parse_matpower_struct no longer advertises kwargs... it cannot forward. Docs: - fix stale claims that parse_file returns "PowerDiff's typed representation" (it returns a PowerIO.Network) in README, getting-started, index, advanced. - powerio-integration.md: costs come from to_powerdata, not raw records, and the reference bus comes from to_powerdata, not a largest-generator promotion. No behavior change for valid inputs: DCNetwork/ACNetwork field values are identical (before/after field dump over pglib case5/14/30 and a non-basic-id case), and the test suite passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Re-expose shunts as a data.shunt table Dropping the ParsedCase layer folded shunts into per-bus gs/bs (which the network constructors consume) but removed the separate data.shunt records. Add a `shunt` field back to the _network_data tables: one (; index, shunt_bus, gs, bs) record per bus with a nonzero shunt admittance, derived from the per-bus values to_powerdata already aggregates (no raw re-read). DCNetwork/ACNetwork are unchanged (field dump byte-identical); the inline parser test asserts the restored shunt. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Cameron Khanpour <99142483+cameronkhanpour@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Address PR #48 review (Copilot + cameronkhanpour) - benchmark/benchmarks.jl: drop the removed `isdefined(:ParsedCase)` branches and parse with `PowerDiff.parse_file` directly. ParsedCase no longer exists, so the old fallback resolved to `PM.make_basic_network(...)` and `DCOPFProblem(::Dict)` raised a bare MethodError. - Add DCNetwork/DCOPFProblem/DCPowerFlowState(::Dict) rejection methods so the DC side gives the same migration hint as the AC side instead of a MethodError. - _poly_cost: zero-pad model-2 gencosts with fewer than 3 terms instead of indexing g.c[3] (BoundsError guard). - Guard marginal_cost_ub against generator-free networks. - Warn on reference-bus fallback when no type-3 bus is present (AC + DC). - Refresh stale docstrings: flatten_variables eta now packs sol.eta_ref; test/common.jl reflects PowerIO.Network. Document the symmetric branch-shunt and the gen/branch dense-index ID contracts. - Migrate test/mwe_unified.jl off the removed dictionary API. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Support PowerIO parser formats * Handle RTS smoke cost conversion * Honor explicit AC slack override --------- Co-authored-by: ckhanpour3 <ckhanpour3@gatech.edu> Co-authored-by: samtalki <10187005+samtalki@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Cameron Khanpour <99142483+cameronkhanpour@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Stacks on
mk/backends(#48).PowerIO.to_powerdataalready returns normalized, per-unit, filtered, slack-inferred, cost-rescaled network data — whichparser.jlreimplemented asParsedCase. This buildsDCNetwork/ACNetworkstraight from aPowerIO.Networkand deletes the duplicated layer (net −191 lines).What stays in PowerDiff is only OPF-solver modeling, not parsing:
rate_a == 0Key changes
parse_file/parse_matpowerreturn aPowerIO.Network._network_data(net)builds the network tables.DCNetwork/ACNetwork/DCOPFProblem/ACOPFProblem/calc_demand_vectoraccept aPowerIO.Networkor the network-tablesNamedTuple.ParsedCase/ParsedBus/… and the old normalization helpers are removed.IDMappingdrops per-load/shunt ids (loads/shunts are aggregated per bus byto_powerdata).to_powerdatamangles costs withncost>3.[sources]git pin, set[compat] PowerIO = "0.1".ParsedCase; examples already used theparse_file→ constructor path.Supersedes #59 (the PowerIO hard-dependency declaration), which is folded in here; credited to @cameronkhanpour via the commit co-author trailer.
PowerIO issues found (to file upstream)
to_powerdatais not fully drop-in; PowerDiff works around these, but they are real upstream gaps:ncost>3are mangled — it keeps the 3 highest-degree coefficients with the wrong exponents and drops the rest, so a quadratic padded toncost=5(e.g. MATPOWER case14) loses its linear term. Worked around by reading raw cost coefficients fromPowerIO.generators.to_powerdataonly reassigns the slack when an explicit REF was demoted. PowerDiff promotes the largest generator's bus.Float64(::Nothing)instead of a clean error — wrapped as anArgumentError.Verification (zero regressions)
Pkg.test()passes against registered PowerIO 0.1.2 (parser path/IO parity ran, 541/541; MATPOWER semantics 29/29; non-basic 183/183).DCNetwork/ACNetworkfields are byte-identical to the pre-refactorParsedCasepath for pglib case5/14/30 (captured baseline diff).examples/interactive_repl.jlruns end-to-end; the IPP experiment is converted to the new API and parses.🤖 Generated with Claude Code