feat: powerio-dist, a multiconductor distribution crate with dss/PMD/BMOPF converters

[samtalki]

Scope

New workspace crate powerio-dist: a multiconductor distribution network model in wire coordinates with lossless conversion between OpenDSS .dss, PowerModelsDistribution ENGINEERING JSON ("PMD JSON"), and the draft JSON schema of the IEEE PES Task Force on Benchmarking Multiconductor OPF ("BMOPF JSON", frederikgeth/bmopf-report). The surface ships in all bindings: Rust (powerio_dist::{parse_file, parse_str, convert_file, convert_str}), the CLI (powerio convert feeder.dss --to pmd-json), Python (powerio.dist), and the C ABI (pio_dist_* behind a dist cargo feature of powerio-capi, additive, no ABI bump). PowerIO.jl is untouched; a follow-up issue tracks its dist surface.

Closes #2 (reframed per the discussion there). Starts #53.

Design

The canonical model (DistNetwork) is a network in wire coordinates: string bus ids, ordered string terminal names per bus, explicit grounding, terminal maps on every element, SI units and radians internally (BMOPF semantics, the most explicit of the three formats). The transmission Network is positive sequence and stays separate; the two hubs share conventions, not types. dss node 0 connections materialize as a perfectly grounded terminal, matching both PMD and the public BMOPF examples.

Fidelity contract. Same as the transmission crates, two tiers. Writing back to the source format echoes the retained source text byte for byte. Every cross format write regenerates from the typed model and reports each field the target cannot represent in Conversion::warnings; nothing drops silently. The one-shot converters (convert_file/convert_str, and their C and Python forms) merge the reader's parse warnings into the result, since there is no handle to query them from.

Explicit defaults with provenance. The dss reader materializes every OpenDSS class default into an explicit model value and records which fields were defaulted (DistNetwork::defaulted). The default table was read out of the OpenDSS source (epri-dev/OpenDSS-C class constructors) and then verified empirically against opendssdirect; two display strings in the source disagree with the constructor values (generator kW and PF), and the constructor wins. Consequence: BMOPF output is always fully explicit, and a degenerate circuit converts with a warning naming each materialized default.

Format conventions reproduced exactly. dss: TParser semantics (delimiters, quote pairs, ~/More continuations, Redirect/Compile, @var interpolation, RPN expressions, property name prefixes, | matrix rows, bus dotting), per the OpenDSS source. PMD JSON: column major matrices, null for nonfinite with field suffix restoration, uppercase enums, kV/kW scales, degrees, the ENGINEERING data_model marker; PMD parse_file accepts the emitted JSON unchanged (one documented exception below). BMOPF JSON: schema valid output, flat matrix keys, fully explicit values; every emitted document validates against the vendored schema in tests.

Conversion matrix

Generated into powerio-dist/docs/conversion-matrix.md by an ignored test. echo is the byte exact diagonal; ok is a canonical write that reparses to the common projection of the model; warning counts are fidelity losses the conversion reports individually.

fixture	source	→ dss	→ BMOPF	→ PMD
IEEE 13	dss	echo	ok (143 warn)	ok (44 warn)
IEEE 34	dss	echo	ok (374 warn)	ok (218 warn)
IEEE 123	dss	echo	ok (543 warn)	ok (266 warn)
single phase transformer	dss	echo	ok (10 warn)	ok (4 warn)
center tap transformer	dss	echo	ok (23 warn)	ok (12 warn)
wye delta transformer	dss	echo	ok (10 warn)	ok (4 warn)
delta wye transformer	dss	echo	ok (10 warn)	ok (4 warn)
switch states	dss	echo	ok (12 warn)	ok (4 warn)
four wire linecode	dss	echo	ok (25 warn)	ok (12 warn)
constructor defaults	dss	echo	ok (9 warn)	ok (2 warn)
ten conductor linecode	dss	echo	ok (25 warn)	ok (12 warn)
BMOPF IEEE 13 example	BMOPF	ok (1 warn)	echo	ok
BMOPF ENWL example	BMOPF	ok (1035 warn)	echo	ok (1019 warn)
PMD IEEE 13	PMD	ok (9 warn)	ok (152 warn)	echo
PMD four wire	PMD	ok (1 warn)	ok (8 warn)	echo

Diagonal byte identity and canonical idempotence (write(parse(write(parse(x)))) == write(parse(x))) hold on every fixture. Off diagonal round trips reproduce the typed model exactly on the fields both formats carry.

Oracle gates

• OpenDSS re-solve (tools/physics_check.py): every emitted .dss re-solves in opendssdirect (controls off, tolerance 1e-10) and is compared per node against the original. Where conversion fidelity is the only variable the worst deviation meets the 1e-8 pu bound (IEEE 13 canonical 2.2e-9, IEEE 13 through PMD 1.8e-9, IEEE 34 and 123 canonical at 1e-11, micro cases at machine epsilon). Cells above the bound carry a documented cause that the conversion also reports as warnings: BMOPF holds constant power loads only, the center tap collapse to two windings, no vminpu in the ENGINEERING model, PMD's 1e-7 ohm switch convention, regulator banking, and an engine asymmetry where OpenDSS seeds an untouched load's voltage base differently from a written kv= (isolated in a four line reproduction; affects any pipeline that materializes defaults).
• PMD acceptance (tools/pmd/pmdtool.jl): PMD parse_file accepts the emitted PMD JSON and agrees with PMD's own parse of the source .dss (with the documented stripped field list). Exception: PMD v0.16 parse_json cannot read ragged transformer connections (its _fix_arrays! hcats lists of lists), which affects PMD's own print_file output for the same cases identically; verified, and carved out of the gate with a comment.
• Schema validation: every strict BMOPF emission validates against the vendored schema; negative tests cover missing required fields, unknown fields, wrong enum case, wrong types, and negative bounds.

Feedback on the draft BMOPF schema

Observations from building a strict reader and writer against the public frederikgeth/bmopf-report repo, offered as input to the task force. The ten conductor fixture in this PR is the concrete artifact for the first item.

1. Matrix key patterns reject two digit conductor indices. ^R_series_\d_\d matches single digits only, so a 10x10 linecode emits R_series_10_10 keys the schema rejects; the pattern is also unanchored at the tail, so R_series_1_1junk validates. ^R_series_\d+_\d+$ fixes both (same for X_series, G_from/G_to, B_from/B_to, and the shunt G_i_j/B_i_j keys).
2. s_rating units. Documented as kVA, used as VA (their issue #1). The example values only make sense as VA.
3. Optionality markers (their issue Add OpenDSS .dss parser #2): per field required/optional annotations would remove guessing for implementers.
4. Stale $id. The schema's $id points at an earlier repository URL.
5. Counterexamples predate the matrix key rename. The invalid example networks still carry the old concatenated key spelling, so they no longer test what they were written to test.
6. additionalProperties: false vs extensibility. Schema valid output cannot carry any sidecar data, so every conversion into BMOPF is lossy by construction (this PR keeps dropped fields in the in-memory model so round trips through BMOPF stay lossless, and reports each drop). Whether the schema wants an explicit extension mechanism is an open question worth settling before tooling proliferates.
7. switch.i_max items are unconstrained while linecode.i_max items must be nonnegative; a negative switch ampacity validates.
8. Transformer impedances in example_ieee13.json. The substation transformer's series impedance does not pass dimensional analysis against the prose "ohms on the wye side": ohms on the 4.16 kV wye side gives 2.77e-4 for the entered 0.008 percent XHL, while the example holds 2.116e-4, which works out to the delta side base divided by 1000. The single phase regulators show a similar factor. Possibly a kV/V or kVA/VA slip in the example generation; the OpenDSS re-solve gate in this PR is consistent with the prose interpretation.
9. Fixed generation encoding. Generators carry bounds and cost only, with no dispatch setpoint; this PR maps fixed injections to pinned p_min = p_max. A normative statement on whether pinned bounds are the intended encoding would help.
10. Wye-wye three phase decomposition. The example decomposes XFM1 into per phase single_phase units; this convention is currently inferable only from the example and deserves a sentence in the spec.
11. Grounding in example_ieee13.json. The example marks the highest numbered terminal of every bus perfectly grounded, including three wire buses where that terminal is a phase (bus 632, terminals ["1","2","3"], grounded ["3"]). This PR grounds only where OpenDSS grounds (node 0 connections).
12. b_fr/g_fr split vs a single b/g on lines and shunts remains an open modeling question; either answer is implementable, but the choice affects every converter.

Deferred, with follow-up issues

• dss classes outside phase A (reactor, pvsystem, storage, loadshape and time series, fault, and others) parse into the raw layer and convert with an explicit unsupported warning; typed support is a follow-up issue.
• PowerIO.jl bindings for the dist C ABI surface: follow-up issue on the PowerIO.jl side workflow.
• RegControl converts to a fixed tap with a warning; OLTC modeling is out of scope for the BMOPF spec itself.

Release notes for the maintainer

• The release binary workflow currently builds powerio-capi --features arrow for the five platform tarballs; shipping the dist surface to Julia and other C consumers means adding dist to that feature list at the next release. Decide alongside whether powerio-dist joins the crates.io publish list.
• Python wheels need no workflow change: the wheel depends on powerio-dist unconditionally.

🤖 Generated with Claude Code

[samtalki]

summoning @deakinmt @amritanshup7 @frederikgeth @MartaVanin for their wisdom

[deakinmt]

1. Matrix key patterns reject two digit conductor indices. ^R_series_\d_\d matches single digits only, so a 10x10 linecode emits R_series_10_10 keys the schema rejects; the pattern is also unanchored at the tail, so R_series_1_1junk validates. ^R_series_\d+_\d+$ fixes both (same for X_series, G_from/G_to, B_from/B_to, and the shunt G_i_j/B_i_j keys).

Updated and pushed to the experimental branch here

2. s_rating units. Documented as kVA, used as VA (their issue #1). The example values only make sense as VA.

Updated and pushed that that same branch

3. Optionality markers (their issue Add OpenDSS .dss parser #2): per field required/optional annotations would remove guessing for implementers.

Also updated

4. Stale $id. The schema's $id points at an earlier repository URL.

Also updated (although as it is draft so not taking too seriously so I am not for example point to the branch)

5. Counterexamples predate the matrix key rename. The invalid example networks still carry the old concatenated key spelling, so they no longer test what they were written to test.

I assume it is referring to the models in the (private) OpendssToPmd repo, yes this is correct that it would be good to get some counterexamples which function as counterexamples (todo)

6. additionalProperties: false vs extensibility. Schema valid output cannot carry any sidecar data, so every conversion into BMOPF is lossy by construction (this PR keeps dropped fields in the in-memory model so round trips through BMOPF stay lossless, and reports each drop). Whether the schema wants an explicit extension mechanism is an open question worth settling before tooling proliferates.

To discuss (even the Optional parameters at present need confirming)

7. switch.i_max items are unconstrained while linecode.i_max items must be nonnegative; a negative switch ampacity validates.

Updated

8. Transformer impedances in example_ieee13.json. The substation transformer's series impedance does not pass dimensional analysis against the prose "ohms on the wye side": ohms on the 4.16 kV wye side gives 2.77e-4 for the entered 0.008 percent XHL, while the example holds 2.116e-4, which works out to the delta side base divided by 1000.

Transformer impedances are on the todo list / are derived from a (non-standard) ieee 13 bus pmd case file

9. The single phase regulators show a similar factor. Possibly a kV/V or kVA/VA slip in the example generation; the OpenDSS re-solve gate in this PR is consistent with the prose interpretation.

Transformer impedances are on the todo list / are derived from a (non-standard) ieee 13 bus pmd case file

10. Fixed generation encoding. Generators carry bounds and cost only, with no dispatch setpoint; this PR maps fixed injections to pinned p_min = p_max. A normative statement on whether pinned bounds are the intended encoding would help.

The section of the spec where this is covered has been clarified to make the point that generators with fixed P, Q make sense to be captured by a load element with negative value for the powers.

11. Wye-wye three phase decomposition. The example decomposes XFM1 into per phase single_phase units; this convention is currently inferable only from the example and deserves a sentence in the spec.

We won't have IEEE 13 node as an example in the long-run due to the issues around licensing, and probably we will support wye-wye in the next spec version, so this can be fixed but not super high priority.

12. Grounding in example_ieee13.json. The example marks the highest numbered terminal of every bus perfectly grounded, including three wire buses where that terminal is a phase (bus 632, terminals ["1","2","3"], grounded ["3"]). This PR grounds only where OpenDSS grounds (node 0 connections).

There is are no '0' nodes in the ieee case example shared. I have updated this so that only nodes '4' are perfectly grounded (in the PMD file I am taking these from, it appears ambiguous which nodes are perfectly grounded).

NB: I am wondering if we explicitly park the ieee 13 bus case as one of these examples as long-term we are not going to be able to support it anyway due to the licence issue.

13. b_fr/g_fr split vs a single b/g on lines and shunts remains an open modeling question; either answer is implementable, but the choice affects every converter.

Happy either way here - not sure there are that many use-cases exist for different from / to values of these parameters, so perhaps we combine.

NB we spell out *_from in bmopf rather than *_fr (change from pmd).

[samtalki]

Naming note for when this revives, following the taxonomy now codified in docs/languages.md (#93): BMOPF and PMD engineering data are single-document JSON formats, so they belong to the parse_*/to_format family through the dist hub, like the five classical formats. The read_*/write_* verbs are reserved for filesystem datasets (multi-file directories, e.g. the gridfm Parquet tables) — read_gridfm is the only reader there because a directory can't go through parse_str. Keeping #82's converters on the hub verbs keeps all four language bindings consistent.

🤖 Generated with Claude Code

[samtalki]

Thanks for the quick turnaround on all of this @deakinmt .

• 1-4, 7, 12: tracked in BMOPF: update schema fixtures after upstream schema merge #96 .
• I set it up so that when june26-report-updates merges we will now re-vendor the schema and examples in one go. Then, the ten conductor case flips from rejected passing.
• 10: we'll switch to the negative load encoding in the same update.
• 13: good catch, we already export *_from.
• 5: regenerated counterexamples will go straight into our negative test suite.
• 6: agreed, worth settling early. Happy to write up what conversions actually need from an extension mechanism if that helps the discussion.

On licensing:

• The PR now has per directory license files.
• The IEEE feeders keep the BSD notice from the OpenDSS distribution, our own micro cases are CC BY 4.0.
• The bmopf directory now documents data lineage and will follow whatever license the task force publishes.

[frederikgeth]

Awesome, thanks. This is very exciting progress!

[frederikgeth]

Here's some notes on transformer models and load models I generated a while back, to serve as extensions of the spec.
transformer_integrated.pdf

Haven't been critically reviewed yet, so feel free to ask Fable to do that first.

update: new load model note
load_model_note_v2.pdf

[frederikgeth]

@samtalki Do you have any BMOPF JSONs for the IEEE test cases? Then I can use those to test my Julia parser and data (quality) profiler?

[frederikgeth]

Confirming that based on preliminary testing in frederikgeth/BMOPFTools.jl#11, there are no significant mismatches between parsing through PMD and through Powerio. (And the shunt issue should be resolved when merging the changes on main back into this branch).

For now I recommend to stick with a raw conversion from OpenDSS into the BMOPF JSON. We need to do inference on the OpenDSS data to tag the neutral wires/terminals and tag things like voltage regulators. Let me first work through that in my package. We can port / move that functionality down the line from BMFOPFTools to PowerIO.

[frederikgeth]

by the way, in case you are not aware, there's more test cases hiding here: https://github.com/tshort/OpenDSS/tree/master/Distrib/IEEETestCases
and also in the parent folder there