refactor(ledger): give each ledger version its own wrapper - #2059
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 294f681d57
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| } | ||
|
|
||
| #[derive(Debug, Encode, Decode, DecodeWithMemTracking, Clone, TypeInfo, PalletError, PartialEq)] | ||
| pub enum LedgerApiError { |
There was a problem hiding this comment.
Preserve the error-encoding invariant across copies
When either independently editable copy gains or reorders a version-specific error variant, the SCALE discriminants can diverge, but host_api/ledger_9.rs::as_ledger_9_error still converts v8 errors by encoding and decoding them as this v9 enum. The existing test samples only a few variants and does not cover nested transaction errors, so historical v8 reads can silently return the wrong v9 error or fall back to HostApiError. Keep the encoding-bearing error definitions shared, use an explicit conversion, or exhaustively enforce their compatibility.
Useful? React with 👍 / 👎.
This comment has been minimized.
This comment has been minimized.
|
/bot rebuild-metadata |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9425e310a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 60fb008a52
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| @@ -0,0 +1,67 @@ | |||
| #[cfg(feature = "std")] | |||
There was a problem hiding this comment.
Add the required license header
This newly added Rust source file starts directly with a cfg attribute and omits the repository’s mandatory license header. Prepend the contents of LICENSE_HEADER.txt so the new source file complies with the project’s licensing requirements.
AGENTS.md reference: AGENTS.md:L239-L241
Useful? React with 👍 / 👎.
…pper
`ledger/src/versions/common/**` was compiled twice through a
`#[path = "versions"]` module-parameterization trick — once bound to the
ledger-8 crates, once to the ledger-9 crates — with `super::` resolving
differently in each instantiation. A reader of a file under `versions/common/`
could not tell which ledger crate `mn_ledger_local` referred to, and an edit
meant for one generation silently applied to both.
Each generation now owns its files under `ledger/src/ledger_8/` and
`ledger/src/ledger_9/`, with absolute `crate::ledger_N::…` imports so every
file states the generation it binds. `ledger/src/common/` — the SCALE types
crossing the runtime/client interface, version-independent by design and
compiled once — becomes `ledger/src/boundary/`, so no folder is called
`common` any more. All `#[path]` and `#[allow(clippy::duplicate_mod)]`
scaffolding is gone from `lib.rs`.
No behaviour change: rustc already emitted both instantiations, so this only
makes the existing duplication visible in the source tree. There was no
non-test version branching in the shared half to reinterpret, public paths
(`ledger_8::…`, `ledger_9::…`, `latest::…`, `types::…`, `host_api::…`) are
unchanged, and the only edits outside `ledger/src/{lib.rs,ledger_8,ledger_9}`
are two `common::` -> `boundary::` import lines in `host_api/`.
Normalising the version path away, `diff -r src/ledger_8 src/ledger_9` leaves
exactly five differing files — `error_ext.rs`, `guaranteed_validation.rs`,
`post_block_update.rs`, `system_tx.rs`, and `mod.rs`'s crate aliases,
`CRATE_NAME` and `TransactionSignature`. Everything else is identical, which
is the check this layout buys.
Issue: #1768
Assisted-by: Claude:claude-opus-5
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Assisted-by: Claude:claude-opus-5 Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
`scale-info`'s `#[derive(TypeInfo)]` records `module_path!()` in each type's
`Path`, so the runtime metadata embeds Rust module paths verbatim. Moving the
ledger wrapper into per-generation directories therefore changes the metadata's
type registry even though nothing about the SCALE encoding moved, and
`+check-metadata` byte-diffs the blob.
Decoding every `midnight_node_ledger`-rooted path out of the old and new blobs
gives 18 renames, 18 unchanged type names, and nothing added or removed:
common::types::{Op, Tx, UtxoInfo}
-> boundary::types::{Op, Tx, UtxoInfo}
ledger_9::common::types::{DeserializationError, DisjointCheckErrorCode,
EffectsCheckErrorCode, FeeCalculationErrorCode, InvalidError, LedgerApiError,
MalformedContractDeployErrorCode, MalformedError, MalformedZswapErrorCode,
SequencingCheckErrorCode, SerializationError, SystemTransactionError,
TransactionApplicationErrorCode, TransactionError, ZswapInvalidErrorCode}
-> ledger_9::types::{...}
The whole blob shrinks by exactly 99 bytes (148858 -> 148759), which is the
arithmetic of those renames and nothing else: 15 paths drop the `common`
segment (-7 bytes each, one compact length prefix plus six characters) and 3
grow from `common` to `boundary` (+2 bytes each).
No variant, discriminant, field, or field order changed -- `boundary/types.rs`
is a 100% rename of `common/types.rs`, and `ledger_9/types.rs` differs from the
`versions/common/types.rs` it came from by a single `use` line. Anything already
encoded on chain still decodes identically; only the names a fresh subxt codegen
emits are different.
`midnight_metadata_2.1.0.scale` is regenerated alongside `midnight_metadata.scale`
because the two are the same blob (that file tracks main rather than the release
it names).
Assisted-by: Claude:claude-opus-5
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
60fb008 to
d692478
Compare
…e helpers `ledger/helpers/src/versions/common/**` was compiled twice via a `#[path = "versions"]` module-parameterization trick, once bound to the ledger-8 crates and once to the ledger-9 crates, with `super::` resolving differently in each instantiation. A reader of a file under `versions/common/` could not tell which ledger crate `mn_ledger` referred to, and an edit meant for one version silently applied to both — the same footgun #2059 removed one layer down. Each version now has its own directory, `ledger_8/` and `ledger_9/`, with absolute `crate::ledger_N::…` imports so every file states the version it binds. The two inline modules in `lib.rs` become the respective `mod.rs` files, and the version-specific single files are renamed to line up across the two trees: versions/block_context/post_ledger_8.rs -> ledger_N/block_context.rs versions/ecdsa_unimpl.rs -> ledger_8/ecdsa.rs versions/test_utilities_compat.rs -> ledger_8/test_utilities_local.rs versions/ecdsa_wallet_tests.rs -> ledger_9/ecdsa_wallet_tests.rs `ledger_9/ecdsa.rs` and `ledger_9/test_utilities_local.rs` are new one-line re-exports of what `lib.rs` used to alias inline, so the two directories hold the same filenames and `diff -r` is meaningful. No behaviour change: rustc already emitted both instantiations. Public paths are unchanged (`pub use common::*;` already flattened `common` away), so no consumer needed an edit and the runtime metadata is untouched. The same 150 unit tests and 2 doctests run, under the same names minus the `common::` segment. Assisted-by: Claude:claude-opus-5 Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
`tx_generator/builder/builders/common/**` and `commands/fork/common/**` were
each compiled twice via a `#[path = "common"] pub mod inner { … }` trick, once
bound to the ledger-8 helpers and once to the ledger-9 helpers, with
`ledger_helpers_local` resolving differently in each instantiation. A reader of
a file under `common/` could not tell which ledger version it was looking at,
and an edit meant for one version silently applied to both. This finishes the
job #2059 and the ledger-helpers split started.
Each version now has its own directory — `builders/{ledger_8,ledger_9}/` and
`commands/fork/{ledger_8,ledger_9}/` — and each file names its own version with
a `use midnight_node_ledger_helpers::ledger_N as ledger_helpers_local;` line, so
the two copies stay byte-identical apart from that one word and `diff -r` can
police them. The only genuine divergence left is `builders/*/mod.rs`, which
carries the per-version `serialize_tx`.
The `inner` wrapper module is gone (nothing referenced it), so every external
path — `builders::ledger_8::SingleTxBuilder`, `fork::ledger_9::show_wallet`, … —
is unchanged, as is the CLI surface.
`impl_encoded_zswap_conversions!` stays in `builders/mod.rs`: `ledger_storage`
still aliases to `ledger_storage_ledger_8` in both versions, so duplicating the
impls per version would still hit E0119.
With this, `grep -r '#\[path' --include='*.rs'` over the workspace returns
nothing, and no directory is compiled more than once.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
…e helpers `ledger/helpers/src/versions/common/**` was compiled twice via a `#[path = "versions"]` module-parameterization trick, once bound to the ledger-8 crates and once to the ledger-9 crates, with `super::` resolving differently in each instantiation. A reader of a file under `versions/common/` could not tell which ledger crate `mn_ledger` referred to, and an edit meant for one version silently applied to both — the same footgun #2059 removed one layer down. Each version now has its own directory, `ledger_8/` and `ledger_9/`, with absolute `crate::ledger_N::…` imports so every file states the version it binds. The two inline modules in `lib.rs` become the respective `mod.rs` files, and the version-specific single files are renamed to line up across the two trees: versions/block_context/post_ledger_8.rs -> ledger_N/block_context.rs versions/ecdsa_unimpl.rs -> ledger_8/ecdsa.rs versions/test_utilities_compat.rs -> ledger_8/test_utilities_local.rs versions/ecdsa_wallet_tests.rs -> ledger_9/ecdsa_wallet_tests.rs `ledger_9/ecdsa.rs` and `ledger_9/test_utilities_local.rs` are new one-line re-exports of what `lib.rs` used to alias inline, so the two directories hold the same filenames and `diff -r` is meaningful. No behaviour change: rustc already emitted both instantiations. Public paths are unchanged (`pub use common::*;` already flattened `common` away), so no consumer needed an edit and the runtime metadata is untouched. The same 150 unit tests and 2 doctests run, under the same names minus the `common::` segment. Assisted-by: Claude:claude-opus-5 Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
`tx_generator/builder/builders/common/**` and `commands/fork/common/**` were
each compiled twice via a `#[path = "common"] pub mod inner { … }` trick, once
bound to the ledger-8 helpers and once to the ledger-9 helpers, with
`ledger_helpers_local` resolving differently in each instantiation. A reader of
a file under `common/` could not tell which ledger version it was looking at,
and an edit meant for one version silently applied to both. This finishes the
job #2059 and the ledger-helpers split started.
Each version now has its own directory — `builders/{ledger_8,ledger_9}/` and
`commands/fork/{ledger_8,ledger_9}/` — and each file names its own version with
a `use midnight_node_ledger_helpers::ledger_N as ledger_helpers_local;` line, so
the two copies stay byte-identical apart from that one word and `diff -r` can
police them. The only genuine divergence left is `builders/*/mod.rs`, which
carries the per-version `serialize_tx`.
The `inner` wrapper module is gone (nothing referenced it), so every external
path — `builders::ledger_8::SingleTxBuilder`, `fork::ledger_9::show_wallet`, … —
is unchanged, as is the CLI surface.
`impl_encoded_zswap_conversions!` stays in `builders/mod.rs`: `ledger_storage`
still aliases to `ledger_storage_ledger_8` in both versions, so duplicating the
impls per version would still hit E0119.
With this, `grep -r '#\[path' --include='*.rs'` over the workspace returns
nothing, and no directory is compiled more than once.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Oscar Bailey <79094698+ozgb@users.noreply.github.com>
Overview
Closes #1768.
ledger/src/versions/common/**(4451 lines) was compiled twice through a#[path = "versions"]module-parameterization trick — once bound to the ledger-8crates, once to the ledger-9 crates — with
super::resolving differently in eachinstantiation:
A reader of
versions/common/api/ledger.rscould not tell whatmn_ledger_localreferred to, and an edit meant for one version silently applied to both. Per the
issue, that footgun cost several days of investigation and crashed nodes.
Each version now owns its files, with absolute
crate::ledger_N::…imports soevery file states the version it binds:
Nothing is called
commonany more, which is the other half of the issue: thecrate-level
common/was never shared behaviour — it is the runtime/client wireformat, deliberately version-agnostic, and the pallet has one decoder for it. It is
renamed to
boundary/rather than duplicated; versioning that wire format would bea design change, not a refactor.
No behaviour change. rustc already emitted both instantiations, so the +4.6k
lines only make the existing duplication visible in the source tree — no new machine
code, no binary-size or compile-time change. There was no non-test version branching
in the shared half to reinterpret (the only
CRATE_NAMEcomparisons areif CRATE_NAME != crate::latest::CRATE_NAME { return }test guards, which stillwork unchanged). Every import rewrite is compile-checked, so there is no
silent-failure mode.
Public paths are untouched —
ledger_8::…,ledger_9::…,latest::…,types::…(incl.
types::active_versionandtypes::active_ledger_bridge),host_api::….The 27 files outside
ledger/that import from this crate needed zero changes;the only edits outside
lib.rs/ledger_8/ledger_9are twocommon::→boundary::import lines inhost_api/.🗹 TODO before merging
📌 Submission Checklist
git commit -s) for the DCO🧪 Testing Evidence
The review artifact
The point of the layout is that the two directories can be diffed. Normalise the
version path away, then diff:
Exactly five files differ, and they are precisely the ones that were already
per-version before this PR:
error_ext.rsGenerationInfoAlreadyPresent(v8) vsInitialNonceAlreadyPresent(v9)guaranteed_validation.rsapply()dry-run (v8) vsapply_guaranteed_only(v9)post_block_update.rsprevalidate_post_block_update(v9)system_tx.rsSystemTransactionvariants;PayBlockRewardsToTreasuryonly on v8mod.rsCRATE_NAME,TransactionSignatureEverything else —
types.rs,storage.rs,conversions.rs,utxo_ordering_override.rs,block_context.rs,api/{mod,ledger,transaction}.rs—is byte-identical modulo the version path. That is the evidence the copy is faithful,
and going forward it is the standing check for what the two versions actually do
differently.
Builds and tests
cargo check -p midnight-node-ledger --all-targetscargo clippy --workspace --all-targets --features runtime-benchmarks,try-runtimeSKIP_WASM_BUILD=1 cargo check --workspace --all-targetscargo build -p midnight-node-runtime(real no_std / WASM path)cargo test -p midnight-node-ledgercargo test -p pallet-midnight -p pallet-midnight-system -p pallet-c2m-bridge(+ ledger)rg 'midnight_node_ledger::' -g '!ledger/**'Note: bare
cargo check -p midnight-node-ledger --no-default-featuresfails, but itfails identically on
main(405 errors) — it is not a supported featureconfiguration. The no_std path is exercised by the runtime WASM build above.
Metadata rebuild required
scale-info's#[derive(TypeInfo)]recordsmodule_path!()in each type'sPath,so the runtime metadata embeds Rust module paths. Moving these modules therefore
changes the metadata even though nothing about the encoding changed — no variant,
discriminant, field, or field order moved, and
boundary/types.rsis a 100% renamewhile
ledger_9/types.rsdiffers from its old copy by a singleuseline.Decoding every
midnight_node_ledger-rooted path out of the old and new blobsgives 18 renames, 18 unchanged type names, nothing added or removed:
common::types::{Op, Tx, UtxoInfo}->boundary::types::{…}ledger_9::common::types::{DeserializationError, DisjointCheckErrorCode, EffectsCheckErrorCode, FeeCalculationErrorCode, InvalidError, LedgerApiError, MalformedContractDeployErrorCode, MalformedError, MalformedZswapErrorCode, SequencingCheckErrorCode, SerializationError, SystemTransactionError, TransactionApplicationErrorCode, TransactionError, ZswapInvalidErrorCode}->
ledger_9::types::{…}The blob shrinks by exactly 99 bytes (148858 -> 148759), which is the arithmetic of
those renames and nothing else: 15 paths drop the
commonsegment (-7 bytes each, acompact length prefix plus six characters) and 3 grow from
commontoboundary(+2 bytes each). Anything already encoded on chain still decodes identically.
Renaming
ledger_9::common::typesis unavoidable — that module is what the issueasks to remove — so metadata had to be rebuilt either way; the crate-root
common->boundaryrename adds three more paths to the same churn. Downstreamsubxt codegen will see the new
runtime_types::…module paths. No in-repo consumerreferences them (the indexer's
runtimes/v*.rsare pinned to older metadatasnapshots), but it is worth a heads-up to anyone generating fresh bindings.
No genesis rebuild: no storage item, extrinsic, or runtime API changed.
🔱 Fork Strategy
Source-layout only. The runtime WASM blob may differ byte-wise since panic locations
embed source paths, but
spec_versionand behaviour are unchanged, so no coordinatedupdate is needed.
Links
Closes #1768
Follow-ups (deliberately not in this PR)
are separate you can diff them and decide per item — but that judgement call is the
multi-day analysis the issue describes. Pure duplication first.
ledger/helpers/src(6166 lines) andutil/toolkit/src(
commands/fork/common,tx_generator/builder/builders/common, 4003 lines) use thesame trick. Same recipe, separate PRs.
latesttest copies. Theif CRATE_NAME != crate::latest::CRATE_NAME { return }guards already make themno-ops.
🤖 Generated with Claude Code