gRPC: Update to the latest UTxO RPC v1beta spec - #1303
Open
carbolymer wants to merge 8 commits into
Open
Conversation
carbolymer
changed the title
carbolymer
force-pushed
the
mgalazyn/utxorpc-spec-update
branch
from
August 20, 2026 14:43
8d38d97 to
b6cf60a
Compare
carbolymer
force-pushed
the
mgalazyn/utxorpc-spec-update
branch
from
August 27, 2026 13:32
b6cf60a to
a7c0ece
Compare
Sync the vendored proto definitions with utxorpc/spec main (v0.19.2 plus unreleased EvalReport tweaks): governance vote messages and Tx.votes, TxOutput.original_cbor, the ReadState ledger-state query machinery, and all upstream service methods restored in the service blocks. Regenerate the proto-lens code with buf.
The new AnyChainStateQuery/AnyChainStateData oneofs in query.proto generate maybe'query/maybe'result lenses that collide with the ones from cardano.proto's StateQuery/StateData envelopes, making the wholesale re-export of both Fields modules ambiguous. Hide the Cardano_Fields copies, following the module's existing convention.
Wire every method of QueryService, SubmitService and SyncService into the grapesy method tables. Methods without an implementation (ReadData, ReadEraSummary, ReadState, ReadTx, ReadMempool, WaitForTx, WatchMempool, DumpHistory) are declared with UnsupportedMethod, which makes the server respond with the UNIMPLEMENTED gRPC status; the previous hand-rolled dumpHistory stub is converted to the same mechanism. Document the behaviour and the new ReadState method in the README support matrix.
carbolymer
force-pushed
the
mgalazyn/utxorpc-spec-update
branch
from
August 27, 2026 14:46
5213a15 to
d0d5314
Compare
carbolymer
marked this pull request as ready for review
August 27, 2026 14:46
carbolymer
requested review from
CarlosLopezDeLara,
disassembler,
erikd and
palas
as code owners
August 27, 2026 14:46
Convert the voting procedures of Conway-onwards transactions to the UTxO RPC VoterVotes messages: DRep and constitutional committee voters map to stake credentials, stake pool voters to their pool key hash, each with their votes, gov action ids and optional anchors. Read through the any-era getter, so earlier eras yield the empty list.
Fill the field with the era-encoded CBOR of the output. This is a canonical re-encoding, not guaranteed to be the original on-chain bytes: the ledger does not memoise TxOut and its decoders accept non-canonical encodings, so decode-then-encode may differ for historical outputs. No protocol hash uses a standalone TxOut as preimage, and the memoised components inside it (inline datums, plutus scripts) keep their original bytes, so datum and script hashes remain verifiable.
The single-item FetchBlock variant moved to the upcoming utxorpc v1 (utxorpc/spec#208 was retargeted there), so v1beta keeps the repeated request refs and response blocks. Regenerate the proto-lens Sync modules and make the handler fetch every referenced block, failing the whole call with NOT_FOUND naming the first missing ref's slot and header hash, matching Dolos's all-or-nothing behaviour. Mark the changelog fragment as breaking accordingly.
ReadData and ReadTx need a whole-chain index (datum by hash, transaction by hash) that cardano-node does not maintain, so they cannot be implemented without building an external chain indexer into the node. Distinguish them from the methods that are merely not yet implemented.
carbolymer
force-pushed
the
mgalazyn/utxorpc-spec-update
branch
from
August 27, 2026 14:48
d0d5314 to
670894e
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates cardano-rpc to the latest upstream UTxO RPC v1beta spec by syncing the vendored .proto definitions, regenerating the proto-lens Haskell bindings, and wiring new/changed fields and service surfaces into the server and tests (notably governance votes, TxOutput.original_cbor, and the repeated-shape FetchBlock).
Changes:
- Sync vendored UTxO RPC v1beta protos to upstream and regenerate proto-lens output (new query/state messages, updated service method sets, repeated
FetchBlockrequest/response shape). - Implement newly available surfaces: populate
Tx.votes(Conway+) andTxOutput.original_cbor(canonical re-encoding), and expose all v1beta service methods with grapesyUnsupportedMethodfor unimplemented RPCs. - Extend tests and docs to cover votes/original CBOR behavior and document UNIMPLEMENTED exposure in the README; add a Herald changelog fragment marking the breaking
FetchBlockshape reset.
Reviewed changes
Copilot reviewed 21 out of 22 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/TxOutput.hs | Adds a property asserting original_cbor decodes back to the ledger TxOut. |
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/FetchBlockTx.hs | Extends tx projection tests to cover Conway voting procedures and vote counts/contents. |
| cardano-rpc/test/cardano-rpc-test/Test/Cardano/Rpc/ByronTx.hs | Asserts Byron-era defaults for new fields (original_cbor empty, votes empty). |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/TxOutput.hs | Populates TxOutput.originalCbor via canonical ledger serialization (with caveat). |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Tx.hs | Populates Tx.votes from Conway voting procedures (era-gated). |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Type/Governance.hs | Introduces mapping from ledger voters/votes to v1beta VoterVotes / VotingProcedure. |
| cardano-rpc/src/Cardano/Rpc/Server/Internal/UtxoRpc/Sync.hs | Updates FetchBlock handler to process repeated block refs and return repeated blocks. |
| cardano-rpc/src/Cardano/Rpc/Server.hs | Exposes all v1beta methods and uses UnsupportedMethod for unimplemented endpoints (ordered per ServiceMethods). |
| cardano-rpc/src/Cardano/Rpc/Proto/Api/UtxoRpc/Query.hs | Adjusts field-lens imports/hiding for newly generated maybe'query / maybe'result. |
| cardano-rpc/README.md | Documents UNIMPLEMENTED exposure semantics and updates QueryService support matrix (incl. ReadState). |
| cardano-rpc/proto/utxorpc/v1beta/sync/sync.proto | Resets FetchBlock to upstream repeated request/response shape. |
| cardano-rpc/proto/utxorpc/v1beta/submit/submit.proto | Updates SubmitService to include the full upstream v1beta method set/order. |
| cardano-rpc/proto/utxorpc/v1beta/query/query.proto | Vendors new ledger-state query envelope/messages and updated QueryService method set. |
| cardano-rpc/proto/utxorpc/v1beta/cardano/cardano.proto | Vendors new TxOutput.original_cbor, governance vote messages, and state query messages; tweaks EvalReport index docs. |
| cardano-rpc/gen/Proto/Utxorpc/V1beta/Sync/Sync.hs | Regenerated proto-lens code for Sync service/messages (notably repeated ref/block). |
| cardano-rpc/gen/Proto/Utxorpc/V1beta/Sync/Sync_Fields.hs | Regenerated field lenses reflecting repeated ref and new vector lens names. |
| cardano-rpc/gen/Proto/Utxorpc/V1beta/Submit/Submit.hs | Regenerated Submit service methods/order and streaming signatures. |
| cardano-rpc/gen/Proto/Utxorpc/V1beta/Query/Query.hs | Regenerated Query service/messages including state query/data envelopes and ReadState request/response. |
| cardano-rpc/gen/Proto/Utxorpc/V1beta/Query/Query_Fields.hs | Regenerated field lenses for query/state-related additions. |
| cardano-rpc/gen/Proto/Utxorpc/V1beta/Cardano/Cardano_Fields.hs | Regenerated field lenses for governance votes and state query/distribution additions. |
| .changes/20260819_160000_cardano-rpc_carbolymer_utxorpc_spec_update.yml | Herald changelog fragment marking the update as feature + breaking and describing the API reset. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Jimbo4350
approved these changes
Aug 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
This PR syncs the vendored UTxO RPC v1beta proto definitions with the latest upstream utxorpc/spec (v0.19.2 plus the post-release
EvalReportoptionality fix), regenerates the proto-lens code, and implements the newly available surface:Vote/VotingProcedure/VoterVotesmessages are vendored and the newTx.votesfield is populated for Conway-onwards transactions.DRep and constitutional committee voters map to stake credentials, SPO voters to their pool key hash, each with their votes, governance action ids and optional anchors.
TxOutput.original_cbor(upstream #201): populated with the era-encoded CBOR of the output.Note this is a canonical re-encoding, not guaranteed to be byte-identical to the on-chain encoding: the ledger does not memoise
TxOutand its decoders accept non-canonical input.No protocol hash uses a standalone
TxOutas preimage, and the memoised components inside it (inline datums, plutus scripts) keep their original bytes, so datum and script hashes remain verifiable.The caveat is documented at the computation site.
ReadState/StateQuery/StakePoolDistributionmessages are vendored, schema-only for now.The unimplemented ones (
ReadData,ReadTx,ReadEraSummary,ReadState,ReadMempool,WaitForTx,WatchMempool,DumpHistory) are declared with grapesy'sUnsupportedMethod, which makes the server respond with theUNIMPLEMENTEDgRPC status; the previous hand-rolleddumpHistorystub is converted to the same mechanism.The README support matrix documents this behaviour and gains the
ReadStaterow.Breaking:
FetchBlockis reset to the upstream v1beta shape withrepeatedrequest refs and response blocks, and the handler now fetches every referenced block.The previous single-item variant has moved to the upcoming utxorpc v1 (utxorpc/spec#208 was retargeted there), so v1beta stays with the repeated form.
Clients built against the single-item shape (e.g. cardano-node's RPC integration tests) need adapting when they pick up this version.
How to trust this PR
proto/utxorpc/v1betaagainst upstreammainshows no differences.ServiceMethodsorder, not proto declaration order; each table inCardano/Rpc/Server.hsdocuments its expected order in the haddock.cabal test cardano-rpc-test: all 98 tests pass, including new coverage: injected DRep/SPO votes with exact content assertions at Conway, vote-count projections, pre-Conwayvotes == []totality arms, anoriginal_cbordecode-back round-trip, and Byron empty-field assertions.grpcurl -plaintext localhost:<rpc-port> utxorpc.v1beta.query.QueryService/ReadDataresponds with theUnimplementedstatus.Checklist
.changes/