Skip to content

feat: add ledger - #182

Closed
rvolosatovs wants to merge 19 commits into
LFDT-Nightstream:mainfrom
rvolosatovs:feat/ledger
Closed

rvolosatovs wants to merge 19 commits into
LFDT-Nightstream:mainfrom
rvolosatovs:feat/ledger

Conversation

@rvolosatovs

@rvolosatovs rvolosatovs commented Jul 29, 2026 •

Copy link
Copy Markdown
Contributor

This is an in-progress draft of the ledger work, aiming to replace the mock-ledger

You can find the Wizer instrumentation here for snapshotting UTXO state (tested, works!)
Each coordination script invocation produces a "transaction" and UTXOs are identified as just (transaction index, UTXO index, assigned in order of construction)
Coordination scripts and UTXO methods can be invoked over HTTP using wRPC

A new wasmtime::Store is created per coordination script invocation; UTXOs are constructed into that top-level store and then snapshotted (producing a new Wasm component) once coordination script returns

LLM-generated description below:

This adds starstream-ledger: a standalone ledger server built on the starstream-runtime-next host,
aiming to replace the mock-ledger. docs/ledger.md (added here) is the authoritative spec for the
HTTP API, CLI flags, and ledger model — the summary below is just the tour.

  • Contracts are content-addressed by the SHA-256 of their component Wasm, encoded as a multibase
    multihash (canonically base32-lower). The canonical form is also a valid component-model label, so it
    embeds verbatim in Wasm and wRPC instance names.
  • Publishing is signed and paid for. PUT /contracts/<digest> takes a COSE_Sign1 envelope
    (Ed25519, kid = raw public key) over ["starstream:publish", network, nonce, wasm]. The admin account
    (--admin-key/--admin-balance, defaulting to a well-known pre-seeded key with u64::MAX) is charged
    one unit per Wasm byte; nonces are strictly increasing (replay protection).
  • Coordination scripts and UTXO methods are invoked over wRPC via a single POST /rpc, dispatched
    on the instance: starstream:contract/<digest> for scripts, starstream:utxo/<digest> for
    persisted-UTXO methods. Script UTXO imports are bound via X-Starstream-Utxo: <instance>=<contract-digest> request headers.
  • UTXO state is snapshotted with Wizer (tested, works!). A fresh wasmtime::Store is created per
    coordination-script invocation; UTXOs are constructed into it and snapshotted into new Wasm components
    when the script returns. UTXOs are content-addressed too: each successful script invocation is
    recorded as a transaction in a new block, and the persisted UTXO digests (output order) come back in
    X-Starstream-Utxo response headers alongside X-Starstream-Transaction/X-Starstream-Block. Blocks
    are recorded in memory but not (yet) addressable over HTTP.
  • ABIs are served as WIT: GET /contracts/<digest>/rpc and GET /utxos/<digest>/rpc, as WIT text
    or a Wasm-encoded WIT package via Accept. The interface ID is exactly the wRPC instance to invoke, so
    a world importing it feeds straight into wit-bindgen-wrpc.
  • A client CLI (starstream-ledger-cli) is gated behind the cli feature; arguments and results
    are WAVE-encoded. E2E tests drive the in-process server
    (tests/ledger.rs) and the compiled binaries (tests/cli.rs).
  • Cardano context (block height, current slot) is injected into running contracts via the
    starstream:std/cardano host interface (--cardano-block-height, --cardano-current-slot).

Usage

Setup

# Build the server + CLI, compile the contract to a component
cargo build -p starstream-ledger --features cli
./starstream wasm -c examples/score.star --output-component score.wasm

# The default admin account: the Ed25519 key whose private key is SHA-256("admin")
KEY=$(printf admin | sha256sum | cut -d' ' -f1)
URL='http://[::1]:9000'

Terminal 1 — the ledger

# Admin account pre-funded with u64::MAX by default;
# override with --admin-key PUBKEY / --admin-balance N
target/debug/starstream-ledger --network dev

Terminal 2 — the flow

CLI=target/debug/starstream-ledger-cli    # talks to http://[::1]:9000 by default

# 1. Publish: signs the COSE_Sign1 envelope, prints the content-address
DIGEST=$($CLI publish --key $KEY --network dev --nonce 1 score.wasm)

# 2. What can this contract do? No script name -> the served WIT
$CLI script $DIGEST
#   package starstream:contract;
#
#   interface bciqfbwcy4cmf5td7mbayvlymywvvq72cyjlqvccaswu6rtfm2d3fixa {
#     example: func();
#   }

# 3. Run the coordination script; constructs the ScoreProgress UTXO.
#    The persisted UTXO digest, transaction index, and block height
#    are reported on stderr:
$CLI script --utxo score-progress=$DIGEST $DIGEST example
UTXO=bciq...   # the digest reported above — the UTXO's address

# 4. What can the UTXO do? No method name -> the served WIT
$CLI method $UTXO
#   package starstream:utxo;
#
#   interface bciqelzhvansl7qzql6bbqpkluky3ryapr2sf2aoewquyusjra2dfz4q {
#     plus-chips: func(chips2: u64);
#     plus-mult: func(mult2: u64);
#     mult-mult: func(mult-pct: u64);
#     finish: func();
#   }

# 5. Invoke methods (arguments are WAVE, one per parameter)
$CLI method $UTXO plus-chips 7
$CLI method $UTXO plus-mult 42
$CLI method $UTXO mult-mult 200
$CLI method $UTXO finish

The same data over plain HTTP

# The durable record: the COSE_Sign1 publish envelope (the default representation)
curl $URL/contracts/$DIGEST -o envelope.cose

#   }

# 5. Invoke methods (arguments are WAVE, one per parameter)
$CLI method $UTXO plus-chips 7
$CLI method $UTXO plus-mult 42
$CLI method $UTXO mult-mult 200
$CLI method $UTXO finish

The same data over plain HTTP

# The durable record: the COSE_Sign1 publish envelope (the default representation)
curl $URL/contracts/$DIGEST -o envelope.cose
# The raw contract component, disassembled on the fly
curl -H 'accept: application/wasm' $URL/contracts/$DIGEST | wasm-tools print | less

# The ABIs the CLI showed above are just GETs on the /rpc URLs
curl $URL/contracts/$DIGEST/rpc    # script ABI, WIT text
curl $URL/utxos/$UTXO/rpc          # UTXO ABI, WIT text

# The UTXO ABI as a Wasm-encoded WIT package, decoded back by wasm-tools
curl -H 'accept: application/wasm' $URL/utxos/$UTXO/rpc | wasm-tools component wit

Notes

  • The fetched application/wasm contract bytes hash back to $DIGEST
    (curl -s -H 'accept: application/wasm' $URL/contracts/$DIGEST | sha256sum) —
    the content-addressing guarantee in action. Re-publishing an already-stored
    digest is an idempotent 200: nothing is charged, no nonce consumed.
  • UTXO method invocations currently run against the persisted snapshot: mutated
    state is not (yet) written back as a new transaction, so every invocation
    observes the state captured at persistence time.

@rvolosatovs
rvolosatovs force-pushed the feat/ledger branch 3 times, most recently from 3ca95be to 3d592c6 Compare July 29, 2026 14:11
Comment thread docs/ledger.md
Comment on lines +106 to +109
- protected `alg` header: EdDSA;
- protected `kid` header: the signer's raw 32-byte Ed25519 public key
(its lowercase hex is the paying account's identifier);
- payload: the CBOR publish transaction, a four-element array

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember all the validation rules required for COSE, but just be careful to avoid bad keys, insecure keys, non-canonical keys, and a few other weird cases you have to deal with

Comment thread docs/ledger.md
Comment thread docs/ledger.md
Comment thread docs/ledger.md Outdated

```
starstream-ledger [--addr ADDR] [--network NETWORK]
[--account PUBKEY=BALANCE]...

@rvolosatovs rvolosatovs Jul 29, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make --admin-account singular.
FundAccount can only be submitted signed by that key.
Add initial contracts
Add initial UTXOs

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Implement `GET /transactions/<tx>/utxos/<id>/rpc`: render the methods the
persisted UTXO declared via `implements-method` as functions at the root
of a WIT world named after the UTXO's exported instance, in the
`starstream:utxo` package — the client view, so feeding the served WIT to
`wit-bindgen-wrpc` yields working invocation bindings. The e2e test does
exactly that: the score UTXO methods are invoked through bindings
generated from the same WIT file the response is asserted against
verbatim, both as text and as the Wasm-encoded package.

Method invocations are addressed to the root (empty) wRPC instance,
matching how wrpc-wasmtime serves component root exports. The pinned
wit-bindgen-wrpc revision instead derives a `<package>/<world>` instance
for world-root functions; until that is fixed upstream the test routes the
generated bindings through a `RootInvoke` adapter pinning the instance to
the root one.

The representation is negotiated via the `Accept` header: `text/plain`
(the default) serves the WIT text, `application/wasm` serves the same
package encoded as Wasm via `wit-component`.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Add `docs/ledger.md` — the authoritative behaviour spec for the ledger
server, in the style of `docs/cli.md` and picked up by the website via
the shared `docs/` tree: the CLI flags, the content-addressing, account
and transaction model, and every endpoint with its request shape,
content negotiation, and status codes.

Give the crate matching rustdoc: a crate-level overview pointing at the
spec and doc comments on the public items (`Ledger`, `Account`,
`X_STARSTREAM_UTXO`).

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Bump the wrpc dependencies to the revision fixing wit-bindgen-wrpc to
address world-root functions on the root (empty) instance
(bytecodealliance/wrpc#1434) and remove the `RootInvoke` adapter that
papered over the previous behaviour — the generated bindings now invoke
the served UTXO methods directly.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Implement `GET /contracts/<digest>/rpc`: render every coordination-script
export of a published contract as a function import at the root of the
`contract` world in the `starstream:contract` package — the same client
view the UTXO ABI endpoint serves, negotiated the same way (`text/plain`
WIT text by default, `application/wasm` for the `wit-component`-encoded
package). Document the endpoint in `docs/ledger.md`.

The e2e test now exercises both served WITs symmetrically: `example` is
invoked through `wit-bindgen-wrpc` bindings generated from the same WIT
file the response is asserted against verbatim, replacing the hand-rolled
generic `invoke` helper, and the UTXO bindings module is renamed
`score_progress_bindings` next to the new `score_bindings`.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Add `starstream-ledger-cli` (`src/bin/starstream-ledger-cli.rs`, gated
behind the new `cli` cargo feature) driving the ledger HTTP API from the
command line:

- `publish` signs the CBOR publish transaction into a COSE_Sign1
  envelope with a hex-encoded Ed25519 key, PUTs it at the
  content-addressed URL, and prints the contract digest;
- `script` invokes a coordination script, mapping UTXO imports via
  repeatable `--utxo <instance>=<digest>` headers and reporting the
  persisted UTXOs;
- `method` invokes a method on a persisted UTXO;
- `wit` prints a persisted UTXO's ABI as served.

Invocation parameters are given in WAVE, one argument per parameter, and
results are printed back as WAVE — the parameter and result types are
recovered by fetching the published contract component back from the
ledger and introspecting its exports, exactly as the server itself types
the calls. The wRPC invocation body is framed by hand (invocation header
on the root instance plus one parameter frame), keeping the response
headers accessible.

Expose the wRPC value codec as `starstream_ledger::codec` so the client
shares the server's implementation, and document the CLI in
`docs/ledger.md`.

The new `tests/cli.rs` reproduces the whole `score_contract_flow` test
through the two compiled binaries alone; CI enables the feature in the
workspace test run.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Replace the bare lowercase-hex SHA-256 contract identifier with a
multibase-encoded sha2-256 multihash, canonically base32-lower: `b`
followed by 55 lowercase alphanumeric characters. Unlike hex, the
canonical form is a valid component-model label by construction (it
always starts with a letter and contains no separators), so the digest
can be embedded verbatim in Wasm import/export and wRPC instance names,
e.g. `starstream:contract/<digest>`.

`encode_digest`/`parse_digest` implement the codec on top of the
`multibase` and `multihash` crates. Every route taking a digest —
`PUT`/`GET /contracts/<digest>`, `GET`/`POST /contracts/<digest>/rpc`,
and the `X-Starstream-Utxo` import mapping header — parses it with
`parse_digest`, accepting any multibase base, and normalizes to the
canonical encoding for storage, lookups, responses, and errors. The CLI
`publish` prints the canonical digest, and `docs/ledger.md` documents
the format.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Replace the hand-written `Display` and empty `core::error::Error` impls
on the eight ledger error enums (`DigestParseError` and the per-route
errors in `ledger.rs`) with `#[derive(thiserror::Error)]`, keeping the
messages and the `http_status_code` mappings unchanged. Variants that
forwarded `Display` to their inner error use `#[error(transparent)]`,
which now also forwards `source()`.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Replace the hand-rolled invocation plumbing in `tests/ledger.rs` — raw
`encode_invocation` bodies, a manual `FrameDecoder` loop, and
hand-encoded LEB128 parameter bytes — with the stock `wrpc_http::Client`
over the hyper-util legacy client, invoked through the typed
`InvokeExt::invoke_values_blocking` API, so UTXO method parameters are
plain `u64`s and result emptiness is expressed as the `()` results type.

The `X-Starstream-*` response envelope and rejection statuses are not
observable through the wRPC client abstraction (`handle_response` drops
the response parts), so the coordination-script invocation carrying the
persisted-UTXO digest and the not-found cases are asserted over plain
HTTP via a single `post_invocation` request builder.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Replace the repeatable `--account PUBKEY=BALANCE` genesis flag with a
single admin account configured via `--admin-key PUBKEY` and
`--admin-balance BALANCE` (default `u64::MAX`). When `--admin-key` is
not specified, a well-known pre-seeded development key is used: the
Ed25519 key whose private key is the SHA-256 digest of `admin`.

`Ledger::new` now requires the admin key and balance instead of taking
an account `HashMap`; the genesis account is constructed internally
with a zero nonce.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
Add `POST /accounts/<account>`: a fund-account transaction submitted as
a tagged `COSE_Sign1` envelope over the CBOR payload
`["starstream:fund-account", network, nonce, account, amount]` that
must be signed by the configured admin key (403 for any other signer).
On success `amount` is transferred from the admin balance (402 when it
does not cover the amount) to the addressed account — created on first
funding — the admin nonce is advanced (409 on replay), and the transfer
is recorded in a block as the pre-existing `Action::FundAccount`.

The account is addressed by the lowercase-hex encoding of its raw
32-byte Ed25519 public key, which must match the key signed in the
payload; the context string domain-separates the signature from publish
transactions, which share the admin nonce sequence.

Expose the flow as a `fund-account` CLI subcommand mirroring
`publish`, and cover it in the e2e suites: transfer/replay/publish-from-
funded-account over HTTP, rejection statuses, and the CLI score flow now
funds a fresh account that pays for the publish.

Assisted-by: anthropic:claude-fable-5
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
rvolosatovs added a commit that referenced this pull request Sep 6, 2026
Some of this is extracted from #182, and a big part is
reworked/refactored.

This first, minimal ledger supports:
- funding accounts (used to pay for publishing)
- publishing contracts
- fetching contracts (COSE envelope or raw Wasm via `Accept`
negotiation; `HEAD` supported)

Transactions are Ed25519-signed COSE_Sign1 envelopes (verified with
`verify_strict`), bound to a network identifier and a
strictly-increasing per-account nonce. The server rejects unprotected
COSE headers and re-encodes each envelope into canonical tagged form
before charging, storing, or serving it, so the retained bytes are fully
covered by the signature. Publishing costs one unit per envelope byte.

The ledger produces blocks, which are stored in memory. Each block
contains exactly one action (currently either contract publish or
account fund).

Contract digests are represented as multibase-multihash sha256. The
client verifies what it fetches against the requested digest: Wasm by
hash, envelopes by signature and payload digest. Accounts are Ed25519
public keys; funding a weak (small-order) key is rejected, since
`verify_strict` would refuse its signatures at publish time anyway.

Default private admin key is the sha256 digest of the word `admin` and
default network is `dev`.

Server behavior worth noting: `content-type: application/cose` is
required on submissions; concurrent connections are bounded by
`--max-requests`. The node shuts down gracefully on ^C or SIGTERM,
bounded by `--shutdown-timeout` (10 s default).

The CLI takes signing keys as a file path (`--key <PATH>`), never on the
command line, and zeroizes key material after signing.

Example usage (against ledger node running with default config with
compiled score example at `./score.wasm`):

```
$ starstream-ledger-cli key generate > key
 INFO starstream_ledger_cli: generated key pair public_key="ea1fe9aef4b7bcf768fe7614dae347db3911f33d389f7da83532de08783397c5"

$ printf admin | sha256sum | cut -d' ' -f1 > admin.key

$ starstream-ledger-cli account fund --key admin.key --nonce 1 "ea1fe9aef4b7bcf768fe7614dae347db3911f33d389f7da83532de08783397c5" 1000000

$ starstream-ledger-cli contract publish --key key --nonce 1 ./score.wasm

$ curl -s -H 'Accept: application/wasm' http://[::1]:9000/contracts/$(starstream-ledger-cli contract digest ./score.wasm) | wasm-tools print | tail
      (with "import-type-utxo" (type $"#type14 utxo"))
      (with "import-type-s-utxo-context" (type $utxo-context))
      (with "import-type-storage" (type 15))
    )
  )
  (export $score-progress (;6;) "score-progress" (instance $score-progress-shim-instance))
  (@producers
    (processed-by "wit-component" "0.254.0")
  )
)
```

---------

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
rvolosatovs added a commit that referenced this pull request Oct 15, 2026
Both the server and the client implemented `block-height` and
`current-slot` with `bail!("TODO")`, so any contract reading them
trapped, whether in a coordination script run by the client or in a UTXO
method called over `/rpc`.

Port the approach of the original ledger draft (#182): a static
`CardanoCtx` reported to every store. The node takes it from
`--cardano-block-height` and `--cardano-current-slot` (default `0`) and
uses it both for `/rpc` UTXO method calls and for genesis scripts.
`ledger contract script call` takes the same flags for the scripts it
runs.

The values aren't recorded in the transaction or checked when it is
submitted, so the client and the node can disagree.

Breaking: `Ledger::new` takes a `CardanoCtx`, and the client `Ctx` has a
new public `cardano` field.

Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
@rvolosatovs

Copy link
Copy Markdown
Contributor Author

All of the relevant work from this branch has been already merged in main or in open PRs (#262 and #267). Closing

@rvolosatovs
rvolosatovs deleted the feat/ledger branch October 20, 2026 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants