Skip to content

refactor(runtime): Runtime struct, join on the main loop, unified community replies, cheaper state sync - #607

Open
stormer78 wants to merge 27 commits into
mainfrom
refactor/runtime
Open

stormer78 wants to merge 27 commits into
mainfrom
refactor/runtime

Conversation

@stormer78

Copy link
Copy Markdown
Contributor

PR 9 of the remediation plan on #589: the runtime restructure.

Closes #528, closes #529, closes #534, closes #555, closes #556, closes #565, closes #557. Part of #535, part of #537.

Changes

#528 StateHandler::main_loop becomes state_handler/runtime/: a Runtime struct holding the loop-local state, one method per select! arm (arms.rs, inbound_arm.rs, timers.rs), and a short router in Runtime::run. The arm bodies moved unchanged. There is a harness that drives single arms and the whole loop.

#529 The join is driven from the loop that owns it, with no nested event loop:

  • join_flow/driver.rs is a JoinDriver driven by both the runtime and the State-A loop.
  • Resolve, discovery and the five launch stages run off the loop on DispatchDomain::Join.
  • A JoinSaga in core (openvtc-core/src/join/) records each step and its compensation. Network steps are split into I/O and apply halves. SubjectSigner reads the signer off the config and signs without it.
  • Esc, F10 or an error cancels any stage before the submit, and the saga undoes it. A generation counter drops stale results. Keys pressed while a job runs are queued and replayed in order.
  • Submit is past the point of no return. An exit during it waits for it to finish, so the request is recorded and the persona kept. A loop stop mid-submit keeps the persona and listener instead of forgetting them. Inbound from the community being submitted to is held until the pending record exists, then processed. The old nested loop got this by not reading inbound at all.
  • Bug fixed: on main, a failed submit (other than a signer failure) forgot the persona even when it was reused, and could remove another persona's key_info. The saga forgets only a persona this join minted.

#534 A PendingReplies table owned by the runtime replaces the three process-global reply-correlation statics in core (renewal, profile queries, elections). TTLs and caps are the same; entries are recorded on the loop before the spawn. community_request.rs holds the shared request/reply engine; rooms and elections run on it, and repos uses its helpers.

#535 (part) Every TUI save goes through SaveScheduler, and settings_actions::save_config is gone. Forced saves flush, which is awaited and still durable before success is reported. flush now leaves the config dirty while an older background save is in flight, so that save cannot have the last word. One spawn shape: a Job trait plus background_dispatch::dispatch, used by most spawners and by community-context's VTA work. Remaining:

  • Some spawners still claim and spawn directly, because preparation with side effects precedes the claim: DeleteDid, VRC request, inbox, relationship, witness, mediator reconnect.
  • Elections ticket and cast, the Rooms VTA call and the repos probe each need their own job type.
  • The vetting jobs claim their domain elsewhere.
  • Vetting's and community_context's loop handlers are not reducer + effect, and sync_from_config is not split per panel.

#556 The published state is Arc<State> behind a Shared<T> view, instead of three deep copies per turn. There is no dirty tracking (R23's objection stands). The session-reconcile arm skips the broadcast only when it changed nothing; the 2 s VTA-sync tick always republishes, so time-based displays keep moving.

#555 The post-quantum proof check is remembered per credential. did-git-sign detection runs off the loop, at startup and when a view that shows it opens; sync_from_config no longer reads files.

#565 VTA retry back-off is jittered. OPENVTC_DEBUG_LOG rotates past 50 MiB. Relationship keys are looked up by id prefix through a sorted map. Persona labels and membership counts are maps built once per sync.

#557 (rest) regenerate_persona_keys takes &TDK. Personas hydrate three at a time, in a fixed order; one that fails no longer stops the rest.

#537 (part)

  • Runtime arm tests and a join-saga test with a failure injected at every stage.
  • The cancel, resolve-queue and stale-outcome tests, and the submit-stage tests.
  • A new test-ignored CI job runs the #[ignore]d e2e suites on every PR (SHA-pinned, --locked, 60 min timeout).
  • CONTRIBUTING states the policy: a test is ignored only for being slow or needing an in-process service, never for being flaky.
  • Coverage floors are not set.

Also: bip39 moves to openvtc-core's dev-dependencies (only tests use it; cargo 1.100 warned).

Review

An independent review traced:

  • every exit of main's join flow against the driver: outcomes, messages and persisted effects;
  • compensation;
  • key queuing;
  • PendingReplies;
  • Shared<T>;
  • the save path.

It found two medium issues. Exit during the submit forgot the persona after the request was sent, and an early community reply could be dropped. Both are fixed here with tests. It also found three low issues, all fixed: the overlapping flush, guards that left a launch locked, and slower time-based redraws.

Behaviour changes (R3.6)

  • The app no longer freezes during a join. Esc, F10 or an error cancels and undoes any stage before the submit. During the submit, an exit waits for it.
  • "Joining a community failed (internal error)." appears if a join job panics.
  • A failed submit no longer forgets a reused persona.
  • Personas after one that fails to load are still loaded. Their progress lines can interleave.
  • No wire types or config formats changed.

Testing

2,369 passed / 37 ignored on main, and 2,436 passed / 37 ignored on this branch (default features, and also with --no-default-features). --include-ignored: 2,473 passed, 0 failed. -p openvtc --features dev-overrides: 1,100 passed.

All exit 0: cargo fmt --all --check, clippy -D warnings (default, --no-default-features, dev-overrides), RUSTDOCFLAGS=-D warnings cargo doc --workspace --no-deps, cargo machete. Not tried against a live stack.

Pre-merge checklist

  • No new reqwest::Client::new() / bare fetch(); all clients have finite timeouts (R1.2): none added
  • No lock held across a network await (R1.3): join I/O runs in off-loop jobs with owned inputs
  • No local state committed before its remote effect, or the flow is resumable with an idempotency key (R2.1): the saga records each step before its effect and compensates in reverse; the submit is not compensated once it may have gone out
  • Every retry is bounded + backed off; non-idempotent ops are not blind-retried (R1.4): VTA retry is bounded at 3 and now jittered; persona hydration is bounded at 3 concurrent
  • Accept/poll/listen loops survive transient errors (R1.5): the inbound and timer arms are unchanged; held inbound is bounded (64)
  • Acks/deletes happen only after durable handoff (R1.6): unchanged
  • New/changed wire types: camelCase, deny_unknown_fields where security-relevant, schema registered, all consumers (incl. JS) updated (R3.*): none
  • Config absence = most restrictive; fail-closed if enforcement can't start (R5.*): unchanged
  • Logs/status claim only what was verified; background-job failures are surfaced (R6.*): a join-job panic is reported; a stopped mid-submit join says the request may have been sent
  • "Process dies on the next line" answered for every mutation touched (R2.1): saga compensation plus forced flushes; a flush overlapped by an older save is re-saved
  • Deviations from this guide flagged explicitly with rule numbers: none

…#528)

The ~2,500-line StateHandler::main_loop kept some thirty loop-local
variables and 21 select! arms in one function nothing could call from a
test. Those locals are now the fields of state_handler::runtime::Runtime,
each arm is one method on it (runtime/arms.rs, inbound_arm.rs, timers.rs),
and Runtime::run is a router over them plus the end-of-iteration work.

Pure restructuring: the arm bodies moved verbatim (each method
destructures the fields it uses, so the bodies read as they did with
locals), the setup after the State-A branch became Runtime::assemble, and
the shutdown sequence Runtime::shutdown. No behaviour change.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
Assembles a real Runtime on an offline TDK, a listener-less messaging
runtime and no admin session, and drives single arms (listener and
lifecycle events, a failed save, a dispatch outcome, the VTA sync tick, a
pong, the reply-window sweep, the pending-expiry sweep) and the whole loop
until Exit or an interrupt stops it.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
The state channel is now watch::Sender<Arc<State>>. Publishing clones the
state once, into the Arc (PublishState::publish), and the UI's copies
share it: the main page and join flow props hold a Shared<T> projection of
the snapshot instead of cloning the main-page model and the join state on
every update, and the setup and loading props are refreshed only while
their page is on screen. A page that edits its props locally still can:
the first write copies the part, as every update used to.

Every publish still sends — no dirty tracking, so R23's stale-UI objection
does not apply. The two timer arms that can prove they changed nothing
(the VTA sync tick that starts no cycle, the session reconcile that
changes no session) skip the end-of-iteration work and its broadcast.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…ves (#529)

learn_over_http, learn_signing and submit each become an owned I/O half
that needs no Config (fetch_over_http, resolve_document, submit_prepared)
and a half that takes the Config on the caller's thread (learn_fetched,
note_signing, prepare_submit), so the TUI can run the I/O off its runtime
loop. The one-call forms remain for community join and are those halves
in sequence; no behaviour change.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…529)

Records ContextRegistered -> Minted -> ListenerUp -> Submitted -> Persisted
as a launch completes them and derives, from that record alone, what undoes
the launch where it stopped: remove the listener it installed, forget the
persona it minted; nothing once the request is out. Replaces the TUI's
out-parameters and hand-placed rollbacks (wired in the next commit).

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
Equal jitter over the existing exponential step: each retry waits between
half and all of 0.5s * 2^(n-1), so clients that failed together do not
retry in lockstep. Refs #565.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
At open, a log already over the cap is moved to <path>.1 (replacing any
earlier rotation) and a fresh file is started, so long debug sessions
keep at most two files. Refs #565.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…g key_info

generate_profiles and repair_key_info_ids scanned the whole key_info map
per relationship with starts_with. Sort the ids once and range-query
them: exactly the same matches, O((keys + relationships) log keys).
Refs #565.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…sync

finish_credential_rows serialised every VRC to read its proofs on each
sync_from_config. Remember the answer by credential id (a VRC's id is
its proof value) across syncs; each sync keeps only the ids it shows,
so the memo is bounded by the credentials on display. Refs #555.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
finish_credential_rows scanned the personas for every self-subject row,
and the persona list counted memberships with a filter per persona.
Build both maps once per sync_from_config. Refs #565.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
… sync

sync_from_config read did-git-sign's SigningConfig and allowed_signers on
every config change. Detection now runs on a blocking thread as a
DidGitSign dispatch and is kept in SettingsState with the persona DID it
was read for: a sync for another persona hides it, and a refresh is due
on a persona switch, after a Repos signing change, or when Help / Status
or Settings is opened. The Repos tick checks for a due refresh; a check
is a few comparisons, no I/O. Refs #555.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
It only reads the TDK (the secrets resolver is shared state), so a
shared borrow lets personas be hydrated concurrently. Refs #557.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
Each persona's document resolve, key fetch and profile registration ran
one persona after another. They share only read-only handles, so run
them with a bound of three and record the results in account order:
the degraded report and identities are the same whichever finishes
first, and a fault no persona can work around still fails the load.
Refs #557.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…ws it (#555)

Wires the off-loop did-git-sign refresh into the runtime: once when the
loop starts, and after each action when a view that shows it has just
opened (a cheap check, no I/O). The 5-second sweep still catches anything
else.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
… config, sign without it (#529)

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
… with no nested event loop (#529)

The join flow was a nested select! loop the runtime awaited inline, parking
every other arm while the join screen was up. It is now a JoinDriver state
machine both the runtime loop and the State-A loop drive: page logic runs
on the loop, network work runs as off-loop jobs on DispatchDomain::Join
(matched by generation, aborted on Esc/F10), join keys pressed during a job
are queued and replayed, and a launch is a JoinSaga whose failures, cancels
and interrupts undo it from its compensations. A reused persona is no
longer forgotten on a failed submit. The old StateHandler::join_flow,
run_join_sequence and race_against_interrupt are removed.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…and a resolve queued behind and aborted (#529)

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…by thread

One in-memory table (thread -> community, purpose, when) for the replies
admitted only because we asked: renewals, community profiles, and election
answers from a community we hold no membership with. Each kind keeps the
window, cap, thread normalisation and community comparison of the
process-global static it replaces (#534).

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…ndingReplies

renewal.rs PENDING, join.rs PROFILE_QUERIES and elections.rs AWAITING are
gone. The runtime loop owns one PendingReplies and passes it explicitly:
the send paths record into it on the loop thread before spawning (a
renewal's id and a profile question's id are now chosen there), and
inbound dispatch (verification_job, the community-reply gate, the renewal,
refusal and profile handlers) consumes from it. Elections records and
forgets through its Loop, its outcome and its reply path; a failed renewal
send forgets its request in CommunityOutcome::apply. Headless commands
(join, repos) hold their own table for apply_inline, and the core tests
each use a local one instead of sharing process globals (#534).

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…community

Rooms and Elections now share one request/reply engine
(state_handler/community_request.rs) behind a small CommunityFeature
trait: arming the view before the send is spawned, the send job and its
outcome, taking an answer only on its thread from the community asked,
the 30 s reply window, job loss, and keeping a step whose lane is busy.
Elections' threads are recorded at, and forgotten from, the runtime's
PendingReplies by the engine (RECORDS_THREAD). Each panel keeps what is
its own: building requests, what an answer, a timeout or a failure says,
and its VTA calls.

One CommunityLoop replaces the three Loop structs (re-exported as
rooms_actions::Loop, elections_actions::Loop, repos_actions::Loop).
Repos keeps its two-slot wait (request and link poll, per-purpose
windows) but takes the shared loop, Sender/sender/sign_and_send and
answered_by from the module; did_part and capitalise exist once.
No message, timeout or correlation rule changes (#534).

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…hrough SaveScheduler (#535)

Delete settings_actions::save_config. The passphrase set/remove and the
export/import log entries force-flush with SaveScheduler::flush (awaited,
so still durable before success is reported), which also clears the
pending dirty mark; clear_after_external_save has no callers left and goes.

A standalone persona mint is written in memory (MintedPersona::write_into
over the new ConfigExtension::mint_persona_in_memory) and settle_after_apply
force-flushes it through the scheduler, off the runtime. The headless join
keeps its own save. mint_persona_into (join driver, setup) still saves
itself.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…_dispatch::dispatch (#535)

A job names its DispatchDomain and its I/O (Job::execute, returning the
DispatchOutcome); dispatch(job, in_flight, dispatch_tx, on_busy) claims
the domain and spawns it, or drops it unrun and hands on_busy the busy
message. Work due only once the claim succeeded is done by the caller on
true, in the same loop turn as the spawn, so no outcome can overtake it.

On it: spawn_persona_mint, spawn_community_job, spawn_capability_job,
spawn_vic_mutation, spawn_agent_name_job, spawn_vic_refresh (its
begin_vic_refresh guard folds in; the test drives spawn_vic_refresh),
spawn_persona_effect, dispatch_binding_refresh, the timer sweeps (agent
names, community contexts, device grants, VTA transports), the
did-git-sign detection, community_request's sends, the Repos send and
workspace change, and the hidden-vetting job.

Tests: dispatch claims the job's own domain and frees it when applied; a
job for a busy domain is refused unrun with the domain's busy message.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…#535)

The preview, delete, grant listing, grant and revoke jobs were ad-hoc
futures spawned after a claim helper that checked the session and the
domain together. They are now one ContextJob enum on the Job trait: the
session is checked first (same message), then dispatch claims and spawns,
and the panel is committed only once the claim succeeded — as before.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
Also moves bip39 to openvtc-core's dev-dependencies: only the BIP32 tests use it.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
…ity's replies; a flush survives an older save

From review of the runtime restructure:
- F10/Esc/an error during the submit stage wait for it instead of aborting
  it and forgetting a persona the community may now hold; a loop stop
  mid-submit keeps the persona and listener.
- Inbound messages from the community being submitted to are held until
  the pending record exists, then processed (the nested loop got this by
  not reading inbound).
- SaveScheduler::flush leaves the config dirty while an older background
  save is in flight, so that save cannot have the last word.
- Launch guards that found no persona/applicant now fail and undo instead
  of leaving the page locked.
- The 2 s VTA-sync tick republishes state again, so time-based displays
  keep moving.

Signed-off-by: Glenn Gore <glenn.g@affinidi.com>
@stormer78
stormer78 requested a review from a team as a code owner November 15, 2026 12:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment