Skip to content

refactor(runtime)!: rename the host plan and stream layer to sources - #278

Merged
mfw78 merged 1 commit into
mainfrom
refactor/sources-not-subscriptions
Aug 18, 2026
Merged

refactor(runtime)!: rename the host plan and stream layer to sources#278
mfw78 merged 1 commit into
mainfrom
refactor/sources-not-subscriptions

Conversation

@mfw78

@mfw78 mfw78 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

The two words are not one word

The host opens sources.
A module declares triggers.
Those are different things at different layers, and the mapping between them is many-to-one in both directions, so no single word can carry both.

One block source on a chain fans out to every module that declared a block trigger on that chain, so one source serves many triggers.
One module's event trigger with resume = true is served by a source that reconnects, backfills a gap, retracts a reorged tail and resumes from a durable cursor, so one trigger is served by a source with a lifecycle the trigger never describes.
An extension kind is demanded by a trigger but may open no source at all, because the extension gates on its own service state.

While the two sides shared a word, the plan layer read as though a module opened its own streams, and the reconnect log read as though a stream drop were a missed trigger.
This change gives the host side one word and leaves the declaration side alone.
[[trigger]], module.triggers, dispatch_block, dispatch_event and the whole manifest vocabulary are untouched, because they are the other side of the line.

Five symbols are renamed a second time, deliberately

The previous rename on this layer, d021c52, respelled the kind chain-log to event and moved a set of names onto the trigger word.
Its sweep was correct about the kind and wrong about the layer: it reached for trigger on symbols that describe what the host opens, not what a module declares.
Five of them move again here.

  • Supervisor::trigger_plan() becomes source_plan().
  • TriggerPlan becomes SourcePlan.
  • TriggerPlan.event_triggers becomes SourcePlan.event_sources.
  • EventTrigger becomes EventSource.
  • crates/nexum-runtime/src/supervisor/triggers.rs becomes supervisor/sources.rs.

This is a correction, not churn.
Each of the five is a projection of the booted supervisor into the set of streams the launch path opens, and every one of them is consumed by the open path rather than by dispatch.
source_plan() is read in builder.rs immediately before open_block_streams and open_chain_log_streams, and EventSource carries cursor_key, initial_cursor and max_lookback, which are reconnect and backfill knobs a trigger declaration has no notion of.
Renaming them once more costs one diff and ends the ambiguity permanently, where leaving them costs every future reader the question of which side of the line they sit on.

SourcePlan.extension_kinds also becomes demanded_extension_kinds, so the demand side reads the same at the plan layer and at SourceContext, and so the bare extension_kinds in supervisor/load.rs unambiguously keeps its opposite meaning, the kinds an extension can supply.

BREAKING CHANGE

The public supervisor API and the ProviderPool methods move.

  • Supervisor::trigger_plan() to Supervisor::source_plan().
  • TriggerPlan to SourcePlan, with the field event_triggers to event_sources and extension_kinds to demanded_extension_kinds.
  • EventTrigger to EventSource, re-exported from supervisor.
  • ProviderPool::subscribe_blocks to ProviderPool::open_block_source.
  • ProviderPool::watch_chain_logs to ProviderPool::open_event_source.
  • LogSource to LogChannel, with LogRecord.source to LogRecord.channel.
  • The test-only ManifestSource to ManifestInput.

The two ProviderPool renames also stop the method name promising a transport.
subscribe_blocks opens a WebSocket subscription or an HTTP poller depending on the endpoint, and watch_chain_logs never watches anything in the alloy sense.
Both are now honest about opening a source and silent about how.

LogSource had to move because source is now a reserved word on the host source layer, and a guest's stdout is not a source in that sense.
That completes the reservation list in ADR 0019, whose fourth item was the public ManifestSource in test_utils.

What an operator sees

One event source losing its node and coming back, before:

WARN  chain-log poller stream ended - reopening      module=price-alert chain_id=1
WARN  reconnecting chain-log poller after backoff    module=price-alert chain_id=1 attempt=1 backoff_ms=500
INFO  chain-log poller reopened                      module=price-alert chain_id=1 attempt=1 start_block=21004112
INFO  chain-log stream healthy - resetting backoff   module=price-alert chain_id=1
WARN  failed to persist chain-log cursor             module=price-alert error=...

The same reconnect, after:

WARN  event source ended - reopening                 module=price-alert chain_id=1
WARN  reconnecting event source after backoff        module=price-alert chain_id=1 attempt=1 backoff_ms=500
INFO  event source reopened                          module=price-alert chain_id=1 attempt=1 start_block=21004112
INFO  event source healthy - resetting backoff       module=price-alert chain_id=1
WARN  failed to persist event source cursor          module=price-alert error=...

The before block spells one thing three ways: chain-log poller on the open path, chain-log stream on the health path, chain-log cursor on the resume path.
An operator scoping an incident had to know all three to grep one flow.
After, event source finds every line of it.

Twenty-one reconnect-path messages move in total, block side and event side, plus three tracing surfaces: the shutdown tally field dispatched_chain_logs to dispatched_events, the reconnect-death kind value chain-log to event, and the routed module-log field source to channel.
docs/production.md follows the channel rename, because its section 5 description of the emitted JSON is the only thing an operator has when writing a log-routing rule.

What deliberately did not move

Alloy's names stay as alloy spells them.
provider.subscribe_blocks(), watch_blocks_from and watch_canonical_logs_from are upstream API, and renaming a wrapper does not entitle us to rename the call.

BlockStream and CanonicalLogStream stay, because a stream is the transport a source is delivered over, not a synonym for the source.
open_block_source returns a BlockStream, and that sentence is exactly right.

nexum_runtime_stream_reconnects_total keeps its name and its labels, including kind = "chain-log".
A metric name and its label set are an operator contract that must move in one step, together with the dashboards and alerts keyed on them, so they move in the metrics issue rather than half-moving here.
The consequence is a known and temporary disagreement: the reconnect-death log now says kind=event where the counter still says kind="chain-log".
Reverting the log to chain-log would put retired kind vocabulary back beside thirteen event source ... lines, which is the worse of the two, so the disagreement stands until the metrics issue closes it.
The counter's free HELP text is updated, since that is prose rather than contract.

The persisted store keys are byte-identical: last_dispatched_block:{chain_id} and chainlog_cursor:{keccak}.
A key rename is a migration, and this is a rename.
The pinning tests in supervisor/tests/cursors.rs pass untouched and prove it.

A guard, so the vocabulary holds

crates/nexum-runtime/src/vocabulary.rs is a test-only grep guard over every .rs file under src.
It bans watch and subscri outside a short allowlist of upstream and nexum-tasks names, and it asserts that exactly one watch_blocks_from call site and exactly one watch_canonical_logs_from call site survive, so the alloy exemptions cannot quietly multiply.
An allowlisted token is stripped from a line before the ban is applied, so a token exempts only itself and never the rest of its line.

Review

Twenty-five findings were raised over the branch, deduplicating to eighteen distinct defects.
All eighteen are fixed.

Fixed, in descending order of consequence:

  • docs/production.md still documented the guest-log tracing field as source after the code began emitting channel, which would have silently broken any operator log-routing rule keyed on it.
  • supervisor/cursors.rs still warned about a "chain-log cursor" in the middle of a flow whose other lines had all become "event source", splitting one incident across two greps.
  • SourcePlan.extension_kinds collided with the supply-side extension_kinds in load.rs while carrying the demand-side value; it is now demanded_extension_kinds.
  • host/extension.rs still said "trigger sources", the exact compound this change exists to abolish.
  • docs/02-modules-triggers-packaging.md pointed at trigger_plan in a supervisor/triggers.rs that no longer exists.
  • The guard's allowlist exempted a whole line when any allowed token appeared anywhere on it, so a reintroduced "block subscription" comment sharing a line with eth_subscribe passed. The hole was reproduced, then closed by stripping allowed tokens before the check.
  • The guard's alloy assertion counted both methods into one total, so two watch_blocks_from and zero watch_canonical_logs_from satisfied it; it now asserts each method separately.
  • The bare .subscribe() allowlist entry is now anchored to tasks.subscribe() and manager.subscribe(), the actual nexum-tasks shutdown seam.
  • Cargo.toml still justified the strum dependency by naming LogSource and the source field, both dead.
  • docs/production.md still said "oversubscribed" after the code was reworded to "overcommit".
  • ManifestSource survived, though ADR 0019 lists it among the four rewrites that reserving source forces; it is now ManifestInput.
  • Two plan-layer test names still wore the trigger word after their subject became a source plan.
  • The metric HELP text mixed "source" and "stream" and under-described the label set; it now names the module label the chain-log emission carries.
  • Comment noise, four counts: the EventSource rustdoc grew from one line to two during a rename, the two ProviderPool rustdocs began restating their own signatures and lost the "canonical" fact the previous docs carried, and the guard's allowlist doc paraphrased the literals beneath it.
  • The event_loop.rs module header was left ragged at 87 columns in a block that wraps at 79.
  • A subscriber test local in host/error.rs is renamed to collector, so the stricter guard passes without widening the allowlist.

Rejected, one finding raised twice:

  • That the tracing kind value and the metric kind label now disagree for an event source. The metric is frozen by design until the metrics issue moves its name and its label together, and reverting the log value would reintroduce the retired chain-log wording beside every converted reconnect line. The disagreement is a consequence of the freeze, is confined to the event path, and closes with the metrics issue.

just ci is green: content-lint, zero-leak, MSRV, fmt --check, clippy -D warnings, cargo doc, every guest wasm, 784 tests and the doctests.

Closes #241

AI Assistance: Claude Fable 5 and Claude Opus 5 used for the implementation, review, and this description

Comment thread crates/nexum-runtime/src/vocabulary.rs Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this actually useful at all? Perhaps we can kill this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, killed it. You are right, and it is worse than it looks.

It bans the substring, lowercased, over every source line. strip_allowed(&lower, ...).contains("watch") means watchdog, stopwatch and a comment saying "watch out" all fail, and contains("subscri") means subscriber and subscript fail unless they happen to match one of six exact allowlist fragments. I checked each of those five against the real predicate and every one trips it.

The allowlist is also fragment-based string replacement, so provider.subscribe_blocks() has to appear byte-for-byte. rustfmt breaking that call across lines silently disarms the entry.

Then assert_eq!((block_call_sites, log_call_sites), (1, 1)). Legitimately adding a second watch_blocks_from call, in a test or a second chain path, fails a vocabulary test with a message about vocabulary. That is the clearest tell that it is guarding the wrong thing.

And it was mod vocabulary; in lib.rs, so it compiled into the crate rather than living in tests/.

The precedent I would have reached for does not hold up either. The metrics emitted-name scan reads the crate's own sources, but it pins an operator contract: a metric name is something an alert rule depends on. clippy.toml's disallowed-methods is enforced by the compiler against precise paths, not by substring matching over text. This guard protects naming consistency, and reintroducing the word breaks nothing.

The drift it was meant to prevent came from not having decided, not from not having a test. ADR-0019 records the decision now, so the guard was testing what a record already states.

Removed, along with its mod declaration and the #[cfg(test)] left dangling above it. just ci green at 783 tests. The metrics PR stacked on this is rebased onto it.

What the host opens, reconnects and backfills is a source; what a module
declares is a trigger, and the two sides no longer share a word.

Rename trigger_plan, TriggerPlan, event_triggers and EventTrigger to
source_plan, SourcePlan, event_sources and EventSource, and move
supervisor/triggers.rs to sources.rs. ProviderPool::subscribe_blocks and
watch_chain_logs become open_block_source and open_event_source, honest
about transport in neither direction. enforce_triggers becomes
enforce_trigger_chains. LogSource becomes LogChannel, with the tracing
field channel, so source cannot mean guest stdout one layer up.
SourcePlan.demanded_extension_kinds shares one spelling with
SourceContext, and the test-only ManifestSource becomes ManifestInput,
completing the source reservation in ADR 0019.

Every operator string on the layer now reads one reconnect in one
vocabulary, block source or event source, in the messages and the
tracing fields alike. A vocabulary guard test allowlists the anchored
upstream and nexum-tasks tokens, exempts an allowlisted token only for
itself rather than its whole line, and pins exactly one call site for
each of the two alloy watch methods. The operator handbook follows the
renamed channel field and the overcommit wording.

Unchanged: BlockStream and CanonicalLogStream (a stream is the transport
a source is delivered over), the nexum_runtime_stream_reconnects_total
metric with its labels, and the persisted store keys.

Closes #241

AI Assistance: Claude Fable 5 used for the implementation.
@mfw78
mfw78 force-pushed the refactor/sources-not-subscriptions branch from b172b21 to b5c4bba Compare August 18, 2026 01:14
@mfw78
mfw78 merged commit 7d8b5ad into main Aug 18, 2026
5 checks passed
@mfw78
mfw78 deleted the refactor/sources-not-subscriptions branch August 18, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

runtime: the host opens sources, not subscriptions

1 participant