refactor(runtime)!: rename the host plan and stream layer to sources - #278
Conversation
There was a problem hiding this comment.
Is this actually useful at all? Perhaps we can kill this?
There was a problem hiding this comment.
Agreed, killed it. You are right, and it is worse than it looks.
It bans the substring, lowercased, over every source line. strip_allowed(&lower, ...).contains("watch") means watchdog, stopwatch and a comment saying "watch out" all fail, and contains("subscri") means subscriber and subscript fail unless they happen to match one of six exact allowlist fragments. I checked each of those five against the real predicate and every one trips it.
The allowlist is also fragment-based string replacement, so provider.subscribe_blocks() has to appear byte-for-byte. rustfmt breaking that call across lines silently disarms the entry.
Then assert_eq!((block_call_sites, log_call_sites), (1, 1)). Legitimately adding a second watch_blocks_from call, in a test or a second chain path, fails a vocabulary test with a message about vocabulary. That is the clearest tell that it is guarding the wrong thing.
And it was mod vocabulary; in lib.rs, so it compiled into the crate rather than living in tests/.
The precedent I would have reached for does not hold up either. The metrics emitted-name scan reads the crate's own sources, but it pins an operator contract: a metric name is something an alert rule depends on. clippy.toml's disallowed-methods is enforced by the compiler against precise paths, not by substring matching over text. This guard protects naming consistency, and reintroducing the word breaks nothing.
The drift it was meant to prevent came from not having decided, not from not having a test. ADR-0019 records the decision now, so the guard was testing what a record already states.
Removed, along with its mod declaration and the #[cfg(test)] left dangling above it. just ci green at 783 tests. The metrics PR stacked on this is rebased onto it.
What the host opens, reconnects and backfills is a source; what a module declares is a trigger, and the two sides no longer share a word. Rename trigger_plan, TriggerPlan, event_triggers and EventTrigger to source_plan, SourcePlan, event_sources and EventSource, and move supervisor/triggers.rs to sources.rs. ProviderPool::subscribe_blocks and watch_chain_logs become open_block_source and open_event_source, honest about transport in neither direction. enforce_triggers becomes enforce_trigger_chains. LogSource becomes LogChannel, with the tracing field channel, so source cannot mean guest stdout one layer up. SourcePlan.demanded_extension_kinds shares one spelling with SourceContext, and the test-only ManifestSource becomes ManifestInput, completing the source reservation in ADR 0019. Every operator string on the layer now reads one reconnect in one vocabulary, block source or event source, in the messages and the tracing fields alike. A vocabulary guard test allowlists the anchored upstream and nexum-tasks tokens, exempts an allowlisted token only for itself rather than its whole line, and pins exactly one call site for each of the two alloy watch methods. The operator handbook follows the renamed channel field and the overcommit wording. Unchanged: BlockStream and CanonicalLogStream (a stream is the transport a source is delivered over), the nexum_runtime_stream_reconnects_total metric with its labels, and the persisted store keys. Closes #241 AI Assistance: Claude Fable 5 used for the implementation.
b172b21 to
b5c4bba
Compare
The two words are not one word
The host opens sources.
A module declares triggers.
Those are different things at different layers, and the mapping between them is many-to-one in both directions, so no single word can carry both.
One block source on a chain fans out to every module that declared a block trigger on that chain, so one source serves many triggers.
One module's event trigger with
resume = trueis served by a source that reconnects, backfills a gap, retracts a reorged tail and resumes from a durable cursor, so one trigger is served by a source with a lifecycle the trigger never describes.An extension kind is demanded by a trigger but may open no source at all, because the extension gates on its own service state.
While the two sides shared a word, the plan layer read as though a module opened its own streams, and the reconnect log read as though a stream drop were a missed trigger.
This change gives the host side one word and leaves the declaration side alone.
[[trigger]],module.triggers,dispatch_block,dispatch_eventand the whole manifest vocabulary are untouched, because they are the other side of the line.Five symbols are renamed a second time, deliberately
The previous rename on this layer, d021c52, respelled the kind
chain-logtoeventand moved a set of names onto the trigger word.Its sweep was correct about the kind and wrong about the layer: it reached for
triggeron symbols that describe what the host opens, not what a module declares.Five of them move again here.
Supervisor::trigger_plan()becomessource_plan().TriggerPlanbecomesSourcePlan.TriggerPlan.event_triggersbecomesSourcePlan.event_sources.EventTriggerbecomesEventSource.crates/nexum-runtime/src/supervisor/triggers.rsbecomessupervisor/sources.rs.This is a correction, not churn.
Each of the five is a projection of the booted supervisor into the set of streams the launch path opens, and every one of them is consumed by the open path rather than by dispatch.
source_plan()is read inbuilder.rsimmediately beforeopen_block_streamsandopen_chain_log_streams, andEventSourcecarriescursor_key,initial_cursorandmax_lookback, which are reconnect and backfill knobs a trigger declaration has no notion of.Renaming them once more costs one diff and ends the ambiguity permanently, where leaving them costs every future reader the question of which side of the line they sit on.
SourcePlan.extension_kindsalso becomesdemanded_extension_kinds, so the demand side reads the same at the plan layer and atSourceContext, and so the bareextension_kindsinsupervisor/load.rsunambiguously keeps its opposite meaning, the kinds an extension can supply.BREAKING CHANGE
The public supervisor API and the
ProviderPoolmethods move.Supervisor::trigger_plan()toSupervisor::source_plan().TriggerPlantoSourcePlan, with the fieldevent_triggerstoevent_sourcesandextension_kindstodemanded_extension_kinds.EventTriggertoEventSource, re-exported fromsupervisor.ProviderPool::subscribe_blockstoProviderPool::open_block_source.ProviderPool::watch_chain_logstoProviderPool::open_event_source.LogSourcetoLogChannel, withLogRecord.sourcetoLogRecord.channel.ManifestSourcetoManifestInput.The two
ProviderPoolrenames also stop the method name promising a transport.subscribe_blocksopens a WebSocket subscription or an HTTP poller depending on the endpoint, andwatch_chain_logsnever watches anything in the alloy sense.Both are now honest about opening a source and silent about how.
LogSourcehad to move becausesourceis now a reserved word on the host source layer, and a guest's stdout is not a source in that sense.That completes the reservation list in ADR 0019, whose fourth item was the public
ManifestSourceintest_utils.What an operator sees
One event source losing its node and coming back, before:
The same reconnect, after:
The before block spells one thing three ways:
chain-log polleron the open path,chain-log streamon the health path,chain-log cursoron the resume path.An operator scoping an incident had to know all three to grep one flow.
After,
event sourcefinds every line of it.Twenty-one reconnect-path messages move in total, block side and event side, plus three tracing surfaces: the shutdown tally field
dispatched_chain_logstodispatched_events, the reconnect-deathkindvaluechain-logtoevent, and the routed module-log fieldsourcetochannel.docs/production.mdfollows thechannelrename, because its section 5 description of the emitted JSON is the only thing an operator has when writing a log-routing rule.What deliberately did not move
Alloy's names stay as alloy spells them.
provider.subscribe_blocks(),watch_blocks_fromandwatch_canonical_logs_fromare upstream API, and renaming a wrapper does not entitle us to rename the call.BlockStreamandCanonicalLogStreamstay, because a stream is the transport a source is delivered over, not a synonym for the source.open_block_sourcereturns aBlockStream, and that sentence is exactly right.nexum_runtime_stream_reconnects_totalkeeps its name and its labels, includingkind = "chain-log".A metric name and its label set are an operator contract that must move in one step, together with the dashboards and alerts keyed on them, so they move in the metrics issue rather than half-moving here.
The consequence is a known and temporary disagreement: the reconnect-death log now says
kind=eventwhere the counter still sayskind="chain-log".Reverting the log to
chain-logwould put retired kind vocabulary back beside thirteenevent source ...lines, which is the worse of the two, so the disagreement stands until the metrics issue closes it.The counter's free HELP text is updated, since that is prose rather than contract.
The persisted store keys are byte-identical:
last_dispatched_block:{chain_id}andchainlog_cursor:{keccak}.A key rename is a migration, and this is a rename.
The pinning tests in
supervisor/tests/cursors.rspass untouched and prove it.A guard, so the vocabulary holds
crates/nexum-runtime/src/vocabulary.rsis a test-only grep guard over every.rsfile undersrc.It bans
watchandsubscrioutside a short allowlist of upstream andnexum-tasksnames, and it asserts that exactly onewatch_blocks_fromcall site and exactly onewatch_canonical_logs_fromcall site survive, so the alloy exemptions cannot quietly multiply.An allowlisted token is stripped from a line before the ban is applied, so a token exempts only itself and never the rest of its line.
Review
Twenty-five findings were raised over the branch, deduplicating to eighteen distinct defects.
All eighteen are fixed.
Fixed, in descending order of consequence:
docs/production.mdstill documented the guest-log tracing field assourceafter the code began emittingchannel, which would have silently broken any operator log-routing rule keyed on it.supervisor/cursors.rsstill warned about a "chain-log cursor" in the middle of a flow whose other lines had all become "event source", splitting one incident across two greps.SourcePlan.extension_kindscollided with the supply-sideextension_kindsinload.rswhile carrying the demand-side value; it is nowdemanded_extension_kinds.host/extension.rsstill said "trigger sources", the exact compound this change exists to abolish.docs/02-modules-triggers-packaging.mdpointed attrigger_planin asupervisor/triggers.rsthat no longer exists.eth_subscribepassed. The hole was reproduced, then closed by stripping allowed tokens before the check.watch_blocks_fromand zerowatch_canonical_logs_fromsatisfied it; it now asserts each method separately..subscribe()allowlist entry is now anchored totasks.subscribe()andmanager.subscribe(), the actualnexum-tasksshutdown seam.Cargo.tomlstill justified thestrumdependency by namingLogSourceand thesourcefield, both dead.docs/production.mdstill said "oversubscribed" after the code was reworded to "overcommit".ManifestSourcesurvived, though ADR 0019 lists it among the four rewrites that reservingsourceforces; it is nowManifestInput.modulelabel the chain-log emission carries.EventSourcerustdoc grew from one line to two during a rename, the twoProviderPoolrustdocs began restating their own signatures and lost the "canonical" fact the previous docs carried, and the guard's allowlist doc paraphrased the literals beneath it.event_loop.rsmodule header was left ragged at 87 columns in a block that wraps at 79.subscribertest local inhost/error.rsis renamed tocollector, so the stricter guard passes without widening the allowlist.Rejected, one finding raised twice:
kindvalue and the metrickindlabel now disagree for an event source. The metric is frozen by design until the metrics issue moves its name and its label together, and reverting the log value would reintroduce the retiredchain-logwording beside every converted reconnect line. The disagreement is a consequence of the freeze, is confined to the event path, and closes with the metrics issue.just ciis green: content-lint, zero-leak, MSRV,fmt --check,clippy -D warnings,cargo doc, every guest wasm, 784 tests and the doctests.Closes #241
AI Assistance: Claude Fable 5 and Claude Opus 5 used for the implementation, review, and this description