Leios prototype: Update tx-firehose to take a --color and add a mempool-monitor to visualize these - #6660
Merged
Merged
Conversation
--color ff0000 or --color auto puts three bytes of RGB into metadata label 1022, so a mempool observer can attribute each tx to the generator that made it. That is what makes mempool fragmentation visible when several generators feed different corners of a network. auto takes a hue from the verification key hash and fixes saturation and lightness, since hash bytes read directly as RGB leave a good share of keys dark or muddy, which is the opposite of what colouring is for. Builds the auxiliary data through cardano-api's toAuxiliaryData for the same reason signing already goes through cardano-api: it case-analyses the era, so the aux-data constraints resolve where an era-generic build cannot. The hash lands in the body before the witness is computed, which the witness covers.
Walks a node's mempool over the node-to-client LocalTxMonitor protocol and reports which colours it holds, tallying the metadata tag tx-firehose writes. One instance per node on purpose: fragmentation is a statement about how pools differ, so an aggregate hides the thing under test. A snapshot is acquired once per round and walked within it, so the tally is consistent. Depth and capacity come from GetSizes, a single message, which leaves the per-tx round trips paying only for colour. Hence the ten second default interval. The drained count doubles as a check against the size GetSizes reports for the same snapshot. Reads the ledger tx's auxiliary data directly rather than through cardano-api's body view, which would build a record per tx across tens of thousands of them, and is deprecated besides. Output follows the handle rather than a flag: a repainting pane on a terminal, one line per snapshot in a log, plus optional TSV for later analysis.
Both submit events now carry a color field, so a transaction is attributable from the log and not only from its metadata. Useful when reconciling what a generator sent against what an observer found in a mempool. Absent rather than null when --color is unset, so uncoloured runs keep the log shape the digest scripts already read.
GetSizes reports capacityInBytes as txMeasureByteSize of the mempool's capacity, that is one projection of a multi-dimensional measure. The mempool's own measure is TxMeasureWithDiffTime, and every TxMeasureMetrics method forgets the DiffTime, so the validation-time budget reaches neither GetSizes nor GetMeasures. That budget is what binds first on the Leios prototype, where the bounded configuration stopped accepting around 34k transactions on roughly five seconds of validation time rather than on bytes. An unlabelled bar therefore reads as fullness while showing a dimension with plenty of room left.
The delay ran after the drain, so a 2 s drain on a 10 s interval gave a 12 s cadence: the requested rate was never the actual one, and the error grew with mempool depth. Now it sleeps only the remainder of the period, so the cadence is what was asked for, and a drain that overruns degrades to draining continuously rather than quietly stretching.
A restart reopens the file in append mode and wrote another header into the middle of it, which breaks every reader downstream. A dozen-devnet run already had six of them in one file.
ch1bo
force-pushed
the
ch1bo/firehose-color
branch
from
August 29, 2026 15:12
c34a7c7 to
e0bfe2a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Used in input-output-hk/ouroboros-leios#1057 for input-output-hk/ouroboros-leios#1022