Repository navigation
Conversation
A run can finish while some records were permanently rejected by the store, or while a STREAM_EVIDENCE stream left keys unaccounted. The server already records both: per-batch and run-total records_permanently_rejected, outcomes.unaccounted on run.stream_evidence_declared, and per-connection record-rejection receipts. The run page showed none of them, so a run could read as clean while records were missing. The page now has a section that lists the rejected-record receipts for the run (stream, reason, first input index, size) and the unaccounted count per stream. No server change: the receipts route and the events exist on main, so a new console works with an older server (the receipt read fails soft and the count from run events remains) and an older console is unaffected. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
The run page summed rejected records from one timeline page, and rendered nothing when the sum was 0. For a long run, or for events that predate the field, that looked the same as "nothing was rejected". - Run status now carries the terminal event's records_permanently_rejected (window-independent, like known_gaps). The console reads it first. - The summary states a count only when it can prove it: final, so far, at least, or unavailable. An unreadable count renders as unavailable. - Unaccounted streams are flagged as not listed when the page is not the whole timeline, and a stream that declares evidence twice is merged. - The receipt list reports a cut-short read and an unreadable list. - Drop the claim that the other records were stored: retryable and unaccounted records are not. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
…nknown counts The rejected-record summary kept its own list of terminal events and left out run.abandoned. An abandoned run with no total on its event, or a finished run whose terminal event was not on the loaded page, could be reported as "so far", which says the run is still going. The summary now takes the page's authoritative liveness as an input, reports "so far" only for an active run with every event loaded, and reads the total from run.abandoned like the other terminal events. The page read the run's receipts only when the count was above zero, and an unknown count counted as zero. An unknown count is when the receipts are the only readable evidence, so they are now read when the count is unknown or positive and the connection is known. A known zero still reads none. The data flow moves into loadRunRejectionView so a test can drive it without rendering the page. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The run detail page now shows records a run could not store or account for:
Why
A run can succeed while the store permanently rejected some records (per-record isolation keeps the cursor moving), or while a stream left keys unaccounted (spec behavior). Both are recorded server-side, but the run page rendered neither, so such a run looked clean.
This PR keeps both behaviors and makes them visible. It does not hold the cursor and adds no table.
How
Data already on main:
run.batch_ingestedcarriesrecords_permanently_rejected, and the terminal event carries the run total.run.stream_evidence_declaredcarriesoutcomes.unaccounted.GET /_ref/connections/:id/record-rejectionslists receipts, oldest first, with no run filter.One timeline page holds only part of a long run, and the terminal event is emitted last. A count summed from one page can therefore miss the batch and terminal events, and a page without them is not evidence of zero. So the run total comes from the run status, the same window-independent source as
known_gaps:GET /_ref/runs/:runIdpasses the terminal event'srecords_permanently_rejectedthrough, only when it is a non-negative integer. It is absent while the run is active and for runs that predate the field.summarizeRunRejectionsreturns one of four states and never reports a zero it cannot prove. A terminal event of any kind, includingrun.abandoned, can supply the total.final: the run status total, or the terminal event's total. A reported zero renders no section.so_far: the run is still active and the whole timeline is loaded, so the batch sum is the total to now. Liveness is the page's own (run status, else the timeline envelope's terminal status); the summary keeps no list of terminal events for it. A finished run with no total, such as an abandoned run, is neverso_far.at_least: part of the run is not on this page; the batch sum is a lower bound.unknown: nothing readable. The section says the count is not available here.Mixed versions: a new console with an older server has no status total, so it shows
so_far,at_leastorunknownfrom the events it has, never a silent zero. An older console with a newer server ignores the extra field.Tests
run-rejections.test.ts(18): each state, a reported zero, the terminal total beating the batch sum, an abandoned run with and without a total, a finished run whose terminal event is not on the page (neverso_far), events without the field, evidence merged by stream, receipts narrowed to the run with the cursor kept, and the page's receipt read throughloadRunRejectionView(old-server status with a partial timeline reads the receipts; a known zero, or no connection, reads none; a failed read gives no list).rejected-records-section.test.ts(11): count and receipts, unaccounted count, no "other records" claim, lower bound, in progress, unavailable count, an unknown count that still lists receipts, other-page streams not listed, receipts unavailable, receipts partial, nothing rendered for a known zero.run-status-body-builders-pure.test.tsandref-run-status-route.test.ts: the total passes through from the terminal event (also a reported zero), and is omitted when absent or invalid. The route test uses the file-backed database; PostgreSQL was not run, since the new field readsterminal.data, the objectknown_gapsalready comes from.The new console tests fail on
main, where the modules do not exist.ultracite checkreports no lint-rule diagnostics for the touched files. It reports formatting errors (it expects tab indentation, while existing files in these directories, including unmodified ones, use spaces) and unused-suppression warnings, all present on main.Limits
{events.length} events, the terminal known gaps, and the failure and stderr sections.Assisted-by: AI