Repository navigation
fix(vintage): --retry recovers skipped snapshots, not just failed ones - #86
Merged
Merged
Conversation
`skipped` means no canonicaliser existed for that report type at ingest time, and it was exactly as terminal as `failed`: --pending selects neither, and nothing else ever wrote parse_status back. The design notes promise that adding a canonicaliser later just means re-marking these pending and re-running, and there was no way to do the re-marking. Nearly bit today. The disagg and TFF bytes captured 2026-07-30 sat unparsed for two days because no canonicaliser existed yet, and this morning's run recovered 51,735 observations from them. That only worked because the producer's install predated the drain that marks them `skipped`, so they were still `pending`. On a current install they would have been stranded with no command able to reach them. The live case is the weekly static: every retained copy is `skipped`, so all of them would be unreachable the day it gets a canonicaliser, which is listed as still-deferred work. --retry now resets both states. Re-skipping is harmless and quiet: a source that still has no canonicaliser simply drains again, and the alert fatigue the drain was added to prevent does not apply to a flag somebody typed on purpose. The output names the breakdown so an operator can see which kind was recovered. --retry-failed is kept as an alias. It shipped earlier the same day under the narrower name and the new behaviour is a superset, so nothing anyone already wrote breaks. 259 tests pass, ruff clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
skippedmeans no canonicaliser existed for that report type at ingest time, and it was exactly as terminal asfailed:--pendingselects neither, and nothing else ever wroteparse_statusback. The design notes promise that adding a canonicaliser later just means re-marking these pending and re-running, and there was no way to do the re-marking.It nearly bit today
The disagg and TFF bytes captured 2026-07-30 sat unparsed for two days because no canonicaliser existed yet. This morning's producer run recovered 51,735 observations from them, which is the immutable landing zone working as designed.
That only worked because the producer's install predated the drain that marks unparseable sources
skipped, so they were stillpending. On a current install they would have been stranded with no command able to reach them.The live case is the weekly static: every retained copy of it is
skipped, so all of them would be unreachable the day it gets a canonicaliser, which is listed as still-deferred work.Change
--retryresets bothfailedandskipped. Re-skipping is harmless and quiet: a source that still has no canonicaliser simply drains again, and the alert fatigue the drain exists to prevent does not apply to a flag somebody typed on purpose. Output names the breakdown so the operator sees which kind was recovered.--retry-failedkept as an alias. It shipped earlier the same day under the narrower name and the new behaviour is a superset.259 tests pass (2 new), ruff clean.
Generated with Claude Code