fix(vintage): drain unparseable snapshots; add wrapper preflight - #80
Merged
Merged
Conversation
Both found from the first real production capture, which retained 4 snapshots but ingested only the Legacy annual zip. DRAIN BUG. disagg/TFF/weekly-static have no canonicaliser yet, so ingest 'continue'd past them and left them parse_status=pending forever. Two consequences: --pending never drains, and -- the real problem -- a pending snapshot carrying restatement_suspect would re-fire the alert on EVERY subsequent run, forever. Scoping suspects to the snapshots a run processed only works if snapshots actually drain out of pending, which these never did. They are now marked 'skipped' with a reason, so they surface once and go quiet. Raw bytes are retained, so adding a canonicaliser later just means re-marking them pending. PREFLIGHT. run-vintage.cmd now checks up front that cotdata-vintage.exe and cotdata-schedule.exe exist and that the store path is real, naming the fix (git pull && pip install -e .) instead of failing four times with 'cannot find path'. Those two entry points are newer than the rest of the CLI, so a venv installed before they existed has neither. Also expands the comment on why the vintage dir must be created BEFORE the first redirect: cmd opens a >> target before running the command, so without it the first line fails, nothing runs, and the task looks like it never fired -- which is exactly what happened on the first real setup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both issues surfaced by the first real production capture, which retained 4 snapshots but ingested only the Legacy annual zip.
Drain bug (the substantive one)
disagg / TFF / weekly-static have no canonicaliser yet, so
ingestcontinued past them and left themparse_status=pendingforever.--pendingnever drains, so every run re-selects and re-skips them.restatement_suspectwould re-fire the alert on every subsequent run, forever. Scoping suspects to the snapshots a run processed (added in feat(vintage): COT vintage store & revision tracking (capture + change-only ingest/PIT) #78) only works if snapshots actually drain out ofpending— and these never did. That is the alert-fatigue failure the scoping was meant to prevent.They are now marked
skippedwith a reason, so they surface once and go quiet. Raw bytes are retained, so adding a canonicaliser later just means re-marking them pending and re-running.Preflight
run-vintage.cmdnow checks up front thatcotdata-vintage.exe/cotdata-schedule.exeexist and the store path is real, naming the fix (git pull && pip install -e .) instead of failing four times withcannot find path. Those entry points are newer than the rest of the CLI, so a venv installed before they existed has neither.Also expands the comment on why the vintage dir must be created before the first redirect — cmd opens a
>>target before running the command, so without it the first line fails, nothing runs, and the task looks like it never fired. That is exactly what happened on the first real setup.203 tests pass (2 new, pinning that unparseable types drain and that a suspect on one alerts once then goes quiet).
ruff check src testsclean.🤖 Generated with Claude Code