feat(visibility): surface per-collector errors to Flow Doctor#191
Merged
Conversation
Saturday SF DataPhase1 PARTIAL run on 2026-05-09 fired no per-failure Flow Doctor alert. The arcticdb backfill regression was stored in the result dict but never logged at ERROR level — only main()'s generic "Weekly collection finished with non-ok status=partial" summary fires, which produces a single dedup signature across every partial run and contains no actual error text for Flow Doctor's LLM diagnose pipeline. Fix: _finalize() now calls alpha_engine_lib.collector_results.report_collector_errors(), which emits one logger.error() per error-status entry with the collector name + original message. Each emitted record carries a distinct dedup signature, restoring per-failure alert granularity. - Pin alpha-engine-lib v0.5.1 → v0.6.2 (helper landed in lib PR #34) - Wire call inside _finalize after status computation, before manifest write — fires for every code path that finalizes (phase 1, phase 2, daily, morning enrich) and runs even if postflight raises afterward - New wiring test test_collector_error_visibility.py pins the call so a future refactor can't silently drop it 552 unit tests pass locally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2 tasks
cipher813
added a commit
that referenced
this pull request
May 24, 2026
…ctions (#298) AST-walk regression pin: every production function whose name matches fresh|stale|preflight|postflight must not contain ``.days`` calendar arithmetic. Closes the cross-repo defect class surfaced by the 2026-05-24 Sunday SF recovery: calendar-day gates trip on every post-Saturday redrive even when data carries the most recent NYSE close. Escape hatch: inline ``# noqa: trading-day`` marker on the same line documents calendar-day correctness at that specific call site. Explicit allowlist: ``collectors/prices._find_stale_fast`` checks S3 LastModified timestamp (write-recency, not data-freshness) and is correctly calendar-day; allowlist verified by a second test that ensures the named function actually exists in the named file. Pin passes on current clean state (all 5 freshness sites migrated to ``alpha_engine_lib.dates.{trading_days_stale, is_fresh_in_trading_days}`` in PR #297). Would catch any future PR that adds calendar-day arithmetic to a freshness-named function. Composes with the lib v0.27.0 chokepoint (lib #59) + the cross-repo migration arc (predictor #191, research #222). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
alpha_engine_lib.collector_results.report_collector_errors()intoweekly_collector._finalizeso per-collector error-status entries emit onelogger.erroreach, with the collector name + original error message.alpha-engine-libpin v0.5.1 → v0.6.2 (helper added in feat(collectors): report_collector_errors helper for Flow Doctor visibility (v0.6.2) alpha-engine-lib#34).test_collector_error_visibility.pyto pin the call site.Why
The 2026-05-09 Saturday SF DataPhase1 PARTIAL run produced no per-failure Flow Doctor alert despite
arcticdbfailing on a backfill regression preflight. Tracing showed the onlylogger.error()that fires on partial status ismain()'s end-of-run generic summary:Two problems:
"Backfill regression preflight failed: 38 symbols would regress...") lives only in the result dict. Flow Doctor's LLM diagnosis + GitHub-issue body has nothing specific to work with.Fix placement
Call inside
_finalizeimmediately after the status block:Dependency
collector_resultsmodule and tags v0.6.2. Merge order:@v0.6.2resolvesTest plan
Follow-up
🤖 Generated with Claude Code