Emit daily_data module health stamp after daily_closes.collect#41
Merged
Conversation
Adds a module-scoped health stamp (health/daily_data.json) that matches
the schema consumed by alpha-engine's executor upstream gate
(executor/health_status.py::check_upstream_health). Complements the
direct LastModified check added in alpha-engine#53 — stamp catches
"ran and failed" state with structured reason/summary; direct check
catches "ran ok by the stamp but the artifact is stale."
Scoped to daily_closes only, not the whole daily pipeline — the
executor consumes predictor/daily_closes/{date}.parquet, which is
written by this collector. Feature store + ArcticDB append run
after and have their own concerns; their failures don't invalidate
the daily_closes write for the executor's purposes.
Emitted on BOTH success and failure paths so downstream can
distinguish "ran and failed today" (last_success=null, status=failed)
from "hasn't run in N hours" (stamp absent).
- new _write_module_health() helper using executor's schema
- kept _write_health_marker() phase stamps for backward compat
- 5 tests covering ok / failed / degraded / key pattern / defaults
Pairs with:
- cipher813/alpha-engine#53 (executor-side LastModified gate)
- cipher813/alpha-engine-predictor#33 (predictor-side LastModified gate)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds `health/daily_data.json` emission after `daily_closes.collect()` in `_run_daily()`, using the schema consumed by the executor's `check_upstream_health()` (`alpha-engine/executor/health_status.py:91`).
Belt-and-suspenders with the direct `LastModified` gates in alpha-engine#53 and alpha-engine-predictor#33:
The stamp alone would leave the silent-upstream-staleness hole — stamp says green while the blob is yesterday's. That's why both PRs exist.
Change
Test plan
Deploy sequencing
Follow-up (not in this PR)
🤖 Generated with Claude Code