Fix DataPreflight: SPY lives in macro library, not universe#32
Merged
Conversation
The daily-mode preflight was checking ArcticDB `universe/SPY` freshness, but SPY (and other market-wide series — VIX, sector ETFs, yields) lives in the `macro` library. The `universe` library holds per-stock OHLCV for S&P 500/400 constituents. Caught by the first real run on EC2 dashboard: RuntimeError: Pre-flight: ArcticDB universe/SPY read failed: E_NO_SUCH_VERSION ... version matching query 'latest' not found for symbol 'SPY' daily_append writes to both libraries in the same run, so checking macro/SPY freshness is a sufficient signal for the write path being healthy end-to-end. If we later want asymmetric coverage (macro dies but universe keeps writing, or vice versa), we can add a second check_arcticdb_fresh call pointed at a universe ticker like AAPL. No lib change needed — the primitive already takes library+symbol as args. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The daily-mode preflight was checking ArcticDB `universe/SPY` freshness, but SPY (and other market-wide series) lives in the `macro` library. Caught by the first real run on EC2:
```
RuntimeError: Pre-flight: ArcticDB universe/SPY read failed:
E_NO_SUCH_VERSION ... version matching query 'latest' not found for symbol 'SPY'
```
One-line fix: `"universe"` → `"macro"`. The primitive already takes library+symbol as args.
Why macro/SPY is sufficient
`daily_append` writes to both the `universe` (per-stock OHLCV + features) and `macro` (market-wide series) libraries in the same run. If daily_append fails, neither library gets today's row. Checking one library is enough signal.
If we later want asymmetric coverage (macro write succeeds but universe doesn't, or vice versa), we can add a second `check_arcticdb_fresh` call pointed at a universe ticker like AAPL. Not needed for the bug we were actually trying to catch.
Test plan
🤖 Generated with Claude Code