feat(wave3): try reference/price_cache/ first in regime/features._read_parquet_close (ROADMAP L1401)#181
Merged
Merged
Conversation
…d_parquet_close (ROADMAP L1401)
Wave-3 reader-side migration for the regime substrate fetcher
(`regime/features.fetch_macro_feature_history` + its daily sibling
+ all callers in `regime/handler.py`, `regime/retrospective_eval_handler.py`,
the backfill scripts). Companion to producer write-both PR1
(alpha-engine-data#270, shipped 2026-05-19).
The legacy single-key read is funneled through one chokepoint —
`_read_parquet_close(ticker, ..., prefix)` — so this PR is a
surgical change there:
- When `prefix == DEFAULT_PRICE_CACHE_PREFIX` (the production
default used by handler.py + retrospective_eval_handler.py + the
backfill scripts), iterate `(reference/, predictor/)` in order
and return the first hit. New prefix consulted first; legacy is
the soak-window fallback.
- When `prefix` is anything else (a custom test/override prefix),
single-prefix semantics — opt-out from the fallback chain.
Mirrors the write-side `price_cache_write_prefixes` opt-out.
After Wave-3 PR4 retires legacy + runs `aws s3 rm --recursive
predictor/price_cache/`, the fallback entry in
`_PRICE_CACHE_FALLBACK_PREFIXES` becomes dead and is dropped in the
same one-line cutover edit.
Tests (+3 new, suite 13 → 16 in test_regime_features.py)
- `test_wave3_reader_prefers_new_prefix_over_legacy` — plants both
prefixes with distinct Close values; asserts the new prefix wins.
- `test_wave3_reader_falls_back_to_legacy_when_new_missing` — early-
soak ticker only in legacy; reader must serve it.
- `test_wave3_reader_explicit_custom_prefix_does_not_fall_back` —
custom prefix opts out; empty result when its key is missing.
Existing tests unaffected — they seed to
`DEFAULT_PRICE_CACHE_PREFIX` (legacy) and the fallback chain still
finds the data on the second attempt.
Composes with
- alpha-engine-data#270 (producer write-both, prerequisite).
- alpha-engine#197 (IAM ARN add for `reference/price_cache/`).
- alpha-engine-data#272 + alpha-engine-backtester#230 (sibling
Wave-3 reader migrations).
- Wave-3 PR4 cutover (drops the fallback entry).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cipher813
added a commit
that referenced
this pull request
May 20, 2026
…onstant + sector_map (ROADMAP L1401) (#182) Completes the predictor-repo side of Wave-3 PR3 reader migration. The deployed Lambda chokepoint shipped 2026-05-19 via #181 (regime/features ._read_parquet_close); this PR closes the analysis-tool sites the L1401 PR3 follow-up plan flagged + a dead module-level constant. - ``config.PRICE_CACHE_KEY = "predictor/price_cache/{ticker}.parquet"`` deleted. ``git grep PRICE_CACHE_KEY`` returned zero references outside the constant itself — classic dead-code scaffolding. A new regression-pin test ``test_config_no_longer_exports_price_cache_key _constant`` prevents a future ``from config import PRICE_CACHE_KEY`` from silently bypassing the Wave-3 read-prefix chain. - ``analysis/triple_barrier_cutover_runner._load_prices_from_s3`` iterates a per-callsite read-prefix chain (new → legacy) for the per-ticker parquet lookup. Inlines the same shape #181 used in ``regime/features._read_parquet_close`` rather than importing — keeps analysis tools decoupled from Lambda-deployed modules. - ``analysis/triple_barrier_cutover_runner._load_sector_map_from_s3`` prepends ``reference/price_cache/sector_map.json`` to the existing fallback list (data/sector_map.json → predictor/price_cache/sector _map.json). sector_map.json now lives at THREE keys post alpha-engine-data #272's write-both gap fix; the reader prefers the new home but cascades through both legacy locations for the soak. - New test file ``tests/test_triple_barrier_cutover_runner_wave3 _readers.py`` — 7 tests covering: prefers-new, falls-back-legacy, absent-in-both (price-cache 3-test discipline); sector_map prefers- new, cascades-through-data-then-predictor, returns-empty-on-total- miss; PRICE_CACHE_KEY deletion guard. Suite 1113 → 1120 green. Discovery (surfaced + handled in main checkout, NOT in this PR): ``scripts/momentum_ic_study_21d.py`` is named in the L1401 PR3-wave-2 plan but ``git status`` confirms it is **untracked / never committed** — a local-only operator analysis script. Migrated in the main repo working copy so it survives PR4 cutover, but not added to git here. Brian's call whether to git-track it later; the migration is reversible either way. Composes with: alpha-engine-predictor #181 (the deployed Lambda chokepoint this mirrors), alpha-engine-data #272 (sector_map.json 3-key write-both that this reads from), alpha-engine-data #273 (sibling PR3-wave-2 — data-repo invasive sites). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ROADMAP: L1401 —
predictor/S3 namespace rationalization Wave 3 — predictor-repo half of the PR3+ reader migrations.Companion to producer write-both PR1 alpha-engine-data#270 (shipped 2026-05-19) + IAM ARN add alpha-engine#197 + the one-shot backfill executed 2026-05-19 22:13Z.
Surgical single-site change
The predictor regime substrate fetcher has a single chokepoint —
regime/features._read_parquet_close(ticker, ..., prefix)— that funnels every per-ticker price-cache read from:regime/handler.py(Lambda — weekly substrate writer)regime/retrospective_eval_handler.py(eval Lambda)scripts/backfill_regime_substrate.pyscripts/backfill_regime_fast_signal.pyregime/features.fetch_macro_feature_history+fetch_macro_feature_history_dailyMigrating that one function migrates the whole regime read path.
Behavior
prefixargDEFAULT_PRICE_CACHE_PREFIX(production default)(reference/, predictor/)— new first, legacy fallback. Mirrors the data-sideprice_cache_read_prefixessemantic.price_cache_write_prefixesopt-out semantic.After Wave-3 PR4 retires legacy + runs
aws s3 rm --recursive predictor/price_cache/, the legacy entry in_PRICE_CACHE_FALLBACK_PREFIXESbecomes dead and is dropped in a one-line cutover edit.Tests (+3 new, test_regime_features.py 13 → 16)
test_wave3_reader_prefers_new_prefix_over_legacy— plants both prefixes with distinct Close values; asserts the new prefix wins (would catch a regression that flipped the order back).test_wave3_reader_falls_back_to_legacy_when_new_missing— early-soak state where a ticker is only in legacy; reader must serve it.test_wave3_reader_explicit_custom_prefix_does_not_fall_back— custom prefix opts out; empty result when its key is missing.Existing tests unaffected — they seed to
DEFAULT_PRICE_CACHE_PREFIX(legacy) and the fallback chain finds the data on attempt #2.Suite: 1113 passed.
Sites audited but not migrated in this PR
config.py:67(PRICE_CACHE_KEYconstant)grep PRICE_CACHE_KEYacross the repo returns only its definition. Will be deleted in PR4 cutover.analysis/triple_barrier_cutover_runner.py:138,164scripts/momentum_ic_study_21d.pyComposes with
reference/price_cache/).🤖 Generated with Claude Code