PR4 (data, DRAFT): delete slim writer + load_slim_cache API + fallbacks [merge-gated 5/23]

WAVE4_SLIM_DELETION_RUNBOOK.md

@@ -0,0 +1,72 @@
+# Wave-4 — `predictor/price_cache_slim/` S3 prefix deletion runbook
+
+**Status: GATED — do NOT execute until the gate below passes.**
+
+This PR removes the slim *code* (writer, `load_slim_cache` API, consumer
+fallbacks). The destructive S3 prefix deletion is a separate, manual,
+gated step documented here so it is reviewable and auditable (it is **not**
+run by CI or any pipeline).
+
+## Gate (all must hold before deletion)
+
+1. **This PR merged** — slim writer gone, every consumer ArcticDB-only
+   (data macro-breadth + feature-compute; backtester exit_timing #226;
+   dashboard health-check retired #88).
+2. **≥1 clean Saturday-SF parity observation.** Before this PR merges, the
+   migrated consumers still ran with slim + emitted
+   `WAVE4_PARITY_METRIC {breadth,compute,exit_timing}` JSON lines. Pull the
+   most recent Saturday SF logs (first observable **2026-05-23**) and
+   confirm, for each of the three streams:
+   - `passed: true`
+   - `max_abs_value_delta` ≤ epsilon (effectively `0.0`)
+   - `only_in_b` empty (nothing the ArcticDB read lacked vs slim);
+     `only_in_a` (slim-only legacy symbols) is expected and acceptable.
+   If any stream shows value divergence, **stop** — investigate before
+   deleting the fallback's data.
+3. **No remaining live reader.** `spot_backtest.sh:528`'s slim `aws s3
+   sync` was verified dead (predictor_backtest.py loads from ArcticDB;
+   only `sector_map.json` is read from the cache dir) and is removed in
+   the backtester PR4. Re-confirm no new consumer via the terminal guard
+   `tests/test_wave4_slim_arctic_parity.py`.
+
+## Procedure (single-dev, paper-trading; bounded-reversible)
+
+```
+# 1. Pre-deletion byte-equal backup (Wave-5 precedent).
+aws s3 cp --recursive \
+  s3://alpha-engine-research/predictor/price_cache_slim/ \
+  s3://alpha-engine-research/backups/price_cache_slim.pre-deletion-260523/ \
+  --only-show-errors
+
+# 2. Verify the backup is byte-equal (object count + total bytes).
+aws s3 ls --recursive --summarize \
+  s3://alpha-engine-research/predictor/price_cache_slim/ \
+  | tail -2
+aws s3 ls --recursive --summarize \
+  s3://alpha-engine-research/backups/price_cache_slim.pre-deletion-260523/ \
+  | tail -2
+# Object count + Total Size MUST match before proceeding.
+
+# 3. Delete the prefix.
+aws s3 rm --recursive \
+  s3://alpha-engine-research/predictor/price_cache_slim/ --only-show-errors
+
+# 4. Confirm empty.
+aws s3 ls s3://alpha-engine-research/predictor/price_cache_slim/ \
+  | wc -l   # -> 0
+```
+
+## Rollback
+
+`git revert` the Wave-4 PR(s) + `aws s3 cp --recursive` the backup prefix
+back to `predictor/price_cache_slim/`. The slim writer resumes on the next
+weekly SF. Worst case from a missed divergence in this single-dev
+paper-trading context: degraded features for ~one week until noticed — no
+capital or data-loss risk (per the CLAUDE.md severity posture).
+
+## Follow-ups (cosmetic, batch after deletion)
+
+- Dashboard architecture-page slim *labels* (`public/pages/2_Architecture.py`,
+  `pages/10_Architecture.py`) — descriptive topology text only.
+- Comment-only slim mentions in `builders/backfill.py`,
+  `validators/price_validator.py` — historical context, harmless.

collectors/macro.py

@@ -26,9 +26,7 @@
 import requests
 import yfinance as yf
 
-from store.parquet_loader import load_slim_cache
 from alpha_engine_lib.arcticdb import load_universe_ohlcv
-from alpha_engine_lib.reconcile import reconcile_frame_dicts
 
 logger = logging.getLogger(__name__)
 
@@ -48,55 +46,24 @@
 
 
 def _load_breadth_prices(bucket: str) -> Optional[dict]:
-    """Load the ~900-ticker price set for breadth — ArcticDB primary,
-    slim-cache fallback, parity-observed.
-
-    Wave 4 of the predictor/price_cache_slim deletion arc. ArcticDB (via the
-    lib ``load_universe_ohlcv`` slim-equivalent reader) is the single source
-    of truth; the legacy ``predictor/price_cache_slim/`` parquet read is kept
-    as a fallback so breadth cannot break if ArcticDB is unavailable. While
-    both sources still exist we dual-read and emit a ``reconcile`` ParityReport
-    so the eventual slim deletion (PR4) is a data-driven cutover, not an
-    eyeballed one. The slim side — and this dual-read — are removed in PR4.
-
-    Returns ``None`` only if BOTH sources fail (caller then omits the breadth
-    key, preserving the existing no-null contract).
+    """Load the ~900-ticker price set for breadth from the ArcticDB
+    universe library.
+
+    Wave-4 terminal state: ``predictor/price_cache_slim/`` is deleted;
+    ArcticDB (via the lib ``load_universe_ohlcv`` reader) is the sole
+    source. The 5/23 parity observation confirmed slim<->ArcticDB
+    equivalence before the slim fallback + dual-read were removed here.
+
+    Returns ``None`` if the ArcticDB read fails (caller then omits the
+    breadth key — the existing no-null contract; Research has its own
+    fallback). This matches the pre-Wave-4 behaviour when the single
+    price source was unavailable.
     """
-    arctic_prices = None
     try:
-        arctic_prices = load_universe_ohlcv(bucket)
-    except Exception as exc:  # noqa: BLE001 - fall back, don't break breadth
+        return load_universe_ohlcv(bucket) or None
+    except Exception as exc:  # noqa: BLE001 - omit breadth, don't write null
         logger.warning("ArcticDB universe read for breadth failed: %s", exc)
-
-    slim_prices = None
-    try:
-        slim_prices = load_slim_cache(boto3.client("s3"), bucket)
-    except Exception as exc:  # noqa: BLE001 - parity/fallback only
-        logger.warning(
-            "Slim cache read for breadth (parity/fallback) failed: %s", exc
-        )
-
-    # SOTA observation: while both exist, emit the quantitative parity metric
-    # every run. Grep ``WAVE4_PARITY_METRIC breadth`` over the observation
-    # window before PR4 retires slim.
-    if arctic_prices and slim_prices:
-        report = reconcile_frame_dicts(
-            slim_prices, arctic_prices, value_cols=("Close",)
-        )
-        logger.info("breadth slim<->arctic %s", report.summary())
-        logger.info(
-            "WAVE4_PARITY_METRIC breadth %s", json.dumps(report.as_metrics())
-        )
-
-    if arctic_prices:
-        return arctic_prices
-    if slim_prices:
-        logger.warning(
-            "breadth falling back to slim cache — ArcticDB universe "
-            "unavailable (Wave-4 migration fallback path)"
-        )
-        return slim_prices
-    return None
+        return None
 
 
 def collect(