diff --git a/CHANGELOG.md b/CHANGELOG.md index 3776e2d..969cc32 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,22 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). off ETF proxies. Adds the MSCI EM (MME→EEM) and EAFE (MFS→EFA) held-out generalization markets. ([#24](https://github.com/mspinola/cotdata/pull/24)) +### Changed +- **`--require-final` price gate is now data-driven, not a wall-clock cutoff.** + `finals_ready` previously deferred until Norgate's `Futures` and `Continuous + Futures` databases were both refreshed at/after a fixed local time + (`--final-cutoff`, default 20:55). That is fragile by construction: the cutoff + must sit below the earliest evening final yet above any daytime interim, and + Norgate's publish time drifts. On 2026-07-27 Norgate finalized the Futures DB at + 8:49pm, so the `>= 20:55` check never turned true and prices went stale for the + day. The gate now asks the robust question — does Norgate hold a **newer settled + continuous bar** than the store already has? — across a liquid reference quorum + (ES, CL, ZC). It is immune to publish-time drift (early publish → ready early, + late publish → a retry catches it) and needs no trading calendar (weekends and + holidays simply produce no new bar). `--final-cutoff` is accepted but ignored + (deprecated) so existing schedulers do not break. See + `docs/design/finals_ready_data_driven.md`. + ### Fixed - **`databento.fetch_daily_ohlc`'s `start_date` parameter now actually does something** (dormant provider). It was previously silently ignored on every diff --git a/docs/design/finals_ready_data_driven.md b/docs/design/finals_ready_data_driven.md new file mode 100644 index 0000000..8225bcc --- /dev/null +++ b/docs/design/finals_ready_data_driven.md @@ -0,0 +1,168 @@ +# Data-driven `finals_ready` (replace the fixed-clock cutoff) + +Status: IMPLEMENTED, pending live validation on the Windows producer. The daytime probe +answered the open question (Norgate never shows an in-progress session's bar); the gate is +wired data-driven with unit tests. What remains is a few nights confirming it flips ready at +the right moment against the live feed. + +## Problem + +`--require-final` gates the nightly Norgate price capture so it never stores an +interim (non-settled) bar. Today it does so with a fixed **local-clock cutoff** +(`--final-cutoff`, default `20:55`): `finals_ready` is true only once *both* the +`Futures` and `Continuous Futures` databases were refreshed at/after today's cutoff +(`providers/norgate.py::_finals_ready`). + +This is calibration, not robustness, and it broke in production on 2026-07-27: + +- Norgate's evening publish finalized the **Futures** database at **8:49 PM** and did + not touch it again that night; **Continuous Futures** finalized at 8:55 PM. +- The check requires *both* `>= 20:55`. Futures at 8:49 `< 8:55` → never ready → the + task deferred (exit 1) on every attempt, the chained store sync was skipped, and + prices went stale (stuck on the prior Friday's bar). + +The failure mode is intrinsic to a wall-clock cutoff: it must sit *below* the earliest +evening-final refresh and *above* any daytime interim refresh, and Norgate's publish +time drifts night to night. If Norgate finalizes at 20:44, a `20:45` cutoff misses it; +if you lower the cutoff to catch early nights, you risk accepting a pre-settlement bar. +There is no single clock value that is safe. + +## Available Norgate API (v1.0.77) + +`norgatedata` exposes **no** holiday / trading-calendar / session function (confirmed on +PyPI). It does expose data-content signals we are not yet using: + +- `last_quoted_date(symbol, datetimeformat='iso')` — the latest trading date that has a + bar for `symbol`. For a continuous symbol (never expires) this is the newest available + daily bar. +- `second_last_quoted_date(symbol)` — the prior trading day's date. +- `last_price_update_time(symbol)` — local PC time the symbol was last refreshed. +- `last_database_update_time(db)` — local PC time the database was refreshed (current + approach). +- `status()` — is NDU running. + +`last_quoted_date` is the key: it lets us ask "**has Norgate's latest bar advanced to the +session we expect?**" instead of "was the file touched after a magic minute?" That is +immune to publish-time drift. + +## Proposed design + +Replace the clock-threshold with a **data + session** check: + +``` +finals_ready(now) := last_quoted_date(ref) >= expected_last_session(now) +``` + +- `ref` = one (or a small quorum) of liquid continuous symbols that trade every US + futures session (e.g. `&ES`, `&CL`). Use the min across the quorum so a single lagging + symbol cannot green-light the whole book. +- `expected_last_session(now)` = the most recent trading day whose session has **closed** + as of `now`. Weekend/holiday aware. + +Why this is robust where the cutoff is not: + +- **Publish-time drift disappears.** If Norgate finalizes early, `last_quoted_date` + advances early and we are ready early. If it finalizes late, the bar is simply not + there yet and the run defers — retries then catch it whenever it lands. The gate no + longer depends on Norgate hitting a target minute. +- **The only remaining clock element is coarse and stable.** `expected_last_session` + needs to know a session has *closed*, i.e. "we are past the exchange's daily settlement + window." That threshold has hours of slack (settlement is done well before the evening + publish), unlike the razor-thin publish cutoff. It is a property of the market, not of + Norgate's schedule. + +### Two pieces to nail down + +1. **`expected_last_session` / the trading calendar.** `norgatedata` has no calendar API, + so options are: + - `pandas_market_calendars` (mature, has CME/ICE calendars) — adds a dependency to a + public package. Cleanest correctness. + - A maintained holiday list in-repo (the NDU app shows NYSE/exchange holidays 6 months + each way; not exposed to Python) — no dependency, but must be maintained. + - Derive from Norgate: treat `expected_last_session` as "the max `last_quoted_date` + across a broad basket," and require the ref quorum to have caught up to it. Avoids a + calendar entirely, at the cost of a softer definition. + A wrong calendar only causes a **false defer** (safe: it self-heals next session), never + a false capture, so this is a robustness/convenience tradeoff, not a correctness risk. + +2. **Interim/preliminary bars — the open question below.** + +## Open question (Windows probe required) + +Does Norgate ever expose a **provisional current-day bar before final settlement**? +- If **no** (today's bar appears only once settled): `last_quoted_date == today` is a + sufficient, clean "finals are in" signal, and the design above is complete. +- If **yes** (staged/preliminary then final settlement): bar *presence* is not enough, + because the value can still be revised. We would then keep a **coarse** "past settlement + window" guard (e.g. only trust today's bar after ~18:00 ET) on top of the date check — + still far more forgiving than the current publish-time cutoff. + +The probe (`scripts/probe_norgate_finals.py`, run on the Windows producer) answers this by +sampling `last_quoted_date` and the tail of `price_timeseries` for a couple of continuous +symbols during the day and again after the evening publish, and by dumping `dir(norgatedata)` +to confirm no calendar function exists in the installed build. + +## Compatibility / rollout + +- `cotdata-prices` is a public CLI. Keep `--final-cutoff` accepted (as the fallback + "settlement window" guard, or deprecated-but-honored) so no caller breaks. +- Keep the pure-core split: `_finals_ready(...)` stays a norgatedata-free, unit-tested + function operating on plain dates/times; `finals_ready(...)` does the norgatedata I/O. +- Default behavior change (clock → data) is the point, but gate it so it can be rolled + back with a flag if a first-night surprise appears. + +## Test plan + +Unit-test the pure core on any OS (no norgatedata): weekend/holiday rollover, early vs +late publish, ref-quorum lag, and the interim guard if needed. Live-validate `finals_ready` +against the real feed on Windows across a few evenings before flipping the default. + +## Probe results (Windows, 2026-07-28 daytime) — design simplified + +The daytime run (`scripts/probe_norgate_finals.py`) settled the open questions and made the +design simpler than the draft above: + +- **No calendar API.** The only `*session*` functions (`futures_market_session_*`, + `session_type`) describe which contracts trade in a session, not holidays. Confirmed. +- **`last_quoted_date` is unusable for the gate** — it returns `None` for continuous + symbols (it is meant for expiring instruments). The signal must come from + `price_timeseries`. +- **Norgate (EOD) does not publish an in-progress session's bar.** At 11 AM Tue the latest + continuous bar was **Mon 7/27** (final OHLC), with **`Open Interest == 0`** while every + prior day was ~1.9M — `OI==0` simply marks the newest bar (OI publishes T+1). No Tuesday + bar existed intraday. + +So the gate needs **neither a trading calendar nor a wall-clock cutoff**: + +``` +finals_ready := norgate_latest_bar_date > store_latest_bar_date +``` + +Implemented as the pure core `_finals_ready_by_date(norgate_last, store_last)` (unit-tested, +no norgatedata). Weekends/holidays produce no new bar (correctly no capture); publish-time +drift is absorbed by retries; the 2026-07-27 failure could not recur (Monday's bar was +available all along — only the clock cutoff blocked it). + +### What shipped + +- `_finals_ready_by_date(norgate_last, store_last)` — pure per-symbol core. +- `_finals_ready_quorum(norgate_dates, store_dates)` — pure combiner; ready only when + EVERY reference symbol (`_FINALS_REF_SYMBOLS = ES, CL, ZC`) has a newer settled bar than + the store, so a session is captured once and complete. +- `finals_ready()` — thin norgatedata I/O: reads each ref's latest continuous bar date + (`_norgate_last_bar_date`, a short trailing `price_timeseries` window) and the store's + last captured date (`_store_last_bar_date`, from the prices manifest), then delegates to + the quorum. Guards on `_require_norgate_service()`. +- CLI: `--require-final` now uses this; `--final-cutoff` is accepted-but-ignored + (deprecated) so no scheduler breaks. The legacy `_finals_ready` clock core is retained, + unused, for reference/rollback. +- Unit tests cover the core, the quorum (all-advance vs one lagging), the wiring, and the + ignored `cutoff` arg — all norgatedata-free. + +### Remaining: live validation (Windows) + +A couple of evenings confirming `finals_ready` flips to ready when the new settled bar lands +(and that a same-session re-run stays not-ready). The optional two-point evening probe +(before ~8:45 PM, after ~9:15 PM) is a nice confirmation of the transition but is no longer a +blocker — the daytime probe already showed Norgate does not expose an in-progress session's +bar. diff --git a/scripts/probe_norgate_finals.py b/scripts/probe_norgate_finals.py new file mode 100644 index 0000000..37d995f --- /dev/null +++ b/scripts/probe_norgate_finals.py @@ -0,0 +1,68 @@ +#!/usr/bin/env python +"""Probe norgatedata's data-content signals for the data-driven finals_ready redesign. + +Run on the WINDOWS producer (needs norgatedata + a live Norgate subscription). + +Run it TWICE and compare the output: + 1. during the trading day (before Norgate's evening publish), and + 2. after the evening publish (e.g. ~9:30 PM ET). + +The key question: does `last_quoted_date` for a continuous symbol show TODAY's date +only after settlement, or already during the day (a provisional bar)? That decides +whether bar-presence alone is a sufficient "finals are in" signal. See +docs/design/finals_ready_data_driven.md. +""" +import datetime as dt + +import norgatedata + +REF_SYMBOLS = ["&ES", "&CL", "&ZC"] # liquid continuous: S&P, WTI, corn — trade every session +FINAL_DATABASES = ["Futures", "Continuous Futures"] + +print(f"# probe at {dt.datetime.now().isoformat()} (local PC time)") +print(f"norgatedata version: {getattr(norgatedata, '__version__', '?')}") +try: + print(f"NDU running (status()): {norgatedata.status()}") +except Exception as e: # noqa: BLE001 + print(f"status() ERROR: {e!r}") + +# 1) Confirm whether this build has ANY calendar/holiday/session function. +api = sorted(x for x in dir(norgatedata) if not x.startswith("_")) +cal = [x for x in api if any(k in x.lower() for k in + ("holiday", "calendar", "session", "business", "market_day", "trading_day", "busday"))] +print(f"\ncalendar-ish functions in this build: {cal or 'NONE'}") +print(f"full API: {api}") + +# 2) Database refresh times — the current (fragile) signal. +print("\n-- last_database_update_time --") +for db in FINAL_DATABASES: + try: + print(f" {db}: {norgatedata.last_database_update_time(db)}") + except Exception as e: # noqa: BLE001 + print(f" {db}: ERROR {e!r}") + +# 3) Data-content signals per ref symbol — the proposed signal. +print("\n-- per-symbol date/time signals --") +for s in REF_SYMBOLS: + try: + lq = norgatedata.last_quoted_date(s, datetimeformat="iso") + slq = norgatedata.second_last_quoted_date(s, datetimeformat="iso") + lpu = norgatedata.last_price_update_time(s) + print(f" {s}: last_quoted_date={lq} second_last={slq} last_price_update_time={lpu}") + except Exception as e: # noqa: BLE001 + print(f" {s}: ERROR {e!r}") + +# 4) The actual tail — does today's bar exist yet, and do its values look settled? +print("\n-- price_timeseries tail (last 4 bars) --") +for s in REF_SYMBOLS: + try: + df = norgatedata.price_timeseries( + s, + padding_setting=norgatedata.PaddingType.NONE, + timeseriesformat="pandas-dataframe", + start_date="2026-07-20", + ) + print(f" {s}:") + print(" " + df.tail(4).to_string().replace("\n", "\n ")) + except Exception as e: # noqa: BLE001 + print(f" {s}: ERROR {e!r}") diff --git a/src/cotdata/providers/norgate.py b/src/cotdata/providers/norgate.py index 26d214d..79c59b8 100644 --- a/src/cotdata/providers/norgate.py +++ b/src/cotdata/providers/norgate.py @@ -28,6 +28,9 @@ # Default local-time cutoff after which Norgate's "Final" futures prices are in # (≈ Continuous Futures Final at ~8:55pm ET per Norgate's update schedule). DEFAULT_FINAL_CUTOFF = "20:55" +# Liquid continuous reference symbols for the data-driven finals gate: each trades every +# US futures session, so a newer settled bar on all of them means a session has landed. +_FINALS_REF_SYMBOLS = ("ES", "CL", "ZC") # If roll-day overnight moves exceed this multiple of the normal-day median, the # series looks UNADJUSTED (calendar-spread gaps not stitched). Self-calibrating # per symbol, so it works across products with different spread magnitudes. @@ -253,7 +256,11 @@ def _to_naive_local(t): def _finals_ready(db_times: dict, cutoff: str = DEFAULT_FINAL_CUTOFF, now=None): - """Pure core of :func:`finals_ready` — testable without norgatedata. + """LEGACY wall-clock finals core, superseded by the data-driven gate + (:func:`_finals_ready_by_date` / :func:`finals_ready`). Retained for reference and + rollback; no CLI path uses it. It broke in production because a fixed clock cutoff + must sit below the earliest evening final yet above any daytime interim, and Norgate's + publish time drifts (see docs/design/finals_ready_data_driven.md). db_times maps database name → its last-update datetime (tz-aware or naive local, or None). Ready when every database was refreshed at/after today's `cutoff` @@ -271,15 +278,85 @@ def _finals_ready(db_times: dict, cutoff: str = DEFAULT_FINAL_CUTOFF, now=None): return ready, detail -def finals_ready(cutoff: str = DEFAULT_FINAL_CUTOFF, now=None): - """True once Norgate has this day's FINAL futures prices — i.e. it has refreshed - both the 'Futures' and 'Continuous Futures' databases at/after today's local - `cutoff`. Uses norgatedata.last_database_update_time (the local PC time of the - last DB refresh). Lets a scheduled run avoid capturing interim (non-final) bars. - Returns (ready: bool, detail: dict).""" +def _finals_ready_by_date(norgate_last, store_last): + """Data-driven finals gate (pure core): ready when Norgate's latest continuous bar is + a NEWER completed session than the store already holds. + + Both args are dates (or datetimes, normalized to their date; or None). Norgate is + end-of-day and only publishes a session's bar once that session is complete, so a + bar date newer than the store's is a new *settled* session to capture. This needs no + trading calendar (weekends/holidays simply produce no new bar) and no wall-clock + cutoff (early publish → ready early; late publish → not there yet → a retry catches + it), which is what makes it immune to Norgate's publish-time drift. Returns + (ready: bool, detail: dict).""" + def _d(x): + if x is None: + return None + return x.date() if isinstance(x, dt.datetime) else x + + nl, sl = _d(norgate_last), _d(store_last) + detail = { + "norgate_last": nl.isoformat() if nl else None, + "store_last": sl.isoformat() if sl else None, + } + if nl is None: + return False, detail # Norgate has no bar to offer → defer + return (sl is None or nl > sl), detail + + +def _finals_ready_quorum(norgate_dates: dict, store_dates: dict): + """Combine per-reference results into the finals gate (pure/testable): ready only when + EVERY reference symbol has a newer settled bar in Norgate than the store already holds. + Requiring the whole quorum means a session is captured once and complete, and one + lagging reference cannot green-light a partial capture. Returns (ready, detail).""" + per, ready_all = {}, True + for sym in norgate_dates: + r, d = _finals_ready_by_date(norgate_dates.get(sym), store_dates.get(sym)) + per[sym] = {**d, "ready": r} + ready_all = ready_all and r + return ready_all, {"mode": "data", "per_symbol": per} + + +def _norgate_last_bar_date(sym: str): + """Latest continuous (back-adjusted) bar date Norgate holds for internal `sym`, or None. + Pulls a short trailing window (cheap) and takes the last index. Norgate is end-of-day + and only publishes a session's bar once settled, so this date advances exactly when a + new final session lands — the signal the gate keys on.""" import norgatedata # Windows producer only - times = {db: norgatedata.last_database_update_time(db) for db in FINAL_DATABASES} - return _finals_ready(times, cutoff, now) + ng_sym = REGISTRY[sym].norgate + CCB_SUFFIX + start = (dt.date.today() - dt.timedelta(days=10)).isoformat() + df = norgatedata.price_timeseries( + ng_sym, + padding_setting=norgatedata.PaddingType.NONE, + timeseriesformat="pandas-dataframe", + start_date=start, + ) + if df is None or len(df) == 0: + return None + return pd.to_datetime(df.index[-1]).tz_localize(None).normalize().date() + + +def _store_last_bar_date(sym: str): + """Latest date already captured in the store for internal `sym` (back-adjusted), or + None if the store has never seen it. Read from the prices manifest — no price I/O.""" + prices = store.load_manifest().get("prices", {}) + ld = (prices.get(f"{sym}_backadj") or {}).get("last_date") + return pd.to_datetime(ld).date() if ld else None + + +def finals_ready(cutoff=None, now=None, ref_symbols=_FINALS_REF_SYMBOLS): + """Ready once Norgate has a NEWER settled continuous bar than the store, for a quorum of + liquid reference symbols (data-driven finals gate). Replaces the old wall-clock cutoff: + immune to Norgate's publish-time drift (early publish → ready early; late publish → not + there yet → a retry catches it) and needs no trading calendar (weekends and holidays + simply produce no new bar). See docs/design/finals_ready_data_driven.md. + + `cutoff` and `now` are accepted for backward compatibility and IGNORED — the clock gate + is deprecated. Returns (ready: bool, detail: dict).""" + _require_norgate_service() # NDU-down guard: norgatedata calls bare sys.exit otherwise + norgate_dates = {s: _norgate_last_bar_date(s) for s in ref_symbols} + store_dates = {s: _store_last_bar_date(s) for s in ref_symbols} + return _finals_ready_quorum(norgate_dates, store_dates) def _norgate_covered(symbols): diff --git a/src/cotdata/update.py b/src/cotdata/update.py index 270d6cc..a25d9a8 100644 --- a/src/cotdata/update.py +++ b/src/cotdata/update.py @@ -69,13 +69,14 @@ def main(argv=None, half=None) -> None: help="Full rebuild of reconstructed volume (ignore the incremental " "60-day window). Use after a reconstruction-logic change.") p.add_argument("--require-final", action="store_true", - help="For --prices: only fetch once Norgate's FINAL futures prices are " - "in (last_database_update_time for 'Futures' and 'Continuous Futures' " - ">= --final-cutoff today). Otherwise defer with a non-zero exit so a " - "scheduler retries. Avoids capturing interim (non-final) bars.") + help="For --prices: only fetch once Norgate has a NEWER settled continuous " + "bar than the store already holds (a new session has landed). " + "Otherwise defer with a non-zero exit so a scheduler retries. Immune " + "to Norgate's publish-time drift; needs no cutoff or calendar.") p.add_argument("--final-cutoff", default="20:55", metavar="HH:MM", - help="Local time after which Norgate's futures Finals are expected " - "(default 20:55, ≈ Continuous Futures Final in ET).") + help="DEPRECATED and ignored: the finals gate is now data-driven " + "(newer-bar-than-store), not a wall-clock cutoff. Accepted so " + "existing schedulers do not break.") p.add_argument("--check", action="store_true", help="Print store status (row counts, newest data, staleness) from " "the manifest and exit. Read-only, cross-platform, no network.") @@ -170,10 +171,10 @@ def main(argv=None, half=None) -> None: if args.prices: ready = True if args.require_final: - ready, detail = norgate.finals_ready(args.final_cutoff) + ready, detail = norgate.finals_ready() if not ready: - print(f"prices: Norgate Finals not in yet (--require-final, cutoff " - f"{args.final_cutoff}) — deferring. {detail}") + print(f"prices: no new settled Norgate bar yet (--require-final) " + f"— deferring. {detail}") deferred.append("prices") if ready: last_run = norgate.update(symbols=args.symbols, full=args.full) diff --git a/tests/test_norgate_provider.py b/tests/test_norgate_provider.py index 677020a..4d469fb 100644 --- a/tests/test_norgate_provider.py +++ b/tests/test_norgate_provider.py @@ -449,3 +449,72 @@ def test_finals_ready_handles_tz_aware_times(): assert ok is True ng, _ = _finals_ready({"Futures": after, "Continuous Futures": before}, "20:55", now) assert ng is False + + +def test_finals_ready_by_date_pure_logic(): + """Data-driven gate: ready iff Norgate has a newer settled bar than the store.""" + import datetime as d + + from cotdata.providers.norgate import _finals_ready_by_date + mon, tue = d.date(2026, 7, 27), d.date(2026, 7, 28) + # a newer settled session is available -> ready + ok, det = _finals_ready_by_date(tue, mon) + assert ok is True and det["norgate_last"] == "2026-07-28" + # store already holds the latest -> nothing new, not ready + assert _finals_ready_by_date(mon, mon)[0] is False + # Norgate somehow behind the store -> not ready + assert _finals_ready_by_date(mon, tue)[0] is False + # empty store -> ready on any Norgate bar + assert _finals_ready_by_date(mon, None)[0] is True + # Norgate offers nothing -> defer + assert _finals_ready_by_date(None, mon)[0] is False + + +def test_finals_ready_by_date_normalizes_datetimes(): + """datetimes collapse to their date: same calendar day is not 'newer'.""" + import datetime as d + + from cotdata.providers.norgate import _finals_ready_by_date + same_day = _finals_ready_by_date(d.datetime(2026, 7, 27, 23, 0), d.datetime(2026, 7, 27, 8, 0)) + assert same_day[0] is False + next_day = _finals_ready_by_date(d.datetime(2026, 7, 28, 9, 0), d.datetime(2026, 7, 27, 20, 0)) + assert next_day[0] is True + + +def test_finals_ready_quorum(): + """Ready only when EVERY reference symbol has a newer settled bar than the store.""" + import datetime as d + + from cotdata.providers.norgate import _finals_ready_quorum + mon, tue = d.date(2026, 7, 27), d.date(2026, 7, 28) + ok, det = _finals_ready_quorum( + {"ES": tue, "CL": tue, "ZC": tue}, {"ES": mon, "CL": mon, "ZC": mon}) + assert ok is True and det["mode"] == "data" + # one reference still lagging (Norgate hasn't published it) -> defer, wait for all + ng, _ = _finals_ready_quorum( + {"ES": tue, "CL": mon, "ZC": tue}, {"ES": mon, "CL": mon, "ZC": mon}) + assert ng is False + # nothing new anywhere -> not ready + assert _finals_ready_quorum({"ES": mon, "CL": mon}, {"ES": mon, "CL": mon})[0] is False + + +def test_finals_ready_gathers_and_delegates(monkeypatch): + """finals_ready reads norgate+store dates per ref symbol and delegates to the quorum; + the legacy `cutoff` arg is accepted but ignored.""" + import datetime as d + + from cotdata.providers import norgate + monkeypatch.setattr(norgate, "_require_norgate_service", lambda: None) + ng = {s: d.date(2026, 7, 28) for s in norgate._FINALS_REF_SYMBOLS} + st = {s: d.date(2026, 7, 27) for s in norgate._FINALS_REF_SYMBOLS} + monkeypatch.setattr(norgate, "_norgate_last_bar_date", lambda s: ng[s]) + monkeypatch.setattr(norgate, "_store_last_bar_date", lambda s: st[s]) + ready, detail = norgate.finals_ready() + assert ready is True + assert set(detail["per_symbol"]) == set(norgate._FINALS_REF_SYMBOLS) + assert norgate.finals_ready("20:55")[0] is True # cutoff ignored + + # a lagging reference (Norgate hasn't advanced it) keeps the whole gate closed + monkeypatch.setattr(norgate, "_norgate_last_bar_date", + lambda s: d.date(2026, 7, 27) if s == norgate._FINALS_REF_SYMBOLS[0] else ng[s]) + assert norgate.finals_ready()[0] is False