tradeyog-validation

Frequency-aware Stage-4 statistical validation battery for cross-sectional (monthly) and intraday equity strategies. Calibrated for Indian F&O 209 long-only quintile portfolios and Indian intraday futures strategies, but designed to apply to any monthly or intraday strategy with appropriate factor benchmarks.

The methodology document is docs/methodology.html -- open it in any browser. It explains each gate, its literature anchor, the threshold, and how to interpret failures.

Choosing the right gate set

Strategy type	Frequency	Gates
Monthly cross-sectional long-only	monthly	8 (+1 optional)
Daily-resolution swing / momentum	daily	10
Intraday futures / opening-range / lead-lag	intraday	12

The runner dispatches on the frequency argument. The same gate name often applies at both frequencies but with frequency-specific parameters (annualisation factor, block length, factor model, friction).

The gates

Monthly (8 core + 1 optional)

1. Reproducibility -- saved vs regenerated series byte-match
2. Walk-forward -- 3 non-overlapping windows all positive Sharpe
3. Bootstrap 95% CI -- lower bound > 0.5 (block-bootstrap, n=2000, block=6m)
4. Alpha -- FF3+WML Newey-West HAC-6 t(alpha) > 3.0 (Indian FF3+WML via IFFD)
5. Permutation null -- K=300 sign-flip, p < 0.05
6. Orthogonality -- |rho| < 0.7 against Amihud Q5 (Indian liquidity-premium orthogonality)
7. Friction stress -- Sharpe > 0 at 2x baseline friction
8. MaxDD vs buy-hold -- strategy MaxDD less negative than equal-weight benchmark
9. Multiple-testing (optional) -- observed Sharpe > Bailey - Lopez de Prado expected null-best

Intraday (12)

1. Reproducibility -- trade-level series byte-match
2. Walk-forward -- 3 non-overlapping daily-aggregated windows, all Sharpe > 0
3. Bootstrap 95% CI -- daily aggregation, 5-day blocks, sqrt(252) annualisation
4. Alpha -- Lou-Polk-Skouras (2019) two-factor decomposition vs NIFTY OC + CO, Newey-West HAC-3 t(alpha) > 2.0
5. Permutation null -- trade-shuffle (preserves timing) if trades supplied, else daily sign-flip; K=300, p < 0.05
6. Orthogonality -- |rho| < 0.7 against naive intraday benchmark
7. Friction stress -- 4 bp/RT for index futures, 30 bp/RT for delivery intraday; 2x stress
8. MaxDD -- across-session vs buy-hold AND (if trades supplied) within-session peak-to-trough summary
9. Trade count / capacity -- trades/year >= 100 (statistical power threshold)
10. Regime stability -- pre/post regime_split_date both positive Sharpe
11. Multiple-testing (caveat) -- observed Sharpe vs Bailey - Lopez de Prado expected null-best at N candidate variants searched
12. Time-of-day -- requires trades; at least 3 of 12 30-minute session buckets must have positive Sharpe (not single-window concentrated)

Install

pip install -e .

Dependencies: pandas, numpy, statsmodels, scipy.

Run the demos

python examples/run_demo_monthly.py
python examples/run_demo_intraday.py

Monthly demo runs the battery on the bundled Momentum Q5 sample (F&O 209, ~261 monthly observations) -- expected 6/8 PASS.

Intraday demo runs on:

• leadlag_daily_net.csv (Sharpe ~ 1.12) -- expected ~8/11 PASS
• orb_zarattini_daily_net.csv (Sharpe < 0) -- expected ~5/11 PASS

Reports + JSON dumps land in examples/output_example/.

Validate your own strategy

Monthly

import pandas as pd
from validation import run_validation, load_iffd_csv

my_returns = (pd.read_csv("my_strategy.csv", parse_dates=["date"])
                .set_index("date")["net_return"])

iffd     = load_iffd_csv("sample/monthly/iffd_monthly_sba.csv")
amihud   = pd.read_csv("sample/monthly/amihud_q5_net.csv",
                         parse_dates=["date"]).set_index("date")["net_return"]
buy_hold = pd.read_csv("sample/monthly/fno_buy_hold_monthly.csv",
                         parse_dates=["date"]).set_index("date")["monthly_return"]

result = run_validation(
    strategy_net_returns=my_returns,
    frequency="monthly",
    iffd_factors=iffd,
    amihud_benchmark=amihud,
    buy_hold_benchmark=buy_hold,
    friction_bp_per_rt=15,
    friction_stress_bp_per_rt=30,
)
print(result["report_md"])

Intraday

import pandas as pd
from validation import run_validation

my_returns = (pd.read_csv("my_intraday_daily_net.csv", parse_dates=["date"])
                .set_index("date")["net_return"])

# Optional: trade-level dataframe for richer gates
# trades = pd.read_parquet("my_trades.parquet")
# columns expected: entry_ts, exit_ts, side, ret_net

factors = pd.read_csv("sample/intraday/nifty50_daily_oc_co.csv",
                       parse_dates=["date"]).set_index("date")
naive_bench = factors["open_to_close"].dropna()
buy_hold = ((1 + factors["close_to_open"].fillna(0.0)) *
            (1 + factors["open_to_close"].fillna(0.0)) - 1)

result = run_validation(
    strategy_net_returns=my_returns,
    frequency="intraday",
    intraday_factor_returns=factors,
    naive_intraday_benchmark=naive_bench,
    buy_hold_benchmark=buy_hold,
    friction_bp_per_rt=4,            # index futures intraday
    friction_stress_multiplier=2.0,
    regime_split_date=pd.Timestamp("2025-12-08"),
    n_tests_searched=10,
    multiple_testing_hard_fail=False,
)
print(result["report_md"])

Promotability thresholds

• Monthly: 6-of-8 PASS is promotable; 5-of-8 borderline; below 5 retire/redesign.
• Intraday: 8-of-12 (or 8-of-11 without time-of-day) is promotable; 6-7 borderline; below 6 retire.

Sample data

Monthly (sample/monthly/)

• momentum_q5_net.csv -- 12-1 Jegadeesh-Titman momentum Q5 long-only monthly net returns on F&O 209, 15 bp/RT friction
• amihud_q5_net.csv -- Amihud illiquidity Q5 reference benchmark
• fno_buy_hold_monthly.csv -- F&O 209 equal-weight monthly buy-hold
• iffd_monthly_sba.csv -- IIMA Indian Fama-French Database (SBA, monthly)

Intraday (sample/intraday/)

• leadlag_daily_net.csv -- daily-aggregated net returns from a lead-lag cross-asset strategy on Indian index futures (2014-05 to 2026-05).
• orb_zarattini_daily_net.csv -- daily-aggregated net returns from a Zarattini-style opening-range breakout (failed; bundled as a FAIL demo)
• nifty50_daily_oc_co.csv -- NIFTY-50 daily open-to-close + close-to-open
  • realized intraday volatility, derived from 1-min spot bars

All derived from public Indian market data.

Library API

from validation import run_validation, Frequency

# All gates dispatched by frequency
result: dict = run_validation(
    strategy_net_returns=...,
    frequency: Frequency = "monthly",  # or "daily" / "intraday"
    # ...frequency-appropriate inputs...
)

Or use individual gate functions:

from validation import (
    reproducibility_gate, walk_forward_gate, bootstrap_ci_gate,
    alpha_gate, permutation_gate, orthogonality_gate,
    friction_stress_gate, maxdd_gate,
    trade_count_gate, regime_stability_gate,
    multiple_testing_gate, time_of_day_gate,
)

For backwards compatibility with v0.1, the legacy monthly entry point still works:

from validation.runner import run_stage4_battery     # v0.1 signature
from validation.ff3 import load_iffd_csv             # v0.1 path

Caveats

• Monthly battery is calibrated for Indian F&O 209 monthly long-only strategies. Short-side or non-Indian universes need different orthogonality benchmarks and possibly different gate thresholds.
• Intraday battery is calibrated for Indian index futures intraday. Single-stock intraday strategies should switch friction to 30 bp/RT delivery-style cost.
• Bootstrap CI is sample-power-dependent. Samples below ~60 monthly observations (or ~250 daily) will produce wide CIs that may fail the 0.5 lower bound even on real strategies.
• The multiple-testing gate is informational by default (multiple_testing_hard_fail=False). Set hard_fail=True to make it binding only if you have a defensible value for n_tests_searched.

License

MIT -- see LICENSE.