Skip to content

README.md

Latest commit

 

History

History
257 lines (205 loc) · 11.4 KB

README.md

File metadata and controls

257 lines (205 loc) · 11.4 KB
Raw

backtester

Tick-level futures backtester for NinjaTrader Market Replay data (Parquet, see M:\NinjaTrader_DataRepo\RawData\Parquet\README.txt). Built for fast iteration on intraday prop-firm strategies before porting them to NinjaTrader 8.

New here?

  1. Copy or download this whole backtester folder onto your machine (it's self-contained — code, tests, and docs; the tick data itself lives separately, see BACKTESTER_DATA_ROOT below).
  2. Open Claude Code in this folder as your working directory. It auto-loads CLAUDE.md, which has the architecture, conventions, and a full writeup of the validated GodZillaKilla confluence settings — you don't need to paste anything in for Claude to see it.
  3. Point your own tick data at it: set BACKTESTER_DATA_ROOT to wherever your NinjaTrader Market Replay Parquet repo lives (same folder layout as the path above), or ask Claude to help you get data in place.

A good first prompt to paste in:

I just set up this backtester repo. Read CLAUDE.md and README.md to get
oriented, then walk me through running my first backtest. I'd like to
experiment with variations on the GodZillaKilla confluence strategy
settings documented in CLAUDE.md — help me set up a few of my own configs
to test.

Quick start

.venv\Scripts\python cli.py strategies\ema_cross.py --start 2026-06-01 --end 2026-06-17

Produces a console summary and an HTML tearsheet in reports\ (equity curve with the Apex trailing floor overlaid, drawdown, daily P&L, trade distribution, full trade list).

First touch of each day reduces the raw ~24M-event file to trade events with prevailing bid/ask attached and caches it under .cache\ (plus per-period bar caches). First pass over a day costs a few seconds; cached runs are ~0.1 s/day.

Writing a strategy

from backtester import EMA, Strategy

class MyStrat(Strategy):
    symbol = "MNQ"
    period = "1m"                    # time: 30s/1m/5m; tick: 500t; renko: r8
    session = ("09:30", "16:00")     # US/Eastern; None = full day
    flat_at_session_end = True
    qty = 2

    def on_start(self):
        self.fast, self.slow = EMA(9), EMA(21)

    def on_bar(self, bar, bars):     # bar.open/high/low/close/volume/ts
        f, s = self.fast.update(bar.close), self.slow.update(bar.close)
        if self.slow.ready and self.flat and f > s:
            self.buy_bracket(stop_ticks=40, target_ticks=80)

Hooks: on_start, on_bar(bar, bars), on_fill(fill), on_session_end(date), on_finish. Orders: buy/sell (market), buy_bracket/sell_bracket, buy_limit/sell_limit, buy_stop/sell_stop (all accept stop_ticks/target_ticks brackets), close_position(), cancel_all(). Order management (ATM-style): move_stop(price), move_target(price), move_stop_to_breakeven(offset_ticks=), and stop_order / target_order / working_orders for direct inspection — call from on_bar to trail stops. State: self.position, self.flat, self.avg_price, self.balance. Indicators (incremental, NT8-style): EMA, SMA, Bollinger, ATR, RSI, EfficiencyRatio, Highest, Lowest.

Multi-timeframe & tick-level strategies: declare secondary_periods = ["15m"] to get on_secondary_bar(bar, bars, period) fired the instant each secondary bar closes (no look-ahead) and self.secondary(period) for its history. Defining on_tick(ts, price, index) switches that run to a per-event resolver — orders submitted in on_tick fill on later events only — for strategies that need intrabar reaction; strategies that define neither stay on the fast vectorized path.

Confluence / NT8-template-driven strategies: backtester/nt8config.py parses saved NT8 ATM templates and strategy-template XML (brackets, breakeven, tiered trailing, bar type, time windows) so a live NT8 config can drive a Python backtest directly; backtester/atm.py executes the resulting multi-bracket exits. sweep_confluence.py sweeps which signal engines are required and how many must agree. See strategies/godzilla_killa.py for a worked example (six independent signal engines voting per bar).

Bar types

  • Time — 30s, 1m, 5m, 1h (bar timestamp = close time, NT8-style; empty bars omitted).
  • Tick — 500t: fixed trade-count bars.
  • Renko (ninZaRenko) — r8-4: brick size 8 ticks (every bar's body height), trend threshold 4 ticks (with-trend close distance from the previous close). r8 defaults trend to brick/2. Implements the published ninZaRenko manual: open offset = brick − trend (bars overlap), reversal threshold = 2·brick − trend, equal bodies both directions. Manual's recommended configs: 8-4, 15-5, 12-4, 20-5, 30-10. High/low include the synthetic open — matching what NT8 indicators see on ninZaRenko bars.

Bar type only changes when the strategy is asked to decide. Orders always fill against the real tick stream, so none of NT8's Renko fantasy-fill problem applies — a Renko strategy backtested here gets honest fills.

Fixed (2026-07-11): renko bars reset incorrectly at midnight ET. Raw data is stored as one file per ET calendar day, and the renko builder used to reset its brick anchor at the start of every file — correct behavior for a real session gap (e.g. the daily 17:00–18:00 ET halt), but wrong for an overnight session (e.g. ("18:00", "16:55")) that keeps trading straight through midnight ET with no actual gap there. The result: renko geometry was correct for the evening leg of each session but silently wrong for the rest of the day, every day, for any strategy spanning midnight. Confirmed against a real NT8 chart export — bar mismatches were ~0% right after the real halt reset, then jumped to 45–69% at midnight and stayed wrong until the next halt. Fixed by carrying the brick state across day-file boundaries and resetting only on a genuine gap (Catalog.load_bars_sequence in backtester/data.py); verified back up to 99.8% bar-for-bar match on the same export. If you pulled this repo before that fix and have a populated .cache\bars\, no action needed — the cache version bump forces a transparent rebuild on next use. Headline strategy results computed before the fix should be treated as approximate for any renko-bar strategy using an overnight session; see NinjaScript/TerminatorV2/TerminatorV2.md and strategy/GodZillaKilla.md for the specific before/after numbers on this project's two reference strategies (both moved only slightly).

Order flow (what NT8 backtests can't see)

Every trade in the reduced cache is classified by aggressor side (at/above ask = buy, at/below bid = sell), and every bar — any type — carries bar.buy_volume, bar.sell_volume, and bar.delta. bars.delta / bars.cum_delta (session-cumulative) are available as history arrays for delta-divergence and order-flow filters. Prevailing bid/ask queue sizes are also cached per trade for order-imbalance (OIB) research.

Position sizing & risk

  • self.vol_target_contracts(daily_atr_points) — Carver volatility targeting (15% annual default). Pass a daily ATR.
  • --daily-loss-limit 600 — flatten and stand down for the rest of the day when the day's loss touches the limit; hit days are listed in the summary.

Parameter sweeps

python sweep.py strategies\ema_cross.py --param fast_period=6,9,12 ^
    --param slow_period=18,21,27 --start 2026-03-01 --end 2026-06-17

Runs the full grid in parallel, ranks by --metric (sharpe default), writes reports\sweep_*.csv (columns include prop-firm min headroom), and prints a per-parameter sensitivity plateau around the best combo — a spike at one value with collapse next door is flagged FRAGILE (data-snooping, per Chan). Combos with fewer than --min-trades rank last.

Walk-forward analysis

python walkforward.py strategies\ema_cross.py --param fast_period=6,9,12 ^
    --param slow_period=18,21,27 --windows 5 --ratio 5

Rolling IS/OOS windows (5:1 default): optimize the grid in-sample, run the best combo out-of-sample, roll forward. Reports per-window IS vs OOS, the stitched OOS net/Sharpe (the only numbers that haven't seen their own data), and walk-forward efficiency with Davey's verdict (< 0.5 = likely curve-fit).

Fill model

  • Strategy logic runs on bar closes; orders resolve against the underlying trade-event stream inside each bar (no look-ahead — fills happen before the strategy sees the bar).
  • Market orders fill on the next trade event at the prevailing ask (buy) / bid (sell), plus --slippage ticks if set. The spread is a real cost.
  • Limit orders fill when price trades through the limit (touch alone never fills — approximates queue risk). Marketable limits fill at the quote.
  • Stops trigger on last, fill at the quote, never better than the stop price.
  • Commissions are per-contract round-turn defaults in backtester/contracts.py — adjust to your firm's rates.

Prop-firm simulation

The trailing threshold (modeled on Apex's real rule set) trails the intratrade equity peak (unrealized included) and, by default, locks at start balance + a small buffer. A breach is equity touching the floor.

--balance 50000 --prop-threshold 2000     # 50K account defaults ($2,000)
--prop-halt                               # stop the test at the breach
--prop-threshold 0                        # disable

The console summary reports either the breach timestamp or the minimum headroom that survived; the tearsheet plots the floor under the equity curve.

Two further Apex rules are modeled:

  • Max position size — ContractSpec.apex_max_position (6 full-size minis / 60 micros) is enforced by the broker automatically per symbol; override via Strategy.max_position (0 disables).
  • 30-second minimum hold — Strategy.min_hold_s = 30 blocks close_position() until a position has been held that long (force=True bypasses it for risk stand-downs like a daily-loss lock). Every run also reports sub-30-second exposure (trade count, $ P&L) regardless of whether it's enforced, since a real account may flag or void those trades even when the backtest doesn't gate them — check this before trusting a result built on very short holds.

Monte Carlo

Every run (unless --mc 0) resamples the closed-trade P&L 2,000× to separate skill from ordering luck: 5/50/95th-percentile final P&L, max-drawdown distribution, P(breaching the Apex trailing threshold) across orderings, and with --mc-target 3000 the eval race — P(hitting the target before a breach). Block bootstrap is used automatically when trade returns are serially correlated (|r| > 0.2, per Davey).

CLI

python cli.py <strategy.py> [--symbol MNQ] [--period 1m] [--start D] [--end D]
              [--balance 50000] [--apex-threshold 2500] [--apex-halt]
              [--slippage 0] [--out report.html] [--no-report] [--data-root P]

Env overrides: BACKTESTER_DATA_ROOT, BACKTESTER_CACHE.

Each report also writes <name>_trades.csv. To validate fills against NinjaTrader, export the same strategy's trades from NT8 Strategy Analyzer (tick replay) and run:

python tools\compare_nt8.py reports\MyStrat_MNQ_trades.csv nt8_export.csv --symbol MNQ

It matches trades by direction + entry time and reports entry/exit price deltas in ticks.

Tests

.venv\Scripts\python -m pytest tests -q

Covers fill semantics (market/limit/stop/bracket/OCO/reversals), account math, bar building, and the Apex trailing/lock/halt behavior on synthetic tick streams.

Not yet implemented

  • Multi-symbol portfolios (one symbol per Backtest run)