📈 LOB Feed Handler

A C++20 market data feed handler built to HFT-grade standards. Parses binary ITCH 5.0-style messages at 5M+ msgs/sec, maintains an in-memory limit order book with sub-microsecond update latency using lock-free queues, cache-aligned data structures, and a custom memory pool.

---

Architecture

Market Data Feed  →  ItchParser  →  BookManager  →  StrategyApi
   (binary ITCH)     (zero-copy)    (per-symbol)     (SPSC queue)
                        ↓
                  LatencyHistogram
                  (p50/p99/p99.9)

Components

Layer	File(s)	Description
Types	include/common/types.hpp	Price, Qty, OrderId, Side, Bbo
Ring Buffer	include/common/ring_buffer.hpp	Lock-free SPSC, power-of-2 capacity, cache-line padded
Memory Pool	include/common/memory_pool.hpp	Fixed-size slab allocator, O(1) alloc/free, zero heap in hot path
RDTSC Timer	include/common/rdtsc.hpp	TSC calibration, cycles_to_ns(), RAII ScopeTimer
Logger	include/common/logger.hpp	Async lock-free logger (ring buffer + drain thread)
ITCH Messages	include/parser/itch_messages.hpp	#pragma pack(1) structs matching ITCH 5.0 wire layout
ITCH Parser	include/parser/itch_parser.hpp	Zero-copy, template-visitor (no vtable), inline byte-swap
Order	include/orderbook/order.hpp	alignas(64) — exactly one cache line, intrusive linked list
Price Level	include/orderbook/price_level.hpp	FIFO doubly-linked list of orders at a price
Order Book	include/orderbook/order_book.hpp	Sorted bid/ask maps, O(1) order lookup, pool-backed
Book Manager	include/orderbook/book_manager.hpp	Multi-symbol manager, ITCH visitor, BBO publisher
Feed Handler	include/feed/feed_handler.hpp	Pipeline: ring buffer → parser → book manager
Strategy API	include/strategy/strategy_api.hpp	BBO updates via SPSC ring buffer to strategy threads
Histogram	include/metrics/latency_histogram.hpp	HDR-style fixed-bucket, p50/p99/p99.9/max
Feed Generator	src/simulation/feed_generator.cpp	Synthetic ITCH stream (60% add / 20% cancel / 20% exec)
Replay Engine	src/simulation/replay_engine.cpp	Binary ITCH file replay at max speed

---

Performance Design

Technique	Where
Lock-free SPSC ring buffer	RingBuffer<T, N> — wait-free push/pop, false-sharing eliminated by cache-line padding on head/tail
Slab memory pool	MemoryPool<T, N> — zero heap calls in hot path after warmup
Cache-aligned Order struct	alignas(64) Order — exactly 64 bytes; intrusive linked list avoids secondary pointer chasing
Zero-copy parsing	ItchParser reads directly from wire buffer, no copies; template visitor inlined at compile time
Integer prices	int64_t with 4 implied decimal places — no floating-point in the hot path
RDTSC latency	Per-message cycle-accurate latency recording, converted via calibrated TSC frequency
[[likely]]/[[unlikely]]	Error paths and pool exhaustion marked unlikely for branch predictor hints

---

Build Requirements

Tool	Minimum version
C++ compiler	GCC 11+, Clang 14+, or MSVC 2022 (C++20)
CMake	3.20+
Git	For FetchContent (GoogleTest, Google Benchmark)
Internet	Required on first build to fetch test/benchmark deps

---

Building

# Configure (Release mode, native CPU optimizations)
cmake -B build -DCMAKE_BUILD_TYPE=Release

# Build all targets
cmake --build build --config Release -j$(nproc)

On Windows with MinGW:

cmake -B build -G "MinGW Makefiles" -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

On Windows with Visual Studio:

cmake -B build -G "Visual Studio 17 2022" -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release

---

Running

Simulate mode (synthetic feed)

./build/lob_feed_handler --mode simulate --symbols 100 --msgs 5000000

# Expected output:
# Calibrating TSC... 3.800 GHz
# Generating 5000000 synthetic ITCH messages (100 symbols)...
# Stream built: 218000000 bytes
# ■ Processing complete
#   Messages : 5000000
#   Time     : 0.921 s
#   Rate     : 5.43 M msgs/sec
#   BBO upd  : 1234567
#   Symbols  : 100

Replay mode (binary ITCH file)

./build/lob_feed_handler --mode replay --file /path/to/itch_data.bin

CLI Options

--mode simulate|replay   Operation mode (default: simulate)
--file <path>            ITCH binary file for replay mode
--symbols <N>            Number of symbols (default: 100)
--rate <N>               Target msgs/sec — informational (default: 5000000)
--msgs <N>               Total messages to generate
--duration <s>           Duration (used if --msgs not set, default: 5)
--no-stats               Suppress latency histogram
--help                   Show help

---

Tests

cd build
ctest --output-on-failure

# Or run directly:
./run_tests

Test suites:

Suite	What's verified
RingBuffer	Push/pop, full/empty states, SPSC stress (100K items, 2 threads)
MemoryPool	Allocate all slots, exhaustion, LIFO reuse, alignment
ItchParser	Add/execute/cancel parsing, multi-message stream, unknown message handling
OrderBook	Price priority, time priority, cancel/delete/execute, BBO updates, stats

---

Benchmarks

./build/lob_benchmarks --benchmark_filter=BM_Itch
./build/lob_benchmarks --benchmark_filter=BM_OrderBook

Expected results on modern hardware (3–4 GHz, L3 cache warm):

Benchmark	Result
BM_ItchParse_NullVisitor/1M	~150–200 ms → 5–6 M msgs/sec
BM_ItchParse_WithBookManager/1M	~250–400 ms → 2.5–4 M msgs/sec
BM_OrderBook_BestBidAsk	< 10 ns per query
BM_OrderBook_AddOrder/1K	< 500 ns per add

---

Project Structure

lob_feed_handler/
├── CMakeLists.txt
├── README.md
├── include/
│   ├── common/
│   │   ├── types.hpp          # Fundamental types
│   │   ├── ring_buffer.hpp    # Lock-free SPSC queue
│   │   ├── memory_pool.hpp    # Slab allocator
│   │   ├── rdtsc.hpp          # TSC timer
│   │   └── logger.hpp         # Async lock-free logger
│   ├── parser/
│   │   ├── itch_messages.hpp  # Wire-layout ITCH structs
│   │   └── itch_parser.hpp    # Zero-copy template parser
│   ├── orderbook/
│   │   ├── order.hpp          # Cache-aligned Order (64 bytes)
│   │   ├── price_level.hpp    # FIFO price level
│   │   ├── order_book.hpp     # Per-symbol book
│   │   └── book_manager.hpp   # Multi-symbol manager
│   ├── feed/
│   │   └── feed_handler.hpp   # Pipeline orchestrator
│   ├── strategy/
│   │   └── strategy_api.hpp   # BBO publisher interface
│   ├── metrics/
│   │   └── latency_histogram.hpp
│   └── simulation/
│       ├── feed_generator.hpp
│       └── replay_engine.hpp
├── src/
│   ├── main.cpp
│   ├── parser/itch_parser.cpp
│   ├── orderbook/
│   │   ├── order_book.cpp
│   │   └── book_manager.cpp
│   ├── feed/feed_handler.cpp
│   ├── metrics/latency_histogram.cpp
│   └── simulation/
│       ├── feed_generator.cpp
│       └── replay_engine.cpp
├── tests/
│   ├── test_ring_buffer.cpp
│   ├── test_memory_pool.cpp
│   ├── test_parser.cpp
│   └── test_order_book.cpp
└── benchmarks/
    ├── bench_parser.cpp
    └── bench_order_book.cpp

---

Key Design Decisions

Why SPSC ring buffer?
A single-producer / single-consumer design is the correct model for the network-thread → book-engine pipeline: it achieves wait-free semantics with no locking, and the cache-line separation of head/tail eliminates all false sharing.

Why integer prices?
Floating-point arithmetic introduces rounding and is slower on many paths. Scaled integers (int64_t × 10000) are exact, compare in one instruction, and enable deterministic order book logic.

Why a slab pool for orders?
malloc inside a microsecond-critical hot path is unacceptable. The pool pre-allocates all slots at startup; allocate() and deallocate() are single array-indexing operations with no system calls.

Why template visitor in the parser?
Avoids vtable dispatch. The compiler inlines the visitor callbacks at the call site — the entire parse → book-update path can be a single inlined sequence with no indirect calls.