Skip to content

Add compiled batch ingestion for C++ engine#46

Merged
tradingexpert merged 2 commits into
mainfrom
feature/cpp-batch-ingestion
May 18, 2026
Merged

Add compiled batch ingestion for C++ engine#46
tradingexpert merged 2 commits into
mainfrom
feature/cpp-batch-ingestion

Conversation

@tradingexpert
Copy link
Copy Markdown
Owner

What changed

  • add a public CompiledEventColumns helper that turns canonical MBOEvent rows into primitive columns
  • add a compiled CppMatchingEngine.apply_events_batch(...) path over contiguous buffer inputs
  • keep ordinary Replay(...) event-by-event so per-event valuation marks and equity curves stay unchanged
  • document the distinction between replay semantics and direct compiled batch ingestion

Why

The private research engine gets its real C++ throughput from batch ingestion, while the public extraction only had one Python-to-C++ crossing per event. This ports the missing capability without weakening the inspectable replay contract.

Validation

  • python -m ruff check .
  • python -m pytest
  • native C++ core test binary
  • local targeted parity check against the private futures engine on the queue-fill scenario from PR Fix Databento trade normalization #45
  • local throughput smoke test: public compiled batch path ~28.6M events/sec vs public scalar ~1.9M events/sec on this machine; private research batch path ~33.7M events/sec on the same broad workload shape

@tradingexpert tradingexpert self-assigned this May 18, 2026
@tradingexpert tradingexpert merged commit de83c64 into main May 18, 2026
8 checks passed
@tradingexpert tradingexpert deleted the feature/cpp-batch-ingestion branch May 20, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant