Production-Grade C++17 AI Serving Engine with Arena Memory Pool, Dynamic Batching, Paged KV-Cache & Dual OpenAI/Ollama Protocols.
high-performance-computing cpp17 pybind11 fastapi vllm llm-inference paged-kv-cache ai-serving-engine
-
Updated
Sep 30, 2026 - Python