Skip to content
View Aravind0403's full-sized avatar
2 followers · 2 following

Achievements

Achievement: YOLOAchievement: QuickdrawAchievement: Pull Shark

Achievements

Achievement: YOLOAchievement: QuickdrawAchievement: Pull Shark

Block or report Aravind0403

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Aravind0403/README.md

Aravind Sundaresan

AI Systems Engineer · Heterogeneous LLM Inference & Engine Internals · Distributed GPU Systems

LinkedIn arXiv Portfolio Email

⚡ About Me

AI Systems Engineer specialising in heterogeneous LLM inference: hardware-aware scheduling, disaggregated serving, and engine internals on vLLM and SGLang.

  • 7+ years of distributed-systems infrastructure at Microsoft and Amazon combined with 3 years of self-funded inference systems research.
  • Upstream contributor to vLLM and SGLang core schedulers, memory managers, quantization backends, and cache hot paths.
  • Author of HeteroDisagg (asymmetric disaggregated serving) and Clairvoyant (ML-driven SJF inference scheduling, arXiv:2606.07248).

🚀 Upstream Open-Source Contributions

Core engine internals contributed to vLLM and SGLang:

Engine Pull Request Component Architectural Impact Status
SGLang PR #39623 P/D Disaggregation Fixed unbounded memory growth and worker OOM crashes via admission guard, thread-safe RLock state purge, and an $O(1)$ min-heap TTL sweeper across Mooncake, Mori, and Nixl backends. In Review
SGLang PR #39741 Quantization & Marlin Resolved Marlin GEMM tile divisibility aborts ($N \pmod{64} == 0$) and dropped weights for Qwen3.5 GDN hybrid models via automated shape validation & dynamic unquantized fallback. In Review
vLLM PR #56843 / #51574 Core Scheduler Fixed preemption re-admission ordering in PriorityRequestQueue to preserve computed KV cache; added starvation cap min(n, 3) and anti-thrashing victim selection. In Review
vLLM PR #56964 MoE & Speculative Decoding Resolved MoE oracle selection aborts for unquantized draft layers during MTP speculative decoding by falling through quantization-only backends (Marlin/Hummingbird) to auto. In Review
SGLang PR #37067 Prefix Cache / RadixTree Optimized RadixCache.total_size() from $O(N)$ recursive tree traversal to $O(1)$ by maintaining incremental mutation invariants, eliminating DFS overhead and GIL contention. In Review

🛠️ Featured Systems & Research Projects

Disaggregated Serving & LLM Inference Engines

Project Description Key Performance & Benchmarks
HeteroDisagg
Disaggregated LLM Serving 		Research prototype for hardware-adaptive disaggregated LLM serving across mixed-silicon clusters (A100 + RTX 3090). Features asymmetric TP and wire quantization. • Asymmetric TP: HeteroKVConnector for GQA bipartite P2P re-sharding across mismatched prefill/decode ranks (prefill_tp != decode_tp).
• In-Flight FP8: Vectorised FP8 E4M3 quantization halved wire payload (128 MB → 64 MB) at 99.98% cosine fidelity and $\Delta\text{PPL} \le 0.0018$.
• Accelerator Capability Descriptor (ACD): Roofline planner deriving chunk sizing and asymmetric TP across silicon.
Clairvoyant Scheduler
📄 arXiv:2606.07248 		Zero-overhead Go sidecar proxy eliminating Head-of-Line blocking in LLM inference via ML-driven Shortest Job First (SJF) scheduling. • 70–76% P50 latency reduction under burst load.
• 83.6% latency reduction under realistic trace replay.
• 0.029 ms inline ONNX inference overhead with zero backend modifications.
AetherControl
Full LLM Lifecycle Control Plane 		4-stage platform: dataset pre-flight guard (trainsight) → GRPO post-training (rlhf-pipeline) → vLLM serving (vllm-engine) → K8s governance. • 1,141 tok/s throughput · TTFT P50 22.1ms.
• 100% OOM prevention via pre-flight profiler.
• GSM8K: 42% → 70% via GRPO · −98.6% DCGM mutex contention.

Cluster Orchestration & Infrastructure

Project Description Key Performance & Benchmarks
Kube-Sentinel
Zero-Trust K8s GPU Scheduler 		Native Go Scheduling Framework plugin (Score/Reserve/Bind) + Python gRPC telemetry daemon cross-checking node telemetry against scheduler cache truth. • 46.3% GPU cost reduction ($120/hr → $64.44/hr) on 32-node Alibaba cluster trace.
• 1,252 pods/sec peak throughput with sub-millisecond P99 latency.
• 100% failover availability under daemon SIGKILL.
ACO Orchestrator Predictive job scheduler for heterogeneous compute using Ant Colony Optimization + LSTM spike prediction. • <10ms latency · 95%+ SLA adherence across 202 test scenarios.
ServiceScope v2 AI-native blast-radius analysis for Python microservices: AST parsing + local LLM inference. • 190 files/sec · 0% inference failure · Zero external API dependencies.

🔬 Research & Preprints

  • Clairvoyant: ML-Driven Shortest Job First Scheduling for LLM Inference
    Aravind Sundaresan — arXiv:2606.07248
    Target: NeurIPS 2026 Workshop → MLSys 2027

💻 Technical Competencies

  • Inference Engines & Serving: vLLM internals, SGLang internals, PagedAttention, RadixAttention, Continuous Batching, Chunked Prefill, Disaggregated Prefill/Decode, Asymmetric TP, GQA re-sharding, KV-cache lifecycle & quantization.
  • Quantization & Acceleration: Marlin GEMM kernels, GPTQ, AWQ, FP8 (E4M3), MoE kernel dispatch, Triton kernels, CUDA C++, CUDA IPC, NCCL, Roofline analysis.
  • Languages & Frameworks: Python, Go, C++, CUDA, Java, PyTorch, Triton, ONNX Runtime.
  • Orchestration & Distributed: Kubernetes (custom Go scheduler plugins), Docker, DCGM, Prometheus, Grafana, AWS, Azure, GCP, InfiniBand/RoCE, NVLink.

📬 Connect

Pinned Loading

  1. aether-control-llm-infra aether-control-llm-infra Public

    Enterprise LLM Serving Platform, Control Plane & Post-Training Pipeline (vLLM, GRPO, K8s).

    Python

  2. Kube-Sentinel Kube-Sentinel Public

    Trust-weighted, cost-optimal Kubernetes scheduler plugin with zero-trust telemetry verification.

    Python

  3. clairvoyant-scheduler clairvoyant-scheduler Public

    Go sidecar proxy that eliminates Head-of-Line Blocking in LLM inference via ML-driven SJF scheduling — zero backend modification. Paper in preparation

    Python 1

  4. ServiceScope-v2 ServiceScope-v2 Public

    AI-native blast-radius analysis for Python microservices — AST parsing + local LLM inference + dependency graph. No service mesh needed

    Python

  5. ACO_Adaptive_Compute_Orchestrator ACO_Adaptive_Compute_Orchestrator Public

    Predictive job scheduler for heterogeneous compute — ACO + LSTM spike prediction + intent-aware routing. <10ms latency, 95%+ SLA adherence, 202 tests

    Python