Skip to content
View Alex-Wengg's full-sized avatar
🍔
🍔
100 followers · 46 following

Achievements

Achievement: Starstruckx3Achievement: Pull Sharkx3Achievement: Pair Extraordinairex3Achievement: YOLOAchievement: QuickdrawAchievement: Arctic Code Vault Contributor

Achievements

Achievement: Starstruckx3Achievement: Pull Sharkx3Achievement: Pair Extraordinairex3Achievement: YOLOAchievement: QuickdrawAchievement: Arctic Code Vault Contributor

Organizations

@job-pulse @slipboxai @FluidInference

Block or report Alex-Wengg

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Alex-Wengg/README.md

Hello, I'm Alex

AI and machine learning engineer building agentic systems and inference engines from client side to server side.

Contributions

Open-source collaborations and apps

  • vLLM (100K ⭐)

    • Added NVIDIA Canary (canary-1b-v2): speech recognition and X↔English speech translation in 25 European languages, served through the OpenAI-compatible audio API; output is token-identical to Transformers, with 12.34 WER on earnings22 (#64215)
    • Built vllm-parakeet-plugin for NVIDIA Parakeet TDT v3 (parakeet-tdt-0.6b-v3), now listed in vLLM's supported models (#65005): batched greedy TDT decoding on the GPU with its own CUDA graphs; 73/73 transcripts token-identical to Transformers, and LibriSpeech WER 1.93% / 3.61% through vllm serve (model card: 1.93% / 3.59%)
    • Built vllm-nemotron-diarization-plugin for NVIDIA Nemotron-3-Diarization, now listed in vLLM's supported models (#65199): speaker diarization of up to 8 speakers at 10 ms resolution through vLLM's LLM API with one pip install; logits within 2e-5 of Transformers, with identical speaker segments
    • Fixed main's failing pre-commit after two merged PRs collided on Muse Spark's weight-loading hook, which also restored its ROCm monokernel path (#65080)

  • SGLang — fixed the diffusion LoRA merge cache silently reusing stale merged weights after an adapter is rewritten in place, with regression tests (#44204), and made a diffusion worker exit after an unrecoverable CUDA error, instead of serving on a broken device while /health still reported ok (#45065) (40K+ ⭐)

  • Supertonic — ported Supertonic 3 to Core ML for Apple Neural Engine inference on iOS and macOS (conversion mobius#64, runtime FluidAudio#617) (13.7K+ ⭐)

  • NVIDIA Dynamo — improved disaggregated serving and inference infrastructure: made the prefill router fail closed with a retryable 503 while a target activates (#16736), sized multimodal cache budgets for DeepStack embeddings (#17249), and fixed DCGM metrics scrape connection handling (#17387) (8.4K+ ⭐)

  • mlx-audio — added Kokoro-82M v1.1 Mandarin support in MLX (#341) (8.1K+ ⭐)

  • vLLM-Omni — added Kokoro-82M, the most-downloaded open TTS model, as a native single-stage TTS: loads hexgrad's checkpoint as-is with output identical to the reference package, all 54 voices in 9 languages, streaming behind an OpenAI-compatible endpoint (#9637) (7.4K+ ⭐)

  • vLLM Semantic Router (6.3K+ ⭐)

    • Member of two workgroups: Router Models & Inference Runtime and Developer Experience & Ecosystem
    • Early access to Decision 3.0: ported d3 to Core ML (d3-coreml)
    • Wrote the Decision 3.0 on Apple Silicon blog post: d3-lite, nano and mini on Core ML, matching upstream on JevBench and typed-decisions, 2.5–4× faster per request on a Mac (#4905, #4975)
    • Fixed external RAG result ranking across query windows (#3750)
    • Secured Playground feedback for read users (#3755)
    • Hardened Dashboard API authorization with mutation-boundary rechecks (a3174df)

  • KServe — made KEDA autoscaling honor the target type an InferenceService sets for External and PodMetric metrics, instead of silently defaulting to AverageValue (#6185), and fixed in-flight gRPC requests failing on SIGTERM by unloading models only after the servers drain (#6488) (6.1K+ ⭐)

Projects I work on — at Fluid Inference, which was acquired by Levenza

  • FluidAudio — speech AI SDK for iOS and macOS (3.2K+ ⭐)
  • FluidUse — Core ML runtimes for decision models in Swift, including Decision 3.0 (D3Manager)
  • möbius — model conversion and deployment across accelerators
    • 500,000+ monthly model downloads across the Core ML models on FluidInference
1,001 Twitter followers

Some Experiment Highlights (Swift runtimes in FluidUse)

  • laya — Convai's typed decision model with 99.5% of ops on the Apple Neural Engine, 3.7 ms per decision on an M5 Pro (demo) · 1,323 likes · 111 reposts · 120K views
@jackzampolin
image		 @JiliJeanlouis
image		 @ClementDelangue reposts
image		 @victormustar
image
  • GLiNER2.5-Decide — Fastino's 340M decision model on Core ML: ~4× faster, ~5× less peak RAM, half the size (demo) · 880 likes · 83 reposts · 69K views

https://x.com/fastinoAI

image
  • EmbeddingGemma 2 — text encoder 100% on the Apple Neural Engine; sorts 10,080 posts into topics live at ~620/s on a MacBook, any topic splits into subtopics in under a second (demo) · 693 likes · 51 reposts · 34K views

https://x.com/SeraAndroid/status/2108629662238961694?s=20

https://x.com/googlegemma/status/2115150306032193954

image image
  • clef-text-0.6B — distilled Cloudflare's 9B clef-flash into a 0.6B decision model that runs 100% on the Apple Neural Engine; 1,000 support tickets triaged in 57 s on an M5 Pro, 48 ms per ticket, within a few points of the 9B on classification (demo) · 494 likes · 45 reposts · 32K views
@art_zucker reposts
image		 @ClementDelangue reposts
image
  • Decision 3.0 — d3-lite, nano and mini on Core ML; d3-lite triages 1,000 pull requests into vLLM-SR workgroups live on an M5 Pro at 0.12 s per PR, 8.3 PRs/s (demo) · 432 likes · 42 reposts · 28K views
@ClementDelangue
image		 @victormustar
image

Find me

Resume · Twitter · LinkedIn · HuggingFace

Swift Python Rust C++ PyTorch CoreML MLX ONNX HuggingFace Metal NeMo

Pinned Loading

  1. vllm-project/vllm vllm-project/vllm Public

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Python 96.2k 24.3k

  2. sgl-project/sglang sgl-project/sglang Public

    SGLang is a high-performance serving framework for large language models and multimodal models.

    Python 38.4k 10.1k

  3. ai-dynamo/dynamo ai-dynamo/dynamo Public

    A Datacenter Scale Distributed Inference Serving Framework

    Rust 8.4k 1.8k

  4. vllm-project/vllm-omni vllm-project/vllm-omni Public

    A framework for efficient model inference with omni-modality models

    Python 7.5k 2k

  5. vllm-project/semantic-router vllm-project/semantic-router Public

    An open, programmable decision layer for models and compute.

    Go 6.4k 1.1k

  6. FluidInference/FluidAudio FluidInference/FluidAudio Public

    Frontier CoreML audio models in your apps — text-to-speech, speech-to-text, voice activity detection, and speaker diarization. In Swift, powered by SOTA open source.

    Swift 3.3k 502