Race NVIDIA NIM models. Watch the leaderboard reorder live.
A real-time streaming benchmarker for NVIDIA NIM LLMs — pick models, type a prompt, and see token throughput, latency, and rank update as responses stream in.
| 🚀 Live leaderboard | Ranked by tokens/sec, reordering while results stream in |
| 🎛 Parallel benchmarking | Benchmark N models at once — or sequential, clean per-model timings |
| 🗂 All NIM models fetched live | Grouped by provider (meta, nvidia, deepseek, qwen, …), searchable |
| 📊 Deep stats | tok/s, TTFT, p50/p95 latency, run counts — per model |
| 💾 Persisted history | Runs auto-save to your browser; compare any two models on charts |
| 📤 Export | One click to CSV or JSON |
| 🔐 Key stays local | API key lives in localStorage, sent only to your own origin |
npm install
npm run devOpen http://localhost:3000, paste your NVIDIA NIM API key (nvapi-…), pick a
provider, pick models, type a prompt — and Run.
No NIM key? Fetch one free at build.nvidia.com.
Every load opens with a small piece of motion: the NVIDIA mark glides
left, NIMbench pops out from its edge, holds a beat, then tucks back in —
and the mark returns to center before the app fades in. Built with
framer-motion's useAnimate as a strict async sequence (transform/opacity only,
prefers-reduced-motion respected).
- Streaming SSE — responses are parsed token-by-token; the board updates live.
- Worker pool —
parallelism(how many models at once) ×concurrency(requests per model). Setparallelism > 1to race models in parallel. - Anti-hang guards — a 10s idle timeout and a 30s hard deadline guarantee no single request (or keepalive-only stream) can hang the run forever.
- Retry with backoff —
429/5xxretry up to 3× with exponential backoff; auth4xxfail fast.
| Knob | Default | Meaning |
|---|---|---|
parallelism |
1 |
Models benchmarked at once (1 = clean sequential timings) |
iterations |
3 |
Times each prompt is re-sent per model (the sample size) |
concurrency |
2 |
Parallel requests per model |
parallelism > 1disablesiterations— parallel racing is about speed, sequential repetition is about statistical confidence. They don't mix.
Every completed run auto-saves to browser history (capped at 100). The Data tab aggregates them into a persistent leaderboard, filters by provider, and lets you select any two models to compare — token throughput, TTFT, and p50/p95 latency on Recharts panels. Click a row to open its full run history.
The gear menu gives you:
- Refresh models — re-run NIM model discovery
- Remember API key — toggle key persistence in this browser
- Delete stored API key / Delete data & reset settings
NVIDIA's NIM API (integrate.api.nvidia.com) sends no CORS headers, so
browsers can't call it directly. NIMbench ships a same-origin proxy route
(app/api/nim/[...path]/route.ts) that forwards to NIM. Your API key is sent by
the browser per-request and never stored server-side.
The proxy only forwards /v1/models and /v1/chat/completions to the fixed NIM
host — deny-by-default.
app/
page.tsx the whole experience (bench + data tabs)
layout.tsx shell, fonts, metadata
api/nim/[...path] server-side proxy → integrate.api.nvidia.com
components/
StartupAnimation choreographed splash (useAnimate)
ProviderPicker searchable provider dropdown (shared, bench + data)
Leaderboard live ranked board + running/error chips
DataLeaderboard persisted, aggregated board + compare select
ComparePanel Recharts side-by-side charts
ModelDetail per-model run history overlay
SmoothScroll Lenis smooth scrolling
lib/
benchmark.ts SSE runner + worker pool + retry/backoff
sse.ts minimal SSE parser with anti-hang timeouts
metrics.ts per-model aggregates (tok/s, TTFT, p50/p95)
aggregate.ts across-run aggregation for the Data tab
models.ts NIM model catalog + live discovery
config.ts persisted settings + API key (localStorage)
history.ts run history (capped at 100)
export.ts CSV / JSON download
log.ts [area]-prefixed debug logger (localStorage.log)
Next.js 16 (App Router, Turbopack) · React 19 · HeroUI v3 · Tailwind CSS v4 · framer-motion (Motion) · Lenis · Recharts · simple-icons · @gravity-ui/icons · TypeScript
- The API key is stored in
localStorageand sent only as theAuthorizationheader to the app's own proxy route, which forwards to your configured NIM base URL (https://integrate.api.nvidia.com/v1by default). It is never persisted server-side. - The proxy forwards only
/v1/modelsand/v1/chat/completionsto the fixed NIM host — nothing else.
npm install
vercel # or: npx vercel --prodThe API route deploys with the app — no extra services, no configuration.
Made with ⚡ and NVIDIA green #76B900.