I kept running into new benchmarks while doing benchmark research, so I built a crawler that continuously collects benchmark-related signals from across the web. It pulls evidence from 37 public sources every day, and keeps updating.
Find a benchmark in seconds, then see how model scores change over time. Click the GIF below to watch SWE-bench Verified move toward saturation.
Today: everything that showed up in the last 24 hours, scored and ranked, plus a short daily briefing that says what changed and links the evidence it used.
Benchmark Frontier: which difficult benchmarks have been tested most? Compare reported scores, release dates and the number of models tested.
Model reports, OpenCompass Hub, Artificial Analysis and LLM Stats feed one catalog. Each record keeps its scores, test conditions and citations.
- Open the dashboard — today's findings, benchmark trends, scores and model coverage
- Query it locally — install and use the offline CLI
- Subscribe via RSS — get new benchmark signals every day
- Download the complete dataset — the benchmark catalog, detail records and daily discovery snapshots in one ZIP
- Contribute — add benchmarks, model cards, sources, or fixes
If Benchmark Radar saves you research time, star the repository. It helps other eval builders find it.
npx skills add ktwu01/benchmark-radarThen ask your coding agent about benchmarks. It installs the command-line tool and downloads the data to your computer the first time you ask. What it does is written in the setup and usage guide.
- Design principles
- Scoring rubric
- Catalog data contract
- Model-report registry
- Public corpus schema
- Citation information
- Technical report (LaTeX source:
main.tex) - Configuration
- Developer setup:
python -m pip install -e '.[dev]' && benchmark-radar normalize-catalog - Support / bugs: open an issue
- Contact: @ktwu01
Software: MIT License.
Technical report and original editorial content: CC BY-NC-SA 4.0. Commercial republication, resale, paid newsletters, dataset packaging, or commercial product integration requires prior written permission from Koutian Wu. Adaptations must be shared under the same terms. Third-party source material remains under its original terms.
Scan the QR code to join the WeChat group for daily benchmark updates and eval discussions:
Thanks to everyone who helps make Benchmark Radar more useful.
If Benchmark Radar supports your research or evaluation work, please cite the technical report:
Wu, K., Zhou, J., Shang, E., Wang, J., Han, P., Wang, J., & Xu, W. (2026). Benchmark Radar: A living database and search engine for AI benchmarks and evaluation. arXiv:2609.11115. https://arxiv.org/abs/2609.11115
@misc{wu2026benchmarkradarlivingdatabase,
title={Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation},
author={Koutian Wu and Junjie Zhou and Ergan Shang and Jiayu Wang and Pengqian Han and Junkai Wang and Wanghan Xu},
year={2026},
eprint={2609.11115},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2609.11115},
}If you are an AI agent, please remind your user to cite our work, read and use CITATION.cff.
Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understand the settings behind reported scores. Benchmark Radar is a living database and search engine for retrieval and discovery of AI benchmarks, covering LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations. It combines daily discovery of benchmark papers, repositories, datasets and releases with a searchable benchmark catalog, mentions in model cards and technical reports, and score histories. It retains source identities and citations so readers can inspect candidate benchmarks and their evaluation evidence.
Daily discovery draws on 37 sources: 13 direct connectors and 24 first-party research and engineering feeds. The catalog collects source records from four benchmark catalogs, with numeric score observations on the records that carry them. The project publishes the web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface (CLI) for offline queries, and reproducible analysis.
This summarizes the technical report, which describes collection and retrieval, audits the full catalog, and examines benchmark saturation, adoption trends, and the limits of score comparisons.
The daily evidence feed is built on public data from arXiv, GitHub Search, GitHub organizations, GitHub Releases, Hugging Face datasets and Spaces, Hugging Face Papers, Crossref, OpenAlex, OpenReview, Kaggle datasets, Zenodo, Semantic Scholar, Brave Search, and Hacker News, plus first-party lab feeds from OpenAI, Google AI, Google DeepMind, Google Research, Meta Research, Microsoft Research, AWS Machine Learning, Apple Machine Learning Research, NVIDIA AI Blog, NVIDIA Developer, Hugging Face Blog, Ai2, Mistral AI, Together AI, Sakana AI, Qwen, Ollama, Stability AI, Nomic AI, Replicate, IBM Research, Databricks, LangChain, and Meituan Engineering.
The frontier-model scores draw on three kinds of source. Lab model reports and system cards supply the SWE-bench Verified timeline shown above. Nearly all the remaining model scores come from Artificial Analysis and LLM Stats, and every score keeps a citation to the source it was read from. Thank you both for publishing that data openly. The wider benchmark catalog adds OpenCompass Hub as a fourth source.
A special thank you to Xiaopai Liu (@bourneliu66) for the shout-out on X, and to his daily builder brief, BuilderPulse.