Crowdsource human preferences, benchmark models, and build open datasets in any language.
Try the arena · Datasets · Deploy your own · Roadmap · Contribute
compar:IA is a blind arena for large language models. You type a prompt, two anonymous models reply, and you vote for the answer you prefer. Only after voting do you see which models wrote them.
The platform is used in a few ways:
- Teaching people about model diversity, bias, sovereignty, and the energy cost of generative AI.
- Ranking models on real use instead of English-first benchmarks, so you can see which ones work best in your own language or in your sector or even company.
- Building open datasets of prompts, votes, and reactions that anyone can reuse to study how people use these tools, or to train and align models.
Most models are weaker outside English. compar:IA is a way to measure that gap in public and collect the preference data needed to narrow it.
- Oct 2024: The French government launches comparia.beta.gouv.fr, an LLM arena open to the public.
- Mar 2025: 50,000 votes reached, and the first dataset goes up on Hugging Face.
- Nov 2025: The first public leaderboard ships, and a second instance goes live: AI-arenaen in Denmark.
- Today: Over 700,000 conversations and 250,000 votes (about 89% in French), several datasets, and new instances starting up regularly.
compar:IA is now built to be run by anyone: a company, a sector, or a language community.
| Instance | Region | Live | Datasets |
|---|---|---|---|
| compar:IA, French Government | 🇫🇷 | comparia.beta.gouv.fr | Hugging Face, data.gouv.fr, Mozilla |
| AI-arenaen, Denmark | 🇩🇰 | ai-arenaen.dk | coming soon |
| Yours? | 🌍 | Deploy one | your own |
Run the whole platform yourself — your own models, your own language, your own datasets and leaderboard.
Self-host with Docker (single server, automatic HTTPS via Caddy): see devops/standalone_docker_install/DOCKER_INSTALL.md.
Local development:
cp .env.example .env # configure your environment variables
make install # install all dependencies
source .env
make dev-backend # backend -> http://localhost:8008
make dev-frontend # frontend -> http://localhost:5173For the full setup guide (instances, KeePass, Docker, testing, models, i18n, architecture), see CONTRIBUTING.md.
In progress
- Authentication (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Style control, #532 (🇫🇷 Ministry of Culture)
- Prompt moderation, #542 (🇫🇷 Ministry of Culture)
- Improved model cards (🇫🇷 Ministry of Culture)
- Live use-case mapping (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Message history (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Socio-demographic data collection (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Back-office management (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
Done
- New voting system (🇪🇺 ALT-EDIC, 🇫🇷 Ministry of Culture)
- Web search (🇪🇺 ALT-EDIC)
- Separation of all platforms into separate instances (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Ranking consolidation and internationalization (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Language / platform-specific model support (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Gradio to FastAPI migration (🇫🇷 Ministry of Culture, 🇫🇷 DINUM, 🇪🇺 ALT-EDIC)
- EcoLogits update (🇪🇺 ALT-EDIC, 🇫🇷 DINUM)
- Dataset publishing pipeline v1 (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- Leaderboard v1 (🇫🇷 DINUM, 🇫🇷 Ministry of Culture, with 🇫🇷 PEReN)
- Archived models (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- Blog section (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- Internationalization foundations (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
- compar:IA v1 (🇫🇷 DINUM, 🇫🇷 Ministry of Culture)
compar:IA is a digital common. You can help with money, code, translations, or ideas.
- Financially. compar:IA is funded by DINUM and the French Ministry of Culture, with European support from ALT-EDIC. We are looking for new partners and funders to keep the infrastructure running, add languages, and keep the project independent. Reach us at contact@comparia.beta.gouv.fr.
- In code. The whole platform is open source and we welcome contributions of any size: bug fixes, features, translations, documentation. Open a pull request.
- In discussions. Share ideas, flag issues, or ask questions on GitHub Discussions.
- Any other way. Partnerships, academic work, media coverage, or spreading the word. Get in touch.
DINUM, French Ministry of Culture, and ALT-EDIC 🇪🇺, with AI-arenaen (Denmark), PIX, PEReN, and other contributors.
Read the paper: compar:IA: The French Government's LLM arena
A digital common. Reuse it, improve it, or run your own.