A curated list of reinforcement learning environments - from classic control to the LLM-agent training environments powering today's frontier models.
Environments are the new bottleneck. As RL has moved from games to language agents, the limiting factor is no longer the algorithm but the environment: a sandbox that gives an agent a task, lets it act, and returns a reward. This list collects the best of them - battle-tested classics, modern embodied simulators, and the new wave of web, coding, and tool-use environments built specifically for training and evaluating LLM agents.
Contributions welcome - see Contributing.
- Standards & Infrastructure
- General-Purpose Frameworks
- LLM & Agent Environments
- Embodied AI & Robotics
- Games & Arcade
- Multi-Agent
- Board, Card & Strategy Games
- Autonomous Driving
- Domain-Specific
- JAX-Accelerated & High-Throughput
- Building Your Own
- Related Awesome Lists
- Contributing
- License
The APIs and runtimes everything else builds on.
- Gymnasium - The maintained fork of OpenAI Gym and the de-facto standard single-agent RL API.
- PettingZoo - The Gymnasium of multi-agent RL; a standard API for environments with many interacting agents.
- OpenEnv - An open standard and hub from Meta/PyTorch for agentic environments, designed for LLM-agent RL at scale.
- EnvPool - A C++-based, highly parallel environment executor; runs Atari/MuJoCo thousands of steps faster than vanilla Python.
- Shimmy - Compatibility layer that wraps legacy and third-party environments (DM Control, OpenSpiel, etc.) into the Gymnasium API.
- Minari - A standard format and hosting for offline-RL / imitation datasets collected from environments.
- DeepMind Control Suite (dm_control) - Continuous-control benchmark tasks built on MuJoCo; a longtime standard for control research.
- Unity ML-Agents - Build environments in the Unity game engine and train agents with built-in RL algorithms.
- Griddly - A high-performance engine for building grid-world games with rich observation and rendering options.
- MiniGrid - Lightweight, fast grid-world environments for goal-directed and instruction-following research.
- Gym-Retro / Stable-Retro - Turns classic console games into RL environments via emulators.
The fast-moving frontier: environments designed to train and evaluate language-model agents that browse, code, use tools, and operate computers.
- Meta Agents Research Environments (ARE) - A platform for building and running complex, tool-rich environments to study LLM agents.
- AgentGym - A framework spanning many interactive environments for evolving and training generally-capable LLM agents.
- AgentBench - A multi-environment benchmark evaluating LLMs as agents across OS, DB, web, and games.
- BALROG - A benchmark of long-horizon game environments (NetHack, MiniHack, BabyAI, Crafter) for agentic LLM/VLM reasoning.
- WebArena - A realistic, self-hostable web environment (shopping, forums, CMS, GitLab) for autonomous web agents.
- VisualWebArena - Extends WebArena with visually-grounded tasks that require understanding page screenshots.
- BrowserGym - A Gym-style environment for web agents, unifying WebArena, MiniWoB, WorkArena and more behind one API.
- WebShop - A simulated e-commerce site with 1M+ products for training instruction-following shopping agents.
- Mind2Web - A dataset/environment for generalist web agents acting across hundreds of real websites.
- WorkArena - Enterprise knowledge-work tasks on the ServiceNow platform, for agents doing real office workflows.
- OSWorld - A real computer environment (Ubuntu/Windows/macOS) benchmarking agents on open-ended desktop tasks across apps.
- AndroidWorld - A live Android environment with 100+ hand-built tasks across 20 apps for mobile-device agents.
- AndroidEnv - DeepMind's RL environment exposing the Android OS as a touchscreen control problem.
- AppWorld - A simulated world of 9 apps and 457 APIs for benchmarking interactive coding/tool agents.
- Terminal-Bench - A benchmark + environment for agents that accomplish real tasks in a command-line terminal.
- SWE-bench - Resolve real GitHub issues in real repositories; the standard benchmark for software-engineering agents.
- SWE-Gym - The first training environment for SWE agents, with executable repos and reward via the test suite.
- debug-gym - A text environment from Microsoft for agents that interactively debug code using tools like a Python debugger.
- MLE-bench - OpenAI's benchmark of 75 Kaggle competitions measuring how well agents do end-to-end ML engineering.
- τ-bench (tau-bench) - Evaluates agents on tool-use and policy-following in dynamic, user-in-the-loop customer-service domains.
- τ²-bench (tau2-bench) - A dual-control successor where both the agent and a simulated user can act on the environment.
- ToolBench - A large-scale environment/dataset for training and evaluating tool-use over 16k+ real APIs.
- Gorilla / APIBench - Connect LLMs to thousands of APIs; a reference environment for function-calling research.
- Verifiers - A library for writing RL environments for LLMs with verifiable rewards; popular for GRPO-style training.
- TextArena - 100+ competitive and cooperative text games for training and evaluating LLMs through self-play.
- Reasoning Gym - Procedurally-generated reasoning tasks with verifiable answers, built for RL with verifiable rewards.
- TextWorld - Microsoft's engine for generating text-adventure games as language-grounded RL environments.
- ALFWorld - Aligns TextWorld tasks with embodied ALFRED tasks, letting agents learn abstractly then act.
- ScienceWorld - A text environment testing whether agents can perform grade-school science experiments and reasoning.
- MuJoCo - DeepMind's fast, accurate physics engine underpinning much of continuous-control RL.
- Isaac Lab - NVIDIA's GPU-accelerated framework for robot learning, supporting thousands of parallel simulated environments.
- Habitat - A high-performance simulator for embodied agents navigating photorealistic 3D indoor scenes.
- ManiSkill - GPU-parallelized robotic manipulation environments with thousands of objects.
- robosuite - A modular MuJoCo-based simulation framework for robot manipulation benchmarks.
- Meta-World - 50 robotic manipulation tasks designed for meta-learning and multi-task RL.
- Gymnasium-Robotics - A collection of robotics environments (Fetch, Shadow Hand, maze) under the Gymnasium API.
- Arcade Learning Environment (ALE) - The Atari 2600 benchmark that launched deep RL; still a standard for general competence.
- Procgen Benchmark - 16 procedurally-generated arcade games for measuring generalization, not memorization.
- NetHack Learning Environment (NLE) - The famously hard roguelike NetHack as a fast RL environment for long-horizon exploration.
- MiniHack - A sandbox built on NLE for designing custom, controllable roguelike RL tasks.
- Crafter - An open-world survival game that evaluates a broad spectrum of agent abilities in one benchmark.
- Craftax - A JAX reimplementation of Crafter that runs orders of magnitude faster for open-ended RL.
- MineDojo - A massively multitask Minecraft environment with thousands of tasks and an internet-scale knowledge base.
- MineRL - Minecraft environments paired with large human-demonstration datasets for imitation + RL.
- Voyager - An open-ended, LLM-powered lifelong-learning agent for Minecraft; a reference for agent-in-environment design.
- SMACv2 - The StarCraft Multi-Agent Challenge; the standard cooperative MARL micromanagement benchmark.
- Melting Pot - DeepMind's suite for evaluating multi-agent generalization to novel social situations.
- Google Research Football - A physics-based 11-vs-11 football environment for cooperative and competitive RL.
- Overcooked-AI - A cooperative cooking game widely used to study human-AI coordination.
- Neural MMO - A massively-multi-agent environment simulating many agents competing for resources in a persistent world.
- VMAS - A vectorized 2D physics simulator for fast multi-robot / swarm MARL.
- OpenSpiel - DeepMind's collection of 70+ games and algorithms for research in RL and search/planning.
- PySC2 - DeepMind's StarCraft II Learning Environment, exposing the full game as an RL challenge.
- RLCard - A toolkit of card games (poker, blackjack, mahjong, UNO) for RL with imperfect information.
- Pgx - Vectorized, JAX-native board games (chess, Go, shogi, backgammon) for high-throughput self-play.
- CARLA - An open-source simulator for autonomous-driving research with sensors, maps, and traffic.
- MetaDrive - A lightweight, procedurally-generated driving simulator for generalizable RL.
- highway-env - Minimalist 2D autonomous-driving and tactical-decision environments under the Gymnasium API.
- FinRL - A framework of market environments for reinforcement learning in quantitative trading.
- Flatland - A train-scheduling environment for multi-agent path-finding on railway networks.
- Gym-ANM - Environments for RL control of electricity distribution / active network management.
- CityLearn - A benchmark for RL on building energy coordination and demand response.
End-to-end GPU/TPU environments that remove the CPU-simulation bottleneck.
- Brax - A differentiable, massively-parallel rigid-body physics engine for continuous control in JAX.
- Gymnax - JAX reimplementations of classic-control, bsuite, and MinAtar environments for fully on-accelerator RL.
- Jumanji - A suite of scalable, JAX-based combinatorial and routing environments from InstaDeep.
- JaxMARL - Multi-agent environments in JAX (incl. SMAX, Overcooked, MPE) for orders-of-magnitude faster MARL.
- PGX - JAX-native board games for high-throughput self-play (also listed above).
- Gymnasium - Creating environments - The official guide to wrapping any task in the standard RL API.
- PettingZoo - Environment creation - How to author a custom multi-agent environment.
- Verifiers - environment guide - Patterns for writing verifiable-reward environments for LLM RL.
- OpenEnv - A spec for packaging agentic environments so they are portable across training stacks.
- Awesome Reinforcement Learning - Broad RL resources: papers, courses, and libraries.
- Awesome Deep RL - Deep-RL frameworks, algorithms, and applications.
- Awesome LLM-Powered Agent - Papers and resources on LLM agents (complements the environments here).
Contributions are very welcome! Please read CONTRIBUTING.md first. In short: one environment per entry, keep descriptions to a single neutral sentence, link to the canonical source, and add it to the most fitting section (alphabetical-ish within a section). Suggestions, corrections, and new categories are all appreciated - open an issue or a PR.
To the extent possible under law, the contributors have waived all copyright and related or neighboring rights to this work. See LICENSE.