PredPreyGrassContinuous

About 240 decisions (1,900 physics steps) of add_walls without walls, with the policies trained in run T (speed actions, fixed energy cost), the healthiest ecosystem so far: predators hunt prey, and prey graze clusters of grass. The two shaded cones are the view of one predator (blue, 150°) and one prey (grey, 240°); both wrap across the arena's edges.

A continuous-space counterpart to the grid-based PredPreyGrass: multi-agent reinforcement learning for predators and prey, built on the physics-based Aquarium environment and trained with RLlib's new API stack (RLModule + Learner).

Layout

• dying_prey/ — Aquarium PPO training with permanent prey deaths. See its README for installation and evaluation.
• add_grass/ — a contained experiment adding stationary grass, food rewards, delayed regrowth, and nearest-visible-grass inputs for prey. See its README.
• add_energy/ — add_grass plus metabolic energy: decay, food from grass and catches, starvation for both species, and an own-energy input. See its README.
• add_reproduction/ — add_energy plus births at an energy threshold, populations that change during an episode, PredPreyGrass's starting populations, its sparse +10-per-birth reward, and grass that grows in clusters that move as the prey graze them. See its README, and its RESULTS for the log of all training runs and what they showed.
• add_walls/ — add_reproduction plus walls that block movement and sight, so animals can hide, actions at full speed, half speed or stopped with movement energy costs, and observations computed once per decision to reduce environment-stepping overhead. See its README, and its RESULTS for its training runs.
• add_group_hunting/ — add_walls (walls off) plus group hunting: a catch is a contest between the summed energy of nearby predators and the prey's energy, and is shared among the hunters; every observed animal shows its energy. See its README, and its RESULTS for its training runs.
• base_environment_continuous/ — a standalone environment that replaces Aquarium and add_walls's patches for the milestone world (no walls, the arms-race settings), reproducing add_walls exactly. See its README and SPEC.

Grass: from static patches to moving clusters

In every module, grass consists of patches that prey eat when they come close. An eaten patch disappears and regrows after a delay. The modules differ in where grass grows:

• add_grass, add_energy: patches are scattered at random and always regrow in exactly the same spot, so the food landscape stays fixed for a whole episode.

• add_reproduction: grass can grow in clusters that move, driven by the prey themselves through two mechanisms modeled on vegetation ecology:

  • Seed dispersal: an eaten patch regrows next to a surviving patch of its own cluster. Grass spreads where it survives and retreats where it is eaten, so a cluster creeps away from heavy grazing.
  • Overgrazing: a cluster grazed completely bare dies back and, after a delay, regrows in a different, unoccupied part of the arena.

  Together these create a shifting food landscape: prey must keep finding food, and predators can learn where prey gather. Each mechanism has its own switch, alongside scattered placement and random regrowth, so static and moving grass can be compared. The add_reproduction README describes the mechanisms step by step, with all settings.

Population dynamics within an episode

Exact counts over 8,002 physics steps in a new evaluation of the add_reproduction run 2026-10-07_09-38-10_103867, checkpoint 990, seed 0. Orange: predators, left axis. Blue: prey, right axis. The axes use different scales; line heights and crossings do not represent equal populations. Counts include the initial state and every subsequent physics step, with no averaging.

The populations show irregular predator–prey oscillations resembling a Lotka–Volterra pattern: prey can grow when food is available and predator numbers are low, while predator growth follows successful feeding. Near the end of this episode, prey peak around step 7,000 and predators around step 7,500. Grass availability, energy reserves, starvation and threshold-based reproduction modify the feedback; this single rollout does not establish a classical Lotka–Volterra cycle.

Prey turn over much faster than predators. The saved first evaluation episode for this checkpoint recorded 458 prey births and 449 catches, compared with 36 predator births and 35 predator starvation deaths. Prey feed directly on grass; predators receive 50% of a caught prey's energy and reproduce at an energy threshold of 12, versus 8 for prey. Energy reserves delay predator starvation after prey decline. Predator counts therefore change less often, even while individual predators are replaced. They still range from 2 to 12 in the plotted rollout, compared with 1 to 46 prey: a sixfold predator change, so their smaller absolute swings should not be mistaken for constant numbers.

Walls: places to hide

add_walls adds walls: obstacles that block both movement and sight, as in PredPreyGrass's walls-and-occlusion experiments. Animals slide along walls instead of passing through them. Anything behind a wall is hidden from view, so prey can hide and predators can ambush. Every agent senses nearby walls through a few distance inputs. Layouts include a "forest" of square blocks, four rooms connected by doorways, and custom rectangles. Occlusion uses a precomputed cell-to-cell line-of-sight table, so it costs a table lookup per check during an episode. The viewer clips the view cones of one predator and one prey at the walls, so you can watch what each animal can and cannot see. Walls are set with walls_layout (default "blocks"). See the add_walls README for how they work.

The arms race

With realistic movement costs, prey a little faster than their predators and grass spread over many small clusters, predators and prey keep improving against each other for about 2,000 iterations of training while staying evenly matched, the "Red Queen" pattern, before the race levels off. The result held in five independent runs. Details in the add_walls RESULTS (sections 22 to 25).

Tournament over runs AW-17 and AW-18: every predator checkpoint plays every prey checkpoint. Orange: the predators of a checkpoint against all prey (rising: better hunters). Blue: the prey of a checkpoint against all predators (falling: better at escaping). Green: each checkpoint against its own training partner (flat: still evenly matched). Made with add_walls/analysis/plot_arms_race.py.

All subprojects share one Conda environment at the repo root (.conda/). Training settings live in each module's config/config_env.py and config/config_ppo.py; run training without command-line arguments.