COMRAD

COMRAD is a cooperative multi-agent reinforcement learning benchmark built on ViZDoom for first-person visual coordination. This repository contains the benchmark runtime assets, COMRAD-specific environment and model code, a modified Sample Factory training stack, paper artifacts, and the scripts used to produce figures, tables, and diagnostic media.

Repository map

Path	Purpose
comrad/	Benchmark package: training entrypoints, environment registration, wrappers, models, shipped scenario assets, launchers, tests.
sample_factory/	Modified training engine used by COMRAD.
scripts/	Figure-generation, result aggregation, WAD rendering, and experiment helper scripts.
results/	Committed figures, tables, result summaries, and selected media artifacts.
DoomGen/	Procedural generator used to build and vary scenario WADs.

Directory-level guides are provided in:

• comrad/README.md
• sample_factory/README.md
• scripts/README.md
• results/README.md

Setup

COMRAD is packaged through pyproject.toml and is intended to be run with uv.

uv sync --all-extras
uv run pre-commit install

The project requires Python 3.10 or newer. The package metadata and CI workflow are the authoritative sources for dependency resolution.

Single-run training

The main training entrypoint is comrad.train.

uv run python -m comrad.train \
  --env=stag_hunt_arena \
  --algo=IPPO \
  --train_for_env_steps=10000 \
  --num_workers=4 \
  --num_envs_per_worker=4 \
  --device=cpu

Training outputs are written under train_dir/<experiment>/. The main runtime log is:

tail -n 120 train_dir/<experiment>/sf_log.txt

For launcher-owned benchmark sweeps, use comrad/train_all.py instead of manually reproducing those long commands.

Recording and evaluation

High-resolution checkpoint recording:

uv run python -m comrad.record_video \
  --env=stag_hunt_arena \
  --train_dir=train_dir \
  --experiment=<experiment_name> \
  --load_checkpoint_kind=best \
  --resolution=1280x720

Automap/top-down heatmap rendering:

uv run python -m comrad.record_topdown_heatmap \
  --env=ammo_carrier \
  --train_dir=<train_root> \
  --experiment=<experiment_name> \
  --output_dir=results/videos/ammo_carrier_navigation_heatmap \
  --device=cpu \
  --overwrite

Both commands expect a full experiment directory with at least config.json and the corresponding checkpoint directory.

Benchmark launchers

COMRAD ships several paper-facing launcher profiles through comrad/train_all.py.

• benchmark: full benchmark suite
• agent_scaling: Armory Siege agent-count sweep
• platform_chain_curriculum: curriculum comparison on Platform Chain
• qmix_lr_sensitivity_pilot: Short sensitivity study
• qmix_lr_sensitivity_full: Full sensitivity study

Run a launcher profile locally with:

COMRAD_TRAIN_ALL_PROFILE=benchmark \
uv run python -m sample_factory.launcher.run \
  --run=comrad.train_all \
  --backend=processes \
  --max_parallel=1 \
  --pause_between=1

The exact benchmark definitions, algorithms, root directory names, and profile constants live at the top of comrad/train_all.py.

Scenario assets and DoomGen

Runtime scenario assets are stored in comrad/scenarios/ as .cfg and .wad pairs, plus pre-generated batch directories such as batch_platform_chain_curriculum/.

Generation workflows are in DoomGen/. The usual split is:

• DoomGen/examples/benchmark: procedural generators and geometry logic
• comrad/scenarios/: runtime WADs and configs consumed by training and evaluation

If a scenario generator changes, rebuild the corresponding WAD in DoomGen/ and copy the regenerated artifact back into comrad/scenarios/ before training or evaluation.

Using DoomGen as a Python library

DoomGen is declared as an optional dependency in the uv workspace. It is not installed by a plain uv sync:

uv sync --extra doomgen      # install doomgen + its deps
# or
uv run --extra doomgen python -m comrad.train ...   # one-off with doomgen available
# or
uv pip install -e DoomGen/   # editable install directly from local checkout

After any of these, import doomgen works from the root Python environment. Without --extra doomgen, the core COMRAD package remains self-contained and DoomGen is not pulled in.

Tests

Run COMRAD and Sample Factory tests with uv:

uv run pytest comrad/tests -v
uv run pytest sample_factory/tests -v

Tests in comrad/tests/ focus on benchmark logic, launchers, and recorder behavior. Tests in sample_factory/tests/ cover the modified training stack.

Outputs and artifacts

• train_dir/: experiment directories, configs, checkpoints, logs
• results/: committed figures, tables, result summaries, selected media
• wandb/: local W&B run state and uploads

See results/README.md for the structure of the committed artifacts.

License and upstream assets

The repository is released under the MIT license in LICENSE. Upstream and third-party assets are documented in the repository itself, including the bundled scripts/wad2image/ tool and the paper's citations to ViZDoom, Sample Factory, HARL, PyMARL, and QPLEX.

Acknowledgements

We would like to thank all researchers, engineers, advisors involved in this work; and the TU/e Supercomputing Center for providing the infrastructure to run the experiments. The environments are built on top of ViZDoom. Our infrastructure is built on top of Sample Factory. Our experiments were managed using Weights & Biases. For on-policy actor-critic methods, we implement IPPO and MAPPO from the original implementation in Sample Factory, and HAPPO following the HARL codebase. For off-policy value decomposition methods, we implement IDQN, VDN, QMIX following the PyMARL framework, and QPLEX following official QPLEX reference implementation.