Research engineer working on multi-agent reinforcement learning, with a background spanning systems programming, applied ML, robotics, and hardware.
I didn't set out to build a multi-agent RL testbed from scratch — I got there by using the existing tools first. An earlier project benchmarked three MARL frameworks (Tianshou, RLlib, MARLlib) against PettingZoo's predator-prey environments and ran tabular Q-learning by hand before reaching for deep RL. That work is the direct ancestor of the project I maintain now: when the existing frameworks made it hard to isolate what was actually causing a result, I built an environment where every layer — dynamics, perception, incentives, learning — is separated by construction, and reproducibility is enforced rather than assumed.
Before MARL, that same instinct for understanding systems from the inside shows up across a Nav2 planner comparison for a TurtleBot3, a from-scratch VGA-timing FPGA game, an analytic raytracer, a lip-to-speech synthesis pipeline wiring together three pretrained models, and a shell implementation with real fork/pipe/signal handling.
Predator-Prey Archetype Gridworld Environment — a deterministic, modular multi-agent RL testbed built for controlled experimentation and teaching. Six learning baselines (IQL, CQL, MixedTrainer, DQN with Double/Dueling variants, Actor-Critic, A2C), a pluggable observation/reward/action architecture with an immutable core, and a companion research paper studying how embodiment constraints reshape multi-agent coordination.
| Project | What it is | Stack |
|---|---|---|
| PPAGE | Deterministic modular MARL testbed, six baselines, companion paper | Python, PyTorch |
| ROS2 Comparative Planner Analysis | Nav2 global/local planner comparison on a TurtleBot3 | ROS 2, Python |
| Chrome T-Rex on FPGA | VGA-timing game driven by EMG input, built from scratch in HDL | Verilog, FPGA |
| MARL framework benchmark | Tianshou / RLlib / MARLlib against PettingZoo, plus hand-rolled tabular Q-learning | Python |
| Lip-to-speech pipeline | Three pretrained models wired into one inference path | Python, PyTorch |
| Analytic raytracer | Closed-form intersection renderer, no external graphics libs | C++ |
| Unix shell | Real fork/exec, pipes, redirection, signal handling | C |
Live counters, so these update themselves as the work gets picked up.
Open an issue on a project you find here if something catches your interest.