This library is based on JSBSim software (https://github.com/JSBSim-Team/jsbsim). This library's primary purpose is to allow users to explore Beyond Visual Range (BVR) tactics using Reinforcement learning while utilizing JSBSim high-fidelity flight dynamics models.
The following libraries are required to run BVRGym. The code has been tested with Python 3.11
pip install gymnasium jsbsim pymap3d pandas py_trees stable_baselines3 tensorboard
BVRGym has a 3-level structure. Level 1 is the lowest level where Python interacts with the JSBSim flight dynamics simulator. Examples of Level 1 objects are located in the BVRGYM/jsb_gym/simObjects directory. Here we have both the aircraft class and the missile class. Level 2 represents agents, a wrapper that has both missiles and aircraft objects, i.e., they use level 1 objects. And level 3 is the environments that use agents. Each level has its own configuration files.
To test these objects in an isolated scenario, install Flightgear (https://www.flightgear.org/), place yourself in the BVRGYM directory, and then run commands as explained below
FlightGear offers an excellent tool for visualizing the units present in BVRGym.
sudo add-apt-repository ppa:saiarcot895/flightgear
sudo apt update
More details on https://launchpad.net/~saiarcot895/+archive/ubuntu/flightgear
After installing FlightGear, start the program, go to the Aircraft tab, and download the General Dynamics F16 aircraft.
To have a missile model as well, copy the ogel folder from this repo (BVRGYM/fg/ogel) to the Flightgears aircraft directory. To visualize a missile in FlightGear, I used the ogel (1) model within FGFS, replaced the graphical representations of the ogel with a missile available in FGFS (I think it was from (2)), and added a trail from Santa Claus (3) to see the trajectory.
Given that you have both the f16 and ogel directories in the fgfs directory (home/.fgfs/Aircraft/...), you will be able to visualize both units in the next steps
terminal 1 (Visualize F16):
fgfs --fdm=null --native-fdm=socket,in,60,,5550,udp --aircraft=f16-block-52 --airport=ESSA --multiplay=out,10,127.0.0.1,5000 --multiplay=in,10,127.0.0.1,5001
aircraft.mp4
terminal 2:
python -m tests.test_aircraft
test_aircraft.mp4
terminal 1:
fgfs --fdm=null --native-fdm=socket,in,60,,5550,udp --aircraft=f16-block-52 --airport=ESSA --multiplay=out,10,127.0.0.1,5000 --multiplay=in,10,127.0.0.1,5001
terminal 2:
fgfs --fdm=null --native-fdm=socket,in,60,,5551,udp --aircraft=ogel --airport=ESSA --multiplay=out,10,127.0.0.1,5001 --multiplay=in,10,127.0.0.1,5000
terminal 3:
python -m tests.test_missile
test_missile.mp4
Check that all the ammo is loaded.
python -m tests.test_agent
RL agent takes random actions, while the red aircraft is controlled by a behavior tree.
python -m tests.test_tacview
Open tacview (https://www.tacview.net/) and load data in BVRGym/data_output/tacview
test_tacview.mp4
To start training an agent for BVR air combat, run the following command
python main.py
Additional details can be found in the following article
BVR Gym: A Reinforcement Learning Environment for Beyond-Visual-Range Air Combat
https://arxiv.org/abs/2403.17533
Note that this article relates to an older version of the BVRGym, but it is still valid with remarks to understand Behavior Trees and other part of the environments.
Even though this library focuses on BVR (beyond-visual-range) air combat, WVR (within-visual-range) combat remains a critical aspect of aerial warfare, and pilots continue to train extensively for it. One of my favorite articles in this field is https://arxiv.org/pdf/2105.00990, where the authors applied hierarchical reinforcement learning (HRL) to WVR air combat scenarios.
In this section, I will reimplement parts of that work; however, the results will not match the performance reported in the original study. The goal here is to keep the implementation as lightweight and simple as possible while still capturing the core ideas.
python -m tests.test_wvr
Uncomment for example train_WvrControlZone() in the main file and run
python main.py