MAVNet

A modern PyTorch reimplementation of MAVNet, fixing the architectural and methodological issues found in the original 2018 paper, and validated end-to-end on a public driving dataset.

MAVNet (arXiv:1809.00396, "Learning to Navigate Autonomously in Outdoor Environments", kept locally at paper/1809.00396v1.pdf) was an imitation-learning approach for camera-based robot/drone navigation. Revisiting it years later surfaced real problems: three inconsistent architectures across the paper/code/README, an output head that didn't match the paper's own description, a headline "tomographic reconstruction" preprocessing step that was dead code, and a shuffle-before-split bug that leaked test data into training. This repo is a clean rewrite that fixes each of those, with tests and a public-data pipeline to prove it.

This project makes no drone/UAV hardware claims. There is no flight hardware, drone-collected data, or trained model available anymore, so the rebuilt pipeline is validated on a public driving dataset (SullyChen's driving_dataset, a common Udacity behavioral-cloning stand-in) as an honest substitute for the original sensor data — same core imitation-learning problem (predict a driving/steering action from a camera frame), same architecture and pipeline, different domain.

What's fixed vs. the original paper/code

Issue in the original	Fix here
Three inconsistent architectures (Inception-v3 / LRCN / AlexNet) across paper, README, and code	One canonical architecture per run (MAVNetCNN, single-frame), with MAVNetLRCN as an explicit opt-in temporal variant
Output head mismatch: paper describes a 5-value multi-hot vector, code trains a single 7-way softmax that conflates action and scene labels	Two independent heads: softmax movement_head (forward/yaw_left/yaw_right/halt) + sigmoid junction_head
"Tomographic reconstruction" (Radon transform) was the paper's headline contribution but was commented-out dead code	A real, working Radon transform + filtered back-projection step (preprocessing.py), toggled with --preprocessing raw|radon, so it can actually be A/B tested
shuffle(train_data) before slicing train/test — data leakage across near-duplicate adjacent frames	Route-based splitting (route_based_split, route_based_split_three_way) that splits by whole route/segment, never by individual shuffled frames
Regression metrics (EVA/RMSE) mixed into what is fundamentally a classification task	Classification-only metrics (metrics.py): accuracy/precision/recall/F1 for both heads
No validation split; test set implicitly used for tuning	Proper train/val/test split; val checked every epoch, test evaluated exactly once at the end

Repo layout

pytorch_mavnet/
├── model.py             # MAVNetCNN, MAVNetLRCN architectures
├── preprocessing.py     # raw vs. Radon transform / filtered back-projection
├── dataset.py           # dummy dataset + route-based split helpers
├── udacity_dataset.py   # real SullyChen driving_dataset loader
├── metrics.py           # classification metrics for both heads
├── train.py             # CLI training loop
├── tests/                # full test suite
├── paper/1809.00396v1.pdf  # original paper, kept for reference
└── requirements.txt

Quickstart

git clone https://github.com/<your-org>/MAVNet.git
cd MAVNet
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# smoke test on dummy tensors
python train.py --architecture cnn --preprocessing raw

Smoke tests for individual modules:

python model.py            # shape smoke test
python preprocessing.py    # raw vs radon smoke test
python dataset.py          # dummy dataset + route split smoke test
python metrics.py          # metrics smoke test
python train.py --architecture cnn --preprocessing radon # full loop, dummy data

Since the dummy dataset uses random labels, accuracy stays near chance — that's expected; it only proves the plumbing works.

Tests

PYTHONPATH="" python -m pytest tests/ --cov=. --cov-report=term-missing

(PYTHONPATH="" avoids a conflicting global ROS pytest11 plugin on machines with a ROS install on the default Python path.) 66 tests, 96% coverage.

Dataset

This repo ships with a real driving dataset at 07012018/ (SullyChen-style dashcam frames + steering angles): 07012018/data/*.jpg + 07012018/data.txt (filename.jpg angle,timestamp, 63,825 frames). A symlink 07012018/data/data.txt -> ../data.txt is included so the loader (which expects data.txt alongside the images) can read it directly from --data-root 07012018/data.

If you're starting from a fresh clone of the SullyChen driving-datasets release instead, place it anywhere and pass that path as --data-root (just make sure data.txt sits next to the images, symlink it in if not).

Two honest limitations vs. the original paper, handled explicitly rather than faked:

• No "halt" label exists for a moving car — movement labels only ever land in {forward, yaw_left, yaw_right}.
• No junction label exists (that came from Cityscapes originally) — the dataset is flagged has_junction_labels = False, and train.py skips the junction loss/metric entirely rather than training on a fabricated constant-zero target.

Routes are chunked into contiguous blocks (--route-chunk-size, default 500 frames) and split three ways — train/val/test — via route_based_split_three_way, so no near-duplicate adjacent frames leak across splits (fixing the shuffle-then-slice bug from the original code), and the test set is only ever evaluated once, at the very end of training, never used for tuning.

Training / validation / test

train.py runs train + val every epoch, then evaluates the held-out test split exactly once at the end and saves a checkpoint. There's no separate eval/inference script — evaluation is built into the training run.

Smoke test (tiny footprint, confirms the pipeline runs, ~4GB VRAM or less, a few minutes on a small GPU):

python train.py --dataset udacity --data-root 07012018/data \
    --architecture cnn --preprocessing raw \
    --epochs 1 --batch-size 1 --num-workers 2 --seed 0 \
    --checkpoint /tmp/smoke_checkpoint.pt

Full training run (bigger GPU — A5000 / RTX 4070 — larger batch size, more epochs, all quick-win improvements enabled):

python train.py --dataset udacity --data-root 07012018/data \
    --architecture cnn --preprocessing raw \
    --epochs 30 --batch-size 64 --lr 1e-3 --num-workers 8 \
    --class-weighting --augment --seed 0 \
    --checkpoint mavnet_checkpoint.pt

Useful variants:

# Radon-preprocessing ablation (paper's original headline claim)
python train.py --dataset udacity --data-root 07012018/data \
    --architecture cnn --preprocessing radon \
    --epochs 30 --batch-size 64 --num-workers 8 \
    --class-weighting --augment --seed 0 \
    --checkpoint mavnet_radon_checkpoint.pt

# Different train/val/test split sizes or route chunking
python train.py --dataset udacity --data-root 07012018/data \
    --val-fraction 0.15 --test-fraction 0.15 --route-chunk-size 300 \
    --epochs 30 --batch-size 64 --num-workers 8 --seed 0

All CLI flags: --architecture {cnn,lrcn}, --preprocessing {raw,radon}, --dataset {dummy,udacity}, --epochs, --batch-size, --lr, --val-fraction, --test-fraction, --seed, --num-workers, --class-weighting, --augment, --data-root, --angle-threshold-deg, --route-chunk-size, --checkpoint.

Verified end-to-end on GPU: 1 epoch on the full dataset reaches ~56-63% train / ~51-54% val/test movement accuracy (vs. 33% chance baseline for 3 classes), on a genuinely held-out, non-overlapping set of route chunks.

Further improvements added (quick wins)

• Reproducibility: set_seed() + --seed seeds python/numpy/torch (CPU + CUDA) at the start of main().
• DataLoader performance: --num-workers (default 4) and pin_memory=True (when CUDA is available) on all three loaders.
• Train/val/test three-way split: route_based_split_three_way() in dataset.py. Val is evaluated every epoch; test is evaluated exactly once, after training finishes, so it can't be (even accidentally) tuned against.
• Class-weighted loss: --class-weighting computes inverse-frequency weights from the training split (compute_class_weights() in train.py) and passes them to CrossEntropyLoss, addressing the yaw_left under-representation (~26% vs ~32%/42%).
• Horizontal-flip augmentation: --augment (Udacity dataset only, train split only) flips the image and swaps yaw_left/yaw_right, cheaply doubling effective data and further balancing left/right turns.

Not yet done

• MAVNetLRCN needs a sequence-shaped dataset (batch, timesteps, C, H, W); the current datasets only yield single frames. Wire this up if/when the LRCN variant is actually compared.
• Confusion matrix (movement head) instead of aggregate accuracy alone.
• Multi-seed ablation harness for raw-vs-radon / cnn-vs-lrcn comparisons.
• Steering angle as an auxiliary regression head (multi-task).
• Checkpointing on best val loss / early stopping (currently only the final-epoch checkpoint is saved).

Contributing

Contributions are welcome — see CONTRIBUTING.md. This project follows the Contributor Covenant Code of Conduct.

Citation

If you use this software or the original paper, see CITATION.cff.

License

Apache License 2.0 — see LICENSE.