I've been using evo to evaluate FusionCore (a ROS 2 UKF for IMU + wheel odometry + GPS fusion) against robot_localization on the NCLT dataset: 6 outdoor sequences with GPS, IMU, and wheel encoders.
evo made it straightforward to compare APE/RPE across sequences and produce consistent plots from rosbag output. The benchmark configs and reproduce instructions are in the repo:
https://github.com/manankharwar/fusioncore/tree/main/fusioncore_datasets
Summary across 6 NCLT sequences: FusionCore wins 5, loses 1 (the sequence with extended GPS outage where dead-reckoning accumulated drift).
Two questions:
- Is there a 'showcase' or 'used by' section where real-world evaluations like this could be referenced? I think a concrete IMU + wheel odom + GPS comparison might be useful for users figuring out how to structure multi-sensor trajectory evaluation with evo.
- Any recommendations for handling the GPS-anchored reference trajectory comparison? Currently converting the GPS ground truth to ENU and aligning with
evo_ape... wondering if there's a cleaner approach for GPS-referenced benchmarks.
Happy to write up the evaluation methodology as a wiki example if that would be useful.
I've been using evo to evaluate FusionCore (a ROS 2 UKF for IMU + wheel odometry + GPS fusion) against robot_localization on the NCLT dataset: 6 outdoor sequences with GPS, IMU, and wheel encoders.
evo made it straightforward to compare APE/RPE across sequences and produce consistent plots from rosbag output. The benchmark configs and reproduce instructions are in the repo:
https://github.com/manankharwar/fusioncore/tree/main/fusioncore_datasets
Summary across 6 NCLT sequences: FusionCore wins 5, loses 1 (the sequence with extended GPS outage where dead-reckoning accumulated drift).
Two questions:
evo_ape... wondering if there's a cleaner approach for GPS-referenced benchmarks.Happy to write up the evaluation methodology as a wiki example if that would be useful.