Terra

This repository contains the code for Terra: Hierarchical Terrain-Aware 3D Scene Graph for Task-Agnostic Outdoor Mapping

Table of Contents

• Terra
• Table of Contents
• Paper
• Setup
  • Requirements
  • Using Docker Container
• Datasets and Metric Mapping
• Building Terra
• Task-Execution with Terra

Paper

Preprint Citation:

C. R. Samuelson, A. Austin, S. Knoop, B. Romrell, G. R. Slade, T. W. McLain, and J. G. Mangelson, “Terra: Hierarchical Terrain-Aware 3D Scene Graph for Task-Agnostic Outdoor Mapping,” Sept 2025. [Online]. Available: https://arxiv.org/abs/2509.19579

Accepted to ICRA 2026. Citation Pending.

Setup

Requirements

• Docker
• NVIDIA GPU

Our repository is designed for ROS 2 Humble. To handle dependency issues, our repository is built in Docker using the following container.

Using Docker Container

First pull our Terra docker container

docker pull frostlab/ros2_terra:latest
xhost +local:docker

Start the container in your terminal

docker run --rm -it --gpus all --net host -v /tmp/.X11-unix:/tmp/.X11-unix -v <path/to/terra_repo>:/docker_ros2_ws/src/terra -v <ros2_bags_folder>:/<ros2_bags_folder> -e DISPLAY=$DISPLAY frostlab/ros2_terra

Build and source the ros2 repository

tmux

cd docker_ros2_ws

colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release

source install/setup.bash

Datasets and Metric Mapping

The ros launch code uses LIO-SAM to build the metric map, extracts and saves relevant data in the dataset folder structure below.

Dataset folder structure

my_dataset
├── global_pc
|   ├── global_pc_{timestamp}.npy
|   ├── ...
├── camera1_images
|   ├── cam1_img_{timestamp}.jpg
|   ├── ...
├── lidar_pc
|   ├── lidar_pc_{timestamp}.npy
|   ├── ...
├── transformations_lidar2cam1
|   ├── transform_lidar_to_cam1_{timestamp}.npy
|   ├── ...
├── transformations_lidar2global
|   ├── transform_lidar_to_map_{timestamp}.npy
|   └── ...

Simulation: Business Campus Dataset

• Download the dataset: Dense 1/8 (3.30 GB), Sparse 1/3 (1.66 GB), and Sparse Full (13.34 GB).
  • These links enable downloading the extracted metric data from each rosbag collected in HoloOcean.
    • For more information about the Business Campus World in HoloOcean see here.
    • For more information on installing this world in HoloOcean see here.
    • For more information on runing ros2 in HoloOcean see here.
  • Can extract subfolders into a single dataset folder with: unzip holoocean_<dataset-name>.zip -d holoocean_<dataset-name>
  • Make sure the location of this downloaded metric data has been volumed into the docker container.
• Download the simulated YOLO model here.
  • After unzipping, the model is located in: holo_3cls_nano_stepsz25_256imgsz_500epochs/weights/best.pt.
  • Identify your container name in a terminal with: docker ps
  • Copy the model into the container with:

    docker cp <yolo_folder_location>/holo_3cls_nano_stepsz25_256imgsz_500epochs/weights/best.pt <container_name>:/holoocean_3cls.pt
    

• Given this metric data, proceed to Building Terra.

Real-World: South Campus Dataset

• Download the dataset (in progress)
• Put the rosbag in the volumed ros2_bags_folder so you can access it in the docker container
• Copy the contents of the config/south_campus/liosam_params.yaml file to replace the contents in the params.yaml file in the LIO-SAM/config folder
• Update the config/south_campus/params.yaml to match the correct rosbag_path and save_folder.
• Now to build the metric point cloud map with LIO-SAM and save the data into our folder structure, run

ros2 launch terra_ros build_metric_map_south_campus.launch.py

Custom Datasets

If you have a ROS 2 bag of your Ouster OS1-128 LiDAR and RGB Camera data, then do the following to run LIO-SAM:

• Edit the params.yaml file in the LIO-SAM/config folder to match your LiDAR and IMU ros topics as well as the extrinsic transformation between the two.
• Copy the contents config/rviz2.rviz file to replace the rviz2.rviz file in the LIO-SAM/config folder.
• Update the params.yaml ros parameters in terra_ros/config for your dataset and rosbag
• Now to build the metric point cloud map with LIO-SAM and save the data into our folder structure, run

ros2 launch terra_ros build_metric_map_multicam_rate.launch.py params_file:=<path_to_updated_params_file>.yaml

Building Terra

Change into the Terra repo in container with: cd /docker_ros2_ws/src/terra

Building Metric-Semantic Map (MS Map)

Provided that you have all the data saved in the file structure shown above, you can build the metric-semantic map (ms map) as follows:

• Update the terra/config/<dataset_name>/msmap.yaml arguments to match the saved data folder path and YOLO terrain model location.
  • We provide different msmap yaml files showing capabilities of handling more than 1 camera image data as well as using YOLOE for datasets where you don't have a trained YOLO terrain model.
• Run the MS Map code with the correct yaml filepath as an argument as follows:

python3 -m terra.ms_map --params=terra/config/<dataset_name>/msmap.yaml

Note: This will take a while to process at about 1 second per lidar-image pair.

To visualize the resulting MS Map, we have provided a helper script where you just need to pass in the filepath to your saved data folder. Each different semantic CLIP id will have a different color. For example:

python3 -m terra.visualize_msmap --data_folder=/data/folder --output_folder=/directory/to/ms_map/output --num_terrain=3 --pt_size=2.0

Building Region and Place Layers from MS Map

With the results saved from MS Map, you can build the terrain-aware places and region layers of the 3DSG as follows:

• Update the terra/config/<dataset_name>/build_terra.yaml arguments to match the saved MS Map files location and other relevant arguments.
• Run Build Terra code with your updated yaml file as an argument as follows:

python3 -m terra.build_terra --params=/path/to/build_terra.yaml

To visualize the resulting 3D Scene Graph, run:

python3 -m terra.visualize_terra --terra_3dsg=/path/to/saved/terra_nxgraph.pkl --global_pc=/folder/path/to/global_pc/ --num_terrains=num_terrains --view_json=view.json

• The arguments are defined as:
  • terra_3dsg: Path to the saved 3DSG.
    • Note: build_terra.py saves a terra_nxgraph.pkl and a Terra.pkl file. The first is just the nx.Graph 3DSG object and the latter is our Terra 3DSG class. This is asking for the first one.
  • global_pc: Path to the folder that contains all of the global point clouds saved from the metric mapping step.
  • num_terrains: Integer of the number of terrains used by the YOLO model
  • view_json: (Optional) Saved json file to tell Open3D to display the 3DSG at a certain angle and distance.

Task-Execution with Terra

Object Retrieval Tasks

To perform object retrieval tasks with the Terra 3DSG saved from Building Terra, is done as follows:

• Modify the object_retrieval.yaml file for as many object retrieval tasks of interest as well as other parameters described below.
• Run the object retrieval task as follows:

python3 -m terra.object_retrieval_task --params=/path/to/object_retrieval.yaml

• YAML parameters are defined as:
  • terra: Path to the saved Terra 3DSG.
    • Note: build_terra.py saves a terra_3dsg.pkl and a Terra.pkl file. The first is just the nx.Graph 3DSG object and the latter is our Terra 3DSG class. This is asking for the latter.
  • object_tasks: YAML list of object tasks
  • prediction_method: Pass a string of the method to use from the following [ms_avg, ms_max, 3dsg]. These methods are explained in detail in the paper. (Default: ms_avg)
  • alpha: Threshold to determine whether an object is task relevant (i.e. if its cosine-similarity score is above alpha then it is task-relevant). (Default: 0.23)

Region Monitoring Tasks

To perform region monitoring tasks with the Terra 3DSG saved from Building Terra, is done as follows:

• Modify the region_querying.yaml file for as many region monitoring tasks of interest as well as other parameters described below.
• Run the region monitoring task as follows:

python3 -m terra.region_querying_task --params=/path/to/region_querying.yaml

• YAML parameters are defined as:
  • terra: Path to the saved Terra 3DSG.
    • Note: build_terra.py saves a terra_3dsg.pkl and a Terra.pkl file. The first is just the nx.Graph 3DSG object and the latter is our Terra 3DSG class. This is asking for the latter.
  • region_tasks: YAML list of region monitoring tasks
  • prediction_method: Pass a string of the method to use from the following [max, thresh, mix, aib]. These methods are explained in detail in the paper. (Default: max)
  • alpha: Threshold to determine whether a region is task relevant (i.e. if its cosine-similarity score is above alpha then it is task-relevant). Only used for methods [thresh, mix, aib]. (Default: 0.23)
  • k: Parameter for selecting the top-K task relevant regions. Only used for methods [max, mix]. (Default: 1)

Terrain-Aware Path Planning to Destination Query

To perform path planning to a destination query, do the following:

• Modify the path_panning.yaml file based on your destination query and the terrain preferences
• Run the path planning task as follows:

python3 -m terra.path_planning_task --params=/path/to/path_planning.yaml

• YAML parameters are defined as;
  • terra: Path to the saved Terra 3DSG.
    • Note: build_terra.py saves a terra_3dsg.pkl and a Terra.pkl file. The first is just the nx.Graph 3DSG object and the latter is our Terra 3DSG class. This is asking for the latter.
  • terrain_preferences: List the terrain indices that are preferred and forbidden based on the terrain input and ordering used in build_terra.yaml. The penalties are defined by each terrain index and an associated weight.
  • queries:
    • destination: String explaining the destination query
    • start: (Optional) string explaining start location. If not used, default's to start being the first place node in the graph
  • prediction_method: Pass a string of the method to use from the following [ms_avg, ms_max, 3dsg_avg, 3dsg_max, ms_medoid, ms_trim]. These methods are explained in detail in the paper. (Default: ms_avg)
  • alpha: Threshold to determine whether an object is task relevant (i.e. if its cosine-similarity score is above alpha then it is task-relevant). (Default: 0.23)