cpp-depth2normals

Depth → Surface Normals (C++)

A reproducible, geometry-based pipeline that converts floating-point depth maps into per-pixel surface normal maps, then evaluates them against ground-truth normals.

---

It includes:

• an interactive viewer for per-frame inspection (estimated/GT normals + angular error heatmap)

• a CLI for batch evaluation (dataset-level metrics)

Derivative operators currently supported: Sobel (default), Scharr, Prewitt.

---

Contents

• Quickstart (Windows prebuilt binary)
  • UI (interactive viewer)
  • CLI (batch evaluation)
• Method overview
• Evaluation
• Datasets
• Documentation
• Dependencies
• Roadmap
• License
• References

---

Quickstart (Windows prebuilt binary)

This first public release ships with a precompiled Windows executable (no CMake for now).

• Download the latest Windows archive from GitHub Releases

---

UI (interactive viewer)

The viewer is intended for qualitative inspection and debugging. It displays:

• depth map
• ground-truth normal map
• estimated normal map
• angular error heatmap (+ per-frame stats)

Controls:

Left / Right arrows : change frame
Up / Down arrows    : change dataset

CLI (batch evaluation)

Supported options:

-h
    Show usage instructions.

-dataset <file>.zip
    Process a specific dataset archive (repeatable).

-dataset
    Process all datasets located in assets/datasets.

-filter <sobel|prewitt|scharr>
    Select derivative operator (default: sobel).

Examples:

# Process all datasets in assets/datasets with default filter (sobel)
depth2normals.exe -dataset

# Process one dataset with Scharr
depth2normals.exe -dataset assets/datasets/easy_torus.zip -filter scharr

Results are written to:

results/<dataset>_results_<filter>.txt

---

Method overview

1) Back-project depth to a 3D point map

Given pixel coordinates (u, v), depth Z, and intrinsics (fx, fy, cx, cy) (pinhole camera model):

X = (u - cx) * (Z / fx)
Y = (v - cy) * (Z / fy)
Z = Z

2) Estimate tangents via spatial derivatives, then normals

Let P(u, v) = (X(u,v), Y(u,v), Z(u,v)) be the reconstructed 3D point map.

tu = dP/du
tv = dP/dv
n  = normalize( tu × tv )

Spatial derivatives are approximated using Prewitt, Sobel, or Scharr kernels. Invalid / non-finite depth values are masked out consistently across the pipeline.

---

Evaluation

Normals are compared against ground-truth normals using unsigned angular error:

theta = arccos( clamp( abs(ne · ngt), -1, 1 ) )

Reported statistics (per dataset and per filter) include:

• mean / median angular error
• standard deviation
• percentage of pixels below fixed thresholds (5°, 10°, 20°)

---

Datasets

This project uses synthetic depth sequences that provide:

• floating-point depth maps
• camera intrinsics
• ground-truth normal maps

Full dataset collection (3F2N):

• https://sites.google.com/view/3f2n/datasets

For convenience, this repository may include a temporally subsampled subset of the original sequences to keep the repo lightweight and evaluations fast.

---

Documentation

Extended documentation (full tables and discussion) can be found HERE.

---

Dependencies

• OpenCV (image I/O, matrices, general CV utilities)
• raylib + raygui (interactive viewer rendering/UI)
• miniz (ZIP dataset handling)

In the Windows precompiled release, runtime dependencies are distributed alongside the executable.

---

Roadmap

• CMake build + cross-platform support (planned)
• Discontinuity-aware normal estimation (to be explored)
  (investigating simple, heuristic approaches to handle depth jumps during derivative estimation)

---

License

See LICENSE.md

---

References

• Rui Fan et al., Three-Filters-to-Normal: An Accurate and Ultrafast Surface Normal Estimator, 2021.

This project provides an independent implementation inspired by the referenced paper.