Projector-camera calibration with non-overlapping fields of view using a planar mirror

Implementation of the Springer Virtual Reality journal publication Projector-camera calibration with non-overlapping fields of view using a planar mirror: doi:10.1007/s10055-024-01089-7

Tested on Ubuntu 22.04 LTS, with a Realsense D455f camera and a Kodak LUMA 450 projector. We used a first-surface mirror, since this implies there is no need to take refraction indices into account. The current project supports only a single projector and camera.

Installation

Prerequisites

• OpenCV with contrib installation (tested with 4.11.0, installed)

If OpenCV is not installed globally, you can add -DOpenCV_DIR="<path_to_opencv_build_dir>" to the cmake command below. You also need to add the library folder to your library path in .bashrc: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<path_to_lib_dir>

git clone https://github.com/EDM-Research/ProcamCalib.git
cd ProcamCalib/src
mkdir build
cd build
cmake .. 
sudo make install

Usage

In ProcamCalib/ run:

python calibrate.py ./data/recordings/recording/S0_0 ./data/patterns/Asym_4_9 -m ./data/recordings/mirrorRecording/M_14_0

This will use the saved recordings to calibrate the camera, the mirror and the projector and saves the results under ./data/estimation

Live RealSense Capture

You can also capture calibration images directly from a connected RealSense camera:

python calibrate.py ./data/recordings/recording/S0_live ./data/patterns/Asym_4_9 -m ./data/recordings/mirrorRecording/M_live -p 3

• -p/--capturesPerPattern enables live capture mode for all calibration steps.
• --camid <serial> is optional. If omitted, the first connected RealSense device is used.

Camera Calibration Source

The calibrate.py script now supports three camera calibration sources via --camera-calibration-source:

• capture: run CamCalib first (default behavior).
• file: use an existing camera calibration JSON file.
• realsense: export factory intrinsics from a connected RealSense camera and use those for mirror/projector calibration.

Examples:

# 1) Capture camera calibration first (default)
python calibrate.py ./data/recordings/recording/S0_0 ./data/patterns/Asym_4_9 -m ./data/recordings/mirrorRecording/M_14_0 --camera-calibration-source capture -p 3

# 2) Use an existing camera calibration file
python calibrate.py ./data/recordings/recording/S0_0 ./data/patterns/Asym_4_9 -m ./data/recordings/mirrorRecording/M_14_0 --camera-calibration-source file --camera-calibration-file ./data/estimation/camCalib/S0_0.json -p 3

# 3) Use RealSense factory intrinsics
python calibrate.py ./data/recordings/recording/S0_0 ./data/patterns/Asym_4_9 -m ./data/recordings/mirrorRecording/M_14_0 --camera-calibration-source realsense -p 3 --camid <serial>

For --camera-calibration-source realsense, Python packages pyrealsense2, opencv-python and numpy are required.

Docker installation

To run the application with docker use the following two commands:

docker build -t procamcalib:latest .

docker run --rm -v ./data:/workspace/data procamcalib:latest   ./data/recordings/recording/S0_0   ./data/patterns/Asym_4_9   -m ./data/recordings/mirrorRecording/M_14_0

Citation

@article{projectorCameraCalibrationNonOverlappingFOV,
  author = {Vanherck, Joni and Jorissen, Lode and Zoomers, Brent and Michiels, Nick},
  title = {Projector-camera calibration with non-overlapping fields of view using a planar mirror},
  year = {2025},
  issue_date = {Dec 2024},
  publisher = {Springer-Verlag},
  address = {Berlin, Heidelberg},
  volume = {29},
  number = {1},
  issn = {1359-4338},
  url = {https://doi.org/10.1007/s10055-024-01089-7},
  doi = {10.1007/s10055-024-01089-7},
  journal = {Virtual Real.},
  month = jan,
  numpages = {13}
}

About

Developed by Hasselt University, Digital Future Lab in the scope of the Flanders Make funded project ALARMM SBO, FWO and funds on behalf of Hasselt University.