Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions .github/workflows/docs-update.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
name: Notify central docs of update

on:
push:
paths:
- 'docs/**'
push:
paths:
- "docs/**"

jobs:
notify-docs-repo:
runs-on: ubuntu-latest
steps:
- name: Trigger docs site rebuild
run: |
curl -X POST \
-H "Authorization: Bearer ${{ secrets.DOCS_DEPLOY_TOKEN }}" \
-H "Accept: application/vnd.github+json" \
https://api.github.com/repos/vrui-vr/docs/actions/workflows/build-and-deploy.yml/dispatches \
-d '{"ref":"main"}'
notify-docs-repo:
runs-on: ubuntu-latest
steps:
- name: Trigger docs site rebuild
run: |
curl -X POST \
-H "Authorization: Bearer ${{ secrets.DOCS_DEPLOY_TOKEN }}" \
-H "Accept: application/vnd.github+json" \
https://api.github.com/repos/vrui-vr/vrui-vr.github.io/actions/workflows/build-and-deploy.yml/dispatches \
-d '{"ref":"main"}'
16 changes: 15 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
@@ -1 +1,15 @@
# VRUI Kinect package
# Kinect

This application includes the software necessary to connect an unmodified, off-the-shelf, Kinect device to a regular computer, and use it as a 3D camera for a variety of 3D graphics and virtual reality applications.

The software is composed of several classes wrapping aspects of the underlying libusb library into an exception-safe `C++` framework, classes encapsulating control of the Kinect's tilt motor and color and depth cameras, and a class encapsulating the operations necessary to reproject a combined depth and color video stream into 3D space. It also contains several utility applications, including a simple calibration utility.

This software is based on the reverse engineering work of Hector Martin Cantero (marcan42 on twitter and YouTube). The Kinect package doesn't use any of his code, but the "magic incantations" that need to be sent to the Kinect to enable the cameras and start streaming. Without owning an XBox, those incantations were essential to learn its USB protocol. Thanks Hector!

The Kinect driver code and the 3D reconstruction code are entirely written from scratch in `C++`, using the [Vrui VR toolkit](https://vrui-vr.github.io/vrui) for 3D rendering management and interaction.

The Kinect is an accessory for Microsoft's Xbox game console, containing an array of microphones, an active-sensing depth camera using structured light, and a color camera. The Kinect is intended to be used as a controller-free game controller, tracking the body or bodies of one or more players in its field of view.

This software converts the Kinect into a 3D camera by combining the depth and color image streams received from the device, and projecting them back out into 3D space in such a way that real 3D objects inside the cameras' field of view are recreated virtually, at their proper sizes (see the below video).

<iframe width="560" height="315" src="https://www.youtube.com/embed/VJ9Ih-49O3o?si=rlILsY4rUtTTDAbG" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
121 changes: 73 additions & 48 deletions docs/installation/guide.md
Original file line number Diff line number Diff line change
@@ -1,83 +1,108 @@
!!! tip
It is recommended to download or move the source packages for Vrui and the Kinect 3D Video Capture Project into a `src` directory underneath the user's home directory. Otherwise, references to `~/src` in the following instructions need to be changed.

<!-- todo add collab infrastructure to docs and link here -->
<!-- ## Step 1: Install Vrui's collaboration infrastructure (optional)

<!-- todo add collab infrastructure to docs and link here -->
More info about the collaboration infrastructure and the installation guide can be found at [github.com/vrui-vr/collaboration/blob/main/README](https://github.com/vrui-vr/collaboration/blob/main/README). -->

## Step 1: Unpack the Kinect 3D Video Capture
As stated on Kinect's [Getting Started](./index.md) page, you must have the Vrui VR Toolkit installed first in order to follow these instructions to install Vrui's application, Kinect. Check out [these instructions](../../../vrui/installation) to install Vrui.

### Option 1: Downloading and unpacking a zip file from GitHub

On [the Kinect repository's main page](https://github.com/vrui-vr/kinect), click on the green "<> Code" button, and then click on "Download ZIP" in the menu that pops up in response.
## Option 1: Installing with PullPackage (easy)

![Downloading a ZIP from a GitHub repo](download_zip.png)
??? info "Heads up!"
If you haven't used DataLab's package manager, PullPackage, to install either Vrui or one of its applications before, installing PullPackage will allow you to follow simpler installation steps for all following applications, as you will see below. <!-- "DataLab at UC Davis" ? unsure abt branding/monikers -->

Depending on your browser settings, you may be asked where to store the file being downloaded, or it might be stored in a default location, such as your `Downloads` directory. Take note of what the zip file is called and where it is stored.
--8<--
pullpackage.md
pullpackage_kinect.md
--8<--

Assuming that you already created the `src` directory *according to Vrui's installation instructions*, enter the following line into a terminal window once the file is completely downloaded:
You're done!

```sh
cd ~/src
```
## Option 2: Manual install (advanced)

Then enter into the same terminal window:
### Step 1: Unpack the Kinect 3D Video Capture

```sh
unzip <path to downloaded zip file>
```
The Kinect code repository can be downloaded either by:

Replace `<path to downloaded zip file>` with the full path to the zip file, for example `~/Downloads/kinect-main.zip`.
1. downloading the zip file and unpacking it **OR**
2. cloning the repository with `git clone`

Finally, check for the name of your new Kinect directory by entering:
!!! warning
If you are unfamiliar with git and/or GitHub, you should probably go the zip file route.

```sh
ls
```
=== "Option 1: Downloading and unpacking a zip file from GitHub"

which will list all files in the `src` directory, which should include a new directory called `kinect-main`. Take note of this name, and then enter into that directory by typing this command into the terminal window:
On [the Kinect repository's main page](https://github.com/vrui-vr/kinect), click on the green "<> Code" button, and then click on "Download ZIP" in the menu that pops up in response.

```sh
cd <Kinect directory>
```
<!-- todo fix this: ![Downloading a ZIP from a GitHub repo](download_zip.png) -->

where you replace `<Kinect directory>` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`.
Depending on your browser settings, you may be asked where to store the file being downloaded, or it might be stored in a default location, such as your `Downloads` directory. Take note of what the zip file is called and where it is stored.

### Option 2: Clone the repository from GitHub
Assuming that you already created the `src` directory *according to Vrui's installation instructions*, enter the following line into a terminal window once the file is completely downloaded:

Assuming that you already created the `src` directory *according to Vrui's installation instructions*, navigate to the `src` directory on your computer in the terminal window.
```sh
cd ~/src
```

```sh
cd ~/src
```
Then enter into the same terminal window:

Then, clone the repository from GitHub:
```sh
unzip <path to downloaded zip file>
```

```sh
git clone https://github.com/vrui-vr/kinect.git
```
Replace `<path to downloaded zip file>` with the full path to the zip file, for example `~/Downloads/kinect-main.zip`.

Finally, check for the name of your new Kinect directory by entering:
Finally, check for the name of your new Kinect directory by entering:

```sh
ls
```
```sh
ls
```

which will list all files in the `src` directory, which should include a new directory called `kinect`. Take note of this name, and then enter into that directory by typing this command into the terminal window:
which will list all files in the `src` directory, which should include a new directory called `kinect-main`. Take note of this name, and then enter into that directory by typing this command into the terminal window:

```sh
cd <Kinect directory>
```
```sh
cd <Kinect directory>
```

where you replace `<Kinect directory>` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`.
where you replace `<Kinect directory>` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`.

!!! warning
If your installed Vrui version is not 13.0, or Vrui's installation directory was changed from the default of `/usr/local`, adapt the makefile using a text editor.
=== "Option 2: Clone the repository from GitHub"

Assuming that you already created the `src` directory *according to Vrui's installation instructions*, navigate to the `src` directory on your computer in the terminal window.

```sh
cd ~/src
```

Then, clone the repository from GitHub:

```sh
git clone https://github.com/vrui-vr/kinect.git
```

Finally, check for the name of your new Kinect directory by entering:

```sh
ls
```

which will list all files in the `src` directory, which should include a new directory called `kinect`. Take note of this name, and then enter into that directory by typing this command into the terminal window:

```sh
cd <Kinect directory>
```

where you replace `<Kinect directory>` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`.

!!! warning
If your installed Vrui version is not 13.0, or Vrui's installation directory was changed from the default of `/usr/local`, adapt the makefile using a text editor.

Change the value of `VRUI_MAKEDIR` close to the beginning of the file as follows: `VRUI_MAKEDIR := <Vrui install dir>/share/make`, where <Vrui install dir> is the installation directory chosen when you installed Vrui. Use `$(HOME)` to refer to the user's home directory instead of `~`.
Change the value of `VRUI_MAKEDIR` close to the beginning of the file as follows: `VRUI_MAKEDIR := <Vrui install dir>/share/make`, where <Vrui install dir> is the installation directory chosen when you installed Vrui. Use `$(HOME)` to refer to the user's home directory instead of `~`.

## Step 2: Build the Kinect 3D Video Capture Project
### Step 2: Build the Kinect 3D Video Capture Project

Run the following from inside the `<Kinect directory>`:

Expand All @@ -93,7 +118,7 @@ If everything went well the first time, the second run will print:
make: Nothing to be done for 'all'.
```

## Step 3: Install the Kinect 3D Video Capture Project
### Step 3: Install the Kinect 3D Video Capture Project

If Vrui was installed in `/usr/local` or elsewhere outside the user's home directory:

Expand All @@ -109,7 +134,7 @@ If Vrui was installed inside the user's home directory:
make install
```

## Step 4: Install a udev rule file to give access to all users (optional, Linux-only)
### Step 4: Install a udev rule file to give access to all users (optional, Linux-only)

By default, Kinect devices can only be accessed by the root user. This is inconvenient and a security risk, as all Kinect applications must be run as root.

Expand Down
6 changes: 0 additions & 6 deletions docs/installation/index.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,5 @@
# Getting started with the Kinect package

!!! info "Heads up!"
VRUI must be installed before the Kinect package can be installed.

1. Install VRUI [(go to instructions $\rightarrow$)](https://vrui-vr.github.io/docs/vrui/installation/)
2. Install Kinect package!!!

## Prerequisites

This software requires the Vrui VR toolkit, version 13.0 build 001 or newer. It also requires `libusb` version 1.0, and that Vrui was configured with support for `libusb` prior to its installation. To properly work with multiple Kinect-for-Xbox version 1473 and/or Kinect-for-Windows devices, the `libusb` library needs to provide USB bus topology query functions such as those provided by the `libusbx` fork. This software can optionally create Kinect 3D video streaming plug-ins for Vrui's collaboration infrastructure (version 2.9 or newer). The presence of Vrui's collaboration infrastructure will be detected automatically during the build process.
15 changes: 15 additions & 0 deletions docs/nav.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# docs/kinect/nav.toml
nav = [
{ "Kinect" = [
"index.md",
{ "Getting started" = [
"installation/index.md",
{ "Installation guide" = "installation/guide.md" },
] },
{ "Usage" = [
"usage/index.md",
{ "Recording 3D Movies" = "usage/recording_3d_movies.md" },
{ "Merging 3D Facades from Multiple Kinects" = "usage/merging_3d_facades.md" },
] },
] },
]
9 changes: 0 additions & 9 deletions docs/nav.yml

This file was deleted.

46 changes: 40 additions & 6 deletions docs/usage/index.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,47 @@
# Running the Kinect package

## Step 1. Run ./bin/KinectUtil list to list all Kinect devices connected to the host computer.
## Step 1: Find all connected Kinect devices

## Step 2a. Run sudo ./bin/KinectUtil getCalib <camera index> to download factory calibration data from the <camera index>-th Kinect device on the host's USB bus (with the first device having index 0) and create a intrinsic camera calibration file for that device in the Kinect package's configuration file directory.
Run

> **OR**
```sh
./bin/KinectUtil list
```

## Step 2b. Run ./bin/RawKinectViewer <camera index> to intrinsically calibrate the <camera index>-th Kinect device on the host's USB bus (with the first device having index 0), using a semi-transparent checkerboard calibration target (see next section for details). This generates a binary IntrinsicParameters-<serial number>.dat file in <Vrui install dir>/etc/Kinect-<version>, where <serial number> is the unique factory-assigned serial number of the Kinect device as displayed in step 1.
to list all Kinect devices connected to the host computer.

Step 2 (a or b) only have to be performed once for each Kinect device, unless the resulting calibration proves unsatisfactory. Kinect devices typically retain their intrinsic calibration over time.
## Step 2: Camera calibration

## Step 3 Run ./bin/KinectViewer -c <camera index>, where <camera index> is the zero-based index of the Kinect device to use if there are multiple ones.
### Option 1: Factory calibration

Run

```sh
sudo ./bin/KinectUtil getCalib <camera index>
```

to download factory calibration data from the `<camera index>-`th Kinect device on the host's USB bus (with the first device having index 0) and create an intrinsic camera calibration file for that device in the Kinect package's configuration file directory.

### Option 2: Intrinsic calibration

Run

```sh
./bin/RawKinectViewer <camera index>
```

to intrinsically calibrate the `<camera index>-`th Kinect device on the host's USB bus (with the first device having index 0), using a semi-transparent checkerboard calibration target (see next section for details). This generates a binary `IntrinsicParameters-<serial number>.dat` file in <`Vrui install dir>/etc/Kinect-<version>`, where `<serial number>` is the unique factory-assigned serial number of the Kinect device as displayed in step 1.

!!! Note
Either of the above options only have to be performed once for each Kinect device, unless the resulting calibration proves unsatisfactory. Kinect devices typically retain their intrinsic calibration over time.

<!-- ? what does this step do -->
## Step 3

Run

```sh
./bin/KinectViewer -c <camera index>
```

where `<camera index>` is the zero-based index of the Kinect device to use if there are multiple ones.