diff --git a/.github/workflows/docs-update.yml b/.github/workflows/docs-update.yml index fef52a6..d2fc6eb 100644 --- a/.github/workflows/docs-update.yml +++ b/.github/workflows/docs-update.yml @@ -1,18 +1,18 @@ name: Notify central docs of update on: - push: - paths: - - 'docs/**' + push: + paths: + - "docs/**" jobs: - notify-docs-repo: - runs-on: ubuntu-latest - steps: - - name: Trigger docs site rebuild - run: | - curl -X POST \ - -H "Authorization: Bearer ${{ secrets.DOCS_DEPLOY_TOKEN }}" \ - -H "Accept: application/vnd.github+json" \ - https://api.github.com/repos/vrui-vr/docs/actions/workflows/build-and-deploy.yml/dispatches \ - -d '{"ref":"main"}' + notify-docs-repo: + runs-on: ubuntu-latest + steps: + - name: Trigger docs site rebuild + run: | + curl -X POST \ + -H "Authorization: Bearer ${{ secrets.DOCS_DEPLOY_TOKEN }}" \ + -H "Accept: application/vnd.github+json" \ + https://api.github.com/repos/vrui-vr/vrui-vr.github.io/actions/workflows/build-and-deploy.yml/dispatches \ + -d '{"ref":"main"}' diff --git a/docs/index.md b/docs/index.md index 0003f6f..e5e776c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1 +1,15 @@ -# VRUI Kinect package \ No newline at end of file +# Kinect + +This application includes the software necessary to connect an unmodified, off-the-shelf, Kinect device to a regular computer, and use it as a 3D camera for a variety of 3D graphics and virtual reality applications. + +The software is composed of several classes wrapping aspects of the underlying libusb library into an exception-safe `C++` framework, classes encapsulating control of the Kinect's tilt motor and color and depth cameras, and a class encapsulating the operations necessary to reproject a combined depth and color video stream into 3D space. It also contains several utility applications, including a simple calibration utility. + +This software is based on the reverse engineering work of Hector Martin Cantero (marcan42 on twitter and YouTube). The Kinect package doesn't use any of his code, but the "magic incantations" that need to be sent to the Kinect to enable the cameras and start streaming. Without owning an XBox, those incantations were essential to learn its USB protocol. Thanks Hector! + +The Kinect driver code and the 3D reconstruction code are entirely written from scratch in `C++`, using the [Vrui VR toolkit](https://vrui-vr.github.io/vrui) for 3D rendering management and interaction. + +The Kinect is an accessory for Microsoft's Xbox game console, containing an array of microphones, an active-sensing depth camera using structured light, and a color camera. The Kinect is intended to be used as a controller-free game controller, tracking the body or bodies of one or more players in its field of view. + +This software converts the Kinect into a 3D camera by combining the depth and color image streams received from the device, and projecting them back out into 3D space in such a way that real 3D objects inside the cameras' field of view are recreated virtually, at their proper sizes (see the below video). + + diff --git a/docs/installation/guide.md b/docs/installation/guide.md index 8433cff..f24e7f8 100644 --- a/docs/installation/guide.md +++ b/docs/installation/guide.md @@ -1,83 +1,108 @@ !!! tip It is recommended to download or move the source packages for Vrui and the Kinect 3D Video Capture Project into a `src` directory underneath the user's home directory. Otherwise, references to `~/src` in the following instructions need to be changed. + More info about the collaboration infrastructure and the installation guide can be found at [github.com/vrui-vr/collaboration/blob/main/README](https://github.com/vrui-vr/collaboration/blob/main/README). --> -## Step 1: Unpack the Kinect 3D Video Capture +As stated on Kinect's [Getting Started](./index.md) page, you must have the Vrui VR Toolkit installed first in order to follow these instructions to install Vrui's application, Kinect. Check out [these instructions](../../../vrui/installation) to install Vrui. -### Option 1: Downloading and unpacking a zip file from GitHub -On [the Kinect repository's main page](https://github.com/vrui-vr/kinect), click on the green "<> Code" button, and then click on "Download ZIP" in the menu that pops up in response. +## Option 1: Installing with PullPackage (easy) -![Downloading a ZIP from a GitHub repo](download_zip.png) +??? info "Heads up!" + If you haven't used DataLab's package manager, PullPackage, to install either Vrui or one of its applications before, installing PullPackage will allow you to follow simpler installation steps for all following applications, as you will see below. -Depending on your browser settings, you may be asked where to store the file being downloaded, or it might be stored in a default location, such as your `Downloads` directory. Take note of what the zip file is called and where it is stored. +--8<-- +pullpackage.md +pullpackage_kinect.md +--8<-- -Assuming that you already created the `src` directory *according to Vrui's installation instructions*, enter the following line into a terminal window once the file is completely downloaded: +You're done! -```sh -cd ~/src -``` +## Option 2: Manual install (advanced) -Then enter into the same terminal window: +### Step 1: Unpack the Kinect 3D Video Capture -```sh -unzip -``` +The Kinect code repository can be downloaded either by: -Replace `` with the full path to the zip file, for example `~/Downloads/kinect-main.zip`. +1. downloading the zip file and unpacking it **OR** +2. cloning the repository with `git clone` -Finally, check for the name of your new Kinect directory by entering: +!!! warning + If you are unfamiliar with git and/or GitHub, you should probably go the zip file route. -```sh -ls -``` +=== "Option 1: Downloading and unpacking a zip file from GitHub" -which will list all files in the `src` directory, which should include a new directory called `kinect-main`. Take note of this name, and then enter into that directory by typing this command into the terminal window: + On [the Kinect repository's main page](https://github.com/vrui-vr/kinect), click on the green "<> Code" button, and then click on "Download ZIP" in the menu that pops up in response. -```sh -cd -``` + -where you replace `` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`. + Depending on your browser settings, you may be asked where to store the file being downloaded, or it might be stored in a default location, such as your `Downloads` directory. Take note of what the zip file is called and where it is stored. -### Option 2: Clone the repository from GitHub + Assuming that you already created the `src` directory *according to Vrui's installation instructions*, enter the following line into a terminal window once the file is completely downloaded: -Assuming that you already created the `src` directory *according to Vrui's installation instructions*, navigate to the `src` directory on your computer in the terminal window. + ```sh + cd ~/src + ``` -```sh -cd ~/src -``` + Then enter into the same terminal window: -Then, clone the repository from GitHub: + ```sh + unzip + ``` -```sh -git clone https://github.com/vrui-vr/kinect.git -``` + Replace `` with the full path to the zip file, for example `~/Downloads/kinect-main.zip`. -Finally, check for the name of your new Kinect directory by entering: + Finally, check for the name of your new Kinect directory by entering: -```sh -ls -``` + ```sh + ls + ``` -which will list all files in the `src` directory, which should include a new directory called `kinect`. Take note of this name, and then enter into that directory by typing this command into the terminal window: + which will list all files in the `src` directory, which should include a new directory called `kinect-main`. Take note of this name, and then enter into that directory by typing this command into the terminal window: -```sh -cd -``` + ```sh + cd + ``` -where you replace `` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`. + where you replace `` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`. -!!! warning - If your installed Vrui version is not 13.0, or Vrui's installation directory was changed from the default of `/usr/local`, adapt the makefile using a text editor. +=== "Option 2: Clone the repository from GitHub" + + Assuming that you already created the `src` directory *according to Vrui's installation instructions*, navigate to the `src` directory on your computer in the terminal window. + + ```sh + cd ~/src + ``` + + Then, clone the repository from GitHub: + + ```sh + git clone https://github.com/vrui-vr/kinect.git + ``` + + Finally, check for the name of your new Kinect directory by entering: + + ```sh + ls + ``` + + which will list all files in the `src` directory, which should include a new directory called `kinect`. Take note of this name, and then enter into that directory by typing this command into the terminal window: + + ```sh + cd + ``` + + where you replace `` with the name of the directory where you cloned/unpacked the Kinect in the previous step, as printed by `ls`. + + !!! warning + If your installed Vrui version is not 13.0, or Vrui's installation directory was changed from the default of `/usr/local`, adapt the makefile using a text editor. - Change the value of `VRUI_MAKEDIR` close to the beginning of the file as follows: `VRUI_MAKEDIR := /share/make`, where is the installation directory chosen when you installed Vrui. Use `$(HOME)` to refer to the user's home directory instead of `~`. + Change the value of `VRUI_MAKEDIR` close to the beginning of the file as follows: `VRUI_MAKEDIR := /share/make`, where is the installation directory chosen when you installed Vrui. Use `$(HOME)` to refer to the user's home directory instead of `~`. -## Step 2: Build the Kinect 3D Video Capture Project +### Step 2: Build the Kinect 3D Video Capture Project Run the following from inside the ``: @@ -93,7 +118,7 @@ If everything went well the first time, the second run will print: make: Nothing to be done for 'all'. ``` -## Step 3: Install the Kinect 3D Video Capture Project +### Step 3: Install the Kinect 3D Video Capture Project If Vrui was installed in `/usr/local` or elsewhere outside the user's home directory: @@ -109,7 +134,7 @@ If Vrui was installed inside the user's home directory: make install ``` -## Step 4: Install a udev rule file to give access to all users (optional, Linux-only) +### Step 4: Install a udev rule file to give access to all users (optional, Linux-only) By default, Kinect devices can only be accessed by the root user. This is inconvenient and a security risk, as all Kinect applications must be run as root. diff --git a/docs/installation/index.md b/docs/installation/index.md index 58ac3c9..77db056 100644 --- a/docs/installation/index.md +++ b/docs/installation/index.md @@ -1,11 +1,5 @@ # Getting started with the Kinect package -!!! info "Heads up!" - VRUI must be installed before the Kinect package can be installed. - -1. Install VRUI [(go to instructions $\rightarrow$)](https://vrui-vr.github.io/docs/vrui/installation/) -2. Install Kinect package!!! - ## Prerequisites This software requires the Vrui VR toolkit, version 13.0 build 001 or newer. It also requires `libusb` version 1.0, and that Vrui was configured with support for `libusb` prior to its installation. To properly work with multiple Kinect-for-Xbox version 1473 and/or Kinect-for-Windows devices, the `libusb` library needs to provide USB bus topology query functions such as those provided by the `libusbx` fork. This software can optionally create Kinect 3D video streaming plug-ins for Vrui's collaboration infrastructure (version 2.9 or newer). The presence of Vrui's collaboration infrastructure will be detected automatically during the build process. \ No newline at end of file diff --git a/docs/nav.toml b/docs/nav.toml new file mode 100644 index 0000000..c98e1c8 --- /dev/null +++ b/docs/nav.toml @@ -0,0 +1,15 @@ +# docs/kinect/nav.toml +nav = [ + { "Kinect" = [ + "index.md", + { "Getting started" = [ + "installation/index.md", + { "Installation guide" = "installation/guide.md" }, + ] }, + { "Usage" = [ + "usage/index.md", + { "Recording 3D Movies" = "usage/recording_3d_movies.md" }, + { "Merging 3D Facades from Multiple Kinects" = "usage/merging_3d_facades.md" }, + ] }, + ] }, +] diff --git a/docs/nav.yml b/docs/nav.yml deleted file mode 100644 index 534bb24..0000000 --- a/docs/nav.yml +++ /dev/null @@ -1,9 +0,0 @@ -Kinect: - # - Kinect Overview: index.md - - Getting started: - - installation/index.md - - Installation guide: installation/guide.md - - Usage: - - usage/index.md - - Recording 3D Movies: usage/recording_3d_movies.md - - Merging 3D Facades from Multiple Kinects: usage/merging_3d_facades.md diff --git a/docs/usage/index.md b/docs/usage/index.md index c69d58d..bc1ae88 100644 --- a/docs/usage/index.md +++ b/docs/usage/index.md @@ -1,13 +1,47 @@ # Running the Kinect package -## Step 1. Run ./bin/KinectUtil list to list all Kinect devices connected to the host computer. +## Step 1: Find all connected Kinect devices -## Step 2a. Run sudo ./bin/KinectUtil getCalib to download factory calibration data from the -th Kinect device on the host's USB bus (with the first device having index 0) and create a intrinsic camera calibration file for that device in the Kinect package's configuration file directory. +Run - > **OR** +```sh +./bin/KinectUtil list +``` -## Step 2b. Run ./bin/RawKinectViewer to intrinsically calibrate the -th Kinect device on the host's USB bus (with the first device having index 0), using a semi-transparent checkerboard calibration target (see next section for details). This generates a binary IntrinsicParameters-.dat file in /etc/Kinect-, where is the unique factory-assigned serial number of the Kinect device as displayed in step 1. +to list all Kinect devices connected to the host computer. -Step 2 (a or b) only have to be performed once for each Kinect device, unless the resulting calibration proves unsatisfactory. Kinect devices typically retain their intrinsic calibration over time. +## Step 2: Camera calibration -## Step 3 Run ./bin/KinectViewer -c , where is the zero-based index of the Kinect device to use if there are multiple ones. \ No newline at end of file +### Option 1: Factory calibration + +Run + +```sh +sudo ./bin/KinectUtil getCalib +``` + +to download factory calibration data from the `-`th Kinect device on the host's USB bus (with the first device having index 0) and create an intrinsic camera calibration file for that device in the Kinect package's configuration file directory. + +### Option 2: Intrinsic calibration + +Run + +```sh +./bin/RawKinectViewer +``` + +to intrinsically calibrate the `-`th Kinect device on the host's USB bus (with the first device having index 0), using a semi-transparent checkerboard calibration target (see next section for details). This generates a binary `IntrinsicParameters-.dat` file in <`Vrui install dir>/etc/Kinect-`, where `` is the unique factory-assigned serial number of the Kinect device as displayed in step 1. + +!!! Note + Either of the above options only have to be performed once for each Kinect device, unless the resulting calibration proves unsatisfactory. Kinect devices typically retain their intrinsic calibration over time. + + +## Step 3 + +Run + +```sh +./bin/KinectViewer -c +``` + +where `` is the zero-based index of the Kinect device to use if there are multiple ones. \ No newline at end of file