Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitbook.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
root: ./docs/

structure:
readme: README.md
summary: SUMMARY.md
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ All notable changes to this project are documented here. The format follows
`oak_events-<version>.intoto.jsonl`. Verify a download with
`gh attestation verify <file> --repo cloudlabworks/oak-events`. PyPI's own
PEP 740 attestations are unchanged.
- User documentation in `docs/` (overview, install, quickstart, MCP server,
configuration, events reference, running it for real, security,
contributing), with `docs/SUMMARY.md` and `.gitbook.yaml` so the folder can
be synced to a GitBook site.

## [0.1.2] - 2026-11-02

Expand Down
65 changes: 65 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# oak-events

Privacy-first events from Luxonis OAK cameras for AI agents.

An OAK camera detects people, tracks them and measures depth on its own chip.
`oak-events` turns that into one JSON line per event ("a person entered the
doorway zone", "something 40 cm closer has stood in cell r2c5 for a minute")
and serves those lines to agents through an MCP server.

**Agents never receive a picture.** There is no image tool, no snapshot
command, and no code path that writes an image file.

This documentation covers oak-events 0.1.2, the current release on
[PyPI](https://pypi.org/project/oak-events/). The source is on
[GitHub](https://github.com/cloudlabworks/oak-events).

## Why

Giving an agent a camera usually means giving it a video feed. Most agent
questions do not need one: *is anyone at the bench? did something on the shelf
move? is the camera alive?* Those are answerable from detections and a coarse
depth grid, which carry far less about the people in view than frames do.

## Privacy model

Stated plainly, so you can check it against the code:

1. **Colour frames stay on the camera.** The colour sensor feeds the on-chip
detection network. Its frames are never linked to a host output queue.
Only two streams reach the host: detection results (label, confidence,
3-D position, normalized box) and the stereo depth map
(`src/oak_events/camera.py`).
2. **Depth maps are reduced and dropped.** Each depth frame becomes a grid of
cell medians (default 6 x 8) in memory, then is discarded. No depth map is
stored.
3. **Only JSON lines are written**, to stdout and/or a daily `.jsonl` file.
A test scans the repo and a temporary run directory for image files.
4. **DepthAI telemetry, crash-dump upload and auto-calibration are off** by
default. See [Configuration](configuration.md#privacy-environment).
5. **The MCP server only reads the JSONL files.** It never opens the camera.
6. **No model download at runtime.** You point `--model` at a local NNArchive
you fetched once.

What the events still reveal: when someone was present, for how long, how far
from the camera, and when objects moved. Treat the event files as sensitive
and keep them local.

## Where to go next

| Page | What it covers |
| --- | --- |
| [Install](install.md) | Supported hardware, installing from PyPI, fetching a model once |
| [Quickstart](quickstart.md) | Check the setup, run the camera, read events over MCP |
| [MCP server](mcp-server.md) | The three MCP tools, client configuration, access log |
| [Configuration](configuration.md) | Command line options, zones, model hash check, privacy environment |
| [Events reference](events.md) | Every event kind and its fields |
| [Running it for real](operations.md) | Supervisors, exit codes, PoE notes |
| [Security](security.md) | Reporting vulnerabilities, verifying releases, security and AI documents |
| [Contributing and licence](contributing.md) | Development setup, pull requests, MIT licence |

---

oak-events is an independent community project by CloudLab Works, not
affiliated with or endorsed by Luxonis. OAK and DepthAI are Luxonis trademarks,
used here only to say which hardware and SDK the tool works with.
30 changes: 30 additions & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Table of contents

* [Overview](README.md)

## Getting started

* [Install](install.md)
* [Quickstart](quickstart.md)

## Using oak-events

* [MCP server](mcp-server.md)
* [Configuration](configuration.md)
* [Events reference](events.md)
* [Running it for real](operations.md)

## Security and AI

* [Security](security.md)
* [Threat model](THREAT_MODEL.md)
* [Model card](MODEL_CARD.md)
* [Data flow and privacy impact](DATA_FLOW.md)
* [Software bill of materials](SBOM.md)
* [Shared responsibility](SHARED_RESPONSIBILITY.md)
* [Risk assessment](RISK_ASSESSMENT.md)
* [Hardening](HARDENING.md)

## Project

* [Contributing and licence](contributing.md)
94 changes: 94 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Configuration

oak-events has no configuration file. Everything is a command line option, a
zones file, or an environment variable.

## `oak-events run`

| Option | Default | Meaning |
| --- | --- | --- |
| `--model PATH` | required | Local detection NNArchive (`.tar.xz`), fetched once (see [Install](install.md#fetch-a-model-once)) |
| `--device-id ID` | first device found | Device to open. Prefer it over addresses; the id survives network changes |
| `--out-dir DIR` | none | Also append events to `DIR/YYYY-MM-DD.jsonl` (mode 0600) |
| `--zones FILE` | one zone, `view` | Zones file (see [Zones](#zones)) |
| `--fps N` | `10` | Detection frame rate |
| `--alive-every SECONDS` | `300` | Interval of the `alive` heartbeat event |
| `--person-label NAME` | `person` | Model class treated as a person |
| `--person-confidence X` | `0.5` | Minimum detection confidence for a person |
| `--hold-seconds SECONDS` | `60` | How long a depth change must persist before it is reported |
| `--model-sha256 HEX` | off | Refuse to start unless the model archive has this SHA-256 |
| `--confidence-stats` | off | Add per-zone detection-confidence statistics to `alive` events |
| `--quiet` | off | Do not echo events to stdout |

For `oak-events mcp` options, see [MCP server](mcp-server.md#options).
`oak-events doctor` takes only `--json`.

## Zones

By default everything is reported for one zone named `view`: the whole field
of view at any distance. A zones file names regions by normalized image box
and/or distance:

```json
{
"zones": [
{"name": "doorway", "bbox": [0.0, 0.0, 0.35, 1.0]},
{"name": "desk", "bbox": [0.4, 0.3, 1.0, 1.0], "z_mm": [400, 2500]},
{"name": "far_wall", "z_mm": [3000, null]}
]
}
```

- `bbox` is `[x0, y0, x1, y1]` in 0..1 with the origin top-left, and must
satisfy `0 <= x0 < x1 <= 1` and `0 <= y0 < y1 <= 1`.
- `z_mm` is `[min, max]` distance from the camera in millimetres. Either side
may be `null`; if both are set, `min` must be below `max`.
- A zone may have `bbox`, `z_mm`, both, or neither.
- Numbers must be finite: `NaN` and `Infinity` are rejected.

Each zone gets its own person tracker and depth-change grid. The depth map is
aligned to the colour camera only approximately, so leave some margin.

A malformed file stops `oak-events run` with a one-line `bad zones file` error
naming the zone, and exit code 2.

## Model integrity check

The SHA-256 of the model archive is computed at every start and recorded in
the `camera` `up` event (`model_sha256`), with or without any option.

`--model-sha256 HEX` compares that hash with `HEX` (64 hexadecimal
characters) before the camera is opened. On a mismatch oak-events writes a
`camera` event with state `error`, `model_sha256` and `expected_sha256`, and
exits with code 3. The reference hash of the tested archive is in the
[model card](MODEL_CARD.md).

## Confidence statistics

`--confidence-stats` adds a `confidence` object to each `alive` event: per
zone, `n` counts per-frame person detections at or above
`--person-confidence`, `below` counts those under it, and `mean`, `min` and
`max` describe the accepted ones (`null` when `n` is 0). Each `alive` event
covers the time since the previous one. On a scene that does not change, a
falling mean or a rising `below` count means the model, the camera or the
light changed. The [model card](MODEL_CARD.md) describes a spot-check
procedure.

## Privacy environment

Before `depthai` is imported, oak-events forces these variables, overwriting
any value already set:

| Variable | Forced value | Effect |
| --- | --- | --- |
| `DEPTHAI_TELEMETRY` | `0` | DepthAI telemetry off |
| `DEPTHAI_DISABLE_CRASHDUMP_COLLECTION` | `1` | Crash-dump upload off |
| `DEPTHAI_AUTOCALIBRATION` | `OFF` | On-device auto-calibration off |

The only way to keep DepthAI's own defaults is `OAK_EVENTS_ALLOW_TELEMETRY=1`,
in which case oak-events leaves the environment untouched.

Every run writes a `privacy` event before DepthAI loads: `private` is `false`
if any of the three variables differs from the private value, and
`deviations` names them. `oak-events doctor` shows the same state, and the MCP
`camera_health` tool returns the latest `privacy` event.
69 changes: 69 additions & 0 deletions docs/contributing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Contributing and licence

Bug reports, hardware reports (especially RVC4) and small, focused pull
requests are all welcome on
[GitHub](https://github.com/cloudlabworks/oak-events). The full guide is
[`CONTRIBUTING.md`](https://github.com/cloudlabworks/oak-events/blob/main/CONTRIBUTING.md)
in the repository; this page summarizes it.

## The one rule

**No pictures leave the camera.** Changes must not:

- link a colour, preview or passthrough output to a host queue,
- write images, depth maps or arrays (`.jpg`, `.png`, `.npy`, ...) to disk,
- add an MCP tool or event field that carries image data.

`tests/test_no_images.py` and the privacy tests enforce part of this;
reviewers check the rest.

## Development setup

```bash
python3 -m venv .venv
.venv/bin/pip install -e '.[mcp]' pytest
.venv/bin/pytest -q
```

Tests must pass **without a camera and without `depthai` installed**; that is
what CI runs (Ubuntu and macOS, Python 3.11 and 3.13). CI installs its tools
from the hash-pinned lists in `requirements/`. Put event logic in `events.py`
(pure, testable with synthetic data) and keep `camera.py` a thin adapter
around DepthAI.

The parsing code (zones files, event files) also has an Atheris fuzz harness,
run for 60 seconds on every push and pull request.

If you have hardware, also try `pip install -e '.[camera]'` and
`oak-events doctor`, then a short `oak-events run`, and mention the device and
results in your pull request.

## Pull requests

- One change per pull request, with tests for new logic.
- Update the event schema in `README.md` and in this documentation
([Events reference](events.md)) when you add or change an event.
- Add a line under "Unreleased" in `CHANGELOG.md`.
- Do not include device ids, network addresses, hostnames or personal paths in
code, tests, docs or example output.

## Editing this documentation

These pages live in the `docs/` folder of the repository and are published
with GitBook. Edit the Markdown files and open a pull request like any other
change; `docs/SUMMARY.md` sets the page order.

## Code of conduct

This project follows the
[Contributor Covenant](https://github.com/cloudlabworks/oak-events/blob/main/CODE_OF_CONDUCT.md).

## Licence

oak-events is released under the
[MIT licence](https://github.com/cloudlabworks/oak-events/blob/main/LICENSE).
By contributing you agree your work is released under the same licence.

oak-events is an independent community project by CloudLab Works, not
affiliated with or endorsed by Luxonis. OAK and DepthAI are Luxonis
trademarks, used here only to say which hardware and SDK the tool works with.
92 changes: 92 additions & 0 deletions docs/events.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Events reference

Every event is one JSON object on one line, with `ts` (ISO-8601 local time
with offset, to the second), `kind`, and the fields below. `oak-events run`
prints events to stdout and, with `--out-dir DIR`, appends them to
`DIR/YYYY-MM-DD.jsonl`.

```json
{"ts": "2026-10-11T09:14:03-07:00", "kind": "person_enter", "zone": "doorway", "count": 1, "nearest_mm": 1840}
```

## Event kinds

| kind | fields | meaning |
| --- | --- | --- |
| `person_enter` | `zone`, `count`, `nearest_mm` | A person was seen in 3 consecutive frames |
| `person_exit` | `zone`, `dwell_s` | Nobody seen for 10 seconds; dwell is first to last sighting |
| `depth_baseline` | `zone`, `cells_judged`, `cells_noisy`, `grid` | Baseline built; noisy cells (stereo noise, low texture) are ignored |
| `depth_change` | `zone`, `cells` (for example `["r0c3"]`), `delta_mm` | A cell's depth moved more than 15 % for `--hold-seconds` (60) with nobody present |
| `camera` | `state`, plus the fields below as relevant | Camera lifecycle |
| `alive` | `det_fps`, `person`, `occupied`; with `--confidence-stats` also `confidence` | Heartbeat every `--alive-every` seconds (300) |
| `privacy` | `private`, `telemetry_opt_in`, `deviations` | Written once per run, before DepthAI loads |

## People

A person tracker runs per zone. A detection counts when its label is
`--person-label` (`person`), its confidence is at least `--person-confidence`
(0.5), and it falls inside the zone.

- `person_enter` fires after 3 consecutive frames with a person. `count` is
the number of people in that frame, `nearest_mm` the distance of the
nearest one.
- `person_exit` fires once nobody has been seen for 10 seconds. `dwell_s` is
the time from first to last sighting, in whole seconds.

## Depth

Each depth frame is reduced to a grid of cell medians (default 6 rows by 8
columns) per zone and then dropped. Cells are named `r<row>c<column>`,
starting at `r0c0` top-left.

- `depth_baseline`: the first 60 depth frames build a baseline. `grid` is
`[rows, columns]`, `cells_judged` the cells that will be watched, and
`cells_noisy` the cells ignored because their depth is unstable or often
invalid.
- `depth_change`: one or more cells differ from the baseline by more than
15 % for `--hold-seconds`. `cells` lists them and `delta_mm` gives the
signed change for each, in the same order (negative means closer). The
baseline of those cells then moves to the new depth.

Depth-change detection pauses while a person is present in that zone: people
are not scenery.

## Camera lifecycle

`camera` events carry `state`:

| state | When | Extra fields |
| --- | --- | --- |
| `up` | Pipeline started | `device`, `device_model`, `bootloader`, `model`, `model_sha256`, `model_sha256_checked`, `zones` |
| `not_found` | No device found after up to 240 seconds of discovery | `device` |
| `reconnect` | DepthAI is reconnecting (up to 5 attempts) | `detail` |
| `pipeline_stopped` | The pipeline stopped running | |
| `error` | Model archive missing, model hash mismatch, or an unexpected error | `detail`; on a hash mismatch also `model`, `model_sha256`, `expected_sha256` |
| `stopped` | Stopped with Ctrl+C | |

For the exit code that goes with each state, see
[Running it for real](operations.md#exit-codes).

## Heartbeat

`alive` events report `det_fps` (detection frames per second since the
previous heartbeat), `person` (`true` if any zone has a person present) and
`occupied` (the zones with a person present).

With `--confidence-stats`, `confidence` holds per zone `n`, `mean`, `min`,
`max` and `below`; see
[Configuration](configuration.md#confidence-statistics).

## Privacy

The `privacy` event records whether the DepthAI privacy defaults held for this
run: `private` is `false` if any DepthAI privacy variable differs from the
private value, `telemetry_opt_in` is `true` if `OAK_EVENTS_ALLOW_TELEMETRY=1`
was set, and `deviations` names the variables that differ. See
[Configuration](configuration.md#privacy-environment).

## What events never contain

No event carries image data, a depth map, or an array of pixels. The only
spatial data is the per-zone grid size, cell names, distances in millimetres
and normalized zone boxes from your zones file.
Loading
Loading