Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -207,9 +207,17 @@ explaining why: `not_found`, `error`, `pipeline_stopped`.
the camera, what is stored, retention and redaction.
- [Software bill of materials](docs/SBOM.md): CycloneDX SBOM and licence
summary for 0.1.1.
- [Shared responsibility](docs/SHARED_RESPONSIBILITY.md): who owns which
security control (CloudLab Works, you, the model provider), per AI-CAIQ
question.
- [Risk assessment](docs/RISK_ASSESSMENT.md): AI risk register and AI impact
assessment.
- [Hardening](docs/HARDENING.md): running oak-events as a dedicated user, with
a sandboxed, read-only MCP server.
- CloudLab Works [security policies](https://github.com/cloudlabworks/.github/tree/main/policies):
security baseline, security and AI governance policy (including acceptable
use of oak-events) and incident response plan.
use of oak-events), incident response plan, supply chain policy and the
security program index.

Report vulnerabilities as described in [`SECURITY.md`](SECURITY.md).

Expand Down
139 changes: 139 additions & 0 deletions docs/HARDENING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
# Hardening an oak-events deployment

| | |
| --- | --- |
| Owner | Alex Alvord, CloudLab Works |
| Version | Approved by Alex Alvord and published 2026-11-02. Needs oak-events 0.1.2 or later (`--model-sha256`, `--access-log`) |
| Review | At each release that changes a command line option, the MCP tools or the files oak-events writes, and at least once a year (next by 2027-11-01) |
| Files | [`examples/hardening/`](../examples/hardening/): `oak-events-run.service`, `oak-events-mcp-sandbox`, `test/` |

oak-events already limits itself: no pictures leave the camera, the MCP server has three read-only tools
and no path arguments, and event files are owner-only. This guide adds operating-system isolation, so
that a bug in oak-events, in DepthAI or in an agent's use of the MCP tools cannot reach the rest of the
host. It is the customer's part of AI-CAIQ AIS-13.1 (see `SHARED_RESPONSIBILITY.md`).

The recipe is for Linux with systemd. It was tested in a container (section 5), not with a camera.

## 1. Layout

| Path | Owner and mode | Who writes it | Who reads it |
| --- | --- | --- | --- |
| `/opt/oak-events/venv` | root, 0755 | Install step only | Both services |
| `/etc/oak-events/` (model archive, zones file) | root, 0644 files | Admin only | Runner |
| `/var/lib/oak-events/events/` | `oak-events`, 0700; files 0600 | Runner only | MCP server, read-only |
| `/var/log/oak-events/mcp-access.jsonl` | `oak-events`, 0700 directory; file 0600 | MCP server | Admin |

Two processes, one dedicated system user, `oak-events`, with no login shell. The runner can write only the
events directory. The MCP server can write only its access log: the events directory is read-only to it,
and it has no network at all.

## 2. Install

```bash
sudo useradd --system --home-dir /var/lib/oak-events --shell /usr/sbin/nologin oak-events
sudo python3 -m venv /opt/oak-events/venv
sudo /opt/oak-events/venv/bin/pip install 'oak-events[camera,mcp]' # pin the version you tested
sudo install -d -m 0755 /etc/oak-events
sudo install -m 0644 yolov6n-r2-288x512.rvc2.tar.xz /etc/oak-events/
```

Verify the model archive first (`MODEL_CARD.md` section 2.1) and put its SHA-256 in the unit file.

## 3. Runner: a sandboxed system service

Copy `examples/hardening/oak-events-run.service` to `/etc/systemd/system/`, edit the model path, hash and
options, then:

```bash
sudo systemctl daemon-reload
sudo systemctl enable --now oak-events-run.service
journalctl -u oak-events-run.service -f
```

What the unit does:

- runs as `oak-events`, with `NoNewPrivileges`, no capabilities, and a system call filter
(`@system-service`);
- `ProtectSystem=strict`, `ProtectHome=yes`, `PrivateTmp=yes`: the whole file system is read-only or
hidden except the state directory `/var/lib/oak-events/events`, created by systemd with mode 0700;
- `UMask=0077`, so nothing it creates is readable by other users;
- `--model-sha256`: refuses to start if the model archive changed;
- `--quiet`: events go only to the event files, not to the journal (otherwise the journal keeps a
second copy of every event; `DATA_FLOW.md` section 3);
- network limited to IPv4, IPv6, netlink and Unix sockets, which a PoE camera needs. For a USB camera, use
the commented lines instead: no network at all, and only USB devices.

**Not tested with a camera.** The unit was tested without DepthAI and without a camera (section 5). If
DepthAI fails under it, read the journal for `SECCOMP` or address-family errors and relax one option at a
time; please report what you had to change.

## 4. MCP server: read-only, no network, started on demand

An MCP client starts the server over stdio, so it is not a long-running service. The wrapper
`examples/hardening/oak-events-mcp-sandbox` starts it as a transient systemd unit with `systemd-run
--pipe`, which passes stdin and stdout through to the client:

- runs as `oak-events` (the only user that can read the 0600 event files);
- `ReadOnlyPaths=/var/lib/oak-events`: it can read events but cannot change or add any;
- `PrivateNetwork=yes`, `IPAddressDeny=any`, Unix sockets only: no network;
- `PrivateDevices=yes`, `ProtectHome=yes`, `ProtectSystem=strict`, no capabilities, system call filter,
memory and task limits;
- `--access-log /var/log/oak-events/mcp-access.jsonl`: one line per tool call (tool, argument sizes,
result count, outcome, client process id), never values or results. The log directory is the only
writable path.

`systemd-run` needs root, so the desktop user who runs the MCP client gets one `sudo` rule for this exact
command, which takes no arguments:

```bash
sudo install -m 0755 oak-events-mcp-sandbox /usr/local/bin/
echo 'alice ALL=(root) NOPASSWD: /usr/local/bin/oak-events-mcp-sandbox' | sudo tee /etc/sudoers.d/oak-events-mcp
sudo chmod 0440 /etc/sudoers.d/oak-events-mcp
sudo visudo -c
```

Replace `alice` with the user. Client configuration (same shape as `examples/claude-desktop-mcp.json`):

```json
{"mcpServers": {"oak-events": {"command": "sudo", "args": ["-n", "/usr/local/bin/oak-events-mcp-sandbox"]}}}
```

If the MCP client runs on another computer, use `ssh` as the command (`ssh HOST sudo -n
/usr/local/bin/oak-events-mcp-sandbox`); stdio passes through ssh.

## 5. Test results

`bash examples/hardening/test/test.sh WHEEL MODEL` builds a Debian 13 container with systemd as PID 1,
installs oak-events from a wheel, and checks the two recipes. Run on 2026-11-02 with a wheel built from the
code released in oak-events 0.1.2 (branch `csa/logging-integrity`) and the reference model archive; all 23 checks passed:

| Check | Result |
| --- | --- |
| Runner starts as `oak-events`, passes the model hash check, writes the `privacy` event | Pass |
| Event files owned by `oak-events`, mode 0600; directory mode 0700 | Pass |
| Runner can write the events directory, cannot write `/etc`, cannot read `/home`, cannot gain root | Pass |
| A wrong `--model-sha256` stops the runner with a mismatch event, camera not opened | Pass |
| MCP server through the wrapper, started by a desktop user with `sudo`: `camera_health` and `recent_events` answer; `private` is `true` | Pass |
| Access log written, owned by `oak-events`, mode 0600, with argument sizes and result counts only | Pass |
| The desktop user cannot read event files directly | Pass |
| MCP sandbox can read events and write its log, cannot write the events directory, cannot read `/home`, has no network (the same connection works outside the sandbox) | Pass |
| `systemd-analyze security oak-events-run.service` | 1.7 ("OK"; lower is better) |

Not tested: a real camera, DepthAI under these options, USB device access, other distributions.

## 6. Also recommended

1. **Camera network.** Put a PoE camera on its own network segment or VLAN that only the oak-events host
can reach (`DATA_FLOW.md` section 2). Nothing else needs to talk to it.
2. **Direct file access.** The MCP access log records reads through MCP, not someone running `cat` on
the files. To record those too, add a Linux audit rule, for example
`auditctl -w /var/lib/oak-events/events -p rwa -k oak-events`, and keep it in `/etc/audit/rules.d/`.
Not tested by CloudLab Works.
3. **Retention.** Delete old event files on a timer (`DATA_FLOW.md` section 4); run the timer as
`oak-events`.
4. **Alerts.** Forward `privacy` events with `"private": false` and `camera` errors to someone who will
read them (`examples/webhook.md`).
5. **Updates.** Pin the oak-events and DepthAI versions you tested; update both deliberately and re-run
the model spot-check (`MODEL_CARD.md` section 7) after a DepthAI update.

macOS (launchd) is not covered yet.
137 changes: 132 additions & 5 deletions docs/MODEL_CARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
| | |
| --- | --- |
| Owner | Alex Alvord, CloudLab Works |
| Version | Published 2026-11-02; written against commit 00d9053 |
| Version | Published 2026-11-02; written against commit 00d9053. Revised, approved by Alex Alvord and published 2026-11-02: verification steps (section 2.1), provenance (section 2.2), monitoring and spot-check (section 7), bias and fairness (section 8) |
| Review | At each release that changes model guidance, and at least once a year (next by 2027-11-01) |

## 1. What oak-events ships
Expand Down Expand Up @@ -42,6 +42,52 @@ not record which archive file was loaded, so the link between this hash and the
UNVERIFIED. The zoo now lists 4 versions of this model; a new download may differ. Check the hash of what you
download and compare with this card.

### 2.1 Verifying a model archive

Check the archive at every handover: when you download it, when you copy it to the host that runs
oak-events, and every time oak-events loads it.

1. **Hash.** Compare with the SHA-256 above (Linux `sha256sum`, macOS `shasum -a 256`):

```bash
echo "8372e2dec2d5e192e6ad9555b9412c3e31101b06e68121cfd08e92d4a4641122 yolov6n-r2-288x512.rvc2.tar.xz" | sha256sum -c -
```

It must print `OK`. Tested on macOS (`shasum -a 256 -c -`) and Debian 13 (`sha256sum -c -`) on
2026-11-02 against the archive described here.

2. **Contents.** The archive must hold exactly three regular files, and nothing that runs on the host:

```bash
tar -tvJf yolov6n-r2-288x512.rvc2.tar.xz
```

Expected: `config.json`, `buildinfo.json` and `yolov6n-r2-288x512.superblob`, each a regular file
(the listing starts with `-`). Any other file, a link, or a Python, pickle or script file means the
archive is not the one described here: do not use it. Checked 2026-11-02.

3. **At every load** (from the 0.1.2 and later): run with
`--model-sha256 8372e2dec2d5e192e6ad9555b9412c3e31101b06e68121cfd08e92d4a4641122`. oak-events
computes the archive's SHA-256 before opening the camera and refuses to start on a mismatch (a
`camera` event with `state` `error`, exit code 3). Every `camera` `up` event records the hash of
the archive that was loaded, so the event files show which model ran.

If you use another archive, record its hash the day you download it and use that value instead.

### 2.2 Provenance

| Step | Who | Evidence | Signed? |
| --- | --- | --- | --- |
| Architecture and training on COCO | Meituan ([meituan/YOLOv6](https://github.com/meituan/YOLOv6), GPL-3.0) | Upstream repository and paper ([arXiv:2209.02976](https://arxiv.org/abs/2209.02976)) | No signature found for upstream weights |
| Conversion to an RVC2 blob | Luxonis, ModelConverter 0.1.2 with the OpenVINO 2022.3 compile tool, 2024-07-24 | Archive `buildinfo.json` | No |
| Distribution | Luxonis model zoo (HubAI), model marked official | Luxonis model API, read 2026-11-02 | No signature offered; download over HTTPS |
| Download by the deployer | Deployer, once, with `dai.getModelFromZoo` (README) | The archive's SHA-256, compared with this card | Hash only |
| Load | oak-events, on each start | `--model-sha256` (0.1.2 and later) and the hash in the `camera` `up` event | Hash only |

Hash pinning detects a changed file; it does not prove who made it. A signature from the model provider
would. CloudLab Works checks at each review whether Luxonis has started signing model archives, and
verifies signatures once they are offered. **Commitment:** next check by 2027-11-01 (owner: Alex Alvord).

## 3. Intended use and limits

**Intended:** detect that a person is present in a zone, for presence events.
Expand All @@ -56,11 +102,11 @@ people where accuracy matters; safety or security decisions; children's monitori
2. Photos, screens, posters and mannequins can be detected as people.
3. The 512 x 288 input is smaller than the 640 x 640 used for the published accuracy figure.
4. Fairness across skin tones, clothing, body types and mobility aids has not been evaluated by
CloudLab Works. Whether upstream evaluated it is UNKNOWN.
CloudLab Works. No upstream fairness evaluation was found (section 8).
5. Depth is aligned to the colour image only approximately, so zone edges are fuzzy.
6. CloudLab Works has run only a functional live test (camera up, baseline built, about 10 frames per
second). It has not measured accuracy. **Commitment:** a documented spot-check on the test camera by
2027-03-31.
2027-03-31, using the procedure in section 7.

## 4. Failure modes

Expand Down Expand Up @@ -89,5 +135,86 @@ Any model CloudLab Works names as a reference must have, in this card: name and
distributor, licence, training data, input and format, thresholds, reported metrics with their source,
archive hash, intended and out-of-scope uses, known limits, CloudLab Works' own evaluation, failure modes,
open-model risk, and review date. At each review the fields are checked against the archive's `config.json`
and `buildinfo.json`, the Luxonis model page and a fresh SHA-256 of the archive. This card was checked that
way on 2026-11-02.
and `buildinfo.json`, the Luxonis model page and a fresh SHA-256 of the archive, and the archive contents
are checked as in section 2.1. The bias and fairness notes (section 8) are re-checked against upstream and
published research. This card was checked that way on 2026-11-02.

The verification steps in section 2.1 are reviewed with this card: at each release that changes model
guidance, at each change of reference model, and at least once a year.

## 7. Monitoring detection quality

oak-events cannot see pictures, so it cannot score its own detections. Two things help.

**Confidence statistics** (from the 0.1.2 and later). With `oak-events run --confidence-stats`,
every `alive` event carries, per zone, the number of person detections at or above the threshold
(`n`), their `mean`, `min` and `max` confidence, and the number below the threshold (`below`), for the
time since the previous `alive` event. Only numbers are kept.

Watch the trend on a scene that does not change. Signs that something changed (camera moved, lighting,
lens dirt, a different model, a firmware update):

- the mean confidence for a zone drops by more than 0.1 against the same hours of the previous week;
- `below` grows to more than twice its usual share of `n + below`;
- `n` falls to zero in hours when people are normally present.

```bash
jq -c 'select(.kind == "alive" and .confidence) | {ts, confidence}' DIR/*.jsonl
```

**Spot-check procedure.** Run it when you install a camera, after any of the changes above, and at
least once a month. CloudLab Works runs it on its test camera by 2027-03-31 and at each change of
reference model, and records the result in this card.

1. Turn on `--confidence-stats` and note the date, camera position, lighting, model hash and
oak-events version.
2. With the people who use the space, and with their agreement, walk through each zone 20 times:
standing, walking, sitting, and, where it fits the space, lying down, carrying something, and at the
far edge of the zone. Count the `person_enter` events. Expected: 20 of 20 for standing and walking.
3. Show the camera a printed photo and a screen with a person on it for 30 seconds each. Count any
`person_enter` events: each one is a false positive (threat model T9).
4. Leave the zone empty for 10 minutes. Count any `person_enter` events.
5. Record the counts and the confidence statistics. Compare with the last check. If standing or walking
detection is below 18 of 20, or empty-zone or photo tests produce events, fix the cause (position,
light, threshold, model) before relying on the events.

No picture is taken or stored at any step; the check uses the events only.

## 8. Bias and fairness

**Upstream evaluation.** The YOLOv6 README and paper report accuracy on COCO val2017 only. No
evaluation of detection rates across groups of people was found: the upstream README has no mention
of bias, fairness, demographics or skin tone (searched 2026-11-02), nor does the paper's abstract.
The Luxonis model page cites the upstream accuracy figure. Upstream fairness evaluation is therefore
UNKNOWN, and treated as not done.

**Published research on similar detectors.** These studies did not test YOLOv6. They show the kinds of
gaps to expect from person detectors trained on everyday photographs:

- Wilson, Hoffman and Morgenstern, *Predictive Inequity in Object Detection*, 2019
([arXiv:1902.11097](https://arxiv.org/abs/1902.11097)): detectors performed less well on pedestrians
with darker skin tones (Fitzpatrick 4 to 6) on BDD100K, and time of day and occlusion did not explain
the gap.
- Brandão, *Age and gender bias in pedestrian detection algorithms*, 2019
([arXiv:1906.10490](https://arxiv.org/abs/1906.10490)): all 24 top pedestrian detectors tested had
higher miss rates on children; gender differences were not significant.

**Known and possible gaps for oak-events**

| Group or condition | Expected effect | Status |
| --- | --- | --- |
| Children, people small in frame or far away | Lower recall: late or missing `person_enter`, early `person_exit` | Likely (section 3 limit 1; Brandão 2019). Not measured |
| People lying down, sitting low, or seen from above | Lower recall | Likely (section 3 limit 1). Not measured |
| People using wheelchairs or other mobility aids | UNKNOWN | Not measured; COCO coverage UNKNOWN |
| Darker skin tones, especially in low light | Possibly lower recall | Possible (Wilson 2019, on other detectors). UNKNOWN for this model |
| Unusual clothing, costumes, heavy coats | UNKNOWN | Not measured |

**What this means for use.** Events may be less reliable for some people in view. That is one reason
oak-events must not be used for safety, care or security decisions (section 3, and the acceptable use
policy). Deployers should run the spot-check (section 7) with the people who actually use the space.

**Regular evaluation.** CloudLab Works re-checks upstream and published research at each review of
this card, at least once a year. The first CloudLab Works spot-check (by 2027-03-31) will include the
poses and heights available at the test site, with the agreement of the people involved. A household
test cannot measure differences across skin tones or mobility aids with any confidence; CloudLab Works
will say so in the result rather than claim a fairness evaluation it has not done.
Loading
Loading