Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,23 @@ All notable changes to this project are documented here. The format follows
- Configurable `oak-events mcp --stale-after SECONDS` threshold for camera health.
- Atheris fuzz harness `fuzz/fuzz_events.py` for zones files and event JSONL
files, run for 60 s on every push and pull request (`fuzz` workflow).
- `oak-events run` writes a `privacy` event before DepthAI loads, recording
whether the DepthAI privacy defaults hold (`private`), whether telemetry
was opted in (`telemetry_opt_in`) and which variables differ
(`deviations`). `camera_health` returns the latest one and `private`.
- `oak-events run --model-sha256 HEX`: refuse to start (exit code 3, camera
not opened) when the model archive's SHA-256 differs. The archive's
SHA-256 is now recorded in every `camera` `up` event.
- `oak-events run --confidence-stats`: per-zone detection-confidence
statistics (`n`, `mean`, `min`, `max`, `below`) in `alive` events. Off by
default.
- `oak-events mcp --access-log FILE`: optional audit log, one JSON line per
tool call (time, tool, argument names and value sizes, result count,
outcome, MCP client process id). Never argument values, results or image
content. Off by default.
- `examples/webhook.md`: alerting on privacy and model-integrity events.
- Fuzz harness also checks the access log's result count and the
`camera_health` privacy fields.

### Changed

Expand Down
31 changes: 26 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,9 +114,24 @@ the colour camera only approximately, so leave some margin.

| Command | What it does |
| --- | --- |
| `oak-events run --model PATH [--device-id ID] [--out-dir DIR] [--zones FILE] [--fps 10]` | Run the camera; print events, optionally append to `DIR/YYYY-MM-DD.jsonl` |
| `oak-events run --model PATH [--device-id ID] [--out-dir DIR] [--zones FILE] [--fps 10] [--model-sha256 HEX] [--confidence-stats]` | Run the camera; print events, optionally append to `DIR/YYYY-MM-DD.jsonl` |
| `oak-events doctor [--json]` | depthai version, visible devices, bootloader advice, privacy env state |
| `oak-events mcp --events-dir DIR [--stale-after SECONDS]` | MCP stdio server with `recent_events`, `zone_state`, `camera_health`; `camera_health` defaults to stale after 900 seconds |
| `oak-events mcp --events-dir DIR [--stale-after SECONDS] [--access-log FILE]` | MCP stdio server with `recent_events`, `zone_state`, `camera_health`; `camera_health` defaults to stale after 900 seconds |

Optional checks, all off unless you set them:

- `--model-sha256 HEX`: compare the model archive's SHA-256 with `HEX` at
start and refuse to run on a mismatch (`camera` event `error`, exit code 3,
camera not opened). The hash is computed and recorded in the `camera` `up`
event on every start, with or without this option. The reference hash is in
[`docs/MODEL_CARD.md`](docs/MODEL_CARD.md).
- `--confidence-stats`: add per-zone detection-confidence statistics to each
`alive` event (see the spot-check procedure in the model card).
- `--access-log FILE`: append one JSON line per MCP tool call to `FILE` (mode
0600): time, tool name, argument names with the size of each value, result
count, success, and the process id of the MCP client. Argument values and
results are not written, and there is no image content to write. `FILE`
must be outside the events directory.

## Event schema

Expand All @@ -129,8 +144,14 @@ Every line is a JSON object with `ts` (ISO-8601 local time with offset),
| `person_exit` | `zone`, `dwell_s` | nobody seen for `gone_seconds` (10); dwell = first to last sighting |
| `depth_baseline` | `zone`, `cells_judged`, `cells_noisy`, `grid` | baseline built; noisy cells (stereo noise, low texture) are ignored |
| `depth_change` | `zone`, `cells` (e.g. `["r0c3"]`), `delta_mm` | a cell's depth moved > 15 % for `--hold-seconds` (60) with nobody present |
| `camera` | `state` (`up`, `not_found`, `error`, `reconnect`, `pipeline_stopped`, `stopped`), plus `device`, `device_model`, `bootloader`, `model`, `detail` as relevant | camera lifecycle |
| `alive` | `det_fps`, `person`, `occupied` | heartbeat every `--alive-every` seconds (300) |
| `camera` | `state` (`up`, `not_found`, `error`, `reconnect`, `pipeline_stopped`, `stopped`), plus `device`, `device_model`, `bootloader`, `model`, `model_sha256`, `model_sha256_checked`, `expected_sha256`, `detail` as relevant | camera lifecycle |
| `alive` | `det_fps`, `person`, `occupied`; with `--confidence-stats` also `confidence`: per zone `n`, `mean`, `min`, `max`, `below` | heartbeat every `--alive-every` seconds (300) |
| `privacy` | `private`, `telemetry_opt_in`, `deviations` | written once per run, before DepthAI loads: `private` is `false` if any DepthAI privacy variable differs from the private default, `deviations` names them |

In `confidence`, `n` counts per-frame person detections in the zone at or
above `--person-confidence` and `below` counts those under it; `mean`, `min`
and `max` describe the accepted ones (`null` when `n` is 0). Each `alive`
event covers the time since the previous one.

Example:

Expand All @@ -148,7 +169,7 @@ Python SDK). Tools, all returning JSON text:

- `recent_events(limit=50, kind=None)` - latest events, oldest first
- `zone_state()` - per zone: person present or not, latest person and depth events
- `camera_health()` - latest `camera` and `alive` events with their age and a status (`ok`, `stale`, `no_data`, or the last camera state)
- `camera_health()` - latest `camera` and `alive` events with their age and a status (`ok`, `stale`, `no_data`, or the last camera state), plus the latest `privacy` event and `private` (`true`, `false`, or `null` if no run has written one)

See `examples/claude-desktop-mcp.json` for a client config, and
`examples/openclaw-stream.md` / `examples/webhook.md` for wiring the event
Expand Down
27 changes: 27 additions & 0 deletions examples/webhook.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,30 @@ tail -F ~/.local/share/oak-events/events/"$(date +%F)".jsonl | ...

Note that `tail -F` on a dated file does not roll over at midnight; restart it
daily or read stdout instead.

## Alerting on security events

Two events mean something security-relevant changed. Forward them to a place
a person reads:

- `privacy` with `"private": false`: DepthAI telemetry, crash-dump upload or
auto-calibration is not in its private default for this run (`deviations`
says which).
- `camera` with `"state": "error"` and a `detail` starting `model archive
SHA-256 mismatch`: the model file is not the one you pinned with
`--model-sha256`.

```bash
oak-events run --model ~/models/yolov6-nano.rvc2.tar.xz --model-sha256 "$MODEL_SHA256" \
| jq -c --unbuffered 'select((.kind == "privacy" and .private != true)
or (.kind == "camera" and .state == "error"))' \
| while IFS= read -r line; do
curl -sS -X POST -H 'Content-Type: application/json' \
--data-binary "$line" "$ALERT_WEBHOOK_URL" > /dev/null
done
```

An agent can check the same thing through MCP: `camera_health` returns
`"private": false` after such a run. If you run `oak-events mcp --access-log
FILE`, `FILE` is a record of which tool calls the agent made; keep it with
your other logs.
Binary file added fuzz/corpus/events-privacy
Binary file not shown.
3 changes: 3 additions & 0 deletions fuzz/events.dict
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@
"\"person_exit\""
"\"depth_change\""
"\"depth_baseline\""
"\"privacy\""
"\"private\""
"\"telemetry_opt_in\""
"null"
"true"
"false"
Expand Down
7 changes: 6 additions & 1 deletion fuzz/fuzz_events.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@
* events: bytes written as a daily JSONL file -> ``recent_events``,
``zone_state`` and ``camera_health``. Malformed lines are documented as
skipped, so no file content may raise, and every result must still be
JSON-serialisable (the MCP tools return ``json.dumps`` of it).
JSON-serialisable (the MCP tools return ``json.dumps`` of it). The access
log's ``result_count`` must accept every result, and ``camera_health``'s
``private`` must be a boolean or ``None`` whatever the ``privacy`` event says.

No camera and no depthai: only the pure modules are imported.
"""
Expand Down Expand Up @@ -80,6 +82,9 @@ def check_events(data: bytes) -> None:
]
for result in results:
json.dumps(result)
for tool, result in zip(("recent_events", "recent_events", "zone_state", "camera_health"), results):
assert mcp_server.result_count(tool, result) >= 0 # what the access log records
assert results[3]["private"] in (True, False, None), results[3]["private"]
assert results[3]["status"] in ("ok", "stale", "no_data") or results[3]["last_camera"] is not None


Expand Down
56 changes: 47 additions & 9 deletions src/oak_events/camera.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,18 @@
feed the on-chip network and are never linked to a host queue.
* Each depth map is reduced to a coarse grid of cell medians and dropped.
Nothing but JSON lines is ever written.
* Telemetry, crash dumps and auto-calibration are off (see ``privacy``).
* Telemetry, crash dumps and auto-calibration are off (see ``privacy``), and
every run records that state in a ``privacy`` event before DepthAI loads.
"""

from . import privacy # noqa: F401 (must stay first: sets DepthAI env before depthai loads)
from . import privacy # must stay first: sets DepthAI env before depthai loads

import hashlib
import time
from pathlib import Path
from typing import Any, Callable

from .events import DepthChangeDetector, PersonTracker, Zone
from .events import ConfidenceStats, DepthChangeDetector, PersonTracker, Zone, make_event
from .sink import JsonlSink

INSTALL_HINT = "depthai is not installed. Install the camera extra: pip install 'oak-events[camera]'"
Expand Down Expand Up @@ -54,6 +56,15 @@ def discover(dai: Any, device_id: str | None = None, timeout_s: float = DISCOVER
sleep(poll_s)


def file_sha256(path: str | Path, chunk_size: int = 1 << 20) -> str:
"""Lower-case hex SHA-256 of a file, read in chunks."""
digest = hashlib.sha256()
with open(path, "rb") as fh:
for chunk in iter(lambda: fh.read(chunk_size), b""):
digest.update(chunk)
return digest.hexdigest()


def _label_name(labels: list[str], idx: int) -> str:
return labels[idx] if 0 <= idx < len(labels) else str(idx)

Expand Down Expand Up @@ -97,10 +108,19 @@ def run(
hold_seconds: float = 60.0,
noisy_frac: float = 0.10,
discovery_timeout_s: float = DISCOVERY_TIMEOUT_S,
model_sha256: str | None = None,
confidence_stats: bool = False,
) -> int:
"""Run until the pipeline stops. Returns a process exit code.

Exit codes: 1 pipeline stopped, 2 no device found, 3 bad model path.
Exit codes: 1 pipeline stopped, 2 no device found, 3 bad model path or
model hash mismatch.
Just before DepthAI loads, a ``privacy`` event records the privacy state.
The model archive's SHA-256 is
computed at every start and recorded in the ``camera`` ``up`` event; with
``model_sha256`` set, a different hash stops the run before the camera is
opened. With ``confidence_stats``, ``alive`` events carry per-zone
detection-confidence statistics.
Exceptions propagate; the CLI turns them into a ``camera`` ``error`` event.
Run under a supervisor that restarts on exit.
"""
Expand All @@ -109,7 +129,13 @@ def run(
if not model_path.is_file():
sink.emit("camera", state="error", detail=f"model archive not found: {model_path.name}")
return 3
model_digest = file_sha256(model_path)
if model_sha256 is not None and model_digest != model_sha256.strip().lower():
sink.emit("camera", state="error", detail=f"model archive SHA-256 mismatch: {model_path.name}",
model=model_path.name, model_sha256=model_digest, expected_sha256=model_sha256.strip().lower())
return 3

sink.emit("privacy", **privacy.event_fields())
dai = import_depthai()
info = discover(dai, device_id, timeout_s=discovery_timeout_s)
if info is None:
Expand All @@ -120,7 +146,8 @@ def run(
device.setMaxReconnectionAttempts(
MAX_RECONNECT_ATTEMPTS, lambda status: sink.emit("camera", state="reconnect", detail=str(status))
)
trackers = [PersonTracker(confirm_frames, gone_seconds, person_confidence, zone, person_label) for zone in zones]
trackers = [PersonTracker(confirm_frames, gone_seconds, person_confidence, zone, person_label,
stats=ConfidenceStats() if confidence_stats else None) for zone in zones]
detectors = [DepthChangeDetector(grid, baseline_frames, noisy_frac, change_frac, hold_seconds, zone)
for zone in zones]

Expand All @@ -138,7 +165,8 @@ def run(
q_depth = stereo.depth.createOutputQueue(maxSize=2, blocking=False)
p.start()
sink.emit("camera", state="up", device=info.getDeviceId(), device_model=_device_model(device),
bootloader=_bootloader(device), model=model_path.name, zones=[z.name for z in zones])
bootloader=_bootloader(device), model=model_path.name, model_sha256=model_digest,
model_sha256_checked=model_sha256 is not None, zones=[z.name for z in zones])

last_alive, frames = time.monotonic(), 0
while p.isRunning():
Expand All @@ -163,15 +191,25 @@ def run(
sink.write(ev)
del frame # reduced to cell medians above; nothing is kept
if now - last_alive >= alive_every_s:
sink.emit("alive", det_fps=round(frames / (now - last_alive), 1),
person=any(t.present for t in trackers),
occupied=[t.zone.name for t in trackers if t.present])
sink.write(alive_event(frames / (now - last_alive), trackers))
last_alive, frames = now, 0
time.sleep(0.01)
sink.emit("camera", state="pipeline_stopped")
return 1


def alive_event(det_fps: float, trackers: list[PersonTracker]) -> dict[str, Any]:
"""The heartbeat event; adds per-zone ``confidence`` statistics when the trackers collect them."""
fields: dict[str, Any] = {
"det_fps": round(det_fps, 1),
"person": any(t.present for t in trackers),
"occupied": [t.zone.name for t in trackers if t.present],
}
if any(t.stats is not None for t in trackers):
fields["confidence"] = {t.zone.name: t.stats.snapshot() for t in trackers if t.stats is not None}
return make_event("alive", **fields)


def list_devices() -> list[dict[str, Any]]:
"""Describe visible devices for ``doctor`` (raises CameraUnavailable without depthai)."""
dai = import_depthai()
Expand Down
24 changes: 23 additions & 1 deletion src/oak_events/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ def _positive_seconds(value: str) -> float:
return seconds


def _sha256_hex(value: str) -> str:
digest = value.strip().lower()
if len(digest) != 64 or any(ch not in "0123456789abcdef" for ch in digest):
raise argparse.ArgumentTypeError("must be 64 hexadecimal characters (a SHA-256 digest)")
return digest


def cmd_run(args: argparse.Namespace) -> int:
from . import camera # lazy: imports privacy first, depthai only when running

Expand All @@ -68,6 +75,8 @@ def cmd_run(args: argparse.Namespace) -> int:
person_label=args.person_label,
person_confidence=args.person_confidence,
hold_seconds=args.hold_seconds,
model_sha256=args.model_sha256,
confidence_stats=args.confidence_stats,
)
except camera.CameraUnavailable as exc:
print(f"oak-events: {exc}", file=sys.stderr)
Expand Down Expand Up @@ -138,8 +147,15 @@ def _print_doctor(report: dict) -> None:
def cmd_mcp(args: argparse.Namespace) -> int:
from . import mcp_server

extra: dict[str, object] = {}
if args.access_log is not None:
try:
extra["access_log"] = mcp_server.check_access_log_path(args.events_dir, args.access_log)
except ValueError as exc:
print(f"oak-events: bad --access-log: {exc}", file=sys.stderr)
return 2
try:
mcp_server.serve(args.events_dir, stale_after_s=args.stale_after)
mcp_server.serve(args.events_dir, stale_after_s=args.stale_after, **extra)
except RuntimeError as exc:
print(f"oak-events: {exc}", file=sys.stderr)
return 4
Expand Down Expand Up @@ -167,6 +183,10 @@ def build_parser() -> argparse.ArgumentParser:
run.add_argument("--person-confidence", type=float, default=0.5, help="minimum confidence (default 0.5)")
run.add_argument("--hold-seconds", type=float, default=60.0,
help="how long a depth change must persist before it is reported (default 60)")
run.add_argument("--model-sha256", type=_sha256_hex, metavar="HEX",
help="refuse to start unless the model archive has this SHA-256 (see docs/MODEL_CARD.md)")
run.add_argument("--confidence-stats", action="store_true",
help="add per-zone detection-confidence statistics to alive events")
run.add_argument("--quiet", action="store_true", help="do not echo events to stdout")
run.set_defaults(func=cmd_run)

Expand All @@ -178,6 +198,8 @@ def build_parser() -> argparse.ArgumentParser:
mcp.add_argument("--events-dir", required=True, help="directory written by 'run --out-dir'")
mcp.add_argument("--stale-after", type=_positive_seconds, default=900.0, metavar="SECONDS",
help="mark camera health stale after this many seconds (default 900)")
mcp.add_argument("--access-log", metavar="FILE",
help="append one JSON line per tool call to FILE (off by default; outside the events directory)")
mcp.set_defaults(func=cmd_mcp)
return parser

Expand Down
Loading
Loading