Repository navigation
Bench observer: eyes/ears fixture for self-hosted metal CI (xuss-class product face) #102
Description
Problem
Self-hosted metal CI can prove deployed (identity / FW_VERSION) and some metal I/O over USB. Xuss-class GCUs still hang product face acceptance on eyes and ears: side LEDs, IPS paint, boot riffs, short demos.
Agents already struggle there: they stop at version match + an honesty footnote, or file an issue instead of observing. A runner has no eyes/ears unless we add a second device that watches the DUT (device under test).
We want a rough product / harness spec for an observer that hangs off the same self-hosted GitHub runner as the DUT, so optional metal CI can claim instrumented product-face layers without a human at the desk — while human see/hear remains first-class for first ship.
Related: host-always / metal-later CI contract (#101, knowledge ci-host-metal.md). This issue is not host CI; it is the sensor side of the metal plane.
Job (one sentence)
A small eyes + ears peer that sits next to a practice GCU (xuss / xuss-c / …), captures light and sound during boot/demo windows, and returns machine-checkable pass/fail (+ artifacts) to the runner host.
Why xuss is the right hard example
| Product face | Why agents fail | What an observer could measure |
|---|---|---|
| Side LED chase / status color | Wrong GPIO, plate pin vs M5 face | ROI brightness / color / motion on strip |
| IPS “ready” glyph | Partial paint, wrong endian | Camera ROI vs golden thumbnail |
| Boot riff ~N seconds | Silent deploy, wrong DAC path | Mic RMS / band energy in time window |
| Desk demo loop | App parked in REPL | Repeating optical/audio signature |
Cheap device self-report (face_status over UART) is still the best first ROI on the DUT. Observer hardware covers the residual: “firmware claims ready” vs “bench actually lit/sang.”
Existing devices (likely starting points — not a buy decision)
Off-the-shelf boards already combine camera + mic; none are “silico metal CI observer” out of the box, but they are strong candidates:
| Candidate | Eyes | Ears | Notes |
|---|---|---|---|
| Espressif ESP-EYE | 2MP camera | Digital mic | Classic ESP32 vision+audio kit; mature community demos |
| ESP32-P4-EYE | 2MP + LCD | Digital mic | Camera-form factor; USB HS; stronger vision SoC; newer |
| ESP32-P4X-EYE | Camera + LCD | Mic | Related P4 eye family SKU |
| M5Stack Unit CamS3-5MP | 5MP | PDM mic | Grove/USB-C; same ecosystem as M5GO DUTs — convenient on a xuss bench |
| ESP32-CAM + external I2S mic | Camera | Add-on | Cheap; more glue; audio not always onboard |
| Runner-native USB webcam + USB mic | Host CV/audio | Host CV/audio | No second MCU; different ops model (drivers, lighting, flaky CV) |
Likely: start from ESP-EYE / P4-EYE / Unit CamS3 rather than a custom PCB. Silico may only need a thin host probe protocol + fixture geometry (where to point, dark tent, mic distance), not a new silicon product.
Also consider non-camera first: photodiode / light sensor + envelope detector on speaker line — fewer false fails than CV, worse for “IPS glyph correct.”
Rough product shape (observer GCU or spine fixture)
Not a vertical customer product. Closer to a bench instrument that silico (or a tiny public harness repo) drives.
Physical
- Mount: fixed pose relative to DUT (tripod, 3D print, or M5 stacking) so ROI is stable across jobs.
- Lighting: controlled or measured ambient; document “good” bench photo.
- Power: USB from runner host or powered hub; must survive DUT reboot brownouts.
- Isolation: observer must not share the DUT’s CDC port; second USB device (or Wi‑Fi to host).
Host API (runner talks to observer)
Non-interactive, artifact-friendly. Sketch:
observer arm --window-ms 20000 --roi leds,panel --audio on
# … host flashes DUT, resets …
observer result --json
# → { ok, leds_energy, audio_rms_peak, frames_saved, errors[] }
observer save-artifacts ./metal-obs/
Or pure USB serial knocks similar to silico inspect identity.
Accept probes (examples)
| id | Window | Pass heuristic (v0) |
|---|---|---|
boot-audio-energy |
0–20s after reset | Mic RMS above threshold for ≥T ms in band |
side-led-activity |
same | ROI mean luminance delta or hue class change |
panel-not-black |
after face ready | ROI not near-zero vs dark baseline |
optional-glyph-match |
nightly | SSIM / hash vs golden (brittle — keep optional) |
v0 should fail closed on missing observer, timeout, or saturated sensor — never silent green.
Layers (honesty)
| Layer | Who |
|---|---|
| Deployed | DUT via silico deploy/inspect |
Metal I/O / face_status |
DUT self-report |
| Instrumented product face | Observer |
| Human product face | Operator (Stage D1) — never deleted by CI |
GitHub check names must not say “product face accepted” for instrumented-only green.
Integration with self-hosted metal job
Order of work (dependencies):
- Practice GCUs: pristine main + session harness (stop contaminating xuss examples) #101 host always — lab branches get host-gate without dirtying
main. - Metal-deployed on self-hosted — wait-device, inspect, flash, version match (no observer yet).
- DUT
face_status(or equivalent knock) — forces product path exposure. - This observer — optional job step + artifacts on the same runner.
- Golden fixtures — per board profile (m5go side strip location, boot window length).
Concurrency: one DUT + one observer per runner label; lock both.
Non-goals
- Replacing human first-ship see/hear for real product ship.
- Perfect CV / music ID.
- Putting product domain songs/glyphs into silico as the moat.
- Metal CI on untrusted fork PRs.
Open questions
- Observer as silico-owned fixture firmware vs thin host scripts + commodity cam/mic?
- Same M5 ecosystem (CamS3 next to M5GO) vs Espressif eye kit vs runner webcam?
- Minimum viable: audio-only envelope first, or LED ROI first?
- Does the observer live in
tig/silicoknowledge + plate stubs, or a separatetig/bench-eye(name TBD) repo? - Calibration: per-bench golden capture committed (git-lfs?) vs runtime baseline dark frame?
Acceptance for this issue (spec spike)
- Written rough spec (this body refined) with recommended default hardware path after a hands-on spike.
- Explicit layer table (what CI may claim).
- Host API sketch stable enough for a metal-bench step.
- Decision: buy/try 1–2 candidate boards; note failures.
- Link from
ci-host-metal.mdwhen observer work starts (compound).
Related
- Practice GCU / dual surface / host-always CI: Practice GCUs: pristine main + session harness (stop contaminating xuss examples) #101
- Layered DoD / product face: root AGENTS, lexicon product face
- Optional self-hosted metal CI horizon: specs/silicov1.md beta note
Activity
tig commentedon Jul 26, 2026
Opened alongside #101 host-always work (PR #103). Metal-deployed on self-hosted should land before wiring this observer; host plane is intentionally separate.