diff --git a/docs/architecture/lib-map.md b/docs/architecture/lib-map.md index 6239378c..51e04c05 100644 --- a/docs/architecture/lib-map.md +++ b/docs/architecture/lib-map.md @@ -336,6 +336,8 @@ Every current root Python module is listed here so future additions are intentio - `bed_view.py` — the auto view of the printer bed (#303 follow-up): pure picker of the best inspection Z (share of the plate in the camera frame, reference at that Z, clearance floor, axis ceiling), the guarded Z-only move + judge + restore flow `bed_state.py status` runs (`watch.bed.auto_view`). Stdlib only; the pixels stay in `bed_state.py`. - `bed_photo.py` — the table photo of a finished print (#579): plan a lift of the bed to a viewing Z (`--lift MM` or `auto` from the camera calibration, within `bed_view.move_limits`), `bed_view.guarded_z_move`, a fresh camera frame, restore in a `finally`; dry run by default. Backs `3d photo capture`; the camera/printer are injected so tests never touch a printer. - `bed_part.py` — the pixel half of `3d printer watch NAME --job` (#723): the changed share of the plate region against the job's empty-plate baseline (homography-warped to the current Z with a camera calibration; without one the plate is FOLLOWED on the picture — a chained ECC affine transform saved with every healthy footprint, `--track`/`--save-track`, so the baseline survives the bed's drift), and how much of the part's learnt footprint is still changed / how much changed beyond it; `blind` (never "gone") when the plate is out of view, cannot be followed, or the baseline cannot stand. Stateless, read-only; decisions live in `printer_ops/jobwatch.py`. +- `print_quality.py` — the pixel half of the print-quality monitor (#733): in the plate region of a frame (640 px wide) `fleck_frac` (small bright NEUTRAL specks and strands — the top-hat of the minimum colour channel — i.e. stringing debris), `order` (energy-weighted structure-tensor coherence: how regular the bead pattern is) and `luma`; `blind` when the plate is out of view. Stateless, read-only; run by `printer_ops/jobwatch_quality.py` as a subprocess and imported by `vision_quality.py`. Decisions live in `printer_ops/quality.py`; calibration in `docs/research/print-quality-monitor.md`. +- `vision_quality.py` — the print-quality tier of `3d printer vision` (#733): `QualityTier` measures each frame (`print_quality.py`), judges it with `printer_ops/quality.py` against the print's own first layers and returns `Finding`s (`quality__`, warning/severe -> a WARNING event with the marked frame, notice -> info; never critical, never a pause); `--once` reports the absolute verdict of one frame; writes the per-print monitoring report when the print ends. - `lib/geometry/mesh_metrics.py` — pinned-convention 3D shape metrics (Chamfer, F-score@tau, Hausdorff, normal consistency, volumetric IoU) between two meshes; backs `3d metrics geometry`. - `perceptual_metrics.py` — perceptual/semantic image metrics (PSNR, LPIPS, CLIP-sim) with explicit senses and graceful degradation; backs `3d metrics perceptual`. - `metrics.py` — compatibility wrapper re-exporting `registries.metrics`. diff --git a/docs/commands/printer-vision.md b/docs/commands/printer-vision.md index 7417be04..206c9d03 100644 --- a/docs/commands/printer-vision.md +++ b/docs/commands/printer-vision.md @@ -9,7 +9,7 @@ from the device's `watch:` block in `printer-devices.yaml`). ```text 3d printer vision NAME [--interval S] [--once] [--save OUT.jpg] [--dry-run] [--min-severity LEVEL] [--snapshot URL|PATH] [--no-keepalive] [--no-npu] [--evidence DIR] [--verbose] - [--no-verify] [--no-geometry] [--scene] [--no-detach] [--model | --no-model] + [--no-verify] [--no-geometry] [--scene] [--no-detach] [--no-quality] [--model | --no-model] 3d printer vision eval --pos PATH... --neg PATH... [--every N] [--warmup N] [--interval S] [--lead-in N] [--printer NAME] [--no-verify] [--scene] [--no-detach] [--json] [--model [--model-class CLASS]] @@ -37,6 +37,11 @@ from the device's `watch:` block in `printer-devices.yaml`). - `--no-detach` — skip the part-detached check (`vision_part_detached`, below); same as `detach: false` in the device's `watch: vision:` block, which can also hold a mapping that tunes it. With `--once` the report adds `detach_gap` (0..1, null = the check abstained) and `part_detached` (gap at or above the absolute threshold). +- `--no-quality` — skip the print-quality tier (below) and its per-print monitoring report; same as + `watch.quality: false`. With `--once` the report adds a `quality` block: the numbers + (`fleck_frac`, `order`), the ABSOLUTE verdict of that one frame (`level`, `issue`, `basis`) — a single + frame has no job baseline, so it is a hint to look at the frame — and `findings` lists + `quality__`. Without the print position (Z) it says `status: blind` and why. - `--scene` — also run the bed-change detectors (`vision_foreign_object`, `vision_part_moved` — against a 3-12 min reference and, for slow drift, a 15-60 min one — and `vision_no_growth`). Off by default: they are still noisy (see [printer-vision-tuning.md](../research/printer-vision-tuning.md)). Same as `scene: true` in the @@ -95,6 +100,9 @@ from the device's `watch:` block in `printer-devices.yaml`). | `vision_part_detached` | warning, then critical | the printed part looks lifted/peeled off the plate: bare build plate shows where the near wall of the part should be (details below); on by default with the camera calibration; never pauses | | `vision_model_spaghetti` | warning | tier (b): the whole-frame model scores loose filament >= 0.5 on 3 model runs (30 s apart) in a row; a second opinion next to `vision_spaghetti`, never a pause | | `vision_model_part_detached`, `vision_model_empty_bed` | warning | tier (b), **off by default** (no skill zero-shot): only with a fitted head (`fit-head`) or a `threshold` in `watch.vision.model`; `vision_model_empty_bed` is only judged once the print is at least `min_z` (10 mm) high | +| `quality_stringing_warning`, `quality_stringing_severe` | warning | the print-quality tier: white stringing debris on the plate, against the print's own first layers (below); a WARNING event with the marked frame, once per issue and level, **never a pause** | +| `quality_texture_*`, `quality_first_layer_*` | warning | the bead pattern got lost with no debris (stripes, roughness, under/over-extrusion) / an uneven first layer (absolute floor) | +| `quality__notice` | info | the first sign of the same; only subscribers that asked for info | | `vision_foreign_object`, `vision_part_moved` | warning | only with `--scene`: something new on the bed outside the parts / a part's silhouette changed | | `vision_no_growth` | info | only with `--scene`: the newest layers of a part look empty (under- or no extrusion) | | `printer_noodle_verdict` | warning | the U1's own NPU detector logs a defect label (table below) at or above 0.83 (once per episode, 5 min quiet ends it) | @@ -336,6 +344,25 @@ an `auto_pause` event in `printer-agents/NAME/events.jsonl` and the journal. printing and logs each verdict to klippy.log; vision tails that log over HTTP Range requests. Moonraker does not expose the detector itself (`camera.detect_capture` → -32601). +## Print quality (stringing, pattern loss, first layer) + +Tiers (a) and (b) look for FAILURES. None of them knows stringing: the Obico model scored p 0.0 / q 0.0 on all +12 frames of the 2026-11-05 sheet job whose layers 2-4 were covered in white thread debris, so nothing was +flagged until the owner saw the part. The quality tier (`lib/vision_quality.py`, `lib/print_quality.py`, +`printer_ops/quality.py`) measures each frame in the plate region at the print's Z (`fleck_frac`: white +specks and strands; `order`: regularity of the bead pattern) and judges it against the print's OWN first +layers: the first frame of the print is the empty-plate reference (a part stands once a frame differs from +it; past the first layers it always does), the first 4 readings with a part are the baseline. Levels notice +< warning < severe, 2 of the last 4 readings to report one, one WARNING event with the marked frame +(`tg --photo FRAME ""` through the dispatcher) per issue and level — a second level inside the +dispatcher's cooldown folds into its `+N more` line — and **never a pause**. When the print ends +(or another starts) `/--monitor-report/` gets `report.json`, `report.md` and +`worst-frame.jpg`; one stderr line says where. It needs the camera calibration or `watch.bed.roi` for the plate +region and the print context (Z) from Moonraker. Start the supervisor at the beginning of a job: a print adopted +mid-way has no first layers to learn from. Numbers, calibration and limits: +[print-quality-monitor.md](../research/print-quality-monitor.md). The job watch +([`printer watch --job`](printer.md#print-quality-monitor)) runs the same tier on its own frames. + ## Three tiers 1. **(a) cheap checks, every frame.** The pipeline above: stale/dark camera, Obico boxes + CLIP @@ -427,6 +454,11 @@ u1: head: ~/.cache/3d-cli/vision/models/tier-b-head.json # default; from `fit-head` ``` +`watch.quality` (a sibling of `vision:`, shared with `printer watch --job`) tunes the print-quality tier: +`quality: false` turns it off, `quality: {baseline_n: 4, warning_ratio: 2.0, warning_abs: 0.035, severe_ratio: +3.5, severe_abs: 0.06, warning_drop: 0.10, first_layer_order: 0.6, ...}` any `QualityConfig` field +(`printer_ops/quality.py`); a wrong key is an error naming the valid ones. The first-layer floor (`first_layer_order`, 0 = off) is on by default only for a Snapmaker U1, the camera it was calibrated on. + The block is per device, so every Moonraker printer in the fleet (U1, K1 SE, others) gets its own camera `snapshot`, thresholds and `crop`; nothing in tier (b) is U1-specific. diff --git a/docs/commands/printer.md b/docs/commands/printer.md index d2f59d47..773930ec 100644 --- a/docs/commands/printer.md +++ b/docs/commands/printer.md @@ -979,7 +979,7 @@ alerts an agent): this one *acts*, so it needs `--yes` — you authorise a pause ``` 3d printer watch NAME --job --yes [--on-fail auto|pause|cancel] [--gcode FILE.gcode] [--out DIR] [--interval S] [--checkpoints L1@25%,L1@50%,L1@100%,25%,50%,100%] [--dry-run | --tg-dry-run] - [--fixture SCENARIO.json] [--duration S] [--wait-start S] + [--fixture SCENARIO.json] [--duration S] [--wait-start S] [--no-quality] ``` Start it right after `3d printer start NAME FILE --yes` (or give `start` [`--watch`](#start---watch)); @@ -999,6 +999,8 @@ quarter, half and all of the first layer — the first-layer marks need `--gcode - *Signals.* The same Moonraker events as the always-on watch (filament runout, heater loss, bed off, exception, G-code error, homing lost), and the printer's own defect detection (`noodle` verdicts with p >= 0.8, read from klippy.log). +- *Print quality* ([below](#print-quality-monitor)). Is the print GOOD, not just still there: stringing + debris, a lost bead pattern (stripes, roughness), an uneven first layer. Warnings only, never a stop. **Actions on a critical finding**, in this order, each step runs even if an earlier one failed: @@ -1012,6 +1014,40 @@ quarter, half and all of the first layer — the first-layer marks need `--gcode A warning (part moved, nothing seen, a heater drift) is only a tg message with the frame; nothing is touched and nobody is called. + +**The print-quality monitor** (#733; `lib/print_quality.py` measures, `printer_ops/quality.py` decides) runs +on the same frame as the part check, in the plate region at the print's own Z. Why it exists: on 2026-11-05 a +10 h sheet of small tags stringed from layer 2 on — white thread debris all over the part — while the part +check (it only asks "is the part where it was") and the spaghetti model (no stringing class: p 0.0 on all +12 frames) stayed silent. + +| number | what it is | +|---|---| +| `fleck_frac` | share of the plate region covered by small bright **neutral** specks and strands (the top-hat of the minimum colour channel: high only where R, G and B are all high, i.e. white debris — not beige part, not golden specular grain) | +| `order` | structure-tensor coherence: 1 = every neighbourhood has one dominant bead direction, lower = debris, webs, broken or striped lines | + +The judge compares them with the **job's own first layers**: the first 4 readings with the part on the plate +(bed_part says so) are the baseline, never judged against anything but a floor; a clean reading drifts it a +little (bounded at 1.3x debris / 0.04 order), a bad one never does. Issues: `stringing` (debris against the +baseline AND an absolute level), `texture` (the pattern got lost with no debris: stripes, roughness, +under-/over-extrusion), `first_layer` (an absolute floor on `order` while layer <= 3; ON by default only for the U1, whose camera it was calibrated on — `first_layer_order: 0.6` in `watch.quality` turns it on elsewhere, a K1 SE view of a part reads 0.5 normally). Levels: `notice` +(report only) < `warning` < `severe`; a level is reported when 2 of the last 4 readings reach it (3 of 4 +for severe), it is latched and clears after 6 clean readings. A debris trend (ratio to the baseline gained +per 10 min) lifts a notice to a warning. + +A quality finding is **never critical**: one `tg --photo FRAME ""` per issue and level (the frame has the debris marked in red), nobody is rung, the job is not +touched. Part gone, spaghetti and the noodle verdict still act as above. `--no-quality`, or +`.watch.quality: false`, turns it off; `.watch.quality: {baseline_n: 4, warning_ratio: 2.0, +warning_abs: 0.035, ...}` tunes any `QualityConfig` field (a wrong key is an error naming the valid ones). + +**The monitoring report.** When the watch ends — the job completed, a lost print stopped it, or `--duration` +ran out — `DIR/-NAME-monitor-report/` holds `report.json` (readings, counts per severity and per issue, +the alerts that were sent, the baseline, the worst reading, the timeline), `report.md` (the same for a human) +and `worst-frame.jpg`. A job with nothing wrong still gets one ("0 warnings in 41 judged readings"): silence +then means *checked and fine*, not *never ran*. Calibration, limits and the numbers on the real frames: +[print-quality-monitor.md](../research/print-quality-monitor.md). + **`--on-fail`.** `auto` (default): CANCEL when the part is gone, or on a camera finding in the first layers (layer <= 3); PAUSE for everything else (runout, heater, later spaghetti: resumable). `pause` / `cancel`: always that. Defaults can also be set in `.watch.job` (`on_fail`, diff --git a/docs/research/print-quality-monitor.md b/docs/research/print-quality-monitor.md new file mode 100644 index 00000000..40b5be6c --- /dev/null +++ b/docs/research/print-quality-monitor.md @@ -0,0 +1,119 @@ +# Print-quality monitor: why stringing went unnoticed, and the tier that sees it + +Issue [#733](https://github.com/alex-mextner/3d-cli/issues/733). Code: `lib/print_quality.py` (pixels), +`lib/printer_ops/quality.py` (decisions), `quality_report.py`, `jobwatch_quality.py` (job watch), +`lib/vision_quality.py` (printer vision). User docs: [printer.md](../commands/printer.md#print-quality-monitor), +[printer-vision.md](../commands/printer-vision.md#print-quality-stringing-pattern-loss-first-layer). + +## What happened (2026-11-05) + +A 10 h sheet of small tags (Snapmaker U1, 270 x 270 mm, beige PETG "Wooden", 232 °C, bed 80 °C, many islands) +stringed from layer 2 on: white thread debris over the layers, "everything in stripes" on the real part. The part +stayed attached, so nothing was lost — and nothing flagged it. `3d printer vision u1 --once` returned +`findings: []`, `p 0.0`, `q 0.0`, `boxes: []`, `part_detached: false`, `trend 0.0` on every one of 12 frames, and the +job watch (`printer watch u1 --job`, PR #727) stayed silent. The orchestrator noticed the stringing in the chat +only and did not send a `tg --photo` (process gap: a visible defect on the camera is an owner alert, not a chat +remark — the monitor now does exactly that). + +## Why nothing fired (measured on the 12 frames) + +| component | what it asks | on the 12 frames | +|---|---|---| +| Obico spaghetti model (stage 1), `vision_detect.py` | loose filament piles (class "failure") | **0 boxes above CONF_MIN 0.08** on every frame; `p 0.0`. `masked 0`: nothing was dropped by a mask — the model never fired. | +| CLIP verifier (stage 2) | confirms the boxes stage 1 kept | never ran: `q = 0.0` is the **empty-sum** of zero boxes (`verified_score([])`), not a measurement | +| trend tracker | EWM of `q` minus the lifetime mean | 0.0 by construction | +| tier (b), whole-frame CLIP | zero-shot prompts: spaghetti / part detached / empty plate | off by default unless the weights are cached; its prompts have no stringing class; "stringing" is documented as *not an alert* in [printer-vision-tuning.md](printer-vision-tuning.md) | +| part-detached / scene detectors | geometry of the part's silhouette | stringing does not change the silhouette | +| job watch camera (`bed_part.py`) | is the changed footprint still there? | `changed` 0.23-0.52, a part that stands: debris on top of a part is not a missing part. (On the real frames its footprint compare also reads the toolhead/light changes as `partial` -> a `part_moved` WARNING; it is not about stringing.) | +| `z` from Moonraker | used by the camera masks (bed volume at Z) | yes, `ContextTap` supplies it; it only feeds masks, so it could not help | + +Root cause: **no component has a defect class for print *quality*.** The stack detects failures (spaghetti, a part +off the plate, a foreign object). Stringing, a lost bead pattern and an uneven first layer are not failures of +the job, so every detector is correct to stay quiet — and the owner learns of them from the finished part. + +## The quality tier + +Per frame, inside the plate region at the print's Z (camera calibration or `watch.bed.roi`), at 640 px width: + +* `fleck_frac` — share of the region covered by small (5-130 px) bright **neutral** specks and strands: + white top-hat (17 px ellipse) of the **minimum colour channel**, divided by the local mean brightness, over 0.18. + The minimum channel is high only where R, G and B are all high (white debris); beige part, golden specular grain and the + blue-lit toolhead shadow stay low. Chosen by a grid search over channel / kernel / threshold / area against the + 12 frames; plain edge density, gray top-hat and colour (R/B) statistics were tried first and do **not** separate + (auto-white-balance swings R/B between 0.6 and 1.7 from frame to frame; clean first-layer bars are as edgy as debris). +* `order` — energy-weighted structure-tensor coherence (window sigma 5.5 px): how regular the bead pattern is. + Clean parallel beads 0.83-0.85; debris, webs and broken lines lower it; the empty plate and the stringy phone + photos read 0.3-0.7. + +The judge (`quality.py`) compares them with the **job's own first layers** (the first 4 readings with the part on the +plate), with persistence (2 of the last 4 readings, 3 of 4 for severe), one alert per issue and level, a bounded +baseline drift, a debris trend, and an absolute floor on `order` for first layers. Never CRITICAL: a quality finding +is a `tg --photo` WARNING with the frame, never a pause or a cancel. + +### Calibration table (full-size frames of the job, chronological; presence from `bed_part` against f1; the layer column is the replay's assumption: f1-f5 first layer, f6 layer 2, f7-f9 layer 3, f10-f12 layer 4, readings 15 s apart on layer 1 and 60 s after) + +| frame | layer | before: vision `--once` (p / q / findings) | before: job watch | `fleck_frac` | `order` | after: this reading | after: monitor verdict | +|---|---|---|---|---|---|---|---| +| f1 | empty plate | 0.0 / 0.0 / [] | empty (changed 0.00) | 0.089 | 0.68 | not judged (no part) | nothing | +| f2 | 1 | 0.0 / 0.0 / [] | seen | 0.026 | 0.83 | baseline | nothing | +| f3 | 1 | 0.0 / 0.0 / [] | seen | 0.011 | 0.84 | baseline | nothing | +| f4 | 1 | 0.0 / 0.0 / [] | seen | 0.022 | 0.83 | baseline | nothing | +| f5 | 1 | 0.0 / 0.0 / [] | seen | 0.015 | 0.85 | baseline (median 0.018 / 0.84) | nothing | +| f6 | 2 | 0.0 / 0.0 / [] | seen | 0.080 | 0.80 | severe (single reading) | nothing yet (1 of 2 needed) | +| f7 | 3 | 0.0 / 0.0 / [] | armed | 0.085 | 0.74 | severe | **WARNING `quality_stringing_warning`** | +| f8 | 3 | 0.0 / 0.0 / [] | present | 0.073 | 0.73 | severe | **SEVERE `quality_stringing_severe`** (second message) | +| f9 | 3 | 0.0 / 0.0 / [] | present | 0.057 | 0.68 | warning | severe (latched) | +| f10 | 4 | 0.0 / 0.0 / [] | present | 0.073 | 0.68 | severe | severe (latched) | +| f11 | 4 | 0.0 / 0.0 / [] | present | 0.039 | 0.66 | warning | severe (latched) | +| f12 | 4 | 0.0 / 0.0 / [] | present | 0.041 | 0.70 | warning | severe (latched) | + +f1-f5 raise nothing; every frame from f7 on is at least WARNING (f6, the first strings, is one reading too few to +alert: a single frame is never a finding). `vision --once` on one frame has no job history and reports the +**absolute** verdict: f5 none, f7-f10 `quality_stringing_severe`/`warning`, f11-f12 `warning`; above the first layers +only, because the bare plate sparkles (f1 reads 0.089). + +Phone photos of the failed first attempt (whole-frame region, no camera calibration): `599472757` order 0.33, +`599472758` 0.42, the coupon `599472764` 0.47 — all under the first-layer floor 0.6 against 0.83+ for a clean first +layer, so they would read `first_layer` warning/severe. The coupon photos' blob at the end and the bowed frame are +not separate measurements. + +### Limits (read before trusting it) + +* **One camera.** The constants are calibrated on 12 frames of one U1 view and 4 phone photos. Margins on the + weakest frames are real but small (f11/f12 are 2.1-2.2x the first-layer baseline against a warning ratio of 2.0). + JPEG recompression at 640 px moves `fleck_frac` by up to 30 % (the test fixtures use the 640 px copies). Another + camera/lens/light needs its own look; every threshold is a `watch.quality` key. +* **Texture, under-/over-extrusion, first-layer gaps, non-squished beads, bowed corners** are measured only through + `order` (pattern regularity) and, for a lifted part, the existing `part_moved` (footprint). No frame of a + *known* under-extruded layer exists, so `texture` is calibrated for "no false alarm on clean frames", not for recall. + A bowed corner is not told apart from other disorder on one frame. +* **The first-layer floor is per camera.** An absolute `order` floor (0.6) separates the U1's clean first layer (0.83+) from + the stringy phone photos (0.33-0.53), but a K1 SE view of a pasted part reads 0.5 normally (the job-watch fixtures of PR #727), so + the floor is on by default only for the Snapmaker U1 profile (`lib/data/vision/quality_profiles.json`) and a `watch.quality` key elsewhere. +* **Baseline = the job's first layers.** A job whose first layers are already stringy teaches the monitor that this + is normal; the absolute levels and the first-layer floor still apply. A supervisor started mid-print learns from + what it sees then. +* **Toolhead over the plate** removes part of the region and moves both numbers; persistence (2 of 4) absorbs most + of it. +* The job watch's part check (PR #727) produced a `part_moved` WARNING of its own on these frames because the + toolhead and the light differ from frame to frame; that is not part of this change. + +## Recommendations to prevent the stringing itself (hypotheses, NOT measured on the printer) + +None of this was tested on the U1; each is a first thing to try for beige PETG at 232 °C with many islands. + +1. **Temperature.** Lower the nozzle 5-10 °C (to 222-227 °C) or run a temperature tower at 215-235 °C; keep the + bed at 80 °C. Stringing grows steeply with temperature; 232 °C is at the hot end for PETG. +2. **Retraction and wipe.** Raise retraction distance/speed in small steps (direct drive: ~0.8-1.5 mm at 30-40 mm/s; + add 0.2 mm at a time), enable wipe on retract and "wipe before retract"-style small coasting, and avoid zero-length + retractions on short hops; a retract test (one tower of two thin pillars) shows the effect in minutes. +3. **Travel strategy.** Enable *avoid crossing perimeters/walls* and travel inside already printed areas, raise the + minimum travel distance that triggers a retraction less than the island gaps, and use *z-hop only where needed* + (a hop with a hot, oozing nozzle drags a strand across the part). +4. **Fewer islands, fewer travels.** Order the objects/tags so the nozzle finishes a region before leaving it, group + them into fewer, bigger connected islands (join tags with a thin bridge or a shared frame so one wall run covers many), + and print in one pass per layer area instead of hopping between 100+ separate islands each layer. +5. **Wet or hot? Diagnose in this order.** Wet PETG pops, hisses and strings with fine bubbly "hair": dry it + (65 °C for 4-6 h) and repeat the same file — if the debris is gone it was moisture, not settings. If dry filament + still strings on travel moves, it is retraction/temperature (1-4). Change one variable per test and look at the + same camera view; the monitor's `fleck_frac` is a cheap number to compare between runs. diff --git a/lib/commands/printer.py b/lib/commands/printer.py index d88614ce..758076a4 100644 --- a/lib/commands/printer.py +++ b/lib/commands/printer.py @@ -89,7 +89,10 @@ CANCEL (--on-fail), save the frames + klippy lines to print/runs/, `tg call me --say`, tg with the frame. Start it right after `printer start`. Never moves an axis; --dry-run, - --fixture (see `3d printer watch --job --help`) + --fixture (see `3d printer watch --job --help`). Also the + PRINT-QUALITY monitor (stringing debris, lost bead pattern, + uneven first layer): tg --photo warnings only, never a stop, + and a monitor-report/ folder at the end; --no-quality supervise status|install|uninstall|quiet [NAME...|--all] the ALWAYS-ON supervision: one watch + one vision systemd user unit per owned printer from the fleet registry, running without a login, waiting @@ -97,7 +100,8 @@ when a printing/answering printer is unsupervised (see `3d printer supervise --help`) vision NAME [options] camera failure detection (spaghetti, detached part, stale/dark camera) - into the same event sinks as watch; also + and print quality (stringing, lost bead pattern, uneven first layer: + warnings only, --no-quality) into the same event sinks as watch; also `vision eval --pos P.. --neg N..`, `vision record NAME DIR [--timeout S]`, `vision fetch-model [--tier a|b|all]`, `vision fit-head ..`, `vision doctor` (read-only: does any service run --dry-run?) diff --git a/lib/data/vision/quality_profiles.json b/lib/data/vision/quality_profiles.json new file mode 100644 index 00000000..cd955c74 --- /dev/null +++ b/lib/data/vision/quality_profiles.json @@ -0,0 +1,4 @@ +{ + "_doc": "Print-quality monitor (lib/printer_ops/quality.py): what was CALIBRATED beyond the generic QualityConfig defaults, per printer model. A model matches when its name, lowercased and stripped to letters and digits, ends with the key (Snapmaker U1 -> ...u1). Same keys as `.watch.quality`; the printer's own watch.quality wins. See docs/research/print-quality-monitor.md.", + "u1": {"first_layer_order": 0.6} +} diff --git a/lib/print_quality.py b/lib/print_quality.py new file mode 100644 index 00000000..c6b5befd --- /dev/null +++ b/lib/print_quality.py @@ -0,0 +1,370 @@ +"""print_quality.py — how good does the print look? The pixel half of the print-quality monitor. + +Run through cli.pyrun with numpy + OpenCV (BED_DEPS) by `3d printer watch NAME --job` (once per camera check), +and imported by vision_quality.py inside `3d printer vision` (the frame is already decoded there): + + print_quality.py check NAME --frame F.jpg --z Z [--roi x,y;x,y;...] [--annotated OUT.jpg] [--json] + +It measures; it does not decide. The decision (notice / warning / severe, per issue, against the job's own +first-layer baseline, with persistence and one alert per level) is the pure state machine of +printer_ops/quality.py, so this tool stays stateless: a frame in, numbers out. + +WHAT IS MEASURED (all inside the PLATE region of the frame — the plate surface at the print's own Z from the +camera calibration, a `watch.bed.roi` polygon, `--roi`, or the whole frame; frames are resized to PROC_W = 640 +px wide, so every pixel constant below is "px at 640", and the same frame at any camera resolution reads the same): + `fleck_frac` share of the region covered by small bright NEUTRAL specks and strands: the top-hat (SE + TOPHAT_K px) of the minimum colour channel, divided by the local mean brightness, above + FLECK_THR, connected pieces of FLECK_AMIN..FLECK_AMAX px. The minimum channel is high only + where ALL of R, G and B are high, i.e. on white: stringing debris, loose strands, blobs of + natural/white filament. The beige part itself, golden specular grain (blue is low) and the + blue-lit toolhead shadow do not reach it. `fleck_per10k` is the same as a count. + `order` how regular the surface pattern is: the structure-tensor coherence (1 = every neighbourhood has + ONE dominant line direction, 0 = isotropic mush), averaged with gradient energy as weights. Clean + parallel beads read 0.8+; loose debris, stringing webs, rough or striped-and-broken lines pull it + down; an empty plate is isotropic. It is meaningful as a CHANGE against the same job's own first + layers (the decision side does exactly that), and as an absolute floor for a first layer. + `luma` mean brightness of the region (the exposure the numbers were taken at). + +CALIBRATION (docs/research/print-quality-monitor.md): 12 U1 frames of one 10 h job (f1 empty plate, f2-f5 clean +first layer, f6 first strings, f7-f12 heavy stringing debris on layers 2-4) and 4 phone photos of the failed +first attempt. Clean f2-f5: fleck_frac 0.011-0.025, order 0.83-0.85; f7-f12: fleck_frac 0.039-0.087, order +0.63-0.72. The constants are for ONE camera view family (a low oblique plate view); another camera needs its own +calibration run (see the research doc) before its thresholds are trusted. + +BLIND, NOT CLEAN: no region (the plate is not in the frame), a region under MIN_REGION_PX, or an undecodable +frame is `status: blind` with the reason and no numbers: a camera that cannot see must never read as "good". + +INVARIANTS: read-only (files only; no printer, no network); numpy/OpenCV imported lazily. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path +from typing import TYPE_CHECKING, Any + +LIB = Path(__file__).resolve().parent +if str(LIB) not in sys.path: + sys.path.insert(0, str(LIB)) + +if TYPE_CHECKING: + import numpy as np + +PROC_W = ( + 640 # processing width: every constant below is in pixels of a PROC_W-wide frame +) +REGION_ERODE_PX = ( + 7 # the region is eroded by this much: its rim is the plate edge, not the print +) +MIN_REGION_PX = 4000 # under this the plate is too small in the frame to judge (640x360 frame: ~1.7 %) +TOPHAT_K = 17 +FLECK_THR = 0.18 +FLECK_AMIN = 5 +FLECK_AMAX = 130 +MIN_CHANNEL_SIGMA = 0.55 +LOCAL_MEAN_SIGMA = 8.0 +ORDER_GRAD_SIGMA = 0.7 +ORDER_WIN_SIGMA = 5.5 +ANNOTATE_RED = (40, 40, 255) + +USAGE = """print_quality.py check NAME --frame F.jpg --z Z [--roi "x,y;x,y;..."] [--annotated OUT.jpg] [--json] + Measure ONE frame of a running print: fleck_frac (white specks/strands = stringing debris), order (how regular + the bead pattern is), luma, inside the plate region at Z (camera calibration of NAME, a watch.bed.roi polygon, + --roi with normalised 0..1 points, or the whole frame with `--roi whole`). Prints one JSON object: + {"status": "ok", "fleck_frac": .., "fleck_per10k": .., "order": .., "luma": .., "region_px": .., ...} or + {"status": "blind", "reason": ".."}. It decides nothing: printer_ops/quality.py compares it with the job's own + first layers.""" + + +def blind(reason: str) -> dict[str, Any]: + return {"status": "blind", "reason": reason} + + +# ---- decode and region ------------------------------------------------------------------------ + + +def decode(data: bytes) -> np.ndarray: + """BGR frame PROC_W wide (INTER_AREA down from anything wider; a narrower frame is used as it is).""" + import cv2 + import numpy as np + + img = cv2.imdecode(np.frombuffer(data.rstrip(b"\x00"), np.uint8), cv2.IMREAD_COLOR) + if img is None: + raise ValueError("not a decodable image") + return prepare(img) + + +def prepare(img: np.ndarray) -> np.ndarray: + """A decoded BGR frame at the processing width (INTER_AREA down from anything wider).""" + import cv2 + import numpy as np + + if img.shape[1] <= PROC_W: + return img + h = max(1, round(img.shape[0] * PROC_W / img.shape[1])) + return np.asarray(cv2.resize(img, (PROC_W, h), interpolation=cv2.INTER_AREA)) + + +def parse_roi(text: str) -> list[tuple[float, float]]: + """`x,y;x,y;...` (normalised 0..1) -> points. ValueError names the bad part.""" + pts = [] + for item in (x for x in text.split(";") if x.strip()): + x, y = (float(v) for v in item.split(",")) + if not (0.0 <= x <= 1.0 and 0.0 <= y <= 1.0): + raise ValueError(f"--roi point {item!r} is outside 0..1") + pts.append((x, y)) + if len(pts) < 3: + raise ValueError("--roi needs at least 3 points (or `whole`)") + return pts + + +def polygon_region( + points: list[tuple[float, float]], shape: tuple[int, int] +) -> np.ndarray: + import cv2 + import numpy as np + + h, w = shape + mask = np.zeros((h, w), np.uint8) + pts = np.array( + [[round(x * (w - 1)), round(y * (h - 1))] for x, y in points], np.int32 + ) + cv2.fillPoly(mask, [pts], 1) + return mask.astype(bool) + + +def plate_region(name: str, z: float, shape: tuple[int, int]) -> tuple[np.ndarray, str]: + """(the plate surface at Z of printer NAME in a frame of `shape`, where it came from).""" + from printer_ops.devices import get_device + + return device_region( + get_device(name), z, shape + ), f"plate surface of {name} at Z {z:g}" + + +def device_region(device: Any, z: float, shape: tuple[int, int]) -> np.ndarray: + """The plate surface at Z of a configured printer (camera calibration or `watch.bed.roi`).""" + import bed_state as B + import numpy as np + + _, surface, _ = B.build_masks(device, B.bed_config(device), z, shape) + return np.asarray(surface) + + +def shrink(region: np.ndarray) -> np.ndarray: + import cv2 + import numpy as np + + k = np.ones((2 * (REGION_ERODE_PX // 2) + 1,) * 2, np.uint8) + return np.asarray(cv2.erode(region.astype(np.uint8), k) > 0) + + +# ---- the measures ----------------------------------------------------------------------------- + + +def fleck_map(img: np.ndarray) -> np.ndarray: + """Per pixel: how much brighter the WHITE-ness (min channel) is than the local mean brightness.""" + import cv2 + import numpy as np + + f = img.astype(np.float32) + whiteness = cv2.GaussianBlur(f.min(axis=2), (0, 0), MIN_CHANNEL_SIGMA) + local = cv2.GaussianBlur(f.mean(axis=2), (0, 0), LOCAL_MEAN_SIGMA) + 10.0 + se = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (TOPHAT_K, TOPHAT_K)) + return np.asarray(cv2.morphologyEx(whiteness, cv2.MORPH_TOPHAT, se) / local) + + +def fleck_pieces(top: np.ndarray, region: np.ndarray) -> tuple[np.ndarray, np.ndarray]: + """(labels, kept-piece mask): connected pieces of FLECK_AMIN..FLECK_AMAX px over the threshold.""" + import cv2 + import numpy as np + + over = ((top > FLECK_THR) & region).astype(np.uint8) + n, labels, stats, _ = cv2.connectedComponentsWithStats(over, connectivity=8) + area = stats[:, cv2.CC_STAT_AREA] + keep = (area >= FLECK_AMIN) & (area <= FLECK_AMAX) + keep[0] = False + return np.asarray(labels), np.asarray(keep[labels]) if n else np.zeros_like(region) + + +def order_of(img: np.ndarray, region: np.ndarray) -> float: + """Energy-weighted structure-tensor coherence inside the region (0..1).""" + import cv2 + import numpy as np + + g = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY).astype(np.float32) + g /= max(float(g[region].mean()), 1.0) + b = cv2.GaussianBlur(g, (0, 0), ORDER_GRAD_SIGMA) + gx, gy = cv2.Sobel(b, cv2.CV_32F, 1, 0), cv2.Sobel(b, cv2.CV_32F, 0, 1) + jxx = cv2.GaussianBlur(gx * gx, (0, 0), ORDER_WIN_SIGMA) + jyy = cv2.GaussianBlur(gy * gy, (0, 0), ORDER_WIN_SIGMA) + jxy = cv2.GaussianBlur(gx * gy, (0, 0), ORDER_WIN_SIGMA) + coherence = np.sqrt((jxx - jyy) ** 2 + 4 * jxy**2) / (jxx + jyy + 1e-6) + weight = (gx * gx + gy * gy)[region] + total = float(weight.sum()) + return float((coherence[region] * weight).sum() / total) if total > 0 else 0.0 + + +def measure( + img: np.ndarray, region: np.ndarray, source: str = "" +) -> tuple[dict[str, Any], np.ndarray | None]: + """(numbers, the mask of the fleck pixels) for a PROC_W-wide frame and its (already shrunk) region.""" + import cv2 + + px = int(region.sum()) + if px < MIN_REGION_PX: + return blind( + f"the plate covers {px} px of the {img.shape[1]}x{img.shape[0]} frame, under {MIN_REGION_PX}" + ), None + top = fleck_map(img) + _, flecks = fleck_pieces(top, region) + count = cv2.connectedComponents(flecks.astype("uint8"), connectivity=8)[0] - 1 + gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) + out: dict[str, Any] = { + "status": "ok", + "region_px": px, + "region_share": round(px / (img.shape[0] * img.shape[1]), 4), + "fleck_frac": round(float(flecks.sum()) / px, 4), + "fleck_per10k": round(count * 1e4 / px, 2), + "order": round(order_of(img, region), 4), + "luma": round(float(gray[region].mean()), 1), + } + if source: + out["region_source"] = source + return out, flecks + + +def annotated_jpeg( + img: np.ndarray, region: np.ndarray, flecks: np.ndarray | None, label: str +) -> bytes: + """The frame with the region outline and the fleck pixels in red, `label` on top: the evidence picture.""" + import cv2 + + vis = img.copy() + contours, _ = cv2.findContours( + region.astype("uint8"), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE + ) + cv2.drawContours(vis, contours, -1, (0, 200, 0), 1) + if flecks is not None: + vis[flecks] = ANNOTATE_RED + cv2.putText( + vis, label, (8, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.55, (0, 0, 0), 3, cv2.LINE_AA + ) + cv2.putText( + vis, + label, + (8, 20), + cv2.FONT_HERSHEY_SIMPLEX, + 0.55, + (255, 255, 255), + 1, + cv2.LINE_AA, + ) + ok, buf = cv2.imencode(".jpg", vis, [cv2.IMWRITE_JPEG_QUALITY, 85]) + if not ok: + raise ValueError("could not encode the annotated frame") + return bytes(buf) + + +def annotate( + img: np.ndarray, + region: np.ndarray, + flecks: np.ndarray | None, + label: str, + out: Path, +) -> None: + """`annotated_jpeg` written to `out`.""" + out.parent.mkdir(parents=True, exist_ok=True) + out.write_bytes(annotated_jpeg(img, region, flecks, label)) + + +def quality_numbers( + jpeg: bytes, + *, + name: str | None, + z: float, + roi: list[tuple[float, float]] | None = None, + annotated: Path | None = None, + label: str = "", +) -> dict[str, Any]: + """The numbers of one frame. Region: `roi` when given, else printer `name`'s plate surface at Z, else the + whole frame. A frame that cannot be decoded or a plate that is out of view is a `blind` answer.""" + try: + img = decode(jpeg) + except ValueError as exc: + return blind(str(exc)) + shape = (img.shape[0], img.shape[1]) + if roi is not None: + region, source = polygon_region(roi, shape), "--roi polygon" + elif name is not None: + region, source = plate_region(name, z, shape) + else: + import numpy as np + + region, source = np.ones(shape, bool), "whole frame" + numbers, flecks = measure(img, shrink(region), source) + if annotated is not None and numbers["status"] == "ok": + annotate( + img, + shrink(region), + flecks, + label or f"fleck {numbers['fleck_frac']:.3f} order {numbers['order']:.2f}", + annotated, + ) + numbers["annotated"] = str(annotated) + return numbers + + +# ---- command line ----------------------------------------------------------------------------- + + +def build_parser() -> argparse.ArgumentParser: + ap = argparse.ArgumentParser(prog="print_quality.py", add_help=False) + ap.add_argument("sub", choices=("check",)) + ap.add_argument("name") + ap.add_argument("--frame", required=True) + ap.add_argument("--z", type=float, default=0.0) + ap.add_argument("--roi") + ap.add_argument("--annotated") + ap.add_argument("--label", default="") + ap.add_argument("--json", action="store_true") + ap.add_argument("-h", "--help", action="store_true") + return ap + + +def run_check(args: argparse.Namespace) -> dict[str, Any]: + whole = args.roi == "whole" + roi = None if args.roi is None or whole else parse_roi(args.roi) + return quality_numbers( + Path(args.frame).read_bytes(), + name=None if whole else args.name, + z=args.z, + roi=roi, + annotated=Path(args.annotated) if args.annotated else None, + label=args.label, + ) + + +def main(argv: list[str]) -> int: + if not argv or argv[0] in ("-h", "--help", "help"): + print(USAGE) + return 0 if argv else 1 + args = build_parser().parse_args(argv) + if args.help: + print(USAGE) + return 0 + out = run_check(args) + print(json.dumps(out) if args.json else json.dumps(out, indent=1)) + return 0 + + +if __name__ == "__main__": + from errors import ThreeDError + + try: + sys.exit(main(sys.argv[1:])) + except ThreeDError as exc: + print(exc.render(), file=sys.stderr) + sys.exit(exc.exit_code) diff --git a/lib/printer_ops/jobwatch_quality.py b/lib/printer_ops/jobwatch_quality.py new file mode 100644 index 00000000..79b40615 --- /dev/null +++ b/lib/printer_ops/jobwatch_quality.py @@ -0,0 +1,175 @@ +"""jobwatch_quality.py — the print-quality monitor inside `3d printer watch NAME --job`. + +One camera check of the job watch (jobwatch_run.JobWatch.check) already has the frame and the bed_part reading +(is the part on the plate). `QualityWatch.check` takes the same frame, asks lib/print_quality.py for its numbers +(a subprocess with numpy + OpenCV, like bed_part.py), feeds the pure `QualityJudge` (printer_ops/quality.py) and +returns the NON-critical jobwatch Findings it owes the owner: one per issue and level (warning, severe). A +`notice` is not a finding: it only counts in the report. A quality finding is never critical, so `decide` +answers `alert` for it: the owner gets `tg --photo FRAME ""` and the job is not touched. + +`finish` writes the monitoring report (quality_report.py) when the watch ends — also when it ends because a part +was lost: what the monitor saw until then is part of the evidence. + +INVARIANTS: read-only like the rest of the camera side; a tool failure is a `blind` reading (counted in the report), +never an exception that stops the watch. +""" + +from __future__ import annotations + +import json +import shutil +import subprocess +from collections.abc import Callable +from pathlib import Path +from typing import Any + +from printer_ops.jobwatch import Finding, JobConfig, Observation +from printer_ops.quality import ( + NOTICE, + QualityConfig, + QualityFinding, + QualityJudge, + QualityReading, + QualityVerdict, +) +from printer_ops.quality_report import summary_line, write_report + +QualityTool = Callable[[list[str]], dict[str, Any]] +ROTATE = 4 # annotated frames kept (the evidence copies live in the alert's own folder) + + +def run_quality_tool(argv: list[str]) -> dict[str, Any]: + """print_quality.py through cli.pyrun (numpy + OpenCV): its JSON answer, or a blind reading saying why not.""" + from cli.pyrun import tool_argv + from printer_ops.bed import BED_DEPS + + try: + r = subprocess.run( + tool_argv(BED_DEPS, "print_quality.py", argv), + capture_output=True, + text=True, + timeout=120, + ) + if r.returncode != 0: + tail = (r.stderr or "")[-200:] + return { + "status": "blind", + "reason": f"the quality tool failed (exit {r.returncode}): {tail}", + } + return dict(json.loads(r.stdout)) + except (OSError, subprocess.SubprocessError, ValueError) as exc: + return {"status": "blind", "reason": f"the quality tool failed: {exc}"} + + +def part_present(o: Observation, cfg: JobConfig) -> bool: + """Something stands on the plate: bed_part says so (the same test as the part's `seen`); without a bed_part + reading (blind) the print is taken to have a part from layer 2 on.""" + part = o.part + if part is not None and part.status == "ok": + return part.changed >= cfg.present_min and part.blob_px >= cfg.present_blob_px + return (o.layer or 0) >= 2 + + +class QualityWatch: + def __init__( + self, + name: str, + run_dir: Path, + tool: QualityTool, + cfg: QualityConfig, + job_cfg: JobConfig, + job: str, + ) -> None: + self.name, self.dir, self.tool, self.job_cfg = name, run_dir, tool, job_cfg + self.judge = QualityJudge(cfg, printer=name, job=job) + self.n = 0 + self.last: QualityReading | None = None + self.last_verdict: QualityVerdict | None = None + + # -- one reading + + def check(self, jpeg: bytes, o: Observation) -> list[Finding]: + """Measure the frame, judge it, and return the findings to alert (warning / severe only).""" + if self.last is not None and o.now - self.last.now < self.judge.cfg.min_gap_s: + self.last_verdict = None # a recheck of the part check, seconds after the last look: nothing new to say + return [] + reading = self.read(jpeg, o) + v = self.judge.feed(reading) + self.last, self.last_verdict = reading, v + if v.worst: + self.pin(reading) + return [self.finding(f) for f in v.new if f.level != NOTICE] + + def read(self, jpeg: bytes, o: Observation) -> QualityReading: + self.n += 1 + frame = self.dir / "quality-frame.jpg" + annotated = self.dir / f"quality-annotated-{self.n % ROTATE}.jpg" + frame.write_bytes(jpeg) + argv = [ + "check", + self.name, + "--frame", + str(frame), + "--z", + str(o.z if o.z is not None else 0.0), + "--annotated", + str(annotated), + "--json", + ] + numbers = self.tool(argv) + return QualityReading( + o.now, + numbers, + layer=o.layer, + z=o.z, + progress=o.progress, + present=part_present(o, self.job_cfg), + frame=str(frame), + annotated=str(annotated) if numbers.get("annotated") else None, + ) + + def pin(self, r: QualityReading) -> None: + """Keep the worst frame: the rotating files are overwritten by the next readings.""" + src = r.annotated or r.frame + if src and Path(src).is_file(): + dst = self.dir / "quality-worst.jpg" + shutil.copyfile(src, dst) + self.judge.pin_worst(str(dst)) + + @staticmethod + def finding(f: QualityFinding) -> Finding: + return Finding(f.kind, False, f.text, dict(f.data)) + + def alert_files(self) -> dict[str, str | None]: + """The frames to send with the alert: the one with the debris marked first, then the plain one.""" + r = self.last + if r is None: + return {} + return {"frame.quality.jpg": r.annotated, "frame.raw.jpg": r.frame} + + # -- the end + + def note(self) -> str | None: + """The log line for the last reading (None: nothing worth saying).""" + v, r = self.last_verdict, self.last + if v is None or r is None: + return None + n = r.numbers + if v.state in ("blind", "absent"): + return f"quality {v.state}: {'; '.join(v.notes)}" + return ( + f"quality {v.state} (level {v.level}): fleck {n.get('fleck_frac')} order {n.get('order')}" + + ( + f" ratio {v.fleck_ratio} drop {v.order_drop}" + if v.fleck_ratio is not None + else "" + ) + ) + + def finish(self, out_dir: Path) -> tuple[Path | None, str]: + """Write the monitoring report; -> (its folder, the one-line summary). No readings: no report.""" + if self.judge.report.readings == 0: + return None, "print-quality monitor: no camera reading was taken" + return write_report(self.judge.report, out_dir, self.name), summary_line( + self.judge.report + ) diff --git a/lib/printer_ops/jobwatch_run.py b/lib/printer_ops/jobwatch_run.py index 63a85a10..a2e16d5d 100644 --- a/lib/printer_ops/jobwatch_run.py +++ b/lib/printer_ops/jobwatch_run.py @@ -9,6 +9,8 @@ the printer's own signals; on a lost print it stops the job (PAUSE or CANCEL, see `--on-fail`), saves the frames and the klippy.log lines into `--out`, rings `tg call me --say ...` and sends the frame with `tg`. Decisions are in jobwatch.py, the stop/alert steps in jobwatch_actions.py, the readings in jobwatch_sources.py. +The print-QUALITY monitor (stringing debris, a lost bead pattern, an uneven first layer: warnings only, never a +stop) runs on the same frames (jobwatch_quality.py) and writes its monitoring report when the watch ends. NEVER: starts a print, moves an axis, resumes, sends G-code. The camera sees the plate at the print's own Z, so nothing is moved to look. Exit codes: 0 the job ended and nothing was lost, 3 a lost print was found (the stop @@ -46,9 +48,11 @@ parse_checkpoints, ) from printer_ops.jobwatch_actions import Bound, Responder, bind, find_tg +from printer_ops.jobwatch_quality import QualityWatch, run_quality_tool +from printer_ops.quality import QualityConfig, quality_config USAGE = """3d printer watch NAME --job --yes [--on-fail auto|pause|cancel] [--gcode FILE.gcode] [--out DIR] - [--interval S] [--checkpoints L1@25%,L1@50%,L1@100%,25%,50%,100%] [--dry-run | --tg-dry-run] + [--interval S] [--checkpoints L1@25%,L1@50%,L1@100%,25%,50%,100%] [--dry-run | --tg-dry-run] [--no-quality] [--fixture SCENARIO.json] [--duration S] [--wait-start S] Watch ONE running print and ABORT IT when it is lost. Start it right after `3d printer start NAME FILE --yes` (or give `start` --watch): it follows that job until it ends. Unlike the always-on `3d printer watch NAME` @@ -63,6 +67,16 @@ camera says `blind` — never "gone". signals the same Moonraker events as the always-on watch (filament runout, heater loss, bed off, exception, G-code error, homing lost) and the printer's own defect detection (`noodle` verdicts, p >= 0.8). + quality the PRINT-QUALITY monitor (printer_ops/quality.py, lib/print_quality.py): white stringing debris, + a lost bead pattern (stripes, roughness, under-/over-extrusion) and an uneven first layer, judged + against the job's OWN first layers (the first 4 readings with the part on the plate are the + reference). It sends `tg --photo FRAME ""` once per issue and level (warning, then + severe) and NEVER pauses or cancels: a bad surface is the owner's call. A `notice` is only + written to the report. When the job ends (or the watch stops) the MONITORING REPORT is written + to DIR/-NAME-monitor-report/ (report.json, report.md, worst-frame.jpg): readings per + severity and issue, the alerts, the baseline, the worst frame. `--no-quality` or + `.watch.quality: false` turns it off; `.watch.quality: {key: value}` tunes it + (the QualityConfig fields). On a critical finding: the job is stopped, the frames + finding.json + the klippy.log lines of the job are saved to DIR/-NAME-jobwatch/ (default ./print/runs), `tg call me --say ""` rings the owner, then `tg --photo FRAME ""` sends the picture. A warning (part moved, nothing seen, a heater drift) is a tg @@ -77,6 +91,7 @@ --checkpoints LIST extra checks (see above); `--checkpoints none` for the periodic one only --dry-run nothing is sent to the printer or the phone: the stop and both tg commands are printed --tg-dry-run the job is really stopped, the tg commands are only printed + --no-quality skip the print-quality monitor (and its report) --fixture FILE no network at all: the printer and the camera come from a scenario file with saved frames (implies --dry-run for the phone; a fixture printer records the stop it is given) --duration S stop watching after S seconds (default: until the job ends) @@ -99,6 +114,8 @@ "candidate.png", "track.json", "candidate-track.json", + "quality-frame.jpg", + "quality-worst.jpg", ) @@ -118,6 +135,7 @@ class Args: fixture: str | None duration: float | None wait_start: float | None + no_quality: bool = False class _Parser(argparse.ArgumentParser): @@ -145,6 +163,7 @@ def parse_args(argv: list[str]) -> Args: p.add_argument(flag, type=float) p.add_argument("--dry-run", action="store_true") p.add_argument("--tg-dry-run", action="store_true") + p.add_argument("--no-quality", action="store_true") a = p.parse_args(argv) return Args( a.name, @@ -161,6 +180,7 @@ def parse_args(argv: list[str]) -> Args: a.fixture, a.duration, a.wait_start, + a.no_quality, ) @@ -415,6 +435,7 @@ def __init__( clock: Callable[[], float], sleep: Callable[[float], None], echo: Callable[[str], None] = print, + quality: QualityWatch | None = None, ) -> None: self.source, self.camera, self.responder, self.cfg, self.tracker = ( source, @@ -425,6 +446,7 @@ def __init__( ) self.clock, self.sleep, self.echo = clock, sleep, echo self.judge = JobJudge(cfg) + self.quality = quality self.next_check = 0.0 self.recheck_at: float | None = None self.ended = 0 @@ -448,6 +470,7 @@ def due(self, o: Observation, now: float) -> bool: def check(self, o: Observation, now: float) -> Verdict: why = None + jpeg: bytes | None = None try: jpeg = self.source.grab() why = self.camera.ensure_baseline(jpeg, o) @@ -457,6 +480,8 @@ def check(self, o: Observation, now: float) -> Verdict: o.noodles = self.source.noodles() v = self.judge.step(o) self.camera.learn(v) + if self.quality is not None and jpeg is not None: + v.findings.extend(self.quality.check(jpeg, o)) first = o.layer is not None and o.layer <= 1 self.next_check = now + ( self.cfg.first_layer_interval_s if first else self.cfg.interval_s @@ -476,6 +501,8 @@ def remember(self, o: Observation, v: Verdict) -> None: ) + "".join(f"; {x}" for x in v.notes) ) + if self.quality is not None and (note := self.quality.note()): + self.say(note) if o.part is not None and o.part.frame: self.last_files = { "frame.annotated.jpg": o.part.annotated, @@ -535,11 +562,19 @@ def handle(self, findings: list[Finding], o: Observation) -> int | None: "alert", fresh, layer=o.layer, - files=self.last_files, + files=self.alert_files(fresh), extra={"observations": self.history, "action": "alert"}, ) return None + def alert_files(self, fresh: list[Finding]) -> dict[str, str | None]: + """The frames sent with an alert: the quality picture (debris marked) for a quality alert.""" + if self.quality is not None and all( + f.kind.startswith("quality_") for f in fresh + ): + return {**self.quality.alert_files(), **self.last_files} + return self.last_files + def wait_for_job(self, deadline: float) -> bool: while self.clock() < deadline: o = self.poll(self.clock()) @@ -549,6 +584,20 @@ def wait_for_job(self, deadline: float) -> bool: return False def run(self, duration: float | None) -> int: + """Watch until the job ends; the monitoring report is written whatever way the watch ends.""" + try: + return self._run(duration) + finally: + self.report() + + def report(self) -> None: + if self.quality is None: + return + folder, line = self.quality.finish(self.cfg.out_dir) + if folder is not None: + self.say(f"{line} -> {folder}") + + def _run(self, duration: float | None) -> int: start = self.clock() if not self.wait_for_job(start + self.cfg.wait_start_s): self.say("no job is printing: nothing to watch") @@ -708,11 +757,16 @@ def run(argv: list[str]) -> int: args, dict((device.watch or {}).get("job") or {}), default_out_dir(args.name) ) tracker = CheckpointTracker(cfg.checkpoints, first_layer_range(args.gcode)) + quality = quality_config( + (device.watch or {}).get("quality"), getattr(device, "model", None) + ) with one_watch_per_printer(args.name): - return _run_watch(args, cfg, tracker) + return _run_watch(args, cfg, tracker, quality) -def _run_watch(args: Args, cfg: JobConfig, tracker: CheckpointTracker) -> int: +def _run_watch( + args: Args, cfg: JobConfig, tracker: CheckpointTracker, quality_cfg: QualityConfig +) -> int: from cli.paths import state_dir run_dir = state_dir() / "jobwatch" / args.name @@ -729,7 +783,21 @@ def _run_watch(args: Args, cfg: JobConfig, tracker: CheckpointTracker) -> int: ): # a new job starts from nothing: no baseline or footprint of the last one (run_dir / name).unlink(missing_ok=True) camera = CameraCheck(args.name, run_dir, run_part_tool, cfg) - watch = JobWatch(source, camera, responder, cfg, tracker, clock=now, sleep=sleep) + quality = ( + None + if args.no_quality or not quality_cfg.enabled + else QualityWatch( + args.name, + run_dir, + run_quality_tool, + quality_cfg, + cfg, + responder.bound.filename, + ) + ) + watch = JobWatch( + source, camera, responder, cfg, tracker, clock=now, sleep=sleep, quality=quality + ) print( f"watching {args.name}: {responder.bound.filename} (job {responder.bound.job_id or '?'}), every " f"{cfg.interval_s:g} s, on-fail {cfg.on_fail}, evidence -> {cfg.out_dir}" diff --git a/lib/printer_ops/quality.py b/lib/printer_ops/quality.py new file mode 100644 index 00000000..39dc9757 --- /dev/null +++ b/lib/printer_ops/quality.py @@ -0,0 +1,610 @@ +"""quality.py — the decision half of the print-quality monitor: is the print GOOD, not just alive? + +WHY (2026-11-05): a 10 h sheet of small tags on the U1 stringed from layer 2 on — white thread debris all over the + part — and nothing flagged it: the spaghetti model (Obico) scored 0.0 on all 12 frames (stringing is not a + class it knows), and the job watch (jobwatch.py) only asks "is the part still where it was", which a stringy part + is. The print stayed on the plate, so nothing was lost — only a bad part, hours later. This tier watches + the QUALITY of what is on the plate. + +WHAT (pure, stdlib, no clock, no I/O: feed `QualityReading`s, get `QualityVerdict`s and, at the end, a report): + `QualityConfig` every threshold, from `.watch.quality` in printer-devices.yaml (keys = the fields); + `QualityJudge` the state machine over the numbers of lib/print_quality.py (fleck_frac, order): + * BASELINE the first `baseline_n` readings with the part on the plate ARE the reference (the clean first + layer of THIS job under THIS light on THIS plate); never judged against anything but a floor. + A reading drifts the baseline only while it is clean, so a slow legitimate change (infill vs + walls) is followed and debris is not absorbed. + * ISSUES `stringing` (white specks and strands: fleck_frac against the baseline), `texture` (the regular + bead pattern got lost with no debris: stripes, roughness, under- or over-extrusion: `order` + dropped against the baseline), `first_layer` (an uneven first layer: `order` under an absolute + floor while layer <= first_layer_max). + * LEVELS notice (INFO: report only) < warning < severe. A level is REPORTED when `confirm` of the last + `window` readings reach it (`severe_confirm` for severe): one blurry frame or a toolhead over the + part is not a finding. The level is latched; it clears after `clear_after` clean readings. + * TREND the slope of the debris density over the last readings; a rising trend lifts a notice-level + debris to warning. + * ALERTS one `QualityFinding` per (issue, level) per job (`new` on the verdict): the owner is told once + per escalation, never once per frame. + `QualityReport` what the monitor saw, for a human: readings per severity, per issue, the worst frame, the + alerts, the baseline, a timeline (quality_report.py writes it next to the evidence). + +NEVER CRITICAL: a quality finding is at most WARNING. It tells and shows the frame; it does not pause or cancel — + the print is still attached and a bad surface is the owner's call. Pausing/cancelling stays with the detectors + that mean it (part gone, spaghetti, the printer's own noodle verdict). + +THRESHOLDS are calibrated on one camera (docs/research/print-quality-monitor.md): clean first layer fleck_frac + 0.011-0.025 / order 0.83-0.85, stringing 0.039-0.087 / 0.63-0.74. +""" + +from __future__ import annotations + +import json +import re +from collections import deque +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any + +# issues +STRINGING, TEXTURE, FIRST_LAYER = "stringing", "texture", "first_layer" +ISSUES = (STRINGING, TEXTURE, FIRST_LAYER) + +# levels (rank = index) +OK, NOTICE, WARNING, SEVERE = "ok", "notice", "warning", "severe" +LEVELS = (OK, NOTICE, WARNING, SEVERE) +# readings that carry no judgement +BLIND, ABSENT, LEARNING = "blind", "absent", "learning" + +# the severity names of printer_ops.watch / jobwatch (a quality finding is never above WARNING) +WATCH_SEVERITY = {NOTICE: "info", WARNING: "warning", SEVERE: "warning"} + +# Owner-facing text (Russian: the owner's language) per issue: what is seen, what to change. +SEEN = { + STRINGING: "на детали белые ниточки и крошки (стринг)", + TEXTURE: "поверхность потеряла ровный рисунок линий: полосы и шероховатость (недо- или переэкструзия)", + FIRST_LAYER: "первый слой неровный: рваные линии, пропуски или ниточки между островами", +} +CHANGE = { + STRINGING: "снизить температуру сопла на 5-10 °C, подсушить филамент, включить ретракт и вайп, " + "«не пересекать стенки»", + TEXTURE: "проверить сухость филамента и сопло, откалибровать поток и температуру", + FIRST_LAYER: "протереть стол (IPA), проверить Z-offset и поток первого слоя, снизить его скорость", +} +LEVEL_RU = {NOTICE: "заметил", WARNING: "внимание", SEVERE: "усиливается"} + + +@dataclass(frozen=True) +class QualityConfig: + enabled: bool = True + baseline_n: int = 4 # readings with the part on the plate that make the reference + # debris: a level needs BOTH its ratio to the baseline AND its absolute fleck_frac + fleck_floor: float = 0.012 # a baseline below this counts as this (a spotless first layer must not make 0.03 a disaster) + notice_ratio: float = 1.7 + notice_abs: float = 0.03 + warning_ratio: float = 2.0 + warning_abs: float = 0.035 + severe_ratio: float = 3.5 + severe_abs: float = 0.06 + # pattern loss: `order` under the baseline by this much + notice_drop: float = 0.05 + warning_drop: float = 0.10 + severe_drop: float = 0.18 + # first layer: an absolute floor (no baseline can exist before the first layer is there) + first_layer_max: int = 3 + first_layer_z: float = 0.8 # without a layer number: Z up to this is "first layers" + # `order` under this in a first layer is `first_layer` warning (0 = off). Clean beads on the U1 camera read + # 0.83-0.85, a stringy first layer 0.33-0.53 — but another camera/scene has its own normal (a K1 SE view of + # a part reads 0.5), so the floor is ON only f. + first_layer_order: float = 0.0 + first_layer_severe: float = 0.45 + window: int = 4 + confirm: int = 2 + severe_confirm: int = 3 + clear_after: int = 6 + adapt: float = 0.05 # how fast a clean reading moves the baseline ... + adapt_fleck_cap: float = 1.3 # ... which never rises above this multiple of the first-layer value (debris must not be absorbed) + adapt_order_cap: float = 0.04 # ... nor falls further than this under it + trend_n: int = 5 # readings the slope needs + trend_rise: float = ( + 0.5 # ratio-to-baseline gained per 10 minutes that counts as "rising" + ) + min_gap_s: float = 12.0 # a camera check this soon after the last quality reading is skipped (rechecks of the part check) + timeline: int = 200 # readings kept in the report + + +def first_layer_floor(order: float, cfg: QualityConfig) -> str: + """The absolute first-layer level of an `order` reading (`ok` when the floor is off).""" + if cfg.first_layer_order <= 0: + return OK + if order < min(cfg.first_layer_severe, cfg.first_layer_order): + return SEVERE + return WARNING if order < cfg.first_layer_order else OK + + +PROFILES_FILE = ( + Path(__file__).resolve().parents[1] / "data" / "vision" / "quality_profiles.json" +) + + +def profile_for(model: str | None) -> dict[str, Any]: + """The calibrated defaults of a printer model (`device.model`) from lib/data/vision/quality_profiles.json + (matched on the model name stripped to lowercase letters and digits, by its ending), else none.""" + key = re.sub(r"[^a-z0-9]", "", (model or "").lower()) + if not key: + return {} + profiles = json.loads(PROFILES_FILE.read_text(encoding="utf-8")) + hit: dict[str, Any] = next( + (v for k, v in profiles.items() if not k.startswith("_") and key.endswith(k)), + {}, + ) + return dict(hit) + + +def quality_config(settings: Any, model: str | None = None) -> QualityConfig: + """`.watch.quality` (a mapping of QualityConfig fields; `false` turns the monitor off) -> config. + Unknown keys and values of the wrong type are an InvalidArgument naming the accepted keys.""" + from dataclasses import fields, replace + + from errors import InvalidArgument + + if settings is False: + return QualityConfig(enabled=False) + base = replace(QualityConfig(), **profile_for(model)) + if settings is None or settings is True: + return base + defaults = {f.name: getattr(QualityConfig(), f.name) for f in fields(QualityConfig)} + accepted = sorted(defaults) + if not isinstance(settings, dict): + raise InvalidArgument( + "watch.quality", repr(settings), accepted, command="printer watch" + ) + for key, value in settings.items(): + want = type(defaults.get(key)) + numeric = ( + want in (int, float) + and isinstance(value, (int, float)) + and not isinstance(value, bool) + ) + if key not in defaults or not (numeric or isinstance(value, want)): + raise InvalidArgument( + f"watch.quality.{key}", repr(value), accepted, command="printer watch" + ) + return replace(base, **settings) + + +@dataclass +class QualityReading: + """One camera reading: the numbers of print_quality.py plus where in the job it was taken.""" + + now: float + numbers: dict[str, Any] + layer: int | None = None + z: float | None = None + progress: float | None = None + present: bool = True # the part stands on the plate (jobwatch: bed_part says so); False = not judged + frame: str | None = None # the picture the numbers came from (kept as evidence) + annotated: str | None = None + + @property + def ok(self) -> bool: + return self.numbers.get("status") == "ok" + + +@dataclass(frozen=True) +class QualityFinding: + """One thing to tell the owner. Never critical.""" + + kind: str # quality__: one alert per kind per job + issue: str + level: str + text: str # Russian, short: what is seen and what to change + data: dict[str, Any] = field(default_factory=dict) + + @property + def severity(self) -> str: + return WATCH_SEVERITY[self.level] + + +@dataclass +class QualityVerdict: + state: ( + str # ok | notice | warning | severe | blind | absent | learning (this reading) + ) + raw: str = OK # this reading alone + level: str = OK # the latched level after this reading + issue: str | None = None + new: list[QualityFinding] = field(default_factory=list) # alerts to send now + fleck_ratio: float | None = None + order_drop: float | None = None + trend: float | None = ( + None # debris ratio gained per 10 minutes (None: too few readings) + ) + notes: list[str] = field(default_factory=list) + worst: bool = ( + False # the worst reading so far: the caller keeps its frame (`pin_worst`) + ) + + +def rank(level: str) -> int: + return LEVELS.index(level) + + +def _level_of(value: float, steps: tuple[float, float, float]) -> str: + """`ok` below the first step, else the highest step reached.""" + n = sum(value >= s for s in steps) + return LEVELS[n] + + +def _slope_per_10min(points: list[tuple[float, float]]) -> float | None: + """Least-squares slope of (time s, value), per 10 minutes; None without a usable spread of time.""" + n = len(points) + if n < 2: + return None + mt = sum(t for t, _ in points) / n + mv = sum(v for _, v in points) / n + var = sum((t - mt) ** 2 for t, _ in points) + if var <= 0: + return None + return sum((t - mt) * (v - mv) for t, v in points) / var * 600.0 + + +@dataclass +class Baseline: + fleck: float + order: float + n: int + fleck0: float = 0.0 # as first learnt: the drift is bounded around these + order0: float = 0.0 + + def __post_init__(self) -> None: + self.fleck0, self.order0 = self.fleck, self.order + + def fleck_eff(self, floor: float) -> float: + return max(self.fleck, floor) + + def fleck_eff0(self, floor: float) -> float: + return max(self.fleck0, floor) + + +@dataclass +class QualityReport: + """What the monitor saw: counts per severity, the worst reading, the alerts, the timeline.""" + + printer: str = "" + job: str = "" + readings: int = 0 + skipped: dict[str, int] = field(default_factory=dict) # blind / absent / learning + per_level: dict[str, int] = field( + default_factory=lambda: dict.fromkeys(LEVELS, 0) + ) # raw, judged readings + per_issue: dict[str, int] = field(default_factory=lambda: dict.fromkeys(ISSUES, 0)) + max_level: str = OK # highest REPORTED (latched) level + alerts: list[dict[str, Any]] = field(default_factory=list) + worst: dict[str, Any] | None = None + baseline: dict[str, Any] | None = None + timeline: list[dict[str, Any]] = field(default_factory=list) + started: float | None = None + ended: float | None = None + keep: int = 200 + + def to_json(self) -> dict[str, Any]: + return { + "printer": self.printer, + "job": self.job, + "readings": self.readings, + "judged": sum(self.per_level.values()), + "skipped": self.skipped, + "per_level": self.per_level, + "per_issue": self.per_issue, + "max_level": self.max_level, + "alerts": self.alerts, + "worst": self.worst, + "baseline": self.baseline, + "started": self.started, + "ended": self.ended, + "timeline": self.timeline, + } + + +class QualityJudge: + def __init__( + self, cfg: QualityConfig | None = None, printer: str = "", job: str = "" + ) -> None: + self.cfg = cfg or QualityConfig() + self.base: Baseline | None = None + self._learn: list[dict[str, Any]] = [] + self.level = OK + self._raw: deque[tuple[str, str | None]] = deque(maxlen=self.cfg.window) + self._clean = 0 + self._alerted: set[str] = set() + self._ratios: deque[tuple[float, float]] = deque( + maxlen=max(self.cfg.trend_n, 8) + ) + self.report = QualityReport(printer=printer, job=job, keep=self.cfg.timeline) + + # -- one reading + + def feed(self, r: QualityReading) -> QualityVerdict: + v = self._judge(r) + self._record(r, v) + return v + + def _judge(self, r: QualityReading) -> QualityVerdict: + if not r.ok: + return QualityVerdict( + BLIND, notes=[str(r.numbers.get("reason") or "no reading")] + ) + if not r.present: + return QualityVerdict(ABSENT, notes=["nothing stands on the plate yet"]) + first = self._first_layer(r) + floor = self._floor_level(r, first) + if self.base is None: + return self._learning(r, floor) + return self._compare(r, floor) + + def _first_layer(self, r: QualityReading) -> bool: + if r.layer is not None: + return r.layer <= self.cfg.first_layer_max + return r.z is not None and r.z <= self.cfg.first_layer_z + + def _floor_level(self, r: QualityReading, first: bool) -> str: + """The first-layer absolute floor on `order` (needs no baseline).""" + if not first: + return OK + return first_layer_floor(float(r.numbers["order"]), self.cfg) + + def _learning(self, r: QualityReading, floor: str) -> QualityVerdict: + # A reading that already fails the first-layer floor is judged, but never makes the reference. + if floor == OK: + self._learn.append(r.numbers) + if len(self._learn) >= self.cfg.baseline_n: + self.base = self._baseline_of(self._learn) + self.report.baseline = { + "fleck_frac": self.base.fleck, + "order": self.base.order, + "n": self.base.n, + } + v = QualityVerdict( + LEARNING if floor == OK else floor, + raw=floor, + notes=["learning the job's own baseline"], + ) + if floor != OK: + v.issue = FIRST_LAYER + self._confirm(v, r) + else: + v.level = self.level + return v + + @staticmethod + def _baseline_of(readings: list[dict[str, Any]]) -> Baseline: + def median(key: str) -> float: + xs = sorted(float(x[key]) for x in readings) + m = len(xs) // 2 + return xs[m] if len(xs) % 2 else (xs[m - 1] + xs[m]) / 2 + + return Baseline(median("fleck_frac"), median("order"), len(readings)) + + def _compare(self, r: QualityReading, floor: str) -> QualityVerdict: + assert self.base is not None + cfg = self.cfg + fleck = float(r.numbers["fleck_frac"]) + order = float(r.numbers["order"]) + ratio = fleck / self.base.fleck_eff(cfg.fleck_floor) + drop = self.base.order - order + debris = _level_of( + ratio, (cfg.notice_ratio, cfg.warning_ratio, cfg.severe_ratio) + ) + absolute = _level_of(fleck, (cfg.notice_abs, cfg.warning_abs, cfg.severe_abs)) + debris = LEVELS[min(rank(debris), rank(absolute))] + pattern = _level_of(drop, (cfg.notice_drop, cfg.warning_drop, cfg.severe_drop)) + v = QualityVerdict(OK, fleck_ratio=round(ratio, 2), order_drop=round(drop, 3)) + v.trend = self._trend(r.now, fleck / self.base.fleck_eff0(cfg.fleck_floor)) + if debris == NOTICE and v.trend is not None and v.trend >= cfg.trend_rise: + debris = WARNING + v.notes.append(f"debris is rising: +{v.trend:.1f} x baseline per 10 min") + v.issue, v.raw = _pick(debris, pattern, floor) + v.state = v.raw + self._confirm(v, r) + if v.raw == OK and self.level == OK: + self._adapt(fleck, order) + return v + + def _trend(self, now: float, ratio: float) -> float | None: + self._ratios.append((now, ratio)) + pts = list(self._ratios)[-self.cfg.trend_n :] + return _slope_per_10min(pts) if len(pts) >= self.cfg.trend_n else None + + def _adapt(self, fleck: float, order: float) -> None: + """A clean reading drifts the baseline a little (a different infill reads differently) — within bounds: + a debris that grows slowly through the notice band must not be absorbed into 'normal'.""" + assert self.base is not None + cfg, b = self.cfg, self.base + b.fleck = min( + b.fleck + cfg.adapt * (fleck - b.fleck), b.fleck0 * cfg.adapt_fleck_cap + ) + b.order = max( + b.order + cfg.adapt * (order - b.order), b.order0 - cfg.adapt_order_cap + ) + + def _confirm(self, v: QualityVerdict, r: QualityReading) -> None: + """Window the raw levels, latch / clear the reported level, and emit the alert for a new one.""" + cfg = self.cfg + self._raw.append((v.raw, v.issue)) + self._clean = self._clean + 1 if v.raw == OK else 0 + reached = self._reached() + if rank(reached) > rank(self.level): + self.level = reached + v.new = self._alerts(v, reached, r) + elif self._clean >= cfg.clear_after and self.level != OK: + self.level = OK + v.notes.append("quality is back to normal") + v.level = self.level + + def _reached(self) -> str: + """The highest level that `confirm` of the window's readings reach (`severe_confirm` for severe).""" + cfg = self.cfg + raws = [x for x, _ in self._raw] + for level in (SEVERE, WARNING, NOTICE): + need = cfg.severe_confirm if level == SEVERE else cfg.confirm + if sum(rank(x) >= rank(level) for x in raws) >= need: + return level + return OK + + def _alerts( + self, v: QualityVerdict, level: str, r: QualityReading + ) -> list[QualityFinding]: + issue = self._dominant(v) + kind = f"quality_{issue}_{level}" + owed = [kind] if kind not in self._alerted else [] + self._alerted.update( + f"quality_{issue}_{x}" for x in LEVELS[1 : rank(level) + 1] + ) + if not owed: + return [] + finding = QualityFinding( + kind, + issue, + level, + owner_text(issue, level, v), + self._data(r, v, issue, level), + ) + self.report.alerts.append( + { + "kind": kind, + "level": level, + "issue": issue, + "now": r.now, + "layer": r.layer, + "frame": r.frame, + "text": finding.text, + } + ) + return [finding] + + def _dominant(self, v: QualityVerdict) -> str: + """The issue of the strongest recent reading (the current one when it ties).""" + best, issue = -1, v.issue or STRINGING + for raw, iss in self._raw: + if iss is not None and rank(raw) >= best: + best, issue = rank(raw), iss + return issue + + def _data( + self, r: QualityReading, v: QualityVerdict, issue: str, level: str + ) -> dict[str, Any]: + return { + "issue": issue, + "level": level, + "layer": r.layer, + "z": r.z, + "numbers": { + k: r.numbers.get(k) + for k in ("fleck_frac", "fleck_per10k", "order", "luma") + }, + "baseline": self.report.baseline, + "fleck_ratio": v.fleck_ratio, + "order_drop": v.order_drop, + "trend": v.trend, + } + + # -- the report + + def _record(self, r: QualityReading, v: QualityVerdict) -> None: + rep = self.report + rep.readings += 1 + rep.started = r.now if rep.started is None else rep.started + rep.ended = r.now + judged = v.state not in (BLIND, ABSENT, LEARNING) + if v.state in (BLIND, ABSENT, LEARNING): + rep.skipped[v.state] = rep.skipped.get(v.state, 0) + 1 + if judged: + rep.per_level[v.raw] += 1 + if v.issue is not None and v.raw != OK: + rep.per_issue[v.issue] += 1 + if rank(self.level) > rank(rep.max_level): + rep.max_level = self.level + v.worst = self._worst(r, v, judged) + rep.timeline.append(_timeline_row(r, v)) + del rep.timeline[: -self.cfg.timeline] + + def pin_worst(self, path: str) -> None: + """The caller copied the worst frame to a stable `path` (the camera's own file is overwritten by the next one).""" + if self.report.worst is not None: + self.report.worst["frame"] = path + + def _worst(self, r: QualityReading, v: QualityVerdict, judged: bool) -> bool: + if not judged or v.raw == OK: + return False + score = (rank(v.raw), float(r.numbers.get("fleck_frac") or 0.0)) + cur = self.report.worst + if cur is not None and score <= tuple(cur["score"]): + return False + self.report.worst = { + "score": list(score), + "level": v.raw, + "issue": v.issue, + "now": r.now, + "layer": r.layer, + "z": r.z, + "frame": r.annotated or r.frame, + "raw_frame": r.frame, + "numbers": { + k: r.numbers.get(k) + for k in ("fleck_frac", "fleck_per10k", "order", "luma") + }, + } + return True + + +def _pick(debris: str, pattern: str, floor: str) -> tuple[str | None, str]: + """(issue, raw level) of one reading: debris is stringing, pattern loss without debris is texture, the + first-layer floor is its own issue; the strongest wins, ties go to stringing then first_layer.""" + options = [ + (rank(debris), 3, STRINGING), + (rank(floor), 2, FIRST_LAYER), + (rank(pattern), 1, TEXTURE), + ] + best = max(options) + return (best[2], LEVELS[best[0]]) if best[0] > 0 else (None, OK) + + +def absolute_verdict( + numbers: dict[str, Any], z: float | None, cfg: QualityConfig +) -> tuple[str | None, str]: + """(issue, level) of ONE frame with no job history: only what needs no baseline. While the first layers print + (Z <= first_layer_z) the plate is mostly bare and its own glints read as debris, so only the first-layer + floor on `order` applies; above them the absolute debris levels do. `vision --once` uses it; a watch never + does (it has the baseline).""" + if numbers.get("status") != "ok": + return None, OK + if z is not None and z <= cfg.first_layer_z: + return _pick(OK, OK, first_layer_floor(float(numbers["order"]), cfg)) + debris = _level_of( + float(numbers["fleck_frac"]), (cfg.notice_abs, cfg.warning_abs, cfg.severe_abs) + ) + return _pick(debris, OK, OK) + + +def _timeline_row(r: QualityReading, v: QualityVerdict) -> dict[str, Any]: + return { + "now": r.now, + "layer": r.layer, + "state": v.state, + "level": v.level, + "issue": v.issue, + "fleck_frac": r.numbers.get("fleck_frac"), + "order": r.numbers.get("order"), + "fleck_ratio": v.fleck_ratio, + "order_drop": v.order_drop, + } + + +def owner_text(issue: str, level: str, v: QualityVerdict) -> str: + """The tg message, Russian, short: what is seen, how bad, what to change (the sender adds that nothing was touched).""" + numbers = [] + if v.fleck_ratio is not None and issue == STRINGING: + numbers.append(f"мусора в {v.fleck_ratio:g}× больше, чем на первых слоях") + if v.order_drop is not None and issue == TEXTURE: + numbers.append(f"рисунок линий хуже на {v.order_drop:.2f}") + extra = f" ({', '.join(numbers)})" if numbers else "" + return f"{LEVEL_RU[level]}: {SEEN[issue]}{extra}. Что менять: {CHANGE[issue]}." diff --git a/lib/printer_ops/quality_report.py b/lib/printer_ops/quality_report.py new file mode 100644 index 00000000..e4f67fb2 --- /dev/null +++ b/lib/printer_ops/quality_report.py @@ -0,0 +1,123 @@ +"""quality_report.py — the per-job MONITORING REPORT: what the print-quality monitor saw, for a human. + +`3d printer watch NAME --job` writes it when the watch ends (job complete, lost print, or --duration over) and +`3d printer vision` when a print ends, into a `-NAME-monitor-report/` folder next to the other evidence +(`print/runs/` for the job watch, the vision evidence folder for vision): + + report.json QualityReport.to_json(): readings, counts per severity and per issue, the alerts that were + sent, the baseline, the worst reading, the timeline + report.md the same in words (English, short): a table a human can read in ten seconds + worst-frame.jpg the frame of the worst reading (annotated when there is one: debris in red) + +A job nothing was wrong with still gets its report ("0 warnings in 41 judged readings"): the point is that a +human can see what the monitor LOOKED AT, and that "silent" means "checked and fine" rather than "never ran". +""" + +from __future__ import annotations + +import json +import shutil +from datetime import datetime, timezone +from pathlib import Path +from typing import Any + +from printer_ops.quality import LEVELS, QualityReport + + +def summary_line(report: QualityReport) -> str: + judged = sum(report.per_level.values()) + skipped = sum(report.skipped.values()) + counts = ", ".join(f"{report.per_level[x]} {x}" for x in LEVELS[1:]) + worst = report.worst + where = ( + f"; worst: {worst['issue']} {worst['level']} at layer {worst['layer']}" + if worst + else "" + ) + return ( + f"print-quality monitor: {report.readings} readings ({judged} judged, {skipped} not: " + f"{_skips(report)}): {report.per_level['ok']} ok, {counts}; highest reported level {report.max_level}{where}" + ) + + +def _skips(report: QualityReport) -> str: + return ", ".join(f"{n} {k}" for k, n in sorted(report.skipped.items())) or "none" + + +def render_markdown(report: QualityReport) -> str: + doc = report.to_json() + lines = [ + f"# Print-quality monitor: {doc['printer']} {doc['job']}", + "", + f"- readings: {doc['readings']} ({doc['judged']} judged; not judged: {_skips(report)})", + f"- highest level REPORTED to the owner: **{doc['max_level']}**", + "", + "| severity (per reading) | readings |", + "|---|---|", + *[f"| {k} | {v} |" for k, v in doc["per_level"].items()], + "", + "| issue | readings |", + "|---|---|", + *[f"| {k} | {v} |" for k, v in doc["per_issue"].items()], + "", + ] + lines += _baseline_lines(doc) + lines += _alert_lines(doc) + lines += _worst_lines(doc) + return "\n".join(lines) + "\n" + + +def _baseline_lines(doc: dict[str, Any]) -> list[str]: + b = doc["baseline"] + if b is None: + return [ + "Baseline: never formed (fewer than the required readings with the part on the plate).", + "", + ] + return [ + f"Baseline (the job's own first layers, {b['n']} readings): fleck_frac {b['fleck_frac']:.4f}, order {b['order']:.3f}.", + "", + ] + + +def _alert_lines(doc: dict[str, Any]) -> list[str]: + if not doc["alerts"]: + return ["Alerts sent: none.", ""] + out = [ + "Alerts (one per issue and level; a quality alert never pauses or cancels):", + "", + ] + out.extend( + f"- layer {a['layer']}: **{a['level']}** {a['issue']} — {a['text']}" + for a in doc["alerts"] + ) + return [*out, ""] + + +def _worst_lines(doc: dict[str, Any]) -> list[str]: + w = doc["worst"] + if w is None: + return ["Worst reading: none (every judged reading was ok).", ""] + n = w["numbers"] + return [ + f"Worst reading: {w['level']} {w['issue']}, layer {w['layer']}, fleck_frac {n.get('fleck_frac')}, " + f"order {n.get('order')}; frame: worst-frame.jpg", + "", + ] + + +def write_report(report: QualityReport, out_dir: Path, printer: str) -> Path: + """Write report.json / report.md / worst-frame.jpg into `/--monitor-report/`.""" + ended = report.ended if report.ended is not None else 0.0 + stamp = datetime.fromtimestamp(ended, timezone.utc).strftime("%Y%m%dT%H%M%SZ") + d = out_dir / f"{stamp}-{printer}-monitor-report" + d.mkdir(parents=True, exist_ok=True) + (d / "report.json").write_text( + json.dumps(report.to_json(), indent=1, default=str) + "\n", encoding="utf-8" + ) + (d / "report.md").write_text(render_markdown(report), encoding="utf-8") + worst = report.worst + src = worst.get("frame") if worst else None + if src and Path(src).is_file(): + shutil.copyfile(src, d / "worst-frame.jpg") + return d diff --git a/lib/printer_ops/vision_cycle.py b/lib/printer_ops/vision_cycle.py index 9d9181e0..d04a8fcb 100644 --- a/lib/printer_ops/vision_cycle.py +++ b/lib/printer_ops/vision_cycle.py @@ -132,11 +132,13 @@ def __init__( log: Callable[[str], None] = _stderr, force_frame: bool = False, after_keepalive: Callable[[], None] | None = None, + on_print_end: Callable[[], None] | None = None, ) -> None: """`force_frame`: grab a frame even when nothing prints or the printer is silent (`--once`); `after_keepalive`: called when a keepalive was sent in such a cycle (`--once` waits for the first fresh frame of a sleeping camera). `quiet` / `clear_quiet` default to the printer's - intentional-off marker (printer_ops.intentional_off).""" + intentional-off marker (printer_ops.intentional_off). `on_print_end`: called once when a print that + was supervised ends (the printer left printing/paused): the per-print quality report is written there.""" self.printer = printer self.adapter = adapter self.camera = camera @@ -156,6 +158,7 @@ def __init__( self.log = log self.force_frame = force_frame self.after_keepalive = after_keepalive + self.on_print_end = on_print_end self.reachable: bool | None = None # None until the first poll self.unreachable_since: float | None = None self.last_poll_ok: float | None = None @@ -223,7 +226,10 @@ def _track_job( self, state: str, filename: str | None, new_print: bool, now: float ) -> None: if state not in (PRINTING, PAUSED): + ended = self.job_file is not None self.job_file = self.job_started = None + if ended and self.on_print_end is not None: + self.on_print_end() elif new_print or self.job_started is None or filename != self.job_file: self.job_file, self.job_started = filename, now diff --git a/lib/vision_detect.py b/lib/vision_detect.py index f9c7c4e9..a18a823f 100644 --- a/lib/vision_detect.py +++ b/lib/vision_detect.py @@ -246,8 +246,14 @@ def __init__( geometry: PrintGeometry | None = None, model: Any = None, fixed_bed: Any = None, + quality: Any = None, ) -> None: - self.fixed_bed = fixed_bed # vision_detach.FixedBedDetach: uncalibrated part-detached check + self.quality = ( + quality # vision_quality.QualityTier: print-quality Findings, never p/q + ) + self.fixed_bed = ( + fixed_bed # vision_detach.FixedBedDetach: uncalibrated part-detached check + ) self.detector = detector self.verifier = verifier self.ignore = ignore or [] @@ -265,6 +271,13 @@ def reset_print(self) -> None: self.model.reset_print() if self.fixed_bed is not None: self.fixed_bed.reset() + if self.quality is not None: + self.quality.reset_print() # writes the report of the print that just ended first + + def finish_print(self) -> None: + """The print ended: write its monitoring report (no-op without the quality tier).""" + if self.quality is not None: + self.quality.finish() def _allowed(self, b: Box) -> bool: cx, cy = (b.x1 + b.x2) / 2, (b.y1 + b.y2) / 2 @@ -275,24 +288,13 @@ def _allowed(self, b: Box) -> bool: return not any(_inside(cx, cy, poly) for poly in self.ignore) def score_bgr(self, bgr: np.ndarray) -> FrameScore: - findings: list[Finding] = [] - if self.geometry is not None: - if self.context is None: - # No print context this frame (poll failed, frame missing from a replay): no - # bed-volume mask at all, never the previous frame's (it may be for another Z). - self.roi = None - else: - now = time.time() if self.now is None else self.now - self.roi, findings = self.geometry.update(bgr, self.context, now) - if self.fixed_bed is not None: - findings = findings + self.fixed_bed.update( - bgr, time.time() if self.now is None else self.now - ) + findings = self._bed_findings(bgr) s = self.detector.score_bgr(bgr) kept = [b for b in s.boxes if self._allowed(b)] if self.model is not None: - now = time.time() if self.now is None else self.now - findings = findings + self.model.update(bgr, self.context, now) + findings = findings + self.model.update(bgr, self.context, self._now()) + if self.quality is not None: + findings = findings + self.quality.update(bgr, self.context, self._now()) s = FrameScore( p=float(sum(b.conf for b in kept)), boxes=kept, @@ -307,6 +309,25 @@ def score_bgr(self, bgr: np.ndarray) -> FrameScore: s.q = verified_score(kept, s.probs) return s + def _now(self) -> float: + return time.time() if self.now is None else self.now + + def _bed_findings(self, bgr: np.ndarray) -> list[Finding]: + """Bed-change Findings from the geometry and fixed-bed detectors; updates `self.roi`.""" + findings: list[Finding] = [] + if self.geometry is not None: + if self.context is None: + # No print context this frame (poll failed, frame missing from a replay): no + # bed-volume mask at all, never the previous frame's (it may be for another Z). + self.roi = None + else: + self.roi, findings = self.geometry.update( + bgr, self.context, self._now() + ) + if self.fixed_bed is not None: + findings = findings + self.fixed_bed.update(bgr, self._now()) + return findings + def __call__(self, jpeg: bytes) -> FrameScore: return self.score_bgr(decode(jpeg)) @@ -324,7 +345,9 @@ class GeometryParams: pad_mm: float = 15.0 # calibration error is up to ~30 px near the camera (9 px RMS) above_mm: float = 40.0 # spaghetti tangles around the nozzle, above the part top grow_px: int = 40 # at 1920 px: boxes just outside the projected volume still count - band_mm: float = 4.0 # the newest layers; excluded from the change detector, checked for growth + band_mm: float = ( + 4.0 # the newest layers; excluded from the change detector, checked for growth + ) class PrintGeometry: @@ -411,7 +434,9 @@ def update( self, bgr: np.ndarray, ctx: PrintContext, now: float ) -> tuple[np.ndarray, list[Finding]]: """(ROI for the stage-1 boxes, Findings of the bed-change detectors) for one frame.""" - if ctx.filename != self.file: # another file: no reference frames from the old one + if ( + ctx.filename != self.file + ): # another file: no reference frames from the old one self.file = ctx.filename self.reset_print() m = self.masks(ctx, bgr.shape[:2]) @@ -508,9 +533,14 @@ def _device_camera(device: Any, args: argparse.Namespace) -> Any: "Moonraker entry (adapter: moonraker, port 7125), not creality-stock." ], ) - spec = require_camera(device, "printer vision") # watch.vision.camera / model default, never a guess + spec = require_camera( + device, "printer vision" + ) # watch.vision.camera / model default, never a guess return open_camera( - device, spec, snapshot=args.snapshot, keepalive=False if args.no_keepalive else None + device, + spec, + snapshot=args.snapshot, + keepalive=False if args.no_keepalive else None, ) @@ -564,7 +594,11 @@ def fixed_bed_detach(setting: Any, bed_roi: Any) -> Any: if setting in (None, False): return None opts = {} if setting is True else setting - if not isinstance(opts, dict) or set(opts) - {"roi", "baseline_frames", "baseline_gap_s"}: + if not isinstance(opts, dict) or set(opts) - { + "roi", + "baseline_frames", + "baseline_gap_s", + }: raise UsageError( "watch.vision.fixed_bed must be true or a mapping of roi, baseline_frames, baseline_gap_s", command="printer vision", @@ -573,7 +607,10 @@ def fixed_bed_detach(setting: Any, bed_roi: Any) -> Any: try: poly = [(float(x), float(y)) for x, y in roi or []] except (TypeError, ValueError) as exc: - raise UsageError(f"watch.vision.fixed_bed.roi must be [x, y] pairs (0..1): {exc}", command="printer vision") from exc + raise UsageError( + f"watch.vision.fixed_bed.roi must be [x, y] pairs (0..1): {exc}", + command="printer vision", + ) from exc if len(poly) < 3 or not all(0.0 <= v <= 1.0 for pt in poly for v in pt): raise UsageError( "watch.vision.fixed_bed needs a plate polygon of >= 3 points in 0..1: set " @@ -593,6 +630,7 @@ def build_scorer( detach: bool = True, model: bool | None = False, bed_roi: Any = None, + quality: Any = None, ) -> FrameScorer: """The per-frame pipeline from a device's `watch: vision:` settings: `verify: false` turns stage 2 off, `ignore:` lists polygons (normalised [x, y] points) whose boxes are dropped, @@ -604,7 +642,46 @@ def build_scorer( (or `detach=False`, --no-detach) turns it off, `detach: {gap_abs: 0.3, ...}` tunes it. `model`: tier (b), the whole-frame vision model (vision_model.py), driven by `watch.vision.model` — False (default) = not built, True = build it (downloads the weights if - needed), None = build it only when its weights are already cached.""" + needed), None = build it only when its weights are already cached. + `quality`: the print-quality tier (vision_quality.QualityTier; built by run_live from + `watch.quality`), which only ever adds Findings.""" + from errors import UsageError + + polys = _ignore_polygons(cfg) + verifier = _clip_verifier() if verify and cfg.get("verify", True) else None + bed_change = scene or bool(cfg.get("scene", False)) + geometry = _build_geometry(cfg, camera, bed_change, detach) + if bed_change and geometry is None: + raise UsageError( + "the bed-change detectors (--scene / watch.vision.scene) need the camera " + "calibration and `geometry: true`", + command="printer vision", + remediation=[ + "Drop --scene, or see docs/research/printer-vision-geometry.md." + ], + ) + fixed_bed = ( + fixed_bed_detach(cfg.get("fixed_bed"), bed_roi) + if geometry is None and detach + else None + ) + tier_b = None + if model is not False: + from vision_model import build_tier + + tier_b = build_tier(cfg.get("model"), flag=model) + return FrameScorer( + ObicoScorer(ensure_model()), + verifier, + polys, + geometry, + tier_b, + fixed_bed, + quality, + ) + + +def _ignore_polygons(cfg: dict[str, Any]) -> list[list[tuple[float, float]]]: from errors import UsageError try: @@ -627,42 +704,26 @@ def build_scorer( f"watch.vision.ignore polygons need >= 3 points with x, y in 0..1; got {bad[0]}", command="printer vision", ) - verifier = None - if verify and cfg.get("verify", True): - from vision_verify import ClipVerifier, ensure_clip + return polys - verifier = ClipVerifier(ensure_clip(), threads=_threads()) - bed_change = scene or bool(cfg.get("scene", False)) - geometry = ( - PrintGeometry( - camera, - scene=bed_change, - growth=bed_change, - detach=detach_params(cfg.get("detach", True)) if detach else None, - params=geometry_params(cfg.get("head")), - ) - if camera is not None and cfg.get("geometry", True) - else None - ) - if bed_change and geometry is None: - raise UsageError( - "the bed-change detectors (--scene / watch.vision.scene) need the camera " - "calibration and `geometry: true`", - command="printer vision", - remediation=["Drop --scene, or see docs/research/printer-vision-geometry.md."], - ) - fixed_bed = ( - fixed_bed_detach(cfg.get("fixed_bed"), bed_roi) - if geometry is None and detach - else None - ) - tier_b = None - if model is not False: - from vision_model import build_tier - tier_b = build_tier(cfg.get("model"), flag=model) - return FrameScorer( - ObicoScorer(ensure_model()), verifier, polys, geometry, tier_b, fixed_bed +def _clip_verifier() -> Any: + from vision_verify import ClipVerifier, ensure_clip + + return ClipVerifier(ensure_clip(), threads=_threads()) + + +def _build_geometry( + cfg: dict[str, Any], camera: Any, bed_change: bool, detach: bool +) -> PrintGeometry | None: + if camera is None or not cfg.get("geometry", True): + return None + return PrintGeometry( + camera, + scene=bed_change, + growth=bed_change, + detach=detach_params(cfg.get("detach", True)) if detach else None, + params=geometry_params(cfg.get("head")), ) @@ -674,52 +735,150 @@ def _calibration(device: Any, args: argparse.Namespace) -> Any: return load_calibration(device.name) +def _vision_cfg(device: Any) -> dict[str, Any]: + cfg = (getattr(device, "watch", None) or {}).get("vision") or {} + return cfg if isinstance(cfg, dict) else {} + + +def _live_dispatcher(device: Any, args: argparse.Namespace) -> Any: + from printer_ops.watch import severity_rank + from printer_ops.watch_sinks import Dispatcher + + dispatcher = Dispatcher.for_devices([device], dry_run=args.dry_run) + dispatcher.start_startup_test([device.name], "vision") + if args.min_severity: + severity_rank(args.min_severity) + dispatcher.min_severity[device.name] = args.min_severity + return dispatcher + + +def _score_fields(score: FrameScore) -> dict[str, Any]: + return { + "p": round(score.p, 3), + "q": None if score.q is None else round(score.q, 3), + "luma": round(score.luma, 1), + "masked": score.masked, + "boxes": [ + { + **{k: round(v, 3) for k, v in vars(b).items()}, + "verified": round(score.probs[i], 3) if i < len(score.probs) else None, + } + for i, b in enumerate(score.boxes) + ], + "findings": [f.kind for f in score.findings], + } + + +def _detach_fields(scorer: FrameScorer) -> dict[str, Any]: + detach = scorer.geometry.detach if scorer.geometry is not None else None + if detach is None and scorer.fixed_bed is not None: + detach = scorer.fixed_bed.judge + detach.last_gap = scorer.fixed_bed.last_gap + if detach is None: + return {} + # one frame cannot satisfy the persistence rule, so --once reports the raw reading + gap = detach.last_gap + return { + "detach_gap": None if gap is None else round(gap, 3), + "part_detached": gap is not None and gap >= detach.params.gap_abs, + } + + +def _once_report( + args: argparse.Namespace, scorer: FrameScorer, monitor: Any, rep: Any, name: str +) -> dict[str, Any]: + frame, score = rep.frame, rep.score + report = rep.as_json(name) + if score is None and frame is not None: + score = scorer(frame.jpeg) # not printing: still show what the model sees + if score is not None: + report.update(_score_fields(score)) + report.update(_detach_fields(scorer)) + if scorer.quality is not None: + report["quality"] = scorer.quality.once_report() + if scorer.model is not None and scorer.model.last_scores: + report["model"] = scorer.model.report() + report["trend"] = round(monitor.tracker.adjusted, 3) + report["events"] = [e.to_json() for e in rep.events] + if frame is not None and args.save: + out = Path(args.save).expanduser() + out.parent.mkdir(parents=True, exist_ok=True) + out.write_bytes( + annotate(frame.jpeg, score) if score and score.boxes else frame.jpeg + ) + report["frame"] = str(out) + return report + + +def _print_verbose(rep: Any, monitor: Any, name: str) -> None: + score = rep.score + print( + json.dumps( + { + **rep.as_json(name), + "p": None if score is None else round(score.p, 3), + "q": None if score is None or score.q is None else round(score.q, 3), + "trend": round(monitor.tracker.adjusted, 3), + } + ), + file=sys.stderr, + flush=True, + ) + + def run_live(args: argparse.Namespace) -> int: from cli.paths import cache_dir from printer_ops import make_adapter from printer_ops.devices import get_device from printer_ops.vision import ContextTap, KlippyLogTap, VisionMonitor + from printer_ops.vision_config import monitor_limits, npu_enabled from printer_ops.vision_cycle import VisionCycle from printer_ops.vision_status import VisionStatus - from printer_ops.watch import severity_rank - from printer_ops.watch_sinks import Dispatcher + from vision_quality import build_quality device = get_device(args.printer) camera = _device_camera(device, args) adapter = make_adapter(device) - from printer_ops.vision_config import monitor_limits, npu_enabled - # the printer's own detector only where it has one (watch.vision.npu; default: a Snapmaker monitor camera) - tap = KlippyLogTap(device.host, device.port) if npu_enabled(device) and not args.no_npu else None - cfg = (getattr(device, "watch", None) or {}).get("vision") or {} + tap = ( + KlippyLogTap(device.host, device.port) + if npu_enabled(device) and not args.no_npu + else None + ) + cfg = _vision_cfg(device) cam = _calibration(device, args) + evidence = ( + Path(args.evidence) if args.evidence else cache_dir() / "vision" / "evidence" + ) scorer = build_scorer( - cfg if isinstance(cfg, dict) else {}, + cfg, verify=not args.no_verify, camera=cam, scene=args.scene, detach=not args.no_detach, model=args.model, bed_roi=((getattr(device, "watch", None) or {}).get("bed") or {}).get("roi"), + quality=build_quality( + device, evidence, once=args.once, disabled=args.no_quality + ), ) + # the print position (Z) feeds the camera masks and the quality tier's plate region context_tap = ( - ContextTap(device.host, device.port) if scorer.geometry else None - ) - evidence = ( - Path(args.evidence) if args.evidence else cache_dir() / "vision" / "evidence" + ContextTap(device.host, device.port) + if scorer.geometry or scorer.quality + else None ) monitor = VisionMonitor( - device.name, scorer, annotate=annotate, evidence_dir=evidence, - limits=monitor_limits(cfg if isinstance(cfg, dict) else {}), + device.name, + scorer, + annotate=annotate, + evidence_dir=evidence, + limits=monitor_limits(cfg), ) - status = None if args.once else VisionStatus(device.name) # the file 3d-api serves to Home Assistant - dispatcher = None - if not args.once: - dispatcher = Dispatcher.for_devices([device], dry_run=args.dry_run) - dispatcher.start_startup_test([device.name], "vision") - if args.min_severity: - severity_rank(args.min_severity) - dispatcher.min_severity[device.name] = args.min_severity + status = ( + None if args.once else VisionStatus(device.name) + ) # the file 3d-api serves to Home Assistant + dispatcher = None if args.once else _live_dispatcher(device, args) def set_context(ctx: Any) -> None: scorer.context = ctx @@ -734,6 +893,7 @@ def set_context(ctx: Any) -> None: camera=camera, monitor=monitor, reset_print=scorer.reset_print, + on_print_end=scorer.finish_print, interval_s=args.interval, context_tap=context_tap, set_context=set_context, @@ -742,78 +902,34 @@ def set_context(ctx: Any) -> None: dispatcher=dispatcher, force_frame=args.once, **once_quiet, - after_keepalive=lambda: time.sleep(WAKE_S), # a sleeping monitor writes its first fresh frame within ~4 s + after_keepalive=lambda: time.sleep( + WAKE_S + ), # a sleeping monitor writes its first fresh frame within ~4 s ) while True: rep = cycle.cycle() - frame, score, events = rep.frame, rep.score, rep.events - report = rep.as_json(device.name) if args.once: - if score is None and frame is not None: - score = scorer( - frame.jpeg - ) # not printing: still show what the model sees - if score is not None: - report.update( - p=round(score.p, 3), - q=None if score.q is None else round(score.q, 3), - luma=round(score.luma, 1), - masked=score.masked, - boxes=[ - { - **{k: round(v, 3) for k, v in vars(b).items()}, - "verified": round(score.probs[i], 3) - if i < len(score.probs) - else None, - } - for i, b in enumerate(score.boxes) - ], - findings=[f.kind for f in score.findings], - ) - detach = scorer.geometry.detach if scorer.geometry is not None else None - if detach is None and scorer.fixed_bed is not None: - detach = scorer.fixed_bed.judge - detach.last_gap = scorer.fixed_bed.last_gap - if detach is not None: - # one frame cannot satisfy the persistence rule, so --once reports the raw reading - gap = detach.last_gap - report["detach_gap"] = None if gap is None else round(gap, 3) - report["part_detached"] = gap is not None and gap >= detach.params.gap_abs - if scorer.model is not None and scorer.model.last_scores: - report["model"] = scorer.model.report() - report["trend"] = round(monitor.tracker.adjusted, 3) - report["events"] = [e.to_json() for e in events] - if frame is not None and args.save: - out = Path(args.save).expanduser() - out.parent.mkdir(parents=True, exist_ok=True) - out.write_bytes( - annotate(frame.jpeg, score) if score and score.boxes else frame.jpeg - ) - report["frame"] = str(out) - print(json.dumps(report, ensure_ascii=False)) - return 0 - if args.verbose: print( json.dumps( - { - **report, - "p": None if score is None else round(score.p, 3), - "q": None - if score is None or score.q is None - else round(score.q, 3), - "trend": round(monitor.tracker.adjusted, 3), - } - ), - file=sys.stderr, - flush=True, + _once_report(args, scorer, monitor, rep, device.name), + ensure_ascii=False, + ) ) + return 0 + if args.verbose: + _print_verbose(rep, monitor, device.name) time.sleep(rep.next_delay_s) def _frame_score_errors() -> tuple[type[Exception], ...]: """Native pixel/runtime failures only; programming and model-contract errors propagate.""" import cv2 - from onnxruntime.capi.onnxruntime_pybind11_state import EPFail, EngineError, Fail, RuntimeException + from onnxruntime.capi.onnxruntime_pybind11_state import ( + EPFail, + EngineError, + Fail, + RuntimeException, + ) return (FrameDecodeError, cv2.error, Fail, RuntimeException, EngineError, EPFail) @@ -823,30 +939,44 @@ def run_guard(args: argparse.Namespace) -> int: from cli.paths import cache_dir, state_dir from errors import UsageError from printer_ops import make_adapter - from printer_ops.base import PrinterCommError from printer_ops.guard import PauseGuard, _write_result, guard_device, validate_args from printer_ops.vision import ContextTap from printer_ops.vision_config import monitor_limits validate_args(args) device = guard_device(args.printer) - cfg = (getattr(device, "watch", None) or {}).get("vision") or {} - cfg = cfg if isinstance(cfg, dict) else {} + cfg = _vision_cfg(device) if cfg.get("verify", True) is not True: - raise UsageError("guard requires the CLIP verifier; watch.vision.verify is disabled", command="printer guard") - camera_args = argparse.Namespace(snapshot=None, no_keepalive=False, no_geometry=False) + raise UsageError( + "guard requires the CLIP verifier; watch.vision.verify is disabled", + command="printer guard", + ) + camera_args = argparse.Namespace( + snapshot=None, no_keepalive=False, no_geometry=False + ) camera = _device_camera(device, camera_args) # guard pauses on confirmed spaghetti only: no detached-part check is built for it scorer = build_scorer(cfg, camera=_calibration(device, camera_args), detach=False) if scorer.verifier is None: - raise UsageError("guard requires a loaded CLIP verifier", command="printer guard") + raise UsageError( + "guard requires a loaded CLIP verifier", command="printer guard" + ) score_errors = _frame_score_errors() context = ContextTap(device.host, device.port) if scorer.geometry else None root = state_dir() / "guard" - guard = PauseGuard.arm(make_adapter(device), args.job_id, state_dir=root, - evidence_dir=Path(args.evidence).expanduser() if args.evidence else cache_dir() / "guard", - clock=time.time, sleep=time.sleep, dry_run=args.dry_run, interval=args.interval, - limits=monitor_limits(cfg)) + guard = PauseGuard.arm( + make_adapter(device), + args.job_id, + state_dir=root, + evidence_dir=Path(args.evidence).expanduser() + if args.evidence + else cache_dir() / "guard", + clock=time.time, + sleep=time.sleep, + dry_run=args.dry_run, + interval=args.interval, + limits=monitor_limits(cfg), + ) root.mkdir(parents=True, exist_ok=True) def report(value: dict[str, Any]) -> None: @@ -856,22 +986,9 @@ def report(value: dict[str, Any]) -> None: report(guard._report("armed")) while True: started = time.time() - frame, score, error, score_error = None, None, None, None - try: - camera.keepalive(started) - frame = camera.grab() - if context is not None: - scorer.context = context.poll() - if scorer.context.filename != guard.job.filename: - raise PrinterCommError("camera context belongs to another file", command="printer guard") - except PrinterCommError as exc: - error = str(exc) - else: - # Only scoring is recoverable here. Identity, state writes and PAUSE stay outside. - try: - score = scorer(frame.jpeg) - except score_errors as exc: - score_error = f"{type(exc).__name__}: {exc}"[:2000] + frame, score, error, score_error = _guard_observe_inputs( + camera, scorer, context, guard, score_errors, started + ) result = guard.observe(frame, score) if error: result["camera_error"] = error @@ -883,6 +1000,38 @@ def report(value: dict[str, Any]) -> None: time.sleep(max(0.5, args.interval - (time.time() - started))) +def _guard_observe_inputs( + camera: Any, + scorer: FrameScorer, + context: Any, + guard: Any, + score_errors: tuple[type[Exception], ...], + started: float, +) -> tuple[Any, FrameScore | None, str | None, str | None]: + """One guard tick's (frame, score, camera error, score error).""" + from printer_ops.base import PrinterCommError + + frame, score, error, score_error = None, None, None, None + try: + camera.keepalive(started) + frame = camera.grab() + if context is not None: + scorer.context = context.poll() + if scorer.context.filename != guard.job.filename: + raise PrinterCommError( + "camera context belongs to another file", command="printer guard" + ) + except PrinterCommError as exc: + error = str(exc) + else: + # Only scoring is recoverable here. Identity, state writes and PAUSE stay outside. + try: + score = scorer(frame.jpeg) + except score_errors as exc: + score_error = f"{type(exc).__name__}: {exc}"[:2000] + return frame, score, error, score_error + + def run_record(args: argparse.Namespace) -> int: """`vision record NAME DIR`: while a print is running OR paused (a paused printer is where a failure is still on the bed), save every frame plus where the print is (gcode Z, toolhead @@ -892,7 +1041,6 @@ def run_record(args: argparse.Namespace) -> int: frame was saved within S seconds, so a script never waits forever on an idle printer.""" from errors import GateFailure from printer_ops import make_adapter - from printer_ops.base import PAUSED, PRINTING, PrinterCommError from printer_ops.devices import get_device from printer_ops.vision import ContextTap @@ -907,46 +1055,26 @@ def run_record(args: argparse.Namespace) -> int: with (out / "meta.jsonl").open("a") as meta: while args.max is None or n < args.max: t0 = time.time() - state, active = None, False - try: - state = adapter.status().state - active = state in (PRINTING, PAUSED) - if active: - camera.keepalive(t0) - frame = camera.grab() - ctx = context_tap.poll() - except PrinterCommError as exc: - print( - json.dumps({"t": round(t0, 1), "error": str(exc)}), file=sys.stderr - ) - active = False - if state != last_state: # say why nothing is saved (an idle printer was silent) + state, active, frame, ctx = _record_poll(adapter, camera, context_tap, t0) + if ( + state != last_state + ): # say why nothing is saved (an idle printer was silent) last_state = state print( - json.dumps({"t": round(t0, 1), "state": state, "recording": active}), + json.dumps( + {"t": round(t0, 1), "state": state, "recording": active} + ), file=sys.stderr, flush=True, ) if active: - # milliseconds: two frames inside one second must not overwrite each other - stamp = time.strftime("%Y%m%dT%H%M%S", time.gmtime(frame.ts)) - name = f"{stamp}.{int(frame.ts * 1000) % 1000:03d}Z.jpg" - (out / name).write_bytes(frame.jpeg) - row: dict[str, Any] = { - "frame": name, - "t": round(frame.ts, 2), - "file": ctx.filename, - "state": state, - "z": ctx.z, - "x": round(ctx.x, 2), - "y": round(ctx.y, 2), - } - if ctx.objects != objects: # footprints only when they change - row["objects"] = objects = ctx.objects - meta.write(json.dumps(row) + "\n") - meta.flush() + objects = _record_frame(out, meta, frame, ctx, state, objects) n += 1 - elif args.timeout is not None and n == 0 and time.time() - started >= args.timeout: + elif ( + args.timeout is not None + and n == 0 + and time.time() - started >= args.timeout + ): raise GateFailure( f"{device.name}: no frame saved in {args.timeout:g} s " f"(printer state: {state or 'unreachable'})", @@ -960,6 +1088,50 @@ def run_record(args: argparse.Namespace) -> int: return 0 +def _record_poll( + adapter: Any, camera: Any, context_tap: Any, t0: float +) -> tuple[Any, bool, Any, Any]: + """(printer state, recording?, frame, print context); a comms error only skips this tick.""" + from printer_ops.base import PAUSED, PRINTING, PrinterCommError + + state, active, frame, ctx = None, False, None, None + try: + state = adapter.status().state + active = state in (PRINTING, PAUSED) + if active: + camera.keepalive(t0) + frame = camera.grab() + ctx = context_tap.poll() + except PrinterCommError as exc: + print(json.dumps({"t": round(t0, 1), "error": str(exc)}), file=sys.stderr) + active = False + return state, active, frame, ctx + + +def _record_frame( + out: Path, meta: Any, frame: Any, ctx: Any, state: Any, objects: Any +) -> Any: + """Save one frame plus its meta row; returns the last footprints written.""" + # milliseconds: two frames inside one second must not overwrite each other + stamp = time.strftime("%Y%m%dT%H%M%S", time.gmtime(frame.ts)) + name = f"{stamp}.{int(frame.ts * 1000) % 1000:03d}Z.jpg" + (out / name).write_bytes(frame.jpeg) + row: dict[str, Any] = { + "frame": name, + "t": round(frame.ts, 2), + "file": ctx.filename, + "state": state, + "z": ctx.z, + "x": round(ctx.x, 2), + "y": round(ctx.y, 2), + } + if ctx.objects != objects: # footprints only when they change + row["objects"] = objects = ctx.objects + meta.write(json.dumps(row) + "\n") + meta.flush() + return objects + + def load_replay(path: Path) -> dict[str, tuple[PrintContext, float]]: """frame name -> (PrintContext, frame time) from a `vision record` folder's meta.jsonl; empty when the source is not such a folder.""" @@ -972,9 +1144,7 @@ def load_replay(path: Path) -> dict[str, tuple[PrintContext, float]]: try: row = json.loads(line) if "objects" in row: - objects = tuple( - tuple((float(x), float(y)) for x, y in o) for o in row["objects"] - ) + objects = _footprints(row["objects"]) if "frame" not in row: continue ctx = PrintContext( @@ -990,6 +1160,10 @@ def load_replay(path: Path) -> dict[str, tuple[PrintContext, float]]: return out +def _footprints(raw: Any) -> tuple[tuple[tuple[float, float], ...], ...]: + return tuple(tuple((float(x), float(y)) for x, y in o) for o in raw) + + # ---- evaluation on recorded footage ------------------------------------------------------------ IMAGE_EXT = {".jpg", ".jpeg", ".png"} @@ -997,32 +1171,44 @@ def load_replay(path: Path) -> dict[str, tuple[PrintContext, float]]: def iter_frames(path: Path, every: int = 1) -> Any: """(name, bgr) for a directory of images (sorted), one image, or a video file.""" - import cv2 - if path.is_dir(): - files = sorted(p for p in path.iterdir() if p.suffix.lower() in IMAGE_EXT) - for i, p in enumerate(files): - if i % every == 0: - img = cv2.imread(str(p)) - if img is not None: - yield p.name, img + yield from _iter_dir_frames(path, every) elif path.suffix.lower() in IMAGE_EXT: + import cv2 + img = cv2.imread(str(path)) if img is not None: yield path.name, img else: - cap = cv2.VideoCapture(str(path)) - i = 0 - try: - while True: - ok, img = cap.read() - if not ok: - break - if i % every == 0: - yield f"{path.name}#{i}", img - i += 1 - finally: - cap.release() + yield from _iter_video_frames(path, every) + + +def _iter_dir_frames(path: Path, every: int) -> Any: + import cv2 + + files = sorted(p for p in path.iterdir() if p.suffix.lower() in IMAGE_EXT) + for i, p in enumerate(files): + if i % every == 0: + img = cv2.imread(str(p)) + if img is not None: + yield p.name, img + + +def _iter_video_frames(path: Path, every: int) -> Any: + import cv2 + + cap = cv2.VideoCapture(str(path)) + i = 0 + try: + while True: + ok, img = cap.read() + if not ok: + break + if i % every == 0: + yield f"{path.name}#{i}", img + i += 1 + finally: + cap.release() @dataclass @@ -1058,36 +1244,49 @@ def evaluate( frames (score 0) go first, as if the print had run fine before the clip starts: a failure clip then meets the tracker the way a live failure does, after warm-up. `prepare(source, frame name)` runs before each frame (replays set the print position there).""" - out = [] - for path, label in sources: - if new_print is not None: - new_print() - tracker = FailureTracker(params or TrackerParams()) - for _ in range(lead_in): - tracker.update(0.0) - scores, levels, names, verified, found = [], [], [], [], [] - for name, img in iter_frames(path, every): - if prepare is not None: - prepare(path, name) - s = score_bgr(img) - scores.append(s.p) - if s.q is not None: - verified.append(s.q) - levels.append(tracker.update(s.signal)) - names.append(name) - found.append(tuple(sorted(f.kind for f in s.findings))) - out.append( - SeqResult( - str(path), - label, - scores, - levels, - names, - verified if scores and len(verified) == len(scores) else None, - found, - ) + return [ + _evaluate_source( + path, label, score_bgr, every, params, lead_in, prepare, new_print ) - return out + for path, label in sources + ] + + +def _evaluate_source( + path: Path, + label: str, + score_bgr: Any, + every: int, + params: TrackerParams | None, + lead_in: int, + prepare: Any, + new_print: Any, +) -> SeqResult: + if new_print is not None: + new_print() + tracker = FailureTracker(params or TrackerParams()) + for _ in range(lead_in): + tracker.update(0.0) + scores, levels, names, verified, found = [], [], [], [], [] + for name, img in iter_frames(path, every): + if prepare is not None: + prepare(path, name) + s = score_bgr(img) + scores.append(s.p) + if s.q is not None: + verified.append(s.q) + levels.append(tracker.update(s.signal)) + names.append(name) + found.append(tuple(sorted(f.kind for f in s.findings))) + return SeqResult( + str(path), + label, + scores, + levels, + names, + verified if scores and len(verified) == len(scores) else None, + found, + ) def _table( @@ -1114,26 +1313,75 @@ def metrics( thresholds: tuple[float, ...] = (0.1, 0.2, 0.3, 0.5), interval_s: float = 10.0, ) -> dict[str, Any]: - pos = [p for r in results if r.label == "pos" for p in r.scores] - neg = [p for r in results if r.label == "neg" for p in r.scores] - seqs = [ - { - "source": r.source, - "label": r.label, - "frames": len(r.scores), - "max_p": round(max(r.scores, default=0.0), 3), - "max_q": None - if r.verified is None - else round(max(r.verified, default=0.0), 3), - "first_alert_frame": r.first_alert, - "alert_frames": sum(lv is not None for lv in r.levels), - "critical_frames": sum(lv == "critical" for lv in r.levels), - } + pos = _label_scores(results, "pos") + neg = _label_scores(results, "neg") + seqs = [_sequence_summary(r) for r in results] + days = len(neg) * interval_s / 86400.0 + neg_alerts = _negative_alert_edges(results) + out = { + "frames": {"pos": len(pos), "neg": len(neg)}, + "per_frame": _table(pos, neg, thresholds), + "worst_negatives": [ + {"frame": n, "p": round(p, 3)} for p, n in _worst_negatives(results) + ], + "sequences": seqs, + "sequence_alerts": _sequence_alerts(seqs), + "healthy_days": round(days, 3), + "false_alarms_per_day": round(neg_alerts / days, 2) if days else None, + "finding_alarms_per_day": _finding_rates(_finding_episodes(results), days), + } + if results and all(r.verified is not None for r in results): + out["per_frame_verified"] = _table( + _label_scores(results, "pos", verified=True), + _label_scores(results, "neg", verified=True), + thresholds, + ) + return out + + +def _label_scores( + results: list[SeqResult], label: str, *, verified: bool = False +) -> list[float]: + return [ + p for r in results + if r.label == label + for p in (r.verified or [] if verified else r.scores) ] + + +def _sequence_alerts(seqs: list[dict[str, Any]]) -> dict[str, int]: neg_seq = [s for s in seqs if s["label"] == "neg"] pos_seq = [s for s in seqs if s["label"] == "pos"] - worst = sorted( + return { + "pos_detected": sum(s["first_alert_frame"] is not None for s in pos_seq), + "pos_total": len(pos_seq), + "neg_false_alarms": sum(s["first_alert_frame"] is not None for s in neg_seq), + "neg_total": len(neg_seq), + } + + +def _finding_rates(finding_eps: dict[str, int], days: float) -> dict[str, float]: + if not days: + return {} + return {k: round(v / days, 2) for k, v in sorted(finding_eps.items())} + + +def _sequence_summary(r: SeqResult) -> dict[str, Any]: + return { + "source": r.source, + "label": r.label, + "frames": len(r.scores), + "max_p": round(max(r.scores, default=0.0), 3), + "max_q": None if r.verified is None else round(max(r.verified, default=0.0), 3), + "first_alert_frame": r.first_alert, + "alert_frames": sum(lv is not None for lv in r.levels), + "critical_frames": sum(lv == "critical" for lv in r.levels), + } + + +def _worst_negatives(results: list[SeqResult]) -> list[tuple[float, str]]: + return sorted( ( (p, r.names[i] if i < len(r.names) else f"{r.source}#{i}") for r in results @@ -1142,8 +1390,11 @@ def metrics( ), reverse=True, )[:5] - # False alarms per day of printing on healthy footage: every rising edge of the tracker. - neg_alerts = sum( + + +def _negative_alert_edges(results: list[SeqResult]) -> int: + """False alarms on healthy footage: every rising edge of the tracker.""" + return sum( sum( 1 for i, lv in enumerate(r.levels) @@ -1152,8 +1403,10 @@ def metrics( for r in results if r.label == "neg" ) - days = len(neg) * interval_s / 86400.0 - # Bed-change detectors: episodes (rising edges) per kind on healthy footage. + + +def _finding_episodes(results: list[SeqResult]) -> dict[str, int]: + """Bed-change detectors: episodes (rising edges) per kind on healthy footage.""" finding_eps: dict[str, int] = {} for r in results: if r.label != "neg": @@ -1163,34 +1416,7 @@ def metrics( for k in set(kinds) - set(prev): finding_eps[k] = finding_eps.get(k, 0) + 1 prev = kinds - out = { - "frames": {"pos": len(pos), "neg": len(neg)}, - "per_frame": _table(pos, neg, thresholds), - "worst_negatives": [{"frame": n, "p": round(p, 3)} for p, n in worst], - "sequences": seqs, - "sequence_alerts": { - "pos_detected": sum(s["first_alert_frame"] is not None for s in pos_seq), - "pos_total": len(pos_seq), - "neg_false_alarms": sum( - s["first_alert_frame"] is not None for s in neg_seq - ), - "neg_total": len(neg_seq), - }, - "healthy_days": round(days, 3), - "false_alarms_per_day": round(neg_alerts / days, 2) if days else None, - "finding_alarms_per_day": { - k: round(v / days, 2) for k, v in sorted(finding_eps.items()) - } - if days - else {}, - } - if results and all(r.verified is not None for r in results): - out["per_frame_verified"] = _table( - [q for r in results if r.label == "pos" for q in r.verified or []], - [q for r in results if r.label == "neg" for q in r.verified or []], - thresholds, - ) - return out + return finding_eps def _print_metrics(m: dict[str, Any]) -> None: @@ -1233,42 +1459,14 @@ def _print_metrics(m: dict[str, Any]) -> None: def run_eval(args: argparse.Namespace) -> int: - from errors import UsageError - - sources = [(Path(p), "pos") for p in args.pos] + [ - (Path(p), "neg") for p in args.neg - ] - missing = [str(p) for p, _ in sources if not p.exists()] - if not sources or missing: - raise UsageError( - "eval needs existing --pos/--neg paths" - + (f"; missing: {', '.join(missing)}" if missing else ""), - command="printer vision eval", - ) - cam = None - cfg: dict[str, Any] = {} - if args.printer: - from printer_ops.devices import get_device - from vision_geometry import load_calibration - - cam = load_calibration(args.printer) - if cam is None: - raise UsageError( - f"no camera calibration for printer {args.printer}", - command="printer vision eval", - remediation=[ - "See docs/research/printer-vision-geometry.md; the U1 one ships " - "with 3d-cli for devices with `model: Snapmaker U1`." - ], - ) - # the device's live `watch: vision:` settings (ignore polygons, verify, scene), so eval - # scores what the live loop would - vision_cfg = ( - getattr(get_device(args.printer), "watch", None) or {} - ).get("vision") - cfg = vision_cfg if isinstance(vision_cfg, dict) else {} + sources = _eval_sources(args) + cam, cfg = _eval_camera_cfg(args) scorer = build_scorer( - cfg, verify=not args.no_verify, camera=cam, scene=args.scene, detach=not args.no_detach + cfg, + verify=not args.no_verify, + camera=cam, + scene=args.scene, + detach=not args.no_detach, ) replays = {str(p): load_replay(p) for p, _ in sources} @@ -1309,12 +1507,56 @@ def prepare(path: Path, name: str) -> None: return 0 +def _eval_sources(args: argparse.Namespace) -> list[tuple[Path, str]]: + from errors import UsageError + + sources = [(Path(p), "pos") for p in args.pos] + [ + (Path(p), "neg") for p in args.neg + ] + missing = [str(p) for p, _ in sources if not p.exists()] + if not sources or missing: + raise UsageError( + "eval needs existing --pos/--neg paths" + + (f"; missing: {', '.join(missing)}" if missing else ""), + command="printer vision eval", + ) + return sources + + +def _eval_camera_cfg(args: argparse.Namespace) -> tuple[Any, dict[str, Any]]: + """(calibration, the device's live `watch: vision:` settings); (None, {}) without --printer.""" + from errors import UsageError + + if not args.printer: + return None, {} + from printer_ops.devices import get_device + from vision_geometry import load_calibration + + cam = load_calibration(args.printer) + if cam is None: + raise UsageError( + f"no camera calibration for printer {args.printer}", + command="printer vision eval", + remediation=[ + "See docs/research/printer-vision-geometry.md; the U1 one ships " + "with 3d-cli for devices with `model: Snapmaker U1`." + ], + ) + # the device's live `watch: vision:` settings (ignore polygons, verify, scene), so eval + # scores what the live loop would + return cam, _vision_cfg(get_device(args.printer)) + + def main(argv: list[str]) -> int: from printer_ops.guard import add_arguments ap = argparse.ArgumentParser(prog="3d printer vision") sub = ap.add_subparsers(dest="cmd", required=True) - add_arguments(sub.add_parser("guard", help="explicit one-job automatic PAUSE; local evidence only")) + add_arguments( + sub.add_parser( + "guard", help="explicit one-job automatic PAUSE; local evidence only" + ) + ) live = sub.add_parser("run") live.add_argument("printer") live.add_argument("--interval", type=float, default=10.0) @@ -1327,6 +1569,12 @@ def main(argv: list[str]) -> int: live.add_argument("--no-geometry", action="store_true") live.add_argument("--scene", action="store_true") live.add_argument("--no-detach", action="store_true") + live.add_argument( + "--no-quality", + action="store_true", + help="skip the print-quality tier (stringing debris, lost bead pattern, uneven first " + "layer) and its per-print monitoring report; default follows watch.quality", + ) live.add_argument( "--model", action=argparse.BooleanOptionalAction, @@ -1348,8 +1596,14 @@ def main(argv: list[str]) -> int: ev.add_argument("--no-verify", action="store_true") ev.add_argument("--scene", action="store_true") ev.add_argument("--no-detach", action="store_true") - ev.add_argument("--model", action="store_true", help="also score every frame with tier (b)") - ev.add_argument("--model-class", default="spaghetti", help="tier (b) class the --pos frames show") + ev.add_argument( + "--model", action="store_true", help="also score every frame with tier (b)" + ) + ev.add_argument( + "--model-class", + default="spaghetti", + help="tier (b) class the --pos frames show", + ) ev.add_argument("--printer") ev.add_argument("--json", action="store_true") rec = sub.add_parser("record") @@ -1364,7 +1618,11 @@ def main(argv: list[str]) -> int: fm.add_argument("--tier", choices=("a", "b", "all"), default="all") from vision_fit import add_fit_arguments - add_fit_arguments(sub.add_parser("fit-head", help="fit the tier (b) logistic head on labelled frames")) + add_fit_arguments( + sub.add_parser( + "fit-head", help="fit the tier (b) logistic head on labelled frames" + ) + ) args = ap.parse_args(argv) if args.cmd == "guard": return run_guard(args) diff --git a/lib/vision_quality.py b/lib/vision_quality.py new file mode 100644 index 00000000..fbf41099 --- /dev/null +++ b/lib/vision_quality.py @@ -0,0 +1,235 @@ +"""vision_quality.py — the print-quality tier of `3d printer vision`: is the print GOOD, not just alive? + +Tiers (a) and (b) of the vision stack look for FAILURES (spaghetti, a part off the plate, a foreign object). +None of them knows stringing: on 2026-11-05 the Obico model scored p 0.0 / q 0.0 on all 12 frames of a job that +stringed from layer 2 on (docs/research/print-quality-monitor.md). This tier measures the frame the way the job +watch does (lib/print_quality.py: white debris, bead-pattern order) and judges it with the same pure state +machine (printer_ops/quality.py: the job's OWN first layers are the baseline, persistence, one alert per issue +and level, a debris trend), then hands the result to VisionMonitor as `Finding`s: + + quality__ warning / severe -> a WARNING event: tg --photo FRAME "" via the + watch dispatcher, once per issue and level; NEVER a pause (only CRITICAL events + can reach the auto-pauser, and this tier cannot produce one); + notice -> an INFO event (subscribers that asked for info), report only otherwise. + +PRESENCE: during the first layers (Z <= `first_layer_z`) the print's first frame is the empty-plate reference and + the part stands on the plate once a frame differs from it (the same colour difference as the bed check); past + them a part is always there. A print adopted mid-way has no empty reference: judged from its first layer-above + frame on, its baseline is whatever it looks like then. Needs the camera + position context (`ContextTap`: the print's Z and the plate region at that Z); without it the tier is silent + and says why in `--once`. + +`once`: a one-frame look (`printer vision NAME --once`) has no history: it reports the numbers and the ABSOLUTE + verdict only (printer_ops.quality.absolute_verdict) and says so; a quality finding there is a hint to look at + the frame, not a confirmed one. + +REPORT: `finish()` writes the per-print monitoring report (printer_ops/quality_report.py) into the evidence folder + when the print ends or another one starts. + +INVARIANTS: read-only; heavy imports (numpy / OpenCV, bed_state) are lazy; a measurement error is a `blind` + reading in the report, never an exception in the vision loop. +""" + +from __future__ import annotations + +import sys +from pathlib import Path +from typing import TYPE_CHECKING, Any + +LIB = Path(__file__).resolve().parent +if str(LIB) not in sys.path: + sys.path.insert(0, str(LIB)) + +from printer_ops.quality import ( # noqa: E402 + NOTICE, + OK, + QualityConfig, + QualityFinding, + QualityJudge, + QualityReading, + absolute_verdict, + owner_text, + quality_config, +) +from printer_ops.quality_report import summary_line, write_report # noqa: E402 +from printer_ops.vision import Finding, PrintContext # noqa: E402 + +if TYPE_CHECKING: + import numpy as np + +PRESENT_PROC_W = 480 # the width the bed check compares at (bed_state.PROC_W) + + +class QualityTier: + def __init__( + self, + device: Any, + cfg: QualityConfig, + evidence_dir: Path | None = None, + *, + once: bool = False, + ) -> None: + self.device, self.cfg, self.evidence_dir, self.once = ( + device, + cfg, + evidence_dir, + once, + ) + self.judge = QualityJudge(cfg, printer=device.name) + self.base: tuple[np.ndarray, float] | None = ( + None # the first frame of the print at 480 px, its Z + ) + self.last: dict[str, Any] = {} + + # -- print boundaries + + def reset_print(self) -> None: + """A new print: the report of the one before is written first, then everything starts over.""" + self.finish() + self.judge = QualityJudge(self.cfg, printer=self.device.name) + self.base = None + self.last = {} + + def finish(self) -> Path | None: + """Write the monitoring report of the print that just ended (no readings: nothing to write).""" + report = self.judge.report + if report.readings == 0 or self.evidence_dir is None: + return None + folder = write_report(report, self.evidence_dir, self.device.name) + print( + f"{self.device.name}: {summary_line(report)} -> {folder}", + file=sys.stderr, + flush=True, + ) + self.judge = QualityJudge(self.cfg, printer=self.device.name) + return folder + + # -- one frame + + def update( + self, bgr: np.ndarray, ctx: PrintContext | None, now: float + ) -> list[Finding]: + """Measure, judge, and return the Findings that are owed to the owner right now.""" + if ctx is None: + self.last = { + "status": "blind", + "reason": "no print position context (the plate region needs the Z)", + } + return [] + import print_quality as PQ + + img = PQ.prepare(bgr) + shape = (img.shape[0], img.shape[1]) + region = PQ.shrink(PQ.device_region(self.device, ctx.z, shape)) + numbers, flecks = PQ.measure( + img, region, f"plate surface of {self.device.name} at Z {ctx.z:g}" + ) + self.last = dict(numbers) + if numbers["status"] != "ok": + self.judge.feed(QualityReading(now, numbers, z=ctx.z)) + return [] + label = f"fleck {numbers['fleck_frac']:.3f} order {numbers['order']:.2f}" + image = PQ.annotated_jpeg(img, region, flecks, label) + if self.once: + return self._once(numbers, ctx, image) + reading = QualityReading( + now, numbers, z=ctx.z, present=self._present(img, ctx.z) + ) + v = self.judge.feed(reading) + self.last.update( + state=v.state, + level=v.level, + issue=v.issue, + fleck_ratio=v.fleck_ratio, + order_drop=v.order_drop, + ) + if v.worst: + self._pin(image) + return [self._finding(f, image) for f in v.new] + + def _once( + self, numbers: dict[str, Any], ctx: PrintContext, image: bytes + ) -> list[Finding]: + issue, level = absolute_verdict(numbers, ctx.z, self.cfg) + self.last.update( + basis="absolute: one frame, no job baseline", level=level, issue=issue + ) + if issue is None or level in (OK, NOTICE): + return [] + from printer_ops.quality import QualityVerdict + + text = owner_text(issue, level, QualityVerdict(level, issue=issue)) + data = {"issue": issue, "level": level, "numbers": numbers, "basis": "absolute"} + return [ + self._finding( + QualityFinding(f"quality_{issue}_{level}", issue, level, text, data), + image, + ) + ] + + def _present(self, img: np.ndarray, z: float) -> bool: + """The part stands on the plate. Past the first layers it certainly does; during them the frame must differ + from the print's first (empty-plate) frame.""" + import cv2 + + if z > self.cfg.first_layer_z: + return True + w = PRESENT_PROC_W + small = cv2.resize( + img, + (w, max(1, round(img.shape[0] * w / img.shape[1]))), + interpolation=cv2.INTER_AREA, + ) + if self.base is None: + self.base = (small, z) + return False + return self._differs(self.base[0], small, z) + + def _differs(self, base: np.ndarray, small: np.ndarray, z: float) -> bool: + import bed_part as BP + import bed_state as B + + cfg = B.bed_config(self.device) + region = BP.plate_region(self.device, cfg, z, small.shape[:2]) + th = B.thresholds_for(cfg) + d = B.difference( + base, small, region, th.level, th.detrend_px, th.shift_px, th.open_px + ) + return d.frac >= PRESENT_MIN and d.blob * int(region.sum()) >= PRESENT_BLOB_PX + + # -- helpers + + def _pin(self, image: bytes) -> None: + if self.evidence_dir is None: + return + self.evidence_dir.mkdir(parents=True, exist_ok=True) + path = self.evidence_dir / f"{self.device.name}-quality-worst.jpg" + path.write_bytes(image) + self.judge.pin_worst(str(path)) + + @staticmethod + def _finding(f: QualityFinding, image: bytes) -> Finding: + return Finding(f.kind, f.severity, f.text, dict(f.data), image) + + def once_report(self) -> dict[str, Any]: + """The `quality` block of `vision --once`: the numbers, the verdict and what it is based on.""" + return {k: v for k, v in self.last.items() if k != "annotated"} + + +PRESENT_MIN = 0.004 # the changed share of the plate that says "something stands there" (jobwatch.JobConfig.present_min) +PRESENT_BLOB_PX = ( + 150 # ... and the largest changed region, px at 480 (JobConfig.present_blob_px) +) + + +def build_quality( + device: Any, evidence_dir: Path | None, *, once: bool, disabled: bool +) -> QualityTier | None: + """The tier of a printer from `.watch.quality` (False = off); None when it is off.""" + cfg = quality_config( + (getattr(device, "watch", None) or {}).get("quality"), + getattr(device, "model", None), + ) + if disabled or not cfg.enabled: + return None + return QualityTier(device, cfg, evidence_dir, once=once) diff --git a/tests/data/quality/photo_599472757.jpg b/tests/data/quality/photo_599472757.jpg new file mode 100644 index 00000000..8cd468a0 Binary files /dev/null and b/tests/data/quality/photo_599472757.jpg differ diff --git a/tests/data/quality/photo_599472758.jpg b/tests/data/quality/photo_599472758.jpg new file mode 100644 index 00000000..83f900d2 Binary files /dev/null and b/tests/data/quality/photo_599472758.jpg differ diff --git a/tests/data/quality/photo_599472764.jpg b/tests/data/quality/photo_599472764.jpg new file mode 100644 index 00000000..64ff77b9 Binary files /dev/null and b/tests/data/quality/photo_599472764.jpg differ diff --git a/tests/data/quality/u1_f1.jpg b/tests/data/quality/u1_f1.jpg new file mode 100644 index 00000000..a34cfe84 Binary files /dev/null and b/tests/data/quality/u1_f1.jpg differ diff --git a/tests/data/quality/u1_f10.jpg b/tests/data/quality/u1_f10.jpg new file mode 100644 index 00000000..ae42a93a Binary files /dev/null and b/tests/data/quality/u1_f10.jpg differ diff --git a/tests/data/quality/u1_f12.jpg b/tests/data/quality/u1_f12.jpg new file mode 100644 index 00000000..c05f78cd Binary files /dev/null and b/tests/data/quality/u1_f12.jpg differ diff --git a/tests/data/quality/u1_f5.jpg b/tests/data/quality/u1_f5.jpg new file mode 100644 index 00000000..3d8d444e Binary files /dev/null and b/tests/data/quality/u1_f5.jpg differ diff --git a/tests/data/quality/u1_f7.jpg b/tests/data/quality/u1_f7.jpg new file mode 100644 index 00000000..ec9746a4 Binary files /dev/null and b/tests/data/quality/u1_f7.jpg differ diff --git a/tests/test_print_quality.py b/tests/test_print_quality.py new file mode 100644 index 00000000..860725b9 --- /dev/null +++ b/tests/test_print_quality.py @@ -0,0 +1,232 @@ +"""lib/print_quality.py — the pixel half of the print-quality monitor, on REAL frames and synthetic ones. + +Real: tests/data/quality, downscaled to 640 px — the Snapmaker U1 camera during the 10 h SIMBOL sheet job of +2026-11-05 (`u1_f1` empty plate, `u1_f5` clean first layer, `u1_f7` / `u1_f10` / `u1_f12` stringing debris of +layers 2-4 that nothing flagged on the day) and three phone photos of the failed first attempt (a stringy first +layer, a close-up of it, a coupon with a blob at its end). The numbers are calibrated on exactly these (and on the +other seven full-size frames, docs/research/print-quality-monitor.md), so this file pins the calibration: a change +of the constants that stops separating clean from stringy frames fails here. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from printer_ops import quality as Q + +np = pytest.importorskip("numpy") +cv2 = pytest.importorskip("cv2") +import print_quality as PQ # noqa: E402 + +DATA = Path(__file__).parent / "data" / "quality" +DEVICES = """u1: + adapter: moonraker + host: 127.0.0.1 + port: 1 + model: Snapmaker U1 +""" + + +@pytest.fixture +def u1(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> str: + monkeypatch.setenv("XDG_CONFIG_HOME", str(tmp_path / "config")) + (tmp_path / "config" / "3d-cli").mkdir(parents=True) + (tmp_path / "config" / "3d-cli" / "printer-devices.yaml").write_text( + DEVICES, encoding="utf-8" + ) + return "u1" + + +def read(name: str, u1: str, z: float = 0.6) -> dict: + return PQ.quality_numbers((DATA / name).read_bytes(), name=u1, z=z) + + +def whole(name: str) -> dict: + return PQ.quality_numbers((DATA / name).read_bytes(), name=None, z=0.0) + + +def test_clean_first_layer_vs_stringing_debris_separate_on_the_real_camera( + u1: str, +) -> None: + clean = read("u1_f5.jpg", u1) + assert clean["status"] == "ok" and clean["region_source"].startswith( + "plate surface of u1" + ) + assert clean["fleck_frac"] < 0.02 and clean["order"] > 0.8 + for name in ("u1_f7.jpg", "u1_f10.jpg", "u1_f12.jpg"): + bad = read(name, u1) + assert bad["fleck_frac"] > 2.4 * clean["fleck_frac"], name + assert bad["fleck_frac"] > 0.035 and bad["order"] < 0.75, name + + +def test_the_empty_plate_has_its_own_glints_which_is_why_presence_is_not_the_numbers_job( + u1: str, +) -> None: + """The bare textured plate sparkles (fleck_frac 0.1): the decision side only judges a plate with a part.""" + assert read("u1_f1.jpg", u1)["fleck_frac"] > 0.06 + + +def test_the_numbers_do_not_depend_on_the_camera_resolution( + u1: str, tmp_path: Path +) -> None: + img = cv2.imread(str(DATA / "u1_f7.jpg")) + big = cv2.resize(img, (1920, 1080), interpolation=cv2.INTER_LINEAR) + ok, buf = cv2.imencode(".jpg", big, [cv2.IMWRITE_JPEG_QUALITY, 90]) + assert ok + a, b = read("u1_f7.jpg", u1), PQ.quality_numbers(bytes(buf), name=u1, z=0.6) + assert ( + abs(a["fleck_frac"] - b["fleck_frac"]) < 0.03 + and abs(a["order"] - b["order"]) < 0.05 + ) + + +@pytest.mark.parametrize( + "name", ["photo_599472757.jpg", "photo_599472758.jpg", "photo_599472764.jpg"] +) +def test_phone_photos_of_the_failed_first_layer_read_far_below_a_clean_first_layer( + name: str, +) -> None: + """Whole-frame look (no camera calibration for a phone): the beads are a mess, `order` is far under the + first-layer floor, a clean first layer reads 0.83+.""" + n = whole(name) + assert n["status"] == "ok" and n["region_source"] == "whole frame" + cfg = Q.quality_config(None, "Snapmaker U1") + assert n["order"] < cfg.first_layer_order + issue, level = Q.absolute_verdict(n, 0.2, cfg) + assert issue == Q.FIRST_LAYER and level in (Q.WARNING, Q.SEVERE) + + +def synth(white_specks: int = 0, lines: bool = True, seed: int = 1) -> bytes: + """A 640x360 beige surface of parallel beads, optionally peppered with white specks.""" + rng = np.random.default_rng(seed) + img = np.full((360, 640, 3), (120, 165, 205), np.uint8) # BGR beige + if lines: + for y in range(0, 360, 9): + img[y : y + 3] = ( + 104, + 148, + 188, + ) # shallow gaps between beads: far under the fleck threshold + for _ in range(white_specks): + x, y = int(rng.integers(10, 630)), int(rng.integers(10, 350)) + cv2.circle(img, (x, y), int(rng.integers(1, 4)), (245, 245, 245), -1) + ok, buf = cv2.imencode(".png", img) + assert ok + return bytes(buf) + + +def test_synthetic_white_specks_raise_fleck_frac_and_lost_lines_lower_order() -> None: + clean = PQ.quality_numbers(synth(0), name=None, z=0.0) + specks = PQ.quality_numbers(synth(250), name=None, z=0.0) + mush = PQ.quality_numbers(synth(0, lines=False), name=None, z=0.0) + assert clean["fleck_frac"] < 0.005 and specks["fleck_frac"] > 0.015 + assert clean["order"] > 0.9 and specks["order"] < clean["order"] + assert mush["order"] < clean["order"], ( + "an even surface with no bead pattern has nothing to be ordered" + ) + + +def test_a_region_that_is_too_small_or_a_broken_frame_is_blind_never_clean() -> None: + tiny = PQ.quality_numbers( + synth(0), name=None, z=0.0, roi=[(0.1, 0.1), (0.12, 0.1), (0.12, 0.12)] + ) + assert tiny["status"] == "blind" and "under" in tiny["reason"] + broken = PQ.quality_numbers(b"not a jpeg", name=None, z=0.0) + assert broken["status"] == "blind" and "decodable" in broken["reason"] + + +def test_roi_parsing_names_the_bad_part() -> None: + assert PQ.parse_roi("0,0;1,0;1,1") == [(0.0, 0.0), (1.0, 0.0), (1.0, 1.0)] + for bad in ("0,0;1,0", "0,0;2,0;1,1", "a,b;c,d;e,f"): + with pytest.raises(ValueError): + PQ.parse_roi(bad) + + +def test_the_annotated_frame_marks_the_debris_in_red_and_is_a_jpeg( + u1: str, tmp_path: Path +) -> None: + out = tmp_path / "a" / "marked.jpg" + n = PQ.quality_numbers( + (DATA / "u1_f7.jpg").read_bytes(), name=u1, z=0.6, annotated=out + ) + assert n["annotated"] == str(out) and out.read_bytes()[:2] == b"\xff\xd8" + marked = cv2.imread(str(out)) + assert (marked[..., 2].astype(int) - marked[..., 0].astype(int) > 150).sum() > 200 + + +def test_the_command_line_prints_one_json_object_and_help( + u1: str, tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + assert ( + PQ.main( + ["check", u1, "--frame", str(DATA / "u1_f5.jpg"), "--z", "0.6", "--json"] + ) + == 0 + ) + doc = json.loads(capsys.readouterr().out) + assert doc["status"] == "ok" and { + "fleck_frac", + "fleck_per10k", + "order", + "luma", + "region_px", + } <= set(doc) + assert ( + PQ.main( + [ + "check", + u1, + "--frame", + str(DATA / "photo_599472757.jpg"), + "--roi", + "whole", + "--json", + ] + ) + == 0 + ) + assert json.loads(capsys.readouterr().out)["region_source"] == "whole frame" + assert ( + PQ.main(["--help"]) == 0 + and "print_quality.py check NAME" in capsys.readouterr().out + ) + assert PQ.main([]) == 1 + + +# ---- the table of the calibration, end to end through the judge -------------------------------- + + +def test_verdict_table_of_the_session_frames_through_the_judge(u1: str) -> None: + """f1 empty (reference, not judged); the first layer f5 four times = the baseline; then the debris frames in + the order they happened. Nothing before the debris, a WARNING on it, never above WARNING-severity events.""" + judge = Q.QualityJudge(Q.quality_config(None, "Snapmaker U1")) + rows = [] + sequence = [("u1_f1.jpg", False, 1)] + [("u1_f5.jpg", True, 1)] * 5 + sequence += [ + ("u1_f7.jpg", True, 3), + ("u1_f10.jpg", True, 3), + ("u1_f12.jpg", True, 4), + ("u1_f12.jpg", True, 4), + ] + for i, (name, present, layer) in enumerate(sequence): + v = judge.feed( + Q.QualityReading(60.0 * i, read(name, u1), layer=layer, present=present) + ) + rows.append((name, v.state, v.level, [f.kind for f in v.new])) + assert [r[1] for r in rows[:6]] == [ + "absent", + "learning", + "learning", + "learning", + "learning", + "ok", + ] + assert all(r[2] == "ok" for r in rows[:6]) + warned = [r for r in rows if r[3]] + assert [r[3] for r in warned] == [["quality_stringing_warning"]] + assert judge.report.worst is not None + assert judge.report.max_level == "warning" and judge.report.worst["layer"] == 3 + assert all(f["level"] in ("warning", "severe") for f in judge.report.alerts) diff --git a/tests/test_printer_jobwatch.py b/tests/test_printer_jobwatch.py index 0385eb70..541a92a2 100644 --- a/tests/test_printer_jobwatch.py +++ b/tests/test_printer_jobwatch.py @@ -36,6 +36,7 @@ port: 1 model: Creality K1 SE watch: + quality: false vision: {snapshot: "http://127.0.0.1:1/?action=snapshot"} bed: enabled: true @@ -439,7 +440,9 @@ def test_a_camera_that_jumps_beyond_what_can_be_followed_is_blind_never_gone( assert "the plate cannot be followed on the picture" in out -def pixel_check(tmp_path: Path, frame: str, z: float, chained: bool = False) -> dict[str, Any]: +def pixel_check( + tmp_path: Path, frame: str, z: float, chained: bool = False +) -> dict[str, Any]: """bed_part.py on one frame against DATA/empty.jpg (Z 0.2); `chained`: start from the files of the last call.""" argv = [ *("check", "k1se", "--json", "--baseline", str(DATA / "empty.jpg")), diff --git a/tests/test_printer_quality.py b/tests/test_printer_quality.py new file mode 100644 index 00000000..3cd2c373 --- /dev/null +++ b/tests/test_printer_quality.py @@ -0,0 +1,341 @@ +"""printer_ops/quality.py — the decision half of the print-quality monitor, on plain numbers (no pixels). + +Numbers are what lib/print_quality.py reports for a frame: `fleck_frac` (white debris share) and `order` (bead +pattern regularity). The calibration frames are in tests/test_print_quality.py; here the STATE MACHINE is proved: +baseline from the job's own first layers, persistence, one alert per issue and level, never CRITICAL, clear, +trend, the first-layer floor, the report. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +import pytest + +from errors import InvalidArgument +from printer_ops import quality as Q +from printer_ops.quality_report import render_markdown, summary_line, write_report + +U1 = Q.quality_config( + None, "Snapmaker U1" +) # the calibrated camera: the first-layer floor is on +CLEAN = { + "status": "ok", + "fleck_frac": 0.014, + "fleck_per10k": 9.0, + "order": 0.84, + "luma": 140.0, +} + + +def nums(fleck: float = 0.014, order: float = 0.84) -> dict[str, Any]: + return {**CLEAN, "fleck_frac": fleck, "order": order} + + +class Clock: + def __init__(self, step: float = 60.0) -> None: + self.t = 0.0 + self.step = step + + def next(self) -> float: + self.t += self.step + return self.t + + +def judge_with_baseline( + cfg: Q.QualityConfig | None = None, +) -> tuple[Q.QualityJudge, Clock]: + judge, clock = ( + Q.QualityJudge(cfg or Q.QualityConfig(), printer="u1", job="sheet.gcode"), + Clock(), + ) + for _ in range(4): + v = judge.feed(Q.QualityReading(clock.next(), nums(), layer=1)) + assert v.state == Q.LEARNING + assert judge.base is not None + return judge, clock + + +def feed( + judge: Q.QualityJudge, clock: Clock, n: int, **kw: float +) -> list[Q.QualityVerdict]: + return [ + judge.feed(Q.QualityReading(clock.next(), nums(**kw), layer=5, z=1.0)) + for _ in range(n) + ] + + +def test_the_first_readings_with_the_part_on_the_plate_are_the_baseline_and_are_never_judged() -> ( + None +): + judge = Q.QualityJudge(Q.QualityConfig()) + for _ in range(3): + assert ( + judge.feed(Q.QualityReading(0, nums(0.09, 0.84), layer=1)).state + == Q.LEARNING + ) + assert judge.base is None + v = judge.feed(Q.QualityReading(1, nums(0.09, 0.84), layer=1)) + assert ( + v.state == Q.LEARNING + and judge.base is not None + and judge.base.fleck == pytest.approx(0.09) + ) + + +def test_blind_and_absent_readings_are_counted_but_never_judged_and_never_clean() -> ( + None +): + judge = Q.QualityJudge() + assert ( + judge.feed( + Q.QualityReading(0, {"status": "blind", "reason": "plate out of view"}) + ).state + == Q.BLIND + ) + assert judge.feed(Q.QualityReading(1, nums(), present=False)).state == Q.ABSENT + assert ( + judge.report.skipped == {"blind": 1, "absent": 1} + and sum(judge.report.per_level.values()) == 0 + ) + assert judge.report.max_level == Q.OK + + +def test_debris_is_stringing_and_a_single_bad_frame_is_not_a_finding() -> None: + judge, clock = judge_with_baseline() + (one,) = feed(judge, clock, 1, fleck=0.08, order=0.80) + assert ( + one.raw == Q.SEVERE + and one.issue == Q.STRINGING + and one.level == Q.OK + and one.new == [] + ) + (clean,) = feed(judge, clock, 1) + assert clean.new == [] and clean.level == Q.OK + + +def test_two_bad_readings_warn_once_then_severe_once_and_never_again() -> None: + judge, clock = judge_with_baseline() + cfg = judge.cfg + verdicts = feed(judge, clock, 2, fleck=0.045, order=0.83) # warning level twice + assert [f.kind for v in verdicts for f in v.new] == ["quality_stringing_warning"] + sev = feed(judge, clock, 3, fleck=0.09, order=0.70) # severe three times + assert [f.kind for v in sev for f in v.new] == ["quality_stringing_severe"] + assert judge.level == Q.SEVERE + again = feed(judge, clock, 5, fleck=0.09, order=0.70) + assert [f for v in again for f in v.new] == [], "one alert per issue and level" + assert cfg.severe_confirm == 3 and judge.report.max_level == Q.SEVERE + assert [a["kind"] for a in judge.report.alerts] == [ + "quality_stringing_warning", + "quality_stringing_severe", + ] + + +def test_a_quality_finding_is_never_critical_and_maps_to_watch_severities() -> None: + assert set(Q.WATCH_SEVERITY.values()) == {"info", "warning"} + judge, clock = judge_with_baseline() + out = [f for v in feed(judge, clock, 6, fleck=0.1, order=0.6) for f in v.new] + assert out and all(f.severity in ("info", "warning") for f in out) + assert all("critical" not in f.severity for f in out) + + +def test_the_level_clears_after_clean_readings_and_may_warn_again_only_for_a_new_level() -> ( + None +): + judge, clock = judge_with_baseline() + feed(judge, clock, 3, fleck=0.045) + assert judge.level == Q.WARNING + cleared = feed(judge, clock, judge.cfg.clear_after) + assert cleared[-1].level == Q.OK and "back to normal" in " ".join(cleared[-1].notes) + again = feed(judge, clock, 3, fleck=0.045) + assert [f for v in again for f in v.new] == [], ( + "already told once for this issue and level" + ) + + +def test_pattern_loss_without_debris_is_texture() -> None: + judge, clock = judge_with_baseline() + verdicts = feed(judge, clock, 3, fleck=0.015, order=0.66) # order dropped 0.18 + kinds = [f.kind for v in verdicts for f in v.new] + assert kinds and kinds[0].startswith("quality_texture_") + assert "полосы" in [f.text for v in verdicts for f in v.new][0] + + +def test_the_first_layer_floor_needs_no_baseline_and_a_bad_first_layer_never_makes_the_baseline() -> ( + None +): + judge = Q.QualityJudge(U1) + verdicts = [ + judge.feed(Q.QualityReading(float(i), nums(0.02, 0.40), layer=1)) + for i in range(2) + ] + assert verdicts[0].issue == Q.FIRST_LAYER and verdicts[0].raw == Q.SEVERE + assert judge.base is None and judge.report.baseline is None + kinds = [f.kind for v in verdicts for f in v.new] + assert ( + kinds == ["quality_first_layer_warning"] + or kinds == ["quality_first_layer_severe"] + or kinds[0].startswith("quality_first_layer") + ) + assert "первый слой" in [f.text for v in verdicts for f in v.new][0] + + +def test_without_a_layer_number_z_decides_what_a_first_layer_is() -> None: + judge = Q.QualityJudge(U1) + assert ( + judge.feed(Q.QualityReading(0, nums(0.02, 0.5), z=0.3)).issue == Q.FIRST_LAYER + ) + assert judge.feed(Q.QualityReading(1, nums(0.02, 0.5), z=5.0)).issue is None + + +def test_a_rising_debris_trend_lifts_a_notice_to_a_warning() -> None: + judge, clock = judge_with_baseline( + Q.QualityConfig(notice_abs=0.02, warning_abs=0.09, severe_abs=0.2) + ) + verdicts = [] + for i in range(8): # 1.9x .. 2.5x the baseline: notice level by ratio, rising fast + v = judge.feed( + Q.QualityReading( + clock.next(), nums(fleck=0.021 + 0.003 * i), layer=5, z=1.0 + ) + ) + verdicts.append(v) + assert any( + v.trend is not None and v.trend >= judge.cfg.trend_rise for v in verdicts + ) + assert any("rising" in " ".join(v.notes) for v in verdicts) + + +def test_a_clean_reading_moves_the_baseline_a_debris_reading_does_not() -> None: + judge, clock = judge_with_baseline() + before = judge.base.fleck # type: ignore[union-attr] + feed(judge, clock, 3, fleck=0.016) + assert judge.base.fleck > before # type: ignore[union-attr] + grown = judge.base.fleck # type: ignore[union-attr] + feed(judge, clock, 3, fleck=0.09) + assert judge.base.fleck == grown # type: ignore[union-attr] + + +def test_the_notice_level_is_info_only_and_a_spotless_baseline_has_a_floor() -> None: + judge = Q.QualityJudge(Q.QualityConfig()) + for _ in range(4): + judge.feed(Q.QualityReading(0, nums(0.001), layer=1)) + clock = Clock() + # 0.031 is 31x a spotless baseline but only 2.6x its floor, and under the warning's absolute level + new = [ + f + for _ in range(3) + for f in judge.feed( + Q.QualityReading(clock.next(), nums(0.031), layer=5, z=1.0) + ).new + ] + assert [f.kind for f in new] == ["quality_stringing_notice"] + assert new[0].severity == "info" and judge.level == Q.NOTICE + + +def test_a_slowly_growing_debris_is_not_absorbed_into_the_baseline() -> None: + judge, clock = judge_with_baseline() + for i in range(60): + judge.feed( + Q.QualityReading( + clock.next(), nums(fleck=0.014 + 0.0008 * i), layer=5, z=1.0 + ) + ) + assert ( + judge.base is not None + and judge.base.fleck <= 0.014 * judge.cfg.adapt_fleck_cap + 1e-9 + ) + assert judge.level != Q.OK, ( + "0.062 after an hour of creeping is stringing, however slowly it came" + ) + + +def test_absolute_verdict_is_what_a_single_frame_can_say() -> None: + cfg = U1 + assert Q.absolute_verdict(nums(0.014), 1.2, cfg) == (None, Q.OK) + assert Q.absolute_verdict(nums(0.04), 1.2, cfg) == (Q.STRINGING, Q.WARNING) + assert Q.absolute_verdict(nums(0.08), 1.2, cfg) == (Q.STRINGING, Q.SEVERE) + assert Q.absolute_verdict(nums(0.014, 0.5), 0.3, cfg) == (Q.FIRST_LAYER, Q.WARNING) + assert Q.absolute_verdict(nums(0.014, 0.5), 5.0, cfg) == (None, Q.OK), ( + "the floor is for first layers only" + ) + assert Q.absolute_verdict({"status": "blind"}, 1.0, cfg) == (None, Q.OK) + assert Q.absolute_verdict(nums(0.09, 0.84), 0.3, cfg) == (None, Q.OK), ( + "a bare first-layer plate sparkles" + ) + + +def test_owner_text_is_russian_short_and_says_what_to_change() -> None: + v = Q.QualityVerdict(Q.WARNING, fleck_ratio=3.2) + text = Q.owner_text(Q.STRINGING, Q.WARNING, v) + assert ( + "ниточки" in text and "3.2" in text and "Что менять" in text and len(text) < 330 + ) + assert Q.owner_text( + Q.TEXTURE, Q.SEVERE, Q.QualityVerdict(Q.SEVERE, order_drop=0.2) + ).startswith("усиливается") + + +def test_the_first_layer_floor_is_on_only_for_a_calibrated_camera_or_when_configured() -> ( + None +): + assert Q.QualityConfig().first_layer_order == 0.0 and U1.first_layer_order == 0.6 + assert Q.quality_config(None, "Creality K1 SE").first_layer_order == 0.0 + assert ( + Q.quality_config({"first_layer_order": 0.5}, "Creality K1 SE").first_layer_order + == 0.5 + ) + assert ( + Q.quality_config( + {"first_layer_order": 0.0}, "snapmaker u1 (2026)" + ).first_layer_order + == 0.0 + ) + off = Q.QualityJudge(Q.QualityConfig()) + assert off.feed(Q.QualityReading(0, nums(0.02, 0.2), layer=1)).issue is None, ( + "a part that reads 0.2 is normal on some cameras" + ) + + +def test_quality_config_from_devices_yaml() -> None: + assert ( + Q.quality_config(None) == Q.QualityConfig() + and Q.quality_config(False).enabled is False + ) + assert Q.quality_config({"baseline_n": 6, "warning_ratio": 2}).baseline_n == 6 + for bad in ({"nope": 1}, {"baseline_n": "x"}, {"enabled": 1.5}, [1]): + with pytest.raises(InvalidArgument): + Q.quality_config(bad) + + +def test_the_report_counts_everything_and_renders_for_a_human(tmp_path: Path) -> None: + judge, clock = judge_with_baseline() + feed(judge, clock, 2) + feed(judge, clock, 4, fleck=0.09, order=0.62) + judge.feed(Q.QualityReading(clock.next(), {"status": "blind", "reason": "x"})) + rep = judge.report + assert rep.readings == 11 and rep.skipped == {"learning": 4, "blind": 1} + assert rep.per_level["ok"] == 2 and rep.per_level["severe"] == 4 + assert rep.worst is not None and rep.worst["level"] == "severe" + assert "11 readings (6 judged, 5 not" in summary_line(rep) + md = render_markdown(rep) + assert ( + "| severe | 4 |" in md + and "Baseline (the job's own first layers, 4 readings)" in md + ) + frame = tmp_path / "frame.jpg" + frame.write_bytes(b"jpeg") + rep.worst["frame"] = str(frame) + folder = write_report(rep, tmp_path / "runs", "u1") + assert folder.name.endswith("-u1-monitor-report") + assert json.loads((folder / "report.json").read_text())["max_level"] == "severe" + assert (folder / "worst-frame.jpg").read_bytes() == b"jpeg" + + +def test_a_job_with_nothing_wrong_still_gets_an_honest_report() -> None: + judge, clock = judge_with_baseline() + feed(judge, clock, 5) + md = render_markdown(judge.report) + assert "Alerts sent: none." in md and "Worst reading: none" in md diff --git a/tests/test_printer_quality_watch.py b/tests/test_printer_quality_watch.py new file mode 100644 index 00000000..0e0404cf --- /dev/null +++ b/tests/test_printer_quality_watch.py @@ -0,0 +1,308 @@ +"""`3d printer watch NAME --job`: the print-quality tier on REAL U1 frames (tests/data/quality). + +The frames are the camera of the Snapmaker U1 during the 10 h SIMBOL sheet job of 2026-11-05, downscaled to 640 px: +`u1_f1` the empty plate, `u1_f5` the clean first layer, `u1_f7` / `u1_f10` / `u1_f12` the stringing debris of layers +2-4 that NOTHING flagged on the day (the spaghetti model scored 0.0 on all twelve frames, the job watch only asks +whether the part is still there). The printer is a fixture scenario on a virtual clock; the real `tg` is refused +under pytest and the fixture forces its dry run, so nothing here touches a printer, a camera or a phone. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +import pytest + +from printer_ops import jobwatch_run as R + +np = pytest.importorskip("numpy") +cv2 = pytest.importorskip("cv2") +import bed_part # noqa: E402 (needs numpy + OpenCV) +import print_quality # noqa: E402 + +DATA = Path(__file__).parent / "data" / "quality" +DEVICES = """u1: + adapter: moonraker + host: 127.0.0.1 + port: 1 + model: Snapmaker U1 + watch: + vision: {snapshot: "http://127.0.0.1:1/?action=snapshot"} +""" + + +@pytest.fixture(autouse=True) +def xdg(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + for var, sub in ( + ("XDG_CONFIG_HOME", "config"), + ("XDG_STATE_HOME", "state"), + ("XDG_DATA_HOME", "data"), + ("XDG_CACHE_HOME", "cache"), + ): + monkeypatch.setenv(var, str(tmp_path / sub)) + cfg = tmp_path / "config" / "3d-cli" + cfg.mkdir(parents=True) + (cfg / "printer-devices.yaml").write_text(DEVICES, encoding="utf-8") + monkeypatch.chdir(tmp_path) + monkeypatch.setattr( + R, + "run_part_tool", + lambda argv: bed_part.run_check(bed_part.build_parser().parse_args(argv)), + ) + monkeypatch.setattr( + R, + "run_quality_tool", + lambda argv: print_quality.run_check( + print_quality.build_parser().parse_args(argv) + ), + ) + return tmp_path + + +def at(t: float, frame: str, layer: int) -> dict[str, Any]: + return { + "t": t, + "state": "printing", + "layer": layer, + "total_layer": 100, + "progress": layer / 100, + "z": 0.2 + 0.2 * layer, + "frame": frame, + } + + +def end(t: float) -> dict[str, Any]: + return { + "t": t, + "state": "complete", + "layer": 100, + "progress": 1.0, + "frame": "u1_f1.jpg", + } + + +# the first layer is on the plate (the empty plate is the job's baseline frame), then debris grows on layers 2-4 +HEALTHY = [ + at(0, "u1_f1.jpg", 1), + *[at(15 * i, "u1_f5.jpg", 1) for i in range(1, 6)], + at(120, "u1_f5.jpg", 2), +] +STRINGY = [ + at(180, "u1_f7.jpg", 3), + at(240, "u1_f10.jpg", 3), + at(300, "u1_f12.jpg", 4), + at(360, "u1_f12.jpg", 4), +] + + +def watch( + tmp_path: Path, + steps: list[dict[str, Any]], + *flags: str, + capsys: pytest.CaptureFixture[str], +) -> tuple[int, str, list[dict[str, Any]]]: + for name in {s["frame"] for s in steps}: + (tmp_path / name).write_bytes((DATA / name).read_bytes()) + scenario = tmp_path / "scenario.json" + scenario.write_text( + json.dumps({"printer": "u1", "file": "sheet.gcode", "steps": steps}), + encoding="utf-8", + ) + code = R.run( + [ + "u1", + "--job", + "--fixture", + str(scenario), + "--out", + str(tmp_path / "runs"), + "--duration", + "1500", + "--checkpoints", + "none", + *flags, + ] + ) + out = capsys.readouterr().out + outcomes = [ + json.loads(line)["jobwatch"] + for line in out.splitlines() + if line.startswith('{"jobwatch"') + ] + return code, out, outcomes + + +def would_run(out: str) -> list[list[str]]: + return [ + json.loads(line)["would_run"] + for line in out.splitlines() + if line.startswith('{"would_run"') + ] + + +def quality_only(outcomes: list[dict[str, Any]]) -> list[dict[str, Any]]: + """The job watch's own part check (bed_part) also reads these frames: the toolhead and the light differ from + frame to frame, so it may add a `part_moved` WARNING of its own. This file is about the quality tier.""" + return [o for o in outcomes if o["findings"][0]["kind"].startswith("quality_")] + + +def report_dir(tmp_path: Path) -> Path: + (d,) = (tmp_path / "runs").glob("*-u1-monitor-report") + return d + + +def test_stringing_debris_on_real_frames_is_warned_once_and_never_stops_the_job( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + code, out, outcomes = watch( + tmp_path, [*HEALTHY, *STRINGY, end(500)], "--tg-dry-run", capsys=capsys + ) + assert code == 0, "a quality finding never fails the watch" + (alert,) = quality_only(outcomes) + assert alert["findings"][0]["kind"] == "quality_stringing_warning" + assert alert["action"] == "alert" + assert all( + s["name"] not in ("pause", "cancel", "call") + for o in outcomes + for s in o["steps"] + ) + msgs = [m for m in would_run(out) if "ниточки" in m[-1]] + assert ( + len(msgs) == 1 + and msgs[0][:3] == ["tg", "--tag", "problem"] + and "--photo" in msgs[0] + ) + assert all(m[:2] != ["tg", "call"] for m in would_run(out)), ( + "the owner is not rung for a bad surface" + ) + text = msgs[0][-1] + assert "белые ниточки" in text and "Что менять" in text and "температур" in text + assert "Evidence:" in text # the owner can find the frame + + +def test_the_clean_first_layer_raises_nothing_and_the_report_says_it_was_checked( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + code, out, outcomes = watch( + tmp_path, + [*HEALTHY, at(300, "u1_f5.jpg", 4), end(400)], + "--tg-dry-run", + capsys=capsys, + ) + assert code == 0 and quality_only(outcomes) == [] + doc = json.loads((report_dir(tmp_path) / "report.json").read_text()) + assert ( + doc["max_level"] == "ok" + and doc["per_level"]["warning"] == 0 + and doc["per_level"]["severe"] == 0 + ) + assert doc["judged"] >= 1 and doc["baseline"]["n"] == 4 + assert doc["skipped"]["absent"] >= 1, ( + "the empty plate of the first frame is not judged" + ) + assert "print-quality monitor:" in out and "monitor-report" in out + + +def test_the_monitoring_report_counts_severities_and_keeps_the_worst_frame( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + watch(tmp_path, [*HEALTHY, *STRINGY, end(500)], "--tg-dry-run", capsys=capsys) + d = report_dir(tmp_path) + doc = json.loads((d / "report.json").read_text()) + assert doc["max_level"] == "warning" + assert ( + doc["per_level"]["severe"] >= 1 + and doc["per_level"]["warning"] >= 2 + and doc["per_issue"]["stringing"] >= 3 + ) + assert [a["kind"] for a in doc["alerts"]] == ["quality_stringing_warning"] + assert ( + doc["worst"]["issue"] == "stringing" + and doc["worst"]["level"] == "severe" + and doc["worst"]["layer"] == 3 + ) + assert (d / "worst-frame.jpg").is_file() and ( + d / "worst-frame.jpg" + ).stat().st_size > 1000 + md = (d / "report.md").read_text() + assert ( + "| severe |" in md + and "Alerts (one per issue and level" in md + and "worst-frame.jpg" in md + ) + + +def test_a_quality_alert_ships_the_frame_with_the_debris_marked( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + _, _, outcomes = watch( + tmp_path, [*HEALTHY, *STRINGY, end(500)], "--tg-dry-run", capsys=capsys + ) + d = Path(quality_only(outcomes)[0]["evidence_dir"]) + assert (d / "frame.quality.jpg").is_file() and (d / "frame.raw.jpg").is_file() + marked = cv2.imread(str(d / "frame.quality.jpg")) + assert ( + marked[..., 2].astype(int) - marked[..., 0].astype(int) > 150 + ).sum() > 200, "red fleck pixels drawn in" + + +def test_no_quality_turns_the_tier_and_its_report_off( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + code, out, outcomes = watch( + tmp_path, + [*HEALTHY, *STRINGY, end(500)], + "--tg-dry-run", + "--no-quality", + capsys=capsys, + ) + assert code == 0 and quality_only(outcomes) == [] + assert not list((tmp_path / "runs").glob("*monitor-report")) + assert "print-quality monitor" not in out + + +def test_the_report_is_written_even_when_a_lost_part_ends_the_watch( + tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + """What the monitor saw until the part came off is part of the evidence.""" + steps = [ + *HEALTHY, + at(180, "u1_f7.jpg", 3), + at(240, "u1_f10.jpg", 3), + at(300, "u1_f1.jpg", 20), + at(360, "u1_f1.jpg", 21), + ] + code, _, _ = watch(tmp_path, steps, "--tg-dry-run", capsys=capsys) + assert code == 3 + doc = json.loads((report_dir(tmp_path) / "report.json").read_text()) + assert doc["readings"] >= 5 + + +def test_a_watch_quality_block_tunes_the_thresholds_and_a_bad_key_is_refused( + tmp_path: Path, capsys: pytest.CaptureFixture[str], xdg: Path +) -> None: + from errors import InvalidArgument + + devices = xdg / "config" / "3d-cli" / "printer-devices.yaml" + unreachable = "{warning_ratio: 99, severe_ratio: 99, notice_ratio: 99, warning_drop: 9, severe_drop: 9, notice_drop: 9}" + devices.write_text(DEVICES + f" quality: {unreachable}\n", encoding="utf-8") + code, _, outcomes = watch( + tmp_path, [*HEALTHY, *STRINGY, end(500)], "--tg-dry-run", capsys=capsys + ) + assert code == 0 and quality_only(outcomes) == [], ( + "thresholds nobody can reach: nothing alerts" + ) + devices.write_text(DEVICES + " quality: {no_such_key: 1}\n", encoding="utf-8") + with pytest.raises(InvalidArgument, match="no_such_key"): + R.run( + [ + "u1", + "--job", + "--fixture", + str(tmp_path / "scenario.json"), + "--wait-start", + "5", + ] + ) diff --git a/tests/test_printer_vision_cycle.py b/tests/test_printer_vision_cycle.py index 3537dfdf..b5ccff32 100644 --- a/tests/test_printer_vision_cycle.py +++ b/tests/test_printer_vision_cycle.py @@ -651,6 +651,7 @@ class _FakeScorer: geometry = None fixed_bed = None model = None + quality = None context = None def __init__(self) -> None: @@ -659,6 +660,9 @@ def __init__(self) -> None: def reset_print(self) -> None: self.resets += 1 + def finish_print(self) -> None: + pass + def __call__(self, jpeg: bytes) -> FrameScore: return FrameScore( 0.25, [Box(0.1, 0.2, 0.3, 0.4, 0.5)], 77.0, probs=[0.4], q=0.1 @@ -725,6 +729,7 @@ def _live_args(**over: Any) -> argparse.Namespace: "no_verify": True, "scene": False, "no_detach": True, + "no_quality": True, "model": False, "evidence": None, "dry_run": False, @@ -833,3 +838,17 @@ def fake_sleep(seconds: float) -> None: ) doc = read_status("u1") assert doc is not None and doc["reachable"] is False and doc["job"] is None + + +def test_a_finished_print_calls_on_print_end_once(rig: Rig) -> None: + """The per-print quality report is written when the printer leaves printing/paused, once.""" + ends: list[int] = [] + rig.cycle.on_print_end = lambda: ends.append(1) + rig.print_file() + rig.tick() + rig.tick() + assert ends == [] + rig.printer.state, rig.printer.filename = "standby", None + rig.tick() + rig.tick() + assert ends == [1] diff --git a/tests/test_vision_quality.py b/tests/test_vision_quality.py new file mode 100644 index 00000000..6b82dbfe --- /dev/null +++ b/tests/test_vision_quality.py @@ -0,0 +1,221 @@ +"""vision_quality.py — the print-quality tier of `3d printer vision` on REAL U1 frames (tests/data/quality). + +`u1_f1` is the empty plate (the print's first frame), `u1_f5` the clean first layer, `u1_f7` / `u1_f10` / `u1_f12` +the stringing debris of layers 2-4 that nothing flagged on 2026-11-05. The tier is fed the way FrameScorer feeds +it: a decoded frame, the print context (Z), a clock. No printer, no camera, no model, no network. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any, cast + +import pytest + +from printer_ops.vision import PrintContext + +np = pytest.importorskip("numpy") +cv2 = pytest.importorskip("cv2") +import vision_quality as VQ # noqa: E402 + +DATA = Path(__file__).parent / "data" / "quality" +DEVICES = """u1: + adapter: moonraker + host: 127.0.0.1 + port: 1 + model: Snapmaker U1 +""" + + +@pytest.fixture +def device(tmp_path: Path, monkeypatch: pytest.MonkeyPatch): + from printer_ops.devices import get_device + + monkeypatch.setenv("XDG_CONFIG_HOME", str(tmp_path / "config")) + cfg = tmp_path / "config" / "3d-cli" + cfg.mkdir(parents=True) + (cfg / "printer-devices.yaml").write_text(DEVICES, encoding="utf-8") + return get_device("u1") + + +def frame(name: str) -> Any: + return cv2.imread(str(DATA / name)) + + +def ctx(z: float) -> PrintContext: + return PrintContext("sheet.gcode", z, 100.0, 100.0) + + +def feed_first_layer(tier: VQ.QualityTier) -> None: + """The empty plate, then four readings of the clean first layer: the job's baseline.""" + now = 1000.0 + assert tier.update(frame("u1_f1.jpg"), ctx(0.2), now) == [] + for i in range(1, 5): + assert tier.update(frame("u1_f5.jpg"), ctx(0.2), now + 15 * i) == [] + assert tier.judge.base is not None and tier.judge.base.n == 4 + + +def test_stringing_on_real_frames_becomes_one_warning_finding_with_the_marked_frame( + device, tmp_path: Path +) -> None: + tier = VQ.QualityTier(device, VQ.QualityConfig(), tmp_path / "evidence") + feed_first_layer(tier) + found = [] + for i, name in enumerate(("u1_f7.jpg", "u1_f10.jpg", "u1_f12.jpg", "u1_f12.jpg")): + found += tier.update(frame(name), ctx(1.0), 1200.0 + 60 * i) + (f,) = found + assert f.kind == "quality_stringing_warning" and f.severity == "warning" + assert "белые ниточки" in f.text and "Что менять" in f.text + assert ( + f.image is not None + and cv2.imdecode(np.frombuffer(f.image, np.uint8), cv2.IMREAD_COLOR) is not None + ) + assert (tmp_path / "evidence" / "u1-quality-worst.jpg").is_file() + + +def test_the_clean_first_layer_and_the_empty_plate_raise_nothing(device) -> None: + tier = VQ.QualityTier(device, VQ.QualityConfig()) + feed_first_layer(tier) + for i in range(5): + assert tier.update(frame("u1_f5.jpg"), ctx(0.4), 1100.0 + 15 * i) == [] + assert ( + tier.judge.report.per_level["warning"] == 0 + and tier.judge.report.per_level["severe"] == 0 + ) + assert tier.judge.report.skipped.get("absent") == 1, ( + "the empty first frame is the reference, not a reading" + ) + + +def test_no_print_context_is_blind_not_clean(device) -> None: + tier = VQ.QualityTier(device, VQ.QualityConfig()) + assert tier.update(frame("u1_f12.jpg"), None, 0.0) == [] + assert ( + tier.once_report()["status"] == "blind" and "Z" in tier.once_report()["reason"] + ) + + +def test_a_new_print_writes_the_report_of_the_one_before( + device, tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + tier = VQ.QualityTier(device, VQ.QualityConfig(), tmp_path / "evidence") + feed_first_layer(tier) + for i, name in enumerate(("u1_f7.jpg", "u1_f10.jpg", "u1_f12.jpg")): + tier.update(frame(name), ctx(1.0), 1200.0 + 60 * i) + tier.reset_print() + (d,) = (tmp_path / "evidence").glob("*-u1-monitor-report") + doc = json.loads((d / "report.json").read_text()) + assert ( + doc["max_level"] == "warning" + and doc["per_issue"]["stringing"] >= 2 + and (d / "worst-frame.jpg").is_file() + ) + assert "print-quality monitor:" in capsys.readouterr().err + assert tier.judge.report.readings == 0 and tier.base is None, ( + "the next print starts from nothing" + ) + tier.finish() + assert len(list((tmp_path / "evidence").glob("*-u1-monitor-report"))) == 1, ( + "an empty print writes no report" + ) + + +@pytest.mark.parametrize( + ("name", "verdict"), + [ + ("u1_f5.jpg", None), + ("u1_f7.jpg", "severe"), + ("u1_f10.jpg", "warning"), + ("u1_f12.jpg", "warning"), + ], +) +def test_once_gives_the_absolute_verdict_of_one_frame( + device, name: str, verdict: str | None +) -> None: + """`printer vision NAME --once`: no job history, so only the absolute levels; it says so.""" + tier = VQ.QualityTier(device, VQ.QualityConfig(), once=True) + found = tier.update(frame(name), ctx(1.2), 0.0) + report = tier.once_report() + assert "absolute" in report["basis"] + assert [f.kind for f in found] == ( + [] if verdict is None else [f"quality_stringing_{verdict}"] + ) + assert report["fleck_frac"] > 0 + + +def test_a_disabled_tier_is_not_built(device) -> None: + assert VQ.build_quality(device, None, once=False, disabled=True) is None + assert VQ.build_quality(device, None, once=False, disabled=False) is not None + + +class _NoSpaghetti: + """Stage 1 as it behaved on the day: no box on any frame (p 0.0).""" + + def score_bgr(self, bgr: Any) -> Any: + from printer_ops.vision import FrameScore + + return FrameScore(p=0.0, boxes=[], luma=70.0) + + +def test_the_frame_scorer_adds_the_quality_findings_next_to_a_silent_spaghetti_model( + device, tmp_path: Path +) -> None: + """The 2026-11-05 situation: p 0.0 / q 0.0 / no boxes — and now the same frame still says stringing.""" + import vision_detect as VD + + tier = VQ.QualityTier(device, VQ.QualityConfig(), tmp_path, once=True) + scorer = VD.FrameScorer(cast(Any, _NoSpaghetti()), quality=tier) + scorer.context = ctx(1.2) + score = scorer((DATA / "u1_f7.jpg").read_bytes()) + assert score.p == 0.0 and score.boxes == [] + assert [f.kind for f in score.findings] == ["quality_stringing_severe"] + scorer.context = ctx(1.2) + assert scorer((DATA / "u1_f5.jpg").read_bytes()).findings == [] + + +def test_once_report_carries_the_quality_block(device, tmp_path: Path) -> None: + import argparse + from types import SimpleNamespace + + import vision_detect as VD + + tier = VQ.QualityTier(device, VQ.QualityConfig(), tmp_path, once=True) + scorer = VD.FrameScorer(cast(Any, _NoSpaghetti()), quality=tier) + scorer.context = ctx(1.2) + score = scorer((DATA / "u1_f12.jpg").read_bytes()) + rep = SimpleNamespace( + frame=None, + score=score, + events=[], + as_json=lambda name: {"printer": name}, + ) + monitor = SimpleNamespace(tracker=SimpleNamespace(adjusted=0.0)) + out = VD._once_report(argparse.Namespace(save=None), scorer, monitor, rep, "u1") + assert out["findings"] == ["quality_stringing_warning"] + assert out["quality"]["level"] == "warning" and out["quality"]["fleck_frac"] > 0.035 + assert "absolute" in out["quality"]["basis"] + + +def test_the_scorer_resets_and_finishes_the_tier_with_the_print( + device, tmp_path: Path, capsys: pytest.CaptureFixture[str] +) -> None: + import vision_detect as VD + + tier = VQ.QualityTier(device, VQ.QualityConfig(), tmp_path) + scorer = VD.FrameScorer(cast(Any, _NoSpaghetti()), quality=tier) + for i in range(5): + scorer.context = ctx(0.2) + scorer.now = 100.0 + 15 * i + scorer(frame_jpeg("u1_f1.jpg" if i == 0 else "u1_f5.jpg")) + scorer.finish_print() + assert len(list(tmp_path.glob("*-u1-monitor-report"))) == 1 + assert "print-quality monitor:" in capsys.readouterr().err + scorer.finish_print() + assert len(list(tmp_path.glob("*-u1-monitor-report"))) == 1, ( + "ending twice writes one report" + ) + + +def frame_jpeg(name: str) -> bytes: + return (DATA / name).read_bytes()