visual-compare

Cross-compare two images with vision models from two different labs. Get ranked, actionable findings plus a grounded annotated image — the markup is drawn locally onto your untouched frame, never repainted by an image model.

Every box is a coordinate the model returned, rasterised onto the untouched frame. Colour is severity — red 9+, orange 7–8, yellow 5–6 — and findings both critics raised independently are marked (2x).

What you can get

The critique always runs. Everything below it is opt-in — each layer adds images, which costs tokens and dilutes attention, so nothing turns on unasked.

	what it answers	sample
critique always on	what is wrong, where, how bad, and what to change — ranked, with a box on every claim
--tone exact, no API	exposure: false-colour bands, blown highlights, crushed blacks, zones, waveform
--color exact, no API	palette, per-region colour grid with hex, and ΔE match of target against actual
--shape 1 call/image	silhouette and structure — blue is mass missing from your render, red is mass to remove
--depth estimate	spatial layering in five views — count the bands to count the depth layers
--report	self-contained HTML page with the findings and the provenance of exactly what was sent

Also: --tiles auto / --detail push native-resolution crops through the provider's 768 px cap, and --analyze is the recommended package (--tone falsecolor --color palette,grid).

--tone and --color are measurements, not opinions — identical input, identical numbers, safe to gate a build on. The critique and --depth are model output and can be confidently wrong. Use the first to check the second; that is the whole design.

Quick start

git clone https://github.com/sgeier/visual-compare-skill ~/.claude/skills/visual-compare
python ~/.claude/skills/visual-compare/scripts/vcompare.py --check

python scripts/vcompare.py \
  --target ref/mock.png \
  --actual shots/current.png \
  --models gpt-5.6-sol,gemini-3.6-flash \
  --env ~/.claude/visual-compare.env

Python 3.9+ and Pillow; everything else is stdlib. torch only for --depth. --check makes no API calls, installs nothing, and reports which key came from which file. Output lands in .vcompare/ — critique.json, annotated.png, report.html with --report.

Keys: one file with OPENAI_API_KEY and GOOGLE_API_KEY, passed with --env, or left at ~/.claude/visual-compare.env — resolution order.

Pass two critics from different labs. Two models from one provider share failure modes, so their agreement proves little. If you also pass --knobs, know that it anchors the critique to your existing settings.

Why not a pixel diff

Pixel diffs report that channels differ, not that the ledge is too low or the key light is behind the subject. They are also optimisable — you can hill-climb a similarity score for hours while the look drifts further away. This asks models that can see both images what is wrong, and forces every answer to carry coordinates so it can be checked.

How it works

  target ─┐
          ├─▶ prepare    crop (recorded), downscale, log provenance
  actual ─┘
          ├─▶ layers     tone / colour / shape / depth built LOCALLY
          ├─▶ critics    both frames + views → 2 models, forced JSON schema
          ├─▶ consensus  findings merged, agreement tagged
          ├─▶ markup     Pillow draws boxes on the UNTOUCHED actual image
          └─▶ outputs    critique.json · annotated.png · report.html (opt-in)

The model supplies coordinates only. An earlier version let an image model annotate the screenshot — it reconstructed the scene and drew markup on its own repaint, useless as evidence of what the build renders.

The report

--report writes a self-contained HTML page. Off by default, because generating one unprompted buries the answer in a document nobody asked for.

It carries the thing a findings list always omits: what was actually sent to the models. Both images embedded exactly as transmitted, each with its source file, source dimensions, crop region and the size that reached the API — so a critique of a 772×901 crop cannot be mistaken for a critique of the full frame. Below the fold: every finding with its box, and every enabled layer's views and metrics. Full page →

Reading the results

The models will fabricate specifics. One stated a character was "8–9 units tall" when it was 6.33 by construction — right prose, invented number.

1. Check the box before believing the text. Over empty sky = junk, discard it.
2. Magnitudes are directions, not values. Try it, look, keep or revert.
3. Verify anything checkable against your own code, tokens, or the measured layers.
4. (2x) findings are near-always real; single-critic ones are hypotheses.
5. Separate is_bug from style. A bug is rarely a matter of taste.
6. Never optimise similarity. Sanity check, not objective function.

Docs

docs/layers.md	every layer, every view, what the metrics mean
docs/output.md	critique.json schema, how consensus is computed, resolution limits, reports
docs/keys.md	key resolution order and how secrets are handled
docs/critics.md	why two labs, the model bake-off, and the --knobs trade-off
SKILL.md	agent-facing instructions (Claude Code skill)

Not for

Architecture, code structure, or product decisions. Reliable at "does A look like B" because both images are in the prompt. Everything else has no ground truth and it will produce confident, plausible fiction.

Licence

MIT