Skip to content

benchmark: add Figanos comparison and reference-figure regression - #8

Merged
GeoGeekLab merged 70 commits into
mainfrom
benchmark/figanos-cross-library
Sep 23, 2026
Merged

GeoGeekLab merged 70 commits into
mainfrom
benchmark/figanos-cross-library

Conversation

@GeoGeekLab

@GeoGeekLab GeoGeekLab commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds a reproducible Figanos 0.7.0 benchmark, closes concrete layout/fidelity gaps, and adds source-backed reference-figure regression.

Cross-library benchmark

  • official pinned AR6 Chapter 6 methane scenario rows;
  • shared field, mask, Robinson projection, levels, physical size, and IPCC temp_div;
  • shared warming-level data and 180 x 72 mm canvas;
  • display-space SSP label collision detection.

Figanos remains stronger for general-purpose climate plotting and Xarray-native faceting. In the current SSP benchmark, Figanos matches the WGI-2022 colours exactly; its SSP1-1.9 / SSP1-2.6 end labels overlap, while the collision-aware ar6-sciplot labels do not.

Plotting/API changes

  • map_panel_grid() for fixed-size multi-panel maps;
  • add_uncertainty_legend() for agreement, missing-data, and significance encodings;
  • label_line_ends() with vertical collision avoidance;
  • reference width, height, panel-count, and projection checks in the audit API and CLI;
  • concise audit output when optional checks are not requested.

Official colormap provenance

The asset manifest is pinned to
IPCC-WG1/colormaps@b7d3849d4fa521d2583b91360e875e38f191d209.

The benchmark workflow verifies all 43 official RGB assets by Git blob SHA. Verified Matplotlib colormaps carry source commit/blob metadata, and the strict map audit checks that metadata rather than accepting an ipcc_* name alone.

Reference-figure regression

Three representative source-backed reproductions now have structured contracts:

  • Chapter 3 Figure 3.2b: physical size, axis limits, labels/title, panel bbox, required legend content;
  • Chapter 6 Figure 6.18 source: physical size, time range, labels/title, SSP legend content, report-era SSP colours;
  • Chapter 10 Figure 10.20b: physical size, Lambert Conformal projection, geographic viewport in projected coordinates, title and legend order.

The existing committed PNG regression remains in place for all four gallery figures. The workflow now also emits image-difference metrics: exact pixel size, mean absolute error, changed-pixel fraction, and 64 x 64 thumbnail error.

Current reference-regression run:
https://github.com/GeoGeekLab/ipcc-wg1-scientific-plotting-skill/actions/runs/35834845182

All four regenerated PNGs match their committed baselines exactly:

  • mean absolute error: 0
  • changed pixel fraction (>8/255): 0
  • thumbnail mean absolute error: 0
  • pixel dimensions: unchanged

Artifact digest:
sha256:4ea586a8e23bf1aaa05207d6896081cf771bae144f448cb6235b8b80ad7e4878

Current head: f2a6ad407ff9bbbd4df7d35f45372bbc7b11f194

Ordinary CI also passes on the same head:
https://github.com/GeoGeekLab/ipcc-wg1-scientific-plotting-skill/actions/runs/35834845231

@GeoGeekLab GeoGeekLab changed the title benchmark: compare ar6-sciplot with Figanos and close layout gaps benchmark: add Figanos comparison and reference-figure regression Sep 23, 2026
@GeoGeekLab
GeoGeekLab merged commit 012cfe9 into main Sep 23, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant