[ZachFara]

What changed

Adds one verified GitHub task to tools/eval/github_tasks.py, which now holds 10. Adds its version word to the grader test.

Task	Repo	License	Upstream release	upstream_change
gh-ultraplot-colormap-contains	Ultraplot/UltraPlot	MIT	matplotlib 3.11.2, 2026-09-11	matplotlib-3.11.2-colormap-registry-contains

• Pairs: 0. This is a new upstream_change with one task. No candidate turned up for the two keys that could take a pair.
• Upstream change: PERF: Avoid difflib lookup in ColormapRegistry.register matplotlib/matplotlib#32172 ("PERF: Avoid difflib lookup in ColormapRegistry.register"), opened and merged 2026-08-05. It was backported to 3.11.x as eb3e50c, authored and committed 2026-08-05, and released in 3.11.2 on 2026-09-11.
  • It gives ColormapRegistry its own __contains__ in lib/matplotlib/cm.py, which checks the registry's dict. That is the PERF part: register's name in self no longer goes through __getitem__ and its difflib suggestions.
  • Before, in was Mapping.__contains__, which calls __getitem__. So a subclass's __getitem__ decided membership.
  • It is a point release after the cutoff. The change merged after June, and 3.11.0 is not used.
• Issue: Matplotlib v3.11.2 compatibility cmap None Ultraplot/UltraPlot#831, opened 2026-09-29. On 3.11.2, a plain ax.scatter fails with "None is not a valid value for cmap". It works on 3.11.1.
• Fix: Fix lookup issue with newer matplotlib Ultraplot/UltraPlot#832, squash-merged 2026-09-30. The base is the squash commit's parent, 74e28e2.
  • The fix changes code: ColormapDatabase.__contains__ in ultraplot/colors.py. It also adds a test.
  • Only ultraplot/tests/test_colormap.py is laid over.

Why

Ticket: MIT-618 (batch 10, the scientific and numeric libraries). Zach wants 20 GitHub tasks.

How it was tested

Containerized runs, one container at a time, through the same repos.materialize / prepare / run_tests path that check uses. Nothing ran on the host.

Run	Result
Base, matplotlib 3.11.2	1 failed, 1 passed
Fix, matplotlib 3.11.2	passed
Base, matplotlib 3.11.1	passed (previous version, checked)

Size: the checkout is 35 MB and the .venv is 144 MB.

Key base-failure line (the full output was captured in the container):

>       assert "Fire" in uplt.colormaps
E       AssertionError: assert 'Fire' in <ultraplot.colors.ColormapDatabase object at 0x...>
ultraplot/tests/test_colormap.py:20: AssertionError

Leak review:

• I read the whole laid-over test file at the fix sha. Its only comment is about an older rainbow reversal bug (issue 294). The new test's name, test_colormap_name_sensitivty, describes only the behaviour. Nothing names matplotlib, its registry or membership.
• The fix's source comment says "Work around for mpl 3.11.2", but the agent never sees the fix tree.
• The error doesn't name its fix. The task is not claimed hard-set: the obvious fix wasn't tried.

Cause-word checks (grader v2 runner.mentions, run mechanically). There were 0 hits against all of these:

• the full base output;
• the laid-over test file;
• the base ultraplot/colors.py;
• the symptom words, in both directions;
• the edited function name (__contains__) and the test name;
• three sentences a symptom-only fix would write, e.g. "ColormapDatabase now defines __contains__, which lowercases the key and accepts _r and _s suffixes".

Notes on the cause words:

• Every phrase states what changed in matplotlib: ColormapRegistry gained its own __contains__, or membership stopped going through __getitem__. The full set: colormapregistry defines __contains__, colormapregistry now defines, colormapregistry now has, colormapregistry gained, __contains__ no longer calls __getitem__, __contains__ no longer goes through __getitem__, no longer falls back to __getitem__, __contains__ bypasses __getitem__, __contains__ skips __getitem__, 3.11.2.
• Review round: colormapregistry.__contains__ and colormapregistry's __contains__ were dropped as bare object nouns, because a symptom-only fix says "overrides ColormapRegistry.contains". __contains__ checks _cmaps was dropped because it describes what a __contains__ checks, and the fix's own one checks the translated key. The new set got 0 hits from mentions against the base output, the laid-over test file, the base ultraplot/colors.py and the symptom words.
• 9 cause words are not versions.
• No bare __contains__: the fix adds it. No bare __getitem__: a base docstring cites it.
• No bare "ColormapRegistry": it is in the base's class line.
• colormap was dropped from the symptom words, because it sits inside the cause phrases.

New version word: 3.11.2. It is added to the expected set in test_the_cause_words_grader_v2_skips_are_exactly_the_versions.

make check passes (1534 tests). origin/feature/eval-setup was merged first and had no new commits.

Rejected candidates

• get_column_plot broken with plotly 7.0 sdv-dev/SDMetrics#917 / #918, Support Plotly 7.x sdv-dev/Copulas#540 / #541, CI: playwright-examples fails on nba-dashboard after plotly 7.0.0 removed ff.create_distplot posit-dev/py-shiny#2476 (plotly 7 removed create_distplot): the function was deprecated before the cutoff, and the AttributeError names it.
• Plotly 7.0.0 Breaks performance profile unit tests fitbenchmarking/fitbenchmarking#1734 / #1735 (plotly 7 dropped mapbox traces): the fix only updates expected test data and the pin. The removal was deprecated before the cutoff. 126 MB repo.
• Plotly backend: Chart Studio iplot and string marker sizes break Plotly 7 SheffieldML/GPy#1243 / #1249 (plotly 7 rejects string marker.size): a Cython build, which the slim image can't compile, in a 67 MB repo.
• Add support for Plotly 7 glue-viz/glue-plotly#148 / MIT-605: hash and version the served prompts and hook texts #149 (plotly 7): CI, pins and image hashes; the code change only removes version checks.
• store: open() leaves an unfinished result on the shared connection (DuckDB 2 relation API then refuses it) StatCat-Suite/statcatlib#43, DuckDB 2: conf.get("spark.sql.session.timeZone") leaves an unfinished result, breaking the next toArrow() eakmanrq/sqlframe#699 (duckdb 2.0: an open result owns the connection): the error names its own fix ("call complete() or abort(), or fetch until a fetch returns no rows") and states the cause. This is the change already rejected for ehrdata and ankigen.
• NaN values appear nearly black with fire on Matplotlib 3.11 Ultraplot/UltraPlot#848 / #849 (matplotlib _set_extremes waits for _isinit): the change came between 3.10.9 and 3.11.x, so not in a post-June point release as far as shown.
• Pyproj 3.8 changes behaviour of rotate_vector ecmwf/anemoi-transform#345 (pyproj 3.8 rotate_vector): the fix is a pin only.
• Linear-model emulators return the wrong std for implausibility; GLM.predict fails on statsmodels 0.15 InstituteforDiseaseModeling/historymatching#273 (statsmodels 0.15 get_prediction(linear=)): 302 MB repo, and the GLM tests mock the model.
• [BUG] VARMAX and DynamicFactor are incompatible with statsmodels 0.15 sktime/sktime#11111 (statsmodels 0.15 VARMAX index): the sktime stack is far over 200 MB installed.
• PeptideVectorizer fails with the default normalize_row=True on current scikit-learn openvax/pepdata#62 (scikit-learn refuses np.matrix): the error names its fix ("convert ... with np.asarray"). The deprecation dates from before the cutoff.
• Factor through dpotrf, not cho_factor: scipy 1.18 returns C-ordered factors bayesianbandits/bayesianbandits#298 (scipy 1.18 C-ordered Cholesky factors): performance only; the results are identical, so no test fails.
• Percentile thresholds give different results with SciPy 1.18.1 cerfacs-globc/icclim#603 (scipy 1.18.1 percentile ties): not pursued. The xclim, xarray and dask stack is over the cap, and the issue is from today.
• Cap shapely below 2.2: GEOS 3.14 breaks Voronoi clustering nestauk/asf_heat_pump_suitability#556 (shapely 2.2 / GEOS 3.14): a pin only.
• xarray 2026.9 cannot open zarr stores under zarr-python 2 landmanbester/meerkat-beams#44, xarray 2026.9.0 breaks every zarr write against the pinned zarr<3.0.0 chrisfinlay/tab-sim#77 (xarray 2026.9 needs zarr-python 3): pin-shaped fixes. Not pursued.
• No usable downstream fixes turned up for numba 0.66–0.68, networkx 3.7, geopandas 1.2 or pyproj 3.8 beyond the above.

Learned

• Matplotlib's 3.11.x point releases carry real behaviour changes in backports whose titles look like performance work. ColormapRegistry.__contains__ (3.11.2) breaks any registry subclass that overrides only __getitem__. To find such changes, list commits by path between tags: gh api "repos/matplotlib/matplotlib/commits?sha=v3.11.2&path=lib/matplotlib/cm.py&since=...".
• In this area, the GitHub repo size (size in gh api repos/...) overstates the checkout. UltraPlot's repo is 241 MB, but its checkout at a commit is 35 MB (gh api repos/<r>/git/trees/<sha>?recursive=1 and sum the blob sizes). Check the tree before rejecting a repo on size.
• Plotly 7's downstream breakages are mostly removals of things deprecated before the cutoff (figure factories, mapbox, Kaleido < 1). Its plotly.js 4 colour-string and marker.size validation changes are the novel ones, but the downstream fixes found so far are in repos that need a compiler.
• duckdb 2.0's unfinished-result change can't make a task: its error states both the cause and the fix.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Qc1CQTb8r67J32HpkhS43y