CM choropleth: pivot_table(aggfunc="first") over an ISO-code merge draws an arbitrary row when a country has two country_ids

[Polichinel]

Filed from views-pipeline-core after investigating views-pipeline-core#509. Handed to Simon; not acted on from the pipeline-core side.

What

views_reporting/mapping/mapping.py:491-496 merges the forecast frame onto the map geometry by isoab → ADM0_A3, then :586-591 does pivot_table(index=ADM0_A3, aggfunc="first"). The bundled country metadata (views_reporting/metadata/data/country.parquet) carries duplicate isoab for dead/live country pairs: Sudan (country_id 59 pre-2011 vs 245), Indonesia (208 vs 209), Serbia (230 vs 233), Tanzania (236 vs 242) — verified against the file; these are the only four collisions among the 22 dissolved ids in the CM panel.

If a forecast frame ever carries a row for a dead id alongside the live one, "first" picks whichever row comes first — and if row order follows country_id, the dead id sorts first in all four pairs, so the map would show the dead country's value over the living one. Inferred from the pivot, not rendered.

Why it can happen

views-r2darts2 ≥0.2.0 forecasts every entity ever seen, dead ones included (views-r2darts2#49). pipeline-core 3.3.0 now refuses such predictions at the model boundary (ADR-064), so the phantom rows should not reach you from a 3.3.0+ run — but the pivot's "any duplicate wins arbitrarily" is a hazard independent of that: any duplicate row per ISO code, from any source, is resolved silently.

The ask

Make a duplicate ISO code on the way into the pivot loud (refuse, or at least log which country_ids collided and which won), rather than aggfunc="first". Small; theirs to shape.

🤖 Generated with Claude Code — https://claude.ai/code/session_01KQmxyvNnB8AJGdgaSR5xKr

[Polichinel]

Done in #293 → #294 (main == development at e37e2d7).

Confirmed and reproduced: with Sudan [59, 245] in one frame the CM choropleth drew the dead country's value and the hover reported 59. Two corrections to the issue's numbers: the bundled metadata has 11 duplicate-ISO groups (27 rows), not 4 — the CM panel has the four you counted, the file also carries USSR×6, Yemen×3, Yugoslavia, Germany, Ethiopia, Saudi Arabia, South Africa; 9 of them join the shapefile (USSR/YUG sit in the retired allowlist). And ascending country_id makes the dead id win in 4 of those 9 pairs, not all — the pick is arbitrary in every one.

The guard, as you asked — loud: in build_mapping_dataframe (the single CM chokepoint), more than one country_id per ADM0_A3 → logger.error + ValueError naming each collided code and its ids, register C-227. Raise rather than warn: a forecast frame's entity set is fixed at its origin, so it never legitimately carries both members. Per code, not per (code, month) — our own historical fixture shows real transitions put both ids in the same month (59+245 at month 379), so a per-month rule wouldn't rescue anything, and the hover props key on code alone. Detection only; resolution stays at the producer (ADR-064), as you said.

Latent today, confirmed: every real forecast fixture carries exactly one member per pair. The guard's first real catch was our own test double — a positional label mock cycling 10 ISO codes across 191 countries, silently absorbed for months. Those tests now use the real bundle.