§5.1 ruling: is float64 for GAUL codes a wire decision or a pandas artefact? (nullable int64 not considered) #278

FAO's contractor has now asked twice for GAUL codes as integers — in March 2026, when we said yes and shipped the change, and again on 14 August after the global re-upload reverted it. The current answer we are preparing to send is "cast to integer on read", which is defensible but is the second time we tell them no on the same point.

Before that goes out, I want §5.1's authors to rule on a third option that I cannot find considered anywhere in the repo.

The two options that have been considered

float64 (current). views_postprocessing/contract/gaul_schema.py states the ruling directly:

Wire dtypes are the §5.1 ruling: codes are ALWAYS float64 (one stable schema regardless of whether a run contains a missing code)

int64 + sentinel. Already rejected, and rightly. docs/fao_excluded_cells.md says the uncovered cells are dropped at region level "rather than shipped with a placeholder — shipping them would attribute partner rows to a non-country (-1)". These are identifiers, not quantities: a -1 GAUL code is a fake place that groups, joins and aggregates like a real one. Silent corruption. No argument from me.

The option I cannot find discussed: nullable int64

Parquet and Arrow both carry nullable integers natively — values plus a validity bitmap. That gives integers on the wire, a real null, and one stable schema. All three properties §5.1 wants.

The stated justification — that an integer column cannot hold a missing value — is a property of NumPy-backed pandas, which upcasts int64 to float64 the moment a NaN appears. It is not a property of Parquet, which is what we actually deliver.

So the question is whether §5.1's float ruling is a considered wire decision, or an artefact of the build stack that got written down as one. I genuinely do not know which, and the docstring does not say.

What makes this non-trivial, and why I am asking rather than doing

1. Column order and dtype are normative under ADR-013 §5.1, byte-pinned by the §10 golden fixture. Changing dtype re-cuts the fixture deliberately.
2. The consumer hard-validates: views-faoapi handlers.py, FAO_PGMDataset._METADATA_COLS. Both ends move in lockstep or the delivery fails validation.
3. Round-trip behaviour needs checking, not assuming. If FAO read the parquet with default pandas and a nullable Int64 column degrades back to float or object on their side, we have added contract churn for nothing. That is the empirical question I would want answered before anyone writes code.

The part that makes this awkward to defend as-is

For this delivery the code column never contains a null. The 76 GAUL-uncovered cells are removed before delivery, the exclusion list is frozen in delivery/coverage.py, and tests/test_delivery_coverage.py asserts it against the producer.

So the float type is defending against a case this product structurally cannot produce. That is a coherent position — schema should not depend on data — but it is a hard one to explain to a consumer who has never once received a missing code and has asked twice for integers.

What I need

A ruling, not a patch:

• Is the float64 choice a considered wire decision, or inherited from pandas dtype behaviour?
• Does nullable int64 satisfy §5.1's stability requirement? If not, why not — I would like the reason written down where the next person finds it.
• If it does: what does the change actually cost across gaul_schema.py, the §10 fixture, and the faoapi consumer, and is it worth it?
• If it does not: a sentence in the gaul_schema.py docstring saying nullable int was considered and rejected, so this question is not reopened every time a consumer asks.

Either answer is fine. I would just rather tell FAO "fixed" than tell them no twice, and if the answer is no, I want to give them a real reason rather than one that only holds in pandas.

Related: views-postprocessing#272, views-faoapi#407.