explore profile no longer offers composite keys that are artifacts of a near-unique column or of a continuous measure

[marcociav-exmergo]

Closes #292.

Summary

A profile's grain is acted on: callers write deduplication and joins from it, and
transform plan --scaffold keys generated models on candidate_keys[0]. On a
2,037-row orders table where order_id held 1,927 distinct values, the profile
returned five candidate keys and elected a grain from them. Four were one fact
wearing four hats, that order_id is unique on all but 110 rows so any wider
column completes it. (SUBTOTAL, order_id) is not a grain; it is the observation
that two rows sharing an order id happened to differ in their subtotal. The list
carried no order, so a caller could not tell the real key from the filler, and the
one thing worth saying, that this table has duplicate order ids, was the one thing
the profile did not say.

The same bug shipped in the demo warehouse and was quoted in a committed reference
doc: order_items reported a grain of (unit_price, order_id), a DECIMAL(10,2)
money column paired with a foreign key, on a table whose actual story is 1,000
duplicate order_item_id rows from a double-loaded batch.

Two exclusions now apply before any pair is priced, grain comes back null
rather than wrong when nothing survives, and a new key_evidence field carries the
reasoning per key.

Decisions worth a reviewer's attention

The near-unique rule is a test on the pair, not on the column. The obvious
version, "drop any pair containing a column that is already 90% unique", deletes a
real grain. On a parent-line table where most orders have a single line, ORDER_ID
is near-unique and (ORDER_ID, LINE_NO) is still exactly the grain: a two-value
partner separates the duplicate parents on purpose, where a wider one separates
them by accident. So the rule drops an anchored pair only when its partner's domain
is not bounded by the fanout. That exception is verified live on Snowflake in the
dogfood, and it is the single most load-bearing detail in the change.

A near-unique timestamp gets no such exception. The enumeration escape assumes
the anchor is a parent whose rows a position column enumerates, which a per-row
event time is not. Without this, (status, created_at) comes back as a key on any
table with a 98%-unique timestamp and a four-value status column, which the new
fixture demonstrated before it was fixed.

The measure vocabulary is not decoration, it is the Snowflake and BigQuery
fallback. adapters/base.py:is_integer_type already documents that Snowflake's
SHOW COLUMNS renders NUMBER(38,0) and NUMBER(10,2) identically as FIXED;
BigQuery NUMERIC carries no scale either. A type-only measure rule would
therefore either miss the money on both connectors or, if written as "numeric but
not integer", delete every non-id numeric key member across all of Snowflake. So
the type test requires an explicit nonzero scale and goes deliberately silent
where a dialect erases it, and the name vocabulary covers exactly that gap. A name
never decides alone: quantity, amount and total are legitimately
low-cardinality members of real fact grains, so anything with a domain bounded by
the fanout is an enumeration before it is ever considered a measure.

A fixed 100-row floor, and it earns its place. Below it every column looks
near-unique and every measure looks continuous, so both rules would suppress grains
rather than junk, and the probes are nearly free at that size anyway. Without the
floor, test_composite_probe_fills_the_cap_rather_than_discarding_the_grain (6
rows) fails: amount DOUBLE at 4 of 6 distinct would leave three candidates
against an asserted cap of five. A fixed floor rather than a config knob, because
it changes what gets measured rather than how loudly a finding reads.

grain is null, never the near-unique column. Returning order_id would be
more directly useful and is wrong: maintain/drift.py folds a single-column grain
into its proven-key baseline, so it would report key_lost_uniqueness on every run
forever for a key that never held, and scaffold would key a model on a column
with duplicates.

Exclude before probing, not probe then disclaim. The exact answer cannot move a
decision either way, the pair would have to be reported and immediately qualified,
and dropping it frees a capped slot for a pair that might be the real grain. On the
demo that is visible: the pairs actually probed fall from five to two, and the two
that get asked are the ones that could have been a grain.

One new field, and the restraint applied where it holds. data_quality is a
flat list[str] with no per-key association, it is capped per object in
explore map, and prose naming a suppressed key's columns drags them back into the
serialized columns through columns_with_findings. So key_evidence exists. But
there is no flag (a spurious key being worse than nothing is the issue's premise, so
there is nothing to opt out of), no probed boolean (the probe's budget notes
already say when it did not run), and key_evidence is not in explore map's
payload. reason is prose rather than a code enum because the set of causes is open
and a new one should not need a contract change and an exhaustive match in every
consumer.

CACHE_SCHEMA_VERSION moves to 4, and heals itself. An older engine reads a
version-4 cache fine. The direction that breaks is a current engine reading a
version-3 one, where a suppressed combination still reads as a ranked candidate and
keeps maintain grain paying to re-probe it. Rather than document a caveat, the
profile freshness gate treats a pre-4 profile as stale, so the first explore profile or explore map after upgrading re-scans.

The duplicate figure is phrased as rows to remove. The issue asks for "110
duplicate order ids", but non_null_rows - distinct counts surplus rows, not values
that repeat, and the two differ: one id appearing 111 times is 110 surplus rows and
exactly one repeated value. The removal phrasing is exactly true in every case,
needs no new scan, and says what to fix. Two long-standing imprecisions in that
sentence are fixed alongside it: the surplus was computed against the total row
count rather than the non-null count, and it carried a ~ even when derived from
two exact numbers.

Found by dogfooding, not by the suite

• explore diagram was marking a money column PK in committed output.
  unit_price PK is gone and order_id reads FK rather than PK.
• The first cut elided the one column that carries the finding. With no key on
  the table, order_item_id lost its role and dropped out of explore map's
  columns, so the map reported "order_item_id is not unique" while not listing
  order_item_id. notable_columns now keeps a column named by a suppressed
  entry, with no role, because it is not a key.
• Three notes were stating one column's arithmetic three times. The separate
  probe note is gone and its one distinct fact folded into the grain-unknown
  sentence; the two always co-occur.

The demo's map counts moved with it: 7 objects, 5 joins and 6 PII columns unchanged,
data-quality findings 5 to 6. Both READMEs are updated.

Dogfood

DuckDB (dex demo, free, and the reproduction itself):

	before	after
grain	["unit_price", "order_id"]	null
candidate_keys	two artifact pairs	[]
pairs probed	5	2

BigQuery (exmergo-viz.dex_ci): all four artifact pairs measured live at
exactly 2,037 distinct combinations over 2,037 rows, so every one would have been
reported. After: candidate_keys: [], and order_id is the closest thing to one at 94.6% unique (1927 distinct over 2037 rows, 110 rows would have to be removed), the
issue's numbers exactly. NUMERIC money was excluded by name, since the type
carries no scale. Estimate 41,943,040 bytes, and it held.

Snowflake (DEX_CI.DEX292), the connector the design turns on, where every
column reports the bare token FIXED: money excluded, and on a second table
(ORDER_ID, LINE_NO) reported as the grain with ORDER_ID at 90.9% unique. Both
halves proven on one connector in one run. Estimate 2.0s, settled at 1.55s + 0.29s.