Skip to content

Support ClickHouse geo types (Point, Ring, LineString, MultiLineString, Polygon, MultiPolygon) - #144

Open
Maksim-Burtsev wants to merge 2 commits into
maximdanilchenko:masterfrom
Maksim-Burtsev:geo-types
Open

Support ClickHouse geo types (Point, Ring, LineString, MultiLineString, Polygon, MultiPolygon)#144
Maksim-Burtsev wants to merge 2 commits into
maximdanilchenko:masterfrom
Maksim-Burtsev:geo-types

Conversation

@Maksim-Burtsev

@Maksim-Burtsev Maksim-Burtsev commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Note on the first commit. This branch also carries the test fix from #145 (test_explain_with_fetch breaks on ClickHouse 26.7 for everyone, including master, because the CI image is not pinned). Without it CI here is red for a reason that has nothing to do with geo. Once #145 is merged I will rebase and that commit will disappear from this PR — review it there, not here.

Adds the geo types requested in #136. The other types from that issue (JSON/Object, Variant, Dynamic) are untouched here — they need their own serialization work, so I kept this PR to geo.

What's inside

Geo types are named composites: Point is Tuple(Float64, Float64), Ring and LineString are Array(Point), Polygon is Array(Ring), MultiPolygon is Array(Polygon), and ClickHouse serializes them exactly that way on every format. So there are no new codecs here — each geo type is a type class that inherits the composite it is stored as and hands that composite's name to the base:

class PointType(TupleType):
    """``Point`` is ``Tuple(Float64, Float64)``."""

    __slots__ = ()

    def __init__(self, name: str, **kwargs):
        super().__init__("Tuple(Float64, Float64)", **kwargs)

They are registered in CH_TYPES_MAPPING like every other type, so TSV and RowBinary work through the existing machinery in both directions, nesting included (Array(Point), Nullable(Point), Map(String, Polygon)). what_py_type itself is unchanged.

The Native engine dispatches on the type string rather than on a type object — a columnar Array/Tuple is offsets plus sub-columns, not per-value RowBinary — so it resolves the composite through GEO_WIRE_TYPES, a table derived from the geo classes themselves rather than maintained by hand. Both lookups sit where the existing checks have already missed, so nothing extra runs for anyone who doesn't use geo columns.

The Python → ClickHouse direction needed nothing: a point is a tuple, a ring is a list.

One structural change worth your attention

In _types.pyx, TupleType and ArrayType now parse their name in __init__ instead of __cinit__. Cython runs a base __cinit__ before the subclass and with the original arguments, so otherwise a geo subclass has no way to pass its composite name down (it blows up on RE_TUPLE.findall("Point")[0]). The bodies are unchanged, and the whole Tuple/Array/Map/Nested suite covers the change. Type objects are built once per column per query, so __cinit__ vs __init__ makes no measurable difference — benchmarks before/after are within noise.

TupleType.name and ArrayType.name became readonly so GEO_WIRE_TYPES can read the composite back out of the classes.

Tests

New test_geo_types in TestTypes, which is already parametrized over all three engines and both inserts and reads through the engine under test.

TestNative.test_unsupported_type_raises and test_gc_state_restored_after_decode_error used Point as their "not in the type mapping" example; they now use INTERVAL 1 DAY.

I also ran, outside the suite: all nine insert-engine × read-engine combinations; nested geo (Array(Point), Tuple(String, Polygon), Map(String, Ring), and Nullable(Point) with enable_nullable_tuple_type=1); empty rings and polygons; a polygon with a hole; negative and fractional coordinates. Full suite passes in both builds (pure Python and with the Cython extension).

LineString/MultiLineString require ClickHouse 24.x+; the other four are fine on 22.x+. README and CHANGELOG updated.

Unrelated, but noticed while running the suite: TestFetching::test_explain_with_fetch fails on current master against ClickHouse 26.7 — EXPLAIN SELECT 1 now returns an empty line in the middle of its output, which becomes an empty Record. Not touched here.

Maksim-Burtsev and others added 2 commits August 22, 2026 00:38
ClickHouse 26.7 made `explain_query_plan_default = 'pretty'` the default,
and the pretty plan carries a blank line between the output columns and
the plan tree. A blank TSV line decodes to an empty Record (that is what
the `WITH TOTALS` separator looks like), so indexing every row of an
EXPLAIN result stopped working.

EXPLAIN SYNTAX also renders operators as function calls now
("SELECT plus(1, 1)"), so the test no longer asserts the exact spelling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Point, Ring, LineString, MultiLineString, Polygon and MultiPolygon are
named composites, so each one is a type class inheriting the Tuple/Array
it is stored as. Works on all three engines (TSV, RowBinary, Native).

In _types.pyx TupleType and ArrayType now parse their name in __init__
instead of __cinit__: Cython runs a base __cinit__ before the subclass
and with the original arguments, so a geo subclass could not otherwise
hand its composite name down.

Closes maximdanilchenko#136 partially (geo part only).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant