Skip to content

json type does not preserve the document text it was given #3208

Description

@reltuk

Postgres treats json as validated text: the document is stored exactly as it was given, and only jsonb normalizes it. Doltgres parses both types into an in-memory document on input, so a json value loses the properties Postgres guarantees for it — whitespace, key order, duplicate keys, and the exact form of escapes and numbers. The rendering also depends on whether the value came from storage or not, so the same input can print two different ways.

SELECT '{"b":1,   "a":2}'::json;
-- postgres: {"b":1,   "a":2}
-- doltgres: {"a": 2, "b": 1}

SELECT '{"a":1,"a":2}'::json;
-- postgres: {"a":1,"a":2}
-- doltgres: {"a": 2}

SELECT '{"a":"\u003c"}'::json;
-- postgres: {"a":"\u003c"}
-- doltgres: {"a": "<"}

SELECT '{"a":1.000,"b":1e2}'::json;
-- postgres: {"a":1.000,"b":1e2}
-- doltgres: {"a": 1, "b": 100}

The same value prints differently once it has been through storage, because a stored json value and an in-flight one are rendered by different paths:

CREATE TABLE t (id int PRIMARY KEY, j json);
INSERT INTO t VALUES (1, '{"b":1,   "a":2}');

SELECT j FROM t;        -- doltgres: {"a":2,"b":1}      (compact, keys sorted lexically)
SELECT j::text FROM t;  -- doltgres: {"a":2,"b":1}
SELECT '{"b":1,   "a":2}'::json;  -- doltgres: {"a": 2, "b": 1}  (spaced, keys sorted by length)
-- postgres returns the input text verbatim in all three

Numbers are the part that is not merely cosmetic, and it affects jsonb as well: numeric literals are round-tripped through a float, so a value can come back changed. Postgres keeps the token text for json and stores jsonb numbers as numeric, so both preserve the value.

SELECT '{"n":12345678901234567890,"d":0.1234567890123456789}'::json;
-- postgres: {"n":12345678901234567890,"d":0.1234567890123456789}
-- doltgres: {"d": 0.12345678901234568, "n": 12345678901234567000}

SELECT '{"n":12345678901234567890,"d":0.1234567890123456789}'::jsonb;
-- postgres: {"d": 0.1234567890123456789, "n": 12345678901234567890}
-- doltgres: {"d": 0.12345678901234568, "n": 12345678901234567000}

Functions that inspect a json document inherit the gap and cannot be made compliant on their own, since the information is already gone by the time they see the value:

SELECT * FROM json_object_keys('{"z":0,"a":1,"a":2,"bb":3}');
-- postgres: z, a, a, bb   (document order, duplicate reported twice)
-- doltgres: a, z, bb

SELECT json_strip_nulls('{"b":1,"a":null,"a":2}');
-- postgres: {"b":1,"a":2}
-- doltgres: {"a": 2, "b": 1}

Worth noting for whoever picks this up: making json carry its text is a change to what gets persisted for a given input, so it needs to be weighed as a storage-semantics change rather than a function-level fix, and existing rows would keep whatever text was written for them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions