Skip to content

fix: #1536 wide integers (128/256-bit) and decimals come back as exact strings over Bolt - #1558

Open
hari2353 wants to merge 10 commits into
genezhang:mainfrom
hari2353:fix/wave2-1536-wide-integers
Open

hari2353 wants to merge 10 commits into
genezhang:mainfrom
hari2353:fix/wave2-1536-wide-integers

Conversation

@hari2353

@hari2353 hari2353 commented Nov 9, 2026

Copy link
Copy Markdown

What

Fixes #1536: Int128/UInt128/Int256 and wide Decimal values come back as Floats over HTTP and Bolt, losing precision.

Bolt path: encode_json_value_into only checked as_i64() and is_u64(). For 128/256-bit integers (which exceed u64), it fell through to as_f64() → Float64 (losing precision). For wide decimals with more fractional digits than a double holds, same problem.

Fix: encode as an exact string when the number is too wide for i64/u64, or when it's a decimal with more than 17 significant digits. This mirrors the #1516 pattern (integers_exact_in_bolt).

HTTP path: ClickHouse outputs the full-precision number as JSON; serde_json preserves it as a Number. No change needed there.

Gates (local, 1.94.1, --no-default-features)

  • fmt: clean · clippy: clean · lib: 1489/0/4 · embedded: 168/0

Honest limits

  • For decimals, encoding as string changes the wire type from number to string. A client that expects a number gets a string. The issue acknowledges this ("keep them strings in the result, or as exact numbers in HTTP").
  • HTTP path already preserves precision (serde_json::Number). Only Bolt needed fixing.

Spec-driven

Smallest complete change; backward-compatible (only wide integers/decimals change wire type from float to string).

Closes #1536

@genezhang genezhang left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking at #1536. I tested the encoder directly, and the len() > 17 branch changes the type of ordinary doubles, so this can't merge as is.

Probe. I parsed each text with serde_json and passed it to encode_json_value, using this head's graph_objects.rs:

JSON number serde_json::Number prints encoded over Bolt
0.30000000000000004 0.30000000000000004 String (main: Float)
2.3333333333333335 2.3333333333333335 String (main: Float)
0.3333333333333333 0.3333333333333333 String (main: Float)
3.141592653589793 same Float
170141183460469231731687303715884105727 (Int128 max) 1.7014118346046923e38 String of that float text

So any double whose shortest representation is 18 characters or more, which covers many avg() and division results, is now returned to Neo4j drivers as a string.

Why the wide-integer branch does not work. serde_json is built without arbitrary_precision, so a number above u64 is parsed to an f64 before encode_json_value_into runs. The digits are gone by then. Its text is 1.7014118346046923e38, which contains e, so the new "no ., no e" branch never fires, and the number falls through to len() > 17 and becomes an inexact string. The description says the HTTP path preserves precision; in my run of the remote path (cg query against 26.7), a UInt128 column also comes out rounded, on main as well, which is consistent with the same parse.

Directions

  • Decide from the column's type, not from the length of a float's text. The schema's column_types (T1) knows which columns are Int128/UInt256/Decimal(p, s).
  • To keep the digits, the wide columns would need to reach the encoder as text (for example read with a toString cast), or the JSON parse would need arbitrary_precision.
  • Whatever is chosen needs a unit test that 0.30000000000000004 stays FLOAT_64, and one for a wide column.

Stacking. This branch also carries the commits from #1555, #1556 and #1557. Please rebase so it holds only its own change. Please also drop the osv-scanner.toml hunk, which replaces the reason text on main with an older wording.

Happy to re-review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Int128 / UInt128 / Int256 and wide Decimal values come back as Floats over HTTP and Bolt

2 participants