Repository navigation
Conversation
…ty uses anyIf instead of anyIfOrNull on ClickHouse 25.8 (genezhang#1490)
…_integers=0 (whole-number floats no longer come back as integers) (genezhang#1436)
…_integers=0 (whole-number floats no longer come back as integers) (genezhang#1436)
…atetime/time no-args and string args) (genezhang#1487)
…k as exact strings over Bolt (genezhang#1536)
genezhang
left a comment
There was a problem hiding this comment.
Thanks for looking at #1536. I tested the encoder directly, and the len() > 17 branch changes the type of ordinary doubles, so this can't merge as is.
Probe. I parsed each text with serde_json and passed it to encode_json_value, using this head's graph_objects.rs:
| JSON number | serde_json::Number prints |
encoded over Bolt |
|---|---|---|
0.30000000000000004 |
0.30000000000000004 |
String (main: Float) |
2.3333333333333335 |
2.3333333333333335 |
String (main: Float) |
0.3333333333333333 |
0.3333333333333333 |
String (main: Float) |
3.141592653589793 |
same | Float |
170141183460469231731687303715884105727 (Int128 max) |
1.7014118346046923e38 |
String of that float text |
So any double whose shortest representation is 18 characters or more, which covers many avg() and division results, is now returned to Neo4j drivers as a string.
Why the wide-integer branch does not work. serde_json is built without arbitrary_precision, so a number above u64 is parsed to an f64 before encode_json_value_into runs. The digits are gone by then. Its text is 1.7014118346046923e38, which contains e, so the new "no ., no e" branch never fires, and the number falls through to len() > 17 and becomes an inexact string. The description says the HTTP path preserves precision; in my run of the remote path (cg query against 26.7), a UInt128 column also comes out rounded, on main as well, which is consistent with the same parse.
Directions
- Decide from the column's type, not from the length of a float's text. The schema's
column_types(T1) knows which columns areInt128/UInt256/Decimal(p, s). - To keep the digits, the wide columns would need to reach the encoder as text (for example read with a
toStringcast), or the JSON parse would needarbitrary_precision. - Whatever is chosen needs a unit test that
0.30000000000000004staysFLOAT_64, and one for a wide column.
Stacking. This branch also carries the commits from #1555, #1556 and #1557. Please rebase so it holds only its own change. Please also drop the osv-scanner.toml hunk, which replaces the reason text on main with an older wording.
Happy to re-review.
What
Fixes #1536: Int128/UInt128/Int256 and wide Decimal values come back as Floats over HTTP and Bolt, losing precision.
Bolt path:
encode_json_value_intoonly checkedas_i64()andis_u64(). For 128/256-bit integers (which exceed u64), it fell through toas_f64()→ Float64 (losing precision). For wide decimals with more fractional digits than a double holds, same problem.Fix: encode as an exact string when the number is too wide for i64/u64, or when it's a decimal with more than 17 significant digits. This mirrors the #1516 pattern (
integers_exact_in_bolt).HTTP path: ClickHouse outputs the full-precision number as JSON; serde_json preserves it as a Number. No change needed there.
Gates (local, 1.94.1,
--no-default-features)Honest limits
Spec-driven
Smallest complete change; backward-compatible (only wide integers/decimals change wire type from float to string).
Closes #1536