fix: stop long float values becoming strings and corrupting weight totals (#377) - #383
Conversation
…ght totals The home page crashed with "TypeError: e.toFixed is not a function" (#377), but the crash was the second symptom, not the bug: the totals were already wrong. json-bigint's storeAsString option gates on raw literal length (>15 characters), not on integer-ness, so an ordinary gram value carrying float64 noise — 0.30000000000000004 is 19 characters — parsed as a *string* despite the declared number type. The home page then does `sum + spoolStockWeight(s)`, and once one value is a string, `+` becomes concatenation: every later spool's weight is appended as digits. That concatenation, not any stored value, is where the reporter's 55-digit number came from. Sub-1000-gram values then reached .toFixed() as strings and took the render tree down. Parse with storeAsString: false so long literals arrive as BigNumber instances, which — unlike strings — can be told apart from genuine string fields, and revive each one individually: only whole numbers beyond the safe integer range become strings, preserving the CockroachDB id behaviour (#69) exactly. Also harden formatWeight/formatWeightCompact/formatLength with the Number() + Number.isFinite coercion numberFormatter in the same file already used, so no future bad value can crash a page; and round used_weight to 6 decimals on every write path. Six, not one: the goal is to strip float64 representation noise without discarding real sub-gram increments, since a slicer can legitimately report ~0.03 g per layer. The accumulator stays a single atomic UPDATE — rounding it in Python would reintroduce the read-before-write that deadlocks MariaDB and forces CockroachDB serialization retries — so it rounds in SQL, casting to Numeric first because PostgreSQL has no round(double precision, integer). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
_round6 cast its result back to Float "to match the used_weight column", but that cast never did anything on any supported backend. MySQL/MariaDB cannot CAST to FLOAT, so SQLAlchemy dropped it and warned; on SQLite and PostgreSQL the UPDATE ... SET assignment already coerces to the column type. Compiling every dialect with and without it yields byte-identical SQL, so its only observable effect was the warning. The inner cast to Numeric stays — that one is load-bearing, since PostgreSQL and CockroachDB have no round(double precision, integer) overload. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two commits this branch carries were written inside the chat tool-layer branch, where a changelog entry would have landed under the wrong feature. This is a user-facing crash fix against a filed issue, so it belongs in the Unreleased section on its own terms. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Fixes a frontend crash and silent corruption of weight totals caused by long float literals being parsed as strings by json-bigint, and reduces future risk by rounding used_weight on write paths and making client formatters defensive.
Changes:
- Client: replace
json-bigint“store long literals as strings” behavior with BigNumber + reviver so only oversized integers become strings (preserves CockroachDB ID behavior). - Client: harden
formatWeight,formatWeightCompact, andformatLengthagainst string/NaN/undefined inputs (avoid.toFixed()crashes). - Backend + tests: round
used_weightconsistently (including atomic SQL accumulator) and add regression coverage across client and integration tests; add changelog entry.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests_integration/tests/spool/test_use.py | Adds integration regression test ensuring repeated small use increments don’t accumulate float noise. |
| spoolman/database/spool.py | Adds SQL-side rounding helper and applies rounding across used_weight write paths. |
| client/src/utils/parsing.tsx | Makes weight/length formatters defensively coerce non-numeric inputs. |
| client/src/utils/parsing.test.ts | Adds tests for formatter coercion and noisy-float rounding behavior. |
| client/src/utils/bigintJson.ts | Reworks bigint JSON parsing to revive BigNumbers and only stringify oversized integers. |
| client/src/utils/bigintJson.test.ts | Adds tests for long noisy floats, MAX_SAFE_INTEGER boundary, and oversized integers. |
| client/src/pages/home/analytics.test.ts | Adds end-to-end test covering JSON parsing + home weight summation (prevents string concatenation). |
| CHANGELOG.md | Documents the crash + silent-total-corruption fix and the multi-layer remediation. |
Suppressed comments (1)
spoolman/database/spool.py:411
SpoolUpdateParametersalso allowsused_weight: nullon PATCH. With the new rounding branch,round(v, ...)will throw ifvisNone, producing a 500. Add a null guard so the API rejects this invalid update instead of crashing.
elif k == "used_weight":
# A caller can also set used_weight directly (#377): round it the same way.
spool.used_weight = round(v, WEIGHT_ROUND_DECIMALS)
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| numeric_expr = sqlalchemy.cast(expr, Numeric(18, WEIGHT_ROUND_DECIMALS)) | ||
| return func.round(numeric_expr, WEIGHT_ROUND_DECIMALS) |
| elif k == "remaining_weight": | ||
| if spool.initial_weight is None: | ||
| raise ItemCreateError("remaining_weight can only be used if initial_weight is set.") | ||
| spool.used_weight = max(spool.initial_weight - v, 0) | ||
| new_used_weight = _used_weight_from_remaining(spool.initial_weight, v) | ||
| # Rounded like every other used_weight write path (#377): the subtraction can carry | ||
| # the same float64 noise as the SQL accumulator in use_weight_safe does. | ||
| spool.used_weight = round(new_used_weight, WEIGHT_ROUND_DECIMALS) |
Note on the automated review finding about
|
| field | master (696483c) |
this branch |
|---|---|---|
used_weight: null |
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float' |
TypeError: type NoneType doesn't define __round__ method |
remaining_weight: null |
TypeError: unsupported operand type(s) for -: 'float' and 'NoneType' |
TypeError: unsupported operand type(s) for -: 'float' and 'NoneType' |
Both fields already 500 on master. This branch changes only the message of the used_weight failure — the null now meets round() a few lines earlier than it previously met the arithmetic — not whether it fails or what status the caller sees.
So it is a genuine pre-existing bug (a 500 where a clean 422 belongs) and worth its own issue, but it is not a regression from this fix and is out of scope for a crash fix that is being kept deliberately small.
🤖 Generated with Claude Code
The docstring claimed compiling _round6 with and without the outer Float cast produced byte-identical SQL on all three dialects. Verified directly with SQLAlchemy against all four supported dialects: that's only true for MySQL/MariaDB, where CAST to FLOAT is unsupported and silently dropped. On PostgreSQL, CockroachDB and SQLite the cast does survive compilation and shows up in the SQL text (CAST(round(...) AS FLOAT)) -- the SQL is not identical there, even though the cast is still a no-op in effect, since the UPDATE ... SET assignment already coerces the result to the column's type on those backends. The conclusion (drop the cast) was already correct and stays; only the overstated evidence is corrected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build(), update()'s remaining_weight branch, and update()'s used_weight
branch each round used_weight to strip float64 noise, but had no coverage
in the fast suite -- only the SQL-side _round6 accumulator (use_weight_safe)
was tested, and that suite needs a live server plus Docker so it doesn't
run in the fast job. Proved by mutation: deleting all three round() calls
left the full backend suite at 1069 passed before this fix.
Each of the three new tests was confirmed to fail when its corresponding
round() call was deleted, then the code was restored:
- build(): removing its round() call failed
test_build_rounds_used_weight_noise_on_creation, others unaffected.
- update()'s remaining_weight branch: removing its round() call failed
test_update_remaining_weight_branch_rounds_derived_used_weight, others
unaffected.
- update()'s used_weight branch: removing its round() call failed
test_update_used_weight_branch_rounds_noise, others unaffected.
Inputs are chosen to land on genuine float64 representation noise (e.g.
0.1 + 0.2 == 0.30000000000000004, and 1000 - 999.7 == 0.2999999999999545)
so the assertions can't pass by coincidence.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… the CASE
use_weight_safe's refill path builds `CASE WHEN ... THEN round(CAST(... AS
NUMERIC(18,6)), 6) ELSE 0.0 END`. CockroachDB requires every branch of a CASE to
resolve to the same type and rejects the mixed NUMERIC/FLOAT8 statement:
asyncpg.exceptions.InvalidParameterValueError: incompatible value type:
expected $5::FLOAT8 to be of type decimal, found type float
PostgreSQL, SQLite and MySQL/MariaDB all accept it, so only the CockroachDB leg
of the integration matrix failed (tests/spool/test_measure.py::
test_measure_spool_sequence[measurements3] -> HTTP 500; its final measurement
raises the weight, i.e. a refill, the only path through the CASE).
5a71a7a removed an outer `CAST(... AS FLOAT)` as a no-op that only bought a
MySQL/MariaDB compile warning. It was a no-op for the plain consumption
assignment, but not here: it was what made both CASE branches the same type.
Rather than restore it and the warning with it, cast the zero-clamp literal to
the same Numeric so the branches agree at the source -- CockroachDB accepts it
and MySQL compiles with zero warnings.
Extract the expression into _used_weight_after_refill() so the regression test
exercises the production expression rather than a copy of it, and rewrite the
_round6 docstring, whose claim that the cast "never changes what ends up in the
database on any backend" is now demonstrably false.
The new tests compile the real expressions for the cockroachdb, postgresql,
mysql and sqlite dialects and assert both CASE branches declare the same
fixed-point type, so the fast (SQLite) suite catches this class of bug instead
of leaving it to the Docker matrix. Verified by mutation: reverting the else_
cast turns 5 of the 7 new tests red, and restoring the pre-5a71a7a outer Float
cast turns 6 red.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes #377.
Split out of #378 (the chat tool-layer branch) so a user-facing crash fix does not wait behind a 37-commit feature. The two commits are cherry-picked unmodified; the third adds the changelog entry that would otherwise have landed under the wrong feature heading.
The crash was the second symptom, not the bug
The report was
TypeError: e.toFixed is not a functionon the home page, with a spool weight total of2913718.838604448929592810524802001000955455597751000200— 50-odd decimal digits. That number was never stored anywhere. It was built in the browser.json-bigint'sstoreAsStringoption decides what to keep as a string by looking at the length of the raw literal (>15 characters), not at whether the value is an integer. So an ordinary gram value carrying float64 representation noise —0.30000000000000004is 19 characters — parsed as astringdespiteused_weightbeing a declared number.The home page then computes
sum + spoolStockWeight(s). One string operand turns+into concatenation, so from that spool onward every weight was appended as digits instead of added. That is the reporter's 55-digit number: a running concatenation, not a stored value. Any total downstream of the first noisy spool was silently wrong long before anything crashed. Sub-1000 g values then reached.toFixed()as strings and took the render tree down — which is the part that got noticed.Fix
Three layers, because each is independently wrong:
storeAsString: false, so long literals arrive asBigNumberinstances. Unlike strings, those can be told apart from genuine string fields, so each is revived individually: only whole numbers beyondNumber.MAX_SAFE_INTEGERbecome strings. The CockroachDB large-id behaviour from Fix JS integer-precision loss on large CockroachDB IDs that makes the UI 404 on vendor_id #69 is preserved exactly — that is what the option was there for, and it still works.formatWeight,formatWeightCompactandformatLengthnow use the sameNumber()+Number.isFinitecoercion thatnumberFormatterin the same file already used. A bad value renders as a dash instead of unmounting the page.used_weightis rounded on every write path. Six decimals, not one: enough to strip float64 representation noise, not so aggressive that it discards real sub-gram increments (a slicer can legitimately report ~0.03 g per layer). It rounds in SQL so the accumulator stays a single atomicUPDATE— rounding in Python would reintroduce the read-before-write that deadlocks MariaDB and forces CockroachDB serialization retries. The cast toNumericis load-bearing: PostgreSQL and CockroachDB have noround(double precision, integer)overload.The second commit removes the outer cast back to
Floatfrom that expression. It was a no-op on every supported backend — MySQL/MariaDB cannotCASTtoFLOATso SQLAlchemy dropped it with a warning, and on SQLite/PostgreSQL theUPDATE ... SETassignment already coerces to the column type. Compiling every dialect with and without it produces byte-identical SQL, so the warning was its only observable effect.Tests
client/src/utils/bigintJson.test.ts— long non-integer literals stay numbers; oversized integers still become strings (Fix JS integer-precision loss on large CockroachDB IDs that makes the UI 404 on vendor_id #69 unregressed).client/src/pages/home/analytics.test.ts— the concatenation path: a noisy weight in a list of spools must sum, not concatenate.client/src/utils/parsing.test.ts— formatters survive string/NaN/undefined input.tests_integration/tests/spool/test_use.py— repeated smallusecalls leaveused_weightfree of float64 noise, on every supported database.Verified on this branch: 925 backend tests and 90 client tests in the touched files, all green.
🤖 Generated with Claude Code