Store the wide decimal on the sixteen byte plane - #785
Merged
Merged
Conversation
A decimal is one declared type stored two ways. Up to eighteen digits the column is a lane word of unscaled units; above that it is sixteen little endian bytes of them at a fixed stride, which is the plane INT128 opened and the layout BINARY(16) already used. The precision in the declared type is what tells writer and reader which, and both ask the same function, fixed_octets, so they cannot disagree. That moves the frontier for the type to where an i128 ends. Thirty eight digits is the widest an i128 holds, so it is the widest a value of one can be, and DECIMAL(39,2) is refused at the declaration rather than at the table. A bare DECIMAL, which is DECIMAL(38,0), becomes a declarable and storable property type for the first time. DECIMAL(38,2) was the canonical spelled and not storable example in six places. All six now use INT256, which is durable: it will not become storable without a two hundred and fifty six bit value carrier. The directory version goes to 14.
8 of 19 tasks
tamnd
added a commit
that referenced
this pull request
Oct 1, 2026
#785 made zu1::props::fixed_octets public without regenerating docs/api/model.json, which is why api-model is red on main.
tamnd
added a commit
that referenced
this pull request
Oct 1, 2026
* Publish to crates.io as zudb, from a tag The crate name zu is taken on crates.io, so every crate that goes up is zudb or zudb-*, which is what the overview already promised. The packages are renamed and the libraries are not: the workspace table names each one under its old key with `package = "zudb-..."`, and the inner crates keep `[lib] name = "zu_..."`, so no source outside the engine crate changes. The engine crate's library is zudb because that is what `use zudb::Database` in the README needs, so its own tests, benches and doctests say zudb. Fourteen crates are published: zudb, the twelve under it that it or the CLI reaches, and zudb-cli, which installs the zu binary. The internal pins are exact, so a version bump that misses one fails to resolve rather than publishing against the previous release. The release workflow gains three real steps. verify checks the tag against the workspace and the changelog and runs a full publish dry run, so a crate that does not build from its tarball stops the release before anything is uploaded. crates-io runs scripts/publish-crates.sh, which is rudb's: it asks the index what is up, excludes it, and waits out the new crate limit, so the first release is about an hour and a half once and a re-run finishes a partial one. The GitHub release is created after crates.io, with the changelog section as its notes and provenance on the archives. The token is a secret of the crates-io environment, which only a v* tag can deploy to, and sits in the environment of one step. CI gains a package job running the same dry run on every pull request. The rustdoc model keeps calling the engine crate zu, so model.json does not move. The semver gate skips a baseline that has no zudb package, which is only this change. * Name the CLI's arrow feature by the package it is now A feature on the command line is spelled package/feature, and the package is zudb-cli. * Point the fuzz crate at the renamed packages It is outside the workspace, so it names each package itself rather than through the workspace table. * Bring the API model up to fixed_octets #785 made zu1::props::fixed_octets public without regenerating docs/api/model.json, which is why api-model is red on main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the note left open on the decimal row of S2 (#700).
A decimal was already one declared type: the column holds unscaled units and the scale in the catalog is what makes them a number. This makes it one declared type stored two ways. Up to eighteen digits the units ride a lane word, as they have since #765. Above that they are sixteen little endian bytes at a fixed stride, which is the plane INT128 opened in #777 and the layout
BINARY(16)has always used. Nothing new was needed on the storage side:fixed_octetsis the single hook that puts a column on that layout,write_columnsis generic over it, and answeringSome(16)gets padding, the segment write and the reader for free.The precision in the declared type is what decides which of the two a column is, and writer and reader both ask
fixed_octetsrather than deciding for themselves, so they cannot disagree about a column they are looking at together.Where the frontier is now
At the value carrier, which is where the wide integers landed too.
Value::Decimalcarries ani128of unscaled units, thirty eight digits is the widest ani128holds, so thirty eight digits is the widest a value of one can be.DECIMAL(39,2)has no declared form and is refused at the declaration with 42000, the same place and the same condition the wide decimal's own refusal used to land. Below that line every precision a statement can spell is a column a statement can fill.One consequence worth naming: a bare
DECIMALisDECIMAL(38,0)by the parse cap, so a bareDECIMALis now a declarable and storable property type. It was not before.The example that had to move
DECIMAL(38,2)was the canonical spelled and not storable example in six places, and it stores now. All six moved toINT256, which is durable in a way the decimal was not: the decimal was one carrier away from storing and did store, whereas anINT256column would take a row in and be unable to give it back, and that does not change until the engine has a two hundred and fifty six bit carrier. The six aresession.rs,refusals.rs,refusal_shape.rswith its regenerated snapshot,catalog_statements.rs,graph_type.rsand the corpus casea-property-type-no-column-can-hold.What is checked
check_declaredreads sixteen octets back into aDecimalat the declared scale and compares its digit count to the declared precision, so a wide column is held to its declaration the same way the narrow one is, on the same code path the narrow one uses.frontier()inprops.rsnow pinsDECIMAL(38,2)storable andDECIMAL(39,2)not, and the extended type test walks precisions 1, 9, 18, 19, 38, 39 and 76 and asserts which plane each is on. The new end to end test declaresDECIMAL(32,4), inserts a twenty eight digit value, a rescale up fromDECIMAL(5,1)and a bare integer, reads all three back in order, and pins the two 22003 refusals: thirty three digits, and a fifth decimal place.The directory version goes to 14.
Local gates green: fmt, clippy with
-D warnings,xtask terms,xtask api-map, and 89test result: okacross the workspace with no failures.