New reflection primitives, e.g. the Facet trait provided by facet, allow for simpler runtime inspection of deserialized data.
This project uses facet to encode data of arbitrary shapes into Git trees.
The invariants and functionality described here will be upheld (and expanded) while the project is developed.
The following requirements define the expected behavior of the Git tree serialization and deserialization targets.
|
Caution
|
This set of requirements is not yet stable, and will grow (and change) over time. |
The facet-git-tree crate MUST satisfy all requirements of general facet adapter crates.
The facet-git-tree crate MUST strictly adhere to semantic versioning.
The facet-git-tree crate MUST be compatible with Git’s v2 object format.
Object identities MUST be computed as 20-byte SHA-1 digests over Git’s canonical object encoding, matching Git’s default (non-objectFormat) repositories.
|
Note
|
SHA-256 repositories are currently out of scope, but support SHOULD be added in the future as the project matures. |
The facet-git-tree crate MUST provide a struct which can be serialized into Git trees using only the Facet trait.
A scalar value — one that is neither a composite nor a collection — MUST be stored in a UTF-8 encoded Git blob.
Booleans MUST be serialized as true or false strings.
A serialized tree MUST record values only.
The encoding MUST NOT embed any representation of a type’s SHAPE — no schema object, type identifier, or definition table.
Both serialization and deserialization are driven entirely by the Facet type supplied at the call site, exactly as facet-toml and facet-json rely on the target type rather than embedding a schema in their output.
The structure of every tree and blob is recovered from that type on read.
|
Note
|
Because no schema is stored, type definitions cannot introduce cycles into the encoding; values are inherently cycle-free under Git’s DAG data model.
Self-describing reads — recovering a value without its |
A composite value — a struct or tuple — MUST be encoded as a Git tree, with each field recorded as an entry whose value follows the leaf, composite, collection, and variant rules recursively. A struct’s fields are named by their field name; a tuple’s fields are named by their zero-based positional index, encoded as a zero-padded ordinal name.
An enum value MUST be encoded as follows, externally tagged by the active variant’s name.
-
A unit variant (no payload) MUST be encoded as a single Git blob holding the variant name’s bytes verbatim, followed by the mandatory trailing newline defined by leaf byte encoding — this blob is a leaf blob like any other — with no wrapping tree. The variant name is the entire information content of a unit variant, and a blob is what makes it visible to blob-oriented tooling: a tree entry whose target is a tree with no blob underneath it does not appear at all in
git ls-tree -r’s recursive blob listing and contributes nothing to `git diff, so a unit variant written as an (empty) tree is indistinguishable, to that tooling, from the field never having existed. Encoding it as a blob instead means a field changing between two unit variants (or between a unit variant and any other payload shape) shows as an ordinary content change. -
Every other variant MUST be encoded as a Git tree holding exactly one entry, named after the active variant, whose value encodes the variant’s payload as follows.
-
A newtype variant (one field) resolves directly to the encoding of that field.
-
A struct variant resolves to a tree whose entries are named by field name.
-
A tuple variant resolves to a tree whose entries are named by zero-based positional index, encoded as a zero-padded ordinal name.
-
The variant name is a value, not schema: it records which variant is live, information not present in the type’s SHAPE.
|
Note
|
A unit variant and every other variant therefore have different tree shapes at the same field position — a blob for the former, a tree for the latter. This diverges from a uniform externally-tagged-tree scheme (every variant, unit included, as a single-entry tree); a reader MUST branch on the fetched object’s kind (blob vs. tree) to tell them apart, rather than assuming one shape ahead of fetching. See the empty-tree marker for the parallel treatment of |
None, a dynamic Null, and an empty collection (Array, Vec, slice, or Map, per collection encoding; likewise an empty dynamic Array or Object, per dynamic value encoding) MUST be encoded as a Git tree holding exactly one entry, rather than a literal empty tree: a blob entry named _ (the presence marker) whose content is the empty byte sequence.
The marker blob MUST remain the literal empty blob unconditionally: it is a structural presence sentinel, not a value leaf, and is therefore exempt from the leaf trailing-newline requirement that applies to every other blob this encoding writes.
A literal empty tree — the encoding these cases used before this marker was introduced — contributes nothing to git ls-tree -r (which lists only reachable blobs, recursively, and an empty tree recurses to zero lines) and nothing to git diff (which compares blob content by path). A field that becomes empty, or that was always empty, is therefore indistinguishable, to that tooling, from a field that never existed. Writing the single marker entry instead keeps the path visible — git ls-tree -r shows (for example) tags/_ — and keeps it participating in blob-level diffs when a collection transitions to or from empty.
A single shared marker (one reserved name, one empty-content blob) covers every one of these cases; no case gets a distinct marker. The marker’s only role is presence-signaling for git tooling, not disambiguation: every reader decodes a possibly-marked tree against a known target (a Facet type or a schema node) that already determines, independent of the tree’s content, whether None/Null/an empty List/an empty Map/etc. is expected at that path. The one reader without such a target — the dynamic value read heuristic — already documented Null and an empty Array/Object as collapsing to the same lossy empty-Object reading before this marker existed; a shared marker preserves that reading exactly, rather than introducing a new distinction the heuristic has no way to use.
Because every leaf blob (leaf byte encoding) now carries a mandatory trailing \n while the marker blob never does, an empty String — which serializes to the one-byte blob \n — can no longer collide with the marker’s blob: the two are always distinguishable, on disk, from a byte length alone (1 versus 0). This is a deliberate side effect of the trailing-newline requirement, not a separate rule.
The marker name is reserved: a dynamic (map or dynamic-object) key MUST NOT equal it, and MUST be rejected if it does, for the same reason a key MUST NOT contain / — a real entry named exactly would otherwise be indistinguishable, on read, from the marker. A schema-declared struct field name MUST be rejected on the same terms: a schema document is data, and one authored by hand rather than derived from a Rust type can name a field anything at all. An ordinal name, being always decimal digits, can never collide and needs no such check.
|
Note
|
A unit enum variant (variant encoding) is deliberately handled differently: it collapses its entire entry to a bare blob rather than keeping a tree with a marker. The two cases differ in what information the "empty" state carries: |
The following standard Rust collections MUST be supported, each encoded as a Git tree — an empty one per the presence-marker requirement rather than as a literal empty tree.
-
Array,Vec, and slices — sequence elements are recorded as entries named by their zero-based index, encoded as a zero-padded ordinal name. A sequence whose element type isu8is the sole exception: it is stored as a single blob rather than a tree, per byte-sequence encoding. -
Map— the entry layout is selected by the shape of the key type after the same transparency collapse defined by smart-pointer encoding and transparent-newtype encoding is applied (a smart pointer collapses to its pointee, repeatedly, and a transparent newtype to its inner shape), with no on-disk marker for which layout was chosen, since that collapsed shape is known to both writer and reader:-
Scalar keys (those whose collapsed
DefisScalar— integers, floats,bool,char,String, and other types stored as a single leaf, including a smart pointer or transparent newtype wrapping one of these, such asArc<str>): each entry is named by the textual form of its key and resolves to the encoded value, exactly as a leaf would be encoded. This is the readable, JSON-like form. A key MUST NOT equal the reserved presence-marker name (_, presence marker). -
Composite keys (structs, tuples, enums, sequences, and any other key whose collapsed shape does not encode to a single leaf): the textual-name form cannot faithfully represent them, so each pair is recorded as a two-entry sub-tree with children named
kandvholding the independently-encoded key and value. The map’s entries point at these pair sub-trees and are named by a zero-padded ordinal name; the ordinals are assigned in ascending order of each pair sub-tree’s object id so that the map remains content-addressed independent of iteration order.
-
No type marker is recorded; element and key types are recovered from the target Facet type on read.
A sequence of u8 — Vec<u8>, [u8; N], or a [u8] slice — MUST be stored as a single Git blob whose content is the bytes verbatim, followed by the mandatory trailing newline defined by leaf byte encoding (this blob is a leaf blob, and the rule applies uniformly regardless of the byte sequence’s own content), rather than as a tree with one entry per byte.
This is the Git-native representation of a byte buffer: it lets identical buffers deduplicate to one object and avoids the cost of a per-byte tree. The distinction is recovered from the element type on read; no marker is stored.
A smart pointer — Box<T>, Arc<T>, Rc<T>, including the unsized slice forms Arc<[T]> and friends — MUST be encoded as the encoding of its pointee T, with no wrapping tree or marker.
The indirection carries no information Git needs to record, so it is transparent: Box<Issue> encodes exactly as Issue, and Arc<[u8]> encodes exactly as [u8] (a single blob). On read the pointer is reconstructed around the decoded pointee.
RawTree, a type provided by this crate wrapping an ObjectId, MUST be encoded as a tree entry pointing at that object id directly, with no re-encoding of the referenced tree’s contents.
RawTree embeds a subtree that was written independently of the Facet encoding this crate performs — an arbitrarily-shaped directory with no fixed layout, alongside ordinarily-encoded fields in the same struct. The wrapped object MUST already exist in the store being serialized into; writing a RawTree value performs no object-store write of its own. On read, the entry’s object id is captured as the RawTree value without decoding what it points to, but is verified to be a tree, not a blob, since that is the only shape a RawTree may hold.
Entries addressed by a positional index — tuple fields, tuple-variant fields, and the elements of sequence collections (Array, Vec) — MUST be named by their zero-based index encoded as zero-padded decimal at least four digits wide (0000, 0001, …, 0010, …).
|
Note
|
The zero-padding aligns Git’s tree-entry ordering with numeric index ordering, which keeps sequentially-accessed entries adjacent for better cache-line locality during tree iteration.
This is purely a performance optimization: correctness MUST NOT depend on tree-entry ordering, and indices MUST be parsed numerically on read.
A collection with more than Two entries whose names parse to the same numeric index (e.g. |
A leaf blob — a scalar, a byte sequence (byte-sequence encoding), or a unit enum variant’s name blob (variant encoding) — MUST be encoded as the value’s raw representation, with no delimiters or quoting, followed by exactly one trailing \n byte.
A String is stored as its UTF-8 bytes verbatim plus that one byte; a scalar is stored as the bytes of its textual form plus that one byte; a byte sequence is stored as its bytes verbatim plus that one byte.
The trailing byte MUST be unconditional: it MUST be appended whether or not the value’s raw representation already ends in \n, and MUST be present exactly once, never zero times and never more than once. On read, exactly one trailing \n MUST be stripped to recover the value; a leaf blob whose final byte is not \n MUST be treated as malformed — necessarily a foreign or corrupt object — and rejected with an error, never accepted leniently as though the byte were merely optional.
The presence marker blob is not a leaf blob and is exempt from this requirement: it is a structural presence sentinel, not a value’s own representation, and remains the literal empty blob.
|
Note
|
The rule is "exactly one, always present" — not "at most one, if present". Only the unconditional form is exactly invertible: under an "at most one" reading, the The purpose is cosmetic but real: without a trailing newline, every leaf blob is a file "without a final newline" to Git, and This is the representation that leaf normalization operates on (before the trailing byte is appended), and the content — trailing byte included — that Git wraps in its canonical |
The following leaf representations MUST be normalized on-write.
-
Negative zero MUST be normalized to positive zero.
-
The not a number (
nan) value MUST be normalized to the unquoted stringnan.
|
Note
|
Normalization operates on the value’s raw representation, before the mandatory trailing newline is appended; it has no bearing on that byte, which is appended unconditionally regardless of what normalization did or did not change. |
A dynamic value — one whose shape is dynamic (Def::DynamicValue, e.g. facet_value::Value), carrying its kind at runtime rather than in a static shape — MUST be encoded according to the kind held at write time, as follows. Every kind below that is stored as a blob (Bool, Number, Char, String, Bytes, DateTime, Uuid, QName) writes a leaf blob and is therefore subject to the mandatory trailing-newline requirement like any other leaf, in addition to the rules given here.
-
NullMUST be encoded as the presence-marker tree, per the presence-marker requirement, rather than a literal empty tree. -
BoolMUST be encoded as atrueorfalseblob, per boolean serialization. -
NumberMUST be encoded as a blob holding its textual form, normalized identically to scalar leaves per leaf normalization:nanfor not a number, and negative zero normalized to positive zero. An integer outside thei64/u64range MUST be rendered exactly, or the write MUST fail; it MUST NOT be rendered lossily through a floating-point conversion. A value that is itself floating-point-backed MUST be rendered per the same float normalization rules regardless of magnitude — including a magnitude beyond what a 64-bit or 128-bit integer could hold — since it is not an integer being approximated. An implementation whose runtime interface cannot distinguish an out-of-range whole value’s exact representation (integer vs. floating-point) MUST fail rather than guess. -
CharMUST be encoded as a blob holding the character’s UTF-8 encoding. -
StringMUST be encoded as a blob holding its UTF-8 bytes verbatim, andBytesas a blob holding its bytes verbatim, exactly per leaf byte encoding and byte-sequence encoding. -
DateTimeMUST be encoded as a blob holding RFC 3339-style text conforming to the following normative grammar. An offset datetime MUST carry its offset (Zor±HH:MM); a local datetime, local date, or local time MUST use the corresponding bare form. A negative (BCE, proleptic Gregorian) year MUST render as"-"followed by its magnitude zero-padded to four digits (year-5→-0005); a year at or beyond10000MUST print all of its digits rather than being truncated to four.year = ["-"] 4*DIGIT date = year "-" MM "-" DD time = HH ":" MM ":" SS ["." 1*9DIGIT] offset = "Z" / ("+" / "-") HH ":" MM offset-datetime = date "T" time offset local-datetime = date "T" time local-date = date local-time = time -
UuidandQNameMUST be encoded as blobs holding their defined textual forms: aUuidas its canonical hyphenated lowercase form; aQNamewith a non-empty namespace as Clark notation ({namespace}local), and aQNamewith no namespace — including one whose namespace is present but the empty string — as the bare local name. -
ArrayMUST be encoded as a Git tree whose entries are named by zero-padded ordinal names, one per element. -
ObjectMUST be encoded as a Git tree whose entries are named by their member keys, each key subject to the same entry-name validity rule enforced for scalar map keys and struct field names.
|
Note
|
No on-disk marker distinguishes a dynamic value from the same data written through a typed |
Deserialization of data serialized via facet-git-tree into its original typed Facet target MUST result in the original data. Typed Facet round trips are expected to be faithful. This requirement does not apply to a schemaless dynamic-value target; its read behavior is defined by the dynamic value read heuristic.
When the deserialization target is a dynamic value, no schema is available on disk and none is supplied by the target type, so the reader MUST apply the following heuristic.
-
A blob MUST end with the mandatory trailing
\nbyte required of every leaf blob (leaf byte encoding); its absence MUST be an error, not a lenient accept. That byte MUST be stripped first; the remaining content, if valid UTF-8, MUST be read as aString, otherwise asBytes. -
A non-empty tree in which every entry name parses as a decimal ordinal MUST be read as an
Array, with elements ordered by numeric index per entry name ordering. -
Any other tree — including the presence-marker tree (presence marker), which MUST be stripped before this classification so it is not misread as a one-member
Object— MUST be read as anObject, with one member per entry.
This mapping is best-effort and lossy: it recovers structure, not the writer’s dynamic kind. In particular, scalar numeric text does not retain the original numeric width, signedness, or scalar type, so that information cannot be recovered without a schema or typed target. Null and empty dynamic Array/Object values share the same marker and are all read with the empty-Object interpretation; the heuristic cannot distinguish those empty states.
The following writes do not survive a dynamic round-trip.
| Written | Read back as |
|---|---|
|
empty |
|
|
|
|
empty |
empty |
|
|
Recovering full fidelity requires either a typed read — supplying the original Facet type — or a schema-driven read per schema-driven deserialization.
Given a schema document, the schema-driven reader (deserialize_value_with_schema) MUST recover a faithful dynamic value from a tree written by this crate’s encoding, without the original Facet type, according to the following mapping from schema node to value.
Every blob referenced in the table below is a leaf blob per leaf byte encoding: "the blob’s textual form" or "the blob’s content" means that blob’s bytes with the mandatory trailing \n already stripped, and a blob missing that byte MUST fail the read rather than be accepted with the byte folded into the value.
| Schema node | Value produced |
|---|---|
|
|
|
an exact |
|
a |
|
the corresponding dynamic kind parsed from the blob. |
|
|
|
an |
|
an |
|
an |
|
an |
|
an |
|
|
|
a single-member |
|
a |
|
the result of the dynamic value read heuristic applied at that point. |
|
the value read via the definition named |
Recursion depth MUST be bounded by the same maximum-depth limit that bounds ordinary deserialization, and every hop MUST count against that limit — including Ref resolution — so that Ref-to-Ref chains cannot recurse unboundedly.
Node and Schema MUST be ordinary Facet values, self-hosted: a schema document is stored as a Git tree via this crate’s own encoding, with no special-cased representation.
Their on-disk form is a public contract: any change to the encoded shape of the schema types is a breaking change and MUST be released as a semver-major version, per compatibility with prior releases.
A schema document also carries its embedded kind name. The name is part of
the schema document and is validated as a Git ref-name segment; it MUST NOT be
inferred from a publication ref. A publication ref is an authoring/history
index, not the type definition of an existing bound data document.
The schema types are defined as follows.
pub struct Schema {
pub kind: String,
pub root: Node,
pub defs: BTreeMap<String, Node>,
}
#[repr(u8)]
pub enum Node {
Unit, Bool, Char, String,
I8, I16, I32, I64, I128, ISize,
U8, U16, U32, U64, U128, USize,
F32, F64,
Bytes,
Struct(BTreeMap<String, StructField>),
Tuple(Vec<Node>),
List(Box<Node>),
Array { elem: Box<Node>, len: usize },
Map { key: Box<Node>, value: Box<Node> },
Optional(Box<Node>),
Enum(BTreeMap<String, VariantKind>),
RawTree,
Dynamic,
Ref(String),
}
pub struct StructField {
pub node: Node,
pub has_default: bool,
}
#[repr(u8)]
pub enum VariantKind {
Unit,
Newtype(Box<Node>),
Tuple(Vec<Node>),
Struct(BTreeMap<String, Node>),
}A struct node’s fields and an enum node’s variants MUST be keyed by name, so the name IS the tree entry name under the scalar-key map layout — no ordinal indirection, and no special case in the encoder: the schema types are ordinary Facet values walked by the same generic encoding, which is what this requirement’s self-hosting claim already demands.
A Node::Struct field is a StructField, not a bare Node: has_default marks a field a write MAY leave without a tree entry, per schema-directed serialization's Struct row, and a read finds such an omitted entry simply absent rather than failing, per schema-driven deserialization's. A struct enum variant’s fields (VariantKind::Struct) carry no such marker and stay bare `Node`s.
Field and variant declaration order is consequently NOT recorded and MUST NOT be relied upon. It is not load-bearing anywhere in the encoding — struct values encode name-keyed and enum variants tag by name — so nothing a reader does depends on it.
A field or variant name MUST be a legal tree entry name — the same constraint a dynamic map key is under — and MUST be rejected otherwise.
A schema tree therefore reads like a type declaration, as git ls-tree -r of a published Issue schema shows (trimmed to the relevant entries):
defs/Issue/Struct/id/has_default -> "false\n"
defs/Issue/Struct/id/node -> "String\n"
defs/Issue/Struct/labels/has_default -> "false\n"
defs/Issue/Struct/labels/node/List -> "String\n"
defs/Issue/Struct/parent/has_default -> "false\n"
defs/Issue/Struct/parent/node/Optional -> "String\n"
root/Ref -> "Issue\n"Every stored schema document MUST carry, at the fixed top-level tree entry name schema, a reference to the tree of the schema-schema generation it was written against.
That entry is a storage-layer splice added when a document is written, NOT a field of Schema: a schema: field on the type would make schema_of::<Schema>() describe the pin and recurse without bottom.
This is the same construction as gix-store’s subtree schema binding, which splices `{value/, schema/} onto a data commit’s tree.
A generation’s tree is the serialization of schema_of::<Schema>() — the document describing `Schema’s own shape — under that generation’s encoding.
A generation’s tree MUST also carry a codec entry: a {schema/, value/} pair holding the schema and the encoded value of a fixture designed to exercise every construct this format’s codec can encode.
The fixture’s schema alone binds only the shape language Node/VariantKind describe; its value, produced by the same serializer rather than hand-built, additionally binds how each construct is actually spelled — an f64’s text, `Bytes’ framing, an empty collection’s marker, `None’s marker, and so on — so a codec change that leaves `Schema’s own encoded shape untouched still moves `codec/value’s object id, and therefore the generation’s own tree id.
`codec/schema/ MUST NOT itself be pinned: it is inside the generation being defined, and pinning it would recurse; it is identified by containment instead.
The recursion bottoms out at a root generation, whose tree is written with no pin spliced in.
Absence of a schema entry therefore means "this is the root of the tower", and is legitimate ONLY when the tree’s own object id is one the reader recognizes as a root generation.
A reader MUST reject an unpinned tree whose own id it does not recognize as a root: that is a truncated or hand-written document, not the bottom.
A reader MUST carry the recognized generations as compiled-in constants; the genesis tree of this format version is ea875f69726986da822cdb2670a089eddd09b6ce.
Generation N’s own tree MUST pin generation N−1 under its own schema entry.
Ancestry through those entries is what restores the ordering a version number gave and a bare object id does not; there is one generation today, so the chain has length zero, but the format reserves the shape.
A reader MUST determine what a document was written against, and refuse it if it does not recognize it, WITHOUT deserializing the document.
The read is git ls-tree <tree> schema — one tree entry lookup, strictly cheaper than the git cat-file blob <tree>:version it replaces, since the pinned id is in the entry itself and no object need be read.
This ordering is load-bearing, for the same reason the old version pre-read was: a document written by a newer binary may contain a Node variant the reader has never heard of, and a typed deserialize attempted first fails with an opaque reflection error before the check is ever reached.
An object-id pin admits equality only.
A reader that supports the current self-hosted schema contract cannot repair a
historical schema object that predates the embedded kind field by guessing a
name from refs/schema/<publication>. Such a schema needs an explicit
compatibility conversion or republishing before it can serve as the embedded
schema of newly written self-contained documents.
A reader MUST NOT infer that an unrecognized pin is newer or older than what it speaks, and the diagnostic MUST say what it honestly knows: this document was written against schema-schema <oid>, which I do not recognize; I speak <the recognized set>.
Publishing a schema is governed by the same refusal as reading one: a writer MUST NOT replace a published schema pinned to a schema-schema it does not recognize, since overwriting a document whose meaning was never established asserts the same guarantee that reading it would have. A published schema that is unpinned or otherwise unreadable MUST remain replaceable, republishing being the remedy those conditions name.
The schema-schema pin binds Schema’s own on-disk shape and the value codec it
describes together, in lockstep: a reader that rejects an unrecognized pin
refuses to misread a value written by a newer codec, with no caller-supplied
version field required. The `codec entry above is what lets the pin cover the
same ground: `Schema’s shape and the codec’s spelling of every construct it
can produce both live inside the generation’s own tree, so either changing
moves the generation id.
The converter from a Facet shape to a schema document — schema_of::<T>() / Schema::from_shape — MUST mirror the encoder’s dispatch order exactly: transparency collapse first (smart pointers and transparent newtypes resolve to their pointee or inner shape), then RawTree, then dynamic values, then the scalar table, then byte sequences, then composites (structs, sequences, maps, options, enums).
A shape and its transparent wrappers therefore always produce the same schema, just as they produce the same encoding.
This is the same collapse map-key layout selection applies, so both agree on what a given shape actually encodes to.
The scalar table MUST refuse () (the unit type) exactly as the encoder’s scalar leaf encoding does: () has no textual (Display/FromStr) rendering, so it cannot reach a leaf blob and cannot be described by a schema either.
This does not remove Node::Unit from the format — a unit struct still resolves to it, describing the (always, statically) empty tree its composite encoding writes; only the scalar table’s direct mapping from () is refused.
|
Important
|
A unit enum variant does NOT resolve to That reasoning is about a fixed path, and does not extend to a unit struct reached through a varying one. A |
A plain (non-variant) named struct’s fields MUST carry StructField::has_default set from the underlying facet_core::Field::has_default() — true exactly when the field’s declaration carries [facet(default)] or [facet(default = expr)]. The marker records only that a default exists, never the default value itself: the value may be computed at write time (a current timestamp, for instance), so baking a snapshot into a published schema would let a stale schema silently misrepresent what a fresh write actually produces. A struct enum variant’s fields carry no such marker.
Named user types — structs and enums — MUST be deduplicated into the document’s defs table and referenced by Node::Ref(name).
Names MUST be assigned deterministically: the key is the type’s type_identifier, and when distinct types share an identifier, later occurrences in pre-order traversal of the shape are disambiguated with _2, _3, … suffixes.
Recursive types are representable because a type’s name is registered before its body is computed; the cycle is broken by the Ref.
Generation MUST be deterministic: the same input shape always yields an identical Schema, so a generated schema serializes to a stable object id.
Shape recursion MUST be bounded by the same maximum nesting depth that bounds ordinary deserialization (schema-driven deserialization already applies this bound to reads); a shape nested deeper than that limit MUST fail generation rather than recurse unboundedly, since data written that deep could never be read back regardless of what schema described it.
Given a schema document, the schema-directed writer (serialize_value_with_schema) MUST encode a dynamic value as the Git objects the schema describes, producing byte-for-byte — and therefore object-id — identical output to dynamic encoding of the equivalent typed value, according to the following mapping from schema node to accepted value.
Every blob produced in the table below is a leaf blob per leaf byte encoding: the mandatory trailing \n is appended automatically by the encoding described and is not itself part of the "textual form" named in the table.
Encoding and conformance are one operation: a value that does not conform MUST fail — naming the path within the value at which the mismatch occurred — rather than be encoded lossily or ambiguously.
The accepted set MUST be exactly the image of schema-driven deserialization — every value that read can produce MUST re-encode — together with the two deterministic bridges a value authored without type information (for example, parsed from JSON) requires: a lossless integer supplied for a floating-point node, and a String supplied for a Bytes node.
| Schema node | Accepted value and encoding |
|---|---|
|
|
|
an integer |
|
a |
|
the corresponding kind, encoded as the matching leaf blob. |
|
|
|
an |
|
an |
|
an |
|
an |
|
an |
|
|
|
a single-member |
|
a |
|
any value, encoded by dynamic encoding. |
|
the value encoded via the definition named |
Recursion depth MUST be bounded by the same maximum-depth limit that bounds deserialization, and every hop MUST count against that limit — including Ref resolution — so that a Ref-to-Ref cycle in the schema fails rather than recursing unboundedly.
|
Note
|
Publishing a schema under a ref — e.g. |
A schema migration MUST be applied when a value is read, and MUST NOT rewrite the stored value.
Attestations bind to object hashes. Rewriting a stored value to conform to a newer schema changes its tree hash, and thereby silently voids every claim already made about it. No operation defined by this section may produce a new stored value tree from an old one: application takes a value already read against the old schema and returns a value conforming to the new one, touching no object store and writing no object.
Migration is deliberately one-directional. New readers read old data; old readers reading new data is not a case this format has, so reverse lenses are out of scope and MUST NOT be inferred from the operators defined below.
A migration MUST be data — an ordinary Facet value stored through this crate’s own encoding, self-hosted exactly as a schema document is.
It MUST NOT be, or contain, a function pointer, a closure, or any other Rust-only escape hatch: the canonical artifact has to be interpretable by a consumer that is not Rust and is verifying a claim.
The migration types are defined as follows.
pub struct Migration {
pub ops: Vec<Op>,
}
pub struct Op {
pub at: Target,
pub change: Change,
}
#[repr(u8)]
pub enum Target {
Def(String),
Variant { def: String, variant: String },
}
#[repr(u8)]
pub enum Change {
Rename { from: String, to: String },
Add { field: String, default: Constant },
Remove { field: String },
Wrap { field: String },
}
#[repr(u8)]
pub enum Constant {
Null,
Bool(bool),
Integer(i64),
Float(f64),
Text(String),
List(Vec<Constant>),
Object(BTreeMap<String, Constant>),
}A migration document describes exactly one schema edge. It does NOT carry its two endpoint documents: an edge is identified by where it is stored — the plan of record is the schema commit’s own tree, keyed by the parent→child edge of `refs/schema/<kind>’s commit chain — so embedding the endpoints would duplicate trees the reader already has in hand.
Constant is the closed JSON data model, so an added field’s default is interpretable by a consumer with no Rust type available.
Integer is 64-bit; a default outside that range is out of scope and MUST be treated as a new type plus a backfill, not a migration.
The vocabulary MUST remain minimal. Every operator is semantics that every consumer, in every language, must implement forever; a change that does not fit the operators below is a new type plus a backfill, not a migration.
The operators are exactly four:
Rename { from, to }-
the target’s field
toholds what the source’s fieldfromheld. Add { field, default }-
the target has a field the source lacks; every upcast value takes
default. Remove { field }-
the source has a field the target lacks; the upcast drops it.
Wrap { field }-
the target’s field schema is
Optionalof the source’s.
Wrap is the identity on a dynamic value.
Schema-driven deserialization reads Optional as null or as the inner value directly, so Some(x) and x are the same value.
The operator exists regardless, for two reasons: it records the encoding change — a some tree entry appears — for a consumer working at the tree altitude rather than the value altitude, and without it the most common schema evolution after add, remove, and rename would be unclassifiable by derivation.
The inverse of Wrap is deliberately absent: a stored None has no image under an unwrap, so Optional<T> → T is not a migration.
Adding an enum variant requires no operator: no stored value can hold a variant the source schema never defined, so the edge is a pure widening. Removing an enum variant has no operator, and MUST be reported as unclassifiable: values holding it have no image.
An operation MUST address a definition, never a root-relative path into the value.
Target::Def(name) names an entry in the source document’s defs table; Target::Variant { def, variant } names a struct variant of an enum definition.
Every named user type is a definition (schema generation), so every named field an operator can address is reachable this way.
Root-relative paths are rejected because they cannot express a recursive type: struct Node { children: Vec<Node>, label: String } renaming label needs the change to apply at every depth, and the set of paths reaching those occurrences is unbounded.
A definition scope also applies once to a type used at several positions, which is what the change actually means.
The consequence is that a migration MUST be applied by walking the value guided by its source schema document, applying each target’s operations wherever the walk resolves that definition.
Derivation over two schema documents MUST produce either a complete migration or a typed report of what it could not classify, and MUST make the two outcomes distinct in the type system so a caller cannot accidentally treat a partial derivation as complete.
Derivation MUST be deterministic: the same document pair and the same authoring hints always yield an identical migration, and therefore a stable object id.
Definitions, fields, and variants are compared in sorted order, and operations are emitted in the phase order Rename, Remove, Wrap, Add, so that every phase after the first addresses target-side names.
Because struct and enum nodes are name-keyed (schema representation), added and removed fields fall out of a key-set comparison.
What a diff cannot tell is whether a {remove: a, add: b} pair is a rename or two unrelated changes, and what constant an added field should take.
Those are authoring facts and MUST be supplied by the author, either as a hint passed to derivation or, for renames, declared on the type as #[facet(migrate::renamed_from = "old_name")] and compiled into the document.
An added field whose target schema is Optional MUST default to Null without authoring: absence is that schema’s canonical inhabitant.
Every other added field without a supplied default MUST be reported as unclassifiable rather than given an invented value.
Application MUST take a value read against the source document and return a value conforming to the target schema, per read-time upcast.
Operations at a target apply in document order, each a total transformation of that node, so conflicting operations are well-defined rather than an error.
Rename and Remove on a field the value does not carry MUST be no-ops: application walks a value already read, not a tree, so a field’s absence there is not rechecked against the source schema.
Add MUST insert the constant, overwriting any member already present.
Children MUST be upcast before their parent’s operations run, so that operations address source-side field names exactly as they were derived.
Recursion depth MUST be bounded by the same maximum-depth limit that bounds deserialization, and every hop MUST count against it — including Ref resolution.
A chain of edges A→B→C MUST be applicable by applying each edge in series, in order.
Edges are applied, not composed into a single document. Each edge’s operations are scoped by the definition names of its own source document, and two documents in a chain need not agree on those names, so composing the documents would require a name reconciliation that applying in series does not need. Concatenating two edges' operation lists is therefore NOT a valid composition and MUST NOT be relied upon.
Every stored migration MUST carry, at the fixed top-level tree entry name schema, a reference to the tree of the migration-schema generation it was written against, under exactly the construction the schema-schema pin defines: a storage-layer splice, read by one ls-tree entry lookup, checked before the document is deserialized, with absence legitimate only for a recognized root generation.
The migration tower MUST be separate from the schema-schema tower.
The two documents evolve independently, and adding a Change variant must not invalidate every stored schema document.
A migration-schema generation’s tree MUST carry a codec entry under exactly the same construction the schema-schema tower’s does (the schema-schema pin) — the identical fixture, spliced under the identical name, so the two towers share one content-addressed codec object regardless of which one writes it first.
The genesis tree of this format version is f7f20e16f50e0363863d6322abe6c49a24706711.
The check is load-bearing in a way the schema case is not: a reader that silently ignored an operator it did not recognize would not fail, it would produce a wrong value — one that looks conformant and is not.
|
Note
|
|
A schema commit that advances over a predecessor MUST record the migration off that predecessor at the fixed top-level tree entry name migration in its own tree, and a commit that establishes a kind MUST NOT carry one.
Storing the migration in the schema commit rather than at a ref of its own is what keeps it reachable.
A value’s commit binds its schema tree; the schema commit carrying that tree also carries the migration off its predecessor; so a fetch of one data ref brings the schema and the lineage needed to upcast it, with no dependence on any further ref having been fetched.
A separate migration ref would reintroduce exactly the failure gix-store’s subtree schema binding — the `{value/, schema/} split on a data commit’s tree — exists to prevent.
An edge whose derivation is incomplete MUST NOT be recorded. A reader that then needs that edge MUST fail, naming the schema commit that lacks it, rather than applying a partial migration: a silently lossy upcast produces a value that looks conformant and is not.
Upcasting a value MUST locate the value’s bound schema tree in the schema ref’s history and apply each recorded migration from there forward, in order. A bound schema tree absent from that history MUST be refused — there is no chain to the current schema — rather than read as though no migration were needed.
If two data instances serialize to the same Git tree, they MUST be considered equal for purposes of data retention.
The serialization layer is intentionally separate from the repository plumbing
layer. facet-git-tree remains oid-in/oid-out; gix-store and git-store
compose those objects with schemas, documents, refs, commits, and indexes.
The unbound value operations MUST accept an explicit schema tree or a Git
revision that resolves to one. The CLI spellings are
git store value encode --schema <schema-tree-or-commit> and
git store value decode <value-tree> --schema <schema-tree-or-commit>.
git store document bind <value-tree> --schema <schema-tree-or-commit> MUST
use that same explicit schema to construct the bound document root.
These operations MUST NOT infer a schema from a kind ref, schema history,
commit trailers, or a caller-selected entity name. A bound document MUST carry
exactly schema/ and value/ at its root, with the schema subtree being the
one used to validate the value subtree.
git store schema show <kind> --at <commit> MUST address the named schema
publication commit directly. Omitting --at selects the current publication.
Historical inspection MUST report the publication commit OID and the
schema-tree OID; it MUST NOT replace the historical selection with the current
schema.
The CLI MUST provide Git-shaped inspection without a migration-specific
traversal workflow: ref list, ref resolve, object inspect, and
object tree. Ref output MUST use full ref names and object output MUST use
stable full object IDs. These commands MUST not mutate refs or objects.
Additive plumbing commands MUST support human-readable text and machine-readable
JSON and NDJSON output through --format text|json|ndjson; --json MAY be a
compact-JSON shorthand. Successful records MUST be written to stdout, and
failure diagnostics MUST be written to stderr. Machine output MUST use stable
object IDs and full ref names rather than abbreviated display values.
The current CLI exit categories are 1 for other operational errors, 2 for
invalid arguments or object shape, 3 for missing refs, objects, schemas, or
entities, 4 for compare-and-swap conflicts, and 5 for schema, value, or
document failures. A failed command MUST NOT emit a success record.
git store document publish <kind> <document-tree> --expected <absent|OID> MUST
perform a one-shot compare-and-swap. --alias <name> selects the ref to
publish at, defaulting to the document’s content-derived name. The expectation
applies to that ref, and the ref and the materialized per-kind index MUST
advance in one ref-store batch or neither may advance.
A stale expectation MUST fail with the compare-and-swap exit category and MUST NOT be retried by the CLI. Objects written before a lost publication MAY remain unreachable. Traversal, batching, retry, resume, and conflict policy belong to the calling script or program.
The EntityId of a bound document MUST be the object ID of its complete root
{schema/, value/} tree. Both the encoded value and the embedded schema
therefore contribute to identity, and an implementation MUST make that ID
available for any document it can compile.
Identity is a derived value, not a ref layout. Ref names under
refs/store/<kind>/ are application policy: the store MUST publish an entity
at whatever name the caller selects and MUST NOT attach meaning to that name.
An application that wants content-addressed storage MUST be able to select
<entity-id> as the name, and the store MUST treat that name like any other.
The per-kind index is a materialized cache of the entity refs; entity refs remain authoritative and a reader MUST be able to fall back when the index is missing, malformed, or stale.
git store entity delete <kind> <name> MUST publish a typed tombstone over
the named ref rather than prune it, so the name’s history stays reachable. The
tombstone MUST use the bound {schema/, value/} frame and carry an explicit
deleted state, kind, and the EntityId the name addressed when it was
deleted. A repeated deletion MUST be distinguishable from an absent entity.
The CLI MUST NOT provide or imply a hard-coded migrate workflow. Bash, Git,
or the calling program owns traversal, source and target schema selection,
transforms, batching, retry/resume, and policy. The CLI plumbing MAY construct
new objects and publish them under explicit CAS expectations, but base reads and
inspection MUST NOT silently rewrite stored objects.