Skip to content

Repository files navigation

OxJSON

OxJSON is a typed, schema-directed JSON codec written specifically for OxCaml. It parses, validates, and constructs application values in one pass, and encodes application values directly without creating an intermediate JSON tree.

This is a working prototype rather than a published package. Its public API is in oxjson.mli, its semantic tests are in test_oxjson.ml, and bench_oxjson.ml applies the library to the repository's 1.4 MB catalog fixture.

OxJSON's API is inspired by Jsont: both build a schema value describing a type once, then derive decoding and encoding from it rather than hand-writing each direction. See BENCHMARK.md for a direct performance comparison against Jsont.

Example

type user =
  { id : int;
    name : string;
    tags : string array }

let user_schema =
  let make id name tags = { id; name; tags } in
  Oxjson.Object.map ~kind:"user" make
  |> Oxjson.Object.mem "id" Oxjson.int ~enc:(fun user -> user.id)
  |> Oxjson.Object.mem "name" Oxjson.string ~enc:(fun user -> user.name)
  |> Oxjson.Object.mem "tags" Oxjson.(array string)
       ~enc:(fun user -> user.tags)
  |> Oxjson.Object.finish

let user_codec = Oxjson.compile user_schema

let user = Oxjson.decode_string_exn user_codec json
let json = Oxjson.encode_string_exn user_codec user

Call compile once and reuse the resulting immutable plan. Compilation lowers the schema to specialized closures, open-addressed object-name tables, dense constructor slots, required-member masks, immutable lookup arrays, and pre-escaped member prefixes such as "name":.

For a large array with a predictable size, a schema can include an allocation hint:

Oxjson.array ~initial_capacity:5000 record_schema

An exact hint avoids both growth and the final shrinking copy. It does not change validation or impose a maximum length.

OxCaml-specific implementation

The hot path deliberately uses OxCaml features that affect this workload:

  • parser and encoder state are explicitly stack allocated with stack_ and consumed through @ local functions;
  • hot loop counters use OxCaml's let mutable, avoiding heap-allocated refs;
  • an unboxed record carries an input-slice object key and an unboxed product carries number spans;
  • compiled tables and defaults use immutable arrays (iarray);
  • input references stored in local state are declared global_;
  • leaf routines have zero-allocation checks, while the whole native library is built with the zero-allocation checker enabled;
  • Flambda2, -O3, three optimization rounds, and closure unboxing are enabled for the release build.

The common string path validates UTF-8 while scanning, allocates the resulting OCaml string exactly once on decode, and copies safe string runs in bulk on encode. Object names are hashed and compared directly against the input slice, so known unescaped names are not allocated.

Semantics

The decoder is strict about JSON grammar, integer overflow, duplicate object members, required members, trailing data, Unicode escapes, UTF-8 scalar values, and maximum nesting depth. Unknown members can be rejected or recursively validated and skipped. Encoding rejects invalid UTF-8, NaN, and infinity.

Supported schemas currently include null, booleans, machine integers, finite floats, strings, nullable/options, arrays, lists, objects, conversions, defaults, unknown-member policy, and recursive schemas.

The heterogeneous constructor slots internally use Obj.t; the typed schema builder fixes each slot's type and position, and values reach a constructor only after the corresponding member codec succeeds. No unsafe value is exposed by the public interface.

Not yet implemented are streaming input/output, arbitrary-precision JSON numbers, variants/tagged unions, location-preserving errors, pretty printing, and omission of optional object members during encoding.

Build and test

The workspace contains a local OxCaml compiler at .toolchains/oxcaml. It is pinned to repository commit 8a009a85e357f42753b54bc2a85d8f7ee6b1ccd6 because OxCaml does not currently publish stable releases. That compiler reports 5.4.0+ox with Flambda2 and stack allocation enabled.

From the repository root:

make -C oxcaml-json test
make -C oxcaml-json bench

The Makefile uses the workspace-local compiler and bootstrap Dune. Override DUNE and PATH if using a separately installed OxCaml toolchain.

Benchmark results

Measured on an Apple M3 Max against the repository's 1.4 MB, 5,000-record catalog fixture. See BENCHMARK.md for exact methodology.

Decode

Implementation Mean per document Throughput Relative to OxJSON
serde_json 2.669 ms 525.3 MB/s 1.19x faster
OxJSON 3.176 ms 441.4 MB/s baseline
Bun JSON + handwritten validation 3.324 ms 421.8 MB/s 1.05x slower
Go encoding/json/v2 5.885 ms 238.2 MB/s 1.85x slower
Jsont 15.180 ms 92.4 MB/s 4.78x slower

Encode

Implementation Mean per document Throughput Relative to OxJSON
serde_json 1.110 ms 1,263.0 MB/s 1.33x faster
Bun JSON.stringify 1.406 ms 997.1 MB/s 1.05x faster
OxJSON 1.472 ms 952.4 MB/s baseline
Go encoding/json/v2 2.741 ms 511.5 MB/s 1.86x slower
Jsont 8.853 ms 158.4 MB/s 6.01x slower

License

OxJSON is available under the MIT License.

About

JSON ser/de for OxCAML

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages