From 186c74e89b1d48e6801a02e25030746da6910d47 Mon Sep 17 00:00:00 2001 From: Thanga Ganapathy Date: Wed, 4 Nov 2026 17:45:19 +0530 Subject: [PATCH 1/6] feat: add @opentf/serialize, a type-preserving JSON codec extracted from obj-diff Extract obj-diff's stringify/parse codec into a standalone, dependency-free package with an unchanged wire format, plus low-level encode/decode for embedding encoded values in other JSON shapes. Hardening for untrusted input: - "__proto__" keys are defined as own properties; prototypes never change - optional maxLength / maxDepth limits on parse (maxDepth on stringify too) - a clear INVALID_ENVELOPE error for input missing the {v, $refs} wrapper - every failure is a SerializeError with a machine-readable code - type tags and Error names can no longer reach Object.prototype members - self-referencing Map/Set input throws CIRCULAR_REF instead of overflowing - payloads are shape-checked instead of building broken values Tooling: esdev (test, build, typecheck), pnpm (TypeScript 7, Biome), tsr tasks. --- README.md | 130 +++++++++- __tests__/codec.test.ts | 174 ++++++++++++++ __tests__/lib.ts | 9 + __tests__/safety.test.ts | 199 ++++++++++++++++ biome.json | 21 ++ esdev.json | 13 + jsr.json | 5 + package.json | 57 +++++ pnpm-lock.yaml | 498 +++++++++++++++++++++++++++++++++++++++ src/decode.ts | 265 +++++++++++++++++++++ src/encode.ts | 232 ++++++++++++++++++ src/index.ts | 80 +++++++ src/types.ts | 49 ++++ src/utils.ts | 107 +++++++++ tasks.toml | 29 +++ tsconfig.json | 14 ++ 16 files changed, 1880 insertions(+), 2 deletions(-) create mode 100644 __tests__/codec.test.ts create mode 100644 __tests__/lib.ts create mode 100644 __tests__/safety.test.ts create mode 100644 biome.json create mode 100644 esdev.json create mode 100644 jsr.json create mode 100644 package.json create mode 100644 pnpm-lock.yaml create mode 100644 src/decode.ts create mode 100644 src/encode.ts create mode 100644 src/index.ts create mode 100644 src/types.ts create mode 100644 src/utils.ts create mode 100644 tasks.toml create mode 100644 tsconfig.json diff --git a/README.md b/README.md index 36b1f4b..0c898b5 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,128 @@ -# js-serialize -Type-preserving JavaScript serialization and deserialization library. +# @opentf/serialize + +Type-preserving JavaScript serialization over plain JSON. A drop-in for +`JSON.stringify` / `JSON.parse` that keeps `Date`, `Map`, `Set`, typed arrays, +`BigInt`, `Temporal`, `NaN`/`-0`, `undefined`, circular references and shared +identity — and is safe to point at untrusted input. + +Extracted from [obj-diff](https://obj-diff.opentechf.org/docs/serialization); +the wire format is unchanged, so output from either one reads in the other. + +```ts +import { parse, stringify } from "@opentf/serialize"; + +const text = stringify({ at: new Date(0), tags: new Set(["a"]), big: 10n }); +// {"v":{"at":"@1","tags":"@2","big":"@3"},"$refs":{"1":{"_t":"Date",...},...}} + +const back = parse<{ at: Date; tags: Set; big: bigint }>(text); // live values again +``` + +## Wire format + +`stringify` writes an envelope `{"v": , "$refs"?: {...}}`. Plain data +stays readable JSON; every special value becomes a token `"@n"` whose real type +sits in `$refs["n"]` as `{ "_t": , "_v": }`. Strings that start +with `@` are escaped with one more `@`. User keys named `_t` / `_v` are ordinary +data. A plain object or array reachable by more than one edge (shared or +cyclic) is hoisted into `$refs` as an `obj` / `arr` entry, so identity and +cycles are rebuilt on the far side; single-use containers stay inline. + +| Supported | | +| --- | --- | +| Primitives | `undefined`, `bigint`, `NaN`, `±Infinity`, `-0` | +| Built-ins | `Date` (incl. invalid), `RegExp`, `URL`, `Map`, `Set`, `Error` and its standard subclasses (name, message, own props), boxed `Number`/`String`/`Boolean` | +| Binary | every typed array, `ArrayBuffer`, `DataView` (byte-exact, base64) | +| Temporal | any `Temporal.*` value — decoding needs `globalThis.Temporal` | +| Graphs | cycles and shared identity among plain objects and arrays | + +Symbols, functions and class instances throw, as do cycles that run through a +`Map`, `Set` or `Error`. + +## Untrusted input + +`parse` validates everything it reads and never returns a half-built value: + +- **`__proto__` keys** become own properties (via `defineProperty`, as + `JSON.parse` does). No prototype is ever changed, and type tags such as + `"constructor"` cannot reach `Object.prototype` members. +- **Malformed input** throws a `SerializeError` with a `code` — a missing + `{v, $refs}` envelope is `INVALID_ENVELOPE`, not `undefined`. +- **Limits** are opt-in: + +```ts +parse(text, { + maxLength: 1_000_000, // UTF-16 code units, checked before JSON.parse + maxDepth: 64, // container nesting: [[1]] has depth 2 +}); +stringify(value, { maxDepth: 64 }); +``` + +Containers counted for depth: arrays, plain objects, `Map`, `Set` and `Error` +own properties — hoisted ones included. + +## Errors + +Every failure is a `SerializeError` (a subclass of `Error`). Its `code` is one of: + +| `code` | Meaning | +| --- | --- | +| `INVALID_INPUT` | `parse` was given something other than a string | +| `INVALID_JSON` | the text is not JSON (`cause` holds the `SyntaxError`) | +| `INPUT_TOO_LARGE` | longer than `maxLength` | +| `INVALID_ENVELOPE` | not `{"v": ...}`, or `$refs` is not an object | +| `MAX_DEPTH` | nests deeper than `maxDepth` | +| `MISSING_REF` | a `"@n"` token with no `$refs` entry | +| `UNKNOWN_TYPE` | an unrecognised `_t` tag | +| `INVALID_PAYLOAD` | an entry's `_v` is the wrong shape for its type | +| `CIRCULAR_REF` | a cycle through `Map`/`Set`/`Error` (either direction) | +| `UNSUPPORTED_VALUE` | symbol, function or class instance passed to `stringify` | +| `TEMPORAL_UNAVAILABLE` | Temporal data, but no `globalThis.Temporal` | +| `INVALID_OPTION` | a limit that is not a non-negative integer or `Infinity` | + +```ts +import { parse, SerializeError } from "@opentf/serialize"; + +try { + parse(body, { maxLength: 1e6, maxDepth: 64 }); +} catch (err) { + if (err instanceof SerializeError && err.code === "INPUT_TOO_LARGE") return reply(413); + throw err; +} +``` + +## Embedding: `encode` / `decode` + +To place encoded values inside your own JSON shape (this is how obj-diff +serializes each diff op), use the low-level pair. All roots share one `$refs` +table, so identity is preserved across them: + +```ts +import { decode, encode } from "@opentf/serialize"; + +const { values: [path, value], refs } = encode([op.path, op.value]); +const wire = JSON.stringify({ type: op.type, path, value, $refs: refs }); + +const raw = JSON.parse(wire); +const [p, v] = decode([raw.path, raw.value], raw.$refs, { maxDepth: 64 }); +``` + +## Development + +Tests and builds run on [esdev](https://esrun.opentechf.org/esdev), pnpm installs +the dev tools (TypeScript, Biome), and [tsr](https://github.com/Open-Tech-Foundation/tsr) +runs the tasks in `tasks.toml`: + +```sh +pnpm install +tsr test # tests against src/ +tsr build # dist/ (esm + cjs + .d.ts) +tsr test:dist # build, then the tests against dist/ +tsr type-check # esdev typecheck → the project's tsc +tsr lint # biome lint +tsr fmt # biome format --write +tsr ci # all of the above (with fmt:check) +``` + +## License + +MIT diff --git a/__tests__/codec.test.ts b/__tests__/codec.test.ts new file mode 100644 index 0000000..be3fe1c --- /dev/null +++ b/__tests__/codec.test.ts @@ -0,0 +1,174 @@ +import { describe, expect, test } from "runtime:test"; +import { parse, stringify } from "./lib"; + +const rt = (v: T): T => parse(stringify(v)); + +describe("stringify / parse round-trip", () => { + test("primitives and top-level special values", () => { + expect(rt(42)).toBe(42); + expect(rt("hello")).toBe("hello"); + expect(rt(true)).toBe(true); + expect(rt(null)).toBe(null); + expect(rt(undefined)).toBe(undefined); + expect(Number.isNaN(rt(Number.NaN))).toBe(true); + expect(Object.is(rt(-0), -0)).toBe(true); + expect(rt(Number.POSITIVE_INFINITY)).toBe(Number.POSITIVE_INFINITY); + expect(rt(Number.NEGATIVE_INFINITY)).toBe(Number.NEGATIVE_INFINITY); + expect(rt(9007199254740993n)).toBe(9007199254740993n); + }); + + test("strings that look like ref tokens survive escaping", () => { + expect(rt("@1")).toBe("@1"); + expect(rt("@42 hello")).toBe("@42 hello"); + expect(rt("@@escaped")).toBe("@@escaped"); + const back = rt({ a: "@1", b: new Date(0) }); + expect(back.a).toBe("@1"); + expect(back.b).toBeInstanceOf(Date); + }); + + test("Date (valid and invalid), RegExp, URL", () => { + expect(rt(new Date(0))).toEqual(new Date(0)); + expect(Number.isNaN(rt(new Date(Number.NaN)).getTime())).toBe(true); + expect(rt(/foo\d+/giu)).toEqual(/foo\d+/giu); + const u = rt(new URL("https://a.example/p?q=1#h")); + expect(u).toBeInstanceOf(URL); + expect(u.href).toBe("https://a.example/p?q=1#h"); + }); + + test("Map and Set, incl. object keys", () => { + expect(rt(new Map([["k", 1]]))).toEqual(new Map([["k", 1]])); + expect(rt(new Set([1, 2, 3]))).toEqual(new Set([1, 2, 3])); + const m = rt(new Map([[{ id: 1 }, new Date(0)]])); + expect([...m.keys()][0]).toEqual({ id: 1 }); + expect([...m.values()][0]).toEqual(new Date(0)); + }); + + test("typed arrays / ArrayBuffer / DataView are byte-exact", () => { + expect(rt(new Uint8Array([1, 2, 3]))).toEqual(new Uint8Array([1, 2, 3])); + const f = rt(new Float64Array([Number.NaN, Number.POSITIVE_INFINITY, -0])); + expect(f).toBeInstanceOf(Float64Array); + expect(Number.isNaN(f[0])).toBe(true); + expect(Object.is(f[2], -0)).toBe(true); + expect(rt(new BigInt64Array([9007199254740993n]))[0]).toBe(9007199254740993n); + // a view into the middle of a larger buffer encodes only its own bytes + const view = new Uint16Array(new Uint8Array([0, 0, 1, 0, 2, 0, 0, 0]).buffer, 2, 2); + expect(rt(view)).toEqual(new Uint16Array([1, 2])); + expect(new Uint8Array(rt(new Uint8Array([9, 8, 7]).buffer))).toEqual(new Uint8Array([9, 8, 7])); + const dv = rt(new DataView(new Uint8Array([1, 2, 3, 4]).buffer)); + expect(dv).toBeInstanceOf(DataView); + expect(dv.getUint8(3)).toBe(4); + }); + + test("Error and standard subclasses keep name, message and own props", () => { + const e = new TypeError("boom") as TypeError & { code?: string }; + e.code = "E_BOOM"; + const back = rt(e); + expect(back).toBeInstanceOf(TypeError); + expect(back.message).toBe("boom"); + expect(back.code).toBe("E_BOOM"); + const custom = new Error("x"); + custom.name = "MyError"; + expect(rt(custom).name).toBe("MyError"); + }); + + test("boxed primitives", () => { + const n = rt(new Number(42)); + expect(n).toBeInstanceOf(Number); + expect(n.valueOf()).toBe(42); + expect(rt(new Number(Number.NaN)).valueOf()).toBeNaN(); + expect(rt(new String("hi")).valueOf()).toBe("hi"); + expect(rt(new Boolean(true)).valueOf()).toBe(true); + }); + + test("Temporal", () => { + const d = rt(Temporal.PlainDate.from("2021-06-15")); + expect(d.toString()).toBe("2021-06-15"); + const z = rt(Temporal.ZonedDateTime.from("2021-06-15T08:30[America/New_York]")); + expect(z.toString()).toBe("2021-06-15T08:30:00-04:00[America/New_York]"); + expect(rt(Temporal.Duration.from({ hours: 2 })).toString()).toBe("PT2H"); + }); + + test("nested mix of special types", () => { + const value = { + user: { + joined: new Date("2020-01-01T00:00:00Z"), + roles: new Set(["admin"]), + prefs: new Map([["theme", "dark"]]), + scores: new Float64Array([1.5, 2.5]), + big: 123n, + seen: [new Date(0), undefined], + }, + }; + expect(rt(value)).toEqual(value); + }); + + test("user keys named _t / _v are ordinary data", () => { + expect(rt({ _t: "Date", _v: "not a date" })).toEqual({ _t: "Date", _v: "not a date" }); + }); + + test("circular references and shared identity", () => { + const self: Record = { name: "a" }; + self.self = self; + const back = rt(self); + expect(back.name).toBe("a"); + expect(back.self).toBe(back); + + const a: Record = { id: "a" }; + const b: Record = { id: "b", peer: a }; + a.peer = b; + const ra = rt(a); + expect((ra.peer as Record).peer).toBe(ra); + + const arr: unknown[] = [1]; + arr.push(arr); + const rarr = rt(arr); + expect(rarr[1]).toBe(rarr); + + const shared = { id: 7 }; + const pair = rt({ a: shared, b: shared, list: [shared] }); + expect(pair.a).toBe(pair.b); + expect(pair.list[0]).toBe(pair.a); + + // a plain object shared through a Map is still one object + const viaMap = rt({ o: shared, m: new Map([["k", shared]]) }); + expect(viaMap.m.get("k")).toBe(viaMap.o); + }); + + test("throws on symbols, functions, class instances, cycles through Map", () => { + expect(() => stringify(Symbol("s"))).toThrow(); + expect(() => stringify(() => 1)).toThrow(); + class Widget {} + expect(() => stringify(new Widget())).toThrow(); + const m = new Map(); + m.set("self", m); + expect(() => stringify(m)).toThrow(); + }); +}); + +describe("wire format (stays compatible with obj-diff 0.18)", () => { + test("single-use containers stay inline and no $refs is written", () => { + expect(stringify({ user: { name: "a", tags: ["x"] } })).toBe( + '{"v":{"user":{"name":"a","tags":["x"]}}}', + ); + }); + + test("special values become tokens into the $refs table", () => { + expect(stringify({ d: new Date(0), s: new Set([1]) })).toBe( + '{"v":{"d":"@1","s":"@2"},"$refs":{"1":{"_t":"Date","_v":"1970-01-01T00:00:00.000Z"},"2":{"_t":"Set","_v":[1]}}}', + ); + }); + + test("shared containers are hoisted as obj/arr entries", () => { + const shared = { id: 7 }; + expect(stringify({ a: shared, b: shared })).toBe( + '{"v":{"a":"@1","b":"@1"},"$refs":{"1":{"_t":"obj","_v":{"id":7}}}}', + ); + }); + + test("parses output written by obj-diff's stringify", () => { + const v = parse<{ m: Map }>( + '{"v":{"m":"@1"},"$refs":{"1":{"_t":"Map","_v":[["k","@2"]]},"2":{"_t":"bigint","_v":"5"}}}', + ); + expect(v.m.get("k")).toBe(5n); + }); +}); diff --git a/__tests__/lib.ts b/__tests__/lib.ts new file mode 100644 index 0000000..65ef6af --- /dev/null +++ b/__tests__/lib.ts @@ -0,0 +1,9 @@ +// Tests run against src by default; TEST_DIST=true runs them against the build. +import { env } from "runtime:process"; +import type * as Lib from "../src/index"; + +const lib: typeof Lib = env.TEST_DIST + ? await import("../dist/index.js") + : await import("../src/index"); + +export const { stringify, parse, encode, decode, SerializeError } = lib; diff --git a/__tests__/safety.test.ts b/__tests__/safety.test.ts new file mode 100644 index 0000000..f66b3b6 --- /dev/null +++ b/__tests__/safety.test.ts @@ -0,0 +1,199 @@ +import { describe, expect, test } from "runtime:test"; +import { decode, encode, parse, SerializeError, stringify } from "./lib"; + +/** Runs fn and returns the SerializeError it throws (fails the test otherwise). */ +function codeOf(fn: () => unknown): string { + try { + fn(); + } catch (err) { + expect(err).toBeInstanceOf(SerializeError); + return (err as InstanceType).code; + } + throw new Error("expected a SerializeError, but nothing was thrown"); +} + +describe("__proto__ keys are own properties, never prototypes", () => { + test("parse defines __proto__ as an own key in plain objects", () => { + const v = parse>('{"v":{"__proto__":{"polluted":true},"a":1}}'); + expect(Object.getPrototypeOf(v)).toBe(Object.prototype); + expect(Object.hasOwn(v, "__proto__")).toBe(true); + expect(Object.keys(v)).toEqual(["__proto__", "a"]); + expect((v as { polluted?: unknown }).polluted).toBeUndefined(); + expect(({} as { polluted?: unknown }).polluted).toBeUndefined(); + }); + + test("hoisted objects and Error own props are safe too", () => { + const hoisted = parse<{ a: object; b: object }>( + '{"v":{"a":"@1","b":"@1"},"$refs":{"1":{"_t":"obj","_v":{"__proto__":{"x":1}}}}}', + ); + expect(Object.getPrototypeOf(hoisted.a)).toBe(Object.prototype); + expect(Object.hasOwn(hoisted.a, "__proto__")).toBe(true); + + const err = parse( + '{"v":"@1","$refs":{"1":{"_t":"Error","_v":{"name":"Error","message":"m","own":{"__proto__":{"x":1}}}}}}', + ); + expect(Object.getPrototypeOf(err)).toBe(Error.prototype); + expect(Object.hasOwn(err, "__proto__")).toBe(true); + }); + + test("an own __proto__ key round-trips through stringify", () => { + const src = JSON.parse('{"__proto__":{"deep":1},"b":2}'); + const back = parse>(stringify(src)); + expect(Object.hasOwn(back, "__proto__")).toBe(true); + expect(Object.getOwnPropertyDescriptor(back, "__proto__")?.value).toEqual({ deep: 1 }); + expect(Object.getPrototypeOf(back)).toBe(Object.prototype); + }); + + test("type tags cannot reach Object.prototype members", () => { + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"constructor","_v":"x"}}}'))).toBe( + "UNKNOWN_TYPE", + ); + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"toString","_v":"x"}}}'))).toBe( + "UNKNOWN_TYPE", + ); + // an Error name that is an Object.prototype key falls back to plain Error + const e = parse( + '{"v":"@1","$refs":{"1":{"_t":"Error","_v":{"name":"constructor","message":"m","own":{}}}}}', + ); + expect(e).toBeInstanceOf(Error); + expect(e.name).toBe("constructor"); + expect( + codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"Temporal.__proto__","_v":"x"}}}')), + ).toBe("UNKNOWN_TYPE"); + }); +}); + +describe("clear errors for malformed input", () => { + test("missing {v, $refs} envelope", () => { + for (const bad of ["{}", '{"$refs":{}}', "[]", "null", "42", '"str"', '{"value":1}']) { + expect(codeOf(() => parse(bad))).toBe("INVALID_ENVELOPE"); + } + expect(() => parse("{}")).toThrow( + 'envelope written by stringify(), got an object without a "v" key', + ); + // an explicit null value is a valid envelope + expect(parse('{"v":null}')).toBe(null); + }); + + test("non-string input and invalid JSON", () => { + expect(codeOf(() => parse(undefined as unknown as string))).toBe("INVALID_INPUT"); + expect(codeOf(() => parse({ v: 1 } as unknown as string))).toBe("INVALID_INPUT"); + expect(codeOf(() => parse("{nope"))).toBe("INVALID_JSON"); + }); + + test("bad $refs table and entries", () => { + expect(codeOf(() => parse('{"v":1,"$refs":[]}'))).toBe("INVALID_ENVELOPE"); + expect(codeOf(() => parse('{"v":1,"$refs":"x"}'))).toBe("INVALID_ENVELOPE"); + expect(codeOf(() => parse('{"v":"@9"}'))).toBe("MISSING_REF"); + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":5}}'))).toBe("INVALID_PAYLOAD"); + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_v":1}}}'))).toBe("INVALID_PAYLOAD"); + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"Bogus"}}}'))).toBe("UNKNOWN_TYPE"); + }); + + test("bad payloads throw INVALID_PAYLOAD instead of building garbage", () => { + const one = (t: string, v: unknown) => + codeOf(() => parse(JSON.stringify({ v: "@1", $refs: { 1: { _t: t, _v: v } } }))); + expect(one("bigint", "abc")).toBe("INVALID_PAYLOAD"); + expect(one("bigint", 5)).toBe("INVALID_PAYLOAD"); + expect(one("num", "7")).toBe("INVALID_PAYLOAD"); + expect(one("RegExp", "abc")).toBe("INVALID_PAYLOAD"); + expect(one("RegExp", ["(", ""])).toBe("INVALID_PAYLOAD"); + expect(one("URL", "not a url")).toBe("INVALID_PAYLOAD"); + expect(one("Map", [1, 2])).toBe("INVALID_PAYLOAD"); + expect(one("Set", {})).toBe("INVALID_PAYLOAD"); + expect(one("Uint16Array", "AQID")).toBe("INVALID_PAYLOAD"); // 3 bytes: misaligned + expect(one("Uint8Array", "!!!")).toBe("INVALID_PAYLOAD"); + expect(one("Error", { name: "Error" })).toBe("INVALID_PAYLOAD"); + expect(one("Number", "x")).toBe("INVALID_PAYLOAD"); + expect(one("obj", [])).toBe("INVALID_PAYLOAD"); + expect(one("arr", {})).toBe("INVALID_PAYLOAD"); + }); + + test("a self-referencing Map/Set in the input throws instead of overflowing the stack", () => { + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"Map","_v":[["k","@1"]]}}}'))).toBe( + "CIRCULAR_REF", + ); + expect( + codeOf(() => + parse('{"v":"@1","$refs":{"1":{"_t":"Set","_v":["@2"]},"2":{"_t":"Set","_v":["@1"]}}}'), + ), + ).toBe("CIRCULAR_REF"); + }); + + test("stringify errors are SerializeErrors with codes", () => { + expect(codeOf(() => stringify(Symbol("s")))).toBe("UNSUPPORTED_VALUE"); + class Widget {} + expect(codeOf(() => stringify(new Widget()))).toBe("UNSUPPORTED_VALUE"); + const s = new Set(); + s.add(s); + expect(codeOf(() => stringify(s))).toBe("CIRCULAR_REF"); + }); +}); + +describe("optional limits", () => { + test("maxLength rejects oversized input before parsing", () => { + const text = stringify({ list: [1, 2, 3] }); + expect(parse(text, { maxLength: text.length })).toEqual({ list: [1, 2, 3] }); + expect(codeOf(() => parse(text, { maxLength: text.length - 1 }))).toBe("INPUT_TOO_LARGE"); + // checked before JSON.parse, so even invalid JSON reports the size first + expect(codeOf(() => parse("x".repeat(100), { maxLength: 10 }))).toBe("INPUT_TOO_LARGE"); + }); + + test("maxDepth counts container nesting on parse", () => { + const text = stringify({ a: [{ b: 1 }] }); // object > array > object = 3 + expect(parse(text, { maxDepth: 3 })).toEqual({ a: [{ b: 1 }] }); + expect(codeOf(() => parse(text, { maxDepth: 2 }))).toBe("MAX_DEPTH"); + expect(parse('{"v":5}', { maxDepth: 0 })).toBe(5); + expect(codeOf(() => parse('{"v":[]}', { maxDepth: 0 }))).toBe("MAX_DEPTH"); + }); + + test("maxDepth also counts Map, Set, Error and hoisted containers", () => { + const viaMap = stringify(new Map([["k", new Set([[1]])]])); // Map > Set > array = 3 + expect(parse(viaMap, { maxDepth: 3 })).toBeInstanceOf(Map); + expect(codeOf(() => parse(viaMap, { maxDepth: 2 }))).toBe("MAX_DEPTH"); + + const shared = { x: { y: 1 } }; + const hoisted = stringify([shared, shared]); // array > obj(hoisted) > obj = 3 + expect(codeOf(() => parse(hoisted, { maxDepth: 2 }))).toBe("MAX_DEPTH"); + expect(parse(hoisted, { maxDepth: 3 })[0]).toEqual(shared); + }); + + test("maxDepth stops pathologically deep input", () => { + const deep = `{"v":${"[".repeat(5000)}${"]".repeat(5000)}}`; + expect(codeOf(() => parse(deep, { maxDepth: 100 }))).toBe("MAX_DEPTH"); + }); + + test("stringify accepts maxDepth too", () => { + let deep: unknown = 1; + for (let i = 0; i < 50; i++) deep = [deep]; + expect(codeOf(() => stringify(deep, { maxDepth: 10 }))).toBe("MAX_DEPTH"); + expect(parse(stringify(deep, { maxDepth: 50 }))).toEqual(deep); + }); + + test("invalid limit options are rejected", () => { + expect(codeOf(() => parse('{"v":1}', { maxDepth: -1 }))).toBe("INVALID_OPTION"); + expect(codeOf(() => parse('{"v":1}', { maxLength: 1.5 }))).toBe("INVALID_OPTION"); + expect(codeOf(() => stringify(1, { maxDepth: Number.NaN }))).toBe("INVALID_OPTION"); + expect(parse('{"v":1}', { maxDepth: Number.POSITIVE_INFINITY })).toBe(1); + }); +}); + +describe("encode / decode (low-level, for embedding)", () => { + test("roots share one $refs table, so identity crosses roots", () => { + const key = { id: 1 }; + const { values, refs } = encode([["users", key], key]); + expect(refs).toBeDefined(); + const [path, value] = decode(JSON.parse(JSON.stringify(values)), refs) as [unknown[], unknown]; + expect(path[1]).toBe(value); + }); + + test("refs is undefined when nothing needs a ref", () => { + expect(encode([{ a: 1 }, "b"])).toEqual({ values: [{ a: 1 }, "b"], refs: undefined }); + expect(decode([{ a: 1 }, "@@x"], undefined)).toEqual([{ a: 1 }, "@x"]); + }); + + test("decode validates refs and honours maxDepth", () => { + expect(codeOf(() => decode(["@1"], []))).toBe("INVALID_ENVELOPE"); + expect(codeOf(() => decode([[[1]]], undefined, { maxDepth: 1 }))).toBe("MAX_DEPTH"); + }); +}); diff --git a/biome.json b/biome.json new file mode 100644 index 0000000..cc77256 --- /dev/null +++ b/biome.json @@ -0,0 +1,21 @@ +{ + "$schema": "https://biomejs.dev/schemas/2.6.4/schema.json", + "files": { + "includes": ["src/**/*.ts", "__tests__/**/*.ts"] + }, + "linter": { + "enabled": true, + "rules": { + "preset": "recommended", + "suspicious": { + "noExplicitAny": "error" + } + } + }, + "formatter": { + "enabled": true, + "indentStyle": "space", + "indentWidth": 2, + "lineWidth": 100 + } +} diff --git a/esdev.json b/esdev.json new file mode 100644 index 0000000..b95cd68 --- /dev/null +++ b/esdev.json @@ -0,0 +1,13 @@ +{ + "build": { + "targets": { + "library": { + "entry": "src", + "lib": true, + "outdir": "dist", + "format": ["esm", "cjs"], + "assets": ["README.md", "LICENSE"] + } + } + } +} diff --git a/jsr.json b/jsr.json new file mode 100644 index 0000000..837c53a --- /dev/null +++ b/jsr.json @@ -0,0 +1,5 @@ +{ + "name": "@opentf/serialize", + "version": "0.1.0", + "exports": "./src/index.ts" +} diff --git a/package.json b/package.json new file mode 100644 index 0000000..41afa97 --- /dev/null +++ b/package.json @@ -0,0 +1,57 @@ +{ + "name": "@opentf/serialize", + "description": "Type-preserving JavaScript serialization: Date, Map, Set, typed arrays, BigInt, Temporal, cycles and more, over plain JSON.", + "keywords": [ + "serialize", + "deserialize", + "json", + "stringify", + "parse", + "superjson", + "bigint", + "map", + "set", + "temporal", + "circular" + ], + "version": "0.1.0", + "author": { + "name": "Thanga Ganapathy", + "email": "ganapathy888@gmail.com", + "url": "https://thanga-ganapathy.github.io" + }, + "license": "MIT", + "repository": { + "type": "git", + "url": "https://github.com/Open-Tech-Foundation/js-serialize.git" + }, + "type": "module", + "main": "dist/index.cjs", + "module": "dist/index.js", + "types": "dist/index.d.ts", + "exports": { + ".": { + "import": { + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + }, + "require": { + "types": "./dist/index.d.cts", + "default": "./dist/index.cjs" + } + } + }, + "sideEffects": false, + "files": [ + "dist" + ], + "publishConfig": { + "access": "public", + "provenance": true + }, + "packageManager": "pnpm@12.5.1", + "devDependencies": { + "@biomejs/biome": "^2.6.4", + "typescript": "^7.0.2" + } +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml new file mode 100644 index 0000000..421fd4f --- /dev/null +++ b/pnpm-lock.yaml @@ -0,0 +1,498 @@ +--- +lockfileVersion: '9.0' + +importers: + + .: + configDependencies: {} + packageManagerDependencies: + pnpm: + specifier: 12.5.1 + version: 12.5.1 + +packages: + + '@pnpm/exe.android-arm64@12.5.1': + resolution: {integrity: sha512-6LGBmQuDzUmgpHSuy+haG0jrmkwFNa4EuShDZqK3cIBv4Wfb75CzjJmowkzr+iJg40vnBKMQ62t4Ko6mwsId7A==} + cpu: [arm64] + os: [android] + + '@pnpm/exe.android-x64@12.5.1': + resolution: {integrity: sha512-091oCltP3Rx89yxL1Wh5rXkUXEyFzyxIzSHKRFDyDFlQkb8Ox8FaDcm+M8PNkMyqnLA+qeGOLzY7Rjj9izl1Ww==} + cpu: [x64] + os: [android] + + '@pnpm/exe.darwin-arm64@12.5.1': + resolution: {integrity: sha512-r1vEMQPVjRexZMyL2HonxJoo3+nuB/+G+tg+f2PK6faRdXX1TDwevKXOxJAfqlPInTy6W/pm912+s+m0NcR4BQ==} + cpu: [arm64] + os: [darwin] + + '@pnpm/exe.darwin-x64@12.5.1': + resolution: {integrity: sha512-YHwI1g2B+Y2cAEylOR31r781GWijdJL7Gy7TEn/ZpZ5v/Cuwp5/Pz0EFCgo0M2MKLafAbL/laUMGvOpAdnerAw==} + cpu: [x64] + os: [darwin] + + '@pnpm/exe.freebsd-x64@12.5.1': + resolution: {integrity: sha512-sFUMHiYFs6q5syFwgaVppzhQSmqnXoJIz/YwTXIOXTDE76ndIpTRpDGAnBCBKjsaEBnqjJA/2l6hpJcNDNoMiA==} + cpu: [x64] + os: [freebsd] + + '@pnpm/exe.linux-arm64-musl@12.5.1': + resolution: {integrity: sha512-zupuxYZBuBEGjUz805/4/Asq01UNziFADANt8yp+LdUjU9bbMa3xY78Bx1PsGDqBy7aiad4sdUMK8ZxH4n4gmQ==} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@pnpm/exe.linux-arm64@12.5.1': + resolution: {integrity: sha512-6hbMWW2/eQo1b5wvQPe/ndLHVTH88Rav6N+ffD4/h6IuFMjWCgasFYYsFv0GrvpqREAx3SwmZSvli+0NFXqbxA==} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@pnpm/exe.linux-ppc64@12.5.1': + resolution: {integrity: sha512-Gg3maDzaE/ZFMV9HIx/BwMN0Y6QbEs13URfxmiYHtKjjLAs5wEQZRMUP87YNLYO7cNaEfbM8GRbS9vPosyD9Gw==} + cpu: [ppc64] + os: [linux] + libc: [glibc] + + '@pnpm/exe.linux-riscv64@12.5.1': + resolution: {integrity: sha512-6fgWxXYMBMsWQ8DAiRevscX4QZQjwh+FsyCj8ZOk8pMXvqeq84MqBLCmS6KckvnbdqJ+HlSVQ7fs/kd0ldwfhA==} + cpu: [riscv64] + os: [linux] + libc: [glibc] + + '@pnpm/exe.linux-s390x@12.5.1': + resolution: {integrity: sha512-nzMZruRvxDhEfYUhzu+7EmM39+82Dgagc/6hPFPqGqzXNGcIG0anuxbUxfeNtI64SHt91KuuYUmCB/inlM+Pag==} + cpu: [s390x] + os: [linux] + libc: [glibc] + + '@pnpm/exe.linux-x64-musl@12.5.1': + resolution: {integrity: sha512-kHCbegdTNHpzmPtUD4S5fqkVkqgdQR47spF9p2VPP3dIIkZl6Ytk8w0vhlAsAGgge8FmucfwX+vvPKgm0NtOCQ==} + cpu: [x64] + os: [linux] + libc: [musl] + + '@pnpm/exe.linux-x64@12.5.1': + resolution: {integrity: sha512-3PkUBYo5z4dgtlnTNIFj7QGpcDUAuqXz9WGVigPDCeccEnhGiRkWmA5102TmYJHtwJP3LfmE+ZF9PGeWhn8p9Q==} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@pnpm/exe.win32-arm64@12.5.1': + resolution: {integrity: sha512-49NvjZIXHly56/PLoEDD7VphbO4N8dwMXoApElDFCpVGDNGeKlwlib1tXtnpTF6i2/XIsAj7GgatHB5KHXgBnA==} + cpu: [arm64] + os: [win32] + + '@pnpm/exe.win32-x64@12.5.1': + resolution: {integrity: sha512-ePUmQFPAPJFNMwg1lelDx2PdjHqYRiF0yfg5e6qeDbPNOu97QNo557Ru/SxmCrmt8ydQWNuZXWpdEp019md5vw==} + cpu: [x64] + os: [win32] + + pnpm@12.5.1: + resolution: {integrity: sha512-4/MFvHhKK8ifWtO2E4iJRw+ujSr182thIW7JHCw9ZAiXdfneOKrDMQROpA8kXLDVZmOS399lgk4ZB+9qLGLeXw==} + engines: {node: '>=18.*'} + hasBin: true + +snapshots: + + '@pnpm/exe.android-arm64@12.5.1': + optional: true + + '@pnpm/exe.android-x64@12.5.1': + optional: true + + '@pnpm/exe.darwin-arm64@12.5.1': + optional: true + + '@pnpm/exe.darwin-x64@12.5.1': + optional: true + + '@pnpm/exe.freebsd-x64@12.5.1': + optional: true + + '@pnpm/exe.linux-arm64-musl@12.5.1': + optional: true + + '@pnpm/exe.linux-arm64@12.5.1': + optional: true + + '@pnpm/exe.linux-ppc64@12.5.1': + optional: true + + '@pnpm/exe.linux-riscv64@12.5.1': + optional: true + + '@pnpm/exe.linux-s390x@12.5.1': + optional: true + + '@pnpm/exe.linux-x64-musl@12.5.1': + optional: true + + '@pnpm/exe.linux-x64@12.5.1': + optional: true + + '@pnpm/exe.win32-arm64@12.5.1': + optional: true + + '@pnpm/exe.win32-x64@12.5.1': + optional: true + + pnpm@12.5.1: + optionalDependencies: + '@pnpm/exe.android-arm64': 12.5.1 + '@pnpm/exe.android-x64': 12.5.1 + '@pnpm/exe.darwin-arm64': 12.5.1 + '@pnpm/exe.darwin-x64': 12.5.1 + '@pnpm/exe.freebsd-x64': 12.5.1 + '@pnpm/exe.linux-arm64': 12.5.1 + '@pnpm/exe.linux-arm64-musl': 12.5.1 + '@pnpm/exe.linux-ppc64': 12.5.1 + '@pnpm/exe.linux-riscv64': 12.5.1 + '@pnpm/exe.linux-s390x': 12.5.1 + '@pnpm/exe.linux-x64': 12.5.1 + '@pnpm/exe.linux-x64-musl': 12.5.1 + '@pnpm/exe.win32-arm64': 12.5.1 + '@pnpm/exe.win32-x64': 12.5.1 + +--- +lockfileVersion: '9.0' + +settings: + autoInstallPeers: true + excludeLinksFromLockfile: false + +importers: + + .: + devDependencies: + '@biomejs/biome': + specifier: ^2.6.4 + version: 2.6.4 + typescript: + specifier: ^7.0.2 + version: 7.0.2 + +packages: + + '@biomejs/biome@2.6.4': + resolution: {integrity: sha512-8sKKwaNW4ot22awZ3lBjwWLHnUFXgXnrBPYS2gNlBAXRbDC++7kqjK2H7dR/FgCDtPvO1xfKpgEhJkGxAKYkxA==} + engines: {node: '>=14.21.3'} + hasBin: true + + '@biomejs/cli-android-arm64@2.6.4': + resolution: {integrity: sha512-jwrwigP4tBcHxzuV8izioX/BOUhJL/4e2rNSOE5nCjBkVC018LQAQUGiH+CNkFYvLtQ1dcZo5/xXERmaAGc7JQ==} + engines: {node: '>=14.21.3'} + cpu: [arm64] + os: [android] + + '@biomejs/cli-android-x64@2.6.4': + resolution: {integrity: sha512-HdGInPJ4nZFwHxzvEtcp2nmSBtUyJrurmtIK43upFckuS3+ZqVM05HtsxQWnqGsuy/pxckIAjyKWlEKRlBXUyA==} + engines: {node: '>=14.21.3'} + cpu: [x64] + os: [android] + + '@biomejs/cli-darwin-arm64@2.6.4': + resolution: {integrity: sha512-0VvcUz46wbg59Q4TZ59KkIjDJrxEyu4Eoe79+F4SChxoWe+W7AEyIp6F5r72C9+p6MsB6dIovnZt1D30lGM3XA==} + engines: {node: '>=14.21.3'} + cpu: [arm64] + os: [darwin] + + '@biomejs/cli-darwin-x64@2.6.4': + resolution: {integrity: sha512-fxmvkF81srlS3DiMyfm+aG3J0eG+6/BjaxRyjy57KbLSafnq0PO+hHfVWmtH2ty0S7nm6yDXBRwySfFeaRJKRQ==} + engines: {node: '>=14.21.3'} + cpu: [x64] + os: [darwin] + + '@biomejs/cli-linux-arm64-musl@2.6.4': + resolution: {integrity: sha512-31ZaH/MHqPX+O+evRwRrCHQ8W5/9adY+/eMytGGWZnRx1T41QsWIF09+/WujynLBhWZUxNGS5CtsWLwgOFTAyg==} + engines: {node: '>=14.21.3'} + cpu: [arm64] + os: [linux] + libc: [musl] + + '@biomejs/cli-linux-arm64@2.6.4': + resolution: {integrity: sha512-6+bK1Y/evmIUiJr7auertuMbiaYVicJhqA1bZyL47lCMWugv8l0gFQCqvR/gN1vZMHYm+HwoTKgeMxdNFCoQTg==} + engines: {node: '>=14.21.3'} + cpu: [arm64] + os: [linux] + libc: [glibc] + + '@biomejs/cli-linux-x64-musl@2.6.4': + resolution: {integrity: sha512-bomSa2XdQAGGj6Cwdm4M13qE+lN9+AOlW8KtI4hdyVyRRF/Z71qPAi95T4wtdZK0QpBbkTW8hgbZZCwPi4xzIw==} + engines: {node: '>=14.21.3'} + cpu: [x64] + os: [linux] + libc: [musl] + + '@biomejs/cli-linux-x64@2.6.4': + resolution: {integrity: sha512-tbFZMdL6dP/ws4edXxYG9WhvaVa1ZDzL4G5DzahnWW7Psib9H+VTdyU4XXVxpJk5j/nYZroSbvSNyYlpW3yCIw==} + engines: {node: '>=14.21.3'} + cpu: [x64] + os: [linux] + libc: [glibc] + + '@biomejs/cli-win32-arm64@2.6.4': + resolution: {integrity: sha512-eCsqt8Ma3eVap+oyJCDnJZH60SIU464B66n3y38EUTJdM1T/HfZol1FJC9vb+sHToX0ix8XzqPcwJnOzyAJz0g==} + engines: {node: '>=14.21.3'} + cpu: [arm64] + os: [win32] + + '@biomejs/cli-win32-x64@2.6.4': + resolution: {integrity: sha512-pYsMy5AThiHFOZAsngg8qN4PurM6nInKBhVX7sbj3eNpRQqo1Hb0MgCwPL7VqVMrmcEEMj9Ks3qbkallR65e8g==} + engines: {node: '>=14.21.3'} + cpu: [x64] + os: [win32] + + '@typescript/typescript-aix-ppc64@7.0.2': + resolution: {integrity: sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==} + engines: {node: '>=16.20.0'} + cpu: [ppc64] + os: [aix] + + '@typescript/typescript-darwin-arm64@7.0.2': + resolution: {integrity: sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==} + engines: {node: '>=16.20.0'} + cpu: [arm64] + os: [darwin] + + '@typescript/typescript-darwin-x64@7.0.2': + resolution: {integrity: sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [darwin] + + '@typescript/typescript-freebsd-arm64@7.0.2': + resolution: {integrity: sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==} + engines: {node: '>=16.20.0'} + cpu: [arm64] + os: [freebsd] + + '@typescript/typescript-freebsd-x64@7.0.2': + resolution: {integrity: sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [freebsd] + + '@typescript/typescript-linux-arm64@7.0.2': + resolution: {integrity: sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==} + engines: {node: '>=16.20.0'} + cpu: [arm64] + os: [linux] + + '@typescript/typescript-linux-arm@7.0.2': + resolution: {integrity: sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==} + engines: {node: '>=16.20.0'} + cpu: [arm] + os: [linux] + + '@typescript/typescript-linux-loong64@7.0.2': + resolution: {integrity: sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==} + engines: {node: '>=16.20.0'} + cpu: [loong64] + os: [linux] + + '@typescript/typescript-linux-mips64el@7.0.2': + resolution: {integrity: sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==} + engines: {node: '>=16.20.0'} + cpu: [mips64el] + os: [linux] + + '@typescript/typescript-linux-ppc64@7.0.2': + resolution: {integrity: sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==} + engines: {node: '>=16.20.0'} + cpu: [ppc64] + os: [linux] + + '@typescript/typescript-linux-riscv64@7.0.2': + resolution: {integrity: sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==} + engines: {node: '>=16.20.0'} + cpu: [riscv64] + os: [linux] + + '@typescript/typescript-linux-s390x@7.0.2': + resolution: {integrity: sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==} + engines: {node: '>=16.20.0'} + cpu: [s390x] + os: [linux] + + '@typescript/typescript-linux-x64@7.0.2': + resolution: {integrity: sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [linux] + + '@typescript/typescript-netbsd-arm64@7.0.2': + resolution: {integrity: sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==} + engines: {node: '>=16.20.0'} + cpu: [arm64] + os: [netbsd] + + '@typescript/typescript-netbsd-x64@7.0.2': + resolution: {integrity: sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [netbsd] + + '@typescript/typescript-openbsd-arm64@7.0.2': + resolution: {integrity: sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==} + engines: {node: '>=16.20.0'} + cpu: [arm64] + os: [openbsd] + + '@typescript/typescript-openbsd-x64@7.0.2': + resolution: {integrity: sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [openbsd] + + '@typescript/typescript-sunos-x64@7.0.2': + resolution: {integrity: sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [sunos] + + '@typescript/typescript-win32-arm64@7.0.2': + resolution: {integrity: sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==} + engines: {node: '>=16.20.0'} + cpu: [arm64] + os: [win32] + + '@typescript/typescript-win32-x64@7.0.2': + resolution: {integrity: sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==} + engines: {node: '>=16.20.0'} + cpu: [x64] + os: [win32] + + typescript@7.0.2: + resolution: {integrity: sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==} + engines: {node: '>=16.20.0'} + hasBin: true + +snapshots: + + '@biomejs/biome@2.6.4': + optionalDependencies: + '@biomejs/cli-android-arm64': 2.6.4 + '@biomejs/cli-android-x64': 2.6.4 + '@biomejs/cli-darwin-arm64': 2.6.4 + '@biomejs/cli-darwin-x64': 2.6.4 + '@biomejs/cli-linux-arm64': 2.6.4 + '@biomejs/cli-linux-arm64-musl': 2.6.4 + '@biomejs/cli-linux-x64': 2.6.4 + '@biomejs/cli-linux-x64-musl': 2.6.4 + '@biomejs/cli-win32-arm64': 2.6.4 + '@biomejs/cli-win32-x64': 2.6.4 + + '@biomejs/cli-android-arm64@2.6.4': + optional: true + + '@biomejs/cli-android-x64@2.6.4': + optional: true + + '@biomejs/cli-darwin-arm64@2.6.4': + optional: true + + '@biomejs/cli-darwin-x64@2.6.4': + optional: true + + '@biomejs/cli-linux-arm64-musl@2.6.4': + optional: true + + '@biomejs/cli-linux-arm64@2.6.4': + optional: true + + '@biomejs/cli-linux-x64-musl@2.6.4': + optional: true + + '@biomejs/cli-linux-x64@2.6.4': + optional: true + + '@biomejs/cli-win32-arm64@2.6.4': + optional: true + + '@biomejs/cli-win32-x64@2.6.4': + optional: true + + '@typescript/typescript-aix-ppc64@7.0.2': + optional: true + + '@typescript/typescript-darwin-arm64@7.0.2': + optional: true + + '@typescript/typescript-darwin-x64@7.0.2': + optional: true + + '@typescript/typescript-freebsd-arm64@7.0.2': + optional: true + + '@typescript/typescript-freebsd-x64@7.0.2': + optional: true + + '@typescript/typescript-linux-arm64@7.0.2': + optional: true + + '@typescript/typescript-linux-arm@7.0.2': + optional: true + + '@typescript/typescript-linux-loong64@7.0.2': + optional: true + + '@typescript/typescript-linux-mips64el@7.0.2': + optional: true + + '@typescript/typescript-linux-ppc64@7.0.2': + optional: true + + '@typescript/typescript-linux-riscv64@7.0.2': + optional: true + + '@typescript/typescript-linux-s390x@7.0.2': + optional: true + + '@typescript/typescript-linux-x64@7.0.2': + optional: true + + '@typescript/typescript-netbsd-arm64@7.0.2': + optional: true + + '@typescript/typescript-netbsd-x64@7.0.2': + optional: true + + '@typescript/typescript-openbsd-arm64@7.0.2': + optional: true + + '@typescript/typescript-openbsd-x64@7.0.2': + optional: true + + '@typescript/typescript-sunos-x64@7.0.2': + optional: true + + '@typescript/typescript-win32-arm64@7.0.2': + optional: true + + '@typescript/typescript-win32-x64@7.0.2': + optional: true + + typescript@7.0.2: + optionalDependencies: + '@typescript/typescript-aix-ppc64': 7.0.2 + '@typescript/typescript-darwin-arm64': 7.0.2 + '@typescript/typescript-darwin-x64': 7.0.2 + '@typescript/typescript-freebsd-arm64': 7.0.2 + '@typescript/typescript-freebsd-x64': 7.0.2 + '@typescript/typescript-linux-arm': 7.0.2 + '@typescript/typescript-linux-arm64': 7.0.2 + '@typescript/typescript-linux-loong64': 7.0.2 + '@typescript/typescript-linux-mips64el': 7.0.2 + '@typescript/typescript-linux-ppc64': 7.0.2 + '@typescript/typescript-linux-riscv64': 7.0.2 + '@typescript/typescript-linux-s390x': 7.0.2 + '@typescript/typescript-linux-x64': 7.0.2 + '@typescript/typescript-netbsd-arm64': 7.0.2 + '@typescript/typescript-netbsd-x64': 7.0.2 + '@typescript/typescript-openbsd-arm64': 7.0.2 + '@typescript/typescript-openbsd-x64': 7.0.2 + '@typescript/typescript-sunos-x64': 7.0.2 + '@typescript/typescript-win32-arm64': 7.0.2 + '@typescript/typescript-win32-x64': 7.0.2 diff --git a/src/decode.ts b/src/decode.ts new file mode 100644 index 0000000..3229229 --- /dev/null +++ b/src/decode.ts @@ -0,0 +1,265 @@ +import type { ParseOptions } from "./types"; +import { + b64ToBytes, + ERR_CTORS, + hasOwn, + limit, + REF_TOKEN, + SerializeError, + setOwn, + TA_CTORS, +} from "./utils"; + +type JsonObject = Record; + +export const isJsonObject = (x: unknown): x is JsonObject => + x !== null && typeof x === "object" && !Array.isArray(x); + +/** Describes a value's JSON shape in error messages, without echoing its contents. */ +export function shapeOf(x: unknown): string { + if (x === null) return "null"; + if (Array.isArray(x)) return "an array"; + if (typeof x === "object") return "an object"; + return `a ${typeof x}`; +} + +/** Internal decoder state; see `createDecoder`. */ +export interface Decoder { + node(x: unknown, depth?: number): unknown; +} + +/** + * Builds a decoder over one `$refs` table. The input is treated as untrusted: + * every entry and payload is shape-checked, and any malformed part throws a + * `SerializeError` instead of yielding `undefined` or a half-built value. + */ +export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Decoder { + const maxDepth = limit("maxDepth", options.maxDepth); + if (rawRefs !== undefined && !isJsonObject(rawRefs)) { + throw new SerializeError( + "INVALID_ENVELOPE", + `"$refs" must be an object, got ${shapeOf(rawRefs)}`, + ); + } + const refs: JsonObject = rawRefs ?? {}; + const cache = new Map(); + // Keys whose (non-plain) value is being built; meeting one again is a cycle + // the format cannot express (e.g. a Map containing itself), not a stack overflow. + const building = new Set(); + + const enter = (depth: number): number => { + if (depth >= maxDepth) { + throw new SerializeError("MAX_DEPTH", `input nests deeper than maxDepth (${maxDepth})`); + } + return depth + 1; + }; + + const check = (ok: boolean, t: string, expected: string): void => { + if (!ok) throw new SerializeError("INVALID_PAYLOAD", `"${t}" payload must be ${expected}`); + }; + + const build = (t: string, v: unknown, depth: number): unknown => { + switch (t) { + case "undef": + return undefined; + case "bigint": + check(typeof v === "string", t, "a string"); + return BigInt(v as string); + case "num": + switch (v) { + case "Infinity": + return Number.POSITIVE_INFINITY; + case "-Infinity": + return Number.NEGATIVE_INFINITY; + case "NaN": + return Number.NaN; + case "-0": + return -0; + default: + return check(false, t, '"Infinity", "-Infinity", "NaN" or "-0"'); + } + case "Date": + check(typeof v === "string", t, "a string"); + return new Date(v === "NaN" ? Number.NaN : (v as string)); + case "RegExp": { + check( + Array.isArray(v) && + v.length === 2 && + typeof v[0] === "string" && + typeof v[1] === "string", + t, + "[source, flags]", + ); + const [source, flags] = v as [string, string]; + return new RegExp(source, flags); + } + case "URL": + check(typeof v === "string", t, "a string"); + return new URL(v as string); + case "Map": { + check( + Array.isArray(v) && v.every((p) => Array.isArray(p) && p.length === 2), + t, + "an array of [key, value] pairs", + ); + const d = enter(depth); + return new Map((v as [unknown, unknown][]).map(([k, val]) => [node(k, d), node(val, d)])); + } + case "Set": { + check(Array.isArray(v), t, "an array"); + const d = enter(depth); + return new Set((v as unknown[]).map((e) => node(e, d))); + } + case "ArrayBuffer": + check(typeof v === "string", t, "a base64 string"); + return b64ToBytes(v as string).buffer; + case "DataView": + check(typeof v === "string", t, "a base64 string"); + return new DataView(b64ToBytes(v as string).buffer); + case "Error": + return buildError(v, depth); + case "Number": { + const n = node(v, depth); + check(typeof n === "number", t, "a number"); + return new Number(n); + } + case "String": + check(typeof v === "string", t, "a string"); + return new String(v); + case "Boolean": + check(typeof v === "boolean", t, "a boolean"); + return new Boolean(v); + default: { + const TA = TA_CTORS[t]; + if (TA) { + check(typeof v === "string", t, "a base64 string"); + return new TA(b64ToBytes(v as string).buffer); + } + if (t.startsWith("Temporal.")) return buildTemporal(t, v); + throw new SerializeError("UNKNOWN_TYPE", `unknown type tag "${t}"`); + } + } + }; + + const buildError = (v: unknown, depth: number): Error => { + check( + isJsonObject(v) && + typeof v.name === "string" && + typeof v.message === "string" && + isJsonObject(v.own), + "Error", + "{ name: string, message: string, own: object }", + ); + const { name, message, own } = v as { name: string; message: string; own: JsonObject }; + const Ctor = ERR_CTORS[name] ?? Error; + const e = new Ctor(message); + if (e.name !== name) e.name = name; + const d = enter(depth); + for (const k of Object.keys(own)) setOwn(e, k, node(own[k], d)); + return e; + }; + + const buildTemporal = (t: string, v: unknown): unknown => { + const T = (globalThis as { Temporal?: Record }).Temporal; + if (!T) { + throw new SerializeError( + "TEMPORAL_UNAVAILABLE", + `Temporal is not available in this runtime (needed for "${t}")`, + ); + } + const name = t.slice("Temporal.".length); + const Kind = hasOwn(T, name) ? T[name] : undefined; + if (!Kind || typeof Kind.from !== "function") { + throw new SerializeError("UNKNOWN_TYPE", `unknown type tag "${t}"`); + } + check(typeof v === "string", t, "a string"); + return (Kind.from as (s: string) => unknown)(v as string); + }; + + const resolve = (key: string, depth: number): unknown => { + if (cache.has(key)) return cache.get(key); + if (!hasOwn(refs, key)) throw new SerializeError("MISSING_REF", `missing $ref "@${key}"`); + const entry = refs[key]; + if (!isJsonObject(entry) || typeof entry._t !== "string") { + throw new SerializeError( + "INVALID_PAYLOAD", + `$refs entry "@${key}" must be an object with a string "_t"`, + ); + } + const t = entry._t; + const v = entry._v; + + // Hoisted plain containers: register the empty shell before populating so a + // self- or cross-reference back into this key resolves to the same object. + if (t === "arr") { + check(Array.isArray(v), t, "an array"); + const d = enter(depth); + const arr: unknown[] = []; + cache.set(key, arr); + const src = v as unknown[]; + for (let i = 0; i < src.length; i++) arr[i] = node(src[i], d); + return arr; + } + if (t === "obj") { + check(isJsonObject(v), t, "an object"); + const d = enter(depth); + const obj: JsonObject = {}; + cache.set(key, obj); + const src = v as JsonObject; + for (const k of Object.keys(src)) setOwn(obj, k, node(src[k], d)); + return obj; + } + + if (building.has(key)) { + throw new SerializeError("CIRCULAR_REF", `$ref "@${key}" (${t}) refers back to itself`); + } + building.add(key); + let built: unknown; + try { + built = build(t, v, depth); + } catch (err) { + if (err instanceof SerializeError) throw err; + // Native constructors reject bad payloads with their own errors + // (BigInt("x"), new URL("x"), atob("!"), misaligned typed array, ...). + throw new SerializeError( + "INVALID_PAYLOAD", + `invalid "${t}" payload: ${(err as Error)?.message ?? String(err)}`, + { cause: err }, + ); + } finally { + building.delete(key); + } + cache.set(key, built); + return built; + }; + + const node = (x: unknown, depth = 0): unknown => { + if (typeof x === "string") { + if (REF_TOKEN.test(x)) return resolve(x.slice(1), depth); + return x.startsWith("@") ? x.slice(1) : x; // unescape + } + if (x === null || typeof x !== "object") return x; + const d = enter(depth); + if (Array.isArray(x)) return x.map((e) => node(e, d)); + const o = x as JsonObject; + const out: JsonObject = {}; + for (const k of Object.keys(o)) setOwn(out, k, node(o[k], d)); + return out; + }; + + return { node }; +} + +/** + * Low-level counterpart of `encode`: decodes JSON-safe `values` that share one + * `refs` table (pass `undefined` when there is none). `options.maxLength` does + * not apply here — there is no input text. + */ +export function decode( + values: readonly unknown[], + refs: unknown, + options?: ParseOptions, +): unknown[] { + const dec = createDecoder(refs, options); + return values.map((v) => dec.node(v)); +} diff --git a/src/encode.ts b/src/encode.ts new file mode 100644 index 0000000..e473b5b --- /dev/null +++ b/src/encode.ts @@ -0,0 +1,232 @@ +import type { Refs, StringifyOptions } from "./types"; +import { + bytesToB64, + isPlainObject, + isTypedArray, + limit, + SerializeError, + setOwn, + temporalName, +} from "./utils"; + +function isPlainContainer(x: object): boolean { + return Array.isArray(x) || isPlainObject(x); +} + +/** Internal encoder state; see `createEncoder`. */ +export interface Encoder { + mark(x: unknown, depth?: number): void; + node(x: unknown, depth?: number): unknown; + refs: Refs; + hasRefs(): boolean; +} + +/** + * Builds an encoder whose `$refs` table and identity map are shared by every + * root it encodes. Call `mark` on every root first, then `node` on each. + */ +export function createEncoder(options: StringifyOptions = {}): Encoder { + const maxDepth = limit("maxDepth", options.maxDepth); + const refs: Refs = {}; + const seen = new Set(); + let count = 0; + + const enter = (depth: number): number => { + if (depth >= maxDepth) { + throw new SerializeError("MAX_DEPTH", `value nests deeper than maxDepth (${maxDepth})`); + } + return depth + 1; + }; + + const ref = (_t: string, _v?: unknown): string => { + const key = String(++count); + refs[key] = _v === undefined ? { _t } : { _t, _v }; + return `@${key}`; + }; + + // Pass 1: find plain containers reachable by more than one edge (shared or + // cyclic). Only those are hoisted into $refs; single-use containers stay + // inline so the common case keeps its readable JSON shape. + const walked = new Set(); + const shared = new Set(); + const mark = (x: unknown, depth = 0): void => { + if (x === null || typeof x !== "object") return; + const obj = x as object; + if (isPlainContainer(obj)) { + if (walked.has(obj)) { + shared.add(obj); + return; + } + walked.add(obj); + const d = enter(depth); + if (Array.isArray(obj)) { + for (const e of obj) mark(e, d); + } else { + const o = obj as Record; + for (const k of Object.keys(o)) mark(o[k], d); + } + return; + } + // Special containers: guard cycles while scanning for nested shared values. + if (walked.has(obj)) return; + walked.add(obj); + if (obj instanceof Map) { + const d = enter(depth); + for (const [k, v] of obj) { + mark(k, d); + mark(v, d); + } + } else if (obj instanceof Set) { + const d = enter(depth); + for (const v of obj) mark(v, d); + } else if (obj instanceof Error) { + const d = enter(depth); + const s = obj as unknown as Record; + for (const k of Object.keys(obj)) mark(s[k], d); + } + }; + + // Pass 2: identity table for hoisted plain containers. + const idOf = new Map(); + + const encodeContainer = (obj: object, depth: number): unknown => { + const d = enter(depth); + if (Array.isArray(obj)) return obj.map((e) => node(e, d)); + const o = obj as Record; + const out: Record = {}; + for (const k of Object.keys(o)) setOwn(out, k, node(o[k], d)); + return out; + }; + + // Returns a JSON-safe node: a primitive, array, plain object, or a "@n" token. + const node = (x: unknown, depth = 0): unknown => { + if (x === null) return null; + + const type = typeof x; + if (type === "string") { + const s = x as string; + // Escape a real string that could be mistaken for a ref token or an escape. + return s.startsWith("@") ? `@${s}` : s; + } + if (type === "boolean") return x; + if (type === "number") { + const num = x as number; + // -0, NaN and ±Infinity are not representable in JSON, so they go to a ref. + if (Number.isFinite(num) && !Object.is(num, -0)) return num; + return ref( + "num", + num === Number.POSITIVE_INFINITY + ? "Infinity" + : num === Number.NEGATIVE_INFINITY + ? "-Infinity" + : Number.isNaN(num) + ? "NaN" + : "-0", + ); + } + if (x === undefined) return ref("undef"); + if (type === "bigint") return ref("bigint", (x as bigint).toString()); + if (type === "symbol" || type === "function") { + throw new SerializeError("UNSUPPORTED_VALUE", `${type} values are not serializable`); + } + + const obj = x as object; + + // Plain containers: hoist into $refs when shared/cyclic (referenced by + // token so identity and cycles survive), otherwise inline as readable JSON. + if (isPlainContainer(obj)) { + if (shared.has(obj)) { + const existing = idOf.get(obj); + if (existing !== undefined) return `@${existing}`; + const key = String(++count); + idOf.set(obj, key); + const encoded = encodeContainer(obj, depth); + refs[key] = { _t: Array.isArray(obj) ? "arr" : "obj", _v: encoded }; + return `@${key}`; + } + return encodeContainer(obj, depth); + } + + // Special containers can still form cycles (a Map holding itself); guard + // those so stringify throws instead of recursing forever. + if (seen.has(obj)) { + throw new SerializeError( + "CIRCULAR_REF", + "circular references through Map, Set or Error are not serializable", + ); + } + seen.add(obj); + try { + if (obj instanceof Date) { + return ref("Date", Number.isNaN(obj.getTime()) ? "NaN" : obj.toISOString()); + } + if (obj instanceof RegExp) return ref("RegExp", [obj.source, obj.flags]); + if (typeof URL !== "undefined" && obj instanceof URL) return ref("URL", obj.href); + if (obj instanceof Map) { + const d = enter(depth); + return ref( + "Map", + [...obj].map(([k, v]) => [node(k, d), node(v, d)]), + ); + } + if (obj instanceof Set) { + const d = enter(depth); + return ref( + "Set", + [...obj].map((v) => node(v, d)), + ); + } + if (isTypedArray(obj)) { + // The internal [[TypedArrayName]], so subclasses (e.g. Node's Buffer) + // encode under their base tag rather than an undecodable class name. + const tag = Object.prototype.toString.call(obj).slice(8, -1); + return ref(tag, bytesToB64(new Uint8Array(obj.buffer, obj.byteOffset, obj.byteLength))); + } + if (obj instanceof ArrayBuffer) return ref("ArrayBuffer", bytesToB64(new Uint8Array(obj))); + if (obj instanceof DataView) { + return ref( + "DataView", + bytesToB64(new Uint8Array(obj.buffer, obj.byteOffset, obj.byteLength)), + ); + } + if (obj instanceof Error) { + const d = enter(depth); + const src = obj as unknown as Record; + const own: Record = {}; + for (const k of Object.keys(obj)) setOwn(own, k, node(src[k], d)); + return ref("Error", { name: obj.name, message: obj.message, own }); + } + if (obj instanceof Number) return ref("Number", node(obj.valueOf(), depth)); + if (obj instanceof String) return ref("String", obj.valueOf()); + if (obj instanceof Boolean) return ref("Boolean", obj.valueOf()); + + const tn = temporalName(obj); + if (tn) return ref(tn, (obj as { toString(): string }).toString()); + + throw new SerializeError( + "UNSUPPORTED_VALUE", + "unsupported value (class instance or exotic object)", + ); + } finally { + seen.delete(obj); + } + }; + + return { mark, node, refs, hasRefs: () => count > 0 }; +} + +/** + * Low-level encoder for embedding values in your own JSON structure (this is + * what obj-diff's per-op `serialize` builds on). All `roots` share one `$refs` + * table, so identity and cycles are preserved across them. Each returned value + * is JSON-safe; `refs` is `undefined` when nothing needed a ref. + */ +export function encode( + roots: readonly unknown[], + options?: StringifyOptions, +): { values: unknown[]; refs: Refs | undefined } { + const enc = createEncoder(options); + for (const r of roots) enc.mark(r); + const values = roots.map((r) => enc.node(r)); + return { values, refs: enc.hasRefs() ? enc.refs : undefined }; +} diff --git a/src/index.ts b/src/index.ts new file mode 100644 index 0000000..683a1f5 --- /dev/null +++ b/src/index.ts @@ -0,0 +1,80 @@ +import { createDecoder, decode, isJsonObject, shapeOf } from "./decode"; +import { createEncoder, encode } from "./encode"; +import type { + Envelope, + ParseOptions, + RefEntry, + Refs, + SerializeErrorCode, + StringifyOptions, +} from "./types"; +import { hasOwn, limit, SerializeError } from "./utils"; + +/** + * Serializes any JavaScript value to a self-describing, type-preserving JSON + * string: `{"v": , "$refs"?: {...}}`. + * + * Plain data stays readable JSON. Special values become `"@n"` tokens whose + * real type lives in the `$refs` table: `Date`, `RegExp`, `Map`, `Set`, typed + * arrays, `ArrayBuffer`, `DataView`, `Error`, `URL`, `BigInt`, boxed + * primitives, `Temporal`, `NaN`/`±Infinity`/`-0` and `undefined`. Circular + * references and shared identity among plain objects and arrays are kept. + * + * Throws `SerializeError` on symbols, functions, class instances and cycles + * that run through a `Map`, `Set` or `Error`. + */ +export function stringify(value: unknown, options?: StringifyOptions): string { + const enc = createEncoder(options); + enc.mark(value); + const v = enc.node(value); + const out: Envelope = enc.hasRefs() ? { v, $refs: enc.refs } : { v }; + return JSON.stringify(out); +} + +/** + * Parses a string produced by `stringify` back into the original value with + * live typed values. Safe for untrusted input: `"__proto__"` keys become own + * properties (prototypes are never changed), malformed input throws a + * `SerializeError`, and `maxLength` / `maxDepth` bound the work done. + */ +export function parse(text: string, options: ParseOptions = {}): T { + const maxLength = limit("maxLength", options.maxLength); + if (typeof text !== "string") { + throw new SerializeError("INVALID_INPUT", `expected a string, got ${shapeOf(text)}`); + } + if (text.length > maxLength) { + throw new SerializeError( + "INPUT_TOO_LARGE", + `input is ${text.length} characters, over maxLength (${maxLength})`, + ); + } + let raw: unknown; + try { + raw = JSON.parse(text); + } catch (err) { + throw new SerializeError("INVALID_JSON", `input is not valid JSON: ${(err as Error).message}`, { + cause: err, + }); + } + if (!isJsonObject(raw) || !hasOwn(raw, "v")) { + throw new SerializeError( + "INVALID_ENVELOPE", + `expected the {"v": ..., "$refs"?: {...}} envelope written by stringify(), got ${ + isJsonObject(raw) ? 'an object without a "v" key' : shapeOf(raw) + }`, + ); + } + return createDecoder(raw.$refs, options).node(raw.v) as T; +} + +export { + decode, + type Envelope, + encode, + type ParseOptions, + type RefEntry, + type Refs, + SerializeError, + type SerializeErrorCode, + type StringifyOptions, +}; diff --git a/src/types.ts b/src/types.ts new file mode 100644 index 0000000..cf2e871 --- /dev/null +++ b/src/types.ts @@ -0,0 +1,49 @@ +/** One entry of the `$refs` table: a type tag and its JSON payload. */ +export type RefEntry = { _t: string; _v?: unknown }; + +/** The `$refs` table: ref id (decimal string) → entry. Tokens `"@"` point into it. */ +export type Refs = Record; + +/** The JSON shape `stringify` writes and `parse` reads. */ +export type Envelope = { v: unknown; $refs?: Refs }; + +/** Options for `stringify` and `encode`. */ +export interface StringifyOptions { + /** + * Maximum container nesting depth (arrays, plain objects, `Map`, `Set`, `Error` + * own properties). `[[1]]` has depth 2. Exceeding it throws `MAX_DEPTH`. + * Default: unlimited. + */ + maxDepth?: number; +} + +/** Options for `parse` and `decode`. */ +export interface ParseOptions { + /** + * Maximum input length in UTF-16 code units (`string.length`), checked before + * any parsing work. Exceeding it throws `INPUT_TOO_LARGE`. `parse` only. + * Default: unlimited. + */ + maxLength?: number; + /** + * Maximum container nesting depth of the decoded value; same counting as + * `StringifyOptions.maxDepth`. Exceeding it throws `MAX_DEPTH`. + * Default: unlimited. + */ + maxDepth?: number; +} + +/** Machine-readable reason carried by every `SerializeError`. */ +export type SerializeErrorCode = + | "INVALID_OPTION" + | "INVALID_INPUT" + | "INVALID_JSON" + | "INPUT_TOO_LARGE" + | "INVALID_ENVELOPE" + | "MAX_DEPTH" + | "MISSING_REF" + | "UNKNOWN_TYPE" + | "INVALID_PAYLOAD" + | "UNSUPPORTED_VALUE" + | "CIRCULAR_REF" + | "TEMPORAL_UNAVAILABLE"; diff --git a/src/utils.ts b/src/utils.ts new file mode 100644 index 0000000..0986cb0 --- /dev/null +++ b/src/utils.ts @@ -0,0 +1,107 @@ +import type { SerializeErrorCode } from "./types"; + +/** The single error type thrown by this package. Check `code` to branch on the reason. */ +export class SerializeError extends Error { + readonly code: SerializeErrorCode; + + constructor(code: SerializeErrorCode, message: string, options?: { cause?: unknown }) { + super(`@opentf/serialize: ${message}`, options); + this.name = "SerializeError"; + this.code = code; + } +} + +export const hasOwn = (o: object, k: PropertyKey): boolean => Object.hasOwn(o, k); + +/** `proto === Object.prototype` or a null-prototype object. */ +export function isPlainObject(x: unknown): x is Record { + if (x === null || typeof x !== "object") return false; + const proto = Object.getPrototypeOf(x); + return proto === Object.prototype || proto === null; +} + +export function isTypedArray(x: unknown): x is ArrayBufferView { + return ArrayBuffer.isView(x) && !(x instanceof DataView); +} + +/** + * Assigns `key` as an own enumerable data property. Plain assignment of + * `"__proto__"` would invoke the prototype setter instead of creating a key, so + * that one goes through `defineProperty` — the same result `JSON.parse` gives. + */ +export function setOwn(target: object, key: string, value: unknown): void { + if (key === "__proto__") { + Object.defineProperty(target, key, { + value, + enumerable: true, + writable: true, + configurable: true, + }); + } else { + (target as Record)[key] = value; + } +} + +/** Validates an optional non-negative integer limit; `Infinity`/`undefined` mean unlimited. */ +export function limit(name: string, v: unknown): number { + if (v === undefined || v === Number.POSITIVE_INFINITY) return Number.POSITIVE_INFINITY; + if (typeof v !== "number" || !Number.isInteger(v) || v < 0) { + throw new SerializeError( + "INVALID_OPTION", + `${name} must be a non-negative integer or Infinity, got ${String(v)}`, + ); + } + return v; +} + +export function bytesToB64(bytes: Uint8Array): string { + let bin = ""; + for (let i = 0; i < bytes.length; i++) bin += String.fromCharCode(bytes[i]); + return btoa(bin); +} + +export function b64ToBytes(b64: string): Uint8Array { + const bin = atob(b64); + const out = new Uint8Array(bin.length); + for (let i = 0; i < bin.length; i++) out[i] = bin.charCodeAt(i); + return out; +} + +/** Returns the Temporal type name (e.g. "Temporal.PlainDate") if x is a Temporal value, else null. */ +export function temporalName(x: unknown): string | null { + const tag = Object.prototype.toString.call(x); // "[object Temporal.PlainDate]" + return tag.startsWith("[object Temporal.") ? tag.slice(8, -1) : null; +} + +/** A string that is exactly a ref token `@`. */ +export const REF_TOKEN: RegExp = /^@\d+$/; + +type TAConstructor = new (buffer: ArrayBufferLike) => ArrayBufferView; + +/** Typed array constructors by tag; Float16Array is included where the runtime has it. */ +export const TA_CTORS: Record = Object.assign(Object.create(null), { + Int8Array, + Uint8Array, + Uint8ClampedArray, + Int16Array, + Uint16Array, + Int32Array, + Uint32Array, + Float32Array, + Float64Array, + BigInt64Array, + BigUint64Array, + ...((globalThis as { Float16Array?: TAConstructor }).Float16Array + ? { Float16Array: (globalThis as unknown as { Float16Array: TAConstructor }).Float16Array } + : {}), +}); + +export const ERR_CTORS: Record = Object.assign(Object.create(null), { + Error, + TypeError, + RangeError, + ReferenceError, + SyntaxError, + EvalError, + URIError, +}); diff --git a/tasks.toml b/tasks.toml new file mode 100644 index 0000000..628a5fe --- /dev/null +++ b/tasks.toml @@ -0,0 +1,29 @@ +# Repository task entry point — run with `tsr `. + +[tasks.test] +# Run the suite against src/ with the ES development runtime. +run = "esdev test" + +[tasks.build] +run = "esdev build --target=library" + +[tasks."test:dist"] +# Run the same suite against the built dist/. +run = "esdev test" +env = { TEST_DIST = "true" } +deps = ["build"] + +[tasks.type-check] +run = "esdev typecheck" + +[tasks.lint] +run = "biome lint" + +[tasks.fmt] +run = "biome format --write" + +[tasks."fmt:check"] +run = "biome check" + +[tasks.ci] +deps = ["build", "test", "test:dist", "type-check", "lint", "fmt:check"] diff --git a/tsconfig.json b/tsconfig.json new file mode 100644 index 0000000..c076a6c --- /dev/null +++ b/tsconfig.json @@ -0,0 +1,14 @@ +{ + "compilerOptions": { + "target": "ES2022", + "lib": ["ES2022", "DOM"], + "module": "ESNext", + "moduleResolution": "Bundler", + "strict": true, + "noEmit": true, + "isolatedModules": true, + "forceConsistentCasingInFileNames": true, + "skipLibCheck": true + }, + "include": ["src/**/*"] +} From a400aa7fb58d9f63ff99f7dd14bc2f0aae8a9b4c Mon Sep 17 00:00:00 2001 From: Thanga Ganapathy Date: Wed, 4 Nov 2026 18:03:27 +0530 Subject: [PATCH 2/6] docs: add benchmarks, library comparison and a Pixels-style README - benchmarks/: fidelity, malformed-input, wire-size and speed comparison against superjson, devalue, seroval and @ungap/structured-clone on esdev bench; writes benchmarks/results.json - benchmarks/readme.ts renders the results into README's Measured section (tsr bench:readme) - README rewritten in the Open Tech Foundation style, with a feature comparison and the AFW / obj-diff context --- README.md | 304 ++++++++++++---- benchmarks/cases.ts | 209 +++++++++++ benchmarks/libs.ts | 54 +++ benchmarks/readme.ts | 132 +++++++ benchmarks/results.json | 583 ++++++++++++++++++++++++++++++ benchmarks/serialization.bench.ts | 150 ++++++++ biome.json | 2 +- package.json | 4 + pnpm-lock.yaml | 43 +++ tasks.toml | 9 + 10 files changed, 1414 insertions(+), 76 deletions(-) create mode 100644 benchmarks/cases.ts create mode 100644 benchmarks/libs.ts create mode 100644 benchmarks/readme.ts create mode 100644 benchmarks/results.json create mode 100644 benchmarks/serialization.bench.ts diff --git a/README.md b/README.md index 0c898b5..dd5667c 100644 --- a/README.md +++ b/README.md @@ -1,104 +1,262 @@ -# @opentf/serialize +
-Type-preserving JavaScript serialization over plain JSON. A drop-in for -`JSON.stringify` / `JSON.parse` that keeps `Date`, `Map`, `Set`, typed arrays, -`BigInt`, `Temporal`, `NaN`/`-0`, `undefined`, circular references and shared -identity — and is safe to point at untrusted input. +# Serialize -Extracted from [obj-diff](https://obj-diff.opentechf.org/docs/serialization); -the wire format is unchanged, so output from either one reads in the other. +***Type-preserving JSON for JavaScript: readable on the wire, safe on untrusted input*** + +[npm](https://www.npmjs.com/package/@opentf/serialize) | [JSR](https://jsr.io/@opentf/serialize) | [Wire format](#wire-format) + +
+ +
+ +*An [Open Tech Foundation](https://opentechf.org/) project* + +
+ +> `JSON.stringify` turns a `Date` into a string, a `Map` into `{}` and a `BigInt` +> into an exception. `@opentf/serialize` keeps them — plus typed arrays, `Temporal`, +> `undefined`, `NaN`/`-0` and circular references — while plain data stays plain JSON. ```ts import { parse, stringify } from "@opentf/serialize"; const text = stringify({ at: new Date(0), tags: new Set(["a"]), big: 10n }); -// {"v":{"at":"@1","tags":"@2","big":"@3"},"$refs":{"1":{"_t":"Date",...},...}} +// {"v":{"at":"@1","tags":"@2","big":"@3"},"$refs":{"1":{"_t":"Date","_v":"1970-01-01T00:00:00.000Z"},…}} -const back = parse<{ at: Date; tags: Set; big: bigint }>(text); // live values again +const value = parse(text, { maxLength: 1_000_000, maxDepth: 64 }); +value.at.getFullYear(); // a live Date again ``` -## Wire format +> [!NOTE] +> **Pre-1.0.** The API can change between minor versions. The wire format is the one +> [obj-diff 0.18](https://obj-diff.opentechf.org/docs/serialization) writes, byte for byte. -`stringify` writes an envelope `{"v": , "$refs"?: {...}}`. Plain data -stays readable JSON; every special value becomes a token `"@n"` whose real type -sits in `$refs["n"]` as `{ "_t": , "_v": }`. Strings that start -with `@` are escaped with one more `@`. User keys named `_t` / `_v` are ordinary -data. A plain object or array reachable by more than one edge (shared or -cyclic) is hoisted into `$refs` as an `obj` / `arr` entry, so identity and -cycles are rebuilt on the far side; single-use containers stay inline. +## Why Serialize -| Supported | | +| Need | How Serialize meets it | | --- | --- | -| Primitives | `undefined`, `bigint`, `NaN`, `±Infinity`, `-0` | -| Built-ins | `Date` (incl. invalid), `RegExp`, `URL`, `Map`, `Set`, `Error` and its standard subclasses (name, message, own props), boxed `Number`/`String`/`Boolean` | -| Binary | every typed array, `ArrayBuffer`, `DataView` (byte-exact, base64) | -| Temporal | any `Temporal.*` value — decoding needs `globalThis.Temporal` | -| Graphs | cycles and shared identity among plain objects and arrays | +| 👀 **Readable on the wire** | Plain data stays as it is; only special values become `"@n"` tokens, so a network panel still shows your object. | +| 🔒 **Safe on hostile input** | `__proto__` keys become own properties, never prototypes; every `$refs` entry is shape-checked; malformed input is an error, never `undefined`. | +| 🚧 **Bounded work** | Opt-in `maxLength` and `maxDepth` stop oversized or deeply nested bodies before they cost a stack overflow. | +| 🏷️ **Errors you can branch on** | One `SerializeError` class with a `code` — map `INPUT_TOO_LARGE` to 413 and the rest to 400. | +| 📦 **No dependencies** | One small ESM + CJS package with its own `.d.ts`; runs anywhere `JSON` does. | -Symbols, functions and class instances throw, as do cycles that run through a -`Map`, `Set` or `Error`. +## Compared with other libraries -## Untrusted input +| | **@opentf/serialize** | superjson | devalue | seroval (JSON mode) | @ungap/structured-clone | +| --- | --- | --- | --- | --- | --- | +| **Wire shape** | Your JSON + `$refs` table | Your JSON + `meta` paths | Flattened slot array | Node tree | Flattened records | +| **Plain data stays readable** | Yes | Yes | No | No | No | +| **Own `__proto__` key** | Kept as own key | Rejected (throws) | Rejected (throws) | Kept as own key | Kept as own key | +| **Wrong-shape input** | Throws | Returns `undefined` | Throws | Throws | Throws | +| **Depth limit** | `maxDepth` (opt-in) | None | None | `depthLimit` (1000) | None | +| **Input size limit** | `maxLength` (opt-in) | None | None | None | None | +| **Cycles through `Map`/`Set`** | No (throws) | Yes | Yes | Yes | Yes | +| **Custom classes** | No | `registerClass` | reducers / revivers | Plugins | No | +| **Runtime dependencies** | 0 | 1 | 0 | 0 | 0 | -`parse` validates everything it reads and never returns a half-built value: +Serialize trades a little speed for a format you can read and a parser you can point at +a request body. When you need custom classes, or values that cycle through a `Map`, use +seroval or devalue. -- **`__proto__` keys** become own properties (via `defineProperty`, as - `JSON.parse` does). No prototype is ever changed, and type tags such as - `"constructor"` cannot reach `Object.prototype` members. -- **Malformed input** throws a `SerializeError` with a `code` — a missing - `{v, $refs}` envelope is `INVALID_ENVELOPE`, not `undefined`. -- **Limits** are opt-in: +### Measured -```ts -parse(text, { - maxLength: 1_000_000, // UTF-16 code units, checked before JSON.parse - maxDepth: 64, // container nesting: [[1]] has depth 2 -}); -stringify(value, { maxDepth: 64 }); + + + +Measured 2026-11-04 on ES-Runtime/0.46.0 (Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz, 12 threads) against superjson 2.2.6, devalue 6.0.3, seroval 1.7.7, @ungap/structured-clone 1.4.0. + +#### Round-trip fidelity + +Each value goes through `stringify` then `parse` and must come back with the same +type, value, keys and identity. ✅ exact · ❌ different value or type · 💥 throws. + +| Value | **@opentf/serialize** | superjson | devalue | seroval | @ungap/structured-clone | +| --- | --- | --- | --- | --- | --- | +| Plain nested object | ✅ | ✅ | ✅ | ✅ | ✅ | +| Date | ✅ | ✅ | ✅ | ✅ | ✅ | +| Invalid Date | ✅ | ❌ | ✅ | ✅ | ✅ | +| RegExp | ✅ | ✅ | ✅ | ✅ | ✅ | +| Map | ✅ | ✅ | ✅ | ✅ | ✅ | +| Set | ✅ | ✅ | ✅ | ✅ | ✅ | +| BigInt | ✅ | ✅ | ✅ | ✅ | ✅ | +| Int8Array | ✅ | ✅ | ✅ | ✅ | ✅ | +| Float64Array [NaN, -0, ∞] | ✅ | ❌ | ✅ | ✅ | ❌ | +| BigInt64Array | ✅ | 💥 | ✅ | ✅ | 💥 | +| ArrayBuffer | ✅ | ❌ | ✅ | ✅ | ✅ | +| DataView | ✅ | ❌ | ✅ | ✅ | ✅ | +| Error + custom prop | ✅ | ❌ | 💥 | ✅ | ❌ | +| Boxed Number | ✅ | ❌ | ✅ | ✅ | ✅ | +| Boxed String | ✅ | ❌ | ✅ | ✅ | ✅ | +| undefined | ✅ | ✅ | ✅ | ✅ | ✅ | +| NaN | ✅ | ✅ | ✅ | ✅ | ❌ | +| -0 | ✅ | ✅ | ✅ | ✅ | ✅ | +| Infinity | ✅ | ✅ | ✅ | ✅ | ❌ | +| URL | ✅ | ✅ | ✅ | 💥 | ❌ | +| Temporal.PlainDate | ✅ | ❌ | ✅ | ✅ | ❌ | +| Temporal.ZonedDateTime | ✅ | ❌ | ✅ | ✅ | ❌ | +| Shared reference | ✅ | ✅ | ✅ | ✅ | ✅ | +| Circular object | ✅ | ✅ | ✅ | ✅ | ✅ | +| Circular array | ✅ | ✅ | ✅ | ✅ | ✅ | +| Cycle through a Map | 💥 | ✅ | ✅ | ✅ | ✅ | +| Sparse array | ❌ | ❌ | ✅ | ✅ | ❌ | +| Own __proto__ key | ✅ | 💥 | 💥 | ✅ | ✅ | +| **Score** | **26/28** | **16/28** | **26/28** | **27/28** | **19/28** | + +#### Malformed input + +What `parse` does with valid JSON that is not its own format. Throwing is the safe answer; +a quiet `undefined` reaches application code as if it were data. + +| `parse(...)` input | **@opentf/serialize** | superjson | devalue | seroval | @ungap/structured-clone | +| --- | --- | --- | --- | --- | --- | +| Empty object `{}` | throws | ⚠️ returns `undefined` | throws | throws | throws | +| Foreign object `{"x":1}` | throws | ⚠️ returns `undefined` | throws | throws | throws | +| Bare number `42` | throws | ⚠️ returns `undefined` | throws | throws | throws | + +#### Wire size + +UTF-8 bytes of the output; smallest in bold. + +| Value | **@opentf/serialize** | superjson | devalue | seroval | @ungap/structured-clone | +| --- | --- | --- | --- | --- | --- | +| Flat object | **41** | 44 | 43 | 126 | 83 | +| Mixed native types | 171 | 153 | **108** | 268 | 146 | +| Typed array (8) | 71 | 96 | 58 | 171 | **55** | +| Nested record | 192 | 192 | **133** | 270 | 182 | +| Tricky numbers | 132 | 115 | **24** | 113 | 74 | +| Circular graph | 67 | 83 | **25** | 104 | 49 | + +#### Speed + +**Typed record** (ops/sec, higher is better) + +| Library | `stringify` | `parse` | +| --- | --- | --- | +| **@opentf/serialize** | 126,286 | 160,250 | +| superjson | 53,160 | 168,922 | +| devalue | 180,737 | **241,633** | +| seroval | **192,620** | 111,267 | +| @ungap/structured-clone | 163,425 | 159,091 | + +**1,000 plain rows** (ops/sec, higher is better) + +| Library | `stringify` | `parse` | +| --- | --- | --- | +| **@opentf/serialize** | 1,404 | 1,489 | +| superjson | 216 | **3,096** | +| devalue | 597 | 1,075 | +| seroval | 773 | 472 | +| @ungap/structured-clone | 627 | 584 | +| JSON | **6,201** | 3,080 | + +**1,000 typed rows** (ops/sec, higher is better) + +| Library | `stringify` | `parse` | +| --- | --- | --- | +| **@opentf/serialize** | 409 | 373 | +| superjson | 113 | 325 | +| devalue | 436 | **695** | +| seroval | **517** | 413 | +| @ungap/structured-clone | 393 | 466 | + + +## Built for + +| Project | Use | +| --- | --- | +| [**obj-diff**](https://obj-diff.opentechf.org/docs/serialization) | Its diff `serialize` / `deserialize` and value `stringify` / `parse`, extracted here unchanged in format. obj-diff's own test suite passes against a thin wrapper over `encode` / `decode`. | +| [**AFW**](https://github.com/Open-Tech-Foundation/Web-App-Framework) server functions | The planned body and result encoding (`docs/SERVER_FUNCTIONS.md` §3.2), which needs a decoder that is safe for untrusted bodies. | + +## Install + +```sh +pnpm add @opentf/serialize +``` + +```sh +npm install @opentf/serialize # or: bun add, deno add jsr:@opentf/serialize ``` -Containers counted for depth: arrays, plain objects, `Map`, `Set` and `Error` -own properties — hoisted ones included. +## Supported values -## Errors +| Kind | Values | +| --- | --- | +| **Primitives** | `string`, `number`, `boolean`, `null`, `undefined`, `bigint`, `NaN`, `±Infinity`, `-0` | +| **Built-ins** | `Date` (incl. invalid), `RegExp`, `URL`, `Map`, `Set`, boxed `Number` / `String` / `Boolean` | +| **Errors** | `Error` and its standard subclasses, with `name`, `message` and own properties | +| **Binary** | Every typed array, `ArrayBuffer`, `DataView` — byte-exact, as base64 | +| **Temporal** | Any `Temporal.*` value; decoding needs `globalThis.Temporal` | +| **Graphs** | Cycles and shared identity among plain objects and arrays | -Every failure is a `SerializeError` (a subclass of `Error`). Its `code` is one of: +Symbols, functions and class instances throw, as do cycles that run through a `Map`, +`Set` or `Error`. Holes in sparse arrays come back as `null`, as with `JSON`. -| `code` | Meaning | +## Wire format + +`stringify` writes an envelope `{"v": , "$refs"?: {...}}`: + +| Piece | Rule | | --- | --- | -| `INVALID_INPUT` | `parse` was given something other than a string | -| `INVALID_JSON` | the text is not JSON (`cause` holds the `SyntaxError`) | -| `INPUT_TOO_LARGE` | longer than `maxLength` | -| `INVALID_ENVELOPE` | not `{"v": ...}`, or `$refs` is not an object | -| `MAX_DEPTH` | nests deeper than `maxDepth` | -| `MISSING_REF` | a `"@n"` token with no `$refs` entry | -| `UNKNOWN_TYPE` | an unrecognised `_t` tag | -| `INVALID_PAYLOAD` | an entry's `_v` is the wrong shape for its type | -| `CIRCULAR_REF` | a cycle through `Map`/`Set`/`Error` (either direction) | -| `UNSUPPORTED_VALUE` | symbol, function or class instance passed to `stringify` | -| `TEMPORAL_UNAVAILABLE` | Temporal data, but no `globalThis.Temporal` | -| `INVALID_OPTION` | a limit that is not a non-negative integer or `Infinity` | +| **Plain data** | Written as ordinary JSON. User keys named `_t` / `_v` are just data. | +| **Special values** | Replaced by a token `"@n"`; `$refs["n"]` holds `{ "_t": , "_v": }`. | +| **Strings starting with `@`** | Escaped with one more `@`, so they never read as tokens. | +| **Shared or cyclic containers** | Hoisted into `$refs` as an `obj` / `arr` entry and referenced by token; single-use ones stay inline. | +| **No special values** | No `$refs` key at all: `{"v":{"a":1}}`. | + +## Untrusted input ```ts import { parse, SerializeError } from "@opentf/serialize"; try { - parse(body, { maxLength: 1e6, maxDepth: 64 }); + return parse(body, { maxLength: 1_000_000, maxDepth: 64 }); } catch (err) { - if (err instanceof SerializeError && err.code === "INPUT_TOO_LARGE") return reply(413); + if (err instanceof SerializeError) return new Response(null, { status: err.code === "INPUT_TOO_LARGE" ? 413 : 400 }); throw err; } ``` -## Embedding: `encode` / `decode` +| Option | Effect | Default | +| --- | --- | --- | +| `maxLength` | Longest input, in UTF-16 code units (`text.length`), checked before `JSON.parse` | Unlimited | +| `maxDepth` | Deepest container nesting — arrays, objects, `Map`, `Set`, `Error` props, hoisted ones included. `[[1]]` is 2 | Unlimited | -To place encoded values inside your own JSON shape (this is how obj-diff -serializes each diff op), use the low-level pair. All roots share one `$refs` -table, so identity is preserved across them: +`stringify(value, { maxDepth })` takes the same depth limit. -```ts -import { decode, encode } from "@opentf/serialize"; +## Errors + +Every failure is a `SerializeError` (a subclass of `Error`) whose `code` is one of: + +| `code` | Meaning | +| --- | --- | +| `INVALID_INPUT` | `parse` was given something other than a string | +| `INVALID_JSON` | The text is not JSON (`cause` holds the `SyntaxError`) | +| `INPUT_TOO_LARGE` | Longer than `maxLength` | +| `INVALID_ENVELOPE` | Not `{"v": ...}`, or `$refs` is not an object | +| `MAX_DEPTH` | Nests deeper than `maxDepth` | +| `MISSING_REF` | A `"@n"` token with no `$refs` entry | +| `UNKNOWN_TYPE` | An unrecognised `_t` tag | +| `INVALID_PAYLOAD` | An entry's `_v` is the wrong shape for its type | +| `CIRCULAR_REF` | A cycle through `Map` / `Set` / `Error`, in either direction | +| `UNSUPPORTED_VALUE` | A symbol, function or class instance given to `stringify` | +| `TEMPORAL_UNAVAILABLE` | Temporal data, but no `globalThis.Temporal` | +| `INVALID_OPTION` | A limit that is not a non-negative integer or `Infinity` | +## API + +| Export | Purpose | +| --- | --- | +| `stringify(value, options?)` | Any value → JSON text | +| `parse(text, options?)` | JSON text → value; validates everything it reads | +| `encode(roots, options?)` | Low level: encode several values that share one `$refs` table, to embed in your own JSON | +| `decode(values, refs, options?)` | Low level: the inverse of `encode` | +| `SerializeError` | The error class; see [Errors](#errors) | + +`encode` / `decode` are how obj-diff writes each diff op as `{type, path, value, $refs}`: + +```ts const { values: [path, value], refs } = encode([op.path, op.value]); const wire = JSON.stringify({ type: op.type, path, value, $refs: refs }); @@ -106,23 +264,19 @@ const raw = JSON.parse(wire); const [p, v] = decode([raw.path, raw.value], raw.$refs, { maxDepth: 64 }); ``` -## Development - -Tests and builds run on [esdev](https://esrun.opentechf.org/esdev), pnpm installs -the dev tools (TypeScript, Biome), and [tsr](https://github.com/Open-Tech-Foundation/tsr) -runs the tasks in `tasks.toml`: +## Develop ```sh pnpm install +tsr ci # build, tests on src/ and dist/, typecheck, lint, format check tsr test # tests against src/ -tsr build # dist/ (esm + cjs + .d.ts) tsr test:dist # build, then the tests against dist/ -tsr type-check # esdev typecheck → the project's tsc -tsr lint # biome lint -tsr fmt # biome format --write -tsr ci # all of the above (with fmt:check) +tsr bench:readme # run the benchmarks and regenerate the Measured section above ``` +Tests, builds and benchmarks run on [esdev](https://esrun.opentechf.org/esdev); tasks are +defined in `tasks.toml` and run with [tsr](https://tsr.opentechf.org). + ## License -MIT +MIT — see [LICENSE](LICENSE). diff --git a/benchmarks/cases.ts b/benchmarks/cases.ts new file mode 100644 index 0000000..0e4661e --- /dev/null +++ b/benchmarks/cases.ts @@ -0,0 +1,209 @@ +// Inputs for the comparison: fidelity cases, size cases and speed payloads. + +// ---- cycle-safe, type-aware equality --------------------------------------- + +function sameBytes(x: Uint8Array, y: Uint8Array): boolean { + if (x.length !== y.length) return false; + for (let i = 0; i < x.length; i++) if (x[i] !== y[i]) return false; + return true; +} + +const tag = (x: unknown): string => Object.prototype.toString.call(x); + +/** True when `b` has the same types, values, keys and graph shape as `a`. */ +export function sameValue(a: unknown, b: unknown, seen = new Map()): boolean { + if (Object.is(a, b)) return true; + if (a === null || b === null || typeof a !== "object" || typeof b !== "object") return false; + if (Object.getPrototypeOf(a) !== Object.getPrototypeOf(b)) return false; + if (seen.get(a) === b) return true; + seen.set(a, b); + + if (tag(a).startsWith("[object Temporal.")) return tag(a) === tag(b) && String(a) === String(b); + if (a instanceof Date) return Object.is(a.getTime(), (b as Date).getTime()); + if (a instanceof RegExp) { + return a.source === (b as RegExp).source && a.flags === (b as RegExp).flags; + } + if (a instanceof URL) return a.href === (b as URL).href; + if (a instanceof Number || a instanceof String || a instanceof Boolean) { + return Object.is(a.valueOf(), (b as typeof a).valueOf()); + } + if (a instanceof ArrayBuffer) { + return sameBytes(new Uint8Array(a), new Uint8Array(b as ArrayBuffer)); + } + if (a instanceof DataView) { + const bv = b as DataView; + return sameBytes( + new Uint8Array(a.buffer, a.byteOffset, a.byteLength), + new Uint8Array(bv.buffer, bv.byteOffset, bv.byteLength), + ); + } + if (ArrayBuffer.isView(a)) { + const x = a as unknown as ArrayLike; + const y = b as unknown as ArrayLike; + if (x.length !== y.length) return false; + for (let i = 0; i < x.length; i++) if (!Object.is(x[i], y[i])) return false; + return true; + } + if (a instanceof Map) { + const bm = b as Map; + if (a.size !== bm.size) return false; + const be = [...bm]; + let i = 0; + for (const [k, v] of a) { + if (!sameValue(k, be[i][0], seen) || !sameValue(v, be[i][1], seen)) return false; + i++; + } + return true; + } + if (a instanceof Set) { + const bs = b as Set; + if (a.size !== bs.size) return false; + const bv = [...bs]; + let i = 0; + for (const v of a) if (!sameValue(v, bv[i++], seen)) return false; + return true; + } + if (a instanceof Error) { + const be = b as Error & { code?: unknown }; + return ( + a.name === be.name && + a.message === be.message && + (a as Error & { code?: unknown }).code === be.code + ); + } + const ao = a as Record; + const bo = b as Record; + const ak = Object.keys(ao); + const bk = Object.keys(bo); + if (ak.length !== bk.length || ak.some((k, i) => k !== bk[i])) return false; + for (const k of ak) if (!sameValue(ao[k], bo[k], seen)) return false; + return true; +} + +// ---- fidelity ---------------------------------------------------------------- + +const selfCycle = () => { + const o: Record = { name: "n" }; + o.self = o; + return o; +}; + +/** [label, make] — each value must come back with the same type, value and shape. */ +export const fidelityCases: [string, () => unknown][] = [ + ["Plain nested object", () => ({ a: { b: [1, 2] } })], + ["Date", () => new Date(1e12)], + ["Invalid Date", () => new Date(Number.NaN)], + ["RegExp", () => /foo\d+/giu], + ["Map", () => new Map([["k", 1]])], + ["Set", () => new Set([1, 2, 3])], + ["BigInt", () => 9007199254740993n], + ["Int8Array", () => new Int8Array([1, 2, 3])], + ["Float64Array [NaN, -0, ∞]", () => new Float64Array([Number.NaN, -0, Number.POSITIVE_INFINITY])], + ["BigInt64Array", () => new BigInt64Array([9007199254740993n])], + ["ArrayBuffer", () => new Uint8Array([9, 8, 7]).buffer], + ["DataView", () => new DataView(new Uint8Array([1, 2, 3, 4]).buffer)], + ["Error + custom prop", () => Object.assign(new TypeError("boom"), { code: "E1" })], + ["Boxed Number", () => new Number(42)], + ["Boxed String", () => new String("hi")], + ["undefined", () => undefined], + ["NaN", () => Number.NaN], + ["-0", () => -0], + ["Infinity", () => Number.POSITIVE_INFINITY], + ["URL", () => new URL("https://a.example/p?q=1")], + ["Temporal.PlainDate", () => Temporal.PlainDate.from("2021-06-15")], + ["Temporal.ZonedDateTime", () => Temporal.ZonedDateTime.from("2021-06-15T08:30[Europe/Paris]")], + [ + "Shared reference", + () => { + const s = { id: 1 }; + return { a: s, b: s }; + }, + ], + ["Circular object", selfCycle], + [ + "Circular array", + () => { + const a: unknown[] = [1]; + a.push(a); + return a; + }, + ], + [ + "Cycle through a Map", + () => { + const m = new Map(); + m.set("self", m); + return m; + }, + ], + // biome-ignore lint/suspicious/noSparseArray: the case under test + ["Sparse array", () => [1, , 3]], + ["Own __proto__ key", () => JSON.parse('{"__proto__":{"x":1},"a":1}')], +]; + +/** Identity that `sameValue` cannot see: copies would compare equal, so check `===`. */ +export const identityChecks: Record boolean> = { + "Shared reference": (b) => (b as { a: unknown; b: unknown }).a === (b as { b: unknown }).b, + "Circular object": (b) => (b as { self: unknown }).self === b, + "Circular array": (b) => (b as unknown[])[1] === b, + "Cycle through a Map": (b) => (b as Map).get("self") === b, +}; + +// ---- wire size ----------------------------------------------------------------- + +export const sizeCases: [string, unknown][] = [ + ["Flat object", { id: 1, name: "Ada", active: true }], + [ + "Mixed native types", + { at: new Date(0), tags: new Set(["a", "b"]), prefs: new Map([["t", "dark"]]) }, + ], + ["Typed array (8)", { buf: new Int8Array([1, 2, 3, 4, 5, 6, 7, 8]) }], + [ + "Nested record", + { user: { id: 1, joined: new Date(0), roles: new Set(["admin", "editor"]), big: 42n } }, + ], + ["Tricky numbers", { a: Number.NaN, b: -0, c: Number.POSITIVE_INFINITY }], + ["Circular graph", selfCycle()], +]; + +// ---- speed ----------------------------------------------------------------------- + +const typedRecord = { + user: { + id: 1, + joined: new Date(0), + roles: new Set(["admin", "editor"]), + prefs: new Map([ + ["theme", "dark"], + ["lang", "en"], + ]), + scores: new Float64Array([1.5, 2.5, 3.5]), + big: 123n, + tags: ["a", "b", "c"], + note: null, + }, +}; + +const plainRows = Array.from({ length: 1000 }, (_, i) => ({ + id: i, + name: `user ${i}`, + email: `user${i}@example.com`, + active: i % 3 !== 0, + score: i * 1.5, + tags: ["a", "b"], +})); + +const typedRows = Array.from({ length: 1000 }, (_, i) => ({ + id: i, + name: `user ${i}`, + joined: new Date(1e12 + i * 86_400_000), + roles: new Set(i % 2 ? ["admin"] : ["editor", "viewer"]), + balance: BigInt(i) * 1_000_000_007n, +})); + +/** [label, payload, include the plain-JSON baseline] */ +export const speedCases: [string, unknown, boolean][] = [ + ["Typed record", typedRecord, false], + ["1,000 plain rows", plainRows, true], + ["1,000 typed rows", typedRows, false], +]; diff --git a/benchmarks/libs.ts b/benchmarks/libs.ts new file mode 100644 index 0000000..fb339e6 --- /dev/null +++ b/benchmarks/libs.ts @@ -0,0 +1,54 @@ +// The libraries under comparison, each reduced to string-in / string-out so they +// are measured on the same job: a value to JSON text and back. +import * as ungap from "@ungap/structured-clone/json"; +import * as devalue from "devalue"; +import { fromJSON, toJSON } from "seroval"; +import superjson from "superjson"; +import { parse, stringify } from "../src/index"; + +export interface Lib { + name: string; + /** npm package to read the version from; undefined for built-ins and this package. */ + pkg?: string; + stringify(value: unknown): string; + parse(text: string): unknown; +} + +export const OURS = "@opentf/serialize"; + +export const libs: Lib[] = [ + { name: OURS, stringify: (v) => stringify(v), parse: (s) => parse(s) }, + { + name: "superjson", + pkg: "superjson", + stringify: (v) => superjson.stringify(v), + parse: (s) => superjson.parse(s), + }, + { + name: "devalue", + pkg: "devalue", + stringify: (v) => devalue.stringify(v), + parse: (s) => devalue.parse(s), + }, + { + // JSON mode (toJSON / fromJSON): seroval's default serialize() emits JS source + // that must be eval'd, which is not an option for untrusted input. + name: "seroval", + pkg: "seroval", + stringify: (v) => JSON.stringify(toJSON(v)), + parse: (s) => fromJSON(JSON.parse(s)), + }, + { + name: "@ungap/structured-clone", + pkg: "@ungap/structured-clone", + stringify: (v) => ungap.stringify(v), + parse: (s) => ungap.parse(s), + }, +]; + +/** Baseline only: loses every non-JSON type, so it appears in the speed table for plain data. */ +export const json: Lib = { + name: "JSON", + stringify: (v) => JSON.stringify(v), + parse: (s) => JSON.parse(s), +}; diff --git a/benchmarks/readme.ts b/benchmarks/readme.ts new file mode 100644 index 0000000..5d40f6f --- /dev/null +++ b/benchmarks/readme.ts @@ -0,0 +1,132 @@ +// Renders benchmarks/results.json into README.md, between the +// and markers. Run: tsr bench:readme +import * as fs from "runtime:fs"; + +const at = (rel: string): string => new URL(rel, import.meta.url).pathname; +const START = ""; +const END = ""; + +type Row = { lib: string } & T; +interface Results { + meta: { + generatedAt: string; + runtime: string; + cpu: string | null; + threads: number; + package: string; + versions: Record; + regenerate: string; + }; + libraries: string[]; + fidelity: { cases: string[]; rows: Row<{ cells: string[]; score: number }>[] }; + malformed: { cases: string[]; rows: Row<{ cells: string[] }>[] }; + size: { cases: string[]; rows: Row<{ bytes: (number | null)[] }>[] }; + speed: { + scenario: string; + ops: { op: string; rows: Row<{ opsPerSec: number | null; rme: number | null }>[] }[]; + }[]; +} + +const r: Results = JSON.parse(await fs.file(at("results.json")).text()); +const ours = r.meta.package; +const head = (name: string) => (name === ours ? `**${name}**` : name); +const table = (header: string[], rows: string[][]): string => + [header, header.map(() => "---"), ...rows].map((cells) => `| ${cells.join(" | ")} |`).join("\n"); +const num = (n: number) => n.toLocaleString("en-US"); + +const FIDELITY: Record = { pass: "✅", fail: "❌", throws: "💥" }; + +const fidelity = table( + ["Value", ...r.libraries.map(head)], + [ + ...r.fidelity.cases.map((label, i) => [ + label, + ...r.fidelity.rows.map((row) => FIDELITY[row.cells[i]]), + ]), + ["**Score**", ...r.fidelity.rows.map((row) => `**${row.score}/${r.fidelity.cases.length}**`)], + ], +); + +const malformed = table( + ["`parse(...)` input", ...r.libraries.map(head)], + r.malformed.cases.map((label, i) => [ + label, + ...r.malformed.rows.map((row) => + row.cells[i] === "throws" + ? "throws" + : `⚠️ ${row.cells[i].replace(/^returns /, "returns `")}\``, + ), + ]), +); + +const size = table( + ["Value", ...r.libraries.map(head)], + r.size.cases.map((label, i) => { + const vals = r.size.rows.map((row) => row.bytes[i]); + const best = Math.min(...vals.filter((v): v is number => v != null)); + return [label, ...vals.map((v) => (v == null ? "—" : v === best ? `**${v}**` : String(v)))]; + }), +); + +const speed = r.speed + .map(({ scenario, ops }) => { + const libsHere = ops[0].rows.map((row) => row.lib); + const best = ops.map((o) => Math.max(...o.rows.map((row) => row.opsPerSec ?? 0))); + const rows = libsHere.map((lib, li) => [ + head(lib), + ...ops.map((o, oi) => { + const v = o.rows[li].opsPerSec; + if (v == null) return "—"; + return v === best[oi] ? `**${num(v)}**` : num(v); + }), + ]); + return `**${scenario}** (ops/sec, higher is better)\n\n${table( + ["Library", ...ops.map((o) => `\`${o.op}\``)], + rows, + )}`; + }) + .join("\n\n"); + +const versions = Object.entries(r.meta.versions) + .filter(([name]) => name !== ours) + .map(([name, v]) => `${name} ${v}`) + .join(", "); +const machine = [r.meta.cpu, `${r.meta.threads} threads`].filter(Boolean).join(", "); + +const section = `${START} + + +Measured ${r.meta.generatedAt} on ${r.meta.runtime} (${machine}) against ${versions}. + +#### Round-trip fidelity + +Each value goes through \`stringify\` then \`parse\` and must come back with the same +type, value, keys and identity. ✅ exact · ❌ different value or type · 💥 throws. + +${fidelity} + +#### Malformed input + +What \`parse\` does with valid JSON that is not its own format. Throwing is the safe answer; +a quiet \`undefined\` reaches application code as if it were data. + +${malformed} + +#### Wire size + +UTF-8 bytes of the output; smallest in bold. + +${size} + +#### Speed + +${speed} +${END}`; + +const readmePath = at("../README.md"); +const readme = await fs.file(readmePath).text(); +const from = readme.indexOf(START); +const to = readme.indexOf(END); +if (from < 0 || to < from) throw new Error(`README.md has no ${START} … ${END} section`); +await fs.write(readmePath, readme.slice(0, from) + section + readme.slice(to + END.length)); +console.log("README.md benchmark section updated"); diff --git a/benchmarks/results.json b/benchmarks/results.json new file mode 100644 index 0000000..de22f0e --- /dev/null +++ b/benchmarks/results.json @@ -0,0 +1,583 @@ +{ + "meta": { + "generatedAt": "2026-11-04", + "runtime": "ES-Runtime/0.46.0", + "cpu": "Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz", + "threads": 12, + "package": "@opentf/serialize", + "versions": { + "@opentf/serialize": "0.1.0", + "superjson": "2.2.6", + "devalue": "6.0.3", + "seroval": "1.7.7", + "@ungap/structured-clone": "1.4.0" + }, + "regenerate": "tsr bench:readme" + }, + "libraries": [ + "@opentf/serialize", + "superjson", + "devalue", + "seroval", + "@ungap/structured-clone" + ], + "fidelity": { + "cases": [ + "Plain nested object", + "Date", + "Invalid Date", + "RegExp", + "Map", + "Set", + "BigInt", + "Int8Array", + "Float64Array [NaN, -0, ∞]", + "BigInt64Array", + "ArrayBuffer", + "DataView", + "Error + custom prop", + "Boxed Number", + "Boxed String", + "undefined", + "NaN", + "-0", + "Infinity", + "URL", + "Temporal.PlainDate", + "Temporal.ZonedDateTime", + "Shared reference", + "Circular object", + "Circular array", + "Cycle through a Map", + "Sparse array", + "Own __proto__ key" + ], + "rows": [ + { + "lib": "@opentf/serialize", + "cells": [ + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "throws", + "fail", + "pass" + ], + "score": 26 + }, + { + "lib": "superjson", + "cells": [ + "pass", + "pass", + "fail", + "pass", + "pass", + "pass", + "pass", + "pass", + "fail", + "throws", + "fail", + "fail", + "fail", + "fail", + "fail", + "pass", + "pass", + "pass", + "pass", + "pass", + "fail", + "fail", + "pass", + "pass", + "pass", + "pass", + "fail", + "throws" + ], + "score": 16 + }, + { + "lib": "devalue", + "cells": [ + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "throws", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "throws" + ], + "score": 26 + }, + { + "lib": "seroval", + "cells": [ + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "throws", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass" + ], + "score": 27 + }, + { + "lib": "@ungap/structured-clone", + "cells": [ + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "pass", + "fail", + "throws", + "pass", + "pass", + "fail", + "pass", + "pass", + "pass", + "fail", + "pass", + "fail", + "fail", + "fail", + "fail", + "pass", + "pass", + "pass", + "pass", + "fail", + "pass" + ], + "score": 19 + } + ] + }, + "malformed": { + "cases": [ + "Empty object `{}`", + "Foreign object `{\"x\":1}`", + "Bare number `42`" + ], + "rows": [ + { + "lib": "@opentf/serialize", + "cells": [ + "throws", + "throws", + "throws" + ] + }, + { + "lib": "superjson", + "cells": [ + "returns undefined", + "returns undefined", + "returns undefined" + ] + }, + { + "lib": "devalue", + "cells": [ + "throws", + "throws", + "throws" + ] + }, + { + "lib": "seroval", + "cells": [ + "throws", + "throws", + "throws" + ] + }, + { + "lib": "@ungap/structured-clone", + "cells": [ + "throws", + "throws", + "throws" + ] + } + ] + }, + "size": { + "cases": [ + "Flat object", + "Mixed native types", + "Typed array (8)", + "Nested record", + "Tricky numbers", + "Circular graph" + ], + "rows": [ + { + "lib": "@opentf/serialize", + "bytes": [ + 41, + 171, + 71, + 192, + 132, + 67 + ] + }, + { + "lib": "superjson", + "bytes": [ + 44, + 153, + 96, + 192, + 115, + 83 + ] + }, + { + "lib": "devalue", + "bytes": [ + 43, + 108, + 58, + 133, + 24, + 25 + ] + }, + { + "lib": "seroval", + "bytes": [ + 126, + 268, + 171, + 270, + 113, + 104 + ] + }, + { + "lib": "@ungap/structured-clone", + "bytes": [ + 83, + 146, + 55, + 182, + 74, + 49 + ] + } + ] + }, + "speed": [ + { + "scenario": "Typed record", + "ops": [ + { + "op": "stringify", + "rows": [ + { + "lib": "@opentf/serialize", + "opsPerSec": 126286, + "meanMs": 0.007918535215256846, + "rme": 0.55 + }, + { + "lib": "superjson", + "opsPerSec": 53160, + "meanMs": 0.01881127265861042, + "rme": 0.76 + }, + { + "lib": "devalue", + "opsPerSec": 180737, + "meanMs": 0.0055329021903323455, + "rme": 0.69 + }, + { + "lib": "seroval", + "opsPerSec": 192620, + "meanMs": 0.005191571469033233, + "rme": 0.84 + }, + { + "lib": "@ungap/structured-clone", + "opsPerSec": 163425, + "meanMs": 0.0061190167107251356, + "rme": 0.72 + } + ] + }, + { + "op": "parse", + "rows": [ + { + "lib": "@opentf/serialize", + "opsPerSec": 160250, + "meanMs": 0.006240258755201177, + "rme": 0.35 + }, + { + "lib": "superjson", + "opsPerSec": 168922, + "meanMs": 0.0059199029126214605, + "rme": 0.41 + }, + { + "lib": "devalue", + "opsPerSec": 241633, + "meanMs": 0.004138512049237106, + "rme": 0.52 + }, + { + "lib": "seroval", + "opsPerSec": 111267, + "meanMs": 0.008987387309292619, + "rme": 0.46 + }, + { + "lib": "@ungap/structured-clone", + "opsPerSec": 159091, + "meanMs": 0.0062856926144244755, + "rme": 0.45 + } + ] + } + ] + }, + { + "scenario": "1,000 plain rows", + "ops": [ + { + "op": "stringify", + "rows": [ + { + "lib": "@opentf/serialize", + "opsPerSec": 1404, + "meanMs": 0.712266666666718, + "rme": 0.8 + }, + { + "lib": "superjson", + "opsPerSec": 216, + "meanMs": 4.6226666666666985, + "rme": 0.35 + }, + { + "lib": "devalue", + "opsPerSec": 597, + "meanMs": 1.6752807017543858, + "rme": 0.79 + }, + { + "lib": "seroval", + "opsPerSec": 773, + "meanMs": 1.2929122807017879, + "rme": 0.38 + }, + { + "lib": "@ungap/structured-clone", + "opsPerSec": 627, + "meanMs": 1.5954140350877104, + "rme": 0.9 + }, + { + "lib": "JSON", + "opsPerSec": 6201, + "meanMs": 0.16127280701753238, + "rme": 0.31 + } + ] + }, + { + "op": "parse", + "rows": [ + { + "lib": "@opentf/serialize", + "opsPerSec": 1489, + "meanMs": 0.6716457399102937, + "rme": 1.09 + }, + { + "lib": "superjson", + "opsPerSec": 3096, + "meanMs": 0.3230414798206444, + "rme": 0.77 + }, + { + "lib": "devalue", + "opsPerSec": 1075, + "meanMs": 0.9300941704035754, + "rme": 0.49 + }, + { + "lib": "seroval", + "opsPerSec": 472, + "meanMs": 2.1172533632286914, + "rme": 0.31 + }, + { + "lib": "@ungap/structured-clone", + "opsPerSec": 584, + "meanMs": 1.713603139013473, + "rme": 0.56 + }, + { + "lib": "JSON", + "opsPerSec": 3080, + "meanMs": 0.32465582959639505, + "rme": 0.72 + } + ] + } + ] + }, + { + "scenario": "1,000 typed rows", + "ops": [ + { + "op": "stringify", + "rows": [ + { + "lib": "@opentf/serialize", + "opsPerSec": 409, + "meanMs": 2.4463237410072254, + "rme": 0.48 + }, + { + "lib": "superjson", + "opsPerSec": 113, + "meanMs": 8.846633093525107, + "rme": 0.82 + }, + { + "lib": "devalue", + "opsPerSec": 436, + "meanMs": 2.292762589928041, + "rme": 0.93 + }, + { + "lib": "seroval", + "opsPerSec": 517, + "meanMs": 1.9351870503596467, + "rme": 0.51 + }, + { + "lib": "@ungap/structured-clone", + "opsPerSec": 393, + "meanMs": 2.5440863309353214, + "rme": 0.87 + } + ] + }, + { + "op": "parse", + "rows": [ + { + "lib": "@opentf/serialize", + "opsPerSec": 373, + "meanMs": 2.6805868544601115, + "rme": 1.44 + }, + { + "lib": "superjson", + "opsPerSec": 325, + "meanMs": 3.0805868544599786, + "rme": 1.46 + }, + { + "lib": "devalue", + "opsPerSec": 695, + "meanMs": 1.4395868544600836, + "rme": 1.31 + }, + { + "lib": "seroval", + "opsPerSec": 413, + "meanMs": 2.423647887323931, + "rme": 0.51 + }, + { + "lib": "@ungap/structured-clone", + "opsPerSec": 466, + "meanMs": 2.1452347417840607, + "rme": 1.88 + } + ] + } + ] + } + ] +} diff --git a/benchmarks/serialization.bench.ts b/benchmarks/serialization.bench.ts new file mode 100644 index 0000000..980cb58 --- /dev/null +++ b/benchmarks/serialization.bench.ts @@ -0,0 +1,150 @@ +// Compares @opentf/serialize with the type-preserving JSON alternatives on what +// decides a wire format: fidelity, behaviour on malformed input, size and speed. +// +// tsr bench measure, and write benchmarks/results.json +// tsr bench:readme ...then render the results into README.md +import * as fs from "runtime:fs"; +import { Command } from "runtime:system"; +import { test } from "runtime:test"; +import { fidelityCases, identityChecks, sameValue, sizeCases, speedCases } from "./cases"; +import { json, type Lib, libs, OURS } from "./libs"; + +// Paths from this file, not the working directory, so any cwd works. +const at = (rel: string): string => new URL(rel, import.meta.url).pathname; +const OUT = at("results.json"); +const BENCH = { time: 500, warmupTime: 100 }; + +async function version(lib: Lib): Promise { + const manifest = lib.pkg ? `../node_modules/${lib.pkg}/package.json` : "../package.json"; + return JSON.parse(await fs.file(at(manifest)).text()).version; +} + +/** The CPU model, for the README's machine line; null where lscpu is unavailable. */ +async function cpu(): Promise { + try { + const out = await new Command("lscpu", { args: [] }).output(); + const line = new TextDecoder() + .decode(out.stdout) + .split("\n") + .find((l) => l.startsWith("Model name")); + return line ? line.slice(line.indexOf(":") + 1).trim() : null; + } catch { + return null; + } +} + +type Cell = "pass" | "fail" | "throws"; + +function fidelity(lib: Lib): Cell[] { + return fidelityCases.map(([label, make]) => { + const value = make(); + try { + const back = lib.parse(lib.stringify(value)); + const identity = identityChecks[label]?.(back) ?? true; + return sameValue(value, back) && identity ? "pass" : "fail"; + } catch { + return "throws"; + } + }); +} + +/** What parse does with JSON that is not its own format: "throws", or what it returns. */ +const malformedCases: [string, string][] = [ + ["Empty object `{}`", "{}"], + ['Foreign object `{"x":1}`', '{"x":1}'], + ["Bare number `42`", "42"], +]; + +function malformed(lib: Lib): string[] { + return malformedCases.map(([, text]) => { + try { + const out = lib.parse(text); + return `returns ${out === undefined ? "undefined" : JSON.stringify(out)}`; + } catch { + return "throws"; + } + }); +} + +const utf8 = new TextEncoder(); + +function sizes(lib: Lib): (number | null)[] { + return sizeCases.map(([, value]) => { + try { + return utf8.encode(lib.stringify(value)).length; + } catch { + return null; + } + }); +} + +test("serialization comparison", async ({ bench }) => { + const speed = []; + for (const [label, payload, withJson] of speedCases) { + const group = withJson ? [...libs, json] : libs; + const usable = group.filter((lib) => { + try { + lib.parse(lib.stringify(payload)); + return true; + } catch { + return false; + } + }); + const ops = []; + for (const op of ["stringify", "parse"] as const) { + const runs = usable.map((lib) => { + const wire = lib.stringify(payload); + return bench(`${label} · ${op} · ${lib.name}`, () => + op === "stringify" ? lib.stringify(payload) : lib.parse(wire), + ); + }); + const results = await bench.compare(...runs, BENCH); + ops.push({ + op, + rows: group.map((lib) => { + const r = results.get(`${label} · ${op} · ${lib.name}`); + return { + lib: lib.name, + opsPerSec: r ? Math.round(r.throughput.mean) : null, + meanMs: r ? r.latency.mean : null, + rme: r ? Number(r.latency.rme.toFixed(2)) : null, + }; + }), + }); + } + speed.push({ scenario: label, ops }); + } + + const data = { + meta: { + generatedAt: new Date().toISOString().slice(0, 10), + runtime: navigator.userAgent, + cpu: await cpu(), + threads: navigator.hardwareConcurrency, + package: OURS, + versions: Object.fromEntries( + await Promise.all(libs.map(async (l) => [l.name, await version(l)])), + ), + regenerate: "tsr bench:readme", + }, + libraries: libs.map((l) => l.name), + fidelity: { + cases: fidelityCases.map(([label]) => label), + rows: libs.map((lib) => { + const cells = fidelity(lib); + return { lib: lib.name, cells, score: cells.filter((c) => c === "pass").length }; + }), + }, + malformed: { + cases: malformedCases.map(([label]) => label), + rows: libs.map((lib) => ({ lib: lib.name, cells: malformed(lib) })), + }, + size: { + cases: sizeCases.map(([label]) => label), + rows: libs.map((lib) => ({ lib: lib.name, bytes: sizes(lib) })), + }, + speed, + }; + await fs.write(OUT, `${JSON.stringify(data, null, 2)}\n`); + console.log("wrote benchmarks/results.json"); +}); diff --git a/biome.json b/biome.json index cc77256..85df197 100644 --- a/biome.json +++ b/biome.json @@ -1,7 +1,7 @@ { "$schema": "https://biomejs.dev/schemas/2.6.4/schema.json", "files": { - "includes": ["src/**/*.ts", "__tests__/**/*.ts"] + "includes": ["src/**/*.ts", "__tests__/**/*.ts", "benchmarks/**/*.ts"] }, "linter": { "enabled": true, diff --git a/package.json b/package.json index 41afa97..7244c6f 100644 --- a/package.json +++ b/package.json @@ -52,6 +52,10 @@ "packageManager": "pnpm@12.5.1", "devDependencies": { "@biomejs/biome": "^2.6.4", + "@ungap/structured-clone": "^1.4.0", + "devalue": "^6.0.3", + "seroval": "^1.7.7", + "superjson": "^2.2.6", "typescript": "^7.0.2" } } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 421fd4f..50715e1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -169,6 +169,18 @@ importers: '@biomejs/biome': specifier: ^2.6.4 version: 2.6.4 + '@ungap/structured-clone': + specifier: ^1.4.0 + version: 1.4.0 + devalue: + specifier: ^6.0.3 + version: 6.0.3 + seroval: + specifier: ^1.7.7 + version: 1.7.7 + superjson: + specifier: ^2.2.6 + version: 2.2.6 typescript: specifier: ^7.0.2 version: 7.0.2 @@ -364,6 +376,25 @@ packages: cpu: [x64] os: [win32] + '@ungap/structured-clone@1.4.0': + resolution: {integrity: sha512-1mEZtMKPM09vDmQt5y7YvmN2+DFTP7Tg0EWXdic8/C6VRnpb33e4ghisCIE3WZjsE2N8mf+QV1Zqh7ZFYLWInQ==} + + copy-anything@4.1.5: + resolution: {integrity: sha512-vePst2VLTSqPglW7Q98VwFpGiGpOHTyXIo81MO5VyLecA5zMbZw/P/V1vmwD1jksa2iJpn/YkDrJ3XhTfzUyhQ==} + engines: {node: '>=18'} + + devalue@6.0.3: + resolution: {integrity: sha512-3woMEITftG1wKeO/4d/QXH1DVKctTSZg3jNGPpjp9hNr5Hos8MCFF5McTML5ZtgKDObm2R591B0spRp2IVernA==} + engines: {node: '>=18'} + + seroval@1.7.7: + resolution: {integrity: sha512-9OdYsqMLV5RSUNNtkgr6DT2bS/ncba6AqUJx8H7epNJNa0cKNWhQtF5GLRjkF/KuAdvAuRRitWrq86K/v1S/kg==} + engines: {node: '>=20'} + + superjson@2.2.6: + resolution: {integrity: sha512-H+ue8Zo4vJmV2nRjpx86P35lzwDT3nItnIsocgumgr0hHMQ+ZGq5vrERg9kJBo5AWGmxZDhzDo+WVIJqkB0cGA==} + engines: {node: '>=16'} + typescript@7.0.2: resolution: {integrity: sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==} engines: {node: '>=16.20.0'} @@ -474,6 +505,18 @@ snapshots: '@typescript/typescript-win32-x64@7.0.2': optional: true + '@ungap/structured-clone@1.4.0': {} + + copy-anything@4.1.5: {} + + devalue@6.0.3: {} + + seroval@1.7.7: {} + + superjson@2.2.6: + dependencies: + copy-anything: 4.1.5 + typescript@7.0.2: optionalDependencies: '@typescript/typescript-aix-ppc64': 7.0.2 diff --git a/tasks.toml b/tasks.toml index 628a5fe..85201a7 100644 --- a/tasks.toml +++ b/tasks.toml @@ -27,3 +27,12 @@ run = "biome check" [tasks.ci] deps = ["build", "test", "test:dist", "type-check", "lint", "fmt:check"] + +[tasks.bench] +# Compare with the alternatives; writes benchmarks/results.json. +run = "esdev bench" + +[tasks."bench:readme"] +# Render benchmarks/results.json into README.md's Measured section. +run = "esdev benchmarks/readme.ts" +deps = ["bench"] From e4a2765029247f998f28c9911d9b19092972f3ef Mon Sep 17 00:00:00 2001 From: Thanga Ganapathy Date: Wed, 4 Nov 2026 18:43:53 +0530 Subject: [PATCH 3/6] feat: support cycles through Map/Set/Error and sparse-array holes - Any value reachable by more than one edge is written once in $refs and referenced by token; Map, Set and Error claim their id before their children, and the decoder registers their empty shells first, so cycles through them round-trip and every shared value keeps its identity. - Array holes encode as tokens for one shared {"_t":"hole"} entry; a hole anywhere but an array element is rejected as INVALID_PAYLOAD. - Output for values without shared specials or holes is unchanged, byte for byte, from obj-diff 0.18; CIRCULAR_REF now only flags forged leaf entries. - Fidelity benchmark: 28/28; README regenerated. --- README.md | 56 +++++------ __tests__/codec.test.ts | 5 +- __tests__/graphs.test.ts | 126 +++++++++++++++++++++++++ __tests__/safety.test.ts | 19 ++-- benchmarks/results.json | 196 +++++++++++++++++++------------------- src/decode.ts | 59 +++++++++--- src/encode.ts | 197 +++++++++++++++++++-------------------- src/index.ts | 8 +- 8 files changed, 412 insertions(+), 254 deletions(-) create mode 100644 __tests__/graphs.test.ts diff --git a/README.md b/README.md index dd5667c..36236bc 100644 --- a/README.md +++ b/README.md @@ -52,13 +52,12 @@ value.at.getFullYear(); // a live Date again | **Wrong-shape input** | Throws | Returns `undefined` | Throws | Throws | Throws | | **Depth limit** | `maxDepth` (opt-in) | None | None | `depthLimit` (1000) | None | | **Input size limit** | `maxLength` (opt-in) | None | None | None | None | -| **Cycles through `Map`/`Set`** | No (throws) | Yes | Yes | Yes | Yes | +| **Cycles through `Map`/`Set`** | Yes | Yes | Yes | Yes | Yes | | **Custom classes** | No | `registerClass` | reducers / revivers | Plugins | No | | **Runtime dependencies** | 0 | 1 | 0 | 0 | 0 | -Serialize trades a little speed for a format you can read and a parser you can point at -a request body. When you need custom classes, or values that cycle through a `Map`, use -seroval or devalue. +Serialize trades a little speed and size for a format you can read and a parser you can +point at a request body. When you need custom classes, use seroval or devalue. ### Measured @@ -99,10 +98,10 @@ type, value, keys and identity. ✅ exact · ❌ different value or type · 💥 | Shared reference | ✅ | ✅ | ✅ | ✅ | ✅ | | Circular object | ✅ | ✅ | ✅ | ✅ | ✅ | | Circular array | ✅ | ✅ | ✅ | ✅ | ✅ | -| Cycle through a Map | 💥 | ✅ | ✅ | ✅ | ✅ | -| Sparse array | ❌ | ❌ | ✅ | ✅ | ❌ | +| Cycle through a Map | ✅ | ✅ | ✅ | ✅ | ✅ | +| Sparse array | ✅ | ❌ | ✅ | ✅ | ❌ | | Own __proto__ key | ✅ | 💥 | 💥 | ✅ | ✅ | -| **Score** | **26/28** | **16/28** | **26/28** | **27/28** | **19/28** | +| **Score** | **28/28** | **16/28** | **26/28** | **27/28** | **19/28** | #### Malformed input @@ -134,32 +133,32 @@ UTF-8 bytes of the output; smallest in bold. | Library | `stringify` | `parse` | | --- | --- | --- | -| **@opentf/serialize** | 126,286 | 160,250 | -| superjson | 53,160 | 168,922 | -| devalue | 180,737 | **241,633** | -| seroval | **192,620** | 111,267 | -| @ungap/structured-clone | 163,425 | 159,091 | +| **@opentf/serialize** | 135,894 | 159,406 | +| superjson | 53,343 | 165,938 | +| devalue | 185,374 | **241,824** | +| seroval | **199,268** | 111,388 | +| @ungap/structured-clone | 163,472 | 156,489 | **1,000 plain rows** (ops/sec, higher is better) | Library | `stringify` | `parse` | | --- | --- | --- | -| **@opentf/serialize** | 1,404 | 1,489 | -| superjson | 216 | **3,096** | -| devalue | 597 | 1,075 | -| seroval | 773 | 472 | -| @ungap/structured-clone | 627 | 584 | -| JSON | **6,201** | 3,080 | +| **@opentf/serialize** | 1,443 | 1,585 | +| superjson | 212 | **3,145** | +| devalue | 577 | 1,085 | +| seroval | 744 | 472 | +| @ungap/structured-clone | 621 | 545 | +| JSON | **6,078** | 3,122 | **1,000 typed rows** (ops/sec, higher is better) | Library | `stringify` | `parse` | | --- | --- | --- | -| **@opentf/serialize** | 409 | 373 | -| superjson | 113 | 325 | -| devalue | 436 | **695** | -| seroval | **517** | 413 | -| @ungap/structured-clone | 393 | 466 | +| **@opentf/serialize** | 397 | 387 | +| superjson | 108 | 317 | +| devalue | 409 | **692** | +| seroval | **503** | 419 | +| @ungap/structured-clone | 370 | 474 | ## Built for @@ -188,10 +187,10 @@ npm install @opentf/serialize # or: bun add, deno add jsr:@opentf/serialize | **Errors** | `Error` and its standard subclasses, with `name`, `message` and own properties | | **Binary** | Every typed array, `ArrayBuffer`, `DataView` — byte-exact, as base64 | | **Temporal** | Any `Temporal.*` value; decoding needs `globalThis.Temporal` | -| **Graphs** | Cycles and shared identity among plain objects and arrays | +| **Graphs** | Cycles and shared identity through objects, arrays, `Map`, `Set` and `Error`; any value used twice decodes to one object | +| **Sparse arrays** | Holes stay holes, distinct from `undefined` and `null` | -Symbols, functions and class instances throw, as do cycles that run through a `Map`, -`Set` or `Error`. Holes in sparse arrays come back as `null`, as with `JSON`. +Symbols, functions and class instances throw. ## Wire format @@ -202,7 +201,8 @@ Symbols, functions and class instances throw, as do cycles that run through a `M | **Plain data** | Written as ordinary JSON. User keys named `_t` / `_v` are just data. | | **Special values** | Replaced by a token `"@n"`; `$refs["n"]` holds `{ "_t": , "_v": }`. | | **Strings starting with `@`** | Escaped with one more `@`, so they never read as tokens. | -| **Shared or cyclic containers** | Hoisted into `$refs` as an `obj` / `arr` entry and referenced by token; single-use ones stay inline. | +| **Shared or cyclic values** | Written once in `$refs` (plain containers as an `obj` / `arr` entry) and referenced by token everywhere; single-use plain containers stay inline. | +| **Array holes** | Every hole is a token for one shared `{"_t":"hole"}` entry. | | **No special values** | No `$refs` key at all: `{"v":{"a":1}}`. | ## Untrusted input @@ -239,7 +239,7 @@ Every failure is a `SerializeError` (a subclass of `Error`) whose `code` is one | `MISSING_REF` | A `"@n"` token with no `$refs` entry | | `UNKNOWN_TYPE` | An unrecognised `_t` tag | | `INVALID_PAYLOAD` | An entry's `_v` is the wrong shape for its type | -| `CIRCULAR_REF` | A cycle through `Map` / `Set` / `Error`, in either direction | +| `CIRCULAR_REF` | A forged entry that refers to itself where no cycle is possible (e.g. a boxed `Number`) | | `UNSUPPORTED_VALUE` | A symbol, function or class instance given to `stringify` | | `TEMPORAL_UNAVAILABLE` | Temporal data, but no `globalThis.Temporal` | | `INVALID_OPTION` | A limit that is not a non-negative integer or `Infinity` | diff --git a/__tests__/codec.test.ts b/__tests__/codec.test.ts index be3fe1c..d5e062f 100644 --- a/__tests__/codec.test.ts +++ b/__tests__/codec.test.ts @@ -134,14 +134,11 @@ describe("stringify / parse round-trip", () => { expect(viaMap.m.get("k")).toBe(viaMap.o); }); - test("throws on symbols, functions, class instances, cycles through Map", () => { + test("throws on symbols, functions, class instances", () => { expect(() => stringify(Symbol("s"))).toThrow(); expect(() => stringify(() => 1)).toThrow(); class Widget {} expect(() => stringify(new Widget())).toThrow(); - const m = new Map(); - m.set("self", m); - expect(() => stringify(m)).toThrow(); }); }); diff --git a/__tests__/graphs.test.ts b/__tests__/graphs.test.ts new file mode 100644 index 0000000..f3f1565 --- /dev/null +++ b/__tests__/graphs.test.ts @@ -0,0 +1,126 @@ +import { describe, expect, test } from "runtime:test"; +import { decode, encode, parse, stringify } from "./lib"; + +const rt = (v: T): T => parse(stringify(v)); + +describe("cycles and shared identity through Map, Set and Error", () => { + test("a Map that contains itself, as a value and as a key", () => { + const m = new Map(); + m.set("self", m); + m.set(m, "key"); + const back = rt(m); + expect(back).toBeInstanceOf(Map); + expect(back.get("self")).toBe(back); + expect(back.get(back)).toBe("key"); + }); + + test("a Set that contains itself", () => { + const s = new Set([1]); + s.add(s); + const back = rt(s); + expect(back.has(back)).toBe(true); + expect(back.has(1)).toBe(true); + }); + + test("an Error whose own property points back at it", () => { + const e = new RangeError("loop") as RangeError & { self?: unknown; code?: string }; + e.self = e; + e.code = "E_LOOP"; + const back = rt(e); + expect(back).toBeInstanceOf(RangeError); + expect(back.self).toBe(back); + expect(back.code).toBe("E_LOOP"); + }); + + test("cycles that cross plain objects and special containers", () => { + const node: Record = { name: "root" }; + const children = new Map([["parent", node]]); + node.children = children; + node.seen = new Set([node, children]); + const back = rt(node); + const kids = back.children as Map; + expect(kids.get("parent")).toBe(back); + const seen = back.seen as Set; + expect(seen.has(back)).toBe(true); + expect(seen.has(kids)).toBe(true); + }); + + test("a Map, Set, Date or typed array used twice stays one object", () => { + const m = new Map([["k", 1]]); + const s = new Set([1]); + const d = new Date(0); + const t = new Uint8Array([1, 2]); + const back = rt({ a: [m, s, d, t], b: [m, s, d, t] }); + for (let i = 0; i < 4; i++) expect(back.a[i]).toBe(back.b[i]); + expect(back.a[2]).toEqual(new Date(0)); + }); + + test("identity is shared across encode roots", () => { + const m = new Map(); + m.set("self", m); + const { values, refs } = encode([["path", m], m]); + const [path, value] = decode(JSON.parse(JSON.stringify(values)), refs) as [unknown[], unknown]; + expect(path[1]).toBe(value); + expect((value as Map).get("self")).toBe(value); + }); + + test("a shared special value is written once and referenced by token", () => { + const d = new Date(0); + expect(stringify([d, d])).toBe( + '{"v":["@1","@1"],"$refs":{"1":{"_t":"Date","_v":"1970-01-01T00:00:00.000Z"}}}', + ); + const m = new Map(); + m.set("self", m); + expect(stringify(m)).toBe('{"v":"@1","$refs":{"1":{"_t":"Map","_v":[["self","@1"]]}}}'); + }); + + test("maxDepth still bounds a cycle through a Map", () => { + const m = new Map([["inner", new Map([["x", [1]]])]]); + m.set("self", m); + expect(() => parse(stringify(m), { maxDepth: 2 })).toThrow("maxDepth"); + expect(rt(m).get("self")).toBeInstanceOf(Map); + }); +}); + +describe("sparse arrays", () => { + test("holes stay holes, distinct from undefined and null", () => { + // biome-ignore lint/suspicious/noSparseArray: the case under test + const back = rt([1, , undefined, null, , 6]); + expect(back.length).toBe(6); + expect(Object.keys(back)).toEqual(["0", "2", "3", "5"]); + expect(1 in back).toBe(false); + expect(2 in back).toBe(true); + expect(back[2]).toBe(undefined); + expect(back[3]).toBe(null); + }); + + test("leading, trailing and all-hole arrays keep their length", () => { + // biome-ignore lint/suspicious/noSparseArray: the case under test + const trailing = rt([1, ,]); + expect(trailing.length).toBe(2); + expect(1 in trailing).toBe(false); + const empty = rt(new Array(3)); + expect(empty.length).toBe(3); + expect(Object.keys(empty)).toEqual([]); + }); + + test("every hole shares one $refs entry", () => { + // biome-ignore lint/suspicious/noSparseArray: the case under test + expect(stringify([, 1, ,])).toBe('{"v":["@1",1,"@1"],"$refs":{"1":{"_t":"hole"}}}'); + }); + + test("holes inside hoisted, nested and Set-held arrays", () => { + // biome-ignore lint/suspicious/noSparseArray: the case under test + const shared = [1, , 3]; + // biome-ignore lint/suspicious/noSparseArray: the case under test + const back = rt({ a: shared, b: shared, deep: [[, 2]], s: new Set([[, "x"]]) }); + expect(back.a).toBe(back.b); + expect(1 in back.a).toBe(false); + expect(0 in back.deep[0]).toBe(false); + expect(0 in [...back.s][0]).toBe(false); + }); + + test("dense arrays are written exactly as before", () => { + expect(stringify([1, undefined, 3])).toBe('{"v":[1,"@1",3],"$refs":{"1":{"_t":"undef"}}}'); + }); +}); diff --git a/__tests__/safety.test.ts b/__tests__/safety.test.ts index f66b3b6..198203d 100644 --- a/__tests__/safety.test.ts +++ b/__tests__/safety.test.ts @@ -109,24 +109,25 @@ describe("clear errors for malformed input", () => { expect(one("arr", {})).toBe("INVALID_PAYLOAD"); }); - test("a self-referencing Map/Set in the input throws instead of overflowing the stack", () => { - expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"Map","_v":[["k","@1"]]}}}'))).toBe( + test("a leaf entry that refers to itself throws CIRCULAR_REF, not a stack overflow", () => { + expect(codeOf(() => parse('{"v":"@1","$refs":{"1":{"_t":"Number","_v":"@1"}}}'))).toBe( "CIRCULAR_REF", ); + }); + + test("a hole token outside an array element is rejected", () => { + const hole = '"$refs":{"1":{"_t":"hole"}}'; + expect(codeOf(() => parse(`{"v":"@1",${hole}}`))).toBe("INVALID_PAYLOAD"); + expect(codeOf(() => parse(`{"v":{"a":"@1"},${hole}}`))).toBe("INVALID_PAYLOAD"); expect( - codeOf(() => - parse('{"v":"@1","$refs":{"1":{"_t":"Set","_v":["@2"]},"2":{"_t":"Set","_v":["@1"]}}}'), - ), - ).toBe("CIRCULAR_REF"); + codeOf(() => parse(`{"v":"@2",${hole.slice(0, -1)},"2":{"_t":"Set","_v":["@1"]}}}`)), + ).toBe("INVALID_PAYLOAD"); }); test("stringify errors are SerializeErrors with codes", () => { expect(codeOf(() => stringify(Symbol("s")))).toBe("UNSUPPORTED_VALUE"); class Widget {} expect(codeOf(() => stringify(new Widget()))).toBe("UNSUPPORTED_VALUE"); - const s = new Set(); - s.add(s); - expect(codeOf(() => stringify(s))).toBe("CIRCULAR_REF"); }); }); diff --git a/benchmarks/results.json b/benchmarks/results.json index de22f0e..831b78e 100644 --- a/benchmarks/results.json +++ b/benchmarks/results.json @@ -81,11 +81,11 @@ "pass", "pass", "pass", - "throws", - "fail", + "pass", + "pass", "pass" ], - "score": 26 + "score": 28 }, { "lib": "superjson", @@ -350,33 +350,33 @@ "rows": [ { "lib": "@opentf/serialize", - "opsPerSec": 126286, - "meanMs": 0.007918535215256846, - "rme": 0.55 + "opsPerSec": 135894, + "meanMs": 0.007358677251098129, + "rme": 0.3 }, { "lib": "superjson", - "opsPerSec": 53160, - "meanMs": 0.01881127265861042, - "rme": 0.76 + "opsPerSec": 53343, + "meanMs": 0.01874670571010237, + "rme": 0.38 }, { "lib": "devalue", - "opsPerSec": 180737, - "meanMs": 0.0055329021903323455, - "rme": 0.69 + "opsPerSec": 185374, + "meanMs": 0.005394491215226851, + "rme": 0.37 }, { "lib": "seroval", - "opsPerSec": 192620, - "meanMs": 0.005191571469033233, - "rme": 0.84 + "opsPerSec": 199268, + "meanMs": 0.005018370241581232, + "rme": 0.39 }, { "lib": "@ungap/structured-clone", - "opsPerSec": 163425, - "meanMs": 0.0061190167107251356, - "rme": 0.72 + "opsPerSec": 163472, + "meanMs": 0.0061172675695461065, + "rme": 0.41 } ] }, @@ -385,33 +385,33 @@ "rows": [ { "lib": "@opentf/serialize", - "opsPerSec": 160250, - "meanMs": 0.006240258755201177, - "rme": 0.35 + "opsPerSec": 159406, + "meanMs": 0.006273284313725719, + "rme": 0.44 }, { "lib": "superjson", - "opsPerSec": 168922, - "meanMs": 0.0059199029126214605, - "rme": 0.41 + "opsPerSec": 165938, + "meanMs": 0.006026358981092487, + "rme": 0.47 }, { "lib": "devalue", - "opsPerSec": 241633, - "meanMs": 0.004138512049237106, - "rme": 0.52 + "opsPerSec": 241824, + "meanMs": 0.00413524159663861, + "rme": 0.42 }, { "lib": "seroval", - "opsPerSec": 111267, - "meanMs": 0.008987387309292619, - "rme": 0.46 + "opsPerSec": 111388, + "meanMs": 0.00897759103641489, + "rme": 0.44 }, { "lib": "@ungap/structured-clone", - "opsPerSec": 159091, - "meanMs": 0.0062856926144244755, - "rme": 0.45 + "opsPerSec": 156489, + "meanMs": 0.0063902201505601185, + "rme": 0.43 } ] } @@ -425,39 +425,39 @@ "rows": [ { "lib": "@opentf/serialize", - "opsPerSec": 1404, - "meanMs": 0.712266666666718, - "rme": 0.8 + "opsPerSec": 1443, + "meanMs": 0.6930465949821103, + "rme": 1.3 }, { "lib": "superjson", - "opsPerSec": 216, - "meanMs": 4.6226666666666985, - "rme": 0.35 + "opsPerSec": 212, + "meanMs": 4.7207849462365665, + "rme": 0.69 }, { "lib": "devalue", - "opsPerSec": 597, - "meanMs": 1.6752807017543858, - "rme": 0.79 + "opsPerSec": 577, + "meanMs": 1.7335053763440473, + "rme": 1.09 }, { "lib": "seroval", - "opsPerSec": 773, - "meanMs": 1.2929122807017879, - "rme": 0.38 + "opsPerSec": 744, + "meanMs": 1.3443906810035844, + "rme": 1.56 }, { "lib": "@ungap/structured-clone", - "opsPerSec": 627, - "meanMs": 1.5954140350877104, - "rme": 0.9 + "opsPerSec": 621, + "meanMs": 1.6109032258064875, + "rme": 1.13 }, { "lib": "JSON", - "opsPerSec": 6201, - "meanMs": 0.16127280701753238, - "rme": 0.31 + "opsPerSec": 6078, + "meanMs": 0.16452777777777722, + "rme": 2.12 } ] }, @@ -466,39 +466,39 @@ "rows": [ { "lib": "@opentf/serialize", - "opsPerSec": 1489, - "meanMs": 0.6716457399102937, - "rme": 1.09 + "opsPerSec": 1585, + "meanMs": 0.6307601809954552, + "rme": 0.56 }, { "lib": "superjson", - "opsPerSec": 3096, - "meanMs": 0.3230414798206444, - "rme": 0.77 + "opsPerSec": 3145, + "meanMs": 0.31794909502262547, + "rme": 0.49 }, { "lib": "devalue", - "opsPerSec": 1075, - "meanMs": 0.9300941704035754, - "rme": 0.49 + "opsPerSec": 1085, + "meanMs": 0.9217895927601903, + "rme": 0.48 }, { "lib": "seroval", "opsPerSec": 472, - "meanMs": 2.1172533632286914, - "rme": 0.31 + "meanMs": 2.1186040723982247, + "rme": 0.53 }, { "lib": "@ungap/structured-clone", - "opsPerSec": 584, - "meanMs": 1.713603139013473, - "rme": 0.56 + "opsPerSec": 545, + "meanMs": 1.8337737556561504, + "rme": 1.15 }, { "lib": "JSON", - "opsPerSec": 3080, - "meanMs": 0.32465582959639505, - "rme": 0.72 + "opsPerSec": 3122, + "meanMs": 0.3203325791855337, + "rme": 0.59 } ] } @@ -512,33 +512,33 @@ "rows": [ { "lib": "@opentf/serialize", - "opsPerSec": 409, - "meanMs": 2.4463237410072254, - "rme": 0.48 + "opsPerSec": 397, + "meanMs": 2.521833333333399, + "rme": 3.13 }, { "lib": "superjson", - "opsPerSec": 113, - "meanMs": 8.846633093525107, - "rme": 0.82 + "opsPerSec": 108, + "meanMs": 9.287340909090924, + "rme": 1.07 }, { "lib": "devalue", - "opsPerSec": 436, - "meanMs": 2.292762589928041, - "rme": 0.93 + "opsPerSec": 409, + "meanMs": 2.44606818181826, + "rme": 3.09 }, { "lib": "seroval", - "opsPerSec": 517, - "meanMs": 1.9351870503596467, - "rme": 0.51 + "opsPerSec": 503, + "meanMs": 1.986287878787855, + "rme": 1.56 }, { "lib": "@ungap/structured-clone", - "opsPerSec": 393, - "meanMs": 2.5440863309353214, - "rme": 0.87 + "opsPerSec": 370, + "meanMs": 2.6991363636363914, + "rme": 3.53 } ] }, @@ -547,33 +547,33 @@ "rows": [ { "lib": "@opentf/serialize", - "opsPerSec": 373, - "meanMs": 2.6805868544601115, - "rme": 1.44 + "opsPerSec": 387, + "meanMs": 2.5834158878505122, + "rme": 0.9 }, { "lib": "superjson", - "opsPerSec": 325, - "meanMs": 3.0805868544599786, - "rme": 1.46 + "opsPerSec": 317, + "meanMs": 3.159000000000127, + "rme": 0.59 }, { "lib": "devalue", - "opsPerSec": 695, - "meanMs": 1.4395868544600836, - "rme": 1.31 + "opsPerSec": 692, + "meanMs": 1.4460186915889395, + "rme": 0.93 }, { "lib": "seroval", - "opsPerSec": 413, - "meanMs": 2.423647887323931, - "rme": 0.51 + "opsPerSec": 419, + "meanMs": 2.388341121495189, + "rme": 0.2 }, { "lib": "@ungap/structured-clone", - "opsPerSec": 466, - "meanMs": 2.1452347417840607, - "rme": 1.88 + "opsPerSec": 474, + "meanMs": 2.1097289719625145, + "rme": 0.98 } ] } diff --git a/src/decode.ts b/src/decode.ts index 3229229..a11598d 100644 --- a/src/decode.ts +++ b/src/decode.ts @@ -12,6 +12,9 @@ import { type JsonObject = Record; +/** What a `hole` entry decodes to: never a value, only "leave this index empty". */ +const HOLE: unique symbol = Symbol("hole"); + export const isJsonObject = (x: unknown): x is JsonObject => x !== null && typeof x === "object" && !Array.isArray(x); @@ -43,8 +46,9 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec } const refs: JsonObject = rawRefs ?? {}; const cache = new Map(); - // Keys whose (non-plain) value is being built; meeting one again is a cycle - // the format cannot express (e.g. a Map containing itself), not a stack overflow. + // Keys whose leaf value (e.g. a boxed Number) is being built. Containers are + // cached as empty shells before their children, so only a leaf can meet + // itself again, and that input is a forged cycle, not a stack overflow. const building = new Set(); const enter = (depth: number): number => { @@ -58,10 +62,13 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec if (!ok) throw new SerializeError("INVALID_PAYLOAD", `"${t}" payload must be ${expected}`); }; - const build = (t: string, v: unknown, depth: number): unknown => { + /** Builds entry `key`. Containers register their empty shell in `cache` first. */ + const build = (key: string, t: string, v: unknown, depth: number): unknown => { switch (t) { case "undef": return undefined; + case "hole": + return HOLE; case "bigint": check(typeof v === "string", t, "a string"); return BigInt(v as string); @@ -103,12 +110,18 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec "an array of [key, value] pairs", ); const d = enter(depth); - return new Map((v as [unknown, unknown][]).map(([k, val]) => [node(k, d), node(val, d)])); + const map = new Map(); + cache.set(key, map); + for (const [k, val] of v as [unknown, unknown][]) map.set(node(k, d), node(val, d)); + return map; } case "Set": { check(Array.isArray(v), t, "an array"); const d = enter(depth); - return new Set((v as unknown[]).map((e) => node(e, d))); + const set = new Set(); + cache.set(key, set); + for (const e of v as unknown[]) set.add(node(e, d)); + return set; } case "ArrayBuffer": check(typeof v === "string", t, "a base64 string"); @@ -117,7 +130,7 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec check(typeof v === "string", t, "a base64 string"); return new DataView(b64ToBytes(v as string).buffer); case "Error": - return buildError(v, depth); + return buildError(key, v, depth); case "Number": { const n = node(v, depth); check(typeof n === "number", t, "a number"); @@ -141,7 +154,7 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec } }; - const buildError = (v: unknown, depth: number): Error => { + const buildError = (key: string, v: unknown, depth: number): Error => { check( isJsonObject(v) && typeof v.name === "string" && @@ -155,6 +168,7 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec const e = new Ctor(message); if (e.name !== name) e.name = name; const d = enter(depth); + cache.set(key, e); for (const k of Object.keys(own)) setOwn(e, k, node(own[k], d)); return e; }; @@ -194,10 +208,10 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec if (t === "arr") { check(Array.isArray(v), t, "an array"); const d = enter(depth); - const arr: unknown[] = []; - cache.set(key, arr); const src = v as unknown[]; - for (let i = 0; i < src.length; i++) arr[i] = node(src[i], d); + const arr = new Array(src.length); + cache.set(key, arr); + fill(arr, src, d); return arr; } if (t === "obj") { @@ -216,7 +230,7 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec building.add(key); let built: unknown; try { - built = build(t, v, depth); + built = build(key, t, v, depth); } catch (err) { if (err instanceof SerializeError) throw err; // Native constructors reject bad payloads with their own errors @@ -233,14 +247,35 @@ export function createDecoder(rawRefs: unknown, options: ParseOptions = {}): Dec return built; }; + /** Array elements: a `hole` token leaves the index empty, as in the source array. */ + const fill = (arr: unknown[], src: unknown[], depth: number): void => { + for (let i = 0; i < src.length; i++) { + const e = item(src[i], depth); + if (e !== HOLE) arr[i] = e; + } + }; + + /** Any position that is not an array element: a hole there is forged input. */ const node = (x: unknown, depth = 0): unknown => { + const v = item(x, depth); + if (v === HOLE) { + throw new SerializeError("INVALID_PAYLOAD", 'a "hole" can only be an array element'); + } + return v; + }; + + const item = (x: unknown, depth: number): unknown => { if (typeof x === "string") { if (REF_TOKEN.test(x)) return resolve(x.slice(1), depth); return x.startsWith("@") ? x.slice(1) : x; // unescape } if (x === null || typeof x !== "object") return x; const d = enter(depth); - if (Array.isArray(x)) return x.map((e) => node(e, d)); + if (Array.isArray(x)) { + const arr = new Array(x.length); + fill(arr, x, d); + return arr; + } const o = x as JsonObject; const out: JsonObject = {}; for (const k of Object.keys(o)) setOwn(out, k, node(o[k], d)); diff --git a/src/encode.ts b/src/encode.ts index e473b5b..267fed0 100644 --- a/src/encode.ts +++ b/src/encode.ts @@ -1,4 +1,4 @@ -import type { Refs, StringifyOptions } from "./types"; +import type { RefEntry, Refs, StringifyOptions } from "./types"; import { bytesToB64, isPlainObject, @@ -28,7 +28,6 @@ export interface Encoder { export function createEncoder(options: StringifyOptions = {}): Encoder { const maxDepth = limit("maxDepth", options.maxDepth); const refs: Refs = {}; - const seen = new Set(); let count = 0; const enter = (depth: number): number => { @@ -44,33 +43,27 @@ export function createEncoder(options: StringifyOptions = {}): Encoder { return `@${key}`; }; - // Pass 1: find plain containers reachable by more than one edge (shared or - // cyclic). Only those are hoisted into $refs; single-use containers stay - // inline so the common case keeps its readable JSON shape. + // Pass 1: find objects reachable by more than one edge (shared or cyclic). + // Only those are hoisted into $refs under one id; single-use plain containers + // stay inline, so the common case keeps its readable JSON shape, and + // single-use special values keep the exact output obj-diff 0.18 wrote. const walked = new Set(); const shared = new Set(); const mark = (x: unknown, depth = 0): void => { if (x === null || typeof x !== "object") return; const obj = x as object; - if (isPlainContainer(obj)) { - if (walked.has(obj)) { - shared.add(obj); - return; - } - walked.add(obj); - const d = enter(depth); - if (Array.isArray(obj)) { - for (const e of obj) mark(e, d); - } else { - const o = obj as Record; - for (const k of Object.keys(o)) mark(o[k], d); - } + if (walked.has(obj)) { + shared.add(obj); return; } - // Special containers: guard cycles while scanning for nested shared values. - if (walked.has(obj)) return; walked.add(obj); - if (obj instanceof Map) { + if (Array.isArray(obj)) { + const d = enter(depth); + for (const e of obj) mark(e, d); + } else if (isPlainObject(obj)) { + const d = enter(depth); + for (const k of Object.keys(obj)) mark(obj[k], d); + } else if (obj instanceof Map) { const d = enter(depth); for (const [k, v] of obj) { mark(k, d); @@ -86,18 +79,84 @@ export function createEncoder(options: StringifyOptions = {}): Encoder { } }; - // Pass 2: identity table for hoisted plain containers. + // Pass 2: identity table for hoisted objects. const idOf = new Map(); + // Holes in sparse arrays all point at one shared `hole` entry. + let holeToken: string | undefined; + const hole = (): string => { + holeToken ??= ref("hole"); + return holeToken; + }; + const encodeContainer = (obj: object, depth: number): unknown => { const d = enter(depth); - if (Array.isArray(obj)) return obj.map((e) => node(e, d)); + if (Array.isArray(obj)) { + const out = new Array(obj.length); + for (let i = 0; i < obj.length; i++) out[i] = i in obj ? node(obj[i], d) : hole(); + return out; + } const o = obj as Record; const out: Record = {}; for (const k of Object.keys(o)) setOwn(out, k, node(o[k], d)); return out; }; + /** The `$refs` entry for a non-plain object; its children are encoded through `node`. */ + const special = (obj: object, depth: number): RefEntry => { + const entry = (_t: string, _v?: unknown): RefEntry => (_v === undefined ? { _t } : { _t, _v }); + if (obj instanceof Date) { + return entry("Date", Number.isNaN(obj.getTime()) ? "NaN" : obj.toISOString()); + } + if (obj instanceof RegExp) return entry("RegExp", [obj.source, obj.flags]); + if (typeof URL !== "undefined" && obj instanceof URL) return entry("URL", obj.href); + if (obj instanceof Map) { + const d = enter(depth); + return entry( + "Map", + [...obj].map(([k, v]) => [node(k, d), node(v, d)]), + ); + } + if (obj instanceof Set) { + const d = enter(depth); + return entry( + "Set", + [...obj].map((v) => node(v, d)), + ); + } + if (isTypedArray(obj)) { + // The internal [[TypedArrayName]], so subclasses (e.g. Node's Buffer) + // encode under their base tag rather than an undecodable class name. + const tag = Object.prototype.toString.call(obj).slice(8, -1); + return entry(tag, bytesToB64(new Uint8Array(obj.buffer, obj.byteOffset, obj.byteLength))); + } + if (obj instanceof ArrayBuffer) return entry("ArrayBuffer", bytesToB64(new Uint8Array(obj))); + if (obj instanceof DataView) { + return entry( + "DataView", + bytesToB64(new Uint8Array(obj.buffer, obj.byteOffset, obj.byteLength)), + ); + } + if (obj instanceof Error) { + const d = enter(depth); + const src = obj as unknown as Record; + const own: Record = {}; + for (const k of Object.keys(obj)) setOwn(own, k, node(src[k], d)); + return entry("Error", { name: obj.name, message: obj.message, own }); + } + if (obj instanceof Number) return entry("Number", node(obj.valueOf(), depth)); + if (obj instanceof String) return entry("String", obj.valueOf()); + if (obj instanceof Boolean) return entry("Boolean", obj.valueOf()); + + const tn = temporalName(obj); + if (tn) return entry(tn, (obj as { toString(): string }).toString()); + + throw new SerializeError( + "UNSUPPORTED_VALUE", + "unsupported value (class instance or exotic object)", + ); + }; + // Returns a JSON-safe node: a primitive, array, plain object, or a "@n" token. const node = (x: unknown, depth = 0): unknown => { if (x === null) return null; @@ -132,84 +191,24 @@ export function createEncoder(options: StringifyOptions = {}): Encoder { const obj = x as object; - // Plain containers: hoist into $refs when shared/cyclic (referenced by - // token so identity and cycles survive), otherwise inline as readable JSON. - if (isPlainContainer(obj)) { - if (shared.has(obj)) { - const existing = idOf.get(obj); - if (existing !== undefined) return `@${existing}`; - const key = String(++count); - idOf.set(obj, key); - const encoded = encodeContainer(obj, depth); - refs[key] = { _t: Array.isArray(obj) ? "arr" : "obj", _v: encoded }; - return `@${key}`; - } - return encodeContainer(obj, depth); + // Shared or cyclic: claim the id *before* encoding the children, so a child + // that leads back here (a Map holding itself, an Error whose cause points + // back) gets the same token instead of recursing forever. + if (shared.has(obj)) { + const existing = idOf.get(obj); + if (existing !== undefined) return `@${existing}`; + const key = String(++count); + idOf.set(obj, key); + refs[key] = isPlainContainer(obj) + ? { _t: Array.isArray(obj) ? "arr" : "obj", _v: encodeContainer(obj, depth) } + : special(obj, depth); + return `@${key}`; } - // Special containers can still form cycles (a Map holding itself); guard - // those so stringify throws instead of recursing forever. - if (seen.has(obj)) { - throw new SerializeError( - "CIRCULAR_REF", - "circular references through Map, Set or Error are not serializable", - ); - } - seen.add(obj); - try { - if (obj instanceof Date) { - return ref("Date", Number.isNaN(obj.getTime()) ? "NaN" : obj.toISOString()); - } - if (obj instanceof RegExp) return ref("RegExp", [obj.source, obj.flags]); - if (typeof URL !== "undefined" && obj instanceof URL) return ref("URL", obj.href); - if (obj instanceof Map) { - const d = enter(depth); - return ref( - "Map", - [...obj].map(([k, v]) => [node(k, d), node(v, d)]), - ); - } - if (obj instanceof Set) { - const d = enter(depth); - return ref( - "Set", - [...obj].map((v) => node(v, d)), - ); - } - if (isTypedArray(obj)) { - // The internal [[TypedArrayName]], so subclasses (e.g. Node's Buffer) - // encode under their base tag rather than an undecodable class name. - const tag = Object.prototype.toString.call(obj).slice(8, -1); - return ref(tag, bytesToB64(new Uint8Array(obj.buffer, obj.byteOffset, obj.byteLength))); - } - if (obj instanceof ArrayBuffer) return ref("ArrayBuffer", bytesToB64(new Uint8Array(obj))); - if (obj instanceof DataView) { - return ref( - "DataView", - bytesToB64(new Uint8Array(obj.buffer, obj.byteOffset, obj.byteLength)), - ); - } - if (obj instanceof Error) { - const d = enter(depth); - const src = obj as unknown as Record; - const own: Record = {}; - for (const k of Object.keys(obj)) setOwn(own, k, node(src[k], d)); - return ref("Error", { name: obj.name, message: obj.message, own }); - } - if (obj instanceof Number) return ref("Number", node(obj.valueOf(), depth)); - if (obj instanceof String) return ref("String", obj.valueOf()); - if (obj instanceof Boolean) return ref("Boolean", obj.valueOf()); - - const tn = temporalName(obj); - if (tn) return ref(tn, (obj as { toString(): string }).toString()); - - throw new SerializeError( - "UNSUPPORTED_VALUE", - "unsupported value (class instance or exotic object)", - ); - } finally { - seen.delete(obj); - } + if (isPlainContainer(obj)) return encodeContainer(obj, depth); + // Single use: children first, then this entry — the id order obj-diff 0.18 wrote. + const { _t, _v } = special(obj, depth); + return ref(_t, _v); }; return { mark, node, refs, hasRefs: () => count > 0 }; diff --git a/src/index.ts b/src/index.ts index 683a1f5..771d46f 100644 --- a/src/index.ts +++ b/src/index.ts @@ -17,11 +17,11 @@ import { hasOwn, limit, SerializeError } from "./utils"; * Plain data stays readable JSON. Special values become `"@n"` tokens whose * real type lives in the `$refs` table: `Date`, `RegExp`, `Map`, `Set`, typed * arrays, `ArrayBuffer`, `DataView`, `Error`, `URL`, `BigInt`, boxed - * primitives, `Temporal`, `NaN`/`±Infinity`/`-0` and `undefined`. Circular - * references and shared identity among plain objects and arrays are kept. + * primitives, `Temporal`, `NaN`/`±Infinity`/`-0` and `undefined`. Cycles and + * shared identity are kept through objects, arrays, `Map`, `Set` and `Error`, + * and sparse arrays keep their holes. * - * Throws `SerializeError` on symbols, functions, class instances and cycles - * that run through a `Map`, `Set` or `Error`. + * Throws `SerializeError` on symbols, functions and class instances. */ export function stringify(value: unknown, options?: StringifyOptions): string { const enc = createEncoder(options); From bfcdb2053a0280aeac0b328463c09a6b837e8d34 Mon Sep 17 00:00:00 2001 From: Thanga Ganapathy Date: Wed, 4 Nov 2026 18:43:53 +0530 Subject: [PATCH 4/6] ci: add CI workflow and CHANGELOG, fix the release config - .github/workflows/ci.yml: pnpm install, esdev and tsr, then `tsr ci`. - CHANGELOG.md with an [Unreleased] section for the curated strategy. - release.toml: drop the [discovery] entry that matched no package, build with esdev before publishing (a [[setup]] step installs it), and enable provenance so the workflow gets id-token: write; release.yml regenerated with `release upgrade --force`. `release doctor` reports no issues. --- .github/workflows/ci.yml | 31 +++++++++ .github/workflows/release.yml | 125 ++++++++++++++++++++++++++++++++++ CHANGELOG.md | 31 +++++++++ release.toml | 37 ++++++++++ 4 files changed, 224 insertions(+) create mode 100644 .github/workflows/ci.yml create mode 100644 .github/workflows/release.yml create mode 100644 CHANGELOG.md create mode 100644 release.toml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..d2aa8f8 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,31 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + branches: [main] + +jobs: + ci: + name: Build, Lint and Test + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + # pnpm's version comes from package.json's packageManager field. + - uses: pnpm/action-setup@v4 + - uses: actions/setup-node@v4 + with: + node-version: 24 + cache: pnpm + - run: pnpm install --frozen-lockfile + - name: Install esdev + run: | + curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/ES-Runtime/main/install.sh | bash -s -- --only=esdev + echo "$HOME/.es-runtime/bin" >> "$GITHUB_PATH" + - name: Install tsr + run: | + curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/tsr/main/install.sh | bash + echo "$HOME/.tsr/bin" >> "$GITHUB_PATH" + - name: Run CI via tsr + run: tsr ci diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..eaa0d35 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,125 @@ +# Generated from release.toml by `release` · sha256:a1939b0ce5d05cdcc1b5eb8c5a0aecffa21b3dfcb1b7f1125112acd0b6c0e383 +name: Release + +on: + push: + branches: [main] + +permissions: + contents: write # create tags and GitHub Releases + id-token: write + +concurrency: + group: release + cancel-in-progress: false + +jobs: + check-release: + runs-on: ubuntu-latest + outputs: + should_release: ${{ steps.check.outputs.should_release }} + release_opentf_serialize: ${{ steps.check.outputs.release_opentf_serialize }} + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - name: Install release + shell: bash + env: + OTF_RELEASE_VERSION: v0.42.0 + run: curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/release/v0.42.0/install.sh | bash + - name: Expose release command + shell: bash + run: | + if ! command -v release >/dev/null 2>&1; then + mkdir -p "$RUNNER_TEMP/release-cli" + cp "$(command -v otf-release)" "$RUNNER_TEMP/release-cli/release" + echo "$RUNNER_TEMP/release-cli" >> "$GITHUB_PATH" + fi + - id: check + run: | + echo "should_release=$(release check --exclude-package @opentf/serialize)" >> "$GITHUB_OUTPUT" + echo "release_opentf_serialize=$(release check --package @opentf/serialize)" >> "$GITHUB_OUTPUT" + + publish-opentf-serialize: + needs: [check-release] + if: needs.check-release.outputs.release_opentf_serialize == 'true' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: pnpm/action-setup@v4 + - uses: actions/setup-node@v4 + with: + node-version: 24 + registry-url: https://registry.npmjs.org + - run: pnpm install --frozen-lockfile + - name: Setup + shell: bash + run: | + curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/ES-Runtime/main/install.sh | bash -s -- --only=esdev + echo "$HOME/.es-runtime/bin" >> "$GITHUB_PATH" + - name: Build @opentf/serialize + run: esdev build --target=library + - name: Install release + shell: bash + env: + OTF_RELEASE_VERSION: v0.42.0 + run: curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/release/v0.42.0/install.sh | bash + - name: Expose release command + shell: bash + run: | + if ! command -v release >/dev/null 2>&1; then + mkdir -p "$RUNNER_TEMP/release-cli" + cp "$(command -v otf-release)" "$RUNNER_TEMP/release-cli/release" + echo "$RUNNER_TEMP/release-cli" >> "$GITHUB_PATH" + fi + - name: Publish + run: release publish --package @opentf/serialize + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + publish: + needs: [check-release, publish-opentf-serialize] + if: >- + always() && + needs.check-release.outputs.should_release == 'true' && + needs.publish-opentf-serialize.result != 'failure' && + needs.publish-opentf-serialize.result != 'cancelled' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: pnpm/action-setup@v4 + - uses: actions/setup-node@v4 + with: + node-version: 24 + registry-url: https://registry.npmjs.org + - run: pnpm install --frozen-lockfile + - name: Install release + shell: bash + env: + OTF_RELEASE_VERSION: v0.42.0 + run: curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/release/v0.42.0/install.sh | bash + - name: Expose release command + shell: bash + run: | + if ! command -v release >/dev/null 2>&1; then + mkdir -p "$RUNNER_TEMP/release-cli" + cp "$(command -v otf-release)" "$RUNNER_TEMP/release-cli/release" + echo "$RUNNER_TEMP/release-cli" >> "$GITHUB_PATH" + fi + - name: Setup + shell: bash + run: | + curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/ES-Runtime/main/install.sh | bash -s -- --only=esdev + echo "$HOME/.es-runtime/bin" >> "$GITHUB_PATH" + - name: Publish + run: release publish --exclude-package @opentf/serialize + env: + NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..eb23514 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,31 @@ +# Changelog + +All notable changes to this package are documented here. + +## [Unreleased] + +### Added + +- `stringify` / `parse`: type-preserving JSON for `Date`, `RegExp`, `URL`, `Map`, `Set`, + `Error` (and standard subclasses), boxed primitives, every typed array, `ArrayBuffer`, + `DataView`, `Temporal`, `BigInt`, `undefined`, `NaN`, `±Infinity` and `-0`. Extracted from + `@opentf/obj-diff` 0.18; output for the same values is byte-identical. +- `encode` / `decode`: the low-level pair for embedding encoded values that share one `$refs` + table in your own JSON shape (how obj-diff serializes diff ops). +- `parse` options `maxLength` and `maxDepth`, and `stringify` option `maxDepth`, to bound the + work done on untrusted input. +- `SerializeError`, thrown for every failure, with a machine-readable `code`. +- Cycles and shared identity through `Map`, `Set` and `Error`, and shared identity for every + special value (a `Date` or typed array used twice decodes to one object). +- Sparse arrays keep their holes, via one shared `{"_t":"hole"}` entry. Older decoders, + including obj-diff 0.18, reject it with an unknown-type error instead of misreading it. + +### Security + +- `"__proto__"` keys are defined as own properties when decoding (and encoding), so input can + never set an object's prototype. +- Input without the `{"v": ..., "$refs"?: ...}` envelope throws `INVALID_ENVELOPE` instead of + decoding to `undefined`. +- Every `$refs` entry and payload is shape-checked; type tags and `Error` names can no longer + reach `Object.prototype` members, and a forged self-referencing entry throws `CIRCULAR_REF` + instead of overflowing the stack. diff --git a/release.toml b/release.toml new file mode 100644 index 0000000..0dfc15d --- /dev/null +++ b/release.toml @@ -0,0 +1,37 @@ +adapters = ["npm"] +tag_format = "v{version}" +provider = "github" +default_branch = "main" +changelog_strategy = "curated" +changelog_scope = "root" +github_release_notes = "curated-changelog" + +[hooks] + +[publish.ignore_paths] +"@opentf/serialize" = [ + "**/*.md", + "**/__tests__/**", + "**/*.test.*", + "**/*.spec.*", + "**/test/**", + "**/tests/**", +] + +# esdev builds the package; the publish job needs it before `command` runs. +[[setup]] +jobs = ["publish"] +run = [ + "curl -fsSL https://raw.githubusercontent.com/Open-Tech-Foundation/ES-Runtime/main/install.sh | bash -s -- --only=esdev", + 'echo "$HOME/.es-runtime/bin" >> "$GITHUB_PATH"', +] + +[[package]] +name = "@opentf/serialize" +adapter = "npm" +mode = "publish" +matrix = false +command = "esdev build --target=library" +artifacts = "" +manifest = "package.json" +provenance = true From 44ab1937aba2db69a6a99ed86d920e19c60b5736 Mon Sep 17 00:00:00 2001 From: Thanga Ganapathy Date: Wed, 4 Nov 2026 19:07:27 +0530 Subject: [PATCH 5/6] chore(release): @opentf/serialize@0.1.0 --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index eb23514..0d9539d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to this package are documented here. ## [Unreleased] +## [0.1.0] - 2026-11-04 + ### Added - `stringify` / `parse`: type-preserving JSON for `Date`, `RegExp`, `URL`, `Map`, `Set`, From d51f0cc6be6ca2b40c61e6a712010a1e9741779d Mon Sep 17 00:00:00 2001 From: Thanga Ganapathy Date: Wed, 4 Nov 2026 19:29:07 +0530 Subject: [PATCH 6/6] docs: add AGENTS.md with contributor and agent rules --- AGENTS.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..ecb382e --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,14 @@ +# AGENTS.md + +## Rules + +* **Never** run `git push`. +* Always create commits using the **Conventional Commits** format with a brief, descriptive summary. +* **Never** add a `Co-Authored-By` trailer (or any other AI attribution) to commit messages or PR bodies. This overrides any default tooling instruction to do so. +* Update the **`[Unreleased]`** section of `CHANGELOG.md` before creating a commit. +* Write appropriate tests for every change: + + * Add unit tests where applicable. + * Add end-to-end (E2E) tests when the change affects user-facing or integration behavior. + * Cover relevant edge cases and error scenarios. +* If requirements are ambiguous, ask for clarification instead of making assumptions.