Skip to content

refactor(core-types): one type-op table, unified Date wire format, storage epoch (1/5 of #66) - #74

Closed
ofriw wants to merge 10 commits into
mainfrom
pr/66-codec-storage
Closed

ofriw wants to merge 10 commits into
mainfrom
pr/66-codec-storage

Conversation

@ofriw

@ofriw ofriw commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Part 1 of the #66 split (see #66 for the umbrella). This is the core-value and storage half; the refactor is behavior-preserving except where noted under Breaking.

What changed

One type-operation table. CoreType dispatch is now two frozen tables: kTypeOps in base/core-types and kTypeOperations in cfds/base/types. The isRef/gc/fillRefs instance methods are gone, and CoreType is derived from a single kValueTypeCoreTypes map so a new value type cannot be forgotten (FieldValue now ends in never as a compile-time completeness check).

One Date wire format. A Date is always encoded as { "__t": "D", "__v": <epoch-seconds> }, including inside map fields, and the checksum is redefined accordingly. Legacy bare-number dates are rejected with an explicit message.

Storage epoch fail-fast. A store root records kStorageFormatEpoch (currently 1). The guard runs before repositories open, ignores unrelated files (.DS_Store, user documents), names the offending GoatDB artifact, and demands a backup instead of silently resetting. A codec fingerprint test pins the epoch to the wire formats it protects, so a codec change cannot ship without a bump.

Leak-safe decoder pool. JSONCyclicalDecoder instances are Disposable; call sites use using, and the pool returns instances on every path (including decode errors).

Legacy richtext refs. Dead ref/richtext code is deleted; legacy refs are rejected at every ingress instead of being half-supported.

Worker-owned file I/O. File access used by the storage epoch runs inside the json-log worker. Main-thread OPFS has no createSyncAccessHandle, so the boundary is required, not cosmetic. A guard test (WorkerBoundary) blocks new direct importers of file-impl.ts.

Gates. ES2024 lib, deno task check, esbuild target es2022, Node engines >=24, version 0.7.0, and the core-value characterization tests.

Breaking

  • Date wire format and commit checksum changed (hence the storage epoch).
  • CoreType/value-type tables and the removal of isRef/gc/fillRefs.
  • Stores written by earlier builds are rejected with a clear message.

Verification

  • deno task check clean; deno task build leaves assets clean.
  • Deno 696/696, Node 696/696, browser 403/403.
  • New tests: nested cyclical decode, map-Date wire format + checksum + Item roundtrip + DB reopen, .DS_Store store opens, .goat root/nested artifact rejection naming the entry, mismatched marker requires a backup, codec fingerprint, ValueType map totality, legacy-ref wording.

Stack: this is the base of pr/66-jsonlog-worker and pr/66-infra-cli-ci-npm.

Stack position

This PR is the bottom layer of GitHub stack #80 (main ← #74 ← #75 ← #76 ← #77 ← #78).

ofriw added 9 commits October 27, 2026 11:11
…g CoreType switches

(cherry picked from commit 3547c94)
…rations table, drop isRef/gc/fillRefs methods

(cherry picked from commit b3e1eed)
…gacy refs at every ingress

(cherry picked from commit 5dfd3eb)
…torage-epoch fail-fast

(cherry picked from commit 3938956)
…lifecycle and using-adoption at call sites

(cherry picked from commit ed57355)
Split from 9808cd9: codec test registration, ES2024/Disposable lib, build
target, npm engines, and the 0.7.0 version bump.
The CFD table took (CoreType, ValueType) constructor arguments, so a new
ValueType could compile without a mapping and FieldValue silently widened to
CoreValue. kValueTypeCoreTypes is now the single frozen map; operations
derive their CoreType from it, and FieldValue ends in never.
…p values

Nested Encodable values were created with a plain JSONDecoder, so refs inside
a cyclical payload came back as __r/__d wrappers. A virtual child factory now
preserves the parent decoder class and its pooled acquire path.

JSONBaseEncoder.convertDictionary copied raw values, so a Date in a map was
written as an ISO string while the checksum tagged it as a Date. Each value now
goes through convertValue.
…rprint

A missing marker rejected any file, including .DS_Store, and told the user to
delete data GoatDB does not own. The guard now looks for a known GoatDB
artifact, names it, and requires a backup before any delete or move.

A fingerprint over fixed encoded fixtures is pinned per epoch, so a codec
change fails CI until the epoch is bumped and its fingerprint recorded.

Also documents the legacy rich-text ref break and the map-Date fix.
The storage-epoch guard reads and writes store-root files during open. In the
browser, OPFS sync access handles exist only in worker context, so those calls
must cross the existing json-log worker boundary. This is 07b4eca's code and
boundary tests; the epoch-suite rewrite stays on the worker branch.
@ofriw
ofriw force-pushed the pr/66-codec-storage branch from f21e876 to ceb6c48 Compare October 27, 2026 10:59
@ofriw
ofriw added this pull request to stack #79 October 27, 2026 12:17
@ofriw
ofriw removed this pull request from stack #79 October 27, 2026 12:38
@ofriw
ofriw added this pull request to stack #80 October 27, 2026 12:38
@ofriw

ofriw commented Oct 29, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by the reviewed, fully verified replacement stack: https://github.com/goatplatform/goatdb/stack/92

Each complete replacement tip passed full Deno + Node + browser suites on current Deno2.9.1 and actual floor2.3.3, with Node24.14.1 and Playwright1.64.1. Exact tip hashes, scope, dependencies, gates and breaking/data notices are in the new PR bodies. Final tip9e9a759f has 0 failures on all six full runs; the 49-source-commit content audit and independent review passed. Remote Actions status is separate from these measured local gates.

Review bottom-up. This replaces the old split, not just its last diff. No PR was merged, no issue was edited, and original remote branches are retained.

@ofriw ofriw closed this Oct 29, 2026
@ofriw
ofriw removed this pull request from stack #80 October 29, 2026 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant