diff --git a/CHANGELOG.md b/CHANGELOG.md index f418cdb2..3c0f09d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,81 +2,153 @@ All notable changes to GoatDB will be documented in this file. -The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and GoatDB adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and GoatDB adheres to +[Semantic Versioning](https://semver.org/spec/v2.0.0.html). -> **Pre-1.0 compatibility notice:** GoatDB is still pre-1.0. Backward compatibility is not guaranteed between releases; APIs, storage formats, and behavior may change without notice. +> **Pre-1.0 compatibility notice:** GoatDB is still pre-1.0. Backward +> compatibility is not guaranteed between releases; APIs, storage formats, and +> behavior may change without notice. ## [Unreleased] ### Added -- `DBInstanceConfig.repoInactivityTimeoutMs` and - `queryInactivityTimeoutMs` auto-close idle repositories and queries. Both - default to `0` (disabled). A repository closes only when it has no in-flight - leases, no external `DocumentChanged` listeners, and no open dependent - queries; `/sys/` repositories are exempt. -- Query loading is lazy: `db.query()` no longer starts a scan. Loading begins - on the first subscription (`onResultsChanged`), `onLoadingFinished`, or +- `DBInstanceConfig.repoInactivityTimeoutMs` and `queryInactivityTimeoutMs` + auto-close idle repositories and queries. Both default to `0` (disabled). A + repository closes only when it has no in-flight leases, no external + `DocumentChanged` listeners, and no open dependent queries; `/sys/` + repositories are exempt. +- Query loading is lazy: `db.query()` no longer starts a scan. Loading begins on + the first subscription (`onResultsChanged`), `onLoadingFinished`, or `loadingFinished()`; reads (`results()`, `has()`, ...) are side-effect free and do not start loading. A fresh unobserved query reports `loading === false` and, when `queryInactivityTimeoutMs` is set, is eligible for auto-close. A closed query is replaced by calling `db.query()` again. +### Fixed + +- `GoatDB.state` reports the instance lifecycle (`booting`, `ready`, `closing`, + `closed`, `failed`) and replaces the `GoatDB.ready` boolean; use + `state === 'ready'` for the old check. `GoatDB.isOpen` is the derived "accepts + work" signal. Repeated `close()` calls return the same promise, and data + operations on a closing, closed, or failed instance fail at the API boundary + with `serviceUnavailable` instead of reopening repositories. `count()`, + `keys()`, `getTrustPool()`, and `logout()` follow the same rule, so a closed + instance can no longer report an empty repo, hand out session keys, or wipe + local data. `loggedIn` reports `false` after teardown. Passive inspection + (`repository()`, `itemLoaded()`, `clientsForRepo()`, `state`, `isOpen`) stays + available. An `open()` deferred behind an in-flight repository close is + cancelled with `serviceUnavailable` when `db.close()` starts, so no repository + can outlive the terminal closed state. Mounted `useDBReady` consumers follow + `LifecycleChanged` through close. Failed bootstraps are logged by the DB + without an unhandled rejection. `readyPromise()` rejects with the original + error, while `getTrustPool()` fails closed with `serviceUnavailable` once the + instance leaves `booting`. `close()` aborts in-flight peer sync, so teardown + never waits on a stalled peer's retries or timeouts; an `open()` that loses + that race fails with `serviceUnavailable`. A failed teardown reports `failed` + and rejects `close()` instead of claiming `closed`; a later `close()` retries + the remaining teardown and settles `closed`. When `close()` aborts a + multi-peer `open()` mid-startup, clients that already armed their sync timers + are closed, so no repeating timer outlives the closed state. +- An auto-close teardown that fails leaves the repository registered and rejects + with a logged error; the next `open()` or `closeRepo()` finishes the teardown + before serving a fresh instance. +- `ManagedItem.readyPromise()` rejects when the initial load fails instead of + resolving as if the item were ready. A close during loading rejects with + `serviceUnavailable`. The rejection is handled internally, so it never + surfaces as an unhandled rejection. +- `ManagedItem` mutators (`set()`, `setMulti()`, `delete()`, the `schema` + setter, and the `isDeleted` setter) throw `serviceUnavailable` when the DB is + closed or closing, so post-close edits fail loudly instead of dropping. + ## [0.6.1] - 2026-09-01 ### Fixed -- Republish 0.6.0 as 0.6.1: npm prohibits re-publishing an unpublished version (tombstone retention in registry `time` metadata), so 0.6.1 supersedes with identical code. +- Republish 0.6.0 as 0.6.1: npm prohibits re-publishing an unpublished version + (tombstone retention in registry `time` metadata), so 0.6.1 supersedes with + identical code. ## [0.6.0] - 2026-08-30 ### Breaking Changes -- **Binary `.goat` storage format**: Commits are now stored in a compact binary format (magic byte `0x47 'G'`, little-endian commit fields, big-endian 4-byte length framing). Both `.jsonl` and binary storage are supported, with the format selected through `DBInstanceConfig.storageFormat`. Tooling that reads `.goat` files directly requires manual migration. +- **Binary `.goat` storage format**: Commits are now stored in a compact binary + format (magic byte `0x47 'G'`, little-endian commit fields, big-endian 4-byte + length framing). Both `.jsonl` and binary storage are supported, with the + format selected through `DBInstanceConfig.storageFormat`. Tooling that reads + `.goat` files directly requires manual migration. -- **Bloom-filter ancestors replaced with explicit string arrays**: The `af` and `ac` commit fields are no longer written or read. Existing commits using Bloom-filter ancestry load with empty ancestor arrays and rebuild their ancestry through new merge commits. +- **Bloom-filter ancestors replaced with explicit string arrays**: The `af` and + `ac` commit fields are no longer written or read. Existing commits using + Bloom-filter ancestry load with empty ancestor arrays and rebuild their + ancestry through new merge commits. -- **Strict `.goat` binary records**: Non-binary records in `.goat` files are rejected instead of being silently loaded. Mixed JSON/binary files require manual migration. +- **Strict `.goat` binary records**: Non-binary records in `.goat` files are + rejected instead of being silently loaded. Mixed JSON/binary files require + manual migration. -- **Ed25519 signing replaces ECDSA P-384**: Existing ECDSA-signed commits cannot be verified after upgrading and require a fresh trust pool. +- **Ed25519 signing replaces ECDSA P-384**: Existing ECDSA-signed commits cannot + be verified after upgrading and require a fresh trust pool. -- **Query cache identity changed**: Query IDs now use Murmur3 and account for relevant options and normalized source paths. Persisted caches from earlier versions may miss and be rebuilt lazily. +- **Query cache identity changed**: Query IDs now use Murmur3 and account for + relevant options and normalized source paths. Persisted caches from earlier + versions may miss and be rebuilt lazily. -- **Query live updates are enabled by default**: Queries now receive uncommitted updates from `set()`. Set `liveUpdates: false` to retain commit-only behavior. +- **Query live updates are enabled by default**: Queries now receive uncommitted + updates from `set()`. Set `liveUpdates: false` to retain commit-only behavior. -- **Public API changes**: The optional `session` parameter was removed from `Repository.leavesForKey()`, and `commitIsHighProbabilityLeaf()` was renamed to `commitIsLeaf()`. +- **Public API changes**: The optional `session` parameter was removed from + `Repository.leavesForKey()`, and `commitIsHighProbabilityLeaf()` was renamed + to `commitIsLeaf()`. -- **Item path components are limited to 39 characters**: Type, repository, item, and embed path components longer than 39 characters are rejected. +- **Item path components are limited to 39 characters**: Type, repository, item, + and embed path components longer than 39 characters are rejected. ### Added -- `GoatDB.insert()` for creating batches of new items without redundant ancestor computation. +- `GoatDB.insert()` for creating batches of new items without redundant ancestor + computation. - `liveUpdates` query configuration for selecting live or commit-only updates. - `WriteFailure` events on `GoatDB` when a commit cannot be persisted. -- Shard configuration through `maxShardCommits`, `splitThreshold`, and `minShardCommits`, with read-only access through `GoatDB.shardConfig`. +- Shard configuration through `maxShardCommits`, `splitThreshold`, and + `minShardCommits`, with read-only access through `GoatDB.shardConfig`. - Public `AppConfig.esbuildPlugins` support for build customization. ### Changed -- CSS is bundled through esbuild during compile and debug builds, including plugin processing, hashed `url()` assets, and composed sourcemaps. -- The normal-accuracy sync Bloom-filter false-positive rate is capped at `0.001` to reduce unnecessary resynchronization rounds. The low-accuracy path retains its previous setting. -- Repository authentication concurrency is captured when the repository is created, preventing inconsistent verification batching. +- CSS is bundled through esbuild during compile and debug builds, including + plugin processing, hashed `url()` assets, and composed sourcemaps. +- The normal-accuracy sync Bloom-filter false-positive rate is capped at `0.001` + to reduce unnecessary resynchronization rounds. The low-accuracy path retains + its previous setting. +- Repository authentication concurrency is captured when the repository is + created, preventing inconsistent verification batching. - The homepage, benchmarks, and documentation have been overhauled. ### Fixed -- Conflict resolution now selects the closest available lowest common ancestor and defers merging when that ancestor is unavailable, avoiding destructive fallback merges. -- Query IDs no longer collide across relevant options or source paths, explicit empty IDs are preserved, stale aliases are avoided, and generated-ID caching is bounded. -- Query ordering and `find()` now handle dates, booleans, numbers, `NaN`, and near-equal values consistently. -- Locally created commits retain monotonic creation order when timestamps collide. -- `CoroutineQueue` now preserves FIFO ordering, runs a single worker, and recovers after task failures. +- Conflict resolution now selects the closest available lowest common ancestor + and defers merging when that ancestor is unavailable, avoiding destructive + fallback merges. +- Query IDs no longer collide across relevant options or source paths, explicit + empty IDs are preserved, stale aliases are avoided, and generated-ID caching + is bounded. +- Query ordering and `find()` now handle dates, booleans, numbers, `NaN`, and + near-equal values consistently. +- Locally created commits retain monotonic creation order when timestamps + collide. +- `CoroutineQueue` now preserves FIFO ordering, runs a single worker, and + recovers after task failures. - Zero-byte writes now fail explicitly instead of retrying indefinitely. ## [0.5.1] - 2026-03-05 ### Breaking Changes -- **`./init` sub-path export removed.** `deno run -A jsr:@goatdb/goatdb/init` no longer works. +- **`./init` sub-path export removed.** `deno run -A jsr:@goatdb/goatdb/init` no + longer works. **Migration:** ```bash @@ -104,7 +176,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Breaking Changes -- **Build tools moved from `@goatdb/goatdb/server`**: `compile`, `startDebugServer`, and `AppConfig` moved to `@goatdb/goatdb/server/build`. Runtime exports (`Server`, `createHttpServer`, etc.) remain in `@goatdb/goatdb/server`. +- **Build tools moved from `@goatdb/goatdb/server`**: `compile`, + `startDebugServer`, and `AppConfig` moved to `@goatdb/goatdb/server/build`. + Runtime exports (`Server`, `createHttpServer`, etc.) remain in + `@goatdb/goatdb/server`. **Migration:** ```typescript @@ -118,7 +193,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Security -- Fixed two vulnerabilities in `persistCommits`: null namespace guard and delta-based record bypass +- Fixed two vulnerabilities in `persistCommits`: null namespace guard and + delta-based record bypass ### Added @@ -130,18 +206,23 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - Loading status exposed on `ManagedItem` - `itemPathIsValid()` exported as a public utility - `getEnvVar` exported as stable cross-runtime API -- `setup` callback in `DebugServerOptions` for configuring server-side logic during development +- `setup` callback in `DebugServerOptions` for configuring server-side logic + during development - Health check endpoint on `Server` - `port` and `address` getters on `Server`; graceful shutdown support - Node.js Single Executable Application (SEA) compilation via `compile()` - `jsr:@goatdb/goatdb/link` now works in Node.js in addition to Deno -- HTTPS support with automatic self-signed certificate generation for secure development +- HTTPS support with automatic self-signed certificate generation for secure + development ### Changed -- Runtime abstraction layer refactored for cross-platform compatibility (Deno, Node.js, Browser) -- Path utilities moved to `base/path.ts` with POSIX-style handling across all environments -- Windows path support — backslashes from `Deno.cwd()`/`process.cwd()` are normalized to forward slashes +- Runtime abstraction layer refactored for cross-platform compatibility (Deno, + Node.js, Browser) +- Path utilities moved to `base/path.ts` with POSIX-style handling across all + environments +- Windows path support — backslashes from `Deno.cwd()`/`process.cwd()` are + normalized to forward slashes - `FileImpl` interface extended with `exists`, `copyFile`, and `readDir` methods - CLI `init` templates externalized to `cli/templates/` directory - CLI commands (`init`, `link`) now work in Node.js @@ -149,9 +230,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - Node.js minimum version raised to 24 (required for SEA support) - Node.js is now production-ready; experimental warning removed - esbuild bumped from 0.24 to 0.25.4 -- Browser `getOS()` uses 3-tier detection (User-Agent Client Hints, `navigator.platform`, User-Agent string) and returns `'unknown'` when platform cannot be detected +- Browser `getOS()` uses 3-tier detection (User-Agent Client Hints, + `navigator.platform`, User-Agent string) and returns `'unknown'` when platform + cannot be detected - Default server port for HTTPS changed to 8443 (HTTP remains 8080) -- `ManagedItem.commit()` now ensures all changes are fully committed before returning +- `ManagedItem.commit()` now ensures all changes are fully committed before + returning ### Removed @@ -164,12 +248,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - First sync awaited when opening a repository for the first time - Query scan now correctly awaited in browser environments - Resource leak in OPFS `copyFile()` — file handles now properly closed on error -- Template `DomainConfig` used incorrect method names (`resolveOrg` instead of `mapToOrg`) +- Template `DomainConfig` used incorrect method names (`resolveOrg` instead of + `mapToOrg`) - Template HTML referenced raw `index.tsx` instead of built `/app.js` -- Sync engine: recursive `serverUrl` expansion, infinite loop, dangling promise, and sync-before-open bugs +- Sync engine: recursive `serverUrl` expansion, infinite loop, dangling promise, + and sync-before-open bugs - `FileImpl`: partial-read loop and resource cleanup on error - No longer crashes when the cache file is missing -- Now throws an error when commit contents are missing instead of failing silently +- Now throws an error when commit contents are missing instead of failing + silently ## [0.3.1] @@ -188,7 +275,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Added -- [Orderstamp](https://github.com/goatplatform/orderstamp-js) is now bundled and exposed directly within GoatDB +- [Orderstamp](https://github.com/goatplatform/orderstamp-js) is now bundled and + exposed directly within GoatDB ## [0.2.2] @@ -211,23 +299,27 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Added - `FieldDef.validate` enables custom field level validation -- Before/After hooks for integrating with external build steps via `LiveReloadOptions.beforeBuild` and `LiveReloadOptions.afterBuild` +- Before/After hooks for integrating with external build steps via + `LiveReloadOptions.beforeBuild` and `LiveReloadOptions.afterBuild` - `CoreTypes` are now exported by the top level `mod.ts` - It's now possible to set a custom user schema via `SchemaManager.userSchema` -- Attempting to commit an item with invalid data now throws with a detailed message when running in debug mode +- Attempting to commit an item with invalid data now throws with a detailed + message when running in debug mode ### Fixed - Live reload in debug server is now working again - Set types being deserialized incorrectly -- Crash in `useItem` React hook when first passing `undefined` then a `ManagedItem` instance +- Crash in `useItem` React hook when first passing `undefined` then a + `ManagedItem` instance ## [0.2.0] ### Added - Initial support for Node.js -- Trusted mode which disables security mechanisms for increased performance in trusted environments +- Trusted mode which disables security mechanisms for increased performance in + trusted environments ### Changed @@ -243,14 +335,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Removed -- Order stamps have been moved to their own repository — https://github.com/goatplatform/orderstamp-js +- Order stamps have been moved to their own repository — + https://github.com/goatplatform/orderstamp-js ## [0.1.6] - 2025-02-26 ### Added - `FileImpl.getCWD` to get the current working directory -- `cli/link.ts` now enables easy development of GoatDB alongside an existing project +- `cli/link.ts` now enables easy development of GoatDB alongside an existing + project ### Fixed @@ -266,7 +360,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - GoatDB is now licensed under Apache 2.0 - `JSONLogFile.append` now automatically scans the log file rather than throw -- `JSONLogFile` worker now uses the transpiled JS version both for the server and the browser +- `JSONLogFile` worker now uses the transpiled JS version both for the server + and the browser - Improved React SPA scaffold - `ServerOptions.domain` enables fine control over domain / organization mapping @@ -279,7 +374,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Added - `Item.isNull` to check if an item has a null schema -- It's now possible to override the authorization rules for `/sys/users` and `/sys/*` +- It's now possible to override the authorization rules for `/sys/users` and + `/sys/*` - `Query.sortDescending` to sort results in descending order ### Changed diff --git a/cfds/base/errors.ts b/cfds/base/errors.ts index 24efdb6f..c3b1ed48 100644 --- a/cfds/base/errors.ts +++ b/cfds/base/errors.ts @@ -132,6 +132,11 @@ export function serviceUnavailable(): ServerError { return new ServerError(Code.ServiceUnavailable); } +/** @internal Throws serviceUnavailable unless the owning DB instance is open. */ +export function assertOpen(isOpen: boolean): void { + if (!isOpen) throw serviceUnavailable(); +} + export function incompatibleCFDSVersion( serverVersion: string, clientVersion: string, diff --git a/db/db.ts b/db/db.ts index 864590f9..f7f50ca7 100644 --- a/db/db.ts +++ b/db/db.ts @@ -49,6 +49,7 @@ import { DataRegistry } from '../cfds/base/data-registry.ts'; import { Emitter } from '../base/emitter.ts'; import { getGoatConfig } from '../base/config.ts'; import { makeShardConfig, type ShardConfig } from '../repo/shard-format.ts'; +import { assertOpen, serviceUnavailable } from '../cfds/base/errors.ts'; /** * The result of a sync operation with all peers for a repository. @@ -73,6 +74,18 @@ export type SyncResult = */ export type DBMode = 'client' | 'server'; +/** + * Lifecycle of a DB instance. This is the single source of truth; `isOpen` is + * the only derived signal. `failed` means bootstrap or teardown failed; a + * failed instance rejects work but `close()` can still retry teardown. + */ +export type DBState = + | 'booting' + | 'ready' + | 'closing' + | 'closed' + | 'failed'; + /** * The on-disk storage format for repository commit logs. * @@ -208,6 +221,12 @@ export type EventWriteFailure = 'WriteFailure'; */ export type EventItemChanged = 'ItemChanged'; +/** + * Emitted on every real lifecycle transition. Payload: (state: DBState). + * @group Database + */ +export type EventLifecycleChanged = 'LifecycleChanged'; + /** * Payload for the {@link EventWriteFailure} event, fired after 3 consecutive * I/O failures cause pending commits to be permanently dropped. @@ -241,8 +260,12 @@ export interface WriteFailureDetail { * @template US The user schema type, defaults to the base Schema type * @group Database */ -export class GoatDB - extends Emitter { +export class GoatDB extends Emitter< + | EventUserChanged + | EventWriteFailure + | EventItemChanged + | EventLifecycleChanged +> { readonly orgId: string; readonly registry: DataRegistry; readonly trusted: boolean; @@ -264,9 +287,20 @@ export class GoatDB >; // Consecutive write-failure count per file; dropped and logged at 3 failures. private readonly _appendFailCounts = new Map(); - // Test-only: when > 0, _drainPendingAppends throws instead of - // writing, simulating I/O failures. Decremented on each triggered failure. + /** + * @internal Test-only. When > 0, _drainPendingAppends throws instead of + * writing, simulating I/O failures. Decremented on each triggered failure. + */ _testForceAppendFailures = 0; + /** + * @internal Test-only. When > 0, _tearDownRepo throws before releasing + * resources, simulating a failed teardown. Decremented per failure. + */ + _testForceTeardownFailures = 0; + /** @internal Test-only. Close-time item commits to allow before the next fails; 0 fails next, -1 disables. */ + _testForceCommitFailures = -1; + /** @internal Test-only. When > 0, the next JSONLogFileClose throws (after detach). */ + _testForceFileCloseFailures = -1; // Tracks the most recent fire-and-forget drain promise per file so that // flush()/closeRepo() can await it before their own drain loop. private readonly _inFlightDrains = new Map>(); @@ -280,17 +314,21 @@ export class GoatDB string, Query >(); - /** In-flight repo closes, keyed by repoId (serializes open/close). */ + /** + * In-flight repo closes, keyed by repoId. Doubles as the per-repo mutex: + * the entry is installed synchronously before the first await, so JS's + * single-threaded scheduling makes concurrent close/open callers observe it. + */ private readonly _closePromises = new Map>(); - /** Mutex guard against concurrent manual closeRepo() calls for the same repo. */ - private readonly _closeLocks = new Set(); private _path: string | undefined; private _settingsProvider: DBSettingsProvider | undefined; queryPersistence?: QueryPersistence; private _trustPool: TrustPool | undefined; private _syncSchedulers: SyncScheduler[] | undefined; private _trustPoolPromise: Promise; - private _ready: boolean = false; + private _state: DBState = 'booting'; + /** Set by the first close(); repeated close() calls return this promise. */ + private _closePromise: Promise | undefined; readonly shardConfig: ShardConfig; // ── Inactivity auto-close config ────────────────────────────── @@ -339,10 +377,19 @@ export class GoatDB : Array.from(new Set(config.peers)); this._repoClients = new Map(); } - this._trustPoolPromise = this._getTrustPoolImpl().catch((err) => { - // Store the error but don't let it become an unhandled rejection - // It will be re-thrown when readyPromise() is called - return Promise.reject(err); + // DB logs bootstrap failures; readyPromise() still rejects. + this._trustPoolPromise = this._getTrustPoolImpl(); + this._trustPoolPromise.catch((err) => { + // Only a genuine bootstrap failure (while still booting) is an error; a + // close() mid-bootstrap cancels via _assertOpen: no ERROR log, no state. + if (this._state === 'booting') { + log({ + severity: 'ERROR', + error: 'StorageError', + message: `Database bootstrap failed: ${err}`, + }); + this._setState('failed'); + } }); // Inactivity is now handled per-resource via one-shot timers @@ -369,6 +416,7 @@ export class GoatDB * that's attached to a known user. */ get loggedIn(): boolean { + if (!this.isOpen) return false; // failed teardown keeps the pool for retry return this._trustPool?.currentSession.owner !== undefined; } @@ -377,39 +425,62 @@ export class GoatDB * anonymous. */ get currentUser(): ManagedItem | undefined { + this._assertOpen(); const userId = this._trustPool?.currentSession.owner; return userId ? this.item('sys', 'users', userId) : undefined; } /** * Returns the current session. - * @throws This method throws if called before db.ready returns true. + * @throws This method throws if called before the DB is ready. */ get currentSession(): ManagedItem { + this._assertOpen(); const sessionId = this._trustPool?.currentSession.id; assert( sessionId !== undefined, - 'Session not available yet. Wait for db.ready before accessing the current session.', + 'Session not available yet. Wait for the DB to be ready before accessing the current session.', ); return this.item('sys', 'sessions', sessionId); } /** - * Returns whether this DB instance is ready to receive commands or is it - * still performing the initial load. + * Lifecycle phase of this instance. Only `ready` runs against fully + * bootstrapped state; see `isOpen` for the narrower "accepts work" question + * and `readyPromise()` for the awaitable form. + */ + get state(): DBState { + return this._state; + } + + /** Single writer: publish every real lifecycle transition to observers. */ + private _setState(state: DBState): void { + if (this._state === state) return; + this._state = state; + this.emit('LifecycleChanged', state); + } + + /** + * True while the instance accepts work. Booting permits pre-ready calls; + * closing, closed, and failed do not. Once closed, create a new GoatDB + * instance instead of reopening this one. */ - get ready(): boolean { - return this._ready; + get isOpen(): boolean { + return this._state === 'booting' || this._state === 'ready'; } /** - * A convenience promise form of the `ready` flag. When the promise returns, - * this DB instance is ready to receive commands. + * A convenience promise form of the ready lifecycle phase. When the promise + * returns, this DB instance is ready to receive commands. * - * @throws ServiceUnavailable if the initial load failed. + * @throws The original bootstrap error if the initial load failed, or + * ServiceUnavailable if the instance is closing or closed. */ async readyPromise(): Promise { await this._trustPoolPromise; + if (this._state !== 'ready') { + throw serviceUnavailable(); + } } /** @@ -428,6 +499,7 @@ export class GoatDB * otherwise. */ async loginWithMagicLinkEmail(email: string): Promise { + this._assertOpen(); return await sendLoginEmail( ( await this.getTrustPool() @@ -438,78 +510,180 @@ export class GoatDB /** * Closes the database, releasing all resources including repositories, - * sync schedulers, queries, and file handles. This method should be called - * when you're done using the database instance. + * sync schedulers, queries, and file handles. Call this method when you're + * done using the database instance. + * + * Single-flight: the first call starts teardown synchronously; calls made + * while it is in flight and after it succeeds return the same promise. + * `state` becomes `closing` immediately and `closed` after teardown settles. + * If teardown fails, `state` becomes `failed` and the promise rejects; a + * later `close()` retries the remaining teardown. */ - async close(): Promise { - // Unschedule all idle timers to prevent concurrent fires during teardown + close(): Promise { + if (this._closePromise) return this._closePromise; + // Publish closing only after installing the single-flight promise. + this._closePromise = this._closeImpl().then( + () => this._setState('closed'), + (e: unknown) => this._onCloseRejected(e), + ); + this._setState('closing'); + return this._closePromise; + } + + /** + * Records a failed teardown. Partial teardown is recoverable: clearing the + * single-flight promise lets a later close() retry, and `failed` replaces a + * false `closed`. + */ + private _onCloseRejected(e: unknown): never { + this._closePromise = undefined; + this._setState('failed'); + log({ + severity: 'ERROR', + error: 'StorageError', + message: `Database close failed: ${e}`, + trace: String(e), + }); + throw e; + } + + /** + * Teardown body. Waits for bootstrap and pre-close opens first so a late + * bootstrap or open cannot install resources after teardown. + */ + private async _closeImpl(): Promise { + // Abort peer sync before waiting on bootstrap so repos opened during a + // close-overlapping bootstrap spawn no sync clients. + this._abortSyncSchedulers(); + // Wait for bootstrap: its sys-repo opens survive a concurrent close + // (allowDuringClose), so accepted pre-ready edits can be persisted below. + // A genuinely failed bootstrap rejects here, retaining dirty items instead + // of silently discarding them. + await this._trustPoolPromise; + // Schedulers born during the overlapping bootstrap (assigned in + // _createTrustPool after several awaits) postdate the pre-wait abort; + // reap them here or their 200ms intervals outlive the resolved close(). + this._abortSyncSchedulers(); + await this._awaitPreCloseOpens(); + this._unscheduleIdleTimers(); + this._closeAllQueries(); + await this._closeAllRepositories(); + + // Commit dirty items last so every opened repo from the prep phase is + // closed before a close-time commit reopens an auto-closed one. + const committed = await this._commitDirtyItems(); + await this._clearQueryPersistence(); + this._finalizeTeardown(committed); + + if (!committed) { + throw new Error('Failed to persist pending item edits during close'); + } + } + + /** Aborts peer sync before teardown so close is not held by a stalled sync. */ + private _abortSyncSchedulers(): void { + if (!this._syncSchedulers) return; + for (const scheduler of this._syncSchedulers) { + scheduler.close(); + } + this._syncSchedulers = undefined; + } + + /** Waits for opens that started before close so teardown sees them. */ + private async _awaitPreCloseOpens(): Promise { + await Promise.allSettled([...this._openPromises.values()]); + } + + /** Unschedules idle timers to prevent concurrent fires during teardown. */ + private _unscheduleIdleTimers(): void { for (const repo of this._repositories.values()) { repo._idleTimer?.unschedule(); } for (const q of this._openQueries.values()) { q._idleTimer?.unschedule(); } + } - // Stop all sync operations (prevents new writes) - if (this._syncSchedulers) { - for (const scheduler of this._syncSchedulers) { - scheduler.close(); - } - this._syncSchedulers = undefined; - } - - // Close all open queries + private _closeAllQueries(): void { for (const query of this._openQueries.values()) { query.close(); } this._openQueries.clear(); + } - // Close all repositories + private async _closeAllRepositories(): Promise { for (const repoPath of [...this._repositories.keys()]) { - await this.closeRepo(repoPath); + await this._closeRepo(repoPath); } + } - // Commit all remaining items (auto-closed repos). Items from - // closeRepo'd repos were already removed from _items. The commit - // reopens auto-closed repos via acquireRepo. After this, close any - // repos still in _repositories (reopened by this loop or by timer - // callbacks that fired during closeRepo). + /** + * Persists dirty managed items (auto-closed repos included) via the + * close-time commit path. Returns true only when every dirty item committed; + * failures are logged and the item stays dirty so a retried close() can + * re-attempt persistence. + */ + private async _commitDirtyItems(): Promise { const pendingRepos = new Set(); + const failedItems = new Set(); for (const item of [...this._items.values()]) { // Skip clean items: committing them would reopen an auto-closed repo // only for _setValueForKeyImpl() to dedup the no-op write. if (!item.isDirty) continue; const repoId = itemPathGetRepoId(item.path); + const hadRepo = this._repositories.has(repoId); + let committed = true; try { - await item.commit(); + await item._commitForClose(); } catch (e) { + committed = false; + failedItems.add(item.path); log({ - severity: 'WARNING', + severity: 'ERROR', error: 'StorageError', message: `Commit of pending item ${item.path} during close() failed: ${e}`, }); + // Only discard a half-open repo THIS commit introduced; a pre-existing + // one holds successful sibling commits that must still be torn down. + if (!hadRepo) await this._discardHalfOpenRepo(repoId); } - if (this._repositories.has(repoId)) { + // Close only repos reopened by SUCCESSFUL commits. A failed reopen left a + // half-open repo; closing it here would discard the pending edit, so a + // retried close() finishes it instead. + if (committed && this._repositories.has(repoId)) { pendingRepos.add(repoId); } } for (const repoId of pendingRepos) { - await this.closeRepo(repoId); + await this._closeRepo(repoId); } - - // Clear query persistence (has flush timer) - if (this.queryPersistence) { - await this.queryPersistence.close(); - this.queryPersistence = undefined; + // Final success is decided after repo-close retries: a failed sibling + // commit may be recovered by _commitDependentItems during teardown. Only + // a failed item still dirty means close truly failed; the half-open + // failure case keeps its item dirty, preserving retryability. + for (const path of failedItems) { + if (this._items.get(path)?.isDirty) return false; } + return true; + } + + private async _clearQueryPersistence(): Promise { + if (!this.queryPersistence) return; + await this.queryPersistence.close(); + this.queryPersistence = undefined; + } - // Deactivate any leftover managed items so their pending commit timers - // cannot reopen a repo after teardown. + /** Deactivates leftover items (kept when persistence failed) and drops session keys. */ + private _finalizeTeardown(committed: boolean): void { for (const item of this._items.values()) { item.deactivate(); } - this._items.clear(); + if (committed) { + this._items.clear(); + // Keep the pool on failure so a retried close() can reopen /sys repos. + this._trustPool = undefined; + } } /** @@ -517,9 +691,11 @@ export class GoatDB * local data. On browsers, this method will also reload the page to ensure * a clean state. * - * @throws ServiceUnavailable if the operation fails. + * @throws ServiceUnavailable while closing or after close. A closed + * instance must not wipe local data as a side effect. */ async logout(): Promise { + this._assertOpen(); await this.close(); await remove(this._basePath); if (isBrowser()) { @@ -531,18 +707,37 @@ export class GoatDB * Opens the given repository, loading all its items to memory. * This method does nothing if the repository is already open. * + * An open deferred behind an in-flight closeRepo()/auto-close is cancelled + * with serviceUnavailable if db.close() starts first: close is terminal, so + * no repository may be installed after its teardown snapshots. + * * @param path The path to the given repository. * @param opts Configuration options when opening this repository. */ - open(path: string, opts?: OpenOptions): Promise { + async open(path: string, opts?: OpenOptions): Promise { + this._assertOpen(); + return await this._openInternal(path, opts); + } + + /** + * @internal Open body that bypasses the DB lifecycle guard. + * + * @param allowDuringClose Set by close-time persistence, which must reopen + * auto-closed repos after `state` left 'ready'. + */ + private _openInternal( + path: string, + opts?: OpenOptions, + allowDuringClose = false, + ): Promise { path = itemPathNormalize(path); const repoId = itemPathGetRepoId(path); const existing = this._repositories.get(repoId); // Only hand out an already-open repo when no close is in flight. During a // manual closeRepo() the state stays 'open' through the commit/flush phase, - // so without the _isCloseInFlight() check open() would return the very - // instance that is about to be torn down (close/open overlap). Auto-close - // sets 'closing' synchronously, but manual close must be guarded here too. + // so without the in-flight check open() would return the very instance + // that is about to be torn down (close/open overlap). Auto-close sets + // 'closing' synchronously, but manual close must be guarded here too. if ( existing && existing._closeState === 'open' && !this._isCloseInFlight(repoId) @@ -560,11 +755,29 @@ export class GoatDB // in-flight open) and then reopening is the only safe path. const closeP = this._closePromises.get(repoId); if (closeP) { - return closeP.catch(() => {}).then(() => this.open(path, opts)); + // A rejected close (failed auto-close teardown) is retried from the + // fresh open below; the recursion re-checks the registry after settling. + return closeP.catch(() => {}).then(() => { + // Close wins: db.close() may have started while this open waited. + // Recursing unguarded would install a repo after close snapshotted + // _openPromises, leaking it behind the cached terminal close promise. + // The guard is the sole post-await step, so close cannot interleave. + if (!allowDuringClose) this._assertOpen(); + return this._openInternal(path, opts, allowDuringClose); + }); + } + if (existing && existing._closeState === 'closing') { + // An auto-close teardown failed and its promise was cleared. Finish the + // teardown before opening, otherwise the fresh instance would orphan the + // old repo's file handle and listeners. + return this._closeRepo(repoId).then(() => { + if (!allowDuringClose) this._assertOpen(); + return this._openInternal(path, opts, allowDuringClose); + }); } let result = this._openPromises.get(repoId); if (!result) { - result = this._openImpl(repoId, opts).finally(() => { + result = this._openImpl(repoId, opts, allowDuringClose).finally(() => { if (this._openPromises.get(repoId) === result) { this._openPromises.delete(repoId); } @@ -585,8 +798,24 @@ export class GoatDB * @group Database */ async acquireRepo(...pathComps: string[]): Promise { - const path = itemPathNormalize(pathComps.join('/')); - const repo = await this.open(path); + this._assertOpen(); + return await this._acquireRepoImpl(pathComps.join('/')); + } + + /** + * @internal Close-time variant of acquireRepo(). Bypasses the lifecycle + * guard so db.close() can persist dirty items after closing begins. + */ + _acquireRepoForClose(...pathComps: string[]): Promise { + return this._acquireRepoImpl(pathComps.join('/'), true); + } + + private async _acquireRepoImpl( + repoPath: string, + allowDuringClose = false, + ): Promise { + const path = itemPathNormalize(repoPath); + const repo = await this._openInternal(path, undefined, allowDuringClose); repo.acquireIdleLease(); return new RepoLease(repo, () => repo.releaseIdleLease()); } @@ -606,26 +835,29 @@ export class GoatDB * @param path Path to the desired repository. */ async closeRepo(path: string): Promise { + this._assertOpen(); + return await this._closeRepo(path); + } + + /** @internal closeRepo() body; db.close() uses it while closing. */ + private async _closeRepo(path: string): Promise { path = itemPathNormalize(path); const repoId = itemPathGetRepoId(path); - // Mutex: acquire the lock synchronously before any await. - if (this._closeLocks.has(repoId)) { - // Another call is already processing; wait for it via the promise. - const p = this._closePromises.get(repoId); - if (p) await p; - return; - } - this._closeLocks.add(repoId); - // Build the close promise synchronously (before any await) so concurrent - // callers and db.open() can properly await it. The promise captures the - // entire close lifecycle including commit and teardown. + // The map entry is the mutex: it is installed synchronously below, before + // any await, so a concurrent caller always joins the in-flight close here. + const inFlight = this._closePromises.get(repoId); + if (inFlight) return await inFlight; + // Build the close promise synchronously so concurrent callers and db.open() + // can await it. It captures the whole lifecycle: commit, flush, teardown. const cp = this._buildClosePromise(repoId); this._closePromises.set(repoId, cp); try { await cp; } finally { - this._closePromises.delete(repoId); - this._closeLocks.delete(repoId); + // Identity check: a later close may have replaced this entry. + if (this._closePromises.get(repoId) === cp) { + this._closePromises.delete(repoId); + } } } @@ -645,10 +877,14 @@ export class GoatDB // Wait for any in-flight open (do not close a half-loaded repo). const openP = this._openPromises.get(repoId); if (openP) await openP; - // _closeLocks guarantees single-entry. Double-check: a previous close - // may have already cleared the repo. + // The caller holds the _closePromises mutex. Double-check: a previous + // close may have already cleared the repo. const repo = this.repository(repoId); - if (!repo || repo._closeState !== 'open') { + // 'closing' means a previous teardown failed part-way; retry it. + if ( + !repo || + (repo._closeState !== 'open' && repo._closeState !== 'closing') + ) { return; } // Unschedule idle timer so it cannot fire during the close. @@ -660,24 +896,26 @@ export class GoatDB // Tear down dependent queries (manual close owns this; auto-close never // reaches here with queries attached because listener pins prevent it). this._closeDependentQueries(repoId); - // Flush pending item edits directly to this closing repo. This deliberately - // bypasses item.commit()/acquireRepo, which would deadlock by awaiting the - // close promise this method itself is running. - await this._commitDependentItems(repo); + // Flush pending item edits directly to this closing repo, bypassing + // item.commit()/acquireRepo (deadlock). Skip when a previous teardown + // already detached listeners: committing would lose the edit silently. + if (!repo._teardownDetached) { + await this._commitDependentItems(repo); + } for (const k of [...this._items.keys()]) { const item = this._items.get(k); - if (item && item.repository === repo) { + // Skip dirty items: they failed to persist and must survive teardown so + // a later close() can retry them (see _commitDirtyItems). + if (item && item.repository === repo && !item.isDirty) { item.deactivate(); this._items.delete(k); } } - // Transition to Closing and tear down. + // Transition to Closing and tear down. Only mark closed after teardown + // resolves; a failure leaves 'closing' so a retry can finish. repo._closeState = 'closing'; - try { - await this._tearDownRepo(repo); - } finally { - repo._closeState = 'closed'; - } + await this._tearDownRepo(repo); + repo._closeState = 'closed'; } /** Closes queries whose source repo matches the given repoId. */ @@ -738,6 +976,10 @@ export class GoatDB private async _tearDownRepo>( repo: Repository, ): Promise { + if (this._testForceTeardownFailures > 0) { + this._testForceTeardownFailures--; + throw new Error('test-forced teardown failure'); + } const repoId = repo.path; repo._idleTimer?.unschedule(); // Flush log file - retry if data was re-queued after a transient failure @@ -759,12 +1001,18 @@ export class GoatDB this._repoClients?.delete(repoId); // Detach event handlers first to prevent new file operations repo.detachAll(); - // Close log file + repo._teardownDetached = true; + // Close log file. Drop the map entry as soon as the close succeeds so a + // retried teardown (after a later throw) does not close it twice. if (fileEntry) { + if (this._testForceFileCloseFailures > 0) { + this._testForceFileCloseFailures--; + throw new Error('test-forced file close failure'); + } await JSONLogFileClose(fileEntry); + this._files.delete(repoId); this._appendFailCounts.delete(fileEntry); } - this._files.delete(repoId); // Identity guard: a concurrent open() may have installed a fresh repo. if (Object.is(this._repositories.get(repoId), repo)) { this._repositories.delete(repoId); @@ -772,6 +1020,15 @@ export class GoatDB this._warnedLegacyRepos.delete(repoId); } + // Disposes a repo reopened only by a failed close-time commit; old code + // deleted only the registry entry, leaking its _files handle and listeners. + private async _discardHalfOpenRepo(repoId: string): Promise { + const repo = this._repositories.get(repoId); + if (!repo) return; + repo._closeState = 'closing'; // teardown failure stays retryable + await this._tearDownRepo(repo); + } + /** * Access an item at the given path. An item's path is typically at the * following format: @@ -787,6 +1044,7 @@ export class GoatDB * @returns A managed item that tracks both local and remote edits. */ item(...pathComps: string[]): ManagedItem { + this._assertOpen(); for (const s of pathComps) { assert(typeof s === 'string'); // Sanity check } @@ -825,6 +1083,7 @@ export class GoatDB schema: S, data?: Partial>, ): ManagedItem { + this._assertOpen(); if (itemPathGetPart(path, 'item') === undefined) { path = itemPathJoin(path, uniqueId()); } @@ -855,6 +1114,7 @@ export class GoatDB schema: S, data: SchemaDataType, ): Promise { + this._assertOpen(); const repo = await this.open(path); let key = itemPathGetPart(path, 'item'); if (!key || key.length <= 0) { @@ -889,6 +1149,7 @@ export class GoatDB schema: S, items: { key?: string; data: SchemaDataType }[], ): Promise { + this._assertOpen(); const repo = await this.open(repoPath); const entries = []; for (const item of items) { @@ -911,8 +1172,10 @@ export class GoatDB * * @param path The full path to count. * @returns The number of items found or -1. + * @throws ServiceUnavailable while closing or after close. */ count(path: string): number { + this._assertOpen(); const repoId = itemPathGetRepoId(path); const repo = this.repository(repoId); if (!repo) return -1; @@ -927,8 +1190,10 @@ export class GoatDB * * @param path Full path to a repository. * @returns The keys at the specified path. + * @throws ServiceUnavailable while closing or after close. */ keys(path: string): Iterable { + this._assertOpen(); path = itemPathNormalize(path); const repoId = itemPathGetRepoId(path); return this.repository(repoId)?.keys() || []; @@ -957,6 +1222,7 @@ export class GoatDB query( config: Omit, 'db'>, ): Query { + this._assertOpen(); const id = resolveQueryId(config); let q = this._openQueries.get(id); if (!q) { @@ -1099,6 +1365,7 @@ export class GoatDB * disk. */ async flush(path: string): Promise { + this._assertOpen(); const repoId = itemPathGetRepoId(itemPathNormalize(path)); path = repoId; if (!path.endsWith('/')) { @@ -1127,6 +1394,7 @@ export class GoatDB * Flushes all pending writes for all repositories to disk. */ async flushAll(): Promise { + this._assertOpen(); const promises = mapIterable( this._repositories.keys(), (path) => this.flush(path), @@ -1156,6 +1424,7 @@ export class GoatDB * } */ async sync(path: string): Promise { + this._assertOpen(); const repoId = itemPathGetRepoId(itemPathNormalize(path)); await this.open(repoId); const clients = Array.from(this.clientsForRepo(repoId)); @@ -1194,6 +1463,7 @@ export class GoatDB * } */ async syncAll(): Promise> { + this._assertOpen(); const repoIds = Array.from(this._repositories.keys()); const results = await Promise.all(repoIds.map((rid) => this.sync(rid))); const out: Record = {}; @@ -1227,8 +1497,12 @@ export class GoatDB * Note: You almost never need to use the trust pool directly. * * @returns The trust pool of this DB instance. + * @throws ServiceUnavailable once the instance leaves `booting` (closing, + * closed, or failed). The pool holds session private keys, so a + * non-open instance must not hand it out. */ - getTrustPool(): Promise { + async getTrustPool(): Promise { + this._assertOpen(); return this._trustPoolPromise; } @@ -1236,7 +1510,13 @@ export class GoatDB await this._createTrustPool(); // Open /sys/sessions so all known sessions are properly loaded into our // new trust pool - const sessionsRepo = await this.open('/sys/sessions'); + // allowDuringClose: close() may overlap bootstrap; its sys-repo opens must + // finish so accepted pre-ready edits persist instead of being discarded. + const sessionsRepo = await this._openInternal( + '/sys/sessions', + undefined, + true, + ); // Although the repository automatically adds new sessions to the trust // pool, the initial bootstrapping must happen explicitly as the chain of // sessions won't be loaded in the correct order. @@ -1246,8 +1526,11 @@ export class GoatDB } // Open /sys/users so we can perform login and basic operations without // waiting - await this.open('/sys/users'); - this._ready = true; + await this._openInternal('/sys/users', undefined, true); + // close() may have started during bootstrap; never revive a closing DB. + if (this._state === 'booting') { + this._setState('ready'); + } return this._trustPool!; } @@ -1320,61 +1603,68 @@ export class GoatDB } } + /** + * @internal Throws serviceUnavailable while close is in flight or after it + * completes. Public data operations call this before side effects so callers + * see the failure at the API boundary, not inside a repository. Close-time + * persistence passes allowDuringClose so it may reopen auto-closed repos. + */ + private _assertOpen(allowDuringClose = false): void { + if (!allowDuringClose) assertOpen(this.isOpen); + } + /** * @internal True while any close (manual or auto) is registered for the repo. * Used by Repository._canWrite() to reject writes that would silently land * on a repo whose teardown is already in flight. */ _isCloseInFlight(repoId: string): boolean { - return this._closeLocks.has(repoId) || this._closePromises.has(repoId); + return this._closePromises.has(repoId); } /** - * @internal Called by a Repository idle timer. Shares the same - * serialization primitives as closeRepo() (_closeLocks, - * _closePromises) so manual closeRepo and db.open() see the + * @internal Called by a Repository idle timer. Uses the same per-repo mutex + * (_closePromises) as closeRepo() so manual close and db.open() see the * in-flight close. Does NOT commit pending items or close dependent - * queries — pending edits reopen the repo on demand via acquireRepo, - * and listener-pinned queries keep repos ineligible for idle close. + * queries — pending edits reopen the repo on demand via acquireRepo, and + * listener-pinned queries keep repos ineligible for idle close. + * + * A failed teardown rejects and leaves the repo in 'closing' with no + * registered promise, so the next open()/closeRepo() retries it instead of + * leaking the file handle. */ async _requestRepoIdleClose>( repo: Repository, ): Promise { + if (!this.isOpen) return; if (repo._closeState !== 'open') return; if (!Object.is(this.repository(repo.path), repo)) return; // stale instance if (!repo._isIdleEligible()) return; - // If manual closeRepo is in-flight, let it handle the close. - if (this._closeLocks.has(repo.path)) { - const p = this._closePromises.get(repo.path); - if (p) await p; - return; - } - // Transition synchronously so concurrent open() and closeRepo() - // see the in-flight close before any await. + // A manual close is in-flight; join it instead of tearing down twice. + const inFlight = this._closePromises.get(repo.path); + if (inFlight) return await inFlight; + // Transition synchronously so concurrent open() and closeRepo() see the + // in-flight close before any await. repo._closeState = 'closing'; - this._closeLocks.add(repo.path); - // Resolve the stored promise even on teardown failure so every awaiter - // (manual closeRepo included) observes the same, already-logged outcome. - const cp = this._tearDownRepo(repo).catch((e) => { - log({ - severity: 'WARNING', - error: 'StorageError', - message: `Auto-close of repo ${repo.path} failed: ${e}`, - }); + const cp = this._tearDownRepo(repo).then(() => { + repo._closeState = 'closed'; }); this._closePromises.set(repo.path, cp); try { await cp; } finally { - this._closePromises.delete(repo.path); - this._closeLocks.delete(repo.path); - repo._closeState = 'closed'; + // Only success marks 'closed' (inside the then above). A rejection + // leaves 'closing' so the next open() retries the teardown. + if (this._closePromises.get(repo.path) === cp) { + this._closePromises.delete(repo.path); + } } } private async _openImpl( repoId: string, opts?: OpenOptions, + allowDuringClose = false, ): Promise { repoId = Repository.normalizePath(repoId); let trustPool: TrustPool; @@ -1383,7 +1673,9 @@ export class GoatDB if (repoId === '/sys/sessions' || repoId === '/sys/users') { trustPool = this._trustPool!; } else { - trustPool = await this.getTrustPool(); + // Read the promise directly: close-time commits reopen repos while the + // public getter rejects (see _acquireRepoForClose). + trustPool = await this._trustPoolPromise; } const repo = new Repository(this as unknown as GoatDB, repoId, trustPool, { ...opts, @@ -1509,43 +1801,95 @@ export class GoatDB } }); repo.unmute(); - if (this._syncSchedulers) { - const clients: RepoClient[] = []; - for (const scheduler of this._syncSchedulers) { - const c = new RepoClient( - repo, - repoId, - scheduler.syncConfig, - scheduler, - this.orgId, - ); - clients.push(c); - c.ready = true; - if ( - !loadedFromBackup && repoId !== '/sys/sessions' && - repoId !== '/sys/users' - ) { - try { - await c.sync(); - } catch (e: unknown) { - log({ - severity: 'WARNING', - error: 'UnknownSyncError', - message: `Initial sync failed for ${repoId}`, - trace: String(e), - }); - } - } - c.startSyncing(); - } - this._repoClients!.set(repoId, clients); - } + // close() nulls _syncSchedulers and aborts fetches. A public open that + // lost the race must not revive clients or return a repo teardown destroys. + this._assertOpen(allowDuringClose); + // Backup loads and sys repos have nothing to reconcile with peers yet. + await this._startRepoClients( + repo, + repoId, + allowDuringClose, + loadedFromBackup || repoId === '/sys/sessions' || + repoId === '/sys/users', + ); // Arm the idle timer only after the repo is fully loaded, so a slow open // cannot immediately expire itself. repo._startIdleTimer(); return repo; } + /** One REST client per peer scheduler; owns this repo's adaptive sync timer. */ + private _newRepoClient( + repo: Repository, + repoId: string, + scheduler: SyncScheduler, + ): RepoClient { + return new RepoClient( + repo, + repoId, + scheduler.syncConfig, + scheduler, + this.orgId, + ); + } + + /** + * Initial peer sync of a freshly opened repo. A close() abort rethrows through + * the guard so the open is abandoned; a real sync error is logged and the + * open continues. + */ + private async _initialSync( + client: RepoClient, + repoId: string, + allowDuringClose: boolean, + ): Promise { + try { + await client.sync(); + } catch (e: unknown) { + this._assertOpen(allowDuringClose); + log({ + severity: 'WARNING', + error: 'UnknownSyncError', + message: `Initial sync failed for ${repoId}`, + trace: String(e), + }); + } + } + + /** + * Creates and starts one client per sync scheduler, then registers them for + * teardown. An abort leaves the clients created so far unregistered, hence + * invisible to _tearDownRepo: close them here or their repeating timers + * outlive the closed state. + */ + private async _startRepoClients( + repo: Repository, + repoId: string, + allowDuringClose: boolean, + skipInitialSync: boolean, + ): Promise { + if (!this._syncSchedulers) { + return; + } + const clients: RepoClient[] = []; + try { + for (const scheduler of this._syncSchedulers) { + const client = this._newRepoClient(repo, repoId, scheduler); + clients.push(client); + client.ready = true; + if (!skipInitialSync) { + await this._initialSync(client, repoId, allowDuringClose); + } + this._assertOpen(allowDuringClose); + client.startSyncing(); + } + } catch (e: unknown) { + for (const client of clients) client.close(); + throw e; + } + this._repoClients!.set(repoId, clients); + } + /** * Checks if an item at the given path is currently loaded in memory. * This is a passive check that does not trigger loading the item. diff --git a/db/managed-item.ts b/db/managed-item.ts index 2e33a093..3856c5bb 100644 --- a/db/managed-item.ts +++ b/db/managed-item.ts @@ -1,4 +1,8 @@ -import type { Schema, SchemaDataType } from '../cfds/base/schema.ts'; +import { + type Schema, + type SchemaDataType, + SchemaEquals, +} from '../cfds/base/schema.ts'; import type { Commit } from '../repo/commit.ts'; import type { Repository } from '../repo/repo.ts'; import { itemPathGetPart, itemPathGetRepoId, itemPathIsValid } from './path.ts'; @@ -12,6 +16,7 @@ import { import { SimpleTimer, type Timer } from '../base/timer.ts'; import type { GoatDB } from './db.ts'; import { assert } from '../base/error.ts'; +import { assertOpen, serviceUnavailable } from '../cfds/base/errors.ts'; import { log } from '../logging/log.ts'; import { ScratchPool } from '../base/scratch-pool.ts'; @@ -30,6 +35,7 @@ export class ManagedItem private _item!: Item; private _commitPromise?: Promise; private _age: number = 0; + private _localEditCount: number = 0; // local edits (not rebase artifacts) private _commitInProgress: boolean = false; // True when local edits have not yet been persisted. Cleared only after a // successful commit that was not overtaken by a newer edit; a failed commit @@ -37,6 +43,8 @@ export class ManagedItem private _dirty: boolean = false; private _ready: boolean = false; private _readyPromiseResolve?: () => void; + private _readyPromiseReject?: (e: unknown) => void; + /** Settles once; stays rejected after a failed load so later awaiters see it. */ private _readyPromise?: Promise; // deno-lint-ignore no-explicit-any private readonly _scratchPool = new ScratchPool>( @@ -48,22 +56,31 @@ export class ManagedItem assert(itemPathIsValid(path), `Invalid item path: ${path}`); this.path = path; this._commitDelayTimer = new SimpleTimer(300, false, () => { + // Close owns persistence for dirty items; any other failure is a real + // fault and must stay visible (ERROR), not be logged away as WARNING. + if (!this.db.isOpen) return; this.commit().catch((e) => { log({ - severity: 'WARNING', + severity: 'ERROR', error: 'StorageError', message: `Auto-commit failed for ${this.path}`, trace: String(e), }); }); }); - this._readyPromise = new Promise((resolve) => { + this._readyPromise = new Promise((resolve, reject) => { this._readyPromiseResolve = resolve; + this._readyPromiseReject = reject; }); + // A failed load must not surface as an unhandled rejection: callers that + // care await readyPromise(), and the failure is logged in _failReady(). + this._readyPromise.catch(() => {}); const repo = db.repository(itemPathGetRepoId(path)); this._item = Item.nullItem(db.registry); if (!repo) { - this.loadRepoAndDoc(); + // Fire-and-forget load. It may lose the race to close(), which rejects + // the open with serviceUnavailable; surface other failures. + this.loadRepoAndDoc().catch((e: unknown) => this._failReady(e)); } else { this.loadInitialDoc(repo); } @@ -96,6 +113,9 @@ export class ManagedItem /** * Returns a promise that resolves when this managed item is ready to use. * If the item is already ready, the promise resolves immediately. + * + * @throws The original error if the initial load failed (also logged), or + * ServiceUnavailable if the instance closed while loading. */ readyPromise(): Promise { return this._readyPromise ?? Promise.resolve(); @@ -129,6 +149,7 @@ export class ManagedItem * item. */ set schema(s: S) { + this._assertOpen(); if (this._item.isLocked) { this._item = this._item.clone(); } @@ -169,6 +190,7 @@ export class ManagedItem * Sets this item's delete marker. Used to delete/un-delete an item. */ set isDeleted(flag: boolean) { + this._assertOpen(); const oldValue = this.isDeleted; if (oldValue !== flag) { if (this._item.isLocked) this._item = this._item.clone(); @@ -232,6 +254,7 @@ export class ManagedItem key: string & T, value: SchemaDataType[T], ): void { + this._assertOpen(); const oldValue = this.has(key) ? this.get(key) : undefined; if (this._item.isLocked) this._item = this._item.clone(); this._item.set(key, value); @@ -267,6 +290,7 @@ export class ManagedItem * if it didn't exist */ delete>(key: string & T): boolean { + this._assertOpen(); const oldValue = this.has(key) ? this.get(key) : undefined; if (this._item.isLocked) this._item = this._item.clone(); if (this._item.delete(key)) { @@ -302,6 +326,18 @@ export class ManagedItem * @returns A promise that resolves when the commit is complete */ commit(): Promise { + return this._commit(false); + } + + /** + * @internal Close-time commit. Bypasses the DB lifecycle guard so db.close() + * can persist dirty items after the instance enters the closing state. + */ + _commitForClose(): Promise { + return this._commit(true); + } + + private _commit(forClose: boolean): Promise { this._item.normalize(); const [valid, error] = this._item.validate(); if (this.db.debug) { @@ -323,10 +359,15 @@ export class ManagedItem // Always flush any scheduled commit and run now this._commitDelayTimer.unschedule(); if (this._commitPromise) { - // Wait for the existing commit to finish, then try again if needed - return this._commitPromise.then(() => this.commit()); + // Wait for the existing commit to finish, then try again if needed. + // Close retries even after a failure so a commit that raced close() + // cannot silently drop the edit; public callers keep failure semantics. + const pending = this._commitPromise; + return (forClose ? pending.catch(() => {}) : pending).then(() => + this._commit(forClose) + ); } - const p = this._commitImpl().finally(() => { + const p = this._commitImpl(forClose).finally(() => { if (this._commitPromise === p) { this._commitPromise = undefined; } @@ -390,6 +431,11 @@ export class ManagedItem return this._dirty || this._commitInProgress; } + /** @internal Throws when the DB is not open; booting stays allowed. */ + private _assertOpen(): void { + assertOpen(this.db.isOpen); + } + /** * @internal * Deactivates the managed item by canceling any pending commits. @@ -415,28 +461,84 @@ export class ManagedItem isRebase = false, ): void { ++this._age; + if (!isRebase) ++this._localEditCount; this.emit('change', mutations); this.db.emit('ItemChanged', this.path, isRebase); this._dirty = true; this._commitDelayTimer.schedule(); } - private async _commitImpl(): Promise { + /** + * @internal True when the in-memory item structurally matches the persisted + * head record. Uses checksum equality, which serializes NaN + * deterministically: matching NaN payloads compare equal while null vs NaN + * differ, so a NaN artifact left by a rebase cannot falsely trip divergence. + */ + private _matchesPersistedHead(repo: Repository): boolean { + const entry = repo.valueForKey(itemPathGetPart(this.path, 'item')!); + if (!entry) { + return false; + } + try { + return ( + SchemaEquals(this._item.schema, entry[0].schema) && + this._item.checksum === entry[0].checksum + ); + } catch (e) { + log({ + severity: 'WARNING', + error: 'StorageError', + message: `Cannot verify persistence convergence for ${this.path}`, + trace: String(e), + }); + // A schema/data mismatch (e.g. a pending auto-upgrade) makes the + // checksum incomputable; treat as not-converged so the item stays + // dirty and the upgrade is re-persisted instead of being dropped. + return false; + } + } + + /** + * @internal Clears the dirty flag only when the just-written state is fully + * persisted: no newer local edit landed during the await, and the + * in-memory item now equals the persisted head record. A rebase (explicit + * or triggered by a new commit) may merge remote fields into the item, so + * the counter alone cannot prove convergence; the NaN-tolerant checksum + * comparison is the ground truth. Otherwise the merged view is still ahead + * of disk and must stay dirty so close()/auto-commit re-persists it instead + * of dropping it. + */ + private _maybeClearDirty(localEditCount: number, repo: Repository): void { + if (this._localEditCount !== localEditCount) { + return; + } + if (this._matchesPersistedHead(repo)) { + this._dirty = false; + } + } + + private async _commitImpl(forClose: boolean): Promise { assert(!this._commitInProgress); this._commitInProgress = true; try { this._commitDelayTimer.unschedule(); const currentDoc = this._item.clone(); - const commitAge = this._age; + const localEditCount = this._localEditCount; const key = itemPathGetPart(this.path, 'item')!; const repoId = itemPathGetRepoId(this.path); // Acquire an open repo + idle lease atomically. Holding the lease // prevents an idle close from tearing down the repo this write targets; // releasing it (in the finally block via dispose()) re-arms the idle // timer. This awaits any close promise registered by closeRepo or - // auto-close, so the commit targets a fully open repo. - const _lease = await this.db.acquireRepo(repoId); + // auto-close, so the commit targets a fully open repo. Close bypasses + // the DB-level lifecycle guard via the internal close variant. + const _lease = forClose + ? await this.db._acquireRepoForClose(repoId) + : await this.db.acquireRepo(repoId); try { + const forced = forClose ? this.db._testForceCommitFailures : -1; + if (forced === 0) throw new Error('test-forced commit failure'); + if (forced > 0) this.db._testForceCommitFailures--; const newHead = await _lease.repo.setValueForKey( key, currentDoc, @@ -446,11 +548,9 @@ export class ManagedItem this._head = newHead; this.rebase(); } - // Persisted: clear dirty unless a newer edit landed during the await - // (age changed) or rebase merged remote fields into the local doc. - if (this._age === commitAge) { - this._dirty = false; - } + // Persisted: clear dirty unless a newer local edit landed during the + // await or a rebase left the item diverging from the persisted head. + this._maybeClearDirty(localEditCount, _lease.repo); } finally { _lease.dispose(); } @@ -467,7 +567,7 @@ export class ManagedItem */ async _commitTo(repo: Repository): Promise { const currentDoc = this._item.clone(); - const commitAge = this._age; + const localEditCount = this._localEditCount; const key = itemPathGetPart(this.path, 'item')!; const newHead = await repo._setValueForKeyForClose( key, @@ -478,9 +578,7 @@ export class ManagedItem this._head = newHead; this.rebase(); } - if (this._age === commitAge) { - this._dirty = false; - } + this._maybeClearDirty(localEditCount, repo); } /** @@ -536,6 +634,28 @@ export class ManagedItem this._readyPromiseResolve?.(); this._readyPromise = undefined; this._readyPromiseResolve = undefined; + this._readyPromiseReject = undefined; this.emit('LoadingFinished'); } + + /** + * Settles a failed initial load. Closing/closed is a cancellation by design + * (serviceUnavailable); anything else, `failed` included, is a real fault + * that is logged and rethrown to readyPromise() awaiters. + */ + private _failReady(e: unknown): void { + const { state } = this.db; + const cancelled = state === 'closing' || state === 'closed'; + if (!cancelled) { + log({ + severity: 'ERROR', + error: 'StorageError', + message: `Failed to load item ${this.path}: ${e}`, + trace: String(e), + }); + } + this._readyPromiseReject?.(cancelled ? serviceUnavailable() : e); + this._readyPromiseResolve = undefined; + this._readyPromiseReject = undefined; + } } diff --git a/deno.json b/deno.json index a05d5aec..f25683e3 100644 --- a/deno.json +++ b/deno.json @@ -66,6 +66,7 @@ "nodemailer": "npm:nodemailer@^6.10.0", "playwright": "npm:playwright@^1.48.0", "react": "npm:react@^19.1.0", + "react-test-renderer": "npm:react-test-renderer@^19.1.0", "source-map": "npm:source-map@^0.7.4", "sql.js": "npm:sql.js@^1.11.0", "postject": "npm:postject@1.0.0-alpha.6", diff --git a/mod.ts b/mod.ts index 376e0b66..ec28bd91 100644 --- a/mod.ts +++ b/mod.ts @@ -54,7 +54,9 @@ export type { }; export type { DBInstanceConfig, + DBState, EventItemChanged, + EventLifecycleChanged, EventWriteFailure, StorageFormat, WriteFailureDetail, diff --git a/net/client.ts b/net/client.ts index c42fd6ea..01aa2f6d 100644 --- a/net/client.ts +++ b/net/client.ts @@ -247,13 +247,17 @@ export class RepoClient extends Emitter { priority, )) as typeof reqMsg; } catch (e: unknown) { - log({ - severity: 'INFO', - error: 'SerializeError', - value: (e as Error).message, - message: (e as Error).message, - trace: (e as Error).stack, - }); + // close() rejects pending sends with serviceUnavailable; log only real + // peer/transport faults so shutdown stays quiet. + if (!this.scheduler.isClosed) { + log({ + severity: 'INFO', + error: 'SerializeError', + value: (e as Error).message, + message: (e as Error).message, + trace: (e as Error).stack, + }); + } this._setIsOnline(false); return false; } finally { diff --git a/net/rest-api.ts b/net/rest-api.ts index 764fa128..428f3fad 100644 --- a/net/rest-api.ts +++ b/net/rest-api.ts @@ -8,8 +8,7 @@ import { } from '../db/session.ts'; import { kSecondMs } from '../base/date.ts'; import type { JSONValue, ReadonlyJSONObject } from '../base/interfaces.ts'; -import { sleep } from '../base/time.ts'; -import { timeout } from '../cfds/base/errors.ts'; +import { serviceUnavailable, timeout } from '../cfds/base/errors.ts'; import { getGoatConfig } from '../base/config.ts'; import { getCrypto } from '../base/common.ts'; @@ -28,15 +27,14 @@ export async function createNewSession( if (resp.status !== 200) { return [undefined, undefined]; } - const body = await resp.json(); + const body = JSON.parse(resp.body) as ReadonlyJSONObject; const encodedRoots = body.roots as EncodedSession[]; const roots: Session[] = []; for (const e of encodedRoots) { roots.push(await decodeSession(e)); } - return [await decodeSession(body.session), roots]; + return [await decodeSession(body.session as EncodedSession), roots]; } catch (_err: unknown) { - debugger; return [undefined, undefined]; } } @@ -77,45 +75,71 @@ export function sendJSONToEndpoint( endpoint: string, session: OwnedSession | undefined, json: ReadonlyJSONObject, -): Promise { +): Promise { return sendJSONToURL(urlForEndpoint(endpoint), session, json); } -export async function sendJSONToURL( - url: string, +export interface JSONResponse { + status: number; + body: string; +} + +/** Builds request headers, signing the request once when a session is given. */ +async function buildRequestHeaders( sessionOrSignature: OwnedSession | undefined | string, - json: JSONValue, - orgId?: string, - timeoutMs = 5 * kSecondMs, -): Promise { - const headers: HeadersInit = { + orgId: string | undefined, +): Promise { + const headers: Record = { 'Content-Type': 'application/json', }; - if (sessionOrSignature !== undefined) { - if (typeof sessionOrSignature !== 'string') { - sessionOrSignature = await generateRequestSignature(sessionOrSignature); + let signature = sessionOrSignature; + if (signature !== undefined) { + if (typeof signature !== 'string') { + signature = await generateRequestSignature(signature); } - headers['x-goat-sig'] = sessionOrSignature; + headers['x-goat-sig'] = signature; } if (orgId) { headers['x-org-id'] = orgId; } - const abortController = new AbortController(); - const fetchPromise = fetch(url, { - method: 'POST', - headers, - body: JSON.stringify(json), - signal: abortController.signal, - }); - let aborted = false; - const timeoutPromise = (async () => { - await sleep(timeoutMs); - aborted = true; - abortController.abort(); - })(); - await Promise.any([fetchPromise, timeoutPromise]); - if (aborted) { - throw timeout(); + return headers; +} + +export async function sendJSONToURL( + url: string, + sessionOrSignature: OwnedSession | undefined | string, + json: JSONValue, + orgId?: string, + timeoutMs = 5 * kSecondMs, + signal?: AbortSignal, +): Promise { + const headers = await buildRequestHeaders(sessionOrSignature, orgId); + // One controller drives both timeout and external cancellation, so the + // fetch has a single abort path with an explicit reason (fetch itself masks + // it as a generic AbortError). + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(timeout()), timeoutMs); + const onExternalAbort = () => controller.abort(serviceUnavailable()); + // Registering after an already-aborted signal would never fire; abort now. + if (signal?.aborted) { + onExternalAbort(); + } else { + signal?.addEventListener('abort', onExternalAbort, { once: true }); + } + try { + const resp = await fetch(url, { + method: 'POST', + headers, + body: JSON.stringify(json), + signal: controller.signal, + }); + // Keep the timeout/abort armed until the body is consumed. + return { status: resp.status, body: await resp.text() }; + } catch (e) { + if (controller.signal.aborted) throw controller.signal.reason; + throw e; + } finally { + clearTimeout(timer); + signal?.removeEventListener('abort', onExternalAbort); } - return await fetchPromise; } diff --git a/net/sync-scheduler.ts b/net/sync-scheduler.ts index 67385ce9..157e577b 100644 --- a/net/sync-scheduler.ts +++ b/net/sync-scheduler.ts @@ -80,6 +80,9 @@ export class SyncScheduler { private _intervalId: ReturnType | undefined; private _fetchInProgress = false; private _closed = false; + // Aborts the in-flight batch-sync fetch so close() cannot wait on the + // network timeout (db close must not inherit peer stall latency). + private readonly _abortController = new AbortController(); constructor( readonly url: string, @@ -99,11 +102,17 @@ export class SyncScheduler { return syncConfigGetCycles(this.syncConfig, this._syncFreqAvg.currentValue); } + /** @internal True after close(): pending work is cancelled, not failing. */ + get isClosed(): boolean { + return this._closed; + } + close(): void { if (this._closed) { return; } this._closed = true; + this._abortController.abort(); if (this._intervalId !== undefined) { clearInterval(this._intervalId); this._intervalId = undefined; @@ -178,8 +187,9 @@ export class SyncScheduler { reqArr, this.orgId, this.syncConfig.timeoutMs, + this._abortController.signal, ); - respText = resp.status === 200 ? await resp.text() : undefined; + respText = resp.status === 200 ? resp.body : undefined; const syncDurationMs = performance.now() - start; this._syncFreqAvg.addValue(syncDurationMs); @@ -193,13 +203,17 @@ export class SyncScheduler { }); } } catch (e: unknown) { - log({ - severity: 'INFO', - error: 'FetchError', - message: (e as Error).message, - trace: (e as Error).stack, - url: this.url, - }); + // A close() aborts the in-flight fetch; that is cancellation, not a peer + // fault, so it must not be logged as a sync error. + if (!this._closed) { + log({ + severity: 'INFO', + error: 'FetchError', + message: (e as Error).message, + trace: (e as Error).stack, + url: this.url, + }); + } this._fetchInProgress = false; } diff --git a/react/hooks.ts b/react/hooks.ts index d1a8fe9a..9da6c95f 100644 --- a/react/hooks.ts +++ b/react/hooks.ts @@ -15,11 +15,14 @@ import type { ReadonlyJSONValue } from '../base/interfaces.ts'; import type { Query, QueryConfig } from '../repo/query.ts'; import { getBaseURL } from '../net/rest-api.ts'; -type GoatDBCtxProps = { +export type GoatDBCtxProps = { db?: GoatDB; }; -const GoatDBContext = React.createContext({}); +/** Exported so tests and hosts can inject a specific DB instance. */ +export const GoatDBContext: React.Context = React.createContext< + GoatDBCtxProps +>({}); /** * Opens a local DB, creating it if necessary. Once opened, the DB is available @@ -320,37 +323,19 @@ export type DBReadyState = 'loading' | 'ready' | 'error'; */ export function useDBReady(): DBReadyState { const db = useDB(); - let error = false; + // GoatDB logs bootstrap failures; do not add a second readyPromise() consumer. const subscribe = useCallback( (onStoreChange: () => void) => { - if (!db.ready) { - let cancelled = false; - db.readyPromise() - .then(() => { - if (!cancelled) { - onStoreChange(); - } - }) - .catch((err) => { - console.error(err); - error = true; - if (!cancelled) { - onStoreChange(); - } - }); - return () => { - cancelled = true; - }; - } - return () => {}; + // Return detach so useSyncExternalStore unsubscribes. + return db.attach('LifecycleChanged', onStoreChange); }, [db], ); const getSnapshot = useCallback(() => { - if (error) { + if (db.state === 'closed' || db.state === 'failed') { return 'error'; } - return db.ready ? 'ready' : 'loading'; + return db.state === 'ready' ? 'ready' : 'loading'; }, [db]); return useSyncExternalStore(subscribe, getSnapshot); } diff --git a/repo/repo.ts b/repo/repo.ts index 6c751a29..e319af8c 100644 --- a/repo/repo.ts +++ b/repo/repo.ts @@ -142,6 +142,8 @@ export class Repository< * before transitioning; combined with _closePromises in GoatDB for * await-based serialization of Open -> Closing -> Closed. */ _closeState: 'open' | 'closing' | 'closed' = 'open'; + /** @internal True once teardown detached listeners; re-committing is unsafe. */ + _teardownDetached = false; /** @internal Active idle leases (e.g. in-flight item commits via acquireRepo). */ _idleLeaseCount = 0; /** Guards timer scheduling until _openImpl completes (no slow-open expiry). */ @@ -305,14 +307,15 @@ export class Repository< /** @internal Called by the idle timer to request close. */ _onIdleTimeout(): void { if (!this._isIdleEligible()) return; - // Fire-and-forget, but never leave the promise unobserved: the lock branch - // of _requestRepoIdleClose() awaits a manual close promise that may - // rethrow a teardown failure. + // Fire-and-forget, but never leave the promise unobserved. A failed + // teardown leaves the repo registered in 'closing' so the next open or + // close retries it; log so the failure is visible meanwhile. void this.db._requestRepoIdleClose(this).catch((e) => { log({ - severity: 'WARNING', + severity: 'ERROR', error: 'StorageError', - message: `Auto-close of repo ${this.path} failed: ${e}`, + message: + `Auto-close of repo ${this.path} failed; repo left open for retry: ${e}`, }); }); } diff --git a/tests/asserts.ts b/tests/asserts.ts index cbd937de..fc79f426 100644 --- a/tests/asserts.ts +++ b/tests/asserts.ts @@ -10,6 +10,22 @@ import { getGlobalLoggerStreams, setGlobalLoggerStreams, } from '../logging/log.ts'; +import type { LogEntry } from '../logging/log.ts'; +import type { NormalizedLogEntry } from '../logging/entry.ts'; + +/** Swaps in a capturing log stream for the duration of fn, then restores. */ +export function withLogCapture( + fn: (captured: NormalizedLogEntry[]) => Promise, +): Promise { + const captured: NormalizedLogEntry[] = []; + const prev = getGlobalLoggerStreams(); + setGlobalLoggerStreams([{ + appendEntry(e: NormalizedLogEntry): void { + captured.push(e); + }, + }]); + return fn(captured).finally(() => setGlobalLoggerStreams(prev)); +} /** * Custom error class for assertion failures. @@ -165,7 +181,9 @@ export function assertThrowsInstanceOf( if (!(caughtError instanceof errorCtor)) { throw new AssertionError( message || - `Expected error to be instance of ${errorCtor.name}, got ${caughtError?.constructor?.name ?? typeof caughtError}`, + `Expected error to be instance of ${errorCtor.name}, got ${ + caughtError?.constructor?.name ?? typeof caughtError + }`, ); } }, @@ -185,7 +203,9 @@ export function assertThrowsInstanceOf( if (!(caughtError instanceof errorCtor)) { throw new AssertionError( message || - `Expected error to be instance of ${errorCtor.name}, got ${caughtError?.constructor?.name ?? typeof caughtError}`, + `Expected error to be instance of ${errorCtor.name}, got ${ + caughtError?.constructor?.name ?? typeof caughtError + }`, ); } } diff --git a/tests/auto-close.test.ts b/tests/auto-close.test.ts index f59a67da..c6500ac4 100644 --- a/tests/auto-close.test.ts +++ b/tests/auto-close.test.ts @@ -24,7 +24,12 @@ * only a small E2E layer uses real timers for scheduler integration. */ -import { assertEquals, assertExists, assertTrue } from './asserts.ts'; +import { + assertEquals, + assertExists, + assertThrows, + assertTrue, +} from './asserts.ts'; import { TEST } from './mod.ts'; import { DataRegistry } from '../cfds/base/data-registry.ts'; import { isBrowser } from '../base/common.ts'; @@ -1169,6 +1174,50 @@ export default function setup(): void { // Part 11: Auto-Close + Manual Close Serialization // ════════════════════════════════════════════════════════════════ + TEST( + 'AutoClose', + 'failed auto-close leaves the repo for retry, then opens a fresh one', + async (ctx) => { + const db = await ctx.createDB('ac-idle-teardown-failure', { + registry: kRegistry, + }); + try { + await db.readyPromise(); + const item = db.create('/data/items/x', kAutoCloseSchema, { + value: 'v1', + }); + await item.commit(); + await db.flushAll(); + const oldRepo = repoFor(db, '/data/items'); + assertExists(oldRepo); + + db._testForceTeardownFailures = 1; + await assertThrows( + () => p(oldRepo)._testTriggerIdleTimeout(), + 'auto-close rejects when teardown fails', + ); + assertTrue( + db.repository('/data/items') === oldRepo, + 'a failed teardown leaves the repo registered for retry', + ); + + const freshRepo = await db.open('/data/items'); + assertTrue(freshRepo !== oldRepo, 'open returns a fresh instance'); + assertTrue( + db.repository('/data/items') === freshRepo, + 'the fresh instance replaced the half-torn repo', + ); + assertEquals( + db.item('/data/items/x').get('value'), + 'v1', + 'committed data survives a failed teardown retry', + ); + } finally { + await db.close(); + } + }, + ); + TEST( 'AutoClose', 'manual closeRepo awaits in-flight auto-close', diff --git a/tests/closed-state.test.ts b/tests/closed-state.test.ts new file mode 100644 index 00000000..7413fa8c --- /dev/null +++ b/tests/closed-state.test.ts @@ -0,0 +1,1064 @@ +/** + * Tests for GoatDB instance lifecycle: booting -> ready -> closing -> closed. + * + * Assertions use the public lifecycle API (`state`, `isOpen`, `close()`, + * `readyPromise()`, and data operations), so they survive internal changes. A + * test that must prove an internal leak states its deviation inline. + */ + +import { + assertEquals, + assertThrows, + assertTrue, + withLogCapture, +} from './asserts.ts'; +import { TEST } from './mod.ts'; +import { GoatDB } from '../db/db.ts'; +import { Code, ServerError } from '../cfds/base/errors.ts'; +import { DataRegistry } from '../cfds/base/data-registry.ts'; +import { Item } from '../cfds/base/item.ts'; +import { kSchemaSession, type SchemaTypeSession } from '../cfds/base/schema.ts'; +import { generateSession, sessionToItem } from '../db/session.ts'; +import { pathExists, writeTextFile } from '../base/json-log/file-impl.ts'; +import { isBrowser } from '../base/common.ts'; +import { sleep } from '../base/time.ts'; +import { RepoClient } from '../net/client.ts'; +import { sendJSONToURL } from '../net/rest-api.ts'; +import { + blockedDBPath, + trackUnhandledRejections, + waitForState, +} from './lifecycle-fixtures.ts'; + +const kClosedStateSchema = { + ns: 'closed-state-test', + version: 1, + fields: { + value: { type: 'string', default: () => '' }, + num: { type: 'number', default: () => 0 }, + }, +} as const; + +// Higher version of the same namespace; proves the schema setter would mutate. +const kClosedStateSchemaV2 = { + ns: 'closed-state-test', + version: 2, + fields: { value: { type: 'string', default: () => '' } }, +} as const; + +const kRegistry = new DataRegistry(); +kRegistry.registerSchema(kClosedStateSchema); +kRegistry.registerSchema(kClosedStateSchemaV2); + +/** + * Asserts that an operation fails at the API boundary with the canonical + * serviceUnavailable error. + */ +async function assertServiceUnavailable( + fn: () => unknown, + message: string, +): Promise { + let caught: unknown; + try { + await fn(); + } catch (e) { + caught = e; + } + assertTrue( + caught instanceof ServerError && caught.code === Code.ServiceUnavailable, + message, + ); +} + +/** + * Observes one repo's RepoClients through the prototype: which clients armed a + * sync timer, and when the stalled peer's client started syncing. The patch is + * process-global, so callers MUST call restore() in a finally block. + */ +function probeRepoClients( + repoPath: string, + peers: ReadonlySet, + stalledPeer: string, +) { + const started: RepoClient[] = []; + const inFlight = Promise.withResolvers(); + const isOurs = (client: RepoClient) => + client.repoPath === repoPath && peers.has(client.serverUrl); + const originalStart = RepoClient.prototype.startSyncing; + const originalSync = RepoClient.prototype.sync; + RepoClient.prototype.startSyncing = function (): RepoClient { + if (isOurs(this)) started.push(this); + return originalStart.call(this); + }; + RepoClient.prototype.sync = function (): Promise { + if (isOurs(this) && this.serverUrl === stalledPeer) { + inFlight.resolve(); + } + return originalSync.call(this); + }; + const restore = () => { + RepoClient.prototype.startSyncing = originalStart; + RepoClient.prototype.sync = originalSync; + }; + return { started, inFlight: inFlight.promise, restore }; +} + +export default function setup(): void { + TEST( + 'ClosedState', + 'state and isOpen track the close lifecycle', + async (ctx) => { + const db = await ctx.createDB('closed-lifecycle', { + registry: kRegistry, + }); + try { + assertEquals(db.state, 'booting', 'booting before bootstrap'); + assertTrue(db.isOpen, 'booting instance accepts work'); + + await db.readyPromise(); + assertEquals(db.state, 'ready', 'ready after bootstrap'); + assertTrue(db.isOpen, 'ready instance accepts work'); + + await db.open('/data/items'); + const closeP = db.close(); + assertEquals(db.state, 'closing', 'closing while teardown runs'); + assertEquals(db.isOpen, false, 'closing instance rejects work'); + + await closeP; + assertEquals(db.state, 'closed', 'closed after teardown'); + assertEquals(db.isOpen, false, 'closed instance rejects work'); + assertEquals( + db.repository('/data/items'), + undefined, + 'teardown released repositories', + ); + } finally { + await db.close(); + } + }, + ); + + TEST('ClosedState', 'close is single-flight and idempotent', async (ctx) => { + const db = await ctx.createDB('closed-idempotent', { + registry: kRegistry, + }); + await db.readyPromise(); + + const first = db.close(); + const second = db.close(); + assertTrue( + first === second, + 'concurrent close() calls share the same promise', + ); + + await first; + await second; + await db.close(); + assertEquals(db.state, 'closed', 'instance stays closed'); + }); + + TEST( + 'ClosedState', + 'data operations on a closed instance reject with serviceUnavailable', + async (ctx) => { + const db = await ctx.createDB('closed-ops', { + registry: kRegistry, + }); + await db.readyPromise(); + await db.close(); + + await assertServiceUnavailable( + () => db.item('/data/items/x'), + 'item() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.create('/data/items/x', kClosedStateSchema, { value: 'x' }), + 'create() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.open('/data/items'), + 'open() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => + db.query({ + source: '/data/items', + predicate: () => true, + schema: kClosedStateSchema, + }), + 'query() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.loginWithMagicLinkEmail('closed@example.com'), + 'loginWithMagicLinkEmail() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.flushAll(), + 'flushAll() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.sync('/data/items'), + 'sync() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.getTrustPool(), + 'getTrustPool() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.count('/data/items'), + 'count() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.keys('/data/items'), + 'keys() throws serviceUnavailable', + ); + await assertServiceUnavailable( + () => db.logout(), + 'logout() throws serviceUnavailable', + ); + assertEquals( + db.loggedIn, + false, + 'closed instance reports no active session', + ); + }, + ); + + TEST( + 'ClosedState', + 'inspection stays available on a closed instance', + async (ctx) => { + const db = await ctx.createDB('closed-inspection', { + registry: kRegistry, + }); + await db.readyPromise(); + await db.close(); + + assertEquals( + db.repository('/data/items'), + undefined, + 'repository() is an honest lookup after close', + ); + assertEquals( + db.itemLoaded('/data/items/x'), + false, + 'itemLoaded() is false after close', + ); + assertEquals( + Array.from(db.clientsForRepo('/data/items')), + [], + 'clientsForRepo() is empty after close', + ); + assertEquals(db.state, 'closed', 'lifecycle stays inspectable'); + assertEquals(db.isOpen, false, 'lifecycle stays inspectable'); + }, + ); + + TEST( + 'ClosedState', + 'logout on a closed instance fails without wiping data', + async (ctx) => { + const dataPath = await ctx.tempDir('closed-logout'); + const db = await ctx.createDB('closed-logout', { registry: kRegistry }); + try { + await db.readyPromise(); + db.create('/data/items/x', kClosedStateSchema, { value: 'x' }); + await db.flushAll(); + await db.close(); + + await assertServiceUnavailable( + () => db.logout(), + 'logout() throws serviceUnavailable', + ); + assertTrue( + await pathExists(dataPath), + 'failed logout must keep the data directory', + ); + } finally { + await db.close(); + } + }, + ); + + TEST( + 'ClosedState', + 'close persists unflushed edits from open and auto-closed repos', + async (ctx) => { + const db = await ctx.createDB('closed-durability', { + registry: kRegistry, + repoInactivityTimeoutMs: 100, + }); + const openItem = db.create('/data/open/item', kClosedStateSchema, { + value: 'initial', + }); + const autoItem = db.create('/data/auto/item', kClosedStateSchema, { + value: 'initial', + }); + await Promise.all([openItem.commit(), autoItem.commit()]); + await db.flushAll(); + + // Force the auto repo closed so close() must persist through the + // _commitForClose/_acquireRepoForClose path; the open repo exercises + // _closeRepo/_commitTo. + const autoRepo = db.repository('/data/auto') as unknown as { + _testTriggerIdleTimeout(): Promise; + }; + await autoRepo._testTriggerIdleTimeout(); + assertEquals( + db.repository('/data/auto'), + undefined, + 'auto repo closed', + ); + + openItem.set('value', 'open-dirty'); + autoItem.set('value', 'auto-dirty'); + await db.close(); + + const db2 = await ctx.createDB('closed-durability', { + registry: kRegistry, + repoInactivityTimeoutMs: 0, + }); + try { + await db2.readyPromise(); + await Promise.all([db2.open('/data/open'), db2.open('/data/auto')]); + assertEquals( + db2.item('/data/open/item').get('value'), + 'open-dirty', + 'unflushed edit in an open repo survives close()', + ); + assertEquals( + db2.item('/data/auto/item').get('value'), + 'auto-dirty', + 'unflushed edit in an auto-closed repo survives close()', + ); + } finally { + await db2.close(); + } + }, + ); + + TEST( + 'ClosedState', + 'close rejects and retries when a pending item commit fails', + async (ctx) => { + // Filesystem trick: replaces the repo log path with a directory so the + // close-time reopen fails. Not portable to OPFS/browser. + if (isBrowser() || typeof Deno === 'undefined') return; + const dataPath = await ctx.tempDir('close-commit-fail'); + const db = await ctx.createDB('close-commit-fail', { + registry: kRegistry, + repoInactivityTimeoutMs: 100, + }); + const repoFile = `${dataPath}/data/items.goat`; + try { + const item = db.create('/data/items/x', kClosedStateSchema, { + value: 'initial', + }); + await item.commit(); + await db.flushAll(); + + // Auto-close the repo so close() must persist the dirty item through + // the close-time reopen (the open-repo path commits via the repo's + // own live file handle and would not hit the directory). + const repo = db.repository('/data/items') as unknown as { + _testTriggerIdleTimeout(): Promise; + }; + await repo._testTriggerIdleTimeout(); + assertEquals( + db.repository('/data/items'), + undefined, + 'repo auto-closed', + ); + + // Make the repo log path a directory so the close-time reopen fails. + await Deno.remove(repoFile); + await Deno.mkdir(repoFile); + + item.set('value', 'pending'); + await assertThrows(() => db.close(), 'close rejects on failed commit'); + assertEquals(db.state, 'failed', 'failed close reports failed'); + + await Deno.remove(repoFile); + await db.close(); // retried close completes + assertEquals(db.state, 'closed', 'retried close completes'); + + const db2 = await ctx.createDB('close-commit-fail', { + registry: kRegistry, + }); + try { + await db2.readyPromise(); + await db2.open('/data/items'); + assertEquals( + db2.item('/data/items/x').get('value'), + 'pending', + 'retried commit persisted', + ); + } finally { + await db2.close(); + } + } finally { + await Deno.remove(repoFile).catch(() => {}); + await db.close().catch(() => {}); + } + }, + ); + + TEST( + 'ClosedState', + 'close cancels an open deferred behind an in-flight repo close', + async (ctx) => { + const db = await ctx.createDB('closed-deferred-open', { + registry: kRegistry, + }); + await db.readyPromise(); + await db.open('/data/items'); + + const [repoClose, deferred, teardown] = await Promise.allSettled([ + db.closeRepo('/data/items'), + db.open('/data/items'), + db.close(), + ]); + + assertEquals(repoClose.status, 'fulfilled', 'closeRepo fulfills'); + assertEquals(teardown.status, 'fulfilled', 'close fulfills'); + assertEquals(deferred.status, 'rejected', 'deferred open is cancelled'); + const reason = (deferred as PromiseRejectedResult).reason; + assertTrue( + reason instanceof ServerError && + reason.code === Code.ServiceUnavailable, + 'deferred open fails with serviceUnavailable', + ); + assertEquals(db.state, 'closed', 'close wins the ordering race'); + assertEquals(db.repository('/data/items'), undefined, 'no repo survives'); + assertTrue(db.close() === db.close(), 'close() reuses the promise'); + }, + ); + + TEST( + 'ClosedState', + 'close aborts an in-flight peer sync instead of waiting on it', + async (ctx) => { + // Two peers: one resets every connection, so its client finishes syncing + // and arms a repeating timer first; one accepts and never answers, so its + // client stalls and close() aborts the open mid-flight. + if (isBrowser() || typeof Deno === 'undefined') return; + const repoPath = '/data/items'; + const resetting = Deno.listen({ hostname: '127.0.0.1', port: 0 }); + const stalled = Deno.listen({ hostname: '127.0.0.1', port: 0 }); + const urlOf = (listener: Deno.Listener) => + `http://127.0.0.1:${(listener.addr as Deno.NetAddr).port}`; + const failPeer = urlOf(resetting); + const stalledPeer = urlOf(stalled); + // The iterator rejects once the listener closes; that is teardown. + const resets = (async () => { + for await (const conn of resetting) conn.close(); + })().catch(() => {}); + try { + const db = await ctx.createDB('closed-peer-stall', { + registry: kRegistry, + peers: [failPeer, stalledPeer], + }); + // An armed sync timer has no public surface, so this regression reads + // clients directly (see the file header). + const probe = probeRepoClients( + repoPath, + new Set([failPeer, stalledPeer]), + stalledPeer, + ); + try { + await db.readyPromise(); + const openP = db.open(repoPath); + // Handle the rejection up front: the open is doomed once close starts. + const openSettled = openP.then( + () => 'resolved' as const, + (e: unknown) => e, + ); + assertTrue( + await Promise.race([ + probe.inFlight.then(() => true), + sleep(3000).then(() => false), + ]), + 'the second peer client stalled on its initial sync', + ); + const raced = await Promise.race([ + db.close().then( + () => 'closed' as const, + (e: unknown) => `failed: ${e}`, + ), + sleep(2000).then(() => 'timeout' as const), + ]); + assertEquals( + raced, + 'closed', + 'close() must not wait on the peer response or its timeout', + ); + assertEquals( + db.state, + 'closed', + 'close finished while the peer stalled', + ); + const open = await openSettled; + assertTrue( + open instanceof ServerError && + open.code === Code.ServiceUnavailable, + 'an open that loses the race to close rejects with ' + + `serviceUnavailable, got ${open}`, + ); + assertEquals( + db.repository(repoPath), + undefined, + 'teardown released the half-open repo', + ); + assertTrue( + probe.started.length > 0 && + probe.started.every((c) => c.closed), + 'every peer client that armed a sync timer is closed', + ); + } finally { + probe.restore(); + await db.close(); + } + } finally { + stalled.close(); + resetting.close(); + await resets; + } + }, + ); + + TEST( + 'ClosedState', + 'a stalled response body cannot outlive the request timeout', + async (ctx) => { + if (isBrowser() || typeof Deno === 'undefined') return; + const server = Deno.serve( + { hostname: '127.0.0.1', port: 0 }, + () => + new Response( + new ReadableStream({ + start(c) { + c.enqueue(new TextEncoder().encode('[')); + }, + }), + ), + ); + try { + const outcome = await Promise.race([ + sendJSONToURL( + `http://127.0.0.1:${server.addr.port}/batch-sync`, + undefined, + {}, + undefined, + 50, + ).then(() => 'resolved', () => 'rejected'), + sleep(2000).then(() => 'timeout'), + ]); + assertEquals( + outcome, + 'rejected', + 'timeout cancels the stalled body read', + ); + } finally { + server.shutdown(); + } + }, + ); + + TEST( + 'ClosedState', + 'a failed teardown reports failed and a later close() retries', + async (ctx) => { + const db = await ctx.createDB('closed-teardown-retry', { + registry: kRegistry, + }); + try { + await db.readyPromise(); + await db.open('/data/items'); + + db._testForceTeardownFailures = 2; + await assertThrows( + () => db.close(), + 'close() rejects when teardown fails', + ); + assertEquals( + db.state, + 'failed', + 'a failed teardown is not reported as closed', + ); + assertEquals(db.isOpen, false, 'failed instance rejects work'); + assertEquals(db.loggedIn, false, 'failed instance reports no session'); + + await assertThrows( + () => db.close(), + 'the retry rejects while teardown still fails', + ); + assertEquals( + db.state, + 'failed', + 'a second failed retry still reports failed', + ); + + await db.close(); + assertEquals( + db.state, + 'closed', + 'a later close() retries the partial teardown', + ); + assertEquals(db.isOpen, false, 'closed instance rejects work'); + assertEquals(db.loggedIn, false, 'no session after retried close'); + assertEquals( + db.repository('/data/items'), + undefined, + 'retry released the repository', + ); + } finally { + await db.close(); + } + }, + ); + + TEST( + 'ClosedState', + 'failed bootstrap close rejects and retains accepted edits', + async (ctx) => { + if (isBrowser()) return; // OPFS has no "path occupied by a file" failure + const dir = await ctx.tempDir('closed-failed'); + const dbPath = `${dir}/not-a-directory`; + assertTrue(await writeTextFile(dbPath, 'x'), 'blocker file written'); + + const db = new GoatDB({ + path: dbPath, + orgId: 'test-org', + registry: kRegistry, + }); + db.create('/data/items/x', kClosedStateSchema, { value: 'pending' }); + let caught: unknown; + try { + await db.readyPromise(); + } catch (e) { + caught = e; + } + assertTrue(caught !== undefined, 'readyPromise() rejects'); + assertEquals(db.state, 'failed', 'state reports the failed bootstrap'); + assertEquals(db.isOpen, false, 'failed instance rejects work'); + await assertServiceUnavailable( + () => db.getTrustPool(), + 'getTrustPool() fails closed after a failed bootstrap', + ); + + await assertThrows( + () => db.close(), + 'close rejects after failed bootstrap', + ); + assertEquals(db.state, 'failed', 'failed close keeps state failed'); + const internals = db as unknown as { _items: Map }; + assertEquals(internals._items.size, 1, 'accepted edit retained'); + }, + ); + + TEST('ClosedState', 'failed bootstrap is handled', async (ctx) => { + if (isBrowser()) return; // OPFS has no "path occupied by a file" failure + const dir = await ctx.tempDir('closed-failed-state-only'); + const dbPath = await blockedDBPath(dir, 'not-a-directory'); + const rejections = trackUnhandledRejections(); + const db = new GoatDB({ path: dbPath, orgId: 'test-org' }); + try { + await waitForState(db, 'failed'); + await new Promise((resolve) => setTimeout(resolve, 0)); + assertEquals(db.state, 'failed', 'failed bootstrap state'); + assertEquals(rejections.reasons.length, 0, 'no unhandled rejection'); + } finally { + rejections.restore(); + await db.close().catch(() => {}); + } + }); + + TEST( + 'ClosedState', + 'item load failure rejects readyPromise without an unhandled rejection', + async (ctx) => { + if (isBrowser()) return; // OPFS has no "path occupied by a file" failure + const dir = await ctx.tempDir('closed-item-load'); + const dbPath = await blockedDBPath(dir, 'not-a-directory'); + const rejections = trackUnhandledRejections(); + const db = new GoatDB({ path: dbPath, orgId: 'test-org' }); + try { + // Created while booting, so the load targets the doomed bootstrap. + const item = db.item('/data/items/thing'); + const outcome = item.readyPromise().then( + () => 'resolved' as const, + (e: unknown) => e, + ); + await waitForState(db, 'failed'); + const caught = await outcome; + assertTrue( + caught instanceof Error, + `readyPromise() must reject with the load error, got ${caught}`, + ); + assertEquals(item.ready, false, 'a failed load never reports ready'); + await new Promise((resolve) => setTimeout(resolve, 0)); + assertEquals(rejections.reasons.length, 0, 'no unhandled rejection'); + } finally { + rejections.restore(); + await db.close().catch(() => {}); + } + }, + ); + + TEST( + 'ClosedState', + 'mutators on a held item throw after close and drop no edits', + async (ctx) => { + const db = await ctx.createDB('closed-item-mutators', { + registry: kRegistry, + }); + const item = db.create('/data/items/x', kClosedStateSchema, { + value: 'committed', + }); + await item.commit(); + await db.close(); + + const mutators: Array<[string, () => unknown]> = [ + ['set', () => item.set('value', 'lost-edit')], + ['setMulti', () => item.setMulti({ value: 'lost-multi' })], + ['delete', () => item.delete('value')], + [ + 'schema setter', + () => (item.schema = + kClosedStateSchemaV2 as unknown as typeof kClosedStateSchema), + ], // V2 proves setter throws; cast bypasses pinned V1 setter type + ['isDeleted setter', () => (item.isDeleted = true)], + ]; + for (const [name, mutate] of mutators) { + await assertServiceUnavailable(mutate, `${name} throws after close`); + } + const db2 = await ctx.createDB('closed-item-mutators', { + registry: kRegistry, + }); + try { + await db2.readyPromise(); + await db2.open('/data/items'); + assertEquals( + db2.item('/data/items/x').get('value'), + 'committed', + 'held item edits were never dropped', + ); + } finally { + await db2.close(); + } + }, + ); + + TEST( + 'ClosedState', + 'close during bootstrap cannot become ready later', + async (ctx) => { + await withLogCapture(async (captured) => { + const db = await ctx.createDB('closed-during-boot', { + registry: kRegistry, + }); + assertEquals(db.state, 'booting', 'bootstrap has not completed'); + + await db.close(); + assertEquals( + db.state, + 'closed', + 'close during boot never becomes ready', + ); + assertEquals(db.isOpen, false, 'closed instance rejects work'); + assertEquals( + db.repository('/sys/sessions'), + undefined, + 'bootstrap repositories are torn down', + ); + await assertServiceUnavailable( + () => db.readyPromise(), + 'readyPromise() rejects after close', + ); + await assertServiceUnavailable( + () => db.getTrustPool(), + 'getTrustPool() rejects after close', + ); + await db.getTrustPool().catch(() => {}); + assertTrue( + !captured.some((e) => e.severity === 'ERROR'), + 'close during bootstrap must not log an ERROR', + ); + }); + }, + ); + + TEST( + 'ClosedState', + 'close during bootstrap with peers', + async (ctx) => { + if (isBrowser() || typeof Deno === 'undefined') return; + // Dead-port peer: bootstrap never connects (sys-repo opens skip initial + // sync), so it only forces schedulers to be born mid-close. + const db = await ctx.createDB('closed-peer-during-boot', { + registry: kRegistry, + peers: ['http://127.0.0.1:1'], + }); + // Pre-wait abort runs before schedulers exist; the post-bootstrap abort + // must reap them. Private-field read: a leaked interval has no public + // surface. + const unhandled = trackUnhandledRejections(); + try { + await db.close(); + assertEquals(db.state, 'closed', 'close completes'); + const schedulers = (db as unknown as { + _syncSchedulers?: unknown[]; + })._syncSchedulers; + assertEquals( + schedulers, + undefined, + 'no sync scheduler survives close-during-bootstrap', + ); + assertTrue( + unhandled.reasons.length === 0, + 'close-during-bootstrap leaves no unhandled rejection', + ); + } finally { + unhandled.restore(); + await db.close(); + } + }, + ); + TEST( + 'ClosedState', + 'close during boot persists accepted edits', + async (ctx) => { + const db = await ctx.createDB('closed-during-boot-durability', { + registry: kRegistry, + }); + db.create('/data/items/x', kClosedStateSchema, { value: 'pending' }); + await db.close(); + assertEquals(db.state, 'closed', 'close completes'); + const db2 = await ctx.createDB('closed-during-boot-durability', { + registry: kRegistry, + }); + await db2.readyPromise(); + await db2.open('/data/items'); + assertEquals( + db2.item('/data/items/x').get('value'), + 'pending', + 'edit created during boot survived close', + ); + await db2.close(); + }, + ); + + TEST( + 'ClosedState', + 'a failed close keeps /sys repos retryable and persists dirty sessions', + async (ctx) => { + const session = await generateSession('f1-user'); + const rec = await sessionToItem(session, kRegistry); + const db = await ctx.createDB('closed-f1-sys', { registry: kRegistry }); + await db.readyPromise(); + const item = db.create(`/sys/sessions/${session.id}`, kSchemaSession, { + id: rec.get('id'), + publicKey: 'bad-key', + expiration: rec.get('expiration'), + owner: rec.get('owner'), + }); + await db.closeRepo('/sys/sessions'); // commit fails; item stays dirty + item.set('publicKey', rec.get('publicKey')); + db._testForceCommitFailures = 0; + await assertThrows(() => db.close(), 'close rejects'); + assertEquals(db.state, 'failed', 'failed close reports failed'); + db._testForceCommitFailures = -1; + await db.close(); + assertEquals(db.state, 'closed', 'retried close completes'); + const db2 = await ctx.createDB('closed-f1-sys', { registry: kRegistry }); + await db2.readyPromise(); + const repo = await db2.open('/sys/sessions'); + const got = repo.valueForKey(session.id); + assertTrue(got?.[0].get('owner') === 'f1-user', 'session persisted'); + await db2.close(); + await db.close(); + }, + ); + + TEST( + 'ClosedState', + 'a failed sibling commit is recovered by repo-close retries', + async (ctx) => { + const db = await ctx.createDB('closed-f2-siblings', { + registry: kRegistry, + repoInactivityTimeoutMs: 100, + }); + await db.readyPromise(); + const a = db.create('/data/items/a', kClosedStateSchema); + const b = db.create('/data/items/b', kClosedStateSchema); + await Promise.all([a.commit(), b.commit()]); + await db.flushAll(); + const repo = db.repository('/data/items') as unknown as { + _testTriggerIdleTimeout(): Promise; + }; + await repo._testTriggerIdleTimeout(); + a.set('value', 'a1'); + b.set('value', 'b1'); + db._testForceCommitFailures = 1; // allow a; fail b + await db.close(); // repo close retries b; close reports success once + assertEquals(db.state, 'closed', 'single close succeeds'); + const db2 = await ctx.createDB('closed-f2-siblings', { + registry: kRegistry, + repoInactivityTimeoutMs: 0, + }); + await db2.readyPromise(); + await db2.open('/data/items'); + assertEquals(db2.item('/data/items/a').get('value'), 'a1', 'a kept'); + assertEquals(db2.item('/data/items/b').get('value'), 'b1', 'b kept'); + await db2.close(); + await db.close(); + }, + ); + + TEST( + 'ClosedState', + 'a NaN-valued sibling commit recovered by repo-close retry closes cleanly', + async (ctx) => { + const db = await ctx.createDB('closed-f1-nan', { registry: kRegistry }); + await db.readyPromise(); + const a = db.create('/data/items/a', kClosedStateSchema); + const b = db.create('/data/items/b', kClosedStateSchema); + await Promise.all([a.commit(), b.commit()]); + const repo = db.repository('/data/items') as unknown as { + _testTriggerIdleTimeout(): Promise; + }; + await repo._testTriggerIdleTimeout(); + a.set('value', 'a1'); + b.set('value', 'b1'); + b.set('num', NaN); // NaN never equals itself; rebase always diffs it + db._testForceCommitFailures = 1; // allow a; fail b + await db.close(); + assertEquals(db.state, 'closed', 'single close succeeds despite NaN'); + // Enforce the durable contract through persisted state only: a + // successful close() must persist every accepted edit. NaN itself is + // not representable in the storage format (JSON coerces it to null), + // so the observable round-trip value is null; the crucial invariant is + // that the retried close neither wedges nor drops the sibling edit. + const db2 = await ctx.createDB('closed-f1-nan', { registry: kRegistry }); + await db2.readyPromise(); + await db2.open('/data/items'); + const entry = db2.repository('/data/items')!.valueForKey('b')![0]; + assertEquals(entry.get('value'), 'b1', 'b kept'); + assertEquals(entry.get('num'), null, 'NaN round-trips to null'); + await db2.close(); + }, + ); + + TEST( + 'ClosedState', + 'a failed close-time commit leaks no half-open repo or file handle', + async (ctx) => { + const db = await ctx.createDB('closed-half-open-leak', { + registry: kRegistry, + repoInactivityTimeoutMs: 100, + }); + await db.readyPromise(); + const item = db.create('/data/items/x', kClosedStateSchema); + await item.commit(); + const repo = db.repository('/data/items') as unknown as { + _testTriggerIdleTimeout(): Promise; + }; + await repo._testTriggerIdleTimeout(); + item.set('value', 'pending'); + db._testForceCommitFailures = 0; // fail the close-time reopen + await assertThrows(() => db.close(), 'close rejects'); + assertEquals(db.state, 'failed', 'failed close reports failed'); + const internals = db as unknown as { + _files: Map; + _repositories: Map; + }; + assertEquals( + internals._files.has('/data/items') || + internals._repositories.has('/data/items'), + false, + 'no half-open repo or leaked handle', + ); + db._testForceCommitFailures = -1; + await db.close(); + const db2 = await ctx.createDB('closed-half-open-leak', { + registry: kRegistry, + }); + await db2.readyPromise(); + await db2.open('/data/items'); + assertEquals( + db2.item('/data/items/x').get('value'), + 'pending', + 'dirty edit persisted on retried close', + ); + await db2.close(); + }, + ); + + TEST( + 'ClosedState', + 'retried close persists dirty edits after a detach-time teardown failure', + async (ctx) => { + const db = await ctx.createDB('cd-retry', { registry: kRegistry }); + await db.readyPromise(); + const item = db.create('/data/items/x', kClosedStateSchema); + await item.commit(); + const repo = db.repository('/data/items')!; + item.set('value', 'v2'); + db._testForceFileCloseFailures = 1; + await assertThrows( + () => db._requestRepoIdleClose(repo), + 'auto-close fails after detach', + ); + await db.close(); + const db2 = await ctx.createDB('cd-retry', { registry: kRegistry }); + await db2.readyPromise(); + await db2.open('/data/items'); + assertEquals(db2.item('/data/items/x').get('value'), 'v2', 'kept'); + await db2.close(); + }, + ); + + TEST( + 'ClosedState', + 'close persists a local edit across a divergent remote write', + async (ctx) => { + const db = await ctx.createDB('closed-f2-divergent', { + registry: kRegistry, + }); + await db.readyPromise(); + const x = db.create('/data/items/x', kClosedStateSchema); + await x.commit(); + x.set('value', 'local'); + const repo = db.repository('/data/items')!; + const remote = new Item( + { schema: kClosedStateSchema, data: { value: 'remote', num: 5 } }, + kRegistry, + ); + // The mid-commit divergence race (the head moves while a write is in + // flight) is not reproducible through the public API: the repo rejects + // concurrent writes to a key while a commit is pending. The parity + // invariant below pins the user-facing contract that replaces it. + await repo.setValueForKey('x', remote, undefined); + // Wait (public reads only) for the remote head to be merged into the + // local view; sync delivery is async in the browser runtime. + const deadline = Date.now() + 1000; + while (Date.now() < deadline && x.get('num') !== 5) { + await sleep(10); + } + // Parity invariant: everything observable on the item just before a + // successful close() must be exactly what a fresh open reads. + const value = x.get('value'); + const num = x.get('num'); + await db.close(); + const db2 = await ctx.createDB('closed-f2-divergent', { + registry: kRegistry, + }); + await db2.readyPromise(); + await db2.open('/data/items'); + const entry = db2.repository('/data/items')!.valueForKey('x')![0]; + assertEquals(entry.get('value'), value, 'observable value persisted'); + assertEquals(entry.get('num'), num, 'observable num persisted'); + await db2.close(); + }, + ); +} diff --git a/tests/db-trusted.test.ts b/tests/db-trusted.test.ts index f963da17..ee667c40 100644 --- a/tests/db-trusted.test.ts +++ b/tests/db-trusted.test.ts @@ -53,9 +53,9 @@ export default function setup(): void { assertEquals(db.registry, kDataRegistry); // Should start not ready - assertEquals(db.ready, false); + assertEquals(db.state, 'booting'); await db.readyPromise(); - assertEquals(db.ready, true); + assertEquals(db.state, 'ready'); // Authentication state depends on environment if (isBrowser()) { diff --git a/tests/db-untrusted.test.ts b/tests/db-untrusted.test.ts index 92a31aed..2eebe44a 100644 --- a/tests/db-untrusted.test.ts +++ b/tests/db-untrusted.test.ts @@ -35,11 +35,11 @@ export default function setup(): void { assertEquals(db.registry, kDataRegistry); // Should start not ready - assertEquals(db.ready, false); + assertEquals(db.state, 'booting'); // Wait for it to be ready await db.readyPromise(); - assertEquals(db.ready, true); + assertEquals(db.state, 'ready'); // Should start with root session assertEquals(db.loggedIn, true); diff --git a/tests/json-log-formats.test.ts b/tests/json-log-formats.test.ts index 130c9a3c..2483a1d6 100644 --- a/tests/json-log-formats.test.ts +++ b/tests/json-log-formats.test.ts @@ -10,7 +10,7 @@ * session. Verification reads therefore use a fresh session (close + reopen). */ import { TEST } from './mod.ts'; -import { assertEquals, assertTrue } from './asserts.ts'; +import { assertEquals, assertTrue, withLogCapture } from './asserts.ts'; import { isBrowser } from '../base/common.ts'; import * as path from '../base/path.ts'; import { FileImplGet } from '../base/json-log/file-impl.ts'; @@ -26,27 +26,6 @@ import { import { GoatDB } from '../db/db.ts'; import { DataRegistry } from '../cfds/base/data-registry.ts'; import { BINARY_MAGIC } from '../base/core-types/encoding/binary-commit.ts'; -import { - getGlobalLoggerStreams, - setGlobalLoggerStreams, -} from '../logging/log.ts'; -import type { LogEntry } from '../logging/log.ts'; -import type { NormalizedLogEntry } from '../logging/entry.ts'; - -// Swaps in a capturing log stream for the duration of fn, then restores. -function withLogCapture( - fn: (captured: NormalizedLogEntry[]) => Promise, -): Promise { - const captured: NormalizedLogEntry[] = []; - const prev = getGlobalLoggerStreams(); - setGlobalLoggerStreams([{ - appendEntry(e: NormalizedLogEntry): void { - captured.push(e); - }, - }]); - return fn(captured).finally(() => setGlobalLoggerStreams(prev)); -} - type LogFormat = 'goat' | 'jsonl'; const gTextEncoder = new TextEncoder(); diff --git a/tests/lifecycle-fixtures.ts b/tests/lifecycle-fixtures.ts new file mode 100644 index 00000000..8d332919 --- /dev/null +++ b/tests/lifecycle-fixtures.ts @@ -0,0 +1,38 @@ +import { writeTextFile } from '../base/json-log/file-impl.ts'; +import type { DBState, GoatDB } from '../db/db.ts'; + +export async function blockedDBPath( + dir: string, + name: string, +): Promise { + const dbPath = `${dir}/${name}`; + if (!await writeTextFile(dbPath, 'x')) { + throw new Error('Blocker write failed'); + } + return dbPath; +} + +export function waitForState(db: GoatDB, state: DBState): Promise { + if (db.state === state) return Promise.resolve(); + return new Promise((resolve) => { + const detach = db.attach('LifecycleChanged', (next: DBState) => { + if (next !== state) return; + detach(); + resolve(); + }); + }); +} + +type Proc = { + on(event: string, listener: (reason: unknown) => void): void; + off(event: string, listener: (reason: unknown) => void): void; +}; + +/** Deno and Node expose the process rejection hook. */ +export function trackUnhandledRejections() { + const reasons: unknown[] = []; + const listener = (reason: unknown) => reasons.push(reason); + const proc = (globalThis as { process: Proc }).process; + proc.on('unhandledRejection', listener); + return { reasons, restore: () => proc.off('unhandledRejection', listener) }; +} diff --git a/tests/react-db-ready.test.ts b/tests/react-db-ready.test.ts new file mode 100644 index 00000000..c5e72499 --- /dev/null +++ b/tests/react-db-ready.test.ts @@ -0,0 +1,80 @@ +import React from 'react'; +import { act, create } from 'react-test-renderer'; +import { assertEquals } from './asserts.ts'; +import { TEST } from './mod.ts'; +import { GoatDBContext, useDBReady } from '../react/hooks.ts'; +import { GoatDB } from '../db/db.ts'; +import { isBrowser } from '../base/common.ts'; +import { blockedDBPath, waitForState } from './lifecycle-fixtures.ts'; +const Probe = () => useDBReady(); +const mount = (db: GoatDB) => { + let r!: ReturnType; + act(() => { + r = create( + React.createElement( + GoatDBContext.Provider, + { value: { db } }, + React.createElement(Probe), + ), + ); + }); + return r; +}; +const text = (r: ReturnType) => r.toJSON() as unknown; +export default function setup(): void { + TEST('ReactDbReady', 'useDBReady tracks close while mounted', async (ctx) => { + const db = await ctx.createDB('react-db-ready'); + await db.readyPromise(); + const early = mount(db); + assertEquals(text(early), 'ready', 'ready before close'); + const closing = db.close(); + const late = mount(db); // while state is 'closing' + await act(async () => await closing); + assertEquals(text(early), 'error', 'mounted consumer sees close'); + assertEquals(text(late), 'error', 'late mount sees close'); + [early, late].forEach((r) => r.unmount()); + }); + + TEST('ReactDbReady', 'useDBReady maps failure to error', async (ctx) => { + if (isBrowser()) return; // OPFS has no "path occupied by a file" failure + const dir = await ctx.tempDir('react-db-failed'); + const dbPath = await blockedDBPath(dir, 'not-a-directory'); + const db = new GoatDB({ path: dbPath, orgId: 'test-org' }); + const rendered = mount(db); + try { + assertEquals(text(rendered), 'loading', 'booting maps to loading'); + await waitForState(db, 'failed'); + await act(async () => {}); + assertEquals(text(rendered), 'error', 'failed maps to error'); + } finally { + rendered.unmount(); + await db.close().catch(() => {}); + } + }); + + TEST( + 'ReactDbReady', + 'does not leak lifecycle listeners across mount/unmount', + async (ctx) => { + const db = await ctx.createDB('react-db-ready-leak'); + await db.readyPromise(); + const baseline = db.listenerCount('LifecycleChanged'); + for (let i = 0; i < 5; i++) { + const rendered = mount(db); + assertEquals( + db.listenerCount('LifecycleChanged'), + baseline + 1, + 'mount adds one listener', + ); + act(() => { + rendered.unmount(); + }); + assertEquals( + db.listenerCount('LifecycleChanged'), + baseline, + 'unmount releases listener', + ); + } + }, + ); +} diff --git a/tests/test-registry.ts b/tests/test-registry.ts index 4a1ad959..96a3a76f 100644 --- a/tests/test-registry.ts +++ b/tests/test-registry.ts @@ -62,6 +62,8 @@ import setupHashMap from './hash-map.test.ts'; import setupCoreValueCompare from './core-value-compare.test.ts'; import setupDocsMdxEscape from './docs-mdx-escape.test.ts'; import setupAutoClose from './auto-close.test.ts'; +import setupClosedState from './closed-state.test.ts'; +import setupReactDbReady from './react-db-ready.test.ts'; /** * Registers all test suites with the default TestsRunner. @@ -117,6 +119,8 @@ export async function registerAllTests(): Promise { // INTEGRATION TESTS (100-500ms each) - Multiple components, file I/O setupLiveQuery(); // Live query membership updates on ManagedItem edits setupAutoClose(); // Auto-close of inactive repos and queries + setupClosedState(); // DB instance closed state + setupReactDbReady(); // useDBReady follows DB lifecycle transitions setupWriteFailure(); // WriteFailure event after 3 consecutive I/O failures setupTrusted(); // Database operations in trusted mode setupUntrusted(); // Database operations in untrusted mode diff --git a/tests/tests-entry-browser.ts b/tests/tests-entry-browser.ts index ed9cc0f4..4a7647f6 100644 --- a/tests/tests-entry-browser.ts +++ b/tests/tests-entry-browser.ts @@ -40,6 +40,8 @@ import setupLiveQuery from './live-query.test.ts'; import setupHashMap from './hash-map.test.ts'; import setupCoreValueCompare from './core-value-compare.test.ts'; import setupAutoClose from './auto-close.test.ts'; +import setupClosedState from './closed-state.test.ts'; +import setupReactDbReady from './react-db-ready.test.ts'; import { getEnvVar } from '../base/os.ts'; import { notReached } from '../base/error.ts'; @@ -63,6 +65,7 @@ class TestConsoleLogStream implements LogStream { * Browser test entry point - all browser-compatible existing tests. */ async function main(): Promise { + (globalThis as any).global = globalThis; // react-test-renderer needs Node's global in browser // Wipe OPFS to prevent stale data from previous runs contaminating results await clearOPFS(); @@ -93,6 +96,8 @@ async function main(): Promise { // INTEGRATION TESTS (100-500ms each) - Multiple components, file I/O setupLiveQuery(); // Live query membership updates on ManagedItem edits setupAutoClose(); // Auto-close of inactive repos and queries + setupClosedState(); // DB instance closed state + setupReactDbReady(); // useDBReady follows DB lifecycle transitions setupTrusted(); // Database operations in trusted mode - CRITICAL for browser setupStaticAssetsEndpoint(); // File serving and asset management