___                 _   ___  ___
 |   \ ___ __ ___ _ _| |_|   \| _ )
 | |) / -_) _/ -_) ' \  _| |) | _ \
 |___/\___\__\___|_||_\__|___/|___/

DecentDB is an embedded relational database engine built with Rust, focused on durable ACID writes, fast reads, and predictable correctness.

It preserves a one writer and many concurrent readers concurrency model under snapshot isolation, with local on-disk databases able to coordinate multiple native OS processes through WAL locks, reader slots, and a rebuildable coordination sidecar.

Features

• 🔒 ACID Transactions - Write-ahead logging with crash-safe recovery
• 🔐 Local Data Security - Transparent data encryption, durable row policies, projection masks, audit context functions, and queryable security audit events
• 🌳 B+Tree Storage - Efficient tables and secondary indexes with page caching
• 🔁 Local-First Sync - Durable change journals, public changeset APIs, production HTTP/WebSocket relay, shape subscriptions, scoped peer replication, conflict workflows, operational doctor tooling, CLI commands, and .NET/web helpers
• 📡 Reactive Subscriptions - In-process table, range, query, and change-stream watches deliver committed invalidation events with LSN boundaries, bounded lag handling, C ABI JSON polling, and Python/Go direct helpers
• 🖥️ Built-In HTTP Server And Web Console - decentdb serve exposes a local HTTP API and embedded browser console for inspection, SQL execution, schema browsing, EXPLAIN, CSV export, and scripting
• 🌐 WASM Browser OPFS Support - @decentdb/web runs DecentDB in a Dedicated Worker with an OPFS-backed VFS, browser-app SQL profile metadata, transactions, prepared statement paging, binary result transport, checkpoint, import/export, persistence helpers, relay apply-before-ack helpers, and browser smoke/benchmark coverage
• 🐘 PostgreSQL-like SQL - Familiar DDL/DML syntax with JOINs (INNER, LEFT, RIGHT, FULL OUTER, CROSS, NATURAL), CTEs (including WITH RECURSIVE), subqueries, window functions, and rich types (UUID, DECIMAL, native TIMESTAMP)
• 🔎 SQLite/PostgreSQL Compatibility Helpers - Safe SQLite-style PRAGMAs, sqlite_schema, minimal information_schema, generate_series, main./temp. qualifiers, and query-time built-in collations for easier tool and migration onboarding
• 🧩 Lua Extensions - Manifest-declared Lua packages add scalar functions, table-valued functions, aggregates, and query-time collations with explicit install, enable, content-hash trust, CLI, Rust, and C ABI lifecycle APIs
• 🕒 Native TIMESTAMP Type - DATE/TIMESTAMP columns stored as int64 microseconds since Unix epoch (UTC); correct ORDER BY and EXTRACT(YEAR|MONTH|DAY|HOUR|MINUTE|SECOND FROM ...), with native bind/read in all bindings
• 👥 Concurrent Reads - Snapshot isolation allows multiple readers with one writer
• 🔐 Cross-Process WAL Coordination - Native byte-range locks and a rebuildable .coord sidecar coordinate local on-disk databases across processes, preserving one-writer/many-reader snapshot safety with diagnostics
• 🚦 Queued Write Concurrency - Engine-owned bounded write queue with timeouts, metrics, and strict durable group commit for predictable in-process concurrent writes
• 📈 Queryable Operational Metrics - Stable sys.* inspection views expose WAL, write-queue, storage, reactive subscription, and sync status snapshots without telemetry writes
• 🌿 Branch, Diff, Restore, And Time Travel - Named snapshots, isolated branch writes, branch diffs, guarded restore, and constrained merge for migration rehearsal and agent sandboxes
• 🔎 Trigram Index - Fast text search for LIKE '%pattern%' queries
• 🔍 Full-Text Search - Native USING fulltext indexes with phrase/prefix queries and BM25 ranking through fulltext_match and bm25
• 🗺️ Native Geospatial - GEOMETRY / GEOGRAPHY values, ST_* functions, and USING spatial indexes
• 🧪 Comprehensive Testing - Unit tests, property tests, crash injection, and differential testing
• 🔄 Foreign Key Constraints - Automatic indexing and referential integrity enforcement
• 📊 Rich Query Support - Aggregates (including DISTINCT), subqueries (FROM, EXISTS, scalar), UPSERT, set operations, generated columns, and scalar functions (string, math, UUID, JSON)
• ⚡ Triggers - AFTER and INSTEAD OF triggers for complex logic
• 💾 Savepoints - Nested transaction control with SAVEPOINT, RELEASE, and ROLLBACK TO
• 🧠 In-Memory Database - Ephemeral :memory: databases for caching and testing, with save-as (CLI) / saveAs (embedded API) to snapshot to disk
• 📦 Single-file DB + WAL sidecar - Primary .ddb file with a -wal sidecar log for durability
• 🌐 Cross-Platform - Native release builds for Linux x86_64/arm64 (including 64-bit Raspberry Pi OS), macOS, and Windows
• 🚀 Bulk Load Operations - Optimized high-performance data loading
• 🛠️ Rich CLI Tool - Unified command-line interface for all database operations
• 📁 Import/Export Tools - CSV and JSON data import/export capabilities
• 🧩 Parameterized Queries - Safe parameter binding to prevent SQL injection
• 🧾 Transaction Support - BEGIN, COMMIT, ROLLBACK for atomic operations
• 📋 Temporary Objects - Session-scoped TEMP tables and views
• 🏗️ EF Core Provider - Full Entity Framework Core integration with LINQ translation, migrations, and NodaTime support
• 🔌 DBeaver Support - Connect to .ddb files from DBeaver via the in-process JNI-backed JDBC driver; browse tables, run queries, and render ER diagrams. See the DBeaver guide.

Languages/Toolkits/SDKs

Language	Toolkit	Description	Documentation
C#	ADO.NET + Dapper + MicroOrm (LINQ) + EF Core	Embedded ADO.NET provider, LINQ Micro-ORM, and EF Core integration with DbContext, migrations, and NodaTime support	decentdb.org/api/dotnet
Dart	Dart FFI (Flutter desktop)	Embedded FFI binding for Flutter desktop apps with cursor paging and schema introspection	decentdb.org/api/dart
Java	JDBC (JNI-backed, in-process)	JDBC driver for connecting to .ddb files from Java and tools like DBeaver	decentdb.org/api/jdbc
Go	database/sql driver	Embedded database/sql driver with $N positional parameters	decentdb.org/api/go
Node.js	N-API + Knex	Embedded native addon + Knex client for building/issuing queries	decentdb.org/api/node
Python 3	SQLAlchemy	Embedded DB-API driver + SQLAlchemy dialect	decentdb.org/api/python
Web	TypeScript + WASM + OPFS	Browser binding with a Dedicated Worker runtime and OPFS persistence	decentdb.org/api/wasm

Tools

Decent Bench - Native cross platform DecentDB Bench SQL tool.

Performance (at a glance)

How this chart is produced

• The native benchmark summary is generated with cargo bench -p decentdb --bench embedded_compare.
• Optional Python-harness engines (for example H2 and HSQLDB) are merged into the README summary with python scripts/aggregate_benchmarks.py.
• The README chart assets are rendered from data/bench_summary.json by python scripts/make_readme_chart.py and python scripts/visualize_alternative.py.
• Checked-in chart assets use the accepted release benchmark snapshot. Local diagnostic benchmark runs should not replace data/bench_summary.json unless they were collected in the release benchmark lane.
• Native runs now keep DecentDB rows separate by profile:
  • decentdb_balanced_durable — current balanced durable config (wal_sync_full, 16 MB cache)
  • decentdb_low_memory_durable — constrained-host durable config (wal_sync_full, 4 MB cache)
  • decentdb_tuned_durable — durability-preserving tuned config (wal_sync_full, 64 MB cache, retain_paged_row_sources_after_commit)
  • decentdb_default_durable — legacy release-snapshot key for historical 4 MB default rows
• Native DecentDB chart rows run with process_coordination=single_process_unsafe because this chart is a single-process embedded-engine comparison. Cross-process coordination is validated separately and should not be inferred from these hot-path bars.
• The benchmark asset workflow also runs the raw rust-baseline full-scale cross-check and validates the merged chart summary before publishing. If the README chart would show a severe tuned-profile drop while the raw engine baseline remains healthy, the workflow fails instead of committing confusing assets.
• SQLite results are reported as sqlite_wal_full (WAL+FULL sync); relaxed durability variants can be added as separate profiles when collected.
• Chart values are normalized vs SQLite (baseline = 1.0), and the charted metric set is the full native set collected in the benchmark JSON:
  • read_p95_ms, join_p95_ms, range_scan_p95_ms, aggregate_p95_ms, concurrent_read_p95_ms, commit_p95_ms, insert_rows_per_sec.
• For "lower is better" latency metrics, inversion is applied so higher bars mean better.
• For missing per-metric coverage, chart scripts render explicit N/A markers and keep H2/HSQLDB out of the all-axis radar panel rather than forcing zero scores.
• Full methodology and all profile metadata lives in crates/decentdb/benches/embedded_compare.rs, and the generated summary lives in data/bench_summary.json.

Regenerate

# Build the native benchmark summary
cargo bench -p decentdb --bench embedded_compare

# After running benchmarks/python_embedded_compare, merge its extra engines
python scripts/aggregate_benchmarks.py \
  --native-summary data/bench_summary.json \
  --python-embedded-compare-results benchmarks/python_embedded_compare/out/results_merged.json \
  --output data/bench_summary.json

# Render the README chart assets from the merged summary
python scripts/make_readme_chart.py
python scripts/visualize_alternative.py

Quick Start

Prerequisites

• Rust (includes cargo + rustc)
• libpg_query (C library + headers)
• Python 3 (optional; for test harness)

Download a prebuilt release

GitHub Releases publish native archives for:

• decentdb-<tag>-Linux-x64.tar.gz
• decentdb-<tag>-Linux-arm64.tar.gz — for 64-bit Raspberry Pi OS on Raspberry Pi 3/4/5 and other aarch64 Linux systems
• decentdb-<tag>-macOS-x64.tar.gz
• decentdb-<tag>-Windows-x64.zip

Each archive contains the DecentDB CLI plus the native C API library. Extract the archive and add decentdb (or decentdb.exe) to your PATH.

JDBC and DBeaver assets are published alongside the CLI/native library bundles as decentdb-jdbc-<tag>-... and decentdb-dbeaver-<tag>-..., including Linux arm64 variants for Raspberry Pi.

Build from source

cargo build
# Optionally: install into ~/.cargo/bin
cargo install --path crates/decentdb-cli

Create a Database

# Create and query a database
# Note: DecentDB auto-assigns an id when you omit a single INT64 PRIMARY KEY column on INSERT.
decentdb exec --db ./my.ddb --sql "CREATE TABLE users (id INT PRIMARY KEY, name TEXT NOT NULL, email TEXT UNIQUE)"
decentdb exec --db ./my.ddb --sql "INSERT INTO users (name, email) VALUES ('Alice', 'alice@example.com') RETURNING id"
decentdb exec --db ./my.ddb --sql "SELECT * FROM users"

REPL Mode

decentdb repl --db ./my.ddb

Usage Examples

SQL Operations

# Create tables with constraints
decentdb exec --db ./my.ddb --sql "CREATE TABLE orders (
    id INT PRIMARY KEY,
    user_id INT REFERENCES users(id),
    amount FLOAT64,
    created_at TIMESTAMP
)"

# Insert data
decentdb exec --db ./my.ddb --sql "INSERT INTO orders VALUES (1, 1, 99.99, TIMESTAMP '2025-01-01 00:00:00')"

# Query with JOINs
decentdb exec --db ./my.ddb --sql "SELECT u.name, SUM(o.amount) AS total
    FROM users u
    JOIN orders o ON u.id = o.user_id
    GROUP BY u.name"

# Text search with trigram index
decentdb exec --db ./my.ddb --sql "CREATE INDEX idx_users_name ON users USING trigram(name)"
decentdb exec --db ./my.ddb --sql "SELECT * FROM users WHERE name LIKE '%ali%'"

# Ranked full-text search
decentdb exec --db ./my.ddb --sql "CREATE INDEX idx_docs_search ON docs USING fulltext(title, body) WITH (prefix = '2,3')"
decentdb exec --db ./my.ddb --sql "SELECT id, bm25('idx_docs_search') AS rank FROM docs WHERE fulltext_match('idx_docs_search', 'database OR search') ORDER BY rank DESC LIMIT 20"

# Geospatial radius query with a spatial index
decentdb exec --db ./my.ddb --sql "CREATE TABLE places (id INT PRIMARY KEY, geog GEOGRAPHY(POINT,4326))"
decentdb exec --db ./my.ddb --sql "CREATE INDEX idx_places_geog ON places USING spatial(geog)"
decentdb exec --db ./my.ddb --sql "SELECT id FROM places WHERE ST_DWithin(geog, ST_GeogPoint(-97.7431, 30.2672), 5000)"

Import/Export

# Import CSV data
decentdb import --table users --input data.csv --db ./my.ddb

# Export to JSON
decentdb export --table users --output users.json --db ./my.ddb --format=json

# Bulk load large datasets
decentdb bulk-load --table users --input large_dataset.csv --db ./my.ddb

There are several tools for DecentDB that provide importing/converting from other databases, read more here

Maintenance

# Force WAL checkpoint
decentdb checkpoint --db ./my.ddb

# View database statistics
decentdb stats --db ./my.ddb

# Collect planner statistics (row counts / index cardinality)
decentdb exec --db ./my.ddb --sql "ANALYZE"

# Rebuild an index
decentdb rebuild-index --index idx_users_name --db ./my.ddb

# Rebuild all indexes
decentdb rebuild-indexes --db ./my.ddb

CLI Reference

DecentDB provides a unified CLI tool. See decentdb --help for all commands.

Common commands:

• exec - Execute SQL statements
• repl - Interactive SQL shell
• extension - Validate, install, enable, and inspect Lua extension packages
• import / export - Data transfer
• bulk-load - High-performance data loading
• checkpoint - WAL maintenance
• save-as - Snapshot backup to a new on-disk file
• list-tables / describe - Schema introspection
• rebuild-index / rebuild-indexes - Index maintenance
• dump - Export database as SQL

Documentation

• User Guide - SQL reference, tutorials, and examples
• Releases - GitHub release workflow, asset naming, and Linux arm64 / Raspberry Pi packages
• Rust API - Embedded API documentation
• Architecture - Design and implementation details
• Contributing - Development guidelines

Architecture

DecentDB is organized into focused modules:

• VFS - OS I/O abstraction with fault injection support
• Pager - Fixed-size pages, LRU cache, and freelist management
• WAL - Append-only log, crash recovery, and checkpointing
• B+Tree - Table storage and secondary indexes
• Record - Typed value encoding with overflow pages
• Catalog - Schema metadata management
• SQL/Planner/Exec - Query parsing, planning, and execution
• Search - Trigram substring search and native full-text search

Development

# Run the main test suite (engine + harness + .NET/Go/Node/Python/Dart bindings)
cargo test

cargo test -p decentdb

# Run benchmarks
cargo bench

# Lint code
cargo clippy

See Contributing Guide for development workflow and guidelines.

License

Apache-2.0. See LICENSE.