Skip to content

README.md

Latest commit

 

History

History
253 lines (210 loc) · 7.88 KB

README.md

File metadata and controls

253 lines (210 loc) · 7.88 KB
Raw

Go RDBMS with BadgerDB

A high-performance, embedded message-based RDBMS written in Go, powered by BadgerDB with a complete TCP server, query language (DSL), and protocol system for entity mapping.

🚀 Key Features

Core Database

  • 3-Layer Architecture: Clean separation between Engine, Store Manager, and Database layers
  • Hybrid Storage: RAM buffering with asynchronous disk persistence (500ms flush)
  • Full Indexing: Automatic reverse indexing for every column
  • Laravel-style Validation & Formatting: Schema-level rules (e.g., required|min:18, trim|upper)
  • Production Ready: Structured logging, graceful shutdown, configuration via environment

Message-Based System

  • TCP Server: Multi-client support with connection pooling
  • Message Router: Routes requests to database, DSL, or extensions
  • Protocol System: Maps entity aliases to actual DB/table names with relationships
  • DSL Query Engine: SQL-like query language with filters, projections, and aggregations

🏗 Architecture

┌─────────────────────────────────────────────────────────┐
│                    TCP Server (Port 8080)                │
│                  Message Protocol (\x04)                 │
└────────────────────┬────────────────────────────────────┘
                     │
                     ▼
┌─────────────────────────────────────────────────────────┐
│                   Message Router                         │
│  Routes: database | onql | protocol | schema            │
└─────┬──────────┬──────────┬────────────┬───────────────┘
      │          │          │            │
      ▼          ▼          ▼            ▼
┌──────────┐ ┌──────┐ ┌──────────┐ ┌──────────┐
│ Database │ │ DSL  │ │ Protocol │ │  Schema  │
│ Handler  │ │Engine│ │Extension │ │Extension │
└────┬─────┘ └──┬───┘ └────┬─────┘ └────┬─────┘
     │          │          │            │
     └──────────┴──────────┴────────────┘
                     │
                     ▼
          ┌──────────────────────┐
          │   Database Layer     │
          │ (Validation/Format)  │
          └──────────┬───────────┘
                     │
                     ▼
          ┌──────────────────────┐
          │   Store Manager      │
          │ (Buffer/Flush/Index) │
          └──────────┬───────────┘
                     │
                     ▼
          ┌──────────────────────┐
          │   BadgerDB Engine    │
          │   (LSM Tree Store)   │
          └──────────────────────┘

📋 Protocol System

The Protocol System allows you to define entity aliases that map to actual database tables with field mappings and relationships.

Protocol Structure

{
  "database": "mydb",
  "entities": {
    "users": {
      "table": "accounts",
      "fields": {
        "id": "account_id",
        "name": "full_name",
        "email": "email_address"
      },
      "relations": {
        "orders": {
          "type": "otm",
          "entity": "orders",
          "fk_field": "id:user_id"
        }
      },
      "context": {
        "user": "mydb.users[id=$1]",
        "admin": "mydb.users[role='admin']"
      }
    }
  }
}

Relationship Types

  • oto (One-to-One): Single related record
  • otm (One-to-Many): Multiple related records
  • mto (Many-to-One): Reverse of one-to-many
  • mtm (Many-to-Many): Through junction table

Protocol Commands

Store a Protocol:

{
  "target": "protocol",
  "payload": ["set", "mypassword", {...protocol_data...}]
}

Retrieve Protocol:

{
  "target": "protocol",
  "payload": ["desc", "mypassword"]
}

Delete Protocol:

{
  "target": "protocol",
  "payload": ["drop", "mypassword"]
}

🔍 DSL Query Language

The DSL (Domain Specific Language) allows SQL-like queries with entity aliases:

mydb.users[age>18].name
mydb.orders[status='pending'].customer.name
mydb.products._count()

Features:

  • Entity-based queries (uses protocol mappings)
  • Filtering with conditions
  • Relationship traversal
  • Aggregations (count, sum, avg, etc.)
  • Projections and field selection

💻 Message Protocol

All communication uses JSON messages with this structure:

{
  "id": "sender_id",
  "target": "database|onql|protocol|schema",
  "rid": "request_id",
  "type": "request|response",
  "payload": "json_string"
}

Messages are delimited by \x04 (EOT character).

Database Functions

{
  "target": "database",
  "payload": {
    "function": "GetDatabases|CreateDatabase|GetTables|...",
    "args": [...]
  }
}

DSL Queries

{
  "target": "onql",
  "payload": {
    "protopass": "mypassword",
    "query": "mydb.users[age>18]",
    "ctxkey": "user",
    "ctxvalues": ["123"]
  }
}

🏁 Getting Started

Prerequisites

  • Go 1.21+

Installation

git clone <repo-url>
cd rdbms
go mod tidy

Running the Server

go run examples/server_demo.go

The server starts on port 8080 by default.

Configuration

Environment variables:

  • DB_PATH: Data storage path (default: ./store)
  • FLUSH_INTERVAL: Buffer flush interval (default: 500ms)
  • LOG_LEVEL: DEBUG|INFO|WARN|ERROR (default: INFO)

📁 Project Structure

rdbms/
├── engine/              # BadgerDB wrapper
├── storemanager/        # Buffer, indexing, schema
├── database/            # Validation, formatting, protocols
├── dsl/                 # Query language parser & evaluator
├── router/              # Message routing
├── server/              # TCP server
├── extensions/          # Extension system
│   ├── protocol/        # Protocol management extension
│   └── schema/          # Schema management extension
├── config/              # Configuration loader
├── logger/              # Structured logging
└── examples/            # Demo applications

🎯 Use Cases

  • Multi-tenant Applications: Use protocols for tenant isolation
  • API Backends: Expose via TCP with custom client libraries
  • Embedded Systems: Lightweight, single-binary database
  • Real-time Applications: Fast RAM access with disk durability
  • Microservices: Message-based communication between services

📚 Examples

See examples/ directory for:

  • protocol_demo.go - Protocol system demonstration
  • server_demo.go - TCP server with client
  • dsl_demo.go - DSL query examples

🔧 Advanced Features

  • Context Queries: Parameterized queries with $1, $2 placeholders
  • Field Mapping: Alias fields to actual column names
  • Relationship Traversal: Navigate through entity relationships
  • Extension System: Add custom handlers via extensions
  • Graceful Shutdown: Ensures data integrity on exit

Built with ❤️ using Go and BadgerDB