A Redis-compatible, in-memory database written from scratch in Rust.
FlashDB speaks the RESP protocol,
so you can talk to it with any Redis client (redis-cli, language bindings, redis-benchmark).
FlashDB has a small dependency set (async runtime + buffers). Run it directly:
cargo run --releaseThis starts the server on 127.0.0.1:6379. Connect with any Redis client:
redis-cli -p 6379 ping
# PONGOr build a release binary and run it:
cargo build --release
./target/release/flashdbThe server accepts a few Redis-style startup flags:
flashdb -p 7000 --dir /var/lib/flashdb --dbfilename snapshot.rdb-p (or --port) sets the listening port (default 6379). --dir and
--dbfilename name where the on-disk snapshot lives; they default to the
current directory and dump.rdb; they locate the on-disk snapshot FlashDB loads
at startup and writes on SAVE / BGSAVE (see Persistence
below). --replicaof <host> <port> starts the server as a replica that syncs
from a master on boot (see Replication below). An unknown flag or
a flag missing its value prints a message and exits non-zero rather than
starting.
PINGECHOSET key value [EX seconds | PX milliseconds]GETDELEXPIRE key secondsTTL keyPERSIST keyTYPE keyRPUSH key value [value ...]LPUSH key value [value ...]RPOP keyLPOP keyLLEN keyLRANGE key start stopHSET key field value [field value ...]HGET key fieldHGETALL keyHDEL key field [field ...]SAVEBGSAVEWAIT numreplicas timeoutMULTIEXECDISCARDWATCH key [key ...]UNWATCHSUBSCRIBE channel [channel ...]UNSUBSCRIBE [channel ...]PUBLISH channel messageXADD key [MAXLEN [=|~] threshold] <id> field value [field value ...]XLEN keyXRANGE key start end [COUNT count]XREVRANGE key end start [COUNT count]XDEL key id [id ...]XTRIM key MAXLEN [=|~] thresholdXREAD [COUNT count] [BLOCK ms] STREAMS key [key ...] id [id ...]
MULTI opens a transaction on the connection. Every command sent after it is
queued rather than run — each is answered with +QUEUED — until EXEC
replays the whole batch in order and returns one array holding each command's
reply. DISCARD throws the queued batch away without running it.
Queuing is validated up front: an unknown command is rejected the moment it is
queued and taints the transaction, so the eventual EXEC aborts the whole
batch with EXECABORT and runs nothing. A command that only fails at run time
(for example a WRONGTYPE error) is not caught early — it runs, and its error
is simply one element of the EXEC reply array; the commands around it still
run, because a Redis transaction is a batch, not a rollback.
WATCH key [key ...] adds optimistic locking (check-and-set). It records the
current version of each named key; if any watched key is written by any
connection before EXEC, the transaction aborts — EXEC replies with the nil
array (*-1) and runs nothing — so the client can retry. Watching a key that
does not exist yet still guards it: if it springs into existence before EXEC,
that counts as a change. UNWATCH clears the watch set, and a completed EXEC
or DISCARD clears it automatically. WATCH is not allowed once MULTI has
opened a transaction.
Transaction state (the queue and the watch set) lives per-connection, not in the shared keyspace, so two clients can run independent transactions at once. Writes inside a transaction replicate to replicas exactly as they would outside one.
FlashDB supports Redis's publish/subscribe messaging. SUBSCRIBE channel [channel ...] puts a connection into subscribe mode and registers it as a
listener on each channel; the server confirms each with a [subscribe, channel, count] frame, where count is how many channels the connection now holds.
PUBLISH channel message, sent from any other connection, delivers [message, channel, payload] to every subscriber of that channel and returns the number
that received it. UNSUBSCRIBE [channel ...] leaves the named channels, or
every channel when given none; once the last subscription is dropped the
connection returns to ordinary request/response mode.
While subscribed, a connection is half push, half request/response: the server
may send it a message at any moment, and it may still SUBSCRIBE,
UNSUBSCRIBE, or PING. Any other command is refused with an error, matching
Redis's RESP2 rule that a subscribed client can only manage its subscriptions.
The channel routing table lives on the shared server (channel → the set of
subscribed connections), while each connection tracks its own channel set on its
task — the same shared-registry-plus-per-connection-state split that replication
and transactions use.
Not yet implemented: pattern subscriptions (PSUBSCRIBE/PUNSUBSCRIBE),
PUBSUB introspection, and propagating PUBLISH across a replication link.
A stream is an append-only log of entries. Each entry has a unique,
monotonically increasing ID of the form <ms>-<seq> (a millisecond timestamp
and a per-millisecond sequence counter) and a set of field/value pairs.
XADD key [MAXLEN [=|~] threshold] <id> field value [field value ...] appends
an entry and returns the ID it was stored under. The <id> may be:
*— both parts auto-generated (the millisecond comes from the clock, or the last entry's millisecond if the clock hasn't advanced, and the sequence is the next free one);<ms>-*or a bare<ms>— the millisecond is fixed and the sequence is auto-generated;<ms>-<seq>— a fully explicit ID.
The resolved ID must be strictly greater than the stream's current top ID (and
greater than 0-0), otherwise XADD errors and the stream is left untouched.
An optional leading MAXLEN [=|~] threshold clause trims the stream down to
threshold entries — using the same trim primitive as the standalone XTRIM
below — after the new entry is appended, so the just-added entry is never
the one trimmed away. As with XTRIM, the =/~ exactness marker is parsed
but doesn't change FlashDB's behavior.
XLEN key returns the number of entries. XRANGE key start end [COUNT count]
returns the entries whose IDs fall in the inclusive range [start, end]; the
bounds accept - and + (the smallest and largest possible IDs), a bare
<ms> (which covers the whole millisecond), or a full <ms>-<seq>, and
COUNT caps how many entries come back. Each entry is returned as
[id, [field, value, ...]]. A bound may be made exclusive by prefixing it
with ( — for example XRANGE key (1-0 (3-0 returns everything strictly
between 1-0 and 3-0, dropping both endpoints.
XREVRANGE key end start [COUNT count] returns the same window as XRANGE but
in reverse, highest ID first. Note the bound order is flipped — end is given
before start — matching Redis; COUNT keeps the highest IDs.
XDEL key id [id ...] removes the named entries and returns the number actually
deleted (IDs that weren't present don't count). Deleting entries never lowers the
stream's high-water mark, so a future XADD can't reuse the ID of a deleted
entry even if the whole stream is emptied.
XTRIM key MAXLEN [=|~] threshold drops the oldest entries until at most
threshold remain, returning the number actually removed. The optional =/~
exactness marker Redis accepts before the threshold is parsed but doesn't change
FlashDB's behavior — trimming is always exact. Like XDEL, trimming never
lowers the stream's high-water mark.
XREAD [COUNT count] [BLOCK ms] STREAMS key [key ...] id [id ...] reads entries
newer than a given ID from one or more streams at once: it returns, for each
stream that has anything, [key, [entries...]] with the entries whose IDs are
strictly greater than the paired id. An id of $ means "only entries newer
than the current end". Without BLOCK, the read is non-blocking — if no stream
has new entries the reply is the nil array (so XREAD on $ returns nothing).
With BLOCK ms the read instead waits for new entries: it resolves each $
against the stream's current end at the moment blocking begins, and then parks
until a matching XADD arrives (on any connection) or the timeout elapses.
BLOCK 0 waits indefinitely. If the timeout passes with nothing new the reply
is the nil array. The wait is genuinely async — a blocked client uses no CPU and
never holds the store lock while waiting, and a single XADD wakes every client
blocked on that stream. Inside a MULTI/EXEC transaction a BLOCK is ignored
(the queued read runs as a non-blocking pass), matching Redis.
Streams are not yet persisted to RDB (real Redis uses a listpack/radix-tree
encoding FlashDB does not emit yet), and the consumer-group commands
(XACK/XCLAIM/…) are still to come.
Every key holds a typed value — a string, a list, a hash, or a stream
today, with sets to follow. Values are modelled as an enum, so a command that
meets the wrong type (say GET on a list, or LPUSH on a string) replies with
a WRONGTYPE error instead of misbehaving. TYPE key reports the kind of value
stored: string, list, hash, stream, or none if the key is missing or
has expired.
A list is an ordered sequence of strings you can grow and shrink from either
end. RPUSH appends to the tail and LPUSH prepends to the head; both create
the list on first use, accept several values at once, and return the list's new
length. RPOP and LPOP remove and return one element from the tail or head
(a null reply if the key is missing or empty) — and when a pop empties the
list, the key is deleted, so an empty list never lingers.
LLEN key returns the length (0 for a missing key), and LRANGE key start stop returns the elements between start and stop inclusive. Indices are
zero-based and may be negative to count back from the end, so LRANGE mylist 0 -1 returns the whole list; an inverted or out-of-range span yields an empty
array rather than an error.
A hash maps string fields to string values under a single key — handy for
representing an object without a key per attribute. HSET key field value [field value ...] sets one or more pairs (creating the hash on first use) and
returns how many fields were newly added, so overwriting an existing field
counts as zero. HGET key field returns a single field's value (a null reply
if the field or key is missing), and HGETALL key returns every field and
value flattened into one array. HDEL key field [field ...] removes fields and
returns how many were actually present; when the last field is removed the key
is deleted, so an empty hash never lingers.
Fields are stored in a HashMap, so HGETALL returns pairs in an unspecified
order — sort client-side if you need a stable order, exactly as you would with
Redis.
Keys can be given a time to live. SET takes an optional EX <seconds> or
PX <milliseconds> to set a lifetime up front, and EXPIRE key seconds sets
one on an existing key. TTL key returns the seconds remaining (-1 if the
key has no expiry, -2 if it doesn't exist), and PERSIST key removes an
expiry so the key lives forever again.
Expiry is passive: a key past its deadline stays in memory until something touches it, at which point the read drops it and reports it as missing — so an expired key is indistinguishable from one that was never set.
FlashDB persists to disk in Redis's own binary RDB format, so snapshots move
freely in both directions between FlashDB and a real redis-server.
On startup FlashDB looks for a Redis RDB snapshot at <dir>/<dbfilename> and,
if one is present, loads its keys into memory before accepting any clients — so
a restart recovers whatever a previous run (or a real Redis server) persisted. A
missing snapshot is a clean first boot with an empty keyspace; a snapshot that
can't be parsed aborts startup rather than silently discarding the data.
Redis's compact integer string encoding is understood on load (a numeric string is stored as an integer and rendered back to text). Each key's expiry metadata is honoured: a still-future deadline is restored as a live TTL, and a key whose deadline has already passed is dropped on load, just as Redis does.
SAVE writes the whole keyspace to <dir>/<dbfilename> synchronously and
replies +OK once the file is on disk. BGSAVE takes a consistent snapshot and
hands the file write to a background thread, replying +Background saving started immediately so the client isn't blocked on the disk (real Redis forks a
child process for the same effect). Both write the file atomically — to a temp
file, then a rename — so a crash mid-write never leaves a half-written snapshot.
Strings, lists, and hashes are all serialized, each with its optional expiry.
Because the on-disk bytes are real RDB — right down to the CRC64 trailer Redis
verifies on load — a snapshot FlashDB writes loads straight back into FlashDB
and into an unmodified redis-server:
redis-cli -p 6379 SET greeting "hello world"
redis-cli -p 6379 RPUSH mylist a b c
redis-cli -p 6379 SAVE # FlashDB writes dump.rdb
# ...point a real redis-server at that directory...
redis-cli -p 6379 LRANGE mylist 0 -1 # 1) "a" 2) "b" 3) "c" — loaded by RedisFlashDB speaks both sides of the replication link: it can start as a replica of another server, and it can act as a master that feeds replicas.
Start a replica pointed at a master:
flashdb -p 6380 --replicaof 127.0.0.1 6379On startup the replica dials the master and performs the Redis replication
handshake — PING, two REPLCONF rounds (announcing its own listening port and
its capabilities), then PSYNC ? -1. The master answers with a
+FULLRESYNC <replid> <offset> line followed by a full RDB snapshot of its
keyspace as a bulk payload. The replica frames that snapshot off the wire and
loads it through the same RDB reader used for on-disk startup loading, so strings,
lists, hashes, and expiry all transfer. The master can be another FlashDB
instance or a real redis-server.
After the snapshot the link stays open. The master streams every write it
applies (SET, DEL, EXPIRE, the list and hash mutations, …) down the
connection as ordinary RESP commands, and the replica replays each one
against its own keyspace, so a change on the master shows up on the replica a
moment later. Reads and rejected commands are never streamed. WAIT numreplicas timeout reports how many replicas are currently connected. The whole sync runs
on a background task, so both servers keep serving their own clients throughout;
if the master is unreachable the replica logs it and serves whatever it had.
Not yet implemented: per-replica acknowledged offsets (so WAIT reports
connection count rather than blocking until writes are acked), replica reconnect
with backoff, a runtime REPLICAOF command, and partial resync.
The RESP parser is incremental: a frame split across multiple TCP segments
is reassembled, and a pipelined batch of commands sent in one segment is
answered in order, one reply per command. Malformed input gets a
Redis-style -ERR Protocol error reply before the connection is closed.
The server logic lives in a library (src/lib.rs); src/main.rs is a thin
binary that binds the listener and calls flashdb::run_with_config (which loads
any RDB snapshot first). The RDB decoder lives in src/rdb.rs. Splitting it
this way lets the integration tests start a real server on an ephemeral port and
talk to it over a genuine TCP socket.
lib.rs grew past four thousand lines as commands piled up in one file, so it's
being split into src/commands/, one module per self-contained command group.
src/commands/persistence.rs (SAVE/BGSAVE/WAIT and the RDB snapshot
helpers) was the first slice out; src/commands/lists.rs
(RPUSH/LPUSH/RPOP/LPOP/LLEN/LRANGE) and src/commands/hashes.rs
(HSET/HGET/HGETALL/HDEL) followed, then src/commands/streams.rs —
the biggest group yet — took XADD/XLEN/XRANGE/XREVRANGE/XDEL/XTRIM/
XREAD plus the async blocking XREAD ... BLOCK path with it. lib.rs is
down to roughly half its peak size. More groups move the same way as they're
touched next — the pub/sub and transaction helpers are the biggest remaining
chunks.
cargo test # parser unit tests + end-to-end integration tests
cargo clippy --all-targets -- -D warnings
cargo fmtTests come in two layers: unit tests in src/resp.rs that prove single RESP
frames parse and serialize correctly (including partial and malformed input),
and integration tests in tests/integration.rs that drive a live server over
TCP and assert on the raw bytes it sends back.
Every push and pull request against master runs the same gate through
GitHub Actions: cargo fmt --check,
cargo clippy --all-targets -- -D warnings, and cargo test. A red build
means one of those failed.
MIT — see LICENSE.