PaperDesk

A multi-service paper centralized exchange built to study what actually happens after someone clicks Buy: matching, balance locks, durable order flow, market-data persistence, and perpetual risk.

It is a systems project: a single-writer matching engine, an asynchronous OMS with a transactional outbox, a gateway that translates Redis Streams and exchange SSE, and a separate TimescaleDB ingester. Spot and perpetual markets run in-process with mark price, liquidation, and funding.

Built to make failure modes visible — duplicate commands, maker/taker fills, reconnect gaps, crash windows between engine execution and event publication instead of hiding them behind a single CRUD API.

What makes it different

Engine owns matching and balances; OMS is a durable projection, not a second matching engine
Place/cancel flow uses Redis Streams + Postgres outbox instead of synchronous “write DB and hope”
Maker and taker fills share a trade id but are stored per order; events carry engine sequence
Live market data is ephemeral (pub/sub); history is durable (md:events → Timescale)
Perps add margin, positions, mark, liquidation, and funding on top of the same engine model
SSE reconnect uses streamSeq catch-up, with reconcile when the in-memory ring was overrun
Browser market streams can EventSource the gateway directly (ticket from the web BFF); the BFF stream proxy is for local/dev fallback

Repository Overview

apps/exchange
Single-writer matching engine. One process hosts spot SOL-USD and perpetual SOL-USD-PERP by default (USD margin, positions, mark, liquidation, funding).
apps/oms
Product-facing order state in Postgres, transactional command outbox, and event-driven status updates.
apps/engine-gateway
Sole client of the exchange: Redis commands → engine HTTP; SSE → orders:events + md:events + live pub/sub.
apps/ingester
Consumes durable market-data events into TimescaleDB and serves historical trades, BBO, and candles.
apps/web
Next.js trading app: Google auth, paper credit, Spot / Perps surfaces, charts, and BFF proxies. With SIM_HEARTBEAT=true this process also runs the shared market sim.
packages/exchange-types
Shared engine domain types: orders, trades, balances, positions, events, commands.
packages/app-contracts
Application-layer Redis Streams / pub/sub contracts.
packages/db
Prisma schema for users and OMS order state. Trading balances live in the exchange, not Postgres.
packages/logger
Structured logs and Redis stream health checks used by the Node services.
infra
Redis, PostgreSQL, TimescaleDB, Prometheus, Grafana, and Loki (Compose). Optional nginx samples for a public host.
.github/workflows
CI (lint / typecheck / unit tests). Optional SSH deploy workflow.
ecosystem.config.cjs
Optional PM2 process file when running all Node apps on one machine.

Current Architecture

apps/web
  └─ user auth + trading UX

apps/exchange
  └─ matching engine + balances + WAL + snapshots + HTTP/SSE

Application layer
  └─ OMS → engine-gateway → exchange
  └─ Redis Streams + Redis pub/sub
  └─ exchange SSE → engine-gateway → md:events + orders:events
  └─ md:events → ingester → TimescaleDB

Exchange engine

The exchange engine is intentionally single-writer per market. It keeps matching logic in memory and uses disk only for crash recovery.

MarketRuntime coordinates live commands, WAL persistence, replay, and checkpoints.
CommandQueue serializes concurrent commands and batches WAL flushes.
FileWal appends CREDIT, PLACE, CANCEL, LIQUIDATE, and FUNDING commands.
Snapshots shorten restart time by restoring state and replaying only the WAL tail.
EventBus publishes live ORDER, BBO, CREDIT, TRADE, POSITION, LIQUIDATION, and FUNDING events for SSE consumers.
SSE includes a monotonic streamSeq and a bounded ring so reconnecting gateways can catch up via ?afterSeq= / Last-Event-ID (gap signal when the ring was overrun).
On SSE gap, the gateway calls GET /v1/markets/:market/reconcile and republishes retained order events, order snapshots, positions, liquidations, and funding to orders:events so OMS can catch up.

Application layer direction

The application layer wraps the engine with service boundaries:

Redis Streams for command/event delivery between OMS and the engine gateway
Exchange SSE as the canonical source for BBO, trades, and maker-side fills
Redis pub/sub for live best bid/ask and trade fan-out
The durable md:events stream and TimescaleDB for historical market data
Postgres for users and OMS order state. Trading balances stay in the exchange WAL.

Monorepo Layout

PaperDesk/
├── apps/
│   ├── exchange/
│   ├── engine-gateway/
│   ├── ingester/
│   ├── oms/
│   └── web/
├── infra/
└── packages/
    ├── app-contracts/
    ├── db/
    ├── exchange-types/
    ├── logger/
    └── typescript-config/

Package names are still @cex/* (@cex/web, @cex/exchange, and the rest). PaperDesk is the product name.

Prerequisites

Node.js >=20
pnpm 10.14.0 (enable with corepack enable)
Docker Desktop (or compatible) for Redis / Postgres / Timescale

Local quickstart

pnpm install
pnpm infra:up
pnpm setup:local          # creates .env files if missing + migrate deploy

Edit apps/web/.env and set:

GOOGLE_CLIENT_ID / GOOGLE_CLIENT_SECRET
NEXTAUTH_SECRET (any random string locally)
NEXTAUTH_URL=http://localhost:3000

Then start the full stack (exchange, gateway, OMS, ingester, web):

pnpm dev:stack

Open http://localhost:3000. Sign in with Google.

Sim controls and the book wipe are limited to the Google emails in SIM_OPERATOR_EMAILS (see .env.example). An empty list in production means nobody. With the variable unset outside production, those controls stay available. PM2 turns the shared sim on with SIM_HEARTBEAT=true; each heartbeat places one missing quote and at most one print.

Command	What it starts
`pnpm infra:up`	Redis `:6379`, Postgres `:5432`, Timescale `:5434`
`pnpm setup:local`	Env templates + `prisma migrate deploy`
`pnpm dev:stack`	All app processes (labeled logs)
`pnpm dev:backend`	Same without Next.js
`pnpm dev`	Web only

Default local tokens and URLs live in .env.example. setup:local copies them to root .env, packages/db/.env, and apps/web/.env when those files are missing (never overwrites).

Backend services load env from cwd / repo root / packages/db/.env. Next.js only reads apps/web/.env.

Ports

Service	Port
Web	`3000`
Exchange	`4010`
Engine gateway	`4020`
OMS	`4030`
Ingester (history)	`4040`
Redis	`6379`
Postgres	`5432`
Timescale	`5434`

Individual services

pnpm dev:exchange
pnpm dev:gateway
pnpm dev:oms
pnpm dev:ingester
pnpm dev

Exchange hosts both SOL-USD and SOL-USD-PERP on :4010 by default. WALs live under apps/exchange/data/<market>.jsonl.

# Spot only
cross-env EXCHANGE_MARKET=SOL-USD pnpm dev:exchange

# Legacy separate perp process on :4011
pnpm dev:exchange:perp

Supported engine environment variables:

EXCHANGE_MARKETS — comma list, default SOL-USD,SOL-USD-PERP
EXCHANGE_MARKET — single-market override
EXCHANGE_PORT — HTTP/SSE port (default 4010)
EXCHANGE_WAL_PATH — only when hosting a single market
EXCHANGE_DATA_DIR — WAL directory (default apps/exchange/data)

Database migrations

pnpm db:migrate:deploy   # apply existing migrations (CI / local setup)
pnpm db:migrate          # prisma migrate dev (schema authors)
pnpm db:generate

Infra only

pnpm infra:up
pnpm infra:down
pnpm infra:logs

See infra/README.md for local Compose.
Cloud-style split deploy (e.g. Vercel + Render): infra/DEPLOY.md §2.

Production shape (optional)

When you deploy to a single public host (not required for local work):

Piece	Role
Docker Compose	Redis, Postgres, Timescale, plus Prometheus, Grafana, and Loki on loopback
PM2	exchange, gateway, OMS, ingester, web (`SIM_HEARTBEAT=true` on web)
nginx	TLS termination + reverse proxy to loopback app ports
Let’s Encrypt (certbot)	Free certificates for the public hostnames
GitHub Actions	CI on push/PR; Deploy workflow SSHs in, builds web, reloads PM2

Public surface is HTTPS for the web UI and gateway SSE. App processes and databases listen on localhost; only the proxy (and SSH) need to be reachable from the internet. Live market streams use a short-lived ticket from the web BFF, then EventSource the gateway origin directly.

Configure matching internal tokens across services (OMS_*, GATEWAY_* / ENGINE_GATEWAY_*, EXCHANGE_GATEWAY_*, market-data / ingester) and set ENGINE_GATEWAY_PUBLIC_URL + CORS_ORIGINS to your HTTPS origins. Helpers: pnpm test:ci, pnpm typecheck, pnpm build:web, pnpm pm2:start / pm2:reload. Full steps: infra/DEPLOY.md §1.

See API.md for request IDs, error envelopes, order pagination, and BFF/internal boundaries.

Exchange API

One exchange process hosts both markets (SOL-USD and SOL-USD-PERP) by default. Command, balance, book, and stream APIs require x-gateway-token. Only /health is public.

Method	Path	Purpose
`GET`	`/health`	Process health and active markets
`POST`	`/v1/markets/:market/credit`	Internal gateway credit operation
`POST`	`/v1/markets/:market/orders`	Place a limit or market order (`leverage` for perps)
`DELETE`	`/v1/markets/:market/orders/:orderId`	Cancel an order
`GET`	`/v1/markets/:market/orders/:orderId`	Fetch one order
`GET`	`/v1/markets/:market/orders?userId=&openOnly=`	Fetch user orders
`GET`	`/v1/markets/:market/balances/:userId`	Fetch engine balances
`GET`	`/v1/markets/:market/positions`	List positions with risk fields (perp)
`GET`	`/v1/markets/:market/positions/:userId`	Fetch one user position
`GET`	`/v1/markets/:market/mark`	Mark price (BBO mid or last trade)
`GET`	`/v1/markets/:market/funding`	Funding rate / interval (perp)
`POST`	`/v1/markets/:market/funding/settle`	Force a funding settle tick (perp)
`GET`	`/v1/markets/:market/book`	Fetch order book snapshot
`GET`	`/v1/markets/:market/reconcile`	Gap recovery snapshot (orders, events, risk)
`GET`	`/v1/markets/:market/stream?userId=`	Subscribe to live SSE

Notable engine rules:

Units are integer-only.
Spot market buys require quoteBudget; perp MARKET orders require quoteBudget on both sides (notional cap for margin).
Perps lock USD margin (ceil(notional / leverage)); fills update positions and realize PnL — no SOL delivery.
Maintenance liquidation force-closes underwater perps at mark vs house (sim-liquidator).
Funding settles periodically (demo: 100 bps / 60s); longs pay shorts when rate > 0.
Spot and perps share one wallet (USD/SOL available + locked). Books and positions stay per market.
Exchange place/credit are idempotent on retry: same orderId+intent returns the prior order; credit with commandId does not double-apply.
Gateway command handling journals the outcome in Redis before publish, then marks processed — crash mid-flight retries replay the outcome (deterministic event ids) instead of relying on a best-effort mark.
FOK_BUDGET is a market-buy-only fill-or-kill order. It must fill the requested quantity within quoteBudget or reject before matching.
The exchange BalanceStore and its WAL are authoritative for trading balances.

Testing

Exchange test suite

pnpm test:exchange
pnpm test:exchange:unit
pnpm test:exchange:integration
pnpm test:exchange:e2e

Unit tests cover core engine modules.
Integration tests cover replay and durability behavior.
End-to-end tests cover the HTTP surface and restart behavior.

OMS test suite

pnpm test:oms
pnpm test:oms:integration

The integration test requires PostgreSQL, Redis, the exchange, the engine gateway, and OMS to be running.

Implemented

Spot exchange engine for SOL-USD and perp engine for SOL-USD-PERP (one multi-market process by default)
Spot balance locking / delivery settlement; perp USD margin + positions + PnL
Mark price, maintenance liquidation (force-close at mark), and periodic funding payments
WAL persistence with checkpoints and replay (CREDIT / PLACE / CANCEL / LIQUIDATE / FUNDING; positions in snapshot v2)
HTTP commands and queries (orders, balances, book, mark, positions + risk fields, funding)
SSE for live order, credit, BBO, trade, position, liquidation, and funding events
Engine gateway multi-market routing; Redis fan-out for POSITION / LIQUIDATION / FUNDING
OMS order APIs with perp leverage persistence + idempotency, Postgres order state, outbox, event-driven status updates
OMS cancel uses a conditional status update (PENDING/ACCEPTED/OPEN/PARTIALLY_FILLED only) so a fill race cannot mark a terminal order CANCEL_REQUESTED
Market-data writer (TimescaleDB history for trades, BBO, and one-minute candles per market)
Web app: Google auth, paper credit, and Spot / Perps desks
Shared market sim in the web process. Production sim controls and wipe require SIM_OPERATOR_EMAILS
Gateway refuses new sim writes while the command stream is behind, and drops sim commands older than 1.5s, so a sim burst cannot stall a user order
Prometheus, Grafana, and Loki for metrics and PM2 logs (loopback; see infra/README.md)

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
.vscode		.vscode
apps		apps
infra		infra
packages		packages
scripts		scripts
.env.example		.env.example
.gitignore		.gitignore
.node-version		.node-version
API.md		API.md
README.md		README.md
ecosystem.config.cjs		ecosystem.config.cjs
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
render.yaml		render.yaml
tsconfig.json		tsconfig.json
turbo.json		turbo.json

Navigation Menu

Repository files navigation

PaperDesk

What makes it different

Repository Overview

Current Architecture

Exchange engine

Application layer direction

Monorepo Layout

Prerequisites

Local quickstart

Ports

Individual services

Database migrations

Infra only

Production shape (optional)

Exchange API

Testing

Exchange test suite

OMS test suite

Implemented

About

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

PaperDesk

What makes it different

Repository Overview

Current Architecture

Exchange engine

Application layer direction

Monorepo Layout

Prerequisites

Local quickstart

Ports

Individual services

Database migrations

Infra only

Production shape (optional)

Exchange API

Testing

Exchange test suite

OMS test suite

Implemented

About

Topics

Resources

Stars

Watchers

Forks

Contributors

Languages