Forge — Lightweight API Gateway & Reverse Proxy

A production-style, config-driven API gateway written in Node.js/TypeScript. It sits in front of multiple upstream services and handles routing, load balancing, edge JWT auth, per-client rate limiting, circuit breaking, and observability — built on raw node:http (no gateway library) to demonstrate the request-lifecycle / networking layer directly.

Portfolio/infra project. The "user" is any backend service behind it; the demo fronts dummy reviews, skills, and chaos mock services.

Performance Summary

200 RPS baseline throughput
3 ms P95 latency
1600+ RPS spike capacity
13 automated k6 benchmarks
0 external framework dependencies (no Express/Fastify)

Features

Feature	Implementation
Reverse Proxy	Streaming `http.request` with keep-alive
Routing	Longest-prefix + optional host match
JWT Authentication	HS256 edge verification
Rate Limiter	Token bucket per (client, route)
Circuit Breaker	Closed / Open / Half-Open state machine
Load Balancer	Round-robin + least-connections + consistent-hash
Health Checks	Active polling with automatic pool management
Service Discovery	DNS / static upstream membership refresh
Metrics	Prometheus `/metrics`
Dashboard	Grafana auto-provisioned
Logging	Pino structured JSON with trace IDs
Bulkhead Isolation	Per-upstream connection pools
Config-driven	YAML routes/upstreams, no code changes
HTTP/2	Optional h2c frontend listener
Observability	Trace IDs + OpenTelemetry spans, never leaked

Tech Stack

Runtime: Node.js
Language: TypeScript
Networking: node:http (raw sockets)
Containerization: Docker Compose
Orchestration: Kubernetes (Helm chart)
Metrics: Prometheus + Grafana
Load Testing: k6 v2.0.0
Testing: Vitest
Logging: Pino
Auth: JWT (HS256)
Config: YAML + Zod
Tracing: OpenTelemetry (Jaeger / Zipkin / OTLP)
Rate Limiting: Token bucket (memory or Redis)

Architecture

High-Level Architecture

Client
    │
    ▼
Forge Gateway
    │
    ▼
JWT Authentication
    │
    ▼
Rate Limiter
    │
    ▼
Router
    │
    ▼
Load Balancer
    │
    ▼
Circuit Breaker
    │
    ▼
Reverse Proxy
    │
    ▼
Upstream Services

Request Flow

flowchart LR
    subgraph Clients
        C1[HTTP Client]
        C2[k6 Benchmarks]
        C3[SDK / curl]
    end

    subgraph Gateway["Forge Gateway"]
        direction LR
        P1[Trace]
        P2[Logger]
        P3[Auth]
        P4[RateLimiter]
        P5[Router]
        P6[LoadBalancer]
        P7[CircuitBreaker]
        P8[Proxy]
        P1 --> P2 --> P3 --> P4 --> P5 --> P6 --> P7 --> P8
    end

    subgraph Upstreams
        U1[reviews-1]
        U2[reviews-2]
        U3[skills-1]
        U4[skills-2]
        U5[chaos-1]
    end

    subgraph Observability
        O1[Prometheus]
        O2[Grafana]
    end

    C1 --> Gateway
    C2 --> Gateway
    C3 --> Gateway

    Gateway --> U1
    Gateway --> U2
    Gateway --> U3
    Gateway --> U4
    Gateway --> U5

    Gateway --> O1
    O1 --> O2

Component Overview

Component	Responsibility
Trace	Resolve/inject `x-trace-id` for every request
Logger	Structured JSON via Pino, bound to trace ID
Auth	Verify JWT (HS256) on protected routes
Rate Limiter	Token bucket per (client, route); returns 429 + Retry-After
Router	Longest-prefix + optional host match
Load Balancer	Pick healthy instance (RR or LC)
Circuit Breaker	Per-upstream closed/open/half-open gate
Proxy	Streaming `http.request` with keep-alive agents

Design Decisions

Why raw `node:http`?

Lower overhead, complete control over sockets, and direct demonstration of networking fundamentals without framework abstractions.

Why configuration-driven routing?

Routes and upstreams are defined in YAML. Adding capacity or changing paths requires no recompilation — just edit gateway.config.yaml.

Why Token Bucket rate limiting?

Constant memory usage, natural burst support, and smooth refill behavior matches real-world API gateway requirements.

Why Circuit Breaker?

Prevents cascading failures. If an upstream degrades, the breaker trips to open, fails fast, and probes for recovery via half-open.

Why Active Health Checks?

The gateway actively polls /health on each upstream. Unhealthy instances are ejected from the pool automatically, ensuring traffic only reaches healthy nodes.

Why Keep-Alive Agents?

Separate keep-alive agents per upstream provide bulkhead isolation and reduce connection-establishment latency under load.

Why Round Robin + Least-connections?

Round-robin for even distribution; least-connections for workloads with variable response times. Both are health-aware.

Why Prometheus?

Standard /metrics exposition enables Grafana dashboards, alerting, and integration with existing observability stacks without custom instrumentation.

Request Lifecycle

Client
    │
    ▼
Trace ID
    │
    ▼
Logger
    │
    ▼
JWT Authentication
    │
    │ 401 if missing/invalid
    ▼
Rate Limiter
    │
    │ 429 + Retry-After if exhausted
    ▼
Router
    │
    │ 502 if no route matches
    ▼
Load Balancer
    │
    │ Selects healthy instance
    ▼
Circuit Breaker
    │
    │ Fast-fail if open
    ▼
Reverse Proxy
    │
    │ Streams request/response
    ▼
Upstream
    │
    ▼
Response

Trace — resolve or inject x-trace-id; bound to every log line.
Logger — structured JSON via Pino.
Auth — verify JWT (HS256); protected routes return 401 on invalid/expired.
Rate Limiter — token bucket per (client, route); 429 + Retry-After when exhausted.
Router — longest-prefix + optional host match → upstream.
Load Balancer — pick a healthy instance (round-robin or least-connections).
Circuit Breaker — per-upstream closed→open→half-open gate; fast-fail when open.
Proxy — streaming http.request with separate keep-alive agent per upstream (bulkhead).

Gateway Components

Reverse Proxy

Streams request bodies and response bodies using http.request. No buffering of the full payload — memory usage stays constant regardless of response size.

Load Balancer

Manages instances per upstream. Supports:

Round-robin: even distribution for homogeneous backends
Least-connections: prefers idle connections for variable-latency services
Health-aware: only routes to instances passing active health checks

Router

Longest-prefix match against configured routes. Optional host-based matching allows the same prefix on different domains.

Circuit Breaker

Per-upstream state machine:

Closed: normal operation; counts failures
Open: fast-fails after threshold; waits for cooldown
Half-Open: allows probe requests; closes on success, re-opens on failure

Rate Limiter

In-memory or Redis-backed token bucket. Each (client IP / API key, route) pair gets an independent bucket. The Redis backend uses an atomic Lua script so the quota is enforced globally across workers and gateway nodes; it fails open on Redis errors so a store outage never takes down the gateway. Configurable tokens-per-second and burst size.

Health Checker

Active polling of each upstream's /health endpoint at a configured interval. Instances that fail health checks are removed from the pool and marked unhealthy.

Observability

Metrics: Prometheus counters, gauges, and histograms via prom-client
Logging: Pino structured JSON with trace IDs
Tracing: OpenTelemetry spans (Jaeger / Zipkin / OTLP) when enabled via FORGE_OTEL_ENABLED
Dashboard: Grafana auto-provisioned with Prometheus datasource

Configuration Loader

Zod-validated YAML loader. Supports environment overrides via FORGE_* env vars. Routes, upstreams, limits, and timeouts are all centralized.

Load Balancing

Forge supports three strategies per upstream:

Strategy	Use Case
Round-robin	Homogeneous backends, even distribution
Least-connections	Variable-latency services, avoids overloading busy nodes
Consistent-hash	Sticky routing — pins a client (API key / IP) to one instance via rendezvous hashing, minimizing remapping when instances join/leave

Both strategies are health-aware. The load balancer only selects instances that have passed active health checks. If no instances are healthy, the proxy returns 502 Bad Gateway.

Automatic failover happens transparently: when an instance fails health checks or trips the circuit breaker, traffic shifts to remaining healthy instances.

Circuit Breaker

Closed
    │
    │ Consecutive failures exceed threshold
    ▼
Open
    │
    │ Cooldown period expires
    ▼
Half-Open
    │
    │ Probe succeeds
    ▼
Closed
    │
    │ Probe fails
    ▼
Open

Each upstream maintains its own breaker state. Consecutive failures trip the breaker to open. During the cooldown window, requests fail fast. After the cooldown, probe requests enter half-open state. Success closes the breaker; failure re-opens it.

Bulkhead Isolation

Gateway
    ├── Reviews Agent (keep-alive pool)
    ├── Skills Agent (keep-alive pool)
    └── Chaos Agent (keep-alive pool)

Each upstream gets its own HTTP agent with independent connection pools. This prevents one slow or misbehaving upstream from exhausting connections intended for another. The pattern is inspired by the bulkhead resilience pattern.

Service Discovery

Upstream membership can be kept in sync with a discovery source so the gateway scales upstreams (Kubernetes Deployments, Consul, DNS-based discovery) without a config reload or restart. Configure a discovery block on an upstream:

upstreams:
  - name: reviews
    strategy: round-robin
    instances: # seed list; overwritten by discovery once it resolves
      - id: reviews-1
        url: http://reviews-1:3001
    discovery:
      type: dns # or "static"
      host: reviews.default.svc.cluster.local
      port: 3001
      intervalSeconds: 15
    healthCheck: { ... }

dns: resolves A/AAAA records for host and builds one instance per IP on port (instance id host-<ip>, stable across refreshes).
static: treats urls as the authoritative membership set.

UpstreamPool.syncInstances reconciles the live set preserving each instance's health/connection state, so existing instances keep their breaker status across refreshes.

HTTP/2 (h2c)

The gateway can expose an additional cleartext HTTP/2 frontend (FORGE_HTTP2_PORT) alongside the HTTP/1.1 listener. Both share the exact same request pipeline; HTTP/2 connections use prior-knowledge negotiation (no TLS). TLS-terminated HTTP/2 is handled by the ingress/LB in front of the gateway.

Distributed Benchmarking

forge benchmark <id> --distributed (or -d) fans a single benchmark across multiple k6 load generators to raise total request rate beyond one machine's limits. Generators are listed in FORGE_LOADGENS (user@host,user@host2,...); the local machine is always generator #0. Each generator runs the same script with a 1/N slice of the VUs/rate, and their summary-export files are pulled back and merged into one combined result that the normal report pipeline renders.

FORGE_LOADGENS="loader1@10.0.0.5,loader2@10.0.0.6" \
  forge benchmark baseline --distributed

Forge is fully config-driven. Key configuration domains:

Routes: prefix, upstream name, auth requirement, rate limit
Upstreams: name, strategy (RR/LC/consistent-hash), instances, discovery, circuit breaker settings
Health Checks: endpoint, interval, timeout, healthy threshold
Rate Limits: tokens per second, burst capacity (memory or Redis-backed)
Timeouts: connect, response, and keep-alive timeouts
Circuit Breakers: failure threshold, cooldown seconds, half-open probes
Listeners: HTTP/1.1 port + optional HTTP/2 (h2c) port

Environment variables (FORGE_*) override config at runtime without restarts where supported.

upstreams:
  - name: reviews
    strategy: round-robin # or least-connections
    instances: [{ id: reviews-1, url: http://reviews:3001 }]
    circuitBreaker:
      { failureThreshold: 5, cooldownSeconds: 10, halfOpenProbes: 2 }
routes:
  - prefix: /api/reviews
    upstream: reviews
    auth: true
    rateLimit: { tokensPerSecond: 30, burst: 60 }

Metrics Glossary

Metric	Description
`gateway_latency`	Gateway processing latency (histogram)
`availability`	Successful responses (rate)
`instance_hits`	Requests per upstream instance
`breaker_trips`	Circuit breaker activations
`breaker_recoveries`	Successful circuit breaker recoveries
`rate_limited_total`	Total HTTP 429 responses

Benchmarking

Benchmark Overview

Benchmark	Purpose
`baseline`	Normal traffic, latency, availability
`spike`	Burst handling, rate limiter, stability
`sustained`	Long-running stability, latency drift
`breaking-point`	Maximum throughput, saturation, graceful degradation
`chaos`	Failover, health check, recovery
`failure-injection`	Circuit breaker, bulkhead, retry
`mixed-traffic`	Realistic workload across all routes
`load-balancer`	Distribution correctness, fairness
`circuit-breaker`	OPEN / HALF_OPEN / CLOSED transitions
`recovery`	Health checker recovery, traffic restoration
`rate-limiter`	Token bucket validation, 429 responses
`payload`	1KB–10MB proxy performance
`timeout`	Upstream timeout (504) + breaker interaction
`soak`	6–24h leak / drift detection

Benchmark Workflow

Run Benchmark
    │
    ▼
Generate JWT
    │
    ▼
Execute k6
    │
    ▼
Summary Export
    │
    ▼
Report Generator
    │
    ├── Markdown
    ├── HTML
    └── JSON
    │
    ▼
Comparison / Regression

Benchmark Architecture

Benchmark CLI
    │
    ▼
Runner
    │
    ▼
k6
    │
    ▼
Gateway
    │
    ▼
Summary Export
    │
    ▼
Parser
    │
    ▼
Report Generator
    │
    ├── Markdown
    ├── HTML
    └── JSON

CLI Usage

# Run a single benchmark
forge benchmark baseline

# Run specific benchmarks
forge benchmark spike
forge benchmark sustained
forge benchmark chaos

# Run all benchmarks
forge benchmark all

# Compare two runs
forge benchmark compare run-a.json run-b.json

# Generate combined report
forge benchmark report

# Check environment readiness
forge benchmark doctor

# Run across multiple load generators (FORGE_LOADGENS)
forge benchmark baseline --distributed

# Clean results
forge benchmark clean

Load Test Results

Measured locally with k6 against a single mock upstream + single gateway instance (local dev config). Capacity numbers will be updated after the Docker Compose run (docker compose up --build), which fronts reviews×2 + skills×2 for real multi-instance capacity.

Latest Results (13 benchmarks)

Benchmark	Throughput	Requests	Error Rate	Availability	HTTP p95	HTTP p99	GW p95	GW p99	Breaker Trips
`baseline`	199.97 RPS	12000	0.00%	100.00%	3.04 ms	4.08 ms	3.04 ms	4.08 ms	0
`breaking-point`	2807.89 RPS	507969	83.57%	16.43%	860.18 ms	1727.77 ms	860.18 ms	1727.77 ms	0
`chaos`	249.99 RPS	75000	0.00%	100.00%	3.02 ms	4.06 ms	3.02 ms	4.06 ms	0
`circuit-breaker`	100.00 RPS	7000	45.37%	54.63%	2.74 ms	3.24 ms	2.74 ms	3.24 ms	50
`failure-injection`	399.95 RPS	36002	0.00%	100.00%	3.27 ms	4.23 ms	3.27 ms	4.23 ms	0
`load-balancer`	399.97 RPS	48002	0.00%	100.00%	3.31 ms	4.40 ms	3.31 ms	4.40 ms	0
`mixed-traffic`	299.99 RPS	90000	0.00%	100.00%	3.01 ms	4.11 ms	3.01 ms	4.11 ms	0
`payload`	48.44 RPS	8994	15.63%	84.37%	5814.55 ms	5869.00 ms	5814.87 ms	5869.02 ms	0
`rate-limiter`	100.00 RPS	12000	0.00%	100.00%	3.25 ms	3.86 ms	3.25 ms	3.86 ms	0
`recovery`	249.99 RPS	75001	0.00%	100.00%	3.06 ms	4.02 ms	3.06 ms	4.02 ms	0
`spike`	1644.47 RPS	131558	76.91%	23.09%	20.60 ms	35.69 ms	20.60 ms	35.69 ms	0
`sustained`	499.99 RPS	300001	3.92%	96.08%	3.94 ms	5.31 ms	3.94 ms	5.31 ms	0
`timeout`	80.99 RPS	15094	50.00%	50.00%	10002.89 ms	10003.40 ms	10002.89 ms	10003.40 ms	300

See loadtests/results/report.md for the full combined report.

Key Observations

Throughput (baseline): sustained 200 req/s at P95 latency 3.0 ms (p99 4.1 ms), 0% error.
Overhead (NFR1): end-to-end gateway P95 ≈ 3.0 ms including JWT verify + proxying; direct-to-upstream comparison pending the Docker run.
Resilience (spike): gateway survived a ~1600 req/s spike — 23% success, no crash; transient errors are expected under saturation (rate limiting + connection limits).
Circuit breaker (NFR3): trips to open after consecutive failures (configurable per upstream); timeout benchmark shows 300 breaker trips after upstream stalls.
Rate limiting (FR5): clients exceeding the per-route token bucket (default 50 tokens/s) receive 429 + Retry-After.
Payload: large payloads (10MB) degrade performance as expected; gateway remains available at 84% with P95 latency ~5.8 s.

Project Layout

src/
  server.ts                 entrypoint: http server, /metrics, /healthz
  core/
    gateway.ts              request pipeline orchestration
    router.ts               path/host matching
    loadBalancer.ts         round-robin + least-connections
    upstreamPool.ts         per-instance health + connection state
    healthCheck.ts          active /health polling
    circuitBreaker.ts       closed/open/half-open state machine
    proxy.ts                streaming proxy + bulkhead agents
    trace.ts / headers.ts   trace id + header injection/stripping
  middleware/
    auth.ts                 JWT verification
    rateLimiter.ts          token-bucket limiter
  observability/
    logger.ts               pino structured logging
    metrics.ts              prom-client collectors
  config/                   zod schema + env-aware loader
mock-upstreams/index.js     generic mock service (chaos hooks via query)
config/gateway.config.yaml  routes, upstreams, limits
loadtests/                  k6 baseline / spike / failure-injection
prometheus/ grafana/        observability stack
docker-compose.yml          gateway + mocks + Prometheus + Grafana
tests/                      unit + integration (vitest)

Quick Start (local, no Docker)

npm install

# Terminal 1: a mock upstream
PORT=3001 SERVICE_NAME=reviews INSTANCE_ID=reviews-1 node mock-upstreams/index.js

# Terminal 2: the gateway (local config points at 127.0.0.1:3001)
FORGE_CONFIG=config/local.config.yaml FORGE_PORT=8090 npx tsx src/server.ts

# Generate a token and call a protected route
TOKEN=$(node -e "console.log(require('jsonwebtoken').sign({sub:'u'},'dev-secret-change-me',{algorithm:'HS256'}))")
curl -i -H "Authorization: Bearer $TOKEN" http://localhost:8090/api/reviews/items
curl http://localhost:8090/metrics

Full Stack (Docker Compose)

cp .env.example .env          # set a real FORGE_JWT_SECRET
docker compose up --build

Gateway: http://localhost:8080
Prometheus: http://localhost:9090
Grafana: http://localhost:3000 (dashboard "Forge Gateway" auto-provisioned)

Chaos / failure testing

The mock chaos service honours query hooks so you can inject failures without code changes:

GET /api/chaos/ping?error=1 → upstream returns 500 (trips the breaker).
GET /api/chaos/ping?latency=3000 → upstream delays 3s (tests timeouts).
Mock env FAIL_RATE=0.3 → 30% of non-health requests fail randomly.

Kill a mock container mid-load (docker compose stop reviews-1) to watch the health checker remove it and the breaker isolate the failure.

Testing

npm test                 # vitest: unit (router, LB, breaker, limiter) + integration
npm run typecheck
npm run build

Load tests (k6):

TOKEN=$(node -e "console.log(require('jsonwebtoken').sign({sub:'u'},'dev-secret-change-me',{algorithm:'HS256'}))")
k6 run -e BASE_URL=http://localhost:8080 -e JWT=$TOKEN loadtests/baseline.js
k6 run -e BASE_URL=http://localhost:8080 -e JWT=$TOKEN loadtests/spike.js
k6 run -e BASE_URL=http://localhost:8080 -e JWT=$TOKEN loadtests/failure-injection.js

Out of Scope (v1)

TLS/ACME, multi-region, gRPC, dashboard UI beyond Grafana, mTLS, dynamic service discovery, and multi-node gateway clustering (single instance is enough to demonstrate the concepts — noted here as explicit future work).

Roadmap

Implemented

✅ Redis-backed distributed rate limiter — atomic Lua token bucket over ioredis, shared across workers/nodes (src/middleware/rateLimiterRedis.ts, src/config/redis.ts). Fails open on Redis errors.
✅ Consistent-hash load balancing — highest-random-weight (rendezvous) hashing keyed on the client (API key / IP), so clients pin to one instance and the mapping is stable under membership changes (src/core/loadBalancer.ts).
✅ OpenTelemetry distributed tracing — gateway.request spans with trace id, route, upstream and method; Jaeger / Zipkin / OTLP exporters, lazy-loaded so there is zero overhead when disabled (src/observability/tracing.ts).
✅ Service discovery — DNS (A/AAAA) and static providers refresh upstream instance membership at runtime without a restart or config reload (src/core/serviceDiscovery.ts, UpstreamPool.syncInstances).
✅ HTTP/2 (h2c) frontend — optional cleartext HTTP/2 listener reusing the same request pipeline (FORGE_HTTP2_PORT, src/server.ts).
✅ Distributed benchmarking — fan a benchmark across multiple k6 load generators (local + SSH remotes) and merge their summary exports (loadtests/framework/distributed.js, forge benchmark <id> --distributed).
✅ Kubernetes Helm chart (helm/forge-gateway/)

Planned

HTTP/3 (QUIC) — requires a QUIC-capable listener; out of scope for raw node:http and deferred until a QUIC transport is adopted.
gRPC proxying
Kubernetes operator

Name		Name	Last commit message	Last commit date
Latest commit History 9 Commits
.kilo/plans		.kilo/plans
benchmarks		benchmarks
bin		bin
config		config
grafana		grafana
helm/forge-gateway		helm/forge-gateway
loadtests		loadtests
mock-upstreams		mock-upstreams
prometheus		prometheus
src		src
tests		tests
.env.example		.env.example
.gitignore		.gitignore
BENCHMARK_PRD.md		BENCHMARK_PRD.md
Dockerfile		Dockerfile
Dockerfile.mock		Dockerfile.mock
PRD.md		PRD.md
README.md		README.md
WORKFLOW.md		WORKFLOW.md
docker-compose.yml		docker-compose.yml
forge		forge
package-lock.json		package-lock.json
package.json		package.json
test_build.js		test_build.js
test_parse.js		test_parse.js
tsconfig.json		tsconfig.json
vitest.config.ts		vitest.config.ts

Folders and files

Latest commit

History

Repository files navigation

Forge — Lightweight API Gateway & Reverse Proxy

Performance Summary

Features

Tech Stack

Architecture

High-Level Architecture

Request Flow

Component Overview

Design Decisions

Why raw node:http?

Why configuration-driven routing?

Why Token Bucket rate limiting?

Why Circuit Breaker?

Why Active Health Checks?

Why Keep-Alive Agents?

Why Round Robin + Least-connections?

Why Prometheus?

Request Lifecycle

Gateway Components

Reverse Proxy

Load Balancer

Router

Circuit Breaker

Rate Limiter

Health Checker

Observability

Configuration Loader

Load Balancing

Circuit Breaker

Bulkhead Isolation

Service Discovery

HTTP/2 (h2c)

Distributed Benchmarking

Metrics Glossary

Benchmarking

Benchmark Overview

Benchmark Workflow

Benchmark Architecture

CLI Usage

Load Test Results

Latest Results (13 benchmarks)

Key Observations

Project Layout

Quick Start (local, no Docker)

Full Stack (Docker Compose)

Chaos / failure testing

Testing

Out of Scope (v1)

Roadmap

Implemented

Planned

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Why raw `node:http`?