Ollama Proxy

A high-performance reverse proxy for Ollama with enterprise-grade features including authentication, rate limiting, token usage tracking, OpenAI/Anthropic API compatibility, and a modern admin dashboard.

Features

Core Features

JWT Authentication: Secure JWT-based authentication with refresh tokens
API Token Authentication: Create and manage API tokens for programmatic access
Role-Based Access Control: Three roles - read, generate, and admin
Rate Limiting: Configurable limits per minute, hour, and day at global, user, and token levels
Token Usage Tracking: Track prompt and completion tokens per user/token
Model Access Control: Allow or deny specific models at global, user, and token levels
Flexible Storage: Redis for production with automatic SQLite fallback
Argon2id Password Hashing: Industry-standard secure password hashing
Audit Logging: Comprehensive audit trail of all system actions

API Compatibility

OpenAI API Compatible: Full support for OpenAI SDKs
- Chat Completions API (/v1/chat/completions)
- Completions API (/v1/completions)
- Embeddings API (/v1/embeddings)
- Models API (/v1/models)
Anthropic API Compatible: Full support for Anthropic Messages API
- Messages API (/v1/messages)
- Streaming support
Ollama Native API: Full proxy support for all Ollama endpoints

Admin Dashboard

Modern UI: Built with Tailwind CSS, Alpine.js, and htmx
Server-Side Rendering: htmx-powered partials for efficient data loading
User Management: Create, edit, delete users with role assignment
Token Management: Full API token lifecycle management
Usage Statistics: User and token statistics modals with charts, daily breakdown, and activity logs
Real-time Statistics: Usage charts and live metrics with 24h/7d/30d filters
Audit Log Viewer: Filterable audit log with search and notifications
Configuration Editing: Edit rate limits and model restrictions
Confirmation Dialogs: Safe destructive actions with confirmation prompts
Auto-refresh: Live updates via htmx polling
Responsive Design: Works on desktop and mobile

Infrastructure

Comprehensive Logging: JSON/text format with file rotation (lumberjack)
Health Endpoints: /health and /ready for monitoring
Graceful Shutdown: Clean shutdown with connection draining
Docker Support: Multi-stage builds and Docker Compose ready

Quick Start

Prerequisites

Docker and Docker Compose (recommended)
Ollama running on the host machine (default port 11434)
Go 1.26 or later (only if building from source rather than using Docker)

Installation

Clone the repository:

git clone https://github.com/anorebel/ollama-proxy.git
cd ollama-proxy

Create your environment file:

cp .env.example .env

Edit .env and set the required values:

# Generate a secure JWT secret (must be at least 32 bytes)
JWT_SECRET=$(openssl rand -base64 32)

# A strong password you generate — not this literal string
ADMIN_PASSWORD=<a strong generated password>

# Where your Ollama backend is. Compose maps host.docker.internal to the
# host gateway, so this reaches an Ollama running on the host machine.
OLLAMA_URL=http://host.docker.internal:11434

Start the services:

docker compose up -d

Access the admin dashboard at http://localhost:8080/admin

For Docker without Compose, running from source, systemd, and putting this behind TLS, see Running and hosting.

Configuration

Environment Variables

Required

Variable	Description	Default
`JWT_SECRET`	Required. Secret key for JWT signing. Must be at least 32 bytes; startup fails otherwise. Generate with `openssl rand -base64 32`	-
`ADMIN_PASSWORD`	Required. Password used to provision the admin user on first start	-

The service refuses to start if JWT_SECRET is shorter than 32 bytes or is left set to one of the placeholder values published in this repository (for example your-super-secret-jwt-key-change-this). The startup error names the variable and shows how to generate a valid value.

Server

Variable	Description	Default
`SERVER_HOST`	Server bind address	`0.0.0.0`
`SERVER_PORT`	Server port	`8080`
`CORS_ALLOWED_ORIGINS`	Comma-separated allowlist of browser origins permitted to call the API. Empty means no cross-origin browser access. The wildcard `*` is rejected at startup	- (none)
`SERVER_READ_HEADER_TIMEOUT`	Maximum time a client may take to send request headers	`10s`
`SERVER_READ_TIMEOUT`	Maximum time to read the full request (headers + body)	`60s`
`SERVER_WRITE_TIMEOUT`	Maximum time to write the response. Leave at `0` — see the warning below	`0` (disabled)
`SERVER_IDLE_TIMEOUT`	How long an idle keep-alive connection is kept before closing	`120s`
`SERVER_MAX_BODY_SIZE`	Largest accepted request body, e.g. `32M`. Oversized requests receive `413`. Generous enough for base64-encoded images	`32M`

CORS

Cross-origin access is restricted to an explicit allowlist. By default the list is empty, so no Access-Control-Allow-Origin header is sent and browsers deny every cross-origin read. To serve a browser front-end from another origin, list the exact origins:

CORS_ALLOWED_ORIGINS=https://admin.example.com,http://localhost:5173

The wildcard * is refused at startup. Because callers authenticate with an Authorization: Bearer header read from client-side storage rather than with cookies, a wildcard would let any site on the internet drive this administrative API from a victim's browser. This is not needed for server-side or CLI clients (curl, SDKs, other backends) — CORS only constrains browsers.

Write timeout and streaming

SERVER_WRITE_TIMEOUT defaults to 0 (disabled) deliberately. Go measures the write timeout from the end of the request header read, not from the first byte written, so any non-zero value caps the total lifetime of a streaming generation — a long response is severed mid-stream and the client cannot tell a truncation from a completed answer. Upstream duration is already bounded by OLLAMA_TIMEOUT, and the read and idle timeouts cover the slow-client and idle-connection cases. Only set a write timeout if you expose no streaming endpoints.

Ollama

Variable	Description	Default
`OLLAMA_URL`	URL to Ollama server	`http://localhost:11434`
`OLLAMA_TIMEOUT`	Request timeout	`120s`

Storage

Variable	Description	Default
`REDIS_URL`	Redis connection URL	`redis://localhost:6379`
`REDIS_PASSWORD`	Redis password	-
`REDIS_DB`	Redis database number	`0`
`STORAGE_SQLITE_PATH`	SQLite fallback path	`./data/ollama-proxy.db`

Authentication

Variable	Description	Default
`AUTH_ENABLED`	Enable authentication	`true`
`ADMIN_USERNAME`	Admin username	`admin`
`ADMIN_PASSWORD_FORCE_RESET`	Overwrite the stored admin password with `ADMIN_PASSWORD` on startup. Off by default so a password rotated in the UI survives restarts	`false`

Admin password lifecycle

ADMIN_PASSWORD provisions the admin account the first time the service starts against an empty database. Once the account exists the value is not re-applied on subsequent restarts — a password changed through the admin UI stays changed.

To recover a lost admin password, set ADMIN_PASSWORD to the new value and start the service once with ADMIN_PASSWORD_FORCE_RESET=true. The reset is recorded in the audit log under the admin_password_reset action. Unset the flag afterwards so the next restart does not revert a subsequent rotation.

JWT

Variable	Description	Default
`JWT_EXPIRATION`	Token expiration time	`24h`

Rate Limits (Defaults)

Variable	Description	Default
`DEFAULT_REQUESTS_PER_MINUTE`	Requests per minute	`60`
`DEFAULT_REQUESTS_PER_HOUR`	Requests per hour	`1000`
`DEFAULT_REQUESTS_PER_DAY`	Requests per day	`10000`
`DEFAULT_TOKENS_PER_DAY`	Tokens per day	`1000000`
`DEFAULT_CONCURRENT_REQUESTS`	Max concurrent requests	`5`
`LIMITS_FAIL_OPEN`	Admit requests when the counter store is unreachable. Off by default, so an outage cannot silently remove all rate limiting	`false`

Model Access

Variable	Description	Default
`DEFAULT_ALLOWED_MODELS`	Comma-separated allowed models	`""` (all)
`DEFAULT_DENIED_MODELS`	Comma-separated denied models	`""` (none)

Logging

Variable	Description	Default
`LOG_LEVEL`	Log level (debug, info, warn, error)	`info`
`LOG_FORMAT`	Log format (json, text)	`json`
`LOG_FILE`	Log file path	`./logs/ollama-proxy.log`

Admin Dashboard

Variable	Description	Default
`ADMIN_ENABLED`	Enable admin UI	`true`
`ADMIN_PATH`	Admin UI path. Assets and the page's `<base href>` follow this value, so a custom path serves correctly	`/admin`

See .env.example for all available options.

Usage

Authentication

Using JWT (Bearer Token)

# Login to get tokens
curl -X POST http://localhost:8080/api/admin/login \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "your-password"}'

# Use the access token
curl http://localhost:8080/api/tags \
  -H "Authorization: Bearer <access_token>"

Using API Token

# Create an API token via the admin dashboard or API
# Then use it in requests
curl http://localhost:8080/api/tags \
  -H "X-API-Key: olp_abc123..."

# An API token may also be presented as a bearer token
curl http://localhost:8080/api/tags \
  -H "Authorization: Bearer olp_abc123..."

API keys must be sent in a header. The ?api_key= query parameter is no longer accepted and returns 401. Query strings are recorded in access logs, browser history and Referer headers, which leaks the credential to anyone who can read them. Use X-API-Key or Authorization: Bearer instead.

Token lookup prefix widened. Tokens are indexed by their first 20 characters (olp_ plus 16 hex digits) instead of the first 8. The old width left only 16 bits of variable entropy, so two tokens collided at around 256 issued tokens — and a collision was silent: on SQLite the non-unique index could resolve a lookup to the wrong record and reject a valid token, and on Redis the newer token overwrote the older one's mapping, permanently orphaning it. The prefix is now a unique index and issuance retries on collision.

Existing tokens keep working — lookup falls back to the old 8-character width, so no re-issuance is needed. On first start against an existing SQLite database the non-unique index is rebuilt as unique. If two stored tokens already share an 8-character prefix that rebuild fails and startup reports it: delete or re-issue one of the colliding tokens, since one of them was already unusable.

Roles and Permissions

Role	Permissions
`read`	List models, show model info, check running models
`generate`	All `read` permissions + generate, chat, embeddings
`admin`	All permissions + admin API access

Roles are resolved from storage on every request, not from the role recorded in the token when it was issued. Changing a user's role therefore takes effect on their next request, including for tokens that were issued before the change and have not yet expired. An API token's role can only narrow its owner's current role, never exceed it — demoting a user immediately demotes their tokens too.

API Endpoints

Ollama Native API

All Ollama endpoints are proxied at /api/*:

# List models
curl http://localhost:8080/api/tags \
  -H "X-API-Key: <your-token>"

# Generate text
curl http://localhost:8080/api/generate \
  -H "X-API-Key: <your-token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama2", "prompt": "Hello!"}'

# Chat
curl http://localhost:8080/api/chat \
  -H "X-API-Key: <your-token>" \
  -H "Content-Type: application/json" \
  -d '{"model": "llama2", "messages": [{"role": "user", "content": "Hello!"}]}'

OpenAI Compatible API

# Chat completions
curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer <your-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama2",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

# List models (OpenAI format)
curl http://localhost:8080/v1/models \
  -H "Authorization: Bearer <your-token>"

Anthropic Compatible API

# Messages API
curl http://localhost:8080/v1/messages \
  -H "Authorization: Bearer <your-token>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama2",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Health Checks

# Health check
curl http://localhost:8080/health

# Readiness check
curl http://localhost:8080/ready

Admin API

Authentication

# Login
POST /api/admin/login
{
  "username": "admin",
  "password": "password"
}

# Refresh token (rotating)
POST /api/admin/refresh
{
  "refresh_token": "<refresh_token>"
}
# Returns a NEW refresh_token alongside the access token. The refresh token you
# submitted is spent and will not work again — store the new one.

# Logout (revokes this session's credentials)
POST /api/admin/logout
Authorization: Bearer <access_token>
{
  "refresh_token": "<refresh_token>"   # optional, revokes the whole session
}

Token revocation

Issued JWTs carry a unique jti and are checked against a revocation denylist on every request, so a credential can be withdrawn before it expires. Tokens are revoked when:

a user logs out (the presented access token, and the refresh family if the refresh token is supplied),
a user is disabled, deleted, or has their role changed — every session that user holds stops working immediately,
an already-redeemed refresh token is presented again.

Denylist entries expire together with the credential they deny, so the list stays bounded and needs no manual pruning.

Refresh token rotation and reuse detection

Each redemption issues a new refresh token and invalidates the one presented. Presenting an already-redeemed refresh token is treated as theft: the entire token family descended from that login is revoked, and both the legitimate user and the attacker must authenticate again. Clients must therefore persist the new refresh_token returned by every refresh call; replaying an old one will end the session.

User Management

# List users
GET /api/admin/users

# Create user
POST /api/admin/users
{
  "username": "user1",
  "password": "password",
  "role": "generate",
  "requests_per_minute": 30,
  "allowed_models": ["llama2", "mistral"]
}

# Get user
GET /api/admin/users/:id

# Update user
PUT /api/admin/users/:id
{
  "role": "admin",
  "enabled": true,
  "requests_per_day": 5000
}

# Update user password
PUT /api/admin/users/:id/password
{
  "password": "new-password"
}

# Delete user
DELETE /api/admin/users/:id

# Get user statistics
GET /api/admin/users/:id/stats?days=30

# Get user activity
GET /api/admin/users/:id/activity?limit=20

# List user tokens
GET /api/admin/users/:user_id/tokens

Token Management

# List tokens
GET /api/admin/tokens

# Create token
POST /api/admin/tokens
{
  "user_id": "user-uuid",
  "name": "Production API",
  "role": "generate",
  "expires_in": "720h",
  "requests_per_minute": 60,
  "allowed_models": ["llama2"]
}

# Get token
GET /api/admin/tokens/:id

# Get token statistics
GET /api/admin/tokens/:id/stats?days=30

# Update token
PUT /api/admin/tokens/:id
{
  "enabled": false,
  "role": "read"
}

# Delete token
DELETE /api/admin/tokens/:id

Statistics & Monitoring

# Get global statistics
GET /api/admin/stats

# Get system health
GET /api/admin/health

# Get configuration
GET /api/admin/config

# Update configuration
PUT /api/admin/config
{
  "limits": {
    "requests_per_minute": 120,
    "tokens_per_day": 2000000
  },
  "models": {
    "allowed": ["llama2", "mistral"],
    "denied": ["model1"]
  }
}

Audit Logs

# Get audit logs (with filters)
GET /api/admin/audit?action=login&user_id=xxx&success=true&limit=50&offset=0

Admin Dashboard

The admin dashboard provides a modern, responsive interface for managing all aspects of the proxy:

Features

Overview: Real-time statistics, usage charts, system health
Users: Full CRUD operations, statistics, activity logs
Tokens: API token management with user assignment
Audit Logs: Searchable and filterable audit trail
Configuration: Edit rate limits and model restrictions

Access

Navigate to http://localhost:8080/admin and login with your admin credentials.

Screenshots

Login Page

Dashboard Overview

User Management

Token Management

Configuration Settings

Audit Logs

Model Tuning

Generation settings are per model, organised as reusable profiles. Assigning a profile to several models means editing it once updates all of them.

Profile Editor

Any field left empty falls through to each model's own default.

Per-Model Settings

The form shows only the settings a model actually supports, prefilled with the value in effect and labelled with where it came from. A reasoning model offers a thinking level alongside the full sampling controls:

An embedding model offers only a context window, since sampling controls do not apply to it:

Mobile

The sidebar collapses into a drawer below the lg breakpoint.

Storage Architecture

The proxy uses a flexible storage abstraction that automatically falls back to SQLite when Redis is unavailable:

Redis (Recommended for Production)

High performance with in-memory operations
Built-in TTL support for rate limits
Atomic operations for consistency
Supports concurrent connections

SQLite (Automatic Fallback)

Used when Redis is not available
File-based persistence
Supports all Redis features via GORM ORM
Automatic schema migration

Running and hosting

The proxy is a single static Go binary that embeds the admin dashboard. It needs an Ollama backend to talk to, and optionally Redis; with no Redis it falls back to SQLite automatically.

Three things are required however you run it:

Variable	Why
`JWT_SECRET`	Signs admin sessions. At least 32 bytes. Generate with `openssl rand -base64 32`. Placeholder values from this repository are rejected at startup.
`ADMIN_PASSWORD`	Provisions the initial admin account on first start.
`OLLAMA_URL`	Where your Ollama backend lives.

Set CORS_ALLOWED_ORIGINS as well if a browser on another origin will call the API. It is empty by default, which denies all cross-origin browser reads; * fails startup deliberately.

Docker Compose (recommended)

cp .env.example .env
# Edit .env: set JWT_SECRET, ADMIN_PASSWORD, and OLLAMA_URL.
#   JWT_SECRET=$(openssl rand -base64 32)

docker compose up -d

This starts the proxy and a Redis instance, with health checks, restart policies and persistent volumes for Redis data and logs. It does not run Ollama — point OLLAMA_URL at an existing backend.

Reaching an Ollama running on the host machine:

docker-compose.yml (bridge networking, the default) maps host.docker.internal to the host gateway, so OLLAMA_URL=http://host.docker.internal:11434 works. A LAN address such as http://192.168.0.10:11434 works too.
docker-compose.host.yml uses host networking, for when the container needs real localhost access:
```
docker compose -f docker-compose.host.yml up -d
```

Then:

docker compose logs -f ollama-proxy     # follow logs
docker compose ps                       # health status
docker compose down                     # stop
docker compose up -d --build            # rebuild after a code change

The dashboard is at http://localhost:8080/admin.

Docker without Compose

docker build -t ollama-proxy .

docker run -d --name ollama-proxy \
  -p 8080:8080 \
  --add-host host.docker.internal:host-gateway \
  -e JWT_SECRET="$(openssl rand -base64 32)" \
  -e ADMIN_PASSWORD='<a strong generated password>' \
  -e OLLAMA_URL=http://host.docker.internal:11434 \
  ollama-proxy

With no REDIS_URL the proxy uses SQLite. That database lives inside the container, so mount a volume to keep users and tokens across restarts:

  -v ollama-proxy-data:/data -e STORAGE_SQLITE_PATH=/data/proxy.db

To stamp the image with a version, see Versioning.

Manually, from source

Requires Go 1.26 or later. No Node toolchain is needed — the dashboard's stylesheet is committed and embedded in the binary.

git clone https://github.com/anorebel/ollama-proxy.git
cd ollama-proxy
go build -o ollama-proxy ./cmd/proxy

JWT_SECRET="$(openssl rand -base64 32)" \
ADMIN_PASSWORD='<a strong generated password>' \
OLLAMA_URL=http://localhost:11434 \
./ollama-proxy

Configuration comes from environment variables, or a YAML file:

cp config.yaml.example config.yaml
./ollama-proxy --config config.yaml

Environment variables win over the file. See Configuration for the full list.

Running as a systemd service

# /etc/systemd/system/ollama-proxy.service
[Unit]
Description=Ollama Proxy
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=ollama-proxy
Group=ollama-proxy
ExecStart=/usr/local/bin/ollama-proxy
Restart=on-failure
RestartSec=5s

# Secrets belong in a root-owned file with mode 0600, not in this unit —
# anything here is world-readable via `systemctl show`.
EnvironmentFile=/etc/ollama-proxy/proxy.env

# The proxy needs no write access outside its state directory.
StateDirectory=ollama-proxy
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true

[Install]
WantedBy=multi-user.target

sudo useradd --system --no-create-home --shell /usr/sbin/nologin ollama-proxy
sudo install -d -m 0750 -o root -g ollama-proxy /etc/ollama-proxy
sudo install -m 0640 -o root -g ollama-proxy /dev/null /etc/ollama-proxy/proxy.env
# Put JWT_SECRET, ADMIN_PASSWORD, OLLAMA_URL and
# STORAGE_SQLITE_PATH=/var/lib/ollama-proxy/proxy.db in that file.

sudo systemctl daemon-reload
sudo systemctl enable --now ollama-proxy
sudo systemctl status ollama-proxy
journalctl -u ollama-proxy -f

Putting it on the internet

The proxy speaks plain HTTP and does not terminate TLS. Bearer tokens would cross the network in cleartext, so run it behind a reverse proxy that terminates TLS and do not expose it directly.

server {
    listen 443 ssl;
    server_name proxy.example.com;

    ssl_certificate     /etc/letsencrypt/live/proxy.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/proxy.example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host $host;

        # Streaming must not be buffered, or responses arrive all at once at
        # the end and the whole point of incremental streaming is lost.
        proxy_buffering off;
        proxy_cache off;

        # Generation can take minutes; the default 60s read timeout would cut
        # long responses off mid-stream.
        proxy_read_timeout 600s;
    }
}

Two things to know when running behind a proxy:

Bind to loopback (PROXY_HOST=127.0.0.1) so the plain-HTTP port is not reachable from outside the host.
The client address comes from the connection, not X-Forwarded-For. That is deliberate — it stops a caller from forging the source address in audit logs or rotating headers to reset their rate-limit bucket. Behind a load balancer every request therefore appears to come from the balancer. Enabling header-based extraction requires an IP extractor scoped to your trusted hops, and must not be enabled while the service is directly reachable.

Verifying a deployment

curl http://localhost:8080/health   # {"status":"healthy","ollama":"connected",...}
curl http://localhost:8080/ready    # {"status":"ready"}

/health reports the running version, which is the quickest way to confirm which build is deployed. Both endpoints are unauthenticated, for probes.

Kubernetes

See FEATURES.md for complete Kubernetes deployment examples.

Dokploy/Railway

Option 1: Docker Compose (Recommended)

Create a new application in Dokploy
Connect your Git repository
Set the build type to "Docker Compose"
Add the required environment variables:
- JWT_SECRET
- ADMIN_PASSWORD
Deploy!

The docker-compose.yml includes Redis automatically with:

Health checks for both the proxy and Redis
Automatic restart on failure
Persistent volumes for Redis data and logs

Option 2: Railpack/Nixpacks (Single Container)

If using Railpack or Nixpacks, you need to deploy Redis separately:

Deploy Redis first:
- Create a new Redis service in Dokploy
- Note the internal hostname (e.g., redis-abc123)
Deploy the proxy:
- Create a new application with Railpack/Nixpacks
- Connect your Git repository
- Add environment variables:
  - REDIS_URL=redis://your-redis-hostname:6379
  - JWT_SECRET — paste the output of openssl rand -base64 32
  - ADMIN_PASSWORD — a strong password you generate
  - OLLAMA_URL=http://host.docker.internal:11434
- Deploy!

Required environment variables for Railpack:

REDIS_URL=redis://your-redis-service:6379    # Required
JWT_SECRET=$(openssl rand -base64 32)        # Required, min 32 bytes
ADMIN_PASSWORD=<a strong generated password> # Required
OLLAMA_URL=http://host.docker.internal:11434 # Ollama on host

Placeholder values such as your-secret-key are rejected at startup — supply real generated secrets.

Development

Requires Go 1.26 or later.

If you plan to open a pull request, read CONTRIBUTING.md — it covers what CI expects, the opt-in test suites, and a few things in this codebase that are load-bearing in non-obvious ways.

Running Locally

go mod download

# JWT_SECRET must be at least 32 bytes.
JWT_SECRET="$(openssl rand -base64 32)" \
ADMIN_PASSWORD='<a strong generated password>' \
OLLAMA_URL=http://localhost:11434 \
go run ./cmd/proxy

Redis is optional — without REDIS_URL the proxy falls back to SQLite, which is usually what you want for a development loop. To exercise the Redis path, add REDIS_URL=redis://localhost:6379.

For deploying rather than developing, see Running and hosting.

Building

# Build binary
go build -o ollama-proxy ./cmd/proxy

# Build Docker image
docker build -t ollama-proxy .

Versioning

The binary reports its own identity:

./ollama-proxy --version
# ollama-proxy 1.4.0 (commit 9fceb02, built 2026-08-13T09:00:00Z, go1.26.5)

The same values appear in the startup log, in GET /health, and in the X-Proxy-Version header sent to the Ollama backend — so a request in Ollama's logs can be traced to the proxy build that made it.

Version, commit and build date are injected at link time into internal/version. Releases are cut by tagging (v*), which drives goreleaser for the binaries and the build args for the container image; both are already wired up, so a tagged release needs no manual version bump anywhere.

To stamp a local build:

PKG=github.com/anorebel/ollama-proxy/internal/version
go build -ldflags "\
  -X $PKG.Version=$(git describe --tags --always) \
  -X $PKG.Commit=$(git rev-parse HEAD) \
  -X $PKG.Date=$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  -o ollama-proxy ./cmd/proxy

Or for a container:

docker build \
  --build-arg VERSION=$(git describe --tags --always) \
  --build-arg COMMIT=$(git rev-parse HEAD) \
  --build-arg BUILD_DATE=$(date -u +%Y-%m-%dT%H:%M:%SZ) \
  -t ollama-proxy .

An unstamped build reports dev rather than inventing a number, and falls back to the commit the Go toolchain records, so even go install builds stay traceable.

If you move or rename anything in internal/version, update the -X paths in .goreleaser.yml and the Dockerfile to match. The linker silently ignores a -X flag naming a symbol that does not exist, so a stale path does not fail the build — it ships a release that reports dev. TestLdflagsPathsResolve guards exactly this.

Dashboard assets

The dashboard's stylesheet is generated from internal/admin/static/src/input.css and committed to internal/admin/static/css/tailwind.css. The Go binary embeds it, so go build ./cmd/proxy produces a working server with no Node or Bun toolchain installed — building and running this project needs Go alone.

Regenerating it is only necessary after changing markup or adding utility classes. Use the standalone Tailwind CLI, which is a single binary and needs no package.json:

curl -sLO https://github.com/tailwindlabs/tailwindcss/releases/latest/download/tailwindcss-linux-x64
chmod +x tailwindcss-linux-x64
./tailwindcss-linux-x64 \
  -i internal/admin/static/src/input.css \
  -o internal/admin/static/css/tailwind.css \
  --minify

input.css lists the files scanned for class names, including the server-rendered partials under internal/admin/templates/. Classes used only there are dropped from the stylesheet if that list is incomplete, so add any new template directory to it. Commit the regenerated CSS alongside the markup change; a stale stylesheet is why several dashboard views once rendered unstyled.

Running Tests

go test ./...

# With the race detector (as CI runs it)
go test -race ./...

The default run is hermetic — nothing reaches the network.

Integration tests against a real Ollama backend

Some behaviour only fails against a real backend: a request the proxy considers well-formed that Ollama rejects, a response that is only incremental over a real connection, a usage counter that is only non-zero when a model actually ran. Those tests are opt-in, gated on TEST_OLLAMA_URL, and skip when it is unset:

TEST_OLLAMA_URL=http://your-ollama:11434 go test ./internal/proxy -run Integration -v

They cover native chat, OpenAI and Anthropic streaming, embeddings on both surfaces, token accounting, capability gating, tool calling, and that model-specific options reach the backend. GET /v1/models returning 500 for every caller was found this way.

They are written to be safe on a shared machine: they use the smallest models carrying the capability under test (smollm2:1.7b, bge-m3), never pull, skip if a model is absent, and unload each model with keep_alive: 0 afterwards — then assert via /api/ps that nothing stayed resident, so a regression in the unload path is reported rather than silently holding memory for Ollama's five-minute keep-alive window.

Redis-backed storage tests

Storage tests run against SQLite by default and additionally against Redis when TEST_REDIS_URL is set:

TEST_REDIS_URL=redis://localhost:6379 go test ./internal/storage

Adding API endpoints

All /v1 endpoints (OpenAI- and Anthropic-compatible) are registered on a single Echo group in proxy.RegisterAPIRoutes. Register new endpoints there.

Echo binds middleware to the group instance rather than to the path, so creating a second e.Group("/v1") produces an independent middleware chain and silently serves requests with no authentication, rate limiting, or endpoint-access checks. TestNoDuplicateV1Group guards against this.

Architecture

┌─────────────────────────────────────────────────────────┐
│                    HTTP/HTTPS Layer                      │
├─────────────────────────────────────────────────────────┤
│                                                        │
│  ┌──────────────────┐      ┌───────────────┐  │
│  │   Admin Handler    │      │  Auth Middleware │  │
│  └──────────────────┘      └───────────────┘  │
│                                                        │
│  ┌──────────────────┐      ┌───────────────┐  │
│  │  Proxy Handler    │      │ Rate Limiter   │  │
│  └──────────────────┘      └───────────────┘  │
│                                                        │
│  ┌──────────────────┐      ┌───────────────┐  │
│  │  Model Access    │      │ Accounting      │  │
│  │  Controller      │      │ Tracker        │  │
│  └──────────────────┘      └───────────────┘  │
│                                                        │
│  ┌──────────────────────────────────────────────┐   │
│  │          Storage Layer                 │   │
│  │  ┌─────────┐  ┌──────────┐      │   │
│  │  │ Redis   │  │ SQLite  │      │   │
│  │  └─────────┘  └──────────┘      │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
└──────────────────┐
   │  Ollama       │
   └──────────────┘

Security

Argon2id: Memory-hard password hashing resistant to GPU/ASIC attacks
JWT HS256: HMAC-SHA256 token signing
API Key Hashing: Tokens are hashed with Argon2id before storage
Token Prefixing: Only a short prefix is recorded, never the token itself
CORS allowlist: Cross-origin access is restricted to explicitly configured origins via CORS_ALLOWED_ORIGINS; the wildcard * is rejected at startup
Security headers: X-Content-Type-Options: nosniff, X-Frame-Options: DENY and Referrer-Policy: no-referrer on every response
Content Security Policy: the admin UI is served with a CSP that restricts scripts, styles and connections to the same origin and sets frame-ancestors 'none', so the dashboard cannot be framed or clickjacked
Connection timeouts: read, read-header and idle timeouts bound slow and idle connections without severing long-running streams
Rate Limiting: Sliding window rate limiting
Audit Logging: Comprehensive audit trail

Performance

Connection Pooling: HTTP client with connection pooling
High-Performance JSON: Uses Sonic for fast JSON parsing
Async Operations: Non-blocking audit logging
Graceful Shutdown: Clean connection draining

Troubleshooting

Redis Connection Failed

Symptom: "Warning: Redis connection failed, falling back to SQLite"
Solution: Check Redis URL, ensure Redis is running
Alternative: Configure only SQLite for single-instance deployments

High Memory Usage

Symptom: Out of memory errors
Solution: Reduce concurrent request limits, increase max memory

Rate Limiting Too Strict

Symptom: "429 Too Many Requests"
Solution: Increase rate limits in configuration

Debug Mode

export LOG_LEVEL=debug

Documentation

For detailed feature documentation, see FEATURES.md.

Acknowledgments

This project was vibe coded with AI assistance and human supervision.

License

MIT License - See LICENSE file for details.

Support

For issues and questions:

GitHub: https://github.com/anorebel/ollama-proxy/issues

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
cmd/proxy		cmd/proxy
docs/screenshots		docs/screenshots
e2e		e2e
internal		internal
scripts		scripts
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
.goreleaser.yml		.goreleaser.yml
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
FEATURES.md		FEATURES.md
LICENSE		LICENSE
README.md		README.md
config.yaml.example		config.yaml.example
docker-compose.host.yml		docker-compose.host.yml
docker-compose.yml		docker-compose.yml
go.mod		go.mod
go.sum		go.sum

Latest commit

History

Folders and files

Repository files navigation

Ollama Proxy

Features

Core Features

API Compatibility

Admin Dashboard

Infrastructure

Quick Start

Prerequisites

Installation

Configuration

Environment Variables

Required

Server

CORS

Write timeout and streaming

Ollama

Storage

Authentication

Admin password lifecycle

JWT

Rate Limits (Defaults)

Model Access

Logging

Admin Dashboard

Usage

Authentication

Using JWT (Bearer Token)

Using API Token

Roles and Permissions

API Endpoints

Ollama Native API

OpenAI Compatible API

Anthropic Compatible API

Health Checks

Admin API

Authentication

Token revocation

Refresh token rotation and reuse detection

User Management

Token Management

Statistics & Monitoring

Audit Logs

Admin Dashboard

Features

Access

Screenshots

Storage Architecture

Redis (Recommended for Production)

SQLite (Automatic Fallback)

Running and hosting

Docker Compose (recommended)

Docker without Compose

Manually, from source

Running as a systemd service

Putting it on the internet

Verifying a deployment

Kubernetes

Dokploy/Railway

Development

Running Locally

Building

Versioning

Dashboard assets

Running Tests

Integration tests against a real Ollama backend

Redis-backed storage tests

Adding API endpoints

Architecture

Security

Performance

Troubleshooting

Redis Connection Failed

High Memory Usage

Rate Limiting Too Strict

Debug Mode

Documentation