CIDX Server Deployment Guide

This guide covers deploying and configuring the CIDX server for multi-user team collaboration with server-side performance optimizations.

Note: This guide covers standalone (single-node) deployment with SQLite storage. For multi-node cluster deployment with PostgreSQL, see Cluster Architecture and Cluster Setup Guide.

Overview

CIDX server provides:

Multi-user semantic code search
Server-side HNSW index caching (100-1800x speedup)
OAuth 2.0 authentication
RESTful API and MCP protocol support
Per-repository isolation
Automatic cache management with TTL-based eviction

System Requirements

Minimum Requirements

Python 3.10 or later
4GB RAM (8GB+ recommended for large repositories)
10GB disk space (scales with repository size)
Linux/macOS/Windows (Linux recommended for production)

Network Requirements

Port 8090 for HTTP API and MCP protocol (configurable; MCP is served on the same port as the HTTP API)
Outbound HTTPS for VoyageAI API or Cohere API (embedding generation)

Installation

Option 1: pipx (Recommended)

# Install the latest release (check https://github.com/LightspeedDMS/code-indexer/releases for the current tag)
pipx install git+https://github.com/LightspeedDMS/code-indexer.git@<latest-tag>

# Verify installation
cidx --version

Option 2: pip with virtual environment

# Create virtual environment
python3 -m venv cidx-venv
source cidx-venv/bin/activate

# Install the latest release (check https://github.com/LightspeedDMS/code-indexer/releases for the current tag)
pip install git+https://github.com/LightspeedDMS/code-indexer.git@<latest-tag>

# Verify installation
cidx --version

Configuration

Environment Variables

Create /etc/cidx-server/config.env (or ~/.cidx-server/config.env for user-level):

# VoyageAI API Key (required for embedding generation)
VOYAGE_API_KEY=your-voyage-api-key-here

# Server Data Directory
CIDX_SERVER_DATA_DIR=/var/lib/cidx-server  # Default: ~/.cidx-server

# Server Port (HTTP API and MCP protocol share the same port)
CIDX_SERVER_PORT=8090     # Default: 8090

# Logging level
CIDX_LOG_LEVEL=INFO       # DEBUG, INFO, WARNING, ERROR
# Note: server application logs are stored in ~/.cidx-server/logs.db (SQLite), not a log file.

Configuration File

Alternative to environment variables, create ~/.cidx-server/config.json.

config.json is for bootstrap-only settings that must be available before the database is accessible. All runtime settings (cache TTL, embedding config, etc.) belong in the database and are managed via the Web UI Config Screen.

Bootstrap keys supported in config.json:

{
  "host": "0.0.0.0",
  "port": 8090,
  "server_dir": "/var/lib/cidx-server",
  "log_level": "INFO",
  "storage_mode": "sqlite"
}

For PostgreSQL cluster mode, add:

{
  "host": "0.0.0.0",
  "port": 8090,
  "server_dir": "/var/lib/cidx-server",
  "log_level": "INFO",
  "storage_mode": "postgres",
  "postgres_dsn": "postgresql://user:pass@db-host:5432/cidx"
}

HNSW Index Cache Configuration

The server includes automatic HNSW index caching for massive query performance improvements.

Cache Behavior

Without Cache (CLI Mode):

Each query loads HNSW index from disk
Typical HNSW lookup time: 200-400ms (with OS page cache)
Suitable for individual developers, single-user workflows

With Cache (Server Mode):

HNSW indexes cached in memory after first query
Cold HNSW lookup (cache miss): ~277ms
Warm HNSW lookup (cache hit): <1ms
HNSW lookup speedup: 100-1800x for repeated queries
Suitable for multi-user teams, high-query workloads

Note: the numbers above measure the in-process HNSW index lookup component only (Story #526 cache benchmark). End-to-end query latency also includes the embedding-provider round trip (50–300ms typical for VoyageAI / Cohere) on every query, plus HTTP and auth overhead in server / cluster modes. Treat these figures as cache-effectiveness signal, not user-observed query time.

Cache Configuration Options

TTL (Time-To-Live)

Configure how long HNSW indexes remain in cache via the Web UI Configuration Screen or config.json:

{
  "cache": {
    "ttl_minutes": 10.0
  }
}

Recommendations:

Small teams (1-5 users): 600 seconds (10 minutes)
Medium teams (5-20 users): 1800 seconds (30 minutes)
Large teams (20+ users): 3600 seconds (1 hour)
High-frequency queries: 7200 seconds (2 hours)

Memory Considerations:

Each cached HNSW index: 50-500MB (depends on repository size)
Monitor memory usage and adjust TTL accordingly
Longer TTL = better performance but higher memory usage

Per-Repository Isolation

Cache automatically isolates HNSW indexes by repository path:

Each repository has independent cache entry
No cross-repository cache contamination
Independent TTL tracking per repository

Example:

# Repository A and Repository B each have separate cache entries
# Query to Repo A doesn't affect Repo B's cache

Background Cleanup

Automatic background thread removes expired cache entries:

{
  "cache": {
    "enable_auto_cleanup": true,
    "cleanup_interval_seconds": 60  # Check every 60 seconds
  }
}

Monitoring Cache Performance

Cache Statistics Endpoint

Query real-time cache statistics:

curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:8090/cache/stats

Response:

{
  "hit_count": 1234,
  "miss_count": 56,
  "hit_ratio": 0.957,
  "cached_repositories": 12,
  "total_memory_mb": 480.5,
  "eviction_count": 3,
  "per_repository_stats": {
    "/path/to/repo1": {
      "access_count": 510,
      "last_accessed": "2025-11-30T12:34:56Z",
      "created_at": "2025-11-30T10:00:00Z",
      "ttl_remaining_seconds": 423
    }
  }
}

Key Metrics

hit_ratio: Percentage of queries served from cache (target: >80%)
hit_count: Number of cache hits (warm queries)
miss_count: Number of cache misses (cold queries)
cached_repositories: Number of repositories currently cached

Performance Expectations

Scenario	HNSW lookup	Cache Status	Notes
First query to repository	200-400ms	Miss	Loads from disk, benefits from OS cache
Subsequent queries (within TTL)	<1ms	Hit	Served from memory cache
Query after TTL expiration	200-400ms	Miss	Cache rebuild required
Concurrent queries to same repo	<1ms	Hit	Shared cache across users

HNSW lookup is the in-process index-search component of a query. End-to-end query response time is dominated by the embedding-provider round trip (50–300ms typical for VoyageAI / Cohere); the table above isolates the cache effect, not user-observed query speed.

Troubleshooting Cache Issues

Cache Not Activating

Symptom: All queries show cache miss behavior (slow)

Diagnosis:

# 1. Check cache configuration via Web UI Configuration Screen

# 2. Verify process is running
PID=$(pgrep -f "code_indexer.server.app")

# 3. Check cache stats endpoint
curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:8090/cache/stats
# If "cached_repositories" is always 0, cache is not working

Solution:

# 1. Configure cache TTL via Web UI Configuration Screen

# 2. Reload systemd configuration
sudo systemctl daemon-reload

# 3. Restart service
sudo systemctl restart cidx-server

# 4. Verify cache is configured
# Check Web UI Configuration Screen for cache settings

# 5. Test cache activation
# Make a query, then check /cache/stats for hit_count > 0

High Memory Usage

Symptom: Server consuming excessive memory

Diagnosis:

# Check number of active cache entries
curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:8090/cache/stats | jq '.cached_repositories'

# Monitor memory usage
ps aux | grep cidx-server

Solutions:

Reduce TTL to evict entries more frequently via Web UI Configuration Screen (set cache TTL to 300 seconds / 5 minutes)
Restart server to clear cache:
```
systemctl restart cidx-server
```
Limit number of repositories indexed on server

Low Hit Ratio

Symptom: hit_ratio below 50%

Diagnosis:

curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:8090/cache/stats | jq '.hit_ratio'

Possible Causes:

TTL too short (entries evicted before reuse)
Low query volume (few repeat queries)
Many different repositories queried (cache fragmentation)

Solutions:

Increase TTL for high-query-volume environments
Analyze query patterns to optimize cache usage
Consider increasing server memory to cache more repositories

Running the Server

Development Mode

# Start server in foreground (for testing)
python3 -m code_indexer.server.app

Production Deployment with systemd

RECOMMENDED: Use the provided deployment script and template:

cd deployment/
sudo ./deploy-server.sh YOUR_VOYAGE_API_KEY

This script automatically handles deployment configuration and verification.

Manual Installation (if automated script cannot be used):

Create /etc/systemd/system/cidx-server.service:

[Unit]
Description=CIDX Semantic Code Search Server
After=network.target

[Service]
Type=simple
User=cidx-server
Group=cidx-server
WorkingDirectory=/var/lib/cidx-server

# Load additional environment variables from file
EnvironmentFile=/etc/cidx-server/config.env

ExecStart=/usr/local/bin/python3 -m code_indexer.server.app

Restart=always
RestartSec=10

# Security hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/cidx-server /var/log/cidx-server

StandardOutput=append:/var/log/cidx-server/server.log
StandardError=append:/var/log/cidx-server/error.log

[Install]
WantedBy=multi-user.target

Start and enable service:

sudo systemctl daemon-reload
sudo systemctl enable cidx-server
sudo systemctl start cidx-server
sudo systemctl status cidx-server

Server Management

# Start server
sudo systemctl start cidx-server

# Stop server
sudo systemctl stop cidx-server

# Restart server (clears cache)
sudo systemctl restart cidx-server

# Check status
sudo systemctl status cidx-server

# View logs
sudo journalctl -u cidx-server -f

Security Considerations

Authentication

CIDX server uses OAuth 2.0 with JWT tokens:

# User authentication flow
1. User logs in via browser
2. Server issues JWT access token
3. Client includes token in API requests
4. Server validates token on each request

Network Security

Recommended production setup:

# Run server behind reverse proxy (nginx/haproxy)
# Terminate SSL at proxy level
# Forward to CIDX server on localhost:8090

# Example nginx config
server {
    listen 443 ssl;
    server_name cidx.example.com;

    ssl_certificate /etc/ssl/certs/cidx.crt;
    ssl_certificate_key /etc/ssl/private/cidx.key;

    location / {
        proxy_pass http://localhost:8090;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

File Permissions

# Create dedicated user for server
sudo useradd -r -s /bin/false cidx-server

# Set directory permissions
sudo mkdir -p /var/lib/cidx-server
sudo chown cidx-server:cidx-server /var/lib/cidx-server
sudo chmod 700 /var/lib/cidx-server

# Set log permissions
sudo mkdir -p /var/log/cidx-server
sudo chown cidx-server:cidx-server /var/log/cidx-server
sudo chmod 750 /var/log/cidx-server

Performance Tuning

Cache Optimization

For maximum cache benefit:

Monitor hit ratio: Target >80% for high-query environments
Adjust TTL: Balance memory usage vs. cache effectiveness
Pre-warm cache: Query common repositories during server startup
Memory allocation: Ensure sufficient RAM for expected cache size

Repository Management

Best practices:

Index frequently-queried repositories: Priority indexing for active projects
Schedule re-indexing: Off-hours re-indexing to minimize cache disruption
Repository isolation: Separate large monorepos to independent cache entries

Scaling Considerations

For large deployments:

Horizontal scaling: Run multiple server instances behind load balancer
Shared storage: Use shared filesystem for repository data
Cache distribution: Each server maintains independent cache (no shared cache needed)
Monitoring: Track per-server cache statistics and memory usage

Monitoring and Maintenance

Health Check Endpoint

# Check server health (authentication required)
curl -H "Authorization: Bearer YOUR_TOKEN" http://localhost:8090/health

Example response:

{
  "status": "healthy",
  "message": "Server is running normally",
  "uptime": 3600.5,
  "active_jobs": 2,
  "job_queue": {
    "active_jobs": 2,
    "pending_jobs": 0,
    "failed_jobs": 0
  },
  "started_at": "2025-11-30T10:00:00Z",
  "maintenance_mode": false,
  "version": "10.141.0"
}

Log Analysis

Server application logs are stored in ~/.cidx-server/logs.db (SQLite). Monitor logs with:

# View recent error logs
sqlite3 ~/.cidx-server/logs.db "SELECT timestamp, level, message FROM logs WHERE level IN ('ERROR','WARNING') ORDER BY id DESC LIMIT 50"

# Search for cache-related logs
sqlite3 ~/.cidx-server/logs.db "SELECT timestamp, message FROM logs WHERE message LIKE '%cache%' ORDER BY id DESC LIMIT 50"

# Count recent errors
sqlite3 ~/.cidx-server/logs.db "SELECT COUNT(*) FROM logs WHERE level='ERROR'"

Backup and Recovery

Critical data to backup:

Repository indexes: ~/.cidx-server/golden-repos/
Configuration: ~/.cidx-server/config.json
Database: ~/.cidx-server/cidx_server.db (users, settings, job history)

Cache data (HNSW indexes) can be rebuilt, no backup needed.

Troubleshooting

Common Issues

Server Won't Start

# Check logs
sudo journalctl -u cidx-server -n 50

# Common causes:
# - Missing VOYAGE_API_KEY
# - Port already in use
# - Permission issues

Slow Query Performance

# Check if cache is working
curl http://localhost:8090/cache/stats

# If hit_ratio is low:
# 1. Increase TTL
# 3. Monitor memory usage

Memory Issues

# Check memory usage
free -h
ps aux | grep cidx-server

# If high memory usage:
# 1. Reduce TTL
# 2. Reduce number of cached repositories
# 3. Restart server to clear cache

Post-Generation Verification Pass

The cidx-server supports an optional post-generation verification pass that re-reads generated dependency-map artifacts and repo descriptions against their actual source code and produces a corrected version with evidence citations.

Default: disabled. Enable via the Admin Web UI Config Screen, Claude CLI Integration section, "Post-generation verification pass (fact-check)" toggle.

Latency cost

Verification invokes Claude CLI once per generated document. Expected cost:

Per-invocation: approximately 20-45 seconds (within the configured timeout).
Per refresh cycle: adds verification time multiplied by the number of generated artifacts (per-domain for dependency maps, once for descriptions).

Before enabling in production, run the baseline measurement procedure described in the story: measure the fabrication rate on a fixed 3-repo corpus both with and without verification to confirm the corrections justify the cost.

Configuration fields

dep_map_fact_check_enabled (bool, default false): master toggle.
fact_check_timeout_seconds (int, default 600): per-invocation timeout. Accepts 60-3600.

Known limitations

Changes to max_concurrent_claude_cli require a service restart to take effect. The shared subprocess semaphore that bounds total Claude CLI concurrency is initialized once at process start and does not resize at runtime. Restart the cidx-server systemd unit after changing this field.
Verification is a heuristic second pass using the same model class as the generator. It reduces hallucinated claims but does not guarantee factual correctness.
Source-code bugs found by verification are NOT automatically corrected in deployed code; the feature only corrects the generated markdown artifacts. Code-level bugs are filed as GitHub issues per the existing bug-report flow.

cidx-meta backup to remote git

The cidx-meta directory holds description files and metadata for every registered golden repository. You can configure the server to keep a continuous git backup of this directory in a remote repository so that the metadata can be recovered after data loss.

Configuration

Navigate to the Web UI Config Screen: /admin/config.
Open the "cidx-meta backup" section.
Set "Enabled" to true.
Enter the remote URL in the "Remote URL" field (see URL formats below).
Click Save.

URL formats accepted:

git@github.com:org/repo.git — SSH (requires SSH key, see below)
https://github.com/org/repo.git — HTTPS
file:///path/to/bare.git — local bare repository (useful for testing)

SSH key prerequisite

For git@host: URLs, the server must have an SSH key registered for that hostname before saving the configuration. To add a key:

Go to the SSH Keys page in the admin UI.
Create or import a key and assign it to the target hostname.
Return to the Config Screen and save the cidx-meta backup settings.

file:// and https:// URLs skip the SSH key check.

Bootstrap behavior

The first time a remote URL is saved (or when the URL changes), the server runs CidxMetaBackupBootstrap.bootstrap():

If .git/ does not exist in the cidx-meta directory, git is initialized, all current files are committed, and a force-push is made to the remote.
If .git/ exists and the remote URL is unchanged, the call is a no-op.
If the URL changed, git remote set-url is issued and a force-push is made.

Bootstrap is idempotent: running it multiple times with the same URL is safe.

Sync on refresh

Every time the cidx-meta refresh job runs and backup is enabled:

MetaDirectoryUpdater creates or removes description files on disk.
CidxMetaBackupSync stages all changes (git add -A), commits, fetches from origin, rebases local commits on top of remote changes, and pushes.
If the rebase produces conflicts, Claude CLI is invoked to resolve them (600 s timeout; SIGTERM then SIGKILL after 30 s if exceeded).
If conflict resolution fails, git rebase --abort reverts the repo and the refresh job is marked FAILED with the resolution error as the reason.
If the push itself fails (e.g. network error), the job is also marked FAILED but indexing still runs (deferred-failure pattern).

Recovery procedure

To recover cidx-meta from the remote backup:

Stop the cidx-server service.
Remove or rename the existing cidx-meta directory: mv ~/.cidx-server/data/golden-repos/cidx-meta ~/.cidx-server/data/golden-repos/cidx-meta.bak
Clone the remote backup into that location: git clone <remote-url> ~/.cidx-server/data/golden-repos/cidx-meta
Restart cidx-server.
Trigger a manual refresh for cidx-meta-global from the admin UI to re-index the recovered metadata.

Operational notes

URL change idempotency: changing the URL in the Web UI triggers CidxMetaBackupBootstrap.bootstrap() at Save time. The next scheduled refresh cycle also runs bootstrap at the start so URL changes applied via direct DB edits are picked up automatically.
Deferred-failure pattern: indexing always runs after sync regardless of whether push succeeded. A failed push surfaces as a FAILED job with the push error included in the failure reason.
Claude conflict resolver timeout: 600 s per invocation. On timeout, SIGTERM is sent first; SIGKILL follows after a 30 s grace period.
The mutable base path for cidx-meta is always <server_data_dir>/data/golden-repos/cidx-meta/. Git operations NEVER run inside .versioned/ snapshot directories.

Additional Resources

Main README - Project overview and features
GitHub Repository - Source code and issues

Uh oh!

FilesExpand file tree

server-deployment.md

Latest commit

History

server-deployment.md

File metadata and controls

CIDX Server Deployment Guide

Overview

System Requirements

Minimum Requirements

Network Requirements

Installation

Option 1: pipx (Recommended)

Option 2: pip with virtual environment

Configuration

Environment Variables

Configuration File

HNSW Index Cache Configuration

Cache Behavior

Cache Configuration Options

TTL (Time-To-Live)

Per-Repository Isolation

Background Cleanup

Monitoring Cache Performance

Cache Statistics Endpoint

Key Metrics

Performance Expectations

Troubleshooting Cache Issues

Cache Not Activating

High Memory Usage

Low Hit Ratio

Running the Server

Development Mode

Production Deployment with systemd

Server Management

Security Considerations

Authentication

Network Security

File Permissions

Performance Tuning

Cache Optimization

Repository Management

Scaling Considerations

Monitoring and Maintenance

Health Check Endpoint

Log Analysis

Backup and Recovery

Troubleshooting

Common Issues

Server Won't Start

Slow Query Performance

Memory Issues

Post-Generation Verification Pass

Latency cost

Configuration fields

Known limitations

cidx-meta backup to remote git

Configuration

SSH key prerequisite

Bootstrap behavior

Sync on refresh

Recovery procedure

Operational notes

Additional Resources