rag-service

A Go microservice that ingests documents, chunks them, embeds each chunk with OpenAI, stores the vectors in Postgres/pgvector, and serves semantic search plus a streaming RAG answer endpoint over Server-Sent Events. Every external service — OpenAI, pgvector, Redis — lives behind a port, so the whole app runs in tests with mock adapters and no network.

Pipeline

flowchart LR
    subgraph Ingest["POST /documents"]
        A[Document] --> B[Chunk<br/>token-aware + overlap]
        B --> C{Redis cache<br/>hash of model+text}
        C -- hit --> E[(pgvector)]
        C -- miss --> D[OpenAI embed]
        D --> E
        D -. store .-> C
    end

    subgraph Query["POST /search · POST /ask"]
        Q[Query] --> R[OpenAI embed]
        R --> S[pgvector<br/>cosine top-k]
        S --> T[Grounded prompt]
        T --> U[OpenAI stream]
        U -- SSE tokens --> Client
    end

Loading

Architecture

Hexagonal / dependency inversion. Dependencies point inward only:

• internal/domain — pure value objects (Document, Chunk) and the deterministic chunking function. Stdlib only; no I/O.
• internal/ports — the interfaces: inbound (IngestUseCase, SearchUseCase, AskUseCase) and outbound (Embedder, VectorStore, EmbeddingCache, LLMClient).
• internal/app — the use cases that orchestrate the ports. Imports domain and ports only.
• internal/adapters — concrete implementations: OpenAI, pgvector, Redis, an HTTP inbound adapter, and in-memory mocks for tests.
• cmd/api/main.go — the only place that constructs concrete adapters and wires them together.

Design decisions & trade-offs

Chunking — fixed-size, token-aware, with overlap. Size and overlap are counted in tokens, not characters, so chunks stay within the embedding model's token budget. Overlap carries trailing tokens of one chunk into the next so a sentence split across a boundary is still retrievable from both sides. The tokenizer is an injectable function (default: whitespace) — a real BPE tokenizer can be plugged in at an adapter without pulling any dependency into the pure domain. Chunk text is the exact original substring, so no formatting is lost.

Cache key = hash(model + normalized chunk text). Embedding is the expensive, rate-limited call, so identical chunks should never be embedded twice. The model name is part of the key, so switching embedding models invalidates the cache naturally instead of serving stale vectors of the wrong dimensionality. Queries are not cached — they are mostly unique and caching them rarely pays off.

Cosine distance + HNSW index. Embeddings are normalized, so cosine distance is the right similarity metric (<=> in pgvector). An ANN index is required — a sequential scan does not scale — and HNSW gives high recall and fast queries without tuning lists/probes the way ivfflat needs. The index is created in a migration, never left to a sequential scan.

SSE over WebSockets for /ask. The answer is a one-way token stream, so the full-duplex machinery of WebSockets buys nothing. SSE is plain HTTP, flushes per token, and rides through proxies. The handler passes the request context to the use case, so a client disconnect cancels the underlying LLM stream instead of burning tokens into a dead connection.

A port for every external service. OpenAI, pgvector, and Redis are swappable implementations behind interfaces, not hardcoded calls. The domain and use cases never import an SDK, a SQL driver, or a Redis client, which is what lets the entire pipeline run in tests against mock adapters with no network.

Running

Module path: github.com/orhan17/rag-service.

git clone https://github.com/orhan17/rag-service
cd rag-service
export OPENAI_API_KEY=sk-...
docker compose up

Compose starts the app, Postgres+pgvector, and Redis. Migrations (extension, table, HNSW index) run automatically on startup. Configuration is read once from the environment: OPENAI_API_KEY, DATABASE_DSN, REDIS_ADDR, EMBEDDING_MODEL, CHAT_MODEL, CHUNK_SIZE, CHUNK_OVERLAP, HTTP_ADDR.

make run    # go run ./cmd/api
make test   # go test ./... -race
make lint   # golangci-lint run

pgvector integration tests are tagged and opt-in:

DATABASE_DSN=postgres://rag:rag@localhost:5432/rag?sslmode=disable \
  go test -tags integration ./internal/adapters/outbound/pgvector/

Example

Ingest a document:

curl -s localhost:8080/documents \
  -H 'content-type: application/json' \
  -d '{"text":"pgvector adds vector similarity search to Postgres. It supports cosine, L2, and inner-product distance."}'
# {"document_id":"7f3a...","chunk_count":1}

Semantic search:

curl -s localhost:8080/search \
  -H 'content-type: application/json' \
  -d '{"query":"how does Postgres do similarity search?","top_k":3}'
# {"results":[{"document_id":"7f3a...","chunk_index":0,"text":"pgvector adds...","score":0.83}]}

Ask (streamed over SSE):

curl -N localhost:8080/ask \
  -H 'content-type: application/json' \
  -d '{"question":"What distances does pgvector support?","top_k":3}'
# data: pg
# data: vector
# data:  supports cosine, L2, and inner-product distance.
# data: [DONE]

API

Method	Path	Body	Response
POST	/documents	{id?, text}	201 {document_id, chunk_count}
POST	/search	{query, top_k?}	200 {results:[{…, score}]}
POST	/ask	{question, top_k?}	200 text/event-stream
GET	/healthz	—	200 ok