Ask the Docs — Technical Documentation RAG

A runnable Retrieval-Augmented Generation use-case over a folder of technical documentation, built with LangChain.js (LCEL), Chroma, OpenRouter embeddings (with a local Transformers.js option) and an OpenRouter chat model.

Ask questions about a corpus/ of Markdown docs and get grounded answers with source citations — switchable between hosted and fully local embeddings.

Stack

Concern	Choice
Language / runtime	TypeScript (strict, ESM), Node ≥ 22
Framework	LangChain.js LCEL
Embeddings	OpenRouter (liquid/lfm-2.5-embedding-350m:free, 1024-dim) by default; local Transformers.js (Xenova/all-MiniLM-L6-v2) via EMBEDDINGS_PROVIDER=local
Vector store	Chroma (@langchain/community), persisted to .chroma/
Chat model	OpenRouter via @langchain/openrouter

Architecture lives in docs/design.md; the PNGs below are rendered from docs/diagrams/*.js via pnpm diagrams.

Architecture at a glance

Components

Ingest pipeline

Query pipeline

Data model

Quickstart

pnpm install
cp .env.example .env            # add OPENROUTER_API_KEY

# 1. start the Chroma server (separate terminal)
pnpm engine:up

# 2. index the corpus
pnpm ingest --reset

# 3. ask a question
pnpm ask "What is the rate limit on the Free plan?"

ask prints the answer followed by a Sources: list (path#chunkIndex).

Commands

Command	Purpose
pnpm engine:up	Start the local Chroma server (persists to ./.chroma)
pnpm ingest [--reset]	Load corpus/ → split → embed → store
pnpm ask "<question>"	Retrieve + generate a cited answer
pnpm diagrams	Render docs/diagrams/*.js → PNG
pnpm typecheck	tsc --noEmit
pnpm test	Unit tests (no network)
pnpm test:integration	Chroma round-trip (needs pnpm engine:up)
pnpm test:model	Local embedding model check (downloads model)
pnpm verify	typecheck && test

Configuration

See .env.example. Key variables: OPENROUTER_API_KEY, CHAT_MODEL, EMBEDDING_MODEL, CHROMA_URL, CHROMA_COLLECTION, CORPUS_DIR, RETRIEVER_K, CHUNK_SIZE, CHUNK_OVERLAP.

Free models: if your OpenRouter key has no credits, set CHAT_MODEL to a :free variant (e.g. liquid/lfm-2.5-2.6b:free). The repo's local .env does this by default, along with the :free embeddings model.

Note: EMBEDDING_MODEL (and EMBEDDINGS_PROVIDER) must match between ingest and query. Changing either requires re-running pnpm ingest --reset (the vector dimension changes).

Replacing the corpus

Drop Markdown/.txt files under corpus/ (recursively) and re-run pnpm ingest --reset. Chunk ids are sha256(sourcePath#chunkIndex), so re-ingesting is safe.

Troubleshooting

• Chroma unreachable at ... — the Chroma server isn't running: pnpm engine:up.
• collection "docs" is empty — run pnpm ingest --reset first.
• OpenRouter key limit — ask requires a key that can call the chosen model. Paid models need credits; set CHAT_MODEL/EMBEDDING_MODEL to :free variants otherwise. Free models are per-minute rate limited (the smoke eval paces calls; tune with EVAL_DELAY_MS).
• Model ... does not exist — provider/model mismatch: EMBEDDINGS_PROVIDER=openrouter needs an OpenRouter model id, local needs a Transformers.js model id.
• The 'path' argument is deprecated — a deprecation notice emitted by LangChain's Chroma integration; harmless.

See also

• docs/intent.md — problem, scope, constraints
• docs/design.md — architecture, interfaces, ADRs
• docs/plan.md — implementation plan and traceability
• docs/testing.md — test strategy and metrics