ContextOS

Context engineering and retrieval for AI coding agents. ContextOS indexes code repositories and their documentation, traces how files and symbols depend on each other, and assembles the smallest, cited, trustworthy context package a coding agent needs for a specific task. Agents consume it through an MCP server; people inspect and tune it in a web studio.

17-second walkthrough (MP4 version).

Core question: how can an autonomous agent receive the smallest, most relevant, trustworthy and sufficiently complete context to solve a task accurately?

What works today

Area	Status
Ingestion	Local folders (restricted to authorised roots) and GitHub repos; .gitignore + exclusion rules; secret-bearing files skipped and inline credentials redacted; commit tracking; content-hash incremental re-indexing
Code intelligence	Tree-sitter for Python, TypeScript/TSX and JavaScript: classes, functions, methods, interfaces, types, imports, re-exports, call sites, base classes
Dependency graph	Imports, calls, inheritance, test→code and doc→symbol edges, each labelled validated, heuristic or syntactic
Retrieval	A keyword (Postgres FTS) · B vector (pgvector) · C hybrid (RRF) · D graph-enhanced · E task-aware (LangGraph) — each benchmarked independently
Context packaging	Token budgets, per-task quotas, de-duplication, non-destructive compression, provenance, uncertainty signals, explicit exclusion reasons
Agent memory	Proposed → verified review flow, provenance, staleness on file change, expiry for working memory
MCP server	8 typed tools over stdio, repository allow-list; verified with the official MCP Python client (9/9 calls)
Evaluation	30 hand-written, repository-grounded queries over two pinned open-source repos; Recall@k, Precision@k, MRR, nDCG, package recall, citation correctness, tokens, latency
Web app	Next.js studio: dashboard, code explorer (Monaco), knowledge graph (React Flow), retrieval playground, context studio, evaluation lab, agent integration
Tests	26 backend tests (unit + integration against real Postgres/pgvector and real git repos, incl. adversarial cases) · 8 Playwright end-to-end tests

Screenshots

All screenshots show the production build running against the real backend, with itsdangerous and ky indexed.

Repositories dashboard

The start page. The headline sits over the atrium photograph; the panel on the right shows how far the indexed text (285k tokens across 146 files) is narrowed for one task (about 3.9k tokens). Typing a task into the field and choosing Build context opens the Context studio with that task.

Each indexed repository with its pinned commit, language mix and file, symbol and chunk counts, plus Explore code, Re-index and run details. The right column imports a GitHub repository or a local folder and shows recent activity and the latest benchmark scores.

Code explorer

The file tree (colour-coded by code, test, doc and config), the source in a read-only Monaco viewer with the cited lines highlighted, and an inspector with the symbols Tree-sitter found, each file's links and the chunks it was split into.

Knowledge graph

Files as nodes, sized by how many links they have. Solid lines are links resolved against the file tree; dashed lines were matched by name. Filters above the graph switch link kinds and confidence levels on and off; selecting a file lists its links and can focus the graph on its neighbourhood.

Retrieval playground

One question, one strategy (A–E). Every result shows its citation, why it was retrieved and a breakdown of the scores behind its rank; the side panel shows latency, the detected task type and the step-by-step workflow trace. Compare all five puts the strategies side by side.

Context studio

A task, a token budget, an optional task type and a strategy go in; a package comes out. The summary shows tokens used against the budget by section, uncertainty notes and the relationships between included files; the right column shows how the workflow built it.

Everything included, grouped by section, with Pin and Exclude for the next build; items that did not fit are listed under Left out with the reason. The memory panel holds verified repository facts and the review queue for facts a model proposed.

Evaluation laboratory

Mean metrics for the five strategies on the 30-query benchmark, switchable between all queries, the 20 held-out queries and the 10 tuning queries. The best value in each column is highlighted, and the embedder used for the run is stated.

nDCG@10 per question category, and every query where a simpler strategy beat the task-aware one, with the files it should have found and what it ranked first instead.

Agent integration

The MCP client configuration to copy, which repositories are exposed, and a one-click compatibility test that starts the MCP server the way an agent host does and calls every tool.

On a phone

Dashboard	Retrieval	Context studio

The layout collapses to one column at phone width, the navigation moves into a menu, and nothing scrolls sideways.

Quick start (macOS)

Requirements: Python 3.11+, Node 20+, PostgreSQL 16 with pgvector, git.

# 1. Database (Homebrew) — or skip and run `docker compose up -d db`
brew install postgresql@16 pgvector
brew services start postgresql@16
createdb contextos

# 2. Backend
cd backend
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"                 # add ".[local-embeddings]" for a neural embedder (see below)
cp ../.env.example .env                 # then edit CONTEXTOS_ALLOWED_ROOTS and the database URL
uvicorn contextos.api.main:app --port 8000

# 3. Frontend (new terminal)
cd frontend
npm install
npm run dev                              # http://localhost:3000

# 4. Optional: index the benchmark repositories and run the evaluation
cd backend && python -m contextos.eval.runner --setup

Or use make db api web, see the Makefile. With Docker: docker compose up --build starts Postgres+pgvector, the API and the web app.

Embeddings

CONTEXTOS_EMBEDDING_PROVIDER=auto uses sentence-transformers (BAAI/bge-small-en-v1.5) when the local-embeddings extra is installed and the model can be downloaded, and otherwise falls back to the built-in hash-ngram embedder. Every index is versioned by embedding provider, model, dimensions and chunking strategy; queries are never compared against vectors from a different version, and the API tells you to re-index if the active embedder changed. An OpenAI-compatible /embeddings endpoint (NVIDIA NIM, Gemini, Ollama) is also supported.

Language model (optional)

Nothing requires a model. When CONTEXTOS_LLM_PROVIDER is anthropic or openai-compatible, the task classifier and query planner can use it (opt-in per request), and python -m contextos.eval.runner --downstream runs the downstream evaluation.

Benchmark results

30 queries, two repositories pinned to commits (pallets/itsdangerous Python, sindresorhus/ky TypeScript), 4 000-token budget, hash-ngram embeddings (HuggingFace was unreachable from the build sandbox), lexical reranker. Full method and per-query results: docs/evaluation.md.

Strategy	Recall@5	Recall@10	MRR	nDCG@10	Package file recall	Citations correct	p50 latency
A · Keyword	0.68	0.76	0.60	0.59	0.80	99.4%	10 ms
B · Vector	0.28	0.42	0.22	0.25	0.58	100%	4 ms
C · Hybrid	0.56	0.79	0.50	0.53	0.79	100%	16 ms
D · Graph	0.56	0.79	0.50	0.53	0.79	100%	23 ms
E · Task-aware	0.76	0.86	0.72	0.70	0.88	99.8%	89 ms

On the 20 held-out queries (never used while tuning) task-aware reaches nDCG@10 0.70 vs 0.49 for keyword. What the numbers also say, plainly:

• Keyword search beats hybrid here, because the hash-ngram "vector" signal is a noisy duplicate of the lexical one. That conclusion is specific to this embedder; re-run with a neural model before generalising it.
• Graph expansion (D) did not change file-level ranking at all in this scoring scheme — neighbours enter below the top 20. Dependency expansion only pays off inside E, where the reranker can promote what it adds (dependency analysis: 0.80 vs 0.46).
• A simpler baseline beats task-aware on 9 of 30 queries by more than 0.05 nDCG (keyword on 6). Usually the reranker lifted a neighbouring file such as exc.py or __init__.py above the file that answers. See failure cases in the evaluation doc.
• No downstream (agent task-success) numbers are claimed: that harness exists but was not run, because no model was available in the build environment.

Using it from an agent (MCP)

{
  "mcpServers": {
    "contextos": {
      "command": "/path/to/ContextOS/backend/.venv/bin/python",
      "args": ["-m", "contextos.mcp_server"],
      "env": {
        "CONTEXTOS_DATABASE_URL": "postgresql://you@localhost:5432/contextos",
        "CONTEXTOS_MCP_ALLOWED_REPOS": "ky,itsdangerous"
      }
    }
  }
}

Tools: list_repositories, search_repository, find_symbol, explain_dependency, get_relevant_tests, build_context_package, inspect_repository_structure, retrieve_architecture_notes. Run python -m contextos.mcp_selftest <repo> (or the button on the Agents page) to check a machine. Details: docs/mcp.md.

Documentation

• Architecture — components, data flow, database schema, retrieval specs, LangGraph state
• Evaluation — methodology, results, failure cases, reproduction
• Security model — untrusted evidence, secrets, access control, memory poisoning
• MCP — tools, schemas, tested compatibility
• Decision records · Limitations and future work

Project layout

backend/contextos/
  ingest/        discovery, Tree-sitter parsing, chunking, graph building, indexer
  retrieval/     baselines (A–C), graph expansion (D), task-aware LangGraph workflow (E), reranker
  context/       packager: budgets, compression, uncertainty, rendering
  api/           FastAPI application
  eval/          benchmark.json and runner
  mcp_server.py  MCP server · mcp_selftest.py  stdio client test
  memory.py  security.py  embeddings.py  graph.py  llm.py
backend/tests/   unit and integration tests (real Postgres, real git)
frontend/src/    Next.js app router pages and components
frontend/e2e/    Playwright tests
docs/            architecture, evaluation results (JSON), screenshots, ADRs

Testing

cd backend && pytest                       # needs a Postgres with pgvector; creates contextos_test
cd frontend && npx playwright test         # needs API + web running and an indexed repository

Design

The interface uses the atrium photograph as a full-screen backdrop, graded with an overlay so white type stays readable, plus a light bloom over the skylight and slow dust that drifts into the aperture — a picture of context being narrowed. Typography is Fraunces for headings, Hanken Grotesk for interface text and JetBrains Mono for code. Motion is limited to one load sequence, page transitions, and responses to actions; it switches off entirely with prefers-reduced-motion.