Skip to content

Latest commit

 

History

652 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

free-context-hub

Persistent memory and guardrails for AI coding agents — so they never start from zero again.

MIT License Node 20+ TypeScript MCP Compatible Self-Hosted


Every AI coding session starts with amnesia. Your agent forgets the architectural decisions you explained yesterday, the workaround you discovered last week, and the deployment rule your team agreed on last month. You re-explain, re-discover, re-enforce — over and over.

free-context-hub is a self-hosted, open-source MCP server that gives AI assistants (Claude Code, Cursor, etc.) persistent knowledge across sessions and agents — decisions, preferences, workarounds, and guardrails that survive after a conversation ends.

It also provides a web GUI for humans to review, approve, and refine AI-generated knowledge — bridging AI-to-AI and AI-to-Human collaboration.

Inspired by ContextStream. Free, local, for small teams.


What this actually is

The visible surface is memory, lessons, and guardrails for AI coding agents. The deeper arc — see ROADMAP.md for the full picture — is a five-phase research program building the MCP-served, software-substrate implementation of the Dead Light Framework (DLF): a governance methodology for hybrid human + AI organizations.

Phase 1 ─── Phase 2 ─── Phase 3 ─── Phase 4 ─── Phase 5
Memory      Eval +      Governance  Benchmark   Runtime
+ Knowledge RAG Quality Primitives  Governance  Enforcement
✅ shipped   🔄 active   🔄 active   ⏳ planned   💡 research

Phase 1 is the core feature surface most users interact with today. Phases 2 and 3 are being built in parallel on separate branches — generation-quality evaluation (Phase 2) and the coordination / governance primitives drawn from DLF (Phase 3: motions, proxies, scope, approval, dispute, multi-tier routing). Phase 4 will benchmark Phase 3 across diverse governance scenarios. Phase 5 — pre-action runtime enforcement and isolation environments for dangerous tasks — is still in literature review.

The differentiated wedge: persistent semantic memory of decisions + DLF-derived collective-decision primitives + tenant-scope isolation under one MIT roof, served via MCP. Several expansion directions are scoped for future phases — see ROADMAP.md.


Screenshots

20 pages, 70+ REST endpoints, full human-in-the-loop workflow.

Dashboard — Knowledge health, insights, activity feed

Dashboard

AI Chat — Markdown rendering, tool calls, conversation history

Chat

Lessons — Search, filter, import/export, feedback signals

Lessons

Review Inbox — Approve AI-generated lessons, trust levels

Review Inbox

Lesson Detail — Rich editor, comments, version history

Lesson Detail

Analytics — Retrieval trends, dead knowledge, agent activity

Analytics

Documents — Upload, link, generate lessons from docs

Documents

Guardrails — Test presets, "What Would Block?" simulate mode

Guardrails


Tech Stack

Layer Technology
Backend Node.js + TypeScript, MCP protocol server
Frontend Next.js 16, React 19, Tailwind CSS 4
Primary Database PostgreSQL 16 + pgvector (data + embeddings)
Knowledge Graph Neo4j 5 (optional — symbol extraction via ts-morph)
Job Queue RabbitMQ (background workers for git ingestion, reflection)
Cache Redis (tiered search caching)
AI Integration MCP (Model Context Protocol) — open standard for agent tool use
Embeddings Any OpenAI-compatible endpoint (LM Studio, Ollama, etc.)
Deployment Docker Compose — single-node, self-hosted, zero cloud dependency

Architecture

 AI Agents (Claude Code, Cursor, etc.)
        │ MCP :3000
        ▼
 ┌─────────────────────────────────┐
 │       ContextHub Backend        │
 │  MCP Server · REST API · Chat   │
 │  Background Worker (jobs)       │
 │─────────────────────────────────│
 │        Services Layer           │
 │  lessons · guardrails · search  │
 │  git · jobs · docs · audit      │
 │─────────────────────────────────│
 │  Postgres   │ Neo4j  (optional) │
 │  + pgvector │ Redis  (optional) │
 │             │ Rabbit (optional) │
 └──────────────┬──────────────────┘
                │ REST :3001
                ▼
 ┌─────────────────────────────────┐
 │      GUI (Next.js :3002)        │
 │  20 pages · AI chat · Review    │
 │  Inbox · Analytics · Agents     │
 └─────────────────────────────────┘
                │
           Human (browser)

Core Features

1. Persistent Lessons & Knowledge

The main value proposition. Store and retrieve team knowledge that persists across sessions and agents.

  • add_lesson — Capture decisions, preferences, workarounds, guardrails
  • search_lessons — Semantic search across all stored knowledge
  • list_lessons — Browse with filters (type, tags, status)
  • update_lesson_status — Lifecycle management (draft → active → superseded → archived)
  • reflect — LLM-synthesized answers from multiple lessons (optional, requires chat model)

Example workflow:

Agent A (Monday):   add_lesson("We use JWT not sessions — legal requires stateless auth")
Agent B (Thursday): search_lessons("authentication approach") → gets the decision instantly
Agent C (Next month): Doesn't waste time debating sessions vs JWT

2. Guardrails

Prevent repeated mistakes by enforcing team rules before risky actions.

  • check_guardrails — Pre-action safety verification (git push, deploy, migrations)
  • Guardrails are derived from lessons with lesson_type: "guardrail"
  • Returns pass/fail + prompt for user confirmation when blocked

3. Session Bootstrap

Quick onboarding for new agent sessions.

  • get_context — Bootstrap with project state + suggested next calls
  • get_project_summary — Full project briefing in one read
  • help — Tool discovery and sample workflows

4. Human-in-the-Loop GUI

A full web dashboard where humans review, approve, and refine AI-generated knowledge.

  • Review Inbox — Batch approve/reject AI-generated lessons with trust levels
  • Rich Content Editor — Markdown toolbar, live preview, keyboard shortcuts (Ctrl+B/I)
  • AI Editor — Clarify, simplify, expand lessons with diff view (accept/reject)
  • Comments & Feedback — Threaded comments, thumbs up/down, bookmarks
  • Activity & Analytics — Timeline, retrieval trends, dead knowledge detection
  • Documents — Upload reference docs, generate lessons from them
  • Global Search — Cmd+K across all knowledge

5. Access Control & Agent Audit

Secure multi-agent environments with role-based access and full audit trails.

  • API Keys with Roles — Admin, editor, viewer permissions via SHA-256 hashed keys
  • Agent Audit Trail — Unified timeline of all agent actions (guardrail checks, lessons created)
  • Custom Lesson Types — Define project-specific types beyond the defaults
  • Feature Toggles — Enable/disable capabilities per project

6. Code Search (Supplementary)

Semantic code search assists agents in finding relevant code. Supplementary — agents already have built-in Grep/Glob.

  • search_code_tiered — Multi-tier search with auto-selected profiles:
    • code-search: ripgrep → symbol → FTS → semantic
    • relationship: convention paths → KG imports → filtered ripgrep
    • semantic-first: vector similarity → FTS (for docs/scripts)

7. Git Intelligence (Supplementary)

Auto-collect insights from commit history.

  • ingest_git_history / suggest_lessons_from_commits — Draft lessons from git
  • analyze_commit_impact — Commit impact over symbol/lesson graph

8. Knowledge Graph (Optional)

Symbol-level code structure for advanced queries. Requires Neo4j.

  • search_symbols / get_symbol_neighbors / trace_dependency_path
  • get_lesson_impact — Which code does a lesson affect?

Quickstart

Prerequisites

  • Node.js 20+ and npm
  • Docker (for PostgreSQL + pgvector)
  • Embeddings server — LM Studio or any OpenAI-compatible endpoint

Setup

# 1. Clone and install
git clone https://github.com/your-username/free-context-hub.git
cd free-context-hub
npm install

# 2. Configure environment
cp .env.example .env
# Edit .env — set DATABASE_URL and EMBEDDINGS_BASE_URL

# 3. Start infrastructure
docker compose up -d

# 4. Launch the backend (MCP + API)
npm run dev

# 5. Verify everything works
npm run smoke-test

# 6. (Optional) Launch the GUI
cd gui && npm install && npm run dev

Connect Your AI Tool

Add the MCP server URL in your tool's settings:

http://localhost:3000/mcp
Service URL
MCP Server http://localhost:3000/mcp
REST API http://localhost:3001
Web GUI http://localhost:3002

Detailed setup: docs/QUICKSTART.md


Self-Hosted Models

ContextHub uses OpenAI-compatible APIs. Run locally with LM Studio or any compatible server.

Recommended Combo (benchmarked)

We tested 8 embedding models and 8 reranker models (embedding/reranker benchmark), plus 6 distillation models (distillation benchmark). Best combo:

Role Model Config
Embeddings qwen3-embedding-0.6b EMBEDDINGS_DIM=1024 — best accuracy (18/18 pass, avg 0.652)
Reranker qwen3-4b-instruct-ranker RERANK_MODEL=qwen3-4b-instruct-ranker — +9% accuracy at 180 lessons
Distillation ibm/granite-4-h-tiny DISTILLATION_ENABLED=true — non-reasoning, ~100K context; reflect, compress, summarize

Alternative Models

Embedding (8 tested, lesson search accuracy):

Model Dims Pass Rate Avg Score Notes
qwen3-embedding-0.6b 1024 18/18 0.652 Recommended
bge-m3 1024 18/18 0.575 Fast, solid all-rounder
mxbai-embed-large-v1 1024 17/18 0.648 Close but 1 failure

Reranker (8 tested, 180 lessons / 33 queries):

Model Type Pass Rate Latency Notes
qwen3-4b-instruct-ranker generative 85% 1.8s Recommended — no thinking overhead
qwen.qwen3-reranker-4b generative 85% 1.9s Thinking mode, same accuracy
rank_zephyr_7b generative 82% ~2s RankGPT format
(no rerank) — 76% 99ms Baseline

Distillation (6 tested — chat model for reflect/compress/summarize; full benchmark):

Model Reasoning Speed Notes
ibm/granite-4-h-tiny No in-stack add_lesson ~5.6s Recommended — 0 reasoning tokens, ~100K context (max 1M)
microsoft/phi-4 No in-stack add_lesson ~3.7s Faster, but only a 16K context window
mistral-nemo-instruct-2407 No non-reasoning Viable alternative, 128K context
qwen3.5-9b Yes — Reasoning cannot be disabled via the API
deepseek-r1-distill-qwen-32b Yes ~5 tok/s Non-thinking is possible (32B) but too slow
nvidia/nemotron-3-nano Yes ~25-35s/call Previous default — chain-of-thought overhead

Note: Code search uses ripgrep/FTS (deterministic), not embeddings. Cross-encoder rerankers (bge-reranker, gte-reranker) don't work via LM Studio — they need a dedicated /v1/rerank API.


Roadmap

For the strategic five-phase arc (Memory → Eval → Governance → Benchmark → Runtime Enforcement) and its DLF lineage, see ROADMAP.md. The list below is the sprint-level history mapped to the strategic phases.

Completed:

  • Phase 1-2: Core MVP — Lessons, Search, Guardrails
  • Phase 3: Knowledge Distillation & Reflection
  • Phase 4: Knowledge Graph (Neo4j, symbol-level)
  • Phase 5: Git Intelligence & Automation
  • Phase 6: Retrieval Quality Tuning & Tiered Search
  • Phase 7: Interactive GUI & Human-in-the-Loop
    • 23 pages, 105 REST endpoints, 45 MCP tools, 41 migrations
    • Dashboard, Chat, Lessons, Review Inbox, Analytics, Documents, Guardrails, Settings, and more
  • Phase 8: Advanced HITL
    • Access control (API keys with roles), custom lesson types, rich content editor, agent audit trail, feature toggles
  • Phase 8D/E: Polish & Testing
    • Feature toggles BE, role enforcement middleware, layout fixes (viewport scroll, page sizing)
    • E2E test suite: 198 tests (API smoke 75, GUI smoke 23, MCP smoke 36, API scenarios 34, GUI scenarios 21, Agent visual 9)
  • Phase 9: Multi-Project UX Redesign
    • "All Projects" first-class mode with project selector V2 (multi-select, checkboxes)
    • ProjectBadge on all 23 pages, cross-project data on 8 pages (Dashboard, Lessons, Guardrails, Review, Analytics, Jobs, Activity, Agents)
    • Per-project guards on 4 pages (Graph Explorer, Code Search, Sources, Settings)
    • 6 backend services extended with project_ids[], 9 *Multi API methods
    • 11 sprints, 26 commits, 41 files changed
  • Phase 10: Multi-Format Extraction Pipeline
    • Fast text (pdf-parse + mammoth), Quality text (pdftotext + pandoc), Vision (LLM via OpenAI-compatible API with per-page async jobs, progress reporting, cancel)
    • Chunking + pgvector embeddings with hierarchical/naive strategies, table/code/mermaid detection
    • Hybrid semantic + FTS chunk search across 4 surfaces: REST endpoint, Cmd+K global search, chat search_documents tool, MCP search_document_chunks tool
    • Chunk edit/delete with optimistic locking + re-embed; bulk vision re-extract
    • Image upload UX (drag-drop + thumbnail preview + auto-Vision preselect)
    • Mermaid diagram rendering everywhere MarkdownContent is used
    • SSRF-hardened URL ingestion (private-range DNS check, redirect re-validation, streaming size cap)
    • 7 sprints, 7 migrations, 47-test E2E suite incl real vision runs
  • Phase 11: Knowledge Portability
    • Zip+JSONL bundle format with sha256 per entry — streaming encoder/decoder
    • Full project export (GET /api/projects/:id/export) via pg-cursor
    • Full project import (POST /api/projects/:id/import) with 3 conflict policies (skip/overwrite/fail) + dry-run + cross-tenant UUID guard
    • GUI Knowledge Exchange panel embedded in Project Settings (drag-drop + Preview + Apply)
    • Cross-instance pull (POST /api/projects/:id/pull-from) with DNS-rebinding pinning + slow-loris body-stall defense
    • Streaming JSONL decode (~99% jsonl peak memory reduction) + streaming base64 encode on import (~45% PDF peak reduction)
    • Batched SELECT on import (~99% SELECT-count reduction)
    • 9 sub-sprints, 61 API e2e + 1 GUI Playwright + 39 unit tests, all through v2.2 12-phase workflow with /review-impl
  • Phase 12: RAG Quality Measurement & Rerank Optimization
    • Retrieval measurement infrastructure: recall@k, MRR, latency budgets, golden-set harness
    • 8 reranker models benchmarked; qwen3-4b-instruct-ranker recommended (+9% accuracy at 180 lessons, 1.8s latency)
    • Rerank arc integrated into tiered search pipeline with configurable model + fallback
    • 7 sprints
  • Phase 13: Multi-Agent Coordination Protocol
    • Artifact ownership / leasing with TTL + fencing tokens + abandoned-claim sweep
    • pending-review lesson lifecycle state + Review Request queue
    • Taxonomy profiles (unified with lesson_types); Dead Light Framework reference profile shipped
    • 19-bug post-hoc review fully cleared; 7 sprints
  • Phase 14: Global Embedding & Distillation Model Swap
    • Embeddings: mxbai-large → bge-m3 (8192-token context, resolved silent 512-token truncation)
    • Distillation: qwen-coder → nemotron-3-nano (later switched to ibm/granite-4-h-tiny per current benchmark)
    • All projects re-embedded in place
  • Phase 15: Multi-Actor Coordination Protocol
    • Coordination substrate: durable append-only event log, Topic / Actor / participant model
    • The Board: tasks with dependency gating, derived-identity artifacts, claims with fencing
    • Request / Approval: multi-level routing, unilateral and collective procedures
    • Collective Decision: motions, votes, tally, veto, quorum
    • Intake mailbox, dispute resolution, 3-phase topic-close drain
    • Primitive-outcome chaining, multi-tier collective routing
    • Authorization model: non-owner level-grant, owner permanence, 3 HARD pre-prod triggers
    • End-to-end tenant-scope enforcement; 12 sprints (15.1–15.12), migrations 0050–0063
  • DEFERRED-029: MCP Tenant-Scope Enforcement
    • CallerScope threaded through ~115 service functions across 8 domain PRs
    • Scoped MCP token model via api_keys.project_scope; legacy CONTEXT_HUB_WORKSPACE_TOKEN retired
    • 5 adversary review passes (4 static + 1 live hardened-mode) found 7 bypasses (2 CRITICAL / 4 HIGH / 1 MED), all fixed before merge
    • 843 unit + 300 E2E tests green; 10 stacked PRs (#20–#29) + 1 orthogonal test-fix (#30)

In Progress:

  • Phase 16: Production-Ready RAG with Generation-Quality Evaluation (separate branch)
    • 152-row gen-eval dataset (127 retrieval queries + 25 hand-curated edge cases)
    • Ragas judge sidecar (Python / FastAPI) running google/gemma-4-26b-a4b-it
    • Metrics: faithfulness, answer_relevancy, context_precision, context_recall, refusal_correctness, groundedness_self_eval
    • Threshold gates in WARN mode until 2 weeks of baseline variance data, then flip to BLOCK
  • Phase 17: Anti-Hallucination Experiments (separate branch)
    • Citation-forced prompt templates, selective abstention
    • Chain-of-Verification synthesizer
    • groundedness_self_eval metric for model-side grounding check

Planned (no detailed design yet):

  • Phase 18 — Governance Benchmark Suite
    • Catalog of governance scenarios across org topologies (flat / hierarchical / federated), conflict patterns (collaborative / adversarial / byzantine), and human + AI mixes
    • Per-scenario success metrics: right outcome, right number of steps, right principals, auditability intact
  • Phase 19 — Runtime Enforcement & Isolation
    • Pre-action policy engine that blocks agent behavior before it occurs
    • Isolated environments for experimental / dangerous tasks
    • Open design questions: policy DSL (DLF-native vs OPA Rego / Cedar), enforcement location (gateway / SDK / OS sandbox), cryptographic identity (DIDs vs scoped API keys), isolation primitive (container / VM / WASM / seccomp)

Intentionally Dropped:

  • Multi-Agent Passive Collection — Parsing agent conversations costs tokens and captures noise. add_lesson captures verified conclusions explicitly.
  • Session History Sharing — Transcripts are 50k-200k tokens. The value is conclusions, not the journey.
  • IDE Extension — Agents use MCP (done). Humans use the web GUI — works in any browser including VS Code's built-in browser.

Project Stats

Metric Count
MCP Tools 45+
REST Endpoints 105+
GUI Pages 23
Database Migrations 63
Unit Tests 843
E2E Tests 300 (api 128 + gui 52 + smoke 111 + agent 9)
Sprint Phases 15 complete + DEFERRED-029 · Phase 16/17 active on separate branches
Strategic Arc Phase 1 shipped · Phase 2 + Phase 3 in progress · Phase 4 + Phase 5 planned (ROADMAP.md)

Contributing

Contributions are welcome! This is a solo project that grew into something useful — fresh eyes and ideas are appreciated.

  1. Fork the repo
  2. Create a feature branch (git checkout -b feature/my-feature)
  3. Make your changes
  4. Run npm run smoke-test to verify
  5. Submit a pull request

Please open an issue first for large changes so we can discuss the approach.


Troubleshooting

  • Unauthorized: invalid workspace_token: Set MCP_AUTH_ENABLED=false in .env or provide the correct token.
  • dimension mismatch: Ensure EMBEDDINGS_DIM=1024 matches your model's output.
  • 401 Unauthorized (LM Studio): Check EMBEDDINGS_API_KEY in your .env.

License

MIT — see LICENSE for details.

Roadmap | Whitepaper | Quickstart Guide | Knowledge Exchange Reference

About

🧠 Self-hosted "persistent memory + semantic search + guardrails" for MCP AI tools. Empower Cursor and Claude Code with long-term memory, intent-based repo indexing (pgvector), and safety checks. Capture decisions, preferences, and team lessons locally, with support for LLM-powered knowledge distillation—all on your own hardware.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages