ContextCortex provides fast, local, syntax-aware semantic and hybrid search over codebases, git repositories, markdown notes, architecture documents, and system documentation. It is built natively on the Model Context Protocol (MCP) SDK 2.0.0+ using FastMCP, with an integrated FastAPI web engine, real-time diagnostic logging, pluggable relational and vector store backends (PostgreSQL 16 with pgvector, Qdrant, ChromaDB, and SQLite), automatic polling daemons, multi-provider webhooks, interactive dependency topology graph explorer, RFC 9728 OAuth 2.1 Protected Resource Server, 3-tier API key RBAC, and a React 19 administrative dashboard.
All backend services and frontend components are modularized into cohesive packages with a strict sub-500 LOC per file maintainability floor.
flowchart TD
subgraph Clients["MCP & Web Clients"]
Claude["AI Coding Agents / MCP Clients\n(Cursor, Claude Desktop, Antigravity, Windsurf)"]
Browser["Admin Dashboard\n(ContextCortex Dashboard - React 19)"]
end
subgraph Server["FastAPI Core & FastMCP 2.0 Application (main.py)"]
FastAPI["FastAPI App (Lifespan Session Manager)"]
AuthMiddleware["Auth & RBAC Layer (app/services/auth/)"]
FastMCP["FastMCP Server (app/mcp/mcp_server.py)"]
SSE["SSE Transport (/sse, /messages/)"]
HTTP["Streamable HTTP Transport (/mcp)"]
WellKnown["RFC 9728 Protected Resource\n(/.well-known/oauth-protected-resource)"]
AdminAPI["Admin REST API Routers (app/api/routers/*)\n(repositories, settings, navigator, auth, storage, ingestion)"]
Webhooks["Webhook Ingestion (app/api/webhooks.py)"]
LogBuffer["Diagnostic Ring Buffer (app/services/logger.py)"]
end
subgraph ModularServices["Core Modular Services (app/services/)"]
subgraph AuthPkg["app/services/auth/"]
AuthSrv["service.py"]
KeySrv["key_service.py"]
JwtVal["jwt_validator.py"]
AuthModels["models.py"]
end
subgraph DatabasePkg["app/services/database/"]
SchemaCore["schema.py (SQLAlchemy Core)"]
EngineMgr["engine.py (Connection Pool & Retries)"]
DBConn["connection.py"]
Creds["credentials.py"]
SyncCfg["sync_config.py"]
ADRDb["adrs.py"]
EmbCacheDb["embedding_cache.py"]
end
subgraph VectorStorePkg["app/services/vector_store/"]
VSBase["base.py"]
VSManager["manager.py"]
PgVectorStore["pgvector_store.py (HNSW Cosine)"]
QdrantStore["qdrant_store.py (Dense + BM25)"]
ChromaStore["chroma_store.py"]
end
subgraph ChunkingPkg["app/services/chunking/"]
TSLoader["tree_sitter_loader.py"]
TextChunk["text_chunker.py"]
SymExtract["symbol_extractor.py"]
RelExtract["relationship_extractor.py"]
RouteExtract["api_route_extractor.py"]
end
subgraph IndexingPkg["app/services/indexing/"]
IdxState["state.py"]
GitSync["git_syncer.py"]
LocalSync["local_syncer.py"]
ProcFile["processor.py"]
end
subgraph TopologyPkg["app/services/topology/"]
GraphBuilder["graph_builder.py"]
NodeDetails["node_details.py"]
TopoHelpers["helpers.py"]
end
NavigatorSrv["Codebase Navigator Service (navigator.py)\nHero Tree, AST Outline & Impact Engine"]
OmniSearchSrv["Omni-Search Command Palette (omni_search.py)\nFuzzy Repo, File & Symbol Ranking"]
FileReaderSrv["Source & Doc Reader (file_reader.py)\nFull Code & Markdown Content Delivery"]
LocalStorageSrv["Local Storage Service (local_storage.py)\nPath Traversal Defense & File Trees"]
GitMgr["Universal Shallow Git Ingestion (git_manager.py)\nEphemeral & Persistent Clones"]
Embeddings["FastEmbed Engine (embeddings.py)\nDense (384d) + Sparse BM25"]
Search["Hybrid & RRF Search (search.py)"]
Poller["Auto-Sync Poller Daemon (poller.py)"]
ADRService["ADR Parser & Lifecycle (adr.py)"]
end
subgraph PersistentStorage["Pluggable Persistent Storage"]
subgraph RelationalDB["Relational Backend (SQLAlchemy 2.0)"]
PostgresDB[("PostgreSQL 16 Engine\n(pgvector/pgvector:pg16)")]
SQLiteDB[("SQLite WAL Engine\n(index_cache.db)")]
end
subgraph VectorEngines["Vector Search Backends"]
PgV["pgvector (HNSW Cosine)"]
Qdrant["Qdrant (Hybrid Dense+BM25)"]
Chroma["ChromaDB (Embedded/Remote)"]
end
LocalStorageDir[("Managed Local Storage\n(DATA_DIR/storage)")]
PersistentRepoDir[("Persistent Shallow Clones\n(DATA_DIR/repos/{repo_name})")]
end
Claude -->|Authorization: Bearer cc_... or JWT| SSE
Claude -->|Authorization: Bearer cc_... or JWT| HTTP
Claude -->|OAuth Discovery| WellKnown
SSE --> AuthMiddleware
HTTP --> AuthMiddleware
AuthMiddleware --> FastMCP
Browser -->|REST API /admin/api/*| AdminAPI
Browser -->|Webhooks /api/webhooks/*| Webhooks
AdminAPI --> AuthPkg
AdminAPI --> DatabasePkg
AdminAPI --> IndexingPkg
AdminAPI --> VectorStorePkg
AdminAPI --> TopologyPkg
AdminAPI --> NavigatorSrv
AdminAPI --> Search
AdminAPI --> LocalStorageSrv
AdminAPI --> LogBuffer
NavigatorSrv --> DatabasePkg
FastMCP --> Search
FastMCP --> SymExtract
FastMCP --> DatabasePkg
FastMCP --> IndexingPkg
FastMCP --> TopologyPkg
FastMCP --> ADRService
FastMCP --> LocalStorageSrv
LocalStorageSrv --> LocalStorageDir
LocalStorageSrv --> IndexingPkg
LocalStorageSrv --> VectorStorePkg
LocalStorageSrv --> DatabasePkg
IndexingPkg --> ChunkingPkg
IndexingPkg --> Embeddings
IndexingPkg --> GitMgr
IndexingPkg --> VectorStorePkg
IndexingPkg --> DatabasePkg
IndexingPkg --> LogBuffer
Poller --> GitMgr
Poller --> IndexingPkg
Webhooks --> IndexingPkg
VectorStorePkg --> VectorEngines
DatabasePkg --> RelationalDB
Search --> Embeddings
Search --> VectorStorePkg
- RFC 9728 Protected Resource Metadata (
/.well-known/oauth-protected-resource):- Exposes resource URI indicator, supported authorization server issuers, token bearer methods (
header), and available MCP scopes (mcp:admin,mcp:editor,mcp:viewer).
- Exposes resource URI indicator, supported authorization server issuers, token bearer methods (
- 3-Tier Role Hierarchy & Permissions:
admin(Level 30 /mcp:admin): Full administrative control over server settings, API key lifecycle, credentials vault, repository syncs, and MCP tools.editor(Level 20 /mcp:editor): Mutation operations including repository triggering, ADR authoring/updating, and read-only searches.viewer(Level 10 /mcp:viewer): Read-only retrieval across code search, documentation search, AST symbols, outlines, and architecture summaries.
- JWT & OIDC Validation (
jwt_validator.py):- Validates RS256/ES256 signed JWTs against OpenID Connect discovery endpoints (
/.well-known/openid-configuration) and JWKS key sets with in-memory caching and automatic stale key refresh. - Verifies token claims:
iss(issuer),audorresource(resource indicator matching RFC 8707 / RFC 9728),exp(expiration with clock skew tolerance), and extracts roles fromroles,groups, orscopeclaims.
- Validates RS256/ES256 signed JWTs against OpenID Connect discovery endpoints (
- Database-Backed API Keys (
key_service.py):- Generates secure random keys prefixed with
cc_(e.g.cc_live_...). - Stores SHA-256 hashes (
key_hash) and 16-character public prefixes (key_prefix) in relational storage. - Tracks
last_used_attimestamps, expiration policies, and active revocation flags.
- Generates secure random keys prefixed with
- Tool Authorization Guards (
enforce_tool_permission):- ContextVar-based per-request security context propagation (
AuthContext). - Guards MCP tool executions with clear
ForbiddenError(403) orAuthenticationError(401) responses when permissions are insufficient.
- ContextVar-based per-request security context propagation (
- Local Development Bypass & Auto-Bootstrap:
- Default bypass mode when
AUTH_ENABLED=falsegrants fulladminrights for zero-friction local development. - Automatically bootstraps admin keys from
ADMIN_INITIAL_KEYenvironment variable during container startup.
- Default bypass mode when
- Canonical Schema Single Source of Truth (
schema.py):- Declares all relational tables using SQLAlchemy 2.0 Core (
MetaData,Table,Column,Index,ForeignKey). - Maintains structural parity between SQLite and PostgreSQL 16.
- Declares all relational tables using SQLAlchemy 2.0 Core (
- Engine Factory & Connection Pool (
engine.py):- Dynamic URL normalization for
postgresql+psycopg://driver connection strings andsqlite:///paths. - Connection pooling with
pool_pre_ping=True, configurablepool_size(default 10) andmax_overflow(default 20) for PostgreSQL. - WAL mode, busy timeout (5000ms), and foreign keys enabled automatically on SQLite connections.
- Cold-boot retry loop (
wait_for_db) with exponential backoff for resilient container orchestration. - Idempotent table creation (
metadata.create_all) and automatic default seeding (system metadata, default prompts, vault paths).
- Dynamic URL normalization for
- Embedding Cache (
embedding_cache.py):- Fast chunk-hash deduplication cache storing dense and sparse embedding representations across reindexing runs.
- Custom Git Host Vault & ADR Storage (
credentials.py,adrs.py,sync_config.py):- Multi-tier credential hierarchy resolution and Architectural Decision Record lifecycle state machine.
- Abstract Vector Interface (
base.py):- Standardized
VectorStorecontract withupsert_documents,search,delete_by_path,delete_by_repo,get_stats, andhealth_check.
- Standardized
- PostgreSQL 16 + pgvector Backend (
pgvector_store.py):- Native
vector(384)column storage with HNSW cosine distance index (vector_cosine_ops). - Cosine similarity ranking (
1 - (embedding <=> query_vec)). - Native
JSONBpayload and tag filtering (tags @> '["tag"]'). - Parameterized chunked batch upserts with
ON CONFLICT (id) DO UPDATE.
- Native
- Qdrant Backend (
qdrant_store.py):- Embedded disk and remote server support with dense + sparse BM25 multi-vectors and Reciprocal Rank Fusion (RRF).
- ChromaDB Backend (
chroma_store.py):- Lightweight persistent disk or remote vector store.
- Vector Store Manager (
manager.py):- Singleton provider supporting dynamic runtime backend switching across
pgvector,qdrant, andchroma.
- Singleton provider supporting dynamic runtime backend switching across
- FastMCP Core (
app/mcp/mcp_server.py): Manages MCP lifespan and session registration. - Dual Transports:
- Server-Sent Events (SSE): Streaming events at
/sseand message exchanges at/messages/. - Streamable HTTP: Bidirectional JSON-RPC at
/mcp.
- Server-Sent Events (SSE): Streaming events at
- Extended Agent Tools (
app/mcp/tools.py&app/mcp/handlers/):search_handlers.py:search_code,search_docshybrid search with RRF.symbol_handlers.py:find_symbol,get_file_outlinesub-50ms AST lookups.repo_handlers.py:list_repositories,sync_repository,index_status.route_handlers.py:get_code_routes,trace_call_pathcross-repo API calls.architecture_handlers.py:get_architecture,manage_adrADR tracking.storage_handlers.py:manage_local_file(upload, replace, delete, read),what_is_ingested(unified multi-source catalog filter).
- Dynamic Resource Providers:
knowledge://catalog/summary: Markdown catalog of repositories, document types, and symbols.
- Agent Prompts:
search_infrastructure_docs,find_implementation_symbol.
tree_sitter_loader.py: Lazy loader for 10 Tree-sitter grammars (Python, TS/JS, Go, Rust, C#, C++, Java, Ruby, PHP).text_chunker.py: Hierarchical markdown breadcrumbs (# > ## > ###) and sliding window text chunking.symbol_extractor.py: AST symbol declaration extraction with 1-indexed line numbers and signatures.relationship_extractor.py: AST relations extraction (CALLS,IMPORTS,EXTENDS).api_route_extractor.py: REST route definitions and client invocations across backend frameworks.
state.py: Global indexing lock, active session notifications (send_tool_list_changed), configuration constants.git_syncer.py: Ephemeral shallow git cloning, remote commit SHA tracking, AST symbol ingestion, vector upserts.local_syncer.py: Local filesystem directories and Obsidian markdown vault indexing with mtime caching.processor.py: Unified file parsing, YAML frontmatter extraction, and AST chunk generation.
graph_builder.py: Builds multi-repo dependency graphs combining files, classes, functions, and API routes with depth-bounded BFS filtering.node_details.py: Formats deep inspection data (code previews, line ranges, neighbor relations, permalinks).helpers.py: Cross-repo node ID generation, URL link normalization, and graph pruning.
- Universal Provider Ingestion: GitHub, GitLab (Cloud & Self-Hosted), Gitea/Forgejo, Bitbucket, and Generic Git HTTP/HTTPS.
- Zero Disk Bloat: Shallow ephemeral clones cleaned immediately after processing.
- Provider-Exact Permalinks: Deep code permalinks across all supported Git hosts.
- Credential Sanitization: URL sanitization masking tokens in logs and client responses.
app/services/poller.py: Background daemon checking remote commit SHAs at configured intervals.app/api/webhooks.py: Authenticated push event ingestion for GitHub, GitLab, Gitea, and Bitbucket.
- Configurable Storage Directory: Managed directory tree located at
DATA_DIR/storageor configured viaLOCAL_STORAGE_PATH. - Path Sanitization & Traversal Defense:
- Validates relative paths using
os.path.abspath(os.path.join(root, rel_path)). - Canonical containment verification via
os.path.commonpath([resolved_path, root_dir]) == root_dir. - Rejects
.., absolute paths, leading slashes, and null bytes (\x00) with 400 Bad Request / error strings.
- Validates relative paths using
- Directory Hierarchy & Tree Inspection:
get_file_tree(subfolder)generates nested directories, file metadata (sizes, mtimes), and aggregate file counts for UI and catalog consumers.
- Role-Based Access Enforcement:
- Mutation actions (
upload,replace,delete) requireRole.EDITOR. - Read actions (
read,tree,catalog) requireRole.VIEWER.
- Mutation actions (
The incremental ingestion engine allows documents, notes, and code files uploaded to Local Storage to be parsed, chunked, and vector-indexed with sub-second latency:
flowchart TD
Client["Client / Agent Request\n(Upload / Replace / Delete)"]
AuthCheck{"RBAC Guard\n(Role.EDITOR)"}
PathGuard{"Path Sanitization\n(commonpath == root)"}
subgraph StorageOps["Local Storage Operations"]
DiskWrite["Save File to Disk\n(LOCAL_STORAGE_PATH / rel_path)"]
DiskDelete["Remove File from Disk"]
end
subgraph IncrementalIndexing["Real-Time Indexing Pipeline (processor.py)"]
AST["Tree-sitter AST Parsing\n(symbols, relationships, routes)"]
Chunk["Semantic Boundary Chunking\n(Markdown headers / code blocks)"]
Embed["Embedding Generation\n(Dense 384d + Sparse BM25)"]
Cache["Chunk-Hash Deduplication\n(embedding_cache)"]
end
subgraph StoragePersistence["Persistent Store Upserts / Purges"]
RelUpsert["Relational Upserts\n(indexed_files, file_summaries,\nast_symbols, ast_relationships,\napi_routes, api_calls)"]
VecUpsert["Vector Store Upsert\n(upsert_points deterministic UUID5)"]
RelDelete["Relational Purge\n(DELETE WHERE filepath = ?)"]
VecDelete["Vector Store Purge\n(delete_by_path)"]
end
Notify["List Changed Notification\n(trigger_list_changed_notification)"]
Client --> AuthCheck
AuthCheck -->|Authorized| PathGuard
%% Upload / Replace Flow
PathGuard -->|Upload / Replace| DiskWrite
DiskWrite --> AST
AST --> Chunk
Chunk --> Cache
Cache --> Embed
Embed --> VecUpsert
AST --> RelUpsert
VecUpsert --> Notify
RelUpsert --> Notify
%% Delete Flow
PathGuard -->|Delete| DiskDelete
DiskDelete --> RelDelete
DiskDelete --> VecDelete
RelDelete --> Notify
VecDelete --> Notify
- Multi-Source Aggregation: Single consolidated inventory querying across:
- Git Repositories (
git_repositoriestable, branches, commit SHAs, URLs, sync status). - Monitored Local Paths (
indexed_pathstable, categories, recursive scan settings). - Managed Local Storage (
local_storagenamespace files inindexed_filesand filesystem tree).
- Git Repositories (
- Multi-Dimensional Query Filtering:
source_type: Filter byall,git,monitored_path, orlocal_storage.repo_name: Exact match repository or namespace alias.path_prefix: Filter files matching directory / prefix path.file_extension: Filter by extension (e.g..md,.py,.ts).detail_level:summaryfor high-level repository stats, file counts, and symbol totals;detailedfor comprehensive file-by-file inventories with doc types and languages.
- MCP Tool & REST Parity: Exposes identical catalog querying functionality via FastMCP tool (
what_is_ingested) and FastAPI endpoint (GET /admin/api/ingestion/catalog) guarded byRole.VIEWER.
13. High-Performance Codebase Navigator & Omni-Search (app/services/navigator.py, app/services/omni_search.py, app/services/file_reader.py)
- Architectural Motivation: Provides an ultra-fast, IDE-grade codebase comprehension and exploration system with split hero layout, instant command palette search, syntax-highlighted code viewing, and deep AST symbol intelligence.
- Hero Split-View Layout Topology:
- Left Sidebar: Dual-Tab Directory Tree & Symbol Outline:
- Files Tab (
NavigatorTree.tsx): QueriesGET /admin/api/navigator/tree?repo=.... Recursively structuresindexed_filesinto hierarchical trees with aggregate symbol counts and API routes. Features instant search filtering, Expand All / Collapse All controls, active file highlighting, and per-repository directory expansion persistence viasessionStorage. - Symbols Tab (
NavigatorOutline.tsx): QueriesGET /admin/api/navigator/file-outline?filepath=...&repo=.... Displays AST symbol declarations fromast_symbolsand REST routes fromapi_routeswith category filtering (All,Functions,Classes,Routes), signature previews, and search.
- Files Tab (
- Center Hero Viewport: Code Viewer, Doc Reader & Intelligence:
- Syntax-Aware Source Code Viewer (
NavigatorCodeViewer.tsx): QueriesGET /admin/api/navigator/file-content?filepath=...&repo=...throughfile_reader.py. Features Prism-powered syntax highlighting across 8+ languages (C#, Python, JavaScript/TypeScript, C++, Go, Rust, SQL, COBOL, JSON, YAML, etc.), line numbering, target line range highlights (targetStartLine-targetEndLine), permalink copying, and collapsible caller impact drawer. - Documentation & Markdown Reader (
NavigatorDocReader.tsx): Renders markdown files with GitHub Flavored Markdown (GFM), automatic table normalization (bridging blank lines and auto-inserting missing separator rows), and interactive Mermaid diagram rendering (flowchart,sequenceDiagram,classDiagram,erDiagram, etc.). - Code Intelligence & Impact Inspector (
NavigatorInspector.tsx): QueriesGET /admin/api/navigator/symbol-impact?symbol_id=.... Immediately accessible and clickable upon file selection without requiring manual outline navigation. Aggregates caller/callee counts, API route mappings, method signatures, docstrings, and cross-file jump navigation.
- Syntax-Aware Source Code Viewer (
- Omni-Search Command Palette (
NavigatorOmniSearch.tsx/Ctrl+K):- Queries
GET /admin/api/navigator/omni-search?query=...&repo=.... - Real-time fuzzy ranked matching across repository aliases, file paths, and AST symbols with hotkey triggers, badge indicators, and instant keyboard selection.
- Queries
- Left Sidebar: Dual-Tab Directory Tree & Symbol Outline:
- Multi-Density Layout Engine & Persistent UX:
Balanced: Default balanced layout optimized for standard desktop viewports.Compact: High-density IDE layout reducing font sizes and padding for large file trees and complex outlines.Spacious: Card-based layout with expanded line heights and spacious typography.- Layout density and last selected repository are persisted in
localStorage.
- Zero Horizontal Overflow & Responsive Adaptability:
- Flexbox and grid CSS architecture with
min-width: 0,overflow-wrap: anywhere, and nested vertical scroll containers ensuring zero horizontal viewport overflow across desktop (1080p), tablet, and mobile displays.
- Flexbox and grid CSS architecture with
erDiagram
GIT_REPOSITORIES {
int id PK
string name UK
string url
string branch
string provider
string auth_user
string auth_token
string commit_sha
string status
string last_error
datetime last_synced
int enabled
int auto_sync
int keep_shallow "0 = Ephemeral (delete on sync), 1 = Retain shallow clone on disk"
string webhook_secret
datetime added_at
}
GIT_HOST_CREDENTIALS {
int id PK
string host UK
string provider
string auth_user
string auth_token
datetime added_at
}
INDEXED_PATHS {
int id PK
string path UK
string type
int recursive
int enabled
string category
string repo
datetime added_at
}
INDEXED_FILES {
string filepath PK
string repo
string doc_type
string language
string commit_sha
real mtime
string hash
}
AST_SYMBOLS {
int id PK
string repo
string filepath
string kind
string name
string full_symbol
string signature
int start_line
int end_line
string language
}
AST_RELATIONSHIPS {
int id PK
string repo
int source_symbol_id FK
string source_filepath
string source_symbol
string target_symbol
string relationship_type
int line_number
}
API_ROUTES {
int id PK
string repo
string filepath
string framework
string http_method
string path_pattern
string handler_symbol
int start_line
int end_line
datetime created_at
}
API_CLIENT_CALLS {
int id PK
string repo
string filepath
string http_method
string url_pattern
string caller_symbol
int line_number
datetime created_at
}
ARCHITECTURE_DECISION_RECORDS {
string id PK
string repo
string title
string status
string context
string decision
string consequences
string superseded_by
datetime created_at
datetime updated_at
}
FILE_SUMMARIES {
string filepath PK
string repo
string title
string folder
string category
string tags
string headings
string keywords
real mtime
}
EMBEDDING_CACHE {
string chunk_hash PK
string model_name PK
string dense_vector
string sparse_indices
string sparse_values
datetime created_at
}
CUSTOM_PROMPTS {
int id PK
string name UK
string description
string arguments_json
string template
datetime added_at
}
API_KEYS {
int id PK
string name
string key_prefix
string key_hash UK
string role
string group_name
datetime expires_at
datetime created_at
datetime last_used_at
boolean is_active
}
SYSTEM_METADATA {
string key PK
string value
}
GIT_REPOSITORIES ||--o{ INDEXED_FILES : "contains"
GIT_REPOSITORIES ||--o{ AST_SYMBOLS : "declares"
GIT_REPOSITORIES ||--o{ AST_RELATIONSHIPS : "traces"
GIT_REPOSITORIES ||--o{ API_ROUTES : "exposes"
GIT_REPOSITORIES ||--o{ API_CLIENT_CALLS : "invokes"
GIT_REPOSITORIES ||--o{ ARCHITECTURE_DECISION_RECORDS : "documents"
GIT_REPOSITORIES ||--o{ FILE_SUMMARIES : "summarizes"
INDEXED_PATHS ||--o{ INDEXED_FILES : "contains"
INDEXED_FILES ||--o{ AST_SYMBOLS : "defines"
INDEXED_FILES ||--o| FILE_SUMMARIES : "has metadata"
AST_SYMBOLS ||--o{ AST_RELATIONSHIPS : "source"
This architecture specification complies with the ASD-STE100 Simplified Technical English (Issue 9) standard.
Interactive system design diagrams, sequence flows, and component layouts are available on the VitePress Documentation Site.