CODEGENOME
Unified Code Reality Graph
- Overview
- Intent & Value Provenance
- Problem Statement
- Architecture
- Multi-Language Support
- Quick Start
- Graph Composition
- Confidence Fusion
- Experiment Engine
- Research Tracker
- Governance
- Agent Memory Alignment
- MCP Integration
- Project Metrics
- Dependencies
- Roadmap
- Contributing
- License
CODEGENOME is a content-addressed, multi-layer program analysis graph that merges syntax, semantics, control/data flow, process traces, and runtime observations into a single queryable substrate. Every node has a cryptographic identity (BLAKE3). Every edge carries a confidence score. Independent evidence sources fuse via noisy-OR. The system hill-climbs on its own accuracy using an adaptive, self-evolving experiment engine.
The intended outcome is not "better code search" or "a smarter assistant." The intended outcome is a universal internal representation from which those functions can be derived as query, traversal, policy, and mutation operations over the same canonical reality model.
Research prototype. Not production-ready. APIs will change.
This repository carries a dual mandate, and both halves are permanent:
- Experimental instrument. CODEGENOME was designed from the start to measure itself — recall/impact accuracy, propagation behavior, graph density, cycle time, and governance integrity are first-class, instrumented metrics. The experiment engine, the fitness functions, the tamper-evident run logs, and the archived raw data in
data/runs/are not scaffolding to be removed on the way to a product; they are the evidence-generating apparatus the product's claims rest on. - Product substrate. CODEGENOME is developed toward a usable product: a governed code-reality graph that AI agents consume as domain-specific memory (via MCP and the Agent Memory doctrine).
The connection between the two is the point: every capability the product claims must trace to the experimental record that established it. Fusion is in the product because RUN-001/RUN-002 showed a structural fitness jump (0.29 → 0.64 baseline) that 224K iterations of parameter tuning could not produce. Freshness gating is in the product because staleness is a measured, computable relation. Future capabilities enter the same way — through experiments whose raw data is retained and whose conclusions are recorded in the governance ledger. Changes that would degrade the repository's capacity to run, log, verify, and archive experiments are treated as regressions, even if they simplify the product.
The current tooling landscape fragments code understanding across incompatible models — syntax-aware parsers, semantic indexers, dependency analyzers, runtime tracing systems, and assistant retrieval pipelines. Each builds a partial internal truth optimized for one surface area. The ecosystem compensates by repeatedly translating, summarizing, or approximating across boundaries.
This produces four persistent failures:
| Failure | Description |
|---|---|
| Identity drift | The same artifact is represented differently across tools and sessions |
| Semantic drift | A relationship recovered in one subsystem cannot be expressed in another without lossy conversion |
| Provenance loss | High-level conclusions outlive the evidence and method that produced them |
| Governance weakness | Read/write operations lack a substrate-level model for confidence, policy, and auditability |
CODEGENOME addresses these failures by replacing tool-local truth with canonical graph reality.
Source Files (.rs, .ts, .tsx, .py)
│
├── [Language Detection] extension-based routing
│
▼
[Syntax Overlay] tree-sitter AST → Contains edges (confidence: 1.0)
[Semantic Overlay] Multi-lang symbol resolution → Calls, Imports, Implements (0.7–0.8)
[Flow Overlay] Intraprocedural CFG/DFG → ControlFlow, DataFlow (1.0)
[Process Overlay] Entrypoint detection → PartOfProcess edges (0.9^depth)
[PDG Overlay] CFG+DFG composition → ControlDependence (1.0)
[Runtime Overlay] TSV trace ingestion → weighted Calls (count/10)
[SCIP Overlay] Compiler index → References, Implements (1.0)
[LSP Overlay] rust-analyzer → References (1.0)
│
▼
[Confidence Fusion] Noisy-OR merge: two 0.7 edges → one 0.94 edge
│
▼
[Signal Propagation] Impact forward, staleness backward
│
▼
[Experiment Engine] Adaptive hill-climbing with LLM advisor
| Crate | Purpose |
|---|---|
codegenome-identity |
Graph types, UOR identity, 9 overlays, multi-lang extraction (lang/), indexing pipeline, fusion, signal propagation, evidence, diff mapping |
codegenome-substrate |
Experiments engine, beliefs, embeddings, federation, governance (policy, Merkle ledger, write gate) |
codegenome-cli |
11 CLI commands: index, query, status, serve, init, verify, analyze, experiment, federate, visualize, workspace-report |
codegenome-mcp |
MCP stdio tool server with 11 governed tools (7 read + 4 write-gated) |
| Layer | Purpose | Modules |
|---|---|---|
| L1: Canonical Reality | Store what exists | graph/, identity/, store/ |
| L2: Extraction | Convert source evidence into graph artifacts | lang/, index/, overlay/ |
| L3: Composition | Fuse observations into coherent state | index/merger, confidence/fusion |
| L4: Query | Make graph reality operationally usable | graph/query, graph/traversal |
| L5: Governance | Constrain and audit actions | codegenome-substrate/src/governance/ |
| L6: Evaluation | Measure and improve substrate quality | codegenome-substrate/src/experiments/ |
UOR Address. Every graph node is identified by BLAKE3(content). The same code always produces the same address, regardless of toolchain version or language.
Overlay. A composable layer that contributes nodes and edges without knowledge of other overlays. Nine overlays exist, each implementing the Overlay trait.
Observer Separation. External systems (tree-sitter, LSP, SCIP, runtime traces) contribute observations about reality. No observer is canonical. The graph is canonical.
Provenance. Every non-trivial graph artifact carries: what created it, when, whether it was inferred or observed, and what evidence supports it.
CODEGENOME extracts code intelligence from three language families through a shared LanguageSupport trait:
| Language | Grammar | Extensions | Extraction |
|---|---|---|---|
| Rust | tree-sitter-rust |
.rs |
Symbols, imports, calls, impls, CFG, DFG |
| TypeScript | tree-sitter-typescript |
.ts, .tsx |
Functions, classes, interfaces, enums, imports, calls, class heritage, CFG, DFG |
| Python | tree-sitter-python |
.py |
Functions, classes (decorator unwrap), imports, calls, base classes, CFG, DFG |
All backends produce the same language-neutral intermediate representation (lang/ir.rs), which feeds into a shared graph builder. The pipeline automatically detects file languages, groups by backend, and dispatches extraction in parallel.
# Clone and build
git clone https://github.com/MythologIQ-Labs-LLC/CodeGenome.git
cd CodeGenome
cargo build --workspace
# Index a mixed-language codebase
cargo run -p codegenome-cli -- index --source-dir ./src
# Query impact from a location
cargo run -p codegenome-cli -- query --file src/lib.rs --line 44
# Check index freshness
cargo run -p codegenome-cli -- status
# Start MCP server for AI assistant integration
cargo run -p codegenome-cli -- serve
# Run the self-evolving experiment engine
cargo run -p codegenome-cli -- experiment --source-dir ./src --max-iterations 100
# Verify experiment chain integrity
cargo run -p codegenome-cli -- verifyNine overlay layers compose through a shared Overlay trait:
| Layer | Edge Types | Source | Confidence |
|---|---|---|---|
| Syntax | Contains |
tree-sitter AST | 1.0 |
| Semantic | Calls, Imports, Implements |
Multi-lang symbol resolution | 0.7–0.8 |
| Flow | ControlFlow, DataFlow |
Intraprocedural AST walk | 1.0 |
| Process | PartOfProcess |
Entrypoint BFS tracing | 0.9^depth |
| PDG | ControlDependence, DataFlow |
CFG+DFG composition | 1.0 |
| Runtime | Calls (weighted) |
TSV trace file | count/10 |
| SCIP | References, Implements |
Compiler index (protobuf) | 1.0 |
| LSP | References |
rust-analyzer subprocess (stub — detects the binary, contributes no edges yet) | 1.0 |
| Fused | All types merged | Noisy-OR fusion | max ≤ fused ≤ 1.0 |
When multiple overlays produce edges between the same nodes with the same relation, their confidences fuse via noisy-OR:
P(fused) = 1 - ∏(1 - cᵢ)
Two independent 0.7 estimates become 0.91. Three become 0.973. This is the mechanism by which structural improvements to the graph produce larger accuracy gains than parameter tuning.
The engine implements a four-tier autoresearch pattern:
| Tier | Behavior | Trigger |
|---|---|---|
| 1 — Mechanical | Hill-climb 3 parameters | Every iteration |
| 1.5 — Adaptive | Widen search exponentially, restart with random params | Plateau (10 no-improvement) |
| 2 — Agent-Guided | Local LLM (4-bit quantized) reads TSV, recommends strategy | After mechanical restart |
| 3 — Autopoietic | Switch fitness function mid-run | LLM recommends SWITCH_FITNESS |
Four fitness functions (switchable at runtime):
| Function | Measures | Range |
|---|---|---|
ImpactAccuracy |
Sibling reachability via propagation | [0, 1] |
PropagationDepth |
Average max hops from sampled symbols | [0, 1] |
CycleTime |
Parse-propagate speed (faster = higher) | [0, 1] |
GraphDensity |
Edge-to-node ratio in fused overlay | [0, 1] |
| Parameter | Default | Range | Effect |
|---|---|---|---|
confidence_threshold |
0.75 | 0.01–0.99 | Minimum confidence to include in propagation |
attenuation_factor |
0.85 | 0.1–2.0 | Signal decay per hop |
max_depth |
5.0 | 1.0–20.0 | Maximum propagation depth |
| ID | Name | Iterations | Duration | Peak Fitness | Status |
|---|---|---|---|---|---|
| RUN-001 | Pre-Fusion Exhaustive Search | 224,703 | ~62 hrs | 0.896 | Complete — raw data archived in data/runs/ |
| RUN-002 | Post-Fusion Baseline | 20+ (early) | — | 0.825 (early) | Paused — baseline recorded, full run not yet resumed |
RUN-001: Pre-Fusion Engine (224,703 iterations)
The first autonomous run used the three-layer engine (syntax + semantic + flow) without confidence fusion.
Duration: ~62 hours (~100 iterations/minute)
Iterations: 224,703
Fitness peak: 0.896
Avg fitness: 0.59
Total KEEPs: 8 (0.004%)
Restarts: ~7,500
Trend analysis (sampled every 25K iterations):
| Iteration | Avg Fitness | Max Fitness | KEEPs | Restarts |
|---|---|---|---|---|
| 0–25K | 0.591 | 0.894 | 6 | 331 |
| 25K–50K | 0.588 | 0.894 | 0 | 334 |
| 50K–75K | 0.620 | 0.894 | 0 | 333 |
| 100K–125K | 0.612 | 0.894 | 0 | 333 |
| 125K–150K | 0.565 | 0.895 | 1 | 333 |
| 175K–200K | 0.596 | 0.896 | 1 | 332 |
| 200K–225K | 0.620 | 0.896 | 0 | 334 |
The 75K–100K and 150K–175K buckets were omitted from the original analysis; the full raw record (all 224,703 iterations) is committed at data/runs/run-001-prefusion-224k.tsv and can be re-derived from it.
Findings:
- The fitness function has a structural ceiling determined by the graph's edge confidence distribution
- The ceiling at 0.896 was confirmed stable across 224K iterations
- The engine found it within the first 1,000 iterations and spent the remaining 223K cycling plateau→widen→restart
- Discovery: The engine converged on
attenuation_factor ≈ 1.3–1.5— signal amplification rather than decay. This compensates for low-confidence heuristic edges. Above ~1.7, BFS scores diverge (NaN), revealing a stability boundary
RUN-002: Post-Fusion Engine (in progress)
After adding confidence fusion (noisy-OR), the engine was restarted with the same parameters.
Baseline: 0.644 (vs 0.292 pre-fusion — 2.2x higher starting point)
Early peak: 0.825 at iteration 20
Architecture: Fused overlay merges duplicate edges
Findings:
- Fusion raises the ceiling by strengthening weak edges — two 0.7 estimates fuse to 0.94
- The baseline jumped from 0.29 to 0.64 without parameter changes — purely from structural improvement
- The engine reached 0.825 in 20 iterations (vs 224K iterations at 0.896 pre-fusion)
Phase 4 (baseline): fitness = 0.29 ← syntax only, 1 overlay
Phase 6 (three-layer): fitness = 0.29 ← 3 overlays, unfused
Phase 7 (adaptive): fitness = 0.35 ← plateau detection + widen
Phase 11 (fusion): fitness = 0.87 ← noisy-OR edge merge (breakthrough)
Run 001 (224K iters): ceiling = 0.896 ← pre-fusion exhaustive search
Run 002 (post-fusion): baseline = 0.64 ← fusion raises the floor
Key insight: Structural changes to the graph (adding overlays, fusing edges) produce larger fitness improvements than parameter optimization. The engine spent 224K iterations improving parameters by 0.006. One architectural change (fusion) improved the baseline by 0.35.
| # | Question | Hypothesis | Status |
|---|---|---|---|
| RQ-1 | What is the post-fusion fitness ceiling? | Higher than 0.896 due to stronger edge confidences | Open (RUN-002 paused) |
| RQ-2 | Does multi-language indexing affect graph density? | Cross-language call edges increase density but may reduce precision | Planned |
| RQ-3 | Can federated cross-repo edges improve impact analysis? | Cross-repo dependency edges should reduce false negatives in blast radius | Planned |
| RQ-4 | Does the autopoietic tier (fitness switching) find better optima? | Dynamic fitness landscapes prevent premature convergence | Planned |
| RQ-5 | What is the optimal confidence floor for write gating? | Too low permits unreliable mutations; too high blocks legitimate operations | Planned |
Every graph operation is governed:
- Merkle Ledger: BLAKE3-chained record of all operations with cryptographic integrity
- Evidence Log: Tamper-evident TSV with chain verification
- Policy Engine: TOML rules for allow/deny/require-approval decisions
- Write Gating: Confidence floors and provenance requirements before mutation
- Index Freshness: Source hash comparison detects stale indexes
# governance.toml
[[rules]]
operation = "query"
condition = "impact_nodes > 10"
action = "require-approval"This repository implements the code reality graph — content-addressed code artifact identity, evidence and provenance bundles, confidence fusion, and impact traversal — within the Agent Memory reference architecture. CODEGENOME supplies domain evidence and graph structure; it does not own general memory lifecycle or authority doctrine.
Canonical doctrine this work maps to:
- ADR-005 — CodeGenome is the code reality substrate
- Source trust is a first-class signal (ADR-009)
- Repo implementation map
- Canonical and derived state — this repository's index-freshness check (source-hash comparison detecting stale indexes) is a working instance of the staleness-as-computable-relation design that document proposes
The governing boundary: confidence is evidence support, not lifecycle permanence. Inferred edges carry estimator provenance and preserve disagreement rather than averaging it away; graph nodes become memory units only when consumed under Agent Memory's lifecycle and authority contracts.
Implementation status: declared
Conformance evidence: none yet
CODEGENOME exposes 11 governed tools via the Model Context Protocol (stdio):
| Tool | Input | Output | Gate |
|---|---|---|---|
codegenome_context |
file, line | Node + neighbors | Read |
codegenome_impact |
file, line | Blast radius with confidence scores | Read |
codegenome_detect_changes |
git ref range | Affected symbols + blast radius + process impacts | Read |
codegenome_trace |
entrypoint name | Process call chain | Read |
codegenome_assert |
claim, file, line, confidence | Belief node creation with provenance | Write |
codegenome_reindex |
actor | Re-index source files | Write |
codegenome_status |
— | Index freshness report | Read |
codegenome_experiment_start |
source dir, iterations | Start async experiment loop | Write |
codegenome_experiment_status |
— | Poll experiment progress | Read |
codegenome_experiment_results |
last N | Recent experiment results | Read |
codegenome_workspace_trace |
workspace, from, to | Cross-repo workspace path trace | Read |
All responses are governed — evidence is automatically compiled and policy is evaluated before returning results. Write-gated tools pass through WriteGatePolicy before mutation.
# Start the MCP server
codegenome index --source-dir .
codegenome serve| Metric | Value |
|---|---|
| Source files | 198 (Rust, ~15.3K LOC) |
| Languages supported | 3 (Rust, TypeScript, Python) |
| Tests | 264 (all passing) |
| Crates | 4 (identity + substrate + cli + mcp) |
| Overlays | 9 (8 active + LSP stub) + beliefs module |
| Edge types | 16 |
| CLI commands | 11 |
| MCP tools | 11 (7 read + 4 write-gated) |
| Fitness functions | 4 |
| Governance entries | 118 |
| Experiment data | 224K+ iterations archived in data/runs/ |
| Package | Purpose |
|---|---|
blake3 |
Content-addressed hashing for UOR identity and chain integrity |
serde + postcard + serde_json |
Serialization for graph store, checkpoints, JSON output |
tree-sitter |
Incremental multi-language parsing engine |
tree-sitter-rust |
Rust grammar for syntax overlay |
tree-sitter-typescript |
TypeScript/TSX grammar for syntax overlay |
tree-sitter-python |
Python grammar for syntax overlay |
clap |
CLI argument parsing |
rand |
Randomized parameter perturbation in experiment loop |
rayon |
Parallel pipeline fan-out (per-language group) |
mistralrs |
Embedded local LLM for Tier 2 experiment advisor (4-bit quantized) |
tokio |
Async runtime for LLM inference and MCP server |
rmcp |
MCP protocol SDK for tool server |
toml |
Governance policy config parsing |
prost |
Protobuf decoding for SCIP index files |
lsp-types |
LSP protocol types for rust-analyzer integration |
git2 |
Git repository operations for diff extraction |
proptest |
Property-based testing (dev only) |
| Capability | Status | Location |
|---|---|---|
| UOR content-addressed identity | Implemented | identity/ |
| 9 overlay types | 8 implemented, LSP overlay stub | overlay/ |
| Multi-language extraction (Rust + TS + Python) | Implemented | lang/ |
| Language-neutral IR + shared graph builder | Implemented | lang/ir.rs, lang/graph_builder.rs |
| Multi-lang pipeline wiring | Implemented | index/parser, index/flow, index/resolver_multi |
| Confidence fusion (noisy-OR) | Implemented | confidence/fusion.rs |
| Signal propagation | Implemented | signal/ |
| Karpathy Loop (Tier 1–3) | Implemented | experiments/ |
| Secured experiment chain | Implemented | BLAKE3 chain + checkpoint |
| CLI (11 commands) | Implemented | codegenome-cli/ |
| MCP server (11 tools) | Implemented | codegenome-mcp/ |
| Governance (ledger + policy + write gating) | Implemented | codegenome-substrate/src/governance/ |
| Evidence log | Implemented | evidence/ |
| Index freshness | Implemented | store/meta.rs |
| Federation (cross-repo identity + symbol resolution) | Implemented | federation/, graph/query_context |
| Federated query (QueryContext polymorphism) | Implemented | federation/query_context.rs |
| Reasoning artifacts (beliefs as graph nodes) | Implemented | belief/ |
| MCP assertion tool (write-gated) | Implemented | codegenome-mcp/tools/assert_belief |
| Impact propagation in detect_changes | Implemented | diff/propagator, tools/detect |
| Capability broker | Planned | Awaiting FailSafe-Pro |
| LSP overlay (rust-analyzer protocol) | Planned | overlay/lsp.rs is a stub |
| SLSA supply-chain attestation (backlog W4) | Planned | — |
| Sigstore signing of graph artifacts (backlog W5) | Planned | — |
| IDE panels + code lenses (backlog W1) | Planned | — |
This is an active research project. The codebase evolves through a governance protocol (plan → audit → implement → substantiate) with cryptographic session seals. See CONTRIBUTING.md for build/test instructions, the pre-commit hook setup, and the PR process. Contact the maintainers before submitting significant changes.
198 files. 264 tests. 118 governance entries. 3 languages. 16 edge types. 11 MCP tools.
The graph fuses independent evidence into stronger signals.
The engine evolves. Beliefs become graph reality. Federation crosses boundaries.