Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
520f5d4
feat: Dynamic Database Switching & Neo4j Integration
SamanPandey-in Apr 2, 2026
43b2f4f
chore: neo4j migrations
SamanPandey-in Apr 2, 2026
2bc5c9e
refactor: Persistence to use graphRepo & Neo4jSeed to use Neo4jDriver
SamanPandey-in Apr 2, 2026
7d062e5
chore: Supervisor integration with graphRepo
SamanPandey-in Apr 2, 2026
b4d9f7d
chore: added neo4j thresholds
SamanPandey-in Apr 15, 2026
71b43bd
feat: added Neo4j DB for dynamic switching bw neo and pg
SamanPandey-in Apr 29, 2026
45f04e7
fix: neo4j aura driver and startup, added db_type in pg
SamanPandey-in May 2, 2026
c8abf86
chore: added optional docker file for running neo4j service on docker
SamanPandey-in May 2, 2026
e4d1fe6
docs: add audit report explaining all new flow and configs for dynami…
SamanPandey-in May 2, 2026
162e1c3
fix: all high priority bug fixes in agents for Neo4j and Pg switch
SamanPandey-in May 2, 2026
70430a1
fix: heatmap route now rejects malformed job IDs before the Postgres …
SamanPandey-in May 2, 2026
227dcae
fix: normalises upstream AI provider errors (429, 503, 401) into user…
SamanPandey-in May 3, 2026
e99aaab
fix: stream caching, lineStart/lineEnd, question+snippet length limits
SamanPandey-in May 3, 2026
ec593e8
fix: debounce, better error message, reset impact state in AIPanel
SamanPandey-in May 3, 2026
375882d
fix: ImpactPanel now has credentials:'include', error message parsing
SamanPandey-in May 3, 2026
7c7753d
fix: graph heatmap has now COALESCE on both sides
SamanPandey-in May 3, 2026
be8cc26
fix: removed _saveQuery which was causing duplicate entries
SamanPandey-in May 3, 2026
01407ae
refactor: changed /health endpoint for better details
SamanPandey-in May 3, 2026
d090940
docs: add bug fixes guide and implementation status
SamanPandey-in May 3, 2026
5ef6a1f
style: improved the UI of the dashboard page, cache metrics now only …
SamanPandey-in May 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
454 changes: 269 additions & 185 deletions client/src/features/ai/components/AiPanel.jsx

Large diffs are not rendered by default.

429 changes: 189 additions & 240 deletions client/src/features/dashboard/pages/DashboardPage.jsx

Large diffs are not rendered by default.

43 changes: 33 additions & 10 deletions client/src/features/graph/pages/ImpactPanel.jsx
Original file line number Diff line number Diff line change
Expand Up @@ -65,32 +65,49 @@ function ImpactGroup({ title, nodes, config }) {
}

export default function ImpactPanel() {
const graphData = useSelector(selectGraphData);
const graphData = useSelector(selectGraphData);
const selectedNodeId = useSelector(selectSelectedNodeId);
const jobId = graphData?.jobId;
const jobId = graphData?.jobId;

const [impact, setImpact] = useState(null);
const [impact, setImpact] = useState(null);
const [loading, setLoading] = useState(false);
const [error, setError] = useState('');
const [error, setError] = useState('');

const apiBase = import.meta.env.VITE_API_BASE_URL || 'http://localhost:5000';
// BUG 2 FIX: use VITE_API_BASE_URL consistently — never hardcode localhost
const apiBase = import.meta.env.VITE_API_BASE_URL || '';

async function runImpact() {
if (!jobId || !selectedNodeId) return;

setLoading(true);
setError('');
setImpact(null);

try {
// BUG 2 FIX: credentials:'include' sends the httpOnly JWT cookie.
// Without this the request returns 401 silently on every call.
const response = await fetch(
`${apiBase}/api/graph/${jobId}/impact?node=${encodeURIComponent(selectedNodeId)}&hops=6`,
{
credentials: 'include',
headers: { 'Content-Type': 'application/json' },
},
);

if (!response.ok) {
throw new Error(await response.text());
// BUG 10 FIX: parse JSON error body instead of showing raw HTML via response.text()
let msg = `Impact analysis failed (HTTP ${response.status})`;
try {
const body = await response.json();
if (body?.error) msg = body.error;
} catch {
// Non-JSON response — keep the generic message above
}
throw new Error(msg);
}

setImpact(await response.json());
const data = await response.json();
setImpact(data);
} catch (err) {
setError(err.message || 'Impact analysis failed.');
} finally {
Expand All @@ -116,7 +133,7 @@ export default function ImpactPanel() {
{selectedNodeId}
</div>
<Button size="sm" onClick={runImpact} disabled={loading} className="gap-2">
{loading ? 'Analysing...' : 'Run Impact Analysis'}
{loading ? 'Analysing' : 'Run Impact Analysis'}
<Zap className="size-3" />
</Button>
</>
Expand All @@ -134,10 +151,16 @@ export default function ImpactPanel() {
<div className="mb-3 flex items-center gap-2 text-xs text-muted-foreground">
<GitBranch className="size-3" />
<span>{impact.totalImpacted} total nodes impacted</span>
<span className="ml-auto opacity-50">via {impact.source}</span>
{impact.source && (
<span className="ml-auto opacity-50">via {impact.source}</span>
)}
</div>

<ImpactGroup title={SEVERITY_CONFIG.direct.label} nodes={impact.direct} config={SEVERITY_CONFIG.direct} />
<ImpactGroup
title={SEVERITY_CONFIG.direct.label}
nodes={impact.direct}
config={SEVERITY_CONFIG.direct}
/>
<ImpactGroup
title={SEVERITY_CONFIG.nearTransitive.label}
nodes={impact.nearTransitive}
Expand Down
4 changes: 3 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ services:
- "6379:6379"

backend:
build: ./server
build:
context: ./server
dockerfile: Dockerfile
container_name: codegraph-backend
depends_on:
- postgres
Expand Down
Binary file added docs/codegraph-phase5-complete-bugfix-guide.docx
Binary file not shown.
103 changes: 103 additions & 0 deletions docs/dynamicDb/Neo4j.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# Neo4j Migrations Guide

This document explains how Neo4j migrations work in this repository and how to run them safely.

Location
- Migration runner: `server/src/infrastructure/db/migrate.js`
- Neo4j driver: `server/src/infrastructure/db/neo4jDriver.js`
- Migration files directory: `server/src/infrastructure/db/migrations/`

Prerequisites
- Set Neo4j connection environment variables in `server/.env` (or system env):
- `NEO4J_URI` (e.g. `neo4j+s://<your-instance>.databases.neo4j.io`)
- `NEO4J_USERNAME` (e.g. `neo4j`)
- `NEO4J_PASSWORD`
- Ensure `neo4j-driver` is installed (it's listed in `server/package.json`).
- If running inside Docker Compose, the backend container must have network access to the Neo4j host.

Migration file conventions
- Files live in `server/src/infrastructure/db/migrations/`.
- File name format: `V001__description.cypher` (version prefix, two underscores, human description).
- Version token is the portion before the first `__` and must be unique (e.g. `V001`, `V002`).
- Files must contain valid Cypher statements. Multiple statements may be separated by one or more blank lines.
- The runner will:
1. Create a uniqueness constraint on `:__Neo4jMigration.version` (if missing).
2. Read `.cypher` files sorted by filename.
3. Skip versions already recorded in the database.
4. Apply new files and record them as applied by creating `(:__Neo4jMigration { version, filename, appliedAt })` nodes.

Creating a migration example
- Path: `server/src/infrastructure/db/migrations/V001__create_constraints.cypher`

Example contents:

```
CREATE CONSTRAINT IF NOT EXISTS FOR (f:CodeFile) REQUIRE (f.jobId, f.path) IS NODE KEY

CREATE CONSTRAINT IF NOT EXISTS FOR (m:__Neo4jMigration) REQUIRE m.version IS UNIQUE
```

Running migrations locally (Node)
1. From the repository root run (Esm dynamic import):

```bash
node -e "import('./server/src/infrastructure/db/migrate.js').then(m => m.runMigrations()).catch(e=>{console.error(e); process.exit(1)})"
```

2. Or create an npm script in `server/package.json` (example):

```json
"scripts": {
"neo4j:migrate": "node -e \"import('./src/infrastructure/db/migrate.js').then(m=>m.runMigrations())\""
}
```

Running migrations inside Docker Compose
- If using `docker compose` (backend service name `backend` or `codegraph-backend`), run:

```bash
# run migrations inside the backend container
docker compose exec backend node -e "import('./src/infrastructure/db/migrate.js').then(m=>m.runMigrations()).catch(e=>{console.error(e); process.exit(1)})"

# or using the built image/container name
docker exec -it codegraph-backend node -e "import('./src/infrastructure/db/migrate.js').then(m=>m.runMigrations()).catch(e=>{console.error(e); process.exit(1)})"
```

Automatic migrations at runtime
- The `SupervisorAgent` calls `runMigrations()` automatically when a job selects Neo4j as the repository backend. This means migrations are applied before seeding large graphs when required.

Verifying applied migrations
- Use Cypher to inspect the `__Neo4jMigration` records:

```cypher
MATCH (m:__Neo4jMigration) RETURN m.version AS version, m.filename AS file, m.appliedAt ORDER BY m.appliedAt;
```

Troubleshooting
- **Connection failed with encryption error**: The migration runner now automatically loads `.env` via `dotenv`. Ensure environment variables are set correctly.
- **"Comment-only statements" parsing error**: Comment lines (`//` and `/*`) are automatically filtered by the runner. No action needed.
- Unauthorized / authentication failure: verify `NEO4J_URI`, `NEO4J_USERNAME`, `NEO4J_PASSWORD` and network access.
- Statement failure: the runner logs which statement failed and throws; fix the Cypher and re-run. Applied migrations are recorded; fix the failing migration file and re-run (runner will skip applied versions).
- If you need to re-apply a migration for testing, you can delete the corresponding `__Neo4jMigration` node first (use with caution):

```cypher
MATCH (m:__Neo4jMigration { version: 'V001' }) DETACH DELETE m;
```

## Status ✅

**Migration System**: OPERATIONAL
- Environment variables properly loaded via `dotenv`
- Comment lines automatically filtered from Cypher files
- Schema successfully created in Neo4j Aura
- All constraints and indexes in place
- Integration tests passing (4/4)

Best practices
- Keep migrations small and idempotent where possible (`CREATE CONSTRAINT IF NOT EXISTS`, `MERGE` instead of `CREATE`).
- Use explicit version prefixes and increment strictly.
- Test Cypher statements in the Neo4j Browser or Aura Console before adding them to migration files.

Change Log
- 2026-04-29: Migrations system operational. Fixed dotenv loading in migrate.js. Added comment filtering for .cypher files. Schema successfully created with 4 constraints and 10 indexes.

136 changes: 136 additions & 0 deletions docs/dynamicDb/VALIDATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# Validation Summary - Dynamic Database Switching Implementation

**Date:** April 29, 2026
**Status:** ✅ COMPLETE AND VALIDATED

## Test Results

### Dynamic Database Selection Tests
```
✔ selectDatabase prefers Postgres for small graphs
✔ selectDatabase switches to Neo4j for large topology signals
✔ selectDatabase respects manual overrides
✔ createGraphRepository follows the selection result

Tests: 4 passed (4)
Duration: 1.29s
```

### Unit Tests (Vitest)
```
✓ src/agents/core/__tests__/confidence.test.js (8 tests)
✓ src/agents/parser/__tests__/ParserAgent.test.js (1 test)
✓ src/agents/graph/__tests__/GraphBuilderAgent.test.js (1 test)
✓ src/agents/core/__tests__/SupervisorAgent.test.js (4 tests)

Test Files: 4 passed (4)
Tests: 14 passed (14)
Duration: 4.49s
```

### Code Style & Formatting
```
✅ Prettier: All 7 modified files pass formatting check
✅ No lint errors in backend code
✅ No type errors
```

## Implementation Checklist

### Core Changes ✅
- [x] Database selector logic aligned with documented thresholds
- [x] Graph topology metrics computation (relationshipTypeCount, largestCycleSize)
- [x] Neo4j repository constructor flexibility
- [x] Postgres connection pool configuration
- [x] Infrastructure bootstrap probe

### Files Modified (7)
1. `server/src/infrastructure/db/dbSelector.js` - Selector logic
2. `server/src/agents/graph/GraphBuilderAgent.js` - Topology metrics
3. `server/src/infrastructure/db/Neo4jGraphRepository.js` - Constructor flexibility
4. `server/src/infrastructure/connections.js` - Pool configuration
5. `server/src/infrastructure/db/startup.js` - Bootstrap probe (NEW)
6. `server/index.js` - Server initialization
7. `server/test/dynamic-db-selection.test.js` - Regression tests (NEW)

### Files Created (2)
1. `server/src/infrastructure/db/startup.js` - Infrastructure bootstrap
2. `server/test/dynamic-db-selection.test.js` - Regression test suite

### Files Documented (2)
1. `docs/dynamicDb/graph_infrastructure.md` - Architecture documentation
2. `docs/dynamicDb/implementation.md` - Implementation report (NEW)

## Key Guarantees

✅ **No Breaking Changes**
- Backward compatible with existing Postgres-only deployments
- All persistence methods remain identical
- Topology aliases support gradual migration

✅ **Safety & Reliability**
- Neo4j is optional, not a blocker
- Non-blocking fallback to Postgres on Neo4j unavailability
- Bootstrap probe catches misconfigurations at startup
- Data never splits between stores

✅ **Code Quality**
- All tests pass
- All code passes Prettier formatting
- No lint errors or type errors
- Safe parsing with Number.isFinite() guards

## Decision Logic

The selector uses these thresholds to choose Neo4j:

| Metric | Threshold | Impact |
|--------|-----------|--------|
| nodeCount | ≥ 500 | Large codebases |
| edgeCount | ≥ 2,000 | Dense graphs |
| density | ≥ 0.05 | Highly connected modules |
| cyclesDetected | ≥ 20 | Circular dependencies |
| largestCycleSize | > 50 | Large cycles |
| relationshipTypeCount | > 3 | Multiple edge types |
| impactAnalysisDepth | > 5 | Deep impact analysis |

## Deployment Notes

### For Operations
```env
# Required (always)
DATABASE_URL=postgres://postgres:postgres@localhost:5433/codegraph

# Optional (for large repos)
NEO4J_URI=neo4j+s://<your-instance>.databases.neo4j.io
NEO4J_USER=neo4j
NEO4J_PASSWORD=your-password

# Tuning (optional)
PG_POOL_MAX=20
```

### Startup Output
```
[GraphInfrastructure] Postgres OK
[GraphInfrastructure] Neo4j connected
```
or
```
[GraphInfrastructure] Postgres OK
[GraphInfrastructure] Neo4j unavailable - falling back to Postgres
```

## What's Working

1. **Automatic Selection**: Graph size automatically selects best database
2. **Manual Overrides**: Testing can force Postgres or Neo4j if needed
3. **Backward Compatibility**: Topology aliases support legacy code
4. **Flexible Dependency**: Neo4j repository auto-imports pgPool if needed
5. **Bootstrap Verification**: Infrastructure health checked at startup
6. **Connection Pooling**: Postgres pool size configurable for production

## Sign-Off

All changes implemented, tested, formatted, and documented.
Ready for production deployment.
Binary file not shown.
Loading
Loading