four-opencode-brain — Evolution Roadmap

Canonical plan for eliminating duplication, extracting atomic modules, and hardening reliability.

Wave Ordering (Revised)

A2 (atomic dedup + module extraction) → A5 (open-source readiness) → A6 (memory fixes) → A7+ (further)

A4 is done — unified updateStatus() with session-scoped status files (#69, #70).
A3 is obsolete — no blocked-directory scanning bug.
A1 is merged into A2 — library extraction now part of the atomic module plan.

Wave A2 — Atomic Dedup + Module Extraction (FIRST)

Priority: HIGH. Eliminates duplication, extracts reusable npm packages, makes the codebase modular.

Background

Brain currently has two types of duplication:

opencode-native duplication — spinner array, hardcoded colors (can use opencode's built-in components)
Monolith architecture — single package with 4 engines (ingest, search, memory, knowledge) + TUI + hooks, making it hard to reuse individual engines

A2 subdivided into two phases:

Phase 1: Spinner/Color Dedup (A2.1–A2.3)

#	Task	Current	Target
A2.1	Replace file polling with TuiEventBus	`setInterval(poll, 200ms)` reading status JSON file	`api.event.on('brain:status', handler)` — push-based
A2.2	Use opencode `<Spinner>`	Handwritten braille `["⠋","⠙","⠹","⠸","⠼","⠴","⠦","⠧","⠇","⠏"]` + manual `spin++`	Import `<Spinner>` from opencode TUI
A2.3	Use `api.theme` colors	Hardcoded `setFg(GREEN)`, `setFg(RED)`, `setFg(YELLOW)`	`theme().success`, `theme().error`, `theme().warning`

Phase 2: Atomic Module Extraction (A2.4)

Each engine becomes its own npm package under @four-bytes/ scope:

@four-bytes/brain-core              — DB schema init, LRU cache, logger, shared utils
@four-bytes/brain-ingest            — File walker, content-hash dedup, chunker, embed pipeline
@four-bytes/brain-search            — FTS5+vec0 hybrid search, query parser, FTS sanitizer
@four-bytes/brain-memory            — Memory CRUD (add/search/list/forget) + diary
@four-bytes/brain-knowledge         — KB entries, confidence gating, REGATE lifecycle
@four-bytes/brain-hooks             — System prompt generator, auto-capture triggers
@four-bytes/brain-tui               — SolidJS BrainStatusBar component, spinner/color integration
@four-bytes/opencode-plugin-lib     — Toast wrapper (createToast), shared plugin utilities

The main @four-bytes/four-opencode-brain becomes a thin composer that imports all modules and wires them together:

// four-opencode-brain.ts (post-extraction)
import { initBrainDatabase } from "@four-bytes/brain-core";
import { ingestPath } from "@four-bytes/brain-ingest";
import { brainSearch } from "@four-bytes/brain-search";
import { memoryAdd, memorySearch } from "@four-bytes/brain-memory";
import { kbAdd, kbSearch } from "@four-bytes/brain-knowledge";
import { brainSystemPrompt } from "@four-bytes/brain-hooks";
import { createToast } from "@four-bytes/opencode-plugin-lib";
import { BrainStatusBar } from "@four-bytes/brain-tui";

Extraction Order (dependency-driven)

1. opencode-plugin-lib  →  Toast wrapper (no deps)
2. brain-core           →  Schema, cache, logger, shared (no deps)
3. brain-ingest         →  Depends on core (DB, cache)
4. brain-search         →  Depends on core (DB)
5. brain-memory         →  Depends on core (DB)
6. brain-knowledge      →  Depends on core (DB)
7. brain-hooks          →  Depends on memory + knowledge
8. brain-tui            →  Depends on core (project-agnostic)
9. four-opencode-brain  →  Composer (depends on all)

Tasks

A2.1 — Replace file polling with TuiEventBus

In src/status.ts: publish event on bus instead of writeFileSync()
In src/tui.tsx: subscribe to event bus instead of setInterval(poll, 200ms)
Remove POLL_MS, setInterval, onCleanup(clearInterval) polling loop
Remove file-based writeFileSync / readFile status mechanism
Verification: Status bar updates in real time. Note: file I/O retained as opencode runs server/TUI in separate Worker contexts — event bus is fast-path within same context, file is cross-context fallback.

A2.2 — Use opencode `<Spinner>` component

Remove const SPINNER = ["⠋","⠙",…] + spin variable from tui.tsx
Use opencode's <Spinner /> component (verify import path from @opentui/solid)
Wire spinner visibility to data.phase === 'busy' or equivalent
Verification: Spinner animates identically to opencode's native spinners

A2.3 — Use `api.theme` colors

Replace all setFg(GREEN) / setFg(RED) / setFg(YELLOW) with theme equivalents
Map: GREEN → theme().success, RED → theme().error, YELLOW/ORANGE → theme().warning
Map: MUTED → theme().textMuted, accent pulse → theme().accent
Remove hardcoded color constants
Verification: Colors match the user's opencode theme

A2.4 — Extract atomic npm packages

Acceptance Criteria (A2)

No handwritten spinner array in brain code
No hardcoded color constants — uses api.theme
No polling loop in TUI — uses TuiEventBus
All 8 sub-packages extracted and independently buildable
four-opencode-brain composer passes all tests
Each package has its own package.json, tsconfig.json, build script

Wave A4 — Unified Status-Update Function ✅

DONE (#69, #70). Single updateStatus() function with directory-scoped status files.

What was built:

src/status.ts — updateStatus(state, opts?) supporting busy/success/warning/error/ready

Session-scoped status files via getBrainStatusFile(directory) (MD5-hash)

Wired to all 10 tool paths (ingest, search, reindex, memory, KB operations)

TUI polls per-directory file via api.state.path.directory

Wave A5 — Open-Source Readiness

Priority: MEDIUM. Required before public announcement.

Tasks

A5.1 — README.md

Project description: "Unified brain plugin — SQLite DB for RAG search, memory, and knowledge base"
Installation: npm install @four-bytes/four-opencode-brain or opencode plugin marketplace
Quick start: /brain ingest, /brain search, etc.
Tool reference table (all 10 brain tools)
Architecture diagram (three engines: ingest, search, memory)
Configuration (env vars: BRAIN_AUTO_INGEST, BRAIN_DEBUG)
Requirements (Bun, opencode, vec0 extension)

A5.2 — CONTRIBUTING.md

Branch workflow: Issue → Branch → PR → Merge
Conventional commits: feat:, fix:, refactor:
Build discipline: bun run build after every change
Testing: bun test
PR template checklist

A5.3 — Issue Templates

.github/ISSUE_TEMPLATE/bug_report.md
.github/ISSUE_TEMPLATE/feature_request.md
.github/PULL_REQUEST_TEMPLATE.md

A5.4 — GitHub Metadata

Repository description, topics, website
About section with key features
License badge, npm version badge

Acceptance Criteria (A5)

README is complete and welcoming
CONTRIBUTING.md covers full workflow
Issue templates guide quality reports
Repository looks professional and discoverable

Wave A6 — Fix Memory Module

Priority: HIGH. Memory (diary, important updates, store) currently silently broken.

Root Causes (ranked)

#	Bug	File	Fix
1	`memories_dedup_bi` trigger silently ignores repeat inserts; `memoryAdd()` returns success with fake ID	`schema.ts:574–582`, `store.ts:85–116`	Check `result.changes` after INSERT; return error if 0
2	No data migration from `~/.four-mem/` to SQLite	Both repos	One-time migration script (personal use only)
3	`memorySearch` returns `[]` without error when no query	`store.ts:123`	Return explicit error like old plugin
4	Diary auto-capture depends on external API call that can fail silently	`four-opencode-brain.ts:738–767`	Add fallback — if API fails, create diary from event properties
5	`memoryAdd()` never verifies INSERT succeeded	`store.ts:85–116`	Check `result.changes` and connect to `updateStatus`
6	`onSessionIdle` creates knowledge entries, not memory entries	`auto-capture.ts:207–362`	Ensure memory patterns also create `memoryAdd` entries
7	Diary API requires `subMode: "add"` (non-obvious)	`four-opencode-brain.ts:486–499`	Improve API: auto-detect add vs get
8	Tool renamed from `memory` → `brain_memory`	N/A	No fix needed — no code references old name

Tasks

A6.1 — Fix dedup: validate INSERT success (Fix #1, #5)

In memoryAdd(): capture const result = db.run("INSERT INTO memories …")
If result.changes === 0, throw error: "Memory not stored: duplicate content detected"
Wire to updateStatus('warning', { text: 'Duplicate', toast: 'Memory already exists' })
Apply same pattern to diaryAdd(), kbAdd(), and any other INSERT

A6.2 — Fix empty query response (Fix #3)

In memorySearch(): if !opts.query, return structured error { error: "query required" }
Match old plugin's behavior

A6.3 — Fix diary resilience (Fix #4)

In event hook handler:
- If client.session.messages() fails → fall back to eventInput.event.properties text
- Log warning but still attempt onSessionIdle
Ensure diary entries are always created on session idle

A6.4 — Fix memory auto-capture (Fix #6)

In onSessionIdle / onChatMessage: after creating knowledge entries, also create memoryAdd entries
Pattern: decisions → memoryAdd with type: "decision" AND kbAdd with kind: "decision"
Ensure both stores are populated

A6.5 — Improve diary API (Fix #7)

Simplify: diaryGet returns today if no date specified
diaryAdd becomes simpler: brain_memory({ mode: "diary", title: "...", content: "..." }) auto-detects add vs get
Backward compatible with current API

A6.6 — Migration script (Fix #2)

Create scripts/migrate-from-four-mem.ts
Reads ~/.four-mem/MEMORY.md → parses entries → inserts into SQLite
Reads ~/.four-mem/diary/*.md → parses entries → inserts into diary_entries
One-time use, documented in README
Note: Personal script — not part of plugin runtime

Acceptance Criteria (A6)

Re-adding the same memory returns a clear error, not fake success
brain_memory({ mode: "search" }) without query returns error
Diary entries created on every session idle
Memory entries created alongside knowledge entries on auto-capture
Diary API is intuitive (auto-detect add vs get)
Migration script available for old data

Dependency Graph (Revised)

A2 (atomic dedup + modules)
 ├─► A5 (open-source readiness) — independent, can run in parallel with A2 phase 1
 └─► A6 (memory fixes) — uses modules from A2

Execution Order

A2 Phase 1 → Replace spinner, colors, polling with opencode-native APIs
A2 Phase 2 → Extract 8 atomic npm packages + refactor composer
A5 → README, CONTRIBUTING, issue templates, repo polish (parallelizable with A2 phase 2)
A6 → Fix memory dedup, search, diary, migration script

Status

Wave	Status	Issue
A2	🔄 In Progress	#71, #76, #98
A4	✅ Done	#69, #70
A5	✅ Done	#85
A6	✅ Done	#86

Historical (Completed/Obsolte)

Wave	Status	Issue
A1	Merged into A2	#49
A3	Obsolete	—

Wave A7 — Gatekeeping & Review Automation

Priority: HIGH. Automated quality gates + AI code review for every PR.

Background

Per P48 — Open Source Github Review Automation: CI, linters, and tests are already solid. Adding an AI reviewer + quality gate pattern on top ensures every PR is automatically reviewed before merge.

Setup

A7.1 — CI Workflow (GitHub Actions)

✅ .github/workflows/ci.yml — builds + runs 247 tests on every PR
Trigger: pull_request (opened, reopened, synchronize, ready_for_review) + push: main

A7.2 — AI Reviewer (cubic GitHub App)

Install cubic GitHub App on the repo
Automatic PR review per commit, inline comments, summary
Model: same as dagu reference implementation

A7.3 — Branch Protection (Quality Gates)

Require: Build & Test (CI workflow) — blocking
Require: cubic AI review — blocking from start
Require: at least 1 human review
Require: conversation resolution before merge

Acceptance Criteria (A7)

CI workflow runs on every PR (build + test)
cubic AI reviewer installed and active
Branch protection enforces both CI + AI review
No PR merges without passing gates

Status

Wave	Status	Issue
A7	✅ Done	#87

Wave: Embedding Sidecar Architecture

Status: 🔄 In Progress (EPIC #117). Replaces node-llama-cpp (in-process) with a Go sidecar binary that manages llama-server as a separate HTTP process. Split into 3 waves:

Wave 1 (#118): Go embed-sidecar binary

Wave 2 (#119): Plugin integration

Wave 3 (#120): Remove node-llama-cpp

Design Decisions (v1.7.1+)

Two modes only: hash-based (default) / Go sidecar (opt-in via BRAIN_EMBED_SIDECAR). No in-process node-llama-cpp.
Non-blocking: embed() returns hash result immediately; sidecar loads async.
Go binary: Bundled via GitHub Releases prebuilt, extracted by build.ts (same pattern as vec0.so).
Self-bootstrapping: Go binary downloads llama-server + GGUF model on first run (cached in ~/.cache/four-opencode-brain/).
Port: 8666, configurable via opencode.json and BRAIN_EMBED_SIDECAR_PORT.
Single-instance: Port bind detection — 2nd instance connects to existing sidecar.
Idle timeout: 30min without requests → auto-shutdown.
Rich /status: {phase, progress, queue_depth} — always responsive even during model loading.

Goal

OpenCode communicates with a local llama.cpp server via HTTP (OpenAI-compatible /v1/embeddings) instead of embedding node-llama-cpp in the main process. This eliminates init-race conditions, separates CPU-heavy embedding from the main event loop, and allows the sidecar to outlive individual OpenCode sessions.

Why

Problem	Sidecar Fix
`initialize()` race — multiple callers trigger parallel `getLlama()`	Single start via Promise-lock + cross-process lockfile
Ingest batch embeddings block search queries	Separate process → separate CPU core; search uses its own HTTP connection
node-llama-cpp addon conflicts in Worker threads	No addon in main process at all
Process crash takes down embeddings	Sidecar is `detached` — survives parent crash/restart

Components

Component	File	Responsibility
EmbeddingSidecarManager	`src/embed/sidecar/EmbeddingSidecarManager.ts`	Process lifecycle: spawn, health poll, restart, stop
LlamaCppEmbeddingClient	`src/embed/sidecar/LlamaCppEmbeddingClient.ts`	HTTP client for `/v1/embeddings` + `/health`
Lockfile	`src/embed/sidecar/lockfile.ts`	Cross-process start guard with PID-based stale detection
Integration	`src/embed/embeddingService.ts`	Mode switch: `OPENCODE_EMBED_SIDECAR=true` → HTTP; else legacy node-llama-cpp

Architecture

┌─────────────────────────────────┐
│  OpenCode Process (Bun)         │
│  ┌───────────────────────────┐  │
│  │ EmbeddingService          │  │
│  │  ├─ Promise-lock init    │  │
│  │  ├─ Priority queue       │  │
│  │  └─ HTTP client ─────────┼──┼──► POST /v1/embeddings
│  └───────────────────────────┘  │     GET  /health
│  ┌───────────────────────────┐  │
│  │ SidecarManager            │  │
│  │  ├─ spawn (detached)     │  │
│  │  ├─ lockfile guard       │  │
│  │  └─ health poll          │  │
│  └───────────────────────────┘  │
└─────────────────────────────────┘
         │ spawn + lock
         ▼
┌─────────────────────────────────┐
│  llama.cpp server (sidecar)     │
│  Port: 8091 (configurable)      │
│  ├─ /health    → 200/503        │
│  └─ /v1/embeddings → vectors   │
│  Model: all-MiniLM-L6-v2.Q8_0  │
└─────────────────────────────────┘

Health Model

State	/health response	Meaning
`live`	Any HTTP response	Process is running, TCP port open
`ready`	HTTP 200	Model loaded, embeddings available
`loading`	HTTP 503 + `"loading model"`	Live but not ready
`down`	Connection refused / timeout	Process not reachable

GPU Strategy

Config	Behavior
`off` (default)	Always CPU — `-ngl 0` (no GPU layers offloaded)
`on`	Try GPU first (`-ngl 999`), fallback to CPU on failure
`auto`	Detect GPU availability; if uncertain, prefer CPU

CPU always works. GPU is a bonus path with mandatory fallback.

Lockfile

Path: /tmp/opencode-embeddings-sidecar.lock
Content: { pid, createdAt, port, binary, model }
Stale detection: PID dead OR lock > 30s old → acquire
Release: only if our PID matches

Configuration (env vars)

Variable	Default	Description
`OPENCODE_EMBED_SIDECAR`	`false`	Enable sidecar mode
`OPENCODE_EMBED_HOST`	`127.0.0.1`	Sidecar bind address
`OPENCODE_EMBED_PORT`	`8091`	Sidecar port
`OPENCODE_EMBED_MODEL`	`~/.cache/.../all-MiniLM-L6-v2.Q8_0.gguf`	Model path
`OPENCODE_EMBED_LLAMA_SERVER`	`llama-server` from PATH	Binary path
`OPENCODE_EMBED_GPU`	`auto`	GPU mode
`OPENCODE_EMBED_GPU_LAYERS`	`999`	GPU layers to offload
`OPENCODE_EMBED_START_TIMEOUT_MS`	`15000`	Max wait for process start
`OPENCODE_EMBED_READY_TIMEOUT_MS`	`120000`	Max wait for model load
`OPENCODE_EMBED_LOG`	`/tmp/opencode-embed-sidecar.log`	Sidecar log file

Integration Plan

EmbeddingService gains sidecar and sidecarClient fields
initialize(): when OPENCODE_EMBED_SIDECAR=true, create SidecarManager + Client instead of loading node-llama-cpp
embedDirect(): swap this.ctx.getEmbeddingFor() → sidecarClient.embed([text])
dispose(): add sidecar.stop() or leave running (detached)
Legacy mode (node-llama-cpp) preserved as default until sidecar is stable

Test Plan

Unit: Lockfile stale detection, status parsing, start-guard Promise dedup
Smoke: Start sidecar → poll /health → POST /v1/embeddings → verify dimensions
Manual: CPU-only test, simulated crash recovery, optional GPU test

Acceptance Criteria

Sidecar starts on first embed call, survives parent restart
Search queries are never blocked by ingest (separate HTTP connections)
Lockfile prevents duplicate sidecar processes
GPU failure gracefully falls back to CPU
Legacy mode continues working unchanged

FilesExpand file tree

ROADMAP.md

Latest commit

History

ROADMAP.md

File metadata and controls

four-opencode-brain — Evolution Roadmap

Wave Ordering (Revised)

Wave A2 — Atomic Dedup + Module Extraction (FIRST)

Background

A2 subdivided into two phases:

Phase 1: Spinner/Color Dedup (A2.1–A2.3)

Phase 2: Atomic Module Extraction (A2.4)

Extraction Order (dependency-driven)

Tasks

A2.1 — Replace file polling with TuiEventBus

A2.2 — Use opencode <Spinner> component

A2.3 — Use api.theme colors

A2.4 — Extract atomic npm packages

Acceptance Criteria (A2)

Wave A4 — Unified Status-Update Function ✅

Wave A5 — Open-Source Readiness

Tasks

A5.1 — README.md

A5.2 — CONTRIBUTING.md

A5.3 — Issue Templates

A5.4 — GitHub Metadata

Acceptance Criteria (A5)

Wave A6 — Fix Memory Module

Root Causes (ranked)

Tasks

A6.1 — Fix dedup: validate INSERT success (Fix #1, #5)

A6.2 — Fix empty query response (Fix #3)

A6.3 — Fix diary resilience (Fix #4)

A6.4 — Fix memory auto-capture (Fix #6)

A6.5 — Improve diary API (Fix #7)

A6.6 — Migration script (Fix #2)

Acceptance Criteria (A6)

Dependency Graph (Revised)

Execution Order

Status

Historical (Completed/Obsolte)

Wave A7 — Gatekeeping & Review Automation

Background

Setup

A7.1 — CI Workflow (GitHub Actions)

A7.2 — AI Reviewer (cubic GitHub App)

A7.3 — Branch Protection (Quality Gates)

Acceptance Criteria (A7)

Status

Wave: Embedding Sidecar Architecture

Design Decisions (v1.7.1+)

Goal

Why

Components

Architecture

Health Model

GPU Strategy

Lockfile

Configuration (env vars)

Integration Plan

Test Plan

Acceptance Criteria

A2.2 — Use opencode `<Spinner>` component

A2.3 — Use `api.theme` colors