Skip to content

testing.md

Latest commit

 

History

History
148 lines (104 loc) · 3.3 KB

testing.md

File metadata and controls

148 lines (104 loc) · 3.3 KB
Raw

Testing

Integration Test Suite

Cecil ships with a full integration test that exercises every major subsystem:

npx tsx scripts/test-v2.ts

This runs 16 tests covering:

Test What it verifies
init() SQLite + Qdrant + world model initialize
getSubjectName() Name resolves from seed or world model
setSubjectName() Runtime name override works
worldModel.summary() World model query works
worldModel.entities() Entity listing
worldModel.entities('person') Entity filtering by kind
worldModel.beliefs() Belief listing (includes validFrom/validTo)
worldModel.beliefsAsOf(date) Temporal belief query
worldModel.openLoops() Open loop listing
worldModel.contradictions() Contradiction listing
recall() Memory search returns results
store() Memory storage works
recall stored memory Stored memory is immediately retrievable
chat() LLM responds with memory context
turn() Full cycle (chat + observe)
maintenance(dryRun) Maintenance pipeline runs without errors
reflect() Reflection agent generates analysis

Requirements

  • Qdrant running (docker compose up -d)
  • LLM endpoint running (local model server or cloud API)
  • Existing memory data (optional — some tests work without it)

Type Checking

npx tsc --noEmit

Must pass with zero errors before any commit.

Linting

npm run lint

Manual Verification

Verify modules load

npx tsx --eval "import { observe } from './cecil/observer'; console.log('ok')"

Verify world model tables exist

npm run world-model

Verify web app starts without seed

Rename or delete identity/seed.md, then:

npm run dev

Open http://localhost:3000 — should go straight to chat.

Verify recall works

npm run memory:inspect -- --query="test query" --window

Verify reflection

npm run reflect -- --patterns

Verify maintenance

npm run maintenance -- --dry-run
npm run maintenance -- --dry-run --semantic-dedup   # Verify semantic dedup reports processed count

Verify MCP server starts

npm run mcp

Should start without errors and wait for stdio input.

Memory Health Check

npm run memory:audit

Reports:

  • Total records and events
  • Memory types present
  • Source pipelines writing
  • Stale or low-quality records
  • Duplicate detection
  • Optional ranked recall preview

This is the first thing to check when Cecil's answers feel generic or blank.

Verify temporal beliefs

npm run world-model -- --beliefs    # Should show from:/to: fields on beliefs

Verify domain tagging

After a conversation, check that memories have domain tags:

npm run memory:inspect              # Look for domain field in output

Verify exchange pairs

After a multi-turn conversation, check for exchange-pair memory keys:

sqlite3 memory/structured-memory.sqlite "SELECT memory_key FROM memory_current WHERE memory_key LIKE '%pair%' LIMIT 5"

Testing Changes to Recall

If you modify recall behavior, verify with:

GET /api/memory?query=your+query&includeWindow=true

This shows the exact recall window Cecil would use for that query, including evidence tiers and source labels.