diff --git a/cludemem/.gitignore b/cludemem/.gitignore new file mode 100644 index 000000000..5ca11dc0e --- /dev/null +++ b/cludemem/.gitignore @@ -0,0 +1,10 @@ +# Model binaries and training outputs — never commit these. +# Corpora (*.jsonl) are handled by data/.gitignore (sample.jsonl whitelisted). +*.gguf +*.safetensors +*.pt +*.bin +*-lora/ +checkpoints/ +runs/ +__pycache__/ diff --git a/cludemem/README.md b/cludemem/README.md new file mode 100644 index 000000000..93082df48 --- /dev/null +++ b/cludemem/README.md @@ -0,0 +1,129 @@ +# CludeMem — training pipeline + +A small, open-weights memory model: fine-tune **Gemma 4 E4B** (Apache 2.0) into a +specialist that runs the agent-memory lifecycle locally (classify, extract, +entities/relations, temporal, consolidate, reconcile, query, answer-with-abstention). +Distributed via Ollama; drops into Clude by setting two env vars. + +Design + claim + benchmark plan: `specs/research/2026-06-13-clude-memory-model/design.md`. +Host integration (already shipped in this repo): `docs/integrations/local-memory-contract.md`. + +## Pipeline + +``` +life scripts ──► data engine (TS, here) ──► train_qlora.py (GPU) ──► GGUF ──► Ollama ──► Clude + planted derive + verify Unsloth QLoRA merge pull MEMORY_MODEL=... + ground truth (offline or teacher) (E4B, ~$100-150) +``` + +| Stage | Where it runs | Cost | +|-------|---------------|------| +| Data engine (`data-engine/`) | here, Node/tsx — offline & deterministic | $0 (templated) / ~$200-500 (teacher, at 100K scale) | +| Training (`training/`) | a GPU box (RunPod H100 ~$2/hr) | ~$100-150 per run | +| Packaging (`packaging/`) | a box with Ollama | $0 | +| Eval (`eval/`) | a box with Ollama + the model | $0 (local) | + +## 1. Generate data + +**Offline smoke (no API, runs anywhere):** +```bash +npx tsx cludemem/data-engine/generate.ts # writes data/sample.jsonl, self-checks +``` +This derives examples from the planted life scripts via the deterministic +`TemplateRenderer` and runs the verification gauntlet. Labels are exact by +construction — the script IS the ground truth, so the model can't be taught a +wrong label. Running it is the test (it asserts 0 gauntlet rejections + full task +coverage + abstention examples). For the full set of v1 invariants (temporal +links, length stratification, hard-negative variety, decontamination, and the +difficulty-quota ratios), run the dedicated check: + +```bash +npx tsx cludemem/data-engine/engine.test.ts +``` + +**Scale the count (offline, $0):** the volume lever is generating more scripts. +```bash +npx tsx cludemem/data-engine/generate.ts --count 3000 --out train.jsonl # ~100K examples +``` +Each script yields ~35 examples (3000 → ~103K). `script-generator.ts` assembles +diverse personas+timelines combinatorially while GUARANTEEING the hard structures +(a supersession, a contradiction, a temporal chain, 2 hard-negative unanswerables) +with exact labels. It's seeded/deterministic — use different `--seed` values for +disjoint dev/test shards (decontamination). To add variety, extend the pools and +archetypes in `script-generator.ts`, or add hand-authored scripts to `SEED_SCRIPTS`. + +**Add naturalness (teacher-rendered):** the `--teacher` flag renders each planted +fact as natural dialogue via a teacher API. Creds live in the gitignored `.env`: +`TEACHER_API_KEY`, `TEACHER_BASE_URL`, `TEACHER_MODEL`. Labels never depend on +phrasing, so they stay exact. +```bash +npx tsx cludemem/data-engine/generate.ts --count 3000 --teacher --out train.jsonl +``` +PERMITTED teachers only — `teacher.ts` hard-refuses GPT/Claude/Gemini (their terms +forbid training a competing model on their outputs; design Section 6.2). Clean +options: DeepSeek (MIT, distillation explicitly allowed), Qwen/Mistral (Apache), +Kimi K2. **On Ollama Cloud, do NOT use the Qwen3 models for rendering** — they are +reasoning models that burn ~2,800 tokens to paraphrase one line (~120x waste). +Use a fast non-thinking instruct model: `ministral-3:8b` (Apache, ~24 tok/render) +or `deepseek-v3.2` (MIT). Teacher data gets ~10% gauntlet rejections (rendering +drift drops a proper noun, breaking grounding) vs 0% on template — that's the +gauntlet filtering bad examples, exactly as intended. + +## 2. Train (GPU) + +```bash +cd cludemem/training && pip install -r requirements.txt +python train_qlora.py --data ../data/train.jsonl --base unsloth/gemma-4-E4B-it \ + --out ./cludemem-e4b-lora --epochs 2 --export-gguf +``` +QLoRA r32 (α=2r), cosine lr 2e-4, 2 epochs, max-seq 16k. Versions are pinned to +carry the Gemma 4 fixes (grad-accum loss explosion; E2B/E4B use_cache gibberish). +Watch early loss ~13-15, not 300+. `--export-gguf` writes merged q4_k_m + q8_0. + +## 3. Package + publish + +```bash +cd cludemem/packaging +./build_and_push.sh cludemem-e4b ../training/cludemem-e4b-lora/unsloth.Q4_K_M.gguf clude +# -> ollama pull clude/cludemem-e4b +``` +Ships **merged weights** (not adapters). Apache 2.0; include the standard NOTICE. + +## 4. Evaluate + +```bash +npx tsx cludemem/eval/memopseval.ts --model cludemem-e4b --data cludemem/data/heldout.jsonl +``` +Calls the local model per example and scores per-task accuracy, schema-adherence, +and abstention calibration against the planted gold. This is the gate every +training iteration must pass before any public benchmark run (design Section 8.1). + +## 5. Activate in Clude + +```bash +MEMORY_MODEL_PROVIDER=ollama MEMORY_MODEL=cludemem-e4b # + EMBEDDING_PROVIDER=ollama for full offline +``` +or `new Cortex({ localModel: { model: 'cludemem-e4b' } })`. Memory ops route to +CludeMem with graceful frontier fallback; personality stays on frontier. + +## Status + +- ✅ **Data engine (v1)** — built, runs, self-validated offline. The real IP. + 9 of 10 task types generated (RERANK deferred; design Section 8.1 ships it + gated/optional). Difficulty quotas, length stratification, hard-negative + variety, register noise, and a decontamination guard are enforced; + `engine.test.ts` locks the invariants. +- ✅ **Training / packaging / eval scripts** — written, runnable on a GPU/Ollama box. +- ⏳ **Real corpus + fine-tune** — needs a teacher API key + a GPU (your resources). +- ⏳ Open decisions before publishing (model name, dataset release): design Section 13. + +## Layout + +``` +cludemem/ + data-engine/ taxonomy.ts life-script.ts script-generator.ts render.ts noise.ts derive.ts gauntlet.ts generate.ts teacher.ts engine.test.ts + training/ train_qlora.py requirements.txt + packaging/ Modelfile build_and_push.sh + eval/ memopseval.ts + data/ generated JSONL (gitignored except samples) +``` diff --git a/cludemem/data-engine/derive.ts b/cludemem/data-engine/derive.ts new file mode 100644 index 000000000..67fe2ea45 --- /dev/null +++ b/cludemem/data-engine/derive.ts @@ -0,0 +1,207 @@ +import type { Example, TaskName } from './taxonomy'; +import type { LifeScript, PlantedFact } from './life-script'; +import type { Renderer } from './render'; +import { applyRegisterNoise, seededRng } from './noise'; + +// ============================================================ +// Mechanical label derivation. +// +// Given a planted script + a renderer, emit supervised examples whose labels +// are EXACT by construction (read straight off the script, never guessed). +// Each task gets a fixed system prompt; the model is trained to emit the +// task's strict JSON. See taxonomy.ts for the per-task output schemas. +// ============================================================ + +const SYS: Record = { + CLASSIFY: 'Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}.', + EXTRACT: 'Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}.', + ENTITIES: 'Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}.', + TEMPORAL: 'Extract the event date and its temporal links to the known events. Output JSON: {event_date, precision, links:[{type,target}]}.', + CONSOLIDATE: 'Consolidate the memories into evidence-linked insights. Output JSON: {insights:[{content, evidence[]}]}.', + COMPACT: 'Compact the old memories into one summary, preserving entities and the date range. Output JSON: {summary, preserved_entities[], date_range:{start,end}}.', + RECONCILE: 'Decide how the two memories relate. Output JSON: {verdict, resolution, weaker_id, confidence}.', + QUERY: 'Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}.', + ANSWER: 'Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}.', +}; + +function tagsFor(fact: PlantedFact): string[] { + const names = fact.entities.map((e) => e.name.toLowerCase()); + return Array.from(new Set([...fact.concepts, ...names])).slice(0, 6); +} + +// Light default sentiment when a fact doesn't pin one: contradictions read +// mildly negative, preferences/events mildly positive, else neutral. +function defaultValence(fact: PlantedFact): number { + if (fact.contradicts) return -0.3; + if (fact.concepts.includes('preference')) return 0.3; + if (fact.memoryType === 'episodic' && fact.concepts.includes('event')) return 0.2; + return 0; +} + +function shiftDays(iso: string, days: number): string { + const d = new Date(`${iso}T00:00:00Z`); + d.setUTCDate(d.getUTCDate() + days); + return d.toISOString().slice(0, 10); +} + +export async function deriveExamples(script: LifeScript, renderer: Renderer): Promise { + const out: Example[] = []; + const push = (task: TaskName, input: string, output: unknown, meta: Partial = {}) => + out.push({ task, system: SYS[task], input, output, meta: { scriptId: script.id, ...meta } }); + + // Long scripts (many planted facts) drive the length-stratified, long-context + // examples the benchmarks actually score (design 6.1). + const isLong = script.facts.length > 20; + + // Render each fact once; build session blocks from the utterances. + const utter = new Map(); + for (const f of script.facts) utter.set(f.id, await Promise.resolve(renderer.render(f))); + const sessions = new Map(); + for (const f of script.facts) (sessions.get(f.session) ?? sessions.set(f.session, []).get(f.session)!).push(f); + + // CLASSIFY, ENTITIES — per fact. + for (const f of script.facts) { + const text = utter.get(f.id)!; + push('CLASSIFY', text, { + type: f.memoryType, + importance: f.importance, + tags: tagsFor(f), + concepts: f.concepts, + emotional_valence: f.emotionalValence ?? defaultValence(f), + }, { renderStyle: f.renderStyle }); + + if (f.entities.length) { + push('ENTITIES', text, { + entities: f.entities.map((e) => ({ name: e.name, type: e.type, aliases: e.aliases ?? [] })), + relations: (f.relations ?? []).map((r) => ({ head: r.head, type: r.type, tail: r.tail })), + }); + } + } + + // TEMPORAL — event date + ordered links to the nearest known dated events. + // Links target ids that are LISTED in the prompt (grounded), and order is read + // off the planted dates, so happens_before/after labels are exact. + const dated = script.facts + .filter((f) => f.eventDate) + .sort((a, b) => (a.eventDate! < b.eventDate! ? -1 : a.eventDate! > b.eventDate! ? 1 : 0)); + for (let i = 0; i < dated.length; i++) { + const f = dated[i]; + const earlier = dated.slice(Math.max(0, i - 3), i); + const later = dated.slice(i + 1, i + 4); + const links = [ + ...earlier.map((o) => ({ type: 'happens_after' as const, target: o.id })), + ...later.map((o) => ({ type: 'happens_before' as const, target: o.id })), + ]; + const known = [...earlier, ...later].map((o) => `[${o.id}] ${o.text} (${o.eventDate})`).join('\n'); + const input = + `Today is ${script.referenceDate}.\nNew event [${f.id}]: ${utter.get(f.id)}` + + (known ? `\nKnown events:\n${known}` : ''); + push('TEMPORAL', input, { event_date: f.eventDate, precision: f.datePrecision ?? 'day', links }); + } + + // EXTRACT — per session (dialogue block -> facts revealed in it), with the + // optional register-noise pass (labels untouched, see noise.ts). + const noiseOn = !!script.registerNoise; + for (const [s, facts] of sessions) { + let block = facts.map((f) => `${script.persona.name}: ${utter.get(f.id)}`).join('\n'); + if (noiseOn) block = applyRegisterNoise(block, seededRng(`${script.id}:${s}`)); + push('EXTRACT', block, { + memories: facts.map((f) => ({ content: f.text, summary: f.text.slice(0, 60), type: f.memoryType })), + }); + } + + // CONSOLIDATE — durable (semantic/self_model/procedural) facts as evidence-linked insights. + const durable = script.facts.filter((f) => f.memoryType !== 'episodic'); + if (durable.length) { + const corpus = script.facts.map((f) => `[${f.id}] ${f.text}`).join('\n'); + push('CONSOLIDATE', corpus, { + insights: durable.map((f) => ({ content: f.text, evidence: [f.id] })), + }); + } + + // COMPACT — squash the episodic event log into one summary, keeping entities + date span. + const episodic = script.facts.filter((f) => f.memoryType === 'episodic'); + if (episodic.length >= 2) { + const corpus = episodic.map((f) => `[${f.id}] ${f.text}`).join('\n'); + const eDates = episodic.map((f) => f.eventDate).filter((d): d is string => !!d).sort(); + const entities = Array.from(new Set(episodic.flatMap((f) => f.entities.map((e) => e.name)))); + push('COMPACT', corpus, { + summary: `${script.persona.name}: ${episodic.length} events covering ${entities.slice(0, 5).join(', ')}.`, + preserved_entities: entities, + date_range: { start: eDates[0] ?? null, end: eDates[eDates.length - 1] ?? null }, + }); + } + + // RECONCILE — supersession / contradiction / duplicate / consistent pairs. + for (const f of script.facts) { + const rel = f.supersedes + ? { older: f.supersedes, verdict: 'supersedes' as const } + : f.contradicts + ? { older: f.contradicts, verdict: 'contradicts' as const } + : f.duplicates + ? { older: f.duplicates, verdict: 'duplicate' as const } + : f.consistentWith + ? { older: f.consistentWith, verdict: 'consistent' as const } + : null; + if (!rel) continue; + const older = script.facts.find((x) => x.id === rel.older); + if (!older) continue; + const input = `Memory A [${older.id}]: ${older.text}\nMemory B [${f.id}]: ${f.text}`; + const resolution = + rel.verdict === 'supersedes' + ? 'Memory B is the current truth; Memory A is outdated.' + : rel.verdict === 'contradicts' + ? 'Memory B conflicts with Memory A; treat B as current.' + : rel.verdict === 'duplicate' + ? 'Memory B restates Memory A; keep one.' + : 'Memory A and Memory B agree; both are kept.'; + push('RECONCILE', input, { + verdict: rel.verdict, + resolution, + weaker_id: rel.verdict === 'consistent' ? null : older.id, + confidence: rel.verdict === 'contradicts' ? 0.8 : 0.9, + }); + } + + // QUERY + ANSWER — per planted QA. + for (const qa of script.qa) { + const ents = Array.from( + new Set(script.facts.flatMap((f) => f.entities).map((e) => e.name).filter((n) => qa.question.includes(n))), + ); + const isTemporal = /\b(19|20)\d{2}\b|when|date|before|after|during|since/i.test(qa.question); + const evFact = qa.evidence + .map((id) => script.facts.find((f) => f.id === id)) + .find((f): f is PlantedFact => !!(f && f.eventDate)); + const temporal_constraints: { after: string | null; before: string | null } = + isTemporal && evFact?.eventDate + ? { after: shiftDays(evFact.eventDate, -31), before: shiftDays(evFact.eventDate, 31) } + : { after: null, before: null }; + push('QUERY', `Today is ${script.referenceDate}.\n${qa.question}`, { + expanded_queries: [qa.question, qa.question.replace(/\?$/, '').trim()], + temporal_constraints, + type_filters: [], + entities: ents, + intent: isTemporal ? 'temporal_lookup' : 'lookup', + }, { answerable: qa.answerable }); + + // Retrieved context: evidence + distractors. Long scripts pull the whole + // fact set as distractors so ANSWER inputs reach the 4-16K-token band. + const pool = script.facts.filter((f) => !qa.evidence.includes(f.id)); + const distractors = isLong ? pool : pool.slice(0, 2); + const retrieved = [...qa.evidence.map((id) => script.facts.find((f) => f.id === id)!), ...distractors] + .filter(Boolean) + .map((f) => `[${f.id}] ${f.text}`) + .join('\n'); + const input = `Question: ${qa.question}\n\nMemories:\n${retrieved}`; + push( + 'ANSWER', + input, + qa.answerable + ? { rationale: `Supported by ${qa.evidence.join(', ')}.`, answer: qa.expectedAnswer ?? '', citations: qa.evidence, confidence: 0.9, abstain: false } + : { rationale: 'No provided memory answers this.', answer: 'I do not have enough information to answer that.', citations: [], confidence: 0.8, abstain: true }, + { answerable: qa.answerable }, + ); + } + + return out; +} diff --git a/cludemem/data-engine/engine.test.ts b/cludemem/data-engine/engine.test.ts new file mode 100644 index 000000000..75cb6cc80 --- /dev/null +++ b/cludemem/data-engine/engine.test.ts @@ -0,0 +1,119 @@ +import { SEED_SCRIPTS } from './life-script'; +import { generateScripts } from './script-generator'; +import { TemplateRenderer } from './render'; +import { deriveExamples } from './derive'; +import { runGauntlet } from './gauntlet'; +import * as gauntlet from './gauntlet'; +import type { Example, TaskName } from './taxonomy'; + +// ============================================================ +// CludeMem data-engine "v1" invariants (design Sections 5-6). +// +// npx tsx cludemem/data-engine/engine.test.ts +// +// No vitest in this standalone dir; this self-runs via tsx and exits non-zero +// on any failure, same idiom as generate.ts. Deterministic (seeded), so a green +// run is a real regression guard, not a flake. +// ============================================================ + +let passes = 0; +let failures = 0; +function check(name: string, cond: boolean, detail = ''): void { + if (cond) { + passes++; + console.log(` ok ${name}`); + } else { + failures++; + console.error(`FAIL ${name}${detail ? ` — ${detail}` : ''}`); + } +} + +async function buildCorpus(count: number, seed: number) { + const scripts = [...SEED_SCRIPTS, ...generateScripts(count, seed)]; + const renderer = new TemplateRenderer(); + const all: Example[] = []; + for (const s of scripts) all.push(...(await deriveExamples(s, renderer))); + return { scripts, all }; +} + +async function main() { + const { scripts, all } = await buildCorpus(150, 7); + const { kept, rejected } = runGauntlet(all); + const byTask = (t: TaskName | string) => kept.filter((e) => e.task === t); + const pct = (n: number, d: number) => (d ? `${((100 * n) / d).toFixed(0)}%` : 'n/a'); + + // Template-rendered data must be perfectly clean (labels are planted). + check('template path: 0 gauntlet rejections', rejected.length === 0, rejected.slice(0, 3).map((r) => r.reason).join(' | ')); + + // --- 1. COMPACT task exists (design Section 5, task #6) --- + check('COMPACT examples generated', byTask('COMPACT').length > 0, `got ${byTask('COMPACT').length}`); + + // --- 2. TEMPORAL links are populated (not always []) --- + const temporal = byTask('TEMPORAL'); + const withLinks = temporal.filter((e) => ((e.output as { links: unknown[] }).links?.length ?? 0) > 0); + check('TEMPORAL: some examples carry temporal links', withLinks.length > 0, `${withLinks.length}/${temporal.length} have links`); + // links must reference ids present in the prompt + use a valid bond type (gauntlet enforces; double-check here) + const linkIdsGrounded = withLinks.every((e) => + (e.output as { links: { target: string }[] }).links.every((l) => e.input.includes(`[${l.target}]`)), + ); + check('TEMPORAL: link targets are grounded in the prompt', withLinks.length > 0 && linkIdsGrounded); + + // --- 3. QUERY temporal_constraints populated for temporal questions --- + const queries = byTask('QUERY'); + const withConstraint = queries.filter((e) => { + const c = (e.output as { temporal_constraints: { after: string | null; before: string | null } }).temporal_constraints; + return !!(c && (c.after || c.before)); + }); + check('QUERY: temporal questions get non-null temporal_constraints', withConstraint.length > 0, `${withConstraint.length}/${queries.length}`); + + // --- 4. Hard-negative variety + ratio (design 6.1) --- + const negs = scripts.flatMap((s) => s.qa).filter((qa) => !qa.answerable); + const kinds = new Set(negs.map((qa) => qa.negativeKind)); + check('hard negatives include temporal_near_miss', kinds.has('temporal_near_miss')); + check('hard negatives include wrong_session', kinds.has('wrong_session')); + const hard = negs.filter((qa) => qa.negativeKind && qa.negativeKind !== 'absent'); + check('>=50% of unanswerables are hard negatives', negs.length > 0 && hard.length / negs.length >= 0.5, `${hard.length}/${negs.length} = ${pct(hard.length, negs.length)}`); + + // --- 5. Register noise on 20-30% of dialogues (design 6.1), labels still clean --- + const extracts = byTask('EXTRACT'); + const noiseMarker = /```|\| ---|\bDEBUG\b|\bWARN\b|\{"|\t/; + const noisy = extracts.filter((e) => noiseMarker.test(e.input)); + check('register noise present on a meaningful fraction of EXTRACT blocks', noisy.length / extracts.length >= 0.15, `${noisy.length}/${extracts.length} = ${pct(noisy.length, extracts.length)}`); + check('register noise stays within ~40% of dialogues', noisy.length / extracts.length <= 0.4, pct(noisy.length, extracts.length)); + + // --- 6. Length stratification: long inputs for the long-context tasks (design 6.1) --- + const longTasks = kept.filter((e) => e.task === 'CONSOLIDATE' || e.task === 'COMPACT' || e.task === 'ANSWER'); + const longOnes = longTasks.filter((e) => e.input.length >= 4000); + check('>=20% of long-context-task examples are >=4000 chars', longTasks.length > 0 && longOnes.length / longTasks.length >= 0.2, `${longOnes.length}/${longTasks.length} = ${pct(longOnes.length, longTasks.length)}`); + + // --- 7. Difficulty quota: >=40% of facts rendered non-explicitly (design 6.1) --- + const facts = scripts.flatMap((s) => s.facts); + const nonExplicit = facts.filter((f) => f.renderStyle !== 'explicit'); + check('>=40% of facts use a non-explicit render style', nonExplicit.length / facts.length >= 0.4, pct(nonExplicit.length, facts.length)); + + // --- 8. Decontamination layer exists + works (design 6.3) --- + const decon = (gauntlet as { decontaminate?: (text: string) => { ok: boolean; reason?: string } }).decontaminate; + check('decontaminate() is exported', typeof decon === 'function'); + if (typeof decon === 'function') { + check('decontaminate flags a blocklisted eval persona name', decon('Caroline went to the concert in 2023.').ok === false); + check('decontaminate passes clean synthetic text', decon('Maya moved to Lisbon on 2026-02-10.').ok === true); + } + + // --- 9. RECONCILE verdict variety (design Section 5, task #7) --- + const verdicts = new Set(byTask('RECONCILE').map((e) => (e.output as { verdict: string }).verdict)); + check('RECONCILE emits a duplicate verdict somewhere', verdicts.has('duplicate'), [...verdicts].join(',')); + check('RECONCILE emits a consistent verdict somewhere', verdicts.has('consistent'), [...verdicts].join(',')); + + // --- 10. CLASSIFY carries emotional_valence (design Section 5, task #1) --- + const classify = byTask('CLASSIFY'); + const haveValence = classify.filter((e) => typeof (e.output as { emotional_valence?: unknown }).emotional_valence === 'number'); + check('every CLASSIFY example has a numeric emotional_valence', classify.length > 0 && haveValence.length === classify.length, `${haveValence.length}/${classify.length}`); + + console.log(`\n${passes} passed, ${failures} failed`); + process.exit(failures ? 1 : 0); +} + +main().catch((err) => { + console.error(err); + process.exit(1); +}); diff --git a/cludemem/data-engine/gauntlet.ts b/cludemem/data-engine/gauntlet.ts new file mode 100644 index 000000000..2d39e9b75 --- /dev/null +++ b/cludemem/data-engine/gauntlet.ts @@ -0,0 +1,111 @@ +import { TASK_SCHEMAS, type Example } from './taxonomy'; + +// ============================================================ +// Verification gauntlet. +// +// Every example must pass before it enters the corpus. For template-rendered +// data these checks should all pass (labels are planted). For teacher-rendered +// data they catch schema drift, hallucinated citations, and abstention bugs — +// expect to discard 50-80% there (NuExtract-style rejection sampling). +// ============================================================ + +export interface GauntletResult { + ok: boolean; + reason?: string; +} + +// Known eval-set persona names + n-grams to keep OUT of training data (design +// 6.3, decontamination). Seeded with illustrative LoCoMo/LongMemEval persona +// names; the full n-gram + embedding decontamination set plugs in here in W1. +// Synthetic generator names are disjoint from this list, so template data passes +// clean. +const EVAL_NAME_BLOCKLIST = ['caroline', 'melanie', 'angela', 'joanna', 'hannah', 'gabriella']; +const EVAL_NGRAMS: string[] = []; + +/** Reject text that overlaps a known evaluation set (benchmark-leakage guard). */ +export function decontaminate(text: string): GauntletResult { + const lower = text.toLowerCase(); + for (const name of EVAL_NAME_BLOCKLIST) { + if (new RegExp(`\\b${name}\\b`).test(lower)) { + return { ok: false, reason: `decontamination: eval persona name "${name}"` }; + } + } + for (const ng of EVAL_NGRAMS) { + if (lower.includes(ng.toLowerCase())) return { ok: false, reason: 'decontamination: eval n-gram overlap' }; + } + return { ok: true }; +} + +export function validateExample(ex: Example): GauntletResult { + const schema = TASK_SCHEMAS[ex.task]; + const parsed = schema.safeParse(ex.output); + if (!parsed.success) { + return { ok: false, reason: `schema: ${parsed.error.issues[0]?.path.join('.')} ${parsed.error.issues[0]?.message}` }; + } + + // Benchmark-leakage guard on the rendered input. + const clean = decontaminate(ex.input); + if (!clean.ok) return clean; + + if (ex.task === 'ANSWER') { + const o = ex.output as { citations: string[]; abstain: boolean }; + for (const c of o.citations) { + if (!ex.input.includes(`[${c}]`)) return { ok: false, reason: `citation ${c} not grounded in context` }; + } + if (o.abstain && o.citations.length) return { ok: false, reason: 'abstain=true but has citations' }; + if (!o.abstain && o.citations.length === 0) return { ok: false, reason: 'answer without citations' }; + } + + if (ex.task === 'CONSOLIDATE') { + const o = ex.output as { insights: { evidence: string[] }[] }; + for (const ins of o.insights) { + for (const e of ins.evidence) { + if (!ex.input.includes(`[${e}]`)) return { ok: false, reason: `evidence ${e} not in corpus` }; + } + } + } + + // Groundedness: an entity label must survive into the (possibly teacher-paraphrased) + // input. Proper nouns survive paraphrase, so this catches a teacher that dropped + // an entity — the main way teacher rendering corrupts an otherwise-exact label. + if (ex.task === 'ENTITIES') { + const o = ex.output as { entities: { name: string }[] }; + for (const e of o.entities) { + if (!ex.input.toLowerCase().includes(e.name.toLowerCase())) { + return { ok: false, reason: `entity "${e.name}" not grounded in rendered input` }; + } + } + } + + // TEMPORAL: every temporal link must point at an event id present in the prompt. + if (ex.task === 'TEMPORAL') { + const o = ex.output as { links: { target: string }[] }; + for (const l of o.links) { + if (!ex.input.includes(`[${l.target}]`)) return { ok: false, reason: `temporal link target ${l.target} not in prompt` }; + } + } + + // COMPACT: preserved entities must actually appear in the compacted corpus. + if (ex.task === 'COMPACT') { + const o = ex.output as { preserved_entities: string[] }; + for (const e of o.preserved_entities) { + if (!ex.input.toLowerCase().includes(e.toLowerCase())) return { ok: false, reason: `compact entity "${e}" not in corpus` }; + } + } + + return { ok: true }; +} + +export function runGauntlet(examples: Example[]): { + kept: Example[]; + rejected: { ex: Example; reason: string }[]; +} { + const kept: Example[] = []; + const rejected: { ex: Example; reason: string }[] = []; + for (const ex of examples) { + const r = validateExample(ex); + if (r.ok) kept.push(ex); + else rejected.push({ ex, reason: r.reason! }); + } + return { kept, rejected }; +} diff --git a/cludemem/data-engine/generate.ts b/cludemem/data-engine/generate.ts new file mode 100644 index 000000000..f2e8d8ca9 --- /dev/null +++ b/cludemem/data-engine/generate.ts @@ -0,0 +1,195 @@ +import { writeFileSync, mkdirSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import dotenv from 'dotenv'; +import { SEED_SCRIPTS, type PlantedFact } from './life-script'; +import { generateScripts } from './script-generator'; +import { TemplateRenderer, TeacherRenderer, PrerenderedRenderer, type Renderer } from './render'; +import { makeTeacher } from './teacher'; +import { deriveExamples } from './derive'; +import { runGauntlet } from './gauntlet'; +import type { Example, TaskName } from './taxonomy'; +import { seededRng } from './noise'; + +// Load worktree-root .env so --teacher can read TEACHER_* creds (never committed). +dotenv.config({ path: fileURLToPath(new URL('../../.env', import.meta.url)) }); + +// --count N add N combinatorially-generated scripts (the volume lever). +// --seed S batch seed (use different seeds for disjoint shards / dev vs test). +function argNum(name: string, def: number): number { + const i = process.argv.indexOf(`--${name}`); + return i >= 0 ? Number(process.argv[i + 1]) : def; +} + +// Bounded-concurrency map: run `fn` over `items` with at most `limit` in flight. +async function mapPool(items: T[], limit: number, fn: (item: T, i: number) => Promise): Promise { + const out: R[] = new Array(items.length); + let next = 0; + const worker = async () => { + for (let i = next++; i < items.length; i = next++) out[i] = await fn(items[i], i); + }; + await Promise.all(Array.from({ length: Math.max(1, Math.min(limit, items.length)) }, worker)); + return out; +} + +// ============================================================ +// Build a CludeMem training shard from the planted scripts. +// +// npx tsx cludemem/data-engine/generate.ts +// +// Offline + deterministic (TemplateRenderer). Emits chat-format JSONL and +// runs the verification gauntlet + self-check assertions, so running it IS +// the test. For scale, swap TemplateRenderer -> TeacherRenderer (DeepSeek/Qwen) +// and multiply scripts; the gauntlet stays identical. +// ============================================================ + +function toChat(ex: Example) { + return { + messages: [ + { role: 'system', content: ex.system }, + { role: 'user', content: ex.input }, + { role: 'assistant', content: JSON.stringify(ex.output) }, + ], + task: ex.task, + meta: ex.meta, + }; +} + +// Per-task quota balancer (--balance): seeded-shuffle each task's pool and keep up +// to its target. CLASSIFY/ENTITIES are 1 per fact (abundant); COMPACT/CONSOLIDATE +// are ~1 per script (rare), so raising the rare ones needs MORE SCRIPTS (--count), +// and this trims the abundant ones down. Without it the abundant tasks swamp the +// corpus (~21x) and the model overtrains on them. Runs AFTER the gauntlet + +// self-checks, so those still validate the full generated set; balancing only +// shapes what gets written. +const BALANCE_TARGETS: Record = { + CLASSIFY: 8000, ENTITIES: 8000, TEMPORAL: 8000, + QUERY: 6000, ANSWER: 6000, EXTRACT: 5000, + RECONCILE: 3000, CONSOLIDATE: 2500, COMPACT: 2500, +}; +function balanceExamplesByTask(examples: Example[], seed: number): Example[] { + const rng = seededRng(`balance:${seed}`); + const byTask = new Map(); + for (const e of examples) (byTask.get(e.task) ?? byTask.set(e.task, []).get(e.task)!).push(e); + const out: Example[] = []; + for (const [task, pool] of byTask) { + for (let i = pool.length - 1; i > 0; i--) { + const j = Math.floor(rng() * (i + 1)); + [pool[i], pool[j]] = [pool[j], pool[i]]; + } + out.push(...pool.slice(0, BALANCE_TARGETS[task] ?? pool.length)); + } + return out; +} + +async function main() { + const count = argNum('count', 0); + const seed = argNum('seed', 0); + const useTeacher = process.argv.includes('--teacher'); + // --no-seeds excludes the 2 hand-authored SEED_SCRIPTS so a held-out shard + // (a different --seed) stays fully disjoint from the training shard. + const noSeeds = process.argv.includes('--no-seeds'); + const scripts = [...(noSeeds ? [] : SEED_SCRIPTS), ...generateScripts(count, seed)]; + + let renderer: Renderer; + if (useTeacher) { + const apiKey = process.env.TEACHER_API_KEY; + if (!apiKey) { + console.error('--teacher needs TEACHER_API_KEY in .env'); + process.exit(1); + } + const t = makeTeacher({ apiKey, baseUrl: process.env.TEACHER_BASE_URL, model: process.env.TEACHER_MODEL }); + const teacher = new TeacherRenderer(t.complete, t.model); + const fallback = new TemplateRenderer(); + const conc = argNum('concurrency', 8); + const allFacts: PlantedFact[] = scripts.flatMap((s) => s.facts); + console.log(`rendering ${allFacts.length} facts with teacher ${t.model} (concurrency ${conc})`); + let done = 0; + let failed = 0; + const rendered = await mapPool(allFacts, conc, async (f) => { + let out: string; + try { + out = await teacher.render(f); + } catch { + failed++; + out = fallback.render(f); // graceful: a failed teacher call degrades to template + } + if (++done % 1000 === 0) console.log(` rendered ${done}/${allFacts.length}${failed ? ` (${failed} fell back)` : ''}`); + return out; + }); + const map = new Map(); + allFacts.forEach((f, i) => map.set(f, rendered[i])); + if (failed) console.log(` note: ${failed}/${allFacts.length} renders fell back to template (teacher errors)`); + renderer = new PrerenderedRenderer(map); + } else { + renderer = new TemplateRenderer(); + } + + const all: Example[] = []; + for (const script of scripts) { + all.push(...(await deriveExamples(script, renderer))); + } + + const { kept, rejected } = runGauntlet(all); + + // Self-checks: the whole pipeline is correct iff these hold. + const tasksSeen = new Set(kept.map((e) => e.task)); + const expectTasks: TaskName[] = ['CLASSIFY', 'EXTRACT', 'ENTITIES', 'TEMPORAL', 'CONSOLIDATE', 'COMPACT', 'RECONCILE', 'QUERY', 'ANSWER']; + const assert = (cond: boolean, msg: string) => { + if (!cond) { + console.error(`SELF-CHECK FAILED: ${msg}`); + process.exit(1); + } + }; + assert(kept.length > 0, 'no examples produced'); + // Template data must be perfectly clean; teacher-rendered data is EXPECTED to + // have some gauntlet rejections (drift) — that's the gauntlet doing its job. + if (!useTeacher) { + assert(rejected.length === 0, `template data should fully pass the gauntlet, but ${rejected.length} rejected: ${rejected.slice(0, 3).map((r) => r.reason).join(' | ')}`); + } + for (const t of expectTasks) assert(tasksSeen.has(t), `task ${t} not represented`); + // Abstention coverage: at least one trained refusal. + assert(kept.some((e) => e.task === 'ANSWER' && (e.output as { abstain: boolean }).abstain), 'no abstention examples'); + + // v1 difficulty quotas (design 6.1) — enforced at scale, where ratios are meaningful. + if (count > 0) { + const allFacts = scripts.flatMap((s) => s.facts); + const nonExplicit = allFacts.filter((f) => f.renderStyle !== 'explicit').length; + assert(nonExplicit / allFacts.length >= 0.4, `difficulty quota: only ${((100 * nonExplicit) / allFacts.length).toFixed(0)}% of facts non-explicit (<40%)`); + + const negs = scripts.flatMap((s) => s.qa).filter((q) => !q.answerable); + const hardNeg = negs.filter((q) => q.negativeKind && q.negativeKind !== 'absent').length; + assert(negs.length > 0 && hardNeg / negs.length >= 0.5, `hard-negative quota: only ${((100 * hardNeg) / Math.max(1, negs.length)).toFixed(0)}% of unanswerables are hard negatives (<50%)`); + + const longTasks = kept.filter((e) => e.task === 'CONSOLIDATE' || e.task === 'COMPACT' || e.task === 'ANSWER'); + const longOnes = longTasks.filter((e) => e.input.length >= 4000).length; + assert(longTasks.length > 0 && longOnes / longTasks.length >= 0.2, `length stratification: only ${((100 * longOnes) / Math.max(1, longTasks.length)).toFixed(0)}% of long-context examples >=4000 chars (<20%)`); + } + + // Optional per-task balancing (--balance): shape the written shard's task mix. + const doBalance = process.argv.includes('--balance'); + const emitted = doBalance ? balanceExamplesByTask(kept, seed) : kept; + + // Write the shard. Default smoke -> sample.jsonl (committed, small); + // any scaled run -> train.jsonl (gitignored). Override with --out . + const outArgIdx = process.argv.indexOf('--out'); + const outName = + outArgIdx >= 0 ? process.argv[outArgIdx + 1] : useTeacher ? 'teacher-sample.jsonl' : count > 0 ? 'train.jsonl' : 'sample.jsonl'; + const outDir = fileURLToPath(new URL('../data/', import.meta.url)); + mkdirSync(outDir, { recursive: true }); + const outPath = `${outDir}${outName}`; + writeFileSync(outPath, emitted.map((e) => JSON.stringify(toChat(e))).join('\n') + '\n'); + + // Summary. + const perTask = expectTasks + .map((t) => `${t}=${emitted.filter((e) => e.task === t).length}`) + .join(' '); + const rate = all.length ? ((100 * rejected.length) / all.length).toFixed(1) : '0'; + console.log(`CludeMem data engine — wrote ${emitted.length} examples (${rejected.length} rejected, ${rate}%) to ${outPath}`); + console.log(` scripts: ${scripts.length} renderer: ${useTeacher ? 'teacher' : 'template'} tasks: ${perTask}`); + if (rejected.length) console.log(` sample rejections: ${rejected.slice(0, 3).map((r) => r.reason).join(' | ')}`); +} + +main().catch((err) => { + console.error(err); + process.exit(1); +}); diff --git a/cludemem/data-engine/life-script.ts b/cludemem/data-engine/life-script.ts new file mode 100644 index 000000000..965483648 --- /dev/null +++ b/cludemem/data-engine/life-script.ts @@ -0,0 +1,212 @@ +import type { Concept, DatePrecision, EntityType, MemoryType } from './taxonomy'; + +// ============================================================ +// Life scripts: the planted ground truth. +// +// A life script is a persona + a timeline of planted facts (with preference +// changes, contradictions, supersessions, temporal chains). Because the +// script IS the source of truth, every task label derives MECHANICALLY from +// it — labels are exact by construction, not a teacher's guess. Teachers +// (DeepSeek/Qwen) only RENDER natural dialogue around this structure. +// ============================================================ + +export interface PlantedEntity { + name: string; + type: EntityType; + aliases?: string[]; +} + +export interface PlantedRelation { + head: string; + type: string; // works_at | lives_in | founded | owns | knows | ... + tail: string; +} + +export type RenderStyle = 'explicit' | 'implicit' | 'hedged' | 'corrected'; + +export interface PlantedFact { + id: string; + /** Canonical statement of the fact (ground truth content). */ + text: string; + memoryType: MemoryType; + importance: number; // 0-1 + concepts: Concept[]; + entities: PlantedEntity[]; + relations?: PlantedRelation[]; + eventDate?: string; // ISO (YYYY-MM-DD); omit if not datable + datePrecision?: DatePrecision; + /** id of an earlier fact this fact replaces (newer truth). */ + supersedes?: string; + /** id of a fact this directly conflicts with. */ + contradicts?: string; + /** id of an earlier fact this one duplicates (same content restated). */ + duplicates?: string; + /** id of a fact this one corroborates without conflict (RECONCILE: consistent). */ + consistentWith?: string; + /** -1..1 sentiment of the fact (drives CLASSIFY.emotional_valence). */ + emotionalValence?: number; + /** Which session reveals this fact. */ + session: number; + /** How the renderer should surface it (difficulty control). */ + renderStyle: RenderStyle; +} + +export interface PlantedQA { + id: string; + question: string; + /** Fact ids that answer it. Empty => unanswerable (abstain). */ + evidence: string[]; + answerable: boolean; + /** Expected short answer (for answerable Qs). */ + expectedAnswer?: string; + /** Distractor type for hard negatives (unanswerable Qs). */ + negativeKind?: 'absent' | 'attribute_miss' | 'temporal_near_miss' | 'wrong_session'; +} + +export interface LifeScript { + id: string; + persona: { name: string; role: string }; + /** "Today" for temporal-relative reasoning. */ + referenceDate: string; + /** If set, the renderer injects register noise (logs/tables/code/JSON) into dialogue. */ + registerNoise?: boolean; + facts: PlantedFact[]; + qa: PlantedQA[]; +} + +// ── Seed scripts (hand-authored; the generator multiplies these for scale) ── + +export const SEED_SCRIPTS: LifeScript[] = [ + { + id: 'maya-relocation', + persona: { name: 'Maya', role: 'product designer' }, + referenceDate: '2026-03-01', + facts: [ + { + id: 'f1', + text: 'Maya lives in Berlin.', + memoryType: 'semantic', + importance: 0.6, + concepts: ['personal_fact', 'location'], + entities: [ + { name: 'Maya', type: 'person' }, + { name: 'Berlin', type: 'location' }, + ], + relations: [{ head: 'Maya', type: 'lives_in', tail: 'Berlin' }], + session: 0, + renderStyle: 'explicit', + }, + { + id: 'f2', + text: 'Maya prefers tea over coffee.', + memoryType: 'semantic', + importance: 0.4, + concepts: ['preference'], + entities: [{ name: 'Maya', type: 'person' }], + session: 0, + renderStyle: 'hedged', + }, + { + id: 'f3', + text: 'Maya moved to Lisbon on 2026-02-10.', + memoryType: 'episodic', + importance: 0.8, + concepts: ['event', 'location', 'temporal_marker'], + entities: [ + { name: 'Maya', type: 'person' }, + { name: 'Lisbon', type: 'location' }, + ], + relations: [{ head: 'Maya', type: 'lives_in', tail: 'Lisbon' }], + eventDate: '2026-02-10', + datePrecision: 'day', + supersedes: 'f1', // now lives in Lisbon, not Berlin + session: 2, + renderStyle: 'explicit', + }, + { + id: 'f4', + text: 'Maya now prefers coffee, having switched from tea.', + memoryType: 'semantic', + importance: 0.4, + concepts: ['preference'], + entities: [{ name: 'Maya', type: 'person' }], + contradicts: 'f2', + session: 3, + renderStyle: 'corrected', + }, + { + id: 'f5', + text: 'Maya started a new job at Nuvio as lead designer on 2026-02-24.', + memoryType: 'episodic', + importance: 0.85, + concepts: ['event', 'task', 'relationship'], + entities: [ + { name: 'Maya', type: 'person' }, + { name: 'Nuvio', type: 'organization' }, + ], + relations: [{ head: 'Maya', type: 'works_at', tail: 'Nuvio' }], + eventDate: '2026-02-24', + datePrecision: 'day', + session: 3, + renderStyle: 'implicit', + }, + ], + qa: [ + { id: 'q1', question: 'Where does Maya live now?', evidence: ['f3'], answerable: true, expectedAnswer: 'Lisbon' }, + { id: 'q2', question: 'Does Maya prefer tea or coffee now?', evidence: ['f4'], answerable: true, expectedAnswer: 'coffee' }, + { id: 'q3', question: 'When did Maya move to Lisbon?', evidence: ['f3'], answerable: true, expectedAnswer: '2026-02-10' }, + { id: 'q4', question: 'What is the name of Maya\'s sister?', evidence: [], answerable: false, negativeKind: 'absent' }, + { id: 'q5', question: 'Did Maya move to Madrid?', evidence: [], answerable: false, negativeKind: 'attribute_miss' }, + ], + }, + { + id: 'devon-project', + persona: { name: 'Devon', role: 'backend engineer' }, + referenceDate: '2026-05-15', + facts: [ + { + id: 'g1', + text: 'Devon is allergic to peanuts.', + memoryType: 'semantic', + importance: 0.9, + concepts: ['personal_fact'], + entities: [{ name: 'Devon', type: 'person' }], + session: 0, + renderStyle: 'explicit', + }, + { + id: 'g2', + text: 'Devon is leading the Atlas migration project.', + memoryType: 'episodic', + importance: 0.75, + concepts: ['task', 'goal_or_plan'], + entities: [ + { name: 'Devon', type: 'person' }, + { name: 'Atlas', type: 'project' }, + ], + relations: [{ head: 'Devon', type: 'leads', tail: 'Atlas' }], + eventDate: '2026-04-01', + datePrecision: 'month', + session: 1, + renderStyle: 'explicit', + }, + { + id: 'g3', + text: 'The Atlas migration shipped on 2026-05-09.', + memoryType: 'episodic', + importance: 0.8, + concepts: ['event', 'temporal_marker'], + entities: [{ name: 'Atlas', type: 'project' }], + eventDate: '2026-05-09', + datePrecision: 'day', + session: 2, + renderStyle: 'explicit', + }, + ], + qa: [ + { id: 'p1', question: 'What is Devon allergic to?', evidence: ['g1'], answerable: true, expectedAnswer: 'peanuts' }, + { id: 'p2', question: 'When did the Atlas migration ship?', evidence: ['g3'], answerable: true, expectedAnswer: '2026-05-09' }, + { id: 'p3', question: 'What database did Devon pick for Atlas?', evidence: [], answerable: false, negativeKind: 'absent' }, + ], + }, +]; diff --git a/cludemem/data-engine/noise.ts b/cludemem/data-engine/noise.ts new file mode 100644 index 000000000..852069d68 --- /dev/null +++ b/cludemem/data-engine/noise.ts @@ -0,0 +1,48 @@ +// ============================================================ +// Register noise (design Section 6.1). +// +// Real agent-memory inputs are not clean prose — they carry tool output, logs, +// markdown tables, code fences, JSON. We inject that texture into ~20-30% of +// dialogue blocks so the model learns to extract facts from messy context. +// +// Noise is added AROUND the planted utterances, never inside them, so labels +// stay exact by construction (the gauntlet's groundedness checks still hold). +// ============================================================ + +// Deterministic string-seeded PRNG so a given (scriptId, session) always noises +// the same way — reproducible corpora, no Math.random. +export function seededRng(seedStr: string): () => number { + let h = 2166136261 >>> 0; + for (let i = 0; i < seedStr.length; i++) { + h ^= seedStr.charCodeAt(i); + h = Math.imul(h, 16777619); + } + let a = h >>> 0; + return () => { + a |= 0; + a = (a + 0x6d2b79f5) | 0; + let t = Math.imul(a ^ (a >>> 15), 1 | a); + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + +const ARTIFACTS: ((r: () => number) => string)[] = [ + // code fence (tool transcript) + () => '```\n$ deploy --env prod\nbuild ok\nexit 0\n```', + // markdown table + () => '| metric | value |\n| --- | --- |\n| latency_ms | 42 |\n| status | ok |', + // log lines + (r) => `[2026-0${1 + Math.floor(r() * 8)}-15T09:00:00Z] ${r() < 0.5 ? 'DEBUG' : 'WARN'} worker: queue drained`, + // tool-output JSON + (r) => `{"tool":"search","status":"${r() < 0.5 ? 'ok' : 'WARN'}","hits":${Math.floor(r() * 5)}}`, +]; + +/** + * Wrap a rendered dialogue block in one register-noise artifact (before or + * after). The planted utterances are untouched, so extraction labels stay exact. + */ +export function applyRegisterNoise(block: string, r: () => number): string { + const artifact = ARTIFACTS[Math.floor(r() * ARTIFACTS.length)](r); + return r() < 0.5 ? `${artifact}\n${block}` : `${block}\n${artifact}`; +} diff --git a/cludemem/data-engine/render.ts b/cludemem/data-engine/render.ts new file mode 100644 index 000000000..2390624c0 --- /dev/null +++ b/cludemem/data-engine/render.ts @@ -0,0 +1,97 @@ +import type { LifeScript, PlantedFact, RenderStyle } from './life-script'; + +// ============================================================ +// Renderers turn a planted fact into a natural dialogue utterance. +// +// TemplateRenderer is deterministic + offline (for tests + a no-API smoke +// dataset). TeacherRenderer (interface) is the scale path: a DeepSeek/Qwen +// call that paraphrases the fact in the requested style. Labels never depend +// on the rendering, so swapping renderers cannot corrupt ground truth. +// ============================================================ + +export interface Renderer { + render(fact: PlantedFact): Promise | string; +} + +const STYLE_TEMPLATES: Record string> = { + // explicit: states the fact plainly + explicit: (t) => t, + // implicit: embeds it in a longer sentence, fact not the headline + implicit: (t) => `By the way, around then ${lowerFirst(t)} It came up while we were chatting.`, + // hedged: softens with uncertainty markers + hedged: (t) => `I think ${lowerFirst(t)} At least that's my sense of it.`, + // corrected: presents as a correction of a prior belief + corrected: (t) => `Actually, scratch what I said before — ${lowerFirst(t)}`, +}; + +function lowerFirst(s: string): string { + return s.length ? s[0].toLowerCase() + s.slice(1) : s; +} + +/** Deterministic, offline renderer. Same input -> same output. */ +export class TemplateRenderer implements Renderer { + render(fact: PlantedFact): string { + return STYLE_TEMPLATES[fact.renderStyle](fact.text); + } +} + +/** + * Scale-path renderer backed by a permitted teacher (DeepSeek/Qwen — never + * Claude/GPT/Gemini, per the design's license rules). The `complete` callback + * is injected so this module stays API-key-free and testable; wire it to your + * teacher client in generate.ts when running at scale. + */ +export class TeacherRenderer implements Renderer { + constructor( + private complete: (prompt: string) => Promise, + private teacher: string, + ) {} + + async render(fact: PlantedFact): Promise { + const styleHint: Record = { + explicit: 'state it plainly', + implicit: 'mention it in passing, not as the main point', + hedged: 'express it with mild uncertainty', + corrected: 'present it as correcting an earlier statement', + }; + const prompt = + `Paraphrase this fact as one natural first-person chat utterance; ${styleHint[fact.renderStyle]}. ` + + `Do not add facts not present. Fact: "${fact.text}"`; + const out = await this.complete(prompt); + return out.trim(); + } + + label(): string { + return this.teacher; + } +} + +/** + * Serves utterances pre-rendered (e.g. by a bounded concurrency pool), keyed by + * fact identity. Falls back to the template renderer for any fact not in the map + * (e.g. a teacher call that failed). Lets deriveExamples run fully synchronously + * after a single parallel rendering pass — see generate.ts mapPool. + */ +export class PrerenderedRenderer implements Renderer { + private fallback = new TemplateRenderer(); + constructor(private map: Map) {} + render(fact: PlantedFact): string { + return this.map.get(fact) ?? this.fallback.render(fact); + } +} + +/** Render every fact of a script into per-session dialogue blocks. */ +export async function renderSessions( + script: LifeScript, + renderer: Renderer, +): Promise { + const sessions: Record = {}; + for (const fact of script.facts) { + const line = await renderer.render(fact); + (sessions[fact.session] ??= []).push(`${script.persona.name}: ${line}`); + } + return Object.keys(sessions) + .map(Number) + .sort((a, b) => a - b) + .map((s) => sessions[s].join('\n')); +} diff --git a/cludemem/data-engine/script-generator.ts b/cludemem/data-engine/script-generator.ts new file mode 100644 index 000000000..b50215eb3 --- /dev/null +++ b/cludemem/data-engine/script-generator.ts @@ -0,0 +1,199 @@ +import type { LifeScript, PlantedFact, PlantedQA, RenderStyle } from './life-script'; + +// ============================================================ +// Combinatorial life-script generator (the volume lever). +// +// Assembles diverse personas + timelines from pools, GUARANTEEING the hard +// structures every batch needs (a supersession, a contradiction, a duplicate, +// a consistent pair, a temporal chain, and varied hard-negative unanswerables). +// Structure is deterministic per seed, so labels stay exact by construction — +// only natural-language phrasing should come from a teacher (TeacherRenderer). +// +// import { generateScripts } from './script-generator'; +// const scripts = generateScripts(4000, 42); // ~120K examples +// +// A slice of scripts is marked LONG (many planted events) for length +// stratification, and a slice carries register noise — both per design 6.1. +// ============================================================ + +// Small seeded PRNG (mulberry32) — reproducible, no Math.random. +function rng(seed: number): () => number { + let a = seed >>> 0; + return () => { + a |= 0; + a = (a + 0x6d2b79f5) | 0; + let t = Math.imul(a ^ (a >>> 15), 1 | a); + t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + +const NAMES = ['Maya', 'Devon', 'Priya', 'Theo', 'Aisha', 'Marco', 'Lena', 'Kai', 'Sofia', 'Omar', 'Nina', 'Hugo', 'Ravi', 'Elena', 'Jonas', 'Yuki', 'Tomas', 'Iris', 'Caleb', 'Zara']; +const ROLES = ['product designer', 'backend engineer', 'data scientist', 'teacher', 'nurse', 'founder', 'analyst', 'writer', 'architect', 'chef']; +const CITIES = ['Berlin', 'Lisbon', 'Madrid', 'Tokyo', 'Austin', 'Nairobi', 'Toronto', 'Oslo', 'Lyon', 'Porto', 'Seoul', 'Denver']; +const ORGS = ['Nuvio', 'Aperture', 'Lumen Labs', 'Kestrel', 'Vantage', 'Orbital', 'Brightwell', 'Cadence', 'Halcyon', 'Northstar']; +const PROJECTS = ['Atlas', 'Beacon', 'Polaris', 'Mercury', 'Sable', 'Quartz', 'Ridge', 'Echo']; +const ALLERGENS = ['peanuts', 'shellfish', 'gluten', 'pollen', 'lactose']; +const DRINKS = ['tea', 'coffee', 'matcha', 'cola']; +const RELATIONS = ['sister', 'mentor', 'manager', 'roommate', 'co-founder']; +const AREAS = ['onboarding', 'billing', 'search', 'infra', 'mobile', 'growth']; +const TEAMS = ['platform', 'design', 'data', 'ops', 'product']; + +function pick(arr: T[], r: () => number): T { + return arr[Math.floor(r() * arr.length)]; +} +function intIn(r: () => number, min: number, max: number): number { + return min + Math.floor(r() * (max - min + 1)); +} +function isoDate(r: () => number): string { + const m = intIn(r, 1, 12).toString().padStart(2, '0'); + const d = intIn(r, 1, 28).toString().padStart(2, '0'); + return `2026-${m}-${d}`; +} +// A wrong-but-close date for the same month (temporal near-miss negatives). +function nearByDays(iso: string, delta: number): string { + const [y, m, d] = iso.split('-').map(Number); + let nd = d + delta; + if (nd < 1) nd += 6; + if (nd > 28) nd -= 6; + return `${y}-${String(m).padStart(2, '0')}-${String(nd).padStart(2, '0')}`; +} +const STYLES: RenderStyle[] = ['explicit', 'implicit', 'hedged', 'corrected']; + +function generateScript(index: number): LifeScript { + const r = rng(index * 2654435761 + 1); + const name = pick(NAMES, r); + const role = pick(ROLES, r); + const facts: PlantedFact[] = []; + const qa: PlantedQA[] = []; + let n = 0; + const fid = () => `f${++n}`; + + // ── Guaranteed: relocation (supersession + dated temporal chain) ── + const cityA = pick(CITIES, r); + let cityB = pick(CITIES, r); + while (cityB === cityA) cityB = pick(CITIES, r); + const live = fid(); + facts.push({ + id: live, text: `${name} lives in ${cityA}.`, memoryType: 'semantic', importance: 0.6, + concepts: ['personal_fact', 'location'], entities: [{ name, type: 'person' }, { name: cityA, type: 'location' }], + relations: [{ head: name, type: 'lives_in', tail: cityA }], session: 0, renderStyle: 'explicit', + }); + const moveDate = isoDate(r); + const moved = fid(); + facts.push({ + id: moved, text: `${name} moved to ${cityB} on ${moveDate}.`, memoryType: 'episodic', importance: 0.8, + concepts: ['event', 'location', 'temporal_marker'], entities: [{ name, type: 'person' }, { name: cityB, type: 'location' }], + relations: [{ head: name, type: 'lives_in', tail: cityB }], eventDate: moveDate, datePrecision: 'day', + supersedes: live, session: 2, renderStyle: pick(STYLES, r), + }); + qa.push({ id: 'qa1', question: `Where does ${name} live now?`, evidence: [moved], answerable: true, expectedAnswer: cityB }); + qa.push({ id: 'qa2', question: `When did ${name} move to ${cityB}?`, evidence: [moved], answerable: true, expectedAnswer: moveDate }); + + // ── Guaranteed: duplicate (restates the relocation destination) ── + const dup = fid(); + facts.push({ + id: dup, text: `${name} currently resides in ${cityB}.`, memoryType: 'semantic', importance: 0.5, + concepts: ['personal_fact', 'location'], entities: [{ name, type: 'person' }, { name: cityB, type: 'location' }], + duplicates: moved, session: 4, renderStyle: 'explicit', + }); + + // ── Guaranteed: consistent pair (corroborates the relocation, no conflict) ── + const cons = fid(); + facts.push({ + id: cons, text: `${name} enjoys living in ${cityB}.`, memoryType: 'semantic', importance: 0.4, + concepts: ['preference', 'location'], entities: [{ name, type: 'person' }, { name: cityB, type: 'location' }], + consistentWith: moved, session: 4, renderStyle: 'hedged', emotionalValence: 0.5, + }); + + // ── Guaranteed: preference flip (contradiction) ── + const dA = pick(DRINKS, r); + let dB = pick(DRINKS, r); + while (dB === dA) dB = pick(DRINKS, r); + const pref = fid(); + facts.push({ + id: pref, text: `${name} prefers ${dA} over ${dB}.`, memoryType: 'semantic', importance: 0.4, + concepts: ['preference'], entities: [{ name, type: 'person' }], session: 0, renderStyle: 'hedged', + }); + const pref2 = fid(); + facts.push({ + id: pref2, text: `${name} now prefers ${dB}, having switched from ${dA}.`, memoryType: 'semantic', importance: 0.4, + concepts: ['preference'], entities: [{ name, type: 'person' }], contradicts: pref, session: 3, renderStyle: 'corrected', + }); + qa.push({ id: 'qa3', question: `Does ${name} prefer ${dA} or ${dB} now?`, evidence: [pref2], answerable: true, expectedAnswer: dB }); + + // ── Optional archetypes (sampled) ── + let org: string | null = null; + if (r() < 0.7) { + org = pick(ORGS, r); + const jobDate = isoDate(r); + const job = fid(); + facts.push({ + id: job, text: `${name} started a new job at ${org} on ${jobDate}.`, memoryType: 'episodic', importance: 0.85, + concepts: ['event', 'task'], entities: [{ name, type: 'person' }, { name: org, type: 'organization' }], + relations: [{ head: name, type: 'works_at', tail: org }], eventDate: jobDate, datePrecision: 'day', + session: 3, renderStyle: pick(STYLES, r), + }); + qa.push({ id: 'qa4', question: `Where does ${name} work?`, evidence: [job], answerable: true, expectedAnswer: org }); + } + if (r() < 0.6) { + const allergen = pick(ALLERGENS, r); + const al = fid(); + facts.push({ + id: al, text: `${name} is allergic to ${allergen}.`, memoryType: 'semantic', importance: 0.9, + concepts: ['personal_fact'], entities: [{ name, type: 'person' }], session: 1, renderStyle: 'explicit', + emotionalValence: -0.4, + }); + qa.push({ id: 'qa5', question: `What is ${name} allergic to?`, evidence: [al], answerable: true, expectedAnswer: allergen }); + } + if (r() < 0.6) { + const proj = pick(PROJECTS, r); + const shipDate = isoDate(r); + const ship = fid(); + facts.push({ + id: ship, text: `The ${proj} project shipped on ${shipDate}.`, memoryType: 'episodic', importance: 0.8, + concepts: ['event', 'temporal_marker'], entities: [{ name: proj, type: 'project' }], + eventDate: shipDate, datePrecision: 'day', session: 2, renderStyle: pick(STYLES, r), + }); + qa.push({ id: 'qa6', question: `When did the ${proj} project ship?`, evidence: [ship], answerable: true, expectedAnswer: shipDate }); + } + + // ── Length stratification: ~25% of scripts get a long activity log ── + const isLong = r() < 0.25; + if (isLong) { + const fillerCount = intIn(r, 40, 55); + for (let k = 0; k < fillerCount; k++) { + const fdate = isoDate(r); + const id = fid(); + facts.push({ + id, + text: `On ${fdate}, ${name} spent the afternoon reviewing the ${pick(PROJECTS, r)} roadmap, triaged the ${pick(AREAS, r)} backlog, and noted follow-ups for the ${pick(TEAMS, r)} sync.`, + memoryType: 'episodic', importance: 0.3, concepts: ['event', 'task'], + entities: [{ name, type: 'person' }], eventDate: fdate, datePrecision: 'day', + session: 5 + (k % 4), renderStyle: pick(STYLES, r), + }); + } + } + + // ── Guaranteed: 2 hard-negative unanswerables + temporal near-miss ── + qa.push({ id: 'qn1', question: `What is the name of ${name}'s ${pick(RELATIONS, r)}?`, evidence: [], answerable: false, negativeKind: 'absent' }); + const otherCity = pick(CITIES.filter((c) => c !== cityA && c !== cityB), r); + qa.push({ id: 'qn2', question: `Did ${name} move to ${otherCity}?`, evidence: [], answerable: false, negativeKind: 'attribute_miss' }); + // temporal near-miss: right move, wrong (close) date. + const nearDate = nearByDays(moveDate, intIn(r, 2, 5)); + qa.push({ id: 'qn3', question: `Did ${name} move to ${cityB} on ${nearDate}?`, evidence: [], answerable: false, negativeKind: 'temporal_near_miss' }); + // wrong-session distractor: real org, but pinned to the pre-move city it never co-occurred with. + if (org) { + qa.push({ id: 'qn4', question: `Did ${name} start at ${org} while living in ${cityA}?`, evidence: [], answerable: false, negativeKind: 'wrong_session' }); + } + + // ~25% of scripts carry register noise in their dialogue blocks. + const registerNoise = r() < 0.25; + + return { id: `gen-${index}-${name.toLowerCase()}`, persona: { name, role }, referenceDate: '2026-06-01', registerNoise, facts, qa }; +} + +/** Generate `count` diverse, deterministic life scripts (seed shifts the batch). */ +export function generateScripts(count: number, seed = 0): LifeScript[] { + return Array.from({ length: count }, (_, i) => generateScript(seed * 100000 + i)); +} diff --git a/cludemem/data-engine/taxonomy.ts b/cludemem/data-engine/taxonomy.ts new file mode 100644 index 000000000..ae7df5fb5 --- /dev/null +++ b/cludemem/data-engine/taxonomy.ts @@ -0,0 +1,165 @@ +import { z } from 'zod'; + +// ============================================================ +// CludeMem taxonomy + task I/O schemas +// +// Memory types and bond types are aligned with Clude's production schema +// (packages/shared/src/utils/constants.ts) so a trained CludeMem drops in. +// Concepts use a GENERAL agent-memory vocabulary (not the bot's crypto +// ontology) because CludeMem is a general agent-memory model. +// ============================================================ + +export const MEMORY_TYPES = [ + 'episodic', + 'semantic', + 'procedural', + 'self_model', + 'introspective', +] as const; +export type MemoryType = (typeof MEMORY_TYPES)[number]; + +export const BOND_TYPES = [ + 'supports', + 'contradicts', + 'elaborates', + 'causes', + 'follows', + 'relates', + 'resolves', + 'happens_before', + 'happens_after', + 'concurrent_with', +] as const; +export type BondType = (typeof BOND_TYPES)[number]; + +// General agent-memory concept vocabulary (12 controlled labels). +export const CONCEPTS = [ + 'personal_fact', + 'preference', + 'goal_or_plan', + 'task', + 'event', + 'relationship', + 'knowledge', + 'skill_or_procedure', + 'self_reflection', + 'location', + 'possession', + 'temporal_marker', +] as const; +export type Concept = (typeof CONCEPTS)[number]; + +export const ENTITY_TYPES = [ + 'person', + 'organization', + 'project', + 'concept', + 'location', + 'event', + 'object', +] as const; +export type EntityType = (typeof ENTITY_TYPES)[number]; + +export const DATE_PRECISION = ['day', 'month', 'year', 'none'] as const; +export type DatePrecision = (typeof DATE_PRECISION)[number]; + +// ── Task output schemas (the strict, flat JSON CludeMem emits) ── + +export const ClassifyOut = z.object({ + type: z.enum(MEMORY_TYPES), + importance: z.number().min(0).max(1), + tags: z.array(z.string()), + concepts: z.array(z.enum(CONCEPTS)), + // -1 (very negative) .. 0 (neutral) .. 1 (very positive) + emotional_valence: z.number().min(-1).max(1), +}); + +export const ExtractOut = z.object({ + memories: z.array( + z.object({ + content: z.string(), + summary: z.string(), + type: z.enum(MEMORY_TYPES), + }), + ), +}); + +export const EntitiesOut = z.object({ + entities: z.array( + z.object({ + name: z.string(), + type: z.enum(ENTITY_TYPES), + aliases: z.array(z.string()), + }), + ), + relations: z.array( + z.object({ head: z.string(), type: z.string(), tail: z.string() }), + ), +}); + +export const TemporalOut = z.object({ + event_date: z.string().nullable(), + precision: z.enum(DATE_PRECISION), + links: z.array( + z.object({ type: z.enum(BOND_TYPES), target: z.string() }), + ), +}); + +export const ConsolidateOut = z.object({ + insights: z.array(z.object({ content: z.string(), evidence: z.array(z.string()) })), +}); + +// COMPACT (task #6): squash an old memory group into one summary, keeping the +// entities and the spanned date range so nothing load-bearing is lost. +export const CompactOut = z.object({ + summary: z.string(), + preserved_entities: z.array(z.string()), + date_range: z.object({ start: z.string().nullable(), end: z.string().nullable() }), +}); + +export const ReconcileOut = z.object({ + verdict: z.enum(['contradicts', 'consistent', 'supersedes', 'duplicate']), + resolution: z.string(), + weaker_id: z.string().nullable(), + confidence: z.number().min(0).max(1), +}); + +export const QueryOut = z.object({ + expanded_queries: z.array(z.string()), + temporal_constraints: z.object({ after: z.string().nullable(), before: z.string().nullable() }), + type_filters: z.array(z.enum(MEMORY_TYPES)), + entities: z.array(z.string()), + intent: z.string(), +}); + +export const AnswerOut = z.object({ + rationale: z.string(), + answer: z.string(), + citations: z.array(z.string()), + confidence: z.number().min(0).max(1), + abstain: z.boolean(), +}); + +// Map task -> output schema, for the verification gauntlet. +export const TASK_SCHEMAS = { + CLASSIFY: ClassifyOut, + EXTRACT: ExtractOut, + ENTITIES: EntitiesOut, + TEMPORAL: TemporalOut, + CONSOLIDATE: ConsolidateOut, + COMPACT: CompactOut, + RECONCILE: ReconcileOut, + QUERY: QueryOut, + ANSWER: AnswerOut, +} as const; + +export type TaskName = keyof typeof TASK_SCHEMAS; + +// A single supervised training example (one CludeMem call). +export interface Example { + task: TaskName; + system: string; + input: string; + output: unknown; // validated against TASK_SCHEMAS[task] + meta: { scriptId: string; renderStyle?: string; answerable?: boolean }; +} diff --git a/cludemem/data-engine/teacher.ts b/cludemem/data-engine/teacher.ts new file mode 100644 index 000000000..1e13d339f --- /dev/null +++ b/cludemem/data-engine/teacher.ts @@ -0,0 +1,78 @@ +// ============================================================ +// Teacher client for the scale path (corpus rendering at 100K+). +// +// PERMITTED teachers only: DeepSeek (MIT), Qwen3 (Apache, self-hosted), Kimi K2, +// Mistral open models. NEVER Claude/GPT/Gemini — their terms forbid training a +// competing model on their outputs (see the design doc, Section 6.2). +// +// This is a thin OpenAI-compatible completion fn you pass to TeacherRenderer. +// Default points at DeepSeek; set TEACHER_BASE_URL/TEACHER_MODEL to self-host Qwen. +// ============================================================ + +export interface TeacherConfig { + apiKey: string; + baseUrl?: string; // default DeepSeek + model?: string; +} + +const BANNED = ['gpt', 'openai', 'claude', 'anthropic', 'gemini', 'google']; + +const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); +// Exponential backoff + jitter for 429 rate limits (e.g. Ollama Cloud's per-window quota). +const backoff = (attempt: number) => Math.min(20_000, 800 * 2 ** attempt) + Math.floor(Math.random() * 400); + +export function makeTeacher(cfg: TeacherConfig): { complete: (prompt: string) => Promise; model: string } { + const baseUrl = cfg.baseUrl ?? 'https://api.deepseek.com/v1'; + const model = cfg.model ?? 'deepseek-chat'; + + // Guardrail: refuse to distill from a license-incompatible teacher. + if (BANNED.some((b) => model.toLowerCase().includes(b) || baseUrl.toLowerCase().includes(b))) { + throw new Error( + `Refusing teacher "${model}" @ ${baseUrl}: Claude/GPT/Gemini outputs may not train a competing model. ` + + `Use DeepSeek/Qwen/Kimi/Mistral (see design Section 6.2).`, + ); + } + + const complete = async (prompt: string): Promise => { + const maxRetries = 6; + for (let attempt = 0; ; attempt++) { + let res: Response; + try { + res = await fetch(`${baseUrl}/chat/completions`, { + method: 'POST', + headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${cfg.apiKey}` }, + body: JSON.stringify({ + model, + messages: [{ role: 'user', content: prompt }], + temperature: 0.7, + max_tokens: 256, + }), + }); + } catch (err) { + // Network error — retry a few times, then give up (caller falls back). + if (attempt < maxRetries) { + await sleep(backoff(attempt)); + continue; + } + throw err; + } + + // 429 (rate limit) / 5xx (transient) — back off and retry. + if (res.status === 429 || res.status >= 500) { + if (attempt < maxRetries) { + await sleep(backoff(attempt)); + continue; + } + throw new Error(`Teacher API ${res.status} after ${maxRetries} retries`); + } + if (!res.ok) throw new Error(`Teacher API ${res.status}: ${await res.text().catch(() => '')}`); + + const data = (await res.json()) as { choices?: { message?: { content?: string } }[] }; + const out = data.choices?.[0]?.message?.content; + if (!out) throw new Error('Teacher returned empty content'); + return out; + } + }; + + return { complete, model }; +} diff --git a/cludemem/data/.gitignore b/cludemem/data/.gitignore new file mode 100644 index 000000000..949df173a --- /dev/null +++ b/cludemem/data/.gitignore @@ -0,0 +1,2 @@ +*.jsonl +!sample.jsonl diff --git a/cludemem/data/sample.jsonl b/cludemem/data/sample.jsonl new file mode 100644 index 000000000..85a996444 --- /dev/null +++ b/cludemem/data/sample.jsonl @@ -0,0 +1,48 @@ +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"Maya lives in Berlin."},{"role":"assistant","content":"{\"type\":\"semantic\",\"importance\":0.6,\"tags\":[\"personal_fact\",\"location\",\"maya\",\"berlin\"],\"concepts\":[\"personal_fact\",\"location\"],\"emotional_valence\":0}"}],"task":"CLASSIFY","meta":{"scriptId":"maya-relocation","renderStyle":"explicit"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"Maya lives in Berlin."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Maya\",\"type\":\"person\",\"aliases\":[]},{\"name\":\"Berlin\",\"type\":\"location\",\"aliases\":[]}],\"relations\":[{\"head\":\"Maya\",\"type\":\"lives_in\",\"tail\":\"Berlin\"}]}"}],"task":"ENTITIES","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"I think maya prefers tea over coffee. At least that's my sense of it."},{"role":"assistant","content":"{\"type\":\"semantic\",\"importance\":0.4,\"tags\":[\"preference\",\"maya\"],\"concepts\":[\"preference\"],\"emotional_valence\":0.3}"}],"task":"CLASSIFY","meta":{"scriptId":"maya-relocation","renderStyle":"hedged"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"I think maya prefers tea over coffee. At least that's my sense of it."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Maya\",\"type\":\"person\",\"aliases\":[]}],\"relations\":[]}"}],"task":"ENTITIES","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"Maya moved to Lisbon on 2026-02-10."},{"role":"assistant","content":"{\"type\":\"episodic\",\"importance\":0.8,\"tags\":[\"event\",\"location\",\"temporal_marker\",\"maya\",\"lisbon\"],\"concepts\":[\"event\",\"location\",\"temporal_marker\"],\"emotional_valence\":0.2}"}],"task":"CLASSIFY","meta":{"scriptId":"maya-relocation","renderStyle":"explicit"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"Maya moved to Lisbon on 2026-02-10."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Maya\",\"type\":\"person\",\"aliases\":[]},{\"name\":\"Lisbon\",\"type\":\"location\",\"aliases\":[]}],\"relations\":[{\"head\":\"Maya\",\"type\":\"lives_in\",\"tail\":\"Lisbon\"}]}"}],"task":"ENTITIES","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"Actually, scratch what I said before — maya now prefers coffee, having switched from tea."},{"role":"assistant","content":"{\"type\":\"semantic\",\"importance\":0.4,\"tags\":[\"preference\",\"maya\"],\"concepts\":[\"preference\"],\"emotional_valence\":-0.3}"}],"task":"CLASSIFY","meta":{"scriptId":"maya-relocation","renderStyle":"corrected"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"Actually, scratch what I said before — maya now prefers coffee, having switched from tea."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Maya\",\"type\":\"person\",\"aliases\":[]}],\"relations\":[]}"}],"task":"ENTITIES","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"By the way, around then maya started a new job at Nuvio as lead designer on 2026-02-24. It came up while we were chatting."},{"role":"assistant","content":"{\"type\":\"episodic\",\"importance\":0.85,\"tags\":[\"event\",\"task\",\"relationship\",\"maya\",\"nuvio\"],\"concepts\":[\"event\",\"task\",\"relationship\"],\"emotional_valence\":0.2}"}],"task":"CLASSIFY","meta":{"scriptId":"maya-relocation","renderStyle":"implicit"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"By the way, around then maya started a new job at Nuvio as lead designer on 2026-02-24. It came up while we were chatting."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Maya\",\"type\":\"person\",\"aliases\":[]},{\"name\":\"Nuvio\",\"type\":\"organization\",\"aliases\":[]}],\"relations\":[{\"head\":\"Maya\",\"type\":\"works_at\",\"tail\":\"Nuvio\"}]}"}],"task":"ENTITIES","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Extract the event date and its temporal links to the known events. Output JSON: {event_date, precision, links:[{type,target}]}."},{"role":"user","content":"Today is 2026-03-01.\nNew event [f3]: Maya moved to Lisbon on 2026-02-10.\nKnown events:\n[f5] Maya started a new job at Nuvio as lead designer on 2026-02-24. (2026-02-24)"},{"role":"assistant","content":"{\"event_date\":\"2026-02-10\",\"precision\":\"day\",\"links\":[{\"type\":\"happens_before\",\"target\":\"f5\"}]}"}],"task":"TEMPORAL","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Extract the event date and its temporal links to the known events. Output JSON: {event_date, precision, links:[{type,target}]}."},{"role":"user","content":"Today is 2026-03-01.\nNew event [f5]: By the way, around then maya started a new job at Nuvio as lead designer on 2026-02-24. It came up while we were chatting.\nKnown events:\n[f3] Maya moved to Lisbon on 2026-02-10. (2026-02-10)"},{"role":"assistant","content":"{\"event_date\":\"2026-02-24\",\"precision\":\"day\",\"links\":[{\"type\":\"happens_after\",\"target\":\"f3\"}]}"}],"task":"TEMPORAL","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}."},{"role":"user","content":"Maya: Maya lives in Berlin.\nMaya: I think maya prefers tea over coffee. At least that's my sense of it."},{"role":"assistant","content":"{\"memories\":[{\"content\":\"Maya lives in Berlin.\",\"summary\":\"Maya lives in Berlin.\",\"type\":\"semantic\"},{\"content\":\"Maya prefers tea over coffee.\",\"summary\":\"Maya prefers tea over coffee.\",\"type\":\"semantic\"}]}"}],"task":"EXTRACT","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}."},{"role":"user","content":"Maya: Maya moved to Lisbon on 2026-02-10."},{"role":"assistant","content":"{\"memories\":[{\"content\":\"Maya moved to Lisbon on 2026-02-10.\",\"summary\":\"Maya moved to Lisbon on 2026-02-10.\",\"type\":\"episodic\"}]}"}],"task":"EXTRACT","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}."},{"role":"user","content":"Maya: Actually, scratch what I said before — maya now prefers coffee, having switched from tea.\nMaya: By the way, around then maya started a new job at Nuvio as lead designer on 2026-02-24. It came up while we were chatting."},{"role":"assistant","content":"{\"memories\":[{\"content\":\"Maya now prefers coffee, having switched from tea.\",\"summary\":\"Maya now prefers coffee, having switched from tea.\",\"type\":\"semantic\"},{\"content\":\"Maya started a new job at Nuvio as lead designer on 2026-02-24.\",\"summary\":\"Maya started a new job at Nuvio as lead designer on 2026-02-\",\"type\":\"episodic\"}]}"}],"task":"EXTRACT","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Consolidate the memories into evidence-linked insights. Output JSON: {insights:[{content, evidence[]}]}."},{"role":"user","content":"[f1] Maya lives in Berlin.\n[f2] Maya prefers tea over coffee.\n[f3] Maya moved to Lisbon on 2026-02-10.\n[f4] Maya now prefers coffee, having switched from tea.\n[f5] Maya started a new job at Nuvio as lead designer on 2026-02-24."},{"role":"assistant","content":"{\"insights\":[{\"content\":\"Maya lives in Berlin.\",\"evidence\":[\"f1\"]},{\"content\":\"Maya prefers tea over coffee.\",\"evidence\":[\"f2\"]},{\"content\":\"Maya now prefers coffee, having switched from tea.\",\"evidence\":[\"f4\"]}]}"}],"task":"CONSOLIDATE","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Compact the old memories into one summary, preserving entities and the date range. Output JSON: {summary, preserved_entities[], date_range:{start,end}}."},{"role":"user","content":"[f3] Maya moved to Lisbon on 2026-02-10.\n[f5] Maya started a new job at Nuvio as lead designer on 2026-02-24."},{"role":"assistant","content":"{\"summary\":\"Maya: 2 events covering Maya, Lisbon, Nuvio.\",\"preserved_entities\":[\"Maya\",\"Lisbon\",\"Nuvio\"],\"date_range\":{\"start\":\"2026-02-10\",\"end\":\"2026-02-24\"}}"}],"task":"COMPACT","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Decide how the two memories relate. Output JSON: {verdict, resolution, weaker_id, confidence}."},{"role":"user","content":"Memory A [f1]: Maya lives in Berlin.\nMemory B [f3]: Maya moved to Lisbon on 2026-02-10."},{"role":"assistant","content":"{\"verdict\":\"supersedes\",\"resolution\":\"Memory B is the current truth; Memory A is outdated.\",\"weaker_id\":\"f1\",\"confidence\":0.9}"}],"task":"RECONCILE","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Decide how the two memories relate. Output JSON: {verdict, resolution, weaker_id, confidence}."},{"role":"user","content":"Memory A [f2]: Maya prefers tea over coffee.\nMemory B [f4]: Maya now prefers coffee, having switched from tea."},{"role":"assistant","content":"{\"verdict\":\"contradicts\",\"resolution\":\"Memory B conflicts with Memory A; treat B as current.\",\"weaker_id\":\"f2\",\"confidence\":0.8}"}],"task":"RECONCILE","meta":{"scriptId":"maya-relocation"}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-03-01.\nWhere does Maya live now?"},{"role":"assistant","content":"{\"expanded_queries\":[\"Where does Maya live now?\",\"Where does Maya live now\"],\"temporal_constraints\":{\"after\":null,\"before\":null},\"type_filters\":[],\"entities\":[\"Maya\"],\"intent\":\"lookup\"}"}],"task":"QUERY","meta":{"scriptId":"maya-relocation","answerable":true}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: Where does Maya live now?\n\nMemories:\n[f3] Maya moved to Lisbon on 2026-02-10.\n[f1] Maya lives in Berlin.\n[f2] Maya prefers tea over coffee."},{"role":"assistant","content":"{\"rationale\":\"Supported by f3.\",\"answer\":\"Lisbon\",\"citations\":[\"f3\"],\"confidence\":0.9,\"abstain\":false}"}],"task":"ANSWER","meta":{"scriptId":"maya-relocation","answerable":true}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-03-01.\nDoes Maya prefer tea or coffee now?"},{"role":"assistant","content":"{\"expanded_queries\":[\"Does Maya prefer tea or coffee now?\",\"Does Maya prefer tea or coffee now\"],\"temporal_constraints\":{\"after\":null,\"before\":null},\"type_filters\":[],\"entities\":[\"Maya\"],\"intent\":\"lookup\"}"}],"task":"QUERY","meta":{"scriptId":"maya-relocation","answerable":true}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: Does Maya prefer tea or coffee now?\n\nMemories:\n[f4] Maya now prefers coffee, having switched from tea.\n[f1] Maya lives in Berlin.\n[f2] Maya prefers tea over coffee."},{"role":"assistant","content":"{\"rationale\":\"Supported by f4.\",\"answer\":\"coffee\",\"citations\":[\"f4\"],\"confidence\":0.9,\"abstain\":false}"}],"task":"ANSWER","meta":{"scriptId":"maya-relocation","answerable":true}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-03-01.\nWhen did Maya move to Lisbon?"},{"role":"assistant","content":"{\"expanded_queries\":[\"When did Maya move to Lisbon?\",\"When did Maya move to Lisbon\"],\"temporal_constraints\":{\"after\":\"2026-01-10\",\"before\":\"2026-03-13\"},\"type_filters\":[],\"entities\":[\"Maya\",\"Lisbon\"],\"intent\":\"temporal_lookup\"}"}],"task":"QUERY","meta":{"scriptId":"maya-relocation","answerable":true}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: When did Maya move to Lisbon?\n\nMemories:\n[f3] Maya moved to Lisbon on 2026-02-10.\n[f1] Maya lives in Berlin.\n[f2] Maya prefers tea over coffee."},{"role":"assistant","content":"{\"rationale\":\"Supported by f3.\",\"answer\":\"2026-02-10\",\"citations\":[\"f3\"],\"confidence\":0.9,\"abstain\":false}"}],"task":"ANSWER","meta":{"scriptId":"maya-relocation","answerable":true}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-03-01.\nWhat is the name of Maya's sister?"},{"role":"assistant","content":"{\"expanded_queries\":[\"What is the name of Maya's sister?\",\"What is the name of Maya's sister\"],\"temporal_constraints\":{\"after\":null,\"before\":null},\"type_filters\":[],\"entities\":[\"Maya\"],\"intent\":\"lookup\"}"}],"task":"QUERY","meta":{"scriptId":"maya-relocation","answerable":false}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: What is the name of Maya's sister?\n\nMemories:\n[f1] Maya lives in Berlin.\n[f2] Maya prefers tea over coffee."},{"role":"assistant","content":"{\"rationale\":\"No provided memory answers this.\",\"answer\":\"I do not have enough information to answer that.\",\"citations\":[],\"confidence\":0.8,\"abstain\":true}"}],"task":"ANSWER","meta":{"scriptId":"maya-relocation","answerable":false}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-03-01.\nDid Maya move to Madrid?"},{"role":"assistant","content":"{\"expanded_queries\":[\"Did Maya move to Madrid?\",\"Did Maya move to Madrid\"],\"temporal_constraints\":{\"after\":null,\"before\":null},\"type_filters\":[],\"entities\":[\"Maya\"],\"intent\":\"lookup\"}"}],"task":"QUERY","meta":{"scriptId":"maya-relocation","answerable":false}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: Did Maya move to Madrid?\n\nMemories:\n[f1] Maya lives in Berlin.\n[f2] Maya prefers tea over coffee."},{"role":"assistant","content":"{\"rationale\":\"No provided memory answers this.\",\"answer\":\"I do not have enough information to answer that.\",\"citations\":[],\"confidence\":0.8,\"abstain\":true}"}],"task":"ANSWER","meta":{"scriptId":"maya-relocation","answerable":false}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"Devon is allergic to peanuts."},{"role":"assistant","content":"{\"type\":\"semantic\",\"importance\":0.9,\"tags\":[\"personal_fact\",\"devon\"],\"concepts\":[\"personal_fact\"],\"emotional_valence\":0}"}],"task":"CLASSIFY","meta":{"scriptId":"devon-project","renderStyle":"explicit"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"Devon is allergic to peanuts."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Devon\",\"type\":\"person\",\"aliases\":[]}],\"relations\":[]}"}],"task":"ENTITIES","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"Devon is leading the Atlas migration project."},{"role":"assistant","content":"{\"type\":\"episodic\",\"importance\":0.75,\"tags\":[\"task\",\"goal_or_plan\",\"devon\",\"atlas\"],\"concepts\":[\"task\",\"goal_or_plan\"],\"emotional_valence\":0}"}],"task":"CLASSIFY","meta":{"scriptId":"devon-project","renderStyle":"explicit"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"Devon is leading the Atlas migration project."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Devon\",\"type\":\"person\",\"aliases\":[]},{\"name\":\"Atlas\",\"type\":\"project\",\"aliases\":[]}],\"relations\":[{\"head\":\"Devon\",\"type\":\"leads\",\"tail\":\"Atlas\"}]}"}],"task":"ENTITIES","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Classify the memory. Output JSON: {type, importance (0-1), tags[], concepts[], emotional_valence (-1..1)}."},{"role":"user","content":"The Atlas migration shipped on 2026-05-09."},{"role":"assistant","content":"{\"type\":\"episodic\",\"importance\":0.8,\"tags\":[\"event\",\"temporal_marker\",\"atlas\"],\"concepts\":[\"event\",\"temporal_marker\"],\"emotional_valence\":0.2}"}],"task":"CLASSIFY","meta":{"scriptId":"devon-project","renderStyle":"explicit"}} +{"messages":[{"role":"system","content":"Extract entities and relations. Output JSON: {entities:[{name,type,aliases}], relations:[{head,type,tail}]}."},{"role":"user","content":"The Atlas migration shipped on 2026-05-09."},{"role":"assistant","content":"{\"entities\":[{\"name\":\"Atlas\",\"type\":\"project\",\"aliases\":[]}],\"relations\":[]}"}],"task":"ENTITIES","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Extract the event date and its temporal links to the known events. Output JSON: {event_date, precision, links:[{type,target}]}."},{"role":"user","content":"Today is 2026-05-15.\nNew event [g2]: Devon is leading the Atlas migration project.\nKnown events:\n[g3] The Atlas migration shipped on 2026-05-09. (2026-05-09)"},{"role":"assistant","content":"{\"event_date\":\"2026-04-01\",\"precision\":\"month\",\"links\":[{\"type\":\"happens_before\",\"target\":\"g3\"}]}"}],"task":"TEMPORAL","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Extract the event date and its temporal links to the known events. Output JSON: {event_date, precision, links:[{type,target}]}."},{"role":"user","content":"Today is 2026-05-15.\nNew event [g3]: The Atlas migration shipped on 2026-05-09.\nKnown events:\n[g2] Devon is leading the Atlas migration project. (2026-04-01)"},{"role":"assistant","content":"{\"event_date\":\"2026-05-09\",\"precision\":\"day\",\"links\":[{\"type\":\"happens_after\",\"target\":\"g2\"}]}"}],"task":"TEMPORAL","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}."},{"role":"user","content":"Devon: Devon is allergic to peanuts."},{"role":"assistant","content":"{\"memories\":[{\"content\":\"Devon is allergic to peanuts.\",\"summary\":\"Devon is allergic to peanuts.\",\"type\":\"semantic\"}]}"}],"task":"EXTRACT","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}."},{"role":"user","content":"Devon: Devon is leading the Atlas migration project."},{"role":"assistant","content":"{\"memories\":[{\"content\":\"Devon is leading the Atlas migration project.\",\"summary\":\"Devon is leading the Atlas migration project.\",\"type\":\"episodic\"}]}"}],"task":"EXTRACT","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Extract atomic memories from the text. Output JSON: {memories:[{content, summary, type}]}."},{"role":"user","content":"Devon: The Atlas migration shipped on 2026-05-09."},{"role":"assistant","content":"{\"memories\":[{\"content\":\"The Atlas migration shipped on 2026-05-09.\",\"summary\":\"The Atlas migration shipped on 2026-05-09.\",\"type\":\"episodic\"}]}"}],"task":"EXTRACT","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Consolidate the memories into evidence-linked insights. Output JSON: {insights:[{content, evidence[]}]}."},{"role":"user","content":"[g1] Devon is allergic to peanuts.\n[g2] Devon is leading the Atlas migration project.\n[g3] The Atlas migration shipped on 2026-05-09."},{"role":"assistant","content":"{\"insights\":[{\"content\":\"Devon is allergic to peanuts.\",\"evidence\":[\"g1\"]}]}"}],"task":"CONSOLIDATE","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Compact the old memories into one summary, preserving entities and the date range. Output JSON: {summary, preserved_entities[], date_range:{start,end}}."},{"role":"user","content":"[g2] Devon is leading the Atlas migration project.\n[g3] The Atlas migration shipped on 2026-05-09."},{"role":"assistant","content":"{\"summary\":\"Devon: 2 events covering Devon, Atlas.\",\"preserved_entities\":[\"Devon\",\"Atlas\"],\"date_range\":{\"start\":\"2026-04-01\",\"end\":\"2026-05-09\"}}"}],"task":"COMPACT","meta":{"scriptId":"devon-project"}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-05-15.\nWhat is Devon allergic to?"},{"role":"assistant","content":"{\"expanded_queries\":[\"What is Devon allergic to?\",\"What is Devon allergic to\"],\"temporal_constraints\":{\"after\":null,\"before\":null},\"type_filters\":[],\"entities\":[\"Devon\"],\"intent\":\"lookup\"}"}],"task":"QUERY","meta":{"scriptId":"devon-project","answerable":true}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: What is Devon allergic to?\n\nMemories:\n[g1] Devon is allergic to peanuts.\n[g2] Devon is leading the Atlas migration project.\n[g3] The Atlas migration shipped on 2026-05-09."},{"role":"assistant","content":"{\"rationale\":\"Supported by g1.\",\"answer\":\"peanuts\",\"citations\":[\"g1\"],\"confidence\":0.9,\"abstain\":false}"}],"task":"ANSWER","meta":{"scriptId":"devon-project","answerable":true}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-05-15.\nWhen did the Atlas migration ship?"},{"role":"assistant","content":"{\"expanded_queries\":[\"When did the Atlas migration ship?\",\"When did the Atlas migration ship\"],\"temporal_constraints\":{\"after\":\"2026-04-08\",\"before\":\"2026-06-09\"},\"type_filters\":[],\"entities\":[\"Atlas\"],\"intent\":\"temporal_lookup\"}"}],"task":"QUERY","meta":{"scriptId":"devon-project","answerable":true}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: When did the Atlas migration ship?\n\nMemories:\n[g3] The Atlas migration shipped on 2026-05-09.\n[g1] Devon is allergic to peanuts.\n[g2] Devon is leading the Atlas migration project."},{"role":"assistant","content":"{\"rationale\":\"Supported by g3.\",\"answer\":\"2026-05-09\",\"citations\":[\"g3\"],\"confidence\":0.9,\"abstain\":false}"}],"task":"ANSWER","meta":{"scriptId":"devon-project","answerable":true}} +{"messages":[{"role":"system","content":"Understand the query. Output JSON: {expanded_queries[], temporal_constraints, type_filters[], entities[], intent}."},{"role":"user","content":"Today is 2026-05-15.\nWhat database did Devon pick for Atlas?"},{"role":"assistant","content":"{\"expanded_queries\":[\"What database did Devon pick for Atlas?\",\"What database did Devon pick for Atlas\"],\"temporal_constraints\":{\"after\":null,\"before\":null},\"type_filters\":[],\"entities\":[\"Devon\",\"Atlas\"],\"intent\":\"lookup\"}"}],"task":"QUERY","meta":{"scriptId":"devon-project","answerable":false}} +{"messages":[{"role":"system","content":"Answer ONLY from the provided memories; abstain if unsupported. Output JSON: {rationale, answer, citations[], confidence, abstain}."},{"role":"user","content":"Question: What database did Devon pick for Atlas?\n\nMemories:\n[g1] Devon is allergic to peanuts.\n[g2] Devon is leading the Atlas migration project."},{"role":"assistant","content":"{\"rationale\":\"No provided memory answers this.\",\"answer\":\"I do not have enough information to answer that.\",\"citations\":[],\"confidence\":0.8,\"abstain\":true}"}],"task":"ANSWER","meta":{"scriptId":"devon-project","answerable":false}} diff --git a/cludemem/eval/memopseval.ts b/cludemem/eval/memopseval.ts new file mode 100644 index 000000000..4013adf0f --- /dev/null +++ b/cludemem/eval/memopseval.ts @@ -0,0 +1,142 @@ +import { readFileSync } from 'node:fs'; +import { z } from 'zod'; +import { TASK_SCHEMAS, type TaskName } from '../data-engine/taxonomy'; + +// ============================================================ +// MemOpsEval — per-task gate for CludeMem. +// +// npx tsx cludemem/eval/memopseval.ts --model cludemem-e4b --data cludemem/data/heldout.jsonl +// +// Calls a local Ollama model per example, scores per-task accuracy + +// schema-adherence + abstention calibration against the planted gold. +// Needs Ollama + the model (the GPU/Ollama stage). See design Section 8.1. +// ============================================================ + +const OLLAMA_URL = process.env.OLLAMA_URL ?? 'http://localhost:11434'; + +interface ChatRow { + messages: { role: string; content: string }[]; + task: TaskName; + meta?: { answerable?: boolean }; +} + +function arg(name: string, def?: string): string { + const i = process.argv.indexOf(`--${name}`); + return i >= 0 ? process.argv[i + 1] : (def ?? ''); +} + +async function callModel(model: string, system: string, user: string, schema: unknown): Promise { + const res = await fetch(`${OLLAMA_URL}/api/chat`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + model, + stream: false, + format: schema, + options: { temperature: 0 }, + messages: [ + { role: 'system', content: system }, + { role: 'user', content: user }, + ], + }), + }); + if (!res.ok) throw new Error(`Ollama ${res.status}`); + const data = (await res.json()) as { message?: { content?: string } }; + return JSON.parse(data.message?.content ?? '{}'); +} + +// Per-task correctness: compare the load-bearing fields to gold. +function score(task: TaskName, gold: any, pred: any): boolean { + switch (task) { + case 'CLASSIFY': + return gold.type === pred.type; + case 'TEMPORAL': + return gold.event_date === pred.event_date && gold.precision === pred.precision; + case 'RECONCILE': + return gold.verdict === pred.verdict && gold.weaker_id === pred.weaker_id; + case 'ENTITIES': { + const g = new Set((gold.entities ?? []).map((e: any) => e.name.toLowerCase())); + const p = new Set((pred.entities ?? []).map((e: any) => e.name.toLowerCase())); + return [...g].every((n) => p.has(n)); + } + case 'EXTRACT': { + const g = (gold.memories ?? []).map((m: any) => m.type).sort(); + const p = (pred.memories ?? []).map((m: any) => m.type).sort(); + return JSON.stringify(g) === JSON.stringify(p); + } + case 'CONSOLIDATE': { + const g = new Set((gold.insights ?? []).flatMap((i: any) => i.evidence)); + const p = new Set((pred.insights ?? []).flatMap((i: any) => i.evidence)); + return [...g].some((e) => p.has(e)); + } + case 'COMPACT': { + // Semantic check, like CONSOLIDATE/ENTITIES: a real summary was written, + // every planted entity is retained (containment-tolerant so "The Ridge" + // still preserves "Ridge"), and the exact date span was recovered — all + // three are derivable from the input by construction (derive.ts). + if (typeof pred.summary !== 'string' || !pred.summary.trim()) return false; + const p = (pred.preserved_entities ?? []).map((e: any) => String(e).toLowerCase()); + const entitiesKept = (gold.preserved_entities ?? []).every((g: any) => + p.some((n: string) => n.includes(String(g).toLowerCase())), + ); + const dateOk = + gold.date_range?.start === pred.date_range?.start && + gold.date_range?.end === pred.date_range?.end; + return entitiesKept && dateOk; + } + case 'QUERY': + return gold.intent === pred.intent; + case 'ANSWER': + if (gold.abstain !== pred.abstain) return false; + if (gold.abstain) return true; // correct refusal + return typeof pred.answer === 'string' && pred.answer.toLowerCase().includes(String(gold.answer).toLowerCase()); + default: + return false; + } +} + +async function main() { + const model = arg('model', 'cludemem-e4b'); + const dataPath = arg('data', 'cludemem/data/heldout.jsonl'); + const rows = readFileSync(dataPath, 'utf8').trim().split('\n').map((l) => JSON.parse(l) as ChatRow); + + const stat: Record = {}; + let abstainTotal = 0, abstainCorrect = 0; + + for (const r of rows) { + const [sys, usr, asst] = r.messages; + const gold = JSON.parse(asst.content); + const schema = TASK_SCHEMAS[r.task]; + const s = (stat[r.task] ??= { n: 0, correct: 0, schemaOk: 0 }); + s.n++; + try { + const pred = await callModel(model, sys.content, usr.content, z.toJSONSchema(schema)); + if (schema.safeParse(pred).success) s.schemaOk++; + if (score(r.task, gold, pred)) s.correct++; + if (r.task === 'ANSWER' && r.meta && r.meta.answerable === false) { + abstainTotal++; + if ((pred as any).abstain === true) abstainCorrect++; + } + } catch (e) { + // schema/parse failure counts as incorrect (schemaOk not incremented) + } + } + + console.log(`MemOpsEval — model=${model} n=${rows.length}`); + let totN = 0, totC = 0; + for (const [task, s] of Object.entries(stat)) { + totN += s.n; totC += s.correct; + console.log(` ${task.padEnd(12)} acc=${pct(s.correct, s.n)} schema=${pct(s.schemaOk, s.n)} (n=${s.n})`); + } + console.log(` ${'OVERALL'.padEnd(12)} acc=${pct(totC, totN)}`); + if (abstainTotal) console.log(` abstention ${pct(abstainCorrect, abstainTotal)} of ${abstainTotal} unanswerable correctly refused`); +} + +function pct(a: number, b: number): string { + return b ? `${((100 * a) / b).toFixed(1)}%` : 'n/a'; +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/cludemem/packaging/Modelfile b/cludemem/packaging/Modelfile new file mode 100644 index 000000000..d69f1db2b --- /dev/null +++ b/cludemem/packaging/Modelfile @@ -0,0 +1,18 @@ +# CludeMem Ollama Modelfile. +# Build: ollama create cludemem-e4b -f Modelfile +# Then: ollama run cludemem-e4b +# +# Point FROM at the merged GGUF produced by train_qlora.py --export-gguf +# (ship merged weights, not adapters). Replace the path/quant as needed. +FROM ./cludemem-e4b-lora/unsloth.Q4_K_M.gguf + +# Deterministic structured output for memory ops. +PARAMETER temperature 0 +PARAMETER top_p 0.95 +PARAMETER top_k 64 +PARAMETER num_ctx 16384 + +# CludeMem is infrastructure: it emits strict JSON for memory operations and +# does not chat. The per-task system prompt is supplied by the caller +# (see core/memory-ops.ts); this is only a safe default. +SYSTEM """You are CludeMem, a memory operations model. Given a memory task and input, respond with ONLY the strict JSON the task specifies. No prose, no markdown.""" diff --git a/cludemem/packaging/build_and_push.sh b/cludemem/packaging/build_and_push.sh new file mode 100755 index 000000000..9a251e992 --- /dev/null +++ b/cludemem/packaging/build_and_push.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# Package a trained CludeMem into Ollama + (optionally) push to a namespace. +# Run on a box with Ollama installed and the merged GGUF present. +# +# ./build_and_push.sh cludemem-e4b ./cludemem-e4b-lora/unsloth.Q4_K_M.gguf clude +# +# Args: [ollama-namespace] +set -euo pipefail + +NAME="${1:?usage: build_and_push.sh [namespace]}" +GGUF="${2:?missing gguf path}" +NS="${3:-}" + +if [[ ! -f "$GGUF" ]]; then + echo "GGUF not found: $GGUF" >&2 + exit 1 +fi + +WORK="$(mktemp -d)" +sed "s#^FROM .*#FROM ${GGUF}#" "$(dirname "$0")/Modelfile" > "${WORK}/Modelfile" + +echo "[cludemem] ollama create ${NAME}" +ollama create "${NAME}" -f "${WORK}/Modelfile" + +echo "[cludemem] smoke test (CLASSIFY)" +ollama run "${NAME}" --format json \ + 'Task: CLASSIFY. Input: Maya moved to Lisbon on 2026-02-10.' || true + +if [[ -n "${NS}" ]]; then + echo "[cludemem] pushing ${NS}/${NAME}" + ollama cp "${NAME}" "${NS}/${NAME}" + ollama push "${NS}/${NAME}" + echo "[cludemem] published: ollama pull ${NS}/${NAME}" +fi + +rm -rf "${WORK}" +echo "[cludemem] done. Activate in Clude: MEMORY_MODEL_PROVIDER=ollama MEMORY_MODEL=${NAME}" diff --git a/cludemem/training/requirements.txt b/cludemem/training/requirements.txt new file mode 100644 index 000000000..171ed471b --- /dev/null +++ b/cludemem/training/requirements.txt @@ -0,0 +1,10 @@ +# CludeMem training deps — pinned to carry the Gemma 4 fixes (design Section 7): +# grad-accumulation loss-explosion fix + E2B/E4B use_cache gibberish fix. +# Install on the GPU box (RunPod H100), not in CI. +unsloth>=0.1.36 +transformers>=5.5.2 +trl>=1.1.0 +datasets>=3.0.0 +accelerate>=1.0.0 +# llama.cpp is pulled in by Unsloth's GGUF export; for a manual conversion path +# use packaging/build_and_push.sh instead. diff --git a/cludemem/training/train_qlora.py b/cludemem/training/train_qlora.py new file mode 100755 index 000000000..15512f80d --- /dev/null +++ b/cludemem/training/train_qlora.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python3 +""" +CludeMem QLoRA fine-tune (Gemma 4 E4B) via Unsloth. + +Runs on a single GPU (RunPod H100 ~$2/hr; QLoRA E4B fits ~10-17GB). NOT runnable +in CI / on the data-engine box — this is the GPU stage of the pipeline. + + pip install -r requirements.txt + python train_qlora.py \ + --data ../data/train.jsonl \ + --base unsloth/gemma-4-E4B-it \ + --out ./cludemem-e4b-lora --epochs 2 + +Pinned versions (requirements.txt) carry the Gemma 4 fixes: the grad-accumulation +loss-explosion fix and the E2B/E4B use_cache gibberish fix (Unsloth >=0.1.36, +transformers >=5.5.2). See design Section 7. +""" +import argparse, json + + +def load_chat_jsonl(path): + rows = [] + with open(path) as f: + for line in f: + line = line.strip() + if line: + rows.append(json.loads(line)) + # Keep only the chat fields; the data engine also stores task/meta. + return [{"messages": r["messages"]} for r in rows] + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--data", required=True, help="chat-format JSONL from the data engine") + ap.add_argument("--base", default="unsloth/gemma-4-E4B-it") + ap.add_argument("--out", default="./cludemem-e4b-lora") + ap.add_argument("--epochs", type=float, default=2.0) + ap.add_argument("--rank", type=int, default=32) # design: r 32-64 + ap.add_argument("--lr", type=float, default=2e-4) # cosine + ap.add_argument("--max-seq", type=int, default=16384) # length-stratified data needs long ctx + ap.add_argument("--batch", type=int, default=2) + ap.add_argument("--grad-accum", type=int, default=8) + ap.add_argument("--export-gguf", action="store_true", help="also write merged GGUF (q4_k_m,q8_0)") + args = ap.parse_args() + + from unsloth import FastModel + from unsloth.chat_templates import get_chat_template, standardize_sharegpt + from datasets import Dataset + from trl import SFTTrainer, SFTConfig + + model, tokenizer = FastModel.from_pretrained( + model_name=args.base, + max_seq_length=args.max_seq, + load_in_4bit=True, # QLoRA + full_finetuning=False, + ) + model = FastModel.get_peft_model( + model, + r=args.rank, + lora_alpha=2 * args.rank, # design: alpha = 2r + lora_dropout=0.0, + bias="none", + target_modules=["q_proj", "k_proj", "v_proj", "o_proj", + "gate_proj", "up_proj", "down_proj"], + use_gradient_checkpointing="unsloth", + ) + tokenizer = get_chat_template(tokenizer, chat_template="gemma-4") + + rows = load_chat_jsonl(args.data) + ds = standardize_sharegpt(Dataset.from_list(rows)) + + def fmt(ex): + # Strip the auto-added ; the processor re-adds it (design Section 7). + text = tokenizer.apply_chat_template(ex["messages"], tokenize=False, add_generation_prompt=False) + return {"text": text.removeprefix(tokenizer.bos_token) if tokenizer.bos_token else text} + + ds = ds.map(fmt) + + trainer = SFTTrainer( + model=model, + tokenizer=tokenizer, + train_dataset=ds, + args=SFTConfig( + dataset_text_field="text", + per_device_train_batch_size=args.batch, + gradient_accumulation_steps=args.grad_accum, + warmup_ratio=0.03, + num_train_epochs=args.epochs, + learning_rate=args.lr, + lr_scheduler_type="cosine", + logging_steps=10, + optim="adamw_8bit", + seed=3407, + output_dir=args.out, + report_to="none", + ), + ) + # Sanity: E4B multimodal loss should sit ~13-15 early, not explode to 300+ + # (that signals the grad-accum bug -> bump Unsloth). See design Section 7. + trainer.train() + + model.save_pretrained(args.out) + tokenizer.save_pretrained(args.out) + print(f"[cludemem] LoRA saved to {args.out}") + + if args.export_gguf: + # Merge + export GGUF. Ship merged weights (NOT adapters) for Ollama. + for q in ("q4_k_m", "q8_0"): + model.save_pretrained_gguf(args.out, tokenizer, quantization_method=q) + print(f"[cludemem] wrote GGUF ({q})") + + +if __name__ == "__main__": + main() diff --git a/docs/integrations/local-memory-contract.md b/docs/integrations/local-memory-contract.md new file mode 100644 index 000000000..17458493b --- /dev/null +++ b/docs/integrations/local-memory-contract.md @@ -0,0 +1,108 @@ +# Clude Local Memory Contract + +**contractVersion: 1** +**Status:** stable for the v1 surface below. Changes follow the versioning policy at the end. + +This document defines the stable interface between Clude's memory engine and an external host (the Clude desktop app, which lives in a separate repository, or any process embedding the `clude` SDK). It lets a host run Clude's memory operations on a **local model** (CludeMem, served by Ollama) with no frontier API key. + +The host depends on this contract, not on Clude's internals. Anything not listed here may change without a version bump. + +--- + +## 1. What "local memory mode" means + +Clude splits LLM work into two classes: + +- **Memory operations** (classification, importance, extraction, entity/relation, temporal, consolidation, compaction, contradiction verdicts, query understanding, reranking, grounded answer). These can run on a local model. +- **Personality generation** (chat replies, reflection journals, emergence). These stay on a frontier provider and are out of scope for local mode. + +In local mode, memory operations route to a local Ollama model; if the local call fails, the router falls back to the configured frontier path (or surfaces the error when no frontier is configured). + +--- + +## 2. Transport + +- **Server:** Ollama, reachable at `OLLAMA_URL` (default `http://localhost:11434`). +- **Chat endpoint:** native `POST {OLLAMA_URL}/api/chat` with `stream: false`. Structured outputs use Ollama's `format` field (a JSON Schema). The OpenAI-compatible `POST {OLLAMA_URL}/v1/chat/completions` is also available. +- **Embedding endpoint:** `POST {OLLAMA_URL}/v1/embeddings` (OpenAI-compatible), used when `EMBEDDING_PROVIDER=ollama`. +- The host is responsible for installing Ollama and pulling the models (`ollama pull `). + +--- + +## 3. Models + +| Role | Selector | v1 value(s) | Notes | +|------|----------|-------------|-------| +| Memory-ops chat | `MEMORY_MODEL` | any Ollama tag, e.g. `gemma3:4b` today; `cludemem-e4b` once published | The fine-tuned CludeMem model is a drop-in: set this tag. | +| Embeddings | `EMBEDDING_MODEL` | `nomic-embed-text` (or `mxbai-embed-large`) | Only when `EMBEDDING_PROVIDER=ollama`. | + +--- + +## 4. Enabling local mode + +### 4a. Environment variables (recommended for a separate-process host, e.g. the desktop app) + +```bash +# Memory operations -> local model +MEMORY_MODEL_PROVIDER=ollama +MEMORY_MODEL=cludemem-e4b # or gemma3:4b today +OLLAMA_URL=http://localhost:11434 # default; override if Ollama runs elsewhere +MEMORY_MODEL_TIMEOUT_MS=20000 # optional, per-request timeout + +# Embeddings -> local (for a fully offline, zero-API-key setup) +EMBEDDING_PROVIDER=ollama +EMBEDDING_MODEL=nomic-embed-text +``` + +Memory operations are **opt-in**: with `MEMORY_MODEL_PROVIDER` unset, behavior is exactly as before (frontier path). A running Ollama server is never auto-detected into use. + +### 4b. Programmatic (in-process `clude` SDK) + +```ts +import { Cortex } from 'clude'; // @clude/brain + +const cortex = new Cortex({ + supabase: { url, serviceKey }, // or your storage config + localModel: { model: 'cludemem-e4b' }, // ollamaUrl optional (defaults to localhost:11434) + embedding: { provider: 'ollama', apiKey: 'ollama', model: 'nomic-embed-text' }, +}); +``` + +`localModel` applies a runtime override equivalent to the env vars in 4a, after config has loaded. + +--- + +## 5. Capability probes (stable exports from `@clude/shared`) + +```ts +import { isMemoryModelEnabled, isOllamaEnabled, getMemoryModelConfig } from '@clude/shared'; +import { pingOllama } from '@clude/shared/core/ollama-client'; + +isMemoryModelEnabled(); // boolean: any memory-model provider configured +isOllamaEnabled(); // boolean: local Ollama memory model configured + model set +getMemoryModelConfig(); // { provider, model, ollamaUrl, timeoutMs } (effective) +await pingOllama('http://localhost:11434'); // boolean: server reachable (GET /api/tags) +``` + +A host should `pingOllama()` and confirm the configured `MEMORY_MODEL` is pulled before relying on local mode, and surface a remediation hint (`ollama pull `) otherwise. The `clude doctor` CLI performs an equivalent check. + +--- + +## 6. Memory-op output shape + +Structured memory ops request a JSON Schema via `format`; the model returns JSON matching it (temperature 0). Schemas are intentionally flat. The set of locally-routed operations (by cognitive function) is: `classify`, `extract`, `entity`, `temporal`, `importance`, `summarize`, `reconcile`, `query`. The per-op JSON schemas are defined by the CludeMem model card and are additive within a contract major version. + +--- + +## 7. Versioning policy + +- `contractVersion` is an integer. The host pins the version it was built against. +- **Patch (no bump):** clarifications, new optional env vars with safe defaults, new probe helpers. +- **Minor (no bump within v1):** new locally-routed operations, additional model tags, additional optional schema fields. +- **Major (bump):** removing or renaming an env var, changing a default, removing a probe, or a breaking change to an op's output schema. Breaking changes are listed in the changelog below and announced before release. + +### Changelog + +| Version | Date | Change | +|---------|------|--------| +| 1 | 2026-06-13 | Initial contract: env + programmatic local mode, Ollama transport, capability probes, opt-in memory-op routing. | diff --git a/packages/brain/src/memory/hosted-dreams.ts b/packages/brain/src/memory/hosted-dreams.ts index a26911aa5..821f2fe29 100644 --- a/packages/brain/src/memory/hosted-dreams.ts +++ b/packages/brain/src/memory/hosted-dreams.ts @@ -20,8 +20,13 @@ import { type MemoryType, } from './memory'; import { isOpenRouterEnabled, generateOpenRouterResponse } from '@clude/shared/core/openrouter-client'; +import { generateMemoryOp } from '@clude/shared/core/memory-ops'; +import { isOllamaEnabled } from '@clude/shared/core/inference'; import { createChildLogger } from '@clude/shared/core/logger'; +// JSON schema for the local model's structured output: a flat array of strings. +const STRING_ARRAY_SCHEMA = { type: 'array', items: { type: 'string' } } as const; + const log = createChildLogger('hosted-dreams'); const MIN_EPISODIC_FOR_DREAM = 20; // Don't dream if agent has < 20 episodic memories @@ -62,22 +67,29 @@ async function dreamForAgent(ownerWallet: string, agentName: string): Promise<{ .map(m => `- [${m.created_at?.slice(0, 10)}] ${m.summary}`) .join('\n'); - if (!isOpenRouterEnabled()) { - log.warn({ agentName }, 'OpenRouter not enabled — cannot run dream cycle'); + if (!isOpenRouterEnabled() && !isOllamaEnabled()) { + log.warn({ agentName }, 'No memory-generation backend enabled — cannot run dream cycle'); return; } + const consolidationPrompt = `Extract 5-8 key factual insights from these agent memories. Facts, patterns, learned knowledge — not events.\n\n${episodicSummaries}`; try { - const consolidationResponse = await generateOpenRouterResponse({ - systemPrompt: 'You extract factual insights from data. Always respond with ONLY a JSON array of strings. No explanation, no markdown, no thinking process.', - messages: [{ - role: 'user', - content: `Extract 5-8 key factual insights from these agent memories. Facts, patterns, learned knowledge — not events.\n\n${episodicSummaries}`, - }], - model: 'meta-llama/llama-3.3-70b-instruct', - maxTokens: 1000, - temperature: 0.2, - }); + const consolidationResponse = isOllamaEnabled() + ? await generateMemoryOp({ + cognitiveFunction: 'summarize', + systemPrompt: 'You extract factual insights from data. Always respond with ONLY a JSON array of strings. No explanation, no markdown, no thinking process.', + userMessage: consolidationPrompt, + format: STRING_ARRAY_SCHEMA, + maxTokens: 1000, + temperature: 0.2, + }) + : await generateOpenRouterResponse({ + systemPrompt: 'You extract factual insights from data. Always respond with ONLY a JSON array of strings. No explanation, no markdown, no thinking process.', + messages: [{ role: 'user', content: consolidationPrompt }], + model: 'meta-llama/llama-3.3-70b-instruct', + maxTokens: 1000, + temperature: 0.2, + }); // Parse insights const jsonMatch = consolidationResponse.match(/\[[\s\S]*\]/); @@ -107,17 +119,24 @@ async function dreamForAgent(ownerWallet: string, agentName: string): Promise<{ // ── Phase 2: Procedural extraction ── // Extract behavioral patterns / rules from episodes + const proceduralPrompt = `Extract 3-4 actionable rules/behaviors from these agent memories. Things to DO or AVOID.\n\n${episodicSummaries}`; try { - const proceduralResponse = await generateOpenRouterResponse({ - systemPrompt: 'You extract actionable rules from data. Always respond with ONLY a JSON array of strings. No explanation.', - messages: [{ - role: 'user', - content: `Extract 3-4 actionable rules/behaviors from these agent memories. Things to DO or AVOID.\n\n${episodicSummaries}`, - }], - model: 'meta-llama/llama-3.3-70b-instruct', - maxTokens: 500, - temperature: 0.2, - }); + const proceduralResponse = isOllamaEnabled() + ? await generateMemoryOp({ + cognitiveFunction: 'extract', + systemPrompt: 'You extract actionable rules from data. Always respond with ONLY a JSON array of strings. No explanation.', + userMessage: proceduralPrompt, + format: STRING_ARRAY_SCHEMA, + maxTokens: 500, + temperature: 0.2, + }) + : await generateOpenRouterResponse({ + systemPrompt: 'You extract actionable rules from data. Always respond with ONLY a JSON array of strings. No explanation.', + messages: [{ role: 'user', content: proceduralPrompt }], + model: 'meta-llama/llama-3.3-70b-instruct', + maxTokens: 500, + temperature: 0.2, + }); const jsonMatch = proceduralResponse.match(/\[[\s\S]*\]/); if (jsonMatch) { diff --git a/packages/brain/src/memory/memory.ts b/packages/brain/src/memory/memory.ts index 72b32406c..f28ac0fa6 100644 --- a/packages/brain/src/memory/memory.ts +++ b/packages/brain/src/memory/memory.ts @@ -43,7 +43,9 @@ import { bm25SearchMemories } from '../experimental/bm25-search'; import { setContentTokens } from './content-tokens'; import { runReconcileShadow } from './reconcile-shadow'; import { encryptForStorage, delegationStateForWrite } from './memory-encryption'; -import { generateOpenRouterResponse, isOpenRouterEnabled } from '@clude/shared/core/openrouter-client'; +import { isOpenRouterEnabled } from '@clude/shared/core/openrouter-client'; +import { generateMemoryOp } from '@clude/shared/core/memory-ops'; +import { isMemoryModelEnabled, isOllamaEnabled } from '@clude/shared/core/inference'; import { isEncryptionEnabled, getEncryptionPubkey, encryptContent } from '@clude/shared/core/encryption'; import { decryptMemories } from './memory-decryption'; import { eventBus } from '../events/event-bus'; @@ -1059,17 +1061,17 @@ async function embedMemory( * Falls back to just the original query if LLM is unavailable or slow. */ async function expandQuery(query: string): Promise { - if (!isOpenRouterEnabled()) return [query]; + // Need at least one generation backend: the local memory model or a frontier router. + if (!isOpenRouterEnabled() && !isMemoryModelEnabled()) return [query]; try { const response = await Promise.race([ - generateOpenRouterResponse({ + generateMemoryOp({ + cognitiveFunction: 'query', // memory op → local model when enabled, else fast frontier slot systemPrompt: 'You are a search query expander. Given a question, output 3 alternative phrasings that would help find relevant information in a memory database. Output ONLY the 3 alternatives, one per line. No numbering, no explanations.', - messages: [{ role: 'user', content: query }], - model: 'meta-llama/llama-3.2-3b-instruct', + userMessage: query, maxTokens: 150, temperature: 0.3, - cognitiveFunction: 'entity', // Use fast model slot }), new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 3000)), ]) as string; @@ -2536,7 +2538,17 @@ export async function scoreImportanceWithLLM( fallbackOpts?: Parameters[0] ): Promise { try { - const response = await generateImportanceScore(description); + // Local memory model (CludeMem) when enabled, else the existing Anthropic-direct path. + const response = isOllamaEnabled() + ? await generateMemoryOp({ + cognitiveFunction: 'importance', + systemPrompt: + 'You rate the importance of events for an AI agent. Respond with ONLY a single integer from 1 to 10. 1 = purely mundane. 5 = moderately important. 10 = extremely significant.', + userMessage: `Rate the importance of this event:\n"${description.slice(0, 500)}"\nRating (1-10):`, + maxTokens: 10, + temperature: 0, + }) + : await generateImportanceScore(description); const parsed = parseInt(response.trim(), 10); if (!isNaN(parsed) && parsed >= 1 && parsed <= 10) { return parsed / 10; diff --git a/packages/brain/src/sdk/cortex.ts b/packages/brain/src/sdk/cortex.ts index 359a9e82b..b4cdf6b6e 100644 --- a/packages/brain/src/sdk/cortex.ts +++ b/packages/brain/src/sdk/cortex.ts @@ -56,6 +56,16 @@ export class Cortex { }); } + // Local memory model (CludeMem) — route memory ops to a local Ollama model. + if (config.localModel) { + const { _configureMemoryModel } = require('@clude/shared/core/memory-model-config'); + _configureMemoryModel({ + provider: 'ollama', + model: config.localModel.model, + ...(config.localModel.ollamaUrl ? { ollamaUrl: config.localModel.ollamaUrl } : {}), + }); + } + // Inject Solana config if provided if (config.solana) { const { _configureSolana, _configureMemoryRegistry } = require('@clude/shared/core/solana-client'); diff --git a/packages/brain/src/sdk/types.ts b/packages/brain/src/sdk/types.ts index 6bcdd1a66..ac07ee07e 100644 --- a/packages/brain/src/sdk/types.ts +++ b/packages/brain/src/sdk/types.ts @@ -30,6 +30,19 @@ export interface CortexConfig { dimensions?: number; }; + /** + * Local memory model (CludeMem via Ollama). Routes memory operations + * (classify/importance/extract/summarize/query/...) to a local model instead + * of a frontier API. Self-hosted only. Pair with an Ollama embedding provider + * (EMBEDDING_PROVIDER=ollama) for a fully local, zero-API-key setup. + */ + localModel?: { + /** Ollama model tag, e.g. 'cludemem-e4b' or 'gemma3:4b'. */ + model: string; + /** Ollama server base URL. Defaults to http://localhost:11434. */ + ollamaUrl?: string; + }; + /** Owner wallet address (Solana public key). Tags all memories with ownership. */ ownerWallet?: string; diff --git a/packages/brain/src/services/upload-processor.ts b/packages/brain/src/services/upload-processor.ts index 7cbaca5be..ab85cb92c 100644 --- a/packages/brain/src/services/upload-processor.ts +++ b/packages/brain/src/services/upload-processor.ts @@ -12,6 +12,8 @@ import { generateOpenRouterResponse, OPENROUTER_MODELS, } from "@clude/shared/core/openrouter-client"; +import { generateMemoryOp } from "@clude/shared/core/memory-ops"; +import { isOllamaEnabled } from "@clude/shared/core/inference"; import { z } from "zod"; const log = createChildLogger("batch-upload"); @@ -57,6 +59,21 @@ const ExtractedNodeSchema = z.object({ type ExtractedNode = z.infer; +// JSON schema for the local model's structured output (mirrors ExtractedNodeSchema). +const NODE_ARRAY_SCHEMA = { + type: "array", + items: { + type: "object", + properties: { + title: { type: "string" }, + content: { type: "string" }, + entities: { type: "array", items: { type: "string" } }, + original_text: { type: "string" }, + }, + required: ["title", "content", "entities", "original_text"], + }, +} as const; + interface ChunkRow { batch_id: string; chunk_index: number; @@ -70,18 +87,24 @@ async function extractNodes( chunk: string, chunkIndex: number, ): Promise<{ nodes: ExtractedNode[]; rawResponse: string }> { - const rawResponse = await generateOpenRouterResponse({ - systemPrompt: EXTRACTION_PROMPT, - messages: [ - { - role: "user", - content: `Extract memory nodes from this text chunk (#${chunkIndex + 1}):\n\n${chunk}`, - }, - ], - model: OPENROUTER_MODELS["claude-haiku-4.5"], - maxTokens: 4096, - temperature: 0.2, - }); + const userPrompt = `Extract memory nodes from this text chunk (#${chunkIndex + 1}):\n\n${chunk}`; + // Local memory model (CludeMem) when enabled, else the existing Haiku path. + const rawResponse = isOllamaEnabled() + ? await generateMemoryOp({ + cognitiveFunction: "extract", + systemPrompt: EXTRACTION_PROMPT, + userMessage: userPrompt, + format: NODE_ARRAY_SCHEMA, + maxTokens: 4096, + temperature: 0.2, + }) + : await generateOpenRouterResponse({ + systemPrompt: EXTRACTION_PROMPT, + messages: [{ role: "user", content: userPrompt }], + model: OPENROUTER_MODELS["claude-haiku-4.5"], + maxTokens: 4096, + temperature: 0.2, + }); let nodes: ExtractedNode[] = []; try { diff --git a/packages/shared/src/config.ts b/packages/shared/src/config.ts index 7b9f91d4e..17cac29ad 100644 --- a/packages/shared/src/config.ts +++ b/packages/shared/src/config.ts @@ -254,6 +254,23 @@ export const config = { | "anthropic" | "openrouter", }, + // Local/pluggable memory-operations model (CludeMem). Empty provider = disabled, + // so memory ops stay on the frontier path. Opt-in only — never auto-enabled even + // when a local Ollama server is running (see specs design Section 9). + memoryModel: { + /** "" (disabled) | "ollama" | "anthropic" | "openrouter" */ + provider: optional("MEMORY_MODEL_PROVIDER", "") as + | "" + | "ollama" + | "anthropic" + | "openrouter", + /** Model id/tag, e.g. an Ollama tag like "cludemem-e4b" or "gemma3:4b" */ + model: optional("MEMORY_MODEL", ""), + /** Reuses OLLAMA_URL (shared with the local embedding provider) */ + ollamaUrl: optional("OLLAMA_URL", "http://localhost:11434"), + /** Per-request timeout (ms) for local inference */ + timeoutMs: parseInt(optional("MEMORY_MODEL_TIMEOUT_MS", "20000"), 10), + }, tavily: { apiKey: optional("TAVILY_API_KEY", ""), }, diff --git a/packages/shared/src/core/__tests__/config-memory-model.test.ts b/packages/shared/src/core/__tests__/config-memory-model.test.ts new file mode 100644 index 000000000..2eef5b983 --- /dev/null +++ b/packages/shared/src/core/__tests__/config-memory-model.test.ts @@ -0,0 +1,40 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; + +// config.ts reads env at import time and rebuilds on resetModules. +// __CLUDE_SDK_MODE makes required() lenient so importing config in a unit +// test never throws on unrelated missing vars (X_API_KEY, etc.). +const KEYS = ['MEMORY_MODEL_PROVIDER', 'MEMORY_MODEL', 'OLLAMA_URL', 'MEMORY_MODEL_TIMEOUT_MS']; + +describe('config.memoryModel', () => { + beforeEach(() => { + (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE = true; + for (const k of KEYS) delete process.env[k]; + vi.resetModules(); + }); + + afterEach(() => { + for (const k of KEYS) delete process.env[k]; + delete (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE; + }); + + it('defaults to a disabled provider (memory ops stay on the frontier path)', async () => { + const { config } = await import('../../config'); + expect(config.memoryModel.provider).toBe(''); + }); + + it('reads provider, model, and ollamaUrl from env', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + process.env.OLLAMA_URL = 'http://127.0.0.1:11434'; + const { config } = await import('../../config'); + expect(config.memoryModel.provider).toBe('ollama'); + expect(config.memoryModel.model).toBe('cludemem-e4b'); + expect(config.memoryModel.ollamaUrl).toBe('http://127.0.0.1:11434'); + }); + + it('parses the request timeout as a number', async () => { + process.env.MEMORY_MODEL_TIMEOUT_MS = '12345'; + const { config } = await import('../../config'); + expect(config.memoryModel.timeoutMs).toBe(12345); + }); +}); diff --git a/packages/shared/src/core/__tests__/inference-ollama.test.ts b/packages/shared/src/core/__tests__/inference-ollama.test.ts new file mode 100644 index 000000000..f4bca8dd8 --- /dev/null +++ b/packages/shared/src/core/__tests__/inference-ollama.test.ts @@ -0,0 +1,79 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; + +// Mock the three provider clients so we assert routing, not network calls. +// vi.hoisted keeps the mock fns available to the hoisted vi.mock factories. +const { ollamaMock, claudeMock, openrouterMock } = vi.hoisted(() => ({ + ollamaMock: vi.fn(async (..._a: unknown[]) => 'OLLAMA_OUT'), + claudeMock: vi.fn(async (..._a: unknown[]) => 'CLAUDE_OUT'), + openrouterMock: vi.fn(async (..._a: unknown[]) => 'OR_OUT'), +})); + +vi.mock('../ollama-client', () => ({ generateOllamaResponse: ollamaMock })); +vi.mock('../claude-client', () => ({ generateResponse: claudeMock })); +vi.mock('../openrouter-client', () => ({ + generateOpenRouterResponse: openrouterMock, + isOpenRouterEnabled: () => false, +})); + +const KEYS = ['MEMORY_MODEL_PROVIDER', 'MEMORY_MODEL', 'OLLAMA_URL']; + +describe('inference: ollama (local memory model) routing', () => { + beforeEach(() => { + (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE = true; + for (const k of KEYS) delete process.env[k]; + ollamaMock.mockClear(); + claudeMock.mockClear(); + openrouterMock.mockClear(); + vi.resetModules(); + }); + + afterEach(() => { + for (const k of KEYS) delete process.env[k]; + delete (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE; + }); + + it('isOllamaEnabled is false with no provider, and false with a model but no provider (no auto-enable)', async () => { + const a = await import('../inference'); + expect(a.isOllamaEnabled()).toBe(false); + + process.env.MEMORY_MODEL = 'm'; // model set, provider still unset + vi.resetModules(); + const b = await import('../inference'); + expect(b.isOllamaEnabled()).toBe(false); + }); + + it('routes generate({ provider: "ollama" }) to the local client', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + const { generate, isOllamaEnabled } = await import('../inference'); + + expect(isOllamaEnabled()).toBe(true); + const out = await generate({ provider: 'ollama', userMessage: 'classify: I moved to Lisbon' }); + + expect(out).toBe('OLLAMA_OUT'); + expect(ollamaMock).toHaveBeenCalledOnce(); + expect(claudeMock).not.toHaveBeenCalled(); + const arg = ollamaMock.mock.calls[0][0] as { model: string }; + expect(arg.model).toBe('cludemem-e4b'); + }); + + it('does NOT fall back to anthropic when an explicit ollama call fails', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + ollamaMock.mockRejectedValueOnce(new Error('ollama down')); + const { generate } = await import('../inference'); + + await expect(generate({ provider: 'ollama', userMessage: 'x' })).rejects.toThrow(/ollama down/); + expect(claudeMock).not.toHaveBeenCalled(); + }); + + it('auto mode never selects ollama even when it is configured', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + const { generate } = await import('../inference'); + + const out = await generate({ provider: 'auto', userMessage: 'x' }); + expect(out).toBe('CLAUDE_OUT'); // openrouter mocked disabled -> anthropic + expect(ollamaMock).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/shared/src/core/__tests__/memory-model-config.test.ts b/packages/shared/src/core/__tests__/memory-model-config.test.ts new file mode 100644 index 000000000..297ef5d55 --- /dev/null +++ b/packages/shared/src/core/__tests__/memory-model-config.test.ts @@ -0,0 +1,44 @@ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; + +// The runtime override lets the SDK / desktop app enable local mode after +// config has frozen (mirrors _configureEmbeddings). +const KEYS = ['MEMORY_MODEL_PROVIDER', 'MEMORY_MODEL', 'OLLAMA_URL']; + +describe('memory-model-config runtime override', () => { + beforeEach(() => { + (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE = true; + for (const k of KEYS) delete process.env[k]; + }); + + afterEach(async () => { + for (const k of KEYS) delete process.env[k]; + delete (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE; + const mod = await import('../memory-model-config'); + mod._resetMemoryModelConfig(); + }); + + it('returns env config (disabled) when no override is set', async () => { + const { getMemoryModelConfig } = await import('../memory-model-config'); + expect(getMemoryModelConfig().provider).toBe(''); + }); + + it('applies a runtime override and enables the local model via isOllamaEnabled', async () => { + const { _configureMemoryModel, getMemoryModelConfig } = await import('../memory-model-config'); + const { isOllamaEnabled, isMemoryModelEnabled } = await import('../inference'); + + expect(isOllamaEnabled()).toBe(false); + _configureMemoryModel({ provider: 'ollama', model: 'cludemem-e4b', ollamaUrl: 'http://localhost:11434' }); + + const mm = getMemoryModelConfig(); + expect(mm.provider).toBe('ollama'); + expect(mm.model).toBe('cludemem-e4b'); + expect(isOllamaEnabled()).toBe(true); + expect(isMemoryModelEnabled()).toBe(true); + }); + + it('merges a partial override over env defaults (ollamaUrl falls back)', async () => { + const { _configureMemoryModel, getMemoryModelConfig } = await import('../memory-model-config'); + _configureMemoryModel({ provider: 'ollama', model: 'm' }); + expect(getMemoryModelConfig().ollamaUrl).toBe('http://localhost:11434'); + }); +}); diff --git a/packages/shared/src/core/__tests__/memory-model-exports.test.ts b/packages/shared/src/core/__tests__/memory-model-exports.test.ts new file mode 100644 index 000000000..dc1a97ed2 --- /dev/null +++ b/packages/shared/src/core/__tests__/memory-model-exports.test.ts @@ -0,0 +1,26 @@ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; + +// Smoke test: the public capability helpers are reachable from the package +// entry (`@clude/shared` → src/index), and the client from its deep subpath. +// __CLUDE_SDK_MODE keeps config import lenient on unrelated required vars. +describe('memory-model public surface', () => { + beforeEach(() => { + (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE = true; + }); + afterEach(() => { + delete (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE; + }); + + it('exposes isOllamaEnabled and isMemoryModelEnabled from the package entry', async () => { + const entry = await import('../../index'); + expect(typeof entry.isOllamaEnabled).toBe('function'); + expect(typeof entry.isMemoryModelEnabled).toBe('function'); + }); + + it('exposes the Ollama client from its deep subpath', async () => { + const client = await import('../ollama-client'); + expect(typeof client.generateOllamaResponse).toBe('function'); + expect(typeof client.askOllama).toBe('function'); + expect(typeof client.pingOllama).toBe('function'); + }); +}); diff --git a/packages/shared/src/core/__tests__/memory-ops.test.ts b/packages/shared/src/core/__tests__/memory-ops.test.ts new file mode 100644 index 000000000..7875e79ed --- /dev/null +++ b/packages/shared/src/core/__tests__/memory-ops.test.ts @@ -0,0 +1,101 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; + +// Mock the underlying clients; assert routing, not network calls. +const { ollamaMock, openrouterMock, claudeMock } = vi.hoisted(() => ({ + ollamaMock: vi.fn(async (..._a: unknown[]) => 'LOCAL'), + openrouterMock: vi.fn(async (..._a: unknown[]) => 'OPENROUTER'), + claudeMock: vi.fn(async (..._a: unknown[]) => 'CLAUDE'), +})); + +vi.mock('../ollama-client', () => ({ generateOllamaResponse: ollamaMock })); +vi.mock('../claude-client', () => ({ generateResponse: claudeMock })); + +// isOpenRouterEnabled is toggled per test via a mutable flag. +let openrouterEnabled = true; +vi.mock('../openrouter-client', () => ({ + generateOpenRouterResponse: openrouterMock, + isOpenRouterEnabled: () => openrouterEnabled, +})); + +const KEYS = ['MEMORY_MODEL_PROVIDER', 'MEMORY_MODEL']; + +describe('memory-ops router', () => { + beforeEach(() => { + (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE = true; + for (const k of KEYS) delete process.env[k]; + openrouterEnabled = true; + ollamaMock.mockClear(); + openrouterMock.mockClear(); + claudeMock.mockClear(); + vi.resetModules(); + }); + + afterEach(() => { + for (const k of KEYS) delete process.env[k]; + delete (globalThis as unknown as { __CLUDE_SDK_MODE?: boolean }).__CLUDE_SDK_MODE; + }); + + it('classifies query/classify/importance as memory ops and reply/reflect as not', async () => { + const { isMemoryOp } = await import('../memory-ops'); + expect(isMemoryOp('query')).toBe(true); + expect(isMemoryOp('classify')).toBe(true); + expect(isMemoryOp('importance')).toBe(true); + expect(isMemoryOp('reply')).toBe(false); + expect(isMemoryOp('reflect')).toBe(false); + expect(isMemoryOp('emergence')).toBe(false); + expect(isMemoryOp('dream')).toBe(false); + }); + + it('uses the frontier path (OpenRouter) when the local model is disabled', async () => { + const { generateMemoryOp } = await import('../memory-ops'); + const out = await generateMemoryOp({ cognitiveFunction: 'query', userMessage: 'x' }); + expect(out).toBe('OPENROUTER'); + expect(ollamaMock).not.toHaveBeenCalled(); + expect(openrouterMock).toHaveBeenCalledOnce(); + }); + + it('uses the local model for a memory op when enabled, passing the format schema', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + const { generateMemoryOp } = await import('../memory-ops'); + const schema = { type: 'object' }; + const out = await generateMemoryOp({ + cognitiveFunction: 'classify', + userMessage: 'I moved to Lisbon', + format: schema, + }); + expect(out).toBe('LOCAL'); + expect(openrouterMock).not.toHaveBeenCalled(); + const arg = ollamaMock.mock.calls[0][0] as { model: string; format?: unknown }; + expect(arg.model).toBe('cludemem-e4b'); + expect(arg.format).toEqual(schema); + }); + + it('does NOT route a personality function to the local model even when enabled', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + const { generateMemoryOp } = await import('../memory-ops'); + const out = await generateMemoryOp({ cognitiveFunction: 'reply', userMessage: 'x' }); + expect(out).toBe('OPENROUTER'); + expect(ollamaMock).not.toHaveBeenCalled(); + }); + + it('falls back to the frontier path when the local call fails', async () => { + process.env.MEMORY_MODEL_PROVIDER = 'ollama'; + process.env.MEMORY_MODEL = 'cludemem-e4b'; + ollamaMock.mockRejectedValueOnce(new Error('ollama down')); + const { generateMemoryOp } = await import('../memory-ops'); + const out = await generateMemoryOp({ cognitiveFunction: 'classify', userMessage: 'x' }); + expect(out).toBe('OPENROUTER'); + expect(ollamaMock).toHaveBeenCalledOnce(); + expect(openrouterMock).toHaveBeenCalledOnce(); + }); + + it('falls back to the Claude client when OpenRouter is also disabled', async () => { + openrouterEnabled = false; + const { generateMemoryOp } = await import('../memory-ops'); + const out = await generateMemoryOp({ cognitiveFunction: 'summarize', userMessage: 'x' }); + expect(out).toBe('CLAUDE'); + expect(claudeMock).toHaveBeenCalledOnce(); + }); +}); diff --git a/packages/shared/src/core/__tests__/ollama-client.test.ts b/packages/shared/src/core/__tests__/ollama-client.test.ts new file mode 100644 index 000000000..86c205bcd --- /dev/null +++ b/packages/shared/src/core/__tests__/ollama-client.test.ts @@ -0,0 +1,108 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +beforeEach(() => { + vi.restoreAllMocks(); + vi.resetModules(); +}); + +function mockFetchOnce(body: unknown, ok = true, status = 200) { + return vi.spyOn(globalThis, 'fetch').mockResolvedValue({ + ok, + status, + json: async () => body, + text: async () => (typeof body === 'string' ? body : JSON.stringify(body)), + } as unknown as Response); +} + +describe('ollama-client', () => { + it('returns message content from /api/chat (stream:false)', async () => { + const f = mockFetchOnce({ message: { role: 'assistant', content: 'hello' }, done: true }); + const { generateOllamaResponse } = await import('../ollama-client'); + + const out = await generateOllamaResponse({ + ollamaUrl: 'http://localhost:11434', + model: 'gemma3:4b', + messages: [{ role: 'user', content: 'hi' }], + }); + + expect(out).toBe('hello'); + const [url, init] = f.mock.calls[0]; + expect(String(url)).toBe('http://localhost:11434/api/chat'); + const sent = JSON.parse((init as RequestInit).body as string); + expect(sent.stream).toBe(false); + expect(sent.model).toBe('gemma3:4b'); + expect(sent.messages).toEqual([{ role: 'user', content: 'hi' }]); + }); + + it('prepends systemPrompt and passes a JSON schema as `format`', async () => { + const f = mockFetchOnce({ message: { content: '{"type":"semantic"}' }, done: true }); + const { generateOllamaResponse } = await import('../ollama-client'); + const schema = { type: 'object', properties: { type: { type: 'string' } } }; + + const out = await generateOllamaResponse({ + ollamaUrl: 'http://localhost:11434', + model: 'm', + systemPrompt: 'You classify memories.', + messages: [{ role: 'user', content: 'x' }], + format: schema, + options: { temperature: 0 }, + }); + + expect(out).toBe('{"type":"semantic"}'); + const sent = JSON.parse((f.mock.calls[0][1] as RequestInit).body as string); + expect(sent.format).toEqual(schema); + expect(sent.options.temperature).toBe(0); + expect(sent.messages[0]).toEqual({ role: 'system', content: 'You classify memories.' }); + }); + + it('omits format and options when not provided', async () => { + const f = mockFetchOnce({ message: { content: 'ok' } }); + const { generateOllamaResponse } = await import('../ollama-client'); + + await generateOllamaResponse({ + ollamaUrl: 'http://localhost:11434', + model: 'm', + messages: [{ role: 'user', content: 'x' }], + }); + + const sent = JSON.parse((f.mock.calls[0][1] as RequestInit).body as string); + expect('format' in sent).toBe(false); + expect('options' in sent).toBe(false); + }); + + it('throws on a non-2xx response', async () => { + mockFetchOnce({ error: 'model not found' }, false, 404); + const { generateOllamaResponse } = await import('../ollama-client'); + + await expect( + generateOllamaResponse({ + ollamaUrl: 'http://localhost:11434', + model: 'missing', + messages: [{ role: 'user', content: 'x' }], + }), + ).rejects.toThrow(/Ollama API error: 404/); + }); + + it('throws on a malformed response shape', async () => { + mockFetchOnce({ nope: true }); + const { generateOllamaResponse } = await import('../ollama-client'); + + await expect( + generateOllamaResponse({ + ollamaUrl: 'http://localhost:11434', + model: 'm', + messages: [{ role: 'user', content: 'x' }], + }), + ).rejects.toThrow(/invalid response shape/); + }); + + it('pingOllama returns true on ok and false when fetch throws', async () => { + vi.spyOn(globalThis, 'fetch') + .mockResolvedValueOnce({ ok: true } as Response) + .mockRejectedValueOnce(new Error('connection refused')); + const { pingOllama } = await import('../ollama-client'); + + expect(await pingOllama('http://localhost:11434')).toBe(true); + expect(await pingOllama('http://localhost:11434')).toBe(false); + }); +}); diff --git a/packages/shared/src/core/inference.ts b/packages/shared/src/core/inference.ts index e2ff8312c..ed9b17f74 100644 --- a/packages/shared/src/core/inference.ts +++ b/packages/shared/src/core/inference.ts @@ -1,6 +1,8 @@ import { createChildLogger } from './logger'; import { generateResponse as generateClaudeResponse } from './claude-client'; import { generateOpenRouterResponse, isOpenRouterEnabled, type OpenRouterMessage } from './openrouter-client'; +import { generateOllamaResponse } from './ollama-client'; +import { getMemoryModelConfig } from './memory-model-config'; const log = createChildLogger('inference'); @@ -13,11 +15,25 @@ const log = createChildLogger('inference'); // Providers: // - anthropic: Claude (direct Anthropic API) // - openrouter: Unified router (all models via single key) +// - ollama: local memory model (CludeMem). Opt-in only — never selected +// by 'auto', and an explicit ollama call surfaces failures rather than +// silently falling back to a frontier provider (see specs design Section 9). // // Priority in auto mode: OpenRouter > Anthropic // ============================================================ -export type InferenceProvider = 'anthropic' | 'openrouter' | 'auto'; +export type InferenceProvider = 'anthropic' | 'openrouter' | 'ollama' | 'auto'; + +/** True only when the local memory model is explicitly configured (opt-in). */ +export function isOllamaEnabled(): boolean { + const mm = getMemoryModelConfig(); + return mm.provider === 'ollama' && Boolean(mm.model); +} + +/** True when any non-empty memory-model provider is configured (capability check). */ +export function isMemoryModelEnabled(): boolean { + return getMemoryModelConfig().provider !== ''; +} export interface InferenceConfig { /** Primary provider (default: 'auto' — tries OpenRouter first, falls back to Anthropic) */ @@ -95,9 +111,12 @@ export async function generate(opts: GenerateOptions): Promise { throw lastError || new Error('All inference providers failed'); } -function getProviderOrder(provider: InferenceProvider): Array<'anthropic' | 'openrouter'> { +type ConcreteProvider = 'anthropic' | 'openrouter' | 'ollama'; + +function getProviderOrder(provider: InferenceProvider): Array { if (provider === 'auto') { - const providers: Array<'anthropic' | 'openrouter'> = []; + // 'auto' never selects the local memory model — opt-in is explicit only. + const providers: Array = []; if (isOpenRouterEnabled()) { providers.push('openrouter'); } @@ -105,15 +124,21 @@ function getProviderOrder(provider: InferenceProvider): Array<'anthropic' | 'ope return providers; } - const providers: Array<'anthropic' | 'openrouter'> = [provider as 'anthropic' | 'openrouter']; + // Explicit local memory model: surface failures, never silently fall back to + // a frontier provider. Graceful fallback is added at the Spec 1b router level. + if (provider === 'ollama') { + return ['ollama']; + } + + const providers: Array = [provider as ConcreteProvider]; if (inferenceConfig.fallback && inferenceConfig.fallback !== provider) { - providers.push(inferenceConfig.fallback as 'anthropic' | 'openrouter'); + providers.push(inferenceConfig.fallback as ConcreteProvider); } return providers; } async function generateWithProvider( - provider: InferenceProvider, + provider: ConcreteProvider, opts: GenerateOptions ): Promise { switch (provider) { @@ -121,11 +146,41 @@ async function generateWithProvider( return generateWithOpenRouter(opts); case 'anthropic': return generateWithAnthropic(opts); + case 'ollama': + return generateWithOllama(opts); default: throw new Error(`Unknown provider: ${provider}`); } } +async function generateWithOllama(opts: GenerateOptions): Promise { + if (!isOllamaEnabled()) { + throw new Error( + 'Ollama memory model not configured — set MEMORY_MODEL_PROVIDER=ollama and MEMORY_MODEL', + ); + } + + let systemPrompt = opts.systemPrompt || 'You are Clude, an AI with persistent memory.'; + if (opts.featureInstruction) { + systemPrompt += `\n\n${opts.featureInstruction}`; + } + + let userContent = opts.userMessage; + if (opts.context) { + userContent = `${opts.context}\n\n---\n\n${opts.userMessage}`; + } + + const mm = getMemoryModelConfig(); + return generateOllamaResponse({ + ollamaUrl: mm.ollamaUrl, + model: mm.model, + systemPrompt, + messages: [{ role: 'user', content: userContent }], + timeoutMs: mm.timeoutMs, + options: opts.maxTokens ? { num_predict: opts.maxTokens } : undefined, + }); +} + async function generateWithOpenRouter(opts: GenerateOptions): Promise { if (!isOpenRouterEnabled()) { throw new Error('OpenRouter not configured'); diff --git a/packages/shared/src/core/memory-model-config.ts b/packages/shared/src/core/memory-model-config.ts new file mode 100644 index 000000000..c5a4fef02 --- /dev/null +++ b/packages/shared/src/core/memory-model-config.ts @@ -0,0 +1,46 @@ +import { config } from '../config'; + +// ============================================================ +// MEMORY-MODEL CONFIG (effective, with SDK runtime override) +// +// The memory model (CludeMem) is configured from env at startup +// (config.memoryModel), but the SDK / desktop app needs to enable it +// programmatically AFTER config has frozen — e.g. Cortex "local mode". +// This module provides a single source of truth: env config, optionally +// overridden at runtime via _configureMemoryModel(). Mirrors the +// _configureEmbeddings() override pattern in embeddings.ts. +// ============================================================ + +export type MemoryModelProvider = '' | 'ollama' | 'anthropic' | 'openrouter'; + +export interface MemoryModelConfig { + provider: MemoryModelProvider; + model: string; + ollamaUrl: string; + timeoutMs: number; +} + +let _override: Partial | null = null; + +/** + * SDK / desktop runtime override for the memory model (e.g. local mode). + * Pass only the fields you want to change; the rest fall back to env config. + */ +export function _configureMemoryModel(cfg: Partial): void { + _override = cfg; +} + +/** @internal Test helper — clears any runtime override. */ +export function _resetMemoryModelConfig(): void { + _override = null; +} + +/** The effective memory-model config: runtime override merged over env config. */ +export function getMemoryModelConfig(): MemoryModelConfig { + return { + provider: (_override?.provider ?? config.memoryModel.provider) as MemoryModelProvider, + model: _override?.model ?? config.memoryModel.model, + ollamaUrl: _override?.ollamaUrl ?? config.memoryModel.ollamaUrl, + timeoutMs: _override?.timeoutMs ?? config.memoryModel.timeoutMs, + }; +} diff --git a/packages/shared/src/core/memory-ops.ts b/packages/shared/src/core/memory-ops.ts new file mode 100644 index 000000000..00792df06 --- /dev/null +++ b/packages/shared/src/core/memory-ops.ts @@ -0,0 +1,105 @@ +import { createChildLogger } from './logger'; +import { generateResponse as generateClaudeResponse } from './claude-client'; +import { + generateOpenRouterResponse, + isOpenRouterEnabled, + type CognitiveFunction, +} from './openrouter-client'; +import { generateOllamaResponse } from './ollama-client'; +import { isOllamaEnabled } from './inference'; +import { getMemoryModelConfig } from './memory-model-config'; + +const log = createChildLogger('memory-ops'); + +// ============================================================ +// MEMORY-OPS ROUTER +// +// Routes memory-operation cognitive functions to the local memory model +// (CludeMem via Ollama) when it is enabled, otherwise to the frontier path. +// Personality functions (reply/reflect/emergence/dream) are NOT memory ops +// and never route here. Structured ops pass a JSON schema (`format`) to the +// local model. On a local failure the router falls back to the frontier path +// (graceful — deliberately distinct from the raw generate({provider:'ollama'}) +// path, which surfaces errors). See specs design Sections 5 and 9. +// ============================================================ + +/** Cognitive functions that are memory operations (eligible for the local model). */ +export const MEMORY_OP_FUNCTIONS: ReadonlySet = new Set([ + 'classify', + 'extract', + 'entity', + 'temporal', + 'importance', + 'summarize', + 'reconcile', + 'query', +]); + +export function isMemoryOp(fn: CognitiveFunction): boolean { + return MEMORY_OP_FUNCTIONS.has(fn); +} + +export interface MemoryOpOptions { + cognitiveFunction: CognitiveFunction; + userMessage: string; + systemPrompt?: string; + context?: string; + /** JSON schema for structured output. Applied on the local path only. */ + format?: Record; + maxTokens?: number; + temperature?: number; +} + +function buildUserContent(opts: MemoryOpOptions): string { + return opts.context ? `${opts.context}\n\n---\n\n${opts.userMessage}` : opts.userMessage; +} + +/** + * Run a memory operation. Uses the local memory model when it is enabled AND + * the function is a memory op; otherwise the frontier path (OpenRouter cognitive + * routing, or the Claude client). Falls back to frontier if the local call fails. + */ +export async function generateMemoryOp(opts: MemoryOpOptions): Promise { + if (isOllamaEnabled() && isMemoryOp(opts.cognitiveFunction)) { + const mm = getMemoryModelConfig(); + try { + return await generateOllamaResponse({ + ollamaUrl: mm.ollamaUrl, + model: mm.model, + systemPrompt: opts.systemPrompt, + messages: [{ role: 'user', content: buildUserContent(opts) }], + format: opts.format, + options: { + temperature: opts.temperature ?? 0, + ...(opts.maxTokens ? { num_predict: opts.maxTokens } : {}), + }, + timeoutMs: mm.timeoutMs, + }); + } catch (err) { + log.warn( + { cognitiveFunction: opts.cognitiveFunction, err: (err as Error).message }, + 'Local memory-op failed; falling back to frontier', + ); + } + } + + return generateFrontier(opts); +} + +async function generateFrontier(opts: MemoryOpOptions): Promise { + if (isOpenRouterEnabled()) { + return generateOpenRouterResponse({ + messages: [{ role: 'user', content: buildUserContent(opts) }], + systemPrompt: opts.systemPrompt, + cognitiveFunction: opts.cognitiveFunction, + maxTokens: opts.maxTokens, + temperature: opts.temperature, + }); + } + return generateClaudeResponse({ + userMessage: opts.userMessage, + context: opts.context, + featureInstruction: opts.systemPrompt, + maxTokens: opts.maxTokens, + }); +} diff --git a/packages/shared/src/core/ollama-client.ts b/packages/shared/src/core/ollama-client.ts new file mode 100644 index 000000000..c2ab4e67c --- /dev/null +++ b/packages/shared/src/core/ollama-client.ts @@ -0,0 +1,134 @@ +import { z } from 'zod'; +import { createChildLogger } from './logger'; + +const log = createChildLogger('ollama-client'); + +// ============================================================ +// LOCAL OLLAMA CHAT CLIENT +// +// Talks to a local Ollama server via its native /api/chat endpoint +// (stream:false). Supports JSON-schema structured output via `format`. +// Zero API cost, fully offline. Mirrors the local embedding provider in +// embeddings.ts. This is the host path for the CludeMem memory model +// (and any stock Ollama chat model). See specs design Sections 5 and 9. +// +// Endpoint contract: +// POST {ollamaUrl}/api/chat +// { model, messages, stream:false, format?, options? } +// → { message: { role, content }, done, ... } +// +// The client is pure: callers pass ollamaUrl/model explicitly (no config +// import) so it stays trivially testable and reusable from the SDK. +// ============================================================ + +export interface OllamaMessage { + role: 'system' | 'user' | 'assistant'; + content: string; +} + +export interface OllamaGenerateOptions { + /** Base URL of the Ollama server, e.g. http://localhost:11434 */ + ollamaUrl: string; + /** Model tag, e.g. "gemma3:4b" or "cludemem-e4b" */ + model: string; + /** Conversation turns */ + messages: OllamaMessage[]; + /** Optional system prompt, prepended as a system message */ + systemPrompt?: string; + /** JSON schema for structured output (Ollama `format`). Omit for free text. */ + format?: Record; + /** Runtime/sampling options (temperature, num_ctx, ...) */ + options?: { temperature?: number; num_ctx?: number; [k: string]: unknown }; + /** Per-request timeout in ms (default 20000) */ + timeoutMs?: number; +} + +const OllamaChatResponse = z.object({ + message: z.object({ + role: z.string().optional(), + content: z.string(), + }), + done: z.boolean().optional(), +}); + +const DEFAULT_TIMEOUT_MS = 20_000; + +/** + * Generate a chat completion from a local Ollama model. + * Throws on non-2xx, timeout, or malformed response. + */ +export async function generateOllamaResponse(opts: OllamaGenerateOptions): Promise { + const { ollamaUrl, model, timeoutMs = DEFAULT_TIMEOUT_MS } = opts; + if (!ollamaUrl) throw new Error('Ollama client: ollamaUrl is required'); + if (!model) throw new Error('Ollama client: model is required'); + + const messages: OllamaMessage[] = []; + if (opts.systemPrompt) messages.push({ role: 'system', content: opts.systemPrompt }); + messages.push(...opts.messages); + + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + + try { + const response = await fetch(`${ollamaUrl}/api/chat`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ + model, + messages, + stream: false, + ...(opts.format ? { format: opts.format } : {}), + ...(opts.options ? { options: opts.options } : {}), + }), + signal: controller.signal, + }); + + if (!response.ok) { + const errorText = await response.text().catch(() => ''); + log.error({ status: response.status, model }, 'Ollama API error'); + throw new Error(`Ollama API error: ${response.status} ${errorText}`); + } + + const parsed = OllamaChatResponse.safeParse(await response.json()); + if (!parsed.success) { + log.error({ model }, 'Ollama returned an invalid response shape'); + throw new Error('Ollama client: invalid response shape'); + } + + log.debug({ model, structured: Boolean(opts.format) }, 'Ollama response generated'); + return parsed.data.message.content; + } catch (err) { + if ((err as Error)?.name === 'AbortError') { + log.error({ model, timeoutMs }, 'Ollama request timed out'); + throw new Error(`Ollama request timed out after ${timeoutMs}ms`); + } + throw err; + } finally { + clearTimeout(timer); + } +} + +/** Single-turn convenience helper. */ +export async function askOllama( + prompt: string, + opts: Omit, +): Promise { + return generateOllamaResponse({ ...opts, messages: [{ role: 'user', content: prompt }] }); +} + +/** + * Liveness probe — true if the Ollama server responds on /api/tags. + * Used by `clude doctor` and the desktop-app integration contract (Spec 1c). + */ +export async function pingOllama(ollamaUrl: string, timeoutMs = 3_000): Promise { + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + try { + const res = await fetch(`${ollamaUrl}/api/tags`, { signal: controller.signal }); + return res.ok; + } catch { + return false; + } finally { + clearTimeout(timer); + } +} diff --git a/packages/shared/src/core/openrouter-client.ts b/packages/shared/src/core/openrouter-client.ts index 86c3a5ccd..a9aa6d530 100644 --- a/packages/shared/src/core/openrouter-client.ts +++ b/packages/shared/src/core/openrouter-client.ts @@ -101,7 +101,14 @@ export type CognitiveFunction = | 'entity' // Entity extraction (fast, lightweight) | 'importance' // Importance scoring (fast, lightweight) | 'summarize' // Memory compaction/summarization (general) - | 'web_search'; // Web-augmented responses (general + search) + | 'web_search' // Web-augmented responses (general + search) + // Memory-op functions (CludeMem). Frontier models below are the fallback + // used when the local memory model is off; see core/memory-ops.ts. + | 'classify' // Memory-type classification + importance/tags + | 'extract' // Atomic memory/fact extraction + | 'temporal' // Event-date + temporal-link extraction + | 'reconcile' // Contradiction/duplicate verdicts + | 'query'; // Query understanding / expansion const COGNITIVE_MODEL_MAP: Record = { reply: OPENROUTER_MODELS['claude-sonnet-4.6'], // Quality matters for public replies @@ -112,6 +119,11 @@ const COGNITIVE_MODEL_MAP: Record = { importance: OPENROUTER_MODELS['llama-3b'], // Fast importance scoring summarize: OPENROUTER_MODELS['llama-70b'], // Good enough for compaction web_search: OPENROUTER_MODELS['llama-70b'], // Fast, good at synthesis + classify: OPENROUTER_MODELS['llama-3b'], // Fast classification fallback + extract: OPENROUTER_MODELS['llama-70b'], // Extraction fallback + temporal: OPENROUTER_MODELS['llama-3b'], // Fast temporal fallback + reconcile: OPENROUTER_MODELS['llama-70b'], // Reconciliation fallback + query: OPENROUTER_MODELS['llama-3b'], // Fast query understanding fallback }; /** diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.ts index 43afe56f0..4c4e3598e 100644 --- a/packages/shared/src/index.ts +++ b/packages/shared/src/index.ts @@ -1 +1,11 @@ export { config } from './config'; + +// Memory-model capability checks (top-level so consumers and the desktop-app +// integration contract can probe without a deep import). The heavy clients stay +// on their deep subpaths, e.g. `@clude/shared/core/ollama-client`. +export { isOllamaEnabled, isMemoryModelEnabled } from './core/inference'; +export { + _configureMemoryModel, + getMemoryModelConfig, + type MemoryModelConfig, +} from './core/memory-model-config';