Feat/memory context v2#60
Merged
Merged
Conversation
Trust + safety + observability layer over the existing memory + context-pack surface. 18 planning phase docs informed the design (kept locally in .claude/Integrations/memory-context-v2/, gitignored).
memory v2 (ADR-033 + ADR-034):
- suggestion-gated AI extraction: MESSAGE_COMPLETED handler writes MemorySuggestion rows; auto-approve fires only for confidence >= threshold AND sensitivity == NORMAL.
- sensitivity classification at write time: regex pre-filter (AWS, JWT, SSN, CC, private-key, Google, GitHub, OpenAI) redacts content before persistence.
- scopes (USER/THREAD/WORKSPACE/PROJECT) enforced at the query layer.
- retention + pause + pinned + qualityScore + useCount + lastUsedAt + provenanceJson.
- new tables: memory_suggestions, memory_usages, memory_audit_logs, memory_preferences.
- new modules: memory-suggestions, memory-preferences, memory-audit, memory-usage.
- audit row outlives memory deletion (memoryId nullable) for RTBF compliance.
context v2 (ADR-035 + ADR-036):
- scope enum + scopeRef; legacy free-text scope preserved as legacy_scope.
- item type enum (TEXT/FILE/URL/MARKDOWN/SNIPPET/MEMORY_REF) + legacy_type.
- visibility (PRIVATE/WORKSPACE/PUBLIC); per-pack and per-item enablement and pinning.
- new tables: context_pack_versions (immutable, pruned at 20), context_pack_usages, context_pack_attachments, context_pack_templates.
- repository auto-increments version on every mutation.
memory + context integration v2 (ADR-037 + ADR-038):
- POST /internal/memories/retrieve returns a typed RetrievalBundle — single source of truth for chat assembly, inspector, and receipts.
- ChatThread gains useMemory / useContext toggles; ChatMessageContextReceipt table persists the bundle; GET /chat-messages/:id/context-receipt enforces per-user ownership and sanitizes REDACTED content.
- 19 new events (memory.*, context_pack.*, context.receipt_written, chat_thread.*_toggled).
frontend:
- /memory rebuilt as 3-tab page (Saved / Suggestions / Audit) with scope, source, sensitivity, search filters.
- suggestion approve / reject / suppress-similar UI; audit timeline; controller hook orchestrates tabs + filters.
- 8 locale files + i18n.types.ts updated atomically with 29 new keys (real translations per locale).
infra:
- .env.example, scripts/install.{sh,ps1} include the 14 new V2 env vars.
- two non-destructive Prisma migrations (additive columns + new tables + safe enum coercion of legacy free-text columns).
validation:
- root typecheck across all 14 workspaces clean.
- root lint clean (0 errors).
- backend tests: 593 passed across all workspaces.
- frontend tests: 512 passed.
- build: all workspaces succeed.
deferred to follow-up sessions (documented in planning docs):
- sensitivity classifier Ollama call (regex pre-filter ships; Ollama fallback enqueued).
- memory + pack embedding manager and cosine ranking (schema is ready; embedding pipeline lands in next session).
- compose-time preview popover UI and receipt-button placement in chat (the read endpoint is in place).
- context-pack version revert UI / template gallery / NDJSON import-export (Phase 3 slice).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up the 5 items the initial v2 commit deferred and ships them end-to-end on the same branch. Embeddings (memory + context-pack): - new migration: embedding vector(768) + embedded_at columns on memory_records and context_pack_items, plus ivfflat cosine indexes (matches nomic-embed-text dimension already used by workspace embeddings). - MemoryEmbeddingManager wraps fetchEmbedding + repo cosine search; embedding upsert is fire-and-forget on create / update-with-content-change. - ContextPackEmbeddingManager mirrors the pattern for pack items. - MemoryRetrievalService now blends semantic cosine into candidate scoring (lexical fallback retained) AND pulls attached pack items into the bundle (scope-filtered + pinned-first + cosine-ranked). Ollama-backed sensitivity classifier: - classifyWithOllama() runs after the regex pre-filter on AI-extracted memories; verdicts NORMAL / SENSITIVE / REDACTED with bounded prompt + 5s timeout + Zod schema validation; fail-soft falls back to NORMAL. - AppConfig gains MEMORY_SENSITIVITY_MODEL (default gemma3:4b). Context-pack versions: - new ContextPackVersionsModule: list, get-one, snapshot, revert, diff endpoints. - snapshot stored as JSON payload; revert restores items inside one transaction and writes a forward version. - diff returns ADDED/REMOVED/CHANGED/UNCHANGED per item plus packMetadataChanged. - auto-prune to CONTEXT_VERSION_RETENTION_COUNT (default 20). Context-pack templates: - new ContextPackTemplatesModule + 6 seed system templates (engineering, product, sales, support, personal, research). - POST /context-pack-templates/:id/clone copies template payload into a new private pack. Import / export: - /memories-portable/export NDJSON + /memories-portable/import (re-runs the full create pipeline per row). - /context-packs/:id/export JSON + /context-packs/import (creates a new pack + items). Chat-service: preview + receipt write + thread toggles: - POST /chat-threads/:id/preview-context dry-runs memory + pack retrieval for a draft. - ChatMessagesService writes a context receipt after every assistant message. - ChatThread update DTO + service publish CHAT_THREAD_MEMORY_TOGGLED / CHAT_THREAD_CONTEXT_TOGGLED on flip. Frontend: - receipt button on every assistant message bubble opens a dialog with memories + pack items + warnings + token budget. - preview button on the composer dry-runs the bundle for the current draft. - thread settings drawer gains memory/context switches. - new types + 2 new enums (RetrievalReason + VersionDiffStatus), 5 new repos, 7 new hooks. i18n: receipt (15 keys) + preview (12 keys) + 4 chat thread-toggle keys translated to en/ar/de/es/fr/hi/it/pt/ru; i18n.types.ts updated in the same commit. Shared utility: toPrismaJsonInput / fromPrismaJsonValue eliminates `as unknown as Prisma.InputJsonValue` across version + template + portable services. Validation: - root typecheck: 0 errors across all 14 workspaces. - root lint: 0 errors. - memory-service: 66 tests pass; chat-service: 367 tests pass; full root: 593 backend tests pass; frontend: 512 tests pass. Out of scope for this session (functionally available via API, UI follow-up): - /context page UI for version list / diff modal / template gallery / export-import buttons (backend endpoints + frontend hooks + repos are all in place; only the page-level wiring is deferred). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brief description of changes.
Type
Checklist
npm run lintpassesnpm run buildpassesnpm run testpasses