Release 1.6: universal local document imports and hardening

.claude-plugin/marketplace.json

@@ -9,7 +9,7 @@
       "name": "engraphis-memory",
       "source": "./",
       "description": "Discipline for giving agents durable, scoped, explainable memory across sessions and repos with the Engraphis MCP tools.",
-      "version": "1.5"
+      "version": "1.6"
     }
   ]
 }

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "engraphis-memory",
-  "version": "1.5",
+  "version": "1.6",
   "description": "Give agents durable, scoped, explainable memory across sessions and repos via the Engraphis MCP tools. Use when you learn something worth keeping, need prior context before acting, or ask why/how a fact changed. Covers remember/recall, why/timeline, forget/pin/correct, sessions, and code search.",
   "author": {
     "name": "The Engraphis Authors",

.claude-plugin/skill-assets.sha256

@@ -1,5 +1,5 @@
-d30ad152dcc4c82ce10e7167fdfe67e709358e5f435293939125f2d6cffc5b7e  .claude-plugin/marketplace.json
-28dcd15a7a186f8cb8a15705f1bd7734086167991c4acc28ec2cfea59a2374ab  .claude-plugin/plugin.json
+4c18cdb509babb853ac7e5283ca9b309e2669b82ff098f3acf33238a9e4c1114  .claude-plugin/marketplace.json
+94bfa06317a8fe6a6a7e204bb70c5abdc9e4bbc34d79dd6f8447a30140bc8b85  .claude-plugin/plugin.json
 055655db84af07561d002f0c69744313d8413c39f3e873f941f0fa0b1e76dc66  skills/engraphis-memory/references/CONVENTIONS.md
 62019760766ff472a76a0f81437898f39e3c1fe2631732b7b7733e50c1ad837f  skills/engraphis-memory/references/SCOPING.md
 4ce83a2768680ec84488a767fc3bd6cd62688d785010a0abd1d4b3edbf14d03a  skills/engraphis-memory/references/TOOLS.md

AGENTS.md

@@ -21,7 +21,7 @@ most common mistake here.
 | Status | Primary scoped, bi-temporal, interface-driven implementation. | Compatibility/reference implementation with flat namespaces. |
 | Model | Scoped + bi-temporal + typed; interface-driven. | Single flat `namespace` string per memory. |
 | Code | `engraphis/core/`, `engraphis/backends/`, `eval/`, `tests/`, `scripts/migrate_to_v2.py` | `engraphis/app.py`, `config.py`, `models.py`, `routes/`, `stores/`, `engines/`, `llm/`, `static/` |
-| Data | new v2 schema (`SCHEMA_VERSION = 15`) | `engraphis_v1.db` |
+| Data | new v2 schema (`SCHEMA_VERSION = 16`) | `engraphis_v1.db` |
 | Entry | `engraphis.MemoryEngine.create()` / `engraphis.create_memory_engine()` → `engraphis/factory.py` → `core/engine.py` | Internal reference only; never a public launcher |
 
 **Rule:** build new capability on **v2** (`core/` + `backends/`) behind the interfaces.
@@ -198,19 +198,25 @@ These are pure, unit-tested functions — change them only with a corresponding
 
 ---
 
-## 5. Data model cheat-sheet (`core/interfaces.py`, `core/schema.py` — `SCHEMA_VERSION = 15`)
+## 5. Data model cheat-sheet (`core/interfaces.py`, `core/schema.py` — `SCHEMA_VERSION = 16`)
 
 - **Scope hierarchy:** `workspace → repo → session → memory`. Scopes: `session|repo|workspace|user`.
 - **Bi-temporal validity on every record:** world-time `valid_from/valid_to` +
   system-time `ingested_at/expired_at`. Reads hide facts outside their validity window
   unless `include_invalid=True` or an `as_of` anchor is given.
 - **IDs:** ULID, time-sortable, **typed prefixes** (`ws_`, `repo_`, `ses_`, `mem_`, `ent_`,
-  `edg_`, `sym_`, `evt_`, `job_`, `aud_`, `dev_`, `rcpt_`) — `core/ids.py`.
+  `edg_`, `sym_`, `evt_`, `job_`, `aud_`, `dev_`, `rcpt_`, `vlt_`, `src_`) — `core/ids.py`.
   Lexicographic sort == chronological.
 - **Tables:** `workspaces`, `repos`, `sessions`, `memories`, `mem_vectors`, `embedding_state`,
   `mem_fts` (FTS5 + plain-table fallback), `entities`, `edges` (bi-temporal), `mem_links`,
   `memory_entities`, `symbols`, `code_edges`, `code_files`, `code_memory_links`,
-  `operation_receipts`, `events`, `audit`, `memory_tombstones`, `schema_migrations`.
+  `operation_receipts`, `events`, `audit`, `memory_tombstones`, `source_vaults`,
+  `source_imports`, `source_import_items`, `schema_migrations`.
+- **Local document sources:** source collections are scope-bound, resumable manifests. Their
+  paths, digests, and import state are provenance; source folders never create implicit memory
+  scopes. `kind="documents"` is the source-neutral adapter, while `kind="obsidian"` retains
+  the rich Markdown adapter and its existing lineage. Import jobs persist the optional session
+  target, and schema checks keep source-job lineage and per-job items in that exact session.
 - **Erasure markers contain no memory content.** `memory_tombstones.export_class` is strictly
   `never_export|remote_erasure`; only `remote_erasure` may cross a sync boundary.
 - **Vectors are stored L2-normalized** so cosine similarity == dot product.
@@ -250,6 +256,12 @@ These are pure, unit-tested functions — change them only with a corresponding
   with `engraphis/mcp_server.py`.
 - **`docs/SYNC.md`** — cloud sync (Pro): architecture, the convergent merge, CLI usage, and the
   untrusted-bundle security model.
+- **`docs/DOCUMENT_IMPORT.md`** — universal local-document import, source safety, re-import,
+  conflicts, format adapters, and dashboard/CLI flows. Keep it synchronized with the parser,
+  importer, dashboard, and import-report schema.
+- **`docs/OBSIDIAN_IMPORT.md`** — the rich Obsidian Markdown adapter (frontmatter, wikilinks,
+  aliases, attachments) and its compatibility command. It supplements, rather than replaces,
+  the universal document-import guide.
 - **`AGENTS.md`** (this file) + **`CLAUDE.md`** — how to work in the repo.
 - **`skills/engraphis-memory/`** — portable Agent Skill (SKILL.md + `references/`) that teaches any
   MCP-capable agent the *memory discipline* (when to remember/recall, scoping, tool selection).

CHANGELOG.md

@@ -5,12 +5,35 @@ All notable changes to Engraphis are documented here. Format loosely follows
 
 ## [Unreleased]
 
+## [1.6] - 2026-08-08
+
+Minor release advancing the v2 engine through schema 16 with deterministic sync state, trusted
+local document and Obsidian import, tighter trust boundaries, synchronized agent guidance, and
+stronger release and evaluation evidence.
+
+### Added
+
+- A dependency-free, source-neutral local document importer for Markdown, plain text,
+  reStructuredText, HTML, JSON/JSONL, CSV/TSV, configuration/XML text, and stdlib-readable
+  source code, RTF, DOCX/ODT, XLSX/ODS, PPTX/ODP, and EPUB documents, with existing local
+  adapters for PDF text, image OCR, and explicitly local-model audio/video transcription.
+  `engraphis import documents` and the
+  dashboard’s **Import local documents** flow
+  provide strict previews, safe per-file reporting, resumable source manifests, temporal
+  re-import history, and explicit conflict choices. Obsidian remains the rich Markdown adapter.
+- Offline, repeatable Obsidian-vault import with strict dry-run previews, source
+  safety exclusions, resumable per-note progress, temporal re-import history, and
+  a trusted-owner dashboard wizard that uploads only `.md` note bytes plus content-free
+  attachment manifests. It ships through
+  `engraphis import obsidian`, the `engraphis-import` console alias, and a deprecated
+  v1 seed-script wrapper that maps legacy namespaces to v2 workspaces.
+
 ### Security
 
 - Fail closed on new `user`-scope memory writes until records carry an immutable owner identity;
   preserve historical reads and the existing promotion rejection instead of presenting
   workspace-bound rows as private personal memory.
-- Load optional dotenv configuration only from the owner-private
+- Parse bounded dotenv-style configuration without an optional runtime dependency, and load it only from the owner-private
   `~/.engraphis/config.env` or an absolute owner-private file selected by
   `ENGRAPHIS_ENV_FILE`; arbitrary working-directory `.env` files are not a trust boundary.
 - Clarify Cloud Sync credential-origin binding, secret-manager-only unattended credentials,
@@ -19,11 +42,30 @@ All notable changes to Engraphis are documented here. Format loosely follows
 - Advance through schema 15: schema 12 classifies content-free erasure markers so local-only
   `never_export` markers remain private and only validated `remote_erasure` markers may cross
   sync boundaries; schema 13 adds per-memory hybrid logical clocks for deterministic
-  descriptive-state sync; schema 14-15 add the local source-import manifest for document
-  and note-collection tracking with scope-security triggers.
+  descriptive-state sync and durable, content-free proof that a memory crossed a sync boundary;
+  schema 14 adds Obsidian collection and import manifests; schema 15 generalizes them to
+  source-neutral `documents` and `obsidian` adapters, preserves temporal source lineage, enforces
+  adapter/job and target-scope integrity, and retains only bounded, content-free per-job
+  format/result metadata. Schema 16 persists the optional session target on import jobs and
+  enforces exact session equality for source lineage and job items.
+- Bind each trusted-owner dashboard document or Obsidian run to an expiring, owner-session-bound,
+  one-time preview token over the exact note/document bytes, attachment manifest, target, source,
+  and conflict policy; invalidate changed client previews and keep job polling and cancellation
+  bound to the workspace where the job started.
+- Make read-only Store inspection write-free for SQLite and injected/SQLCipher connectors:
+  require injected connectors to expose `open_read_only(path)`, open existing checkpointed files
+  with `mode=ro&immutable=1` plus `PRAGMA query_only=ON`, and reject missing paths or active
+  WAL/rollback journals before a connector can create or recover state.
 
 ### Fixed
 
+- Publish separately backed vector-index changes for service memory-title edits only after the
+  canonical Store row, FTS mirror, portable vector, audit, and commit succeed; late Store failures
+  publish nothing, while post-commit provider failures preserve canonical state and record
+  content-free repair debt.
+- Defer separately backed vector-index upserts and deletes during sync until each canonical apply
+  batch commits, coalesce repeated IDs, publish nothing on late Store failure, and record
+  content-free repair debt if the provider fails after commit.
 - Synchronize the portable memory skill with the live Smart nine-tool and Classic 34-tool
   surfaces, including the two intentionally narrower Smart overlap schemas, trust/origin fields,
   planner and response bounds, context-savings filters, receipt anchors, and expanded health
@@ -47,6 +89,24 @@ All notable changes to Engraphis are documented here. Format loosely follows
   including clean-checkout completion receipts, exact source-question coverage, privacy-safe
   export binding, matched `context_k=2` comparators, and memory-type count evidence.
 
+### Added
+
+- Dashboard Settings panel and startup banner now display the running Engraphis
+  version, fetched from the existing `/api/info` endpoint.
+
+### Fixed
+
+- Wrap `engraphis_get_memory` post-inspect body in error-redaction try/except
+  matching all other Smart gateway tools, preventing internal SQL errors and
+  file paths from leaking through FastMCP error responses.
+- Fix malformed SQLite URI on Windows in `_keyword_search` and `/api/memories`
+  fallback paths: use `Path.resolve().as_uri()` instead of bare string
+  interpolation, matching the store's URI construction.
+- Apply `_graph_csv()` limit enforcement to the `/graph` endpoint's `layers`
+  parameter, matching all other graph endpoints.
+- Log a warning when `ENGRAPHIS_LLM_EXTRA_HEADERS` contains invalid JSON
+  instead of silently dropping the headers.
+
 ## [1.5] - 2026-08-04
 
 Minor release advancing the v2 engine to schema 11 with governed recall recovery,

MANIFEST.in

@@ -10,6 +10,7 @@ recursive-include engraphis/dashboard_assets/vendor *
 include engraphis/commercial_manifest.json
 include LICENSE NOTICE README.md CHANGELOG.md BENCHMARKS.md
 include docs/RECALL_RECOVERY.md
+include docs/DOCUMENT_IMPORT.md docs/OBSIDIAN_IMPORT.md
 include docs/images/context-efficiency.svg
 include pyproject.toml
 include .env.example requirements.txt

README.md

@@ -164,15 +164,19 @@ selection, set `ENGRAPHIS_UPDATE_EXTRAS` to a comma-separated list (for example
 > **Upgrading to 1.5:** schema 10 bounds legacy retention state and schema 11 backfills explicit
 > approval only for eligible pre-review local memories. Pending and quarantined evidence remains
 > gated. Existing 1.4.x databases migrate automatically when Engraphis 1.5 opens them; see the
-> [1.5 release notes](https://github.com/Coding-Dev-Tools/engraphis/blob/main/CHANGELOG.md#150---2026-08-04).
-
-> **Current source:** schema 15 adds the local source-import manifest (vaults, imports,
-> job items) for document and note-collection tracking. Schema 14 introduced the
-> initial note-collection manifest; schema 15 generalized it to support multiple source
-> kinds with content-free scope-security triggers.
-> Schema 12 classifies content-free erasure markers before sync: existing markers migrate to
-> local-only `never_export`; new secure erasures become `remote_erasure` only for non-secret
-> `workspace`/`repo` records that were already eligible for sharing.
+> [1.5 release notes](https://github.com/Coding-Dev-Tools/engraphis/blob/main/CHANGELOG.md#15---2026-08-04).
+
+> **Upgrading to 1.6:** existing 1.5 databases migrate automatically through schema 12, which
+> classifies content-free erasure markers before sync: existing markers become local-only
+> `never_export`, while new secure erasures become `remote_erasure` only for non-secret
+> `workspace`/`repo` records already eligible for sharing. Schema 13 adds per-memory hybrid
+> logical clocks for deterministic descriptive-state sync and durable, content-free proof that a
+> memory crossed a sync boundary. Schema 14 adds the Obsidian collection and import manifests;
+> schema 15 generalizes them to source-neutral local documents, preserves temporal source lineage
+> across re-imports, binds adapters and target scopes, and retains only bounded, content-free
+> per-job format/result metadata. The schema 16 migration persists each import job's optional session target
+> and requires source lineage and job-item attachments to remain in that exact session. See the
+> [1.6 release notes](https://github.com/Coding-Dev-Tools/engraphis/blob/main/CHANGELOG.md#16---2026-08-08).
 
 ---
 
@@ -638,11 +642,36 @@ oversized key files rather than following an unexpected filesystem object.
 
 ## Import files and folders
 
-Import supported documents and code through the dashboard, a local folder, or MCP. Optional
-extractors add offline chunking, structured LLM extraction, document OCR, transcription, and
-PostgreSQL schema ingestion. See the [MCP tool reference](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/MCP_TOOLS.md),
-[architecture guide](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/ARCHITECTURE_V3.md), and [security policy](https://github.com/Coding-Dev-Tools/engraphis/blob/main/SECURITY.md) for formats,
-configuration, and local-resource safeguards.
+The dependency-free universal core scans Markdown, plain text, RST, HTML, JSON/JSONL, CSV/TSV,
+configuration/XML text, source code, RTF, DOCX/ODT, XLSX/ODS, PPTX/ODP, and EPUB into the normal
+v2 memory path. Installed local resource adapters add PDF text, image OCR, and explicitly
+local-model audio/video transcription.
+Start with a zero-write
+preview, then confirm the same source collection explicitly:
+
+```bash
+engraphis import documents /path/to/collection --workspace acme --dry-run
+engraphis import documents /path/to/collection --workspace acme --repo product --yes
+```
+
+The CLI never downloads an embedding model during import. Use a model that is already cached,
+set `ENGRAPHIS_EMBED_MODEL=local:/absolute/model/path`, or explicitly set
+`ENGRAPHIS_EMBED_MODEL` to an empty value to use dependency-free deterministic hashing in
+lexical degraded mode.
+
+The dashboard’s **Import local documents** flow offers the same preview, target scope, source
+label, conflict policy, cancellation, and resumable progress. Re-imports are idempotent,
+preserve temporal history, and report source removals without hard-deleting memories. Obsidian
+remains the rich Markdown adapter for frontmatter, aliases, wikilinks, and attachment references:
+
+```bash
+engraphis import obsidian /path/to/vault --workspace acme --dry-run
+```
+
+See the [document import guide](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/DOCUMENT_IMPORT.md)
+for supported formats, source safety, resume and conflict behavior, optional adapters, and
+limitations; see the [Obsidian adapter guide](https://github.com/Coding-Dev-Tools/engraphis/blob/main/docs/OBSIDIAN_IMPORT.md)
+for Markdown-specific behavior.
 
 ---
 
@@ -663,7 +692,7 @@ file. It never searches the working directory for `.env`, and explicit process v
 
 | Env Var | Default | Description |
 |---------|---------|-------------|
-| `ENGRAPHIS_ENV_FILE` | `~/.engraphis/config.env` | Optional trusted config leaf selected before dotenv values load. An explicit value must be an absolute path to an owner-private regular file; arbitrary working-directory `.env` files are ignored. |
+| `ENGRAPHIS_ENV_FILE` | `~/.engraphis/config.env` | Optional trusted config leaf selected before trusted values load. Its bounded dependency-free parser performs no interpolation. An explicit value must be an absolute path to an owner-private regular file; arbitrary working-directory `.env` files are ignored. |
 | `ENGRAPHIS_DB_PATH` | Source: `<repo>/engraphis.db`; installed: platform user-data directory | SQLite database file. Installed defaults are `%LOCALAPPDATA%\engraphis\engraphis.db` (Windows), `~/Library/Application Support/engraphis/engraphis.db` (macOS), and `$XDG_DATA_HOME/engraphis/engraphis.db` or `~/.local/share/engraphis/engraphis.db` (Linux). The environment variable overrides every default. |
 | `ENGRAPHIS_HOST` | `127.0.0.1` | Server bind address |
 | `ENGRAPHIS_PORT` | `8700` | Dashboard port |