feat: add structured extraction with provenance receipts (#156)

[ezutfen]

Closes #156.

Automatic capture currently splits text without preserving speaker roles or evidence spans. This change adds POST /v1/extract and stored provenance receipts. It supports preview and idempotent proposed writes. The SDK and OpenAPI expose the versioned contract.

The Hermes feature flag sends structured messages through the new API. Provider failures preserve the request and retry key in volatile storage. Disabling the flag restores the existing classifier pipeline. The flag defaults to false.

Extraction preserves assertion mode, caller-supplied roles, evidence spans, shared roots, and literal source cues. The #162D-shaped regression retains this provenance when the suggested kind is fact. Extraction does not assign risk, consequence, sensitivity, or admission authority. Production admission behavior remains unchanged.

Migration 036 explicitly revokes UPDATE and DELETE inherited from migration 003's default grants. Both extraction tables are append-only to engram_app, which has SELECT/INSERT only. The existing UPDATE triggers and link constraints provide additional protection. Owner/migration-role maintenance remains available.

Verification

• Verified commit: 5d8af6a64f0d425613881f863e4b43f11b687d91.
• Exact-head GitHub CI passed: Actions run 33999122992. All eight jobs passed. Both new privilege regressions ran and passed in database shard 4-of-4.
• All 33 focused extraction tests passed. The new tests verify effective SELECT/INSERT-only privileges on both tables after initial migration and reapplication. Actual UPDATE/DELETE attempts on visible owned rows fail with SQLSTATE 42501. The run and link remain unchanged after each attempt, including the attempted cascading run deletion.
• make check passed inside Docker Compose.
• The full make compose-ci run passed: 3,543 service tests, 56 SDK tests, 190 hooks tests, and 36 MCP tests.
• All 34 tests initially skipped for missing Node.js or ShellCheck passed in a supplemental Compose run. Total: 3,859 distinct tests passed, with no unresolved skips.
• Migration, FORCE RLS, concurrency, idempotency, API, and integration checks used Compose PostgreSQL 16 with pgvector 0.8.5 and application-role RLS proofs.
• Full-schema downgrade and re-upgrade preserved written memory, tenant configuration, and memory kinds. Hermes rollback and provider fallback passed.
• Production-image health, readiness, OpenAPI, and disabled-provider checks passed. All 561 tracked-file hashes matched the verified commit in both images.
• Lock and Python/JavaScript conformance checks passed.

Live evaluation

The configured classification provider used deepseek-ai/DeepSeek-V4-Flash with prompt engram.extract.3 and schema engram.extraction.v1. The frozen set contains 19 synthetic cases. All expected HTTP outcomes passed. Candidate precision and recall were 100%; kind accuracy was 78.9%; retention accuracy was 89.5%. Attribution and evidence checks scored 100%. The provider reported $0.002690244 and a median latency of 4.410 seconds.

The recorded outputs, earlier development runs, and reproducibility checks are included. This small synthetic set does not certify production admission. Caller-supplied roles do not authenticate speakers, and batch roots do not prove independent evidence.

See the completion report and the ADR.