feat(continuity): land one governed observe-to-episodic-recall vertical for #228

[joshyorko]

Existing-corpus continuity amendment — September 8, 2026

The first vertical must work against a mixed pre-change corpus, not only newly observed events. Reuse #234's shared legacy compatibility/preview reconciliation and #231's identity repair; no separate importer, event store or migration framework.

Core guarantees: existing records, turns, conclusions, retained imports and transferred origins remain accessible under policy without mandatory invented new metadata. Represent what the retained source said, its actual known speaker/producer and source time separately from whether the claim is current, uncertain, corrected, withdrawn or superseded. No bulk conversion of old facts into fresh observations, fabricated original event IDs, reassignment to the new consumer, or assumption that every old /conclusions row was explicitly user-confirmed. Missing evidence of adoption/independent verification stays unknown; existing known supporting evidence must not be discarded either.

Introduce documented compatibility projections for supported legacy shapes. A stable migration/mapping identifier may identify the reconciliation operation, but must never masquerade as a historical producer event or independent corroboration. Preserve handles, bundle origins, source attribution, original time basis and correction links. Support old/new event identity namespaces without #231's lossy fallback collapsing distinct new content.

Backfilled old material does not get a new event date or automatic priority in the recent episodic lane because it was imported today. Keep useful authorized historical/imported/other evidence available through bounded relevant recall/search, with visible limits; retention and current-truth admission are separate. When source time is absent, use an explicit unknown-time compatibility policy rather than inventing recency or dropping all such content.

Reconciliation and corrections: preview proposed source/lineage/status mappings, affected references, conflicts and unresolved cases before apply. No silent re-key, promotion, overwrite or deletion. Existing corrections/withdrawals apply to proven equivalent material arriving through replay, archive, bundle or client paraphrase. Possible equivalence stays visibly unresolved, not falsely independently confirmed and not silently merged/erased. Later source linking, builtin-memory mirroring or restore must not launder legacy assistant prose into primary user evidence. Preserve the existing mirror-path trust audit in this issue's comments.

Adapter and consumer requirements: adapters emit truthful capture metadata for new events while carrying known legacy provenance unchanged. Older accepted request shapes cannot widen authority or clear dispositions; unsupported write combinations fail explicitly. Consumers validate compatibility on an isolated pre-change snapshot and apply only reviewed reconciliation plans, with rollback/restore receipts and actual loaded-version evidence. Enabling observation intake must not trigger a whole-corpus reimport.

Additional acceptance gates

• Begin with pre-change Codex/Hermes data, assistant-authored conclusions, imported summaries, bundle-origin records, useful other material, missing lineage/timestamps and existing corrections. Add new events from two supported clients with Dreamer disabled.
• Existing useful memories and new episodic evidence are both recallable within scope/budget, without invented currentness, freshness, original speaker or confirmation.
• Preview/cancel/apply/retry and a concurrent correction preserve stable references/origins, truthful mappings and separate source/status timelines.
• Old archive reimport, mirrored paraphrase, event replay, bundle round trip and restored backup do not reinstate proven withdrawn claims as current; insufficient ancestry is reported explicitly.
• Compare known independent evidence weight before/after backfill and cross-client repetition: no increase solely from import, mapping, copying or reconciliation. Report supported legacy shapes and unresolved lineage; no perfect semantic-loop suppression claim.

---

Outcome / parent

First bounded implementation child of #228: a supported client contributes one safe observation, a different session/client can retrieve it immediately as attributed episodic evidence with Dreamer disabled, and no host assertion becomes a promoted fact or new execution authority.

Priority: P1 after identity/authorization contract review. This is not the entire ChatGPT App, a sync service, or a Codex fork rebase.

Baseline and reuse

At 25d617b5f08593c76d336f90b8ffd1e78f77afbd, src/server.rs exposes turns, conclusions, checkpoints, subjects and episodes, but no dedicated host-observation endpoint. src/service.rs, src/recall.rs, src/protocol.rs, src/store.rs, policy and evidence ledger already own their respective semantics. Some convention-shaped turns already create records; this issue must not claim all current turns are unrecallable.

Use one typed service operation with thin CLI/HTTP adapters and the existing store/search/recall admission path. Name it once in the protocol PR. Codex-side integration remains joshyorko/codex#317; Hermes integration remains this repository's generic adapter. #193 owns public-reference authorization, not a second store.

Contract to settle before implementation

• Stable contribution/event identity scoped by authenticated/configured principal, producer instance, profile/workspace and session. Same ID + same canonical payload => same receipt; same ID + different payload => explicit conflict. Crash after commit/before acknowledgement must not duplicate evidence or confidence.
• Preserve Cross-client continuity: host-native memory contributions + ChatGPT App bridge #228's first-class producer dimensions: harness/app, surface, adapter, transport, version; model is separate telemetry. An assistant speaking about a user is not a user-origin event.
• Caller-supplied actor, confidence, lineage and source_kind are claims, not credentials. Only a trusted host capture/adoption path can assert primary user evidence. Unknown-source evidence stays weak.
• Preserve event/observed time, ingestion time and claimed-valid time separately where supplied. A newly imported old conversation is not a newly verified fact. Missing times remain unknown; future-clock and late-arrival cases are explicit.
• Carry source references and derived_from links through recall and re-contribution. A host-native summary, assistant paraphrase or MemoryD-derived observation cannot count as independent primary corroboration.
• Hash dedupe is not semantic lineage proof. Unknown paraphrases must not gain promotion weight merely because a new ID/client supplied them; no promise of perfect semantic ancestry detection.
• Whole request has policy-before-persistence and closed/size-bounded fields. Rejected content must not appear in logs, error strings, receipts or debug metadata.
• Receipt distinguishes accepted evidence, duplicate, rejected, not immediately recallable, and pending consolidation. Never report an unacknowledged write as saved.

Fast recall

Return a bounded explicitly labelled episodic section, separate from promoted facts. Safe-but-weak evidence can be recalled under policy; unsafe/quarantined evidence stays withheld. Reuse current relevance and scope machinery, with known omissions/gaps reported rather than a second ranking engine. No automatic replay of a remembered task or old permission.

Required tests

• Two different client/session fixtures: observe -> immediate recall with Dreamer disabled.
• Before/after core policy snapshots prove zero automatic promoted records from host-only evidence.
• Same event retry, concurrent retry, commit-then-lost-response, changed-payload conflict and partial-batch rejection.
• Round trip A -> MemoryD -> B -> MemoryD has zero new primary weight. Paraphrased unknown lineage stays weak. Independent actual user confirmation is represented distinctly.
• Late imported evidence versus recent correction; planned/reported/verified work; ambiguous subjects; similar repository names and changed branches/heads.
• Profile/workspace/principal bleed attempts, revoked grants, supplied source-ID spoofing and forbidden fields are rejected.
• Poisoned incident/runbook text cannot select tools, grant permissions, or become a trusted procedure through repetition.
• Query-with-no-support abstains; unrelated recall and omitted candidates are measured, not rewarded.

Delivery and gates

Split protocol/intake and retrieval if either is independently reviewable. Land local typed tests first; don't delay on remote OAuth or UI. Before enabling a host, validate producer/grant identity end to end. Leave #228 open for native Codex coexistence, MCP/ChatGPT security and full cross-client acceptance. No new DB backend, scheduler, embeddings requirement, auto-Dreamer apply, or whole-history ingestion.

Research-informed tests: LongMemEval (updates, temporal reasoning, abstention), memory poisoning study. These motivate failure cases, not claims that MemoryD already passes them.

[joshyorko]

Include the existing built-in-memory mirror in the trust audit

The current Hermes adapter's on_memory_write() sends add/replace content through /v1/conclusions. A native memory tool call can be model-initiated; the existence of that tool call is not itself proof of an explicit human adoption event.

Before declaring the new weak-observation boundary complete, trace this existing path through Service::conclusions and Dreamer evidence weighting. Preserve convenience and acknowledged durability, but distinguish source user assertion, assistant-authored note, inherited memory and explicitly user-adopted conclusion. Do not allow a weaker contribution to bypass its promotion ceiling by being mirrored back through a stronger endpoint.

Add a test where recalled assistant-derived content is written through the builtin memory tool and mirrored, then consolidated: no new independent primary weight or changed author should arise solely from that round trip. A separately captured genuine user correction/adoption remains eligible under the normal core policy. This is a source-identified trust boundary to test, not a claim that a live poisoning exploit was executed.