v0.9.4: Adopt Moraine as CodeWhale's memory backend (ingestion adapter + MCP recall)

[Hmbown]

Goal

Adopt Moraine (https://github.com/eric-tramel/moraine, Apache-2.0) as CodeWhale's long-term
agent-memory backend: Moraine ingests CodeWhale's persisted sessions losslessly and exposes them as
searchable MCP recall tools (search, search_conversations, list_sessions, get_session,
open). This supersedes the in-repo push/inject memory path (crates/tui/src/memory.rs, which
prepends a <user_memory> block into the system prompt) in favor of pull/recall via Moraine MCP.

Local-first, no prompt injection (MCP is read-only retrieval). CodeWhale keeps persisting sessions;
Moraine indexes them. Supersedes community memory PRs #3381 (@pkeging, memory tags) and
#2933 (@cy2311, hippocampal v2 — glossary/namespaces/rollback/auto-inject) — their ideas inform
the recall layer but the implementation is Moraine-backed, not in-repo.

This is scoped for v0.8.66, not 0.8.65.

Research already done (design is concrete)

Moraine fork: Hmbown/moraine (upstream eric-tramel/moraine, on main @ v0.6.2).

Key finding — the session_json ingest format already fits CodeWhale

crates/moraine-ingest-core/src/dispatch.rs::process_session_json_file reads a whole-file JSON
snapshot and extracts session_doc["messages"] — exactly CodeWhale's shape
(~/.codewhale/sessions/{id}.json → { schema_version, metadata, messages[], artifacts, system_prompt }).
It builds synthetic session_meta + per-session_message records and checkpoints by message index
(incremental ingest), then dispatches each to the harness adapter's normalize().

The fork is already half-wired

crates/moraine-config/src/lib.rs::default_sources() already contains the CodeWhale entry:
harness: "codewhale", glob: "~/.codewhale/sessions/*.json", format: SOURCE_FORMAT_SESSION_JSON.

Work items (Moraine fork)

1. Add adapter crates/moraine-ingest-core/src/sources/codewhale.rs implementing IngestSource
   (harness()="codewhale", format()=SessionJson), mirroring the Hermes session handlers
   (normalize_hermes_session_meta / normalize_hermes_session_message at hermes.rs:791/891).
   Handle CodeWhale content-part types text / thinking / tool_use / tool_result; roles
   user / assistant.
2. Register in sources/mod.rs: pub(crate) mod codewhale; + .register(&codewhale::CODEWHALE)
   in registry(), and add "codewhale" to moraine-config::KNOWN_INGEST_HARNESSES
   (the registry_matches_config_known_harnesses test enforces these agree).
3. Generalize the synthetic builders build_session_meta_record/build_session_message_record
   (dispatch.rs:883/950) to read CodeWhale's nested identity with Hermes-safe fallbacks:
   session_id ← session_doc.session_id OR metadata.id; model ← session_doc.model OR metadata.model;
   session_start ← session_start OR metadata.created_at; last_updated ← last_updated OR metadata.updated_at.
   Hermes files have no metadata, so it's unaffected. compose_hermes_model is a no-op for CodeWhale
   (no base_url).
4. Golden fixture + test mirroring tests/hermes_session_fixture.rs, using a redacted real
   CodeWhale session snapshot.
5. Upstream the adapter to eric-tramel/moraine (PR) so it ships in a Moraine release; keep the
   fork in sync.

Work items (CodeWhale side)

6. Wire moraine-mcp as a recall tool source (stdio moraine mcp server) via CodeWhale's MCP
   config so agents gain the recall tools.
7. Config-gate deprecate the memory.rs push/inject (call sites crates/tui/src/core/engine.rs:801
   and :2996 via crate::memory::compose_block; render prompts.rs:1209; already off by default via
   MemoryConfig.memory_enabled=false) in favor of Moraine pull/recall. Update docs/config comments.

Validation

moraine up ingests ~/.codewhale/sessions; MCP list_sessions/search return CodeWhale sessions
with correct session id / model / timestamps; end-to-end recall in a CodeWhale agent turn.

Known risks / gaps

• CodeWhale session files are whole-file snapshots rewritten on every save; verify the session_json
  "shrank → ignore" guard (dispatch.rs:758) doesn't drop data when a new session has fewer messages
  than a prior one (different file, so likely fine — confirm).
• CodeWhale messages carry no per-message timestamps (only session created_at/updated_at);
  events need synthetic monotonic ts from session start + message index (like hermes_event_dt).
• tool_result content-part exact shape needs confirmation (Anthropic-style
  {type, tool_use_id, content}) — handle defensively.
• Cost/tokens live in metadata.cost / metadata.total_tokens — map into Moraine token accounting if
  desired (non-blocking for recall).

[pkeging]

@Hmbown Thank you for the thorough design. I'm glad the tagging/namespace ideas from #3381 informed the direction — adopting a dedicated memory backend like Moraine makes a lot of sense.
I'd like to help with items 6 and 7 on the CodeWhale side (wiring moraine-mcp as a recall tool source and config-gating the legacy memory.rs path). My experience integrating the WeCom bridge has given me familiarity with CodeWhale's MCP config surface and the relevant code paths.
If you have any early pointers or a rough timeline for v0.8.66, I'll start studying the MCP config and the memory.rs call sites to hit the ground running.
Looking forward to contributing.

[cy2311]

I have implemented the CodeWhale ingestion adapter for Moraine.

PR: Hmbown/moraine#1

Covers all 5 Moraine-side work items from the issue:

1. codewhale.rs - IngestSource implementation handling text/thinking/tool_use/tool_result
2. Registered in SourceRegistry
3. Added to KNOWN_INGEST_HARNESSES + default_sources()
4. 10 integration tests with redacted fixture (all passing)
5. Ready to upstream

Implemented by @cy2311

[findshan]

Nice to see this moving — props to @cy2311 for the Moraine-side adapter (#1) and @pkeging for taking items 6/7. To de-risk the ingest mapping I read CodeWhale's actual session serialization (crates/tui/src/session_manager.rs + crates/tui/src/models.rs); sharing verified specs for the parts the issue flags as unconfirmed, plus one edge case I think matters.

tool_result shape — resolved (it isn't Anthropic-style)

ContentBlock::ToolResult (crates/tui/src/models.rs:111-119) is:

#[serde(rename = "tool_result")]
ToolResult {
    tool_use_id: String,
    content: String,                              // flat string, NOT an array
    #[serde(skip_serializing_if = "Option::is_none")]
    is_error: Option<bool>,
    #[serde(skip_serializing_if = "Option::is_none")]
    content_blocks: Option<Vec<serde_json::Value>>,
},

So on disk: { "type":"tool_result", "tool_use_id", "content": "<string>", "is_error"?, "content_blocks"? }. content is a plain string; is_error and content_blocks are omitted when None. Worth confirming the adapter (a) carries is_error through so failed tool calls are distinguishable in recall, and (b) falls back to content_blocks for multi-part results rather than only reading content.

Content-part coverage — the enum has more than four variants

ContentBlock (models.rs:81-136) is an externally-tagged enum (#[serde(tag = "type")]) with eight variants: text, thinking, tool_use, tool_result, plus image_url, server_tool_use, tool_search_tool_result, code_execution_tool_result. An adapter matching only the first four will silently drop the rest (web-search / code-execution results are exactly the kind of thing you'd want recallable). Suggest a catch-all that preserves unknown parts as text (or logs) instead of discarding.

Per-message timestamps — confirmed absent

Message { role: String, content: Vec<ContentBlock> } (models.rs:74-79) — no per-message timestamp; only session-level metadata.created_at / updated_at (session_manager.rs:98-135). So the synthetic-monotonic-ts plan is required. Rather than created_at + idx·ε, consider spreading indices across the real span — ts(i) = created_at + (updated_at − created_at) · i/(n−1) — so recall ordering reflects actual session duration.

Open question — compaction vs. the "shrank → ignore" guard

This is the one I'd most want a maintainer call on. update_session (session_manager.rs:746-750) does:

session.messages.clear();
session.messages.extend_from_slice(messages);
session.metadata.message_count = messages.len();

i.e. a full in-place replace under the same {id}.json. CodeWhale auto-compacts (context_budget::should_compact, runtime_threads::compact_thread), so a compacted session rewrites the same file with fewer messages and a lower message_count. That looks like exactly the session_json "shrank → ignore" case at dispatch.rs:758 — but it's the same file, not a different one, so the "different file, likely fine" note in the issue may not cover it. Should compaction-shrink be a guard exception (re-checkpoint from 0 on detected shrink), or should CodeWhale persist compacted turns under a new session id so Moraine treats them as a fresh document?

Minor

metadata.cost is a struct (SessionCostSnapshot, six f64 USD/CNY fields), not a scalar — the scalar to map into token accounting is metadata.total_tokens: u64. schema_version is currently 1 (session_manager.rs:19); a forward-compat default keeps old/new files loadable.

Happy to turn any of this into a redacted golden fixture (current schema_version: 1, with tool_result.content_blocks + a non-text part) to extend @cy2311's test set if that's useful.

[Hmbown]

Status update after #3681: the CodeWhale-side Moraine fallback gates are now on main in 9cef270.

Landed pieces:

• moraine-mcp is present in the default MCP templates as a disabled-by-default stdio server using moraine mcp (crates/tui/src/main.rs:1926-1936, crates/tui/src/mcp.rs:2998-3010).
• [memory] moraine_fallback = true now explicitly disables the legacy <user_memory> push/inject path, remember tool registration, and # foo quick-add even when [memory] enabled = true (crates/tui/src/config.rs:1374-1379, crates/tui/src/config.rs:3597-3608, crates/tui/src/memory.rs:120-126).
• Model-facing memory guidance now only recommends Moraine tools when the MCP server/tools are actually present, so agents should not hallucinate recall tools (crates/tui/src/prompts/memory_guidance.md:25-37).

Keeping this issue open because the full acceptance criteria still need the Moraine-side adapter/upstreaming and end-to-end moraine up + MCP recall verification. Thank you @pkeging for the fallback/MCP wiring work that #3681 preserved from #3584, @cy2311 for the Moraine adapter work, and @findshan for the session-shape verification that helps de-risk ingestion.