Repository navigation
Conversation
Add a memory MCP server (Streamable HTTP at /mcp/memory) so external agents (coding agents, bots) can read/write Octop expert memory directly, aligned with the in-process MemoryService capabilities. Tools (per expert, bound at connect time via X-Octop-Agent-Id header): - memory_recall(query, limit): full recall pipeline (tokenize + FTS + rerank), returns structured memories + rendered markdown - memory_save(content, source, topic?): persist a structured fact directly into the atom/tree (durable, no extraction) - memory_capture(content, source, session_id?): write an L0 raw event (extraction pipeline); visible immediately via memory_search_raw - memory_search_raw(query, limit): FTS-search L0 raw events (capture visible before extraction) - memory_update(atom_id, new_content, source): deprecate old atom + save new Auth: independent token via OCTOP_MEMORY_MCP_TOKEN (fail-closed if unset); authorization via Authorization: Bearer or X-Octop-Memory-Token. Implementation: - Lives in infra/agents/memory_mcp.py (no api-layer dependency; opens the agent Memory instance via open_memory_kwargs) - One FastMCP per agent, routed by X-Octop-Agent-Id header at /mcp/memory - DNS rebinding protection disabled (server runs behind a reverse proxy) - streamable_http task groups wired into the FastAPI lifespan Tests: tests/unit/agents/test_memory_mcp.py (tools, header routing, token middleware, unified mount).
保留两者: - main: enable_mobile 路由挂载 - PR: memory_mcp MCP server 挂载
- Add assert for server.services to satisfy mypy strict mode - Add return type annotation to _memory closure
基于社区 feat/memory-mcp-server (fea4c59) 的独立扩展分支,仅改 memory_mcp.py: - memory_capture 写 L0 后自动触发提取流水线(extract -> promote -> atom) - 缺省 session 派生 ext:{source}:{user},外部调用无需 octop 原生 session - X-Octop-User-Id header / user 参数,全部工具支持调用者追溯 - 工具描述区分日常(recall/capture)与显式(save/update) - stateless HTTP + mypy strict 修复
参考 DSH 记忆工具设计,把记忆生产流水线每环节暴露为 MCP 工具: - memory_raws: L0 原始事件结构化列表(session/host/user 过滤) - memory_candidates: L1 候选队列查询(pending/promoted/rejected) - memory_extract: 手动调度提取(L0→L1,promote 可直达 L2) - memory_promote: 审核晋升候选(L1→L2) - memory_reject: 拒绝候选(标记+审计) - memory_atoms: L2 原子记忆结构化查询 全部工具经 MemoryService / Memory 分层 API 实现,与 capture 自动流水线互补, 支持外部按需调度记忆生产。
- 删除 memory_atoms(memory_recall 已覆盖 L2 查询) - 删除 memory_search_raw(memory_raws 增加 query 参数走 FTS,覆盖 L0 搜索) - 保留 9 个:recall/save/capture/update(读写)+ raws/candidates/extract/promote/reject(分层查询+流水线调度)
覆盖 9 工具面(recall/save/capture/update/raws/candidates/extract/promote/reject): - 连接信息:/mcp/memory endpoint、OCTOP_MEMORY_MCP_TOKEN 鉴权(fail-closed)、X-Octop-Agent-Id 专家绑定、调用者追溯 - 工具参数表 + 与 MemoryService 的层级映射(L0/L1/L2) - 使用示例:curl 握手、capture/save/recall、MCP SDK 客户端 - 部署配置与开启步骤
|
这里有没有其他目标agent接入使用手册呢?想接入试试外部效果评审一下 |
ce72fba to
dc3f641
Compare
CandidateStatus 是 typing.Literal 而非 Enum: - memory_candidates 用 CandidateStatus(status) 会抛 TypeError: Cannot instantiate typing.Literal(仅 except ValueError,未捕获) - memory_reject 用 CandidateStatus.REJECTED 会抛 AttributeError,工具 100% 失败 改为校验字面量取值后直接把字符串透传给 list_candidates; reject 直接传字面量 "rejected"。 新增 3 个回归测试:status 传参透传、非法 status 抛 ValueError、reject 字面量。
- 专家绑定改为按请求校验(X-Octop-Agent-Id → agent_repo,须存在且 enabled), 改用单一共享 FastMCP app;新建/停用专家即时生效,无需重启进程 - 读面补齐 memory_search(corpus=all|memory|atom|raw,返回带虚拟 path 的命中) 与 memory_get(path → markdown,支持 start/lines 分页,坏路径返回 error+hint) - 写入按 X-Octop-User-Id 在正文加 `<user>说:` 前缀(随原子落库、可被 FTS 命中、 召回时可见);memory_capture 保留同 session+content 幂等 - 工具描述统一中文,明确 recall / search / get / raws 的分工与覆盖边界 - docs/memory-mcp.md:11 工具面、记忆分层与选工具、Kiro / Claude Code / DSH 接入 (含 hooks 与「共享记忆」预设的自动化配置) - tests/unit/agents/test_memory_mcp.py:31 项(新增 search/get/前缀/幂等用例)
`_memory_mcp_lifespan`(asynccontextmanager 包装)缺返回类型,让 `make typecheck` 与 CI 在 api/app.py:284 报 no-untyped-def。补 `-> AsyncIterator[None]` 并加对应局部 import。纯类型标注,行为不变。 验证:mypy src/octop(474 文件)0 error; pytest -n auto -m "not live" → 2978 passed / 15 skipped。
dc3f641 to
f7ca99a
Compare
hook 场景暴露的两个缺口: 1. memory_recall 接受 session_id / thread_id 并透传给 recall_for_prompt: - session_id 把本会话的 raw 从召回里排除,避免"注入 → 被记录 → 下轮再召回"的回声 - thread_id 启用共指消解(active-entity stack)并把命中写回实体栈 2. memory_capture 增加回声保护:内容含召回注入标记("[memory] Earlier in this workspace" / "## Memory Recall")时直接丢弃,返回 skipped=recall_echo。 MCP 写路径走 Memory.add_raw,绕过了 MemoryRuntime.capture 的 skip_memory_echo, 不补这层就会把 hook 注入的召回块当新事件采集,形成放大环。 docs/memory-mcp.md:同步 memory_recall 的参数与 capture 回声保护契约。 验证:pytest -n auto -m "not live" → 2983 passed / 15 skipped; tests/unit/agents/test_memory_mcp.py → 36 passed;mypy / ruff 干净。
`memory_recall_patch.py` 随本 PR 一起进仓库,但一直没有单测(内网分支同样没有)。 补两个 wiring 断言: - `apply_memory_recall_patch()` 后 `router.route` 与 `multi_source` 的两个 私有函数确实被替换,且上游原始实现保留为兜底路径; - 重复调用幂等:不会把补丁套两层(`_orig_*` 不被覆盖)。 补丁的实际召回效果由 memory eval 语料覆盖,不在单测里重复。 验证:`pytest tests/unit/agents/test_memory_recall_patch.py` → 2 passed。
@Bosheng0422 已经更新到了 docs/memory-mcp.md |
…nto feat/memory-mcp-extension-b7
…op_* packages - merge tag v1.0.2b7 (664c1fb): 1635 files, +146486/-15034, 0 conflicts - fix import after upstream reorg: agents.memory_backend -> agents.memory.backend - rename harness_memory/harness_agent -> octop_memory/octop_harness (pyproject now requires octop-memory>=1.0.1) - verified: pytest tests/unit/agents/test_memory_{mcp,recall_patch,backend}.py -> 50 passed
…session archive)
- /mcp/memory/setup + /uninstall installer for Claude Code / Kiro / DSH / Codex / Cursor
(client_setup.py), per-host hook scripts, and the DSH shared-memory preset
- transcript import and server-side session archive (claude_transcript.py,
memory_session_archive.py)
- auth stays a single configurable header: OCTOP_TOKEN_HEADER / OCTOP_ENTRY_TOKEN_HEADER
(default Authorization -> Bearer <token>); the deployment-specific token dialect is gone
- verified: pytest tests/unit/agents/test_{client_setup,hook_scripts,memory_session_archive,claude_transcript}.py -> 97 passed
- memory_mcp.py: 10 tools; memory_update is gone (memory_save(supersedes=...) covers
corrections), caller-preference ordering on recall/search, transcript-shaped
memory_capture with session archive, per-request expert binding
- docs/memory-mcp.md rewritten to the shipped surface (tool list, installer, env vars)
- no tracing backend and no hard-coded gateway endpoints: the client entry defaults to
this instance's mount point, overridable via OCTOP_MEMORY_PUBLIC_MCP_URL
- verified: pytest tests/unit/agents/test_memory_{mcp,recall_patch,backend}.py -> 111 passed
memory_search already covers the read path (the shipped hooks inject its output), so
the rendered-markdown recall tool was redundant surface.
- read surface is now search / get / raws: 9 tools total
- removed the tool, its docs and its tests, plus the read-side session/thread scoping
it was the only carrier of
- kept the write-side echo guard and the legacy transcript filter: both still match the
[octop memory ...] / upstream recall markers recorded by earlier installs
verified:
- pytest tests/unit/agents/test_memory_{mcp,recall_patch,backend}.py + terminal
integration tests -> 205 passed
- pytest tests/unit/agents tests/unit/gateway/test_slash_memory.py
tests/integration/test_memory_api.py -> 962 passed, 2 skipped
- ruff check / ruff format --check / mypy -> clean
- new 快速上手 section before §1: interactive one-liner, zero-interaction env form, post-install self-check and uninstall - §4.7 now points at it and keeps the implementation detail (?agent/?terminals/?url/?hook_url) - every env var in the table was checked against the generated installer script, and the documented zero-interaction line was executed end to end (Claude Code + Kiro, exit 0, entries carry X-Octop-Agent-Id / Authorization / X-Octop-User-Id)
…token Answers the recurring question: direct mode needs just OCTOP_MEMORY_MCP_TOKEN (baked in as DEFAULT_TOKEN, overridable via OCTOP_MEMORY_TOKEN). A personal token (OCTOP_ENTRY_TOKEN) is only required when the client entry is pointed at a self-hosted OAuth front, where the proxy identifies the caller from the token.
…tial
The fork is a plain self-hosted instance: one address, one credential.
- removed OCTOP_DISABLE_OAUTH / the OAuth-front branch and the personal-token path
(OCTOP_ENTRY_TOKEN / OCTOP_ENTRY_TOKEN_HEADER); hooks and the client MCP entry now
always share the instance token (OCTOP_MEMORY_MCP_TOKEN, overridable via
OCTOP_MEMORY_TOKEN, header name via OCTOP_TOKEN_HEADER)
- ?url= / OCTOP_MEMORY_PUBLIC_MCP_URL survive as a plain entry-address override
- no second server-name env: the MCP entry is always named from the entry address
- also dropped the last internal artefacts found while editing: an employee's Chinese
name used as a non-ASCII test fixture, and 'oauth' in the server-name example list
- docs: quick start without the removed variables, single-credential note, §5 table trimmed
verified:
- pytest tests/unit/agents/{test_client_setup,test_memory_mcp,test_memory_recall_patch,
test_memory_backend,test_hook_scripts,test_memory_session_archive,test_claude_transcript}.py
-> 203 passed
- pytest tests/unit/agents tests/unit/gateway/test_slash_memory.py
tests/integration/test_memory_api.py -> 960 passed, 2 skipped
- ruff check / ruff format --check / mypy -> clean
- live server: generated installer has 0 OAuth/ada/internal markers; the documented
zero-interaction line installs Claude Code + Kiro with the shared token (exit 0);
the installed UserPromptSubmit hook still injects memory_search hits
|
请把这个变更重新提交到 Please open this change against |
|
已按 review 要求把同一份变更重新提交到 |
|
@jubaoliang 重提的 #2887(base=develop)刚被仓库的临时作者限制机器人自动关闭:
@ArvinQi 目前没有已合并的 PR,也不在 allowlist 里。麻烦把 |
Summary(摘要)
本次改动基于
feat/memory-mcp-server分支(基线fea4c59),对专家记忆 MCP 服务(/mcp/memory,Streamable HTTP)做三方面增强,全部改动集中在src/octop/infra/agents/memory_mcp.py一个文件:memory_capture写入 L0 原始事件后,自动触发专家记忆提取流水线(extract → candidate → promote → atom),外部 Agent / Bot 无需 octop 原生 session 也能通过 MCP 沉淀长期记忆。X-Octop-User-IdHTTP Header(或工具user参数),所有记忆工具(recall / search_raw / capture / save / update)都能感知"是谁在调用",每次写入落库可追溯,按人区分使用偏好。FastMCP启用 stateless HTTP,避免服务重启后Mcp-Session-Id失效导致的-32600 Session not found。Why(背景与动机)
Octop 专家的记忆(事实、对话、决策)目前只有 Octop 控制台 / 进程内 Agent 能访问。社区
feat/memory-mcp-server已把记忆通过 MCP 暴露给外部 Agent,但存在两个缺口:session_id分组,而外部接入方(coding agent、bot)没有 octop 原生 session,memory_capture记录后永远停留在 L0 原始层,无法蒸馏成可召回的 atom。本 PR 解决这两个问题,同时顺带提升 MCP 服务的稳定性(stateless HTTP)。
What(改动明细)
memory_capture写 L0 后调用_trigger_extract,复用专家进程内MemoryService异步执行extract(incremental=True, promote=True),raw 事件自动蒸馏 → 候选 → 晋升为 atom(best-effort,依赖运行时的 MemoryService,缺失时静默跳过不报错)session_id可选;缺省时派生为ext:{source}:{user}——同来源同用户的多次 capture 落入同一分组,提取器可聚合上下文蒸馏;不同用户自动隔离X-Octop-User-IdHeader 经 ASGI 中间件 → contextvar → 工具读取(或显式user参数覆盖);add_raw/store落库 user 字段;recall/search_raw返回caller字段memory_recall/memory_capture标记为日常使用(自动提取);memory_save/memory_update标记为显式记忆 / 更新(明确知道事实时才用)FastMCP(stateless_http=True):每次请求独立会话,杜绝服务重启后Mcp-Session-Id失效(代价是每次调用多一次 initialize)server.services/app_runtime非空断言、_memory() -> Any类型、ctx: Context泛型兼容(保持 mcp 工具注入可用)How it works(工作机制)
外部调用 → 原子记忆的完整链路:
调用者隔离:
Verification(验证结果)
memory_capture不带 session_id → 自动派生ext:{source}:{user},extract_scheduled: trueX-Octop-User-IdHeader 生效:L0 落库user字段(user-alice / user-bob 分属不同 session 组)memory_recall返回caller字段(识别调用者)Compatibility(兼容性说明)
stateless_http:依赖mcp>=1.28(上游 uv.lock 已锁 1.28.1)✅_trigger_extract:依赖 harness 运行时注入的agent._memory_runtime.service(octop 仓库内无此符号)。代码用getattr防御性访问——上游环境若无该属性会静默跳过提取(extract_scheduled: false),不会报错;提取触发为 best-effort_AgentRouter中间件写入 contextvar → 工具读取,不依赖 mcp SDK 内部request_context(stateless 模式下 SDK 不提供),跨 ASGI→工具稳定传递memory_mcp.py,不新增挂载、不改其他文件Target branch
develop(feature / fix — default)main(release/*orhotfix/*only)Type of change
Test plan
make allpasses locallyChecklist
CHANGELOG.md(if user-facing)