feat: 记忆召回反馈/加权/RRF融合/TTL过期/巩固 - #4
Merged
Merged
Conversation
P0 召回反馈(recall_count 递增)+ 信号加权(importance/频次/时效)+ 稠密稀疏 RRF 融合;P1 TTL 过期标记 + 低频记忆 LLM 巩固为摘要;新增 9 项配置与中英 i18n
Reviewer's Guide实现带有每条记忆统计的召回反馈、按信号加权的稠密/稀疏 RRF 融合召回排序，以及后台 TTL 过期和通过 LLM 总结进行的低频记忆合并，并通过新的配置和 i18n 条目接入记忆注入生命周期。 带反馈的信号加权稠密/稀疏 RRF 召回的时序图sequenceDiagram
actor User
participant MemoryPlugin
participant MemoryManager
participant VecDB
participant DocumentStorage
participant RerankProvider
User->>MemoryPlugin: inject_memories(event, request)
MemoryPlugin->>MemoryManager: recall_memories(event, query, top_k, bump=True)
alt personal scope
MemoryManager->>MemoryManager: _retrieve_with_filter(query, fetch_k, filters)
else multi-scope
MemoryManager->>MemoryManager: _build_recall_filters(...)
MemoryManager->>MemoryManager: _retrieve_with_filter(query, fetch_k, filters_list[i]) *
end
activate MemoryManager
MemoryManager->>VecDB: retrieve(query, k=top_k, rerank=False, metadata_filters)
VecDB-->>MemoryManager: dense_results
MemoryManager->>MemoryManager: _parse dense_results to dense_memories
alt recall_sparse_fusion enabled
MemoryManager->>MemoryManager: _sparse_retrieve(query, top_k, filters)
activate MemoryManager
MemoryManager->>DocumentStorage: search_sparse(query_tokens, limit)
DocumentStorage-->>MemoryManager: sparse docs
MemoryManager->>MemoryManager: _matches_filters(metadata, filters)
MemoryManager->>MemoryManager: _rrf_fuse(dense_memories, sparse_memories, limit=top_k)
deactivate MemoryManager
else fallback
MemoryManager->>MemoryManager: use dense_memories only
end
opt use_reranker
MemoryManager->>RerankProvider: rerank(query, docs)
RerankProvider-->>MemoryManager: reranked results
MemoryManager->>MemoryManager: reorder memories by relevance_score
end
MemoryManager->>MemoryManager: _rerank_by_signal(memories)
opt bump
MemoryManager->>MemoryManager: _bump_recall_stats(recalled_uris)
MemoryManager->>MemoryManager: _exec_metadata_update(set_clause, where_clause, params)
end
MemoryManager-->>MemoryPlugin: memories
MemoryPlugin-->>User: inject recalled memories into request context
TTL 过期与基于 LLM 的记忆合并时序图sequenceDiagram
actor User
participant MemoryPlugin
participant MemoryManager
participant VecDB
participant DocumentStorage
participant LLM
User->>MemoryPlugin: inject_memories(event, request)
MemoryPlugin->>MemoryPlugin: _maybe_expire_stale_memories()
MemoryPlugin->>MemoryManager: _expire_stale_memories(ttl_days)
MemoryManager->>MemoryManager: _exec_metadata_update(set_clause, where_clause, params)
MemoryManager-->>MemoryPlugin: expired count
MemoryPlugin->>MemoryPlugin: asyncio.create_task(_maybe_consolidate_memories(event))
activate MemoryPlugin
MemoryPlugin->>MemoryManager: _fetch_consolidation_candidates(min_age_days, max_recall, limit, owner_user_id, memory_scope=personal)
MemoryManager->>VecDB: vec_db.document_storage
MemoryManager->>DocumentStorage: get_session() / SELECT text, metadata FROM documents ...
DocumentStorage-->>MemoryManager: candidate rows
MemoryManager-->>MemoryPlugin: candidates
MemoryPlugin->>MemoryPlugin: _get_llm_provider_id(event, "summarization" / "extraction")
MemoryPlugin->>LLM: context.llm_generate(chat_provider_id, MEMORY_CONSOLIDATION_PROMPT)
LLM-->>MemoryPlugin: summary text
MemoryPlugin->>MemoryManager: _mark_consolidated(source_uris)
MemoryManager->>MemoryManager: _exec_metadata_update(set_clause, where_clause, params)
MemoryManager-->>MemoryPlugin: marked count
MemoryPlugin->>MemoryManager: store_memory(event, summary, domain="consolidated", memory_type=CONTEXT, disclosure, importance=4, memory_scope=personal)
MemoryManager-->>MemoryPlugin: stored consolidated memory
deactivate MemoryPlugin
MemoryPlugin-->>User: consolidation runs in background, user flow unaffected
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your Experience进入你的 dashboard 以：
Getting HelpOriginal review guide in EnglishReviewer's GuideImplements recall feedback with per-memory statistics, signal-weighted and dense/sparse RRF-fused recall ranking, plus background TTL expiration and low-frequency memory consolidation via LLM summaries, wired into the memory injection lifecycle with new configuration and i18n entries. Sequence diagram for signal-weighted dense/sparse RRF recall with feedbacksequenceDiagram
actor User
participant MemoryPlugin
participant MemoryManager
participant VecDB
participant DocumentStorage
participant RerankProvider
User->>MemoryPlugin: inject_memories(event, request)
MemoryPlugin->>MemoryManager: recall_memories(event, query, top_k, bump=True)
alt personal scope
MemoryManager->>MemoryManager: _retrieve_with_filter(query, fetch_k, filters)
else multi-scope
MemoryManager->>MemoryManager: _build_recall_filters(...)
MemoryManager->>MemoryManager: _retrieve_with_filter(query, fetch_k, filters_list[i]) *
end
activate MemoryManager
MemoryManager->>VecDB: retrieve(query, k=top_k, rerank=False, metadata_filters)
VecDB-->>MemoryManager: dense_results
MemoryManager->>MemoryManager: _parse dense_results to dense_memories
alt recall_sparse_fusion enabled
MemoryManager->>MemoryManager: _sparse_retrieve(query, top_k, filters)
activate MemoryManager
MemoryManager->>DocumentStorage: search_sparse(query_tokens, limit)
DocumentStorage-->>MemoryManager: sparse docs
MemoryManager->>MemoryManager: _matches_filters(metadata, filters)
MemoryManager->>MemoryManager: _rrf_fuse(dense_memories, sparse_memories, limit=top_k)
deactivate MemoryManager
else fallback
MemoryManager->>MemoryManager: use dense_memories only
end
opt use_reranker
MemoryManager->>RerankProvider: rerank(query, docs)
RerankProvider-->>MemoryManager: reranked results
MemoryManager->>MemoryManager: reorder memories by relevance_score
end
MemoryManager->>MemoryManager: _rerank_by_signal(memories)
opt bump
MemoryManager->>MemoryManager: _bump_recall_stats(recalled_uris)
MemoryManager->>MemoryManager: _exec_metadata_update(set_clause, where_clause, params)
end
MemoryManager-->>MemoryPlugin: memories
MemoryPlugin-->>User: inject recalled memories into request context
Sequence diagram for TTL expiration and LLM-based memory consolidationsequenceDiagram
actor User
participant MemoryPlugin
participant MemoryManager
participant VecDB
participant DocumentStorage
participant LLM
User->>MemoryPlugin: inject_memories(event, request)
MemoryPlugin->>MemoryPlugin: _maybe_expire_stale_memories()
MemoryPlugin->>MemoryManager: _expire_stale_memories(ttl_days)
MemoryManager->>MemoryManager: _exec_metadata_update(set_clause, where_clause, params)
MemoryManager-->>MemoryPlugin: expired count
MemoryPlugin->>MemoryPlugin: asyncio.create_task(_maybe_consolidate_memories(event))
activate MemoryPlugin
MemoryPlugin->>MemoryManager: _fetch_consolidation_candidates(min_age_days, max_recall, limit, owner_user_id, memory_scope=personal)
MemoryManager->>VecDB: vec_db.document_storage
MemoryManager->>DocumentStorage: get_session() / SELECT text, metadata FROM documents ...
DocumentStorage-->>MemoryManager: candidate rows
MemoryManager-->>MemoryPlugin: candidates
MemoryPlugin->>MemoryPlugin: _get_llm_provider_id(event, "summarization" / "extraction")
MemoryPlugin->>LLM: context.llm_generate(chat_provider_id, MEMORY_CONSOLIDATION_PROMPT)
LLM-->>MemoryPlugin: summary text
MemoryPlugin->>MemoryManager: _mark_consolidated(source_uris)
MemoryManager->>MemoryManager: _exec_metadata_update(set_clause, where_clause, params)
MemoryManager-->>MemoryPlugin: marked count
MemoryPlugin->>MemoryManager: store_memory(event, summary, domain="consolidated", memory_type=CONTEXT, disclosure, importance=4, memory_scope=personal)
MemoryManager-->>MemoryPlugin: stored consolidated memory
deactivate MemoryPlugin
MemoryPlugin-->>User: consolidation runs in background, user flow unaffected
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - 我发现了 1 个问题，并给出了一些整体反馈：
- 这个插件与
MemoryManager的内部实现紧密耦合（例如调用_expire_stale_memories、_fetch_consolidation_candidates、_mark_consolidated、_current_owner_user_id），这会增加未来重构的难度；建议将这些提升为显式的公共方法，或者引入一个小的生命周期接口来替代对受保护成员的依赖。 - 在
_tokenize_query中，一旦加载分词器失败，你会把结果缓存为False并且永远不再重试，即使发生了热重载或环境变化；如果这个对象在进程内是长生命周期的，建议增加基于时间的重试机制，或者提供一种重置缓存的方式，让稀疏检索能够从临时的初始化错误中恢复。
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The plugin is tightly coupled to `MemoryManager` internals (e.g., calling `_expire_stale_memories`, `_fetch_consolidation_candidates`, `_mark_consolidated`, `_current_owner_user_id`), which makes future refactors harder; consider promoting these to explicit public methods or introducing a small lifecycle interface instead of relying on protected members.
- In `_tokenize_query`, once tokenizer loading fails you cache `False` and never retry, even after a hot reload or environment change; if this is meant to be long-lived within a process, consider adding a time-based retry or a way to reset the cache so sparse retrieval can recover from transient initialization errors.
## Individual Comments
### Comment 1
<location path="memory_manager.py" line_range="382-386" />
<code_context>
+ except Exception as e:
+ logger.warning(f"[简单长期记忆] 读取巩固候选失败: {e}")
+ return []
+ candidates: list[dict[str, Any]] = []
+ for row in rows:
+ text_val = row[0] if row else ""
+ meta = _safe_parse_metadata(row[1] if len(row) > 1 else {})
+ candidates.append({"text": text_val, "metadata": meta})
+ return candidates
+
</code_context>
<issue_to_address>
**suggestion:** 优先通过字段名而不是位置索引访问 SQLAlchemy 的行字段，以便在模式或查询发生变化时让代码更加健壮。
在 `_fetch_consolidation_candidates` 中，`row[0]` 和 `row[1]` 被用于 `text` 和 `metadata`，这会让行为与列顺序紧密绑定。请改用 `row.text` / `row.metadata`（或 `row._mapping['text']`），这样代码就能与 SELECT 子句保持一致，避免在查询变更时出现静默的字段错位。
建议实现：
```python
except Exception as e:
logger.warning(f"[简单长期记忆] 读取巩固候选失败: {e}")
return []
candidates: list[dict[str, Any]] = []
for row in rows:
# 使用字段名而不是位置索引，使代码在查询/列顺序变更时更健壮
text_val = getattr(row, "text", "") or ""
raw_metadata = getattr(row, "metadata", {}) or {}
meta = _safe_parse_metadata(raw_metadata)
candidates.append({"text": text_val, "metadata": meta})
return candidates
```
1. 确认生成 `rows` 的查询 `SELECT` 子句中包含 `text` 和 `metadata` 字段名，与这里的属性访问保持一致。
2. 如果查询中使用了别名（例如 `SELECT text AS memory_text`），需要将这里的属性名改为对应的别名（如 `row.memory_text`）。
3. 在使用 SQLAlchemy 1.4+/2.0 的情况且 `row` 是 `Row` 对象时，上述代码依赖其支持属性访问；如果项目统一约定使用 `row._mapping["text"]` 访问字段，可将两行改为：
- `text_val = row._mapping.get("text", "") or ""`
- `raw_metadata = row._mapping.get("metadata", {}) or {}`
</issue_to_address>Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Original comment in English
Hey - I've found 1 issue, and left some high level feedback:
- The plugin is tightly coupled to
MemoryManagerinternals (e.g., calling_expire_stale_memories,_fetch_consolidation_candidates,_mark_consolidated,_current_owner_user_id), which makes future refactors harder; consider promoting these to explicit public methods or introducing a small lifecycle interface instead of relying on protected members. - In
_tokenize_query, once tokenizer loading fails you cacheFalseand never retry, even after a hot reload or environment change; if this is meant to be long-lived within a process, consider adding a time-based retry or a way to reset the cache so sparse retrieval can recover from transient initialization errors.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The plugin is tightly coupled to `MemoryManager` internals (e.g., calling `_expire_stale_memories`, `_fetch_consolidation_candidates`, `_mark_consolidated`, `_current_owner_user_id`), which makes future refactors harder; consider promoting these to explicit public methods or introducing a small lifecycle interface instead of relying on protected members.
- In `_tokenize_query`, once tokenizer loading fails you cache `False` and never retry, even after a hot reload or environment change; if this is meant to be long-lived within a process, consider adding a time-based retry or a way to reset the cache so sparse retrieval can recover from transient initialization errors.
## Individual Comments
### Comment 1
<location path="memory_manager.py" line_range="382-386" />
<code_context>
+ except Exception as e:
+ logger.warning(f"[简单长期记忆] 读取巩固候选失败: {e}")
+ return []
+ candidates: list[dict[str, Any]] = []
+ for row in rows:
+ text_val = row[0] if row else ""
+ meta = _safe_parse_metadata(row[1] if len(row) > 1 else {})
+ candidates.append({"text": text_val, "metadata": meta})
+ return candidates
+
</code_context>
<issue_to_address>
**suggestion:** Prefer accessing SQLAlchemy row fields by name instead of positional indices to make the code more robust to schema/query changes.
In `_fetch_consolidation_candidates`, `row[0]` and `row[1]` are used for `text` and `metadata`, which couples behavior to column order. Please switch to `row.text` / `row.metadata` (or `row._mapping['text']`) so the code tracks the SELECT clause and avoids silent misalignment if the query changes.
Suggested implementation:
```python
except Exception as e:
logger.warning(f"[简单长期记忆] 读取巩固候选失败: {e}")
return []
candidates: list[dict[str, Any]] = []
for row in rows:
# 使用字段名而不是位置索引，使代码在查询/列顺序变更时更健壮
text_val = getattr(row, "text", "") or ""
raw_metadata = getattr(row, "metadata", {}) or {}
meta = _safe_parse_metadata(raw_metadata)
candidates.append({"text": text_val, "metadata": meta})
return candidates
```
1. 确认生成 `rows` 的查询 `SELECT` 子句中包含 `text` 和 `metadata` 字段名，与这里的属性访问保持一致。
2. 如果查询中使用了别名（例如 `SELECT text AS memory_text`），需要将这里的属性名改为对应的别名（如 `row.memory_text`）。
3. 在使用 SQLAlchemy 1.4+/2.0 的情况且 `row` 是 `Row` 对象时，上述代码依赖其支持属性访问；如果项目统一约定使用 `row._mapping["text"]` 访问字段，可将两行改为：
- `text_val = row._mapping.get("text", "") or ""`
- `raw_metadata = row._mapping.get("metadata", {}) or {}`
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a memory lifecycle management and consolidation system, featuring TTL-based memory expiration, LLM-driven memory consolidation, and hybrid search fusing dense vector search with FTS5 sparse retrieval via Reciprocal Rank Fusion (RRF). The review feedback highlights several critical issues, including an invalid MemoryType.CONTEXT reference that will cause an AttributeError, potential garbage collection of unreferenced background tasks created via asyncio.create_task, and potential runtime crashes (ValueError, TypeError, or OverflowError) during signal-based re-ranking. Additionally, it suggests optimizing SQLite queries by simplifying nested json_set calls.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Comment on lines
+297
to
+301
| set_clause = ( | ||
| "json_set(json_set(metadata, '$.recall_count', " | ||
| "CAST(COALESCE(json_extract(metadata,'$.recall_count'),0) AS INTEGER) + 1), " | ||
| "'$.last_recalled_at', :now)" | ||
| ) |
There was a problem hiding this comment.
SQLite 的 json_set 函数原生支持传入多组 path 和 value 参数。使用嵌套的 json_set(json_set(...)) 会导致 SQLite 内部进行多次 JSON 解析和序列化，影响性能。建议将其简化为单次 json_set 调用，以提高 SQL 执行效率和代码可读性。
Suggested change
| set_clause = ( | |
| "json_set(json_set(metadata, '$.recall_count', " | |
| "CAST(COALESCE(json_extract(metadata,'$.recall_count'),0) AS INTEGER) + 1), " | |
| "'$.last_recalled_at', :now)" | |
| ) | |
| set_clause = ( | |
| "json_set(metadata, " | |
| "'$.recall_count', CAST(COALESCE(json_extract(metadata,'$.recall_count'),0) AS INTEGER) + 1, " | |
| "'$.last_recalled_at', :now)" | |
| ) |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| return 0 | ||
| kb_id = self._kb_helper.kb.kb_id | ||
| cutoff_iso = (datetime.now(timezone.utc) - timedelta(days=ttl_days)).isoformat() | ||
| set_clause = "json_set(metadata, '$.deprecated', 1)" |
There was a problem hiding this comment.
🚩 Deprecated field type inconsistency: boolean False vs integer 1
Memory records are created with deprecated: False (Python bool) in _build_memory_metadata at memory_manager.py:865. But _expire_stale_memories at memory_manager.py:320 and _mark_consolidated at memory_manager.py:395 use json_set(metadata, '$.deprecated', 1) which stores integer 1. In SQLite JSON, false (boolean) and 0 (integer) are distinct types; similarly true and 1 are distinct. The recall filters at memory_manager.py:674 use deprecated: False which is passed to the vec_db's metadata filter. Whether deprecated: False matches records with deprecated: 0 (JSON false) but not deprecated: 1 (JSON integer) depends on the vec_db filter implementation. In practice, most implementations treat these as equivalent via truthiness checks, but this type mismatch is fragile and could cause issues if the filter implementation changes.
Was this helpful? React with 👍 or 👎 to provide feedback.
- MemoryType.CONTEXT→NORMAL(原不存在,巩固必崩+原文已mark无摘要=数据丢失) - asyncio.create_task 加 _background_tasks 强引用防 GC(LLM 长任务) - _rerank 数值安全转换+max(0,Δt) 防 int() 崩溃与 exp 溢出 - TTL 排除 permanent/global 记忆(管理员全局记忆不再被误过期) - domain consolidated→context(进白名单,不再回退 facts) - row 字段名访问(getattr)+ 嵌套 json_set 改单次调用
expire_stale_memories/fetch_consolidation_candidates/mark_consolidated 去掉 _ 前缀升为公共 API;fetch_consolidation_candidates 改接 event,owner 由内部 _current_owner_user_id 推导(main.py 不再跨边界调受保护成员)。回应 PR#4 sourcery 反馈。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P0 召回反馈(recall_count 递增)+ 信号加权(importance/频次/时效)+ 稠密稀疏 RRF 融合;P1 TTL 过期标记 + 低频记忆 LLM 巩固为摘要;新增 9 项配置与中英 i18n
Summary by Sourcery
通过引入召回反馈、多信号重排、稠密/稀疏融合、基于 TTL 的生命周期管理，以及由 LLM 驱动的低频记忆整合，并配套相关配置与国际化更新，增强记忆系统。
New Features:
recall_count并更新last_recalled_at来跟踪记忆召回反馈。Enhancements:
Documentation:
Original summary in English
Summary by Sourcery
Enhance the memory system with recall feedback, multi-signal reranking, dense/sparse fusion, TTL-based lifecycle management, and LLM-driven consolidation of low-frequency memories, along with supporting config and i18n updates.
New Features:
Enhancements:
Documentation: