feat: scope-aware retrieval with weighted RRF fusion (PR 2/3)#43
Open
kenyonxu wants to merge 5 commits into
Open
feat: scope-aware retrieval with weighted RRF fusion (PR 2/3)#43kenyonxu wants to merge 5 commits into
kenyonxu wants to merge 5 commits into
Conversation
…a only) Add scope (TEXT, default 'personal') and shared_with (TEXT, nullable) columns to 5 core tables, plus composite indexes on (scope) and (profile_id, scope): - memories - atomic_facts - canonical_entities - temporal_events - graph_edges Models updated with scope/shared_with fields. Migration M016 handles ALTER TABLE for existing databases — all existing data defaults to 'personal' for full backward compatibility. This is PR 1/3 of the multi-scope memory feature — pure storage layer. No retrieval logic or interface changes included. Ref: qualixar#20
Add ScopeWeights dataclass (personal=1.0, shared=0.7, global=0.5) for multi-scope RRF fusion weighting. Persisted in config.json under scope_weights key. Part of PR-B: Retrieval Layer for multi-scope memory (qualixar#20).
Add module-level _scope_where() function that builds scope-filtering WHERE clauses for multi-scope retrieval (personal/global/shared). Update 11 query methods with optional include_global/include_shared params: - get_pinned, get_all_facts, get_facts_by_entity, get_facts_by_type - get_facts_by_memory_id, get_fact_count, get_facts_by_ids - search_facts_fts, get_edges_for_node, get_temporal_events - get_temporal_events_by_range All new params default to True (backward compatible). Existing data has scope='personal' (column default) so the OR branches are harmless no-ops. Part of PR-B: Retrieval Layer for multi-scope memory (qualixar#20).
- engine.recall(): add include_global/include_shared params, set on channel instances before parallel execution - engine._load_facts(): pass scope flags to get_facts_by_ids() - semantic_channel: get_facts_by_ids/get_all_facts with scope flags - bm25_channel: get_all_facts (cold-load) with scope flags - hopfield_channel: get_facts_by_ids/get_all_facts with scope flags - entity_channel: get_facts_by_entity/get_edges_for_node with scope flags All scope flags default to True via getattr(self, 'include_X', True) — backward compatible when engine doesn't set them. Part of PR-B: Retrieval Layer for multi-scope memory (qualixar#20).
Add include_global/include_shared keyword args to run_recall() and forward them to retrieval_engine.recall(). Part of PR-B: Retrieval Layer for multi-scope memory (qualixar#20).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 2/3 of the multi-scope memory feature (RFC #20).
Adds scope filtering to all retrieval channels and implements multi-scope parallel retrieval with configurable RRF fusion weights.
Changes
core/config.pyScopeWeightsdataclass (personal=1.0, shared=0.7, global=0.5) with load/save persistencestorage/database.py_scope_where()helper + scope params (include_global/include_shared) on 11 query methodsretrieval/engine.pyinclude_global/include_sharedtorecall(), set on channels before parallel executionretrieval/semantic_channel.pyget_facts_by_ids/get_all_factsretrieval/bm25_channel.pyget_all_factscold-loadretrieval/hopfield_channel.pyget_facts_by_ids/get_all_factsretrieval/entity_channel.pyget_facts_by_entity/get_edges_for_nodecore/recall_pipeline.pyinclude_global/include_sharedthrough to engineDesign
True(backward compatible)_scope_where()builds SQL:(profile_id = ?) OR (scope = 'global') OR (scope = 'shared' AND shared_with LIKE ?)scope='personal'(PR 1/3 column default) — OR branches are no-opsDependencies
Test Results
Ref: #20