Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"name": "engraphis-memory",
"source": "./",
"description": "Discipline for giving agents durable, scoped, explainable memory across sessions and repos with the Engraphis MCP tools.",
"version": "1.2.0"
"version": "1.2.1"
}
]
}
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "engraphis-memory",
"version": "1.2.0",
"version": "1.2.1",
"description": "Give agents durable, scoped, explainable memory across sessions and repos via the Engraphis MCP tools. Use when you learn something worth keeping, need prior context before acting, or ask why/how a fact changed. Covers remember/recall, why/timeline, forget/pin/correct, sessions, and code search.",
"author": {
"name": "The Engraphis Authors",
Expand Down
12 changes: 6 additions & 6 deletions .claude-plugin/skill-assets.sha256
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
81840a090ae1b8b14fff4eac3bf7ff1832760f4806c510070b78687cb15a0f99 .claude-plugin/marketplace.json
db1b72e67e25bc29e3220d75ff5ab03a0ab954ebdbb93dd74d715ca509317a0c .claude-plugin/plugin.json
656caf07c9064b219eb974e018180e6a7a88f2c058fb1b1c6a8a36074d67e9cc skills/engraphis-memory/SKILL.md
9751b6e7310151c14e6bb7f5d683943a53a95951ff207876053d082e65eecfd5 skills/engraphis-memory/references/CONVENTIONS.md
45f4b4ad9dbfd39f2b377083d9b3eec5eed7cba7cb8fa139e3f420bdd6105343 skills/engraphis-memory/references/SCOPING.md
dc83c48d1a57122e7b58da29d32ae3b8ebd4ffff0b6d0570a8833e920e365109 skills/engraphis-memory/references/TOOLS.md
3fb76735c2b1d2fa0b70ba9554d6c76a1bfae0edef8a5632be18af8fb15447e1 .claude-plugin/marketplace.json
a837836347bdb17330292c071dc2908527435e9f7defa21d6ab4f4f91acaffde .claude-plugin/plugin.json
696fe737e83a8d073c8dac77704ada7261332ede2527b4c4d426b1e657e034da skills/engraphis-memory/SKILL.md
7ee71fb5ff9bd2b02f50b3ee8dc62f390a0e1bcd849a55739c4a376ac03d9784 skills/engraphis-memory/references/CONVENTIONS.md
8aafd2daba872be38ec8d42377e886d795d8941bf7c6a39795937ffc1d1f0d88 skills/engraphis-memory/references/SCOPING.md
4c1478453237643e7b4ee2ab4484b9fea8fd759f19f9a5fbf9eeda216d1d6f1a skills/engraphis-memory/references/TOOLS.md
18 changes: 9 additions & 9 deletions BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,30 +9,30 @@ been written; when this and the code disagree, the code wins (CLAUDE.md).
Engraphis's eval harness scores **retrieval**, not end-to-end QA. That distinction is deliberate
and stated everywhere the numbers appear (`eval/external.py`).

- **Correctness gate** `eval/harness.py` over `eval/datasets/sample.jsonl` and
- **Correctness gate**: `eval/harness.py` over `eval/datasets/sample.jsonl` and
`codemem.jsonl` (conflict resolution) and `graph_multihop.jsonl` (multi-hop graph recall).
Runs on the deterministic embedder, so it is a plumbing/regression floor, not a public
performance claim. This is the gate CI enforces.
- **Ablation** `eval/ablation.py`: vector-only vs. 1-hop graph vs. Personalized-PageRank arm,
- **Ablation**: `eval/ablation.py`: vector-only vs. 1-hop graph vs. Personalized-PageRank arm,
to show the graph arm actually earns its place.
- **External benchmarks** `eval/external.py` loads **LoCoMo** and **LongMemEval** and pushes
- **External benchmarks**: `eval/external.py` loads **LoCoMo** and **LongMemEval** and pushes
them through the *real* `MemoryEngine` write path (conflict resolution + evolution) and hybrid
recall with a real sentence-transformers embedder. It reports `recall_at_k` / `hit_at_k` /
`answer_token_recall` i.e. *did the evidence come back*, not *did an LLM answer correctly*.
`answer_token_recall`: i.e. *did the evidence come back*, not *did an LLM answer correctly*.
It retains source categories and abstention/no-evidence questions as explicit exclusions from
retrieval-only aggregates rather than silently dropping them. `eval.longmemeval_v2` is a local,
text-only adapter for the official LongMemEval-V2 `insert(trajectory)` / `query(query,
query_image=None)` memory interface; it does not download data or call a model.
- **Grounded** `eval/grounded.py`: answerable → cite, off-topic → abstain.
- **Chunking (quality per token)** `eval/chunking_eval.py` over `eval/datasets/longdoc.jsonl`
ingests a multi-topic corpus twiceone memory per document (`whole`) vs. sub-file
`ChunkingExtractor` (`chunked`) — and queries both through the real recall pipeline. This is
- **Grounded**: `eval/grounded.py`: answerable → cite, off-topic → abstain.
- **Chunking (quality per token)**: `eval/chunking_eval.py` over `eval/datasets/longdoc.jsonl`
ingests a multi-topic corpus twice: once as one memory per document (`whole`) and once with
sub-file `ChunkingExtractor` (`chunked`), then queries both through the real recall pipeline. This is
the first cut of the context-reduction metric (item 3 below). On the deterministic embedder:
**recall@5 1.000 for both, at ~73% fewer context tokens (809 → 219) and ~4× smaller
tokens-to-evidence (162 → 42).** Pass `--embed-model sentence-transformers/all-MiniLM-L6-v2`
for a real retrieval number (recall should then favour chunked on larger corpora, not just
tie).
- **Full-pipeline latency + quality** `eval/performance.py` times the shipped semantic +
- **Full-pipeline latency + quality**: `eval/performance.py` times the shipped semantic +
lexical + graph + fusion + scoring + rerank + packing path after warmup, with reinforcement
disabled so repeated measurements do not mutate their corpus. It reports p50/p95/p99 latency,
retrieval quality, and packed context tokens in one JSON-safe schema. `--filler-memories`
Expand Down
Loading
Loading