Skip to content

feat(forgetting): opt-in LLM judge that can veto decay deletions - #33

Closed
Nitjsefnie wants to merge 3 commits into
mienetic:mainfrom
Nitjsefnie-OSC:feat/smart-forgetting
Closed

Nitjsefnie wants to merge 3 commits into
mienetic:mainfrom
Nitjsefnie-OSC:feat/smart-forgetting

Conversation

@Nitjsefnie

Copy link
Copy Markdown
Contributor

Summary

Adds opt-in, LLM-assisted forgetting: when enabled, a configured judge can veto the decay formula's
deletion candidates, so a memory the heuristic would drop can be rescued. Disabled by default β€” with
no configuration the behaviour is byte-for-byte what it is today, and no LLM is ever constructed or
called.

Related issue

Closes #14

Changes

  • New mnema.judge β€” an OpenAI-compatible chat judge over httpx (already a core dependency, so
    this adds nothing to install). Split pure from I/O the way decay.py and summarize.py are: the
    prompt builder and parse_forget_verdict are pure functions, only the client touches the network.
  • make_judge factory β€” the single construction site, gated on smart_forget_enabled. When
    smart forgetting is off, no judge object exists at all.
  • MemoryService.apply_decay wiring β€” one injection point, so the CLI, the dream loop, the API
    and the dashboard all inherit the behaviour with zero route changes.
  • Config β€” MNEMA_SMART_FORGET_ENABLED plus MNEMA_JUDGE_* settings, validated in config.py.
  • Docs β€” README.md section covering the flags and the veto-only semantics.
  • Tests: new tests/test_judge.py, a TestSmartForgetting class in tests/test_service.py, config
    cases, and a FakeJudge in tests/fakes.py.

The judge can only rescue, never delete. Candidates are still selected purely by
score <= threshold; the judge is handed that list and may only remove entries from it. There is no
path by which enabling this deletes something the formula would have kept.

Fail-safe is KEEP. Any error, timeout, non-JSON body, wrong-shaped JSON, empty choices, null
content, HTTP 500, or an unparseable verdict results in keeping the memory. This follows the repo's
existing conservative-deletion posture (dry_run=True by default, the importance floor, the
deliberately conservative Dreamer).

Checklist

  • ruff check . is clean
  • pytest passes (with the relevant [extra] installed if you touched a backend)
  • New public functions / classes have docstrings
  • Docs updated (README.md, SKILL.md, docs/) if user-facing behavior changed
  • If this adds a backend or embedding provider: registered in the factory (__init__.py::make_*),
    added an [extra] in pyproject.toml, and added a test class β€” partially: it is registered
    via a make_judge factory and has its own test class, but it needs no [extra] because it
    introduces no new dependency. See the note below.

Bugs Discovered

Notes for reviewer

Design calls I made from your own precedent β€” overrule any of them and I will change it:

  • Veto-only, not judge-everything. The formula still decides what is a candidate; the LLM can only
    rescue. Judging every memory, or judging above-threshold records, is a wider behaviour change and
    felt like your call, not mine.
  • One concrete provider, OpenAI-compatible over httpx. Modelled on embeddings/ollama.py and
    the Cohere worked example in CONTRIBUTING. It works against a local Ollama, so it needs no paid
    key. Because httpx is already a core dependency, adding a pyproject.toml extra would declare a
    dependency that does not exist β€” which is why that checklist box is only partly ticked rather than
    ticked or ignored.
  • Tests never touch the network. They use FakeJudge plus the httpx monkeypatch pattern from
    tests/test_embeddings.py, so this is exercised with no key and no connection.

Deliberately out of scope, as follow-ups for you to decide on rather than things I bundled in:
LLM-assisted merging ("should these two be merged?"), judging above-threshold records, batching /
caching / cost caps, and any CLI, MCP or dashboard surface for smart mode.

Verification detail, in case it is useful: the new tests were checked for bite by reverting
service.py and config.py to main while keeping the tests β€” 19 fail and 20 error, including the
rescue and the never-construct-when-disabled cases. The fail-safe paths were exercised against a real
local HTTP server and a real connect timeout rather than mocks. The whole branch was also verified
from a git archive of the commit rather than the working tree, to be sure nothing that only exists
locally is needed to run it.

Footer

Generated by Kimi K3 (brief, implementation, verification), Claude Opus 5 (1M context) (review)

Nitjsefnie and others added 3 commits July 31, 2026 14:45
Add MNEMA_SMART_FORGET_ENABLED (default off) plus judge model/key/URL
config, a new mnema.judge module (pure prompt/verdict helpers and a thin
OpenAI-compatible httpx client), and wire it into apply_decay: the judge
can only rescue formula-selected candidates, and any failure keeps the
memory. When disabled no judge is constructed and no LLM call is possible.

Co-Authored-By: Kimi K3 <noreply@kimi.com>
Add a scripted FakeJudge to the fakes, unit tests for the pure prompt and
verdict helpers plus the httpx-mocked OpenAI-compatible client, service
tests proving the judge only rescues and fails safe to KEEP, and config
tests for the MNEMA_JUDGE_MODEL requirement. No network, no API key.

Co-Authored-By: Kimi K3 <noreply@kimi.com>
Add a 'Smart forgetting (opt-in)' subsection covering the four
MNEMA_JUDGE_*/MNEMA_SMART_FORGET_ENABLED env vars, the rescue-only and
fail-safe-KEEP guarantees, and a local Ollama example.

Co-Authored-By: Kimi K3 <noreply@kimi.com>
@Nitjsefnie Nitjsefnie closed this Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Smart (LLM-assisted) forgetting

1 participant