feat(capture-attention): Phase A — recurrence-weighted preserve+relate+boost#153
Merged
Conversation
…e+boost
Adds an embedding-only capture-attention path that detects cross-session
restatements at save time. When a new memory's content is semantically
close to ≥2 prior memories spanning distinct sessions, the new memory is
preserved, 'restates' relations are inserted from new → each cluster
member, and the canonical neighbor's confidence + recurrence_count
accrete. Default-off behind CAPTURE_ATTENTION_ENABLED; soak-gated.
Driver: 2026-05-22 finding that load-bearing facts stated across many
sessions land as fragmented memories rather than a single canonical
assertion — the operator was implicitly substituting for a missing
mechanism. Plan: private/mnemon-capture-attention-plan-260522.md.
SOTA invariant: preserve+relate+boost, NEVER skip-the-save. Each
restatement carries different framing; the institutional pattern is
keep the data, link via relations, accrete the importance signal on
the canonical. Operator-reviewed merge is Phase C, not Phase A's job.
Embedding-only (FastEmbed, no LLM dependency) per the 2026-05-21
public-release decision. Composes with existing layered defenses:
runs after Layer 0 (is_well_shaped), respects Layer 4 ceiling
(HOOK_SOURCE_CONFIDENCE_CEILING clamps the boost for hook-sourced).
'restates' is a new relation type (no collision with existing
'supersedes' / 'contradicts' / 'related').
Schema additive only: documents.recurrence_count INTEGER NOT NULL
DEFAULT 0 via _migrate_recurrence_count(). Pre-existing rows get
count=0; harmless if flag stays off.
correction_of parameter on Store.save() reserved for salience-tier
Phase 2 — when set, skips capture attention (operator gesture beats
automated recurrence detection).
Failure mode: CaptureAttentionUnavailableError (named) on
embedder/vecstore failure; save() catches + logger.warning + continues
per acceptable swallow category (secondary observability hung off a
primary save path that records the failure).
Soak monitor: mnemon attention-status — boost-rate over 7d,
recurrence-count distribution, top-10 canonicals, last-10 'restates'
audit trail. Acceptance criteria for default-on flip: boost_rate ≤
0.25 + ≥80% precision on 20-canonical manual review.
Calibration: scripts/calibrate_capture_threshold.py — samples N
operator vault pairs, prompts for same/different tagging, computes
precision-recall at {0.70, 0.75, 0.80, 0.85, 0.90}, recommends
precision-leaning sweet spot. Tagged pairs persist to
tests/fixtures/capture_attention_pairs.json (regression lock).
13 new tests in tests/test_capture_attention.py covering: preserve-
everything invariant, feature-flag-off no-behavior-change, distinct-
sessions trigger, same-session no-trigger, threshold respected, hook
ceiling, user uncapped, pinned-canonical selection, correction_of
override, fail-loud on embedder failure, schema migration idempotency.
Suite 801 → 814 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 22, 2026
cipher813
added a commit
that referenced
this pull request
May 22, 2026
Seals the 2026-05-22 substrate arc: - #153 capture-attention Phase A (recurrence-weighted preserve+relate+boost) - #154 salience-tier Phase 1 (first-class standing tier, +3 MCP tools) - #155 build_standing_set.py exemplar bias fix Both new feature paths gated default-off (CAPTURE_ATTENTION_ENABLED, STANDING_TIER_ENABLED) — operator flips per the soak workflow. Post-merge ritual: - tag v0.7.0rc1 + GitHub Release - twine upload (dist/mnemon-memory-0.7.0rc1.{tar.gz,whl}) - mnemon upgrade web --app-name mnemon-memory --mnemon-version 0.7.0rc1 - mnemon doctor 7/7 against live remote - operator promotes 5 career memories + flips MNEMON_STANDING_TIER_ENABLED for ≥1 week soak Suite 836 passing. mnemon --version returns 0.7.0rc1. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cipher813
added a commit
that referenced
this pull request
May 24, 2026
…n fixture (0.7.0rc4) Activation infrastructure for the Phase A capture-attention soak. env-var override: - New MNEMON_CAPTURE_ATTENTION_ENABLED env var takes precedence over config.CAPTURE_ATTENTION_ENABLED. Mirrors the standing-tier pattern (MNEMON_STANDING_TIER_ENABLED) so operators can flip activation on Fly via `flyctl secrets set` without a code change + redeploy, and the next save picks it up without restarting the server. New store._capture_attention_enabled() helper called at request time from Store.save and cli attention-status. - cli attention-status now reports the EFFECTIVE flag value (env-var override applied), not the static config default — Fly secret flips show up here immediately. - 15 new tests (5 helper resolution cases x parametrize over truthy/falsy aliases + whitespace). Privacy hardening of calibration output: - tests/fixtures/capture_attention_pairs.json is now gitignored — every operator run overwrites it with real vault titles + snippets (personal context) that must not land in a public-repo commit. PR #153 originally shipped the path tracked with a placeholder schema; the placeholder moves to capture_attention_pairs.example.json so the format is still discoverable. Suite 858 → 873 passing; coverage 86.46% (gate ≥80%). Version 0.7.0rc3 → 0.7.0rc4.
cipher813
added a commit
that referenced
this pull request
May 24, 2026
…n fixture (0.7.0rc4) (#162) * fix(vecstore): add get(vec_id) — unblock capture-attention calibration scripts/calibrate_capture_threshold.py:79 calls vs.get(vec_id) but VecStore had no such method, blocking the Phase A soak activation workflow (calibration is the gate before flag-flip). Script was added in #153 (capture-attention Phase A) but never run end-to-end. Add get(vec_id) -> np.ndarray | None mirroring the has/delete single-id shape; returns a defensive copy matching export_all's mutation-safety contract. 3 new tests (returns vector, missing→None, defensive-copy invariant). Suite 855 → 858 passing. * fix(calibrate): sample near-neighbor pairs, not uniform-random Random sampling across a 2510-memory vault produces pair cosines clustered at 0.1-0.4 (clearly-different topics) — operator verdicts on those pairs carry no information about whether CAPTURE_ATTENTION_THRESHOLD should be 0.80 or 0.85. The decision region lives at cosine 0.70-0.95. Replace with near-neighbor sampling: pick a random anchor, take its top non-self neighbor via vs.search(), accept if cosine ≥ 0.55 (well below the lowest calibration threshold so edge-negatives survive). Pairs sorted by cosine descending so the operator sees high-confidence near-dupes first. Verified against /tmp/mnemon-prod-snap.sqlite (2510 live memories): 20-pair sample now spans cosine 0.751-0.999 — entirely in the calibration-relevant range, vs. 4/4 obvious-different of the prior uniform-random run. * feat(capture-attention): env-var override + privacy-harden calibration fixture (0.7.0rc4) Activation infrastructure for the Phase A capture-attention soak. env-var override: - New MNEMON_CAPTURE_ATTENTION_ENABLED env var takes precedence over config.CAPTURE_ATTENTION_ENABLED. Mirrors the standing-tier pattern (MNEMON_STANDING_TIER_ENABLED) so operators can flip activation on Fly via `flyctl secrets set` without a code change + redeploy, and the next save picks it up without restarting the server. New store._capture_attention_enabled() helper called at request time from Store.save and cli attention-status. - cli attention-status now reports the EFFECTIVE flag value (env-var override applied), not the static config default — Fly secret flips show up here immediately. - 15 new tests (5 helper resolution cases x parametrize over truthy/falsy aliases + whitespace). Privacy hardening of calibration output: - tests/fixtures/capture_attention_pairs.json is now gitignored — every operator run overwrites it with real vault titles + snippets (personal context) that must not land in a public-repo commit. PR #153 originally shipped the path tracked with a placeholder schema; the placeholder moves to capture_attention_pairs.example.json so the format is still discoverable. Suite 858 → 873 passing; coverage 86.46% (gate ≥80%). Version 0.7.0rc3 → 0.7.0rc4.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an embedding-only capture attention path to mnemon that detects cross-session restatements at save time. When a new memory's content is semantically close (cosine ≥ 0.85) to ≥2 prior memories spanning distinct sessions, the new memory is preserved,
'restates'relations are inserted from new → each cluster member, and the canonical neighbor's confidence + newrecurrence_countcolumn accrete the load-bearing signal.Default-off behind
CAPTURE_ATTENTION_ENABLEDfor soak. Sibling track to the salience-tier work — both target the same failure mode (load-bearing facts under-weighted) from different angles: salience tier is recall-side weighting, capture attention is upstream curation.Driver
Surfaced 2026-05-22 during salience-tier Phase 0 validation attempts: load-bearing facts (runway composition, career posture) stated "numerous times" in chat landed in mnemon as scattered fragments, never as a single canonical assertion. The operator was implicitly substituting for a missing mechanism — that's the gap this PR closes. Brian's framing: "I shouldn't have to compose memories for them to be accurately treated."
Plan doc (gitignored,
private/mnemon-capture-attention-plan-260522.md) contains full invariants, phase table, and soak gates.SOTA correction baked in
An earlier draft of this work considered "boost canonical + skip the new save" as the auto-apply path. Rejected during pre-implementation SOTA audit: each restatement carries different framing, and discarding it throws away the very signal the recurrence detector is honoring. The institutional pattern for dedup-with-importance in production retrieval systems is preserve+relate+boost — keep the data, link via relations, accrete importance on the canonical. MMR diversity at recall naturally suppresses near-dups without us dropping them at capture. Operator-reviewed merge stays in Phase C, not auto-applied here.
What ships
documents.recurrence_countschema migration (additive, idempotent on re-open, harmless if flag stays off)Store.apply_capture_attention()+ private helpers (_resolve_neighbor_docs,_pick_canonical,_boost_confidence,_increment_recurrence)Store.save()integration — gated byCAPTURE_ATTENTION_ENABLED; skipped whencorrection_ofis setcorrection_ofparameter onStore.save()(forward-compat for salience-tier Phase 2 promotion signals — operator gesture beats automated detection)CaptureAttentionUnavailableErrornamed exception;save()catches + WARNs per acceptable-swallow category (secondary observability hung off primary save path)mnemon attention-statusCLI — soak monitor showing boost-rate ratio, recurrence distribution, top canonicals, recent'restates'audit trailscripts/calibrate_capture_threshold.py— data-tuned threshold selection from a vault snapshot (sample N pairs → tag → precision-recall at {0.70, 0.75, 0.80, 0.85, 0.90} → recommend precision-leaning sweet spot)tests/fixtures/capture_attention_pairs.json— regression-lock seed; replaced by operator-tagged pairs via the calibration scripttests/test_capture_attention.pyComposability
is_well_shaped) — runs BEFORE capture attention; scaffolding never reaches the pathHOOK_SOURCE_CONFIDENCE_CEILING) — boost respects the 0.5 ceiling for hook-sourced canonicals'supersedes'/'contradicts'/'related') — new'restates'type, no collisionSoak gates (before flipping default-on)
boost_rate ≤ 0.25over a 7-day window (measured viamnemon attention-status)Test plan
pytest tests/test_capture_attention.py)mnemon attention-statussmoke-tested against a temp vault (flag-off path)scripts/calibrate_capture_threshold.pyagainst prod-snapshot vault to validate threshold = 0.85 is reasonable for their content distribution (post-merge, before flipping flag on)CAPTURE_ATTENTION_ENABLED=Truefor ≥1 week soak; observesmnemon attention-statusweekly🤖 Generated with Claude Code