Implement durable admission assessments (issue #159) - #182
Open
ezutfen wants to merge 2 commits into
Open
Conversation
Make every promotion/admission decision a durable, inspectable, versioned artifact instead of a transient calculation reconstructed from mutable item columns and job state. This persists the decision current Path A policy produces; it does not replace that policy. assess_promotion_candidate() and auto_promote_proposed_memories() remain the sole production authority, and no threshold, weight, cooling period, lane rule or blocker code changes. #157 memory_assessments are recorded as diagnostic references only and never enter the input or policy digests. Schema (migration 038, additive, no row-per-item reconstruction): - append-only admission_assessments with FORCE RLS, a no-rewrite trigger and revoked UPDATE grant. Reads follow item read eligibility; writes are tenant scoped so the worker can record decisions for items it does not author. - admission_assessment_current, a one-row projection keyed by (tenant, item, policy_profile_key), carrying identity plus the precedence metadata needed to resolve current vs stale deterministically. - nullable item_events.admission_assessment_id; historical events stay unlinked. Contract (engram.admission-assessment.v1, ADR + JSON schema + golden vectors): - one policy profile, path_a_compat / path-a-compat-v1; no #158 semantics. - decision_hash is SHA-256 over RFC 8785 canonical bytes of a deterministic envelope; invocation identity and projection state are excluded, and blocker/reason codes are sorted so discovery order cannot change the hash. - outcome precedence stale > blocked > review_required > cooling > insufficient_evidence > unknown, with admitted/would_admit handled separately. cooling requires a lane that would otherwise qualify, so it is never inferred from an age blocker alone, and unknown is never coerced into insufficient evidence. Integration: - capture is gated by ENGRAM_ADMISSION_ASSESSMENT_CAPTURE_ENABLED (default false); disabled, promotion behavior and audit JSON are unchanged. - enabled, a proposed -> active mutation commits atomically with its assessment, linked review_change event and projection, or fails closed. - the assessment insert is ordered after the guarded mutation so a lost race cannot produce a false admitted row; the loser appends a truthful non-mutating result instead. - policy/config is revalidated under the item lock: a change mid-evaluation makes the pre-change result stale history and forces reevaluation before any mutation. A stale decision never becomes current. - dry-run previews are written after the pass rolls back, as shadow rows that can never mutate state or be projected. Also adds current/history/detail/reevaluate API routes, safe summaries on promotion readiness and the review queue with the scoped filters, and a bounded, restartable, idempotent legacy import that fabricates no historical evaluation, evidence or conflict fact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q9TdJoC37r48XLwhUjYGqf
1. Evaluated-input identity was on the wrong side of lifecycle mutations. The guarded proposed -> active UPDATE synchronizes back onto the live ORM object, so the decision was hashed against state its own mutation produced: an admitted decision recorded decision_inputs.review_status='active', asserting that policy evaluated an already-active item. The conflict-recheck path had the mirror problem — it wrote the blocked decision, then mutated the conflict fields that participate in the input digest, so the decision was stale against its own effect the instant it committed. The evaluated state is now snapshotted before any mutation in the pass and the hashing path consumes that snapshot, never a live object. A new nullable resulting_state_digest records what the decision's own authorized mutation was expected to produce, and freshness resolves against it when present. Both claims are hashed, separately. A shadow preview can never carry one; the database enforces that alongside the outcome restriction. The conflict marking moved ahead of its capture (extracted to _mark_promotion_conflict, audit JSON unchanged) so the blocked decision can declare what its recheck changed. 2. Canonical promotion.evaluate retry was not idempotent. UNIQUE (tenant_id, evaluation_id) existed but nothing reused the bound row, so a job whose decision committed before its worker died would collide and dead-letter work that was already durably complete. insert_assessment now resolves a supplied identity back to its bound decision first. The identity also no longer binds the wrong row: a superseded pre-lock result is recorded with no evaluation_id, and the reevaluation that replaces it claims it. 3. Linked admitted assessments could break item deletion. linked_item_event_id used ON DELETE SET NULL, whose referential action attempts an UPDATE on immutable history — refused by the no-rewrite trigger. Deleting the linked event directly failed outright, and the parent-item cascade survived only by incidental FK ordering. It is now a deferred NO ACTION constraint: the parent cascade removes event and decision together, while destroying the event alone still violates it. History is never rewritten either way. 4. Review-queue filters ran after the limit. Outcome, blocker, next-action and due-before were applied in Python to an already-limited page, so a matching item past the limit produced a false "nothing matches". Those are stored facts and now become SQL predicates over a correlated EXISTS; missing is a SQL predicate too. Only the computed current/stale state needs the digest comparison, and it walks the narrowed queue in bounded keyset batches to an explicit cap instead of stopping at the first window. Migration 038 is convergent, so a database created by its earlier shape picks up the new column and constraint in place. Golden vectors, JSON schema and ADR updated for the envelope change. Adds regressions for every fix, including the retry crash window, the conflict-blocked-stays-current case, the linked-pair cascade, and a queue match sitting past the first unfiltered window. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q9TdJoC37r48XLwhUjYGqf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements issue #159: durable, inspectable, versioned admission decisions. Every promotion/admission decision made by current Path A policy is now recorded as an append-only
admission_assessmentsrow plus a mutable one-rowadmission_assessment_currentprojection, enabling post-facto audit and analysis of what policy decided, from which inputs, under which policy identity, and what must happen next.This is not a new promotion policy. The existing
assess_promotion_candidate()andauto_promote_proposed_memories()remain the sole production authority. Capture is gated byENGRAM_ADMISSION_ASSESSMENT_CAPTURE_ENABLED(defaultfalse), so with capture disabled, production behavior and audit JSON are byte-for-byte unchanged.Key Changes
New module
engram/admission_assessment.py: Core decision contract with outcome/action classification, digest/hash computation, and database persistence logic. Defines the closed vocabulary for outcomes, next actions, reason codes, and blocker mappings.Database schema (migration 038): Adds
admission_assessments(immutable history) andadmission_assessment_current(mutable projection) tables with tenant isolation via RLS, structural constraints preventing dishonest decisions, and a trigger enforcing immutability.Integration with promotion flow: Modified
engram/promotion.pyto capture lane qualification details and invoke assessment recording when capture is enabled. Recomputes lane facts needed for audit without changing any policy thresholds or rules.API routes (
engram/api/routes/admission_assessments.py): New endpoints for viewing current/historical assessments and triggering reevaluation. Implements two authority tiers: basic readers see safe summaries; review-scoped users see full normalized inputs and evidence references.Legacy import (
engram/admission_backfill.py): Bounded, restartable, idempotent snapshot import of currently observable state for pre-ENG-PROMOTION-003D — Persist promotion assessments and expose next-action state #159 items. Never reconstructs historical policy; useslegacy_importmode withunknownoutcomes andunavailable_legacyrecheck status.Comprehensive test coverage:
test_admission_assessment_unit.py: Pure contract tests (outcome precedence, next-action mapping, digest determinism)test_admission_assessments_postgres.py: Real PostgreSQL behavior (atomicity, concurrency, staleness, policy changes, shadow previews)test_admission_assessments_migration.py: Schema, RLS, privilege, and immutability proofstest_admission_assessment_api.py: API contract and authorizationDocumentation: ADR-159 explaining the decision, policy identity, digest contents, and invariants.
Schema and validation: JSON Schema for the decision envelope (used for RFC 8785 canonical hashing) and Pydantic models for API responses.
Notable Implementation Details
Decision hash: SHA-256 of RFC 8785 (JCS) canonical JSON, deterministic across runtimes and time. Identifies a decision over inputs and policy, not one execution of it.
Policy identity: Single production profile (
path_a_compat/path-a-compat-v1) covering both legacy and evidence lanes. Digest includes thresholds, lane rules, accepted receipt versions, and kind eligibility—but excludes timestamps and invocation IDs.Outcome precedence: Mutation always outranks blockers; shadow mode never admits; stale projections cannot displace current ones; unknown is never coerced to insufficient_evidence.
Atomic mutation: When capture is enabled, a
proposed -> activemutation fails closed if its assessment, linked audit event, and projection cannot commit atomically.Evidence references: ENG-CLASSIFY-003 — Separate retention value from epistemic confidence and support versioned reassessment #157
memory_assessmentsare recorded as diagnostic references only (max 8 per decision); their epistemic dimensions carry no admission authority in v1.Conflict recheck: Distinguishes
clear,blocked,not_run,not_run_preview(shadow), andunavailable_legacy(pre-ENG-PROMOTION-003D — Persist promotion assessments and expose next-action state #159 import).https://claude.ai/code/session_01Q9TdJoC37r48XLwhUjYGqf