From 472a4c822e2c065d308e8a746f83fa15592e2fdf Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Thu, 29 Oct 2026 19:28:51 +0530
Subject: [PATCH 1/8] feat(pipeline): audit background failures and cascade
state changes through observers
---
AGENTS.md | 2 +-
CHANGELOG.md | 2 +
docs/upgrades/iii-0.25.md | 2 +-
scripts/docs-sync.state.json | 6 +-
src/functions/cascade-audit.ts | 214 +++++++++++++++++++++++++++++
src/functions/export-import.ts | 7 +-
src/functions/facets.ts | 19 +++
src/functions/status.ts | 3 +
src/index.ts | 4 +-
src/runtime/bindings.ts | 12 ++
src/runtime/pipeline-observers.ts | 35 +++++
src/state/schema.ts | 1 +
src/types.ts | 19 +++
test/cascade-audit.test.ts | 220 ++++++++++++++++++++++++++++++
14 files changed, 539 insertions(+), 7 deletions(-)
create mode 100644 src/functions/cascade-audit.ts
create mode 100644 test/cascade-audit.test.ts
diff --git a/AGENTS.md b/AGENTS.md
index 556121405b..6d57fb572a 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -117,5 +117,5 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).
- 146 REST endpoints
- 6 MCP resources, 3 MCP prompts
- 12 hooks, 17 skills
-- 327+ iii functions
+- 330+ iii functions
- 4,100+ tests
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c3e9a6beb5..945d1266fc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
Boot waits for `state::get` up to `AGENTMEMORY_STATE_READY_TIMEOUT_MS` (default 60 s) instead of failing the first read, and an `engine::on-workers-changed` observer (or health-tick probes in trigger mode) re-arms index and capture recovery when the state worker returns.
+- **Failure audit, delete cascade and relation reactions** (#766, #1624, #938, #768). Failure observers on `mem::compress|summarize|graph-extract|consolidate-pipeline|reflect|skill-extract` record classified `*-failed` audit rows (60s dedupe, retryable vs permanent) and status `failures` counters. `mem::on-record-deleted` cascades state deletes to BM25/vector/facet removal with session `observationCount` reconciliation, `mem::import` with `strategy:"replace"` schedules a keyword index rebuild (#938), and `event::graph::edge-reaction` applies supersedes/contradicts/extends memory reactions with cycle guard and per-relation idempotency (un-reacted on delete when no same-type relation remains).
+
- **`AGENTMEMORY_LOCAL_DISPATCH` runtime switch.** iii 0.25 runs a worker's calls to its own functions in-process by default, so AgentMemory's internal calls produce only the SDK's execute span and skip the engine round trip. Set `AGENTMEMORY_LOCAL_DISPATCH=false` for full engine call spans at the cost of a round trip per call.
- **`npm run gate:large-store`**, a release-gate harness that seeds `mem:semantic`, `mem:graph:nodes`, `mem:graph:edges`, `mem:insights` and `mem:sessions` to configurable sizes directly through the engine, then measures the semantic, insights, sessions, counts, graph-index migration, export and mesh-export probes against a real Compose runtime. `--small` selects CI-friendly sizes; `--state-version`/`--http-version` allow A/B runs against different registry worker pins and `--assert` turns it into a gate.
- **`--repeat N` for the startup release gate**, which runs the whole scenario suite N times in fresh temp dirs and reports a per-run pass/fail summary.
diff --git a/docs/upgrades/iii-0.25.md b/docs/upgrades/iii-0.25.md
index 79a077f7f1..d885425c41 100644
--- a/docs/upgrades/iii-0.25.md
+++ b/docs/upgrades/iii-0.25.md
@@ -26,7 +26,7 @@ Existing file state remains at `state_store.db`; this upgrade does not convert o
### Background work runs through observers
-With `AGENTMEMORY_BINDINGS` unset or `auto` (and an SDK that supports `observe`), background work is dispatched by engine observers instead of inline fire-and-forget triggers: per-observation LLM compression follows `mem::observe` output (`agentmemory:pipeline:compress-on-capture`, bounded by `AGENTMEMORY_COMPRESS_MAX_CONCURRENT`, default 2), post-session graph extraction/reflection/consolidation follow `event::session::stopped`, and graph extraction on `mem::remember` is observe-only. `AGENTMEMORY_COMPRESSION_SWEEP_MS` (default 600000, 0 disables) runs a recovery sweep for observations still pending compression. `AGENTMEMORY_GRAPH_EXTRACT_COOLDOWN_MS` (default 0) rate-limits per-session delta extraction, `AGENTMEMORY_GRAPH_STRUCTURAL=false` disables the heuristic graph pass (LLM pass only), and per-session markers in `mem:graph:extract-marks` dedupe extraction to new observations only. In `trigger` mode the existing inline fan-out is unchanged.
+With `AGENTMEMORY_BINDINGS` unset or `auto` (and an SDK that supports `observe`), background work is dispatched by engine observers instead of inline fire-and-forget triggers: per-observation LLM compression follows `mem::observe` output (`agentmemory:pipeline:compress-on-capture`, bounded by `AGENTMEMORY_COMPRESS_MAX_CONCURRENT`, default 2), post-session graph extraction/reflection/consolidation follow `event::session::stopped`, and graph extraction on `mem::remember` is observe-only. `AGENTMEMORY_COMPRESSION_SWEEP_MS` (default 600000, 0 disables) runs a recovery sweep for observations still pending compression. `AGENTMEMORY_GRAPH_EXTRACT_COOLDOWN_MS` (default 0) rate-limits per-session delta extraction, `AGENTMEMORY_GRAPH_STRUCTURAL=false` disables the heuristic graph pass (LLM pass only), and per-session markers in `mem:graph:extract-marks` dedupe extraction to new observations only. Function failures feed `mem::record-failure` through error and soft-failure observers (`agentmemory:pipeline:failure::*`) that write `*-failed` audit rows and move the status `failures` counters (#766). State observers cascade deletes (`agentmemory:cascade:deleted` → `mem::on-record-deleted` clears BM25/vector/facet entries and reconciles session counts, #1624) and react to `mem:relations` rows (`agentmemory:cascade:relation-reaction` → `event::graph::edge-reaction` applies supersedes/contradicts/extends effects, #768). In `trigger` mode the existing inline fan-out is unchanged.
- iii 0.25 can dispatch calls to a worker's own functions locally. Engine-side spans differ for those calls; SDK tracing and state-worker persistence still apply.
- Local dispatch is on by default, so AgentMemory's internal calls to its own functions run in-process and produce only the SDK's execute span, not an engine call span. Set `AGENTMEMORY_LOCAL_DISPATCH=false` for full engine traces at the cost of a round trip per call.
diff --git a/scripts/docs-sync.state.json b/scripts/docs-sync.state.json
index 0080d05d33..625d9a44df 100644
--- a/scripts/docs-sync.state.json
+++ b/scripts/docs-sync.state.json
@@ -23,16 +23,16 @@
"4100+"
],
"functions": [
- "327"
+ "330"
],
"sourceFiles": [
- "262"
+ "263"
],
"loc": [
"59000"
],
"kvScopes": [
- "66"
+ "67"
]
},
"engineVersion": "0.25.0"
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
new file mode 100644
index 0000000000..c401b266b9
--- /dev/null
+++ b/src/functions/cascade-audit.ts
@@ -0,0 +1,214 @@
+import type { IIIClient } from "iii-sdk";
+import type { StateKV } from "../state/kv.js";
+import { collectAll } from "../state/kv.js";
+import { KV } from "../state/schema.js";
+import { withKeyedLock } from "../state/keyed-mutex.js";
+import type { AuditEntry, Memory, MemoryRelation, RelationReactionMark, Session } from "../types.js";
+import { recordAudit } from "./audit.js";
+import { deleteFacetsCascade } from "./facets.js";
+import { getSearchIndex, vectorIndexRemove } from "./search.js";
+
+const FAILURE_OPS: Record = {
+ "mem::compress": "compress-failed",
+ "mem::summarize": "summarize-failed",
+ "mem::graph-extract": "graph-extract-failed",
+ "mem::consolidate-pipeline": "consolidate-failed",
+ "mem::reflect": "reflect-failed",
+ "mem::skill-extract": "skill-extract-failed",
+};
+
+const RETRYABLE_CODES = new Set([
+ "compression_timeout", "timeout", "ECONNRESET", "ECONNREFUSED", "ETIMEDOUT",
+ "circuit_breaker_open", "parse_failed", "empty_response", "overloaded", "compression_failed",
+]);
+const PERMANENT_CODES = new Set([
+ "source_unavailable", "invalid_raw_observation", "invalid_observation_identity",
+ "observation_changed", "observation_not_found", "prompt_too_long", "content_policy",
+]);
+
+function failureCode(envelope: Record): string {
+ const err = envelope.error as { code?: string; status?: number } | undefined;
+ if (err?.code) return err.code;
+ const output = envelope.output as { error?: string; code?: string } | undefined;
+ if (typeof output?.error === "string") return output.error;
+ if (typeof output?.code === "string") return output.code;
+ if (typeof err?.status === "number") return `http_${err.status}`;
+ return "unknown";
+}
+
+function isRetryable(code: string, envelope: Record): boolean {
+ if (RETRYABLE_CODES.has(code)) return true;
+ if (PERMANENT_CODES.has(code)) return false;
+ const err = envelope.error as { status?: number; statusCode?: number } | undefined;
+ const status = err?.status ?? err?.statusCode;
+ if (typeof status === "number") return status === 429 || status >= 500;
+ if (code.startsWith("invalid_")) return false;
+ if (code.includes("timeout") || code.includes("unavailable")) return true;
+ return false;
+}
+
+const DEDUPE_WINDOW_MS = 60_000;
+const DEDUPE_MAX = 512;
+const dedupe = new Map();
+const counters = { transient: 0, permanent: 0, lastAt: null as string | null };
+
+export function failureCounters(): { transient: number; permanent: number; lastAt: string | null } {
+ return { ...counters };
+}
+
+function extractTargetIds(envelope: Record): string[] {
+ const input = (envelope.input ?? {}) as Record;
+ const ids = [input.observationId, input.sessionId, input.memoryId, input.id]
+ .filter((v): v is string => typeof v === "string" && v.length > 0);
+ const obs = input.observations;
+ if (Array.isArray(obs)) {
+ for (const o of obs.slice(0, 10)) {
+ if (o && typeof o.id === "string" && !ids.includes(o.id)) ids.push(o.id);
+ }
+ }
+ return [...new Set(ids)].slice(0, 20);
+}
+
+function pruneDedupe(now: number): void {
+ if (dedupe.size <= DEDUPE_MAX) return;
+ for (const [k, v] of dedupe) {
+ if (now - v.seenAt > DEDUPE_WINDOW_MS) dedupe.delete(k);
+ }
+ while (dedupe.size > DEDUPE_MAX) {
+ const first = dedupe.keys().next().value;
+ if (first === undefined) break;
+ dedupe.delete(first);
+ }
+}
+
+export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
+ sdk.registerFunction("mem::record-failure", async (envelope: Record) => {
+ const meta = envelope.metadata as { function_id?: string; source_fn?: string; observer_id?: string } | undefined;
+ const observerId = (envelope.observer_id ?? meta?.observer_id ?? envelope.observer) as string | undefined;
+ const fromObserverId = typeof observerId === "string" ? /agentmemory:pipeline:failure:(mem::[a-z-]+):(?:error|soft)/.exec(observerId)?.[1] : undefined;
+ const functionId = meta?.source_fn ?? meta?.function_id ?? fromObserverId ?? (envelope.caller as { function_id?: string } | undefined)?.function_id ?? "unknown";
+ const code = failureCode(envelope);
+ const retryable = isRetryable(code, envelope);
+ const now = Date.now();
+ const key = `${functionId}|${code}`;
+ pruneDedupe(now);
+ const prior = dedupe.get(key);
+ const dedupedCount = prior?.skipped ?? 0;
+ if (prior && now - prior.seenAt < DEDUPE_WINDOW_MS) {
+ prior.skipped++;
+ prior.seenAt = now;
+ return { deduped: true, code };
+ }
+ dedupe.set(key, { seenAt: now, skipped: 0 });
+ if (retryable) counters.transient++; else counters.permanent++;
+ counters.lastAt = new Date().toISOString();
+ const errMsg = (envelope.error as { message?: string } | undefined)?.message ??
+ (envelope.output as { error?: string } | undefined)?.error ?? "";
+ const op = FAILURE_OPS[functionId] ?? "cascade";
+ await recordAudit(kv, op, functionId, extractTargetIds(envelope), {
+ code,
+ message: String(errMsg).slice(0, 300),
+ retryable,
+ durationMs: (envelope.metadata as { duration_ms?: number } | undefined)?.duration_ms,
+ dedupedCount,
+ });
+ return { recorded: true, code, retryable };
+ });
+
+ sdk.registerFunction("mem::on-record-deleted", async (event: Record) => {
+ const scope = String(event.scope ?? "");
+ const id = String(event.key ?? "");
+ if (!id) return { skipped: "no_key" };
+ try { getSearchIndex().remove(id); } catch {}
+ try { vectorIndexRemove(id); } catch {}
+ await deleteFacetsCascade(kv, [id]);
+ let sessionReconciled = false;
+ if (scope.startsWith("mem:obs:")) {
+ const sessionId = scope.slice("mem:obs:".length);
+ const keys = await kv.listKeys(scope);
+ const session = await kv.get(KV.sessions, sessionId);
+ if (session && keys.length !== session.observationCount) {
+ await kv.update(KV.sessions, sessionId, [
+ { op: "set", path: "observationCount", value: keys.length },
+ ]);
+ sessionReconciled = true;
+ }
+ }
+ await recordAudit(kv, "cascade", "mem::on-record-deleted", [id], {
+ scope,
+ eventType: event.event_type,
+ sessionReconciled,
+ });
+ return { removed: id, sessionReconciled };
+ });
+
+ sdk.registerFunction("event::graph::edge-reaction", async (event: Record) => {
+ const rel = (event.event_type === "state:deleted" ? event.old_value : event.new_value) as MemoryRelation | undefined;
+ if (!rel || typeof rel.targetId !== "string") return { skipped: "no_relation" };
+ const relationId = String(event.key ?? `${rel.sourceId}->${rel.targetId}:${rel.type}`);
+ const deleted = event.event_type === "state:deleted";
+ if (!deleted) {
+ const existing = await kv.get(KV.relationReactions, relationId);
+ if (existing) return { skipped: "already_applied", relationId };
+ }
+ const all = await collectAll(kv, KV.relations);
+ if (!deleted) {
+ const cyclic = all.find((r) => r.type === rel.type && r.sourceId === rel.targetId && r.targetId === rel.sourceId && `${r.sourceId}->${r.targetId}:${r.type}` !== relationId);
+ if (cyclic) {
+ await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId, rel.sourceId], {
+ skipped: "cycle", type: rel.type, relationId,
+ });
+ return { skipped: "cycle", relationId };
+ }
+ }
+ if (deleted) {
+ const remaining = all.find((r) => r.type === rel.type && r.targetId === rel.targetId && `${r.sourceId}->${r.targetId}:${r.type}` !== relationId);
+ if (remaining) return { unreacted: false, remaining: true };
+ }
+ const mark = deleted ? await kv.get(KV.relationReactions, relationId) : null;
+ const result = await withKeyedLock(`mem:${rel.targetId}`, async () => {
+ const target = await kv.get(KV.memories, rel.targetId);
+ if (!target) {
+ await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId], {
+ skipped: "missing_target", type: rel.type, relationId,
+ });
+ return { skipped: "missing_target" as const };
+ }
+ if (deleted) {
+ if (!mark) return { skipped: "no_mark" as const };
+ if (mark.isLatestBefore !== undefined) target.isLatest = mark.isLatestBefore;
+ if (mark.strengthBefore !== undefined) target.strength = mark.strengthBefore;
+ if (mark.confidenceBefore !== undefined) target.confidence = mark.confidenceBefore;
+ if (mark.relatedIdsBefore !== undefined) target.relatedIds = mark.relatedIdsBefore;
+ await kv.set(KV.memories, rel.targetId, target);
+ return { unreacted: true as const };
+ }
+ const appliedAt = rel.createdAt;
+ const nextMark: RelationReactionMark = { relationId, type: rel.type, targetId: rel.targetId, appliedAt };
+ if (rel.type === "supersedes") {
+ nextMark.isLatestBefore = target.isLatest;
+ nextMark.strengthBefore = target.strength;
+ target.isLatest = false;
+ target.strength = target.strength * 0.5;
+ } else if (rel.type === "contradicts") {
+ nextMark.confidenceBefore = target.confidence;
+ target.confidence = Math.max(0.05, target.confidence - (rel.confidence ?? 0.5) * 0.3);
+ } else if (rel.type === "extends") {
+ nextMark.relatedIdsBefore = target.relatedIds ?? [];
+ target.relatedIds = [...new Set([...(target.relatedIds ?? []), rel.sourceId])];
+ } else {
+ return { skipped: "unsupported_type" as const };
+ }
+ await kv.set(KV.memories, rel.targetId, target);
+ await kv.set(KV.relationReactions, relationId, nextMark);
+ return { reacted: true as const };
+ });
+ if (deleted && (result as { unreacted?: boolean }).unreacted) {
+ await kv.delete(KV.relationReactions, relationId).catch(() => {});
+ }
+ await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId], {
+ type: rel.type, relationId, deleted, ...result,
+ });
+ return { relationId, ...result };
+ });
+}
diff --git a/src/functions/export-import.ts b/src/functions/export-import.ts
index 0627e09fef..0b4f60c7e6 100644
--- a/src/functions/export-import.ts
+++ b/src/functions/export-import.ts
@@ -40,7 +40,8 @@ import {
import { withKeyedLocks } from "../state/keyed-mutex.js";
import { VERSION } from "../version.js";
import { recordAudit } from "./audit.js";
-import { indexRecords } from "./search.js";
+import { indexRecords, rebuildKeywordIndex } from "./search.js";
+import { singleFlight } from "../runtime/recovery.js";
import { resetLessonIndex } from "./lessons.js";
import { logger } from "../logger.js";
import { budgetImportedObservationSources } from "./observation-source-budget.js";
@@ -289,6 +290,9 @@ export function registerExportImportFunction(sdk: IIIClient, state: StateKV): vo
},
);
+ const rebuildAfterReplace = singleFlight(async () => {
+ await rebuildKeywordIndex(kv);
+ });
sdk.registerFunction("mem::import",
async (data: {
exportData: ExportData;
@@ -816,6 +820,7 @@ export function registerExportImportFunction(sdk: IIIClient, state: StateKV): vo
strategy,
stats,
});
+ if (strategy === "replace") void rebuildAfterReplace(kv).catch((err) => logger.warn("Post-replace index rebuild failed", { error: err instanceof Error ? err.message : String(err) }));
return { success: true, strategy, ...stats };
},
);
diff --git a/src/functions/facets.ts b/src/functions/facets.ts
index bdd90a15a4..f5120d870d 100644
--- a/src/functions/facets.ts
+++ b/src/functions/facets.ts
@@ -50,6 +50,25 @@ export async function deleteFacetsForTargets(
}
}
+export async function deleteFacetsCascade(
+ kv: StateKV,
+ targetIds: Iterable,
+): Promise {
+ const ids = new Set(targetIds);
+ ids.delete("");
+ if (ids.size === 0) return;
+ try {
+ const all = await kv.list(KV.facets);
+ const matches = all.filter((f) => ids.has(f.targetId));
+ await Promise.all(matches.map((f) => kv.delete(KV.facets, f.id)));
+ } catch (err) {
+ logger.warn("Facet cascade cleanup failed", {
+ targetIds: ids.size,
+ error: err instanceof Error ? err.message : String(err),
+ });
+ }
+}
+
export function registerFacetsFunction(sdk: IIIClient, kv: StateKV): void {
sdk.registerFunction("mem::facet-tag",
async (data: {
diff --git a/src/functions/status.ts b/src/functions/status.ts
index faa9ec17e7..3fbbe913c7 100644
--- a/src/functions/status.ts
+++ b/src/functions/status.ts
@@ -3,6 +3,7 @@ import type { VectorBackfillState } from "./search.js";
import { describeGraphCompactBoot, type GraphCompactBootStatus } from "./graph-compact-boot.js";
import type { CaptureStatus } from "./capture.js";
import type { GraphIndexMigrationStatus } from "../state/graph-index-migration.js";
+import { failureCounters } from "./cascade-audit.js";
export type StatusLevel = "ok" | "info" | "warn" | "error";
@@ -133,6 +134,7 @@ export interface StatusReport {
capture: CaptureStatus | null;
observeDedup: ObserveDedupStatus | null;
runtime: StatusRuntimeInfo | null;
+ failures: { transient: number; permanent: number; lastAt: string | null };
functions: Array;
flags: Array;
problems: StatusProblem[];
@@ -528,6 +530,7 @@ export function evaluateStatus(input: StatusInputs): StatusReport {
capture: input.capture ?? null,
observeDedup: input.observeDedup ?? null,
runtime: input.runtime ?? null,
+ failures: failureCounters(),
functions,
flags: input.flags.map((flag) =>
flag.enabled && flag.needsLlm && noLlm
diff --git a/src/index.ts b/src/index.ts
index 959fba5222..2dd2a4ee74 100644
--- a/src/index.ts
+++ b/src/index.ts
@@ -105,7 +105,6 @@ import { registerCrystallizeFunction } from "./functions/crystallize.js";
import { registerDiagnosticsFunction } from "./functions/diagnostics.js";
import { registerFacetsFunction } from "./functions/facets.js";
import { registerVerifyFunction } from "./functions/verify.js";
-import { registerCascadeFunction } from "./functions/cascade.js";
import { registerLessonsFunctions } from "./functions/lessons.js";
import { registerObsidianExportFunction } from "./functions/obsidian-export.js";
import { registerReflectFunctions } from "./functions/reflect.js";
@@ -139,6 +138,8 @@ import { checkDuplicateWorkers } from "./runtime/duplicates.js";
import { registerPipelineObservers } from "./runtime/pipeline-observers.js";
import { registerPostSessionFunctions } from "./functions/post-session.js";
import { registerCompressionSweepFunction } from "./functions/compression-sweep.js";
+import { registerCascadeFunction } from "./functions/cascade.js";
+import { registerCascadeFunctions } from "./functions/cascade-audit.js";
import { compressionSweepMs } from "./config.js";
import { ensureServerSecret, explicitSecret, secretFilePath } from "./secret-store.js";
import { mkdirSync, writeFileSync, unlinkSync } from "node:fs";
@@ -395,6 +396,7 @@ async function main() {
registerWorkingMemoryFunctions(sdk, kv, config.tokenBudget);
registerSkillExtractFunctions(sdk, kv, provider);
registerCascadeFunction(sdk, kv);
+ registerCascadeFunctions(sdk, kv);
registerSlidingWindowFunction(sdk, kv, provider);
registerQueryExpansionFunction(sdk, provider);
diff --git a/src/runtime/bindings.ts b/src/runtime/bindings.ts
index f70105decb..f75221f749 100644
--- a/src/runtime/bindings.ts
+++ b/src/runtime/bindings.ts
@@ -33,6 +33,7 @@ interface PipelineSpec {
source: Record;
target: string;
options?: Record;
+ stateOnly?: boolean;
}
export interface BindingReport {
@@ -125,6 +126,16 @@ export class Bindings {
return handle;
}
+ observeStateChanges(filters: Record, target: string, options?: Record): ObserverHandle | null {
+ const source = { function_id: "state::on-changed", filters, capture: ["input"], deliver: "input" };
+ if (!this.resolved && !this.immediate()) {
+ this.pipelinePending.push({ source, target, options, stateOnly: true });
+ return null;
+ }
+ if (this.modes.state !== "observe") return null;
+ return this.observePipeline(source, target, options);
+ }
+
private bindHttp(spec: HttpTriggerSpec): void {
if (this.modes.http === "observe") return this.observeHttp(spec);
this.sdk.registerTrigger?.(spec);
@@ -263,6 +274,7 @@ export class Bindings {
else this.bindState(spec as StateSpec);
}
for (const spec of this.pipelinePending.splice(0)) {
+ if (spec.stateOnly && this.modes.state !== "observe") continue;
this.observePipeline(spec.source, spec.target, spec.options);
}
return modes;
diff --git a/src/runtime/pipeline-observers.ts b/src/runtime/pipeline-observers.ts
index cbee0ce8c5..501a134f50 100644
--- a/src/runtime/pipeline-observers.ts
+++ b/src/runtime/pipeline-observers.ts
@@ -1,5 +1,6 @@
import type { IIIClient } from "iii-sdk";
import { getBindings } from "./bindings.js";
+import { KV } from "../state/schema.js";
import { compressMaxConcurrent, isAutoCompressEnabled, isGraphExtractionEnabled } from "../config.js";
import { isReflectEnabled } from "../functions/slots.js";
@@ -42,4 +43,38 @@ export function registerPipelineObservers(sdk: IIIClient): void {
{ id: "agentmemory:pipeline:post-remember-graph", max_in_flight: 1 },
);
}
+ for (const fn of ["mem::compress", "mem::summarize", "mem::graph-extract", "mem::consolidate-pipeline", "mem::reflect", "mem::skill-extract"]) {
+ bindings.observePipeline(
+ { function_id: fn, on: "error", capture: ["input", "output", "error"], deliver: "envelope" },
+ "mem::record-failure",
+ { id: `agentmemory:pipeline:failure:${fn}:error`, max_in_flight: 4, metadata: { source_fn: fn } },
+ );
+ bindings.observePipeline(
+ { function_id: fn, on: "success", filters: { "output.success": { $eq: false } }, capture: ["input", "output", "error"], deliver: "envelope" },
+ "mem::record-failure",
+ { id: `agentmemory:pipeline:failure:${fn}:soft`, max_in_flight: 4, metadata: { source_fn: fn } },
+ );
+ }
+ bindings.observeStateChanges(
+ {
+ $or: [
+ { "input.event_type": { $eq: "state:deleted" }, "input.scope": { $prefix: "mem:obs:" } },
+ { "input.event_type": { $eq: "state:deleted" }, "input.scope": { $eq: KV.memories } },
+ { "input.event_type": { $eq: "state:deleted" }, "input.scope": { $eq: KV.lessons } },
+ { "input.scope": { $eq: KV.lessons }, "input.new_value.deleted": { $eq: true }, "input.old_value.deleted": { $ne: true } },
+ ],
+ },
+ "mem::on-record-deleted",
+ { id: "agentmemory:cascade:deleted", max_in_flight: 4 },
+ );
+ bindings.observeStateChanges(
+ {
+ $or: [
+ { "input.event_type": { $eq: "state:created" }, "input.scope": { $eq: KV.relations }, "input.new_value.type": { $in: ["supersedes", "contradicts", "extends"] } },
+ { "input.event_type": { $eq: "state:deleted" }, "input.scope": { $eq: KV.relations }, "input.old_value.type": { $in: ["supersedes", "contradicts", "extends"] } },
+ ],
+ },
+ "event::graph::edge-reaction",
+ { id: "agentmemory:cascade:relation-reaction", max_in_flight: 1 },
+ );
}
diff --git a/src/state/schema.ts b/src/state/schema.ts
index 885f772434..e44526ad2d 100644
--- a/src/state/schema.ts
+++ b/src/state/schema.ts
@@ -29,6 +29,7 @@ export const KV = {
graphIndexMigration: "mem:graph:index-migration",
graphIndexManifest: "mem:graph:index-manifest",
graphExtractMarks: "mem:graph:extract-marks",
+ relationReactions: "mem:relation-reactions",
semantic: "mem:semantic",
procedural: "mem:procedural",
teamShared: (teamId: string) => `mem:team:${teamId}:shared`,
diff --git a/src/types.ts b/src/types.ts
index 392b53e505..ad6b57a14d 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -1,3 +1,14 @@
+export interface RelationReactionMark {
+ relationId: string;
+ type: MemoryRelation["type"];
+ targetId: string;
+ isLatestBefore?: boolean;
+ strengthBefore?: number;
+ confidenceBefore?: number;
+ relatedIdsBefore?: string[];
+ appliedAt: string;
+}
+
export interface GraphExtractMark {
sessionId: string;
lastObservationTimestamp: string;
@@ -735,6 +746,14 @@ export interface AuditEntry {
| "core_add"
| "core_remove"
| "auto_page"
+ | "compress-failed"
+ | "summarize-failed"
+ | "graph-extract-failed"
+ | "consolidate-failed"
+ | "reflect-failed"
+ | "skill-extract-failed"
+ | "cascade"
+ | "relation_reaction"
| "vision_embed"
| "slot_append"
| "slot_replace"
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
new file mode 100644
index 0000000000..f26e68906d
--- /dev/null
+++ b/test/cascade-audit.test.ts
@@ -0,0 +1,220 @@
+import { describe, it, expect, vi, beforeEach } from "vitest";
+import { registerCascadeFunctions, failureCounters } from "../src/functions/cascade-audit.js";
+import { registerPipelineObservers } from "../src/runtime/pipeline-observers.js";
+import { getBindings } from "../src/runtime/bindings.js";
+import { KV } from "../src/state/schema.js";
+import type { Memory, MemoryRelation, Facet } from "../src/types.js";
+
+vi.mock("../src/logger.js", () => ({
+ logger: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() },
+}));
+
+const audits: Array> = [];
+vi.mock("../src/functions/audit.js", () => ({
+ recordAudit: vi.fn(async (_kv: unknown, operation: string, functionId: string, targetIds: string[], details: Record) => {
+ audits.push({ operation, functionId, targetIds, details });
+ return { id: "aud_x" };
+ }),
+}));
+
+const removedBm25: string[] = [];
+const removedVector: string[] = [];
+vi.mock("../src/functions/search.js", () => ({
+ getSearchIndex: () => ({ remove: (id: string) => removedBm25.push(id) }),
+ getVectorIndex: () => ({ size: 0 }),
+ vectorIndexRemove: (id: string) => { removedVector.push(id); },
+ isMemoryIndexReady: () => true,
+}));
+
+function memoryKv(seed: Record> = {}) {
+ const store = seed;
+ return {
+ store,
+ get: vi.fn(async (scope: string, key: string) => store[scope]?.get(key) ?? null),
+ set: vi.fn(async (scope: string, key: string, v: unknown) => { (store[scope] ??= new Map()).set(key, v); }),
+ delete: vi.fn(async (scope: string, key: string) => { store[scope]?.delete(key); }),
+ update: vi.fn(async (scope: string, key: string, ops: Array<{ path: string; value: unknown }>) => {
+ const row = store[scope]?.get(key) as Record | undefined;
+ if (row) for (const op of ops) row[op.path] = op.value;
+ return row;
+ }),
+ list: vi.fn(async (scope: string) => [...(store[scope]?.values() ?? [])]),
+ listKeys: vi.fn(async (scope: string) => [...(store[scope]?.keys() ?? [])]),
+ iterate: async function* (scope: string) { yield [...(store[scope]?.values() ?? [])]; },
+ backend: "file",
+ };
+}
+
+function fakeFns() {
+ const fns = new Map Promise>();
+ return {
+ functions: fns,
+ registerFunction(id: string, h: (d: unknown) => Promise) { fns.set(id, h); },
+ };
+}
+
+describe("mem::record-failure", () => {
+ beforeEach(() => { audits.length = 0; });
+
+ it("classifies retryable vs permanent and writes the op audit", async () => {
+ const sdk = fakeFns();
+ const kv = memoryKv();
+ registerCascadeFunctions(sdk as never, kv as never);
+ const fn = sdk.functions.get("mem::record-failure")!;
+ await fn({ input: { observationId: "o1", sessionId: "s1" }, output: {}, error: { code: "circuit_breaker_open", message: "breaker" }, metadata: { function_id: "mem::compress" } });
+ expect(audits[0].operation).toBe("compress-failed");
+ expect(audits[0].details).toMatchObject({ code: "circuit_breaker_open", retryable: true });
+ expect(audits[0].targetIds).toEqual(["o1", "s1"]);
+ const c = failureCounters();
+ expect(c.transient).toBeGreaterThan(0);
+ audits.length = 0;
+ await fn({ input: {}, output: { success: false, error: "source_unavailable" }, metadata: { function_id: "mem::summarize" } });
+ expect(audits[0].operation).toBe("summarize-failed");
+ expect(audits[0].details).toMatchObject({ code: "source_unavailable", retryable: false });
+ });
+
+ it("dedupes by (function_id, code) within 60s", async () => {
+ const sdk = fakeFns();
+ registerCascadeFunctions(sdk as never, memoryKv() as never);
+ const fn = sdk.functions.get("mem::record-failure")!;
+ const env = { input: {}, output: {}, error: { code: "parse_failed" }, metadata: { function_id: "mem::graph-extract" } };
+ await fn(env);
+ const count = audits.length;
+ const r = await fn(env);
+ expect(r).toMatchObject({ deduped: true });
+ expect(audits.length).toBe(count);
+ });
+});
+
+describe("pipeline observer specs", () => {
+ it("failure observers: error + soft specs per function", async () => {
+ const observed: Array<{ source: Record; target: string; options?: Record }> = [];
+ const sdk = {
+ observed,
+ registerTrigger: vi.fn(),
+ registerFunction: vi.fn(),
+ observe: vi.fn((s: Record, t: string, o?: Record) => { observed.push({ source: s, target: t, options: o }); return { id: "h", state: "active", ready: () => Promise.resolve(), onStateChange: () => () => {}, dispose: () => Promise.resolve() }; }),
+ trigger: vi.fn(async () => ({ functions: ["http::on-request", "state::on-changed", "state::get"] })),
+ };
+ const b = getBindings(sdk as never);
+ registerPipelineObservers(sdk as never);
+ await b.activate();
+ const compressErr = observed.find((o) => o.options?.id === "agentmemory:pipeline:failure:mem::compress:error")!;
+ expect(compressErr.source).toMatchObject({ function_id: "mem::compress", on: "error", capture: ["input", "output", "error"], deliver: "envelope" });
+ expect(compressErr.options?.max_in_flight).toBe(4);
+ const summarizeSoft = observed.find((o) => o.options?.id === "agentmemory:pipeline:failure:mem::summarize:soft")!;
+ expect(summarizeSoft.source).toMatchObject({ on: "success", filters: { "output.success": { $eq: false } } });
+ const deleted = observed.find((o) => o.options?.id === "agentmemory:cascade:deleted")!;
+ const or = deleted.source.filters as { $or: Array> };
+ expect(or.$or[0]).toMatchObject({ "input.event_type": { $eq: "state:deleted" }, "input.scope": { $prefix: "mem:obs:" } });
+ const rel = observed.find((o) => o.options?.id === "agentmemory:cascade:relation-reaction")!;
+ const or2 = rel.source.filters as { $or: Array> };
+ expect(or2.$or[0]["input.new_value.type"]).toEqual({ $in: ["supersedes", "contradicts", "extends"] });
+ expect(rel.options?.max_in_flight).toBe(1);
+ });
+});
+
+describe("mem::on-record-deleted", () => {
+ beforeEach(() => { audits.length = 0; removedBm25.length = 0; removedVector.length = 0; });
+
+ it("removes bm25/vector entries, facets (incl. lessons), reconciles observationCount, audits", async () => {
+ const sdk = fakeFns();
+ const facets = new Map([
+ ["f1", { id: "f1", targetId: "m1", targetType: "memory", dimension: "d", value: "v", createdAt: "t" }],
+ ["f2", { id: "f2", targetId: "l1", targetType: "memory", dimension: "d", value: "v", createdAt: "t" }],
+ ["f3", { id: "f3", targetId: "other", targetType: "memory", dimension: "d", value: "v", createdAt: "t" }],
+ ]);
+ const kv = memoryKv({
+ [KV.facets]: facets,
+ [KV.sessions]: new Map([["s1", { id: "s1", observationCount: 5 }]]),
+ "mem:obs:s1": new Map([["o1", {}], ["o2", {}]]),
+ });
+ registerCascadeFunctions(sdk as never, kv as never);
+ const fn = sdk.functions.get("mem::on-record-deleted")!;
+ await fn({ event_type: "state:deleted", scope: "mem:obs:s1", key: "o9" });
+ expect(removedBm25).toContain("o9");
+ expect(removedVector).toContain("o9");
+ expect(kv.update).toHaveBeenCalledWith(KV.sessions, "s1", [{ op: "set", path: "observationCount", value: 2 }]);
+ expect(audits.find((a) => a.operation === "cascade")).toBeTruthy();
+ await fn({ scope: KV.lessons, key: "l1", event_type: "state:updated", new_value: { deleted: true }, old_value: { deleted: false } });
+ expect(kv.store[KV.facets].has("f2")).toBe(false);
+ expect(kv.store[KV.facets].has("f3")).toBe(true);
+ await fn({ event_type: "state:deleted", scope: KV.memories, key: "m1" });
+ expect(kv.store[KV.facets].has("f1")).toBe(false);
+ });
+});
+
+describe("event::graph::edge-reaction", () => {
+ const base: Memory = { id: "tgt", createdAt: "t", updatedAt: "t", type: "fact", title: "x", content: "c", concepts: [], files: [], sessionIds: [], strength: 1, version: 1, isLatest: true, confidence: 0.8, relatedIds: [] } as unknown as Memory;
+ const rel: MemoryRelation = { type: "supersedes", sourceId: "src", targetId: "tgt", createdAt: "2026-01-01T00:00:00Z", confidence: 0.6 };
+
+ function seeded(seedRel = true) {
+ const sdk = fakeFns();
+ const kv = memoryKv({
+ [KV.memories]: new Map([["tgt", { ...base }]]),
+ [KV.relations]: new Map(seedRel ? [["r1", { ...rel }]] : []),
+ });
+ registerCascadeFunctions(sdk as never, kv as never);
+ return { sdk, kv, fn: sdk.functions.get("event::graph::edge-reaction")! };
+ }
+
+ it("supersedes: isLatest false + strength halved, mark stored", async () => {
+ const { kv, fn } = seeded();
+ await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ const mem = kv.store[KV.memories].get("tgt") as Memory;
+ expect(mem.isLatest).toBe(false);
+ expect(mem.strength).toBeCloseTo(0.5);
+ expect(kv.store[KV.relationReactions]?.get("r1")).toMatchObject({ type: "supersedes", isLatestBefore: true, strengthBefore: 1 });
+ });
+
+ it("contradicts: confidence floored reduction; extends: relatedIds union", async () => {
+ const { kv, fn } = seeded();
+ await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel, type: "contradicts", confidence: 0.5 } });
+ expect((kv.store[KV.memories].get("tgt") as Memory).confidence).toBeCloseTo(0.65);
+ await fn({ event_type: "state:created", scope: KV.relations, key: "r2", new_value: { ...rel, type: "extends", sourceId: "src2" } });
+ expect((kv.store[KV.memories].get("tgt") as Memory).relatedIds).toEqual(["src2"]);
+ });
+
+ it("idempotent on redelivery", async () => {
+ const { kv, fn } = seeded();
+ await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ const r2 = await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ expect(r2).toMatchObject({ skipped: "already_applied" });
+ expect((kv.store[KV.memories].get("tgt") as Memory).strength).toBeCloseTo(0.5);
+ });
+
+ it("delete un-reacts only when no same-type relation remains", async () => {
+ const { kv, fn } = seeded();
+ await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ kv.store[KV.relations].delete("r1");
+ const r = await fn({ event_type: "state:deleted", scope: KV.relations, key: "r1", old_value: { ...rel } });
+ expect(r).toMatchObject({ unreacted: true });
+ expect((kv.store[KV.memories].get("tgt") as Memory).isLatest).toBe(true);
+ expect((kv.store[KV.memories].get("tgt") as Memory).strength).toBeCloseTo(1);
+ });
+
+ it("delete keeps reaction when a same-type relation remains", async () => {
+ const { kv, fn } = seeded();
+ kv.store[KV.relations].set("r2", { ...rel, sourceId: "other" });
+ await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ const r = await fn({ event_type: "state:deleted", scope: KV.relations, key: "r1", old_value: { ...rel } });
+ expect(r).toMatchObject({ unreacted: false, remaining: true });
+ expect((kv.store[KV.memories].get("tgt") as Memory).isLatest).toBe(false);
+ });
+
+ it("cycle guard skips and audits", async () => {
+ const { kv, fn } = seeded();
+ kv.store[KV.relations].set("back", { type: "supersedes", sourceId: "tgt", targetId: "src", createdAt: "t" });
+ const r = await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ expect(r).toMatchObject({ skipped: "cycle" });
+ expect((kv.store[KV.memories].get("tgt") as Memory).isLatest).toBe(true);
+ expect(audits.find((a) => (a.details as { skipped?: string }).skipped === "cycle")).toBeTruthy();
+ });
+
+ it("missing target skips and audits", async () => {
+ const { kv, fn } = seeded();
+ kv.store[KV.memories].delete("tgt");
+ const r = await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ expect(r).toMatchObject({ skipped: "missing_target" });
+ });
+});
From 8e92f621052c28e3264943f98ffecd899db1d92c Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Thu, 29 Oct 2026 20:20:17 +0530
Subject: [PATCH 2/8] fix(pipeline): coalesce relation reactions per target and
batch delete cascades
---
src/functions/cascade-audit.ts | 194 ++++++++++++++++++++-------------
src/functions/export-import.ts | 2 +-
src/functions/relations.ts | 28 +++--
src/types.ts | 9 +-
test/cascade-audit.test.ts | 121 +++++++++++---------
5 files changed, 211 insertions(+), 143 deletions(-)
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
index c401b266b9..32e185f1fb 100644
--- a/src/functions/cascade-audit.ts
+++ b/src/functions/cascade-audit.ts
@@ -7,6 +7,7 @@ import type { AuditEntry, Memory, MemoryRelation, RelationReactionMark, Session
import { recordAudit } from "./audit.js";
import { deleteFacetsCascade } from "./facets.js";
import { getSearchIndex, vectorIndexRemove } from "./search.js";
+import { singleFlight } from "../runtime/recovery.js";
const FAILURE_OPS: Record = {
"mem::compress": "compress-failed",
@@ -81,7 +82,53 @@ function pruneDedupe(now: number): void {
}
}
+const DELETE_BATCH_MS = 500;
+const DELETE_BATCH_MAX = 200;
+const pendingDeletes = new Set();
+const pendingSessions = new Set();
+let deleteTimer: ReturnType | null = null;
+let deleteFlusher: (() => Promise) | null = null;
+
+function queueDeleteCascade(id: string, sessionId?: string): void {
+ pendingDeletes.add(id);
+ if (sessionId) pendingSessions.add(sessionId);
+ if (pendingDeletes.size >= DELETE_BATCH_MAX) void flushDeleteCascade();
+ else if (!deleteTimer) {
+ deleteTimer = setTimeout(() => { deleteTimer = null; void flushDeleteCascade(); }, DELETE_BATCH_MS);
+ deleteTimer.unref();
+ }
+}
+
+export async function flushDeleteCascade(): Promise {
+ if (!deleteFlusher) return;
+ await deleteFlusher();
+}
+
export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
+ deleteFlusher = singleFlight(async () => {
+ if (deleteTimer) { clearTimeout(deleteTimer); deleteTimer = null; }
+ const ids = [...pendingDeletes];
+ const sessions = [...pendingSessions];
+ pendingDeletes.clear();
+ pendingSessions.clear();
+ if (ids.length === 0 && sessions.length === 0) return;
+ await deleteFacetsCascade(kv, ids);
+ const reconciled: string[] = [];
+ for (const sessionId of sessions) {
+ const keys = await kv.listKeys(KV.observations(sessionId));
+ const session = await kv.get(KV.sessions, sessionId);
+ if (session && keys.length !== session.observationCount) {
+ await kv.update(KV.sessions, sessionId, [
+ { type: "set", path: "observationCount", value: keys.length },
+ ]);
+ reconciled.push(sessionId);
+ }
+ }
+ await recordAudit(kv, "cascade", "mem::on-record-deleted", ids.slice(0, 20), {
+ batched: ids.length,
+ sessionsReconciled: reconciled,
+ });
+ });
sdk.registerFunction("mem::record-failure", async (envelope: Record) => {
const meta = envelope.metadata as { function_id?: string; source_fn?: string; observer_id?: string } | undefined;
const observerId = (envelope.observer_id ?? meta?.observer_id ?? envelope.observer) as string | undefined;
@@ -94,14 +141,13 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
pruneDedupe(now);
const prior = dedupe.get(key);
const dedupedCount = prior?.skipped ?? 0;
+ if (retryable) counters.transient++; else counters.permanent++;
+ counters.lastAt = new Date(now).toISOString();
if (prior && now - prior.seenAt < DEDUPE_WINDOW_MS) {
prior.skipped++;
- prior.seenAt = now;
return { deduped: true, code };
}
dedupe.set(key, { seenAt: now, skipped: 0 });
- if (retryable) counters.transient++; else counters.permanent++;
- counters.lastAt = new Date().toISOString();
const errMsg = (envelope.error as { message?: string } | undefined)?.message ??
(envelope.output as { error?: string } | undefined)?.error ?? "";
const op = FAILURE_OPS[functionId] ?? "cascade";
@@ -115,57 +161,23 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
return { recorded: true, code, retryable };
});
- sdk.registerFunction("mem::on-record-deleted", async (event: Record) => {
+ sdk.registerFunction("mem::on-record-deleted", (event: Record) => {
const scope = String(event.scope ?? "");
const id = String(event.key ?? "");
- if (!id) return { skipped: "no_key" };
+ if (!id) return Promise.resolve({ skipped: "no_key" });
try { getSearchIndex().remove(id); } catch {}
try { vectorIndexRemove(id); } catch {}
- await deleteFacetsCascade(kv, [id]);
- let sessionReconciled = false;
- if (scope.startsWith("mem:obs:")) {
- const sessionId = scope.slice("mem:obs:".length);
- const keys = await kv.listKeys(scope);
- const session = await kv.get(KV.sessions, sessionId);
- if (session && keys.length !== session.observationCount) {
- await kv.update(KV.sessions, sessionId, [
- { op: "set", path: "observationCount", value: keys.length },
- ]);
- sessionReconciled = true;
- }
- }
- await recordAudit(kv, "cascade", "mem::on-record-deleted", [id], {
- scope,
- eventType: event.event_type,
- sessionReconciled,
- });
- return { removed: id, sessionReconciled };
+ const sessionId = scope.startsWith("mem:obs:") ? scope.slice("mem:obs:".length) : undefined;
+ queueDeleteCascade(id, sessionId);
+ return Promise.resolve({ removed: id, queued: true });
});
sdk.registerFunction("event::graph::edge-reaction", async (event: Record) => {
const rel = (event.event_type === "state:deleted" ? event.old_value : event.new_value) as MemoryRelation | undefined;
if (!rel || typeof rel.targetId !== "string") return { skipped: "no_relation" };
const relationId = String(event.key ?? `${rel.sourceId}->${rel.targetId}:${rel.type}`);
+ const markKey = `${rel.targetId}:${rel.type}`;
const deleted = event.event_type === "state:deleted";
- if (!deleted) {
- const existing = await kv.get(KV.relationReactions, relationId);
- if (existing) return { skipped: "already_applied", relationId };
- }
- const all = await collectAll(kv, KV.relations);
- if (!deleted) {
- const cyclic = all.find((r) => r.type === rel.type && r.sourceId === rel.targetId && r.targetId === rel.sourceId && `${r.sourceId}->${r.targetId}:${r.type}` !== relationId);
- if (cyclic) {
- await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId, rel.sourceId], {
- skipped: "cycle", type: rel.type, relationId,
- });
- return { skipped: "cycle", relationId };
- }
- }
- if (deleted) {
- const remaining = all.find((r) => r.type === rel.type && r.targetId === rel.targetId && `${r.sourceId}->${r.targetId}:${r.type}` !== relationId);
- if (remaining) return { unreacted: false, remaining: true };
- }
- const mark = deleted ? await kv.get(KV.relationReactions, relationId) : null;
const result = await withKeyedLock(`mem:${rel.targetId}`, async () => {
const target = await kv.get(KV.memories, rel.targetId);
if (!target) {
@@ -174,41 +186,77 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
});
return { skipped: "missing_target" as const };
}
- if (deleted) {
- if (!mark) return { skipped: "no_mark" as const };
- if (mark.isLatestBefore !== undefined) target.isLatest = mark.isLatestBefore;
- if (mark.strengthBefore !== undefined) target.strength = mark.strengthBefore;
- if (mark.confidenceBefore !== undefined) target.confidence = mark.confidenceBefore;
- if (mark.relatedIdsBefore !== undefined) target.relatedIds = mark.relatedIdsBefore;
+ const mark = await kv.get(KV.relationReactions, markKey);
+ if (!deleted) {
+ if (mark?.sources && Object.hasOwn(mark.sources, relationId)) {
+ return { skipped: "already_applied" as const, relationId };
+ }
+ if (rel.type === "supersedes") {
+ const reverseMark = await kv.get(KV.relationReactions, `${rel.sourceId}:supersedes`);
+ if (reverseMark && Object.values(reverseMark.sources).includes(rel.targetId)) {
+ await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId, rel.sourceId], {
+ skipped: "cycle", type: rel.type, relationId,
+ });
+ return { skipped: "cycle" as const };
+ }
+ }
+ const next: RelationReactionMark = mark ?? {
+ targetId: rel.targetId,
+ type: rel.type,
+ sources: {},
+ firstAppliedAt: rel.createdAt,
+ lastAppliedAt: rel.createdAt,
+ };
+ next.sources[relationId] = rel.sourceId;
+ next.lastAppliedAt = rel.createdAt;
+ if (rel.type === "supersedes") {
+ if (!mark) {
+ next.isLatestBefore = target.isLatest;
+ next.strengthBefore = target.strength;
+ target.isLatest = false;
+ target.strength = target.strength * 0.5;
+ }
+ } else if (rel.type === "contradicts") {
+ if (!mark) next.confidenceBefore = target.confidence;
+ target.confidence = Math.max(0.05, target.confidence - (rel.confidence ?? 0.5) * 0.3);
+ } else if (rel.type === "extends") {
+ const related = target.relatedIds ?? [];
+ if (!related.includes(rel.sourceId)) {
+ target.relatedIds = [...related, rel.sourceId];
+ next.relatedIdsAdded = [...(next.relatedIdsAdded ?? []), rel.sourceId];
+ }
+ } else {
+ return { skipped: "unsupported_type" as const };
+ }
await kv.set(KV.memories, rel.targetId, target);
- return { unreacted: true as const };
+ await kv.set(KV.relationReactions, markKey, next);
+ return { reacted: true as const, contributors: Object.keys(next.sources).length };
}
- const appliedAt = rel.createdAt;
- const nextMark: RelationReactionMark = { relationId, type: rel.type, targetId: rel.targetId, appliedAt };
- if (rel.type === "supersedes") {
- nextMark.isLatestBefore = target.isLatest;
- nextMark.strengthBefore = target.strength;
- target.isLatest = false;
- target.strength = target.strength * 0.5;
- } else if (rel.type === "contradicts") {
- nextMark.confidenceBefore = target.confidence;
- target.confidence = Math.max(0.05, target.confidence - (rel.confidence ?? 0.5) * 0.3);
- } else if (rel.type === "extends") {
- nextMark.relatedIdsBefore = target.relatedIds ?? [];
- target.relatedIds = [...new Set([...(target.relatedIds ?? []), rel.sourceId])];
- } else {
- return { skipped: "unsupported_type" as const };
+ if (!mark || !Object.hasOwn(mark.sources, relationId)) {
+ return { skipped: "no_mark" as const };
+ }
+ delete mark.sources[relationId];
+ if (Object.keys(mark.sources).length > 0) {
+ await kv.set(KV.relationReactions, markKey, mark);
+ return { unreacted: false as const, remaining: Object.keys(mark.sources).length };
+ }
+ if (mark.isLatestBefore !== undefined) target.isLatest = mark.isLatestBefore;
+ if (mark.strengthBefore !== undefined) target.strength = mark.strengthBefore;
+ if (mark.confidenceBefore !== undefined) target.confidence = mark.confidenceBefore;
+ if (mark.relatedIdsAdded?.length) {
+ const added = new Set(mark.relatedIdsAdded);
+ target.relatedIds = (target.relatedIds ?? []).filter((id) => !added.has(id));
}
await kv.set(KV.memories, rel.targetId, target);
- await kv.set(KV.relationReactions, relationId, nextMark);
- return { reacted: true as const };
+ await kv.delete(KV.relationReactions, markKey);
+ return { unreacted: true as const };
});
- if (deleted && (result as { unreacted?: boolean }).unreacted) {
- await kv.delete(KV.relationReactions, relationId).catch(() => {});
+ const skip = (result as { skipped?: string }).skipped;
+ if (skip !== "cycle" && skip !== "missing_target") {
+ await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId], {
+ type: rel.type, relationId, deleted, ...result,
+ });
}
- await recordAudit(kv, "relation_reaction", "event::graph::edge-reaction", [rel.targetId], {
- type: rel.type, relationId, deleted, ...result,
- });
- return { relationId, ...result };
+ return { relationId, markKey, ...result };
});
}
diff --git a/src/functions/export-import.ts b/src/functions/export-import.ts
index 0b4f60c7e6..2be81e2dee 100644
--- a/src/functions/export-import.ts
+++ b/src/functions/export-import.ts
@@ -820,7 +820,7 @@ export function registerExportImportFunction(sdk: IIIClient, state: StateKV): vo
strategy,
stats,
});
- if (strategy === "replace") void rebuildAfterReplace(kv).catch((err) => logger.warn("Post-replace index rebuild failed", { error: err instanceof Error ? err.message : String(err) }));
+ if (strategy === "replace") void rebuildAfterReplace().catch((err) => logger.warn("Post-replace index rebuild failed", { error: err instanceof Error ? err.message : String(err) }));
return { success: true, strategy, ...stats };
},
);
diff --git a/src/functions/relations.ts b/src/functions/relations.ts
index 5f9b02c2ed..e0750d78a1 100644
--- a/src/functions/relations.ts
+++ b/src/functions/relations.ts
@@ -91,21 +91,25 @@ export function registerRelationsFunction(sdk: IIIClient, kv: StateKV): void {
const relationId = generateId("rel");
await kv.set(KV.relations, relationId, relation);
- if (!source.relatedIds) source.relatedIds = [];
let sourceUpdated = false;
- if (!source.relatedIds.includes(data.targetId)) {
- source.relatedIds.push(data.targetId);
- await kv.set(KV.memories, data.sourceId, source);
- sourceUpdated = true;
- }
+ await withKeyedLock(`mem:${data.sourceId}`, async () => {
+ const fresh = await kv.get(KV.memories, data.sourceId);
+ if (fresh && !(fresh.relatedIds ?? []).includes(data.targetId)) {
+ fresh.relatedIds = [...(fresh.relatedIds ?? []), data.targetId];
+ await kv.set(KV.memories, data.sourceId, fresh);
+ sourceUpdated = true;
+ }
+ });
- if (!target.relatedIds) target.relatedIds = [];
let targetUpdated = false;
- if (!target.relatedIds.includes(data.sourceId)) {
- target.relatedIds.push(data.sourceId);
- await kv.set(KV.memories, data.targetId, target);
- targetUpdated = true;
- }
+ await withKeyedLock(`mem:${data.targetId}`, async () => {
+ const fresh = await kv.get(KV.memories, data.targetId);
+ if (fresh && !(fresh.relatedIds ?? []).includes(data.sourceId)) {
+ fresh.relatedIds = [...(fresh.relatedIds ?? []), data.sourceId];
+ await kv.set(KV.memories, data.targetId, fresh);
+ targetUpdated = true;
+ }
+ })
await safeAudit(kv, "relation_create", "mem::relate", [relationId], {
type: data.type,
diff --git a/src/types.ts b/src/types.ts
index ad6b57a14d..0457792063 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -1,12 +1,13 @@
export interface RelationReactionMark {
- relationId: string;
- type: MemoryRelation["type"];
targetId: string;
+ type: MemoryRelation["type"];
+ sources: Record;
isLatestBefore?: boolean;
strengthBefore?: number;
confidenceBefore?: number;
- relatedIdsBefore?: string[];
- appliedAt: string;
+ relatedIdsAdded?: string[];
+ firstAppliedAt: string;
+ lastAppliedAt: string;
}
export interface GraphExtractMark {
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
index f26e68906d..613959e789 100644
--- a/test/cascade-audit.test.ts
+++ b/test/cascade-audit.test.ts
@@ -73,16 +73,23 @@ describe("mem::record-failure", () => {
expect(audits[0].details).toMatchObject({ code: "source_unavailable", retryable: false });
});
- it("dedupes by (function_id, code) within 60s", async () => {
+ it("dedupes by (function_id, code) in a fixed 60s window and counts every failure", async () => {
+ vi.useFakeTimers();
const sdk = fakeFns();
registerCascadeFunctions(sdk as never, memoryKv() as never);
const fn = sdk.functions.get("mem::record-failure")!;
const env = { input: {}, output: {}, error: { code: "parse_failed" }, metadata: { function_id: "mem::graph-extract" } };
await fn(env);
- const count = audits.length;
- const r = await fn(env);
- expect(r).toMatchObject({ deduped: true });
- expect(audits.length).toBe(count);
+ vi.advanceTimersByTime(30_000);
+ const r1 = await fn(env);
+ expect(r1).toMatchObject({ deduped: true });
+ expect(audits.length).toBe(1);
+ vi.advanceTimersByTime(31_000);
+ const r2 = await fn(env);
+ expect(r2).toMatchObject({ recorded: true });
+ expect(audits.length).toBe(2);
+ expect(audits[1].details).toMatchObject({ dedupedCount: 1 });
+ vi.useRealTimers();
});
});
@@ -134,87 +141,95 @@ describe("mem::on-record-deleted", () => {
await fn({ event_type: "state:deleted", scope: "mem:obs:s1", key: "o9" });
expect(removedBm25).toContain("o9");
expect(removedVector).toContain("o9");
- expect(kv.update).toHaveBeenCalledWith(KV.sessions, "s1", [{ op: "set", path: "observationCount", value: 2 }]);
- expect(audits.find((a) => a.operation === "cascade")).toBeTruthy();
await fn({ scope: KV.lessons, key: "l1", event_type: "state:updated", new_value: { deleted: true }, old_value: { deleted: false } });
+ await fn({ event_type: "state:deleted", scope: KV.memories, key: "m1" });
+ const { flushDeleteCascade } = await import("../src/functions/cascade-audit.js");
+ await flushDeleteCascade();
+ expect(kv.update).toHaveBeenCalledWith(KV.sessions, "s1", [{ type: "set", path: "observationCount", value: 2 }]);
+ expect(audits.find((a) => a.operation === "cascade")).toBeTruthy();
expect(kv.store[KV.facets].has("f2")).toBe(false);
expect(kv.store[KV.facets].has("f3")).toBe(true);
- await fn({ event_type: "state:deleted", scope: KV.memories, key: "m1" });
expect(kv.store[KV.facets].has("f1")).toBe(false);
});
});
describe("event::graph::edge-reaction", () => {
- const base: Memory = { id: "tgt", createdAt: "t", updatedAt: "t", type: "fact", title: "x", content: "c", concepts: [], files: [], sessionIds: [], strength: 1, version: 1, isLatest: true, confidence: 0.8, relatedIds: [] } as unknown as Memory;
+ const base: Memory = { id: "tgt", createdAt: "t", updatedAt: "t", type: "fact", title: "x", content: "c", concepts: [], files: [], sessionIds: [], strength: 1, version: 1, isLatest: true, confidence: 0.8, relatedIds: ["pre"] } as unknown as Memory;
const rel: MemoryRelation = { type: "supersedes", sourceId: "src", targetId: "tgt", createdAt: "2026-01-01T00:00:00Z", confidence: 0.6 };
- function seeded(seedRel = true) {
+ function seeded(marks: Record = {}) {
const sdk = fakeFns();
const kv = memoryKv({
- [KV.memories]: new Map([["tgt", { ...base }]]),
- [KV.relations]: new Map(seedRel ? [["r1", { ...rel }]] : []),
+ [KV.memories]: new Map([["tgt", { ...base }], ["src", { ...base, id: "src" }]]),
+ [KV.relations]: new Map(),
+ [KV.relationReactions]: new Map(Object.entries(marks)),
});
registerCascadeFunctions(sdk as never, kv as never);
- return { sdk, kv, fn: sdk.functions.get("event::graph::edge-reaction")! };
+ return { kv, fn: sdk.functions.get("event::graph::edge-reaction")! };
}
+ const created = (key: string, r: Partial = {}) => ({ event_type: "state:created", scope: KV.relations, key, new_value: { ...rel, ...r } });
+ const deleted = (key: string, r: Partial = {}) => ({ event_type: "state:deleted", scope: KV.relations, key, old_value: { ...rel, ...r } });
+ const mem = (kv: ReturnType["kv"]) => kv.store[KV.memories].get("tgt") as Memory;
+ const mark = (kv: ReturnType["kv"], type = "supersedes") => kv.store[KV.relationReactions]?.get(`tgt:${type}`) as Record | undefined;
- it("supersedes: isLatest false + strength halved, mark stored", async () => {
+ it("two supersedes: second adds contributor only; delete restores originals when empty", async () => {
const { kv, fn } = seeded();
- await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
- const mem = kv.store[KV.memories].get("tgt") as Memory;
- expect(mem.isLatest).toBe(false);
- expect(mem.strength).toBeCloseTo(0.5);
- expect(kv.store[KV.relationReactions]?.get("r1")).toMatchObject({ type: "supersedes", isLatestBefore: true, strengthBefore: 1 });
+ await fn(created("rA"));
+ await fn(created("rB", { sourceId: "srcB" }));
+ expect(mem(kv).strength).toBeCloseTo(0.5);
+ const r1 = await fn(deleted("rA"));
+ expect(r1).toMatchObject({ unreacted: false, remaining: 1 });
+ expect(mem(kv).isLatest).toBe(false);
+ expect(mem(kv).strength).toBeCloseTo(0.5);
+ const r2 = await fn(deleted("rB", { sourceId: "srcB" }));
+ expect(r2).toMatchObject({ unreacted: true });
+ expect(mem(kv).isLatest).toBe(true);
+ expect(mem(kv).strength).toBeCloseTo(1);
+ expect(mark(kv)).toBeUndefined();
});
- it("contradicts: confidence floored reduction; extends: relatedIds union", async () => {
+ it("contradicts: cumulative floored decay, undo restores original, mutual contradicts allowed", async () => {
const { kv, fn } = seeded();
- await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel, type: "contradicts", confidence: 0.5 } });
- expect((kv.store[KV.memories].get("tgt") as Memory).confidence).toBeCloseTo(0.65);
- await fn({ event_type: "state:created", scope: KV.relations, key: "r2", new_value: { ...rel, type: "extends", sourceId: "src2" } });
- expect((kv.store[KV.memories].get("tgt") as Memory).relatedIds).toEqual(["src2"]);
+ await fn(created("rA", { type: "contradicts", confidence: 0.5 }));
+ await fn(created("rB", { type: "contradicts", sourceId: "srcB", confidence: 0.5 }));
+ expect(mem(kv).confidence).toBeCloseTo(0.5);
+ const cycle = await fn(created("rC", { type: "contradicts", sourceId: "tgt", targetId: "src", confidence: 0.5 }));
+ expect(cycle).toMatchObject({ reacted: true });
+ await fn(deleted("rA", { type: "contradicts" }));
+ await fn(deleted("rB", { type: "contradicts", sourceId: "srcB" }));
+ expect(mem(kv).confidence).toBeCloseTo(0.8);
});
- it("idempotent on redelivery", async () => {
+ it("supersedes cycle guard via reverse mark", async () => {
const { kv, fn } = seeded();
- await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
- const r2 = await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
- expect(r2).toMatchObject({ skipped: "already_applied" });
- expect((kv.store[KV.memories].get("tgt") as Memory).strength).toBeCloseTo(0.5);
- });
-
- it("delete un-reacts only when no same-type relation remains", async () => {
- const { kv, fn } = seeded();
- await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
- kv.store[KV.relations].delete("r1");
- const r = await fn({ event_type: "state:deleted", scope: KV.relations, key: "r1", old_value: { ...rel } });
- expect(r).toMatchObject({ unreacted: true });
- expect((kv.store[KV.memories].get("tgt") as Memory).isLatest).toBe(true);
- expect((kv.store[KV.memories].get("tgt") as Memory).strength).toBeCloseTo(1);
+ await fn(created("rA"));
+ const r = await fn(created("rB", { sourceId: "tgt", targetId: "src" }));
+ expect(r).toMatchObject({ skipped: "cycle" });
+ expect(audits.find((a) => (a.details as { skipped?: string }).skipped === "cycle")).toBeTruthy();
});
- it("delete keeps reaction when a same-type relation remains", async () => {
+ it("extends: undo removes only the added ids", async () => {
const { kv, fn } = seeded();
- kv.store[KV.relations].set("r2", { ...rel, sourceId: "other" });
- await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
- const r = await fn({ event_type: "state:deleted", scope: KV.relations, key: "r1", old_value: { ...rel } });
- expect(r).toMatchObject({ unreacted: false, remaining: true });
- expect((kv.store[KV.memories].get("tgt") as Memory).isLatest).toBe(false);
+ await fn(created("rA", { type: "extends" }));
+ await fn(created("rB", { type: "extends", sourceId: "srcB" }));
+ expect(mem(kv).relatedIds).toEqual(["pre", "src", "srcB"]);
+ await fn(deleted("rA", { type: "extends" }));
+ await fn(deleted("rB", { type: "extends", sourceId: "srcB" }));
+ expect(mem(kv).relatedIds).toEqual(["pre"]);
});
- it("cycle guard skips and audits", async () => {
+ it("replayed create applies once", async () => {
const { kv, fn } = seeded();
- kv.store[KV.relations].set("back", { type: "supersedes", sourceId: "tgt", targetId: "src", createdAt: "t" });
- const r = await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
- expect(r).toMatchObject({ skipped: "cycle" });
- expect((kv.store[KV.memories].get("tgt") as Memory).isLatest).toBe(true);
- expect(audits.find((a) => (a.details as { skipped?: string }).skipped === "cycle")).toBeTruthy();
+ await fn(created("rA"));
+ const r = await fn(created("rA"));
+ expect(r).toMatchObject({ skipped: "already_applied" });
+ expect(mem(kv).strength).toBeCloseTo(0.5);
});
- it("missing target skips and audits", async () => {
+ it("missing target audits and skips", async () => {
const { kv, fn } = seeded();
kv.store[KV.memories].delete("tgt");
- const r = await fn({ event_type: "state:created", scope: KV.relations, key: "r1", new_value: { ...rel } });
+ const r = await fn(created("rA"));
expect(r).toMatchObject({ skipped: "missing_target" });
});
});
From 59d13d58742db46a0bd8b2d6bed1dc86d576f0d1 Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Thu, 29 Oct 2026 20:29:22 +0530
Subject: [PATCH 3/8] fix(pipeline): keep delete flushes draining and relate
locks reentrant
---
src/functions/cascade-audit.ts | 58 +++++++++++++++--------
src/functions/relations.ts | 10 ++--
test/cascade-audit.test.ts | 84 +++++++++++++++++++++++++++++++++-
3 files changed, 128 insertions(+), 24 deletions(-)
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
index 32e185f1fb..2fdbb14ef0 100644
--- a/src/functions/cascade-audit.ts
+++ b/src/functions/cascade-audit.ts
@@ -8,6 +8,7 @@ import { recordAudit } from "./audit.js";
import { deleteFacetsCascade } from "./facets.js";
import { getSearchIndex, vectorIndexRemove } from "./search.js";
import { singleFlight } from "../runtime/recovery.js";
+import { logger } from "../logger.js";
const FAILURE_OPS: Record = {
"mem::compress": "compress-failed",
@@ -89,14 +90,21 @@ const pendingSessions = new Set();
let deleteTimer: ReturnType | null = null;
let deleteFlusher: (() => Promise) | null = null;
+function scheduleDeleteFlush(): void {
+ if (deleteTimer) return;
+ deleteTimer = setTimeout(() => { deleteTimer = null; void flushDeleteCascade().catch(() => reportFlushFailure()); }, DELETE_BATCH_MS);
+ deleteTimer.unref();
+}
+
+function reportFlushFailure(): void {
+ scheduleDeleteFlush();
+}
+
function queueDeleteCascade(id: string, sessionId?: string): void {
pendingDeletes.add(id);
if (sessionId) pendingSessions.add(sessionId);
- if (pendingDeletes.size >= DELETE_BATCH_MAX) void flushDeleteCascade();
- else if (!deleteTimer) {
- deleteTimer = setTimeout(() => { deleteTimer = null; void flushDeleteCascade(); }, DELETE_BATCH_MS);
- deleteTimer.unref();
- }
+ if (pendingDeletes.size >= DELETE_BATCH_MAX) void flushDeleteCascade().catch(() => reportFlushFailure());
+ else scheduleDeleteFlush();
}
export async function flushDeleteCascade(): Promise {
@@ -112,22 +120,34 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
pendingDeletes.clear();
pendingSessions.clear();
if (ids.length === 0 && sessions.length === 0) return;
- await deleteFacetsCascade(kv, ids);
- const reconciled: string[] = [];
- for (const sessionId of sessions) {
- const keys = await kv.listKeys(KV.observations(sessionId));
- const session = await kv.get(KV.sessions, sessionId);
- if (session && keys.length !== session.observationCount) {
- await kv.update(KV.sessions, sessionId, [
- { type: "set", path: "observationCount", value: keys.length },
- ]);
- reconciled.push(sessionId);
+ try {
+ await deleteFacetsCascade(kv, ids);
+ const reconciled: string[] = [];
+ for (const sessionId of sessions) {
+ const keys = await kv.listKeys(KV.observations(sessionId));
+ const session = await kv.get(KV.sessions, sessionId);
+ if (session && keys.length !== session.observationCount) {
+ await kv.update(KV.sessions, sessionId, [
+ { type: "set", path: "observationCount", value: keys.length },
+ ]);
+ reconciled.push(sessionId);
+ }
}
+ await recordAudit(kv, "cascade", "mem::on-record-deleted", ids.slice(0, 20), {
+ batched: ids.length,
+ sessionsReconciled: reconciled,
+ });
+ } catch (err) {
+ logger.warn("Delete cascade flush failed", {
+ ids: ids.length,
+ sessions: sessions.length,
+ error: err instanceof Error ? err.message : String(err),
+ });
+ for (const sessionId of sessions) pendingSessions.add(sessionId);
+ throw err;
+ } finally {
+ if (pendingDeletes.size > 0 || pendingSessions.size > 0) scheduleDeleteFlush();
}
- await recordAudit(kv, "cascade", "mem::on-record-deleted", ids.slice(0, 20), {
- batched: ids.length,
- sessionsReconciled: reconciled,
- });
});
sdk.registerFunction("mem::record-failure", async (envelope: Record) => {
const meta = envelope.metadata as { function_id?: string; source_fn?: string; observer_id?: string } | undefined;
diff --git a/src/functions/relations.ts b/src/functions/relations.ts
index e0750d78a1..770a7555c8 100644
--- a/src/functions/relations.ts
+++ b/src/functions/relations.ts
@@ -10,7 +10,7 @@ import {
} from "./search.js";
import { KV, generateId } from "../state/schema.js";
import { StateKV } from "../state/kv.js";
-import { withKeyedLock } from "../state/keyed-mutex.js";
+import { isKeyedLockHeld, withKeyedLock } from "../state/keyed-mutex.js";
import { safeAudit } from "./audit.js";
import { recordAccessBatch } from "./access-tracker.js";
import { logger } from "../logger.js";
@@ -65,6 +65,8 @@ export function registerRelationsFunction(sdk: IIIClient, kv: StateKV): void {
const lockKey =
firstId === secondId ? `mem:${firstId}` : `mem:${firstId}:${secondId}`;
+ const memoryLock = (id: string, fn: () => Promise): Promise =>
+ isKeyedLockHeld(`mem:${id}`) ? fn() : withKeyedLock(`mem:${id}`, fn);
return withKeyedLock(lockKey, async () => {
const source = await kv.get(KV.memories, data.sourceId);
const target = await kv.get(KV.memories, data.targetId);
@@ -92,7 +94,7 @@ export function registerRelationsFunction(sdk: IIIClient, kv: StateKV): void {
await kv.set(KV.relations, relationId, relation);
let sourceUpdated = false;
- await withKeyedLock(`mem:${data.sourceId}`, async () => {
+ await memoryLock(data.sourceId, async () => {
const fresh = await kv.get(KV.memories, data.sourceId);
if (fresh && !(fresh.relatedIds ?? []).includes(data.targetId)) {
fresh.relatedIds = [...(fresh.relatedIds ?? []), data.targetId];
@@ -102,14 +104,14 @@ export function registerRelationsFunction(sdk: IIIClient, kv: StateKV): void {
});
let targetUpdated = false;
- await withKeyedLock(`mem:${data.targetId}`, async () => {
+ await memoryLock(data.targetId, async () => {
const fresh = await kv.get(KV.memories, data.targetId);
if (fresh && !(fresh.relatedIds ?? []).includes(data.sourceId)) {
fresh.relatedIds = [...(fresh.relatedIds ?? []), data.sourceId];
await kv.set(KV.memories, data.targetId, fresh);
targetUpdated = true;
}
- })
+ });
await safeAudit(kv, "relation_create", "mem::relate", [relationId], {
type: data.type,
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
index 613959e789..4fcad9387d 100644
--- a/test/cascade-audit.test.ts
+++ b/test/cascade-audit.test.ts
@@ -1,5 +1,5 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
-import { registerCascadeFunctions, failureCounters } from "../src/functions/cascade-audit.js";
+import { registerCascadeFunctions, failureCounters, flushDeleteCascade } from "../src/functions/cascade-audit.js";
import { registerPipelineObservers } from "../src/runtime/pipeline-observers.js";
import { getBindings } from "../src/runtime/bindings.js";
import { KV } from "../src/state/schema.js";
@@ -15,6 +15,10 @@ vi.mock("../src/functions/audit.js", () => ({
audits.push({ operation, functionId, targetIds, details });
return { id: "aud_x" };
}),
+ safeAudit: vi.fn(async (_kv: unknown, operation: string, functionId: string, targetIds: string[], details: Record) => {
+ audits.push({ operation, functionId, targetIds, details });
+ return { id: "aud_x" };
+ }),
}));
const removedBm25: string[] = [];
@@ -233,3 +237,81 @@ describe("event::graph::edge-reaction", () => {
expect(r).toMatchObject({ skipped: "missing_target" });
});
});
+
+describe("mem::relate lock safety", () => {
+ it("resolves when sourceId === targetId and a follow-up relate resolves", async () => {
+ const { registerRelationsFunction } = await import("../src/functions/relations.js");
+ const sdk = fakeFns();
+ const kv = memoryKv({
+ [KV.memories]: new Map([["m1", { id: "m1", relatedIds: [], sessionIds: [], concepts: [], files: [] }]]),
+ [KV.relations]: new Map(),
+ });
+ registerRelationsFunction(sdk as never, kv as never);
+ const fn = sdk.functions.get("mem::relate")!;
+ const timed = (p: Promise) =>
+ Promise.race([p, new Promise((_, rj) => setTimeout(() => rj(new Error("deadlock")), 2000))]);
+ const r1 = await timed(fn({ sourceId: "m1", targetId: "m1", type: "extends" }));
+ expect(r1).toMatchObject({ success: true });
+ const r2 = await timed(fn({ sourceId: "m1", targetId: "m1", type: "contradicts" }));
+ expect(r2).toMatchObject({ success: true });
+ });
+});
+
+describe("delete cascade batcher resilience", () => {
+ function seededDelete() {
+ const sdk = fakeFns();
+ const kv = memoryKv({
+ [KV.sessions]: new Map([
+ ["s1", { id: "s1", observationCount: 5 }],
+ ["s2", { id: "s2", observationCount: 3 }],
+ ]),
+ [KV.observations("s1")]: new Map([["o1", {}], ["o2", {}]]),
+ [KV.observations("s2")]: new Map([["o3", {}]]),
+ [KV.facets]: new Map(),
+ });
+ registerCascadeFunctions(sdk as never, kv as never);
+ return { kv, fn: sdk.functions.get("mem::on-record-deleted")! };
+ }
+
+ it("a failed flush logs, requeues sessions, and reconciles on the next flush", async () => {
+ const { kv, fn } = seededDelete();
+ const rejections: unknown[] = [];
+ const onRej = (e: unknown) => { rejections.push(e); };
+ process.on("unhandledRejection", onRej);
+ try {
+ kv.listKeys.mockRejectedValueOnce(new Error("state worker restarting"));
+ await fn({ event_type: "state:deleted", scope: "mem:obs:s1", key: "o9" });
+ await flushDeleteCascade().catch(() => {});
+ expect(rejections).toEqual([]);
+ await flushDeleteCascade();
+ expect(kv.update).toHaveBeenCalledWith(KV.sessions, "s1", [
+ { type: "set", path: "observationCount", value: 2 },
+ ]);
+ } finally {
+ process.off("unhandledRejection", onRej);
+ }
+ });
+
+ it("items queued during an in-flight flush are drained by the next batch", async () => {
+ vi.useFakeTimers();
+ try {
+ const { kv, fn } = seededDelete();
+ let release!: () => void;
+ const gate = new Promise((r) => { release = r; });
+ kv.list.mockImplementationOnce(() => gate.then(() => []));
+ await fn({ event_type: "state:deleted", scope: "mem:obs:s1", key: "o1" });
+ await vi.advanceTimersByTimeAsync(600);
+ await fn({ event_type: "state:deleted", scope: "mem:obs:s2", key: "o3" });
+ release();
+ await vi.advanceTimersByTimeAsync(600);
+ await vi.advanceTimersByTimeAsync(600);
+ expect(kv.update).toHaveBeenCalledWith(KV.sessions, "s2", [
+ { type: "set", path: "observationCount", value: 1 },
+ ]);
+ const cascades = audits.filter((a) => a.operation === "cascade");
+ expect(cascades.length).toBeGreaterThanOrEqual(2);
+ } finally {
+ vi.useRealTimers();
+ }
+ });
+});
From cbe12a8d931159da0f381da49be91318fdac965a Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Thu, 29 Oct 2026 21:16:59 +0530
Subject: [PATCH 4/8] fix(pipeline): scope delete cascades by kind and make
relation reactions reversible
---
src/functions/cascade-audit.ts | 88 ++++++++++++++++++++++++++--------
src/functions/facets.ts | 14 ++----
src/types.ts | 1 +
test/cascade-audit.test.ts | 20 ++++++--
4 files changed, 89 insertions(+), 34 deletions(-)
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
index 2fdbb14ef0..c93248d03f 100644
--- a/src/functions/cascade-audit.ts
+++ b/src/functions/cascade-audit.ts
@@ -6,7 +6,8 @@ import { withKeyedLock } from "../state/keyed-mutex.js";
import type { AuditEntry, Memory, MemoryRelation, RelationReactionMark, Session } from "../types.js";
import { recordAudit } from "./audit.js";
import { deleteFacetsCascade } from "./facets.js";
-import { getSearchIndex, vectorIndexRemove } from "./search.js";
+import { getSearchIndex, scheduleIndexSave, vectorIndexAddGuarded, vectorIndexRemove } from "./search.js";
+import { memoryToObservation } from "../state/memory-utils.js";
import { singleFlight } from "../runtime/recovery.js";
import { logger } from "../logger.js";
@@ -85,7 +86,8 @@ function pruneDedupe(now: number): void {
const DELETE_BATCH_MS = 500;
const DELETE_BATCH_MAX = 200;
-const pendingDeletes = new Set();
+type PendingDelete = { id: string; kind: "memory" | "observation" | "lesson"; sessionId?: string };
+const pendingDeletes = new Map();
const pendingSessions = new Set();
let deleteTimer: ReturnType | null = null;
let deleteFlusher: (() => Promise) | null = null;
@@ -100,13 +102,27 @@ function reportFlushFailure(): void {
scheduleDeleteFlush();
}
-function queueDeleteCascade(id: string, sessionId?: string): void {
- pendingDeletes.add(id);
- if (sessionId) pendingSessions.add(sessionId);
+function queueDeleteCascade(item: PendingDelete): void {
+ pendingDeletes.set(`${item.kind}:${item.id}`, item);
+ if (item.sessionId) pendingSessions.add(item.sessionId);
if (pendingDeletes.size >= DELETE_BATCH_MAX) void flushDeleteCascade().catch(() => reportFlushFailure());
else scheduleDeleteFlush();
}
+async function deleteTargetGone(kv: StateKV, item: PendingDelete): Promise {
+ if (item.kind === "memory") return (await kv.get(KV.memories, item.id)) === null;
+ if (item.kind === "observation" && item.sessionId) {
+ return (await kv.get(KV.observations(item.sessionId), item.id)) === null;
+ }
+ if (item.kind === "lesson") {
+ const lesson = await kv.get<{ deleted?: boolean }>(KV.lessons, item.id);
+ return lesson === null || lesson.deleted === true;
+ }
+ return true;
+}
+
+const KIND_TARGET_TYPE: Record = { memory: "memory", observation: "observation", lesson: "memory" };
+
export async function flushDeleteCascade(): Promise {
if (!deleteFlusher) return;
await deleteFlusher();
@@ -115,34 +131,44 @@ export async function flushDeleteCascade(): Promise {
export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
deleteFlusher = singleFlight(async () => {
if (deleteTimer) { clearTimeout(deleteTimer); deleteTimer = null; }
- const ids = [...pendingDeletes];
+ const items = [...pendingDeletes.values()];
const sessions = [...pendingSessions];
pendingDeletes.clear();
pendingSessions.clear();
- if (ids.length === 0 && sessions.length === 0) return;
+ if (items.length === 0 && sessions.length === 0) return;
try {
- await deleteFacetsCascade(kv, ids);
+ const gone = (await Promise.all(items.map((item) => deleteTargetGone(kv, item).catch(() => false))))
+ .map((ok, i) => (ok ? items[i]! : null))
+ .filter((item): item is PendingDelete => item !== null);
+ await Promise.all(gone.map(async (item) => {
+ const targetType = KIND_TARGET_TYPE[item.kind];
+ if (!targetType) return;
+ await deleteFacetsCascade(kv, [item.id], targetType);
+ }));
const reconciled: string[] = [];
for (const sessionId of sessions) {
- const keys = await kv.listKeys(KV.observations(sessionId));
- const session = await kv.get(KV.sessions, sessionId);
- if (session && keys.length !== session.observationCount) {
- await kv.update(KV.sessions, sessionId, [
- { type: "set", path: "observationCount", value: keys.length },
- ]);
- reconciled.push(sessionId);
- }
+ await withKeyedLock(`obs:${sessionId}`, async () => {
+ const keys = await kv.listKeys(KV.observations(sessionId));
+ const session = await kv.get(KV.sessions, sessionId);
+ if (session && keys.length !== session.observationCount) {
+ await kv.update(KV.sessions, sessionId, [
+ { type: "set", path: "observationCount", value: keys.length },
+ ]);
+ reconciled.push(sessionId);
+ }
+ });
}
- await recordAudit(kv, "cascade", "mem::on-record-deleted", ids.slice(0, 20), {
- batched: ids.length,
+ await recordAudit(kv, "cascade", "mem::on-record-deleted", items.map((item) => item.id).slice(0, 20), {
+ batched: items.length,
sessionsReconciled: reconciled,
});
} catch (err) {
logger.warn("Delete cascade flush failed", {
- ids: ids.length,
+ ids: items.length,
sessions: sessions.length,
error: err instanceof Error ? err.message : String(err),
});
+ for (const item of items) pendingDeletes.set(`${item.kind}:${item.id}`, item);
for (const sessionId of sessions) pendingSessions.add(sessionId);
throw err;
} finally {
@@ -188,7 +214,8 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
try { getSearchIndex().remove(id); } catch {}
try { vectorIndexRemove(id); } catch {}
const sessionId = scope.startsWith("mem:obs:") ? scope.slice("mem:obs:".length) : undefined;
- queueDeleteCascade(id, sessionId);
+ const kind = sessionId ? "observation" : scope === KV.lessons ? "lesson" : "memory";
+ queueDeleteCascade({ id, kind, sessionId });
return Promise.resolve({ removed: id, queued: true });
});
@@ -233,12 +260,18 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
if (!mark) {
next.isLatestBefore = target.isLatest;
next.strengthBefore = target.strength;
+ const wasLatest = target.isLatest === true;
target.isLatest = false;
target.strength = target.strength * 0.5;
+ if (wasLatest) {
+ try { getSearchIndex().remove(rel.targetId); } catch {}
+ try { vectorIndexRemove(rel.targetId); } catch {}
+ }
}
} else if (rel.type === "contradicts") {
- if (!mark) next.confidenceBefore = target.confidence;
+ const before = target.confidence;
target.confidence = Math.max(0.05, target.confidence - (rel.confidence ?? 0.5) * 0.3);
+ next.contradictDeltas = { ...(next.contradictDeltas ?? {}), [relationId]: before - target.confidence };
} else if (rel.type === "extends") {
const related = target.relatedIds ?? [];
if (!related.includes(rel.sourceId)) {
@@ -256,13 +289,26 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
return { skipped: "no_mark" as const };
}
delete mark.sources[relationId];
+ if (rel.type === "contradicts" && mark.contradictDeltas) {
+ const delta = mark.contradictDeltas[relationId];
+ if (delta !== undefined) {
+ target.confidence = Math.min(1, target.confidence + delta);
+ delete mark.contradictDeltas[relationId];
+ }
+ }
if (Object.keys(mark.sources).length > 0) {
+ await kv.set(KV.memories, rel.targetId, target);
await kv.set(KV.relationReactions, markKey, mark);
return { unreacted: false as const, remaining: Object.keys(mark.sources).length };
}
+ const restoredLatest = mark.isLatestBefore !== undefined && target.isLatest !== true && mark.isLatestBefore === true;
if (mark.isLatestBefore !== undefined) target.isLatest = mark.isLatestBefore;
if (mark.strengthBefore !== undefined) target.strength = mark.strengthBefore;
if (mark.confidenceBefore !== undefined) target.confidence = mark.confidenceBefore;
+ if (restoredLatest) {
+ try { getSearchIndex().add(memoryToObservation(target), "memory"); scheduleIndexSave(); } catch {}
+ try { void vectorIndexAddGuarded(rel.targetId, memoryToObservation(target)); } catch {}
+ }
if (mark.relatedIdsAdded?.length) {
const added = new Set(mark.relatedIdsAdded);
target.relatedIds = (target.relatedIds ?? []).filter((id) => !added.has(id));
diff --git a/src/functions/facets.ts b/src/functions/facets.ts
index f5120d870d..e72543985b 100644
--- a/src/functions/facets.ts
+++ b/src/functions/facets.ts
@@ -53,20 +53,14 @@ export async function deleteFacetsForTargets(
export async function deleteFacetsCascade(
kv: StateKV,
targetIds: Iterable,
+ targetType?: string,
): Promise {
const ids = new Set(targetIds);
ids.delete("");
if (ids.size === 0) return;
- try {
- const all = await kv.list(KV.facets);
- const matches = all.filter((f) => ids.has(f.targetId));
- await Promise.all(matches.map((f) => kv.delete(KV.facets, f.id)));
- } catch (err) {
- logger.warn("Facet cascade cleanup failed", {
- targetIds: ids.size,
- error: err instanceof Error ? err.message : String(err),
- });
- }
+ const all = await kv.list(KV.facets);
+ const matches = all.filter((f) => ids.has(f.targetId) && (targetType === undefined || f.targetType === targetType));
+ await Promise.all(matches.map((f) => kv.delete(KV.facets, f.id)));
}
export function registerFacetsFunction(sdk: IIIClient, kv: StateKV): void {
diff --git a/src/types.ts b/src/types.ts
index 0457792063..b9ccb54f65 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -5,6 +5,7 @@ export interface RelationReactionMark {
isLatestBefore?: boolean;
strengthBefore?: number;
confidenceBefore?: number;
+ contradictDeltas?: Record;
relatedIdsAdded?: string[];
firstAppliedAt: string;
lastAppliedAt: string;
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
index 4fcad9387d..9d7b6c5c8d 100644
--- a/test/cascade-audit.test.ts
+++ b/test/cascade-audit.test.ts
@@ -142,10 +142,13 @@ describe("mem::on-record-deleted", () => {
});
registerCascadeFunctions(sdk as never, kv as never);
const fn = sdk.functions.get("mem::on-record-deleted")!;
+ kv.store[KV.observations("s1")]?.delete("o9");
await fn({ event_type: "state:deleted", scope: "mem:obs:s1", key: "o9" });
expect(removedBm25).toContain("o9");
expect(removedVector).toContain("o9");
+ kv.store[KV.lessons]?.set("l1", { deleted: true });
await fn({ scope: KV.lessons, key: "l1", event_type: "state:updated", new_value: { deleted: true }, old_value: { deleted: false } });
+ kv.store[KV.memories]?.delete("m1");
await fn({ event_type: "state:deleted", scope: KV.memories, key: "m1" });
const { flushDeleteCascade } = await import("../src/functions/cascade-audit.js");
await flushDeleteCascade();
@@ -192,18 +195,29 @@ describe("event::graph::edge-reaction", () => {
expect(mark(kv)).toBeUndefined();
});
- it("contradicts: cumulative floored decay, undo restores original, mutual contradicts allowed", async () => {
+ it("contradicts: cumulative floored decay, partial undo adds back per-relation delta", async () => {
const { kv, fn } = seeded();
await fn(created("rA", { type: "contradicts", confidence: 0.5 }));
await fn(created("rB", { type: "contradicts", sourceId: "srcB", confidence: 0.5 }));
expect(mem(kv).confidence).toBeCloseTo(0.5);
- const cycle = await fn(created("rC", { type: "contradicts", sourceId: "tgt", targetId: "src", confidence: 0.5 }));
- expect(cycle).toMatchObject({ reacted: true });
+ const mutual = await fn(created("rC", { type: "contradicts", sourceId: "tgt", targetId: "src", confidence: 0.5 }));
+ expect(mutual).toMatchObject({ reacted: true });
await fn(deleted("rA", { type: "contradicts" }));
+ expect(mem(kv).confidence).toBeCloseTo(0.65);
await fn(deleted("rB", { type: "contradicts", sourceId: "srcB" }));
expect(mem(kv).confidence).toBeCloseTo(0.8);
});
+ it("supersedes removes the target from search indexes and the final undo re-adds it", async () => {
+ const { kv, fn } = seeded();
+ removedBm25.length = 0; removedVector.length = 0;
+ await fn(created("rA"));
+ expect(removedBm25).toContain("tgt");
+ expect(removedVector).toContain("tgt");
+ await fn(deleted("rA"));
+ expect(mem(kv).isLatest).toBe(true);
+ });
+
it("supersedes cycle guard via reverse mark", async () => {
const { kv, fn } = seeded();
await fn(created("rA"));
From 74e889e67a422f304e7c85772fefa83177b27eec Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Sun, 8 Nov 2026 13:30:56 +0530
Subject: [PATCH 5/8] fix(cascade): batch facet scans, retry failed reads and
keep the contradiction floor
- Delete facets for a flushed batch with one scan per target type
instead of one full facet scan per deleted record.
- Requeue a delete whose target read failed, so its facets are cleaned
up on the next flush instead of being dropped.
- Track each contradiction's raw penalty against a base confidence and
apply the change in the floored value, so undoing one contradiction
does not lift confidence while another still holds it at the floor.
- Cover search and vector restoration on the final supersedes undo.
---
src/functions/cascade-audit.ts | 46 ++++++++++++++++++++++------------
src/types.ts | 3 ++-
test/cascade-audit.test.ts | 40 ++++++++++++++++++++++++++++-
3 files changed, 71 insertions(+), 18 deletions(-)
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
index c93248d03f..4b4288f9f1 100644
--- a/src/functions/cascade-audit.ts
+++ b/src/functions/cascade-audit.ts
@@ -121,6 +121,10 @@ async function deleteTargetGone(kv: StateKV, item: PendingDelete): Promise): number {
+ return Math.max(0.05, base - Object.values(penalties).reduce((sum, p) => sum + p, 0));
+}
+
const KIND_TARGET_TYPE: Record = { memory: "memory", observation: "observation", lesson: "memory" };
export async function flushDeleteCascade(): Promise {
@@ -137,14 +141,21 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
pendingSessions.clear();
if (items.length === 0 && sessions.length === 0) return;
try {
- const gone = (await Promise.all(items.map((item) => deleteTargetGone(kv, item).catch(() => false))))
- .map((ok, i) => (ok ? items[i]! : null))
- .filter((item): item is PendingDelete => item !== null);
- await Promise.all(gone.map(async (item) => {
+ const checks = await Promise.all(items.map((item) => deleteTargetGone(kv, item).then(
+ (isGone) => (isGone ? "gone" : "live"),
+ () => "retry",
+ )));
+ const byType = new Map();
+ items.forEach((item, i) => {
+ if (checks[i] === "retry") {
+ pendingDeletes.set(`${item.kind}:${item.id}`, item);
+ return;
+ }
const targetType = KIND_TARGET_TYPE[item.kind];
- if (!targetType) return;
- await deleteFacetsCascade(kv, [item.id], targetType);
- }));
+ if (checks[i] !== "gone" || !targetType) return;
+ byType.set(targetType, [...(byType.get(targetType) ?? []), item.id]);
+ });
+ for (const [targetType, ids] of byType) await deleteFacetsCascade(kv, ids, targetType);
const reconciled: string[] = [];
for (const sessionId of sessions) {
await withKeyedLock(`obs:${sessionId}`, async () => {
@@ -269,9 +280,13 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
}
}
} else if (rel.type === "contradicts") {
- const before = target.confidence;
- target.confidence = Math.max(0.05, target.confidence - (rel.confidence ?? 0.5) * 0.3);
- next.contradictDeltas = { ...(next.contradictDeltas ?? {}), [relationId]: before - target.confidence };
+ const base = next.contradictBase ?? target.confidence;
+ const penalties = next.contradictPenalties ?? {};
+ const before = contradictFloor(base, penalties);
+ const updated = { ...penalties, [relationId]: (rel.confidence ?? 0.5) * 0.3 };
+ target.confidence = Math.max(0.05, target.confidence + contradictFloor(base, updated) - before);
+ next.contradictBase = base;
+ next.contradictPenalties = updated;
} else if (rel.type === "extends") {
const related = target.relatedIds ?? [];
if (!related.includes(rel.sourceId)) {
@@ -289,12 +304,11 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
return { skipped: "no_mark" as const };
}
delete mark.sources[relationId];
- if (rel.type === "contradicts" && mark.contradictDeltas) {
- const delta = mark.contradictDeltas[relationId];
- if (delta !== undefined) {
- target.confidence = Math.min(1, target.confidence + delta);
- delete mark.contradictDeltas[relationId];
- }
+ if (rel.type === "contradicts" && mark.contradictPenalties && mark.contradictBase !== undefined) {
+ const before = contradictFloor(mark.contradictBase, mark.contradictPenalties);
+ delete mark.contradictPenalties[relationId];
+ const after = contradictFloor(mark.contradictBase, mark.contradictPenalties);
+ target.confidence = Math.min(1, target.confidence + after - before);
}
if (Object.keys(mark.sources).length > 0) {
await kv.set(KV.memories, rel.targetId, target);
diff --git a/src/types.ts b/src/types.ts
index 8237e58c88..264dac0b37 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -5,7 +5,8 @@ export interface RelationReactionMark {
isLatestBefore?: boolean;
strengthBefore?: number;
confidenceBefore?: number;
- contradictDeltas?: Record;
+ contradictBase?: number;
+ contradictPenalties?: Record;
relatedIdsAdded?: string[];
firstAppliedAt: string;
lastAppliedAt: string;
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
index 9d7b6c5c8d..48eed69c00 100644
--- a/test/cascade-audit.test.ts
+++ b/test/cascade-audit.test.ts
@@ -23,10 +23,14 @@ vi.mock("../src/functions/audit.js", () => ({
const removedBm25: string[] = [];
const removedVector: string[] = [];
+const addedBm25: string[] = [];
+const addedVector: string[] = [];
vi.mock("../src/functions/search.js", () => ({
- getSearchIndex: () => ({ remove: (id: string) => removedBm25.push(id) }),
+ getSearchIndex: () => ({ remove: (id: string) => removedBm25.push(id), add: (obs: { id: string }) => addedBm25.push(obs.id) }),
getVectorIndex: () => ({ size: 0 }),
vectorIndexRemove: (id: string) => { removedVector.push(id); },
+ vectorIndexAddGuarded: async (id: string) => { addedVector.push(id); },
+ scheduleIndexSave: () => {},
isMemoryIndexReady: () => true,
}));
@@ -214,8 +218,23 @@ describe("event::graph::edge-reaction", () => {
await fn(created("rA"));
expect(removedBm25).toContain("tgt");
expect(removedVector).toContain("tgt");
+ addedBm25.length = 0; addedVector.length = 0;
await fn(deleted("rA"));
expect(mem(kv).isLatest).toBe(true);
+ expect(addedBm25).toContain("tgt");
+ expect(addedVector).toContain("tgt");
+ });
+
+ it("contradicts: undoing one relation keeps the floor a remaining relation still holds", async () => {
+ const { kv, fn } = seeded();
+ kv.store[KV.memories].set("tgt", { ...base, confidence: 0.2 });
+ await fn(created("rA", { type: "contradicts", confidence: 0.5 }));
+ await fn(created("rB", { type: "contradicts", sourceId: "srcB", confidence: 0.5 }));
+ expect(mem(kv).confidence).toBeCloseTo(0.05);
+ await fn(deleted("rA", { type: "contradicts" }));
+ expect(mem(kv).confidence).toBeCloseTo(0.05);
+ await fn(deleted("rB", { type: "contradicts", sourceId: "srcB" }));
+ expect(mem(kv).confidence).toBeCloseTo(0.2);
});
it("supersedes cycle guard via reverse mark", async () => {
@@ -287,6 +306,25 @@ describe("delete cascade batcher resilience", () => {
return { kv, fn: sdk.functions.get("mem::on-record-deleted")! };
}
+ it("scans the facet scope once per target type for a batch of deletes", async () => {
+ const { kv, fn } = seededDelete();
+ for (const id of ["m1", "m2", "m3"]) await fn({ event_type: "state:deleted", scope: KV.memories, key: id });
+ kv.list.mockClear();
+ await flushDeleteCascade();
+ expect(kv.list.mock.calls.filter((c) => c[0] === KV.facets)).toHaveLength(1);
+ });
+
+ it("retries facet cleanup for a target whose read failed", async () => {
+ const { kv, fn } = seededDelete();
+ kv.store[KV.facets].set("f1", { id: "f1", targetId: "m1", targetType: "memory", dimension: "d", value: "v", createdAt: "t" });
+ kv.get.mockRejectedValueOnce(new Error("state worker restarting"));
+ await fn({ event_type: "state:deleted", scope: KV.memories, key: "m1" });
+ await flushDeleteCascade();
+ expect(kv.store[KV.facets].has("f1")).toBe(true);
+ await flushDeleteCascade();
+ expect(kv.store[KV.facets].has("f1")).toBe(false);
+ });
+
it("a failed flush logs, requeues sessions, and reconciles on the next flush", async () => {
const { kv, fn } = seededDelete();
const rejections: unknown[] = [];
From cb446358494b8cdd1646ba3757458cb9d311461b Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Sun, 8 Nov 2026 14:04:49 +0530
Subject: [PATCH 6/8] fix(cascade): restore vectors with the right call and
keep confidence finite
- Re-add an un-superseded memory to the vector index with its session,
embed text and memory context; the previous two-argument call never
embedded it.
- Treat a memory without a stored confidence as 1 in the contradicts
reaction, so the stored value never becomes NaN, and type the field.
- Rebuild the keyword index after a replace import from the store passed
to the handler; the previous closure referenced an undefined name.
---
src/functions/cascade-audit.ts | 16 +++++----
src/functions/export-import.ts | 2 +-
src/types.ts | 1 +
test/cascade-audit.test.ts | 19 +++++++++-
test/import-replace-rebuild.test.ts | 56 +++++++++++++++++++++++++++++
5 files changed, 86 insertions(+), 8 deletions(-)
create mode 100644 test/import-replace-rebuild.test.ts
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
index 4b4288f9f1..7fce861202 100644
--- a/src/functions/cascade-audit.ts
+++ b/src/functions/cascade-audit.ts
@@ -1,13 +1,12 @@
import type { IIIClient } from "iii-sdk";
import type { StateKV } from "../state/kv.js";
-import { collectAll } from "../state/kv.js";
import { KV } from "../state/schema.js";
import { withKeyedLock } from "../state/keyed-mutex.js";
import type { AuditEntry, Memory, MemoryRelation, RelationReactionMark, Session } from "../types.js";
import { recordAudit } from "./audit.js";
import { deleteFacetsCascade } from "./facets.js";
import { getSearchIndex, scheduleIndexSave, vectorIndexAddGuarded, vectorIndexRemove } from "./search.js";
-import { memoryToObservation } from "../state/memory-utils.js";
+import { memoryToObservation, observationEmbedText } from "../state/memory-utils.js";
import { singleFlight } from "../runtime/recovery.js";
import { logger } from "../logger.js";
@@ -280,11 +279,11 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
}
}
} else if (rel.type === "contradicts") {
- const base = next.contradictBase ?? target.confidence;
+ const base = next.contradictBase ?? target.confidence ?? 1;
const penalties = next.contradictPenalties ?? {};
const before = contradictFloor(base, penalties);
const updated = { ...penalties, [relationId]: (rel.confidence ?? 0.5) * 0.3 };
- target.confidence = Math.max(0.05, target.confidence + contradictFloor(base, updated) - before);
+ target.confidence = Math.max(0.05, (target.confidence ?? base) + contradictFloor(base, updated) - before);
next.contradictBase = base;
next.contradictPenalties = updated;
} else if (rel.type === "extends") {
@@ -308,7 +307,7 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
const before = contradictFloor(mark.contradictBase, mark.contradictPenalties);
delete mark.contradictPenalties[relationId];
const after = contradictFloor(mark.contradictBase, mark.contradictPenalties);
- target.confidence = Math.min(1, target.confidence + after - before);
+ target.confidence = Math.min(1, (target.confidence ?? mark.contradictBase) + after - before);
}
if (Object.keys(mark.sources).length > 0) {
await kv.set(KV.memories, rel.targetId, target);
@@ -321,7 +320,12 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
if (mark.confidenceBefore !== undefined) target.confidence = mark.confidenceBefore;
if (restoredLatest) {
try { getSearchIndex().add(memoryToObservation(target), "memory"); scheduleIndexSave(); } catch {}
- try { void vectorIndexAddGuarded(rel.targetId, memoryToObservation(target)); } catch {}
+ void vectorIndexAddGuarded(
+ rel.targetId,
+ target.sessionIds?.[0] ?? "memory",
+ observationEmbedText({ title: target.title, narrative: target.content, concepts: target.concepts }),
+ { kind: "memory", logId: rel.targetId },
+ ).catch(() => {});
}
if (mark.relatedIdsAdded?.length) {
const added = new Set(mark.relatedIdsAdded);
diff --git a/src/functions/export-import.ts b/src/functions/export-import.ts
index ab910a0571..e8beae1818 100644
--- a/src/functions/export-import.ts
+++ b/src/functions/export-import.ts
@@ -291,7 +291,7 @@ export function registerExportImportFunction(sdk: IIIClient, state: StateKV): vo
);
const rebuildAfterReplace = singleFlight(async () => {
- await rebuildKeywordIndex(kv);
+ await rebuildKeywordIndex(state);
});
sdk.registerFunction("mem::import",
async (data: {
diff --git a/src/types.ts b/src/types.ts
index 264dac0b37..66b4fa9cf9 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -167,6 +167,7 @@ export interface Memory {
files: string[];
sessionIds: string[];
strength: number;
+ confidence?: number;
version: number;
recurrence?: number;
parentId?: string;
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
index 48eed69c00..0ba4f0704d 100644
--- a/test/cascade-audit.test.ts
+++ b/test/cascade-audit.test.ts
@@ -25,11 +25,12 @@ const removedBm25: string[] = [];
const removedVector: string[] = [];
const addedBm25: string[] = [];
const addedVector: string[] = [];
+const addedVectorCalls: unknown[][] = [];
vi.mock("../src/functions/search.js", () => ({
getSearchIndex: () => ({ remove: (id: string) => removedBm25.push(id), add: (obs: { id: string }) => addedBm25.push(obs.id) }),
getVectorIndex: () => ({ size: 0 }),
vectorIndexRemove: (id: string) => { removedVector.push(id); },
- vectorIndexAddGuarded: async (id: string) => { addedVector.push(id); },
+ vectorIndexAddGuarded: async (...args: unknown[]) => { addedVector.push(args[0] as string); addedVectorCalls.push(args); return true; },
scheduleIndexSave: () => {},
isMemoryIndexReady: () => true,
}));
@@ -223,6 +224,22 @@ describe("event::graph::edge-reaction", () => {
expect(mem(kv).isLatest).toBe(true);
expect(addedBm25).toContain("tgt");
expect(addedVector).toContain("tgt");
+ const call = addedVectorCalls.find((args) => args[0] === "tgt")!;
+ expect(call).toHaveLength(4);
+ expect(typeof call[1]).toBe("string");
+ expect(String(call[2])).toContain("c");
+ expect(call[3]).toEqual({ kind: "memory", logId: "tgt" });
+ });
+
+ it("contradicts on a memory without a stored confidence starts from 1 and stays finite", async () => {
+ const { kv, fn } = seeded();
+ const plain = { ...base } as Record;
+ delete plain.confidence;
+ kv.store[KV.memories].set("tgt", plain);
+ await fn(created("rA", { type: "contradicts", confidence: 0.5 }));
+ expect(mem(kv).confidence).toBeCloseTo(0.85);
+ await fn(deleted("rA", { type: "contradicts" }));
+ expect(Number.isFinite(mem(kv).confidence)).toBe(true);
});
it("contradicts: undoing one relation keeps the floor a remaining relation still holds", async () => {
diff --git a/test/import-replace-rebuild.test.ts b/test/import-replace-rebuild.test.ts
new file mode 100644
index 0000000000..6d317358ed
--- /dev/null
+++ b/test/import-replace-rebuild.test.ts
@@ -0,0 +1,56 @@
+import { describe, it, expect, vi } from "vitest";
+
+const { warn, rebuilt } = vi.hoisted(() => ({ warn: vi.fn(), rebuilt: [] as unknown[] }));
+vi.mock("../src/logger.js", () => ({
+ logger: { info: vi.fn(), warn, error: vi.fn() },
+}));
+
+vi.mock("../src/functions/search.js", async (importOriginal) => {
+ const actual = await importOriginal();
+ return { ...actual, rebuildKeywordIndex: vi.fn(async (kv: unknown) => { rebuilt.push(kv); return {}; }) };
+});
+
+import { registerExportImportFunction } from "../src/functions/export-import.js";
+import type { ExportData } from "../src/types.js";
+
+function mockKV() {
+ const store = new Map>();
+ return {
+ get: async (scope: string, key: string) => store.get(scope)?.get(key) ?? null,
+ set: async (scope: string, key: string, data: unknown) => {
+ if (!store.has(scope)) store.set(scope, new Map());
+ store.get(scope)!.set(key, data);
+ return data;
+ },
+ delete: async (scope: string, key: string) => { store.get(scope)?.delete(key); },
+ list: async (scope: string) => Array.from(store.get(scope)?.values() ?? []),
+ };
+}
+
+describe("replace import keyword rebuild", () => {
+ it("rebuilds the keyword index from the imported store after a replace import", async () => {
+ const functions = new Map Promise>();
+ const sdk = {
+ registerFunction: (id: string | { id: string }, handler: (payload: unknown) => Promise) => {
+ functions.set(typeof id === "string" ? id : id.id, handler);
+ },
+ registerTrigger: () => {},
+ trigger: async (input: { function_id: string; payload: unknown }) => functions.get(input.function_id)?.(input.payload),
+ };
+ const kv = mockKV();
+ registerExportImportFunction(sdk as never, kv as never);
+ const exportData: ExportData = {
+ version: "0.3.0",
+ exportedAt: new Date().toISOString(),
+ sessions: [{ id: "ses_new", project: "p", cwd: "/tmp/p", startedAt: "2026-03-01T00:00:00Z", status: "active", observationCount: 0 }],
+ observations: {},
+ memories: [],
+ summaries: [],
+ };
+ const result = (await functions.get("mem::import")!({ exportData, strategy: "replace" })) as { success: boolean };
+ expect(result.success).toBe(true);
+ await vi.waitFor(() => expect(rebuilt).toHaveLength(1));
+ expect(rebuilt[0]).toBe(kv);
+ expect(warn).not.toHaveBeenCalledWith("Post-replace index rebuild failed", expect.anything());
+ });
+});
From e1326a30ab5af6761c09ab21eefabfa4316ce41a Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Sun, 8 Nov 2026 14:20:24 +0530
Subject: [PATCH 7/8] fix(cascade): start an unset memory confidence at the
default salience
A contradiction on a memory with no stored confidence started from 1, so
the stored value could rise above the type salience and promote a
contradicted fact in retention. Start from 0.5, the default salience.
---
src/functions/cascade-audit.ts | 4 +++-
test/cascade-audit.test.ts | 7 ++++---
2 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/src/functions/cascade-audit.ts b/src/functions/cascade-audit.ts
index 7fce861202..69cb4bc280 100644
--- a/src/functions/cascade-audit.ts
+++ b/src/functions/cascade-audit.ts
@@ -120,6 +120,8 @@ async function deleteTargetGone(kv: StateKV, item: PendingDelete): Promise): number {
return Math.max(0.05, base - Object.values(penalties).reduce((sum, p) => sum + p, 0));
}
@@ -279,7 +281,7 @@ export function registerCascadeFunctions(sdk: IIIClient, kv: StateKV): void {
}
}
} else if (rel.type === "contradicts") {
- const base = next.contradictBase ?? target.confidence ?? 1;
+ const base = next.contradictBase ?? target.confidence ?? UNSET_MEMORY_CONFIDENCE;
const penalties = next.contradictPenalties ?? {};
const before = contradictFloor(base, penalties);
const updated = { ...penalties, [relationId]: (rel.confidence ?? 0.5) * 0.3 };
diff --git a/test/cascade-audit.test.ts b/test/cascade-audit.test.ts
index 0ba4f0704d..e701c3f548 100644
--- a/test/cascade-audit.test.ts
+++ b/test/cascade-audit.test.ts
@@ -231,15 +231,16 @@ describe("event::graph::edge-reaction", () => {
expect(call[3]).toEqual({ kind: "memory", logId: "tgt" });
});
- it("contradicts on a memory without a stored confidence starts from 1 and stays finite", async () => {
+ it("contradicts on a memory without a stored confidence starts from the default salience and stays finite", async () => {
const { kv, fn } = seeded();
const plain = { ...base } as Record;
delete plain.confidence;
kv.store[KV.memories].set("tgt", plain);
await fn(created("rA", { type: "contradicts", confidence: 0.5 }));
- expect(mem(kv).confidence).toBeCloseTo(0.85);
+ expect(mem(kv).confidence).toBeCloseTo(0.35);
+ expect(mem(kv).confidence!).toBeLessThanOrEqual(0.5);
await fn(deleted("rA", { type: "contradicts" }));
- expect(Number.isFinite(mem(kv).confidence)).toBe(true);
+ expect(mem(kv).confidence).toBeCloseTo(0.5);
});
it("contradicts: undoing one relation keeps the floor a remaining relation still holds", async () => {
From 574f57fcd8714038e05350b86c0bdade7d75f7b4 Mon Sep 17 00:00:00 2001
From: Rohit Ghumare
Date: Sun, 8 Nov 2026 21:24:58 +0530
Subject: [PATCH 8/8] docs: sync documented counts after merging main
---
AGENTS.md | 4 ++--
README.md | 4 ++--
READMEs/README.ar-SA.md | 4 ++--
READMEs/README.bn-BD.md | 4 ++--
READMEs/README.cs-CZ.md | 4 ++--
READMEs/README.da-DK.md | 4 ++--
READMEs/README.de-DE.md | 4 ++--
READMEs/README.el-GR.md | 4 ++--
READMEs/README.es-ES.md | 4 ++--
READMEs/README.fi-FI.md | 4 ++--
READMEs/README.fr-FR.md | 4 ++--
READMEs/README.he-IL.md | 4 ++--
READMEs/README.hi-IN.md | 4 ++--
READMEs/README.hu-HU.md | 4 ++--
READMEs/README.id-ID.md | 4 ++--
READMEs/README.it-IT.md | 4 ++--
READMEs/README.ja-JP.md | 4 ++--
READMEs/README.ko-KR.md | 4 ++--
READMEs/README.nb-NO.md | 4 ++--
READMEs/README.nl-NL.md | 4 ++--
READMEs/README.pl-PL.md | 4 ++--
READMEs/README.pt-BR.md | 4 ++--
READMEs/README.pt-PT.md | 4 ++--
READMEs/README.ro-RO.md | 4 ++--
READMEs/README.ru-RU.md | 4 ++--
READMEs/README.sv-SE.md | 4 ++--
READMEs/README.th-TH.md | 4 ++--
READMEs/README.tl-PH.md | 4 ++--
READMEs/README.tr-TR.md | 4 ++--
READMEs/README.uk-UA.md | 4 ++--
READMEs/README.ur-PK.md | 4 ++--
READMEs/README.vi-VN.md | 4 ++--
READMEs/README.zh-CN.md | 4 ++--
READMEs/README.zh-TW.md | 4 ++--
assets/tags/light/stat-tests.svg | 4 ++--
assets/tags/stat-tests.svg | 4 ++--
scripts/docs-sync.state.json | 6 +++---
37 files changed, 75 insertions(+), 75 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index eb8682c450..89391501e6 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -105,7 +105,7 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).
## Testing
-- Before every commit run, in order: `npm run build`, `npm run skills:gen`, `npm run docs:sync`, `npm run skills:check`, `npm test` (4,600+ tests). Commit anything the generators changed. CI runs the same build, skills check and tests.
+- Before every commit run, in order: `npm run build`, `npm run skills:gen`, `npm run docs:sync`, `npm run skills:check`, `npm test` (4,700+ tests). Commit anything the generators changed. CI runs the same build, skills check and tests.
- Before any npm publish: `npm run release:gate` must pass. It installs the packed tarballs into a clean home and runs capture, recovery, restart, export/import, MCP and status checks against the installed CLI (see CONTRIBUTING.md, Release process)
- Mock pattern: `vi.mock("iii-sdk")` with mock `sdk.trigger`, `kv.get/set/list`
- Test files go in `test/` with `.test.ts` extension
@@ -118,4 +118,4 @@ Hook scripts in `src/hooks/` are standalone Node.js scripts (no iii-sdk import).
- 6 MCP resources, 3 MCP prompts
- 12 hooks, 17 skills
- 322+ iii functions
-- 4,600+ tests
+- 4,700+ tests
diff --git a/README.md b/README.md
index 31ad6a5546..2ceb64b016 100644
--- a/README.md
+++ b/README.md
@@ -69,7 +69,7 @@
-
+
@@ -1311,7 +1311,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.ar-SA.md b/READMEs/README.ar-SA.md
index 13a79a42b6..82665ff7d4 100644
--- a/READMEs/README.ar-SA.md
+++ b/READMEs/README.ar-SA.md
@@ -71,7 +71,7 @@
-
+
@@ -1242,7 +1242,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.bn-BD.md b/READMEs/README.bn-BD.md
index 98bcbaa9ce..e11408e63f 100644
--- a/READMEs/README.bn-BD.md
+++ b/READMEs/README.bn-BD.md
@@ -69,7 +69,7 @@
-
+
@@ -1255,7 +1255,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.cs-CZ.md b/READMEs/README.cs-CZ.md
index 5a64f8b08a..d0799f4279 100644
--- a/READMEs/README.cs-CZ.md
+++ b/READMEs/README.cs-CZ.md
@@ -69,7 +69,7 @@
-
+
@@ -1241,7 +1241,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.da-DK.md b/READMEs/README.da-DK.md
index 2b4f540c29..4590701e7c 100644
--- a/READMEs/README.da-DK.md
+++ b/READMEs/README.da-DK.md
@@ -69,7 +69,7 @@
-
+
@@ -1249,7 +1249,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.de-DE.md b/READMEs/README.de-DE.md
index f4053424a3..0f2d218040 100644
--- a/READMEs/README.de-DE.md
+++ b/READMEs/README.de-DE.md
@@ -69,7 +69,7 @@
-
+
@@ -1247,7 +1247,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.el-GR.md b/READMEs/README.el-GR.md
index 87964c0e5c..bf07a19b33 100644
--- a/READMEs/README.el-GR.md
+++ b/READMEs/README.el-GR.md
@@ -69,7 +69,7 @@
-
+
@@ -1241,7 +1241,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.es-ES.md b/READMEs/README.es-ES.md
index ec06c15d98..7a7f31e2b5 100644
--- a/READMEs/README.es-ES.md
+++ b/READMEs/README.es-ES.md
@@ -69,7 +69,7 @@
-
+
@@ -1243,7 +1243,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.fi-FI.md b/READMEs/README.fi-FI.md
index e31f85b642..117a90888e 100644
--- a/READMEs/README.fi-FI.md
+++ b/READMEs/README.fi-FI.md
@@ -69,7 +69,7 @@
-
+
@@ -1241,7 +1241,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.fr-FR.md b/READMEs/README.fr-FR.md
index 9af69d3629..57db486f2c 100644
--- a/READMEs/README.fr-FR.md
+++ b/READMEs/README.fr-FR.md
@@ -69,7 +69,7 @@
-
+
@@ -1250,7 +1250,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.he-IL.md b/READMEs/README.he-IL.md
index 034ed34170..edbb7fb81b 100644
--- a/READMEs/README.he-IL.md
+++ b/READMEs/README.he-IL.md
@@ -71,7 +71,7 @@
-
+
@@ -1242,7 +1242,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.hi-IN.md b/READMEs/README.hi-IN.md
index 117ad51d40..d4a2b4d88c 100644
--- a/READMEs/README.hi-IN.md
+++ b/READMEs/README.hi-IN.md
@@ -69,7 +69,7 @@
-
+
@@ -1256,7 +1256,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.hu-HU.md b/READMEs/README.hu-HU.md
index d3cdef2c89..d960ff8d8b 100644
--- a/READMEs/README.hu-HU.md
+++ b/READMEs/README.hu-HU.md
@@ -69,7 +69,7 @@
-
+
@@ -1239,7 +1239,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.id-ID.md b/READMEs/README.id-ID.md
index c4e5f5a9ef..c7ddb2699d 100644
--- a/READMEs/README.id-ID.md
+++ b/READMEs/README.id-ID.md
@@ -69,7 +69,7 @@
-
+
@@ -1256,7 +1256,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.it-IT.md b/READMEs/README.it-IT.md
index 1db24899f8..a2d82fc254 100644
--- a/READMEs/README.it-IT.md
+++ b/READMEs/README.it-IT.md
@@ -69,7 +69,7 @@
-
+
@@ -1249,7 +1249,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.ja-JP.md b/READMEs/README.ja-JP.md
index 679309b6b8..16e0465f7e 100644
--- a/READMEs/README.ja-JP.md
+++ b/READMEs/README.ja-JP.md
@@ -69,7 +69,7 @@
-
+
@@ -1255,7 +1255,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.ko-KR.md b/READMEs/README.ko-KR.md
index 68a70a99f8..e323c8cba1 100644
--- a/READMEs/README.ko-KR.md
+++ b/READMEs/README.ko-KR.md
@@ -69,7 +69,7 @@
-
+
@@ -1254,7 +1254,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.nb-NO.md b/READMEs/README.nb-NO.md
index 906c9afed5..b4e085ac40 100644
--- a/READMEs/README.nb-NO.md
+++ b/READMEs/README.nb-NO.md
@@ -69,7 +69,7 @@
-
+
@@ -1247,7 +1247,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.nl-NL.md b/READMEs/README.nl-NL.md
index 82f7752da7..a3e71f316c 100644
--- a/READMEs/README.nl-NL.md
+++ b/READMEs/README.nl-NL.md
@@ -69,7 +69,7 @@
-
+
@@ -1243,7 +1243,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.pl-PL.md b/READMEs/README.pl-PL.md
index bc03deaebe..8a9e9b64a6 100644
--- a/READMEs/README.pl-PL.md
+++ b/READMEs/README.pl-PL.md
@@ -69,7 +69,7 @@
-
+
@@ -1239,7 +1239,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.pt-BR.md b/READMEs/README.pt-BR.md
index c057540f82..6319ece79a 100644
--- a/READMEs/README.pt-BR.md
+++ b/READMEs/README.pt-BR.md
@@ -69,7 +69,7 @@
-
+
@@ -1247,7 +1247,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.pt-PT.md b/READMEs/README.pt-PT.md
index 4df36d9384..c4d3d10adb 100644
--- a/READMEs/README.pt-PT.md
+++ b/READMEs/README.pt-PT.md
@@ -69,7 +69,7 @@
-
+
@@ -1243,7 +1243,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.ro-RO.md b/READMEs/README.ro-RO.md
index 04daaa1692..3aa1227664 100644
--- a/READMEs/README.ro-RO.md
+++ b/READMEs/README.ro-RO.md
@@ -69,7 +69,7 @@
-
+
@@ -1235,7 +1235,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.ru-RU.md b/READMEs/README.ru-RU.md
index 3dc2bd036b..aff111cef8 100644
--- a/READMEs/README.ru-RU.md
+++ b/READMEs/README.ru-RU.md
@@ -69,7 +69,7 @@
-
+
@@ -1241,7 +1241,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.sv-SE.md b/READMEs/README.sv-SE.md
index a3cc9e47c3..5c9987d186 100644
--- a/READMEs/README.sv-SE.md
+++ b/READMEs/README.sv-SE.md
@@ -69,7 +69,7 @@
-
+
@@ -1249,7 +1249,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.th-TH.md b/READMEs/README.th-TH.md
index 2c33f0df62..b1f7d5e460 100644
--- a/READMEs/README.th-TH.md
+++ b/READMEs/README.th-TH.md
@@ -69,7 +69,7 @@
-
+
@@ -1242,7 +1242,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.tl-PH.md b/READMEs/README.tl-PH.md
index 60847a4785..e3a6ed3e5a 100644
--- a/READMEs/README.tl-PH.md
+++ b/READMEs/README.tl-PH.md
@@ -69,7 +69,7 @@
-
+
@@ -1251,7 +1251,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.tr-TR.md b/READMEs/README.tr-TR.md
index c414df3087..34dfa45d57 100644
--- a/READMEs/README.tr-TR.md
+++ b/READMEs/README.tr-TR.md
@@ -69,7 +69,7 @@
-
+
@@ -1241,7 +1241,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.uk-UA.md b/READMEs/README.uk-UA.md
index f6c7cbeffe..9aa78d3f01 100644
--- a/READMEs/README.uk-UA.md
+++ b/READMEs/README.uk-UA.md
@@ -69,7 +69,7 @@
-
+
@@ -1242,7 +1242,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.ur-PK.md b/READMEs/README.ur-PK.md
index 5f04968cc3..47ec42e323 100644
--- a/READMEs/README.ur-PK.md
+++ b/READMEs/README.ur-PK.md
@@ -71,7 +71,7 @@
-
+
@@ -1246,7 +1246,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.vi-VN.md b/READMEs/README.vi-VN.md
index a58f67266f..6b5d7c2de4 100644
--- a/READMEs/README.vi-VN.md
+++ b/READMEs/README.vi-VN.md
@@ -69,7 +69,7 @@
-
+
@@ -1251,7 +1251,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.zh-CN.md b/READMEs/README.zh-CN.md
index d065aa3500..3a6515e0f2 100644
--- a/READMEs/README.zh-CN.md
+++ b/READMEs/README.zh-CN.md
@@ -69,7 +69,7 @@
-
+
@@ -1252,7 +1252,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/READMEs/README.zh-TW.md b/READMEs/README.zh-TW.md
index 1c5117fbb5..5f3686941d 100644
--- a/READMEs/README.zh-TW.md
+++ b/READMEs/README.zh-TW.md
@@ -69,7 +69,7 @@
-
+
@@ -1244,7 +1244,7 @@ curl -X POST http://localhost:3111/agentmemory/graph/compact -H "Authorization:
```bash
npm run dev # Hot reload
npm run build # Production build
-npm test # 4,600+ tests
+npm test # 4,700+ tests
npm run test:integration # API tests (requires running services)
```
diff --git a/assets/tags/light/stat-tests.svg b/assets/tags/light/stat-tests.svg
index bf36e50014..1b4179cfc2 100644
--- a/assets/tags/light/stat-tests.svg
+++ b/assets/tags/light/stat-tests.svg
@@ -1,5 +1,5 @@
-