Repository navigation
feat(graph): delta graph extraction #1309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -685,17 +685,63 @@ export function registerGraphFunction( | |
| provider: MemoryProvider, | ||
| ): void { | ||
| sdk.registerFunction("mem::graph-extract", | ||
| async (data: { observations: CompressedObservation[] }) => { | ||
| if (!data.observations || data.observations.length === 0) { | ||
| async (data: { | ||
| observations: CompressedObservation[]; | ||
| sessionId?: string; | ||
| force?: boolean; | ||
| }) => { | ||
| if (!data || !data.observations || data.observations.length === 0) { | ||
| return { success: false, error: "No observations provided" }; | ||
| } | ||
|
|
||
| const obsIds = data.observations.map((o) => o.id); | ||
| const sessionId = | ||
| (typeof data.sessionId === "string" && data.sessionId.trim()) || | ||
| data.observations[0]?.sessionId; | ||
|
|
||
| let targetObs = data.observations; | ||
| if (!data.force && sessionId) { | ||
| const extractedSet = new Set<string>(); | ||
| try { | ||
| const extractedList = await kv.list< | ||
| { id?: string; obsId?: string } | string | ||
| >(KV.graphExtracted(sessionId)); | ||
|
Comment on lines
+705
to
+707
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- repository convention scope ---'
find /tmp/coderabbit-repo-knowledge/rohitg00-agentmemory-ce01373c -type f -name '*.md' -print \
| sort
printf '%s\n' '--- graph structure ---'
ast-grep outline src/functions/graph.ts
printf '%s\n' '--- target and surrounding code ---'
sed -n '660,850p' src/functions/graph.ts
printf '%s\n' '--- relevant bindings and callers ---'
rg -n -C 3 'graphExtracted|mem::graph-extract|registerFunction|sdk\.trigger\(|recordAudit|force' src/functions/graph.ts srcRepository: rohitg00/agentmemory Length of output: 50377 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- applicable convention and learning files ---'
find /tmp/coderabbit-repo-knowledge/rohitg00-agentmemory-ce01373c \
-type f -name '*.md' -print \
| while read -r f; do
if grep -qE 'graph|src/functions|concurr|lease|trigger|KV' "$f"; then
echo "--- $f"
cat "$f"
fi
done
printf '%s\n' '--- graph extraction path ---'
sed -n '660,850p' src/functions/graph.ts
printf '%s\n' '--- exact graph-related definitions and registrations ---'
rg -n -C 5 'graphExtracted|graph-extract|extractGraph|provider|graphData|recordAudit' \
src/functions/graph.ts src/functions src/triggers
printf '%s\n' '--- SDK and KV contract files ---'
rg -l 'interface.*SDK|type.*SDK|registerFunction|class.*KV|async list|function list' \
. --glob '!node_modules/**' --glob '!dist/**' --glob '!build/**' \
| head -80Repository: rohitg00/agentmemory Length of output: 50377 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- graph persistence implementation ---'
ast-grep outline src/functions/graph.ts
sed -n '500,670p' src/functions/graph.ts
printf '%s\n' '--- lock implementation and usages around graph persistence ---'
cat -n src/state/keyed-mutex.ts
rg -n -C 4 'persistGraphDelta|withKeyedLock|graphNodes|graphEdges' src/functions/graph.ts src/functions src/state
printf '%s\n' '--- SDK dependency and local scheduling evidence ---'
rg -n -C 3 '"iii-sdk"|trigger\s*\(' package.json package-lock.json npm-shrinkwrap.json pnpm-lock.yaml yarn.lock src test 2>/dev/null \
| head -240Repository: rohitg00/agentmemory Length of output: 50376 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- graph heuristic identifiers and imports ---'
sed -n '1,30p' src/functions/graph.ts
sed -n '450,555p' src/functions/graph.ts
printf '%s\n' '--- identifier implementations and graph types ---'
rg -n -C 5 'function (generateId|fingerprintId)|export function (generateId|fingerprintId)|interface Graph(Node|Edge)|type Graph(Node|Edge)' \
src/state/schema.ts src/types.ts
printf '%s\n' '--- all graph extraction tests and concurrency-related tests ---'
rg -n -C 5 'graph-extract|persistGraphDelta|graphExtracted|concurr|duplicate|idempot' \
test src/functions/graph.tsRepository: rohitg00/agentmemory Length of output: 50376 🌐 Web query:
💡 Result: In the iii-sdk ecosystem, function management and invocation are handled through the iii-engine [1][2]. registerFunction The registerFunction method makes a function available within the system [3][4]. Once registered, a function can be invoked from any connected worker or via the CLI [4][5]. Signature: registerFunction(functionId: string, handler: RemoteFunctionHandler | HttpInvocationConfig, options?: RegisterFunctionOptions) => FunctionRef [6][7][8] The handler can be a local asynchronous function or an HTTP invocation configuration for external services (e.g., Lambda) [6][8][9]. sdk.trigger The trigger method is the primary mechanism for invoking registered functions [9][5]. Usage: trigger({ function_id: string, payload: any, action?: TriggerAction }) [10][9] For synchronous calls, it waits for the result [9][11]. For fire-and-forget invocations, use TriggerAction.Void [10][9][2]. The engine handles routing and coordination [1][4][2]. Concurrency and Serialization The iii-engine manages function invocation and routing [1][4]. When a function is triggered: 1. The engine assigns an invocation_id for tracking synchronous calls [11][2]. 2. For synchronous requests, the engine forwards the call to the worker and waits for an InvocationResult [11][2]. 3. For asynchronous (Void) actions, the engine does not expect a reply [11][2]. 4. Serialization is managed via JSON protocol frames exchanged between the SDK and the engine (e.g., InvokeFunction, InvocationResult) [7][11][2]. Legacy Note: Older SDK versions included methods like triggerVoid, which have been deprecated in favor of using trigger with explicit actions [10]. Citations:
Make the cache claim atomic with extraction. 🤖 Prompt for AI Agents |
||
| for (const item of extractedList) { | ||
| if (typeof item === "string") { | ||
| extractedSet.add(item); | ||
| } else if (item && typeof item === "object") { | ||
| if (typeof item.id === "string") extractedSet.add(item.id); | ||
| else if (typeof item.obsId === "string") extractedSet.add(item.obsId); | ||
| } | ||
| } | ||
| } catch (err) { | ||
| logger.warn("failed to read graph extracted observations", { | ||
| sessionId, | ||
| error: err instanceof Error ? err.message : String(err), | ||
| }); | ||
| } | ||
|
|
||
| targetObs = data.observations.filter((o) => !extractedSet.has(o.id)); | ||
| if (targetObs.length === 0) { | ||
| logger.info("Graph extraction skipped — all observations already extracted", { | ||
| sessionId, | ||
| count: data.observations.length, | ||
| }); | ||
| return { | ||
| success: true, | ||
| nodesAdded: 0, | ||
| edgesAdded: 0, | ||
| cached: true, | ||
| skipped: true, | ||
| }; | ||
| } | ||
| } | ||
|
|
||
| const obsIds = targetObs.map((o) => o.id); | ||
|
|
||
| let nodes: GraphNode[] = []; | ||
| let edges: GraphEdge[] = []; | ||
| try { | ||
| const heuristic = extractGraphHeuristics(data.observations); | ||
| const heuristic = extractGraphHeuristics(targetObs); | ||
| nodes = heuristic.nodes; | ||
| edges = heuristic.edges; | ||
| } catch (err) { | ||
|
|
@@ -709,7 +755,7 @@ export function registerGraphFunction( | |
| let llmError: string | undefined; | ||
| if (llmEnabled) { | ||
| const prompt = buildGraphExtractionPrompt( | ||
| data.observations.map((o) => ({ | ||
| targetObs.map((o) => ({ | ||
| title: o.title, | ||
| narrative: o.narrative, | ||
| concepts: o.concepts, | ||
|
|
@@ -732,9 +778,20 @@ export function registerGraphFunction( | |
| } | ||
|
|
||
| if (nodes.length === 0 && edges.length === 0) { | ||
| return llmError | ||
| ? { success: false, error: llmError } | ||
| : { success: true, nodesAdded: 0, edgesAdded: 0 }; | ||
| if (llmError) { | ||
| return { success: false, error: llmError }; | ||
| } | ||
| if (sessionId) { | ||
| await Promise.all( | ||
| targetObs.map((o) => | ||
| kv.set(KV.graphExtracted(sessionId), o.id, { | ||
| id: o.id, | ||
| extractedAt: new Date().toISOString(), | ||
| }), | ||
| ), | ||
| ); | ||
|
Comment on lines
+785
to
+792
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win Write the audit record before committing extraction markers. The empty-result path changes KV state without As per coding guidelines, Also applies to: 805-814 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
| } | ||
| return { success: true, nodesAdded: 0, edgesAdded: 0 }; | ||
| } | ||
|
|
||
| try { | ||
|
|
@@ -745,6 +802,17 @@ export function registerGraphFunction( | |
| obsIds, | ||
| ); | ||
|
|
||
| if (sessionId) { | ||
| await Promise.all( | ||
| targetObs.map((o) => | ||
| kv.set(KV.graphExtracted(sessionId), o.id, { | ||
| id: o.id, | ||
| extractedAt: new Date().toISOString(), | ||
| }), | ||
| ), | ||
| ); | ||
| } | ||
|
|
||
| await recordAudit(kv, "observe", "mem::graph-extract", obsIds, { | ||
| nodesExtracted: nodes.length, | ||
| edgesExtracted: edges.length, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: rohitg00/agentmemory
Length of output: 2649
🏁 Script executed:
Repository: rohitg00/agentmemory
Length of output: 40148
🏁 Script executed:
Repository: rohitg00/agentmemory
Length of output: 50377
Require literal
trueto bypass deduplication.The HTTP graph-extract endpoint forwards
req.bodywithout validatingforce. Sincedata.forceis checked by truthiness, values such as"false"or1skip theKV.graphExtracted(sessionId)lookup and reprocess observations. Usedata.force !== true.🤖 Prompt for AI Agents