Skip to content

feat(episodic): persist whitelisted episode_metadata provenance keys as node properties (FalkorDB/Neo4j)#2

Merged
calebjacksonhoward merged 2 commits into
mainfrom
feat/episodic-provenance-properties
Jul 8, 2026
Merged

feat(episodic): persist whitelisted episode_metadata provenance keys as node properties (FalkorDB/Neo4j)#2
calebjacksonhoward merged 2 commits into
mainfrom
feat/episodic-provenance-properties

Conversation

@calebjacksonhoward

Copy link
Copy Markdown

Summary

Watercooler A1-full (completion-sequence Wave 4, Codex-approved plan 01KX0AMDMN0R0VX335PQYDW4M3 in audit-transport-modes-hosted-db-2026-07): the four provenance keys (entry_id, thread_id, chunk_index, total_chunks) from episode_metadata become first-class Episodic properties, because the FalkorDB/Neo4j save queries write a full property map (SET n = {…}) — a post-hoc stamp is wiped on the next re-save. First-class properties are also indexable (:Episodic(entry_id) for provenance backtraces).

Changes

  • add_episode and RawEpisode gain episode_metadata passthrough into EpisodicNode — the field existed on the model but was never plumbed and never persisted on FalkorDB.
  • EpisodicNode.save flattens EPISODIC_PROVENANCE_KEYS into save params, provider-gated to FalkorDB/Neo4j: Kuzu has a typed schema (unknown params error), Neptune untested — both skipped with in-code notes.
  • Single + bulk save queries for FalkorDB/Neo4j write the four fields; Kuzu/Neptune queries byte-identical.
  • Bulk path (add_nodes_and_edges_bulk_tx) flattens per episode dict under the same gate.

Validation

  • Driver-free unit tests (tests/test_episodic_provenance.py): params carry/omit per provider, query-text references, Kuzu/Neptune untouched. Dependency-free (asyncio.run).
  • Live against a real FalkorDB: properties persist, survive the full-map re-save, and default to null when no metadata is supplied.

🤖 Generated with Claude Code

…as node properties (FalkorDB/Neo4j)

Watercooler A1-full (thread audit-transport-modes-hosted-db-2026-07,
completion sequence Wave 4): entry_id/thread_id/chunk_index/total_chunks
from episode_metadata become first-class Episodic properties so they
survive the providers' full-map 'SET n = {...}' writes (a post-hoc
stamp is wiped on the next re-save) and are indexable
(:Episodic(entry_id) for provenance backtraces).

- add_episode / RawEpisode gain episode_metadata passthrough into
  EpisodicNode (the field existed on the model but was never plumbed).
- EpisodicNode.save flattens EPISODIC_PROVENANCE_KEYS into the save
  params, gated to FalkorDB/Neo4j (Kuzu has a typed schema — unknown
  params error; Neptune untested — both skipped with in-code notes).
- Single + bulk save queries for FalkorDB/Neo4j write the four fields;
  Kuzu/Neptune queries unchanged.
- bulk path: add_nodes_and_edges_bulk_tx flattens per episode dict,
  same provider gate.

Tests: driver-free unit tests (params carry/omit per provider; query
text references; Kuzu/Neptune untouched) — dependency-free via
asyncio.run so they pass with or without pytest-asyncio. Live-validated
against a real FalkorDB: properties persist, survive full-map re-save,
and default to null without metadata.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: calebjacksonhoward <calebjhoward@gmail.com>
@cursor

cursor Bot commented Jul 8, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@calebjacksonhoward

Copy link
Copy Markdown
Author

Evidence basis for merge (the inherited unit-tests/pyright/ruff/database-integration-tests jobs request upstream's depot-ubuntu-22.04 paid runners, which this fork has no access to — they will sit queued forever; CLAAssistant/triage failures are upstream bots):

  1. Fork unit suite, failure-set parity with unmodified main — identical runs of the suite (minus optional-provider modules whose deps our env doesn't carry) on this branch vs main: branch 242 passed / 11 failed / 35 errors; main 237 passed / 11 failed / 35 errors. The +5 delta is exactly this PR's new tests. A targeted re-run of main's precise failing test IDs against this branch confirms the failure sets are identical identity-for-identity — every failure pre-exists this change and is environmental (live Neo4j/DB connection tests).
  2. New unit tests green (tests/test_episodic_provenance.py, 5 tests): params carry/omit per provider, query-text assertions, Kuzu/Neptune byte-identical.
  3. Live FalkorDB validation: the four provenance properties persist, survive the full-map re-save, and default to null without metadata.
  4. CodeQL + analyzers green on this head.

Follow-up chore filed separately: point the inherited workflows at ubuntu-latest so future fork PRs get runnable CI.

@calebjacksonhoward calebjacksonhoward left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one blocking issue.

graphiti_core/models/nodes/node_db_queries.py now makes the FalkorDB/Neo4j single-save query require $entry_id, $thread_id, $chunk_index, and $total_chunks, but the provider operations paths were not updated. graphiti_core/driver/falkordb/operations/episode_node_ops.py and graphiti_core/driver/neo4j/operations/episode_node_ops.py still build the old params dict in save(), then call the same changed query, so graphiti.nodes.episode.save(...) will execute with missing parameters. Their save_bulk() paths also serialize dict(node) without flattening the whitelisted episode_metadata keys, so the new bulk query will not persist the intended provenance through the operations/namespace API.

Why this changes the decision likelihood: this PR’s stated purpose is to make the whitelisted provenance survive episode saves for FalkorDB/Neo4j. If the operations/namespace save paths either fail at runtime or omit those fields, the feature is only partially implemented and callers using the new operations API can still lose or fail to write the provenance the PR is meant to preserve.

Please update the FalkorDB and Neo4j EpisodeNodeOperations.save() and save_bulk() implementations to use the same provenance flattening/defaulting as EpisodicNode.save() and add_nodes_and_edges_bulk_tx(), and add coverage for at least one operations-path save so this does not regress.

I also tried to run python -m pytest tests/test_episodic_provenance.py -q, but this local clone is missing the neo4j dependency, so pytest stopped during conftest import before running the test file.

…deNodeOperations save paths (PR #2 review)

The review caught that the FalkorDB/Neo4j operations-interface save()
built its own params dict against the changed full-map query — missing
the now-required provenance params (runtime failure) — and save_bulk()
serialized dict(node) without flattening episode_metadata (silent
provenance drop). Both now apply the same EPISODIC_PROVENANCE_KEYS
flattening/defaulting as EpisodicNode.save() and
add_nodes_and_edges_bulk_tx(). Five operations-path tests added (save
carry/default, bulk flatten, both providers); note the plain FalkorDB
driver does not activate the operations interface, so the classic path
remains the one our deployment exercises — this closes the gap for
callers that do enable it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: calebjacksonhoward <calebjhoward@gmail.com>
@calebjacksonhoward

Copy link
Copy Markdown
Author

Blocking finding addressed in eb3bfc4 — and it was a good catch: the operations-interface save() would have failed at runtime with missing parameters against the changed query, and save_bulk() would have silently dropped provenance.

Both FalkorEpisodeNodeOperations and Neo4jEpisodeNodeOperations now apply the identical EPISODIC_PROVENANCE_KEYS flattening/defaulting as EpisodicNode.save() / add_nodes_and_edges_bulk_tx(), in save() and save_bulk(). Five operations-path tests added (carry + default-to-null on save, bulk flattening, both providers) — provenance suite now 10/10.

Scope note verified live: the plain FalkorDriver does not activate graph_operations_interface (checked against a real FalkorDB from this head — ops interface active: False, classic-path read-back still correct), so our deployment exercises the originally-patched path; this fix closes the gap for callers that enable the operations interface.

@cursor

cursor Bot commented Jul 8, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@calebjacksonhoward calebjacksonhoward left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rereviewed the fix at eb3bfc4080b42a3c2ea261821c60cee986f25953.

The prior blocking concern is addressed. The FalkorDB and Neo4j EpisodeNodeOperations.save() paths now add the whitelisted provenance params with None defaults, and their save_bulk() paths flatten the same keys into each episode dict before calling the updated bulk queries. The added tests cover the operations-path regression for single and bulk saves.

I do not have further code findings from this pass.

Verification note: I retried python -m pytest tests/test_episodic_provenance.py -q, but this local clone still fails during conftest import because the environment is missing neo4j (ModuleNotFoundError: No module named 'neo4j'). GitHub currently still reports CLAAssistant/triage failing and pyright/unit-tests/database-integration-tests/ruff pending, so those checks still need to settle independently.

@calebjacksonhoward
calebjacksonhoward merged commit 69ac0dd into main Jul 8, 2026
7 of 13 checks passed
@calebjacksonhoward
calebjacksonhoward deleted the feat/episodic-provenance-properties branch July 8, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant