feat(store): keep the graph in FalkorDB as well as SQLite

[cl0ver012]

Summary

Milestone 10: a FalkorDB backend. With CHATLORE_STORE=falkordb the graph lives in a FalkorDB server instead of the SQLite file in the library folder. Every command, the web interface, the API, and the MCP server work on it unchanged. It passes the same contract tests, and CI runs the whole test suite with it as the store.

Changes

Store (chatlore.store.falkordb.FalkorDBStore):

• Nodes. Every node has the label Node, with _id and _label as indexed properties and all props as JSON in _props, since FalkorDB properties cannot hold maps or nulls. Strings, numbers, and booleans are copied into p_<name> properties for find_nodes and the search filters. Edges keep their ChatLore type, with props as JSON. Store-wide state sits on Meta nodes.
• Search indexes. Word search uses a FalkorDB full-text index over folded copies of the title (weight 2) and text. Vectors sit in a euclidean vector index created with the first embedding and dropped by clear_embeddings, so a new dimension works after clearing, as with SQLite.

Where FalkorDB differs, and what the store does about it. Each of these was found by probing a real FalkorDB 4.20.7:

• Full-text search does not fold accents ("cafe" did not find "café"), so indexed text and queries are folded here, and snippets with [matches] are built here. Stemming and stop words are turned off, so "index" does not match "indexing" and "the" is searchable, as in SQLite.
• A vector of the wrong length and an edge to a missing node are accepted without error, so both are checked first and raise like SQLite.
• A query returns at most 10,000 rows, so long reads (find_nodes, neighbors, nodes_without_embedding) are fetched in pages, and writes go in batches.
• Two writes to the same node or edge in one query keep the first, so the last one given is kept, as in SQLite.
• A client made from a URL does not close its sockets, so the store closes them.
• There are no transactions across queries. transaction() only groups calls; each write is atomic.

Choosing a store (chatlore.store):

• open_store reads CHATLORE_STORE (sqlite, the default, or falkordb), plus CHATLORE_FALKORDB_URL and CHATLORE_FALKORDB_GRAPH. It says to install chatlore[falkordb] when the client is missing, and refuses unknown values.
• New drop_store and describe_store. chatlore index --rebuild now drops either store, and chatlore doctor shows which is in use.

Speed on a distant server. Running the demo against a free FalkorDB Cloud server a continent away (0.16 s per round trip, about 20 KB/s) showed pages sending one query per item and data they did not use. SQLite's cost per query is near zero, so this had gone unnoticed.

• Batch methods on GraphStore. Each has a default that goes one item at a time, so any backend keeps working; FalkorDB answers each in one query, in pages:
  • get_nodes, neighbors_many, and edges_many read many nodes, neighbours, or links at once;
  • count_by_label counts every label in one call;
  • upsert_conversations and set_embeddings write many at once.
• Who uses them. Search, the passages gathered for a question, the graph view, stats, and entity and topic lists. Importing, indexing, and archive imports hand conversations to a ConversationWriter 200 at a time, which writes what is waiting even when an import stops with an error. Embedding sync writes a batch at a time.
• Finding the entities a question names. This read every entity. An entity's id is made from the same normal form of its name the question is matched in, so only the ids its words could name are fetched, each checked against its name again.
• The graph view. It now fetches links without every neighbour's full record, takes the outgoing links from the ones already fetched, and reads each topic once.
• FalkorDB store internals:
  • it shares one client per server for the process, closed at exit;
  • it checks connections idle for 30 s before reuse, since networks drop them;
  • it reads the embedding dimension only when a request needs it;
  • it pages the chunk lookup when deleting messages, which could have been cut off at 10,000 rows.
• Security. chatlore doctor hid nothing before; it now shows the server URL with the password as ***.

Dependencies. falkordb>=1.7 as the optional extra falkordb. mypy skips its missing type information.

Tests.

• The contract tests are parametrized over both stores. FalkorDB runs when CHATLORE_TEST_FALKORDB_URL is set and is skipped otherwise, with a fresh graph per test.
• With CHATLORE_STORE=falkordb, an autouse fixture gives every test its own graph, so the whole suite runs against FalkorDB.
• Three tests assumed the SQLite file, by deleting chatlore.db or expecting a second folder to be a second database. They now call drop_store, or pin SQLite where the test is about SQLite.
• New tests without a server cover word folding, snippets, and choosing the store.

CI. A new "FalkorDB backend" job starts falkordb/falkordb:v4.20.7 as a service. It runs the contract tests on both stores, then every test with FalkorDB as the store.

Docs.

• New docs/falkordb.md: setup, the settings, and moving a library in from the caches without computing anything again. It also describes the graph's shape, with a Cypher example, and the differences from SQLite, with timings.
• README docs link and roadmap (M10 done), and CHANGELOG.

How it was tested

• Against FalkorDB 4.20.7 in GitHub Actions (there is no Docker on the development machine):
  • all 41 contract tests passed, on SQLite and FalkorDB;
  • the whole suite passed with FalkorDB as the store.
• The demo on FalkorDB:
  • chatlore demo loaded 32 conversations, 143 entities, and 17 topics;
  • word search, hybrid search by meaning, entity Tidewater, topics, and export all gave the same results as on SQLite.
• 12,000 made-up notes (over 24,000 nodes, so paging was exercised), SQLite against FalkorDB:
  • import: 8 s against 31 s;
  • chunking: 145 s against 26 s;
  • search: 0.8 s against 0.9 s;
  • re-import with nothing changed: 2 s on both;
  • searches returned the same notes.
• The 145 s chunking on SQLite comes from a lookup by conversation that scans every chunk. It predates this PR and is noted as a separate task.
• Speed against the distant server, before and after:
  • search: 46 queries and 10.6 s, now 7 queries and 1 s;
  • graph view: about 400 queries and 105 s, now 4 queries and 2.4 s;
  • gathering the passages for a question: 5.7 s, now 0.9 s;
  • stats: 6 queries, now 2;
  • loading the demo: 96 s, now 41 s.
• Same answers. On a SQLite copy of the demo, 22 answers were compared between the code before the speed work and after, byte for byte: stats, the graph view (overview, 20 entities, around an entity, of a topic), topics, entities, five searches, and the passages and entities gathered for four questions. All 22 were identical.
• FalkorDB against SQLite. On FalkorDB, 18 of the 22 answers are identical to SQLite. The other 4 are searches whose word matches FalkorDB ranks with its own scores (noted in the docs), so equal matches can come in another order.
• Contract tests on the server. A contract test holds each batch method to its one-at-a-time version, and all 26 FalkorDB contract tests pass against the FalkorDB Cloud server too.
• ruff, ruff format --check, strict mypy, and pytest pass locally on Python 3.14 (FalkorDB tests skipped there).

Checklist

• uv run ruff check . and uv run ruff format --check . pass
• uv run mypy passes
• uv run pytest passes
• No real exports, databases, or keys are included
• CHANGELOG.md updated under Unreleased