RediSearch index overhead

[alexluong]

Summary

The tenant RediSearch index uses prefix tenant: which matches more keys than necessary, causing destination summary hashes to be indexed and filtered out at query time.

Details

Index configuration:

FT.CREATE tenant_idx ON HASH PREFIX 1 tenant: FILTER @entity == "tenant"

Keys matching prefix tenant::

Key pattern	Entity field	Indexed?
tenant:{id}:tenant	"tenant"	Yes
tenant:{id}:destination:{destId}	"destination"	No (index FILTER excludes)
tenant:{id}:destinations	(none)	Yes (missing field ≠ filtered)

The index FILTER @entity == "tenant" excludes destination hashes (which have entity = "destination"), but does NOT exclude destination summary hashes (which have no entity field at all).

The query filter @entity:{tenant} in FT.SEARCH is what actually excludes destination summaries from results.

Impact

• Index size: Destination summary hashes are indexed unnecessarily (one per tenant)
• Query overhead: Each query filters them out via @entity:{tenant}
• Low overall - one extra indexed doc per tenant, filtered at query time

Considered alternatives

Add entity: "summary" to destination summary hash:

• Not clean - this hash is used with HGETALL where keys are destination IDs and values are serialized summaries. Adding entity pollutes the structure.
• Creates implicit convention: "every hash under tenant: prefix needs an entity field" - easy to forget, not enforced.

Use a more specific prefix (e.g., tenant:entity:{id}):

• Requires migration path for existing data
• Index recreation

Decision

Accept the overhead and rely on query filter @entity:{tenant} for now. Worth tracking but not worth the complexity to fix.