Skip to content

Enforce MinHash construction compatibility#2

Closed
dipeshbabu wants to merge 1 commit into
masterfrom
agent/enforce-minhash-compatibility
Closed

Enforce MinHash construction compatibility#2
dipeshbabu wants to merge 1 commit into
masterfrom
agent/enforce-minhash-compatibility

Conversation

@dipeshbabu

Copy link
Copy Markdown
Owner

What

Fingerprint the actual permutation matrix and hash callable, then enforce those fingerprints in MinHash operations and all LSH variants.

Why

Sketches with the same seed, length, and scheme can still be incompatible when callers provide different permutations or hash functions. Comparing or indexing them together silently produces invalid similarity results.

Impact

Jaccard, merge, union, LSH, async LSH, LSH Forest, and LSH Bloom now fail clearly when known construction metadata differs. WeightedMinHash remains exempt, and old indexes carrying only scheme metadata remain readable.

Root cause

Compatibility checks treated the seed as a complete identity for permutation generation and did not track the input hash function.

Checks

  • uv run pytest -q test/test_minhash.py test/test_minhash_schemes.py test/test_lsh.py test/test_lshforest.py (119 passed)
  • uv run pytest -q test/test_lshbloom.py (9 skipped; optional dependency unavailable)
  • uv run ruff check on changed Python files
  • uvx pyright (0 errors)
  • git diff --check

@dipeshbabu
dipeshbabu marked this pull request as ready for review July 19, 2026 03:50
@dipeshbabu

Copy link
Copy Markdown
Owner Author

Superseded by upstream pull request ekzhu#326.

@dipeshbabu dipeshbabu closed this Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant