Skip to content

Enforce MinHash construction compatibility#326

Open
dipeshbabu wants to merge 2 commits into
ekzhu:masterfrom
dipeshbabu:agent/enforce-minhash-compatibility
Open

Enforce MinHash construction compatibility#326
dipeshbabu wants to merge 2 commits into
ekzhu:masterfrom
dipeshbabu:agent/enforce-minhash-compatibility

Conversation

@dipeshbabu

Copy link
Copy Markdown
Contributor

What

Fingerprint the actual permutation matrix and hash callable, then enforce those fingerprints in MinHash operations and all LSH variants.

Why

Sketches with the same seed, length, and scheme can still be incompatible when callers provide different permutations or hash functions. Comparing or indexing them together silently produces invalid similarity results.

Impact

Jaccard, merge, union, LSH, async LSH, LSH Forest, and LSH Bloom now fail clearly when known construction metadata differs. WeightedMinHash remains exempt, and old indexes carrying only scheme metadata remain readable.

Root cause

Compatibility checks treated the seed as a complete identity for permutation generation and did not track the input hash function.

Checks

  • uv run pytest -q test/test_minhash.py test/test_minhash_schemes.py test/test_lsh.py test/test_lshforest.py (119 passed)
  • uv run pytest -q test/test_lshbloom.py (9 skipped; optional dependency unavailable)
  • uv run ruff check on changed Python files
  • uvx pyright (0 errors)
  • git diff --check

@codecov-commenter

codecov-commenter commented Jul 19, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (master@3da96a5). Learn more about missing BASE report.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #326   +/-   ##
=========================================
  Coverage          ?   71.75%           
=========================================
  Files             ?       21           
  Lines             ?     2857           
  Branches          ?        0           
=========================================
  Hits              ?     2050           
  Misses            ?      807           
  Partials          ?        0           
Flag Coverage Δ
unittests 71.75% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants