Add robust similarity metrics and normalization to compare.text

[mborodii-prog]

Add robust similarity metrics and normalization to compare.text

Closes #1134

Summary

Adds method: similarity to compare.text — a symmetric, 0.0–1.0 bounded similarity score for comparing product descriptions that may be reordered, differently punctuated, or contain typos. Also fixes several existing issues in difference/intersection/overlap along the way: a real (not deprecated) case_sensitive toggle defaulting to case-insensitive, a cleaner two-column output for overlap's include_ratio, a casing-preservation bug, and a crash on quoted decimal_places.

What's in this PR

New method: similarity with three metrics (wrangles/compare.py):

• token_sort (default) — ignores token order, keeps duplicates, penalizes missing/extra content
• token_set — ignores token order and duplicates; a shorter description fully contained in a longer one can score 1.0
• damerau_levenshtein — character edit distance where an adjacent transposition counts as one edit
• Built on RapidFuzz (fuzz.token_sort_ratio, fuzz.token_set_ratio, distance.DamerauLevenshtein.normalized_similarity) — new dependency: rapidfuzz>=3.0,<4.0
• Standard normalization pipeline (normalize_similarity_text): NFKC Unicode normalization → Unicode-aware case folding → punctuation/separators converted to spaces (not concatenated) → whitespace collapsed. Deliberately does not equate AB-12/AB12, doesn't touch units/numbers, doesn't do synonym/acronym expansion.
• Missing/blank input returns null, never stringified into "nan"/"None" — checked before any str() conversion, unlike the legacy .astype(str) path.
• The metric schema property uses oneOf with a const+description per value instead of a flat enum, so IDE/schema tooltips show each metric's description individually instead of one wall of text under "Allowed Values".

case_sensitive behavior, changed twice over the life of this branch, landed as:

• It's a real, functioning parameter again — not deprecated/ignored.
• Default is now false (case-insensitive) for difference/intersection/overlap — previously defaulted to true (case-sensitive). Pass case_sensitive: true to restore exact-case matching.

overlap with include_ratio: true now requires a two-column output:

• output must be [mask_column, ratio_column] — the mask and ratio are written to separate columns instead of being packed into one cell as [mask, ratio], which was awkward for Excel filtering/numeric operations.
• Raises a clear ValueError if output isn't a two-item list when include_ratio is true. overlap without include_ratio still accepts a single output column name as before.

Bug fixes required to make the above correct, not scope creep:

• Fixed contrast()/overlap() casing: case-insensitive matching was also lowercasing the output in difference/overlap. Matching is case-insensitive when requested, but original casing is now preserved in the output either way.
• Fixed the quoted decimal_places bug (int(decimal_places)'s result was discarded, causing a later TypeError).

Tests

61 tests in tests/recipes/wrangles/test_compare.py (35 pre-existing behavior preserved + new coverage added across the branch): symmetry, score bounds, exact-match, reordered/duplicate/subset tokens, conflicting attributes, each Damerau-Levenshtein edit type individually (insertion/deletion/substitution/transposition), anagram guard, punctuation (AB-12 vs AB12), Unicode normalization, nulls (both library-level None and recipe-level non-stringified), the quoted-decimal_places and casing regression bugs, case_sensitive true/false/default equivalence and divergence, and the new include_ratio output-shape validation (both the error case and the still-works-without-include_ratio case).

Also updated tests/recipes/test_recipes.py: the three tests that were hitting permission-restricted live models (test_recipe_by_production_version, test_recipe_by_version_latest, test_recipe_by_latest_version) now point at newly created replacement model IDs (e954717c-fb9c-4c47, 1b41d016-7129-4b66) instead of being mocked, so they still exercise the real model-version-resolution path end-to-end.

[mborodii-prog]

@ebhills @thomasstvr PR was trested in QA to test it you can use image: dev-1.20.0rc59 . test file
compare_text_data.xlsx
recipe:
compare_text_recipe.wrgl.yml