Repository navigation
docs(b9): say that search folds case over all of Unicode, at every length - #112
Merged
Merged
Conversation
…ngth B9 described two matching arms -- the index for a query long enough to tokenise, a bounded scan for one that is not -- and said nothing about how either compares. Measured in the product, they did not compare alike: the index folds case over the whole of Unicode and the fallback folded ASCII, so a reader typing their merchant's name in the case they see it found the row at three characters and not at two. That is reader-visible behaviour with no requirement over it, so B9-R19 states it: one fold, Unicode-wide, the same in every arm, and case only -- folding case is not stripping accents, and `o` still does not find `ö`. Signed-off-by: Wessel Verheij <info@nightworks.io>
|
lessevv
enabled auto-merge (squash)
September 13, 2026 14:38
github-actions
Bot
added
the
awaiting-maintainer
github-actions
Bot
added
awaiting-maintainer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
B9 describes two matching arms — the index for a query long enough to tokenise, a bounded scan for one that is not — and says nothing about how either compares. Measured in the product, they do not compare alike.
SQLite's
LIKEandLOWER()fold ASCII and nothing else; FTS5's trigram tokeniser folds over the whole of Unicode. The arm is picked by the query's length, so the length of a word decided whether its accents mattered. Over a body containingMÖRK BAR, on the shipped SQLite build:örörkÖRÖRKA reader typing their merchant's name in the case they see it on screen got nothing, and the app offered no reason. That is reader-visible behaviour with no requirement over it.
B9-R19 states the rule: case-insensitive matching folds over the whole of Unicode rather than ASCII alone, applies the same fold in every arm, and does not strip diacritics —
öfindsÖ,ostill does not findö, which is what the index has always done.Also adds the matching bullet to Search behaviour and the edge-case row, so the requirement is not the only place the page says it.
No existing identifier is renumbered or reused.