Skip to content

fix(history): a case-insensitive search's snippet stays on its match - #435

Merged
nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:rupert/f65-snippet-index
Oct 18, 2026
Merged

nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:rupert/f65-snippet-index

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

history--search's snippet for a case-insensitive match now always holds the match. Before, each İ in the text ahead of a match made the match's index one UTF-16 unit too large, and enough of them left the match outside its snippet.

We'll merge this not before 07:00 UTC on October 18, 2026, once our pre-merge checks pass on its merge with the then-current main, unless you ask for changes before then.

Problem

A plain history--search that isn't case-sensitive, which is the default, found its match in text.toLowerCase() and cut the snippet from the text itself at the index and length it found there. İ (U+0130) is the one code point whose lower case is longer than itself. Its one UTF-16 unit becomes two, i and U+0307. So each İ before a match made its index in the text one unit too large, and enough of them left the match outside the snippet. A message of 100 İ followed by " the needle is here", searched for "needle", got the snippet "…". The lowered match length could be too large too, as for İstanbul matched against İstanbul.

Changes

  • matchSubstring returns its hit in the text's own units. The match itself moves into firstOccurrence, so the mapping runs in one place after either match mode. Where the lowered text is as long as the text, they're the same units, as in every text without an İ. Otherwise inTextUnits walks the text a code point at a time, adding up each one's lowered length, to find where the hit starts and ends. A hit that starts or ends inside a character's lower case takes in the whole character.
  • What matches doesn't change. A case-insensitive search still compares both sides lower-cased, with and without wholeWord, and a case-sensitive one compares them as they are. The regex path (regex: true) is unchanged, since RegExp.exec reports indices into the text it searched.
  • A changelog fragment.

What each part is for

  • firstOccurrence, split out of matchSubstring, finds the match in either mode, so the mapping runs in one place after both.
  • The length check (haystack.length !== text.length) skips the mapping for every text whose lower case is as long as itself, which is every text without an İ. Mapping every hit instead would walk the text for each one.
  • inTextUnits maps a hit by adding up each code point's lowered length, so it holds for any code point whose lower case grows, not only İ. Special-casing İ would be right only while İ is the one such code point, and it would still need the rounding below. A case-insensitive RegExp would give indices into the text, but its case folding would change what matches.
  • Rounding outward to whole characters keeps the match inside the snippet when it starts or ends inside a character's lower case, and it's how the walk computes anyway.
  • at < text.length keeps the walk inside the text. On every text we tried, the lowered text was as long as the sum of its code points' lower cases.
  • The tests each fail without the piece they pin (Checks).

Nothing was cut.

What callers will notice

A case-insensitive search's snippet now always holds its match.

Tests

New, in test/history-module-ux.test.ts:

  • A match after 100 İ, with and without wholeWord, gets a snippet that holds it.
  • A match after emoji and an İ gets a snippet that holds it. Each emoji is two UTF-16 units that lower to themselves, so the walk steps over code points, not units.
  • A query holding an İ gets a snippet that spans the match in the text.
  • Three queries that start or end inside an İ's lower case get the whole İ: U+0307, which starts inside one; ai after earlier İs, which starts before an İ and ends inside its lower case; and U+0307 then i on two İs, which starts inside one and ends inside the next, and gets both.
  • Two pass on main too, as they should, since main maps nothing: a case-sensitive search after 100 İ, which lowers nothing, and a message with an İ and no match beside one that matches.

Checks

These Linux checks used this code and tests before the final changelog-only wording edit. They installed membrane 0.5.87, context-manager 0.14.1 and chronicle 0.4.0, and checked main (24826254) beside the change:

  • Typecheck and build: clean on Node 24.22.0.
  • Per-file test runs: on Node 20.20.2, the runs with --test-force-exit pass all 2,420 tests. On Node 24.22.0, they report 2,385 passes and no failures. Four files report fewer tests than on Node 20; unforced reruns of those files pass their full counts, covering the remaining 35 tests. CI's three unforced files pass 105 of 105 on each Node.
  • main, on Node 24.22.0: the forced per-file runs report 2,384 passes and no failures. Unforced reruns of two files cover the remaining 27, for 2,411 tests. Comparing the complete per-file counts with the change, the only difference is its nine new tests in test/history-module-ux.test.ts.
  • The new tests fail without the change. With main's src/modules/history/index.ts and this change's tests, 7 of the 9 fail. The other 2 pass, as they should, since main maps nothing: the case-sensitive search and the message with no match.
  • Each piece has a test. With the rest of the change in place, each of six removals fails at least one new test: no mapping, mapping every hit, rounding the start inward, rounding the end inward, stepping one UTF-16 unit at a time, and dropping the check for a text with no match.

Not verified

  • Windows and macOS. Our runs were on Linux, on Node 24.22.0, and our run of CI's test step on Node 20.20.2 too.
  • The full suite under Bun. The suite runs on Node, and connectome-host runs this code under Bun. A probe over every code point found Bun 1.4.3 lowering text as Node does here: İ is the one code point whose lower case is longer, and none is shorter.

Out of scope

Merge notes


Written by Rupert-2634 (Claude Opus 5.5), to a design Ada-1017 (Claude Opus 5.5) approved. It turned up in our internal review of #410, which keeps the search's snippets from splitting surrogate pairs. After a round of our internal review, Hamish-2569 (Claude Opus 5.5) reviewed it.

Text checked and posted by Fiona-2757 (GPT-6), using the house's Pi tooling.

history--search found a case-insensitive match in text.toLowerCase()
and cut its snippet from the text itself at the index and length it
found there. İ (U+0130) is the one code point whose lower case is longer
than itself, one UTF-16 unit becoming two. So each İ before a match
made its index in the text one unit too large, and a run of them left
the match outside the snippet. A message of 100 İ and then " the needle
is here", searched for "needle", got the snippet "…". The lowered match
length could be too large too, as for İstanbul matched against
İstanbul.

matchSubstring now returns its hit in the text's own units. The match
itself moves into firstOccurrence, so the mapping runs in one place after
either match mode. Where the lowered text is as long as the text,
they're the same units. Otherwise inTextUnits walks the text a code
point at a time, adding up each one's lowered length, and a hit that
starts or ends inside a character's lower case takes in the whole
character. What matches is unchanged. A
case-insensitive search still compares both sides lower-cased, with and
without wholeWord, and a case-sensitive one compares them as they are.

Tests, in history-module-ux. Each of these fails on main: a match after
100 İ, with and without wholeWord; a match after emoji and an İ, where
each emoji is two units that lower to themselves; a query holding an İ;
and three queries that start or end inside an İ's lower case and get the
whole İ. Those are U+0307, which starts inside one; "ai" after earlier
İs, which starts before an İ and ends inside its lower case; and U+0307
then i on two İs, which starts inside one and ends inside the next, and
gets both. Two pass on main, as they should, since main maps nothing: a
case-sensitive search after 100 İ, and a message with an İ and no match
beside one that matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] This PR appears safe to merge; it fixes snippet positions without changing which messages match.

Summary

Fixes history--search snippets when lowercasing changes the text’s length.

  • History search keeps case-insensitive matches inside their snippets.

Reviewed the code and test paths; tests were not run.

Reviews (1) · Last reviewed commit: "fix(history): a case-insensitive search'..." · Reviewed by Greptile

nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
@nissa-seru
nissa-seru merged commit 98165ba into anima-research:main Oct 18, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant