Skip to content

fix(grep): document="EX-10.1" selects exactly EX-10.1; literal matches keep source offsets - #1397

Merged
dgunning merged 1 commit into
mainfrom
fix/grep-exact-document-and-literal-offsets
Oct 1, 2026
Merged

dgunning merged 1 commit into
mainfrom
fix/grep-exact-document-and-literal-offsets

Conversation

@dgunning

@dgunning dgunning commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Beads edgartools-5qqy and edgartools-6th2 (salvaged from closed PR #1380).

5qqy: document="EX-10.1" also searched EX-10.10 through EX-10.19

Filing._attachment_matches matched the document type by substring. It's replaced by _select_attachments, which looks at the whole attachment list and applies a precedence: an exact filename wins, then an exact document type, and the substring match applies only when nothing matched exactly. That means document="EX-10" still selects every EX-10 exhibit. PR #1380 only added an exact-filename check ahead of the substring test, which left the type collision in place.

Ground truth, SeeQC's S-1 (0001213900-26-073222): "agreement" in EX-10.1 (a Stockholder Support Agreement) gives 133 matches. The old filter returned 378, adding the EX-10.10/11/12 employment agreements.

6th2: literal grep shifted matches after characters whose lowercase is longer

The literal branch found positions in text.lower() and sliced the original text with them. "İ".lower() is two characters, so each one before a match shifted the result by one. Literal search now runs re.escape(pattern) with IGNORECASE over the original text.

It keeps the old scan's overlapping matches ("aa" in "aaa" gives two) by searching again from m.start() + 1 rather than using finditer, which would have silently dropped them. Regex mode is unchanged.

Verification

  • New fast tests: document selection (exact type, exact filename, substring fallback, primary, no filter, end-to-end grep()) and literal offsets (the İ case, source casing, overlap, regex metacharacters in a literal).
  • Network test: the SeeQC S-1 case, pinned by accession.
  • Against main: the offset test fails on the old grep.py; the end-to-end and SeeQC tests fail on the old _filings.py.
  • Existing tests: the grep tests and the Filing search/grep don't work on text (non-HTML) filings #819 regression tests pass (129). test-fast: the only failure is the dt1f1 20-F test, which also fails on main.
  • Docs: the grep() docstring and docs/guides/full-text-search.md describe the precedence.

🤖 Generated with Claude Code

…s keep source offsets (5qqy, 6th2)

5qqy: Filing._attachment_matches tested document.upper() in doc_type, a
substring match, so "EX-10.1" also selected EX-10.10..EX-10.19. Replaced by
_select_attachments, which sees the whole list: an exact filename wins,
then an exact document type, and the old substring match applies only when
nothing matched exactly, so "EX-10" still selects every EX-10 exhibit.
PR #1380 added only an exact-filename check ahead of the substring test,
which leaves the type collision in place. SeeQC's S-1
(0001213900-26-073222): "agreement" in EX-10.1 is 133 matches; the old
filter returned 378, adding EX-10.10/11/12.

6th2: the literal branch of _grep_text found positions in text.lower() and
sliced the original text with them. "İ".lower() is two characters, so each
one before a match shifted it by one. Literal search now runs
re.escape(pattern) with IGNORECASE over the original text. It keeps the
old scan's overlapping matches ("aa" in "aaa" is two) by searching again
from m.start() + 1 rather than using finditer; regex mode is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dgunning
dgunning enabled auto-merge (squash) October 1, 2026 21:34
@dgunning
dgunning merged commit fc3d0d8 into main Oct 1, 2026
10 checks passed
@dgunning
dgunning deleted the fix/grep-exact-document-and-literal-offsets branch October 1, 2026 21:49
@dgunning dgunning mentioned this pull request Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant