Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,6 @@
## 2025-02-12 - [Fast Path Execution in Directory Traversal and Log Parsing]
**Learning:** Checking for string existence (`if "silence_" not in stderr`) before invoking regex matchers provides significant speed improvements when parsing large blocks of text. Similarly, moving expensive I/O operations like `os.path.realpath` inside conditional blocks prevents redundant disk access when configuration (like path exclusions) isn't utilized.
**Action:** When working on large text processing or disk operations, verify if early exit conditions or conditional execution can bypass the expensive system or library calls.
## 2024-10-08 - [Optimize Search Set Intersection and Loops]
**Learning:** In tight loops involving `set`s (like when computing the intersection of matching documents for multi-word search terms), creating a new `set` with the `&` operator allocates memory and iterates needlessly. Modifying a target `set` in place using `.intersection_update()` avoids this allocation. In the same vein, using `sum()` with a generator expression introduces significant overhead due to object allocation of the generator. Replacing it with a simple `for` loop that accumulates in a local variable provides a substantial speedup.
**Action:** Always prefer `intersection_update` over `&` when working with sets you can modify safely. For performance-sensitive loops, replace `sum(a for b in c)` with a basic inline `for` loop to eliminate the generator overhead.
11 changes: 7 additions & 4 deletions transcript_search.py
Original file line number Diff line number Diff line change
Expand Up @@ -241,16 +241,19 @@ def search(self, query: str) -> list[Match]:
postings = self._postings.get(term)
if not postings:
return []
candidates = (
set(postings) if candidates is None else candidates & postings
)
if candidates is None:
candidates = set(postings)
else:
candidates.intersection_update(postings)
if not candidates:
return []

matches = []
for position in candidates or ():
entry = self._entries[position]
score = sum(entry.counts[term] for term in unique_terms)
score = 0
for term in unique_terms:
score += entry.counts[term]
matches.append(
Match(
recording_id=entry.recording_id,
Expand Down
Loading