perf(files): reduce tombstone query overhead by JustYannicc · Pull Request #617 · SuperCmdLabs/SuperCmd

JustYannicc · 2026-07-05T09:03:24Z

What changed

Compacts deleted-heavy primary prefix buckets after watcher tombstone batches, gated by tombstone count and bucket density so broad low-value buckets stay untouched.
Tracks exact deleted normalized names on the active snapshot and skips macOS Spotlight fallback for exact deleted-name queries with no live indexed result.
Extends delete-batch tests and benchmark output to cover post-delete query cost and touched bucket size.

Why

Post-delete queries could spend tens of milliseconds after watcher delete batches because tombstoned IDs remained in prefix buckets and exact deleted-name misses fell through to Spotlight. The new path keeps dense tombstone buckets lean and avoids asking Spotlight to rediscover a name the index just tombstoned.

Compatibility impact

Raycast-facing result shapes and delete semantics are preserved: exact file deletes, directory descendant deletes, unknown-root descendant tombstones, prefix-like sibling safety, and nested delete collapse still pass. The Spotlight skip is limited to exact deleted-name tombstones when the live index has no result.

How tested

node scripts/test-file-search-delete-batch.mjs
node scripts/test-file-search-index.mjs
node scripts/test-file-search-perf-harness.mjs
./node_modules/.bin/tsc -p tsconfig.main.json --noEmit --pretty false
npm run build:main via real cached npm CLI
npm run build:renderer via real cached npm CLI
node scripts/build-native.mjs
npm test via real cached npm CLI
LSP diagnostics for src/main/file-search-index.ts and workspace diagnostics; LSP references checked for tombstoneDeletedPaths
git diff --check

Note: top-level npm run build was also attempted, but this machine's nested npm resolution fails before project build steps with Cannot find module ... npm-prefix.js; running the component build steps directly passed.

Performance evidence

Parent-sized harness (node scripts/file-search-perf-harness.mjs --projects 40 --modules 6 --files-per-module 10 --query-runs 8 --updates 64 --deletes 64 --limit 20 --json):

Before: post-delete query 39.066 ms; delete batch 3.569 ms; normal query p95 3.895 ms; path-like p95 4.270 ms.
After: post-delete query 2.583 ms; delete batch 3.597 ms; normal query p95 3.511 ms; path-like p95 3.875 ms.

Larger delete-heavy harness (--projects 80 --modules 8 --files-per-module 16 --query-runs 6 --updates 96 --deletes 256 --limit 20 --json):

Before: post-delete query 33.386 ms; delete batch 5.257 ms; event-loop lag p95 2.109 ms.
After: post-delete query 2.645 ms; delete batch 5.575 ms; event-loop lag p95 1.915 ms.

Focused delete benchmark after change (node scripts/benchmark-file-search-delete-batch.mjs):

direct-file-batch: 80,161 entries, 500 delete paths, 500 deleted, delete 5.997 ms, post-delete query 8.599 ms, touched bucket 80,161 -> 80,161.
directory-tree-batch: 68,162 entries, 4,041 delete paths, 8,041 deleted, delete 13.509 ms, post-delete query 15.947 ms, touched bucket 8,041 -> 0.

Stack validation

Base was exact ac90ce403aeddab98d0a1b245afce268c48b8bfd from codex/perf-wave4-integrated-stack-20260705. Branch codex/perf-file-index-tombstones-wave5 contains one commit: 90df743 perf(files): reduce tombstone query overhead.

Replaces

N/A

(cherry picked from commit 39b567a)

(cherry picked from commit 23ddb55)

(cherry picked from commit 59166b4)

(cherry picked from commit 83c2a89)

(cherry picked from commit 0a7560b)

(cherry picked from commit 17fb535)

… churn

# Conflicts: # src/renderer/src/raycast-api/hooks/use-cached-promise.ts

…launcher surfaces # Conflicts: # src/renderer/src/raycast-api/list-runtime-renderers.tsx # src/renderer/src/raycast-api/list-runtime.tsx

…IO stalls # Conflicts: # src/renderer/src/CameraExtension.tsx

…verhead

…istry churn

… overhead

…nd icon churn

# Conflicts: # src/renderer/src/raycast-api/list-runtime.tsx

…resh overhead # Conflicts: # scripts/bench-script-command-discovery.mjs # scripts/benchmark-file-search-delete-batch.mjs # scripts/lib/script-command-runner-harness.mjs # scripts/lib/ts-import.mjs # scripts/test-file-search-delete-batch.mjs # scripts/test-script-command-runner.mjs # src/main/file-search-index.ts # src/main/main.ts

…hurn # Conflicts: # scripts/lib/script-command-runner-harness.mjs # scripts/lib/ts-import.mjs # scripts/test-extension-lifecycle-sandbox.mjs # scripts/test-script-command-runner.mjs # src/main/main.ts # src/renderer/src/ExtensionView.tsx

… hot paths # Conflicts: # scripts/test-use-frecency-sorting.mjs # src/renderer/src/raycast-api/form-runtime.tsx # src/renderer/src/raycast-api/grid-runtime-hooks.ts # src/renderer/src/raycast-api/grid-runtime-items.tsx # src/renderer/src/raycast-api/hooks/use-frecency-sorting.ts

# Conflicts: # .github/workflows/project-checks.yml # package.json # scripts/file-search-perf-harness.mjs # scripts/test-browser-search-performance.mjs # scripts/test-file-search-perf-harness.mjs # scripts/test-root-search-perf.mjs # src/renderer/src/CameraExtension.tsx # src/renderer/src/hooks/useBrowserSearch.ts # src/renderer/src/raycast-api/list-runtime.tsx

…cle and media fixes # Conflicts: # scripts/lib/ts-import.mjs # scripts/measure-extension-bundle-cache.mjs # src/main/commands.ts # src/main/extension-runner.ts # src/main/main.ts # src/renderer/src/ExtensionView.tsx # src/renderer/src/raycast-api/grid-runtime-hooks.ts # src/renderer/src/raycast-api/list-runtime-hooks.ts

JustYannicc · 2026-07-05T10:05:42Z

Superseded by consolidated PR #620: #620

JustYannicc added 30 commits July 3, 2026 22:19

Batch AI chat stream updates

5ae51b3

Batch cursor prompt stream updates

18481a2

perf(ai): batch Raycast useAI streaming

be66d99

ci: add project regression checks

a1d8c7b

Add bundled TS import helper for tests

a1225c6

fix(types): make renderer typecheck clean

0799e71

ci(claude): fix review action on fork PRs

835bf3a

fix(i18n): restore Italian parity checks

4605c88

fix(pnpm): approve native build scripts

d4c6a9f

perf: cache local icon asset existence checks

986c87a

Virtualize emoji grid list rendering

9e4eed3

Memoize Phosphor icon resolution

5245253

Cache emoji caret session state

cfe2ec3

Stabilize list item registry order

83d6b0b

perf(launcher): window command list rendering

e03ad3e

perf(grid): virtualize grid rendering

f07b554

perf(settings): coalesce AI status polling

6f06eb4

perf(icons): coalesce file icon IPC

949a6fd

Remove root search ranking diagnostics

8ca7ec9

perf: optimize file search delete batches

fb857f2

perf: memoize frecency sorting defaults

dee7108

Avoid interval file-index rebuilds

68cb050

Avoid full script reads for command headers

9f246ce

Optimize root search command scoring

12f951e

perf(runtime): reduce extension and menubar churn

a2737ce

Speed up path-like file search queries

7ed6c2a

Add file search perf harness

dff8539

perf(browser-search): index hot query path

caa78e1

fix(browser-search): narrow indexed nickname candidates

321d103

test: trim benchmark script eof whitespace

644e72d

JustYannicc added 27 commits July 5, 2026 02:26

fix(raycast): guard registry microtask teardown

d89cd33

perf: target extension command refresh

7905622

(cherry picked from commit 39b567a)

fix: clean up extension spawn lifecycle

977dd98

(cherry picked from commit 23ddb55)

Fix extension fetch proxy cancellation

ce93d8d

(cherry picked from commit 59166b4)

fix: clean up main abort listeners

c448044

(cherry picked from commit 83c2a89)

perf: stream cloud media buffers

a5e7148

(cherry picked from commit 0a7560b)

fix(renderer): validate Raycast wave4 consolidation

a18e78b

fix(raycast): guard registry microtask teardown

aac9a64

(cherry picked from commit 17fb535)

fix(validation): finish consolidated checks

0635377

fix: align runtime consolidation integration

b4e5dab

Merge PR SuperCmdLabs#530 perf(runtime): reduce extension and menubar…

0fb756d

… churn

Merge PR SuperCmdLabs#531 perf(ai): batch streaming and status updates

01eab4b

Merge PR SuperCmdLabs#532 fix(validation): restore project checks

38fe371

# Conflicts: # src/renderer/src/raycast-api/hooks/use-cached-promise.ts

Merge PR SuperCmdLabs#533 perf(renderer): cache icons and virtualize …

1adbb01

…launcher surfaces # Conflicts: # src/renderer/src/raycast-api/list-runtime-renderers.tsx # src/renderer/src/raycast-api/list-runtime.tsx

Merge PR SuperCmdLabs#534 perf(mac): reduce native polling and async …

81814e6

…IO stalls # Conflicts: # src/renderer/src/CameraExtension.tsx

Merge PR SuperCmdLabs#535 perf(search): reduce indexing and ranking o…

b34c6fb

…verhead

Merge PR SuperCmdLabs#555 perf(raycast): reduce renderer hook and reg…

bef5071

…istry churn

Merge PR SuperCmdLabs#556 perf(ai): reduce streaming and native media…

36d7f28

… overhead

Merge PR SuperCmdLabs#558 perf(runtime): reduce extension lifecycle a…

d3e0bcc

…nd icon churn

Merge PR SuperCmdLabs#572 perf(renderer): reduce UI runtime churn

6da8e78

# Conflicts: # src/renderer/src/raycast-api/list-runtime.tsx

chore: finalize performance integration stack

ac90ce4

perf(files): reduce tombstone query overhead

90df743

JustYannicc mentioned this pull request Jul 5, 2026

perf(commands): consolidate command search validation #620

Open

JustYannicc closed this Jul 5, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf(files): reduce tombstone query overhead - #617

perf(files): reduce tombstone query overhead#617
JustYannicc wants to merge 121 commits into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-file-index-tombstones-wave5

JustYannicc commented Jul 5, 2026

Uh oh!

JustYannicc commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

JustYannicc commented Jul 5, 2026

What changed

Why

Compatibility impact

How tested

Performance evidence

Stack validation

Replaces

Uh oh!

JustYannicc commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant