perf(ai): bound stream parser buffers by JustYannicc · Pull Request #603 · SuperCmdLabs/SuperCmd

JustYannicc · 2026-07-05T01:59:01Z

What changed

Replaced per-chunk toString() decoding in AI SSE and NDJSON streams with streaming UTF-8 TextDecoder handling.
Added AI_STREAM_PARSER_MAX_BUFFER_CHARS at 4 MiB to cap retained incomplete stream lines.
Added scripts/test-ai-provider-stream-parser-bounds.mjs for split UTF-8, valid long payloads, and bounded incomplete-line behavior across SSE and NDJSON.

Why

A provider that split a multi-byte character across chunks could corrupt streamed JSON text under per-chunk decoding.
A malformed provider could send an unterminated SSE/NDJSON line indefinitely and grow the retained parser buffer without bound.

Compatibility impact

Provider extractors and request lifecycle behavior are unchanged for OpenAI, OpenAI-compatible, Anthropic, Gemini, Ollama, and LM Studio routes.
Valid completed long-ish provider payloads below the 4 MiB retained-buffer limit continue to stream.
Malformed streams that exceed 4 MiB without a line break now fail with a bounded parser error instead of retaining more memory.

How tested

node --test scripts/test-ai-provider-stream-parser-bounds.mjs
node --test scripts/test-ai-provider-gemini-prompt-streaming.mjs
node --test scripts/test-ai-stream-ipc-batching.mjs
node --test scripts/test-raycast-ai-abort-listeners.mjs
node --test scripts/test-ai-provider-gemini-prompt-streaming.mjs scripts/test-ai-stream-ipc-batching.mjs scripts/test-raycast-ai-abort-listeners.mjs scripts/test-ai-provider-stream-parser-bounds.mjs
node /opt/homebrew/lib/node_modules/npm/bin/npm-cli.js run build:main
Codex LSP diagnostics on src/main/ai-provider.ts: no diagnostics found.

Performance evidence

New harness diagnostics show legacy per-chunk decoding does not round-trip split UTF-8 for SSE or NDJSON, while streaming TextDecoder does.
New harness diagnostics show legacy retained incomplete SSE/NDJSON text could grow past 4,194,305 characters; bounded parser retention stops at 4,194,304 characters and rejects over-limit incomplete lines.
Existing IPC batching evidence remains intact: 600 provider chunks coalesce to 1 IPC chunk event in the focused batching harness.

Stack validation

Verified starting git rev-parse HEAD was ac90ce403aeddab98d0a1b245afce268c48b8bfd before branching from codex/perf-wave4-integrated-stack-20260705.
Baseline dependency state: node_modules was absent, so TS-import harnesses could not use local esbuild until dependencies were installed.
npm ci through the local npm shim was blocked by machine policy; direct Homebrew npm ci reached dependency scripts but nested npm calls hit the same shim. Installed locked dependencies with node /opt/homebrew/lib/node_modules/npm/bin/npm-cli.js ci --ignore-scripts for focused tests.

Replaces

None.

(cherry picked from commit 39b567a)

(cherry picked from commit 23ddb55)

(cherry picked from commit 59166b4)

(cherry picked from commit 83c2a89)

(cherry picked from commit 0a7560b)

(cherry picked from commit 17fb535)

… churn

# Conflicts: # src/renderer/src/raycast-api/hooks/use-cached-promise.ts

…launcher surfaces # Conflicts: # src/renderer/src/raycast-api/list-runtime-renderers.tsx # src/renderer/src/raycast-api/list-runtime.tsx

…IO stalls # Conflicts: # src/renderer/src/CameraExtension.tsx

…verhead

…istry churn

… overhead

…nd icon churn

# Conflicts: # src/renderer/src/raycast-api/list-runtime.tsx

…resh overhead # Conflicts: # scripts/bench-script-command-discovery.mjs # scripts/benchmark-file-search-delete-batch.mjs # scripts/lib/script-command-runner-harness.mjs # scripts/lib/ts-import.mjs # scripts/test-file-search-delete-batch.mjs # scripts/test-script-command-runner.mjs # src/main/file-search-index.ts # src/main/main.ts

…hurn # Conflicts: # scripts/lib/script-command-runner-harness.mjs # scripts/lib/ts-import.mjs # scripts/test-extension-lifecycle-sandbox.mjs # scripts/test-script-command-runner.mjs # src/main/main.ts # src/renderer/src/ExtensionView.tsx

… hot paths # Conflicts: # scripts/test-use-frecency-sorting.mjs # src/renderer/src/raycast-api/form-runtime.tsx # src/renderer/src/raycast-api/grid-runtime-hooks.ts # src/renderer/src/raycast-api/grid-runtime-items.tsx # src/renderer/src/raycast-api/hooks/use-frecency-sorting.ts

# Conflicts: # .github/workflows/project-checks.yml # package.json # scripts/file-search-perf-harness.mjs # scripts/test-browser-search-performance.mjs # scripts/test-file-search-perf-harness.mjs # scripts/test-root-search-perf.mjs # src/renderer/src/CameraExtension.tsx # src/renderer/src/hooks/useBrowserSearch.ts # src/renderer/src/raycast-api/list-runtime.tsx

…cle and media fixes # Conflicts: # scripts/lib/ts-import.mjs # scripts/measure-extension-bundle-cache.mjs # src/main/commands.ts # src/main/extension-runner.ts # src/main/main.ts # src/renderer/src/ExtensionView.tsx # src/renderer/src/raycast-api/grid-runtime-hooks.ts # src/renderer/src/raycast-api/list-runtime-hooks.ts

JustYannicc · 2026-07-05T10:27:25Z

Superseded by the consolidated runtime/lifecycle/media PR: #622

JustYannicc added 30 commits July 3, 2026 22:19

Batch AI chat stream updates

5ae51b3

Batch cursor prompt stream updates

18481a2

perf(ai): batch Raycast useAI streaming

be66d99

ci: add project regression checks

a1d8c7b

Add bundled TS import helper for tests

a1225c6

fix(types): make renderer typecheck clean

0799e71

ci(claude): fix review action on fork PRs

835bf3a

fix(i18n): restore Italian parity checks

4605c88

fix(pnpm): approve native build scripts

d4c6a9f

perf: cache local icon asset existence checks

986c87a

Virtualize emoji grid list rendering

9e4eed3

Memoize Phosphor icon resolution

5245253

Cache emoji caret session state

cfe2ec3

Stabilize list item registry order

83d6b0b

perf(launcher): window command list rendering

e03ad3e

perf(grid): virtualize grid rendering

f07b554

perf(settings): coalesce AI status polling

6f06eb4

perf(icons): coalesce file icon IPC

949a6fd

Remove root search ranking diagnostics

8ca7ec9

perf: optimize file search delete batches

fb857f2

perf: memoize frecency sorting defaults

dee7108

Avoid interval file-index rebuilds

68cb050

Avoid full script reads for command headers

9f246ce

Optimize root search command scoring

12f951e

perf(runtime): reduce extension and menubar churn

a2737ce

Speed up path-like file search queries

7ed6c2a

Add file search perf harness

dff8539

perf(browser-search): index hot query path

caa78e1

fix(browser-search): narrow indexed nickname candidates

321d103

test: trim benchmark script eof whitespace

644e72d

JustYannicc added 27 commits July 5, 2026 02:26

fix(raycast): guard registry microtask teardown

d89cd33

perf: target extension command refresh

7905622

(cherry picked from commit 39b567a)

fix: clean up extension spawn lifecycle

977dd98

(cherry picked from commit 23ddb55)

Fix extension fetch proxy cancellation

ce93d8d

(cherry picked from commit 59166b4)

fix: clean up main abort listeners

c448044

(cherry picked from commit 83c2a89)

perf: stream cloud media buffers

a5e7148

(cherry picked from commit 0a7560b)

fix(renderer): validate Raycast wave4 consolidation

a18e78b

fix(raycast): guard registry microtask teardown

aac9a64

(cherry picked from commit 17fb535)

fix(validation): finish consolidated checks

0635377

fix: align runtime consolidation integration

b4e5dab

Merge PR SuperCmdLabs#530 perf(runtime): reduce extension and menubar…

0fb756d

… churn

Merge PR SuperCmdLabs#531 perf(ai): batch streaming and status updates

01eab4b

Merge PR SuperCmdLabs#532 fix(validation): restore project checks

38fe371

# Conflicts: # src/renderer/src/raycast-api/hooks/use-cached-promise.ts

Merge PR SuperCmdLabs#533 perf(renderer): cache icons and virtualize …

1adbb01

…launcher surfaces # Conflicts: # src/renderer/src/raycast-api/list-runtime-renderers.tsx # src/renderer/src/raycast-api/list-runtime.tsx

Merge PR SuperCmdLabs#534 perf(mac): reduce native polling and async …

81814e6

…IO stalls # Conflicts: # src/renderer/src/CameraExtension.tsx

Merge PR SuperCmdLabs#535 perf(search): reduce indexing and ranking o…

b34c6fb

…verhead

Merge PR SuperCmdLabs#555 perf(raycast): reduce renderer hook and reg…

bef5071

…istry churn

Merge PR SuperCmdLabs#556 perf(ai): reduce streaming and native media…

36d7f28

… overhead

Merge PR SuperCmdLabs#558 perf(runtime): reduce extension lifecycle a…

d3e0bcc

…nd icon churn

Merge PR SuperCmdLabs#572 perf(renderer): reduce UI runtime churn

6da8e78

# Conflicts: # src/renderer/src/raycast-api/list-runtime.tsx

chore: finalize performance integration stack

ac90ce4

perf(ai): bound stream parser buffers

d866fdf

JustYannicc mentioned this pull request Jul 5, 2026

perf(runtime): consolidate lifecycle and media hot paths #622

Open

JustYannicc closed this Jul 5, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

perf(ai): bound stream parser buffers - #603

perf(ai): bound stream parser buffers#603
JustYannicc wants to merge 121 commits into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-ai-stream-parser-bounds

JustYannicc commented Jul 5, 2026

Uh oh!

JustYannicc commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

JustYannicc commented Jul 5, 2026

What changed

Why

Compatibility impact

How tested

Performance evidence

Stack validation

Replaces

Uh oh!

JustYannicc commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant