Skip to content

perf(ai): batch stream IPC chunks - #537

Closed
JustYannicc wants to merge 1 commit into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-ai-stream-ipc-batching-pr
Closed

perf(ai): batch stream IPC chunks#537
JustYannicc wants to merge 1 commit into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-ai-stream-ipc-batching-pr

Conversation

@JustYannicc

Copy link
Copy Markdown
Collaborator

What changed

  • Added a small main-process AI stream IPC coalescer that batches provider chunks for ai-stream-chunk sends on a short interval.
  • Wired ai-ask and ai-chat through the coalescer, with synchronous flushes before ai-stream-done, ai-stream-error, cancellation cleanup, and same-request replacement.
  • Kept Ollama model pull cancellation behavior by moving pull controllers into a separate request map while ai-cancel still aborts them.
  • Added focused regression/perf coverage for burst batching and done/error/cancel/replacement flush ordering.

Why

Renderer-side AI updates were already batched, but the main process still emitted one IPC event per provider chunk. Long streams could therefore produce hundreds of ai-stream-chunk IPC messages even when renderer state commits were coalesced.

Compatibility impact

  • Existing renderer APIs, request IDs, final concatenated text, error handling, and abort behavior are preserved.
  • Chunk boundaries may be larger because adjacent provider chunks can be coalesced, but content order is unchanged.
  • No provider-specific behavior changes for OpenAI, Anthropic, Gemini, Ollama, OpenAI-compatible, or LM Studio streams.

Metrics

  • Focused baseline: 600 provider chunks -> 600 ai-stream-chunk IPC sends.
  • Batched burst path: 600 provider chunks -> 1 ai-stream-chunk IPC send.
  • The focused test asserts identical final concatenated text for baseline and batched paths.

How tested

On the clean PR branch from origin/main:

  • node --test scripts/test-ai-stream-ipc-batching.mjs -> 5 pass.
  • ./node_modules/.bin/tsc -p tsconfig.main.json --noEmit -> pass.
  • LSP diagnostics for src/main/ai-stream-ipc.ts and src/main/main.ts -> no diagnostics.
  • node --test 'scripts/test-*.mjs' -> 29 pass, 1 skipped.

On the integration-stack worker branch before clean cherry-pick:

  • pnpm test -> 166 pass, 1 skipped live Electron test.
  • ./node_modules/.bin/tsc -p tsconfig.main.json --noEmit -> pass.
  • ./node_modules/.bin/tsc -p tsconfig.renderer.json --noEmit -> pass.

Stack validation / dependency notes

This PR is a clean cherry-pick onto upstream main so it contains only the AI stream IPC batching change. Full pnpm test and renderer typecheck on upstream main are currently blocked by unrelated stack dependencies that are present on the integration branch but not included here:

  • pnpm test stops before running tests with ERR_PNPM_IGNORED_BUILDS because upstream main lacks the native build-script approvals.
  • Renderer typecheck reports existing upstream-main errors unrelated to this main-process IPC change.

@JustYannicc

Copy link
Copy Markdown
Collaborator Author

Superseded by #556, which consolidates this AI/native media performance batch.

@JustYannicc JustYannicc closed this Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant