Skip to content

perf: stream cloud media buffers - #593

Closed
JustYannicc wants to merge 115 commits into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-wave4-cloud-media-buffering-20260704
Closed

perf: stream cloud media buffers#593
JustYannicc wants to merge 115 commits into
SuperCmdLabs:mainfrom
JustYannicc:codex/perf-wave4-cloud-media-buffering-20260704

Conversation

@JustYannicc

Copy link
Copy Markdown
Collaborator

What changed

  • Streams ElevenLabs and Mistral Voxtral STT multipart uploads by writing the existing multipart parts sequentially instead of concatenating a full request body.
  • Streams successful ElevenLabs TTS MP3 responses directly to the target temp file while keeping HTTP error text bounded.
  • Adds focused cloud media buffering tests with before/after byte diagnostics for STT multipart copies and TTS response buffering.

Why

  • Wave4 closes the remaining cloud STT/TTS media buffering gaps left after the optimized OpenAI Whisper upload path.
  • Long Read/TTS sessions and large STT captures should avoid retaining avoidable full-body copies in the Electron main process.

Compatibility impact

  • Preserves provider endpoints, request field order, request headers, multipart Content-Length accuracy, API key handling, Mistral timeout behavior, JSON/text parsing branches, and successful output semantics.
  • ElevenLabs TTS still rejects empty audio and now removes partial streamed files on stream/empty-audio failure.
  • Non-2xx ElevenLabs TTS/STT detected_unusual_activity messaging is preserved; error messages continue to expose the first 500 response characters.

How tested

  • node --test scripts/test-cloud-media-buffering.mjs
  • node --test scripts/test-ai-provider-whisper-upload.mjs
  • node --test scripts/test-whisper-file-buffer-read.mjs
  • ./node_modules/.bin/tsc -p tsconfig.main.json --noEmit --pretty false
  • Codex LSP diagnostics for src/main/main.ts: no diagnostics found.

Performance evidence

  • ElevenLabs STT test: legacy full multipart body copy would allocate 6,291,868 bytes; streamed path writes 6 buffers, reuses the 6,291,456 byte audio Buffer, and allocates 412 framing bytes.
  • Mistral Voxtral STT test: legacy full multipart body copy would allocate 7,340,475 bytes; streamed path writes 6 buffers, reuses the 7,340,032 byte audio Buffer, and allocates 443 framing bytes.
  • ElevenLabs TTS test: legacy response handling would retain a 5,767,168 byte MP3 Buffer before fs.writeFile; streamed path writes 5,767,168 bytes directly to a 5,767,168 byte temp file without a provider-frame Buffer.concat.

Stack validation

  • Started from codex/perf-wave3-integration-stack-20260704 at commit 206eab8280a2eb91a57e4e74ea6df99e45a55cb4.
  • Implementation branch: codex/perf-wave4-cloud-media-buffering-20260704.
  • No coordinator markdown, playbooks, prompts, or generated locks are included.

Replaces

  • Builds on and replaces the wave3 integration stack for the cloud media buffering portion of the perf work.

…perf-wave2-integration-stack-20260704

# Conflicts:
#	src/renderer/src/raycast-api/hooks/use-cached-promise.ts
…perf-wave2-integration-stack-20260704

# Conflicts:
#	src/renderer/src/raycast-api/list-runtime-renderers.tsx
#	src/renderer/src/raycast-api/list-runtime.tsx
…perf-wave2-integration-stack-20260704

# Conflicts:
#	src/renderer/src/CameraExtension.tsx
…perf-wave2-integration-stack-20260704

# Conflicts:
#	src/renderer/src/raycast-api/list-runtime.tsx
…perf-wave2-integration-stack-20260704

# Conflicts:
#	scripts/bench-script-command-discovery.mjs
#	scripts/benchmark-file-search-delete-batch.mjs
#	scripts/lib/script-command-runner-harness.mjs
#	scripts/lib/ts-import.mjs
#	scripts/test-file-search-delete-batch.mjs
#	scripts/test-script-command-runner.mjs
#	src/main/file-search-index.ts
#	src/main/main.ts
…perf-wave2-integration-stack-20260704

# Conflicts:
#	scripts/lib/script-command-runner-harness.mjs
#	scripts/lib/ts-import.mjs
#	scripts/test-extension-lifecycle-sandbox.mjs
#	scripts/test-script-command-runner.mjs
#	src/main/main.ts
#	src/renderer/src/ExtensionView.tsx
…perf-wave2-integration-stack-20260704

# Conflicts:
#	.github/workflows/project-checks.yml
#	scripts/test-root-search-perf.mjs
#	src/renderer/src/CameraExtension.tsx
#	src/renderer/src/raycast-api/list-runtime.tsx
…erf-wave3-integration-stack-20260704

# Conflicts:
#	package.json
#	scripts/measure-extension-bundle-cache.mjs
#	scripts/test-icon-runtime-file-icon-cache.mjs
#	src/main/extension-runner.ts
#	src/renderer/src/components/HiddenExtensionRunners.tsx
#	src/renderer/src/raycast-api/icon-runtime-render.tsx
…/perf-wave3-integration-stack-20260704

# Conflicts:
#	scripts/test-browser-search-performance.mjs
#	scripts/test-root-search-perf.mjs
@JustYannicc

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #601, which fully replaces this granular PR in the wave4 runtime/main consolidation. Literal validation now passes with /opt/homebrew/opt/node@22/bin/npm run build:main, and #601 is ready for review.

@JustYannicc JustYannicc closed this Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant