Prewarm API connections while recording + latency instrumentation

[swimmer-303]

Split out from #254. Stacked on #263 (the timeout fix) since both touch LLMAPITransport session handling — this diff will show the timeout changes too until #263 merges, then it reduces to just the prewarming + instrumentation.

Prewarm API connections while recording
Transcription uploads deliberately use a fresh URLSession per upload, which put a full DNS + TCP + TLS handshake (~100–300 ms) on the critical stop-to-paste path of every dictation. A one-shot session is now created and its connection opened when recording starts; the next upload to the same host consumes it — still exactly one session per upload, so the poisoning defense is unchanged. The shared data session is warmed the same way for post-processing/context requests.

Latency instrumentation
Transcription uploads and LLM requests now log durations to the existing com.zachlatta.freeflow os_log subsystem, so pipeline latency is visible in Console.

🤖 Generated with Claude Code

Summary by CodeRabbit

• Improvements

  • Reduced the delay between stopping dictation and receiving pasted text by preparing connections in advance.
  • Added timing details for transcription uploads and post-processing requests to help diagnose performance issues.

• Bug Fixes

  • Corrected timeout handling so configured transcription, post-processing, and context timeouts are honored beyond 30 seconds.
  • Improved reliability for longer transfers, including requests to slower local models.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds timeout-aware URL session reuse, API connection prewarming during recording startup, configurable service timeouts, request-duration logging, and release notes describing latency and timeout fixes.

Changes

Transport and recording pipeline

Layer / File(s)	Summary
Timeout-aware sessions and prewarming Sources/LLMAPITransport.swift	Replaces the fixed request session with timeout-keyed shared sessions and adds host-matched upload and shared connection prewarming.
Service timeout configuration and request timing Sources/TranscriptionService.swift, Sources/PostProcessingService.swift	Adds configurable timeout accessors and logs elapsed transcription and post-processing request durations.
Recording startup integration and release notes Sources/AppState.swift, CHANGELOG.md	Prewarms transcription and post-processing connections when recording begins and documents the latency, logging, and timeout changes.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AppState
  participant LLMAPITransport
  participant TranscriptionService
  participant PostProcessingService
  participant URLSession
  AppState->>LLMAPITransport: prewarm transcription upload connection
  AppState->>LLMAPITransport: prewarm post-processing shared connection
  LLMAPITransport->>URLSession: send asynchronous HEAD handshakes
  TranscriptionService->>LLMAPITransport: upload audio with configured timeout
  PostProcessingService->>LLMAPITransport: send processing request with configured timeout
  LLMAPITransport->>URLSession: reuse timeout-matched sessions

Loading

Possibly related PRs

• zachlatta/freeflow#85: Introduces the transport abstraction extended here with shared sessions and connection prewarming.
• zachlatta/freeflow#216: Also changes transcription and post-processing timeout configuration through UserDefaults.
• zachlatta/freeflow#219: Overlaps with this PR’s transcription timeout behavior changes.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is clear, concise, and matches the main changes: prewarming API connections during recording and adding latency instrumentation.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

⚔️ Resolve merge conflicts

• Resolve merge conflict in branch split/prewarm-latency

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (2)

Sources/LLMAPITransport.swift (1)

27-29: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Minor duplication of the resource-timeout clamp.

resourceTimeout(for:) and prewarmUploadConnection's let timeout = max(expectedRequestTimeout, minimumResourceTimeout) compute the identical clamp from two different inputs. Consider a single helper, e.g. clampedResourceTimeout(_ interval: TimeInterval) -> TimeInterval, called from both resourceTimeout(for:) and prewarmUploadConnection, to keep the floor logic in one place.

Also applies to: 54-56

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/LLMAPITransport.swift` around lines 27 - 29, Extract the shared
minimum-timeout clamp into a helper such as clampedResourceTimeout(_:), then
update resourceTimeout(for:) and prewarmUploadConnection to call it instead of
using max(..., minimumResourceTimeout) directly.

Sources/PostProcessingService.swift (1)

469-477: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Duplicate timing/logging block between process and processCommandTransform.

The t0/os_log completion-timing block is identical in both methods. Since this file already duplicates most of the request-building logic between these two methods, consider extracting a small helper (e.g. LLMAPITransport.data(for:) wrapped in a timedLLMRequest(_:) on this service) that performs the request and logs elapsed time/model in one place, to avoid a third copy if another call site is added later.

Also applies to: 608-616

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@Sources/PostProcessingService.swift` around lines 469 - 477, Extract the
duplicated timed LLM request logic from process and processCommandTransform into
a shared helper such as timedLLMRequest(_:), using LLMAPITransport.data(for:)
internally and logging elapsed time with the model. Replace both existing
t0/os_log blocks with calls to this helper, preserving their returned data and
response behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@Sources/LLMAPITransport.swift`:
- Around line 27-29: Extract the shared minimum-timeout clamp into a helper such
as clampedResourceTimeout(_:), then update resourceTimeout(for:) and
prewarmUploadConnection to call it instead of using max(...,
minimumResourceTimeout) directly.

In `@Sources/PostProcessingService.swift`:
- Around line 469-477: Extract the duplicated timed LLM request logic from
process and processCommandTransform into a shared helper such as
timedLLMRequest(_:), using LLMAPITransport.data(for:) internally and logging
elapsed time with the model. Replace both existing t0/os_log blocks with calls
to this helper, preserving their returned data and response behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9a9127a5-9226-4844-b78b-a25afc5644ca

📥 Commits

Reviewing files that changed from the base of the PR and between 7427ca9 and f429cde.

📒 Files selected for processing (5)

• CHANGELOG.md
• Sources/AppState.swift
• Sources/LLMAPITransport.swift
• Sources/PostProcessingService.swift
• Sources/TranscriptionService.swift