Skip to content

fix(google_gemini): fix non-streaming response persistence and add usage tracking#170

Open
olivier-lacroix wants to merge 1 commit into
owndev:mainfrom
olivier-lacroix:fix/non-streaming-disappearing-message
Open

fix(google_gemini): fix non-streaming response persistence and add usage tracking#170
olivier-lacroix wants to merge 1 commit into
owndev:mainfrom
olivier-lacroix:fix/non-streaming-disappearing-message

Conversation

@olivier-lacroix
Copy link
Copy Markdown
Contributor

This PR fixes the issue where model responses would disappear from the browser chat window when streaming was disabled.

Problem:

In non-streaming mode, returning a dictionary (OpenAI-compatible format) was leading to empty content fields in subsequent background events (like chat:outlet or follow-up question generation). When these events reached the browser, they would overwrite the previously displayed response with an empty string, causing it to vanish.

Solution:

  1. Return a String: Switched the return type of the pipe method to a simple string for non-streaming calls. Open WebUI recognizes this as a finalized message and definitively sets the message content, preventing background tasks from overwriting it.
  2. Usage Tracking: Since we are now returning a string, we manually emit the usage data via the __event_emitter__ right before the return. This ensures token counts are still captured and saved to the database.
  3. Avoid Duplication: Removed manual chat:message event emission in the non-streaming path as the string return already handles UI display correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant