Skip to content

fix: recover reasoning-only compact summaries instead of failing with EmptyResponse - #220

Open
minhtq1234 wants to merge 1 commit into
iOfficeAI:mainfrom
minhtq1234:fix/compact-reasoning-empty-response
Open

fix: recover reasoning-only compact summaries instead of failing with EmptyResponse#220
minhtq1234 wants to merge 1 commit into
iOfficeAI:mainfrom
minhtq1234:fix/compact-reasoning-empty-response

Conversation

@minhtq1234

Copy link
Copy Markdown

Problem

Running /compact (and autocompact) fails with Compact failed: Empty response from LLM when the conversation model is a reasoning model served via an OpenAI‑compatible endpoint. Observed in AionUi with MiniMax M2.5 (custom provider): the model generates for ~17s, then the turn ends with event_type="Error" text_len=0.

Root cause

  • The compact request sets thinking: ThinkingConfig::Disabled (crates/aion-agent/src/compact/auto.rs), but the OpenAI projector only acts on ThinkingConfig::Enabled (crates/aion-providers/src/projector.rs), so "disabled" never reaches the wire — reasoning models reason anyway.
  • The OpenAI-compatible stream parser maps reasoning_content deltas to LlmEvent::ThinkingDelta (crates/aion-providers/src/openai.rs).
  • collect_stream_text in compact/auto.rs accumulated only TextDelta and discarded thinking deltas. A model that emits its whole summary as reasoning yields an empty string → CompactError::EmptyResponse, even though a complete summary was produced (output budget is not the issue — COMPACT_MAX_OUTPUT_TOKENS is 20k).

Fix

collect_stream_text now accumulates thinking deltas alongside text and falls back to the reasoning content when the model produced no plain text. format_compact_summary already degrades gracefully when <summary> tags are absent, so the recovered content flows through the existing parsing unchanged.

Behavior is otherwise preserved:

  • Text output, when present, always wins — reasoning is only used as a fallback.
  • A truly empty response (no text, no reasoning) still fails with EmptyResponse.
  • A channel that closes without Done can now salvage reasoning-only output; if nothing was received it still fails with EmptyResponse as before.

Testing

  • New regression test reasoning_only_response_is_recovered (ThinkingDelta-only stream → compaction succeeds, summary contains the recovered content, failure counter untouched).
  • Existing empty_response_fails still passes (truly empty stream → EmptyResponse).
  • cargo test -p aion-agent — all tests pass.
  • cargo fmt --all -- --check and cargo clippy -p aion-agent -- -D warnings — clean.

Repro context

AionUi desktop → aionrs conversation → MiniMax M2.5 (custom OpenAI-compatible provider) → /compactCompact failed: Empty response from LLM. With this change the reasoning-only summary is recovered and compaction completes.

… EmptyResponse

Reasoning models (e.g. MiniMax M2.5) served via OpenAI-compatible
endpoints may emit the entire compact summary as reasoning_content,
which the provider maps to LlmEvent::ThinkingDelta. The compact stream
collector discarded thinking deltas, so the collected text was empty and
/compact and autocompact failed with "Empty response from LLM" even
though the model produced a full summary. ThinkingConfig::Disabled does
not prevent this: not every provider honors it.

Collect thinking deltas alongside text and fall back to the reasoning
content when the model produced no plain text. format_compact_summary
already degrades gracefully when no <summary> tag is present. Truly
empty responses still fail with EmptyResponse.

@jiahe0510 jiahe0510 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. The Done path is the right fix for the reported Sentry case: the compact request completed normally with reasoning deltas but no text deltas, so falling back to reasoning when text is empty will recover the summary.

There is one blocking issue before this can be merged. When the channel closes without a Done event, the new code accepts any partial text/reasoning and returns success with default usage. Autocompact then replaces the complete conversation history with that result. A connection drop after only part of the summary has arrived could therefore be treated as a successful compaction and cause context loss.

Please keep the terminal-event requirement:

  • On LlmEvent::Done, prefer text and fall back to reasoning. This fixes the Sentry case.
  • If the channel closes before Done, return an error even when partial content was received.
  • Add a regression test showing that partial reasoning/text without Done is rejected.
  • It would also be useful to cover mixed text + reasoning and confirm that text wins.

With the EOF behavior narrowed, I am comfortable merging this fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants