fix: recover reasoning-only compact summaries instead of failing with EmptyResponse - #220
Conversation
… EmptyResponse Reasoning models (e.g. MiniMax M2.5) served via OpenAI-compatible endpoints may emit the entire compact summary as reasoning_content, which the provider maps to LlmEvent::ThinkingDelta. The compact stream collector discarded thinking deltas, so the collected text was empty and /compact and autocompact failed with "Empty response from LLM" even though the model produced a full summary. ThinkingConfig::Disabled does not prevent this: not every provider honors it. Collect thinking deltas alongside text and fall back to the reasoning content when the model produced no plain text. format_compact_summary already degrades gracefully when no <summary> tag is present. Truly empty responses still fail with EmptyResponse.
jiahe0510
left a comment
There was a problem hiding this comment.
Thanks for working on this. The Done path is the right fix for the reported Sentry case: the compact request completed normally with reasoning deltas but no text deltas, so falling back to reasoning when text is empty will recover the summary.
There is one blocking issue before this can be merged. When the channel closes without a Done event, the new code accepts any partial text/reasoning and returns success with default usage. Autocompact then replaces the complete conversation history with that result. A connection drop after only part of the summary has arrived could therefore be treated as a successful compaction and cause context loss.
Please keep the terminal-event requirement:
- On
LlmEvent::Done, prefer text and fall back to reasoning. This fixes the Sentry case. - If the channel closes before
Done, return an error even when partial content was received. - Add a regression test showing that partial reasoning/text without
Doneis rejected. - It would also be useful to cover mixed text + reasoning and confirm that text wins.
With the EOF behavior narrowed, I am comfortable merging this fix.
Problem
Running
/compact(and autocompact) fails withCompact failed: Empty response from LLMwhen the conversation model is a reasoning model served via an OpenAI‑compatible endpoint. Observed in AionUi with MiniMax M2.5 (customprovider): the model generates for ~17s, then the turn ends withevent_type="Error" text_len=0.Root cause
thinking: ThinkingConfig::Disabled(crates/aion-agent/src/compact/auto.rs), but the OpenAI projector only acts onThinkingConfig::Enabled(crates/aion-providers/src/projector.rs), so "disabled" never reaches the wire — reasoning models reason anyway.reasoning_contentdeltas toLlmEvent::ThinkingDelta(crates/aion-providers/src/openai.rs).collect_stream_textincompact/auto.rsaccumulated onlyTextDeltaand discarded thinking deltas. A model that emits its whole summary as reasoning yields an empty string →CompactError::EmptyResponse, even though a complete summary was produced (output budget is not the issue —COMPACT_MAX_OUTPUT_TOKENSis 20k).Fix
collect_stream_textnow accumulates thinking deltas alongside text and falls back to the reasoning content when the model produced no plain text.format_compact_summaryalready degrades gracefully when<summary>tags are absent, so the recovered content flows through the existing parsing unchanged.Behavior is otherwise preserved:
EmptyResponse.Donecan now salvage reasoning-only output; if nothing was received it still fails withEmptyResponseas before.Testing
reasoning_only_response_is_recovered(ThinkingDelta-only stream → compaction succeeds, summary contains the recovered content, failure counter untouched).empty_response_failsstill passes (truly empty stream →EmptyResponse).cargo test -p aion-agent— all tests pass.cargo fmt --all -- --checkandcargo clippy -p aion-agent -- -D warnings— clean.Repro context
AionUi desktop → aionrs conversation → MiniMax M2.5 (
customOpenAI-compatible provider) →/compact→Compact failed: Empty response from LLM. With this change the reasoning-only summary is recovered and compaction completes.