Skip to content

Persist Stirrup assistant reasoning in trajectories - #498

Open
ShuxinLin wants to merge 1 commit into
IBM:mainfrom
ShuxinLin:fix/stirrup-persist-reasoning
Open

Persist Stirrup assistant reasoning in trajectories#498
ShuxinLin wants to merge 1 commit into
IBM:mainfrom
ShuxinLin:fix/stirrup-persist-reasoning

Conversation

@ShuxinLin

Copy link
Copy Markdown
Collaborator

Description

Persist the optional AssistantMessage.reasoning payload for Stirrup turns so tool-only turns can retain provider-supplied reasoning separately from visible assistant text.

Fix Details

  • Add a Stirrup-specific turn record with an optional reasoning field.
  • Serialize reasoning as {signature, content} without changing text semantics.
  • Leave the shared turn schema and other agent runners unchanged.
  • Add coverage for populated and missing reasoning payloads.

Impact on Benchmarking

  • No change to baselines: This fix only improves stability/performance.
  • Baseline change: Persisted Stirrup trajectories now include reasoning when the provider supplies it. Agent answers, tool calls, token counts, and operational metrics are unchanged, but LLM-judge inputs may change because the evaluator serializes the complete trajectory. No full before-versus-after benchmark was run.

Related Issues

  • None.

Verification Steps

  1. Run uv run pytest src/agent/stirrup_agent/tests/ -q.
  2. Confirm all 33 Stirrup tests pass.
  3. Construct a Stirrup AssistantMessage with reasoning and confirm trajectory serialization contains reasoning.signature and reasoning.content.

Checklist

  • I have added tests that prove my fix is effective.
  • My changes are Ruff-formatted. The repository has pre-existing lint findings under the latest standalone Ruff release in this test file.
  • I have signed off my commits (DCO).

Signed-off-by: Shuxin Lin <linshuhsin@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant