feat: record accumulated LLM cost per automation run - #280
Open
hieptl wants to merge 3 commits into
Open
Conversation
Contributor
malhotra5
reviewed
Jul 30, 2026
malhotra5
requested changes
Jul 30, 2026
malhotra5
left a comment
Member
There was a problem hiding this comment.
Left a comment regarding the workspace API for registering cost
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
I have verified the changes.
AGENT:
Why
Automation runs don't record what they cost, so users can't see what their automations spend or spot the expensive ones. The value already exists inside the sandbox — both preset scripts compute
conversation.conversation_stats.get_combined_metrics().accumulated_costand print it — but it's a local variable that gets thrown away, andautomation_runshas no column to hold it.One correction to the issue as filed: it proposed making the preset scripts add
costto the callback payload, but the preset scripts never build that payload — the SDK does, inWorkspace.__exit__→_send_completion_callback(). The transport half therefore lands insoftware-agent-sdk; this PR is the service half plus the preset-side call.Summary
costcolumn toAutomationRun(genericsa.Float, cross-database perAGENTS.md) with migration012; acceptcostonRunCompleteRequestand expose it onAutomationRunResponse.complete_runpersists it on bothCOMPLETEDandFAILED— failed runs still spend money.finallyblock and hand it to the workspace viaregister_cost(), so failed runs report their spend too. The read is wrapped intry/except: it falls back to the agent server when no state is cached, and on the failure path that server may already be gone, so it must never mask the original exception.Issue Number
Resolves #242
How to Test
Automated:
Red/green was verified explicitly — with the source changes stashed and migration
012moved aside, the new tests fail:Full suite with the change applied:
Migration verified by hand on real SQLite, both directions:
End-to-end HTTP round trip against the real ASGI app (in-memory SQLite,
get_session/authenticate_requestoverridden the same waytests/conftest.pywires them):That third line is the compatibility case that matters: the currently pinned SDK (1.37.0) sends no
costfield, and the callback must keep succeeding.Preset scripts:
python -m py_compileclean, both files byte-identical in the changed region (diffof thefinallyblock), and every line under the file's de-facto 88-char limit.Lint/format:
uv run ruff check openhands/ migrations/ tests/andruff format --checkclean.Video/Screenshots
N/A — backend only, no UI in this PR. Evidence is the round-trip and migration transcripts above.
Type