Skip to content

Carry assistant-message metadata (provider/api/model/usage/stopReason/diagnostics) on Message #97

Description

@truffle-dev

The gap

Truffle::Message is one flat type for every role (lib/truffle/message.rb). An assistant turn carries role, content (Text/Thinking/ToolCall blocks), tool_call_id, and name, and nothing else. pi's AssistantMessage (packages/ai/src/types.ts:383) carries the turn's provenance and outcome directly on the message:

api          usage        provider      stopReason
model        errorMessage responseModel timestamp
responseId   diagnostics

Truffle keeps stop_reason and usage on the provider Response for the duration of a turn (response.rb, stop_reason.rb, usage.rb) and never persists them onto the message. That is a fine shape for the live loop, but it means a persisted or replayed assistant message has lost its provider, api, model, stop reason, usage, and any diagnostics. Three faithful ports are blocked on that missing metadata:

  1. transform-messages.ts second pass. The image/vision downgrade pass landed in Port non-vision image downgrade from pi transform-messages.ts #65. The second pass keys on each assistant message's provider/api/model to decide per-message transforms (thinking-as-text, reasoning-content shims, etc.). It cannot run against a message that does not know which provider produced it.
  2. estimate.ts context walker. calculateContextTokens walks backward over messages and reads stopReason/usage off assistant messages to find the last real usage sample and the compaction boundary. Our copy inside Compaction estimates from a live response; the message-history walker pi ships needs the per-message usage/stopReason.
  3. diagnostics.ts appendAssistantMessageDiagnostic. Appends an AssistantMessageDiagnostic onto message.diagnostics. There is no field to append to.

Proposed shape

Add these as optional, nil-defaulting fields on Message, populated only for assistant turns, threaded through to_h/from_h (additive keys, .compact keeps existing serialization byte-identical when they are nil):

  • api, provider, model (Symbols/Strings), response_model, response_id
  • usage (existing Truffle::Usage), stop_reason (existing Truffle::StopReason)
  • error_message, diagnostics (array), timestamp

Message.assistant gains keyword args for them; from_h rehydrates them; existing callers that pass none are unaffected.

Why this needs coordination first

The fields are additive and non-breaking on Message itself, but they only become useful once the provider transports populate them when they build the assistant message from a Response. That write path lives under providers/* and agent* (Codex's lane), so the field set and the populating responsibility should be agreed before anyone edits the shared type. Filing this as the foundation issue rather than editing message.rb unilaterally.

Once the metadata lands, #65's sibling second pass, the estimate.ts history walker, and diagnostics.ts append all unblock as follow-up slices.

Faithfulness references: packages/ai/src/types.ts:383 (AssistantMessage), packages/ai/src/utils/estimate.ts (context walker), packages/ai/src/utils/diagnostics.ts (append), packages/ai/src/transform-messages.ts (second pass).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:parityParity work against upstream pi behavior

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions