Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Breaking

- Provider results carry request and response information as the AI SDK does
(`aimux-core`). `GenerateResult` has `request: Option<RequestInfo>` and
`response: Option<ResponseInfo>`; `ResponseInfo` gains `id` and holds the
timestamp, model id, headers and body. `StreamResult` has `request` and
`response: Option<StreamResponseInfo>`. The flat `request_body` and
`response_headers` fields are removed and `response` is optional; the call
layer fills a missing id, timestamp and model id. The user-facing results and
their JSON are unchanged.

- The prompt a provider receives is modelled by role (`aimux-core`), as the AI
SDK's `LanguageModelV4Message`. `LanguageModelPrompt` is
`Vec<LanguageModelMessage>`: `System { content: String }`,
Expand Down
5 changes: 2 additions & 3 deletions MIGRATION-NS.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,9 +155,8 @@ fn f(meta: Option<ProviderMetadata>) -> StreamPart { ... }
```

Local variables typed `Option<Value>` / `HashMap<String, Value>` that end up in these fields: retype them.
Intermediate scratch values that are not themselves stored (e.g. `StreamingToolCallTracker::with_extract_metadata`
closures, which still return `Option<Value>`) stay `Value`; `with_build_provider_metadata` still takes `Option<&Value>`
and returns `Option<ProviderMetadata>`, so build with `provider_namespace("google", json!({...}))`.
This includes `StreamingToolCallDelta::provider_metadata`, which is an `Option<ProviderMetadata>`: build it
with `provider_namespace("google", json!({...}))`.

### 7. Tests comparing against `json!`

Expand Down
3 changes: 3 additions & 0 deletions aimux-core/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,9 @@ pub enum AiMuxError {
#[error("JSON parse error: {0}")]
JsonParse(String),

#[error("{0}")]
NoOutputGenerated(String),

#[error("invalid response data: {0}")]
InvalidResponseData(String),

Expand Down
192 changes: 143 additions & 49 deletions aimux-core/src/generate.rs

Large diffs are not rendered by default.

5 changes: 2 additions & 3 deletions aimux-core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -107,9 +107,8 @@ pub mod prelude {
RerankingCallOptions, RerankingModel, RerankingResult, rerank,
};
pub use crate::result::{
GenerateResponse, GenerateResult, GeneratedFile, RawToolApprovalRequest, ReasoningOutput,
ReasoningPart, Source, StreamResponse, StreamResult, TextContent,
ToolApprovalRequestOutput,
GenerateResult, GeneratedFile, RawToolApprovalRequest, ReasoningOutput, ReasoningPart,
Source, StreamResult, TextContent, ToolApprovalRequestOutput,
};
pub use crate::router::{
FallbackPolicy, Router, RouterConfig, RouterModel, RuleRouter, WeightedRouter,
Expand Down
2 changes: 1 addition & 1 deletion aimux-core/src/moa.rs
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ impl LanguageModel for MoaModel {
}
};

// RFC-0022 §3.4: return None for request_body/response_headers. The
// RFC-0022 §3.4: return None for request/response. The
// aggregator's request body is a synthesized prompt (references
// spliced in), not the user's original — exposing it would mislead
// cache probing (RFC-0015 fingerprinting).
Expand Down
75 changes: 39 additions & 36 deletions aimux-core/src/openai_output.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! OpenAI Chat Completions output format.
//!
//! Converts aimux's internal types ([`GenerateResult`] / [`StreamPart`]) into
//! Converts aimux's internal types ([`GenerateResult`] / [`TextStreamPart`]) into
//! standard OpenAI Chat Completions structures ([`ChatCompletion`] /
//! [`ChatCompletionChunk`]). This lets any provider (OpenAI, Anthropic, Google,
//! …) be consumed via the OpenAI wire format.
Expand All @@ -9,7 +9,7 @@
//!
//! ```text
//! provider.do_generate() → GenerateResult → to_chat_completion() → ChatCompletion
//! provider.do_stream() → Stream<StreamPart> → to_chat_completion_stream() → Stream<Chunk>
//! stream_text() → Stream<TextStreamPart> → to_chat_completion_stream() → Stream<Chunk>
//! ```
//!
//! The conversion is a post-processing step — it does not modify the
Expand Down Expand Up @@ -37,9 +37,9 @@ use crate::error::AiMuxError;
use crate::parse_tool_call::raw_tool_call_text;
use crate::result::{GenerateContent, GenerateResult, GeneratedFile, ReasoningOutput, Source};
use crate::shared::GeneratedFileData;
use crate::stream_part::{StreamPart, TextStreamPart};
use crate::stream_part::TextStreamPart;
use crate::tool::{ToolCall, ToolResult};
use crate::types::{FinishReason, FinishReasonUnified, ResponseMetadata, Usage};
use crate::types::{FinishReason, FinishReasonUnified, Usage};

// ─────────────────────────────────────────────────────────────────────────────
// Non-streaming response types
Expand Down Expand Up @@ -377,7 +377,7 @@ pub fn to_chat_completion(result: &GenerateResult, model: &str) -> ChatCompletio
}

// ─────────────────────────────────────────────────────────────────────────────
// Streaming conversion: Stream<StreamPart> → Stream<ChatCompletionChunk>
// Streaming conversion: Stream<TextStreamPart> → Stream<ChatCompletionChunk>
// ─────────────────────────────────────────────────────────────────────────────

/// A streaming OpenAI Chat Completions result.
Expand All @@ -394,7 +394,7 @@ impl std::fmt::Debug for ChatCompletionStream {
}
}

/// Convert a `StreamPart` stream into a `ChatCompletionChunk` stream.
/// Convert a `TextStreamPart` stream into a `ChatCompletionChunk` stream.
///
/// Uses a stateful converter that maintains tool-call indices and accumulates
/// the final usage / finish_reason. The output follows OpenAI SSE conventions:
Expand Down Expand Up @@ -445,7 +445,7 @@ pub fn to_chat_completion_stream(
}

// After Finish, stop (but we already emitted the final chunk).
if matches!(part, StreamPart::Finish { .. }) {
if matches!(part, TextStreamPart::Finish { .. }) {
return;
}
}
Expand Down Expand Up @@ -536,7 +536,7 @@ impl StreamState {
Some(chunk)
}

/// Process a single StreamPart, returning zero or more output chunks.
/// Process a single TextStreamPart, returning zero or more output chunks.
fn process_part(
&mut self,
part: &TextStreamPart,
Expand All @@ -546,27 +546,30 @@ impl StreamState {
let mut chunks = Vec::new();

match part {
StreamPart::StreamStart { .. } => {
TextStreamPart::StreamStart { .. } => {
if let Some(c) = self.ensure_started() {
chunks.push(c);
}
}

StreamPart::ResponseMetadata(ResponseMetadata { id, model_id, .. }) => {
if let Some(id) = id {
self.id = id.clone();
}
if let Some(m) = model_id {
self.model = m.clone();
TextStreamPart::FinishStep { response, .. } => {
// Response metadata arrives at step end; emitted chunks keep their identity.
if !self.started {
if let Some(id) = &response.id {
self.id = id.clone();
}
if let Some(model_id) = &response.model_id {
self.model = model_id.clone();
}
}
}

StreamPart::TextStart { .. } => {
TextStreamPart::TextStart { .. } => {
if let Some(c) = self.ensure_started() {
chunks.push(c);
}
}
StreamPart::TextDelta { delta, .. } => {
TextStreamPart::TextDelta { delta, .. } => {
if let Some(c) = self.ensure_started() {
chunks.push(c);
}
Expand All @@ -582,10 +585,10 @@ impl StreamState {
}];
chunks.push(chunk);
}
StreamPart::TextEnd { .. } => {}
TextStreamPart::TextEnd { .. } => {}

StreamPart::ReasoningStart { .. } => {}
StreamPart::ReasoningDelta { delta, .. } => {
TextStreamPart::ReasoningStart { .. } => {}
TextStreamPart::ReasoningDelta { delta, .. } => {
if include_reasoning {
if let Some(c) = self.ensure_started() {
chunks.push(c);
Expand All @@ -603,9 +606,9 @@ impl StreamState {
chunks.push(chunk);
}
}
StreamPart::ReasoningEnd { .. } => {}
TextStreamPart::ReasoningEnd { .. } => {}

StreamPart::ToolInputStart { id, tool_name, .. } => {
TextStreamPart::ToolInputStart { id, tool_name, .. } => {
if let Some(c) = self.ensure_started() {
chunks.push(c);
}
Expand Down Expand Up @@ -649,7 +652,7 @@ impl StreamState {
}];
chunks.push(chunk);
}
StreamPart::ToolInputDelta { id, delta, .. } => {
TextStreamPart::ToolInputDelta { id, delta, .. } => {
// Ensure started (shouldn't happen without Start, but be safe).
if let Some(c) = self.ensure_started() {
chunks.push(c);
Expand Down Expand Up @@ -697,9 +700,9 @@ impl StreamState {
}];
chunks.push(chunk);
}
StreamPart::ToolInputEnd { .. } => {}
TextStreamPart::ToolInputEnd { .. } => {}

StreamPart::ToolCall(ToolCall {
TextStreamPart::ToolCall(ToolCall {
tool_call_id,
tool_name,
input,
Expand Down Expand Up @@ -788,7 +791,7 @@ impl StreamState {
}
}

StreamPart::ToolResult(ToolResult {
TextStreamPart::ToolResult(ToolResult {
tool_name, result, ..
}) => {
// Degraded: provider-executed tool result as content.
Expand All @@ -809,7 +812,7 @@ impl StreamState {
chunks.push(chunk);
}

StreamPart::File(GeneratedFile {
TextStreamPart::File(GeneratedFile {
data, media_type, ..
}) => {
if let Some(c) = self.ensure_started() {
Expand All @@ -829,7 +832,7 @@ impl StreamState {
chunks.push(chunk);
}

StreamPart::Source(Source::Url { url, .. }) => {
TextStreamPart::Source(Source::Url { url, .. }) => {
if let Some(c) = self.ensure_started() {
chunks.push(c);
}
Expand All @@ -849,7 +852,7 @@ impl StreamState {
}
}

StreamPart::Finish {
TextStreamPart::Finish {
finish_reason,
usage,
..
Expand All @@ -862,17 +865,17 @@ impl StreamState {
}
}

StreamPart::Error { error } => {
TextStreamPart::Error { error } => {
if let Some(chunk) = self.error_chunk(error) {
chunks.push(chunk);
}
}

StreamPart::Raw { .. }
| StreamPart::Source(Source::Document { .. })
| StreamPart::Custom { .. }
| StreamPart::ReasoningFile(_)
| StreamPart::ToolApprovalRequest(_) => { /* no OpenAI chat equivalent */ }
TextStreamPart::Raw { .. }
| TextStreamPart::Source(Source::Document { .. })
| TextStreamPart::Custom { .. }
| TextStreamPart::ReasoningFile(_)
| TextStreamPart::ToolApprovalRequest(_) => { /* no OpenAI chat equivalent */ }
}

chunks
Expand Down Expand Up @@ -1019,7 +1022,7 @@ fn now_unix() -> u64 {

// Rejected repairs retain the original call. Recover its text from the
// original error, never from the replacement's validation failure.
/// Render a Core-parsed `StreamPart::ToolCall`'s arguments as OpenAI-compatible
/// Render a Core-parsed `TextStreamPart::ToolCall`'s arguments as OpenAI-compatible
/// wire text: the provider's raw argument text verbatim for an invalid call,
/// compact JSON of the parsed value otherwise.
///
Expand Down
Loading
Loading