Skip to content

V4 alignment: Decision model vs upstream DecisionModelV4 #233

Open

Description

@cunninghamcard-bit
Conventions

Why.

  • Upstream shipped this feature first as EvaluationModelV4 / experimental_evaluate:
    • @ai-sdk/provider@4.0.19 and 4.0.21: src/evaluation-model/v4/;
    • ai@7.0.122 and 7.0.127: src/evaluate/.
  • @ai-sdk/provider@4.0.22 (2026-10-05) renamed it to DecisionModelV4. ai@7.0.128 is the first ai release on that provider.
  • aimux has its own Decision API on master (feat: support official decision providers, candidate scoring, and native media #226, RFC-0037/0038).
  • Layer C2 added a second, parallel EvaluationModel that mirrors the pre-rename upstream. The tree now has two models for one feature, and neither matches the current upstream shape.

Fixed in.

  • The alignment migration.
  • The bindings' decision exports move with the binding migration (D6, last).
  • Doc side: the planned alignment-doc PR updates the §5 row and the four "Evaluation out of scope" lines.

Related. RFC-0037 (Decision API) and RFC-0038 (decision providers).

Existing fixes. None.

Checklist

  • C29: Provider::evaluation_model sits on the core Provider trait, and there is no evaluate operation.
    • Evidence: v4: aimux-core/src/provider.rs:109, :192, provider_registry.rs:13, aimux-core/src/evaluation_model.rs:237, :249 (EvaluationModel / do_evaluate, added in layer C2).
    • Doc: doc:86, :392, :705, :1142, :1591. Upstream: @ai-sdk/provider@4.0.19 provider-v4.ts has no evaluation member.
  • N4: bump the baseline to a provider release with DecisionModelV4.
    • Evidence: fixtures/aisdk/VERSIONS.json pins provider 4.0.21 and ai 7.0.127, both before the rename.
    • Upstream: @ai-sdk/provider@4.0.22 src/decision-model/v4/decision-model-v4.ts:7, :11, :13 (specificationVersion 'v4', supportedQuestionTypes, doDecide).
  • N5: rename EvaluationModel to DecisionModel (V4 shape): do_decide and supported_question_types.
    • Upstream:
      • decision-model-v4-call-options.ts:9-10: state: Input, questions: Record<string, Question>, plus abortSignal?, headers?, providerOptions?.
      • decision-model-v4-question.ts: the union is choice{instructions, criteria: Record<string, Input|null>}, score{instructions, criteria: (Input|null)[]} and boolean{instructions, criteria?{true?, false?}}.
      • decision-model-v4-result.ts: answers: Record<string, Answer>, rounding?, usage?, warnings, providerMetadata?, response?.
  • N6: fold the existing aimux Decision API onto that shape.
    • Differences in m: aimux-core/src/decision_model.rs:
      • questions are a Vec of id'd questions (:162), not a keyed record;
      • extra images (:161), max_retries (:167), timeout (:169) and recording_context (:158);
      • DecisionCapabilities (:321) replaces supportedQuestionTypes;
      • answers carry probability_source (:330, :379).
    • Each extra becomes providerOptions / providerMetadata, or a documented product difference.
  • N7: delete the duplicate model, so one decision model remains in core and in the registry (provider_registry.rs:13).
  • N8: re-record the decision fixtures and replay (RFC-0037:119-125) against the new shape. Keep the recording identity-only, as the layers already do.
  • N9: decision recordings stop storing input.decision_capabilities. Supported question types are a model property, rebuilt from the host registry on replay, as upstream DecisionModelV4.supportedQuestionTypes (@ai-sdk/provider@4.0.22 src/decision-model/v4/decision-model-v4.ts:11).
    • Evidence: v4: aimux-core/src/recording.rs:158-162 (InputRecord.decision_capabilities), :187-196 (from_decision_options copies the capabilities into the recording).
    • Expected: drop the field from the recording input; replay looks the model up in the registry. Do it together with N6 and N8.

Acceptance. A type fixture comparison against @ai-sdk/provider@4.0.22 decision-model/v4. The Jev and OpenAI decision providers pass with the new trait. A single decision_model lookup remains in the registry.

Activity

  1. cunninghamcard-bit commented on Oct 14, 2026

    @cunninghamcard-bit
    ContributorAuthor

    按 #255 当前 head 22db0867 对齐条目状态:

    • C29、N5–N7、N9:已实现、待合并。 core/registry 已统一为单一 DecisionModel / decide,questions/answers 使用 ID maps,扩展迁至 providerOptions / providerMetadata,绑定随之迁移,录制不再保存能力快照。
    • N4: [7/8] feat: the remaining upstream model kinds #255 已将 provider pin 升至 4.0.22;ai pin 仍为 7.0.127,decide 实现参照 7.0.128。
    • N8:已实现、待合并,实际验证方式应更新为离线转换与回放。 既有真实 HTTP 录制的 options/result 已确定性转换为新形状,原始 HTTP 请求/响应保持不变;新增覆盖包括录制结果离线回放,以及通过本地 HTTP 回放、registry 重建 provider 后核对结果。详见 fixture 说明。这里沿用既有真实 wire,无需新增网络录制;未重新录制不构成 bug。

    公开 JSON 的 camelCase 由 Draft #256 承载;decide telemetry 按已登记范围后置。以上仅据远程源码核对,本次未运行测试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions