Skip to content

Configure worker and supervisor models per workflow stage #13

Closed

Description

@sam-bretz

What & why

I want to choose which model the worker and the supervisor use at each stage, for example a cheaper, faster model for Task and QA, a stronger one for Design and Code, and a different model supervising than working. Today the choice is one model per role for the whole run. Stages differ a lot in difficulty and in how many tokens they use, so per-stage choice would cut usage without weakening the stages that matter.

Acceptance criteria

  • A workflow node can override the model for each role, for example workflow.nodes.design.agents.worker.model and workflow.nodes.qa.agents.supervisor.model. Anything unset inherits the run-wide agents.worker.model / agents.supervisor.model, then the harness default.
  • Validation rejects unknown keys. Configurations without overrides keep their configuration digests, so existing runs are unaffected.
  • The effective model per stage and role is visible before and during a run: envctl run validate, run show / run readiness (next to the per-node limits), and the dashboard.
  • Each attempt records the model it actually ran. Token usage is attributable per stage and model, so cheaper choices show up in the run's totals.
  • Steering resumes and stall nudges keep the model of the generation they resume. Readiness probes use the cheapest configured model for their role, or a documented choice.
  • Documented in the config reference, including that agents.*.model already exists run-wide.

Context

  • Today: workflow.Harness.Model (internal/workflow/config.go) is set once per role under agents. It's copied into every worker and supervisor invocation (internal/localexec/attempt.go) and into readiness probes (internal/localexec/backend.go), and passed as --model by both adapters (internal/agent/claude.go, internal/agent/codex.go). It isn't documented in config-reference.mdx.
  • Precedent: per-node overrides already exist for limits (workflow.nodes.<id>.limits via Config.NodeLimits), a pattern this could follow.
  • Motivation from dogfooding: unset, Claude runs used the harness default, claude-sonnet-5, for the main work. A single run reached 11.7M–28.1M tokens ($6.65–$14.58 API-equivalent) against the new limits.run_tokens ceiling.
  • Open questions:
    • Should a node also be able to override the harness (kind), for example Codex for Code and Claude for review?
    • Should the prompt tell the supervisor which model produced the work?

Activity

  1. sam-bretz commented on Sep 16, 2026

    @sam-bretz
    OwnerAuthor

    Verified against main (875544e): this is implemented. Going criterion by criterion:

    • Per-node override. NodeAgents / NodeHarness (internal/workflow/config.go:158,169-175) give workflow.nodes.<id>.agents.worker.model and .supervisor.model. Config.NodeAgents(node) resolves node override → run-wide agents.<role>.model → harness default (empty Model omits --model).
    • Unknown keys rejected, digests preserved. V2 decoding rejects unknown keys; Agents is tagged json:"agents,omitzero", so a config without overrides serialises exactly as before and keeps its digest — the same technique NodeLimits uses.
    • Effective model visible per stage. envctl run readiness (cmd/envctl/workflow.go:280) and run show (:593) both emit nodeAgents(rev); the TUI stage view is at internal/tui/model.go:827; the dashboard stage panel at internal/web/assets/app.js.
    • Each attempt records the model it ran. Attempt.Models is populated at Revision.Begin (internal/workflow/state.go:475), resolved once from Config.NodeAgents.
    • Steering / nudges keep the generation's model. Worker and supervisor invocations both resolve Model from Config.NodeAgents(a.Attempt.Node) at internal/localexec/attempt.go:245 and :778, so a resumed generation gets the same model. Readiness probes use the run-wide model, documented as a deliberate choice: "a connection probe has no stage in scope."
    • Documented. "Workflow agent models (version 2, experimental)" in docs/src/content/docs/config-reference.mdx, including the inheritance order and that agents.*.model already existed run-wide.

    Landed in 7b1142c.

    One gap this left is now split out as #38: Attempt.Models omits a role with no resolved model, so harness default still didn't say which model that was. That is fixed in #39.

    The two open questions in the issue were not implemented and are not covered here — per-node harness kind override, and telling the supervisor which model produced the work. Worth separate issues if still wanted.

    Closing as complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions