Skip to content

feat(pricing): add machine-wide pricing overrides via ~/.conductor/pricing.yaml#173

Open
PolyphonyRequiem wants to merge 1 commit into
microsoft:mainfrom
PolyphonyRequiem:feat/machine-wide-pricing-overrides
Open

feat(pricing): add machine-wide pricing overrides via ~/.conductor/pricing.yaml#173
PolyphonyRequiem wants to merge 1 commit into
microsoft:mainfrom
PolyphonyRequiem:feat/machine-wide-pricing-overrides

Conversation

@PolyphonyRequiem
Copy link
Copy Markdown
Member

@PolyphonyRequiem PolyphonyRequiem commented May 10, 2026

Problem

Workflow-level runtime.cost.pricing is per-workflow. If a model isn't in the built-in pricing table, every workflow that uses it either reports $0 silently or fuzzy-matches a coarser ancestor and reports plausible-but-wrong numbers. No shared place exists for pricing on models not yet in DEFAULT_PRICING (the claude-opus-4.7-* family, gpt-5.3 / 5.4 / 5.5, etc.).

Solution

A machine-wide pricing file at ~/.conductor/pricing.yaml, with CONDUCTOR_PRICING_FILE as env-var override (with ~ expansion). The file uses the same PricingOverride schema as runtime.cost.pricing.

Precedence (highest wins):

  1. Exact match in workflow runtime.cost.pricing
  2. Exact match in user ~/.conductor/pricing.yaml
  3. Exact match in built-in DEFAULT_PRICING
  4. Fuzzy (versioned-suffix) match in built-in DEFAULT_PRICING

Override entries are exact-match only — list each concrete model name (e.g. claude-opus-4-20250514, not just claude-opus-4). Keeps the surface narrow; the existing built-in fuzzy match from #143 is unchanged.

One CLI command: conductor pricing path prints the resolved file location.

Failure modes

  • Missing file: silent no-op.
  • Malformed file: hard error at engine construction with the file path and the CONDUCTOR_PRICING_FILE=/dev/null bypass in the error message itself.
  • Bad pricing: value or non-string model key: clear ConfigurationError instead of crashing later with AttributeError.

Diff size

~578 net new lines:

  • config/user_pricing.py (+112) — loader: ~ expansion, env-var, missing→empty, malformed→error, value/key validation
  • engine/workflow.py (+50/-10) — layering in _build_pricing_overrides
  • cli/pricing.py (+28) — single path subcommand
  • Tests: test_user_pricing.py (+195), test_pricing_layering.py (+97), test_pricing_cmd.py (+46)
  • Docs: README (+56), AGENTS.md (+2)

57 new tests pass; lint/format/typecheck clean.

Try it

cat > ~/.conductor/pricing.yaml << 'EOF'
pricing:
  claude-opus-4.7-high:
    input_per_mtok: 15.00
    output_per_mtok: 75.00
EOF
uv run conductor pricing path
uv run conductor run any-workflow.yaml --input ...

History

  • v1 (1142 lines): full CLI (init/show template/table), extra="forbid" schema hardening, override-side fuzzy match, UserPricingFile Pydantic wrapper, $VAR expansion.
  • v2 (651 lines, this PR): kept only what's load-bearing for the core feature. CLI is just path. Schema hardening split out. UserPricingFile and $VAR dropped. Override-side fuzzy initially kept, then dropped in v3 to keep the surface lean — users list concrete model names; built-in fuzzy is unchanged.
  • v3 (578 lines, current): drop override-side fuzzy and its 4 tests. Strict loader validation added (rejects pricing: true, pricing: [], non-string keys). Error message escape hatch surfaced.

Draft for maintainer review.

@PolyphonyRequiem PolyphonyRequiem force-pushed the feat/machine-wide-pricing-overrides branch 2 times, most recently from f47d86f to 2afa98f Compare May 10, 2026 20:07
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@e6f8bd7). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #173   +/-   ##
=======================================
  Coverage        ?   86.63%           
=======================================
  Files           ?       62           
  Lines           ?     8983           
  Branches        ?        0           
=======================================
  Hits            ?     7782           
  Misses          ?     1201           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@PolyphonyRequiem PolyphonyRequiem force-pushed the feat/machine-wide-pricing-overrides branch from 2afa98f to 2d79029 Compare May 10, 2026 20:15
…icing.yaml

## Problem

Workflow-level `runtime.cost.pricing` is per-workflow. If a model isn't in the
built-in pricing table (`engine/pricing.py:DEFAULT_PRICING`), every workflow
that uses it either reports $0 silently or fuzzy-matches a coarser ancestor
and reports plausible-but-wrong numbers. There's no shared place to put
pricing for models that haven't shipped to the default table yet (the
`claude-opus-4.7-*` family, `gpt-5.3` / `5.4` / `5.5`, etc.).

## Solution

A machine-wide pricing file at `~/.conductor/pricing.yaml`, with
`CONDUCTOR_PRICING_FILE` as an env-var override (with `~` expansion). The
file uses the same `PricingOverride` schema as `runtime.cost.pricing`.

Precedence (highest wins):

1. Exact match in workflow `runtime.cost.pricing`
2. Exact match in user `~/.conductor/pricing.yaml`
3. Exact match in built-in `DEFAULT_PRICING`
4. Fuzzy (versioned-suffix) match in built-in `DEFAULT_PRICING`

Override entries are exact-match only — list each concrete model name. The
existing built-in fuzzy match (microsoft#143) is unchanged.

`conductor pricing path` prints the resolved file location (honors the env var).

## Failure modes

- **Missing file**: silent no-op. Workflows that don't need overrides aren't affected.
- **Malformed file**: hard error at engine construction with a pointer to the
  file path and the bypass instruction `CONDUCTOR_PRICING_FILE=/dev/null` in
  the error message itself. Silent acceptance of corrupted overrides was the
  original bug we're solving for.

The loader rejects non-mapping `pricing:` values (e.g. `pricing: true`,
`pricing: []`) and non-string model keys with clear `ConfigurationError`
messages, rather than crashing later with `AttributeError`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PolyphonyRequiem PolyphonyRequiem force-pushed the feat/machine-wide-pricing-overrides branch from 2d79029 to 4210599 Compare May 10, 2026 20:20
@PolyphonyRequiem PolyphonyRequiem marked this pull request as ready for review May 10, 2026 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants