diff --git a/CHANGELOG.md b/CHANGELOG.md index 132f39ad..de408f86 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ adheres to [Semantic Versioning](https://semver.org/). ## Unreleased +### yoagent + +- **DeepSeek is priced** (after `prices::enable_bundled()`): `deepseek-flash`, `deepseek-v4-pro` and the legacy `deepseek-v4-flash` (billed as Flash), from DeepSeek's pricing page. **At the peak rate:** DeepSeek bills half of it off-peak (all hours but 01:00–04:00 and 06:00–10:00 UTC on weekdays), and a `CostConfig` has no time of day, so costs are an upper bound — off-peak they read up to twice the bill, and a `Budget` stops early rather than late. Install the off-peak rates as an override to count those instead (`docs/concepts/pricing.md`). `ModelConfig::deepseek(..)` for those ids now has a `cost`; `Budget::for_model` works for them. The price audit checks the rates as exactly twice models.dev's off-peak ones, and models.dev data (`enable_live`, a fetch or a price cache written by an earlier version) never brings DeepSeek's off-peak rates back over the bundled peak ones. `examples/llm_compaction_live.rs` reads DeepSeek's rates from the snapshot instead of its own literals. + ### yoagent-rutis (0.4.0) - **`yoagent_rutis::adapters`: where the pi and DSH adapters are.** `adapters()` returns the folders of the adapter files the crate ships (`plugins/pi`, `plugins/dsh`, `plugins/yoagent.d.ts`), so a host can copy them and `npm ci` the copy instead of keeping its own copy of them in step with the crate's. diff --git a/docs/concepts/pricing.md b/docs/concepts/pricing.md index 40959612..3e7b9447 100644 --- a/docs/concepts/pricing.md +++ b/docs/concepts/pricing.md @@ -191,7 +191,8 @@ every lookup is `None`. Once it has: `None` otherwise. `ModelConfig::anthropic("claude-sonnet-5", ..)` is priced exactly like `claude_sonnet_5()`; `meta` prices `muse-spark-1.1` and `muse-spark-1.2` and leaves any other id (the far cheaper contributor tier, - say) unknown. + say) unknown. `deepseek` prices `deepseek-flash`, `deepseek-v4-pro` and the + legacy `deepseek-v4-flash` (see below). - **Gateways and custom endpoints** — `custom`, `openai_compat`, `local`, `ollama`, `opencode_zen`, `opencode_go`, `mock` — are always `None`, even for an id the table lists. A gateway's bill is not the vendor's list price. @@ -199,6 +200,25 @@ every lookup is `None`. Once it has: `None` means **unknown**, never free. Set `config.cost` yourself for a model the table does not list. +**DeepSeek is priced at its peak rate.** DeepSeek bills half its peak rate +off-peak: every hour except 01:00–04:00 and 06:00–10:00 UTC on weekdays, +about 79% of the week. A `CostConfig` has no time of day, so the snapshot +carries the peak rate, the upper bound: off-peak, `cost_usd` and +`total_cost_usd()` are up to twice the bill, and a `Budget` stops early rather +than late. To count off-peak spend instead, install the off-peak rates as an +override (`install_override` or `YOAGENT_PRICES`, below): + +```json +{ "schema": 1, "providers": { "deepseek": { + "deepseek-flash": { "input": 0.15, "output": 0.6, "cache_read": 0.003, "cache_write": 0.15 }, + "deepseek-v4-flash": { "input": 0.15, "output": 0.6, "cache_read": 0.003, "cache_write": 0.15 }, + "deepseek-v4-pro": { "input": 0.66, "output": 1.98, "cache_read": 0.022, "cache_write": 0.66 } } } } +``` + +Each model needs its own entry (one left out keeps the peak rate), and +`cache_write` is stated as the input rate: DeepSeek reports no cache-write +tokens, and leaving it out makes the override report a warning. + ## Overriding prices at runtime A price change should not have to wait for a yoagent release. Constructors @@ -444,7 +464,9 @@ for every model it lists. V4 Pro's price. The mapping is conservative: - A model whose cost carries structure `CostConfig` cannot express is **skipped**, not approximated. That covers a separate reasoning rate, a - non-context tier and an unknown key. + non-context tier and an unknown key — and DeepSeek's models, priced by + time of day: models.dev lists the off-peak rate, which must not replace + the snapshot's peak one (they are reported as skipped, without a warning). - A skipped model that the bundled snapshot lists is **named in a `warn` log**. It keeps the snapshot's price when the snapshot is enabled (`enable_live*` always enables it). @@ -528,7 +550,11 @@ drift, on an entry that vanished upstream, and on a comparison that silently checked nothing. The comparison logic also runs offline in CI, against a checked-in slice of models.dev. models.dev is community-maintained, so a failure sends a human to the entry's `source`; the audit never edits the -file. Run the live audit before a release: +file. Where the vendor and models.dev legitimately differ, the audit records +the difference and asserts it both ways rather than skipping the field: for +DeepSeek, the snapshot's rates must be exactly twice models.dev's (which lists +the off-peak ones), and models.dev's separate `reasoning` rate must equal its +output rate. Run the live audit before a release: ```text cargo test --all-features --test price_audit -- --ignored --nocapture diff --git a/examples/llm_compaction_live.rs b/examples/llm_compaction_live.rs index cc3bbcad..89937294 100644 --- a/examples/llm_compaction_live.rs +++ b/examples/llm_compaction_live.rs @@ -34,9 +34,7 @@ use std::sync::Arc; use yoagent::context::ContextConfig; use yoagent::gasp::{GaspRecorder, GoalRef}; use yoagent::llm_compaction::SUMMARY_MARKER; -use yoagent::provider::{ - CostConfig, ModelConfig, ProviderError, StreamConfig, StreamEvent, StreamProvider, -}; +use yoagent::provider::{ModelConfig, ProviderError, StreamConfig, StreamEvent, StreamProvider}; use yoagent::*; /// Dry-run double (`YO_DRY_RUN=1`): bulky deterministic text, so the harness's @@ -110,18 +108,12 @@ fn priced(id: &str) -> ModelConfig { // maps `prompt_cache_hit_tokens` onto `Usage::cache_read`. So a hit // rate here measures *their* caching, not yoagent's placement. // - // Rates live here rather than in `ModelConfig::deepseek` because - // DeepSeek prices by time of day — off-peak is exactly half of peak - // (peak = 01:00-04:00 and 06:00-10:00 UTC). A crate-level preset would - // be wrong half the day. These are the **peak** rates, so the cost - // column over-reports rather than under-reports off-peak runs. - // DeepSeek has no cache-write category at all: populating its cache - // is free, which is the asymmetry the provider comparison turns on. - // `deepseek-v4-flash` is a legacy name DeepSeek still accepts, served - // by V4.1 Flash and billed at the Flash price. - "deepseek-flash" => deepseek_priced("deepseek-flash", 0.30, 1.20, 0.006), - "deepseek-v4-flash" => deepseek_priced("deepseek-v4-flash", 0.30, 1.20, 0.006), - "deepseek-v4-pro" => deepseek_priced("deepseek-v4-pro", 1.32, 3.96, 0.044), + // Priced from the bundled snapshot (`prices::enable_bundled()` in + // `main`) at DeepSeek's **peak** rate: off-peak is exactly half, and a + // `CostConfig` has no time of day, so the cost column over-reports + // rather than under-reports off-peak runs. DeepSeek has no cache-write + // category: populating its cache is free (a miss is plain input), which + // is the asymmetry the provider comparison turns on. other if other.starts_with("deepseek") => { noting_unpriced(ModelConfig::deepseek(other, other)) } @@ -142,15 +134,6 @@ fn note_unpriced(model: &str) { ); } -/// A DeepSeek config carrying peak-window rates. `cache_write` is left unset -/// because DeepSeek has no write category (it would bill at the input rate, -/// but DeepSeek never reports cache-write tokens — a miss is plain input). -fn deepseek_priced(id: &str, input: f64, output: f64, cache_read: f64) -> ModelConfig { - let mut config = ModelConfig::deepseek(id, id); - config.cost = Some(CostConfig::new(input, output).with_cache_read(cache_read)); - config -} - fn env_or(key: &str, default: T) -> T { std::env::var(key) .ok() diff --git a/src/provider/model.rs b/src/provider/model.rs index a540ad87..1a6704db 100644 --- a/src/provider/model.rs +++ b/src/provider/model.rs @@ -131,12 +131,11 @@ pub(crate) fn openai_id_is_reasoning(id: &str) -> bool { /// // config.cost.as_mut()` would silently do nothing — but set every rate /// // you pay, since the rest start at zero (an unset cache rate then bills /// // at the input rate): -/// let mut deepseek = ModelConfig::deepseek("deepseek-flash", "DeepSeek Flash"); -/// assert!(deepseek.cost.is_none()); -/// // DeepSeek Flash peak rates (off-peak is half); DeepSeek has no -/// // cache-write category. -/// deepseek.cost = Some(CostConfig::new(0.30, 1.20).with_cache_read(0.006)); -/// assert_eq!(deepseek.cost.as_ref().unwrap().input_per_million, 0.30); +/// let mut mistral = ModelConfig::mistral("mistral-large-latest", "Mistral Large"); +/// assert!(mistral.cost.is_none()); +/// // The rates you pay, from Mistral's pricing page (these are placeholders). +/// mistral.cost = Some(CostConfig::new(2.0, 6.0).with_cache_read(0.2)); +/// assert_eq!(mistral.cost.as_ref().unwrap().input_per_million, 2.0); /// /// // A model you run for free is `Some` with zero rates, and costs $0: /// let mut local = ModelConfig::local("http://localhost:1234/v1", "qwen3"); @@ -2968,7 +2967,8 @@ mod tests { ModelConfig::qwen("qwen-plus", "Qwen"), ModelConfig::xai("grok-4-1-fast", "Grok"), ModelConfig::groq("llama-3.3-70b-versatile", "Llama"), - ModelConfig::deepseek("deepseek-v4-flash", "DeepSeek"), + // A DeepSeek id the snapshot does not list (the retired names are). + ModelConfig::deepseek("deepseek-chat", "DeepSeek"), ModelConfig::mistral("mistral-large-latest", "Mistral"), ModelConfig::google("gemini-3-pro", "Gemini"), // `meta` looks its id up too: the contributor tier is not listed, @@ -3067,7 +3067,7 @@ mod tests { assert_eq!(sonnet.cost, Some(CostConfig::new(1.8, 9.0))); let opus = ModelConfig::claude_opus_5().with_prices(&table); assert_eq!(opus.cost, ModelConfig::claude_opus_5().cost); - let unknown = ModelConfig::deepseek("deepseek-flash", "D").with_prices(&table); + let unknown = ModelConfig::mistral("mistral-large-latest", "M").with_prices(&table); assert!(unknown.cost.is_none()); // Explicit, so it applies to a local endpoint too: free, not unknown. let local = ModelConfig::local("http://localhost:1234/v1", "qwen3").with_prices(&table); diff --git a/src/provider/prices.json b/src/provider/prices.json index 794a06d2..15c6f2f7 100644 --- a/src/provider/prices.json +++ b/src/provider/prices.json @@ -64,6 +64,38 @@ "verified": "2026-08-19" } }, + "deepseek": { + "deepseek-flash": { + "input": 0.3, + "output": 1.2, + "cache_read": 0.006, + "cache_write": 0.3, + "cache_write_at_input": true, + "note": "Peak rate. DeepSeek bills half of it off-peak: all hours but 01:00-04:00 and 06:00-10:00 UTC on weekdays (about 79% of the week). A flat CostConfig has no time of day, so the peak rate is the upper bound: costs are over-stated off-peak, never under-stated, and a Budget stops early rather than late.", + "source": "https://api-docs.deepseek.com/quick_start/pricing", + "verified": "2026-10-16" + }, + "deepseek-v4-flash": { + "input": 0.3, + "output": 1.2, + "cache_read": 0.006, + "cache_write": 0.3, + "cache_write_at_input": true, + "note": "The legacy name: served by DeepSeek-V4.1-Flash and billed at the deepseek-flash price. Peak rate. DeepSeek bills half of it off-peak: all hours but 01:00-04:00 and 06:00-10:00 UTC on weekdays (about 79% of the week). A flat CostConfig has no time of day, so the peak rate is the upper bound: costs are over-stated off-peak, never under-stated, and a Budget stops early rather than late.", + "source": "https://api-docs.deepseek.com/quick_start/pricing", + "verified": "2026-10-16" + }, + "deepseek-v4-pro": { + "input": 1.32, + "output": 3.96, + "cache_read": 0.044, + "cache_write": 1.32, + "cache_write_at_input": true, + "note": "Peak rate. DeepSeek bills half of it off-peak: all hours but 01:00-04:00 and 06:00-10:00 UTC on weekdays (about 79% of the week). A flat CostConfig has no time of day, so the peak rate is the upper bound: costs are over-stated off-peak, never under-stated, and a Budget stops early rather than late.", + "source": "https://api-docs.deepseek.com/quick_start/pricing", + "verified": "2026-10-16" + } + }, "meta": { "muse-spark-1.1": { "input": 1.25, diff --git a/src/provider/prices/fetch.rs b/src/provider/prices/fetch.rs index 4e0f4369..c319a6b3 100644 --- a/src/provider/prices/fetch.rs +++ b/src/provider/prices/fetch.rs @@ -400,6 +400,10 @@ impl PriceTable { reason, }) }; + if TIME_OF_DAY_PRICED.contains(&provider) { + skip(TIME_OF_DAY_REASON.into()); + continue; + } let Some(cost) = info.get("cost") else { skip("models.dev lists no cost".into()); continue; @@ -418,8 +422,10 @@ impl PriceTable { } } let builtin = builtin_ref(); + // Skipped by design (time-of-day pricing) is not news; unmappable data is. let dropped: Vec = skipped .iter() + .filter(|s| !TIME_OF_DAY_PRICED.contains(&s.provider.as_str())) .filter(|s| builtin.entry(&s.provider, &s.model).is_some()) .map(|s| format!("{}/{} ({})", s.provider, s.model, s.reason)) .collect(); @@ -514,6 +520,16 @@ impl PriceTable { CacheRead::Problem(problem) => (None, Some(problem)), CacheRead::Found(cached) => (Some(cached), None), }; + // A cache written before a provider was priced by time of day still + // holds models.dev's off-peak rates: drop them as the mapping does. + let cached = cached.map(|mut c| { + if matches!(source, PriceSource::ModelsDev | PriceSource::ModelsDevAt(_)) { + c.table + .entries + .retain(|provider, _| !TIME_OF_DAY_PRICED.contains(&provider.as_str())); + } + c + }); if let Some(Cached { table, age: Some(age), @@ -700,6 +716,16 @@ async fn write_cache(path: &Path, url: &str, table: &PriceTable) -> Result<(), P }) } +/// Providers whose rates depend on the time of day (DeepSeek: half price +/// off-peak). models.dev lists one of the two rates and a flat `CostConfig` +/// cannot express both, so their models are skipped: the bundled snapshot +/// carries the peak rate (an upper bound), and a fetched off-peak rate must +/// not silently replace it. +const TIME_OF_DAY_PRICED: &[&str] = &["deepseek"]; + +const TIME_OF_DAY_REASON: &str = "priced by time of day (peak and off-peak): a flat \ + CostConfig cannot express it; the bundled snapshot carries the peak rate"; + /// One models.dev `cost` object as a [`CostConfig`], or why it cannot be /// expressed as one. fn map_models_dev_cost(cost: &Value) -> Result { diff --git a/tests/fixtures/models_dev_slice.json b/tests/fixtures/models_dev_slice.json index b120226c..64cc2be6 100644 --- a/tests/fixtures/models_dev_slice.json +++ b/tests/fixtures/models_dev_slice.json @@ -214,10 +214,26 @@ "output": 393216 }, "cost": { - "input": 0.435, - "output": 0.87, - "reasoning": 0.87, - "cache_read": 0.003625 + "input": 0.66, + "output": 1.98, + "reasoning": 1.98, + "cache_read": 0.022 + } + }, + "deepseek-v4-flash": { + "id": "deepseek-v4-flash", + "name": "DeepSeek V4 Flash (served by V4.1 Flash)", + "reasoning": true, + "tool_call": true, + "limit": { + "context": 1000000, + "output": 393216 + }, + "cost": { + "input": 0.15, + "output": 0.6, + "reasoning": 0.6, + "cache_read": 0.003 } } } diff --git a/tests/price_audit.rs b/tests/price_audit.rs index 903bb59d..d5267b84 100644 --- a/tests/price_audit.rs +++ b/tests/price_audit.rs @@ -90,7 +90,7 @@ const TIER_KEYS: [&str; 2] = ["tiers", "context_over_200k"]; /// from the entries themselves, so it cannot notice the file shrinking — an /// empty table produced "0 compared, 0 drifted" and a green pass. Raise this /// when you add an entry. -const MIN_ENTRIES: usize = 13; +const MIN_ENTRIES: usize = 16; /// A `prices.json` entry and where to look when it drifts. struct Preset { @@ -156,8 +156,49 @@ struct FlatRateGap { why: &'static str, } -/// Recorded gaps, as `(provider, model, gap)`. Empty today. -const FLAT_RATE_GAPS: &[(&str, &str, FlatRateGap)] = &[]; +/// DeepSeek bills reasoning tokens as output; models.dev lists them as a +/// separate `reasoning` rate equal to its output rate. +const DEEPSEEK_REASONING: &str = + "models.dev's `reasoning` rate is its output rate: DeepSeek bills \ + reasoning tokens as output tokens (its pricing page lists no separate rate)."; + +/// Recorded gaps, as `(provider, model, gap)`. +const FLAT_RATE_GAPS: &[(&str, &str, FlatRateGap)] = &[ + ( + "deepseek", + "deepseek-flash", + FlatRateGap { + keys: &["reasoning"], + rates: &[("/reasoning", 0.6), ("/output", 0.6)], + why: DEEPSEEK_REASONING, + }, + ), + ( + "deepseek", + "deepseek-v4-flash", + FlatRateGap { + keys: &["reasoning"], + rates: &[("/reasoning", 0.6), ("/output", 0.6)], + why: DEEPSEEK_REASONING, + }, + ), + ( + "deepseek", + "deepseek-v4-pro", + FlatRateGap { + keys: &["reasoning"], + rates: &[("/reasoning", 1.98), ("/output", 1.98)], + why: DEEPSEEK_REASONING, + }, + ), +]; + +/// Vendors that bill a peak rate and half of it off-peak, which a flat +/// `CostConfig` cannot express: the crate carries the peak rate (an upper +/// bound) and models.dev the off-peak one. For these, `input`, `output` and +/// `cache_read` must be exactly twice models.dev's — asserted, not waived: if +/// models.dev starts listing the peak rate, or the ratio changes, it is drift. +const PEAK_IS_TWICE_UPSTREAM: &[&str] = &["deepseek"]; /// models.dev's provider key for one of ours. The two agree except where /// models.dev names the vendor differently. @@ -219,6 +260,9 @@ fn every_priced_preset_is_an_audited_entry() { ModelConfig::gpt_6_luna(), ModelConfig::meta("muse-spark-1.1", "Muse Spark 1.1"), ModelConfig::meta("muse-spark-1.2", "Muse Spark 1.2"), + ModelConfig::deepseek("deepseek-flash", "DeepSeek Flash"), + ModelConfig::deepseek("deepseek-v4-flash", "DeepSeek V4 Flash"), + ModelConfig::deepseek("deepseek-v4-pro", "DeepSeek V4 Pro"), ] { let entry = table .entry(&config.provider, &config.id) @@ -353,6 +397,11 @@ fn audit(db: &serde_json::Value, all: &[Preset]) -> Outcome { let mut unexpectedly_missing: Vec = Vec::new(); let mut notes: Vec = Vec::new(); let (mut compared, mut absent) = (0usize, 0usize); + for provider in PEAK_IS_TWICE_UPSTREAM { + notes.push(format!( + "{provider}: peak rates, compared as twice models.dev's off-peak ones" + )); + } println!( "\n{:<30} {:<14} {:>10} {:>12}", @@ -463,18 +512,28 @@ fn audit(db: &serde_json::Value, all: &[Preset]) -> Outcome { Upstream::Value(theirs) => { compared += 1; drift.extend(stale_allowance(p, name)); - let same = (ours - theirs).abs() < 1e-9; + // The peak rate, against models.dev's off-peak one. + let twice = PEAK_IS_TWICE_UPSTREAM.contains(&p.provider.as_str()) + && name != "cache_write"; + let expected = if twice { 2.0 * theirs } else { theirs }; + let same = (ours - expected).abs() < 1e-9; println!( - "{:<30} {:<14} {:>10} {:>12} {}", + "{:<30} {:<14} {:>10} {:>12} {}{}", p.model, name, ours, theirs, - if same { "ok" } else { "DRIFT" } + if same { "ok" } else { "DRIFT" }, + if twice { " (peak = 2 × off-peak)" } else { "" } ); if !same { + let rule = if twice { + format!(" (expected the peak rate, {expected}: twice models.dev's off-peak)") + } else { + String::new() + }; drift.push(format!( - "{}: {name} is {ours} in the crate, {theirs} in models.dev — \ + "{}: {name} is {ours} in the crate, {theirs} in models.dev{rule} — \ check {} and edit {} if the vendor agrees", p.model, p.vendor_page, p.constructor )); @@ -748,7 +807,14 @@ fn the_audit_logic_runs_offline_against_the_fixture() { let db: serde_json::Value = serde_json::from_str(FIXTURE).unwrap(); let all = fixture_presets(&db); let names: Vec<&str> = all.iter().map(|p| p.model.as_str()).collect(); - for covered in ["claude-opus-5-5", "gpt-5.5", "gpt-6-sol", "muse-spark-1.2"] { + for covered in [ + "claude-opus-5-5", + "gpt-5.5", + "gpt-6-sol", + "muse-spark-1.2", + "deepseek-flash", + "deepseek-v4-pro", + ] { assert!( names.contains(&covered), "fixture lost {covered}: {names:?}" @@ -769,8 +835,8 @@ fn the_audit_logic_runs_offline_against_the_fixture() { o.notes ); // Unlisted cache_write accepted by the allowance: gpt-5.5 base and tier, - // and Muse Spark 1.2. - assert_eq!(o.absent, 3); + // Muse Spark 1.2, and the three DeepSeek entries. + assert_eq!(o.absent, 6); // Positive controls: each kind of drift is caught. let drift_after = |mutate: &dyn Fn(&mut Preset)| { @@ -800,6 +866,28 @@ fn the_audit_logic_runs_offline_against_the_fixture() { }); // Base, tier and mirror cache_write are now unexplained. assert_eq!(allowance.len(), 3, "{allowance:?}"); + // A peak-priced vendor's entry at the off-peak rate is drift, not a match. + let off_peak = drift_after(&|p| { + if p.model == "deepseek-flash" { + p.cost.input_per_million = 0.15; + } + }); + // Its cache_write (the old input rate) no longer matches the input either. + assert_eq!(off_peak.len(), 2, "{off_peak:?}"); + assert!( + off_peak + .iter() + .any(|d| d.contains("twice models.dev's off-peak")), + "{off_peak:?}" + ); + // Without the recorded gap, models.dev's `reasoning` rate is unknown + // structure. + let reasoning = drift_after(&|p| { + if p.model == "deepseek-v4-pro" { + p.flat_rate_gap = None; + } + }); + assert_eq!(reasoning.len(), 1, "{reasoning:?}"); } /// An entry models.dev lacks is drift unless it records `absent_upstream` — @@ -846,8 +934,19 @@ async fn models_dev_source_still_maps() { "only {} models mapped — has models.dev's schema changed?", fetched.len() ); + // Time-of-day-priced providers are skipped by the mapping on purpose (its + // off-peak rate must not replace the snapshot's peak one), and must stay so. + for (p, m, _) in builtin.iter() { + if PEAK_IS_TWICE_UPSTREAM.contains(&p) { + assert!( + fetched.entry(p, m).is_none(), + "{p}/{m}: the models.dev mapping took an off-peak rate" + ); + } + } let missing: Vec = builtin .iter() + .filter(|(p, _, _)| !PEAK_IS_TWICE_UPSTREAM.contains(p)) .filter(|(p, m, e)| e.absent_upstream.is_none() && fetched.entry(p, m).is_none()) .map(|(p, m, _)| format!("{p}/{m}")) .collect(); diff --git a/tests/price_fetch_test.rs b/tests/price_fetch_test.rs index 0819c67b..d58bbf55 100644 --- a/tests/price_fetch_test.rs +++ b/tests/price_fetch_test.rs @@ -134,13 +134,10 @@ fn models_dev_fixture_maps_into_our_schema() { assert_eq!(gpt.cache_write_per_million, 0.0); let gemini = t.cost("google", "gemini-2.5-pro").unwrap(); assert_eq!(gemini.context_tiers[0].above_prompt_tokens, 200_000); - // reasoning == output is expressible. - assert_eq!( - t.cost("deepseek", "deepseek-v4-pro") - .unwrap() - .output_per_million, - 0.87 - ); + // DeepSeek bills by time of day: never taken from models.dev (it lists + // the off-peak rate), so the snapshot's peak rate stands. + assert!(t.cost("deepseek", "deepseek-v4-pro").is_none()); + assert!(t.cost("deepseek", "deepseek-flash").is_none()); // alibaba is our qwen. assert!(t.cost("qwen", "qwen-flash").is_some()); assert!(t.cost("alibaba", "qwen-flash").is_none()); @@ -149,7 +146,7 @@ fn models_dev_fixture_maps_into_our_schema() { .cost("openrouter", "perplexity/sonar-deep-research") .is_none()); assert!(t.cost("poe", "cerebras/qwen3-32b-cs").is_none()); - assert_eq!(t.len(), 11); + assert_eq!(t.len(), 9); // Against the built-in data: every overlapping model agrees as billed // (gpt-5.5's implicit cache_write equals its explicit input-rate one), so @@ -164,9 +161,10 @@ fn models_dev_fixture_maps_into_our_schema() { .map(ToString::to_string) .collect::>() ); + // A model the snapshot lacks is reported as new. assert!(changes .iter() - .any(|c| c.provider == "deepseek" && c.model == "deepseek-flash")); + .any(|c| c.provider == "google" && c.model == "gemini-2.5-pro")); } #[test] @@ -186,7 +184,7 @@ fn a_changed_models_dev_envelope_is_an_error_not_an_empty_table() { fn the_models_dev_report_says_what_was_skipped_and_why() { let _g = exclusive(); let report = PriceTable::from_models_dev_json_report(MODELS_DEV).unwrap(); - assert_eq!(report.table.len(), 11); + assert_eq!(report.table.len(), 9); let reason = |p: &str, m: &str| { report .skipped @@ -197,7 +195,10 @@ fn the_models_dev_report_says_what_was_skipped_and_why() { }; assert!(reason("openrouter", "perplexity/sonar-deep-research").contains("reasoning")); assert!(reason("poe", "cerebras/qwen3-32b-cs").contains("no cost")); - assert_eq!(report.skipped.len(), 2); + for m in ["deepseek-flash", "deepseek-v4-flash", "deepseek-v4-pro"] { + assert!(reason("deepseek", m).contains("time of day")); + } + assert_eq!(report.skipped.len(), 5); assert!(report.ignored_fields.is_empty()); } @@ -280,7 +281,7 @@ async fn fetches_models_dev_format_from_a_url() { } expected }); - assert_eq!(report.skipped.len(), 2); + assert_eq!(report.skipped.len(), 5); } /// Item 2: a `Url` source is usually hand-maintained, so it is strict. @@ -417,9 +418,9 @@ async fn an_expired_cache_is_refetched_and_rewritten() { "{:?}", got.origin ); - assert_eq!(got.table.len(), 11); + assert_eq!(got.table.len(), 9); // The models.dev skip list reaches the caller. - assert_eq!(got.skipped.len(), 2); + assert_eq!(got.skipped.len(), 5); // The cache now holds the mapped table and serves it, fresh. let again = PriceTable::fetch_cached(&source, &cache, opts(DAY, DAY)).await; assert!(matches!(again.origin, PriceOrigin::Cache { .. })); @@ -722,3 +723,44 @@ async fn an_unreadable_cache_is_reported() { assert!(problem.is_none()); assert_eq!(logged, 0); } + +/// A models.dev cache written before DeepSeek was priced by time of day +/// holds its off-peak rates: they must not come back through the cache, +/// fresh or stale, any more than through a fetch. +#[tokio::test] +async fn a_cached_models_dev_table_never_brings_back_off_peak_deepseek() { + let _g = serial().await; + let dir = tempfile::tempdir().unwrap(); + let cache = dir.path().join("prices.json"); + let down = serve("/api.json", ResponseTemplate::new(500)).await; + let source = PriceSource::ModelsDevAt(format!("{}/api.json", down.uri())); + let old = r#"{"schema": 1, "providers": { + "anthropic": {"claude-sonnet-5": {"input": 1.5, "output": 7.5}}, + "deepseek": {"deepseek-flash": {"input": 0.15, "output": 0.6, "cache_read": 0.003}}}}"#; + write_cache(&cache, &source, old); + + // Fresh: used without a request, DeepSeek dropped. + let got = PriceTable::fetch_cached(&source, &cache, opts(DAY, DAY)).await; + assert!( + matches!(got.origin, PriceOrigin::Cache { .. }), + "{:?}", + got.origin + ); + assert!(got.table.cost("deepseek", "deepseek-flash").is_none()); + assert!(got.table.cost("anthropic", "claude-sonnet-5").is_some()); + + // Stale, the fetch failing: the same. + let got = PriceTable::fetch_cached(&source, &cache, opts(Duration::ZERO, Duration::MAX)).await; + assert!( + matches!(got.origin, PriceOrigin::StaleCache { .. }), + "{:?}", + got.origin + ); + assert!(got.table.cost("deepseek", "deepseek-flash").is_none()); + + // A source that is not models.dev keeps whatever it lists. + let mine = url_source(&down); + write_cache(&cache, &mine, old); + let got = PriceTable::fetch_cached(&mine, &cache, opts(DAY, DAY)).await; + assert!(got.table.cost("deepseek", "deepseek-flash").is_some()); +} diff --git a/tests/price_override_test.rs b/tests/price_override_test.rs index 702f3562..a9fc4848 100644 --- a/tests/price_override_test.rs +++ b/tests/price_override_test.rs @@ -28,7 +28,7 @@ fn sonnet_override() -> PriceTable { table( r#"{"schema": 1, "providers": { "anthropic": {"claude-sonnet-5": {"input": 1.8, "output": 9.0, "cache_read": 0.18, "cache_write": 2.25}}, - "deepseek": {"deepseek-flash": {"input": 0.3, "output": 1.2, "cache_read": 0.006}}}}"#, + "mistral": {"mistral-large-latest": {"input": 0.3, "output": 1.2, "cache_read": 0.006}}}}"#, ) } @@ -91,7 +91,9 @@ fn user_override_beats_builtin_per_model() { let builtin_sonnet = ModelConfig::claude_sonnet_5().cost; assert_eq!(builtin_sonnet.as_ref().unwrap().input_per_million, 2.0); // Positive control: unlisted before the override. - assert!(ModelConfig::deepseek("deepseek-flash", "D").cost.is_none()); + assert!(ModelConfig::mistral("mistral-large-latest", "M") + .cost + .is_none()); let _ = global::install_override(sonnet_override()); @@ -113,7 +115,10 @@ fn user_override_beats_builtin_per_model() { PriceTable::builtin().cost("anthropic", "claude-opus-5") ); // And can price a model the built-in data does not know. - assert_eq!(input(ModelConfig::deepseek("deepseek-flash", "D")), 0.3); + assert_eq!( + input(ModelConfig::mistral("mistral-large-latest", "M")), + 0.3 + ); // Gateways still never look up, whatever the table says. assert!(ModelConfig::opencode_zen("claude-sonnet-5").cost.is_none()); assert_eq!( @@ -126,7 +131,9 @@ fn user_override_beats_builtin_per_model() { global::clear_override(); assert_eq!(ModelConfig::claude_sonnet_5().cost, builtin_sonnet); - assert!(ModelConfig::deepseek("deepseek-flash", "D").cost.is_none()); + assert!(ModelConfig::mistral("mistral-large-latest", "M") + .cost + .is_none()); } /// A cost set on a config is not touched by later installs (only @@ -162,13 +169,13 @@ fn install_override_reports_changes_reverts_and_inert_entries() { global::install_override(table( r#"{"schema": 1, "providers": { "anthropic": {"claude-sonnet-5": {"input": 1.8, "output": 9.0, "cache_read": 0.18, "cache_write": 2.25}}, - "deepseek": {"deepseek-flash": {"input": 0.3, "output": 1.2}}, + "mistral": {"mistral-large-latest": {"input": 0.3, "output": 1.2}}, "openrouter": {"x/y": {"input": 1, "output": 2}}}}"#, )) }); assert_eq!( names(&report.changes), - ["anthropic/claude-sonnet-5", "deepseek/deepseek-flash"] + ["anthropic/claude-sonnet-5", "mistral/mistral-large-latest"] ); // An inert entry is reported, never as a change. assert_eq!(report.inert, ["openrouter/x/y"]); @@ -180,7 +187,7 @@ fn install_override_reports_changes_reverts_and_inert_entries() { assert_eq!(sonnet.before.as_ref().unwrap().input_per_million, 2.0); assert!(report.reverted.is_empty()); - // Replacing it: deepseek-flash reverts to unpriced, sonnet reverts to + // Replacing it: mistral-large-latest reverts to unpriced, sonnet reverts to // its built-in price, and the replacement is warned about. let (report, warns) = warns_of(|| { global::install_override(table( @@ -194,14 +201,14 @@ fn install_override_reports_changes_reverts_and_inert_entries() { names(&report.reverted), [ "anthropic/claude-sonnet-5", - "deepseek/deepseek-flash", + "mistral/mistral-large-latest", "openrouter/x/y" ] ); let flash = report .reverted .iter() - .find(|c| c.model == "deepseek-flash") + .find(|c| c.model == "mistral-large-latest") .unwrap(); assert!(flash.after.is_none()); assert!( @@ -259,7 +266,7 @@ fn an_override_after_a_fetch_reports_against_the_fetched_price() { let _ = global::install_fetched(table( r#"{"schema": 1, "providers": { "anthropic": {"claude-sonnet-5": {"input": 1.5, "output": 7.5}}, - "deepseek": {"deepseek-flash": {"input": 0.15, "output": 0.6, + "mistral": {"mistral-large-latest": {"input": 0.15, "output": 0.6, "tiers": [{"above_prompt_tokens": 128000, "input": 0.3, "output": 1.2}]}}}}"#, )); let builtin = PriceTable::builtin(); @@ -267,7 +274,7 @@ fn an_override_after_a_fetch_reports_against_the_fetched_price() { global::install_override(table(&format!( r#"{{"schema": 1, "providers": {{ "anthropic": {{"claude-sonnet-5": {}}}, - "deepseek": {{"deepseek-flash": {{"input": 0.15, "output": 0.6}}}}}}}}"#, + "mistral": {{"mistral-large-latest": {{"input": 0.15, "output": 0.6}}}}}}}}"#, serde_json::to_string(&serde_json::json!({ "input": 2.0, "output": 10.0, "cache_read": 0.2, "cache_write": 2.5 })) @@ -288,12 +295,12 @@ fn an_override_after_a_fetch_reports_against_the_fetched_price() { sonnet[0].after, builtin.cost("anthropic", "claude-sonnet-5") ); - // Only the fetched layer had deepseek-flash's tier; dropping it warns. + // Only the fetched layer had mistral-large-latest's tier; dropping it warns. assert!( report .warnings .iter() - .any(|w| w.contains("deepseek/deepseek-flash drops the 1 context tier")), + .any(|w| w.contains("mistral/mistral-large-latest drops the 1 context tier")), "{:?}", report.warnings ); @@ -305,17 +312,17 @@ fn install_fetched_reports_against_the_previous_state_and_marks_shadowed() { let first = table( r#"{"schema": 1, "providers": { "anthropic": {"claude-sonnet-5": {"input": 1.5, "output": 7.5}}, - "deepseek": {"deepseek-flash": {"input": 0.15, "output": 0.6}}}}"#, + "mistral": {"mistral-large-latest": {"input": 0.15, "output": 0.6}}}}"#, ); let changes = global::install_fetched(first); assert_eq!( names(&changes), - ["anthropic/claude-sonnet-5", "deepseek/deepseek-flash"] + ["anthropic/claude-sonnet-5", "mistral/mistral-large-latest"] ); assert!(changes.iter().all(|c| !c.shadowed)); // The user layer overrides sonnet; a new fetched layer that drops - // deepseek-flash and moves sonnet again reports both — the revert, and + // mistral-large-latest and moves sonnet again reports both — the revert, and // sonnet's change marked as shadowed (it bills at the override). let _ = global::install_override(table( r#"{"schema": 1, "providers": {"anthropic": {"claude-sonnet-5": {"input": 1.0, "output": 5.0}}}}"#, @@ -327,7 +334,7 @@ fn install_fetched_reports_against_the_previous_state_and_marks_shadowed() { }); assert_eq!( names(&changes), - ["anthropic/claude-sonnet-5", "deepseek/deepseek-flash"] + ["anthropic/claude-sonnet-5", "mistral/mistral-large-latest"] ); let sonnet = changes .iter() @@ -338,7 +345,7 @@ fn install_fetched_reports_against_the_previous_state_and_marks_shadowed() { assert!(sonnet.to_string().ends_with("[shadowed by the user layer]")); let flash = changes .iter() - .find(|c| c.model == "deepseek-flash") + .find(|c| c.model == "mistral-large-latest") .unwrap(); assert!(!flash.shadowed); assert!(flash.after.is_none(), "reverted to unpriced"); @@ -358,30 +365,36 @@ fn install_fetched_reports_against_the_previous_state_and_marks_shadowed() { fn add_only_merges_and_never_overrides() { let _g = exclusive(); let _ = global::install_fetched(table( - r#"{"schema": 1, "providers": {"deepseek": {"deepseek-flash": {"input": 0.15, "output": 0.6}}}}"#, + r#"{"schema": 1, "providers": {"mistral": {"mistral-large-latest": {"input": 0.15, "output": 0.6}}}}"#, )); let (changes, warns) = warns_of(|| { global::install_fetched_with( table( r#"{"schema": 1, "providers": { "anthropic": {"claude-sonnet-5": {"input": 1.5, "output": 7.5}}, - "deepseek": {"deepseek-flash": {"input": 9, "output": 9}, - "deepseek-v4-pro": {"input": 0.435, "output": 0.87}}}}"#, + "mistral": {"mistral-large-latest": {"input": 9, "output": 9}, + "mistral-small-latest": {"input": 0.435, "output": 0.87}}}}"#, ), InstallPolicy::AddOnly, ) }); // Only the model neither the built-in data nor the current fetched layer // listed. - assert_eq!(names(&changes), ["deepseek/deepseek-v4-pro"]); + assert_eq!(names(&changes), ["mistral/mistral-small-latest"]); assert!(warns.is_empty(), "{warns:?}"); assert_eq!( ModelConfig::claude_sonnet_5().cost, PriceTable::builtin().cost("anthropic", "claude-sonnet-5") ); // The earlier fetched entry survived (merged, not replaced). - assert_eq!(input(ModelConfig::deepseek("deepseek-flash", "D")), 0.15); - assert_eq!(input(ModelConfig::deepseek("deepseek-v4-pro", "D")), 0.435); + assert_eq!( + input(ModelConfig::mistral("mistral-large-latest", "M")), + 0.15 + ); + assert_eq!( + input(ModelConfig::mistral("mistral-small-latest", "D")), + 0.435 + ); // Positive control: the default policy replaces the layer and overrides. let changes = global::install_fetched_with( @@ -394,12 +407,14 @@ fn add_only_merges_and_never_overrides() { names(&changes), [ "anthropic/claude-sonnet-5", - "deepseek/deepseek-flash", - "deepseek/deepseek-v4-pro" + "mistral/mistral-large-latest", + "mistral/mistral-small-latest" ] ); assert_eq!(input(ModelConfig::claude_sonnet_5()), 1.5); - assert!(ModelConfig::deepseek("deepseek-flash", "D").cost.is_none()); + assert!(ModelConfig::mistral("mistral-large-latest", "M") + .cost + .is_none()); } /// Disagreements with the bundled snapshot are logged: the first five, then @@ -467,7 +482,7 @@ fn precedence_user_over_fetched_over_builtin() { fn reprice_repeats_the_constructor_lookup() { let _g = exclusive(); let _ = global::install_override(sonnet_override()); - let flash = ModelConfig::deepseek("deepseek-flash", "D"); + let flash = ModelConfig::mistral("mistral-large-latest", "M"); let sonnet = ModelConfig::claude_sonnet_5(); let generic = ModelConfig::anthropic("claude-sonnet-5", "S"); let gateway_table = table(