Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@ BOT_TOKEN=your_telegram_bot_token

# ─── AI providers (all REQUIRED — no hardcoded defaults) ────────────────────

# AI primary — z.ai coding endpoint (GLM 5.3 via OpenAI-compat API)
# AI fallback — z.ai coding endpoint (GLM 5.3 via OpenAI-compat API)
ZAI_API_KEY=your_zai_api_key
ZAI_BASE_URL=https://api.z.ai/api/coding/paas/v4
ZAI_MODEL=glm-5.3
ZAI_FAST_MODEL=glm-5.3-flash

# HuggingFace Router (fallback, tool calling capable)
# HuggingFace Router (last fallback, tool calling capable)
HF_TOKEN=your_hf_token
HF_BASE_URL=https://router.huggingface.co/v1
HF_MODEL=Qwen/Qwen3-235B-A22B
HF_FAST_MODEL=meta-llama/Llama-3.3-70B-Instruct
HF_MODEL=meta-llama/Llama-4-Scout-17B-16E-Instruct:deepinfra
HF_FAST_MODEL=meta-llama/Llama-4-Scout-17B-16E-Instruct:deepinfra

# Anthropic Claude (optional, paid reserve): tried right after z.ai in both default chains,
# Anthropic Claude (primary when configured): tried first in both default chains,
# skipped when both tokens or the chain's model are unset. OpenAI-compatible endpoint.
# CLAUDE_API_TOKEN=sk-ant-...
# Optional backup: prefer an independent organization/workspace for independent quota.
Expand All @@ -25,20 +25,25 @@ HF_FAST_MODEL=meta-llama/Llama-3.3-70B-Instruct
# deadlines remain in force; only opening the AI request and catalog discovery are retried.
# CLAUDE_MODEL=claude-sonnet-5-5
# CLAUDE_FAST_MODEL=claude-haiku-5-5
# AI_SMART_CHAIN / AI_FAST_CHAIN select providers: selected Claude always leads and HF stays last.
# Other providers retain their relative order; omit Claude to select a chain without it.
# AI_SMART_CHAIN=claude,zai,hf
# AI_FAST_CHAIN=claude,zai,hf

# OpenRouter (optional): DeepSeek and GLM Flash behind one key. In NO default chain: it runs only when
# AI_SMART_CHAIN / AI_FAST_CHAIN name `openrouter` and both the key and that chain's model below are set.
# OPENROUTER_API_KEY=sk-or-...
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
# OPENROUTER_MODEL=deepseek/deepseek-v4.1-flash
# OPENROUTER_MODEL=deepseek/deepseek-v4.1-flash-1017
# OPENROUTER_FAST_MODEL=z-ai/glm-5.3-flash

# Role routing (default legacy = off). In `roles` mode agent rounds use AI_EXECUTOR_CHAIN and validator retries
# AI_REPAIR_CHAIN; entries are `provider` or `provider:preset`, and a preset must be qualified for the role.
# Claude leads the selected role providers. The role precedes legacy fallback; HF always stays last.
# AI_ROUTING=legacy
# AI_ROUTING_USER_IDS=123456789 # canary: roles for these Telegram users only
# AI_EXECUTOR_CHAIN=openrouter:glm53-flash-low,claude
# AI_REPAIR_CHAIN=openrouter:glm53-low,claude
# AI_EXECUTOR_CHAIN=claude,openrouter:glm53-flash-low
# AI_REPAIR_CHAIN=claude,openrouter:glm53-low
# AI_PLANNER_CHAIN=

# Groq: voice transcription (Whisper) only — not an AI chat provider
Expand Down
33 changes: 12 additions & 21 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,13 +242,12 @@ All `process.env.*` reads go through `src/config/env.ts` → `loadConfig()` and

When adding a new env var: (1) add it to `EnvConfig` interface in `env.ts`, (2) read and validate in `loadConfig()`, (3) use via `config.VAR_NAME` everywhere else.

**Model ids are env values, refreshed against the live catalogs** (2026-10-24; `ZAI_*`/`HF_*` are required env, `CLAUDE_*` optional). z.ai coding endpoint `/models`: newest are `glm-5.3` (smart) and `glm-5.3-flash` (fast; `glm-5.3-flashx` is not in the Max plan, code 1311). Both answer 429 code 1310 while the weekly premium quota is exhausted (an unknown id would be 400 code 1211), so a 429 does not mean a bad id; the chain falls through to Claude. `glm-5.1` and `glm-4.7-flash` still exist but are superseded. Anthropic `/v1/models`: `claude-sonnet-5-5` and `claude-haiku-5-5` are the newest Sonnet and Haiku (nothing above 5.5 in those tiers). HF router: `meta-llama/Llama-3.3-70B-Instruct` and `Qwen/Qwen3-235B-A22B` are still listed live; HF has no credits (402), so a replacement cannot be probed and the ids are kept until they disappear from `/models`.
**Model ids are env values, checked against current catalogs** (2026-11-13; `ZAI_*`/`HF_*` are required env, `CLAUDE_*` optional). Claude uses `claude-sonnet-5-5` and `claude-haiku-5-5`; z.ai uses `glm-5.3` and `glm-5.3-flash`. HF examples use `meta-llama/Llama-4-Scout-17B-16E-Instruct:deepinfra`. A catalog entry proves availability in the catalog, not successful inference for an account. HF remains the last reserve and its account funding is handled independently of Claude activation.

**Provider order is configuration, not code**: `AI_SMART_CHAIN` and `AI_FAST_CHAIN` list provider ids (`zai`, `claude`, `hf`, optional `openrouter`); the default for both is z.ai → Claude → HF (the owner's order, identical in ExpenseSyncBot); providers without key/model are skipped. The ids come from `PROVIDER_IDS` in
**Provider selection is configuration**: `AI_SMART_CHAIN` and `AI_FAST_CHAIN` list provider ids (`zai`, `claude`, `hf`, optional `openrouter`); the default for both is Claude → z.ai → HF; selected Claude stays first, selected HF last, and other selected providers keep their relative order; providers without key/model are skipped. The ids come from `PROVIDER_IDS` in
`src/services/ai/provider-ids.ts` — a leaf module that imports nothing, so both the config
layer and the AI layer can derive from it without a cycle. Add a provider there and it is
nameable at once. Gemini was removed in full (2026-10): no id, no `GEMINI_*` variable is read, and a
leftover one in `.env` is ignored. Groq as a chat provider was removed in 2026-10 because gpt-oss tested badly
nameable at once. Groq as a chat provider was removed in 2026-10 because gpt-oss tested badly
and there is no replacement Groq model: no `groq` id, no `GROQ_MODEL`/`GROQ_FAST_MODEL`/`GROQ_TPM_LIMITS`
(leftovers in `.env` are ignored, and `groq` in a chain variable is dropped as an unknown id). `GROQ_API_KEY`
remains ONLY for Whisper voice transcription (`src/services/voice/transcription-service.ts`, initialised in
Expand All @@ -265,12 +264,12 @@ a new order in `streaming.ts`.
`OPENROUTER_API_KEY`, `OPENROUTER_BASE_URL` (default `https://openrouter.ai/api/v1`), `OPENROUTER_MODEL` (smart
chain) and `OPENROUTER_FAST_MODEL` (fast chain); no model default. It is in neither default chain: it runs only
when `AI_SMART_CHAIN` / `AI_FAST_CHAIN` name it AND both the key and that chain's model are set, otherwise its slot
is skipped like any unconfigured provider. Live-verified ids (2026-10-23, `/models`): `deepseek/deepseek-v4.1-flash`
is skipped like any unconfigured provider. Live-verified ids (2026-10-23, `/models`): `deepseek/deepseek-v4.1-flash-1017`
and `z-ai/glm-5.3-flash`. Claude-specific request rules (no `temperature`, the 128k `max_tokens`, the visible-token
cap) apply only to the `claude` provider, so do not route Claude models through OpenRouter in a chain without
measuring. Live-model replacement is off for it (it lists hundreds of
differently priced models), and OpenRouter accepts the `content: null` assistant turn that carries tool calls,
so the transcript is sent unchanged. Never reintroduce gpt-oss, Groq chat or Gemini through it. Its circuit is
so the transcript is sent unchanged. Never reintroduce gpt-oss or Groq chat through it. Its circuit is
scoped to the model (`openrouterCircuitScope`) for rate limits and outages, because OpenRouter routes each model to a different
upstream: a limit on one model must not bench the other. A rejected or spent key (401, 402, a non-moderation 403) opens one circuit for the
whole key instead (`CircuitContext.accountKey`, one notice), and any answer closes both.
Expand Down Expand Up @@ -300,7 +299,7 @@ legacy `smart`/`fast` bench, and its ledger `chain` reads `role:<role>`; the gat
reads `preset.qualifications[provider]`, so a preset measured through OpenRouter does not load on z.ai); a preset slot never runs a
live-model replacement (neither a cached nor a fresh heuristic one) because that would serve the role unmeasured; and a role chain is
followed by the legacy chain (minus models the role chain already asked), so a role chain whose slots all fail, are benched or sit behind an
open circuit at runtime hands the round to the legacy chain instead of answering nobody.
open circuit at runtime hands the round to the legacy chain instead of answering nobody. Claude leads the selected providers within each role or legacy chain; legacy Claude never moves ahead of a non-HF role override. HF slots remain last across the combined chain.

**A depleted or rejected provider account is skipped, not retried**: `src/services/ai/provider-circuit.ts`
keeps one durable circuit per provider account for every provider in the chain (key = fingerprint
Expand Down Expand Up @@ -341,24 +340,16 @@ is never written as 0 and a failed attempt that was billed carries its cost. Rea
provider:model counts, errors by class, token sums with the answers that reported none counted apart, cost (`—` when unknown, with a
count of answers without a cost next to a partial sum), p50/p95, a Roles section of cost and latency per role and provider once any
row has a role (role-less rows are `legacy`), and one verdict per provider that says `no traffic in window` for a provider that served nothing; it refuses a database that has not run 067) and the
admin-only Telegram command `/ai_stats [days]` (same report, `t(lang).aiStats`, fail closed). Claude is second in the
default order and gets no traffic while z.ai answers, so prove it alive with the real code path:
admin-only Telegram command `/ai_stats [days]` (same report, `t(lang).aiStats`, fail closed). Claude is first in the
default order. Prove it alive with the real code path:
`AI_SMART_CHAIN=claude AI_FAST_CHAIN=claude bun scripts/probe-ai-chain.ts --usage-database <temp.db> --expect-provider claude [--smart]`
prints the probe result and the `ai_usage_events` row it wrote (never point it at the live database).
Rows (and the report) of retired providers such as `gemini` and `groq` stay readable: the report lists any provider id
found in the rows after the configured ones. The retired `gemini_usage` table lives only in the sidecar
`<DATABASE_PATH>.gemini-quota.sqlite`, which no code opens any more; it is left on disk untouched (delete the file by hand when convenient).
Historical provider rows remain readable: the report lists any provider id found in the rows after the configured ones. Existing database files remain untouched.

**Cost risk of the order**: while z.ai is out of quota (its circuit is open, skipped silently) Claude is the de-facto
primary and every request is paid for. Claude keeps its `CLAUDE_MAX_OUTPUT_TOKENS` (128000) `max_tokens` and its thinking
**Cost of the order**: Claude is the primary and every successful primary request is billed to its API account. Claude keeps its `CLAUDE_MAX_OUTPUT_TOKENS` (128000) `max_tokens` and its thinking
behaviour unchanged. Watch `bun scripts/report-ai-usage.ts` and set `AI_SMART_CHAIN`/`AI_FAST_CHAIN` (e.g. `zai,hf`, or `hf` alone) if the cost
is not acceptable; the ExpenseSyncBot OCR chain is separate and stays Claude vision → HF vision.
Both fallbacks have run dry before: z.ai's quota ran out on 2026-09-24 (its circuit has been open since), and Hugging Face
answered 402 "no credits" in ExpenseSyncBot on 2026-10-12. With z.ai depleted and HF the only provider behind Claude,
Claude carries the traffic; keep an eye on its spend and on HF credit.
z.ai is first in both chains, so a slow-but-not-failing z.ai delays every call before Claude is tried: up to 8 s per provider on the
fast chain (validators, city resolver), up to 15 s on the smart chain; an open circuit costs nothing. The chain order is the owner's choice — shorten
`ZAI_TIMEOUT_MS` in `clients.ts` or reorder via `AI_FAST_CHAIN` if that latency shows up in `/ai_stats`.
is not acceptable. These variables select providers: Claude stays first when selected and HF stays last, regardless of where they are written; the other providers keep their relative order. The effective smart/fast order is logged once per explicit selection. The ExpenseSyncBot OCR chain is separate and stays Claude vision → HF vision.
The fallback accounts have had quota and credit failures before. Those failures stay visible in usage/error telemetry and do not block the primary. A configured OpenRouter route runs before HF. Circuit-open providers are skipped; the remaining selected priority providers keep their relative order. Fast and smart requests retain their existing per-provider deadlines.

Optional features that depend on an env var must deactivate gracefully when the var is absent — never throw at startup. Validate at the point of use, not at startup.

Expand Down
23 changes: 23 additions & 0 deletions docs/plans/2026-11-13-model-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Claude Primary and Reserve Routing

Task: UB-178. Base revision: `128846b7e4c31667ee41b2fc1dae274e0bef3fcd`.

## Changes

Claude is first among selected providers in smart, fast and role chains. Other priority providers keep their relative order; Hugging Face stays last, including the combined role and legacy fallback chain. An explicit role chain precedes the legacy fallback; legacy Claude cannot displace a qualified role override. Provider selection, credential availability, preset qualification, circuits, deadlines and output validation stay active.

Remove obsolete provider references from active source, scripts, fixtures and current configuration guidance. Historical usage rows remain readable through the existing generic provider handling; no database content is changed.

Keep Claude Sonnet 5.5 and Haiku 5.5. Update the optional DeepSeek preset to `deepseek/deepseek-v4.1-flash-1017` and the Hugging Face examples to `meta-llama/Llama-4-Scout-17B-16E-Instruct:deepinfra`. A provider-pinned Hugging Face route cannot silently change upstream through model discovery. No preset receives a new qualification.

## Verification

Run focused Bun tests against the source overlay with network disabled. Cover configured priority, actual provider fallback order, failure visibility, role fallback, existing output validation, preset pricing and pinned model behavior. Full suite and whole-repository typecheck are omitted under the owner's requested hatch. Live Claude account checks are recorded separately; synthetic tests do not prove production delivery.

## Production Plan

Publish to a source branch first. After separate rollout approval, deploy the approved revision and set both smart and fast chains to `claude,zai,hf` (include configured OpenRouter before HF when desired). Keep both current Claude credentials and 5.5 model IDs. Update the two HF model values above and remove obsolete bot-local provider environment entries without revoking shared keys. HF billing repair is outside this change and does not block Claude activation. Verify a bounded Claude text/tool request and fallback evidence after activation; retain the preceding image and environment snapshot for rollback.

## Incident Timeline

Claude 5.5 integration was committed on 2026-10-10 (`c7daca394994e40b65165f5842bded3daa5b4f16`), with 5.5-only model discovery and thinking-budget compatibility on 2026-10-11 (`481b6356d8d0bf2f668ce9a168cb5cc091fd1c84`). Both precede the November 8 requests reported by Anthropic. Current source and retained logs do not identify this bot as that caller.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "hypercalendarbot"
version = "0.3.7"
version = "0.3.8"
requires-python = ">=3.12"
dependencies = [
"pyrofork>=2.3",
Expand Down
1 change: 0 additions & 1 deletion scripts/real-history/pools.py
Original file line number Diff line number Diff line change
Expand Up @@ -711,7 +711,6 @@
"GPT",
"ChatGPT",
"Claude",
"Gemini",
"ИИ",
"Бот",
"Календарь",
Expand Down
25 changes: 13 additions & 12 deletions src/config/env.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// src/config/env.ts

import { MODEL_ROLES } from '../services/ai/model-presets.ts';
import { PROVIDER_IDS, type ProviderId } from '../services/ai/provider-ids.ts';
import { PROVIDER_IDS, type ProviderId, prioritizeProviders } from '../services/ai/provider-ids.ts';
import {
AI_ROUTING_MODES,
type AiRoutingMode,
Expand All @@ -13,15 +13,12 @@ import { TOOL_SCHEMA_MODES, type ToolSchemaMode } from '../services/ai/tool-sche
import { logger, logOnce } from '../utils/logger.ts';
import { GOOGLE_OAUTH_CALLBACK_PATH, publicUrl } from './public-url.ts';

/** Owner-chosen order for main requests: z.ai, then Claude, then Hugging Face.
* Explicit operator order (AI_SMART_CHAIN) wins; token-fit guards, circuits and fallback stay intact.
* A provider without its key or the chain's model is skipped, so a depleted z.ai falls through to Claude.
*/
export const DEFAULT_SMART_CHAIN: ProviderId[] = ['zai', 'claude', 'hf'];
/** Claude is primary; Hugging Face is the last reserve. Selection stays configurable. */
export const DEFAULT_SMART_CHAIN: ProviderId[] = ['claude', 'zai', 'hf'];
/** Short requests use the same owner-chosen order, with each provider's fast model.
* Explicit operator order (AI_FAST_CHAIN) still wins.
* Selected Claude stays first and Hugging Face stays last.
*/
export const DEFAULT_FAST_CHAIN: ProviderId[] = ['zai', 'claude', 'hf'];
export const DEFAULT_FAST_CHAIN: ProviderId[] = ['claude', 'zai', 'hf'];

export const DEFAULT_OPENROUTER_BASE_URL = 'https://openrouter.ai/api/v1';

Expand Down Expand Up @@ -64,15 +61,19 @@ function parseChain(name: string, fallback: ProviderId[]): ChainOrder {
);
return { order: fallback, fromEnv: false, fallback };
}
return { order: parsed, fromEnv: true, fallback };
const order = prioritizeProviders(parsed, (provider) => provider);
logOnce(`${name}:priority:${parsed.join(',')}`, () =>
logger.info({ name, configured: parsed, effective: order }, 'Effective provider order'),
);
return { order, fromEnv: true, fallback };
}

export interface EnvConfig {
BOT_TOKEN: string;
DATABASE_PATH: string;
NODE_ENV: 'development' | 'production';

// AI primary provider (z.ai coding endpoint)
// AI fallback provider (z.ai coding endpoint)
ZAI_API_KEY: string;
ZAI_BASE_URL: string;
ZAI_MODEL: string;
Expand All @@ -97,7 +98,7 @@ export interface EnvConfig {
MTPROTO_SERVICE_USER_ID?: number;
// Whisper voice transcription only; Groq is not an AI chat provider any more
GROQ_API_KEY?: string;
// Anthropic Claude (paid reserve after z.ai; OpenAI-compatible endpoint)
// Anthropic Claude (primary when configured; OpenAI-compatible endpoint)
CLAUDE_API_TOKEN?: string;
/** Independent backup credential, used with the same Claude models and request settings. */
CLAUDE_API_TOKEN2?: string;
Expand Down Expand Up @@ -261,7 +262,7 @@ export function loadConfig(): EnvConfig {
DATABASE_PATH: process.env.DATABASE_PATH || './data/calendar.db',
NODE_ENV: (process.env.NODE_ENV as EnvConfig['NODE_ENV']) || 'development',

// AI primary (z.ai)
// AI fallback (z.ai)
ZAI_API_KEY: requireEnv('ZAI_API_KEY'),
ZAI_BASE_URL: requireEnv('ZAI_BASE_URL'),
ZAI_MODEL: requireEnv('ZAI_MODEL'),
Expand Down
4 changes: 2 additions & 2 deletions src/database/repositories/ai-usage.repository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ export type AiErrorClass =
| 'other';

/**
* Why a provider was passed over without a request being sent. `local_budget` (Gemini's local quota
* admission, removed in #806) and `preflight_size` (the Groq token-fit preflight, removed with Groq as
* Why a provider was passed over without a request being sent. `local_budget` (retired local quota
* admission) and `preflight_size` (the Groq token-fit preflight, removed with Groq as
* a chat provider) are retired and never written any more; they stay so that historical rows keep
* their type.
*/
Expand Down
6 changes: 3 additions & 3 deletions src/services/ai/model-presets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,18 +59,18 @@ export const HISTORY_PASS_REQUIRED = 1;
const OPENROUTER_PRICE_SOURCE = 'OpenRouter /models list price; billed cost can differ, a reported usage.cost wins';

/**
* Seeded from the 2026-09-27 benchmark (PR #429) and the OpenRouter model ids verified on 2026-10-23.
* Seeded from the 2026-09-27 benchmark (PR #429); each route's price records its verification date.
* No preset is qualified: the benchmark was through another host, on reconstructed scenarios.
*/
export const MODEL_PRESETS: readonly ModelPreset[] = [
{
id: 'deepseek41-flash',
models: { openrouter: 'deepseek/deepseek-v4.1-flash' },
models: { openrouter: 'deepseek/deepseek-v4.1-flash-1017' },
prices: {
openrouter: {
inputUsdPerMTok: 0.3,
outputUsdPerMTok: 1.2,
asOf: '2026-10-23',
asOf: '2026-11-13',
source: OPENROUTER_PRICE_SOURCE,
},
},
Expand Down
Loading
Loading