Support per-provider API keys and fallback routing

[roykollensvendsen]

Problem

The current openclaw.json.template uses a single ${CLAWBENCH_LLM_BASE_URL} and ${CLAWBENCH_LLM_API_KEY} for all providers. This means only one provider can be active at a time, with no fallback if it goes down.

We've seen this cause subnet-wide evaluation failures:

• Chutes: 429 - Infrastructure is at maximum capacity → all miners rejected fail-closed
• ZhiPu: 401 - 余额不足 (insufficient balance) → all miners rejected fail-closed

In both cases, a working alternative provider existed but couldn't be used.

Current state

"chutes": { "baseUrl": "${CLAWBENCH_LLM_BASE_URL}", "apiKey": "${CLAWBENCH_LLM_API_KEY}" },
"zhipu":  { "baseUrl": "${CLAWBENCH_LLM_BASE_URL}", "apiKey": "${CLAWBENCH_LLM_API_KEY}" },
"openrouter": { "baseUrl": "${CLAWBENCH_LLM_BASE_URL}", "apiKey": "${CLAWBENCH_LLM_API_KEY}" }

All providers share the same URL and key — only the one matching ${CLAWBENCH_DEFAULT_MODEL} is actually used.

Suggestion

1. Per-provider env vars in template

"chutes": { "baseUrl": "${CHUTES_BASE_URL}", "apiKey": "${CHUTES_API_KEY}" },
"zhipu":  { "baseUrl": "${ZHIPU_BASE_URL}", "apiKey": "${ZHIPU_API_KEY}" },
"openrouter": { "baseUrl": "${OPENROUTER_BASE_URL}", "apiKey": "${OPENROUTER_API_KEY}" }

2. Fallback routing in validator

If the primary provider returns 429/401/5xx, retry with a fallback provider before fail-closed reject. This could be configured as:

CLAWBENCH_DEFAULT_MODEL=chutes/zai-org/GLM-5-TEE
CLAWBENCH_FALLBACK_MODEL=zhipu/glm-5

3. Consistent reference pricing

With multiple providers active, cost reporting must use a single reference price (as proposed in #29) to keep scoring fair.

Impact

• Validators stay operational when one provider goes down
• Miners don't get banned due to validator infrastructure failures
• Subnet evaluation continues uninterrupted

🤖 Generated with Claude Code