Skip to content

feat: Connections hub — unified provider management with model mixing #470

Description

@rexlunae

Inspiration: Unsloth

Unsloth's Connections feature provides a polished UX for managing all model providers in one place:

  • Cloud providers (OpenAI, Anthropic, OpenRouter) with API keys
  • Model servers (llama.cpp, vLLM, Ollama) with base URLs
  • Unified model dropdown: Shows local, cloud, and external models in a single sorted list
  • Prompt caching indicators: Shows which models support prompt caching
  • Per-provider capabilities: Dynamically exposes reasoning levels, image support, thinking modes
  • Provider-native features: OpenAI web search, Anthropic code execution passed through

Current RustyClaw state

RustyClaw has a provider registry in providers.rs with support for Anthropic, OpenAI, Google, xAI, OpenRouter, Ollama, and custom providers. But:

  • No UI for managing providers beyond the onboard wizard
  • No runtime add/remove of providers
  • No capability discovery (just hardcoded per-provider)
  • Ollama integration exists but is separate from the main provider UX

What RustyClaw could do

  1. Connections panel in TUI/Desktop: Add/remove/edit providers with test-connection button
  2. Runtime provider CRUD: Add new providers without restarting the gateway
  3. Capability detection: Auto-detect model capabilities (vision, thinking, tool use, prompt caching) via provider API
  4. Mixed model chat: Let a session switch models mid-conversation (useful for: "this model's bad at math, switch to o1 for this query")
  5. Provider health: Show latency, error rate, rate limit status per provider

Scope

  • rustyclaw-tui + rustyclaw-desktop: Provider management panels
  • rustyclaw-core: Runtime provider add/remove + capability detection
  • rustyclaw-core: Provider health tracking (latency histograms, error counters)

Prior art

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions