Skip to content

feat(llm): add per-model reasoning effort #676

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
allenxu09 wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(llm): add per-model reasoning effort #676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9ce0bbf
feat(llm): add per-model reasoning effort
allenxu09 Aug 2, 2026
5243807
fix(config): address reasoning effort feedback
allenxu09 Aug 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ ocr config model # Pick a model for the active provider

![Provider setup](imgs/providers.jpg)

The interactive UI guides you through provider selection, API key entry, and model configuration, then automatically tests connectivity.
The interactive UI guides you through provider selection, model-specific reasoning effort, and API key entry, then automatically tests connectivity. Choose **Provider default** to omit the effort field.

For CLI setup, environment variables, custom providers, and other advanced configuration, see [Configuration](https://open-codereview.ai/docs/configuration).

Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ ocr config model # 为当前供应商选择模型

![Provider setup](imgs/providers.jpg)

交互式界面会引导你完成供应商选择、API Key 输入和模型配置，完成后自动测试连通性。
交互式界面会引导你完成供应商选择、模型专属的 reasoning effort 和 API Key 输入，完成后自动测试连通性。选择 **Provider 默认值** 时不会发送 effort 字段。

命令行设置、环境变量、自定义供应商等高级配置，详见[配置指南](https://open-codereview.ai/docs/configuration)。

Expand Down
8 changes: 8 additions & 0 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,13 @@ inputs:
llm_model:
description: Model name (mapped to env OCR_LLM_MODEL).
required: true
llm_reasoning_effort:
description: >-
Optional model reasoning effort (mapped to env
OCR_LLM_REASONING_EFFORT). Supported values depend on the selected
protocol and model. Anthropic xhigh/max may require overriding
llm_extra_body to remove disabled thinking.
required: false
llm_use_anthropic:
description: "'true' for Anthropic Claude, 'false' for OpenAI-compatible APIs
(mapped to env OCR_USE_ANTHROPIC). Required to force an explicit choice."
Expand Down Expand Up @@ -267,6 +274,7 @@ runs:
OCR_LLM_URL: ${{ inputs.llm_url }}
OCR_LLM_TOKEN: ${{ inputs.llm_auth_token }}
OCR_LLM_MODEL: ${{ inputs.llm_model }}
OCR_LLM_REASONING_EFFORT: ${{ inputs.llm_reasoning_effort }}
OCR_USE_ANTHROPIC: ${{ inputs.llm_use_anthropic }}
OCR_LLM_AUTH_HEADER: ${{ inputs.llm_auth_header }}
OCR_LLM_EXTRA_HEADERS: ${{ inputs.llm_extra_headers }}
Expand Down
172 changes: 146 additions & 26 deletions cmd/opencodereview/config_cmd.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Examples:
# Provider setup (non-interactive)
ocr config set provider anthropic
ocr config set model claude-opus-4-6
ocr config set reasoning_effort high
ocr config set providers.anthropic.api_key "$ANTHROPIC_API_KEY"

# Custom provider
Expand All @@ -37,7 +38,7 @@ Examples:
var configSetCmd = &cobra.Command{
Use: "set <key> <value>",
Short: "Set a configuration value",
Example: " ocr config set llm.model claude-opus-4-6\n ocr config set provider anthropic",
Example: " ocr config set llm.model claude-opus-4-6\n ocr config set provider anthropic\n ocr config set reasoning_effort high",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
return runConfigSet(args[0], args[1])
Expand All @@ -47,8 +48,8 @@ var configSetCmd = &cobra.Command{
var configUnsetCmd = &cobra.Command{
Use: "unset <key>",
Short: "Remove a configuration value",
Long: "Remove a provider, custom_providers.<name>, or mcp_servers.<name>.",
Example: " ocr config unset provider\n ocr config unset custom_providers.my-provider\n ocr config unset mcp_servers.github",
Long: "Remove a provider, the active model's reasoning_effort, custom_providers.<name>, or mcp_servers.<name>.",
Example: " ocr config unset provider\n ocr config unset reasoning_effort\n ocr config unset custom_providers.my-provider\n ocr config unset mcp_servers.github",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return runConfigUnset(args[0])
Expand All @@ -66,7 +67,7 @@ var configProviderCmd = &cobra.Command{

var configModelCmd = &cobra.Command{
Use: "model",
Short: "Interactive model selection",
Short: "Interactive model and reasoning effort selection",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
return runConfigModel()
Expand Down Expand Up @@ -139,10 +140,24 @@ func runConfigUnset(key string) error {
if key == "provider" {
return unsetActiveProvider(configPath)
}
if key == "reasoning_effort" {
cfg, err := loadOrCreateConfig(configPath)
if err != nil {
return fmt.Errorf("load config: %w", err)
}
if err := setConfigValue(cfg, key, "default"); err != nil {
return err
}
Comment thread
allenxu09 marked this conversation as resolved.
if err := saveConfig(configPath, cfg); err != nil {
return err
}
fmt.Println("Unset reasoning_effort for the active model.")
return nil
}

parts := strings.SplitN(key, ".", 2)
if len(parts) != 2 || parts[1] == "" {
return fmt.Errorf("unset supports provider, custom_providers.<name>, and mcp_servers.<name>")
return fmt.Errorf("unset supports provider, reasoning_effort, custom_providers.<name>, and mcp_servers.<name>")
}

switch parts[0] {
Expand All @@ -151,7 +166,7 @@ func runConfigUnset(key string) error {
case "mcp_servers":
return unsetMCPServer(configPath, parts[1])
default:
return fmt.Errorf("unset supports provider, custom_providers.<name>, and mcp_servers.<name>")
return fmt.Errorf("unset supports provider, reasoning_effort, custom_providers.<name>, and mcp_servers.<name>")
}
}

Expand Down Expand Up @@ -259,15 +274,20 @@ func deleteCustomProvider(cfg *Config, name string) (bool, error) {

// ProviderEntry holds per-provider configuration in the providers map.
type ProviderEntry struct {
APIKey string `json:"api_key,omitempty"`
URL string `json:"url,omitempty"`
Protocol string `json:"protocol,omitempty"`
Model string `json:"model,omitempty"`
Models []string `json:"models,omitempty"`
AuthHeader string `json:"auth_header,omitempty"`
TimeoutSec int `json:"timeout_sec,omitempty"` // per-request HTTP timeout in seconds
ExtraBody map[string]any `json:"extra_body,omitempty"`
ExtraHeaders map[string]string `json:"extra_headers,omitempty"`
APIKey string `json:"api_key,omitempty"`
URL string `json:"url,omitempty"`
Protocol string `json:"protocol,omitempty"`
Model string `json:"model,omitempty"`
Models []string `json:"models,omitempty"`
AuthHeader string `json:"auth_header,omitempty"`
TimeoutSec int `json:"timeout_sec,omitempty"` // per-request HTTP timeout in seconds
ExtraBody map[string]any `json:"extra_body,omitempty"`
ExtraHeaders map[string]string `json:"extra_headers,omitempty"`
ModelSettings map[string]ModelSettings `json:"model_settings,omitempty"`
}

type ModelSettings struct {
ReasoningEffort string `json:"reasoning_effort,omitempty"`
}

// MCPServerConfig holds configuration for a single MCP server.
Expand Down Expand Up @@ -296,15 +316,16 @@ type Config struct {
}

type LlmConfig struct {
URL string `json:"url,omitempty"`
AuthToken string `json:"auth_token,omitempty"`
AuthHeader string `json:"auth_header,omitempty"`
Model string `json:"model,omitempty"`
Protocol string `json:"protocol,omitempty"` // canonical protocol name; takes priority over UseAnthropic
UseAnthropic *bool `json:"use_anthropic,omitempty"` // nil = default true; false = OpenAI protocol (legacy fallback)
TimeoutSec int `json:"timeout_sec,omitempty"` // per-request HTTP timeout in seconds
ExtraBody map[string]any `json:"extra_body,omitempty"`
ExtraHeaders map[string]string `json:"extra_headers,omitempty"`
URL string `json:"url,omitempty"`
AuthToken string `json:"auth_token,omitempty"`
AuthHeader string `json:"auth_header,omitempty"`
Model string `json:"model,omitempty"`
Protocol string `json:"protocol,omitempty"` // canonical protocol name; takes priority over UseAnthropic
UseAnthropic *bool `json:"use_anthropic,omitempty"` // nil = default true; false = OpenAI protocol (legacy fallback)
TimeoutSec int `json:"timeout_sec,omitempty"` // per-request HTTP timeout in seconds
ExtraBody map[string]any `json:"extra_body,omitempty"`
ExtraHeaders map[string]string `json:"extra_headers,omitempty"`
ModelSettings map[string]ModelSettings `json:"model_settings,omitempty"`
}

// TelemetryConfig holds telemetry-specific settings.
Expand Down Expand Up @@ -352,6 +373,7 @@ func LoadAppConfig(path string) (*Config, error) {
var supportedConfigKeys = []string{
"provider",
"model",
"reasoning_effort",
"providers.<name>.<field>",
"custom_providers.<name>.<field>",
"mcp_servers.<name>.<field>",
Expand All @@ -363,6 +385,7 @@ var supportedConfigKeys = []string{
"llm.use_anthropic",
"llm.extra_body",
"llm.extra_headers",
"llm.reasoning_effort",
"language",
"telemetry.enabled",
"telemetry.exporter",
Expand Down Expand Up @@ -423,6 +446,8 @@ func setConfigValue(cfg *Config, key, value string) error {
} else {
cfg.Model = value
}
case "reasoning_effort":
return setActiveModelReasoningEffort(cfg, value)
case "llm.url", "llm.URL":
cfg.Llm.URL = value
case "llm.auth_token", "llm.AuthToken":
Expand All @@ -441,6 +466,19 @@ func setConfigValue(cfg *Config, key, value string) error {
cfg.Llm.ExtraHeaders = parsed
case "llm.model", "llm.Model":
cfg.Llm.Model = value
case "llm.reasoning_effort", "llm.ReasoningEffort":
protocol := cfg.Llm.Protocol
if protocol == "" {
protocol = llm.ProtocolAnthropic
if cfg.Llm.UseAnthropic != nil && !*cfg.Llm.UseAnthropic {
protocol = llm.ProtocolOpenAIChatCompletions
}
}
settings, err := updateReasoningEffort(cfg.Llm.ModelSettings, cfg.Llm.Model, protocol, value)
if err != nil {
return err
}
cfg.Llm.ModelSettings = settings
case "llm.protocol", "llm.Protocol":
normalized := llm.NormalizeProtocol(value)
if err := llm.ValidateProtocol(normalized); err != nil {
Expand Down Expand Up @@ -501,7 +539,7 @@ func setConfigValue(cfg *Config, key, value string) error {
}
cfg.Llm.ExtraBody = m
default:
return fmt.Errorf("unknown config key: %s\nSupported keys: %s\nProvider fields: api_key, url, protocol, model, models, auth_header, extra_body, extra_headers\nProtocol values: anthropic, openai, openai-responses\nMCP server fields: type, command, args, env, url, headers, tools, setup", key, strings.Join(supportedConfigKeys, ", "))
return fmt.Errorf("unknown config key: %s\nSupported keys: %s\nProvider fields: api_key, url, protocol, model, models, model_settings, auth_header, extra_body, extra_headers\nProtocol values: anthropic, openai, openai-responses\nMCP server fields: type, command, args, env, url, headers, tools, setup", key, strings.Join(supportedConfigKeys, ", "))
}
return nil
}
Expand All @@ -526,6 +564,12 @@ func applyProviderField(entry *ProviderEntry, field, key, value string) error {
return fmt.Errorf("invalid model list for %s: %w", key, err)
}
entry.Models = models
case "model_settings":
var settings map[string]ModelSettings
if err := json.Unmarshal([]byte(value), &settings); err != nil {
return fmt.Errorf("invalid JSON for %s: %w", key, err)
}
entry.ModelSettings = settings
case "auth_header":
normalized, err := llm.NormalizeAuthHeader(value)
if err != nil {
Expand All @@ -545,11 +589,87 @@ func applyProviderField(entry *ProviderEntry, field, key, value string) error {
}
entry.ExtraHeaders = parsed
default:
return fmt.Errorf("unknown provider field %q: supported fields are api_key, url, protocol, model, models, auth_header, extra_body, extra_headers", field)
return fmt.Errorf("unknown provider field %q: supported fields are api_key, url, protocol, model, models, model_settings, auth_header, extra_body, extra_headers", field)
}
return nil
}

func setActiveModelReasoningEffort(cfg *Config, value string) error {
if cfg.Provider == "" {
protocol := cfg.Llm.Protocol
if protocol == "" {
protocol = llm.ProtocolAnthropic
if cfg.Llm.UseAnthropic != nil && !*cfg.Llm.UseAnthropic {
protocol = llm.ProtocolOpenAIChatCompletions
}
}
settings, err := updateReasoningEffort(cfg.Llm.ModelSettings, cfg.Llm.Model, protocol, value)
if err != nil {
return err
}
cfg.Llm.ModelSettings = settings
return nil
}

if preset, isPreset := llm.LookupProvider(cfg.Provider); isPreset {
if cfg.Providers == nil {
cfg.Providers = make(map[string]ProviderEntry)
}
entry := cfg.Providers[cfg.Provider]
model := activeModelForProvider(cfg, cfg.Provider, entry)
protocol := preset.Protocol
if entry.Protocol != "" {
protocol = entry.Protocol
}
settings, err := updateReasoningEffort(entry.ModelSettings, model, protocol, value)
if err != nil {
return err
}
entry.ModelSettings = settings
cfg.Providers[cfg.Provider] = entry
return nil
}

entry, ok := cfg.CustomProviders[cfg.Provider]
if !ok {
return fmt.Errorf("provider %q is not configured in custom_providers", cfg.Provider)
}
model := activeModelForProvider(cfg, cfg.Provider, entry)
settings, err := updateReasoningEffort(entry.ModelSettings, model, entry.Protocol, value)
if err != nil {
return err
}
entry.ModelSettings = settings
cfg.CustomProviders[cfg.Provider] = entry
return nil
}

func updateReasoningEffort(settings map[string]ModelSettings, model, protocol, value string) (map[string]ModelSettings, error) {
effort := llm.NormalizeReasoningEffort(value)
model = strings.TrimSpace(model)
if model == "" {
if effort == llm.ReasoningEffortDefault {
return settings, nil
}
return settings, fmt.Errorf("no active model configured; select a model before setting reasoning_effort")
}
if err := llm.ValidateReasoningEffort(protocol, effort); err != nil {
return settings, err
}
if effort == llm.ReasoningEffortDefault {
delete(settings, model)
if len(settings) == 0 {
return nil, nil
}
return settings, nil
}
if settings == nil {
settings = make(map[string]ModelSettings)
}
settings[model] = ModelSettings{ReasoningEffort: effort}
return settings, nil
}

func parseModelListValue(value string) ([]string, error) {
value = strings.TrimSpace(value)
if value == "" {
Expand Down
62 changes: 60 additions & 2 deletions cmd/opencodereview/config_cmd_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,64 @@ func TestSetConfigValueModelWithProvider(t *testing.T) {
}
}

func TestSetConfigValueReasoningEffortIsPerModel(t *testing.T) {
cfg := &Config{
Provider: "anthropic",
Model: "claude-opus-4-8",
Providers: map[string]ProviderEntry{
"anthropic": {
ModelSettings: map[string]ModelSettings{
"claude-opus-4-7": {ReasoningEffort: "low"},
},
},
},
}
if err := setConfigValue(cfg, "reasoning_effort", " XHIGH "); err != nil {
t.Fatal(err)
}
settings := cfg.Providers["anthropic"].ModelSettings
if got := settings["claude-opus-4-8"].ReasoningEffort; got != "xhigh" {
t.Errorf("active model effort = %q, want xhigh", got)
}
if got := settings["claude-opus-4-7"].ReasoningEffort; got != "low" {
t.Errorf("other model effort = %q, want low", got)
}

if err := setConfigValue(cfg, "reasoning_effort", "default"); err != nil {
t.Fatal(err)
}
settings = cfg.Providers["anthropic"].ModelSettings
if _, ok := settings["claude-opus-4-8"]; ok {
t.Error("default should remove only the active model setting")
}
if got := settings["claude-opus-4-7"].ReasoningEffort; got != "low" {
t.Errorf("other model effort after default = %q, want low", got)
}
}

func TestSetConfigValueReasoningEffortValidatesProtocol(t *testing.T) {
cfg := &Config{
Provider: "my-anthropic",
Model: "claude-test",
CustomProviders: map[string]ProviderEntry{
"my-anthropic": {Protocol: llm.ProtocolAnthropic, Model: "claude-test"},
},
}
if err := setConfigValue(cfg, "reasoning_effort", "minimal"); err == nil {
t.Fatal("expected Anthropic to reject OpenAI-only minimal effort")
}
}

func TestUnsetReasoningEffortWithoutActiveModelIsNoOp(t *testing.T) {
cfg := &Config{}
if err := setConfigValue(cfg, "reasoning_effort", "default"); err != nil {
t.Fatalf("unset without active model: %v", err)
}
if cfg.Llm.ModelSettings != nil {
t.Errorf("ModelSettings = %v, want nil", cfg.Llm.ModelSettings)
}
}

func TestSetConfigValueProviderEntry(t *testing.T) {
cfg := &Config{}

Expand Down Expand Up @@ -929,8 +987,8 @@ func TestSetConfigValueUnknownKeyMessage(t *testing.T) {
t.Fatal("expected error for unknown key")
}
want := "unknown config key: bogus.key\n" +
"Supported keys: provider, model, providers.<name>.<field>, custom_providers.<name>.<field>, mcp_servers.<name>.<field>, llm.url, llm.auth_token, llm.auth_header, llm.model, llm.protocol, llm.use_anthropic, llm.extra_body, llm.extra_headers, language, telemetry.enabled, telemetry.exporter, telemetry.otlp_endpoint, telemetry.content_logging\n" +
"Provider fields: api_key, url, protocol, model, models, auth_header, extra_body, extra_headers\n" +
"Supported keys: provider, model, reasoning_effort, providers.<name>.<field>, custom_providers.<name>.<field>, mcp_servers.<name>.<field>, llm.url, llm.auth_token, llm.auth_header, llm.model, llm.protocol, llm.use_anthropic, llm.extra_body, llm.extra_headers, llm.reasoning_effort, language, telemetry.enabled, telemetry.exporter, telemetry.otlp_endpoint, telemetry.content_logging\n" +
"Provider fields: api_key, url, protocol, model, models, model_settings, auth_header, extra_body, extra_headers\n" +
"Protocol values: anthropic, openai, openai-responses\n" +
"MCP server fields: type, command, args, env, url, headers, tools, setup"
if err.Error() != want {
Expand Down
Loading