feat(config): resolve compact context_window from per-model catalog - #268
feat(config): resolve compact context_window from per-model catalog#268salim4n wants to merge 3 commits into
Conversation
When [compact] context_window is not set explicitly, size it from a new per-model rule catalog (model_context_window, mirroring model_max_tokens) instead of the flat 200K fallback. Third-party OpenAI-compatible endpoints (Kimi/Moonshot, DeepSeek, Qwen, MiniMax) get realistic windows, so autocompact thresholds reflect the model actually in use. An explicit [compact] context_window always wins; user-provided model_context_window rules replace the preset, matching model_max_tokens merge semantics.
jiahe0510
left a comment
There was a problem hiding this comment.
Thanks for addressing model-specific context windows. I do not think the current implementation is safe to merge yet because it can silently select incorrect compaction limits.
There are three blocking issues:
- The built-in catalog is already incorrect for current models, and broad substring matching amplifies the problem. For example,
gpt-5is configured as 272K although its documented total context window is 400K. The same pattern also matchesgpt-5.6-*, whose documented window is 1.05M. Likewise, the genericclauderule forces 200K for Claude Sonnet 4.6 and newer applicable Claude models that have a 1M window.CompactConfig::context_windowrepresents the total input+output window, so these are not output-reserve values. This would cause substantially premature compaction, additional summary calls, and avoidable context-quality loss.
Official references:
- https://developers.openai.com/api/docs/models/gpt-5
- https://developers.openai.com/api/docs/models
- https://platform.claude.com/docs/en/build-with-claude/context-windows
Please use model/version-specific ordered rules backed by documented values rather than a single broad family substring.
-
The code cannot determine whether
context_windowwas explicitly configured by comparing it with the default value. Serde materializes both an absent field and an explicitcontext_window = 200000as 200000. As a result, an explicit 200K setting is still overwritten by the catalog, despite the stated contract that explicit configuration always wins. Please preserve field presence/source information (for example via anOptionduring resolution) and add a regression test for an explicitly configured 200000 value. -
The catalog is applied only during
Config::resolve. In standalone AionCLI,AgentEngine::apply_config_updatecan change the model at runtime without updatingcompact_config.context_window, leaving the previous model window active. Please recompute the model-derived window on model changes while preserving explicit user overrides, and add coverage for switching between models with different windows.
Current AionCore compiles against this PR, but once AionCore upgrades the aionrs dependency, issue 1 would directly change autocompact thresholds for embedded AionCLI sessions. Compile compatibility therefore does not remove the runtime risk.
|
Thanks for the detailed review. I agree with all three blockers. The 272K GPT-5 value conflated the maximum input budget with the total context window expected by I’ll revise the PR to:
I’ll add regression coverage for explicit |
|
Implemented all three requested changes in
Validation completed successfully:
Ready for re-review. |
Closes #267
Summary
CompactConfig.context_windowdefaults to a flat 200K regardless of the resolved model, so autocompact thresholds (context_window − output_reserve − autocompact_buffer) are sized for a window the model may not have. This PR extends the existingmodel_max_tokenspattern (output budget) to the input window: amodel_context_windowrule catalog resolved inConfig::resolveonly when the user has not setcontext_windowexplicitly.Changes
aion-config/src/compat.rsModelContextWindowRule { pattern, context_window }(mirrorsModelMaxTokensRule)TransportCompat.model_context_windowfield, merged with the same user-replaces-preset semantics asmodel_max_tokensclaude→ 200K) and OpenAI-compatible endpoints (kimi-for-coding→ 262 144,kimi-k2/kimi/moonshot/deepseek/qwen→ 131 072,minimax,gpt-*,o-*)ProviderCompat::context_window_for_model()aion-config/src/config.rsConfig::resolve: ifcompact.context_windowis still the default and the catalog matches the resolved model, use the catalog value. Explicit config always wins; unknown models keep the 200K fallback.Backward compatibility
[compact] context_windowis never overriddenTests
compat_test.rs(anthropic, bedrock, Kimi/DeepSeek/Qwen endpoints, pattern precedence, user override)config_test.rs(catalog applied when unset, explicit config wins)cargo test -p aion-config: 199 passedcargo clippy -p aion-config: cleanNotes
ContextStatus, documentautocompact_threshold_pctfor the AionUi settings UI.