Skip to content

feat(config): resolve compact context_window from per-model catalog - #268

Open
salim4n wants to merge 3 commits into
iOfficeAI:mainfrom
salim4n:poc/per-model-context-window
Open

feat(config): resolve compact context_window from per-model catalog#268
salim4n wants to merge 3 commits into
iOfficeAI:mainfrom
salim4n:poc/per-model-context-window

Conversation

@salim4n

@salim4n salim4n commented Aug 15, 2026

Copy link
Copy Markdown

Closes #267

Summary

CompactConfig.context_window defaults to a flat 200K regardless of the resolved model, so autocompact thresholds (context_window − output_reserve − autocompact_buffer) are sized for a window the model may not have. This PR extends the existing model_max_tokens pattern (output budget) to the input window: a model_context_window rule catalog resolved in Config::resolve only when the user has not set context_window explicitly.

Changes

  • aion-config/src/compat.rs
    • New ModelContextWindowRule { pattern, context_window } (mirrors ModelMaxTokensRule)
    • New TransportCompat.model_context_window field, merged with the same user-replaces-preset semantics as model_max_tokens
    • Catalogs: Anthropic family (claude → 200K) and OpenAI-compatible endpoints (kimi-for-coding → 262 144, kimi-k2/kimi/moonshot/deepseek/qwen → 131 072, minimax, gpt-*, o-*)
    • New accessor ProviderCompat::context_window_for_model()
  • aion-config/src/config.rs
    • Config::resolve: if compact.context_window is still the default and the catalog matches the resolved model, use the catalog value. Explicit config always wins; unknown models keep the 200K fallback.

Backward compatibility

  • Explicit [compact] context_window is never overridden
  • Behavior unchanged for models outside the catalog
  • No new dependencies, no wire-protocol changes, single crate touched

Tests

  • 5 new unit tests in compat_test.rs (anthropic, bedrock, Kimi/DeepSeek/Qwen endpoints, pattern precedence, user override)
  • 2 new resolution tests in config_test.rs (catalog applied when unset, explicit config wins)
  • cargo test -p aion-config: 199 passed
  • cargo clippy -p aion-config: clean

Notes

  • Motivated by a static audit of the threshold defaults; I have not reproduced a live overflow against a provider. Happy to adjust catalog values to match official provider docs.
  • Follow-up ideas (out of scope): surface the resolved window in ContextStatus, document autocompact_threshold_pct for the AionUi settings UI.

When [compact] context_window is not set explicitly, size it from a new
per-model rule catalog (model_context_window, mirroring model_max_tokens)
instead of the flat 200K fallback. Third-party OpenAI-compatible endpoints
(Kimi/Moonshot, DeepSeek, Qwen, MiniMax) get realistic windows, so
autocompact thresholds reflect the model actually in use.

An explicit [compact] context_window always wins; user-provided
model_context_window rules replace the preset, matching
model_max_tokens merge semantics.

@jiahe0510 jiahe0510 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing model-specific context windows. I do not think the current implementation is safe to merge yet because it can silently select incorrect compaction limits.

There are three blocking issues:

  1. The built-in catalog is already incorrect for current models, and broad substring matching amplifies the problem. For example, gpt-5 is configured as 272K although its documented total context window is 400K. The same pattern also matches gpt-5.6-*, whose documented window is 1.05M. Likewise, the generic claude rule forces 200K for Claude Sonnet 4.6 and newer applicable Claude models that have a 1M window. CompactConfig::context_window represents the total input+output window, so these are not output-reserve values. This would cause substantially premature compaction, additional summary calls, and avoidable context-quality loss.

Official references:

Please use model/version-specific ordered rules backed by documented values rather than a single broad family substring.

  1. The code cannot determine whether context_window was explicitly configured by comparing it with the default value. Serde materializes both an absent field and an explicit context_window = 200000 as 200000. As a result, an explicit 200K setting is still overwritten by the catalog, despite the stated contract that explicit configuration always wins. Please preserve field presence/source information (for example via an Option during resolution) and add a regression test for an explicitly configured 200000 value.

  2. The catalog is applied only during Config::resolve. In standalone AionCLI, AgentEngine::apply_config_update can change the model at runtime without updating compact_config.context_window, leaving the previous model window active. Please recompute the model-derived window on model changes while preserving explicit user overrides, and add coverage for switching between models with different windows.

Current AionCore compiles against this PR, but once AionCore upgrades the aionrs dependency, issue 1 would directly change autocompact thresholds for embedded AionCLI sessions. Compile compatibility therefore does not remove the runtime risk.

@salim4n

salim4n commented Aug 22, 2026

Copy link
Copy Markdown
Author

Thanks for the detailed review. I agree with all three blockers.

The 272K GPT-5 value conflated the maximum input budget with the total context window expected by CompactConfig::context_window.

I’ll revise the PR to:

  1. Replace broad family substring rules with ordered, model/version-specific rules backed by official provider documentation, while leaving unknown models on the existing fallback.
  2. Preserve context_window field presence/provenance through config merging and resolution, so an explicit context_window = 200000 is never treated as an unset default.
  3. Carry that provenance into AgentEngine and recompute the model-derived window during apply_config_update, while preserving explicit overrides.

I’ll add regression coverage for explicit 200000 and runtime model switching, then rerun fmt, clippy, and the relevant workspace tests.

@salim4n

salim4n commented Aug 22, 2026

Copy link
Copy Markdown
Author

Implemented all three requested changes in 611ea82.

  1. Documented, version-specific model rules

  2. Explicit context_window = 200000 is preserved

    • Config loading now carries field presence as Option<usize> through global/project merging.
    • Resolved config records Default, ModelCatalog, or Explicit provenance.
    • Regression coverage verifies that explicit 200K is never overwritten.
  3. Runtime model switches recompute derived windows

    • AgentEngine carries the provenance from Config::resolve.
    • apply_config_update recomputes catalog/default windows on model changes, emits structured debug data, and preserves explicit overrides.
    • Tests cover catalog-to-catalog switching, catalog-to-unknown-to-catalog switching, and explicit 200K preservation through the real resolved-config constructor path.

Validation completed successfully:

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace (clean isolated target, Rust 1.96.1)

Ready for re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Per-model context_window catalog for autocompact thresholds

2 participants