feat(mcp): read and apply the engine settings like the CLI and the server (#1515, part 4)

CHANGELOG.md

@@ -24,6 +24,7 @@
 
 - Embedding calls go through one module, #1538 (part of #1543). Initialisation, the embedder lock and the retry now live in `embedding.rs` (`Uteke::embed`, `embed_once`, `embedder_max_seq_len`); the ten call sites (recall, explain, document search, document chunks, room recall, update, import, `embed_text`, writes, reembed) ask it, and the five `expect("embedder ensured above")` panics are gone. Behaviour change: query, document and room embedding now retry a transient failure like writes did (3 attempts, 200/400/800 ms) instead of failing at once; `reembed` takes the lock per row instead of for the whole loop and still makes one attempt per row, so a repair over a dead backend reports each failed row without waiting. A write that cannot generate a vector is still stored FTS5-only with a warning; an embedder that cannot start is still an error.
 - Internal: tests can supply their own embedder through the public open path (#1542, part of #1543). `test_support` gains `open_with_embedder`, `open_with_embedder_at` and a `RecordingEmbedder` (records every text, can fail its first calls); the 33 core tests that were `#[ignore]`d because `Uteke::open` needs the ONNX runtime now run in CI with the deterministic keyword embedder (the document-chunk, wikilink-enrichment, room, dedup, fusion and verify/repair tests among them). No production code change.
+- **MCP and the server read the engine settings like the CLI (#1515, part 4)** - `uteke-mcp` used to open the store with the built-in defaults and ignore `uteke.toml` for recall, tier, graph rerank, dream, lifecycle and the embedding fallback. It now loads `EngineSettings` (global file, project file with its untrusted keys stripped, `UTEKE_RECALL_*` / `UTEKE_GRAPH_*` / `UTEKE_EMBED_FALLBACK_*`) and applies it, and `uteke_recall` uses `[recall] min_score`, `min_score_strict` and `default_strategy` when a call gives none, as `POST /recall` does; a call's own floor or strategy still wins. MCP over HTTP (`POST /mcp`) gets the same from the server's settings. `uteke-serve` now also honours the `UTEKE_RECALL_*` and `UTEKE_GRAPH_*` variables (it ignored them before). With no `[recall]` section and no variable nothing changes.
 - Internal: one `EngineSettings` and one apply step, #1515 part 3 (round 2 of the architecture deepening, #1523). `uteke_core::EngineSettings` holds `[tier]`, `[recall]`, `[dream]`, `[lifecycle]` and `[embed_fallback]`, and `FallbackSettings` is deserializable like the others; the CLI config and the server config embed it with `#[serde(flatten)]` instead of writing the five sections twice, and `Uteke::apply_settings` replaces the sequence of setters that the CLI and the server each repeated (the CLI no longer applies the fallback in a separate block; the server logs the dream and lifecycle settings on every start, not only when the section exists). MCP does not use it yet (next part). No behaviour change, checked by running the CLI and the server built from `develop` and from this branch on the same config with all five sections, a partial file and an invalid one.
 - Internal: `[recall]` is read into one core type, #1515 part 2 (round 2 of the architecture deepening, #1523). `uteke_core::RecallSettings` owns the nine keys, their defaults and the `UTEKE_RECALL_*` / `UTEKE_GRAPH_*` overrides with their range checks (moved from the CLI, tested with an injected lookup); the CLI config uses it as its `[recall]` type and the server drops `RecallFileSection` and its `Option` chains. The server still does not read the `UTEKE_RECALL_*` variables: that is decided in the apply step of part 3. No behaviour change, checked by running the CLI and the server built from `develop` and from this branch with the same `[recall]` section and environment.
 - Internal: `[tier]`, `[dream]` and `[lifecycle]` are read straight into the core types, #1515 part 1 (round 2 of the architecture deepening, #1523; the first step of closing #1355). `uteke_core::{TierConfig, DreamConfig, LifecycleConfig}` now implement `serde::Deserialize` (each key optional, the core default for the rest; negative tier days are still rejected), so the CLI config uses them as its section types and the server drops its three `Option` mirror structs and the field-by-field default chains. `[recall]` and `[embed_fallback]` follow in the next parts. No behaviour change.

crates/uteke-core/src/config_layers.rs

@@ -126,6 +126,25 @@ fn read_layer(path: &Path) -> Option<Table> {
     }
 }
 
+/// [`resolve`] over the standard places: `{uteke_home}/uteke.toml`, then
+/// `$CWD/.uteke/uteke.toml`, with the project file untrusted unless
+/// `UTEKE_TRUST_PROJECT_CONFIG` is `1` or `true`. For callers that read only
+/// their own sections (the privacy filter, the engine settings of MCP).
+pub fn resolve_standard<T: DeserializeOwned + Default>() -> Resolved<T> {
+    let global = crate::uteke_home().ok().map(|h| h.join("uteke.toml"));
+    let project = std::env::current_dir()
+        .ok()
+        .map(|cwd| cwd.join(".uteke").join("uteke.toml"));
+    resolve::<T>(&Layers {
+        global: global.as_deref(),
+        project: project.as_deref(),
+        trust_project: matches!(
+            std::env::var("UTEKE_TRUST_PROJECT_CONFIG").as_deref(),
+            Ok("1") | Ok("true")
+        ),
+    })
+}
+
 /// Resolve the layers into a `T`.
 pub fn resolve<T: DeserializeOwned + Default>(layers: &Layers<'_>) -> Resolved<T> {
     let mut merged = Table::new();

crates/uteke-core/src/engine_settings.rs

@@ -17,12 +17,36 @@ pub struct EngineSettings {
     pub embed_fallback: FallbackSettings,
 }
 
+impl EngineSettings {
+    /// Read the five sections the way every binary does: the global
+    /// `uteke.toml`, then the project file (its untrusted keys stripped unless
+    /// `UTEKE_TRUST_PROJECT_CONFIG` is set), then the `UTEKE_RECALL_*`,
+    /// `UTEKE_GRAPH_*` and `UTEKE_EMBED_FALLBACK_*` environment variables.
+    /// An unreadable or invalid file is skipped with a warning (defaults).
+    pub fn load() -> Self {
+        let mut engine = crate::config_layers::resolve_standard::<EngineSettings>().value;
+        engine.recall = engine.recall.with_env_overrides();
+        engine.embed_fallback = engine.embed_fallback.with_env_overrides();
+        engine
+    }
+}
+
 impl Uteke {
-    /// Apply what can change after the store is open: the Jaccard weight, the
-    /// dream thresholds, the lifecycle config and, when it is complete (api
-    /// key, base URL and model), the embedding fallback. `tier` and the
-    /// recall thresholds that the constructor takes are not touched here.
+    /// The `[recall]` settings in force: what a request falls back to when it
+    /// gives no similarity floor and no strategy.
+    pub fn recall_settings(&self) -> &RecallSettings {
+        &self.recall_settings
+    }
+
+    /// Apply what can change after the store is open: the tier and graph-rerank
+    /// settings, the Jaccard weight, the recall defaults, the dream thresholds,
+    /// the lifecycle config and, when it is complete (api key, base URL and
+    /// model), the embedding fallback. The embedding backend and the vector
+    /// engine are choices of the constructor and are not touched here.
     pub fn apply_settings(&mut self, settings: &EngineSettings) {
+        self.tier_config = settings.tier;
+        self.graph_rerank_config = settings.recall.graph_rerank().sanitized();
+        self.recall_settings = settings.recall.clone().sanitized();
         self.set_jaccard_weight(settings.recall.jaccard_weight);
         self.set_dream_config(settings.dream);
         self.set_lifecycle_config(settings.lifecycle);

crates/uteke-core/src/engine_settings_tests.rs

@@ -120,6 +120,20 @@ mod recall {
         assert!(r.graph_rerank_enabled);
     }
 
+    #[test]
+    fn an_unknown_default_strategy_becomes_hybrid_and_a_known_one_is_kept() {
+        assert_eq!(
+            parse("default_strategy = \"nonsense\"")
+                .sanitized()
+                .default_strategy,
+            "hybrid"
+        );
+        for known in ["vector", "fts5", "hybrid", "graph", "fusion"] {
+            let r = parse(&format!("default_strategy = \"{known}\"")).sanitized();
+            assert_eq!(r.default_strategy, known);
+        }
+    }
+
     #[test]
     fn conversions_to_the_engine_types() {
         let r = parse(
@@ -232,6 +246,69 @@ mod engine {
         assert_eq!(u.fallback_settings.base_url, "https://example.test");
     }
 
+    #[test]
+    fn apply_settings_also_sets_tier_graph_rerank_and_the_recall_defaults() {
+        let mut u = Uteke::open_with_backend(":memory:", None).expect("open without embedder");
+        let mut e = EngineSettings::default();
+        e.tier.hot_days = 2;
+        e.recall.graph_density_weight = 0.4;
+        e.recall.graph_rerank_enabled = false;
+        e.recall.min_score = 0.3;
+        e.recall.default_strategy = "nonsense".into();
+        u.apply_settings(&e);
+        assert_eq!(u.tier_config.hot_days, 2);
+        assert_eq!(u.graph_rerank_config.density_weight, 0.4);
+        assert!(!u.graph_rerank_config.enabled);
+        assert_eq!(u.recall_settings().min_score, 0.3);
+        assert_eq!(
+            u.recall_settings().default_strategy,
+            "hybrid",
+            "the copy a transport reads is sanitized"
+        );
+    }
+
+    #[test]
+    fn a_fresh_uteke_reports_the_default_recall_settings() {
+        let u = Uteke::open_with_backend(":memory:", None).expect("open without embedder");
+        let r = u.recall_settings();
+        assert_eq!((r.min_score, r.min_score_strict), (0.0, 0.5));
+        assert_eq!(r.default_strategy, "fusion");
+    }
+
+    #[test]
+    fn engine_settings_come_from_layers_then_the_environment() {
+        use crate::config_layers::{Layers, resolve};
+        let dir = std::env::temp_dir().join(format!("uteke-engine-load-{}", std::process::id()));
+        std::fs::create_dir_all(&dir).unwrap();
+        let global = dir.join("uteke.toml");
+        std::fs::write(
+            &global,
+            "[recall]\nmin_score = 0.2\n[embed_fallback]\napi_key = \"file\"\nbase_url = \"https://file.test\"\nmodel = \"m\"\n",
+        )
+        .unwrap();
+        let mut e = resolve::<EngineSettings>(&Layers {
+            global: Some(&global),
+            project: None,
+            trust_project: false,
+        })
+        .value;
+        assert_eq!(e.recall.min_score, 0.2);
+        let env = |k: &str| match k {
+            "UTEKE_RECALL_MIN_SCORE" => Some("0.6".to_string()),
+            "UTEKE_EMBED_FALLBACK_API_KEY" => Some("from-env".to_string()),
+            _ => None,
+        };
+        e.recall = e.recall.with_overrides_from(env);
+        e.embed_fallback = e.embed_fallback.with_overrides_from(env);
+        assert_eq!(
+            e.recall.min_score, 0.6,
+            "the environment wins over the file"
+        );
+        assert_eq!(e.embed_fallback.api_key, "from-env");
+        assert_eq!(e.embed_fallback.base_url, "https://file.test");
+        std::fs::remove_dir_all(&dir).ok();
+    }
+
     #[test]
     fn an_incomplete_fallback_is_not_applied() {
         let mut u = Uteke::open_with_backend(":memory:", None).expect("open without embedder");

crates/uteke-core/src/lib.rs

@@ -681,6 +681,9 @@ pub struct Uteke {
     /// Jaccard token reranking weight (#719). Additive boost applied
     /// post-RRF based on query-content token overlap. Default 0.0 (off).
     jaccard_weight: f32,
+    /// The `[recall]` settings in force: the defaults a transport falls back to
+    /// when a request does not give a floor or a strategy (set by `apply_settings`).
+    recall_settings: RecallSettings,
     /// Dream pipeline thresholds (#731). Used by contradiction scan,
     /// dedup/consolidate, and orphan detection phases.
     dream_config: DreamConfig,
@@ -1109,6 +1112,7 @@ impl Uteke {
             salience_recency_config: salience_recency::SalienceRecencyConfig::default(),
             recall_cache: recall_cache::RecallCache::new(recall_cache::RecallCacheConfig::default()),
             jaccard_weight: 0.0,
+            recall_settings: RecallSettings::default(),
             dream_config: DreamConfig::default(),
             privacy_config: privacy::PrivacyConfig::default(),
             lifecycle_config: LifecycleConfig::default(),

crates/uteke-core/src/privacy.rs

@@ -134,19 +134,7 @@ impl PrivacyConfig {
 /// A bad value is logged and replaced by the default (`redact`): the filter
 /// never turns itself off because of a typo.
 pub fn load_config() -> PrivacyConfig {
-    let global = crate::uteke_home().ok().map(|h| h.join("uteke.toml"));
-    let project = std::env::current_dir()
-        .ok()
-        .map(|cwd| cwd.join(".uteke").join("uteke.toml"));
-    let mut settings =
-        crate::config_layers::resolve::<PrivacyFile>(&crate::config_layers::Layers {
-            global: global.as_deref(),
-            project: project.as_deref(),
-            trust_project: matches!(
-                std::env::var("UTEKE_TRUST_PROJECT_CONFIG").as_deref(),
-                Ok("1") | Ok("true")
-            ),
-        })
+    let mut settings = crate::config_layers::resolve_standard::<PrivacyFile>()
         .value
         .privacy;
     if let Ok(mode) = std::env::var("UTEKE_PRIVACY_MODE") {

crates/uteke-core/src/recall_settings.rs

@@ -77,6 +77,21 @@ impl RecallSettings {
         }
     }
 
+    /// These settings with an unknown `default_strategy` replaced by `hybrid`
+    /// (and a warning): a typo in `uteke.toml` must not make every recall
+    /// request fail with a message that blames the request (#1034).
+    pub fn sanitized(mut self) -> Self {
+        if crate::RecallStrategy::from_str_opt(&self.default_strategy).is_none() {
+            tracing::warn!(
+                "Invalid [recall] default_strategy='{}' in config - falling back to 'hybrid'. \
+                 Expected: vector | fts5 | hybrid | graph | fusion.",
+                self.default_strategy
+            );
+            self.default_strategy = "hybrid".to_string();
+        }
+        self
+    }
+
     /// Apply the `UTEKE_RECALL_*` and `UTEKE_GRAPH_*` overrides from `lookup`.
     /// A value that is out of range or does not parse is logged and ignored.
     pub fn with_overrides_from(mut self, lookup: impl Fn(&str) -> Option<String>) -> Self {

crates/uteke-mcp/src/lib.rs

@@ -1337,15 +1337,20 @@ fn exec_recall(uteke: &Uteke, args: &Value) -> Result<ToolResult, String> {
 
     let tag_vec = req.tag_filter();
     let tags_ref = tag_vec.as_deref();
-    let min_score = req.resolve_min_score(None, None);
+    // The similarity floor and the strategy fall back to `[recall]` of
+    // uteke.toml (defaults equal the built-ins), as `POST /recall` does.
+    let recall = uteke.recall_settings();
+    let min_score = req.resolve_min_score(Some(recall.min_score), Some(recall.min_score_strict));
 
     // Parse optional search type (#531)
     let search_type = req.search_type().map_err(|e| e.to_string())?;
 
     // Parse optional recall strategy (#1035): default fusion since 0.16.0
     // (#1123), matching the CLI and HTTP defaults. Unknown values are a loud
     // error (JSON-RPC -32603), never a silent fallback.
-    let strategy = req.strategy(None).map_err(|e| e.to_string())?;
+    let strategy = req
+        .strategy(Some(recall.default_strategy.as_str()))
+        .map_err(|e| e.to_string())?;
 
     // Explain mode (#1160): memory-only, bypasses unified results and
     // returns full ranking signals per result. An omitted `type` (the
@@ -4333,3 +4338,66 @@ mod privacy_mcp_tests {
         assert!(err.starts_with("secret_detected:"), "{err}");
     }
 }
+
+/// `uteke_recall` falls back to the `[recall]` settings of `uteke.toml` (#1515):
+/// the similarity floor and the strategy, as `POST /recall` does.
+#[cfg(test)]
+mod recall_settings_mcp_tests {
+    use super::*;
+    use uteke_core::test_support::open_keyword;
+    use uteke_core::{EngineSettings, Uteke};
+
+    fn store(configure: impl FnOnce(&mut EngineSettings)) -> Uteke {
+        let mut u = open_keyword();
+        for text in ["rust ownership and borrowing", "python garbage collector"] {
+            u.remember_detailed(text, &[], None, None).unwrap();
+        }
+        let mut e = EngineSettings::default();
+        configure(&mut e);
+        u.apply_settings(&e);
+        u
+    }
+
+    fn recall(u: &Uteke, args: Value) -> String {
+        match &exec_recall(u, &args).unwrap().content[0] {
+            McpContent::Text { text, .. } => text.clone(),
+        }
+    }
+
+    #[test]
+    fn without_a_recall_section_nothing_changes() {
+        let u = store(|_| {});
+        let t = recall(&u, serde_json::json!({"query": "rust borrowing"}));
+        assert!(t.contains("rust ownership"), "{t}");
+    }
+
+    #[test]
+    fn the_configured_floor_filters_and_a_call_floor_wins_over_it() {
+        let u = store(|e| e.recall.min_score = 0.99);
+        let t = recall(&u, serde_json::json!({"query": "rust borrowing"}));
+        assert!(!t.contains("rust ownership"), "the floor is 0.99: {t}");
+        let t = recall(
+            &u,
+            serde_json::json!({"query": "rust borrowing", "min_score": 0.0}),
+        );
+        assert!(
+            t.contains("rust ownership"),
+            "the call's own floor wins: {t}"
+        );
+    }
+
+    #[test]
+    fn the_configured_default_strategy_is_used_and_a_call_strategy_wins() {
+        let u = store(|e| e.recall.default_strategy = "vector".into());
+        // Both strategies answer; what matters is that neither is rejected and
+        // an explicit strategy still overrides the configured one.
+        assert!(recall(&u, serde_json::json!({"query": "rust"})).contains("rust ownership"));
+        assert!(
+            recall(&u, serde_json::json!({"query": "rust", "strategy": "fts5"}))
+                .contains("rust ownership")
+        );
+        // An unknown configured strategy was sanitized to hybrid at apply time.
+        let u = store(|e| e.recall.default_strategy = "nonsense".into());
+        assert!(recall(&u, serde_json::json!({"query": "rust"})).contains("rust ownership"));
+    }
+}

crates/uteke-mcp/src/main.rs

@@ -36,8 +36,13 @@ fn main() {
         std::process::exit(1);
     });
 
+    // The engine sections of uteke.toml, read and applied like the CLI and the
+    // server do (#1515): recall defaults, tier, graph rerank, dream, lifecycle,
+    // embedding fallback.
+    let engine = uteke_core::EngineSettings::load();
     let uteke = match Uteke::open(&store_path) {
         Ok(mut u) => {
+            u.apply_settings(&engine);
             // #1464: secret filter on the write path ([privacy] in uteke.toml)
             u.set_privacy_config(uteke_core::privacy::load_config());
             u