feat(core): read [privacy] from uteke.toml in the CLI, server and MCP (#1464, part 2b-2a)

CHANGELOG.md

@@ -8,6 +8,7 @@
 
 ### Added
 
+- **`[privacy]` in `uteke.toml` (#1464, part 2b-2a)** - `mode` (`redact`, the default, `strict` or `off`) and `allowlist` (regexes) now configure the secret filter in the CLI, `uteke-serve` and `uteke-mcp`; `UTEKE_PRIVACY_MODE` overrides the mode. The section is read from the global file only (a project file could otherwise switch the filter off), and a bad value falls back to the default instead of disabling it.
 - **`author` on every write path (#1427, phase 2)** - a memory records who wrote it (the agent, user or tool name), with or without a room. `POST /remember`, MCP `uteke_remember` and `uteke remember --author` accept it (free text, trimmed, 1 to 128 characters; an invalid value is a 400, a tool error or a CLI error before anything is stored). A room write stamps the named writer on the memory as well as on the room link; the `anonymous` / `user` fallback of the room link is not an author. `author` is returned on the memory object by recall, search, list and get (omitted when unset), echoed by `POST /remember`, and kept by export and import. Stores migrate to schema v22 on first open (see Internal below): back up a production store first. Filtering by author is not part of this phase. Contract: uteke-cloud#219 (cloud already has the field).
 
 ### Fixed

crates/uteke-cli/src/main.rs

@@ -247,6 +247,8 @@ fn main() {
             });
             // #928: apply lifecycle configuration from config
             u.set_lifecycle_config(config.lifecycle.to_core());
+            // #1464: secret filter on the write path ([privacy] in uteke.toml)
+            u.set_privacy_config(uteke_core::privacy::load_config());
             u
         }
         Err(e) => {

crates/uteke-core/src/config_layers.rs

@@ -42,6 +42,9 @@ pub const UNTRUSTED_PROJECT_KEYS: &[&str] = &[
     "extraction.api_key",
     "extraction.base_url",
     "extraction.endpoint_path",
+    // A cloned repository must not switch the secret filter off or allowlist
+    // everything.
+    "privacy",
     "server",
 ];
 

crates/uteke-core/src/privacy.rs

@@ -97,6 +97,89 @@ pub struct PrivacyConfig {
     pub allowlist: Vec<Regex>,
 }
 
+/// `[privacy]` in `uteke.toml`.
+#[derive(Debug, Clone, Default, serde::Deserialize)]
+#[serde(default)]
+pub struct PrivacySettings {
+    /// `redact` (default), `strict` or `off`.
+    pub mode: Option<String>,
+    /// Regexes; a value matching any of them is kept as it is.
+    pub allowlist: Vec<String>,
+}
+
+#[derive(Debug, Default, serde::Deserialize)]
+#[serde(default)]
+struct PrivacyFile {
+    privacy: PrivacySettings,
+}
+
+impl PrivacyConfig {
+    /// Build the config from `[privacy]` settings. An unknown mode or an
+    /// invalid regex is an error; callers that must keep running use
+    /// [`load_config`], which falls back to the safe default.
+    pub fn from_settings(settings: &PrivacySettings) -> Result<Self, Error> {
+        Ok(Self {
+            mode: match settings.mode.as_deref() {
+                Some(m) => PrivacyMode::parse(m)?,
+                None => PrivacyMode::default(),
+            },
+            allowlist: compile_allowlist(&settings.allowlist)?,
+        })
+    }
+}
+
+/// The privacy config of this installation: `[privacy]` from the global
+/// `uteke.toml` (the project-local file is never trusted for it, so a cloned
+/// repository cannot switch the filter off), then `UTEKE_PRIVACY_MODE`.
+/// A bad value is logged and replaced by the default (`redact`): the filter
+/// never turns itself off because of a typo.
+pub fn load_config() -> PrivacyConfig {
+    let global = crate::uteke_home().ok().map(|h| h.join("uteke.toml"));
+    let project = std::env::current_dir()
+        .ok()
+        .map(|cwd| cwd.join(".uteke").join("uteke.toml"));
+    let mut settings =
+        crate::config_layers::resolve::<PrivacyFile>(&crate::config_layers::Layers {
+            global: global.as_deref(),
+            project: project.as_deref(),
+            trust_project: matches!(
+                std::env::var("UTEKE_TRUST_PROJECT_CONFIG").as_deref(),
+                Ok("1") | Ok("true")
+            ),
+        })
+        .value
+        .privacy;
+    if let Ok(mode) = std::env::var("UTEKE_PRIVACY_MODE") {
+        settings.mode = Some(mode);
+    }
+    config_or_default(&settings)
+}
+
+fn config_or_default(settings: &PrivacySettings) -> PrivacyConfig {
+    // Each setting falls back on its own: a bad allowlist entry must not
+    // lose a `strict` mode, and a bad mode must not drop the allowlist.
+    let mode = match settings.mode.as_deref().map(PrivacyMode::parse) {
+        Some(Ok(m)) => m,
+        Some(Err(e)) => {
+            tracing::warn!("Invalid [privacy] mode, using redact: {e}");
+            PrivacyMode::default()
+        }
+        None => PrivacyMode::default(),
+    };
+    let allowlist = settings
+        .allowlist
+        .iter()
+        .filter_map(|p| match compile_allowlist(std::slice::from_ref(p)) {
+            Ok(mut v) => v.pop(),
+            Err(e) => {
+                tracing::warn!("Ignoring [privacy] allowlist entry: {e}");
+                None
+            }
+        })
+        .collect();
+    PrivacyConfig { mode, allowlist }
+}
+
 /// Content and metadata ready to store, plus what the filter did.
 #[derive(Debug, Clone, PartialEq)]
 pub struct Prepared {
@@ -1545,4 +1628,59 @@ mod tests {
         let comment = format!("# {b}\n# {run}\n# {e}");
         assert_eq!(redact(&comment, &[]).text, "# [REDACTED:private-key]");
     }
+
+    #[test]
+    fn settings_become_a_config_and_bad_values_fall_back_one_by_one() {
+        let ok = PrivacyConfig::from_settings(&PrivacySettings {
+            mode: Some("Strict".into()),
+            allowlist: vec!["^dev-".into()],
+        })
+        .unwrap();
+        assert_eq!(ok.mode, PrivacyMode::Strict);
+        assert_eq!(ok.allowlist.len(), 1);
+        assert!(
+            PrivacyConfig::from_settings(&PrivacySettings {
+                mode: Some("maybe".into()),
+                allowlist: vec![],
+            })
+            .is_err()
+        );
+
+        // A bad mode keeps the allowlist and never turns the filter off; a bad
+        // regex keeps the mode.
+        let c = config_or_default(&PrivacySettings {
+            mode: Some("none".into()),
+            allowlist: vec!["^dev-".into()],
+        });
+        assert_eq!((c.mode, c.allowlist.len()), (PrivacyMode::Redact, 1));
+        let c = config_or_default(&PrivacySettings {
+            mode: Some("strict".into()),
+            allowlist: vec!["(".into(), "^dev-".into()],
+        });
+        assert_eq!((c.mode, c.allowlist.len()), (PrivacyMode::Strict, 1));
+    }
+
+    #[test]
+    fn privacy_is_read_from_the_global_file_only() {
+        use crate::config_layers::{Layers, resolve};
+        let dir = std::env::temp_dir().join(format!("uteke-privacy-cfg-{}", std::process::id()));
+        std::fs::create_dir_all(&dir).unwrap();
+        let global = dir.join("global.toml");
+        let project = dir.join("project.toml");
+        std::fs::write(&global, "[privacy]\nmode = \"strict\"\n").unwrap();
+        std::fs::write(
+            &project,
+            "[privacy]\nmode = \"off\"\nallowlist = [\".*\"]\n",
+        )
+        .unwrap();
+        let r = resolve::<PrivacyFile>(&Layers {
+            global: Some(&global),
+            project: Some(&project),
+            trust_project: false,
+        });
+        assert_eq!(r.value.privacy.mode.as_deref(), Some("strict"));
+        assert!(r.value.privacy.allowlist.is_empty());
+        assert!(r.ignored_untrusted.contains(&"privacy".to_string()));
+        std::fs::remove_dir_all(&dir).ok();
+    }
 }

crates/uteke-mcp/src/main.rs

@@ -37,7 +37,11 @@ fn main() {
     });
 
     let uteke = match Uteke::open(&store_path) {
-        Ok(u) => u,
+        Ok(mut u) => {
+            // #1464: secret filter on the write path ([privacy] in uteke.toml)
+            u.set_privacy_config(uteke_core::privacy::load_config());
+            u
+        }
         Err(e) => {
             eprintln!("Failed to open uteke store: {e}");
             std::process::exit(1);

crates/uteke-server/src/main.rs

@@ -285,6 +285,10 @@ fn main() {
         Ok(mut u) => {
             // [recall] jaccard_weight (#719, #1355 step B); the core clamps to 0..=1.
             u.set_jaccard_weight(recall_tuning.jaccard_weight);
+            // [privacy] (#1464): secret filter on the write path.
+            let privacy = uteke_core::privacy::load_config();
+            info!("Privacy filter: {:?}", privacy.mode);
+            u.set_privacy_config(privacy);
             // [embed_fallback] + UTEKE_EMBED_FALLBACK_* (#1355): only when complete.
             if let Some(fallback) = resolve_embed_fallback(&config, |k| std::env::var(k).ok()) {
                 info!("Embedding fallback: enabled ({})", fallback.base_url);

docs/configuration.md

@@ -309,6 +309,7 @@ Resolution order (highest priority first):
 | Env Var | Config Equivalent | Default | Description |
 |---------|-------------------|---------|-------------|
 | `UTEKE_HOME` | — | `~/.codecora/uteke` | Data directory |
+| `UTEKE_PRIVACY_MODE` | `[privacy] mode` | `redact` | Secret filter mode: `redact`, `strict`, `off` |
 | `UTEKE_NAMESPACE` | `[store] namespace` | `default` | Default namespace (applied in CLI) |
 | `UTEKE_AUTH_TOKEN` | — | — | Server auth token (applied in server) |
 | `UTEKE_LOG_LEVEL` | `[logging] level` | `warn` | Log level (trace/debug/info/warn/error) |
@@ -400,10 +401,27 @@ A project directory can be an untrusted clone, so `.uteke/uteke.toml` is **not**
 
 - `[embedding]`: `backend`, `api_key`, `base_url`, `endpoint_path`
 - `[embed_fallback]` and `[extraction]`: `api_key`, `base_url`, `endpoint_path`
+- the whole `[privacy]` section (a cloned repository must not switch the secret filter off)
 - the whole `[server]` section (`enabled`, `host`, `port`, `auth_token`, `read_only_token`, `cors_origins`, `allowed_hosts`)
 
 Everything else (store path, namespace, logging, recall tuning, lifecycle, ...) can be set per project. To trust a project file completely, set `UTEKE_TRUST_PROJECT_CONFIG=1`. The CLI and `uteke-serve` apply the same rules and the same per-key merge.
 
+## Privacy Filter (#1464)
+
+Secrets in the content of a memory are masked before it is embedded or stored (`ghp_[REDACTED:github-token]`, `API_KEY=[REDACTED:env-secret]`). The memory is flagged with `metadata.redacted` and `metadata.redactions` (kinds and counts, never values) and gets a `redacted` timeline event. The original text is not kept anywhere.
+
+```toml
+[privacy]
+mode = "redact"            # redact (default) | strict | off
+allowlist = ["^dev-"]      # regexes; a value matching any of them is kept as it is
+```
+
+- `redact`: mask and store.
+- `strict`: refuse the write (`secret_detected`; import skips the row).
+- `off`: store as given.
+
+`UTEKE_PRIVACY_MODE` overrides `mode`. An unknown mode or an invalid regex is logged and replaced by the default (`redact`) for that setting; the filter never turns itself off because of a typo. The filter covers the content of `remember`, room remember, `update` and `import`; metadata, tags and `source` are not filtered yet.
+
 ## CLI Flag Override
 
 CLI flags always take precedence over config file values: