Governance

CHANGELOG.md

@@ -0,0 +1,107 @@
+# Changelog
+
+All notable changes to the Montycat Python client are documented here.
+
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this
+project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.1.0] - 2026-07-28
+
+### Added
+
+- **Data-mesh governance policy API** on `Engine`:
+  - inspection — `policy_view`, `policy_history`, `policy_explain`, `policy_export`
+  - mutation — `policy_grant`, `policy_revoke`, `policy_deny`, `policy_remove_denial`
+  - dry runs — `policy_preview_grant`, `policy_preview_revoke`
+  - manifests — `policy_validate`, `policy_plan`, `policy_apply` (JSON or YAML documents)
+- New enums, exported from the package root: `PolicyCapability`, `PolicyKeyspaceType`,
+  `SemanticModel`, `PolicyFormat`.
+- `keyspace=` on `enable_semantic_search` / `disable_semantic_search`, for enrolling or
+  dropping a single keyspace instead of the whole store. Raises `ValueError` when a
+  keyspace is given without a store.
+
+### Changed
+
+- `grant_to` and `revoke_from` normalize the permission token (strip + lowercase), so
+  `" ALL "` and `"WrItE"` are accepted and `Permission` enum members serialize correctly
+  instead of reaching the wire as `Permission.ALL`.
+- `policy_explain` and the policy mutations drop `keyspace` for
+  `PolicyCapability.PROVISION_KEYSPACE`, which is a store-level capability.
+- Policy qualifiers are capability-specific: `models`/`model` apply to
+  `PROVISION_KEYSPACE` and `MANAGE_SEMANTIC`; storage-type qualifiers apply to
+  keyspace-scoped capabilities except `MANAGE_SNAPSHOTS`, where in-memory storage is
+  implicit. Invalid combinations raise `ValueError` before a command is sent.
+
+### Breaking
+
+- `enable_semantic_search(model=...)` now takes a `SemanticModel`, not a `str`. Passing a
+  string raises `AttributeError`.
+
+  ```python
+  # before
+  await engine.enable_semantic_search(model="bge-small")
+
+  # now
+  from montycat import SemanticModel
+  await engine.enable_semantic_search(model=SemanticModel.BGE_SMALL)
+  ```
+
+### Internal
+
+- Added `tests/test_governance_response_contract.py` — offline unit tests covering
+  governance error/payload preservation through `recursive_parse_orjson` and permission
+  token normalization. Not part of the distributed package.
+
+## [1.0.7] - 2026-07-20
+
+### Added
+
+- Hybrid semantic search: `semantic_search_get_keys_where` and
+  `semantic_search_get_values_where` take a `filters` dict applied as a hard metadata
+  pre-filter, so only matching items are ranked and scores stay pure cosine. Empty
+  `filters` raises `ValueError` — use the unfiltered methods instead.
+- `min_score` threading through the shared semantic search path.
+
+## [1.0.6] - 2026-07-19
+
+### Changed
+
+- Semantic search response structure: ranked hits are
+  `{'__key__': ..., '__score__': ...}`, and the value-returning variants add
+  `'__value__'`.
+
+## [1.0.5] - 2026-07-16
+
+### Added
+
+- Semantic search — `enable_semantic_search` / `disable_semantic_search` plus the first
+  `semantic_search_get_keys` / `semantic_search_get_values`.
+- Engine controls — `enable_wait_for_index` / `disable_wait_for_index`,
+  `enable_reports` / `disable_reports`, `allow_subscriptions` / `restrict_subscriptions`,
+  `queue_depths`, `set_snapshot_rate`, `set_expiration_check_rate`.
+- `wait_for_index` on the write paths: `insert_value`, `insert_custom_key`,
+  `insert_custom_key_value`, `insert_bulk`, `update_value`, `update_bulk`, `delete_key`,
+  `delete_bulk`.
+
+## [1.0.4] - 2026-05-12
+
+### Changed
+
+- Input validation for key retrieval and keyspace creation, tighter `limit` validation in
+  the generic and persistent KV classes.
+- Bulk key handling uses explicit concatenation; query parameters are passed directly to
+  `convert_to_binary_query` instead of via class attributes.
+- `setup.py` became the single source of truth for package metadata; `build_and_push.sh`
+  derives the version from it.
+
+## [1.0.1] - 2026-02-04
+
+### Added
+
+- Nullable schema fields.
+
+## [1.0.0] - 2025-10-17
+
+### Added
+
+- TLS support and a dedicated subscription port.

NOTES.md

@@ -0,0 +1,2 @@
+RUN OFFLINE TESTS
+python3 -m unittest discover -s tests -v

README.md

@@ -178,7 +178,7 @@ strong = await Sales.semantic_search_get_keys("Show all Bluetooth devices", limi
 
 # Control the DB-wide switch (optional — it's already on):
 # switch the embedding model: 'minilm' | 'bge-small' (default) | 'bge-base' | 'e5-small'
-await connection.enable_semantic_search(model="bge-base")
+await connection.enable_semantic_search(model=SemanticModel.BGE_BASE)
 
 # turn it off (vectors are kept so re-enabling resumes instantly;
 # pass drop_vectors=True to also clear stored vectors)
@@ -222,3 +222,36 @@ matching_values = await Sales.semantic_search_get_values_where(
 - **Is it a Pinecone / Weaviate / Chroma / Qdrant alternative?** Yes — self-hosted and open-source, with a NoSQL store built in.
 - **Which Python versions?** 3.9+ — fully async (`asyncio`).
 
+## Data-mesh governance for shared and multi-tenant deployments
+
+Delegate administration without handing every team full server control. Policies scope
+authority to an owner and store, with optional keyspace, storage-type, and semantic-model
+constraints. This lets platform teams govern shared infrastructure while domain teams
+operate the data products they own.
+
+- Grant, revoke, or explicitly deny keyspace provisioning/removal, schema, semantic,
+  snapshot, and access-management capabilities.
+- Inspect effective permissions and history, or preview a grant/revoke before applying it.
+- Validate, plan, apply, and export JSON or YAML policy manifests for repeatable
+  infrastructure-as-code workflows.
+
+For example, a superowner can separately constrain keyspace provisioning and semantic
+management within one store:
+
+```python
+from montycat import PolicyCapability, PolicyKeyspaceType, SemanticModel
+
+await connection.policy_grant(
+    "alice", PolicyCapability.PROVISION_KEYSPACE, "catalog",
+    types=[PolicyKeyspaceType.IN_MEMORY, PolicyKeyspaceType.PERSISTENT],
+    models=[SemanticModel.BGE_SMALL],
+)
+await connection.policy_grant(
+    "alice", PolicyCapability.MANAGE_SEMANTIC, "catalog",
+    keyspace="products", models=[SemanticModel.BGE_SMALL],
+)
+await connection.policy_view(owner="alice", store="catalog")
+```
+
+Superowners may also call `policy_validate`, `policy_plan`, `policy_apply`, and
+`policy_export` with JSON or YAML policy documents.

montycat/__init__.py

@@ -1,4 +1,4 @@
 from .core.engine import Engine
-from .core.tools import Pointer, Timestamp, Permission
+from .core.tools import Pointer, Timestamp, Permission, PolicyCapability, PolicyKeyspaceType, SemanticModel, PolicyFormat
 from .core.schema import Schema
 from .core.store import Keyspace

montycat/core/engine.py

@@ -1,7 +1,7 @@
 import orjson
 from typing import Union, List, Optional, Any
 from urllib.parse import urlparse
-from .tools import Permission
+from .tools import Permission, PolicyCapability, PolicyKeyspaceType, SemanticModel, PolicyFormat
 from .utils import send_data
 
 class Engine:
@@ -117,7 +117,7 @@ async def remove_store(self) -> Any:
             'remove-store', "store", self.store
         ])
 
-    async def grant_to(self, owner: str, permission: Union[str, Permission], keyspaces: Optional[Union[List[str], str, None]] = None) -> Any:
+    async def grant_to(self, owner: str, permission: Union[str, Permission], keyspaces: Optional[Union[List[str], str, None]] = None) -> Any:
         """
         Grants specific permissions to a user for the current store.
 
@@ -132,10 +132,11 @@ async def grant_to(self, owner: str, permission: Union[str, Permission], keyspac
         Raises:
             ValueError: If an invalid permission is provided.
         """
-        if permission not in self.VALID_PERMISSIONS:
-            raise ValueError(f"Invalid permission: {permission}. Valid permissions are: {self.VALID_PERMISSIONS}")
-
-        command = ['grant-to', "owner", owner, "permission", permission, "store", self.store]
+        normalized_permission = str(permission).strip().lower()
+        if normalized_permission not in self.VALID_PERMISSIONS:
+            raise ValueError(f"Invalid permission: {permission}. Valid permissions are: {self.VALID_PERMISSIONS}")
+
+        command = ['grant-to', "owner", owner, "permission", normalized_permission, "store", self.store]
         if keyspaces:
             command.append("keyspaces")
             if isinstance(keyspaces, str):
@@ -160,10 +161,11 @@ async def revoke_from(self, owner: str, permission: Union[str, Permission], keys
         Raises:
             ValueError: If an invalid permission is provided.
         """
-        if permission not in self.VALID_PERMISSIONS:
-            raise ValueError(f"Invalid permission: {permission}. Valid permissions are: {self.VALID_PERMISSIONS}")
-
-        command = ['revoke-from', "owner", owner, "permission", permission, "store", self.store]
+        normalized_permission = str(permission).strip().lower()
+        if normalized_permission not in self.VALID_PERMISSIONS:
+            raise ValueError(f"Invalid permission: {permission}. Valid permissions are: {self.VALID_PERMISSIONS}")
+
+        command = ['revoke-from', "owner", owner, "permission", normalized_permission, "store", self.store]
         if keyspaces:
             command.append("keyspaces")
             if isinstance(keyspaces, str):
@@ -211,7 +213,7 @@ async def list_owners(self) -> Any:
         """
         return await self._execute_query_with_credentials(['list-owners'])
 
-    async def enable_semantic_search(self, model: Union[str, None] = None, field: Union[str, None] = None, store: Union[str, None] = None) -> Any:
+    async def enable_semantic_search(self, model: Optional[SemanticModel] = None, field: Union[str, None] = None, store: Union[str, None] = None, keyspace: Union[str, None] = None) -> Any:
         """
         Enable semantic (vector similarity) search.
 
@@ -238,17 +240,21 @@ async def enable_semantic_search(self, model: Union[str, None] = None, field: Un
         Returns:
             Any: The server's response describing the enabled model and enrolled keyspaces.
         """
-        command = ['enable-semantic-search']
+        if keyspace and not store:
+            raise ValueError("A store is required when keyspace is specified")
+        command = ['enable-semantic-search']
         if model:
-            command.extend(["model", model])
+            command.extend(["model", model.value])
         if field:
             command.extend(["field", field])
-        if store:
-            command.extend(["store", store])
+        if store:
+            command.extend(["store", store])
+        if keyspace:
+            command.extend(["keyspace", keyspace])
 
         return await self._execute_query_with_credentials(command)
 
-    async def disable_semantic_search(self, drop_vectors: bool = False, store: Union[str, None] = None) -> Any:
+    async def disable_semantic_search(self, drop_vectors: bool = False, store: Union[str, None] = None, keyspace: Union[str, None] = None) -> Any:
         """
         Disable semantic search.
 
@@ -272,13 +278,104 @@ async def disable_semantic_search(self, drop_vectors: bool = False, store: Union
         Returns:
             Any: The server's response confirming the disable.
         """
-        command = ['disable-semantic-search']
+        if keyspace and not store:
+            raise ValueError("A store is required when keyspace is specified")
+        command = ['disable-semantic-search']
         if drop_vectors:
             command.append("drop-vectors")
-        if store:
-            command.extend(["store", store])
-
-        return await self._execute_query_with_credentials(command)
+        if store:
+            command.extend(["store", store])
+        if keyspace:
+            command.extend(["keyspace", keyspace])
+
+        return await self._execute_query_with_credentials(command)
+
+    async def policy_view(self, owner: Optional[str] = None, store: Optional[str] = None) -> Any:
+        command = ['policy-view']
+        if owner:
+            command.extend(['owner', owner])
+        if store:
+            command.extend(['store', store])
+        return await self._execute_query_with_credentials(command)
+
+    async def policy_history(self, owner: Optional[str] = None, store: Optional[str] = None, keyspace: Optional[str] = None) -> Any:
+        command = ['policy-history']
+        if owner:
+            command.extend(['owner', owner])
+        if store:
+            command.extend(['store', store])
+        if keyspace:
+            command.extend(['keyspace', keyspace])
+        return await self._execute_query_with_credentials(command)
+
+    async def policy_explain(self, capability: PolicyCapability, store: str, owner: Optional[str] = None, keyspace: Optional[str] = None, keyspace_type: Optional[PolicyKeyspaceType] = None, model: Optional[SemanticModel] = None) -> Any:
+        if keyspace_type and capability is PolicyCapability.MANAGE_SNAPSHOTS:
+            raise ValueError("keyspace_type is not valid for manage-snapshots policies; snapshots are always in-memory")
+        if model and capability not in (
+            PolicyCapability.PROVISION_KEYSPACE,
+            PolicyCapability.MANAGE_SEMANTIC,
+        ):
+            raise ValueError("model is only valid for provision-keyspace or manage-semantic policies")
+        command = ['policy-explain', 'capability', capability.value, 'store', store]
+        if owner:
+            command.extend(['owner', owner])
+        if keyspace and capability is not PolicyCapability.PROVISION_KEYSPACE:
+            command.extend(['keyspace', keyspace])
+        if keyspace_type:
+            command.extend(['type', keyspace_type.value])
+        if model:
+            command.extend(['model', model.value])
+        return await self._execute_query_with_credentials(command)
+
+    async def _policy_mutation(self, operation: str, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        if types and capability is PolicyCapability.MANAGE_SNAPSHOTS:
+            raise ValueError("types is not valid for manage-snapshots policies; snapshots are always in-memory")
+        if models and capability not in (
+            PolicyCapability.PROVISION_KEYSPACE,
+            PolicyCapability.MANAGE_SEMANTIC,
+        ):
+            raise ValueError("models is only valid for provision-keyspace or manage-semantic policies")
+        command = [operation, 'owner', owner, 'capability', capability.value, 'store', store]
+        if keyspace and capability is not PolicyCapability.PROVISION_KEYSPACE:
+            command.extend(['keyspace', keyspace])
+        if types:
+            command.extend(['types', *(keyspace_type.value for keyspace_type in types)])
+        if models:
+            command.extend(['models', *(model.value for model in models)])
+        return await self._execute_query_with_credentials(command)
+
+    async def policy_grant(self, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        return await self._policy_mutation('policy-grant', owner, capability, store, keyspace, types, models)
+
+    async def policy_revoke(self, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        return await self._policy_mutation('policy-revoke', owner, capability, store, keyspace, types, models)
+
+    async def policy_deny(self, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        return await self._policy_mutation('policy-deny', owner, capability, store, keyspace, types, models)
+
+    async def policy_remove_denial(self, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        return await self._policy_mutation('policy-remove-denial', owner, capability, store, keyspace, types, models)
+
+    async def policy_preview_grant(self, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        return await self._policy_mutation('policy-preview-grant', owner, capability, store, keyspace, types, models)
+
+    async def policy_preview_revoke(self, owner: str, capability: PolicyCapability, store: str, keyspace: Optional[str] = None, types: Optional[List[PolicyKeyspaceType]] = None, models: Optional[List[SemanticModel]] = None) -> Any:
+        return await self._policy_mutation('policy-preview-revoke', owner, capability, store, keyspace, types, models)
+
+    async def _policy_manifest(self, operation: str, document: str, format: PolicyFormat = PolicyFormat.JSON) -> Any:
+        return await self._execute_query_with_credentials([operation, 'format', format.value, 'document', document])
+
+    async def policy_validate(self, document: str, format: PolicyFormat = PolicyFormat.JSON) -> Any:
+        return await self._policy_manifest('policy-validate', document, format)
+
+    async def policy_plan(self, document: str, format: PolicyFormat = PolicyFormat.JSON) -> Any:
+        return await self._policy_manifest('policy-plan', document, format)
+
+    async def policy_apply(self, document: str, format: PolicyFormat = PolicyFormat.JSON) -> Any:
+        return await self._policy_manifest('policy-apply', document, format)
+
+    async def policy_export(self, format: PolicyFormat = PolicyFormat.JSON) -> Any:
+        return await self._execute_query_with_credentials(['policy-export', 'format', format.value])
 
     async def get_structure_available(self) -> Any:
         """