Several vaults per account (phase 3, 3.0.0-alpha.5)

README.md

@@ -28,7 +28,7 @@ SecretSpace lives in four repositories:
 
 ## SecretSpace 3 — a test version, not for real passwords yet
 
-`main` is now **3.0.0-alpha.4**, a test step of SecretSpace 3. **For your real
+`main` is now **3.0.0-alpha.5**, a test step of SecretSpace 3. **For your real
 passwords keep using [2.13.0](https://github.com/usermaneGermane/SecretSpace-Client-Windows/releases/tag/v2.13.0)**,
 which the rest of this README describes. Do not install the test version
 over it: it opens SecretSpace 3 accounts only, so your 2.x vaults would not
@@ -50,6 +50,18 @@ What the test version has:
 - **Importing a 2.x vault.** *Import* lists the 2.x vaults on this PC (or
   takes a `.vault.enc` file) and copies every entry with its history and
   archive state. The 2.x file is only read; its sync settings stay with it.
+- **Several vaults in one account** (new in alpha.5). Besides the
+  *Personal* vault an account can have more — *Home*, *Work*, a project —
+  made, renamed and deleted in the **Vaults** window. The menu above the
+  list picks which vault is shown (*All vaults* shows them together, each
+  row naming its vault, and search covers them all); a new entry goes to
+  the vault picked there, or to the one chosen in the *Add an entry*
+  window. Every vault has its own key. **Sync** covers all of them, each
+  with its own merge and conflict questions; it needs SecretSpace Server
+  **2.0.0-alpha.2** (an older 2.0 server keeps only the Personal vault, and
+  the app says so). A vault deleted on one device is **kept** on the others,
+  marked and no longer synced, until you delete it there too or put it
+  back.
 - **A smoother look** (new in alpha.4). Rounded corners are drawn smoothed
   instead of in pixel steps, the sign-in screen's padlock and key are
   redrawn, and an entry's coloured header has room around its tile.
@@ -66,13 +78,13 @@ What the test version has:
 - Everything else in the window — entries, archive, history, 2FA codes,
   generator, plain export — works as in 2.12.
 
-Not in this test version yet: more than one vault per account,
+Not in this test version yet: sharing a vault with other people (with a
+record of who changed what), moving an entry from one vault to another,
 recovering an account with the 12-word phrase (the phrase is made and
-checked now; the screen that uses it comes later), sharing a vault with
-other people, and *Sync when I unlock* for accounts. SecretSpace for Linux
-has the same test version, 1.0.0-alpha.4. Coming next, in this order:
-several vaults per account, sharing a vault with family or colleagues, and
-recovering an account with the 12-word phrase.
+checked now; the screen that uses it comes later), and *Sync when I
+unlock* for accounts. SecretSpace for Linux has the same test version,
+1.0.0-alpha.5. Coming next, in this order: sharing a vault with family or
+colleagues, and recovering an account with the 12-word phrase.
 
 ## Running it
 

sourceFiles/account_crypto.py

@@ -406,6 +406,49 @@ def decrypt_vault(vault_key: bytes, blob: str, expect_vault_id: bytes = None) ->
     return json.loads(body)
 
 
+# ===========================================================================
+# A vault's name, as the server keeps it
+# ===========================================================================
+#: Longest vault name, in characters and in UTF-8 bytes (the server's limit).
+VAULT_NAME_CHARS = 60
+VAULT_NAME_BYTES = 200
+
+
+def new_vault_key() -> bytes:
+    return os.urandom(KEY_LEN)
+
+
+def _name_key(vault_key: bytes, vault_id: bytes) -> bytes:
+    # Its own key, so a name and the vault's entries never share one.
+    return _hkdf(vault_key, vault_id, b"PassMan3 vault name")
+
+
+def seal_vault_name(vault_key: bytes, vault_id: bytes, name: str) -> str:
+    """
+    ``name`` under the vault's key, for the server: it lists an account's
+    vaults without learning what they are called. Bound to the vault id, so
+    one vault's name cannot be shown as another's.
+    """
+    raw = name.encode("utf-8")
+    if not raw or len(raw) > VAULT_NAME_BYTES:
+        raise CryptoError("A vault name must be 1 to 200 bytes long.")
+    nonce = os.urandom(NONCE_LEN)
+    return _b64(nonce + AESGCM(_name_key(vault_key, vault_id)).encrypt(
+        nonce, raw, vault_id))
+
+
+def open_vault_name(vault_key: bytes, vault_id: bytes, sealed: str) -> str:
+    raw = _unb64(sealed, "vault name")
+    if len(raw) < NONCE_LEN + 1 + 16:
+        raise CryptoError("The vault's name is damaged.")
+    try:
+        return AESGCM(_name_key(vault_key, vault_id)).decrypt(
+            raw[:NONCE_LEN], raw[NONCE_LEN:], vault_id).decode("utf-8")
+    except Exception:
+        raise CryptoError("The vault's name could not be read: wrong key, or "
+                          "it is damaged.") from None
+
+
 # ===========================================================================
 # The 12-word recovery phrase (BIP39, English)
 # ===========================================================================

sourceFiles/account_sync.py

@@ -11,9 +11,11 @@
 Three things live here:
 
 - :class:`AccountServer`, the HTTP calls, on sync_client's transport;
-- :class:`AccountSync`, one sync of the personal vault in two halves, with
-  the same three-way merge, conflict choices and base rules as 2.x
+- :class:`AccountSync`, one sync of one vault in two halves, with the same
+  three-way merge, conflict choices and base rules as 2.x
   (sync.merge_vaults, sync_client.base_matches);
+- :func:`reconcile_vaults` (phase 3), which makes the account's list of
+  vaults the same here and on the server before they are synced;
 - the account lifecycle around them: :func:`connect` (an account made here
   goes onto a server), :func:`sign_in` (a new device, or one catching up
   with a password changed elsewhere) and :func:`change_password`.
@@ -23,6 +25,7 @@
 """
 
 import base64
+import os
 import urllib.parse
 
 import account_crypto
@@ -37,6 +40,10 @@
 #: /health; a 1.x server answers without "api" and has only the 2.x one.
 API_VERSION = 2
 
+#: /health "features" entry of a server that keeps several vaults per
+#: account (SecretSpace Server 2.0.0-alpha.2).
+FEATURE_VAULTS = "vaults"
+
 
 class LoginFailed(SyncError):
     """The server did not take this username + password + Secret Key."""
@@ -52,6 +59,7 @@ class AccountServer(SyncClient):
     def __init__(self, url: str, ca_file: str = "",
                  timeout: int = sync_client.DEFAULT_TIMEOUT):
         super().__init__(url, "", ca_file, timeout)
+        self._features = None
 
     @staticmethod
     def _http_error(e) -> SyncError:
@@ -74,6 +82,24 @@ def check_api(self) -> None:
                             "it needs SecretSpace Server 2.0 or newer. (2.x "
                             "vaults sync with it as before.)")
 
+    def has_vaults(self) -> bool:
+        """Can this server keep more than the personal vault? Asked once
+        per connection."""
+        if self._features is None:
+            self._features = list(self.health().get("features") or [])
+        return FEATURE_VAULTS in self._features
+
+    def create_vault(self, vault_id: str, sealed_name: str, wrapped_key: str) -> dict:
+        return self._request("POST", "/v2/vaults", body={
+            "id": vault_id, "name": sealed_name, "wrapped_key": wrapped_key})
+
+    def rename_vault(self, vault_id: str, sealed_name: str) -> dict:
+        return self._request("PUT", f"/v2/vaults/{vault_id}/name",
+                             body={"name": sealed_name})
+
+    def delete_vault(self, vault_id: str) -> None:
+        self._request("DELETE", f"/v2/vaults/{vault_id}")
+
     def prelogin(self, username: str) -> dict:
         return self._request("GET", "/v2/prelogin/" + urllib.parse.quote(
             username, safe=""))
@@ -202,9 +228,12 @@ class AccountSync:
     dance here -- the vault key does not change with the master password.
     """
 
-    def __init__(self, account: accounts.Account, server: AccountServer = None):
+    def __init__(self, account: accounts.Account, server: AccountServer = None,
+                 vault_id: str = None):
         self.account = account
         self.server = server
+        # The vault this sync is for: the personal one unless told otherwise.
+        self.file = account.vault_file(vault_id)
         self.revision = 0
         self.result = None
         self.first_push = False
@@ -213,7 +242,7 @@ def __init__(self, account: accounts.Account, server: AccountServer = None):
 
     def prepare(self, local_vault: dict):
         self.server = logged_in(self.account, self.server)
-        stored = self.server.get_vault(self.account.vault_id.hex())
+        stored = self.server.get_vault(self.file.hex)
         self.revision = stored["revision"]
         self._fingerprint = sync_client.SyncSession.fingerprint(local_vault)
         blob = stored["blob"]
@@ -222,15 +251,15 @@ def prepare(self, local_vault: dict):
             remote = {"accounts": []}
         else:
             try:
-                remote = self.account.decrypt(blob)
+                remote = self.file.decrypt(blob)
             except (CryptoError, ValueError) as e:
                 raise SyncError(f"The vault on the server cannot be read: {e}") from e
         # The 2.x rules for the base (sync_client.SyncSession.prepare): none
         # when the server is empty, and none if it was agreed elsewhere.
         self.base_ignored = False
         base = None
         if blob is not None:
-            base = self.account.load_base()
+            base = self.file.load_base()
             if base is not None and not sync_client.base_matches(
                     base, self.server.url, self.revision):
                 base, self.base_ignored = None, True
@@ -251,12 +280,11 @@ def commit(self, local_vault: dict, choices=None) -> dict:
             merged_vault(local_vault, resolve(self.result, choices)))
         # Saved here first, then pushed: if the network fails in between,
         # this device holds the merge and the next sync pushes it.
-        self.account.save_vault(merged)
+        self.file.save(merged)
         if self.result.needs_push or self.first_push:
             self.revision = self.server.put_vault(
-                self.account.vault_id.hex(), self.account.encrypt(merged),
-                self.revision)
-        self.account.save_base(merged, sync_client.base_origin_for(
+                self.file.hex, self.file.encrypt(merged), self.revision)
+        self.file.save_base(merged, sync_client.base_origin_for(
             self.server.url, self.revision))
         return merged
 
@@ -265,6 +293,137 @@ def close(self) -> None:
             self.server.logout()
 
 
+# ---------------------------------------------------------------------------
+# The account's list of vaults, here and on the server
+# ---------------------------------------------------------------------------
+class VaultChanges:
+    """What :func:`reconcile_vaults` did, for the user to be told."""
+
+    def __init__(self):
+        self.added = []          # names: came from the server to this device
+        self.uploaded = []       # names: made here, now on the server too
+        self.gone = []           # names: deleted on the server, kept here
+        self.renamed = []        # (old, new): renamed on another device
+        self.server_too_old = False
+
+    def __bool__(self):
+        return bool(self.added or self.uploaded or self.gone or self.renamed
+                    or self.server_too_old)
+
+
+def reconcile_vaults(account: accounts.Account, server: AccountServer) -> VaultChanges:
+    """
+    Make the list of vaults the same on this device and on the server.
+    ``server`` is logged in. Vault *contents* are not touched here; the
+    caller syncs each vault afterwards.
+
+    - A vault the server has and this device does not is added here (its
+      entries arrive with its first sync).
+    - A vault made here goes onto the server.
+    - A vault that was on the server and is no longer there was deleted from
+      another device. It is **kept** on this one, marked, and left out of the
+      sync: deleting someone's entries is their decision, not a sync's.
+    - A name changed here is told to the server; one changed elsewhere is
+      taken over.
+
+    A server older than 2.0.0-alpha.2 keeps only the personal vault; the
+    others then simply stay on this device.
+    """
+    changes = VaultChanges()
+    extra_here = [v for v in account.vault_ids() if v != account.personal_id]
+    listed = {v["id"]: v for v in server.me().get("vaults", [])
+              if v.get("kind") != "personal" and v.get("id") != account.personal_id}
+    if not server.has_vaults():
+        changes.server_too_old = bool(extra_here)
+        return changes
+
+    for vault_id in extra_here:
+        record = account.vault_record(vault_id)
+        file = account.vault_file(vault_id)
+        there = listed.get(vault_id)
+        if there is None:
+            if record.get("on_server"):
+                if not record.get("gone_from_server"):
+                    account.mark_vault(vault_id, gone_from_server=True)
+                    changes.gone.append(file.name)
+                continue
+            try:
+                server.create_vault(vault_id, file.sealed_name(),
+                                    record["wrapped_key"])
+            except Conflict409:
+                raise SyncError(f"The server already has another vault with the "
+                                f"id of “{file.name}”. Nothing was changed.") from None
+            account.mark_vault(vault_id, on_server=True, gone_from_server=False)
+            changes.uploaded.append(file.name)
+            continue
+        if not record.get("on_server") or record.get("gone_from_server"):
+            account.mark_vault(vault_id, on_server=True, gone_from_server=False)
+        if record.get("rename_pending"):
+            server.rename_vault(vault_id, file.sealed_name())
+            account.mark_vault(vault_id, rename_pending=False)
+            continue
+        try:
+            name = accounts.clean_vault_name(file.open_name(there.get("name", "")))
+        except (CryptoError, ValueError):
+            continue                    # an unreadable name changes nothing
+        if name != file.name:
+            try:
+                account.rename_vault(vault_id, name, pending=False)
+                changes.renamed.append((file.name, name))
+            except ValueError:
+                pass                    # taken here: keep this device's name
+
+    for vault_id, there in listed.items():
+        if vault_id in extra_here:
+            continue
+        try:
+            file = account.adopt_vault(vault_id, there.get("wrapped_key", ""),
+                                       there.get("name", ""))
+        except (CryptoError, ValueError) as e:
+            raise SyncError(f"The server lists a vault this account cannot "
+                            f"open ({e}). Nothing was changed.") from e
+        changes.added.append(file.name)
+    return changes
+
+
+def syncable_vaults(account: accounts.Account, server: AccountServer) -> list:
+    """Ids of the vaults a sync covers: the personal one, and every other
+    that is on the server."""
+    if not server.has_vaults():
+        return [account.personal_id]
+    return [v for v in account.vault_ids()
+            if v == account.personal_id or (
+                account.vault_record(v).get("on_server")
+                and not account.vault_record(v).get("gone_from_server"))]
+
+
+def delete_vault(account: accounts.Account, vault_id: str) -> None:
+    """
+    Delete a vault for good. On a server it is deleted there first: if the
+    server cannot be reached nothing is deleted anywhere, so a vault never
+    comes back from the server after being deleted here.
+    """
+    record = account.vault_record(vault_id)
+    if account.server and record.get("on_server") \
+            and not record.get("gone_from_server"):
+        server = logged_in(account)
+        try:
+            server.delete_vault(vault_id)
+        finally:
+            server.logout()
+    account.delete_vault(vault_id)
+
+
+def put_vault_back(account: accounts.Account, vault_id: str) -> None:
+    """A vault deleted on the server and kept here goes onto the server
+    again with the next sync (the user's choice, in the Vaults window)."""
+    account.mark_vault(vault_id, on_server=False, gone_from_server=False)
+    try:
+        os.remove(account.vault_file(vault_id).base_path)   # agreed with a vault that is gone
+    except OSError:
+        pass
+
+
 # ---------------------------------------------------------------------------
 # Signing in from a server
 # ---------------------------------------------------------------------------
@@ -326,6 +485,19 @@ def sign_in(url: str, ca_file: str, username: str, password: str,
         if stored["blob"]:
             account.save_base(data, sync_client.base_origin_for(
                 server.url, stored["revision"]))
+    # The account's other vaults come with it, so a new device opens with
+    # everything; a later sync merges whatever this device already had.
+    reconcile_vaults(account, server)
+    for vault_id in syncable_vaults(account, server):
+        file = account.vault_file(vault_id)
+        if file.personal or file.has_file:
+            continue
+        stored = server.get_vault(vault_id)
+        fetched = file.decrypt(stored["blob"]) if stored["blob"] else {"accounts": []}
+        file.save(fetched)
+        if stored["blob"]:
+            file.save_base(fetched, sync_client.base_origin_for(
+                server.url, stored["revision"]))
     server.logout()
     return account, data