feat(web): import your own data, export the library, and give visitors a private library

.github/workflows/ci.yml

@@ -92,6 +92,23 @@ jobs:
             -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"search","arguments":{"query":"litestream"}}}' \
             | grep -q 'Moving Tidewater from Postgres to SQLite'
 
+      - name: Upload an export as a visitor, then download and delete it
+        run: |
+          curl -fs -c visitor.txt http://127.0.0.1:7860/library/import \
+            -H 'X-ChatLore: 1' -H 'X-Filename: conversations.json' \
+            --data-binary @tests/fixtures/claude/conversations.json
+          for attempt in $(seq 60); do
+            curl -fs -b visitor.txt http://127.0.0.1:7860/library | grep -q '"running":false' && break
+            sleep 2
+          done
+          curl -fs -b visitor.txt http://127.0.0.1:7860/library | grep -q '"stage":"done"'
+          curl -fs -b visitor.txt http://127.0.0.1:7860/stats | grep -q '"conversations":3,'
+          curl -fs http://127.0.0.1:7860/stats | grep -q '"conversations":32,'
+          curl -fs -b visitor.txt -o library.zip http://127.0.0.1:7860/library/export
+          unzip -l library.zip | grep -q manifest.json
+          curl -fs -b visitor.txt -X DELETE -H 'X-ChatLore: 1' http://127.0.0.1:7860/library
+          curl -fs -b visitor.txt http://127.0.0.1:7860/library | grep -q '"own":false'
+
       - name: Show the server's output
         if: always()
         run: docker logs demo || true

CHANGELOG.md

@@ -19,6 +19,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - A `Dockerfile` for the hosted demo, with the demo library and embedding model built in. CI
   builds it, starts it, and checks the web interface, the API, and a tool call over MCP. Guide
   in `docs/hosting.md`.
+- **Your data** in the web interface: upload a ChatGPT, Claude, or Gemini export, Markdown
+  notes, or a ChatLore archive, and watch it be imported, embedded, and read into the knowledge
+  graph in the background; download the library as a ChatLore archive or as Markdown.
+  `POST /library/import`, `GET /library`, `GET /library/export`, and `DELETE /library` do the
+  same over the API. Guide in `docs/web.md`.
+- `chatlore serve --public --uploads` lets visitors try ChatLore on their own conversations:
+  each who uploads gets a private library tied to their browser, deleted after 24 hours
+  (`--keep-hours`) or when they choose, while everyone else sees the server's library.
+  `--max-upload-mb` and `--extract-limit` set how large an upload may be and how much of it the
+  model reads. The hosted demo image turns it on. Guide in `docs/hosting.md`.
+- `export_archive` and `import_archive` take a store that is already open.
 - A FalkorDB store, chosen with `CHATLORE_STORE=falkordb`, with `CHATLORE_FALKORDB_URL` and
   `CHATLORE_FALKORDB_GRAPH` saying where. Every command, the web interface, and the MCP server
   work on it as on SQLite, and it passes the same contract tests. The client is an optional

Dockerfile

@@ -1,5 +1,7 @@
 # The hosted ChatLore demo: the made-up demo library behind the public web
-# interface, API, and MCP endpoint. See docs/hosting.md.
+# interface, API, and MCP endpoint, where visitors can also upload their own
+# export into a private library that is deleted after 24 hours. See
+# docs/hosting.md.
 #
 #   docker build -t chatlore-demo .
 #   docker run -p 7860:7860 -e OPENROUTER_API_KEY=... chatlore-demo
@@ -28,4 +30,4 @@ RUN chatlore demo --no-serve \
 
 EXPOSE 7860
 # PORT is set by hosts such as Render and Fly.io; Hugging Face Spaces uses 7860.
-CMD ["sh", "-c", "exec chatlore --home \"$HOME/.chatlore-demo\" serve --public --host 0.0.0.0 --port \"${PORT:-7860}\""]
+CMD ["sh", "-c", "exec chatlore --home \"$HOME/.chatlore-demo\" serve --public --uploads --host 0.0.0.0 --port \"${PORT:-7860}\""]

README.md

@@ -50,6 +50,9 @@ chatlore stats
 ```
 
 The library is kept in `~/.chatlore`; `--home <folder>` or `CHATLORE_HOME` picks another.
+The web interface can do the same without the terminal: after `chatlore serve`,
+**Your data** uploads an export, builds everything from it, and downloads the
+library again.
 
 The source is detected from the file. Importing is idempotent, so re-running it
 after a fresh export only adds what changed. How to get each export, what is
@@ -98,6 +101,7 @@ extraction is an optional enrichment you can re-run with a better model later.
 | M8 | Easy to try: PyPI package, demo library, export and import | done, v0.1.0 |
 | M9 | Hosted demo | public mode, MCP over HTTP, and Docker image done; deployment next |
 | M10 | FalkorDB backend | done |
+| M11 | Your own data in the web interface: upload, export, private visitor libraries | done |
 
 ## Development setup
 

docs/hosting.md

@@ -56,16 +56,54 @@ front of a hosting service:
   through a visitor's browser.
 - The web interface says it is a demo, and `/health` reports `"public": true`.
 
-The library is only ever read. Never serve your own library this way: anyone
-who finds the address can read every conversation in it.
+The server's library is only ever read. Never serve your own library this way:
+anyone who finds the address can read every conversation in it.
+
+## Visitors' own data
+
+The image also runs with `--uploads`, which lets visitors try ChatLore on their
+own conversations:
+
+```bash
+chatlore --home ~/.chatlore-demo serve --public --uploads --host 0.0.0.0 --port 7860
+```
+
+- A visitor who uploads an export under **Your data** gets a library of their
+  own, and everything they then see, search, and ask is theirs alone. Everyone
+  else still sees the server's library.
+- The library is tied to the visitor's browser by a random token in a cookie
+  (HttpOnly, SameSite=Lax, and Secure over HTTPS). Its folder is named after a
+  hash of the token, so the server's files do not give the token away.
+- It is deleted after 24 hours, or `--keep-hours`, and at once when the visitor
+  chooses **Delete my library**; an import still running is stopped first. The
+  server looks for expired libraries every ten minutes. They are kept in
+  `~/.chatlore-spaces`, or `--spaces-dir`, each with its graph in its own SQLite
+  file, even when the server's library is on FalkorDB.
+- Uploads are imported one at a time, so the server keeps answering while one
+  runs, and each visitor has one import at a time. An upload may be up to
+  200 MB, or `--max-upload-mb`; a zip may unpack to at most 2 GB of text.
+- Every change (an upload, a deletion) must carry the header the web interface
+  sends, `X-ChatLore: 1`, so another website cannot make a visitor's browser
+  upload or delete.
+- Building the knowledge graph from an upload reads all of it with the
+  language model on the server's key, as `chatlore extract` does locally, so a
+  large export costs as much as extracting it at home. `--extract-limit` caps
+  how many passages of each upload the model reads; the rest stays searchable.
+  The dialog tells visitors that their conversations go to that model.
+- Visitors download their library as a ChatLore archive or as Markdown.
+
+Assistants connected over `/mcp` see the server's library, since they carry no
+visitor's cookie.
 
 ## Where to host it
 
 Any service that builds and runs a Dockerfile from a Git repository works, and
 gives the container an HTTPS address. ChatGPT only connects to HTTPS.
 
 CI builds the image on every pull request, starts it, and checks the web
-interface, the API, and a tool call over MCP.
+interface, the API, and a tool call over MCP. It then uploads an export as a
+visitor, waits for the import, checks that only that visitor sees it, downloads
+it, and deletes it.
 
 ## Connecting assistants to it
 

docs/web.md

@@ -28,6 +28,27 @@ follows the system's light or dark setting.
   is a circle sized by how often it is mentioned and coloured by its topic, and
   related entities are linked.
 
+## Your data
+
+**Your data**, in the navigation, imports an export and downloads the library:
+
+- Drop a file on the dialog, or choose one: a ChatGPT or Claude export (.zip),
+  Gemini Takeout (.zip or MyActivity.json), Markdown notes (.zip or .md), or a
+  ChatLore archive. Uploads may be up to 200 MB, or what `--max-upload-mb` sets.
+- The import runs in the background, like `chatlore import`, `chatlore
+  process`, and `chatlore extract` one after another, and the dialog shows each
+  step with its progress: importing, preparing search, reading with the language
+  model, summarising, linking names for the same thing, and topics. Without a
+  model key, everything but the knowledge graph is built. When it is done,
+  **Show the library** reloads the page on it.
+- **ChatLore archive** downloads the whole library, graph and caches included,
+  to import anywhere; **Markdown** downloads one readable file per conversation.
+  See [export.md](export.md).
+
+On your own machine the upload goes into the library itself. On a public server
+that takes uploads, it goes into a private library of the visitor's own; see
+[hosting.md](hosting.md#visitors-own-data).
+
 ## Exploring the graph
 
 The graph fills the page, with a floating toolbar, a legend of the largest