feat(api): serve a public demo with question limits, MCP over HTTP, and a Docker image

.dockerignore

@@ -0,0 +1,8 @@
+# Only what the image needs: pyproject.toml, uv.lock, README.md, LICENSE, src/.
+*
+!pyproject.toml
+!uv.lock
+!README.md
+!LICENSE
+!src/
+**/__pycache__

Dockerfile

@@ -0,0 +1,31 @@
+# The hosted ChatLore demo: the made-up demo library behind the public web
+# interface, API, and MCP endpoint. See docs/hosting.md.
+#
+#   docker build -t chatlore-demo .
+#   docker run -p 7860:7860 -e OPENROUTER_API_KEY=... chatlore-demo
+
+FROM ghcr.io/astral-sh/uv:0.12-python3.12-trixie-slim
+
+ENV UV_COMPILE_BYTECODE=1 UV_LINK_MODE=copy UV_PYTHON_DOWNLOADS=never
+WORKDIR /app
+
+# Dependencies first, so changing the code does not reinstall them.
+COPY pyproject.toml uv.lock ./
+RUN uv sync --locked --no-dev --no-install-project
+COPY README.md LICENSE ./
+COPY src ./src
+RUN uv sync --locked --no-dev --no-editable
+
+# Hugging Face Spaces and many other hosts run containers as user 1000.
+RUN useradd --create-home --uid 1000 chatlore
+USER chatlore
+ENV HOME=/home/chatlore PATH=/app/.venv/bin:$PATH
+
+# Load the demo library and download the embedding model while building, so
+# the container answers as soon as it starts.
+RUN chatlore demo --no-serve \
+    && chatlore --home "$HOME/.chatlore-demo" search --semantic "warm up" > /dev/null
+
+EXPOSE 7860
+# PORT is set by hosts such as Render and Fly.io; Hugging Face Spaces uses 7860.
+CMD ["sh", "-c", "exec chatlore --home \"$HOME/.chatlore-demo\" serve --public --host 0.0.0.0 --port \"${PORT:-7860}\""]

README.md

@@ -93,7 +93,7 @@ extraction is an optional enrichment you can re-run with a better model later.
 | M4 | Entity, topic, and fact extraction with provenance | entities and topics done; facts later |
 | M5 | REST API with streaming chat | done |
 | M6 | Web UI: conversations, graph explorer, chat | basic version done |
-| M7 | MCP server for Claude Desktop, Claude Code, Cursor, ChatGPT | done for local assistants; ChatGPT with the hosted demo |
+| M7 | MCP server for Claude Desktop, Claude Code, Cursor, ChatGPT | done; ChatGPT through /mcp once the demo is online |
 | M8 | Easy to try: PyPI package, demo library, export and import | done, v0.1.0 |
 | M9 | Hosted demo | planned |
 | M10 | FalkorDB backend | done |

docs/hosting.md

@@ -0,0 +1,73 @@
+# Hosting the demo
+
+```bash
+docker build -t chatlore-demo .
+docker run -p 7860:7860 -e OPENROUTER_API_KEY=sk-or-... chatlore-demo
+```
+
+The image in the repository's `Dockerfile` is the hosted demo: the made-up
+library from `chatlore demo`, served to anyone who opens it. Visitors get the
+web interface at `/`, the REST API with its docs at `/docs`, and the MCP server
+at `/mcp`, so they can connect ChatGPT, Claude, or another assistant to it
+without installing anything.
+
+The demo library and the embedding model are built into the image, so a
+container answers as soon as it starts. It used about 400 MB of memory once
+searches had loaded the embedding model; give it 1 GB to be safe. It listens on
+port 7860, or on `$PORT` when the host sets it.
+
+## The model key
+
+Search, conversations, topics, the graph, and every MCP tool work without a
+key. Only asking in the web interface or through `POST /chat` needs a language
+model, and that is paid for with the host's key. Pass it as a secret, never in
+the image:
+
+| Variable | Meaning |
+|---|---|
+| `OPENROUTER_API_KEY` | Key for OpenRouter, the default provider. |
+| `CHATLORE_LLM_MODEL`, `CHATLORE_LLM_BASE_URL` | Another model or provider; see [models.md](models.md). |
+
+Without a key, asking says that a key is needed and everything else works.
+OpenRouter lets a key have a credit limit; give the demo's key one.
+
+## Public mode
+
+The image runs `chatlore serve --public`. Anyone can run the same on a library
+they are happy for anyone to read:
+
+```bash
+chatlore --home ~/.chatlore-demo serve --public --host 0.0.0.0 --port 7860
+```
+
+`--public` is for a server behind a proxy on the internet, such as the HTTPS
+front of a hosting service:
+
+- Questions that reach the language model are limited: 10 an hour for each
+  visitor and 300 a day for everyone together, by default. Change them with
+  `--questions-per-hour` and `--questions-per-day`. A question that finds no
+  passages is not counted, since it never reaches the model. When a limit is
+  reached, the visitor is told why and how to install ChatLore instead.
+- Visitors are told apart by the address the proxy reports, from
+  `X-Forwarded-For`. Without a proxy in front, visitors could claim any address,
+  but the daily limit still holds.
+- `/mcp` accepts requests for any host name. A private server only answers
+  requests addressed to this machine, which keeps websites from reaching it
+  through a visitor's browser.
+- The web interface says it is a demo, and `/health` reports `"public": true`.
+
+The library is only ever read. Never serve your own library this way: anyone
+who finds the address can read every conversation in it.
+
+## Where to host it
+
+Any service that builds and runs a Dockerfile from a Git repository works, and
+gives the container an HTTPS address. ChatGPT only connects to HTTPS.
+
+CI builds the image on every pull request, starts it, and checks the web
+interface, the API, and a tool call over MCP.
+
+## Connecting assistants to it
+
+The MCP endpoint is the demo's address followed by `/mcp`, with no
+authentication. See [mcp.md](mcp.md#over-http) for each assistant.

docs/mcp.md

@@ -80,8 +80,32 @@ project, or `.cursor/mcp.json` for one.
 
 **Other clients.** Any MCP client that starts local servers works the same way:
 the command is `chatlore` with the argument `mcp`. ChatGPT only connects to
-servers on the internet, which a private library should not be; that will come
-with the hosted demo.
+servers on the internet, so it uses the hosted demo, below.
+
+## Over HTTP
+
+`chatlore serve` serves the same tools at `/mcp` over streamable HTTP, for
+assistants that connect to a URL instead of starting a command:
+
+```bash
+chatlore serve                                                        # then:
+claude mcp add --transport http chatlore http://127.0.0.1:8000/mcp
+```
+
+On your machine, `/mcp` only answers requests addressed to `127.0.0.1` or
+`localhost`, so websites cannot reach it through your browser.
+
+The hosted demo serves its made-up library the same way at its address followed
+by `/mcp`, with no authentication (see [hosting.md](hosting.md)):
+
+- **ChatGPT**: in developer mode, add a connector with that URL and no
+  authentication.
+- **Claude**: add it as a custom connector, or in Claude Code with
+  `claude mcp add --transport http chatlore-demo <url>`.
+- **Cursor**: an `mcpServers` entry with `"url": "<url>"` instead of a command.
+
+Never put your own library on the internet this way: anyone with the address
+could read it.
 
 ## Tools
 

src/chatlore/api.py

@@ -6,28 +6,39 @@
 is written, then which sources it cited. The web interface at / is built on
 these endpoints.
 
+The MCP server is served at /mcp too, over streamable HTTP, for assistants
+that connect to a URL instead of starting ``chatlore mcp``.
+
 Each request opens the store and closes it again, so requests never share a
 database connection across threads. The server listens on 127.0.0.1 unless told
-otherwise, because the library is private.
+otherwise, because the library is private. A public server, such as the hosted
+demo, limits how many questions reach the language model, since each one is paid
+for with the host's key.
 """
 
 from __future__ import annotations
 
-from collections.abc import Iterator
+import threading
+import time
+from collections import deque
+from collections.abc import AsyncIterator, Callable, Iterator
+from contextlib import asynccontextmanager
 from pathlib import Path
 from typing import Annotated, Any
 
-from fastapi import FastAPI, HTTPException, Query
+from fastapi import FastAPI, HTTPException, Query, Request
 from fastapi.sse import EventSourceResponse, ServerSentEvent
 from fastapi.staticfiles import StaticFiles
+from mcp.server.transport_security import TransportSecuritySettings
 from pydantic import BaseModel, Field
 from starlette.responses import Response
 from starlette.types import Scope
 
 from chatlore import __version__
-from chatlore.chat import MAX_SOURCES, Context, Source, answer, cited, retrieve
+from chatlore.chat import MAX_SOURCES, ChatLimits, Context, Source, answer, cited, retrieve
 from chatlore.embeddings import Embedder, EmbeddingError, make_embedder, normalise
 from chatlore.llm import LLMError, make_llm
+from chatlore.mcp_server import create_server
 from chatlore.paths import default_home
 from chatlore.search import hybrid_search
 from chatlore.store import Edge, EdgeType, GraphStore, Label, Node, open_store
@@ -49,6 +60,54 @@ async def get_response(self, path: str, scope: Scope) -> Response:
         return response
 
 
+_HOUR = 3600.0
+_MAX_VISITORS = 10_000
+
+
+class _Allowance:
+    """Counts questions against ``ChatLimits``. Shared by the request threads."""
+
+    def __init__(self, limits: ChatLimits, clock: Callable[[], float] = time.time) -> None:
+        self.limits = limits
+        self._clock = clock
+        self._lock = threading.Lock()
+        self._visitors: dict[str, deque[float]] = {}
+        self._day = ""
+        self._today = 0
+
+    def take(self, visitor: str) -> str | None:
+        """Count a question from ``visitor``, or say why it cannot be asked now."""
+        now = self._clock()
+        day = time.strftime("%Y-%m-%d", time.gmtime(now))
+        with self._lock:
+            if day != self._day:
+                self._day, self._today = day, 0
+            asked = self._visitors.setdefault(visitor, deque())
+            while asked and asked[0] <= now - _HOUR:
+                asked.popleft()
+            if len(asked) >= self.limits.per_visitor_hour:
+                return (
+                    f"This demo answers {self.limits.per_visitor_hour} questions an hour for "
+                    "each visitor. Try again later, or install ChatLore to ask your own "
+                    "conversations: pip install chatlore"
+                )
+            if self._today >= self.limits.per_day:
+                return (
+                    "This demo has answered all the questions it can today. Try again "
+                    "tomorrow, or install ChatLore to ask your own conversations: "
+                    "pip install chatlore"
+                )
+            asked.append(now)
+            self._today += 1
+            if len(self._visitors) > _MAX_VISITORS:
+                self._visitors = {
+                    key: times
+                    for key, times in self._visitors.items()
+                    if times and times[-1] > now - _HOUR
+                }
+        return None
+
+
 class ChatRequest(BaseModel):
     question: str = Field(min_length=1, max_length=4000)
     sources: int = Field(default=MAX_SOURCES, ge=1, le=20)
@@ -90,15 +149,39 @@ def _topic(node: Node, entities: int | None = None) -> dict[str, Any]:
     }
 
 
-def create_app(home: Path | None = None) -> FastAPI:
-    """The API over the library in ``home``, or the default ChatLore home."""
+def create_app(
+    home: Path | None = None, *, public: bool = False, limits: ChatLimits | None = None
+) -> FastAPI:
+    """The API over the library in ``home``, or the default ChatLore home.
+
+    A ``public`` app says so at /health, which makes the web interface show a
+    demo note, and accepts MCP requests for any host name, since it is meant to
+    be reached from the internet. Otherwise /mcp only answers requests addressed
+    to this machine, which keeps other websites from reaching it through the
+    browser. ``limits`` caps the questions sent to the language model.
+    """
     library = home or default_home()
+    assistants = create_server(library)
+    security = TransportSecuritySettings(enable_dns_rebinding_protection=False) if public else None
+    # Stateless, with plain JSON replies: any worker can answer any request, and
+    # no tool streams its result.
+    mcp_app = assistants.streamable_http_app(
+        stateless_http=True, json_response=True, transport_security=security
+    )
+
+    @asynccontextmanager
+    async def lifespan(_: FastAPI) -> AsyncIterator[None]:
+        async with assistants.session_manager.run():
+            yield
+
     app = FastAPI(
         title="ChatLore",
         version=__version__,
         description="All your AI conversations, one graph, one chat.",
+        lifespan=lifespan,
     )
     embedders: list[Embedder] = []
+    allowance = _Allowance(limits) if limits is not None else None
 
     def store() -> GraphStore:
         return open_store(library)
@@ -115,8 +198,8 @@ def embed(text: str, graph: GraphStore) -> list[float] | None:
             raise HTTPException(503, str(error)) from error
 
     @app.get("/health")
-    def health() -> dict[str, str]:
-        return {"status": "ok", "version": __version__}
+    def health() -> dict[str, Any]:
+        return {"status": "ok", "version": __version__, "public": public}
 
     @app.get("/stats")
     def stats() -> dict[str, int]:
@@ -385,7 +468,7 @@ def graph(
             }
 
     @app.post("/chat", response_class=EventSourceResponse)
-    def chat(request: ChatRequest) -> Iterator[ServerSentEvent]:
+    def chat(request: ChatRequest, http: Request) -> Iterator[ServerSentEvent]:
         """Answer a question, streaming `sources`, then `token`s, then `done` or `error`.
 
         `sources` lists the numbered passages the answer may cite. Each `token`
@@ -405,6 +488,11 @@ def chat(request: ChatRequest) -> Iterator[ServerSentEvent]:
         if context.empty:
             yield ServerSentEvent(event="done", data={"cited": [], "found": False})
             return
+        if allowance is not None:
+            refusal = allowance.take(http.client.host if http.client else "unknown")
+            if refusal is not None:
+                yield ServerSentEvent(event="error", data={"message": refusal})
+                return
         try:
             llm = make_llm()
         except LLMError as error:
@@ -423,6 +511,9 @@ def chat(request: ChatRequest) -> Iterator[ServerSentEvent]:
         numbers = [source.number for source in cited("".join(written), context)]
         yield ServerSentEvent(event="done", data={"cited": numbers, "found": True})
 
+    # POST /mcp for assistants, next to the API.
+    app.router.routes.extend(mcp_app.routes)
+
     # The web interface: plain files, no build step. Mounted last, so every API
     # route above takes precedence over a file of the same name.
     app.mount("/", _WebFiles(directory=WEB, html=True), name="web")

src/chatlore/chat.py

@@ -32,6 +32,17 @@
 MAX_ANSWER_TOKENS = 1500
 MAX_NAME_WORDS = 4
 
+
+@dataclass(frozen=True, slots=True)
+class ChatLimits:
+    """How many questions a public server sends to the language model."""
+
+    per_visitor_hour: int = 10
+    """Questions one visitor, told apart by address, may ask in any hour."""
+    per_day: int = 300
+    """Questions all visitors together may ask in a day, counted in UTC."""
+
+
 _WORD = re.compile(r"[\w][\w.+#'-]*")
 _CITATION = re.compile(r"\[(\d+)\]")
 _NOT_NAMES = frozenset(