Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .claude/rules/file-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,12 @@ Full module index for LocalChat. **Keep this current** β€” update in the same co
| `src/tools/executor.py` | Ollama tool-call loop (multi-turn until final response) |
| `src/tools/registry.py` | Tool registration with JSON schemas |
| `src/tools/builtin.py` | Built-in tools: document search, calculator, datetime |
| `src/tools/plugin_loader.py` | Loads `.py` plugins from `plugins/` at startup |
| `plugins/example_plugin.py` | Reference plugin demo (`word_count`, `reverse_text` tools) |
| `src/tools/plugin_loader.py` | Loads `.py` plugins from `plugins/` at startup; one with `setup(services)` gets its manifest validated and its services built, one without loads as legacy. A failing plugin is disabled alone, with any tool it registered removed |
| `src/plugins/manifest.py` | PC-1 β€” `read_manifest()`: `PLUGIN_SCOPE`, `PLUGIN_MIN_ROLE`, `PLUGIN_HOOKS` (checked against the catalogue), `PLUGIN_CONFIG`; `PLUGIN_CONTRIBUTES` must be False. Raises `ManifestError` |
| `src/plugins/services.py` | PC-1 β€” the handles a plugin's `setup()` receives: `retrieval` (the request's workspace only, tier capped by the manifest), `llm.embed`, `config`, `identity` (read only), `tools.register` (refuses a taken name) |
| `src/plugins/__init__.py` | `build_services()` β€” types the plugin's settings from `PLUGIN_<PLUGIN>_<KEY>` and assembles its handles |
| `plugins/example_plugin.py` | Example plugin on the PC-1 contract β€” `setup(services)`, a manifest and a setting (`word_count`, `reverse_text` tools) |
| `tests/unit/test_plugin_contract.py` | PC-1 β€” the manifest (every malformed field refused by name), each handle (retrieval reads the request's workspace and refuses a tier beyond its reach or a key's), the executor hiding and refusing a tool below its role, the loader disabling one plugin alone and removing its half-registered tools, and the guard recording who it admitted |
| `plugins/README.md` | Plugin authoring guide |
| **Graph / Memory / Performance** | |
| `src/graph/store.py` | `GraphStore` ABC + `PostgresGraphStore`, which reaches the entities through `db.entities` |
Expand Down Expand Up @@ -242,6 +246,7 @@ Full module index for LocalChat. **Keep this current** β€” update in the same co
| `tests/unit/test_injection_flags.py` | GR-1c β€” each kind is found by the text it names and stays quiet on the near-misses business documents carry (`System: 2`, "you are now logged in") |
| `tests/unit/test_frontend_injection_flag_badge.py` | GR-1c β€” the Documents page badges a flagged document, names the kinds, and leaves an unflagged or pre-scan one alone |
| `tests/integration/test_injection_fixtures.py` | GR-1's acceptance β€” eleven adversarial documents (text, Markdown, Word paragraph and table cell, spreadsheet cells, slide, white-on-white PDF) ingested for real: each flagged with its kind, a control not, the instruction kept inside its fence, nothing reaching memory extraction; PPTX speaker notes asserted not ingested |
| `src/utils/caller.py` | `RequestCaller` and `ROLE_LEVELS` (viewer < editor < owner < admin) β€” who the request was authorised as, bound for the chat stream like the scope, so the tool executor can check a tool's minimum role and a plugin's `identity` handle can read it (PC-1) |
| `src/utils/text.py` | `STOP_WORDS` and `content_terms()` β€” a question's topic words, any script, operator-free; what the lexical arm searches for, and the stop-word set active learning shares |
| **Infra / Config** | |
| `requirements.in` | Runtime dependencies, hand-written β€” the input `requirements.txt` is compiled from |
Expand Down
38 changes: 17 additions & 21 deletions .claude/rules/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,20 @@
How plugins extend LocalChat without destabilising the core. Read alongside the
"Plugin Contract" section in `CLAUDE.md` β€” that states the principle; this is the rule.

> ## ⚠ Status: designed, not built (verified 2026-08-27, re-checked 2026-10-06)
>
> **Everything below describes PC-1..PC-3, which have not been implemented.** They are
> ROADMAP Sprints 10–11, which now run after the quality plan (Initiative 12) has built the
> service layer they wrap. PC-4, the `core-without-plugins` job, was built on 2026-10-23
> (QP-3). Apart from it, none of the following exists in the repository today:
> ## ⚠ Status: partly built (PC-1 built 2026-11-11; PC-4 2026-10-23)
>
> | Described here | Reality |
> |---|---|
> | The services catalogue (`retrieval`, `llm`, `storage`, `config`, `identity`) | No service handles; a plugin imports `src.tools.registry` directly |
> | The hooks catalogue (`on_document_ingested`, `on_scheduler_tick`, …) | No hook bus |
> | The manifest (`PLUGIN_SCOPE`, `PLUGIN_MIN_ROLE`, `PLUGIN_HOOKS`, `PLUGIN_CONFIG`) | `plugin_loader.py` reads none of these; it reads an optional `PLUGIN_META` dict for display only |
> | `plugins/_echo/` reference plugin | Absent. `plugins/` holds `example_plugin.py` |
> | The manifest (`PLUGIN_SCOPE`, `PLUGIN_MIN_ROLE`, `PLUGIN_HOOKS`, `PLUGIN_CONFIG`) | **Built** (PC-1) in `src/plugins/manifest.py`; a malformed one disables that plugin only |
> | The services catalogue | **Built** (PC-1) in `src/plugins/services.py`: `retrieval`, `llm` (`embed` only), `config`, `identity`, and `tools` for registering. `storage` and `llm.complete` are deferred until a consumer needs them |
> | `PLUGIN_MIN_ROLE` | **Enforced by the core** (PC-1): the tool executor hides and refuses a tool below its minimum role |
> | The hooks catalogue (`on_document_ingested`, `on_scheduler_tick`, …) | No hook bus (PC-2). `PLUGIN_HOOKS` is validated against the names below and fires nothing |
> | `plugins/_echo/` reference plugin | Absent (PC-3). `plugins/` holds `example_plugin.py`, on the PC-1 contract |
> | The `core-without-plugins` CI job | **Built** (2026-10-23) in `.github/workflows/tests.yml`; a required check since 2026-10-30 |
>
> **What actually ships** is a much smaller thing, and it is documented accurately in
> [`plugins/README.md`](../../plugins/README.md): drop a `.py` file in `plugins/`, decorate
> functions with `@tool_registry.register`, and they become LLM-callable tools. That is the
> live contract. This file is the target.
> A plugin without `setup(services)` still loads as a *legacy* plugin that imports
> `src.tools.registry` directly. [`plugins/README.md`](../../plugins/README.md) documents
> what ships; the sections below are the target, and say where PC-1 departed from it.
>
> The document is kept in present tense below because it is a specification, and rewriting a
> spec into the conditional makes it harder to build against. Read it as "shall", not "does".
Expand Down Expand Up @@ -53,11 +48,12 @@ with a stable method surface and never imports the core module behind it.

| Service | Surface the plugin gets | Behind it (plugin never touches) |
|---------|------------------------|----------------------------------|
| `retrieval` | `retrieve(query, scope, workspace_id)` -> ranked chunks. `scope` is `local`/`global`/`hybrid`. | `src/rag/retrieval.py`, pgvector, reranker |
| `llm` | `complete(prompt, ...)`, `embed(text)` | `src/ollama_client.py`, model routing, warm-up |
| `storage` | A namespaced handle for the plugin's own tables; optional Clark-Wilson soft-delete helpers | `src/db/connection.py` pool, psycopg3 |
| `config` | Validated access to the plugin's declared keys + defaults | `src/config.py`, `.env` loading |
| `identity` | `require_role(min_role)` -> ok / 403 | `src/security_fastapi.py`, JWT claims |
| `retrieval` | `retrieve(query, knowledge_scope, top_k)` -> ranked chunks. `knowledge_scope` is `local`/`global`/`hybrid`, capped by `PLUGIN_SCOPE`; a workspace API key gets `local` only. **No workspace argument:** it searches the workspace the request was authorised for (decided 2026-11-11; the earlier `workspace_id` parameter would have let a plugin choose its target) | `src/rag/retrieval.py`, pgvector, reranker |
| `llm` | `embed(text)`. `complete(prompt, ...)` is deferred: tools run synchronously inside the chat stream, so a completion from one needs that changed first | `src/ollama_client.py`, model routing, warm-up |
| `storage` | *Deferred until a consumer needs it (2026-11-11).* A namespaced handle for the plugin's own tables; optional Clark-Wilson soft-delete helpers. Open: who migrates those tables, and whether RLS applies | `src/db/connection.py` pool, psycopg3 |
| `config` | Validated access to the plugin's declared keys + defaults, from `PLUGIN_<PLUGIN>_<KEY>` | `src/config.py` (`plugin_setting`), `.env` loading |
| `identity` | `current()` -> the caller, `has_role(min_role)` -> bool. Reads only: `PLUGIN_MIN_ROLE` is enforced by the core before a tool runs | `src/utils/caller.py`, bound from the workspace guard |
| `tools` | `register(name, description, parameters)`, tagged with the plugin and its minimum role; a taken name is refused | `src/tools/registry.py` |

**Adding a service.** Only when a real consumer needs it, and only with a name that
describes the capability generically. A `pricing_lookup` service is a leak; a
Expand All @@ -84,8 +80,8 @@ Every plugin declares a manifest the core reads at load time. Minimal shape:

```python
PLUGIN_SCOPE = "hybrid" # local | global | hybrid (retrieval reach)
PLUGIN_MIN_ROLE = "viewer" # viewer | user | admin (who may invoke)
PLUGIN_CONTRIBUTES = False # may it write to the Global Knowledge Base?
PLUGIN_MIN_ROLE = "viewer" # viewer | editor | owner | admin (who may invoke)
PLUGIN_CONTRIBUTES = False # may it write to the GKB? Must be False: no service writes
PLUGIN_HOOKS = ["on_document_ingested", "on_scheduler_tick", "on_tool_invocation"]
PLUGIN_CONFIG = { # keys the core loads into the config service
"FEEDBACK_THRESHOLD": ("float", 0.30),
Expand Down
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,25 @@ reasoning attached, in [docs/history/LESSONS_LEARNED.md](docs/history/LESSONS_LE

## [Unreleased]

### Added

- **Plugins get a manifest and services (PC-1).** A plugin defines `setup(services)` and
registers its tools through `services.tools.register`, importing nothing from `src`. Its
manifest declares how far its retrieval reaches (`PLUGIN_SCOPE`), who may use it
(`PLUGIN_MIN_ROLE`) and its settings (`PLUGIN_CONFIG`, read from
`PLUGIN_<PLUGIN>_<KEY>`). `services.retrieval` searches only the workspace the request was
authorised for. The core enforces the minimum role: a tool is not offered to a caller below
it, and is refused if the model names it anyway. A malformed manifest disables that plugin
alone. **Upgrading:** nothing to do. A plugin without `setup()` still loads as before, now
listed as legacy in `/api/plugins` with a warning in the log.

### Fixed

- **A failing plugin no longer leaves tools behind, and a reloaded one can be unloaded.** A
plugin that raised after registering a tool kept that tool. A reload recorded none of the
plugin's tools, so unloading it afterwards left them registered. And a loader given an
empty registry used the global one instead, since an empty registry is falsy.

## [3.6.0] β€” 2026-11-11

The release of the QP-9 review's open findings (QP-10): R-8 to R-12 and item 13 are fixed,
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ LocalChat supports plugins (LLM tools, connectors, background work) that extend

The enforcing procedure is a CI job, `core-without-plugins`, that verifies architectural integrity: the core builds, lints, and passes its full test suite **with the `plugins/` directory and all private plugin code absent.** If that job goes red, a plugin dependency has leaked into the core.

> **It exists since 2026-10-23** (PC-4, built with QP-3): it deletes everything in `plugins/` but its README, then runs `ruff` and the fast suite. It has been a required check since 2026-10-30 (see Pull Requests). The rest of the contract β€” the services catalogue, the hook bus, the manifest, the `_echo` reference plugin β€” is likewise designed and unbuilt; what actually ships is the simpler tool-plugin loader described in [`plugins/README.md`](plugins/README.md). See the status banner in [.claude/rules/plugins.md](.claude/rules/plugins.md).
> **It exists since 2026-10-23** (PC-4, built with QP-3): it deletes everything in `plugins/` but its README, then runs `ruff` and the fast suite. It has been a required check since 2026-10-30 (see Pull Requests). The manifest and the services catalogue were built on 2026-11-11 (PC-1, `src/plugins/`), with `PLUGIN_MIN_ROLE` enforced by the core; `storage` and `llm.complete` wait for a consumer. The hook bus (PC-2) and the `_echo` reference plugin (PC-3) are designed and unbuilt. What ships is described in [`plugins/README.md`](plugins/README.md). See the status banner in [.claude/rules/plugins.md](.claude/rules/plugins.md).

Full rules -> [.claude/rules/plugins.md](.claude/rules/plugins.md)

Expand Down
1 change: 1 addition & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -463,6 +463,7 @@ to empty, in which case the router falls back to the active model.
> fixes this product at one node and one process; scale with a second node instead.
| `LOG_FILE` | `logs/app.log` | Path for the file sink |
| `PLUGINS_DIR` | `plugins` | Directory scanned for plugins |
| `PLUGIN_<PLUGIN>_<KEY>` | the plugin's declared default | A setting a plugin declares in `PLUGIN_CONFIG`. `<PLUGIN>` is its file name upper-cased, with anything but letters and digits made `_`. Read by `plugin_setting()` in `config.py`; see [plugins/README.md](../plugins/README.md) |
| `PRESENCE_TTL_SECONDS` | `30` | How long a workspace presence entry stays live |
| `CONNECTOR_LOCAL_ROOTS` | *(empty)* | Comma-separated absolute directories the `local_folder` connector may watch. **Empty disables that connector type** β€” see below |

Expand Down
22 changes: 20 additions & 2 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -594,7 +594,25 @@ The mechanism that lets plugins extend LocalChat without the core ever depending

---

### PC-1 β€” Service catalogue and manifest loader
### PC-1 β€” Service catalogue and manifest loader βœ… (built 2026-11-11)

> **As built.** Four of the spec's points were decided differently by the maintainer on
> 2026-11-11, and `.claude/rules/plugins.md` was amended to match:
> - `retrieval` takes no `workspace_id`. It searches the workspace the request was
> authorised for, so a plugin cannot aim a search at one its caller was never admitted to.
> The tier is capped by `PLUGIN_SCOPE`, and a workspace API key gets `local` only.
> - `PLUGIN_MIN_ROLE` is enforced by the core, not by the plugin: the tool executor hides a
> tool from the model below its minimum role and refuses it if named anyway. The roles are
> the real ones (viewer < editor < owner, then admin); the workspace guard records the
> caller's, and the chat route binds it for the stream. `identity` only reads.
> `require_workspace_role_dep`, which the spec named, was deleted by QP-1.
> - `storage` is deferred until a consumer needs it, and so is `llm.complete`: tools run
> synchronously inside the chat stream, so a completion from one needs that changed first.
> - A plugin without `setup(services)` still loads, as *legacy*, so the release stays a minor.
>
> The built-in tools stay on the core directly: they are core code, and the contract governs
> plugins. Building it found two loader bugs, fixed here: an empty registry was swapped for
> the global one, and a reload recorded no tools, so a later unload left them registered.

**Current foundation (do not rebuild from scratch).** The following already exists and works:
- `src/tools/registry.py` β€” `ToolRegistry` with `@register` decorator, JSON-schema export, execute-by-name, `unregister` for hot-reload.
Expand Down Expand Up @@ -2038,7 +2056,7 @@ the generation path is a deterministic pipeline unless an answer-level eval says
| PG-0..PG-8 | **Production-grade hardening** βœ… β€” see [PRODUCTION_PLAN.md](history/PRODUCTION_PLAN.md). Gated everything below; **gate lifted 2026-08-31**. | done |
| 8 | GKB-1 (schema + two-tier retrieval) βœ… 2026-10-09 β€” read-only global tier under RLS (0019) | 1 week |
| 9 | GKB-2 (contribution workflow) βœ“ backend 2026-10-19 (admin-only contribute, migration 0020), UI 2026-10-20 | 1 week |
| 10 | PC-1 + PC-2 (services, hooks, scheduler) | 1 week |
| 10 | PC-1 βœ… 2026-11-11 (manifest, services, the core's role gate) + PC-2 (hooks, scheduler) | 1 week |
| 11 | PC-3 + PC-4 (echo plugin, CI gate) | 1 week |
| 12 | PR-1 (pricing plugin β€” private repo) | 1–2 weeks |
| 13 | CONN-1 (connector authorisation model β€” decision, no code) | 2–3 days |
Expand Down
Loading
Loading