Provider adapters + telemetry ownership (issue #16, steps 2–3)

Architecture.md

@@ -185,24 +185,27 @@ Thin wrapper over a ChromaDB Collection:
 - **`delete_by_doc_id()`** — removes all chunks for a document via metadata WHERE clause
 - **`get_stats()`** — returns chunk count, backend name, collection name
 
-### `src/llm_handler.py` — LLM Provider Routing
+### `src/llm_handler/` — LLM Provider Routing
 
-Auto-detects provider from model name prefix:
+`LLMHandler` (in `src/llm_handler/__init__.py`) auto-detects the provider from the model-name prefix and selects **one adapter** at construction:
 
-| Prefix | Provider | API Key Env Var |
-|--------|----------|-----------------|
-| `gpt*`, `o1*`, `o3*` | OpenAI | `OPENAI_API_KEY` |
-| `claude*` | Anthropic | `ANTHROPIC_API_KEY` |
-| `glm*` | Zhipu AI (OpenAI-compatible) | `GLM_API_KEY` |
-| everything else | Ollama (localhost:11434) | none |
+| Prefix | Provider | Adapter | API Key Env Var |
+|--------|----------|---------|-----------------|
+| `gpt*`, `o1*`, `o3*` | OpenAI | `OpenAICompatibleAdapter` | `OPENAI_API_KEY` |
+| `claude*` | Anthropic | `AnthropicAdapter` | `ANTHROPIC_API_KEY` |
+| `glm*` | Zhipu AI (OpenAI-compatible) | `OpenAICompatibleAdapter` | `GLM_API_KEY` |
+| everything else | Ollama (localhost:11434) | `OllamaAdapter` | none |
 
-All providers are optional imports with graceful fallback to a dummy response generator.
+Each provider lives behind a `ProviderAdapter` (`src/llm_handler/adapters/`) whose SDK client is **injected** via a zero-arg `client_factory`, so every provider path is unit-testable with a fake (`tests/test_llm_adapters.py`). Adapters return `GenerationResult(text, usage)`; streaming yields text chunks then a terminal `Usage`. Usage is provider-reported where the SDK supplies it, adapter-counted otherwise. See [ADR 0002](docs/adr/0002-provider-adapters.md).
 
-Public API surfaces:
+`LLMHandler` owns provider selection, the single-prompt → messages translation, and the fallback: only `ProviderUnavailableError` (missing SDK, missing GLM key, Ollama connection refused) routes to the `DummyAdapter`; real API errors propagate.
+
+Public API surfaces (unchanged for callers):
 - `generate()` / `stream_response()` — single prompt string
 - `generate_messages()` / `stream_messages()` — OpenAI-style messages list (for multi-turn chat)
+- `generate_with_usage()` — returns `(text, prompt_tokens, completion_tokens)` from provider-reported usage
 
-GPT-5 family and o-series models use `max_completion_tokens` instead of `max_tokens` and omit the `temperature` parameter (constrained to default).
+GPT-5 family and o-series models use `max_completion_tokens` instead of `max_tokens` and omit the `temperature` parameter (constrained to default) — handled inside `OpenAICompatibleAdapter`.
 
 ### `src/database.py` — SQLite/SQLModel
 
@@ -452,7 +455,7 @@ The `src/eval/` package provides a reproducible evaluation system over labeled g
 | Module | Responsibility |
 |--------|----------------|
 | `src/eval/schemas.py` | Pydantic contracts: `EvalQuestion`, `EvalResult`, `AggregatedMetric`, `RunMetadata`, `MetricDelta`, `CompareResult`. |
-| `src/eval/pricing.py` | Hard-coded model price table + `cost_usd()` helper. |
+| `src/telemetry/pricing.py`, `src/telemetry/tokens.py` | **Core** (not eval): model price table + `cost_usd()`, and `count_tokens()`. Imported by both production telemetry and the eval harness (see [ADR 0003](docs/adr/0003-telemetry-ownership.md)). |
 | `src/eval/statistics.py` | `bootstrap_ci()` and `paired_permutation_test()` for run-level confidence intervals and two-run significance testing. |
 | `src/eval/metrics/retrieval.py` | Recall@k, MRR@k, nDCG@k over `(gold_chunk_ids, retrieved_chunk_ids)`. |
 | `src/eval/metrics/operational.py` | Per-stage latency p50/p95/p99, cost, token aggregation. |
@@ -462,7 +465,7 @@ The `src/eval/` package provides a reproducible evaluation system over labeled g
 | `src/eval/datasets/ml_papers.py` | Hand-labeled dev set loader + manifest SHA-256 verification. |
 | `src/eval/config.py` | YAML-loaded `EvalConfig`. |
 | `src/eval/storage.py` | Run-directory CRUD over `eval_runs/<run_id>/`. |
-| `src/eval/pipeline_factory.py` + `src/eval/_telemetry.py` | Builds an isolated RAG pipeline per (config, dataset) using ephemeral Chroma. |
+| `src/eval/pipeline_factory.py` | Builds an isolated RAG pipeline per (config, dataset) using ephemeral Chroma. Token counting and pricing come from `src/telemetry/` (core), not the eval package. |
 | `src/eval/aggregator.py` | Per-dataset + combined `AggregatedMetric` rows from per-question results. |
 | `src/eval/runner.py` | Orchestrates `git_sha`, ingest, query+score loop, aggregation, persistence. |
 | `src/eval/compare.py` | Two-run diff with paired permutation tests + per-question regressions/wins. |
@@ -520,7 +523,7 @@ The system exports per-stage spans for every chat query via OpenTelemetry to [Ar
 
 ### Telemetry Payload
 
-The same numbers are returned to the client as a `StageTelemetry` Pydantic model (`src/api/schemas/telemetry.py`):
+The same numbers are returned to the client as a `StageTelemetry` Pydantic model (`src/api/schemas/telemetry.py`). Token counts are the **provider-reported usage** for the answer pass (from the LLM adapter's `GenerationResult` / streaming terminal `Usage`), with the adapter's local count as fallback — never a reconstructed prompt. Telemetry covers the answer pass only, not the chain-of-thought reasoning pass.
 
 - REST `POST /api/query` — includes a `telemetry` field in the response JSON.
 - WebSocket `/api/chat` — emits a final `{"type": "telemetry", "content": {...}}` event after the existing `done` event.

CONTEXT.md

@@ -0,0 +1,33 @@
+# CONTEXT — domain glossary
+
+The vocabulary of this codebase, so names stay consistent across modules, tests,
+and ADRs. New module names enter here as the architecture-deepening spec
+([issue #16](https://github.com/elkaix/rag-document-qa/issues/16)) lands them.
+
+Design terms use the `/codebase-design` vocabulary: **module** (a unit with an
+interface hiding behaviour), **interface** (the public surface), **depth** (much
+behaviour behind a small interface), **seam** (a boundary you can substitute at),
+**adapter** (a module presenting one interface over another), **leverage**,
+**locality**.
+
+## Modules & types
+
+- **ProviderAdapter** — the interface (Protocol) one LLM provider hides behind:
+  `generate(messages) -> GenerationResult` and `stream(messages)` yielding text
+  chunks then a terminal `Usage`. Implementations: `OpenAICompatibleAdapter`
+  (OpenAI + GLM), `AnthropicAdapter`, `OllamaAdapter`, `DummyAdapter`. Each is
+  constructed with an injected `client_factory` so it is testable with a fake.
+  See [ADR 0002](docs/adr/0002-provider-adapters.md).
+- **Usage** — a value object of `(prompt_tokens, completion_tokens)` for one
+  generation call. Provider-reported where the SDK returns it, adapter-counted
+  otherwise.
+- **GenerationResult** — the `(text, usage)` pair returned by a non-streaming
+  generation, so callers never rebuild a prompt to estimate what was billed.
+- **LLMHandler** — owns provider *selection* (from the model-name prefix), the
+  single-prompt → messages translation, and the unconfigured-provider fallback
+  to `DummyAdapter`. Delegates all SDK work to one selected `ProviderAdapter`.
+- **telemetry** (`src/telemetry/`) — core token counting (`count_tokens`) and
+  cost pricing (`cost_usd`, `MODEL_PRICES`). Owned by the core so both production
+  telemetry assembly and the eval harness use one source of truth; the eval
+  package imports from here, never the reverse. See
+  [ADR 0003](docs/adr/0003-telemetry-ownership.md).

docs/adr/0002-provider-adapters.md

@@ -0,0 +1,63 @@
+# ADR 0002 — LLM provider adapters with injected clients
+
+- **Status:** Accepted
+- **Sequencing:** Step 2 of the RAG architecture deepening spec ([issue #16](https://github.com/elkaix/rag-document-qa/issues/16)); resolves [#10](https://github.com/elkaix/rag-document-qa/issues/10).
+- **Date:** 2026-07-12
+
+> ADR numbering follows issue #16's sequencing steps. Step 1 (deletions +
+> vector-store lookup, PR #17) landed without a written ADR; this is the first.
+
+## Context
+
+`src/llm_handler.py` was an 812-line class that repeated the same three-branch
+provider dispatch **four times** (`generate`, `stream_response`,
+`generate_messages`, `stream_messages`) and built every provider client from
+module globals. Two consequences:
+
+1. **Untestable providers.** The only way to fake a provider was to monkeypatch a
+   module global (`openai.OpenAI`). That worked for OpenAI in CI but left
+   **Anthropic and Ollama with zero coverage** — there was no seam to inject a
+   fake at.
+2. **No usage at the source.** Generation returned only `str`, so callers that
+   needed token counts (cost telemetry) rebuilt the prompt and re-tokenized it —
+   counting a *reconstruction*, not what the provider actually billed.
+
+## Decision
+
+Introduce a **`ProviderAdapter` Protocol** (`generate` -> `GenerationResult`,
+`stream` -> text chunks then a terminal `Usage`) with one adapter per provider:
+
+- **`OpenAICompatibleAdapter`** serves both OpenAI and GLM. The token-parameter
+  and temperature quirks live here; GLM differs only by the injected client's
+  base URL, so it is not a second code path.
+- **`AnthropicAdapter`** owns the system-message split.
+- **`OllamaAdapter`** owns the `/api/chat` payload shape.
+- **`DummyAdapter`** is the always-available fallback.
+
+**Clients are injected** via a zero-arg `client_factory` per adapter, so every
+provider is unit-testable with a fake (see `tests/test_llm_adapters.py`) — no
+module-global monkeypatching. `LLMHandler` selects one adapter at construction;
+the four dispatch tables collapse to that single selection. Its public interface
+is unchanged; prompt variants translate to the messages form the adapters speak.
+
+**Usage is provider-reported where available, adapter-counted as fallback**
+(`base.counted_usage`). Streaming reports usage in its terminal `Usage` event.
+
+## Consequences
+
+- Anthropic and Ollama gain their first real unit coverage; OpenAI/GLM quirks are
+  asserted directly against a fake client.
+- **Preserved behavior (deliberately):** only `ProviderUnavailableError` (missing
+  SDK / missing GLM key / Ollama connection refused) triggers the dummy fallback;
+  real API errors (auth, quota, malformed) still propagate. A missing *OpenAI*
+  key is left to the SDK to reject, exactly as before — the asymmetry with GLM's
+  explicit pre-check is intentional and preserved.
+- **Consolidation:** the single-prompt Ollama `/api/generate` endpoint is gone —
+  every call is translated to a messages list first, so `/api/chat` covers both.
+  Semantically equivalent; the wire format differs.
+- In this step, `LLMHandler`'s public `stream_*` methods still yield text only
+  (they drop the adapter's terminal `Usage`), keeping the backend untouched.
+  Step 3 surfaces that usage to the telemetry layer.
+- The injected-client type is intentionally loose (`object`) at the SDK seam —
+  the adapters call methods on external SDK objects we do not own; no `Any` and
+  no `# type: ignore` are used.

docs/adr/0003-telemetry-ownership.md

@@ -0,0 +1,57 @@
+# ADR 0003 — Telemetry ownership and usage-driven cost
+
+- **Status:** Accepted
+- **Sequencing:** Step 3 of the RAG architecture deepening spec ([issue #16](https://github.com/elkaix/rag-document-qa/issues/16)); resolves [#12](https://github.com/elkaix/rag-document-qa/issues/12).
+- **Date:** 2026-07-12
+
+## Context
+
+Two coupled problems in the cost-telemetry path:
+
+1. **Inverted dependency.** Token counting (`count_tokens`) and pricing
+   (`cost_usd`, `MODEL_PRICES`) lived in `src/eval/` (`_telemetry.py`,
+   `pricing.py`), yet **production** imported them — `src/backend.py` and
+   `src/llm_handler` reached into the eval package for core utilities.
+2. **Cost computed from a reconstruction.** `RAGBackend` did not have the tokens
+   the provider billed, so it **rebuilt the prompt string** (a third copy of the
+   system + user prompt) purely to re-tokenize it with `count_tokens`. The number
+   shown to users was an estimate of a *copy*, in four duplicated assembly sites
+   (sync query, streaming conversation branch, streaming single-turn branch, and
+   `LLMHandler.generate_with_usage`).
+
+## Decision
+
+**Move token/pricing utilities to a core `src/telemetry/` package** (`tokens.py`,
+`pricing.py`). The eval package now imports from core (`from src.telemetry ...`),
+never the reverse; `src/eval/__init__.py` re-exports the pricing symbols so its
+public API is unchanged. The old `src/eval/_telemetry.py` and `src/eval/pricing.py`
+are deleted (moved with `git mv`, not shimmed).
+
+**Consume provider-reported usage** (from ADR 0002's adapters):
+
+- The **synchronous** path calls `LLMHandler.generate_with_usage(...)`, which
+  returns the answer plus real `(prompt_tokens, completion_tokens)`. The
+  reconstructed prompt and its `count_tokens` calls are gone; the now-orphaned
+  `generate_with_context` is deleted.
+- The **streaming** path reads the `Usage` carried on the stream's terminal event
+  (`LLMHandler.stream_response` / `stream_messages` now yield `str | Usage`). The
+  backend discriminates with `isinstance(item, Usage)`.
+
+`cost_usd` is still computed in the backend from those counts.
+
+## Consequences
+
+- Production no longer imports from the eval package — a clean dependency
+  direction (verified: `grep` finds no `src.eval` import in backend/llm_handler).
+- The cost footer reflects what the provider actually billed (or the adapter's
+  local count when a provider omits usage), not a re-tokenized copy of a
+  reconstructed prompt.
+- **Behavior preserved (deliberately):** telemetry still covers the **answer pass
+  only** — the streaming reasoning pass's terminal `Usage` is discarded, keeping
+  the per-query cost display to one model's spend, exactly as before. The
+  no-documents early-return still emits zero telemetry. `StageTelemetry`'s shape
+  is unchanged, so the API and frontend are untouched. `test_backend_telemetry.py`
+  passes unchanged as the behavior-preservation proof.
+- The four duplicated token-assembly sites collapse to two usage reads (one per
+  path); further consolidation of the answer prompt itself is step 4's
+  QueryEngine work, not this step.