Feature/epic/q

CHANGELOG.md

@@ -5,6 +5,53 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- **`lpgcast-bench compare A.json B.json`**: run B read against run A episode by episode, with
+  pooled rate deltas, paired bootstrap intervals, score rises and falls, and status changes.
+  `lpgcast-bench` alone still runs the benchmark.
+- **`LLMTruncatedError` and `LLMRefusalError`**: a reply cut off at the output limit or refused
+  is raised as such, not re-asked; `UsageTracker` counts both.
+- **`EpisodeResult.repair_failure`**: why a repair request failed.
+- **`value_not_in_quote` finding**: a number written as digits, or a date, must be what its quote
+  states; typed repair as for other findings. The benchmark reports it as *Unfaithful*.
+- Repair guidance for a unit other than the declared one: leave the property out, never convert.
+- **Identity values ground in context**: a key of an entity the text refers to may be quoted from
+  `Episode.context`; its grounding carries `in_context` (the context item's source id) and the
+  `grounded_in_context` fix is recorded. `manifest.json` gains `output_format_version` (2).
+- A concept page for `Episode.context`.
+- **`lpgcast serve`** (the `server` extra: FastAPI, uvicorn): a stateless `POST /cast` taking a
+  schema inline, episodes and options, returning graph, groundings, pending edges, quarantine,
+  episode results and the manifest; fail-closed bearer key (`LPGCAST_SERVE_API_KEY`), a batch cap,
+  `GET /health`. Guide: *Running as a service*.
+- **`schema_from_mapping`**: a schema, or a manifest's schema, from a mapping.
+- **Benchmark corpus 1.1.0**: `selection` (wide schemas, inflected type names) and `context` (a key
+  quoted from an earlier text) families; an expected property may name its context item. Each case
+  carries a content digest, and `compare` pairs runs on different corpus versions over unchanged
+  cases (`--pair-by-name` for reports from before digests).
+
+### Changed
+
+- A repair that returns the same findings is not repeated.
+- Selection stems `-es`, `-ies`, `-ed` and `-ing` beside the plural `-s`, so `shipped` meets
+  `ship` and `routing` meets `route`.
+- `write_cast` is a thin wrapper over graflo's `write_container`, and `WriteReport` is graflo's
+  `ContainerWriteReport`; requires graflo ≥ 1.18.0, whose `subschema()` keeps the edges between
+  seeds under a tight budget.
+- The prompt asks for plain numbers and ISO 8601 dates, with the quote in the text's own form, so
+  `1,250` or `4 March 2024` no longer cost a repair. Cached replies from earlier versions miss.
+
+### Fixed
+
+- A provider fault (rate limit, server error, a second timeout, context length) fails only its
+  episode, quarantined as `provider_error`, `truncated` or `refused`; the run goes on.
+- A fault during a repair keeps the best earlier reply instead of failing the episode.
+- `quarantine.jsonl` is written in episode order, so a replay reproduces it byte for byte.
+- Quotes match across typographic quotes and dashes (`“…”`, `’`, `–`, `−`).
+- A property's quote takes the occurrence nearest its vertex's or edge's quote, not the first.
+
 ## [0.1.1] - 2026-10-11
 
 ### Added

docs/_build/gen_cli.py

@@ -57,7 +57,10 @@ def _section(command: click.Command, invocation: str) -> list[str]:
 assert isinstance(lpgcast, click.Group)
 for name, command in lpgcast.commands.items():
     lines += _section(command, f"lpgcast {name}")
-lines += _section(bench, "lpgcast-bench")
+assert isinstance(bench, click.Group)
+for name, command in bench.commands.items():
+    invocation = "lpgcast-bench" if name == "run" else f"lpgcast-bench {name}"
+    lines += _section(command, invocation)
 
 with mkdocs_gen_files.open("reference/cli.md", "w") as f:
     f.write("\n".join(lines))

docs/_build/gen_settings.py

@@ -10,12 +10,13 @@
 import mkdocs_gen_files
 from pydantic_settings import BaseSettings
 
-from lpgcast.config import CacheSettings, LLMSettings, LPGCastSettings
+from lpgcast.config import CacheSettings, LLMSettings, LPGCastSettings, ServeSettings
 
 SECTIONS: list[tuple[str, type[BaseSettings], str]] = [
     ("Model", LLMSettings, "LPGCAST_LLM_"),
     ("Response cache", CacheSettings, "LPGCAST_CACHE_"),
     ("Cast", LPGCastSettings, "LPGCAST_"),
+    ("Service", ServeSettings, "LPGCAST_SERVE_"),
 ]
 
 

docs/concepts/context.md

@@ -0,0 +1,22 @@
+# Context
+
+An episode is one short text. Its `context` holds earlier texts of the same
+thread, so the model can tell what the text refers to:
+
+```json
+{"source_id": "m2", "text": "It is leaking again.",
+ "context": [{"source_id": "m1", "text": "Press S-7 was serviced on Monday."}]}
+```
+
+The model reads the context; the cast records only what the episode's own text
+states. "It" in `m2` can become a mention of press `S-7`, and an edge from
+`m2`'s note to it, but nothing `m1` alone says is written.
+
+The one thing context may supply is a key: the identity value (`id:` or
+`alt id:`) of an entity the text refers to. Its grounding then has `in_context`
+set to the context item's `source_id`, and its offsets point into that item's
+text. Any other quote found only in context is a `quote_not_found` finding.
+
+Context is shown to the model as data, like the text, and counts towards
+`LPGCAST_MAX_EPISODE_TOKENS`. LPGCast does not assemble it: the caller chooses
+the earlier texts, and should keep them few and recent.

docs/concepts/groundings.md

@@ -14,18 +14,20 @@ graph, not into it.
 | `edge`, `source_identity`, `target_identity` | for an edge or its property |
 | `property` | for a property: its name |
 | `char_start`, `char_end`, `quote` | the span of the text, and its characters |
+| `in_context` | set when the span is in a [context](context.md) item: its `source_id`; offsets then point into that item's text |
 | `extractor`, `model`, `schema_ref` | what produced it: the LPGCast version, the model, and `name@version` of the schema |
 | `reference_time` | the episode's time |
 
 The model gives quotes, never offsets; LPGCast finds each quote in the text,
-exactly first, then ignoring case, accents and runs of whitespace. A quote it
-cannot find is a `quote_not_found` finding, so every grounding points at text
-that is there.
+exactly first, then ignoring case, accents, runs of whitespace and typographic
+quotes and dashes. When a quote occurs more than once, a property takes the
+occurrence nearest its vertex's or edge's quote. A quote it cannot find is a
+`quote_not_found` finding, so every grounding points at text that is there.
 
 ## Quarantine
 
-Nothing is dropped silently. `quarantine.jsonl` holds one record per item left
-out, per failed episode, and per `unmapped` entry:
+`quarantine.jsonl` holds one record per item left out, per failed episode, and
+per `unmapped` entry, in episode order:
 
 | `record` | Means |
 |---|---|

docs/concepts/index.md

@@ -34,14 +34,22 @@ chunking, no encoder and no vector store. An episode longer than
 | `undeclared_property` | a property its type does not declare |
 | `duplicate_property` | a property given twice |
 | `coercion` | a value its field type refuses, or a unit other than the declared one |
+| `value_not_in_quote` | a number written as digits, or a date, that its quote does not state (`37.5` on "3.75 kW", `2024-03-01` on "March 2024"); spelled-out and relative values are not checked |
 | `quote_not_found` | an item whose quote is not in the text |
 | `endpoint_invalid` | an edge whose endpoint was removed; never shown to the model, it goes when the endpoint is fixed |
 
+A repair that returns the same findings is not repeated: the slice does not let
+the model fix them. A repair request that fails keeps the best earlier reply and
+records why in the episode's `repair_failure`.
+
 A failure of the whole episode is recorded with its reason: the selection found
-nothing (`selection_*`), the text is over the ceiling (`too_long`), the reply was
-not JSON (`unparseable`) or not the reply shape (`wire_shape`), or nothing in it
-survived (`nothing_valid`). A model that is misconfigured, or a cache miss while
-replaying, stops the run instead: every other episode would fail the same way.
+nothing (`selection_*`), the text is over the ceiling (`too_long`), the provider
+rejected or dropped the request (`provider_error`), the reply stopped at the
+output limit (`truncated`), the model refused (`refused`), the reply was not
+JSON (`unparseable`) or not the reply shape (`wire_shape`), or nothing in it
+survived (`nothing_valid`). A truncated or refused reply is not re-asked. A model
+that is misconfigured, or a cache miss while replaying, stops the run instead:
+every other episode would fail the same way.
 
 ## Beside OntoCast
 

docs/guides/benchmark.md

@@ -8,6 +8,7 @@ lpgcast-bench                              # the oracle row only; free
 lpgcast-bench --model gpt-5.6-luna         # plus a model row (LPGCAST_LLM_* settings)
 lpgcast-bench --split test --model a --model b
 lpgcast-bench --model a --replay           # from the cache; makes no request
+lpgcast-bench compare a.json b.json        # read run b against run a
 ```
 
 Each run writes `<stamp>.json` (counts per episode, rates per case and row) and `<stamp>.md`
@@ -31,9 +32,12 @@ domains:
 | `recast` | a second cast with a fresh model call |
 | `injection` | instructions inside the text |
 | `distractors` | a wide schema sliced under a tight budget, with types the text names but says nothing about |
+| `selection` | a wide schema sliced under a budget, its types named in inflected forms (`parcels`, `shipped`) and related to each other |
+| `context` | a key stated only in an earlier text of the thread (`Episode.context`); everything else in the text itself |
 
 `expected.yaml` maps each episode to `vertices` (keyed by a handle), `edges` between handles, and
-`forbidden` items a cast must not write. Every quote occurs in the text and every value is what
+`forbidden` items a cast must not write. Every quote occurs in the text (a property may instead
+name the `context` item its quote is in, for an identity value only) and every value is what
 its field makes of it; a test checks both, and that the oracle scores perfectly on the whole
 schema.
 
@@ -58,6 +62,7 @@ Every metric is a ratio of counts pooled over episodes.
 | Conformance | items of the first reply with no schema finding, over items emitted; *final* is the attempt kept |
 | Type / relation / property hallucination | `unknown_type`, `undeclared_edge`, `undeclared_property` findings on the first reply, over vertices, edges, properties emitted |
 | Ungrounded | first-reply quotes not found in the text, over items emitted |
+| Unfaithful | first-reply numbers or dates their quote does not state (`value_not_in_quote`), over properties emitted |
 | Entity, edge, property P/R/F1 | matched over produced and over expected; properties on matched entities and edges only |
 | Direction | edges matched, over edges matched or found the wrong way round |
 | Span | matched items whose quote overlaps the expected one |
@@ -74,6 +79,19 @@ Entities match by identity: a natural type's key, a hash type's digest, or any c
 identity. An assigned or blank type matches by equal property values or an overlapping quote.
 Values compare after coercion by their field; strings ignore case and runs of whitespace.
 
+## Comparing two runs
+
+`lpgcast-bench compare A.json B.json` reads run B against run A. Episodes pair on case and source
+id, and only paired episodes count; the rest are listed. For each rate it gives A, B and B − A
+over the pooled counts, with a 95% paired bootstrap interval: episodes are resampled together
+with their pair, so an interval that spans zero says the corpus cannot tell the two runs apart.
+Per episode it counts score rises and falls, tabulates status changes and lists the largest
+drops and rises. Across corpus versions, only cases whose content is unchanged are paired (each
+case carries a digest); with none in common the comparison is refused. A report from before case
+digests (corpus 1.0.0) pairs with a later one only under `--pair-by-name`, which asserts that its
+cases did not change. `--row-a` and `--row-b` pick a row
+when a report holds more than one model; `--json` also writes the comparison as JSON.
+
 ## Adding a case
 
 In a clone: write the three files under a new instance directory, add it to

docs/guides/replay.md

@@ -18,7 +18,8 @@ quarantine, pending edges and episode results are byte-identical to the first
 run; only `manifest.json` differs, since it reports usage.
 
 A miss means the request changed: the prompt, the schema slice or a model
-setting.
+setting. An episode quarantined as `provider_error` has no cached reply, so its
+replay misses too; cast it again without `--replay` to fill the cache.
 
 ## Settings
 

docs/guides/schema.md

@@ -41,7 +41,9 @@ Every property should have a `type`. Values are coerced to it before writing:
 
 A numeric field with `semantics.unit` (`unit: kW`) accepts a value written with
 that unit and strips it. A value in any other unit is refused rather than
-converted. The sheet shows the unit, so the model knows what to expect.
+converted, and units are case-sensitive (`mW` is not `MW`). The sheet shows the
+unit, so the model knows what to expect. The model writes numbers plain
+(`1250`) and dates as ISO 8601; the quote keeps the text's own form.
 
 ## Large schemas
 

docs/guides/serve.md

@@ -0,0 +1,55 @@
+# Running as a service
+
+`lpgcast serve` casts over HTTP. It suits a platform that holds the schema and
+writes the graph itself: the service receives the schema with the episodes,
+casts them, and returns the result. It never writes to a database and never
+calls back, so it holds no database credentials.
+
+```bash
+pip install 'lpgcast[server]'
+export LPGCAST_SERVE_API_KEY=…            # required; see below
+export LPGCAST_LLM_MODEL=… LPGCAST_LLM_API_KEY=…
+lpgcast serve --port 8997
+```
+
+## `POST /cast`
+
+```json
+{
+  "schema": {"metadata": {…}, "core_schema": {…}},
+  "episodes": [{"source_id": "m1", "text": "Ann operates press S-7."}],
+  "options": {"user_instruction": "…", "slice_max_elements": 40}
+}
+```
+
+`schema` is a graflo schema, or a manifest carrying one. `options` may set
+`user_instruction`, `slice_max_tokens`, `slice_max_elements` and
+`max_episode_tokens`; anything left out takes the service's
+[setting](../reference/settings.md). A request cannot choose the model or its
+endpoint.
+
+The reply holds what [`lpgcast cast --out`](../reference/output.md) writes:
+`graph` (graflo's `GraFloOutput`), `groundings`, `pending`, `quarantine`,
+`episodes` and `manifest`. To write the graph, pass `graph` and `pending` to
+graflo's `write_container`, the function `write_cast` uses.
+
+| Status | Means |
+|---|---|
+| 200 | cast; an episode that failed is in `episodes` and `quarantine`, not an error |
+| 401 | missing or wrong bearer token |
+| 413 | more episodes than `LPGCAST_SERVE_MAX_EPISODES` |
+| 422 | the body, or its `schema`, is not valid |
+| 503 | no key configured, or the model is misconfigured (no key, unknown model) |
+
+The request is synchronous: the reply comes when every episode is cast. Keep
+batches small enough for the caller's timeout.
+
+## Authentication
+
+Every route but `GET /health` needs `Authorization: Bearer
+$LPGCAST_SERVE_API_KEY`. Without a configured key the service refuses to cast
+(503) rather than serve anyone who can reach it; set
+`LPGCAST_SERVE_ALLOW_ANONYMOUS=true` only on a development machine.
+
+`GET /health` reports the versions, whether a key is configured and the batch
+cap.

docs/index.md

@@ -75,7 +75,7 @@ uv add lpgcast
 </div>
 
 !!! note "Early development"
-    LPGCast is not released yet. Until it is on PyPI, install it from a clone; see
+    LPGCast is on PyPI (`pip install lpgcast`) and its API may still change; see
     [Installation](getting_started/installation.md).
 
 ## What it does

docs/reference/output.md

@@ -25,7 +25,8 @@ JSON is written with sorted keys, so two casts compare with `diff`.
 | `retrieval` | how selection went: hits per lane, seeds fused, gated out, capped and kept, and the schema's size |
 | `requests`, `repairs` | model requests, re-asks and repairs included; typed repair rounds |
 | `vertices`, `edges`, `pending`, `quarantined`, `unmapped` | counts |
-| `fixes` | deterministic corrections applied without asking the model: `coerced`, `edge_reversed`, `quote_normalized` |
+| `repair_failure` | why a repair request failed, when one did; the best earlier reply was kept |
+| `fixes` | deterministic corrections applied without asking the model: `coerced`, `edge_reversed`, `quote_normalized`, `grounded_in_context` |
 
 ## `pending.jsonl`
 
@@ -35,7 +36,8 @@ An edge with an endpoint known only by a secondary identity: its `edge_id`, its
 
 ## `manifest.json`
 
-The LPGCast and graflo versions, the schema as `name@version`, every model
+The output format version (2: groundings may point into context), the LPGCast
+and graflo versions, the schema as `name@version`, every model
 setting that reaches the provider, the token ceiling and slice budget, the
 counts by status, and `usage`: calls, cache hits, failures, re-asks, and
 tokens, `billed` apart from `replayed`.

docs/reference/python/index.md

@@ -9,7 +9,7 @@ programs need are importable from `lpgcast` itself:
 | `Episode`, `read_episodes` | the input ([`episode`](episode.md)) |
 | `load_schema` | a graflo schema or manifest from YAML or JSON ([`schema_io`](schema_io.md)) |
 | `save_cast` | a cast on disk ([`output`](output.md)) |
-| `write_cast`, `load_connection`, `WriteReport` | writing to a database ([`write`](write.md)) |
+| `write_cast`, `load_connection`, `WriteReport` (graflo's `ContainerWriteReport`) | writing to a database through graflo's `write_container` ([`write`](write.md)) |
 | `Grounding` | where an assertion was read ([`ground`](ground.md)) |
 | `JsonlQuarantineSink`, `MemoryQuarantineSink` | where left-out items go ([`sink`](sink.md)) |
 | `LPGCastSettings`, `LLMSettings`, `CacheSettings` | settings ([`config`](config.md)) |

lpgcast/bench/cases/SPLITS.yaml

@@ -21,6 +21,10 @@ dev:
 -   recast/lab
 -   injection/plant
 -   injection/lab
+-   context/plant
+-   context/logistics
+-   selection/logistics
+-   selection/plant
 test:
 -   distractors/logistics
 -   distractors/lab
@@ -36,3 +40,7 @@ test:
 -   recast/logistics
 -   injection/helpdesk
 -   injection/logistics
+-   context/helpdesk
+-   context/lab
+-   selection/helpdesk
+-   selection/lab

lpgcast/bench/cases/context/helpdesk/case.yaml

@@ -0,0 +1,2 @@
+description: A key stated only in an earlier text of the thread; everything else is stated in the text itself.
+schema: helpdesk

lpgcast/bench/cases/context/helpdesk/episodes.jsonl

@@ -0,0 +1,2 @@
+{"source_id": "context-helpdesk-1", "text": "That ticket is a duplicate of T-5490.", "context": [{"source_id": "context-helpdesk-0", "text": "Ticket T-5521 was opened by Dana (account ACC-77) about the router firmware."}]}
+{"source_id": "context-helpdesk-2", "text": "She has handled it since 2024-05-02.", "context": [{"source_id": "context-helpdesk-0b", "text": "Agent @mira picked up ticket T-6100 yesterday."}]}

lpgcast/bench/cases/context/helpdesk/expected.yaml

@@ -0,0 +1,46 @@
+context-helpdesk-1:
+    vertices:
+        a:
+            type: Ticket
+            quote: That ticket
+            mention: true
+            props:
+                number:
+                    value: T-5521
+                    context: context-helpdesk-0
+        b:
+            type: Ticket
+            quote: T-5490
+            mention: true
+            props:
+                number: T-5490
+    edges:
+    -   relation: DUPLICATE_OF
+        from: a
+        to: b
+        quote: That ticket is a duplicate of T-5490
+context-helpdesk-2:
+    vertices:
+        a:
+            type: Agent
+            quote: She
+            mention: true
+            props:
+                handle:
+                    value: '@mira'
+                    context: context-helpdesk-0b
+        t:
+            type: Ticket
+            quote: it
+            mention: true
+            props:
+                number:
+                    value: T-6100
+                    context: context-helpdesk-0b
+    edges:
+    -   relation: HANDLES
+        from: a
+        to: t
+        quote: She has handled it since 2024-05-02
+        props:
+            since: '2024-05-02'

lpgcast/bench/cases/context/lab/case.yaml

@@ -0,0 +1,2 @@
+description: A key stated only in an earlier text of the thread; everything else is stated in the text itself.
+schema: lab