Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,23 @@ MAX_VISITS=1
# against the catalog you already have. Overridable per request.
RENDER_MODE=ontology_and_facts # ontology | facts | ontology_and_facts

# Encoding the LLM uses for RDF graph payloads. `turtle` suits providers whose
# structured output handles strings more reliably than nested objects.
LLM_GRAPH_FORMAT=jsonld # jsonld | turtle
# Encoding the LLM uses for RDF graph payloads. `turtle` (recommended) spends
# fewer tokens per triple and cannot turn an IRI object into a string by
# accident; `jsonld` suits providers whose structured output handles long
# strings worse than nested objects. Watch llm/parse_retry when switching.
LLM_GRAPH_FORMAT=turtle # turtle | jsonld

# Whitespace the LLM is asked to use in its responses. `compact` (recommended)
# asks for minified JSON and one-line-per-subject Turtle; `free` gives no
# instruction, and models typically indent their JSON. Indentation is billed as
# output tokens and carries nothing the parser reads.
LLM_OUTPUT_LAYOUT=compact # compact | free

# Syntax of the `# ONTOLOGY` chapter in the facts render and critic prompts.
# `auto` (default) takes the cheapest chapter the render mode can legally read:
# `term_sheet` under RENDER_MODE=facts, `inherit` otherwise. `inherit` follows
# LLM_GRAPH_FORMAT; `turtle` shows the ontology as Turtle whatever the wire
# format is, which spends fewer characters per triple than pretty-printed
# JSON-LD; `term_sheet` replaces the serialized graph with a line-per-term
# format is; `term_sheet` replaces the serialized graph with a line-per-term
# listing -- name, surface forms, type, hierarchy, domain/range and usage
# contract -- and is by a wide margin the cheapest of the three.
# Context only: the graph payloads the model emits stay in LLM_GRAPH_FORMAT.
Expand Down
6 changes: 3 additions & 3 deletions .env.example.minimal
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,9 @@ ONTOLOGY_CONTEXT_MODE=selected_single_ontology
# it does not change the mode itself.
#ONTOLOGY_CONTEXT_FIXED_ONTOLOGY_ID= # IRI, ontology_id, or author prefix

# Encoding the LLM writes graphs in. `turtle` is worth trying if a provider's
# structured output handles strings better than nested objects.
LLM_GRAPH_FORMAT=jsonld # jsonld | turtle
# Encoding the LLM writes graphs in. Keep `turtle`; try `jsonld` if a
# provider's structured output handles long strings worse than nested objects.
LLM_GRAPH_FORMAT=turtle # turtle | jsonld


# -- 4. Cost and quality --------------------------------------------------
Expand Down
38 changes: 36 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,41 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
## [0.6.7]

### Added

- **`LLM_OUTPUT_LAYOUT`** (`compact` | `free`): `compact` asks for minified JSON responses and
one-line-per-subject Turtle; recorded in the run manifest's `prompting` block.
- **Counters** `rdf/turtle_repair` and `rdf/jsonld_rdflib_fallback` for graph payloads that
parsed only after recovery, and `repair/compact_iri_literal`.

### Changed

- **Default output encoding** is `LLM_GRAPH_FORMAT=turtle` with `LLM_OUTPUT_LAYOUT=compact`
(was `jsonld` with no layout instruction). Set `LLM_GRAPH_FORMAT=jsonld` and
`LLM_OUTPUT_LAYOUT=free` to keep the old behaviour; cached LLM responses are not reused.

### Fixed

- **JSON-LD instructions** no longer offer a plain string for an IRI object, which JSON-LD parses
as a literal; every JSON-LD instruction now requires `{"@id": …}`.
- **Compact IRIs written as plain strings** (`"unit:NanoM"`) on an IRI-valued position are coerced
to IRIs after a facts render and in critic and completion patches, which skipped every
literal check.
- **`ontocast_extract` and `make_ontocast_node`** honour `LLM_GRAPH_FORMAT`; both ran on the
class default.
- **Facts aggregation scales linearly with the document**: an entity's representation reads
only the triples that mention it, through the graph's indexes, instead of whole unit graphs
merged per mention. Long documents no longer stall before clustering; ontology anchor
selection and entity alignment no longer scan the whole graph per entity.
- **`RDFGraph +=`** adds in place, at the cost of the right-hand side (it rebuilt the whole graph).
- **`CHUNK_MIN_SIZE` held only within a labeled section**: neighbouring unlabeled sections are
now joined up to it, and a heading with no text of its own opens the unit that follows instead
of becoming a unit. Documents with sparse section labels yield far fewer, larger units; their
cached LLM responses are not reused.

## [0.6.6] - 2026-10-04

### Added

Expand Down Expand Up @@ -33,7 +67,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Removed `CONVERTER_PROFILE` values** `default` and `born_digital` dropped from shipped env
examples.

## [0.6.5]
## [0.6.5] - 2026-10-04

### Added

Expand Down
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
# OntoCast <img src="https://raw.githubusercontent.com/growgraph/ontocast/refs/heads/main/docs/assets/logo.png" alt="OntoCast logo" style="height: 32px; width:32px;"/>
# OntoCast <img src="https://raw.githubusercontent.com/growgraph/ontocast/refs/heads/main/docs/assets/logo.png" alt="OntoCast logo" height="32" width="32"/>

**Ontology-guided extraction of RDF knowledge graphs from documents.**

![Python](https://img.shields.io/badge/python-3.12%2B-blue.svg)
[![PyPI version](https://badge.fury.io/py/ontocast.svg)](https://badge.fury.io/py/ontocast)
[![PyPI Downloads](https://static.pepy.tech/badge/ontocast)](https://pepy.tech/projects/ontocast)
[![Docs](https://img.shields.io/badge/docs-growgraph.github.io-224777.svg)](https://growgraph.github.io/ontocast/)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![pre-commit](https://github.com/growgraph/ontocast/actions/workflows/pre-commit.yml/badge.svg)](https://github.com/growgraph/ontocast/actions/workflows/pre-commit.yml)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.17796467.svg)](https://doi.org/10.5281/zenodo.17796467)
[![tests](https://img.shields.io/github/actions/workflow/status/growgraph/ontocast/tests.yml?branch=main&label=tests)](https://github.com/growgraph/ontocast/actions/workflows/tests.yml)
[![pre-commit](https://img.shields.io/github/actions/workflow/status/growgraph/ontocast/pre-commit.yml?branch=main&label=pre-commit)](https://github.com/growgraph/ontocast/actions/workflows/pre-commit.yml)
[![PyPI](https://img.shields.io/pypi/v/ontocast?color=224777)](https://pypi.org/project/ontocast/)
[![Python](https://img.shields.io/pypi/pyversions/ontocast?color=224777)](https://pypi.org/project/ontocast/)
[![Downloads](https://img.shields.io/pepy/dt/ontocast?color=224777)](https://pepy.tech/projects/ontocast)
[![Docs](https://img.shields.io/badge/docs-growgraph.github.io-224777)](https://growgraph.github.io/ontocast/)
[![License](https://img.shields.io/pypi/l/ontocast?color=224777)](https://github.com/growgraph/ontocast/blob/main/LICENSE)
[![DOI](https://img.shields.io/badge/DOI-10.5281%2Fzenodo.17796467-224777)](https://doi.org/10.5281/zenodo.17796467)

OntoCast reads documents and writes an RDF knowledge graph: an ontology that
describes the domain, and the facts the documents state in its terms. Give it
Expand Down
8 changes: 5 additions & 3 deletions docs/concepts/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,11 @@ A document is too long for one prompt, so OntoCast cuts it into parts (content
units) of between
[`CHUNK_MIN_SIZE`](../reference/configuration/chunking.md#chunk_min_size) and
[`CHUNK_MAX_SIZE`](../reference/configuration/chunking.md#chunk_max_size)
characters. Cuts follow the document's sections, so no unit spans two of them,
and every unit carries the label of the section it came from, such as
`methods` or `results`.
characters. Cuts follow the document's sections, so no unit spans two labeled
sections, and every unit carries the label of the section it came from, such as
`methods` or `results`. Neighbouring sections that got no label are joined
until the unit reaches the minimum size, and a heading with no text of its own
opens the unit that follows it.

Section labels decide what is extracted at all: reference lists are skipped by
default, and you can ask for only some sections of a document. [Structured
Expand Down
4 changes: 2 additions & 2 deletions docs/concepts/ontologies_and_facts.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,8 +50,8 @@ graph, deletions first. Two things follow:
an explicit delete.

The facts critic works the same way: its fixes are applied as a patch to the
unit's facts. The model writes patches as compact JSON-LD by default, or as
Turtle; see
unit's facts. The model writes patches as Turtle by default, or as compact
JSON-LD; see
[`LLM_GRAPH_FORMAT`](../reference/configuration/pipeline.md#llm_graph_format).
Either way, the patch that is applied is the same.

Expand Down
8 changes: 5 additions & 3 deletions docs/concepts/structured_documents.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ document it is reading, and how you choose the sections to extract.

While chunking, OntoCast gives every content unit the label of the section it
came from, such as `methods`, `results` or `risk_factors`. Units never span two
sections, so each has at most one label. A unit OntoCast cannot place confidently
gets no label at all: an unlabeled unit is merely never selected by a filter,
while a wrongly labeled one would be dropped or kept by mistake.
labeled sections, so each has at most one label. A unit OntoCast cannot place
confidently gets no label at all: an unlabeled unit is merely never selected by
a filter, while a wrongly labeled one would be dropped or kept by mistake. For
the same reason, size never joins unlabeled text to a labeled unit; neighbouring
unlabeled sections are joined with each other instead.

Labels are recorded in the facts' [provenance](ontologies_and_facts.md#provenance),
together with how each was decided and how confident that decision was.
Expand Down
6 changes: 3 additions & 3 deletions docs/guides/llm_caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ entries instead of misreading them.

Because the whole prompt is part of the key, anything that changes the prompt
misses the cache: a different document part, a changed ontology in the
catalog, a user instruction, `LLM_GRAPH_FORMAT`, `ONTOLOGY_CHAPTER_FORMAT`, or
an OntoCast upgrade that changes prompt wording. Settings that act only after
catalog, a user instruction, `LLM_GRAPH_FORMAT`, `LLM_OUTPUT_LAYOUT`,
`ONTOLOGY_CHAPTER_FORMAT`, or an OntoCast upgrade that changes prompt wording. Settings that act only after
the last LLM call, such as entity disambiguation (`AGG_*`), leave the cache
warm.
`LLM_PROMPT_CACHE_KEY` and `LLM_JSON_MODE` are not part of the key.
`LLM_PROMPT_CACHE_KEY` is not part of the key; `LLM_JSON_MODE` is, when it is on.

Each entry is a JSON file holding the prompt, the response, the provider's
response metadata and its token usage, so a hit reports the same usage as the
Expand Down
40 changes: 29 additions & 11 deletions docs/guides/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,19 +83,37 @@ Every lever below changes the prompt text, so the first run after a change
misses the [LLM response cache](llm_caching.md). [How prompts are
built](../internals/prompt_construction.md) describes the mechanics.

### How much a triple costs
### How the model reads and writes the graph

The defaults pair what the model reads with what it writes:

| Run | Ontology the model reads | Graph the model writes |
|---|---|---|
| `RENDER_MODE=facts` | A term sheet ([below](#the-ontology-chapter-as-a-term-sheet)) | Turtle, compact layout |
| `ontology` or `ontology_and_facts` | Turtle: its chapter follows the output format, because the loop patches what it reads | Turtle, compact layout |

[`LLM_GRAPH_FORMAT`](../reference/configuration/pipeline.md#llm_graph_format)
sets the syntax the model writes graphs in: `jsonld` (default) or `turtle`.
Turtle spends fewer characters per triple and changes nothing about what is
extracted. The two fail differently: a JSON-LD answer with mismatched brackets
is repaired or lost, while in Turtle an IRI the model builds from a phrase
containing a delimiter is invalid. Run both on your documents and compare
`llm/parse_retry` and `llm/parse_abandoned`. On a reasoning model, read
`reasoning_share_of_output` first: when most output tokens are reasoning, the
encoding matters less than the reasoning budget
([`LLM_REASONING_EFFORT`](../reference/configuration/llm.md#llm_reasoning_effort)).
The format can also be set per request.
sets the syntax the model writes graphs in. **Keep the default, `turtle`.** It
spends fewer tokens per triple than JSON-LD, and an IRI object cannot turn into
a string by accident: `unit:NanoM` is an IRI and `"unit:NanoM"` is visibly a
literal. In JSON-LD a bare `"unit:NanoM"` is a literal too, and nothing in the
syntax warns the model. Switch to `jsonld` for a provider whose structured
output handles long strings worse than nested objects: escaping or truncating
a long string shows up as `llm/parse_retry` and `rdf/turtle_repair`. Read those
counters on your own documents before and after switching. The format can also
be set per request.

[`LLM_OUTPUT_LAYOUT`](../reference/configuration/pipeline.md#llm_output_layout)
sets the whitespace of the response. **Keep the default, `compact`.** It asks
for minified JSON and, under Turtle, one line per subject. Without it, models
indent their JSON, and every indentation token is billed output that the parser
discards. `free` exists to compare against; it changes no content, so compare
`output_tokens` with the layout on and off.

On a reasoning model, read `reasoning_share_of_output` first: reasoning tokens
are output that no encoding touches, so when they dominate, the reasoning budget
([`LLM_REASONING_EFFORT`](../reference/configuration/llm.md#llm_reasoning_effort))
matters more than either setting.

### The ontology chapter as a term sheet

Expand Down
3 changes: 2 additions & 1 deletion docs/guides/telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,8 @@ their manifests instead of rerunning them. The HTTP routes return the same
"render_mode": "ontology_and_facts",
"loops": {"max_visits": 1, "facts_critic_passes": 1, "ontology_critic_passes": 0},
"llm": {"provider": "<provider>", "model_name": "<model>", "temperature": 0.0, "max_inflight": 16},
"prompting": {"llm_graph_format": "jsonld", "ontology_chapter_format": "inherit",
"prompting": {"llm_graph_format": "turtle", "llm_output_layout": "compact",
"ontology_chapter_format": "inherit",
"ontology_context_scope": "unit", "fanout_warmup_units": 0,
"parallel_workers": 16, "embedding_model_name": "<model>"},
"budget": {"calls_count": 0, "input_tokens": 0, "output_tokens": 0,
Expand Down
5 changes: 4 additions & 1 deletion docs/internals/telemetry_counters.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,9 @@ wall × effective workers ≈ llm/provider + llm/inflight_wait
| `llm/parse_retry` | `ontocast/agent/common.py` | Calls re-sent because the previous answer did not parse or validate |
| `llm/json_bracket_repair` | `ontocast/agent/common.py` | Answers recovered by correcting mismatched closing brackets |
| `llm/parse_abandoned` | `ontocast/agent/common.py` | Calls given up after the retries ran out or the same JSON error recurred |
| `rdf/turtle_repair` | `ontocast/onto/rdfgraph.py` | Turtle payloads that parsed only after repair (truncated statement, unknown prefix, update-query wrapper) |
| `rdf/jsonld_rdflib_fallback` | `ontocast/onto/rdfgraph.py` | JSON-LD payloads that failed URDNA2015 normalization and were parsed by rdflib instead |
| `repair/compact_iri_literal` | `ontocast/tool/facts_validation/literal_repair.py` | Compact IRIs written as plain strings on an IRI-valued position and coerced back to IRIs, in renders and in critic and completion patches |
| `llm/ontology_selection` | `ontocast/agent/select_ontology_catalog.py` | Ontology selection calls, one per unit in `selected_single_ontology` mode |
| `chapter/text_chars_before` | `ontocast/agent/render_facts.py` | Summed length of capped text literals before the text caps |
| `chapter/text_chars_after` | `ontocast/agent/render_facts.py` | The same after the caps |
Expand Down Expand Up @@ -204,7 +207,7 @@ left out.
| `ontocast_version`, `render_mode`, `current_domain`, `doc_iri`, `tenant`, `project` | What ran, and where its IRIs and graphs went |
| `loops` | The effective `max_visits`, `facts_critic_passes`, `ontology_critic_passes` |
| `llm` | Provider, model, temperature, the Ollama and reasoning settings, `requests_per_second`, `max_retries`, `prompt_cache_key`, `max_inflight` |
| `prompting` | `llm_graph_format`, the resolved `ontology_chapter_format`, `ontology_context_scope`, `fanout_warmup_units`, `parallel_workers`, `embedding_model_name`. Scope and warm-up are left out on the single-unit path |
| `prompting` | `llm_graph_format`, `llm_output_layout`, the resolved `ontology_chapter_format`, `ontology_context_scope`, `fanout_warmup_units`, `parallel_workers`, `embedding_model_name`. Scope and warm-up are left out on the single-unit path |
| `budget` | The full budget, as above |
| `critic`, `ontology_critic` | Per loop: `calls`, `accepted`, score minimum, median, maximum and histogram, fix severity histograms, `accept_reason_histogram`, `incumbent_accepted`, patch passes, fix outcomes (`fixes_applied`, `fixes_noop`, `fixes_rolled_back`, `fixes_junk_refused`, `fixes_unresolved_prefix`, `patches_rolled_back`), `units_unreviewed`, `units_skipped`, triples deleted and inserted |
| `completion` | `calls`, `units`, `subjects_inserted`, `subjects_rolled_back`, `triples_inserted`, `measurements_recovered` |
Expand Down
3 changes: 2 additions & 1 deletion docs/internals/validation_layers.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ machine can decide without the model:
|---|---|
| Numeric retyping | An untyped number on a property with a numeric range gets that datatype |
| `rdf:type` literal coercion | A type written as a string becomes the class IRI when it resolves unambiguously |
| Compact-IRI literal coercion | A plain string such as `"unit:NanoM"` on another predicate becomes the IRI, when its prefix is bound and either the schema says the predicate takes an IRI or the IRI is a catalog or fallback-vocabulary term. Critic and completion patches get the same repair before they are judged |
| Near-miss predicate rewrite | A predicate not in the catalog is rewritten to the one catalog term whose name tokens contain, are contained in, or equal its own. `FACTS_PROPERTY_ALIAS_MIN_RATIO` only breaks ties among such candidates; string similarity alone never triggers a rewrite. A predicate the full catalog declares is never rewritten, even when the unit's snapshot lacks it |
| Code resolution | A node carrying a code from `FACTS_CODE_PREDICATES` (such as `qudt:ucumCode "d"`) but no link to the coded individual gains that link, when exactly one catalog individual declares the code. The linking property comes from the schema's domain and range, or else from how the graph already links such nodes |
| Degenerate bounds | Equal lower and upper bounds become one value, when the quantity fallback vocabulary names `numeric_value`, `lower_bound` and `upper_bound` |
Expand All @@ -121,7 +122,7 @@ to the critic for repair: typed literals whose lexical form is invalid for the
datatype, and string literals on a property whose range is a class
(`FACTS_OBJECT_PROPERTY_LITERAL_CHECK`).

Type coercions, predicate rewrites and code resolutions are recorded as
Type and compact-IRI coercions, predicate rewrites and code resolutions are recorded as
`GraphRepairRecord`s and returned per unit as `facts_repairs`, so a consumer can
tell a machine rewrite from what the model asserted. Ambiguity is never
resolved by guessing: two catalog terms claiming one code means no repair.
Expand Down
1 change: 1 addition & 0 deletions ontocast/agent/complete_facts.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ async def complete_facts(
profile = get_graph_format_profile(
state.llm_graph_format,
ontology_chapter_format=state.ontology_chapter_format,
output_layout=state.llm_output_layout,
)
parser = PydanticOutputParser(pydantic_object=FactsCompletionReport)

Expand Down
1 change: 1 addition & 0 deletions ontocast/agent/criticise_facts.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ async def criticise_facts(
profile = get_graph_format_profile(
state.llm_graph_format,
ontology_chapter_format=state.ontology_chapter_format,
output_layout=state.llm_output_layout,
)
parser = PydanticOutputParser(pydantic_object=FactsCritiqueReport)

Expand Down
Loading
Loading