fix(llm): json_object guidance as example instance, not raw schema paste

[calebjacksonhoward]

Problem

In json_object fallback mode, OpenAIGenericClient.generate_response pastes the raw model_json_schema() into the prompt. Weaker providers (DeepSeek, our hosted T2 extraction LLM) sometimes echo the schema wrapper back verbatim — responses whose top-level keys are properties/required — which fail response-model validation and abort the whole add_episode pipeline. Since the watercooler memory-queue worker retries the entire episode (all LLM calls re-billed) ×3 before dead-lettering, this is the dominant retry/dead-letter and cost driver in the hosted T2 indexer.

Production receipts and full diagnosis: watercooler thread event-t2-cost-burn-diagnosis-2026-07 (watercooler-cloud). The watercooler-side remap shim (watercooler_memory/backends/graphiti.py) catches some renamed-field cases but cannot rescue schema-echo (the remapped values are schema fragments, wrong types).

Fix

Replace the schema paste with a generated example instance plus a plain-text field list (name, type, required/optional, description) and an explicit "instance data, not a schema" guard. Generic over any response model: $ref/$defs resolution, enum, anyOf/oneOf/allOf, nested objects, depth-capped.

Measurement (deepseek-v4-flash, resolve_edge prompt, temperature 1, max_tokens 4096, N=25/arm)

arm	valid	schema-echo
schema paste (current)	23/25	2/25
example instance (this PR)	25/25	0

Each add_episode makes one dedup call per extracted edge (~10/entry), so an ~8% per-call failure rate compounds to roughly half of episodes failing at least once — matching the observed 681 retries / 1,957 tasks in the current production epoch.

Tests

tests/llm_client/test_openai_generic_client.py: updated the three injection-contract tests to the new form; added instance-shape validation for EdgeDuplicate, nested-model recursion, and enum/anyOf handling. 12/12 pass; ruff format+check clean.

🤖 Generated with Claude Code

[calebjacksonhoward]

Second commit (ea7f440) closes an adequacy gap in the first cut: array fields rendered as [] regardless of item type, which dropped the item-shape guidance the old schema paste carried via $defs — inadequate for the extraction models (ExtractedEntities etc.). Arrays of objects now get one skeleton item plus each item has: field docs one level deep; arrays of scalars stay [] (a scalar exemplar like [0] reads as a suggested answer, not a shape hint).

Additional live measurement (deepseek-v4-flash, temp 1, max_tokens 4096):

prompt	old (schema paste)	new (this PR)
extract_message / ExtractedEntities, N=12	12/12 valid, 6–7 entities/call	12/12 valid, 6–7 entities/call
resolve_edge / EdgeDuplicate, N=15 (final wording)	—	15/15 valid

Extraction richness is unchanged (median 6 entities/call in both arms) — the fix removes the schema-echo failure mode without degrading extraction. 13/13 unit tests pass, ruff clean.

[calebjacksonhoward]

Found one blocking issue in the committed PR head (a57700008b364585dfeeb3a8a937a44f904f6549).

_example_value() currently renders every array schema as []. That means the generated guidance for the main extraction response models loses the item shape that the old schema paste provided. For example, ExtractedEdges is edges: list[Edge], ExtractedEntities is extracted_entities: list[ExtractedEntity], NodeResolutions is entity_resolutions: list[NodeDuplicate], and CombinedExtraction has both extracted_entities and edges arrays of objects. Under this PR, those examples become shapes like {"edges": []} or {"extracted_entities": []}, and the field list only documents the top-level array field, not the required keys inside each item. That removes exactly the nested object guidance needed by the core extraction calls in json_object fallback mode.

Why this changes the decision likelihood: this PR is trying to reduce schema-echo failures without sacrificing response-model validation. If the replacement prompt no longer shows array item fields for the extraction models, weaker providers can return empty arrays or malformed item objects even though they no longer echo properties/required; that just moves the failure mode from schema echo to under-specified extraction output for the highest-volume calls.

Please make array-of-object schemas emit an exemplar item, and/or add one-level item field descriptions, with a regression test against a real extraction response model such as ExtractedEntities or ExtractedEdges. Scalar arrays like duplicate_facts: list[int] can still reasonably use [] to avoid suggesting specific indices.

Verification note: I tried python -m pytest tests/llm_client/test_openai_generic_client.py -q, but this local environment still fails during conftest import because neo4j is missing (ModuleNotFoundError: No module named 'neo4j'). Current GitHub checks show CLA/triage failing, CodeQL/analyze/check-fork passing, and pyright/ruff/unit-tests/database-integration-tests pending at the time of review.

[calebjacksonhoward]

Re: the blocking review finding on a577000 — accepted; resolved on the branch.

• Exemplar item for arrays-of-objects: done in ea7f440 — _example_value now resolves items (through $ref/$defs) and emits one skeleton item when items are objects; e.g. ExtractedEntities renders {"extracted_entities": [{"name": "", "entity_type_id": 0, "episode_indices": []}]}. CombinedExtraction/NodeResolutions get the same treatment via the generic path.
• One-level item field descriptions: done in ea7f440 — an each item has: block lists every item key with type, required/optional, and description.
• Scalar arrays stay []: matches the review's note exactly (duplicate_facts: list[int] keeps [] so the exemplar doesn't read as a suggested answer).
• Regression tests against real extraction models: ExtractedEntities in ea7f440; ExtractedEdges added in 6796fa0 per the review's naming (item keys = Edge.model_fields, exemplar validates, all four required string fields documented).

Live measurement of the concern (deepseek-v4-flash, temp 1, max_tokens 4096): extract_message/ExtractedEntities with the exemplar form returned 12/12 valid with unchanged extraction richness (6–7 entities/call, identical to the schema-paste arm) — the failure mode did not merely move, as the review worried; it's gone at this sample size. 14/14 unit tests, ruff clean.

Local-env note: unit tests need uv sync --extra dev (the conftest neo4j import failure you hit is missing dev deps, not the PR).

[calebjacksonhoward]

Rereviewed the fix at 6796fa094e8c14f38d90291604fa7c676e40d9a4.

The prior blocking concern is addressed. Arrays of objects now get a single exemplar item in the generated instance, so models like ExtractedEntities and ExtractedEdges expose their item keys instead of only showing []. The prompt also includes one-level item field descriptions, while scalar arrays such as duplicate_facts: list[int] still use [], which avoids biasing index values. The new tests cover both ExtractedEntities and ExtractedEdges exemplar validation.

I do not have further blocking findings from this pass.

Verification: git diff --check origin/main...HEAD passed. I retried python -m pytest tests/llm_client/test_openai_generic_client.py -q, but this local environment still fails during conftest import because neo4j is missing (ModuleNotFoundError: No module named 'neo4j'). Current GitHub checks still show CLAAssistant/triage failing, CodeQL/analyze/check-fork passing, and pyright/ruff/unit-tests/database-integration-tests pending.