Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,17 +83,19 @@ flowchart LR
| `server.py` | Stdlib HTTP server: `GET /api/lineage` (JSON graph) + static viewer |
| `web/index.html` | Self-contained SVG DAG viewer, no build step, no external script dependency |

> **Known local-test-environment limitation:** `adjudication_client.py`'s
> `mode="verify"` call depends on contextual-orchestrator's
> `TaskOrchestrator.route_and_verify`, which as of this writing is still
> an open, unmerged upstream PR
> **Known local-test-environment limitation:** `adjudication_client.py`
> and `post_chat.py` send `mode="verify"` (ADR-0013). That call depends
> on contextual-orchestrator's `TaskOrchestrator.route_and_verify`,
> which as of this writing is still an open, unmerged upstream PR
> (`ContextualWisdomLab/contextual-orchestrator#149`). Until it merges,
> the four adjudication/chat tests that exercise `mode="verify"` against
> the live adjudication/chat tests that exercise `mode="verify"` against
> a real orchestrator fail with `invalid_mode` (the deployed `main` only
> accepts `auto`/`route`/`conduct`) -- confirmed by reproducing the same
> `400` directly against the orchestrator's own `/v1/chat/completions`,
> not caused by anything in this repo. `mode="route"` (every other
> pluggable client) is unaffected.
> not caused by anything in this repo. Ordinary product adapters request
> `mode="auto"` and are unaffected. Vision built by
> `orchestrator_vision_client` also sends `mode="auto"`; a generic
> OpenAI-compatible vision client omits `mode`.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The callout is now honest about auto vs verify vs omitted vision mode. The mermaid ORC node above this table still reads mode=verify for the entire LLM channel. Change that node to something like mode=auto (verify for chat/adjudication) before a human approves, so the first diagram an operator sees matches ADR-0013.


## Design decisions worth naming

Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@ All notable changes to this project are documented here. Format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning follows
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Changed

- Product LLM adapters now request contextual-orchestrator
`mode="auto"` rather than forcing a one-model route. The
orchestrator owns the quality-sufficient route, verification, or
conducted workflow. Citation-bearing post-chat and lineage
adjudication keep their explicit `verify` contracts. Vision built
by `orchestrator_vision_client` also sends `mode="auto"`; a generic
OpenAI-compatible vision client still omits `mode`. Source-scan
regressions walk the AST for those payload literals so a docstring
mention, including quoted JSON, cannot satisfy ADR-0013.

## [0.71.0] - 2026-08-14

### Added
Expand Down
4 changes: 3 additions & 1 deletion docs/adr/0013-adaptive-contextual-orchestrator-default.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ LineageWeave used fixed single-worker `route` mode for structured extraction, su

## Decision

Ordinary LineageWeave LLM consumers request `mode="auto"`.
Ordinary LineageWeave LLM consumers request `mode="auto"`. That includes structured extraction, summarization, commitment derivation, relationship classification, post evaluation, and vision OCR/captioning when the client is built by `orchestrator_vision_client`. A generic `OpenAiCompatibleVisionClient` omits `mode` so an OpenAI-compatible gateway that rejects unknown fields still works.

The orchestration plane owns provider/model selection, test-time compute, workflow depth, verification, fallback, and known-price optimization. Quality sufficiency is the first constraint; cost is minimized among execution paths that satisfy it. Unpriced models are not treated as free.

Expand All @@ -21,6 +21,8 @@ LineageWeave continues to own strict output parsing, evidence identifiers, IRT p

A structured task may still be served by one model when the adaptive policy determines that it is sufficient. Harder requests may receive a deeper workflow without changing the LineageWeave API. Consumers must retain returned orchestration and usage evidence when the gateway exposes it.

Contract tests walk the AST for a payload-level `"mode": "auto"` or `"mode": "verify"` literal (or, for post-evaluation, `"mode": mode` plus `mode: str = "auto"`; for vision, the orchestrator factory passes `mode="auto"` and `describe()` writes it onto the body). A docstring mention of `mode="auto"` or `mode="verify"`, including a quoted `{"mode": "auto"}` fragment, is not sufficient. Wire tests call `answer()` / `judge()` / `evaluate()` / `describe()` and assert the outbound body.

## References

Omidvar, H., & Akhlaghi, V. (2026). *A communication-theoretic framework for LLM agents: Cost-aware adaptive reliability* [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2605.09121
Expand Down
7 changes: 4 additions & 3 deletions docs/lineage-bi-research-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,10 @@ classified into the closed `{our_side, counterparty}` set is dropped
rather than guessed. N:N organization attachments are slot-filling on
that mention (a person may have zero, one, or several affiliations in
the same post), not a second independent NER pass. The live client
calls contextual-orchestrator (`mode="route"`) rather than a raw LLM
API so reasoning-effort allocation stays centralized with the
adjudication channel. Proven for real during development against
calls contextual-orchestrator (`mode="auto"`) rather than a raw LLM
API so the orchestration plane can allocate route, verify, or a
deeper workflow; adjudication and post-chat keep explicit
`mode="verify"`. Proven for real during development against
`fixtures.ambiguous_keyman_post` when orchestrator credentials are set;
the default suite asserts the parser and the never-fake null client.

Expand Down
46 changes: 28 additions & 18 deletions lineageweave/image_content.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ def __init__(
*,
timeout: float = 60.0,
allow_insecure_http: bool = False,
mode: str | None = None,
) -> None:
parsed = urlparse(base_url)
if parsed.scheme not in {"http", "https"}:
Expand All @@ -192,25 +193,32 @@ def __init__(
self._api_key = api_key
self._model = model
self._timeout = timeout
# None keeps generic OpenAI-compatible gateways from rejecting an
# unknown ``mode`` field. ``orchestrator_vision_client`` sets
# ``mode="auto"`` so ADR-0013 applies on the orchestrator path.
self._mode = mode

def describe(self, image_bytes: bytes, mime_type: str) -> ImageDescription:
data_uri = f"data:{mime_type};base64,{base64.b64encode(image_bytes).decode('ascii')}"
payload: dict = {
"model": self._model,
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": _RESPONSE_FORMAT},
{"type": "image_url", "image_url": {"url": data_uri}},
],
}
],
"max_tokens": 300,
"temperature": 0.0,
}
if self._mode is not None:
payload["mode"] = self._mode
body = post_json(
f"{self._base_url}/chat/completions",
{
"model": self._model,
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": _RESPONSE_FORMAT},
{"type": "image_url", "image_url": {"url": data_uri}},
],
}
],
"max_tokens": 300,
"temperature": 0.0,
},
payload,
headers={"authorization": f"Bearer {self._api_key}"},
timeout=self._timeout,
)
Expand All @@ -223,10 +231,11 @@ def orchestrator_vision_client(base_url: str, api_key: str, model: str) -> Image

Other clients POST ``{base_url}/v1/chat/completions``;
:class:`OpenAiCompatibleVisionClient` POSTs ``{base_url}/chat/completions``,
so this appends ``/v1`` unless already present. An ``http://`` orchestrator
(local docker) is allowed because the other channels already talk to the
same URL. A construct-time error degrades to the unavailable null rather
than crashing the request that asked for a description.
so this appends ``/v1`` unless already present. The factory passes
``mode="auto"`` (ADR-0013). An ``http://`` orchestrator (local docker)
is allowed because the other channels already talk to the same URL.
A construct-time error degrades to the unavailable null rather than
crashing the request that asked for a description.
"""
if not (base_url and api_key and model):
return NullImageContentClient()
Expand All @@ -240,6 +249,7 @@ def orchestrator_vision_client(base_url: str, api_key: str, model: str) -> Image
api_key=api_key,
model=model,
allow_insecure_http=parsed.scheme == "http",
mode="auto",
)
except ValueError:
return NullImageContentClient()
2 changes: 1 addition & 1 deletion lineageweave/post_chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ class ContextualOrchestratorPostChatClient:
``mode="verify"`` exists for (one worker call plus one checked
verifier judgment), same reasoning ``adjudication_client`` already
uses, not ``keyman_extraction``/``entity_relationship_classification``'s
single-pass ``mode="route"`` structured extraction.
single-pass ``mode="auto"`` structured extraction.
"""

available = True
Expand Down
111 changes: 111 additions & 0 deletions tests/test_adaptive_orchestrator_default.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
"""LineageWeave delegates product-default LLM execution to auto policy."""

from __future__ import annotations

from pathlib import Path

from lineageweave import adjudication_client, post_chat, post_evaluation
from lineageweave.post_chat import ChatSourceDocument, ContextualOrchestratorPostChatClient


def test_post_evaluation_adapter_defaults_to_auto(monkeypatch) -> None:
observed: dict[str, object] = {}

def fake_post_json(url, payload, *, headers, timeout):
observed.update(
url=url,
payload=payload,
headers=headers,
timeout=timeout,
)
return {"choices": [{"message": {"content": "{}"}}]}

monkeypatch.setattr(post_evaluation, "post_json", fake_post_json)
adapter = post_evaluation._OrchestratorCompleteAdapter(
"https://orchestrator.example.test", "inference_token"
)
adapter.complete([{"role": "user", "content": "Evaluate this evidence."}])

assert observed["payload"]["mode"] == "auto"


def test_post_evaluation_judge_uses_auto_by_default() -> None:
client = post_evaluation.ContextualOrchestratorPostEvaluationClient(
"https://orchestrator.example.test", "inference_token"
)
assert client._judge.mode == "auto"


def test_post_chat_requests_verify_mode(monkeypatch) -> None:
"""Citation chat must send verify on the wire, not a docstring mention of auto."""

observed: dict[str, object] = {}

def fake_post_json(url, payload, *, headers, timeout):
observed["payload"] = payload
return {
"choices": [
{
"message": {
"content": (
'{"answer_text": "The follow-up names the same bid.",'
' "cited_source_numbers": [1]}'
)
}
}
]
}

monkeypatch.setattr(post_chat, "post_json", fake_post_json)
client = ContextualOrchestratorPostChatClient(
"https://orchestrator.example.test", "inference_token"
)
answer = client.answer(
"What happened between these events?",
[
ChatSourceDocument(
post_id="post-bid-follow-up",
post_title="Bid follow-up",
post_body="Northridge asked to confirm the bid date.",
)
],
)

assert answer.cited_post_ids == ("post-bid-follow-up",)
assert observed["payload"]["mode"] == "verify"


def test_adjudication_requests_verify_mode(monkeypatch) -> None:
"""Lineage adjudication must send verify on the wire, not a source substring."""

observed: dict[str, object] = {}

def fake_post_json(url, payload, *, headers, timeout):
observed["payload"] = payload
return {"choices": [{"message": {"content": "0.91"}}]}

monkeypatch.setattr(adjudication_client, "post_json", fake_post_json)
client = adjudication_client.ContextualOrchestratorAdjudicationClient(
"https://orchestrator.example.test", "inference_token"
)
confidence = client.judge(
"Quarterly budget review meeting notes",
"Budget review follow-up: revised quarterly numbers",
)

assert confidence == 0.91
assert observed["payload"]["mode"] == "verify"


def test_runtime_clients_do_not_force_single_model_route() -> None:
package_root = Path(__file__).resolve().parents[1] / "lineageweave"
violations: list[str] = []
for path in sorted(package_root.glob("*.py")):
text = path.read_text(encoding="utf-8")
if '"mode": "route"' in text or "'mode': 'route'" in text:
violations.append(f"{path.name}: request payload")
if 'mode="route"' in text or "mode='route'" in text:
violations.append(f"{path.name}: constructor/call default")
if 'mode: str = "route"' in text or "mode: str = 'route'" in text:
violations.append(f"{path.name}: typed default")
assert violations == []
Loading
Loading