adapters.md

Adapters

An adapter is the only integration surface between RAGProof and your pipeline. It exposes up to two capabilities, and declares which ones it supports so the engine knows to skip metrics it cannot compute:

• retrieve(question, k) returns the chunks the pipeline retrieved.
• answer(question) returns the generated answer and its citations.

A pipeline may implement one or both. If it cannot expose retrieval, the retrieval metrics are skipped with a reason rather than faked.

Python adapter

Point target at an import path of the form module:attribute. The attribute may be an instance, or a class or function that builds one.

adapter:
  type: python
  target: my_package.pipeline:build

class MyPipeline:
    name = "my-pipeline"
    supports_retrieval = True
    supports_answer = True

    def retrieve(self, question: str, k: int) -> list[dict]:
        return [{"id": "doc-1", "text": "...", "score": 0.91}]

    def answer(self, question: str) -> dict:
        return {
            "answer_text": "...",
            "citations": [{"chunk_id": "doc-1"}],
            "retrieved_chunks": [{"id": "doc-1", "text": "..."}],
        }


def build() -> MyPipeline:
    return MyPipeline()

Both sync and async implementations are accepted. Sync methods are run in a thread so they never block the event loop. See examples/minimal_python_adapter/adapter.py for a complete working example.

HTTP adapter

For a pipeline behind a REST API, map requests and responses with JSONPath. No code is required. Header values that start with env: are read from the environment at request time and are never logged or stored.

adapter:
  type: http
  http:
    name: my-rag-api
    timeout_seconds: 30
    max_attempts: 3
    answer:
      request:
        url: https://rag.internal.example.com/ask
        method: POST
        headers:
          Authorization: env:RAGPROOF_TARGET_API_KEY
        body:
          question: "{question}"
      answer_path: answer
      citations_path: citations[*].chunk_id

The HTTP adapter retries on 429 and 5xx responses with exponential backoff, honours a Retry-After header, and fails fast on 4xx client errors. A full example with both endpoints is in examples/http_adapter_config.yaml.

Source id alignment

Retrieval metrics compare retrieved ids against each case's expected_source_ids. If your pipeline returns chunk ids that do not match the ids in your dataset, set run.source_match: document and expose a document_id in each retrieved chunk's metadata so the two can be lined up at the document level.