Governable RAG

A minimal retrieval-augmented generation pipeline where Chonkie cleans up the retrieval side with intelligent chunking and instructor guarantees a predictable, typed shape on the generation side. It runs natively on Windows against Azure OpenAI (or standard OpenAI).

"Chonkie cleans up the retrieval side, instructor guarantees a predictable shape — together a governable RAG pipeline that runs natively on Windows and Azure OpenAI. Tools that make AI auditable, not just fast."

The concept

Most RAG breaks in two places: messy chunks poison retrieval, and free-text generation drifts into unverifiable claims. This pipeline governs both ends.

document ─▶ Chonkie chunk ─▶ embed ─▶ cosine top-k retrieve ─▶ instructor generate ─▶ GovernedAnswer

• Chonkie (RecursiveChunker) splits the document along natural boundaries while respecting a token budget, so each chunk is semantically coherent instead of cut mid-thought. Clean chunks → cleaner retrieval.
• instructor patches the OpenAI/Azure client so generation must return a validated Pydantic GovernedAnswer: an answer, an answer_found flag, a bounded confidence, and chunk-level citations. The model is instructed to answer only from retrieved context and to flag when context is insufficient. The shape is guaranteed — downstream code can trust the fields.

That guaranteed, cited, grounded contract is what makes the pipeline auditable, not just fast.

Layout

gov_rag/
  chunking.py   # Chonkie RecursiveChunker wrapper -> DocumentChunk list
  store.py      # pure-Python in-memory cosine vector store
  schema.py     # GovernedAnswer / Citation — the typed output contract
  clients.py    # Azure/OpenAI client + embeddings, all from env vars
  pipeline.py   # GovernableRAG: index -> retrieve -> typed answer
main.py          # runnable end-to-end demo
sample_doc.txt   # sample knowledge source
test_store.py    # offline tests (no key, no network)
.env.example     # the environment variables to set

Setup

python -m venv .venv
.venv\Scripts\activate            # Windows
# source .venv/bin/activate       # macOS/Linux
pip install -r requirements.txt

Configure your provider via environment variables — keys are always read from the environment, never hardcoded. Copy .env.example and fill it in, or export the variables directly.

Azure OpenAI (auto-detected when AZURE_OPENAI_ENDPOINT is set)

$env:AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
$env:AZURE_OPENAI_API_KEY="your-azure-api-key"
$env:AZURE_OPENAI_API_VERSION="2024-10-21"
$env:AZURE_OPENAI_CHAT_DEPLOYMENT="your-chat-deployment-name"
$env:AZURE_OPENAI_EMBEDDING_DEPLOYMENT="your-embedding-deployment-name"

Note: on Azure the model name is your deployment name, not the base model id. You need two deployments — one chat model and one embedding model.

Standard OpenAI (used when AZURE_OPENAI_ENDPOINT is not set)

export OPENAI_API_KEY=sk-...
export OPENAI_CHAT_MODEL=gpt-4o-mini
export OPENAI_EMBEDDING_MODEL=text-embedding-3-small

Run

python main.py "What governance rules does Manifold enforce?"

The script chunks sample_doc.txt, embeds and indexes it, retrieves the top matches, and prints a typed GovernedAnswer with citations and a confidence score.

Test

The chunking-independent pieces — the cosine vector store and the typed schema — are covered by an offline suite that needs no API key:

pytest

License

MIT — see LICENSE.