Skip to content

Add example: rag-knowledge-assistant with source-citation policies #54

@kmadan

Description

@kmadan

Goal

Create examples/rag-knowledge-assistant/ showing a retrieval-augmented assistant evaluated for citation accuracy, hallucination, source disclosure, and data-leakage.

Why this matters

RAG is the most common production LLM pattern. Compliance for RAG is its own discipline — citation accuracy, source-confidentiality, hallucination detection — and a documented example shows AICertify users how to encode those concerns.

Files likely involved

  • New: examples/rag-knowledge-assistant/ (six-file shape) over a small fixture corpus
  • examples/README.md table entry
  • Optionally: a small new gopal policy proposal for RAG-specific concerns (open in gopal repo)

Acceptance criteria

  • Contract with 6-10 interactions including: correctly-cited answer, no-source-available answer, hallucination test, sensitive-doc query
  • policy_config.yaml that exercises content-safety + transparency + accountability
  • Retrieved sources in interaction metadata
  • expected_report.md describes the citation-accuracy pass case and the common failure modes (uncited claim, fabricated source)
  • Honest scope: RAG eval is corpus-dependent — a green report on one corpus is not transferable

Help

Comment here if you want to work on this and I will help you get started. First-time contributors are welcome — open a draft PR early and I'll review.

Metadata

Metadata

Assignees

No one assigned

    Labels

    🏗️ help wantedExtra attention is needed📦 examplesApplication examples and templates🦜 llm-appsLangChain / LlamaIndex / agent-framework integrations

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions