Verify that an AI agent's on-chain claims are actually true. A claim-level groundedness checker for Solana / memecoin agents.
Grounded multi-domain support-triage agent (HackerRank Orchestrate Hackathon). 8-layer DAG fusing ChromaDB vector + BM25 lexical retrieval, with an adversarial-critic pass that blocks ungrounded answers — 93% groundedness across a 772-document corpus.
Single-agent, evidence-grounded claim verification to catch LLM hallucinations — a pluggable fact-gate for agent-arena and any multi-agent system (CrewAI, AutoGen, LangGraph).
Provider-agnostic enterprise RAG and agent evaluation harness for Azure Foundry, vLLM, Ollama, and local demos.
Evaluation patterns, release gates, and anti-hallucination techniques for developer-focused AI workflows.
Measuring Groundedness in RAG over Financial Filings
pip install gauntlet-evals · v0.1.0. Merge-blocking evaluation gates for generative AI features: YAML suites run against any HTTP endpoint or Python callable, fail the build on a miss, and emit both a diffable JSON pack and a reviewer document cross-referenced to California's published GenAI risk framework. Aligned to, never approved by.
v0.2.0. Fail-closed evaluation harness for government-facing chat systems: reproducible, provenance-stamped audit verdicts, byte-identical across Python 3.11 to 3.14, with no third-party dependencies. A silent or unreadable target scores zero rather than passing by absence. Two public projects of my own pin it by exact commit.
In-build reference implementation: an offline-first plant-care assistant and public evaluation harness with cited-corpus answers, calibrated abstention, toxicity guardrails, EN/ES parity, photo plant ID, and local reminders.
Beta. Reduced-fare policy assistant citing dated corpus passages in English and Spanish; the bilingual-parity gate is currently failing (see EVALS.md). Corpus of eighteen California transit agencies, public 385-case evaluation harness. Deployed demo serves five agencies; published evidence run lags the repository.
Scores whether an AI agent's answer is actually supported by the tool results it had access to. An LLM-as-judge harness for catching ungrounded or fabricated claims in agent outputs.
Use Jev instead of an LLM for RAG decisions: relevance grading, query routing, hallucination checks and LLM-as-a-judge evals. Faster, cheaper, measured first.
A local groundedness judge for RAG: QLoRA-distilled to match a frontier judge 100% at $0/call. Ships only if its own evals beat baseline.
Measure whether a RAG system is actually grounded, and fail the build when it stops being: retrieval metrics, claim-level groundedness, hallucination rate and citation accuracy.
An AI-powered RAG groundedness / hallucination checker - segments an LLM answer into atomic claims and flags those NOT supported by the retrieved context, via an LLM judge. Provider-agnostic, BYO-key, offline --dry-run. Python CLI.
RAG evaluation workbench for retrieval recall, citation coverage, groundedness checks, and failure analysis
Deterministic citation and claim-support checks for RAG evaluation datasets.
Your RAG, put to proof — self-correcting answers and CI eval gates that block the merge when hallucination rate crosses your threshold
Groundedness & hallucination checker for LLM/RAG answers: decomposes an answer into atomic claims, verifies each against its sources (LLM judge or offline NLI), and highlights what isn't supported.
RAG that grades every answer: groundedness, helpfulness, accuracy + citations. FastAPI + live browser demo
To associate your repository with the groundedness topic, visit your repo's landing page and select "manage topics."