Frontier LLM DevOps & Infrastructure Benchmark Suite: Deterministic sandboxed evaluations covering Docker, Kubernetes, Terraform, and CI/CD.
-
Updated
Nov 1, 2026 - Python
#
llm-evaluations
Here are 10 public repositories matching this topic...
Red-team harness and live console for Google ADK agent memory: attacks an unprotected pgvector store and adk-memshield side by side, query by query.
python portfolio postgres multi-tenant postgresql multi-tenancy gemma ai-agents row-level-security red-teaming ai-security fastapi pgvector ollama llm-evaluation agent-memory google-adk google-adk-framework llm-evaluations
-
Updated
Oct 26, 2026 - Python
A hyperlocal job board purpose-built for India's Tier-2/3 cities — employers post geotagged jobs, candidates upload resumes that an LLM auto-parses into skill vectors, Elasticsearch matches candidates to nearby jobs within a configurable radius, and job alerts arrive on WhatsApp in the candidate's regional language.
elasticsearch job-board job-portal job-search whatsapp-api geotagging full-stack-web-development llm-evals llm-evaluations
-
Updated
Oct 20, 2026 - Java
Profiles LLM reasoning by turning chain-of-thought into thought graphs, with no generative calls. 22 structural metrics. arXiv:2606.29067
benchmark graph-analysis interpretability llm llms chain-of-thought chain-of-thought-reasoning llm-evaluation llm-evaluation-toolkit llm-evaluation-framework llm-evaluation-metrics llm-evaluation-benchmark llm-evaluations thought-graphs
-
Updated
Oct 6, 2026 - Python
Evaluation suite for a banking RAG assistant — faithfulness, hallucination & relevance scoring with CI regression gates (DeepEval · RAGAS · pytest)
ai-evaluation faithfulness retrieval-augmented-generation llm-evaluation rag-evaluation hallucination-detection ragas ai-evaluation-tools deepeval relevance-scoring ai-evaluation-framework banking-ai llm-evaluations retrieval-augmented-generation-for-business
-
Updated
Nov 2, 2026 - Python
Production-ready enterprise RAG and structured data extraction pipeline using LangGraph, Instructor, and Pydantic with automated CI evaluation metrics.
docker ci-cd pytest data-extraction ci-cd-pipeline rag self-correction github-actions pydantic fastapi vector-search instructor vector-search-engine python3-11 retrieval-augmented-generation rag-pipeline structured-outputs pydantic-ai self-correction-ai llm-evaluations
-
Updated
Aug 28, 2026 - Python
benchmarking jailbreak-dataset fine-tuning-tools llm-evaluation official high-priority benchmarking task evaluation checkpoint shared memory sandbox API endpoints structured data schemas collaborative environments
benchmarking ai-safety llm-evaluation llm-evaluation-toolkit llm-evaluation-framework llm-evaluation-metrics llm-evaluation-benchmark ai-safety-research llm-evaluations jailbreak-dataset fine-tuning-tools ai-safety-sandbox
-
Updated
Sep 9, 2026
Replication-first study of sociolinguistic effects on LLM epistemic judgments, with behavioral validity gates before mechanistic interpretation.
reproducibility gemma ai-safety interpretability sociolinguistics llm mechanistic-interpretability ai-safety-research behavioral-evaluation llm-evaluations
-
Updated
Sep 4, 2026 - Jupyter Notebook
-
Updated
Aug 24, 2026 - Python
Defense-in-depth safety study on Mistral-7B: how much of an LLM's safety lives in the weights vs. external guardrails, and how fragile it is under fine-tuning. Defensive research.
python3 lora robustness alignement peft dpo guardrails defense-in-depth responsible-ai fine-tuning-llm llm-safety mistral-7b-instruct large-languge-models ai-safety-research llm-evaluations
-
Updated
Oct 21, 2026 - Jupyter Notebook
Add this topic to your repo
To associate your repository with the llm-evaluations topic, visit your repo's landing page and select "manage topics."