Vector search diagnostics for RAG systems. Evaluate embedding quality, analyze retrieval failures, benchmark semantic vs BM25, profile reranker performance.
PyVectorHound is a proprietary, production-grade diagnostics toolkit for vector search in RAG systems. Identify root causes of retrieval failures with precision.
The Problem:
- Vector search misses relevant documents
- No visibility into why queries fail
- Embedding quality varies wildly
- Reranker performance is a black box
The Solution:
- Embedding quality scoring
- Retrieval failure root cause analysis
- Semantic vs BM25 benchmarking
- Reranker performance profiling
- Actionable recommendations
Result: Debug retrieval issues in minutes, not days. Improve RAG quality 30-50%.
pip install pyvectorhound
# or with uv
uv pip install pyvectorhound- Python 3.10+
- Precompiled wheels
Proprietary-first distribution:
- ✅ Wheels-only via PyPI (no source code)
- ✅ Production-optimized diagnostics
- ✅ 142 comprehensive tests
- ✅ Used in production RAG systems
from pyvectorhound import RAGDiagnostics
# Initialize diagnostics
diagnostics = RAGDiagnostics()
# Analyze retrieval quality
query = "How do I configure OAuth?"
results = vector_search(query, top_k=10)
analysis = diagnostics.analyze(
query=query,
retrieved=results,
relevant_docs=[doc1, doc2, doc3], # Known relevant docs
)
print(f"Retrieval quality: {analysis.quality_score:.2%}")
print(f"Missing relevant docs: {analysis.missed_documents}")
print(f"Irrelevant results: {analysis.false_positives}")
# Benchmark embedding models
comparison = diagnostics.compare_embeddings(
models=['text-embedding-3-small', 'bge-large-en-v1.5'],
queries=test_queries,
)
print(f"Best model: {comparison.best_model} (score: {comparison.best_score:.2%})")- Embedding Quality Scoring: Is your embedding model good enough?
- Retrieval Failure Analysis: Why did the search miss relevant docs?
- Benchmark Embeddings: Compare embedding models objectively
- Semantic vs BM25: Understand tradeoffs
- Reranker Profiling: How much does reranking help?
- Actionable Recommendations: Concrete next steps
- Analysis time: <1s per query
- Benchmark: <10s per embedding model
- Comprehensive: Tests across multiple dimensions
- 142 tests passing
- Production-grade — production RAG systems
- Accurate — validated against manual analysis
For production deployments: mullassery@gmail.com
Version: 1.1.1
License: Proprietary
Distribution: Wheels-only via PyPI
Python: 3.10+
Built for robust RAG system diagnostics.