Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR adds documentation improvements and video demonstrations comparing RAG evaluation frameworks.
Este PR adiciona melhorias na documentação e vídeos demonstrando a comparação entre frameworks de avaliação de RAG.
The goal is to make the project more explainable, visual, and accessible for learning and analysis.
O objetivo é tornar o projeto mais explicável, visual e acessível para aprendizado e análise.
What was added
Added two video demos to the README explaining the differences between RAGAS and DeepEval.
Adicionados dois vídeos no README explicando as diferenças entre RAGAS e DeepEval.
Videos cover:
Os vídeos abordam:
Comparison of evaluation metrics using the same dataset
Comparação das métricas usando o mesmo dataset
Differences between numeric scoring (RAGAS) and reasoning-based evaluation (DeepEval)
Diferença entre score numérico (RAGAS) e avaliação com reasoning (DeepEval)
Use of GEval for domain-specific evaluation
Uso do GEval para avaliação específica de domínio
Confident AI dashboard for experiment tracking and visualization
Dashboard do Confident AI para rastreabilidade e visualização de experimentos
Key Insight
RAGAS provides numeric scores but lacks explainability for debugging.
O RAGAS fornece scores numéricos, mas não oferece explicabilidade para debug.
DeepEval provides reasoning for each metric, enabling better identification of issues.
O DeepEval fornece reasoning para cada métrica, permitindo melhor identificação de problemas.
This includes identifying retrieval issues such as incorrect chunk ranking.
Isso inclui identificar problemas de recuperação, como ranking incorreto de chunks.
Impact
Improves project documentation and makes evaluation results easier to understand.
Melhora a documentação do projeto e torna os resultados mais fáceis de entender.
Adds visual learning resources for users and recruiters.
Adiciona recursos visuais de aprendizado para usuários e recrutadores.
Strengthens the project as a portfolio demonstrating real-world RAG evaluation practices.
Fortalece o projeto como portfólio demonstrando práticas reais de avaliação de RAG.
No changes to core pipeline or evaluation logic.
Nenhuma alteração no pipeline ou lógica de avaliação.
Changes are limited to documentation (README).
As mudanças são limitadas à documentação (README).
How to review
Open the README and watch the embedded videos.
Abra o README e assista aos vídeos adicionados.
Evaluate clarity of explanations and documentation improvements.
Avalie a clareza das explicações e melhorias na documentação.