RAG Retrieval and Evaluation Experiment

A small notebook-exported Python experiment that embeds an Arabic Q&A example, retrieves it from ChromaDB, generates a response with DialoGPT and evaluates the response using DeepEval.

What the current code contains

untitled21.py uses SentenceTransformers (all-MiniLM-L6-v2), ChromaDB and Transformers (microsoft/DialoGPT-medium). It also illustrates precision@k and recall@k on a toy list of document IDs, then runs DeepEval contextual and answer-quality metrics.

The repository name reflects a broader code-assistant idea. The committed file currently has no LangGraph graph, programming-language detector or dedicated code-generation workflow. For the implemented graph-based assistant, see nlp-code-assistant.

Setup

pip install sentence-transformers chromadb transformers torch deepeval
python untitled21.py

The script downloads models, asks for a query and prints a generated response. The DeepEval stage requires OPENAI_API_KEY in the environment or at the hidden prompt, and may incur provider usage charges. Do not place credentials in the source file.

Limitations

• Retrieval contains one example, so it does not demonstrate performance on a representative corpus.
• The toy precision/recall calculation is separate from the live retrieval output.
• DialoGPT and the Arabic prompt need task/language evaluation; answer quality is not established by this example.
• DeepEval settings and dependencies are not pinned. No reproducible benchmark or deployment is included.