[tracing] Add DeepEval tracing to the deep-research-agent#31
Draft
deepeval-tracer[bot] wants to merge 1 commit into
Draft
[tracing] Add DeepEval tracing to the deep-research-agent#31deepeval-tracer[bot] wants to merge 1 commit into
deepeval-tracer[bot] wants to merge 1 commit into
Conversation
Instrument the LangGraph/deepagents research agent with DeepEval's native LangChain CallbackHandler so each graph run, sub-agent, tool call, and LLM call becomes an inspectable span in Confident AI's Observatory. - Pass CallbackHandler(...) into agent.stream via config callbacks (native LangGraph integration — no app rewrite). - Add deepeval dependency (pinned to match the repo's existing usage). - Document CONFIDENT_API_KEY in env.example. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds DeepEval tracing to the
langchain/deep-research-agentapp so its execution becomes visible span-by-span in Confident AI's Observatory.This is a genuine agentic app — a LangGraph/
deepagentsdeep research agent with a delegating orchestrator, research sub-agents, atavily_searchweb-search tool, and an Ollama LLM — which makes it a strong showcase for tracing (agent → sub-agent → tool → LLM spans).How it was instrumented
Native integration, not manual
@observe. Because the app is built on LangGraph (viadeepagents.create_deep_agent), the recommended DeepEval surface is the LangChainCallbackHandler. It is passed into the existingagent.stream(...)call throughconfig={"callbacks": [...]}— every graph run, sub-agent, tool call, and LLM call is captured automatically, with no rewrite of the agent code.Changes (all scoped to
langchain/deep-research-agent/):agent.py— importCallbackHandlerfromdeepeval.integrations.langchainand pass it intoagent.stream(...)with a tracenameand groupingtags. No secrets are captured.pyproject.toml— add thedeepevaldependency (pinned>=4.1.2to match the repo's existingdeepeval/evalsproject). You may want to runuv lockto refreshuv.lock.env.example— documentCONFIDENT_API_KEYalongside the existing keys.To start seeing traces
Set
CONFIDENT_API_KEYin your environment (or rundeepeval login). DeepEval reads it from the environment automatically — see the new entry inenv.example. Without the key the app runs exactly as before; tracing simply stays local/no-op.Notes
config={"callbacks": [CallbackHandler()]}pattern applies to the other LangGraph/deepagentsexamples in this repo (e.g.langchain/internet-search) if you'd like to extend tracing to them.python -m py_compileon the edited module; the app itself was not executed (it requires Ollama + a Tavily key).