Add Convai Evals analytics drilldown recipe#19
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces integration between the analytics SDK and Convai Evals reports, adding comprehensive documentation, TypeScript examples for row-level drilldowns, and a new prompt recipe for AI-driven analysis. The review feedback identifies several opportunities to improve the robustness of the documentation's code examples by implementing defensive programming patterns, such as optional chaining and nullish coalescing, to safely handle potentially missing or malformed report data.
| const report = JSON.parse(await readFile("report.json", "utf8")); | ||
| const client = new ConvaiAnalytics({ apiKey: process.env.CONVAI_API_KEY }); | ||
|
|
||
| const slowRows = report.per_row |
There was a problem hiding this comment.
The code assumes report.per_row always exists. If the JSON report is malformed or from a different version, this will throw a TypeError. Providing a default empty array makes the example more robust.
| const slowRows = report.per_row | |
| const slowRows = (report.per_row ?? []) |
| const sessionId = row.backend?.session_id; | ||
| if (sessionId) { | ||
| const session = await client.sessions.get(sessionId); | ||
| console.log(row.test_id, session.events.length); |
There was a problem hiding this comment.
Accessing session.events.length without a null check on events can lead to runtime errors if the session data is incomplete. Using optional chaining is recommended for defensive programming.
| console.log(row.test_id, session.events.length); | |
| console.log(row.test_id, session.events?.length ?? 0); |
| Use row-level reports to find regressions, then use analytics for broader context: | ||
|
|
||
| ```ts | ||
| const characterId = report.run_metadata.characterId; |
There was a problem hiding this comment.
The run_metadata property might be missing in some report formats. Using optional chaining here prevents the script from crashing when trying to access characterId.
| const characterId = report.run_metadata.characterId; | |
| const characterId = report.run_metadata?.characterId; |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 207674e62c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const interactionId = row.backend?.character_session_id; | ||
| if (interactionId) { | ||
| const trace = await client.interactions.get(interactionId); |
There was a problem hiding this comment.
For rows produced by the Web/Connect APIs, character_session_id is the conversation-continuation/session identifier, not the analytics interaction id that /interactions/{interaction_id} expects; the actual interaction ids are exposed on session timeline events as interactionId before they can be passed to client.interactions.get(...). With the current recipe, reports that include only backend.character_session_id will drive agents to call the interaction endpoint with a session id and get missing/incorrect traces instead of first loading the session and selecting an event interaction id.
Useful? React with 👍 / 👎.
1 participant
Summary
Validation