Add Practical Lessons for GenAI Evals with Chip Huyen and Vivienne Zhang

[conorbronsdon]

Summary

Add a Chain of Thought Podcast panel with Chip Huyen and Vivienne Zhang to the podcast section. The episode gives practitioners concrete ways to evaluate agent plans and tool use.

Why it belongs

This is an eval-first episode with a public transcript. At 28:00, Huyen explains why generic labels such as coherence and relevance are not stable metrics without clear criteria, fixed judge prompts, and curated examples of good and bad responses. At 39:00, she breaks agent evaluation into task completion under constraints, valid tool selection and parameters, plan efficiency, and metrics for the application's likely failure points. Zhang then discusses human handoffs at 42:00 and applying evals at the failure points in a customer-service workflow.

Disclosure

I host Chain of Thought and am submitting my own episode. At the time of recording, I was Head of Developer Awareness at Galileo. Galileo sponsored the episode; its promotional read begins at 20:00. The panel was recorded at Galileo's Productionize 2.0 event and was moderated by Galileo co-founder and CTO Atindriyo Sanyal.

Verification

• Confirmed the canonical episode URL returns HTTP 200.
• Checked the public transcript against the annotation and PR description.
• Confirmed the episode was published in 2024, so the new-resource marker is omitted.
• Checked the existing list for this episode and found no duplicate.
• Used the repository's podcast entry format and added one resource to README.md.