Description
The examples documentation page (\docs/examples.md) could be expanded with real-world workflow examples that users can copy-paste to get started quickly.
What to Add
Add short, focused code examples showing:
- Quick eval from Python — using the Python SDK directly
- Compare two models — running evaluations with GPT-4o vs Claude and comparing results
- Custom metric — creating and registering a simple custom metric
- CI/CD integration — running eval in a GitHub Action and checking thresholds
- Batch evaluation — running multiple configs programmatically
Files to Look At
- \docs/examples.md\ — where to add
- \examples/rag_evaluation_tutorial.ipynb\ — existing notebook for reference
- \openagent_eval/core/engine.py\ — for accurate API examples
Good First Issue ✅
A documentation task that teaches you the core APIs.
Description
The examples documentation page (\docs/examples.md) could be expanded with real-world workflow examples that users can copy-paste to get started quickly.
What to Add
Add short, focused code examples showing:
Files to Look At
Good First Issue ✅
A documentation task that teaches you the core APIs.