Try out an open-source model

[kcarnold]

Definitely cheaper, potentially much faster.

Maybe: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

might depend on #3

[kcarnold]

Potentially useful tool: typpo/promptfoo: Test your prompts. Evaluate and compare LLM outputs, catch regressions, and improve prompt quality.