You’ll Finally Understand AI Evals After Watching This

What's the difference between an eval and a test? By the end of this video, you'll know the difference with confidence.

📚 RESOURCES

- Mastra docs: https://mastra.ai/docs/

💬 JOIN THE MASTRA COMMUNITY

- https://discord.gg/mastra
- https://twitter.com/mastra
- https://www.linkedin.com/company/mastra-ai

🟣 WHAT IS MASTRA?

- Mastra is a TypeScript framework and platform for building AI agents and workflows. Start locally with the open-source framework, then ship to production with the Mastra platform — including hosting, observability, evals, and more. Everything is built on the same framework, so you can prototype fast and ship with confidence.
- https://mastra.ai

⏱️ TIMESTAMPS

- 00:00 Introduction
- 01:15 Manual vs. automatic evals
- 01:40 What is a scorer?
- 02:51 Code-based scorer
- 04:00 LLM-as-Judge scorer
- 05:08 Code behind the scorers
- 08:11 Offline scorers
- 12:00 Online scorers
- 15:10 Advanced ideas