Learning · Article or course
How to Setup Evals For Agents
Free Maven video lesson on evaluating and benchmarking AI agents
This is a free 30-minute recorded video lesson on Maven led by Harrison Chase (LangChain) and Hamel Husain. It focuses on practical frameworks and tooling for evaluating autonomous and semi-autonomous AI agents. The session covers defining metrics, evaluating trajectories, and using tools like LangSmith and OpenEvalz to measure and improve agent reliability.
What it's for
- Define concrete evaluation metrics and benchmarks for AI agents
- Inspect and evaluate multi-turn agent conversation trajectories
- Use LangSmith trace visualizations and human feedback queues to iterate on LLM systems
Features
- Covers core data types and frameworks for evaluating non-deterministic agents
- Compares testing frameworks and tooling like LangSmith and OpenEvalz
- Walkthrough of trace visualization, state tracking, and annotation queues for human feedback
- Techniques for red-teaming agents and simulating multi-turn conversations
- 30-minute on-demand video recording available upon free sign-up
Pricing
Free to watch with a Maven account
Details
- Type
- Article or course
- Category
- Learning
- Topic
- AI Courses
- Open source
- No
- Platforms
- Web
- For
- AI engineers, ML developers, and technical product managers building AI agents
- Tags
- ai agents, evals, langchain, langsmith, machine learning, free course
Why it was saved
- @mavenhq: Teaches how to measure, align, and systematically improve agent performance over time from the creators of LangChain.
Related
Anthropic's official online learning portal for Claude and AI development
Free 30-minute video lesson on production RAG architecture, security, and scaling.
Free workshop on architecting enterprise-grade AI agent systems.
Free 45-minute lesson on foundational building blocks for AI agents.
Free video lecture on modern IR benchmarks and evaluation for RAG pipelines.
A free 30-minute workshop comparing RAG and fine-tuning architectures.
Updated