# How to Setup Evals For Agents

> Free Maven video lesson on evaluating and benchmarking AI agents

This is a free 30-minute recorded video lesson on Maven led by Harrison Chase (LangChain) and Hamel Husain. It focuses on practical frameworks and tooling for evaluating autonomous and semi-autonomous AI agents. The session covers defining metrics, evaluating trajectories, and using tools like LangSmith and OpenEvalz to measure and improve agent reliability.

Website: https://maven.com/p/a58f3f/how-to-setup-evals-for-agents

## What it is for

- Define concrete evaluation metrics and benchmarks for AI agents
- Inspect and evaluate multi-turn agent conversation trajectories
- Use LangSmith trace visualizations and human feedback queues to iterate on LLM systems

## Features

- Covers core data types and frameworks for evaluating non-deterministic agents
- Compares testing frameworks and tooling like LangSmith and OpenEvalz
- Walkthrough of trace visualization, state tracking, and annotation queues for human feedback
- Techniques for red-teaming agents and simulating multi-turn conversations
- 30-minute on-demand video recording available upon free sign-up

## Pricing

Free to watch with a Maven account

## Details

- Type: Article or course
- Category: [Learning](https://xiry.app/topics/learning.md)
- Topic: [AI Courses](https://xiry.app/topics/learning/ai-courses.md)
- Open source: no
- Platforms: Web
- For: AI engineers, ML developers, and technical product managers building AI agents
- Tags: ai agents, evals, langchain, langsmith, machine learning, free course

## Why it was saved

- [@mavenhq, 15 February 2026](https://www.instagram.com/reel/DUzDXLsj5N1/): Teaches how to measure, align, and systematically improve agent performance over time from the creators of LangChain.

## Related

- [Anthropic Academy](https://xiry.app/resources/anthropic-academy): Anthropic's official online learning portal for Claude and AI development (Article or course, Free)
- [Building Enterprise RAG Solutions](https://xiry.app/resources/building-enterprise-rag-solutions): Free 30-minute video lesson on production RAG architecture, security, and scaling. (Article or course, Free)
- [Designing Agentic AI Systems for Enterprise Use-Cases](https://xiry.app/resources/designing-agentic-ai-systems-for-enterprise-use-cases): Free workshop on architecting enterprise-grade AI agent systems. (Article or course, Free)
- [Intro to Agentic AI – Build Your First AI Agent](https://xiry.app/resources/intro-to-agentic-ai-build-your-first-ai-agent): Free 45-minute lesson on foundational building blocks for AI agents. (Article or course, Free)
- [Modern Information Retrieval Evaluation In The RAG Era](https://xiry.app/resources/modern-information-retrieval-evaluation-in-the-rag-era): Free video lecture on modern IR benchmarks and evaluation for RAG pipelines. (Article or course, Free)
- [RAG and LLM Fine-Tuning](https://xiry.app/resources/rag-and-llm-fine-tuning): A free 30-minute workshop comparing RAG and fine-tuning architectures. (Article or course, Free)

Updated 3 October 2026. Page: https://xiry.app/resources/how-to-setup-evals-for-agents

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.