# You Don't Understand AI Agents

> A framework prioritizing end-user business processes and evals over agent architectures.

An essay and mental model by Wayne Hamadi reframing AI agents as systems that act autonomously and proactively by mirroring a specific human worker's reward system. It argues that agent builders waste time on complex multi-agent frameworks, vector databases, and memory architectures instead of deeply understanding the business process and designing granular evals.

## The framework

Author: Wayne Hamadi (@wayne\_hamadi). Link: x.com/i/article/1867032485768597505.

KEY THESIS: An AI agent is 'a system that can act autonomously and proactively.' The technology (RAG, memory, multi-agents, loops, LLMs) is the least interesting part. The most interesting part is 'Anna' — an end user following a business process. Anna already has agency; the agent is just a bot version of Anna, 'a machine with the same reward system as Anna.' Building good agents means writing evals that force you to understand Anna better than anyone else.

HISTORY:

- Pre-ChatGPT: traditional apps with explicit user interfaces; automation done by engineers or low-code workflow engines (Zapier, MuleSoft, n8n, Windmill) that remained too technical for most knowledge workers.

- Nov 2022–Apr 2023: text-davinci-003 was 'an auto-completer'; GPT-3.5 answered reliably, enabling early self-prompting experiments but not sustained loops.

- March 14, 2023: GPT-4 0314 was a step change — 'the best model until Claude Sonnet 3.5 in June 2024'; excellent at self-prompting, which is what made agentic software possible.

- Toran Bruce Richards (a game developer, used to 'while true' loops) built AutoGPT days later — the first AI agent, which the author calls 'chat with side effects.' All impressive AutoGPT demos came from GPT-4 (most people lacked API access).

WHAT NOT TO FOCUS ON (crossed-out diagram): multi-agent architectures with shared memory, digital business context (vector stores, knowledge graphs, structured/unstructured data), physical environment context.

THE ANNA EXAMPLE: Anna is a Coca-Cola recruiter whose ATS only sends junior applicants for a senior PM role. With an agentic ATS she states the high-level problem; the ATS troubleshoots and returns a diagnostic. The design questions are three evals:

- A (incident): if \>40% of applicants for a job titled 'senior' are junior, report an incident.

- B (diagnostic accuracy): if the diagnostic claims a mismatch between two fields, verify the mismatch actually exists.

- C (fix): for a field-mismatch diagnostic, the agent should alert Anna and keep alerting unless she declines — this is where the agent goes in the loop.

Lesson on evals vs benchmarks: eval C ranks lower — passing A and B beats A and C, because Anna can trivially edit the job description herself. Evals also rank/prioritize what's worth automating.

DIFFERENCE BETWEEN EVAL AND BENCHMARK: evals are the individual pass/fail checks; the benchmark is the ranked combination you optimize the agent against.

## What it is for

- Design practical AI agents by modeling existing human workflows instead of technical architectures
- Define and prioritize agent evaluation criteria (evals) before writing automation loops
- Identify where human-in-the-loop oversight is necessary versus automatable

## Key points

- Core thesis that an agent is a software proxy of an existing knowledge worker (the 'Anna' persona) with identical reward incentives
- Argument de-emphasizing multi-agent frameworks, shared memory, and vector databases in favor of domain understanding
- Distinction between individual pass/fail evals and ranked benchmarks used for optimization
- Three-tier evaluation hierarchy: problem detection (incident), diagnostic accuracy, and autonomous resolution

## Pricing

Free

## Details

- Type: Framework
- Category: [AI](https://xiry.app/topics/ai.md)
- Topic: [Design Systems](https://xiry.app/topics/design/design-systems.md)
- Open source: no
- Platforms: Web
- For: AI engineers, product managers, and founders building autonomous agent systems
- Tags: ai agents, evals, mental model, system design, product management

## Why it was saved

- [@wayne_hamadi, 16 December 2024](https://x.com/wayne_hamadi/status/1868742755402621103): A mental model for designing agents: focus on the user's business process and evals, not frameworks or architectures.

## Related

- [Awesome DESIGN.md](https://xiry.app/resources/awesome-design-md): Curated collection of plain-text design systems extracted from popular tech websites. (GitHub repo, Open source)
- [Minimal Product Design Cheatsheet](https://xiry.app/resources/minimal-product-design-cheatsheet): Quick typography, color palette, and iconography reference for clean product interfaces. (Tip, Free)
- [Modulor](https://xiry.app/resources/modulor): Open-source, token-driven design system and Figma UI kit by GR8 Tech. (Website, Open source)
- [Refero Styles](https://xiry.app/resources/refero-styles): AI-readable DESIGN.md style references extracted from popular websites. (Web app, Free)
- [Building Effective Agents: Workflow and Agent Patterns](https://xiry.app/resources/building-effective-agents-workflow-and-agent-patterns): Architectural framework categorizing LLM workflows and autonomous agent patterns (Framework, Free)
- [Checklist Design](https://xiry.app/resources/checklist-design): Curated UI and UX checklists for websites, apps, and user flows (Website, Free)

Updated 11 October 2026. Page: https://xiry.app/resources/you-don-t-understand-ai-agents

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.