# AutoAgent

> Autonomous harness engineering framework that optimizes AI agent setups overnight

AutoAgent is an open source library that uses a meta-agent loop to autonomously optimize a task agent's harness. Instead of manually tweaking prompts and orchestration, a human specifies goals in program.md while the meta-agent modifies tools, prompts, and code in agent.py, running benchmark evaluations in Docker to keep what works.

Website: https://github.com/kevinrgu/autoagent

## What it is for

- Optimize an AI agent harness autonomously against custom task benchmarks
- Hill-climb benchmark scores by testing prompt, tool, and routing variations automatically
- Evaluate agent failure traces in parallel Docker sandboxes to debug task performance

## Features

- Meta-agent feedback loop that modifies agent code, runs evals, and keeps improvements
- Single-file harness architecture (agent.py) isolating editable logic from adapters
- Directive-driven control via program.md without manual code changes
- Harbor benchmark compatibility for standardized task definitions and verification
- Docker sandboxing for isolated, high-concurrency evaluation runs
- Support for external context engineering skills and tool integrations

## Pricing

Open source. Free (MIT license)

## Details

- Type: GitHub repo
- Category: [AI](https://xiry.app/topics/ai.md)
- Open source: yes
- Platforms: Command line, Linux, macOS
- For: AI engineers and researchers building and optimizing autonomous agents
- Alternative to: DSPy, autoresearch
- Tags: ai agents, agent evaluation, meta agent, benchmarking, prompt engineering, python

## Why it was saved

- [@kevingu, 2 April 2026](https://x.com/kevingu/status/2039843234760073341): First open source evidence that an agent can autonomously beat manual harness tuning: \#1 SpreadsheetBench (96.5%), \#1 GPT-5 TerminalBench score (55.1%) after 24h of self-optimization.

## Related

- [ai-engineering-hub](https://xiry.app/resources/ai-engineering-hub): Open-source tutorials and project templates for LLMs, RAG, and AI agents. (GitHub repo, Open source)
- [AutoAgent Optimization Journey Chart](https://xiry.app/resources/autoagent-optimization-journey-chart): Step chart tracking benchmark gains across autonomous harness-tuning experiments (Framework, Free)
- [Building a Style-Mimicking News Bot with Grok API](https://xiry.app/resources/building-a-style-mimicking-news-bot-with-grok-api): Recipe for scraping news URLs and rewriting them in your voice using Grok (Tip, Free)
- [Self-Improving Agent Loop via skill.md](https://xiry.app/resources/self-improving-agent-loop-via-skill-md): A workflow pattern where agents document completed tasks into reusable shortcuts (Tip, Free)
- [system\_prompts\_leaks](https://xiry.app/resources/system-prompts-leaks): Curated collection of leaked system prompts from major AI assistants and models. (GitHub repo, Open source)
- [Google Opal](https://xiry.app/resources/google-opal): No-code visual canvas for building AI mini-apps from plain English prompts (Web app, Free)

Updated 10 October 2026. Page: https://xiry.app/resources/autoagent

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.