xiry.

AI · GitHub repo

AutoAgent

Autonomous harness engineering framework that optimizes AI agent setups overnight

From the Xiry library · Updated 10 October 2026

Visit github.com ↗

AutoAgent is an open source library that uses a meta-agent loop to autonomously optimize a task agent's harness. Instead of manually tweaking prompts and orchestration, a human specifies goals in program.md while the meta-agent modifies tools, prompts, and code in agent.py, running benchmark evaluations in Docker to keep what works.

What it's for

  • Optimize an AI agent harness autonomously against custom task benchmarks
  • Hill-climb benchmark scores by testing prompt, tool, and routing variations automatically
  • Evaluate agent failure traces in parallel Docker sandboxes to debug task performance

Features

  • Meta-agent feedback loop that modifies agent code, runs evals, and keeps improvements
  • Single-file harness architecture (agent.py) isolating editable logic from adapters
  • Directive-driven control via program.md without manual code changes
  • Harbor benchmark compatibility for standardized task definitions and verification
  • Docker sandboxing for isolated, high-concurrency evaluation runs
  • Support for external context engineering skills and tool integrations

Pricing

Open source. Free (MIT license)

Details

Type
GitHub repo
Category
AI
Topic
Agent Optimization
Open source
Yes
Platforms
Command line, Linux, macOS
For
AI engineers and researchers building and optimizing autonomous agents
Alternative to
DSPy, autoresearch
Tags
ai agents, agent evaluation, meta agent, benchmarking, prompt engineering, python

Why it was saved

  • @kevingu, 2 April 2026: First open source evidence that an agent can autonomously beat manual harness tuning: #1 SpreadsheetBench (96.5%), #1 GPT-5 TerminalBench score (55.1%) after 24h of self-optimization.

Related

Updated