# AutoAgent Optimization Journey Chart

> Step chart tracking benchmark gains across autonomous harness-tuning experiments

This diagram documents the iterative progression of AutoAgent across 32 autonomous experiments on SpreadsheetBench and TerminalBench. It maps which specific interventions, prompt updates, and tool additions produced performance jumps versus which changes were discarded. It serves as a practical reference for designing autonomous agent self-improvement loops and debugging benchmarks.

## The framework

Spreadsheet path: baseline ~39% → structured reasoning workflow (+ex...) → ~83% → compute... → write scripts to files + verify fu... → verify with data\_only=... → strengthen formula-vs-value + mand... → ~88% → retry with exponential backoff for... → bonus verification phase → reasoning effort high → parallel to... → truncation=auto → pre-scan file listing + 50K trunc... → NO\_OUTPUT → actionable error hints in Shell to... → shell time... → re-run of 0.962 code → ~97%. Terminal path: baseline ~27% → detailed prompt + reasoning high → prompt\_cache\_retention=24h → truncation=auto → bonus verification phase (15 tus...) → added read\_file + write\_file tools → ~55% plateau.

## What it is for

- Design iterative optimization loops for autonomous AI agent harnesses
- Identify high-impact harness interventions like prompt caching, verification phases, and error handling
- Analyze common plateau points when benchmarking LLMs on terminal and spreadsheet tasks

## Key points

- Tracks performance progression from baseline (~39% to ~97% on SpreadsheetBench, ~27% to ~55% on TerminalBench)
- Highlights successful tweaks including file verification, high reasoning effort, and retry backoffs
- Maps system-level improvements like truncation controls, file listing pre-scans, and actionable shell error hints
- Visualizes discarded experiments faintly against retained improvements

## Pricing

Free

## Details

- Type: Framework
- Category: [AI](https://xiry.app/topics/ai.md)
- Open source: yes
- Platforms: other
- For: AI researchers, benchmark evaluators, and LLM agent developers
- Tags: agents, benchmarking, prompt engineering, optimization, evaluation, spreadsheetbench

## Why it was saved

- [@kevingu, 2 April 2026](https://x.com/kevingu/status/2039843234760073341): Visualizes the autonomous harness-tuning loop and which interventions produced gains.

## Related

- [AutoAgent](https://xiry.app/resources/autoagent): Autonomous harness engineering framework that optimizes AI agent setups overnight (GitHub repo, Open source)
- [Building Effective Agents: Workflow and Agent Patterns](https://xiry.app/resources/building-effective-agents-workflow-and-agent-patterns): Architectural framework categorizing LLM workflows and autonomous agent patterns (Framework, Free)
- [Interpretable Context Methodology (ICM)](https://xiry.app/resources/interpretable-context-methodology-icm): Folder structure as agent architecture for multi-stage AI workflows (GitHub repo, Open source)
- [Verify Agent Browser Actions with Snapshot Diffs](https://xiry.app/resources/verify-agent-browser-actions-with-snapshot-diffs): Verify browser agent actions by diffing DOM snapshots to cut token usage. (Tip, Free)
- [Prompt Engineering (Google Whitepaper)](https://xiry.app/resources/prompt-engineering-google-whitepaper-2086): A comprehensive guide to prompt engineering techniques and LLM parameters by Google. (Article or course, Free)
- [Hermes 3](https://xiry.app/resources/hermes-3): Open-weights fine-tune of Llama 3.1 built for agentic function calling. (AI model, Open source)

Updated 11 October 2026. Page: https://xiry.app/resources/autoagent-optimization-journey-chart

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.