# AutoHarness

> Framework for LLMs to automatically synthesize their own execution harnesses

AutoHarness is a research framework by Google DeepMind that allows LLMs to automatically write and refine their own code harnesses using iterative tree search and environment feedback. Instead of relying on hand-crafted scaffolding or costly fine-tuning, the agent generates external code (such as action verifiers or complete policies) to prevent illegal actions. In benchmarks across 145 TextArena games, it eliminated illegal moves and enabled smaller models like Gemini-2.5-Flash to outperform larger models.

Website: https://arxiv.org/html/2603.03329v1

## What it is for

- Prevent LLM agents from submitting invalid or illegal actions in constrained environments
- Synthesize executable Python control harnesses without manually writing scaffolding
- Generate standalone Python code policies to eliminate LLM inference costs at test time

## Features

- Thompson sampling-guided tree search over generated program candidates
- Critic and Refiner feedback loop using environment rollout errors
- Harness-as-action-verifier mode acting as a learned rejection sampler
- Harness-as-policy mode generating pure Python code to run inference-free
- 100% legal action success rate across 145 TextArena games

## Pricing

Free

## Details

- Type: Article or course
- Category: [AI](https://xiry.app/topics/ai.md)
- Open source: no
- Platforms: Web
- For: AI researchers and agent developers building tool-use workflows and game-playing systems
- Alternative to: Voyager, Eureka, Reflexion
- Tags: research, agents, code generation, deepmind, llm scaffolding

## Why it was saved

- [@dair_ai, 7 March 2026](https://x.com/dair_ai/status/2030289894808166866): Introduces 'code as harness': instead of hand-crafting the scaffolding (tools, code execution, file systems, APIs) around an LLM agent, the agent itself generates its harness via iterative code refinement with environment feedback. On TextArena games, Gemini-2.5-Flash with an auto-synthesized harness prevents all illegal moves across 145 games and outperforms larger models like Gemini-2.5-Pro; pushed further, the model generates the entire policy in code, beating Gemini-2.5-Pro and GPT-5.2-High on 16 one-player games, cheaper than a larger model.

## Related

- [Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned](https://xiry.app/resources/building-effective-ai-coding-agents-for-the-terminal-scaffol): A research paper formalizing architecture and context engineering for CLI coding agents. (Article or course, Free)
- [Hyperagent](https://xiry.app/resources/hyperagent): Fleet of autonomous AI agents that research, build, and deliver work (Web app)
- [Manus](https://xiry.app/resources/manus): Autonomous AI agent that executes complex research, coding, and workflow tasks (Web app, Freemium)
- [Google DeepMind Veo](https://xiry.app/resources/google-deepmind-veo): Generative AI video model producing cinematic footage with native audio (AI model, Freemium)
- [Gemini Notebook](https://xiry.app/resources/gemini-notebook): AI research assistant grounded strictly in your uploaded notes and documents. (Web app, Freemium)
- [Hermes 3](https://xiry.app/resources/hermes-3): Open-weights fine-tune of Llama 3.1 built for agentic function calling. (AI model, Open source)

Updated 10 October 2026. Page: https://xiry.app/resources/autoharness

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.