# Agent S

> Open-source computer use agent operating GUI interfaces via mouse and keyboard

Agent S is an open-source agentic framework that allows multimodal AI models to operate desktop and web software like a human user. By interpreting screen visuals and generating mouse, keyboard, and scroll actions, it completes multi-step tasks across apps without custom APIs. Its third-generation version (Agent S3) and related hosted agent Sai lead benchmarks like OSWorld.

Website: https://github.com/simular-ai/agent-s

## What it is for

- Automate complex multi-step workflows across desktop and browser apps
- Benchmark and evaluate computer-use agents on OSWorld and WindowsAgentArena
- Conduct autonomous web research, trip planning, and price comparisons
- Execute repetitive cross-application data entry without dedicated APIs

## Features

- GUI interaction via mouse movements, clicks, keystrokes, and scrolling
- Direct visual perception of desktop and browser screens
- Multi-OS support across macOS, Windows, and Linux
- Compatibility with multiple model providers including OpenAI, Anthropic, and open-weight models
- Distribution via the gui-agents Python package
- Autonomous planning and step-by-step reasoning sidebar

## Pricing

Open source. Free and open-source under Apache-2.0; users pay for their own LLM API tokens. Hosted version Sai available separately.

## Details

- Type: Desktop app
- Category: [AI](https://xiry.app/topics/ai.md)
- Open source: yes
- Platforms: macOS, Windows, Linux, Code library
- For: Developers and AI researchers building autonomous computer-use agents or desktop automation workflows.
- Alternative to: Anthropic Computer Use, OpenAI Operator
- Tags: computer use, ai agent, automation, osworld, gui agent, python

## Why it was saved

- [@tectalks.io, 26 April 2025](https://www.instagram.com/p/DI6RCbZPUUp/): It automates repetitive web-based and computer tasks 24/7 without manual intervention, such as price comparison, trip planning, online ordering, and research.

## Compare

- [Alternatives to OpenAI Operator](https://xiry.app/alternatives/openai-operator.md)
- [Alternatives to Anthropic Computer Use](https://xiry.app/alternatives/anthropic-computer-use.md)

## Related

- [ACU - Awesome Agents for Computer Use](https://xiry.app/resources/acu-awesome-agents-for-computer-use): Curated collection of papers, tools, and frameworks for computer-controlling AI agents (Website, Free)
- [OmniParser](https://xiry.app/resources/omniparser): Parses UI screenshots into structured interactive elements for vision LLMs (GitHub repo, Open source)
- [Ace](https://xiry.app/resources/ace): Realtime desktop autopilot that controls mouse and keyboard for native apps. (Desktop app)
- [AutoGPT](https://xiry.app/resources/autogpt): Open-source platform and framework for building autonomous AI agents (GitHub repo, Freemium)
- [Claude Opus Multi-Agent Orchestration Demo (opus.py)](https://xiry.app/resources/claude-opus-multi-agent-orchestration-demo-opus-py): A Python multi-agent pattern pairing Opus orchestration with Haiku worker instances. (Framework, Open source)
- [Gobi](https://xiry.app/resources/gobi): Cloud browser AI agent API to scrape structured data via natural language (Web app)

Updated 3 October 2026. Page: https://xiry.app/resources/agent-s

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.