# Make your site readable by AI agents

> Practical guide to making websites discoverable and actionable for AI agents.

Published on the Vercel Knowledge Base in collaboration with the ora.ai research lab, this guide outlines technical requirements for AI agent discoverability based on 1,000+ real agent runs and controlled fetch probes. It reveals that server response behavior and server-rendered HTML matter significantly more than emerging conventions like llms.txt. It provides actionable advice on HTTP status codes, SSR/prerendering, markdown endpoints, and API design.

Website: https://vercel.com/kb/guide/make-your-site-readable-by-ai-agents

## What it is for

- Audit a website to ensure AI agents and crawlers can retrieve content without executing JavaScript
- Configure server responses, HTTP status codes, and rate limits correctly for automated agent traffic
- Expose machine-readable documentation, OpenAPI contracts, and markdown mirrors for AI workflows
- Evaluate domain readiness for AI agents using the is-agentic testing tool

## Features

- Analysis of 1,033 real agent runs and 190 controlled fetch probes across 19 site configurations
- Detailed recommendations on server-rendered HTML over client-side JavaScript rendering
- Guidance on configuring llms.txt as a discoverable, linked index rather than relying on guessed paths
- Practical tips for supporting Markdown format negotiation and Link headers
- Checklist for exposing authenticated APIs, OpenAPI specs, and Model Context Protocol (MCP) endpoints
- Integration with domain scanning tools like is-agentic.com

## Pricing

Free technical article and public research dataset

## Details

- Type: Article or course
- Category: [Development](https://xiry.app/topics/development.md)
- Open source: no
- Platforms: Web
- For: Web developers, technical SEOs, and API engineers building agent-accessible sites
- Tags: ai agents, seo, web development, api design, vercel, llms.txt

## Why it was saved

- [@vercel_dev, 3 September 2026](https://x.com/vercel_dev/status/2095519669117710417): Grounded in over 1,000 real agent runs and a controlled experiment with the ora.ai research lab; counters the common assumption that llms.txt is the answer - the way servers respond to requests reportedly matters more.

## Related

- [Skills.sh](https://xiry.app/resources/skills-sh): Open directory and CLI for discovering and installing AI agent skills (Web app, Free)
- [react-best-practices](https://xiry.app/resources/react-best-practices): Curated React performance rules and coding-agent skills by Vercel (GitHub repo, Open source)
- [agent-browser Environments](https://xiry.app/resources/agent-browser-environments): Run browser automation agents inside on-demand isolated sandbox environments (Web app, Free)
- [Agent-Ready SaaS Checklist](https://xiry.app/resources/agent-ready-saas-checklist): Practical rules for preparing SaaS APIs and infrastructure for AI agents (Framework, Free)
- [agent-skills](https://xiry.app/resources/agent-skills): Curated skills and performance rules for AI coding agents by Vercel. (GitHub repo, Open source)
- [Chat SDK](https://xiry.app/resources/chat-sdk): Universal TypeScript layer for building bots and agents across chat platforms (GitHub repo, Open source)

Updated 10 October 2026. Page: https://xiry.app/resources/make-your-site-readable-by-ai-agents

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.