# Firecrawl

> Turn websites and documents into clean, LLM-ready markdown or structured data.

Firecrawl is an open-source web scraping and search API designed specifically for AI agents and LLM applications. It strips out boilerplate like navigation, ads, and footers, returning clean Markdown, JSON, or screenshots while managing dynamic JavaScript rendering, anti-bot mechanisms, proxies, and media parsing (PDF, DOCX). It also supports interactive browser actions, autonomous agent-driven navigation, and direct integration via MCP, CLI, and popular AI frameworks.

Website: https://firecrawl.dev/

Also at: https://github.com/mendableai/firecrawl

## What it is for

- Extract clean Markdown from documentation or dynamic web pages for LLM context and RAG pipelines
- Parse multi-page PDFs or DOCX reports into structured data for automated research
- Provide AI coding tools (Cursor, Claude) with live web search and browsing capabilities via MCP
- Automate complex multi-step browser interactions like clicking, form filling, and paginated scraping
- Build automated pipelines that ingest news and live web data for content generation

## Features

- Converts entire websites or individual pages into clean Markdown, JSON, or screenshots
- Built-in handling for anti-bot protections, proxies, dynamic JavaScript rendering, and smart waits
- Interactive actions API to click, scroll, type, wait, and navigate web pages via prompts or code
- Media parsing engine for extracting text and tables from PDFs, DOCX, and images
- Unified search endpoint that returns query results with full-page Markdown included
- Official Model Context Protocol (MCP) server, CLI, and SDKs for Python, Node.js, Go, and Rust
- Integrations with AI frameworks including LangChain, LlamaIndex, and CrewAI

## Pricing

Freemium. Free tier available; paid cloud plans for higher limits, with self-hosting available via open-source repository.

## Details

- Type: Website
- Category: [Development](https://xiry.app/topics/development.md)
- Topic: [Web Scraping](https://xiry.app/topics/development/web-scraping.md)
- Open source: yes
- Platforms: Web, API, Command line, Code library, Self-hosted
- For: Software engineers and AI developers building agents, RAG applications, and data extraction pipelines.
- Alternative to: Apify, Jina Reader, Puppeteer, ScrapingBee
- Tags: scraping, llm, markdown, ai agents, mcp, rag, crawler

## Why it was saved

- [@ericciarla, 18 November 2024](https://x.com/ericciarla/status/1858588268935475305): A complete, runnable pattern for combining web scraping + LLM script generation + voice cloning into an automated podcast pipeline.
- [@akshay_pachaar, 16 December 2024](https://x.com/akshay_pachaar/status/1868635172842918323): Handles messy web-to-data conversion: proxies, anti-bots, dynamic content, batching of thousands of URLs, media parsing (PDF, DOCX, images), and actions (click, scroll, input, wait). Integrates with Langchain, Llama Index, Crew.ai and more.
- [@akshay_pachaar, 16 December 2024](https://x.com/akshay_pachaar/status/1868635172842918323): Use it directly without self-hosting; discount code DDODS gives 10% off.
- [@omarsar0, 16 April 2025](https://x.com/omarsar0/status/1912596779784143002): Solves scraping tasks standard scrapers can't: agent navigates complex pages, paginates, and extracts data to markdown/JSON. Used in the demo to scrape the YC Startup Directory.
- [@firecrawl, 3 March 2026](https://x.com/firecrawl/status/2028878214492541208): Turns 200+ page PDFs with text, charts and tables (earnings calls, whitepapers, market research) into clean Markdown in seconds, as shown converting a World Bank Global Economic Prospects report into Markdown.
- [@_avichawla, 7 February 2025](https://x.com/_avichawla/status/1887750672864714886): Replaces hard-coded scrapers: describe the fields to extract (e.g. url, title, publish\_date of all LLM-related articles) and get LLM-ready structured data back, as shown scraping dailydoseofds.com.

## Compare

- [Alternatives to Apify](https://xiry.app/alternatives/apify.md)
- [Alternatives to Puppeteer](https://xiry.app/alternatives/puppeteer.md)

## Related

- [Parse](https://xiry.app/resources/parse): Turn any website into a structured REST API using natural language (Web app, Freemium)
- [Apify](https://xiry.app/resources/apify): Cloud platform and marketplace for web scraping, automation, and AI data extraction. (Web app, Freemium)
- [ScrapeGraphAI](https://xiry.app/resources/scrapegraphai): Open-source Python library and API for prompt-based web scraping using LLMs (GitHub repo, Freemium)
- [Chat4Data](https://xiry.app/resources/chat4data): AI Chrome extension for scraping web data using natural language chat prompts. (Browser extension, Freemium)
- [Cloud Scraping Browsers Comparison](https://xiry.app/resources/cloud-scraping-browsers-comparison): Feature and cost breakdown across cloud browser sandbox providers for scraping (Framework, Free)
- [Cloudflare /crawl Endpoint](https://xiry.app/resources/cloudflare-crawl-endpoint): Single API endpoint to crawl entire websites to Markdown, HTML, or JSON (Tip, Freemium)

Updated 11 October 2026. Page: https://xiry.app/resources/firecrawl-1984

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.