# ScrapeGraphAI

> Open-source Python library and API for prompt-based web scraping using LLMs

ScrapeGraphAI is a Python library and managed API that uses large language models and graph logic to extract structured data from websites and documents. Instead of writing and maintaining CSS selectors or XPath rules, users specify the data they need using natural language prompts. It handles dynamic pages, multi-page crawling, and integrates with major AI agent frameworks.

Website: https://scrapegraphai.com/

Also at: https://github.com/ScrapeGraphAI/Scrapegraph-ai

## What it is for

- Extract structured JSON from websites using natural language prompts without writing selectors
- Scrape JavaScript-rendered dynamic web pages without manual browser automation setup
- Integrate web scraping capabilities into LLM agents built with CrewAI, LlamaIndex, or Agno

## Features

- Prompt-based data extraction from raw HTML, XML, and document formats
- Graph-based scraping pipelines that dynamically determine extraction workflows
- Managed API option alongside the open-source Python library
- Support for local and commercial LLMs via LangChain and direct integrations
- Compatible with AI frameworks such as CrewAI, LlamaIndex, Smithery, and n8n
- No proxy or selector maintenance required for changing page structures

## Pricing

Freemium. Open-source Python library is free (MIT); managed cloud API offers free tier and paid usage plans

## Details

- Type: GitHub repo
- Category: [Development](https://xiry.app/topics/development.md)
- Topic: [Web Scraping](https://xiry.app/topics/development/web-scraping.md)
- Open source: yes
- Platforms: Code library, API, Linux, macOS, Windows
- For: Developers and data engineers building AI workflows or scraping sites without maintaining brittle selectors
- Alternative to: Apify, Bright Data, Octoparse
- Tags: web scraping, python, llm, data extraction, open source, agents

## Why it was saved

- [@criscatalyst, 23 May 2026](https://www.instagram.com/reel/DYrlQReEbxS/): Replaces paid scraping services like Apify and Bright Data by letting you scrape multi-page sites and extract specific fields just by describing what you need in plain English.
- [@thevibefounder, 5 May 2026](https://www.instagram.com/reel/DX9fGa4SgX_/): Replaces complex scraper development and expensive services like Apify or Octoparse by letting you scrape URLs with a single text prompt without maintaining selectors or code.

## Compare

- [Alternatives to Apify](https://xiry.app/alternatives/apify.md)
- [Alternatives to Octoparse](https://xiry.app/alternatives/octoparse.md)

## Related

- [Firecrawl](https://xiry.app/resources/firecrawl-1984): Turn websites and documents into clean, LLM-ready markdown or structured data. (Website, Freemium)
- [Parse](https://xiry.app/resources/parse): Turn any website into a structured REST API using natural language (Web app, Freemium)
- [Apify](https://xiry.app/resources/apify): Cloud platform and marketplace for web scraping, automation, and AI data extraction. (Web app, Freemium)
- [Chat4Data](https://xiry.app/resources/chat4data): AI Chrome extension for scraping web data using natural language chat prompts. (Browser extension, Freemium)
- [Cloud Scraping Browsers Comparison](https://xiry.app/resources/cloud-scraping-browsers-comparison): Feature and cost breakdown across cloud browser sandbox providers for scraping (Framework, Free)
- [Cloudflare /crawl Endpoint](https://xiry.app/resources/cloudflare-crawl-endpoint): Single API endpoint to crawl entire websites to Markdown, HTML, or JSON (Tip, Freemium)

Updated 11 October 2026. Page: https://xiry.app/resources/scrapegraphai

Make your own library: Xiry takes your saved reels, tweets, Reddit bookmarks and Watch Later videos, makes them searchable and pulls the useful stuff out, like this library. It's invite-only for now; join the waitlist at https://xiry.app/#waitlist.