Development · Tip
Enabling FIRE-1 in the Firecrawl Scrape API
API configuration pattern for Firecrawl's agent-driven web navigation model
This tip demonstrates how to call Firecrawl's scrape endpoint using their FIRE-1 agent model. By passing navigation instructions inside the agent payload, you can scrape dynamic websites that require clicking, searching, or paginating before extracting content.
How it works
Include in the scrape API request: "agent": { "model": "FIRE-1", "prompt": "Your detailed navigation instructions here." } — the prompt field is required for scrape requests, telling FIRE-1 how to interact with the webpage. Pagination example (curl): POST https://api.firecrawl.dev/v1/scrape with Authorization Bearer YOUR_API_KEY, url https://www.ycombinator.com/companies, formats ["markdown"], agent {model FIRE-1, prompt "Search for firecrawl and go to the company page."}. Be very specific in prompting about the results you want.
What it's for
- Scrape web pages hidden behind interactive elements like search inputs or dropdowns
- Navigate multi-step pagination flows automatically during extraction
- Extract clean markdown from dynamic single-page applications without manual browser scripts
Key points
- Uses the agent object with model set to 'FIRE-1' inside scrape requests
- Requires a natural language prompt describing exact interactive navigation steps
- Supports automated interaction such as clicking links, form inputs, and pagination
- Returns parsed page content in markdown format
Pricing
Freemium. Requires Firecrawl API access (free tier available with credit limits)
Details
- Type
- Tip
- Category
- Development
- Topic
- Web Scraping
- Open source
- No
- Platforms
- API
- For
- Developers building LLM pipelines and automated scraping workflows
- Alternative to
- Puppeteer, Playwright, Browserbase
- Tags
- firecrawl, scraping, agents, api, llm, automation
Why it was saved
- @omarsar0: Concrete usage pattern for the agent-powered scraping
Compare
Related
Turn websites and documents into clean, LLM-ready markdown or structured data.
Turn any website into a structured REST API using natural language
Cloud platform and marketplace for web scraping, automation, and AI data extraction.
Open-source Python library and API for prompt-based web scraping using LLMs
AI Chrome extension for scraping web data using natural language chat prompts.
Feature and cost breakdown across cloud browser sandbox providers for scraping
Updated