Marketing & Sales · Website
XML-Sitemaps.com
Free online crawler that generates search engine and text sitemaps.
XML-Sitemaps.com is a web-based crawler that scans websites and outputs structured XML, HTML, and text sitemaps. It handles up to 500 URLs for free without requiring account registration, automatically calculating modification times and page priorities. It is commonly used for SEO indexing and for extracting complete lists of website URLs to train custom AI chatbots.
What it's for
- Generate an XML sitemap to submit to search engines for SEO indexing
- Extract all public URLs and FAQ pages to train custom AI customer support bots
- Create a text or HTML list of all pages across a small website
Features
- Crawls up to 500 URLs on the free tier without registration
- Generates XML, HTML, and text sitemaps
- Automatically calculates page priority based on click depth
- Detects page last-modification timestamps from server responses
- Real-time crawling dashboard showing discovered, added, and skipped pages
- Pro tier available for larger sites, auto-updates, and media sitemaps
Pricing
Freemium. Free up to 500 URLs; Pro plans start at $2.44/month for larger sites and automatic updates
Details
- Type
- Website
- Category
- Marketing & Sales
- Topic
- SEO Tools
- Open source
- No
- Platforms
- Web
- For
- Webmasters, SEO specialists, and builders gathering website URLs for AI knowledge bases.
- Alternative to
- Screaming Frog SEO Spider, Yoast SEO
- Tags
- seo, sitemap, web crawler, indexing, knowledge base
Why it was saved
- @daviefogarty: Quickly discovers and compiles every webpage and FAQ link from a target website to use for AI knowledge base ingestion.
Compare
Related
Browser extension showing instant on-page SEO insights and heading hierarchy
Claude Code skill and CLI tool for Generative Engine Optimization audits
Modular SEO, AEO, and GEO skills plugin for Claude Code
Open-source, self-hosted SEO platform powered by DataForSEO API.
All-in-one SEO tool for keyword research, site auditing, and competitor analysis
Automated SEO tool for keyword research, article generation, and backlink exchanges
Updated