Crawl4AI
Open-source crawler that turns websites into LLM-ready Markdown.
A self-hosted Python crawler that renders pages in a real browser and outputs clean Markdown or structured extractions for RAG and agents. Free to run; you supply the compute and proxies.
- Vendor
- Crawl4AI
- Category
- Search & web data APIs
- Pricing
- Open source
- Open source
- Yes
- License
- Apache 2.0
- Platforms
- Python, Docker, Self-host
- Launched
- 2024
- Website
- crawl4ai.com
Features
- Markdown output
- Browser rendering
- Structured extraction
Best for
- Extraction & classification
- Research
More search & web data apis
- Firecrawl — Firecrawl. Turn websites into LLM-ready data.
- Tavily — Tavily. Real-time search API for agents.
- Exa — Exa. Web search API built for AI.
- Perplexity Sonar API — Perplexity. Search-grounded answers via API.
- Brave Search API — Brave. Independent web index via API.
- Bright Data — Bright Data. Web data, unblocking and scraping infrastructure for AI.
- Parallel — Parallel Web Systems. Web search and research APIs built for AI agents.
- Jina Reader — Jina AI (Elastic). Turn any URL into LLM-ready markdown by prefixing r.jina.ai.