Web crawler and scraper for Rust

🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN

Convert any URL to an LLM-friendly input with a simple prefix https://r.jina.ai/

🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!

Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.

➖ Stripped down, stable version of firecrawl optimized for self-hosting and ease of contribution. Billing logic and AI features are completely removed. Crawl and convert any website into LLM-ready …

Borrow cookies from your browser's authenticated session for use in Python scripts.

An ergonomic Python HTTP Client with TLS fingerprint

Python APIs for web automation, testing, and bypassing bot-detection with ease.

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

List of anti-detect and humanizing tools and browsers, including captcha solvers and sms-activation.

Hide your scrapers IP behind the cloud. Provision proxy servers across different cloud providers to improve your scraping success.