Python-based web crawling script with randomized intervals, user-agent rotation, and proxy server IP rotation to outsmart website bots and prevent blocking.

Sneakpeek is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis

Scraper for https://marvelsnapzone.com to retrieve metadata of Marvel SNAP cards.

A universal and local phishing toolkit for audit purposes

An almost generic web crawler built using Scrapy and Python 3.7 to recursively crawl entire websites.

A Simple Script To Scrape DuckDuckGo Search Results Using Python And Selenium WebDriver.

🕷️ | ReconX is a Live-Website Crawler made to gather critical information with an option to take a picture of each site crawled!

This a project to demonstrate the use of standard python libraries like os, urllib, HTMLParser to create a minimalist webpage crawler that crawls webpages on a website to gather hyperlinks (URLs)

Web Link Crawler: A Python script to crawl websites and collect links based on a regex pattern. Efficient and customizable.

WikiWord is an intelligent Wikipedia navigation tool that finds paths between two topics by following internal links and using semantic similarity to select the most relevant next article.

Recursive website crawler

Sitesweeper is a python package to help you automate your web scraping process, outputting pages to a file

Grabs images off webpages.

Email Harvesting Tool designed to efficiently gather and validate emails from specified websites

a python script that crawls website sitemap in a very quick way with multi threading and extract, write SEO based data to CSV file

Simple website crawler to get Meta tags and <H1> on Python