Crawler

👾 Fast and simple video download library and CLI tool written in Go

Elegant Scraper and Crawler Framework for Golang

A next-generation crawling and spidering framework.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Automatically crawls proxy nodes on the public internet, de-duplicates and tests for usability and then provides a list of nodes

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

A powerful browser crawler for web vulnerability scanners

Gospider - Fast web spider written in Go

Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.

Website Cloner - Utilizes powerful Go routines to clone websites to your computer within seconds.

ScopeSentry-Cyberspace mapping, subdomain enumeration, port scanning, sensitive information discovery, vulnerability scanning, distributed nodes

Polite, slim and concurrent web crawler.

Dynamic server-side rendering using headless Chrome

[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.

The fastest dork scanner written in Go.

Golang短视频去水印：抖音,皮皮虾,火山,微视,最右,快手,全民小视频,皮皮搞笑,西瓜视频,虎牙,梨视频,acfun,好看视频...

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

一个方便安全研究人员获取每日安全日报的爬虫和推送程序，目前爬取范围包括先知社区、安全客、Seebug Paper、跳跳糖、奇安信攻防社区、棱角社区以及绿盟、腾讯玄武、天融信、360等实验室博客，持续更新中。

Open source SEO audit tool.

DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.