🕷Crawler & Anti-crawler
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, an…
Chrome extension that records your browser interactions and generates a Playwright or Puppeteer script.
Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
JavaScript API for Chrome and Firefox
A scalable web crawler framework for Java.
Deploy headless browsers in Docker. Run on our cloud or bring your own. Free for non-commercial uses.
Puppeteer Pool, run a cluster of instances in parallel
Headless Chromium-based web performance metrics collector and monitoring tool
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
A collection of awesome web crawler,spider in different languages
A high-level browser automation library.
CrowdSec - the open-source and participative security solution offering crowdsourced protection against malicious IPs and access to the most advanced real-world CTI.
NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX
teler-waf is a Go HTTP middleware that protects local web services from OWASP Top 10 threats, known vulnerabilities, malicious actors, botnets, unwanted crawlers, and brute force attacks.
ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx. It has a robust event-based programming language which provides protection from a rang…
SafeLine is a self-hosted WAF(Web Application Firewall) / reverse proxy to protect your web apps from attacks and exploits.
🛡️ Open-source and next-generation Web Application Firewall (WAF)
OpenGFW is a flexible, easy-to-use, open source implementation of GFW (Great Firewall of China) on Linux
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
🔥 The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data
ModSecurity v3 Nginx Connector
NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX
High-performance WAF built on the OpenResty stack
Handy, High performance, ModSecurity compatible Nginx firewall module & 方便、高性能、兼容 ModSecurity 的 Nginx 防火墙模块
open-appsec is a machine learning security engine that preemptively and automatically prevents threats against Web Application & APIs. This repo include the main code and logic.