A linter for Scrapy projects.

Official git repository for QGit.

A stub implementation of a subset of Zyte API

Websites for testing spiders

Default Twisted does not ship with a CONNECT-enabled HTTP(s) proxy. This code provides one.

An interactive TLS-capable intercepting HTTP proxy for penetration testers and software developers.

Python client for Zyte API

🎭 Playwright integration for Scrapy

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

Zyte API integration for Scrapy

https://mimesniff.spec.whatwg.org/ implementation for Python

Scrapy Extension for monitoring spiders execution.

Spider templates for automatic crawlers.

Remove DIVs, style stuff and normalize HTML preserving structure information

A pure-Python robots.txt parser with support for modern conventions.

Contains the common item definitions used in Zyte.

Page Object pattern for Scrapy

Web scraping Page Objects core library

Common interface for data container classes

Library to populate items using XPath and CSS with a convenient API

Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

CSS Selectors for Python

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python library of web-related functions

Scrapy, a fast high-level web crawling & scraping framework for Python.