Juriscraper is a scraper library started many years ago that gathers judicial opinions, oral arguments, and PACER data in the American court system. It is currently able to scrape:
- a variety of pages and reports within the PACER system
- opinions from all major appellate Federal courts
- opinions from all state courts of last resort except for Georgia (typically their "Supreme Court")
- oral arguments from all appellate federal courts that offer them
Juriscraper is part of a two-part system. The second part is your code, which calls Juriscraper. Your code is responsible for calling a scraper, downloading and saving its results. A reference implementation of the caller has been developed and is in use at CourtListener.com. The code for that caller can be found here. There is also a basic sample caller included in Juriscraper that can be used for testing or as a starting point when developing your own.
Some of the design goals for this project are:
- extensibility to support video, oral argument audio, etc.
- extensibility to support geographies (US, Cuba, Mexico, California)
- Mime type identification through magic numbers
- Generalized architecture with minimal code repetition
- XPath-based scraping powered by lxml's html parser
- return all meta data available on court websites (caller can pick what it needs)
- no need for a database
- clear log levels (DEBUG, INFO, WARN, CRITICAL)
- friendly as possible to court websites
First step: Install Python 3.10+, then:
On Ubuntu based distributions/Debian Linux:
sudo apt-get install libxml2-dev libxslt-dev libyaml-dev
On Arch based distributions:
sudo pacman -S libxml2 libxslt libyaml
On macOS with Homebrew <https://brew.sh>:
brew install libyaml
pip install juriscraper
You can set an environment variable for where you want to stash your logs (this can be skipped, and /var/log/juriscraper/debug.log will be used as the default if it exists on the filesystem):
export JURISCRAPER_LOG=/path/to/your/log.txt
We welcome contributions! If you'd like to get involved, please take a look at our CONTRIBUTING.md guide for instructions on setting up your environment, running tests, and more.
Juriscraper is licensed under the permissive BSD license.