Always love to see Anubis get flak.

…but don’t love to see a solution that has nearly the same a11y problems (RIP Lynx 1992–2025). At least it doesn’t use proof-of-work.

Always love to see Anubis get flak.

…but don’t love to see a solution that has nearly the same a11y problems (RIP Lynx 1992–2025). At least it doesn’t use proof-of-work.

It

grab-site is an easy preconfigured web crawler designed for backing up websites. Give grab-site a URL and it will recursively crawl the site and write WARC files. Internally, grab-site uses a fork of wpull for crawling.

Example of a simple web scraper and a search application using Manticore.