20 Apr 25

An open list of web crawlers associated with AI companies and the training of LLMs to block, including a number of crawlers sourced from Dark Visitors.

by agnieszka 8 months ago saved 4 times