20 Apr 25

An open list of web crawlers associated with AI companies and the training of LLMs to block, including a number of crawlers sourced from Dark Visitors.

by agnieszka 9 months ago saved 4 times