Anubis is awesome! Stopping (AI)crawlbots
Daniel Quinn @ danielquinn @lemmy.ca Posts 11Comments 655Joined 2 yr. ago

Daniel Quinn @ danielquinn @lemmy.ca
Posts
11
Comments
655
Joined
2 yr. ago
The number of lines for each character by percentage of the series
9 days after writing in defence of a Free Palestine, Paul Biggar is dropped from his director role at CircleCI
This all appears to be based on the user agent, so wouldn't that mean that bad-faith scrapers could just declare themselves to be typical search engine user agent?