TikTok’s parent launched a web scraper that’s gobbling up the world’s online data 25-times faster than OpenAI
zod000 @ zod000 @lemmy.ml Posts 0Comments 546Joined 2 yr. ago
zod000 @ zod000 @lemmy.ml
Posts
0
Comments
546
Joined
2 yr. ago
Bullshit. This bot doesn't identify itself as a bot and doesn't rate limit itself to anything that would be an appropriate amount. We were seeing more traffic from this thing that all other crawlers combined.