Apple considered switching to DuckDuckGo from Google for Safari - Bloomberg News
darth_helmet @ darth_helmet @sh.itjust.works Posts 0Comments 216Joined 2 yr. ago
darth_helmet @ darth_helmet @sh.itjust.works
Posts
0
Comments
216
Joined
2 yr. ago
It’s conventional to do this, but a user agent string is entirely up to the client, and robots.txt is just a suggestion.
So for the best results, you probably want to mock Google’s crawler because it’s suicidal to block that if you want search traffic.