1488
Black Mirror AI
(mander.xyz)
A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.
Rules
This is a science community. We use the Dawkins definition of meme.
Wouldn't Google's crawlers respect robots.txt though? Is it naive to assume that anything would?
It's naive to assume that google crawlers respect robot.txt.
It'd be more naive to have a robot.txt file on your webserver and be surprised when webcrawlers don't stay away. ๐