ricecake , 7 days ago Robots.txt isn't even a rule, it's a request. "Please do not ask for the following content if you are a robot". If you don't want someone to look at your content, you ultimately have to not give it to them, not just ask them to not ask.
Robots.txt isn't even a rule, it's a request.
"Please do not ask for the following content if you are a robot".
If you don't want someone to look at your content, you ultimately have to not give it to them, not just ask them to not ask.