Originally Posted by Muad'Dib

Page validators are not the same as SE bots.

You are basically giving (allowing) the address to page validators to go to and check.

But when SE bots come to your site, they read the instructions first and don't go to those directories, because it's forbidden.