|
Their spiders hammer sites to the point of abuse sometimes. Even Googlebot isn't as rampant and intensive as the MSN spiders. I ended up blocking them from some sites because of the incessant and abusive pounding they were doing.
MSN says they observe robots.txt directions, so if you find them being a problem, you could block them there. In my experience, I found out it was them only after it was thought to be abuse.
__________________
rRhino.com ...social networking for book fans...
|