View Single Post
Old 06-09-2006, 04:49 AM  
Violetta
Affiliate
 
Violetta's Avatar
 
Join Date: Jul 2004
Posts: 28,735
Quote:
Originally Posted by duckduckgoose
Their spiders hammer sites to the point of abuse sometimes. Even Googlebot isn't as rampant and intensive as the MSN spiders. I ended up blocking them from some sites because of the incessant and abusive pounding they were doing.

MSN says they observe robots.txt directions, so if you find them being a problem, you could block them there. In my experience, I found out it was them only after it was thought to be abuse.
ah... I see. Thanks a lot!
__________________
M&A Queen
Violetta is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote