GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)
-   Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)
-   -   Googlebot wtf? (https://gfy.com/showthread.php?t=669117)

fusionx 10-22-2006 01:13 PM

Googlebot wtf?
 
Anyone else get beat up by googlebot like this?

Googlebot 29414+33 1.04 GB

That's hits plus bandwidth so far this month - on one small site.

baddog 10-22-2006 01:32 PM

you aren't complaining are you?

baddog 10-22-2006 01:32 PM

if you are, try robots.txt

fusionx 10-22-2006 01:34 PM

hehe - not complaining in that sense..

Seems like a lot of traffic for a site that doesn't really update.

Could someone be spoofing googlebot?

rigrunner 10-22-2006 01:35 PM

wow i get googlebot everyday and it doesn't run up that much bandwidth...

chaze 10-22-2006 02:07 PM

If you site is a gig then it will spider a little less then a gig if it likes you, it basically downloads you site when you are in there a list.

Congrats :thumbsup

woj 10-22-2006 02:11 PM

wow, you really got owned...

squishypimp 10-22-2006 02:13 PM

google is mad at you...

BSleazy 10-22-2006 03:51 PM

There's a crawl rate option in the google webmaster tools. If a gig a month is a problem for you then slow it down.


All times are GMT -7. The time now is 02:42 PM.

Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123