Google Bot. - GoFuckYourself.com

Walrus · 09-03-2005, 05:22 AM

How often does the Google Bot come scan your web site? I can see when it comes in my stats. But I don't ever remember being albe to tell when Yahoo or MSN has scanned.

BastarD · 09-03-2005, 05:24 AM

it all depends how many and what quality inbound links you have and also site content is very important, how often it is updated etc.

I have google bot, yahoo and msn spiders few times a day on each of my sites.

IceMaster · 09-03-2005, 05:25 AM

Everyday. It depends on the Pr of the site.

Manowar · 09-03-2005, 05:26 AM

everyday

Walrus · 09-03-2005, 05:35 AM

Everyday for you guys, wow. So does your PR for some pages change on a daily basis too? Or is all PR for every page on every web site only done every few months?

Also, what does the url look like in your stats for MSN and Yahoo? Is it similar to Google, where it has "bot" in the link?

- Walrus

Alky · 09-03-2005, 05:41 AM

Pr is updated every few months

65.54.188.72 - - [03/Sep/2005:06:33:03 -0400] "GET / HTTP/1.0" 200 49510 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

66.196.91.167 - - [02/Sep/2005:20:26:25 -0400] "GET / HTTP/1.0" 200 60744 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

Walrus · 09-03-2005, 05:51 AM

Alky, thanks. What did you use to get those stats? Raw logs or did you use something like webalyzer? I am looking for a new host with better raw logs. With WebAir's Virtual Hosting you don't get stats like you showed above that has IP address with referring URL.

- Walrus

Alky · 09-03-2005, 05:52 AM

Quote:

Originally Posted by Walrus

Alky, thanks. What did you use to get those stats? Raw logs or did you use something like analyzer? I am looking for a new host with better raw logs. With WebAir's Virtual Hosting you don't get stats like you showed above that has IP address with referring URL.

- Walrus

just from the raw logs.

Walrus · 09-03-2005, 05:55 AM

Quote:

Originally Posted by Alky

just from the raw logs.

Virtual or dedicated? And which hosting company, and how much $$/month? WebAir's Dedicated servers have it, but I get less than 1k visitors a day to my cam site. I'm not going to pay more just to have better stats when I can get it elsewhere, and for free with Virtual. They are a great hosting company but I need better stats. Right now I'm paying $9.95/month.

- Walrus

wedouglas · 09-03-2005, 07:00 AM

in August on my PR4 that gets about 600 a day

20 1127 0.85% Googlebot/2.1 (+http://www.google.com/bot.html)

09-03-2005, 05:22 AM	#1
Walrus Confirmed User Join Date: May 2005 Location: USA Posts: 2,150	Google Bot. How often does the Google Bot come scan your web site? I can see when it comes in my stats. But I don't ever remember being albe to tell when Yahoo or MSN has scanned.

09-03-2005, 05:24 AM	#2
BastarD Confirmed User Join Date: Apr 2004 Location: Very close to that Jack Daniels bottle.. Posts: 6,341	it all depends how many and what quality inbound links you have and also site content is very important, how often it is updated etc. I have google bot, yahoo and msn spiders few times a day on each of my sites.

09-03-2005, 05:25 AM	#3
IceMaster Confirmed User Join Date: Jan 2005 Posts: 8,920	Everyday. It depends on the Pr of the site.

09-03-2005, 05:26 AM	#4
Manowar jellyfish Join Date: Dec 2003 Posts: 71,528	everyday

09-03-2005, 05:35 AM	#5
Walrus Confirmed User Join Date: May 2005 Location: USA Posts: 2,150	Everyday for you guys, wow. So does your PR for some pages change on a daily basis too? Or is all PR for every page on every web site only done every few months? Also, what does the url look like in your stats for MSN and Yahoo? Is it similar to Google, where it has "bot" in the link? - Walrus

09-03-2005, 05:41 AM	#6
Alky Confirmed User Join Date: Apr 2002 Location: Houston Posts: 5,651	Pr is updated every few months 65.54.188.72 - - [03/Sep/2005:06:33:03 -0400] "GET / HTTP/1.0" 200 49510 "-" "msnbot/1.0 (+http://search.msn.com/msnbot.htm)" 66.196.91.167 - - [02/Sep/2005:20:26:25 -0400] "GET / HTTP/1.0" 200 60744 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

09-03-2005, 05:51 AM	#7
Walrus Confirmed User Join Date: May 2005 Location: USA Posts: 2,150	Alky, thanks. What did you use to get those stats? Raw logs or did you use something like webalyzer? I am looking for a new host with better raw logs. With WebAir's Virtual Hosting you don't get stats like you showed above that has IP address with referring URL. - Walrus

09-03-2005, 07:00 AM	#10
wedouglas So Fucking Banned Join Date: Aug 2004 Location: Dearborn, MI Posts: 5,921	in August on my PR4 that gets about 600 a day 20 1127 0.85% Googlebot/2.1 (+http://www.google.com/bot.html)