how often does google crawl porn sites? - GoFuckYourself.com

mika · 01-20-2002, 09:58 AM

Someone said it crawls all the time but I really dont think so because when you take a look at the google cached sites they all seem to have been crawled within 1-3 days

for example PK, al4a, thehun we're all crawled 13th of December

That's too long time to wait. somebody email the boss of Google

Viper2K1 · 01-20-2002, 10:02 AM

I've read somewhere it depends on your current listing in the SE ... high listed site get spidered more often than "the rest".

I also think a meta tag like this
META NAME="revisit-after" CONTENT="20 days" could help you with it

[This message has been edited by Viper2K1 (edited 01-20-2002).]

Trax · 01-20-2002, 10:07 AM

from what i know google gives a decent fuck about meta tags

mika · 01-20-2002, 10:49 AM

fuck if PK was crawled dec13 and its high ranked , then my site would never get spidered.

no I dont believe that theory
I had some galleries crawled around dec13 or so, too

amadman · 01-20-2002, 11:14 AM

I am no se guru but I think the spider crawl and the updating of the index are two diffrent things.

Maybe?

Chris R · 01-20-2002, 12:47 PM

Google crawls somes sites more often than others. This has been experimental - and for a while they were adding what is known as a "freshness" tage to listings.

This, as someone pointed out, is not the same as updating the index. You were pretty much listed in the same order. Google updates approximately once every 26 - 35 days. Next update should be around January 24th - February 6th.

The sites it crawls more often have nothing to do with Metatags - they have to do with PageRank (something similar to link popularity).

It takes about 4 - 6 weeks to get a site listed in Google. Submitting to Google does not help you. You must have links too your site from other sites in Google.

------------------
TopBucks.com - Converting at better than 1:130
David Lace Content - Highest Quality Teen Content
ConversionCash.com - Make Money off your WebTV Traffic

01-20-2002, 09:58 AM	#1
mika Confirmed User Join Date: Sep 2001 Location: Finland Posts: 1,561	how often does google crawl porn sites? Someone said it crawls all the time but I really dont think so because when you take a look at the google cached sites they all seem to have been crawled within 1-3 days for example PK, al4a, thehun we're all crawled 13th of December That's too long time to wait. somebody email the boss of Google

01-20-2002, 10:02 AM	#2
Viper2K1 Confirmed User Industry Role: Join Date: Aug 2001 Posts: 1,006	I've read somewhere it depends on your current listing in the SE ... high listed site get spidered more often than "the rest". I also think a meta tag like this META NAME="revisit-after" CONTENT="20 days" could help you with it [This message has been edited by Viper2K1 (edited 01-20-2002).]

01-20-2002, 10:07 AM	#3
Trax [----------------------] Join Date: Aug 2001 Posts: 14,486	from what i know google gives a decent fuck about meta tags

01-20-2002, 10:49 AM	#4
mika Confirmed User Join Date: Sep 2001 Location: Finland Posts: 1,561	fuck if PK was crawled dec13 and its high ranked , then my site would never get spidered. no I dont believe that theory I had some galleries crawled around dec13 or so, too

01-20-2002, 11:14 AM	#5
amadman Confirmed User Join Date: Nov 2001 Location: padded room Posts: 738	I am no se guru but I think the spider crawl and the updating of the index are two diffrent things. Maybe?

01-20-2002, 12:47 PM	#6
Chris R Confirmed User Join Date: May 2001 Location: Baltimore, MD USA Posts: 1,151	Google crawls somes sites more often than others. This has been experimental - and for a while they were adding what is known as a "freshness" tage to listings. This, as someone pointed out, is not the same as updating the index. You were pretty much listed in the same order. Google updates approximately once every 26 - 35 days. Next update should be around January 24th - February 6th. The sites it crawls more often have nothing to do with Metatags - they have to do with PageRank (something similar to link popularity). It takes about 4 - 6 weeks to get a site listed in Google. Submitting to Google does not help you. You must have links too your site from other sites in Google. ------------------ TopBucks.com - Converting at better than 1:130 David Lace Content - Highest Quality Teen Content ConversionCash.com - Make Money off your WebTV Traffic