SEO test - GoFuckYourself.com

SmokeyTheBear · 07-20-2008, 10:27 PM

If you register a new domain put up a site with 5 pages , the main page and 4 pages linked from the main domain

the 4 pages are named

apple.html
pear.html
squash.html
tomato.html

what page will google spider first ?

stickyfingerz · 07-20-2008, 10:28 PM

The index?

SmokeyTheBear · 07-20-2008, 10:29 PM

Quote:

Originally Posted by stickyfingerz

The index?

lol i was just thinking i should elaborate or some smartass will reply " the index "

assuming the index is already listed has one backlink and none of the other pages have backlinks other than from the main page of the domain

stickyfingerz · 07-20-2008, 10:30 PM

Quote:

Originally Posted by SmokeyTheBear

lol i was just thinking i should elaborate or some smartass will reply " the index "

assuming the index is already listed has one backlink and none of the other pages have backlinks other than from the main page of the domain

Glad to be at your smartass service.

baddog · 07-20-2008, 10:35 PM

Which page is the first link Google sees on the main page?

WiredGuy · 07-20-2008, 10:37 PM

Robots.txt
WG

fallenmuffin · 07-20-2008, 10:41 PM

If listed in order.. apple.html ?

Makingcoin · 07-20-2008, 10:42 PM

grape.html ;-)

Smokey, hit me up please.

d-null · 07-20-2008, 11:07 PM

all four simultaneously

Iron Fist · 07-20-2008, 11:38 PM

What ever fruit Google Labs says is more popular at the time...

Iron Fist · 07-20-2008, 11:43 PM

Okay i'll back that up.... Apple would be first to index...

AlienQ - BANNED FOR LIFE · 07-21-2008, 01:56 AM

Use Robot.txt

Why leave things to chance? Tell the fucker where to go.

SmokeyTheBear · 07-21-2008, 02:12 AM

Quote:

Originally Posted by baddog

Which page is the first link Google sees on the main page?

same order as listed lets say

SmokeyTheBear · 07-21-2008, 02:13 AM

Quote:

Originally Posted by WiredGuy

Robots.txt
WG

no robots.txt

SmokeyTheBear · 07-21-2008, 02:14 AM

Quote:

Originally Posted by AlienQ

Use Robot.txt

Why leave things to chance? Tell the fucker where to go.

as far as i am aware you cant tell google what order to crawl your site

SmokeyTheBear · 07-21-2008, 02:21 AM

i will narrow it down .

5 choices.

#1 Google will spider the links in alphabetical order.
#2 Google will spider the links by length.
#3 Google will spider the links in the order it found them.
#4 Google will spider the links by importance of keyword
#5 Google will spider them randomly.

Violetta · 07-21-2008, 02:27 AM

Quote:

Originally Posted by SmokeyTheBear

#1 Google will spider the links in alphabetical order.

d-null · 07-21-2008, 02:35 AM

whichever shows up first in the page source so #3

SmokeyTheBear · 07-21-2008, 03:43 AM

the answer is PEAR.html

reason = #2

Traf · 07-21-2008, 03:48 AM

Google will crawl the first link indicated in your index page.

IF this is the order you used in your index page then google will crawl apple.html first.

apple.html
pear.html
squash.html
tomato.html

SmokeyTheBear · 07-21-2008, 04:11 AM

Quote:

Originally Posted by Traf

Google will crawl the first link indicated in your index page.

IF this is the order you used in your index page then google will crawl apple.html first.

apple.html
pear.html
squash.html
tomato.html

in my experience google uses the url length . it starts with the shortest urls first

I just did a test to confirm this. I dropped 500 links to a dynamic page, each page crawled creates a hardfile containing the first person to view it. when i sort the hardfiles by date they also are sorted by length. i.e. google crawled the shortest urls first in order to longest

beta-tester · 07-21-2008, 04:20 AM

We can't be 100% sure which url it'll spider first. But I pretty much think it uses heuristic method in doing that.

DatingGameExpert · 07-21-2008, 04:49 AM

Quote:

Originally Posted by SmokeyTheBear

i will narrow it down .

5 choices.

#1 Google will spider the links in alphabetical order.
#2 Google will spider the links by length.
#3 Google will spider the links in the order it found them.
#4 Google will spider the links by importance of keyword
#5 Google will spider them randomly.

Borrrrrrrrrrrrrring.

WiredGuy · 07-21-2008, 06:48 AM

Is this just from self-testing or something more formal?

My last test I had about 800 pages in total, all pages were of the format:
http://www.domain.com/Category_Name/
http://www.domain.com/Category_Name/index1 - index5 (.shtml)
http://www.domain.com/Category_Name/8-Character-Code (.shtml)

I noticed the first 80 pages indexed were full from the last set, domain, category, 8-character-code.shtml. It was never the category or index pages that got indexed first.

WG

nico-t · 07-21-2008, 07:42 AM

Quote:

Originally Posted by WiredGuy

Is this just from self-testing or something more formal?

My last test I had about 800 pages in total, all pages were of the format:
http://www.domain.com/Category_Name/
http://www.domain.com/Category_Name/index1 - index5 (.shtml)
http://www.domain.com/Category_Name/8-Character-Code (.shtml)

I noticed the first 80 pages indexed were full from the last set, domain, category, 8-character-code.shtml. It was never the category or index pages that got indexed first.

WG

those are dirs, not all same leveled pages in smokeys example. So maybe the deepest dir pages get indexed first? And then their parents?

So the shortest, deepest leveled pages, get indexed first?

V_RocKs · 07-21-2008, 08:08 AM

longer names get love last.

Deej · 07-21-2008, 10:16 AM

assuming google does spider each link... within a timely manner for each...

is there an advantage to having google spider certain pages first?

SmokeyTheBear · 07-21-2008, 10:28 AM

Quote:

Originally Posted by Deej

is there an advantage to having google spider certain pages first?

unfortunately no lol, or none i can think of offhand

Deej · 07-21-2008, 10:57 AM

Quote:

Originally Posted by SmokeyTheBear

unfortunately no lol, or none i can think of offhand

Then quit wasting our god damned time smokey the fucking bear!!!

You insignificant trivia feedin mother fucker!!!

Your deck looks like shit!!!!

SCORE Ralph · 07-21-2008, 11:39 AM

Quote:

Originally Posted by Deej

Then quit wasting our god damned time smokey the fucking bear!!!

You insignificant trivia feedin mother fucker!!!

Your deck looks like shit!!!!

Funny, I was thinking the same thing but was going to be a bit more diplomatic about it

baddog · 07-21-2008, 12:17 PM

So you are suggesting that the spider takes the time to look at all the links, then sort in order of length?

pussyserver - BANNED FOR LIFE · 07-21-2008, 12:26 PM

you are all wrong

Google will spider the link most relative to the content of the index page first...

that being said... there is no such thing as spider first sice the google bot is in fact millions of bots that all spider constantly

then 2 pages or 20 pages on a site can all be indexed at the same time

the page most relative to the index snd thus the backlink pointing to the index will recieve the most 'love"

baddog · 07-21-2008, 12:32 PM

Quote:

Originally Posted by pussyserver

you are all wrong

Google will spider the link most relative to the content of the index page first...

that being said... there is no such thing as spider first sice the google bot is in fact millions of bots that all spider constantly

then 2 pages or 20 pages on a site can all be indexed at the same time

the page most relative to the index snd thus the backlink pointing to the index will recieve the most 'love"

How will it know which is most relevant without spidering all first?

pussyserver - BANNED FOR LIFE · 07-21-2008, 12:37 PM

Quote:

Originally Posted by baddog

How will it know which is most relevant without spidering all first?

I meant to say index first not spider first

it spiders everything by default ... but the algo decides on what gets indexed and in what order

Viper.X08 · 07-21-2008, 12:41 PM

the first page who will give way for Google to visit your page.

baddog · 07-21-2008, 01:56 PM

Quote:

Originally Posted by pussyserver

I meant to say index first not spider first

That wasn't the question.

http://www.gofuckyourself.com/showpo...88&postcount=1

pussyserver - BANNED FOR LIFE · 07-21-2008, 01:59 PM

Quote:

Originally Posted by baddog

That wasn't the question.

http://www.gofuckyourself.com/showpo...88&postcount=1

yes i know that wasnt the question... but since the original question didnt make any sense .... I assumed the OP meant index... not spider

d-null · 07-21-2008, 02:07 PM

Quote:

Originally Posted by pussyserver

I meant to say index first not spider first

it spiders everything by default ... but the algo decides on what gets indexed and in what order

actually I'm not sure about that "spidering everything".... googlebot's algorithm makes decisions on what to spider, especially with newer sites..... a site that gets huge fast will not be completely spidered right away, or completely over and over, it would take up too much bandwidth and I think googlebot puts a relevancy factor on sites to decide how much time to spend on them

I've noticed a major shift in googlebot's spidering activities seems to occur with some sites when they hit a certain stage.... eg. site with 10,000 pages that google doesn't have trust in yet might only get 20% spidered and 5% indexed; then all of the sudden some months down the road much more will get spidered and indexed

(just from what I have seen, other factors like penalties may have been involved I guess, but no way of knowing that from where I'm sitting.... try it if you want to test it, start a brand new site and add 10,000 pages to it right away and see if googlebot spiders the whole works... I doubt that it will, at least not right away, and for sure google won't index all 10,000 pages right away either)

pussyserver - BANNED FOR LIFE · 07-21-2008, 02:18 PM

Quote:

Originally Posted by jetjet

actually I'm not sure about that "spidering everything".... googlebot's algorithm makes decisions on what to spider, especially with newer sites..... a site that gets huge fast will not be completely spidered right away, or completely over and over, it would take up too much bandwidth and I think googlebot puts a relevancy factor on sites to decide how much time to spend on them)

again you are talking about indexing

in order for a url to be process by the algorithm it has to be spidered

The googlebot spiders ( crawls ) everything

But not everything is added to the index

you see in order to really understand SEO you must first understand what google is

Google is a massive automated index... nothing more

just an index

But I should proably stop as I get passionate about SEO and wouldnt want to give anything away

d-null · 07-21-2008, 02:23 PM

I was talking about spidering and indexing

I think googlebot will not completely spider a new site that grows extremely fast, at least not right away

I've noticed that before but I wasn't paying close enough attention to say it with certainty and there are always other possible variables to consider

maybe things have changed now and googlebot will completely spider a brand new 10,000 page site immediately, just saying what I noticed in the past

SmokeyTheBear · 07-21-2008, 11:36 PM

Quote:

Originally Posted by jetjet

things have changed now and googlebot will completely spider a brand new 10,000 page site immediately,

SmokeyTheBear · 07-21-2008, 11:41 PM

Quote:

Originally Posted by pussyserver

that being said... there is no such thing as spider first sice the google bot is in fact millions of bots that all spider constantly

wtf ? obviously something gets hit first , google does not simultaniously spider multiple pages, yes they do use multiple bots but they dont all pound your site at the same time or it would crash your site. Your server works the same way , it doesnt handle all tasks at once, it processes them at the same time yes , but it still takes them in an order

SmokeyTheBear · 07-21-2008, 11:44 PM

Quote:

Originally Posted by baddog

So you are suggesting that the spider takes the time to look at all the links, then sort in order of length?

exactly.

basically if it finds 200 new pages it records the urls , then schedules the order they will be spidered by length.

Snake Doctor · 07-22-2008, 12:31 AM

The answer to the question is

There is no money in SEO

2012 · 07-22-2008, 12:36 AM

the spider does whatever it wants, whenever it wants , however way it wants to do it that day/hour/min/sec

... enjoy

Marky333 · 07-22-2008, 04:29 AM

Quote:

Originally Posted by SmokeyTheBear

exactly.

basically if it finds 200 new pages it records the urls , then schedules the order they will be spidered by length.

Anyone tested this?

07-20-2008, 10:37 PM	#6
WiredGuy Pounding Googlebot Industry Role: Join Date: Aug 2002 Location: Canada Posts: 34,485	Robots.txt WG __________________ I play with Google.

07-20-2008, 10:42 PM	#8
Makingcoin Confirmed User Join Date: Aug 2002 Location: The Ditch Posts: 8,919	grape.html ;-) Smokey, hit me up please. __________________ www.MAKINGCOIN.com icq. 166-662-831 "Start making large coin!" Daddy I Get Paid To Be A Whore - Coming Soon

07-20-2008, 11:07 PM	#9
d-null . . . Industry Role: Join Date: Apr 2007 Location: NY Posts: 13,724	all four simultaneously __________________ __________________ Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad! Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs Check out the #1 WordPress SEO Plugin: CyberSEO Suite

07-21-2008, 01:56 AM	#12
AlienQ - BANNED FOR LIFE best designer on GFY Join Date: Mar 2003 Location: IALIEN.COM - High Definition Video and Photographic Productions -ICQ 78943384 Posts: 30,307	Use Robot.txt Why leave things to chance? Tell the fucker where to go. __________________ NAKED HOSTING FTW!11 I'm On The INSANE PLAN $9.95/mo! \| The Alien Blog Adult News Worth Reading Updated Daily \| Content For Sale! 641 PICS 216 MINUTES OF VIDEO $350.00 \|ICQ: 78943384 \|

07-21-2008, 02:21 AM	#16
SmokeyTheBear ►SouthOfHeaven Join Date: Jun 2004 Location: PlanetEarth MyBoardRank: GerbilMaster My-Penis-Size: extralarge MyWeapon: Computer Posts: 28,609	i will narrow it down . 5 choices. #1 Google will spider the links in alphabetical order. #2 Google will spider the links by length. #3 Google will spider the links in the order it found them. #4 Google will spider the links by importance of keyword #5 Google will spider them randomly. __________________ hatisblack at yahoo.com

07-20-2008, 10:27 PM	#1
SmokeyTheBear ►SouthOfHeaven Join Date: Jun 2004 Location: PlanetEarth MyBoardRank: GerbilMaster My-Penis-Size: extralarge MyWeapon: Computer Posts: 28,609	SEO test If you register a new domain put up a site with 5 pages , the main page and 4 pages linked from the main domain the 4 pages are named apple.html pear.html squash.html tomato.html what page will google spider first ? __________________ hatisblack at yahoo.com

07-20-2008, 10:28 PM	#2
stickyfingerz Doin fine Industry Role: Join Date: Oct 2005 Posts: 24,984	The index?

07-20-2008, 10:35 PM	#5
baddog So Fucking Banned Industry Role: Join Date: Apr 2001 Location: the beach, SoCal Posts: 107,089	Which page is the first link Google sees on the main page?

07-20-2008, 10:41 PM	#7
fallenmuffin Confirmed User Industry Role: Join Date: Nov 2005 Posts: 8,170	If listed in order.. apple.html ?

07-20-2008, 11:38 PM	#10
Iron Fist Too lazy to set a custom title Join Date: Dec 2006 Posts: 23,400	What ever fruit Google Labs says is more popular at the time...

07-20-2008, 11:43 PM	#11
Iron Fist Too lazy to set a custom title Join Date: Dec 2006 Posts: 23,400	Okay i'll back that up.... Apple would be first to index...

07-21-2008, 02:35 AM	#18
d-null . . . Industry Role: Join Date: Apr 2007 Location: NY Posts: 13,724	whichever shows up first in the page source so #3 __________________ __________________ Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad! Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs Check out the #1 WordPress SEO Plugin: CyberSEO Suite

07-21-2008, 03:43 AM	#19
SmokeyTheBear ►SouthOfHeaven Join Date: Jun 2004 Location: PlanetEarth MyBoardRank: GerbilMaster My-Penis-Size: extralarge MyWeapon: Computer Posts: 28,609	the answer is PEAR.html reason = #2 __________________ hatisblack at yahoo.com

07-21-2008, 03:48 AM	#20
Traf Confirmed User Industry Role: Join Date: Mar 2006 Posts: 132	Google will crawl the first link indicated in your index page. IF this is the order you used in your index page then google will crawl apple.html first. apple.html pear.html squash.html tomato.html __________________ SEO Problem? Email: [email protected] ICQ: 686736528

07-21-2008, 04:20 AM	#22
beta-tester Rock 'n Roll Baby! Join Date: Sep 2004 Location: USA, temporarly Posts: 22,562	We can't be 100% sure which url it'll spider first. But I pretty much think it uses heuristic method in doing that. __________________ Sig for sale. Affordable prices. Contact me and get a great deal ;) My contact: ICQ: 944-320-46 e-mail: manca {AT} HotFreeSex4All.com

07-21-2008, 06:48 AM	#24
WiredGuy Pounding Googlebot Industry Role: Join Date: Aug 2002 Location: Canada Posts: 34,485	Is this just from self-testing or something more formal? My last test I had about 800 pages in total, all pages were of the format: http://www.domain.com/Category_Name/ http://www.domain.com/Category_Name/index1 - index5 (.shtml) http://www.domain.com/Category_Name/8-Character-Code (.shtml) I noticed the first 80 pages indexed were full from the last set, domain, category, 8-character-code.shtml. It was never the category or index pages that got indexed first. WG __________________ I play with Google.

07-21-2008, 08:08 AM	#26
V_RocKs Damn Right I Kiss Ass! Industry Role: Join Date: Dec 2003 Location: Cowtown, USA Posts: 32,409	longer names get love last.

07-21-2008, 10:16 AM	#27
Deej I make pixels work Industry Role: Join Date: Jun 2005 Location: I live here... Posts: 24,386	assuming google does spider each link... within a timely manner for each... is there an advantage to having google spider certain pages first? __________________ Deej's Designs n' What Not Hit me up for Design, CSS & Photo Retouching Icq#30096880

07-21-2008, 12:17 PM	#31
baddog So Fucking Banned Industry Role: Join Date: Apr 2001 Location: the beach, SoCal Posts: 107,089	So you are suggesting that the spider takes the time to look at all the links, then sort in order of length?

07-21-2008, 12:26 PM	#32
pussyserver - BANNED FOR LIFE So Fucking Banned Join Date: Oct 2005 Location: I convert perverts like catholic church! Posts: 5,133	you are all wrong Google will spider the link most relative to the content of the index page first... that being said... there is no such thing as spider first sice the google bot is in fact millions of bots that all spider constantly then 2 pages or 20 pages on a site can all be indexed at the same time the page most relative to the index snd thus the backlink pointing to the index will recieve the most 'love"

07-21-2008, 12:41 PM	#35
Viper.X08 Confirmed User Join Date: Nov 2007 Posts: 275	the first page who will give way for Google to visit your page. __________________ Make Money In Adult http://www.pantybucks.com/ http://www.richfetish.com/

07-21-2008, 02:23 PM	#40
d-null . . . Industry Role: Join Date: Apr 2007 Location: NY Posts: 13,724	I was talking about spidering and indexing I think googlebot will not completely spider a new site that grows extremely fast, at least not right away I've noticed that before but I wasn't paying close enough attention to say it with certainty and there are always other possible variables to consider maybe things have changed now and googlebot will completely spider a brand new 10,000 page site immediately, just saying what I noticed in the past __________________ __________________ Looking for a custom TUBE SCRIPT that supports massive traffic, load balancing, billing support, and h264 encoding? Hit up Konrad! Looking for designs for your websites or custom tubesite design? Hit up Zuzana Designs Check out the #1 WordPress SEO Plugin: CyberSEO Suite

07-22-2008, 12:31 AM	#44
Snake Doctor I'm Lenny2 Bitch Join Date: Mar 2001 Location: On top of my soapbox Posts: 13,449	The answer to the question is There is no money in SEO __________________ sig too big

07-22-2008, 12:36 AM	#45
2012 So Fucking What Industry Role: Join Date: Jul 2006 Posts: 17,189	the spider does whatever it wants, whenever it wants , however way it wants to do it that day/hour/min/sec ... enjoy Last edited by 2012; 07-22-2008 at 12:39 AM..