robots.txt file?

**fl_prn_str** · 03-03-2005, 11:52 AM

bump 4 u

**tungsten** · 03-03-2005, 11:53 AM

bump, im pretty sure the meta tags and the file serve the same purpose

**psili** · 03-03-2005, 11:54 AM

I think it's used more to define parts you don't want crawled. The meta tags should take care if you just want to be crawled.

**GigaAlan** · 03-03-2005, 12:03 PM

It's changing also, the spiders are looking for more actual page content to make ranking. I'm starting to notice that my description tags are not getting in the listing but any text from the page is including link text.

It's a changing world for search engines.

**nico-t** · 03-03-2005, 04:00 PM

Originally posted by psili

I think it's used more to define parts you don't want crawled. The meta tags should take care if you just want to be crawled.

yeah thats what i think too... so no robots text file for me

**nico-t** · 03-03-2005, 04:02 PM

Originally posted by GigaAlan

It's changing also, the spiders are looking for more actual page content to make ranking. I'm starting to notice that my description tags are not getting in the listing but any text from the page is including link text.

It's a changing world for search engines.

yeah, when i search for my page on google and my galleries and free sites pop up, some dont have a description text below the link at all. But thats when i do a pure domain search, when u search for words it lists parts of the sentences form the page that contain those words.

robots.txt file?

robots.txt file?

Comment

Comment

Comment

Comment

Comment

Comment