Anyone Familiar with error logs, know what this is?
 

I was checking out my error logs (I'm a bit parinoid after what happened at my last hos[Fri Apr 20 15:12:43 2001] [error] [client ip address] File does not exist: /server path url/robots.txt)

What might be looking for a robots.txt file on my main root domain of my server? I find this error in all my domains. Is this a search engine? Something to worry about?

Web_Mistress =

Most nice searchengine spiders will look for the robot.txt file to see if you have any instructions for them, to as what they can spider, and what you want left alone http://bbs.gofuckyourself.com/board/smile.gif No need to be worried.


Tim

Thanks!

Sooo, given that, should I put something there for them, since I seem to be getting a lot of queries in the past few days.... any recommendations?

Web_Mistress =

Check out http://www.searchtools.com/robots/ they have a basic layout of what the robots.txt does. I myself don't use them, maybe to my detriment http://bbs.gofuckyourself.com/board/smile.gif Maybe someone else with a better understanding of how we can use the robots.txt to increase SE placement can speak up ?? Thanks.


Tim

I just put a simple one in that said "don't search my cgi-bin". I see where you can tell it directories not to search, but not what TOO search.

------------------
~Every time I call myself Webmistress, I feel I should be in spike heels with a whip~

KICK ASS content and sponsor Oliver Klozov ~ Quality babes

Web_Mistress =

off topic, but I was wondering with Oliver's program if the 30% on the recurring hurt's sales, or does he make up for that by keeping the customer longer. Thanks.


Tim

I just signed up for the program, and sold 4 in one day...ok, I'm still small, that was the best I've done.

So I'm still waiting to see with the 30%. Since I tend to do "babe" galleries, he is a good niche for the sites I build. Might not work as well for someone who doesn't.

Personally, I think his photos are on the par of suze Randall which is why I like it. I feel like I am selling quality.

I know it doesn't answer your question, but I'll let you know in a month how they hold...



------------------
~Every time I call myself Webmistress, I feel I should be in spike heels with a whip~

KICK ASS content and sponsor Oliver Klozov ~ Quality babes