|
|
|
|
||||
|
Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us. |
 |
|
|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. |
|
|
Thread Tools |
09-07-2009, 12:00 PM
|
#1 |
|
So Fucking Outlawed
Industry Role:
Join Date: Nov 2001
Posts: 5,114
|
BIZ - Here is an SEO tip for the non-seo expert
Include a robots.txt File at the root of your site.
Open up a text editor, type User-agent: *. Then save the file as robots.txt and upload it to your root directory on your domain. This one command will tell the spiders that hit your site to crawl every page. The search engine analyzes everything it indexes to determine what your website is all about, it is a good idea to block certain folders and files that have nothing to do with the content you want analyzed. You can disallow unrelated files to be read by adding "Disallow: /folder_name/" or "Disallow: /filename.html" User-Agent: * Disallow: /cgi-bin/ Disallow: /img/ 
__________________
ICQ 115433750 |
|
|
|
09-07-2009, 12:07 PM
|
#2 |
|
aliasx
Join Date: Apr 2001
Posts: 19,010
|
Good tip, some people who know about it don't even use it. Myself included, should though.. . It is the first thing google bot looks for every time.
__________________
https://porncorporation.com |
|
|
|
|
09-07-2009, 07:07 PM
|
#3 |
|
Pounding Googlebot
Industry Role:
Join Date: Aug 2002
Location: Canada
Posts: 34,482
|
Why would you tell the SE's to ignore your images? You'll lose out on all image search traffic from yahoo/google.
WG
__________________
I play with Google. |
|
|
|
|
09-07-2009, 07:15 PM
|
#4 | |
|
Confirmed User
Join Date: Nov 2005
Location: ICQ - 703894
Posts: 1,949
|
Quote:
__________________
|
|
|
|
|
|
09-07-2009, 07:22 PM
|
#5 | |
|
So Fucking Outlawed
Industry Role:
Join Date: Nov 2001
Posts: 5,114
|
Quote:
__________________
ICQ 115433750 |
|
|
|
|
|
09-07-2009, 07:25 PM
|
#6 |
|
Rock 'n Roll Baby!
Join Date: Sep 2004
Location: USA, temporarly
Posts: 22,562
|
There is no reason using robots.txt on images. Use it only on folders you really don't like indexed, for example: admin panels, user submission forms, protected folders, and so on. By default spiders will craw whole domain without problems, so no need to include robots.txt, if you don't have intentions to blocking or allowing only certain parts of your site.
__________________
Sig for sale. Affordable prices. Contact me and get a great deal ;) My contact: ICQ: 944-320-46 e-mail: manca {AT} HotFreeSex4All.com |
|
|
|
|
09-07-2009, 07:26 PM
|
#7 |
|
Confirmed User
Join Date: Sep 2002
Location: Los Angeles
Posts: 6,830
|
thanks!!!!
and happy bday!
__________________
I can resist everything except temptation |
|
|
|
|
09-07-2009, 07:26 PM
|
#8 | |
|
♥ ♦ ♣ ♠
Industry Role:
Join Date: Sep 2002
Posts: 10,592
|
Quote:
__________________
 "I'm selflessly supporting the common good, but only coincidentally looking out for No.1." |
|
|
|
|
|
09-07-2009, 07:43 PM
|
#9 |
|
Confirmed User
Industry Role:
Join Date: Feb 2009
Posts: 3,068
|
yea what he said
|
|
|
|
|
09-07-2009, 08:05 PM
|
#10 |
|
So Fucking Banned
Industry Role:
Join Date: Feb 2003
Location: Boston
Posts: 4,160
|
Thank you for the advice. Much appreciated.
|
|
|
|
