Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today!

If you have any problems with the registration process or your account login, please contact us.

Post New Thread Reply

Register GFY Rules Calendar
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed.

 
Thread Tools
Old 10-07-2015, 11:19 AM   #1
2MuchMark
Videochat Solutions
 
2MuchMark's Avatar
 
Industry Role:
Join Date: Aug 2004
Location: Canada
Posts: 49,090
Question for SEO Gurus out there

Hey all... question for the SEO Gurus:

If you have a dynamic link that does not exist all the time, how do you stop Engines from indexing it or validators from flagging it as invalid?

For example, I have a link that looks like this:

/Directory/SubDirectory/ProgramName.cgi/content

Sometimes this content is online and sometimes it is not.

I have tried inserting rel="nofollow" into the link, but validators still see this as 404 error.

What am I missing?
__________________

Custom Coding | Videochat Solutions | Age Verification | IT Help & Support
www.2Much.net
2MuchMark is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 11:36 AM   #2
Rob
I'm a great bowler.
 
Rob's Avatar
 
Industry Role:
Join Date: Nov 2003
Location: Right Outside of Normal.
Posts: 13,309
Use .htaccess to omit.
__________________
Rob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 11:42 AM   #3
Google Expert
Webmaster
 
Google Expert's Avatar
 
Industry Role:
Join Date: Jun 2004
Posts: 14,294
put this into your robots.txt

User-agent: *
Disallow: /Directory/SubDirectory/

or

User-agent: *
Disallow: /Directory/SubDirectory/ProgramName.cgi
Google Expert is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 02:42 PM   #4
2MuchMark
Videochat Solutions
 
2MuchMark's Avatar
 
Industry Role:
Join Date: Aug 2004
Location: Canada
Posts: 49,090
Quote:
Originally Posted by Rob View Post
Use .htaccess to omit.
Can you show me an example?
__________________

Custom Coding | Videochat Solutions | Age Verification | IT Help & Support
www.2Much.net
2MuchMark is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 02:49 PM   #5
Fat Panda
Porn is Dead. Move along.
 
Fat Panda's Avatar
 
Industry Role:
Join Date: Aug 2006
Posts: 13,295
https://en.wikipedia.org/wiki/Noindex
Fat Panda is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 03:12 PM   #6
2MuchMark
Videochat Solutions
 
2MuchMark's Avatar
 
Industry Role:
Join Date: Aug 2004
Location: Canada
Posts: 49,090
Quote:
Originally Posted by Muad'Dib View Post
put this into your robots.txt

User-agent: *
Disallow: /Directory/SubDirectory/

or

User-agent: *
Disallow: /Directory/SubDirectory/ProgramName.cgi


Hi Muad'Dib,

Thanks for the info. For now, page validators keep showing the pages I don't want indexed anyway, so I'm thinking that only htaccess will work in this case. I'll use your suggestion too just to be sure.


Quote:
Originally Posted by Fat Panda View Post
Hi Fat,

I already have rel="nofollow" in the links to those pages. Do you think I should use rel="no index" too?
__________________

Custom Coding | Videochat Solutions | Age Verification | IT Help & Support
www.2Much.net
2MuchMark is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 03:45 PM   #7
Google Expert
Webmaster
 
Google Expert's Avatar
 
Industry Role:
Join Date: Jun 2004
Posts: 14,294
Quote:
Originally Posted by ********** View Post
Hi Muad'Dib,

Thanks for the info. For now, page validators keep showing the pages I don't want indexed anyway, so I'm thinking that only htaccess will work in this case. I'll use your suggestion too just to be sure.
Page validators are not the same as SE bots.

You are basically giving (allowing) the address to page validators to go to and check.

But when SE bots come to your site, they read the instructions first and don't go to those directories, because it's forbidden.
Google Expert is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 07:30 PM   #8
O MARINA
I'm clockin' ya, Versace shade watchin' ya
 
O MARINA's Avatar
 
Industry Role:
Join Date: Mar 2003
Location: internet
Posts: 13,795
Quote:
Originally Posted by Muad'Dib View Post
Page validators are not the same as SE bots.

You are basically giving (allowing) the address to page validators to go to and check.

But when SE bots come to your site, they read the instructions first and don't go to those directories, because it's forbidden.
Oh hello
Seo Guru hit me up sometime!
O MARINA is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 07:33 PM   #9
Fat Panda
Porn is Dead. Move along.
 
Fat Panda's Avatar
 
Industry Role:
Join Date: Aug 2006
Posts: 13,295
u want this between head tags on that page

<meta name="robots" content="noindex">
Fat Panda is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 10-07-2015, 07:34 PM   #10
Fat Panda
Porn is Dead. Move along.
 
Fat Panda's Avatar
 
Industry Role:
Join Date: Aug 2006
Posts: 13,295
rel="nofollow" has nothing to do with what youre trying to do
Fat Panda is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Post New Thread Reply
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >

Bookmarks

Tags
link, validators, question, gurus, seo, online, content, missing, rel=nofollow, inserting, error, exist, time, dynamic, hey, invalid, flagging, indexing, stop, engines



Advertising inquiries - marketing at gfy dot com

Contact Admin - Advertise - GFY Rules - Top

©2000-, AI Media Network Inc



Powered by vBulletin
Copyright © 2000- Jelsoft Enterprises Limited.