Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today!

If you have any problems with the registration process or your account login, please contact us.

Post New Thread Reply

Register GFY Rules Calendar
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
How do I stop Googlebot from raping my site? 8000+ spider hits? How do I stop Googlebot from raping my site? 8000+ spider hits?
Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed.

 
Thread Tools
Old 09-16-2006, 09:24 AM   #1
Solid Bob
Confirmed User
 
Join Date: Apr 2006
Location: Easy Webcam Pro
Posts: 1,213
How do I stop Googlebot from raping my site? 8000+ spider hits?
Site is under 5 days old and I get this crap!

Googlebot 3221 49.17 MB 16 Sep 2006 - 04:52
Inktomi Slurp 3045 24.24 MB 16 Sep 2006 - 04:53
Unknown robot (identified by 'crawl') 1610 31.45 MB 16 Sep 2006 - 04:53
MSNBot 59 557.32 KB 16 Sep 2006 - 04:53
AskJeeves 39 574.24 KB 16 Sep 2006 - 04:24


Just kidding. If you want to invest in building some automated traffic targetted to any market email me with your needs and lets chat.
__________________
[email protected]
Solid Bob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 09:30 AM   #2
elron
Confirmed User
 
Join Date: Jul 2003
Posts: 1,553
easy to do with .htaccess
elron is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 09:53 AM   #3
Marshal
Biz Dev and SEO
 
Marshal's Avatar
 
Industry Role:
Join Date: Jun 2005
Posts: 15,181
that's more than good... congrats!
__________________
---
Busy ranking websites on Google...
Marshal is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 09:59 AM   #4
Barefootsies
Choice is an Illusion
 
Barefootsies's Avatar
 
Industry Role:
Join Date: Feb 2005
Location: Land of Obama
Posts: 42,635
:2cents
Quote:
Originally Posted by elron
easy to do with .htaccess

__________________
Should You Email Your Members?

Link1 | Link2 | Link3

Enough Said.

"Would you rather live like a king for a year or like a prince forever?"
Barefootsies is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 10:12 AM   #5
scottybuzz
Too lazy to set a custom title
 
scottybuzz's Avatar
 
Industry Role:
Join Date: May 2006
Location: NY
Posts: 14,800
its been a long walk
scottybuzz is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 10:16 AM   #6
Deej
I make pixels work
 
Deej's Avatar
 
Industry Role:
Join Date: Jun 2005
Location: I live here...
Posts: 24,386
Quote:
Originally Posted by scottybuzz
its been a long walk
__________________

Deej's Designs n' What Not
Hit me up for Design, CSS & Photo Retouching


Icq#30096880
Deej is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 10:18 AM   #7
Deej
I make pixels work
 
Deej's Avatar
 
Industry Role:
Join Date: Jun 2005
Location: I live here...
Posts: 24,386
Code:
<META NAME="robots" CONTENT="No, Google noooo!! Dont Rape ME!!!!">
__________________

Deej's Designs n' What Not
Hit me up for Design, CSS & Photo Retouching


Icq#30096880
Deej is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 10:22 AM   #8
martinsc
Too lazy to set a custom title
 
Industry Role:
Join Date: Jun 2005
Location: 127.0.0.1
Posts: 27,047
Quote:
Originally Posted by Deej
__________________
Make Money
martinsc is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 11:56 AM   #9
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
robots.txt
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)


All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 11:57 AM   #10
baddog
So Fucking Banned
 
Industry Role:
Join Date: Apr 2001
Location: the beach, SoCal
Posts: 107,089
Quote:
Originally Posted by borked
robots.txt

that's the one
baddog is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:04 PM   #11
Solid Bob
Confirmed User
 
Join Date: Apr 2006
Location: Easy Webcam Pro
Posts: 1,213
Thanks for all the help! That'll show you google.com
__________________
[email protected]
Solid Bob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:21 PM   #12
spasmo
Confirmed User
 
Join Date: Dec 2005
Location: Couch
Posts: 2,678
That's where Googlebot went. Send it back my way.

Yahoo Slurp 4890+744 26.77 MB 16 Sep 2006 - 15:00
Googlebot 3265+18 18.58 MB 16 Sep 2006 - 12:07
Alexa (IA Archiver) 430+28 1.68 MB 16 Sep 2006 - 11:41
MSNBot 179+65 964.07 KB 15 Sep 2006 - 02:36
GigaBot 113+67 371.31 KB 10 Sep 2006 - 00:10
Google AdSense 167+1 869.29 KB 16 Sep 2006 - 10:23
__________________

Surfers: Go here for hot babes.
spasmo is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:34 PM   #13
Fizzgig
Registered User
 
Fizzgig's Avatar
 
Join Date: Feb 2004
Location: In Your Dreams
Posts: 9,649
What about that Google sitemap thing?
__________________
---'-,-{@ Sassy Grrrl @}-'-,---
Fizzgig is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:36 PM   #14
Solid Bob
Confirmed User
 
Join Date: Apr 2006
Location: Easy Webcam Pro
Posts: 1,213
Quote:
Originally Posted by spasmo
That's where Googlebot went. Send it back my way.

Yahoo Slurp 4890+744 26.77 MB 16 Sep 2006 - 15:00
Googlebot 3265+18 18.58 MB 16 Sep 2006 - 12:07
Alexa (IA Archiver) 430+28 1.68 MB 16 Sep 2006 - 11:41
MSNBot 179+65 964.07 KB 15 Sep 2006 - 02:36
GigaBot 113+67 371.31 KB 10 Sep 2006 - 00:10
Google AdSense 167+1 869.29 KB 16 Sep 2006 - 10:23
Nice stats but as a note the ones I posted are from a 4.5 day old site that I never have to touch again in my life. ;)
__________________
[email protected]
Solid Bob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:43 PM   #15
spasmo
Confirmed User
 
Join Date: Dec 2005
Location: Couch
Posts: 2,678
Quote:
Originally Posted by Solid Bob
Nice stats but as a note the ones I posted are from a 4.5 day old site that I never have to touch again in my life. ;)
Exactly, which is why I want you to send Googlebot back to me, you botnapper!
__________________

Surfers: Go here for hot babes.
spasmo is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:47 PM   #16
Solid Bob
Confirmed User
 
Join Date: Apr 2006
Location: Easy Webcam Pro
Posts: 1,213
Quote:
Originally Posted by spasmo
Exactly, which is why I want you to send Googlebot back to me, you botnapper!
$5000 and I'll hook it up.
__________________
[email protected]
Solid Bob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:47 PM   #17
chadglni
Confirmed User
 
Join Date: Dec 2002
Location: PEI, Canada
Posts: 6,924
You got arachnophobia or something?
chadglni is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:48 PM   #18
woj
<&(©¿©)&>
 
woj's Avatar
 
Industry Role:
Join Date: Jul 2002
Location: Chicago
Posts: 47,882
good job
__________________
Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000
Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager
Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip Manager
woj is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:54 PM   #19
wedouglas
So Fucking Banned
 
Join Date: Aug 2004
Location: Dearborn, MI
Posts: 5,921
25352 1.74% Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.co
43779 3.01% Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/
wedouglas is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 12:56 PM   #20
gooddomains
Too lazy to set a custom title
 
Join Date: Jul 2003
Location: Netherlands
Posts: 10,127
who needs google
gooddomains is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 01:22 PM   #21
Solid Bob
Confirmed User
 
Join Date: Apr 2006
Location: Easy Webcam Pro
Posts: 1,213
Quote:
Originally Posted by gooddomains
who needs google
I don't need google. I consider google the same as my woman bringing a sexy girlfriend home for some late night fun. Just an extra bonus that makes my pecker hard.
__________________
[email protected]
Solid Bob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 01:26 PM   #22
chadglni
Confirmed User
 
Join Date: Dec 2002
Location: PEI, Canada
Posts: 6,924
chadglni is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 01:35 PM   #23
tranza
ICQ: 197-556-237
 
Join Date: Jun 2003
Location: BRASIL !!!
Posts: 57,559
Congrats I guess....
__________________
I'm just a newbie.
tranza is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 01:35 PM   #24
Alky
Confirmed User
 
Alky's Avatar
 
Join Date: Apr 2002
Location: Houston
Posts: 5,651
Quote:
Originally Posted by Solid Bob
I don't need google. I consider google the same as my woman bringing a sexy girlfriend home for some late night fun. Just an extra bonus that makes my pecker hard.
and you'd tell her to leave?
Alky is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 01:36 PM   #25
Solid Bob
Confirmed User
 
Join Date: Apr 2006
Location: Easy Webcam Pro
Posts: 1,213
Quote:
Originally Posted by Alky
and you'd tell her to leave?
Not if she knew how to read.
__________________
[email protected]
Solid Bob is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 03:09 PM   #26
chaze
Confirmed User
 
Industry Role:
Join Date: Aug 2002
Posts: 9,752
Jesus how 45 megs of html, you must need some sleep right about now.
chaze is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 03:15 PM   #27
borked
Totally Borked
 
borked's Avatar
 
Industry Role:
Join Date: Feb 2005
Posts: 6,284
Quote:
Originally Posted by chaze
Jesus how 45 megs of html, you must need some sleep right about now.
html or image caching?
__________________

For coding work - hit me up on andy // borkedcoder // com
(consider figuring out the email as test #1)


All models are wrong, but some are useful. George E.P. Box. p202
borked is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 03:37 PM   #28
squishypimp
PostMaster General
 
Join Date: Aug 2006
Posts: 10,781
im interested in this googlebot deal!
__________________
squishypimp is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 09-16-2006, 04:14 PM   #29
free4porn
Confirmed User
 
Join Date: Jun 2005
Posts: 4,654
google bot is a whore
__________________
Switch To Fling Now! I'm on 1:201 paid signups sending little traffic! Make $$$
Free Porn
free4porn is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Post New Thread Reply
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
How do I stop Googlebot from raping my site? 8000+ spider hits? How do I stop Googlebot from raping my site? 8000+ spider hits?
« Previous Thread | Next Thread »

Bookmarks



Advertising inquiries - marketing at gfy dot com

Contact Admin - Advertise - GFY Rules - Top

©2000-, AI Media Network Inc



Powered by vBulletin
Copyright © 2000- Jelsoft Enterprises Limited.