How do I stop Googlebot from raping my site? 8000+ spider hits?

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Solid Bob
    Confirmed User
    • Apr 2006
    • 1213
    #1

    How do I stop Googlebot from raping my site? 8000+ spider hits?

    Site is under 5 days old and I get this crap!

    Googlebot 3221 49.17 MB 16 Sep 2006 - 04:52
    Inktomi Slurp 3045 24.24 MB 16 Sep 2006 - 04:53
    Unknown robot (identified by 'crawl') 1610 31.45 MB 16 Sep 2006 - 04:53
    MSNBot 59 557.32 KB 16 Sep 2006 - 04:53
    AskJeeves 39 574.24 KB 16 Sep 2006 - 04:24


    Just kidding. If you want to invest in building some automated traffic targetted to any market email me with your needs and lets chat.
    [email protected]
    Tags: None
    • elron
      Confirmed User
      • Jul 2003
      • 1553
      #2
      easy to do with .htaccess

        Comment

        • Marshal
          Biz Dev and SEO
          • Jun 2005
          • 15224
          #3
          that's more than good... congrats!
          ---
          Busy ranking websites on Google...

            Comment

            • Barefootsies
              Choice is an Illusion
              • Feb 2005
              • 42635
              #4
              Originally posted by elron
              easy to do with .htaccess

              Should You Email Your Members?

              Link1 | Link2 | Link3

              Enough Said.

              "Would you rather live like a king for a year or like a prince forever?"

                Comment

                • scottybuzz
                  Too lazy to set a custom title
                  • May 2006
                  • 14799
                  #5
                  its been a long walk
                  $$$$$ MAKE HUGE MONEY IN CAMS - CLICK HERE $$$$$

                    Comment

                    • Deej
                      I make pixels work
                      • Jun 2005
                      • 24386
                      #6
                      Originally posted by scottybuzz
                      its been a long walk

                      Deej's Designs n' What Not
                      Hit me up for Design, CSS & Photo Retouching


                      Icq#30096880

                        Comment

                        • Deej
                          I make pixels work
                          • Jun 2005
                          • 24386
                          #7
                          Code:
                          <META NAME="robots" CONTENT="No, Google noooo!! Dont Rape ME!!!!">

                          Deej's Designs n' What Not
                          Hit me up for Design, CSS & Photo Retouching


                          Icq#30096880

                            Comment

                            • martinsc
                              Too lazy to set a custom title
                              • Jun 2005
                              • 27043
                              #8
                              Originally posted by Deej
                              Make Money

                                Comment

                                • borked
                                  Totally Borked
                                  • Feb 2005
                                  • 6284
                                  #9
                                  robots.txt

                                  For coding work - hit me up on andy // borkedcoder // com
                                  (consider figuring out the email as test #1)


                                  All models are wrong, but some are useful. George E.P. Box. p202

                                    Comment

                                    • baddog
                                      So Fucking Banned
                                      • Apr 2001
                                      • 107089
                                      #10
                                      Originally posted by borked
                                      robots.txt

                                      that's the one

                                        Comment

                                        • Solid Bob
                                          Confirmed User
                                          • Apr 2006
                                          • 1213
                                          #11
                                          Thanks for all the help! That'll show you google.com
                                          [email protected]

                                            Comment

                                            • spasmo
                                              Confirmed User
                                              • Dec 2005
                                              • 2678
                                              #12
                                              That's where Googlebot went. Send it back my way.

                                              Yahoo Slurp 4890+744 26.77 MB 16 Sep 2006 - 15:00
                                              Googlebot 3265+18 18.58 MB 16 Sep 2006 - 12:07
                                              Alexa (IA Archiver) 430+28 1.68 MB 16 Sep 2006 - 11:41
                                              MSNBot 179+65 964.07 KB 15 Sep 2006 - 02:36
                                              GigaBot 113+67 371.31 KB 10 Sep 2006 - 00:10
                                              Google AdSense 167+1 869.29 KB 16 Sep 2006 - 10:23

                                              Surfers: Go here for hot babes.

                                                Comment

                                                • Fizzgig
                                                  Registered User
                                                  • Feb 2004
                                                  • 9649
                                                  #13
                                                  What about that Google sitemap thing?
                                                  ---'-,-{@ Sassy Grrrl @}-'-,---

                                                    Comment

                                                    • Solid Bob
                                                      Confirmed User
                                                      • Apr 2006
                                                      • 1213
                                                      #14
                                                      Originally posted by spasmo
                                                      That's where Googlebot went. Send it back my way.

                                                      Yahoo Slurp 4890+744 26.77 MB 16 Sep 2006 - 15:00
                                                      Googlebot 3265+18 18.58 MB 16 Sep 2006 - 12:07
                                                      Alexa (IA Archiver) 430+28 1.68 MB 16 Sep 2006 - 11:41
                                                      MSNBot 179+65 964.07 KB 15 Sep 2006 - 02:36
                                                      GigaBot 113+67 371.31 KB 10 Sep 2006 - 00:10
                                                      Google AdSense 167+1 869.29 KB 16 Sep 2006 - 10:23
                                                      Nice stats but as a note the ones I posted are from a 4.5 day old site that I never have to touch again in my life. ;)
                                                      [email protected]

                                                        Comment

                                                        • spasmo
                                                          Confirmed User
                                                          • Dec 2005
                                                          • 2678
                                                          #15
                                                          Originally posted by Solid Bob
                                                          Nice stats but as a note the ones I posted are from a 4.5 day old site that I never have to touch again in my life. ;)
                                                          Exactly, which is why I want you to send Googlebot back to me, you botnapper!

                                                          Surfers: Go here for hot babes.

                                                            Comment

                                                            • Solid Bob
                                                              Confirmed User
                                                              • Apr 2006
                                                              • 1213
                                                              #16
                                                              Originally posted by spasmo
                                                              Exactly, which is why I want you to send Googlebot back to me, you botnapper!
                                                              $5000 and I'll hook it up.
                                                              [email protected]

                                                                Comment

                                                                • chadglni
                                                                  Confirmed User
                                                                  • Dec 2002
                                                                  • 6924
                                                                  #17
                                                                  You got arachnophobia or something?


                                                                  Sign up here - Dating Site affiliate program

                                                                    Comment

                                                                    • woj
                                                                      <&(©¿©)&>
                                                                      • Jul 2002
                                                                      • 47880
                                                                      #18
                                                                      good job
                                                                      Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000
                                                                      Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager
                                                                      Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip Manager

                                                                        Comment

                                                                        • wedouglas
                                                                          So Fucking Banned
                                                                          • Aug 2004
                                                                          • 5921
                                                                          #19
                                                                          25352 1.74% Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.co
                                                                          43779 3.01% Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/

                                                                            Comment

                                                                            • gooddomains
                                                                              Too lazy to set a custom title
                                                                              • Jul 2003
                                                                              • 10127
                                                                              #20
                                                                              who needs google

                                                                                Comment

                                                                                • Solid Bob
                                                                                  Confirmed User
                                                                                  • Apr 2006
                                                                                  • 1213
                                                                                  #21
                                                                                  Originally posted by gooddomains
                                                                                  who needs google
                                                                                  I don't need google. I consider google the same as my woman bringing a sexy girlfriend home for some late night fun. Just an extra bonus that makes my pecker hard.
                                                                                  [email protected]

                                                                                    Comment

                                                                                    • chadglni
                                                                                      Confirmed User
                                                                                      • Dec 2002
                                                                                      • 6924
                                                                                      #22


                                                                                      Sign up here - Dating Site affiliate program

                                                                                        Comment

                                                                                        • tranza
                                                                                          ICQ: 197-556-237
                                                                                          • Jun 2003
                                                                                          • 57559
                                                                                          #23
                                                                                          Congrats I guess....
                                                                                          I'm just a newbie.

                                                                                            Comment

                                                                                            • Alky
                                                                                              Confirmed User
                                                                                              • Apr 2002
                                                                                              • 5651
                                                                                              #24
                                                                                              Originally posted by Solid Bob
                                                                                              I don't need google. I consider google the same as my woman bringing a sexy girlfriend home for some late night fun. Just an extra bonus that makes my pecker hard.
                                                                                              and you'd tell her to leave?

                                                                                                Comment

                                                                                                • Solid Bob
                                                                                                  Confirmed User
                                                                                                  • Apr 2006
                                                                                                  • 1213
                                                                                                  #25
                                                                                                  Originally posted by Alky
                                                                                                  and you'd tell her to leave?
                                                                                                  Not if she knew how to read.
                                                                                                  [email protected]

                                                                                                    Comment

                                                                                                    • chaze
                                                                                                      Confirmed User
                                                                                                      • Aug 2002
                                                                                                      • 9774
                                                                                                      #26
                                                                                                      Jesus how 45 megs of html, you must need some sleep right about now.
                                                                                                      Like the desert needs the rain
                                                                                                      We do fully manged WordPress, VPS, and Servers. Adult Host Pro https://adulthostpro.com/ Since 2001

                                                                                                        Comment

                                                                                                        • borked
                                                                                                          Totally Borked
                                                                                                          • Feb 2005
                                                                                                          • 6284
                                                                                                          #27
                                                                                                          Originally posted by chaze
                                                                                                          Jesus how 45 megs of html, you must need some sleep right about now.
                                                                                                          html or image caching?

                                                                                                          For coding work - hit me up on andy // borkedcoder // com
                                                                                                          (consider figuring out the email as test #1)


                                                                                                          All models are wrong, but some are useful. George E.P. Box. p202

                                                                                                            Comment

                                                                                                            • squishypimp
                                                                                                              PostMaster General
                                                                                                              • Aug 2006
                                                                                                              • 10781
                                                                                                              #28
                                                                                                              im interested in this googlebot deal!

                                                                                                                Comment

                                                                                                                • free4porn
                                                                                                                  Confirmed User
                                                                                                                  • Jun 2005
                                                                                                                  • 4654
                                                                                                                  #29
                                                                                                                  google bot is a whore
                                                                                                                  Switch To Fling Now! I'm on 1:201 paid signups sending little traffic! Make $$$
                                                                                                                  Free Porn

                                                                                                                    Comment

                                                                                                                    Previous template Next
                                                                                                                    Working...