.htaccess list of those damn content stealing bot things

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • xanx
    Confirmed User
    • Apr 2002
    • 3711

    #1

    .htaccess list of those damn content stealing bot things

    anyone want to share their list. search function is disabled as I know these have been posted before.
  • Lycanthrope
    Confirmed User
    • Jan 2004
    • 4517

    #2
    For bots and offline leachers you could add

    RewriteCond %{HTTP_REFERER} ^$
    ReWriteRule .*[Jj][Pp][Gg]$|.*[Jj][Pp][Ee][Gg]$|.*[Gg][Ii][Ff]$ - [F]

    This will return a 403 error, but this will also stop SEs from picking up your images.

    Comment

    • xanx
      Confirmed User
      • Apr 2002
      • 3711

      #3
      Originally posted by Lycanthrope
      For bots and offline leachers you could add

      RewriteCond %{HTTP_REFERER} ^$
      ReWriteRule .*[Jj][Pp][Gg]$|.*[Jj][Pp][Ee][Gg]$|.*[Gg][Ii][Ff]$ - [F]

      This will return a 403 error, but this will also stop SEs from picking up your images.
      I was worried about theat SE part as I get a lot of traffic from google images =(

      Comment

      • peter_mansion
        Registered User
        • Jan 2004
        • 45

        #4
        A lot of guys(browsers) do not send http_referer it can be yp to 10%..
        They are dsiabled by firewall, proxies or browser(e.g OPERA)
        Bare this in mind..

        Comment

        • directfiesta
          Too lazy to set a custom title
          • Oct 2002
          • 30217

          #5
          Is this what you want???

          RewriteEngine On
          RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Iria.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Pockey.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*DA.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Ninja.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*DISCoPump.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Drip.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Gets.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*IBrowse.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*InternetNinja.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*JustView.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*MIDownool.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*MisterPiX.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*OfflineExplorer.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*PapaFoto.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Pockey.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Slurp.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*SpaceBison.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebImageCollector.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*WebSucker.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*lftp.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ [OR]
          RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$
          RewriteRule /*$ http://www.redirectURLhere [L,R]
          I know that Asspimple is stoopid ... As he says, it is a FACT !

          But I can't figure out how he can breathe or type , at the same time ....

          Comment

          • xanx
            Confirmed User
            • Apr 2002
            • 3711

            #6
            yes, thanks man

            Comment

            • Clovis
              Confirmed User
              • Jan 2001
              • 412

              #7
              Originally posted by directfiesta
              Is this what you want???
              a very thorough rewrite list
              thanx
              Thank God for little rebills
              www.jadecandle.com

              just fucking submit

              Comment

              • Fabuleux
                Confirmed User
                • May 2003
                • 2543

                #8
                Is that going to use a lot of CPU power?

                Comment

                • Mr. Mike
                  Confirmed User
                  • Aug 2004
                  • 913

                  #9
                  excellent list, directfiesta! Thanks.
                  Sig for sale. ICQ: 163-545-054

                  Comment

                  • Webmistress Pookie
                    Confirmed User
                    • Feb 2004
                    • 243

                    #10
                    what does the %haha123 stand for?

                    Comment

                    Working...