| 
		
			
			
				
			
			
				 
			
			
				
			
		 | 
		
			
			
				 
			
				
			
		 | 
	||||
| 
				Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.  You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us.  | 
		
		 
		![]()  | 
	
		
			
  | 	
	
	
		
		|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. | 
| 
		 | 
	Thread Tools | 
| 
			
			 | 
		#1 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: May 2001 
				Location: San Diego 
				
				
					Posts: 1,550
				 
				
				
				
				 | 
	
	
	
	
		
			
			 
				
				Could Someone Share Their .htaccess File For Blocking Offline Browsers?
			 
			Hi, 
		
	
		
		
		
		
		
	
	I've been getting clobbered lately, especially by Teleport Pro. But, from what I've read, Teleport masks itself as a browser so it's very hard to block. Anyone successfully figured this out? thanks, marc  | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#2 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: Jun 2002 
				
				
				
					Posts: 3,739
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 Here is my current .htaccess file with all the Offline Browsers I've come across so far ... 
		
	
		
		
		
		
		
	
	[small]RewriteEngine On RewriteCond %{HTTP_USER_AGENT} ^attach [OR] RewriteCond %{HTTP_USER_AGENT} ^BackWeb [OR] RewriteCond %{HTTP_USER_AGENT} ^Bandit [OR] RewriteCond %{HTTP_USER_AGENT} ^BatchFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Buddy [OR] RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR] RewriteCond %{HTTP_USER_AGENT} ^Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Copier [OR] RewriteCond %{HTTP_USER_AGENT} ^Bot\mailto:[email protected] [OR] RewriteCond %{HTTP_USER_AGENT} ^DA [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\Demon [OR] RewriteCond %{HTTP_USER_AGENT} ^Download\Wonder [OR] RewriteCond %{HTTP_USER_AGENT} ^Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^Drip [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^Express\WebPictures [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR] RewriteCond %{HTTP_USER_AGENT} ^FileHound [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR] RewriteCond %{HTTP_USER_AGENT} ^GetSmart [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR] RewriteCond %{HTTP_USER_AGENT} ^gotit [OR] RewriteCond %{HTTP_USER_AGENT} ^Grabber [OR] RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR] RewriteCond %{HTTP_USER_AGENT} ^HMView [OR] RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^Iria [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR] RewriteCond %{HTTP_USER_AGENT} ^JOC [OR] RewriteCond %{HTTP_USER_AGENT} ^JustView [OR] RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR] RewriteCond %{HTTP_USER_AGENT} ^lftp [OR] RewriteCond %{HTTP_USER_AGENT} ^likse [OR] RewriteCond %{HTTP_USER_AGENT} ^Magnet [OR] RewriteCond %{HTTP_USER_AGENT} ^Mag-Net [OR] RewriteCond %{HTTP_USER_AGENT} ^Mass\Downloader [OR] RewriteCond %{HTTP_USER_AGENT} ^Memo [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool [OR] RewriteCond %{HTTP_USER_AGENT} ^Mirror [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX [OR] RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR] RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR] RewriteCond %{HTTP_USER_AGENT} ^Net\Vampire [OR] RewriteCond %{HTTP_USER_AGENT} ^NetZip [OR] RewriteCond %{HTTP_USER_AGENT} ^Ninja [OR] RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto [OR] RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR] RewriteCond %{HTTP_USER_AGENT} ^Pockey [OR] RewriteCond %{HTTP_USER_AGENT} ^Pump [OR] RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^Reaper [OR] RewriteCond %{HTTP_USER_AGENT} ^Recorder [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR] RewriteCond %{HTTP_USER_AGENT} ^Siphon [OR] RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR] RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR] RewriteCond %{HTTP_USER_AGENT} ^Snake [OR] RewriteCond %{HTTP_USER_AGENT} ^SpaceBison [OR] RewriteCond %{HTTP_USER_AGENT} ^Stripper [OR] RewriteCond %{HTTP_USER_AGENT} ^Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR] RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR] RewriteCond %{HTTP_USER_AGENT} ^Vacuum [OR] RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR] RewriteCond %{HTTP_USER_AGENT} ^Website [OR] RewriteCond %{HTTP_USER_AGENT} ^Webster [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget [OR] RewriteCond %{HTTP_USER_AGENT} ^Whacker [OR] RewriteCond %{HTTP_USER_AGENT} ^Widow [OR] RewriteCond %{HTTP_USER_AGENT} ^Xaldon RewriteRule /*$ http://www.yourdomain.com [L,R][/small]  | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#3 | 
| 
			
			
			
			 So Fucking Banned 
			
		
			
			
			Industry Role:  
				Join Date: Apr 2001 
				Location: N.Y. -Long Island -- 
				
				
					Posts: 122,992
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 sec ill grab mine also 
		
	
		
		
		
		
		
	
	 | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#4 | 
| 
			
			
			
			 So Fucking Banned 
			
		
			
			
			Industry Role:  
				Join Date: Apr 2001 
				Location: N.Y. -Long Island -- 
				
				
					Posts: 122,992
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 RewriteEngine On 
		
	
		
		
		
		
		
	
	RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR] RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja.* [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR] RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX.* [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Need\To\Know.* [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer.* [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR] RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR] RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.* RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Backstreet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Caitoo.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Eclipt.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*FlashSite.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GetBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GetSmart.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Grab.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Greed.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HtGet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*iFox98.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Vampire.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Eater.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snarf.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Devil.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Retriever.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WebVCR.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WWWCopy.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WWWoffle.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Aculinx.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*DownloadIt.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Express.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*FlipBrowser.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Weazel.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*IPhoto.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*MemoWeb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*NetDrag.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Commander.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Webexe.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Extractor.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Hoover.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Keepoint.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Down2Web.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Burner.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snatcher.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$ RewriteRule /* http://www.fastsize.com/?rid=1438 [R,L]  | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#5 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: May 2004 
				Location: South Florida 
				
				
					Posts: 4,134
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 has anyone figured out a way to successfully block offline browsers? I was looking into quotas and such but never got around to implimenting it. 
		
	
		
		
		
		
		
	
	 | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#6 | |
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: Jun 2002 
				
				
				
					Posts: 3,739
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 Quote: 
	
  | 
|
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#7 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: May 2001 
				Location: San Diego 
				
				
					Posts: 1,550
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 I don't think that method works properly for the modern offline browsers... like Teleport Pro for example.  I just did a test on juicylinks (sorry Juicy) and it sucked the whole thing down in 30 seconds. 
		
	
		
		
		
		
		
	
	Anyone else have any successful way of combatting this stuff? marc  | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#8 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: Jun 2002 
				
				
				
					Posts: 3,739
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 Limit the bandwidth per IP or you could  replace links with Jscript: 
		
	
		
		
		
		
		
	
	e.g. replace <a href="bigfile.zip">Foo</a> with : <script language="javascript"> <!-- document.write('<a h' + 're' + 'f="'); document.write('bigfile' + '.' + 'zip">'); // --> </script> Foo </a>  | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#9 | |
| 
			
			
			
			 [----------------------] 
			
		
			
			
			Join Date: Aug 2001 
				
				
				
					Posts: 14,486
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 Quote: 
	
 unless the links are uncrawlable you will have a hard time.  | 
|
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#10 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Industry Role:  
				Join Date: Sep 2002 
				Location: In your mind 
				
				
					Posts: 3,766
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 For security reasons I won't paste my entire block list, heres an extract that may help: 
		
	
		
		
		
		
		
	
	(Had to cut down the number due to 10k length) RewriteEngine On RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro.* [OR] RewriteCond %{HTTP_USER_AGENT} ^DISCo\Pump.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Drip.* [OR] RewriteCond %{HTTP_USER_AGENT} ^EirGrabber.* [OR] RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE.* [OR] RewriteCond %{HTTP_USER_AGENT} ^FlashGet.* [OR] RewriteCond %{HTTP_USER_AGENT} ^GetRight.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Gets.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Grafula.* [OR] RewriteCond %{HTTP_USER_AGENT} ^IBrowse.* [OR] RewriteCond %{HTTP_USER_AGENT} ^InterGET.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Internet\Ninja.* [OR] RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [OR] RewriteCond %{HTTP_USER_AGENT} ^JustView.* [OR] RewriteCond %{HTTP_USER_AGENT} ^MIDown\tool.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Mister\PiX.* [OR] RewriteCond %{HTTP_USER_AGENT} ^NearSite.* [OR] RewriteCond %{HTTP_USER_AGENT} ^NetSpider.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Offline\Explorer.* [OR] RewriteCond %{HTTP_USER_AGENT} ^PageGrabber.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Papa\Foto.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ReGet.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Slurp.* [OR] RewriteCond %{HTTP_USER_AGENT} ^SpaceBison.* [OR] RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebAuto.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebFetch.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebReaper.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebStripper.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebWhacker.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebZIP.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Image\Collector.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Web\Sucker.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Webster.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Wget.* [OR] RewriteCond %{HTTP_USER_AGENT} ^eCatch.* [OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archiver.* [OR] RewriteCond %{HTTP_USER_AGENT} ^lftp.* [OR] RewriteCond %{HTTP_USER_AGENT} ^tAkeOut.* RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Backstreet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Caitoo.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Eclipt.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*FlashSite.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GetBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GetSmart.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GetWeb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Grab.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Greed.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HtGet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*iFox98.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Vampire.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Eater.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snarf.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Devil.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Retriever.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WebVCR.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WWWCopy.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WWWoffle.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Aculinx.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*DownloadIt.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Express.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*FlipBrowser.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Weazel.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*IPhoto.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*MemoWeb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*NetDrag.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Commander.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Webexe.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Extractor.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Hoover.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Keepoint.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Enterprise.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Down2Web.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Burner.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snatcher.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$ RewriteCond %{HTTP_USER_AGENT} ^.*Backweb.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Ants.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snagger.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Pump.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*leech.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*likse.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR] RewriteCond %{HTTP_USER_AGENT} ^DA\ 5.0.* [OR] RewriteCond %{HTTP_USER_AGENT} ^DA\ 5.3.* [OR] RewriteCond %{HTTP_USER_AGENT} ^DA\ 4.0.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger\ 1.20b.* [OR] RewriteCond %{HTTP_USER_AGENT} ^WebSauger\ 1.20b.* [OR] RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR] RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR] RewriteCond %{HTTP_USER_AGENT} ^CherryPickerSE [OR] RewriteCond %{HTTP_USER_AGENT} ^CherryPickerElite [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR] RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR] RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archive [OR] RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR] RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR] RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]  | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 | 
| 
			
			 | 
		#11 | 
| 
			
			
			
			 Confirmed User 
			
		
			
			
			Join Date: Sep 2001 
				Location: Boston 
				
				
					Posts: 4,873
				 
				
				
				
				 | 
	
	
	
	
		
		
		
		 its probly easier to do a if not - MSIE / mozilla etc... 
		
	
		
		
		
		
		
	
	 | 
| 
		 | 
	
	
	
		
                 
		
		
		
		
		
		
		
			
			
		
	 |