Help!! Why would I be getting 50,000 (302 error) redirects a day!!? - GoFuckYourself.com

Damian_Maxcash · 10-21-2006, 07:00 PM

Its driving me nuts - Im not getting close to that traffic on the site so I dont think its to do with a couple of redirects in the htaccess file......

Is there a way to see what this traffic is and see where its coming from and going to?

Thanks

squishypimp · 10-21-2006, 07:01 PM

check your AWstats or Webalizer to see were the errors are occuring.

Damian_Maxcash · 10-21-2006, 07:05 PM

Quote:

Originally Posted by squishypimp

check your AWstats or Webalizer to see were the errors are occuring.

AWStats dosnt show them - it just gives you a count

I havnt looked at Webalizer - I will try it now.

GrouchyAdmin · 10-21-2006, 07:05 PM

Mentioning the site, and/or posting your htaccess might help.

SmokeyTheBear · 10-21-2006, 07:07 PM

Quote:

Originally Posted by damian2001

AWStats dosnt show them - it just gives you a count

I havnt looked at Webalizer - I will try it now.

try looking at the last 300 visitors if you have that option in cpanel you should notice

Chris · 10-21-2006, 07:09 PM

download log files?

Damian_Maxcash · 10-21-2006, 07:15 PM

Quote:

Originally Posted by SmokeyTheBear

try looking at the last 300 visitors if you have that option in cpanel you should notice

After calming down (watching 50000 uniques disappear can have an adverse effect on your health)

It turns out its bloody search engine bots looking for a robots.txt file.

The bots have pulled over a gig of that server today - really strange.....

but at least Im not losing traffic.

Thanks guys

Tempest · 10-21-2006, 07:15 PM

what type of site? tgp? running links thru a script?

Tempest · 10-21-2006, 07:19 PM

Quote:

Originally Posted by damian2001

It turns out its bloody search engine bots looking for a robots.txt file.

Put an empty robots.txt file on there.. that's what I do.

GrouchyAdmin · 10-21-2006, 07:20 PM

Quote:

Originally Posted by Tempest

Put an empty robots.txt file on there.. that's what I do.

Or, you know, actually make one.

User-Agent: *
Disallow: /sekret

woj · 10-21-2006, 07:37 PM

look in the raw log files...

Tempest · 10-21-2006, 08:09 PM

Quote:

Originally Posted by toonpornblog

Or, you know, actually make one.

User-Agent: *
Disallow: /sekret

true.. on the other hand, now every smart person can come along and pull up your robots.txt file and see you have this "hidden" sekret directory.

GrouchyAdmin · 10-21-2006, 08:23 PM

Quote:

Originally Posted by Tempest

true.. on the other hand, now every smart person can come along and pull up your robots.txt file and see you have this "hidden" sekret directory.

That's what .htaccess/.htpasswd is for. A few sites penalize you for a 0 byte robots.txt; webcrawler is no longer so they don't count, but I know it still makes Yahoo unhappy.

Tempest · 10-22-2006, 12:59 AM

Quote:

Originally Posted by toonpornblog

That's what .htaccess/.htpasswd is for. A few sites penalize you for a 0 byte robots.txt; webcrawler is no longer so they don't count, but I know it still makes Yahoo unhappy.

Interesting.. I have a site that Yahoo seems to love and it's got a 0 length robots.txt file in there.

.htpasswd etc. is ok.. but it's even better when they don't even know where to look in the first place.

10-21-2006, 07:00 PM	#1
Damian_Maxcash So Fucking Banned Join Date: Oct 2002 Location: MaxCash.com Posts: 12,745	Help!! Why would I be getting 50,000 (302 error) redirects a day!!? Its driving me nuts - Im not getting close to that traffic on the site so I dont think its to do with a couple of redirects in the htaccess file...... Is there a way to see what this traffic is and see where its coming from and going to? Thanks

10-21-2006, 07:01 PM	#2
squishypimp PostMaster General Join Date: Aug 2006 Posts: 10,781	check your AWstats or Webalizer to see were the errors are occuring. __________________ Signature For Sale

10-21-2006, 07:05 PM	#4
GrouchyAdmin Now choke yourself! Industry Role: Join Date: Apr 2006 Posts: 12,085	Mentioning the site, and/or posting your htaccess might help. __________________ AIM: GrouchyGfy

10-21-2006, 07:09 PM	#6
Chris Too lazy to set a custom title Industry Role: Join Date: May 2003 Location: icq: 71462500 Skype: Jupzchris Posts: 27,880	download log files? __________________ [email protected]

10-21-2006, 07:37 PM	#11
woj <&(©¿©)&> Industry Role: Join Date: Jul 2002 Location: Chicago Posts: 47,882	look in the raw log files... __________________ Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000 Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip Manager

10-21-2006, 07:15 PM	#8
Tempest Too lazy to set a custom title Industry Role: Join Date: May 2004 Location: West Coast, Canada. Posts: 10,217	what type of site? tgp? running links thru a script?