Help!! Why would I be getting 50,000 (302 error) redirects a day!!?
 

Its driving me nuts - Im not getting close to that traffic on the site so I dont think its to do with a couple of redirects in the htaccess file......

Is there a way to see what this traffic is and see where its coming from and going to?

Thanks

check your AWstats or Webalizer to see were the errors are occuring.

AWStats dosnt show them - it just gives you a count

I havnt looked at Webalizer - I will try it now.

Mentioning the site, and/or posting your htaccess might help.

try looking at the last 300 visitors if you have that option in cpanel you should notice

download log files?

After calming down (watching 50000 uniques disappear can have an adverse effect on your health)

It turns out its bloody search engine bots looking for a robots.txt file.

The bots have pulled over a gig of that server today - really strange.....

but at least Im not losing traffic.

Thanks guys :)

what type of site? tgp? running links thru a script?

Put an empty robots.txt file on there.. that's what I do.

Or, you know, actually make one.

User-Agent: *
Disallow: /sekret

look in the raw log files...

true.. on the other hand, now every smart person can come along and pull up your robots.txt file and see you have this "hidden" sekret directory.

That's what .htaccess/.htpasswd is for. A few sites penalize you for a 0 byte robots.txt; webcrawler is no longer so they don't count, but I know it still makes Yahoo unhappy. :thumbsup

Interesting.. I have a site that Yahoo seems to love and it's got a 0 length robots.txt file in there.

.htpasswd etc. is ok.. but it's even better when they don't even know where to look in the first place. :2 cents: