How to do a batch link check for 404's?
 

Hi guys...
I was just wondering if anybody has a website or software that can take a list of URL's and check for 404's, while making note of which URL's are 404'ing or redirecting. Basically, I'm looking for something like:
http://www.seoconsultants.com/tools/...ch.asp#results

...But it will need to take more than 25 links (hopefully thousands at once), and hopefully only note what URL's are 404'ing or redirecting. Any assistance is appreciated! Thanks! :thumbsup

Example:
http://www.url1.com
http://www.url2.com
http://www.url3.com
http://www.url4.com (maybe this one is a 404 link and redirects, so it gets reported)

I think somebody wrote a script here on GFY to do this once... it's very simple, I'm sure one of the programmers could write one up quick.

http://home.snafu.de/tilman/xenulink.html

By url, by text file, by page. Pretty much does all error codes I think too, I've had lots of scans hit in the mid hundred thousand, so more then 25 easily.

What he said. XENU is the shit

This is a great tool, but not exactly what I need. I need to put in a list of URL's and see if they redirect the main page...Not check the links on an already loading page. In essence, this is a TGP/MGP gallery check.

Thanks for the suggestion though! This is helpful as well, once I get past step 1. :thumbsup

My only suggestion is to hire an outsourced programmer on one of the sites to get it done for cheap and fast.

I agree with ry0t

I can write that kind of script for you very fast...

Do you just want it to check for redirects? That's easy.

It accepts one URL per line. If you typo or fuck up, it'll give a generic error. The last one always returns "does not redirect" due to a threading problem with the PHP CGI on my host.

Please don't kill my machine, this simple little thing isn't designed to handle thousands of requests. :thumbsup