Sitemap maker? Lists all pages in excel or text
 

I am looking to get a list of all url's in a website. There are about 100000 or so product ID's that are spread out in no specific order. Example www.website.com/productid=554887 can be a product and www.website.com/productid=554888 can be a 404 error but then www.website.com/productid=554899 can be a product again. Basically I want to rip the website by providing the urls to all the products but since the id's are random I need to build a list of links first. Hope I explained this right. Anyone know of a good software I can use to build this list?

You should just pull it straight out of the db, rather than use some software to crawl your site... if you are interested in some custom solution, hit me up icq:33375924

i don't have access to the db, it's hosted on another server, one of those turnkey websites. will hit you up later though

try http://www.drk.com.ar/spider.php . It's free and pretty good, not sure if it will work with that amount of pages, but you can give it a try, it's 100% free

download the sitemap, and parse the urls and grab them based on that.