What is the best way to mass download an entire site?
 

I'd like to be able to log-in, and have this download every file with as little effort as possible.

I know this is done everyday, what is the software called?

frontpage :p





ur killin me ya know this

this is why you put a virus in your members area that can't be seen but will be downloaded when a program is used to suck your site dry :)

Ohhhhh. I like the sound of that.

You have one to sell?

we been beat up for a few days.. i'll take one too

the virus or the program that downloads entire sites? I do know of a members area that has this in it. The program you can probably find on google

wget will do it.

Thats a great idea actually, some scum bag ripped one of my sites and put it up on a bt.

It works well

Never tried it, but might have to agree with "wget" or the ton of graphical clients that crawl and download from sites.

You find a site that still uses basic authentication and offers video downloads that aren't DRM'd, you could pretty much own their shit with free software out there.

Fuck it. Hit up a password sharing site, find all that use basic auth who you have a grudge against, and work their shit.

Using basic auth and allowing free downloads is like dropping a wallet on the street. -- You find it. It's yours.

wget fo shizzle

wget, it even has a windows port now.

Theres hundreds of progs that download entire sites - d/l limits and block lists cure most of the problems...... like the sound of the virus though, any more info?

wget -r [site]

or

man wget

I'd like to wget a fuckin boot up their ass!

You can lead wget down the garden path and fill their disk space up pretty quickly with junk - apparently well formed jpeg files that are really large and don't contain images. You can use links that normal surfers won't see to do this (1x1 pixels linked, example, near the top of each page). Track whoever hits those links, and ban the IP. It will certainly slow them down.

Good luck.

That's pretty slick.

Wget, of course. wget can be set to ignore robots.txt, so a good thing to do is make a hidden directory full of crap, say /d/, and just fill it with a bunch of random data, and have your analyzer watch for that. There are many programs written for UNIX that can even realtime watch your apache logs and fire off an email with the person's IP (and if using normal authentication methods, their username).

jact knows his shit.

That is an awesome idea. :thumbsup

i used webzip for mirroring or downloading websites when i was a pc user