Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums.

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today!

If you have any problems with the registration process or your account login, please contact us.

Post New Thread Reply

Register GFY Rules Calendar
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
best download spider? best download spider?
Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed.

 
Thread Tools
Old 03-08-2003, 06:23 PM   #1
strainer
Confirmed User
 
Join Date: Oct 2002
Location: Philadelphia
Posts: 418
best download spider?
There are a ton of these out there, which works the best for spidering a members site and downloading the content. (So I have to be able to feed it a username/password).

I'm trying to backup a site for a technically challenged webmaster...
strainer is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 06:27 PM   #2
completetosser
Confirmed User
 
Join Date: Jan 2003
Location: Planet Earth
Posts: 1,419
Why don't you just tar it up?
completetosser is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 06:29 PM   #3
strainer
Confirmed User
 
Join Date: Oct 2002
Location: Philadelphia
Posts: 418
Quote:
Originally posted by completetosser
Why don't you just tar it up?
He doesn't have ftp access, much less ssh. long story. I know those dowload spiders work well cause my members use them all the time to suck the whole site down...
strainer is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 06:32 PM   #4
p00p
Confirmed User
 
Join Date: Dec 2002
Location: CanaDUH
Posts: 5,125
Quote:
Originally posted by strainer
There are a ton of these out there, which works the best for spidering a members site and downloading the content. (So I have to be able to feed it a username/password).

I'm trying to backup a site for a technically challenged webmaster...
I've tried quite a few, and this one best replicates directory structure, and gets everything, no missed files.
http://www.httrack.com/index.php
p00p is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 06:46 PM   #5
3Xguru
Confirmed User
 
Join Date: Sep 2001
Location: Iasi
Posts: 1,281
WebCopier is very good and can submit forms too
__________________
-=Free Webmaster Tools=-
3Xguru is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 06:48 PM   #6
SpaceAce
Confirmed User
 
Join Date: Jul 2002
Location: Magrathea
Posts: 6,493
Quote:
Originally posted by strainer


He doesn't have ftp access, much less ssh. long story. I know those dowload spiders work well cause my members use them all the time to suck the whole site down...
Just pick the one from your logs that does the most damage.

SpaceAce
SpaceAce is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 08:33 PM   #7
lil2rich4u2
ICQ: 175171926
 
Join Date: Oct 2002
Location: New York, NY
Posts: 11,046
theres a program on download.com called GetPics
lil2rich4u2 is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 09:26 PM   #8
strainer
Confirmed User
 
Join Date: Oct 2002
Location: Philadelphia
Posts: 418
Quote:
Originally posted by p00p

I've tried quite a few, and this one best replicates directory structure, and gets everything, no missed files.
http://www.httrack.com/index.php
Pretty cool... Its running now. I had already wasted my money on Teleport which doesn't seem to work too well and put everything in one directory.
strainer is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 09:30 PM   #9
p00p
Confirmed User
 
Join Date: Dec 2002
Location: CanaDUH
Posts: 5,125
Quote:
Originally posted by strainer


Pretty cool... Its running now. I had already wasted my money on Teleport which doesn't seem to work too well and put everything in one directory.
Yeah. Once you figure that program out (and it isn't hard) it will download everything, and do a perfect replication. It is the only one I have used over the years that actually does this everytime consistently. I used to use offline explorer, and it would never go deep enough into the directory structure to get everything....
p00p is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Old 03-08-2003, 09:33 PM   #10
X37375787
Guest
 
Posts: n/a
Teleport Pro works well on certain teen hardcore vid sites...
  Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote
Post New Thread Reply
Go Back   GoFuckYourself.com - Adult Webmaster Forum > >
best download spider? best download spider?
« Previous Thread | Next Thread »

Bookmarks



Advertising inquiries - marketing at gfy dot com

Contact Admin - Advertise - GFY Rules - Top

©2000-, AI Media Network Inc



Powered by vBulletin
Copyright © 2000- Jelsoft Enterprises Limited.