How to map / get idea of a large site?

**clickity click** · 02-13-2015, 10:18 AM

Just use something like httrack which will make a directory on your PC with all the files. If you don't need images you can disable scanning for them.
Most sites have a sitemap as well. Usually something like site.com/sitemap.xml

Might be worth checking first.

**Harmon** · 02-13-2015, 10:22 AM

So basically, and let me get this straight... you are 2 posts deep into this forum and you are trying to copy somebody elses business?

You'll fit right in.

**The Porn Nerd** · 02-13-2015, 10:27 AM

Originally posted by Harmon

So basically, and let me get this straight... you are 2 posts deep into this forum and you are trying to copy somebody elses business?

You'll fit right in.

**sandman!** · 02-13-2015, 11:11 AM

Originally posted by Harmon

So basically, and let me get this straight... you are 2 posts deep into this forum and you are trying to copy somebody elses business?

You'll fit right in.

**Bladewire** · 02-13-2015, 01:41 PM

Originally posted by Harmon

So basically, and let me get this straight... you are 2 posts deep into this forum and you are trying to copy somebody elses business?

You'll fit right in.

Originally posted by The Porn Nerd

Originally posted by sandman!

**AdultKing** · 02-13-2015, 01:49 PM

Originally posted by Harmon

So basically, and let me get this straight... you are 2 posts deep into this forum and you are trying to copy somebody elses business?

You'll fit right in.

This forum only attracts assholes now.

**Barry-xlovecam** · 02-13-2015, 03:48 PM

learn to hack :P

ssh user@target
user@target password:

omitted

scp user@target:

omitted

ssh again

user@target:~$/path to document route/$ rm remote-tree.csv
user@target:~$ cd~
user@target:~$ nano .bash_history
You want root so you can clean your connections from the system logs
su {or sudo}
??? look it up l4m3r :P
nano pico vi whatever ...
delete your tracks ^x save
^d

And don't use your mother's IP

**epitome** · 02-13-2015, 03:53 PM

How the fuck was the name AdultSites not registered on GFY until 2015?

Seems like such an obvious choice.

**freecartoonporn** · 02-13-2015, 09:34 PM

are you scraping Wikipedia ?

post the url so i can give some input,

**AdultSites** · 02-14-2015, 02:28 AM

Originally posted by clickity click

Just use something like httrack which will make a directory on your PC with all the files. If you don't need images you can disable scanning for them.
Most sites have a sitemap as well. Usually something like site.com/sitemap.xml

Might be worth checking first.

This program does not seem to be perfect, but I may be doing it wrong too. I will look at the settings, and see what I can make of it. It looks like Teleport Pro, and other download managing programs. They dont seem to be 100% good. They should just work, but there seem to be a lot of bs with that. In the end I end up checking on these programs a lot, instead of getting everything done in the background. Again, I may be doing it wrong, as I did not pay too much attention to this, I was just testing.

**clickity click** · 02-14-2015, 03:55 AM

what exactly are you trying to do. What it the site. You do know you can't just copy a site and have it running unless it's HTML only.

**Roald** · 02-14-2015, 04:55 AM

Originally posted by AdultKing

This forum only attracts assholes now.

this forum always attracted assholes. nothing new really.

How to map / get idea of a large site?

How to map / get idea of a large site?

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment