GoFuckYourself.com - Adult Webmaster Forum

GoFuckYourself.com - Adult Webmaster Forum (https://gfy.com/index.php)

- Fucking Around & Business Discussion (https://gfy.com/forumdisplay.php?f=26)

- - Need a link stripper (https://gfy.com/showthread.php?t=585253)

pr0	03-10-2006 01:19 PM

Need a link stripper

Any programs out there that can hit a url, & export a text list of every link on the page

in this fashion

http://url1.com
http://url2.com
http://url3.com

pr0	03-10-2006 01:33 PM

bump bitch!

Love Sex

03-10-2006 01:43 PM

yes anyone know, that could be really useful

pr0	03-10-2006 01:49 PM

Quote:

Originally Posted by Love Sex

yes anyone know, that could be really useful

well help me keep this bumped...someones got one

PolySix

03-10-2006 02:50 PM

I use Velocityscape's webscraper. It is rather spendy, but has a lot of bells and whistles to it. It's automation features are rather nice.

If you want free, and don't mind processing offline, then you can use this from the perl handbook:

Code:

#!/usr/bin/perl

  

use HTML::LinkExtor;

  

my $FILENAME = 'file.html';

  

$parser = HTML::LinkExtor->new(undef, $base_url);

$parser->parse_file($FILENAME);

@links = $parser->links;

foreach $linkarray (@links) {

    my @element = @$linkarray;

    my $elt_type = shift @element; # element type

  

    # possibly test whether this is an element we're interested in

    while (@element) {

        # extract the next attribute and its value

        my ($attr_name, $attr_value) = splice(@element, 0, 2);

        if ($elt_type eq 'a' && $attr_name eq 'href') {

            print "ANCHOR: $attr_value\n" 

        }

    }

}

Love Sex

03-10-2006 04:14 PM

http://www.velocityscape.com/ nice software
Wouldnt a link stripper and a site scrapper be two diffent items in a sense?

PolySix

03-10-2006 10:50 PM

Quote:

Originally Posted by Love Sex

www velocityscape com nice software
Wouldnt a link stripper and a site scrapper be two diffent items in a sense?

Yes, they would be considered two different items. I guess a more accurate way to say it would be that, link stripping (link extracting), is one basic example of the many things that can be done with a product like webscraper.

It's probably overkill for what the original poster wants to accomplish. But, if you need to do this sort of thing a lot, and want to automate the process as much as possible, then it's an excellent tool.

All times are GMT -7. The time now is 12:45 AM.