Help with link-text extractor

[Webmaster-Toolkit]

Hi

I'm needing a bit of help with a certain part of a script I'm writing. The code I'm using to extract the link text of a specified web-page seems to work on most pages, but unfortunately it doesn't pick up everything when you try it out on our illustrious administrator's (;)) page - http://www.freemoneyservices.com/

The code I'm using is this (simplified a bit):

$url = (the url inputted)
$html = get($url);

# remove newlines and multiple spaces
$html =~ s/\n//g;
$html =~ s/\r//g;
$html =~ s/\s+/ /g;

# get link text from page
@links = ($html =~ m/<a[^>]*>([^<\/a]*)<\/a>/ig);


Could anyone be so kind as to point out my error, or suggest any better way?

Thanks in advance!

[bigDugan]

It might make it easier if you re-post your link here.

[plattypus]

I must be bonkers - why am I writing this for you?!

Okay..this is a very "feature empty" version - it doesn't handle linked images/alt tags etc. Basically, it doesn't pay to simplify! And you need to dig out the old regular expressions manual.

Here's some code:

#!/usr/bin/perl
use LWP::Simple;

$url = "http://www.freemoneyservices.com/";
$html = get($url);

# remove newlines and multiple spaces
$html =~ s/\n/\ /g;
$html =~ s/\r/\ /g;
while ($html=~/\s\s/) {$html=~s/\s\s/ /gi;}


# get link text from page
@links = ($html =~ m/<a[^>]*>([^<]*)/ig);

# Post - process these links. Remove additional spaces, blanks lines etc.
@linksfinal=undef;
foreach $anchor(@links) {
while ($anchor=~/\s\s/) {$anchor=~s/\s\s/ /gi;}
$anchor=~ s/^\s//gi;
if ($anchor ne "") { push @linksfinal, $anchor; }
}


# This just lists the anchors
foreach $anchor(@linksfinal) {
print $anchor,"\n";
}

[Webmaster-Toolkit]

Wow. Thanks plattypus :cheers:

[highman]

I must be bonkers - why am I writing this for you?!


:thumb: great stuff!

[motiv8x]

you may want to look into HTML::Parser