SPUG: downloading from web pages

[Nicholas Melnick]

On Nov 24, 2008, at 11:15 PM, Yitzchak Scott-Thoennes wrote:

> I would use wget, not perl.

[It would help if I responded to the list, eh?]

He did mention parsing the page and downloading files matching a  
certain pattern. This sounds like something perl would be good at with  
minimal fuss. ;)

LWP::Simple sounds like a good plan here, as it's fundamentally simple.

use strict;
use warnings;

use LWP::Simple;

my $page = get('http://www.example.com');
while ( $page =~ /pattern with (http:\/\/stuff I want)/g ) {
    my $newpage = get($1);
    # do something with the result
}

1;


- Nick