LWP::UserAgent and referer?
Robert L. Harris
Robert.L.Harris at rdlg.net
Mon Nov 26 11:42:47 CST 2001
Hmm, I did this one. Looks nice. Problem is now I'm getting a 404. I put
a few prints in. If I go to the index.html and select "save as" it downloads
fine. If I go directly to the link I'm trying to get I get a 403, denied. When
I run this segment of code I get a 404. If I check the URL I'm accessing in
"$url" below against my save as, they're identicle.
Thoughts?
Thus spake John Evans (evansj at kilnar.com):
> On Wed, 21 Nov 2001, Robert L. Harris wrote:
>
> >
> >
> > I'm trying to use a perl script to mirror some patches. The maker wishes
> > you to manually go and download each time, which gets a bit tedious.
> >
> > I've got it getting a list of patches, etc, but when I use "getstore"
> > from LWP::Simple I get an error message:
> >
>
> Here's how I do it and an example:
>
> Imagine that the page http://www.foo.com/patches.html has links to:
> http://www.foo.com/files/patch011119.tar.gz
> http://www.foo.com/files/patch011120.tar.gz
> http://www.foo.com/files/patch011121.tar.gz
>
> You want the tar.gz files:
>
> Try this script. It was written quickly and not tested:
>
> #!/usr/local/bin/perl
>
> use LWP::UserAgent;
> use HTTP::Request;
> use HTTP::Response;
> use URI::Heuristic;
>
> ### These two lines make sure that the URL is properly formatted
> my $raw_url = shift or die "usage: $0 [URL to Fetch]\n";
> my $url = URI::Heuristic::uf_urlstr($raw_url);
>
> ### Build a new web client that will make our requests.
> $client = LWP::UserAgent->new();
>
> ### Make our perl script look like IE 5.5 under Windows 98
> $client->agent("Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)");
>
> ### Build the request for the file we want.
> $request = HTTP::Request->new(GET => $url);
>
> ### Make it look like we came from their site
> $request->referer("http://www.foo.com/patches.html");
>
> $response = $client->request($request);
>
> if ($response->is_error()) {
> printf("%s\n", $response->status_line);
> }
> else {
> my $file = $response->content();
> }
>
>
> --
> John Evans
> http://evansj.kilnar.com/
> http://www.foo.com/files/patch011121.tar.gz
>
> -----BEGIN GEEK CODE BLOCK-----
> Version: 3.1
> GCS d- s++:- a- C+++>++++ ULSB++++$ P+++$ L++++$
> E--- W++ N+ o? K? w O- M V PS+ !PE Y+ PGP t(--) 5-- X++(+++)
> R+++ tv+ b+++(++++) DI+++ D++>+++ G+ e h--- r+++ y+++
> ------END GEEK CODE BLOCK------
>
>
>
:wq!
---------------------------------------------------------------------------
Robert L. Harris | Micros~1 :
Senior System Engineer | For when quality, reliability
at RnD Consulting | and security just aren't
\_ that important!
DISCLAIMER:
These are MY OPINIONS ALONE. I speak for no-one else.
FYI:
perl -e 'print $i=pack(c5,(41*2),sqrt(7056),(unpack(c,H)-2),oct(115),10);'
More information about the Pikes-peak-pm
mailing list