[Chicago-talk] Testing if Page exists.

Jim Jacobus JJacobus at PonyX.com
Mon Mar 30 10:10:00 PDT 2015

At 11:46 AM 3/30/2015, you wrote:

>>On Mar 30, 2015, at 11:17 AM, Jim Jacobus 
>><<mailto:JJacobus at PonyX.com>JJacobus at PonyX.com> wrote:
>>Tried LWP::Simple and HTTP::Request with various results.
>What were those various results?  Did they not work for you?  What was wrong?

LWP::Simple & LWP::Useragent returned the page, but the pages are 
fairly dense with a lot of embedded javascript, embedded forms and 
ads the are being served up. All of which I don't need. It's just 
taking a lot of time and memory. I was just looking for something 
that would just give me a 404 or 200 or stop reading at the some 
place like the end of the /head tag. I'm trying to test out thousands 
of URLs which is the real problem. (This may not be possible.)

>Andy Lester => <http://www.petdance.com>www.petdance.com
>Chicago-talk mailing list
>Chicago-talk at pm.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20150330/1f0f963c/attachment.html>

More information about the Chicago-talk mailing list