<html>
<body>
<font size=3>At 01:30 PM 3/30/2015, you wrote:<br><br>
<blockquote type=cite class=cite cite="">How about the Command
"curl"?</font></blockquote><br>
Curl or wget would take longer since I'd have to make a system() call and
then parse the resulting file. <br><br>
<br>
<blockquote type=cite class=cite cite=""><font size=3>On Mon, Mar 30,
2015 at 1:00 PM, Andy Lester
<<a href="mailto:andy@petdance.com">andy@petdance.com</a>>
wrote:<br>
</font><blockquote type=cite class=cite cite="">
<dl>
<dd>LWP::Simple & LWP::Useragent returned the page, but the pages are
fairly dense with a lot of embedded javascript, embedded forms and ads
the are being served up. All of which I don't need. It's just taking a
lot of time and memory. I was just looking for something that would just
give me a 404 or 200 or stop reading at the some place like the end of
the /head tag. I'm trying to test out thousands of URLs which is the real
problem. (This may not be possible.)</blockquote><br>
<dd>You can use the LWP::Simple head() function like David said, but
head() vs. get() is all-or-nothing. There’s no way to say
“Give me the page up to the and of the <head> tag”.<br><br>
<dd>I’m curious as to how these pages are taking a lot of
memory. You’re not storing them, are you? What memory
problems are you running into?<br><br>
<dd>What’s the problem that you’re actually trying to solve?Â
Is it taking too long to do those 1000 URL checks? How long is it
taking, and how long wouldx you like it to take?<br><br>
<dd>--<br>
<dd>Andy Lester =>Â
<a href="http://www.petdance.com">www.petdance.com</a> <br><br>
<br>
<dd>_______________________________________________<br>
<dd>Chicago-talk mailing list<br>
<dd><a href="mailto:Chicago-talk@pm.org">Chicago-talk@pm.org</a><br>
<dd>
<a href="http://mail.pm.org/mailman/listinfo/chicago-talk" eudora="autourl">
http://mail.pm.org/mailman/listinfo/chicago-talk</a><br><br>
</dl><br><br>
<br>
-- <br>
-----------------<br>
Hal Wigoda<br>
Chicago<br>
_______________________________________________<br>
Chicago-talk mailing list<br>
Chicago-talk@pm.org<br>
<a href="http://mail.pm.org/mailman/listinfo/chicago-talk" eudora="autourl">
http://mail.pm.org/mailman/listinfo/chicago-talk</a></blockquote></body>
</html>