OK, aqui vai o spoiler: estou fazendo benchmark (de overhead) de todos os HTTP agents que conheço. E o LWP está *MUITO* feio na fita:<div><div><div><a href="https://metacpan.org/module/AnyEvent::Net::Curl::Queued#OVERHEAD">https://metacpan.org/module/AnyEvent::Net::Curl::Queued#OVERHEAD</a></div>
<div><br>ABS()<br><br>
<br><br><div class="gmail_quote">2011/10/20 Rafael Prenzier <span dir="ltr"><<a href="mailto:rafaelprenzier@gmail.com">rafaelprenzier@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
inclusive, estou lendo a Doc aqui no Cpan e o exemplo que ele dá é justamente paralelizando Downloads<div><br></div><div><a href="http://search.cpan.org/~dlux/Parallel-ForkManager-0.7.5/ForkManager.pm" target="_blank">http://search.cpan.org/~dlux/Parallel-ForkManager-0.7.5/ForkManager.pm</a><br>
<div><br></div><div><div> use LWP::Simple;</div><div> use Parallel::ForkManager;</div><div><br></div><div> ...</div><div> </div><div> @links=( </div><div> ["<a href="http://www.foo.bar/rulez.data" target="_blank">http://www.foo.bar/rulez.data</a>","rulez_data.txt"], </div>
<div> ["<a href="http://new.host/more_data.doc" target="_blank">http://new.host/more_data.doc</a>","more_data.doc"],</div><div> ...</div><div> );</div><div><br></div><div> ...</div><div><br></div>
<div> # Max 30 processes for parallel download</div>
<div> my $pm = new Parallel::ForkManager(30); </div><div><br></div><div> foreach my $linkarray (@links) {</div><div> $pm->start and next; # do the fork</div><div><br></div><div> my ($link,$fn) = @$linkarray;</div>
<div> warn "Cannot get $fn from $link"</div><div> if getstore($link,$fn) != RC_OK;</div><div><br></div><div> $pm->finish; # do the exit in the child process</div><div> }</div><div> $pm->wait_all_children;</div>
</div><div><br><br></div><div>Abraço</div><div><div></div><div class="h5"><div><br><div class="gmail_quote">2011/10/20 Rafael Prenzier <span dir="ltr"><<a href="mailto:rafaelprenzier@gmail.com" target="_blank">rafaelprenzier@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hummmm se você usar o Parallel::ForkManager com qualquer método de Download, resolve?<br><br><div class="gmail_quote">2011/10/20 Stanislaw Pusep <span dir="ltr"><<a href="mailto:creaktive@gmail.com" target="_blank">creaktive@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div></div><div><div>Perlssoal, alguém conhece agente(s) HTTP, preferencialmente em Perl, e preferencialmente capazes de paralelizar downloads, que não sejam estes?</div>
<ul><li>LWP::UserAgent</li><li>HTTP::Tiny</li><li>HTTP::Lite</li><li>
AnyEvent::Curl::Multi</li><li>lftp</li><li>curl</li><li>wget</li></ul><div>P.S. - vulgo: "complete a lista" :P</div><br>ABS()<br><br>
<br></div></div>=begin disclaimer<br>
Sao Paulo Perl Mongers: <a href="http://sao-paulo.pm.org/" target="_blank">http://sao-paulo.pm.org/</a><br>
SaoPaulo-pm mailing list: <a href="mailto:SaoPaulo-pm@pm.org" target="_blank">SaoPaulo-pm@pm.org</a><br>
L<<a href="http://mail.pm.org/mailman/listinfo/saopaulo-pm" target="_blank">http://mail.pm.org/mailman/listinfo/saopaulo-pm</a>><br>
=end disclaimer<br>
<br></blockquote></div><font color="#888888"><br><br clear="all"><div><br></div>-- <br>Rafael Prenzier dos Santos | Engenheiro e Perl Monger <br><br><div><img> <a href="mailto:rafaelprenzier@gmail.com" target="_blank">rafaelprenzier@gmail.com</a></div>
<div><img> <a href="mailto:rafael@prenzier.net" target="_blank">rafael@prenzier.net</a><br></div><div><a href="mailto:rafaelprenzier@gmail.com" target="_blank"></a><br>
<img> <a href="tel:%2813%29%208131-8212" value="+551381318212" target="_blank">(13) 8131-8212</a><br><div><div><br><div><a href="http://gplus.to/prenzier" target="_blank"><img></a> <a href="http://www.facebook.com/rafaelprenzier" target="_blank"><img></a> <a href="http://twitter.com/prenzier/" target="_blank"><img></a> <a href="http://www.orkut.com.br/Main#Profile?uid=13496291347057199080" target="_blank"><img></a> <a href="http://cid-eacb613d80ce8dba.profile.live.com/" target="_blank"><img></a> <a href="http://www.delicious.com/rafaelprenzier/" target="_blank"><img></a> <a href="http://www.google.com/profiles/rafaelprenzier" target="_blank"><img></a> <a href="http://br.linkedin.com/pub/rafael-prenzier-dos-santos/29/6a2/478" target="_blank"><img></a> <a href="http://www.youtube.com/prenzier" target="_blank"><img></a> <a href="http://www.diigo.com/user/prenzier" target="_blank"><img></a> <a href="http://www.shelfari.com/rprenzier/shelf" target="_blank"><img></a> <a href="http://flavors.me/prenzier" target="_blank"><img></a> </div>
<div><a href="http://www.diigo.com/user/prenzier" target="_blank"></a><br><div><br></div></div></div></div></div><br>
</font></blockquote></div><br><br clear="all"><div><br></div>-- <br>Rafael Prenzier dos Santos | Engenheiro e Perl Monger <br><br><div><img> <a href="mailto:rafaelprenzier@gmail.com" target="_blank">rafaelprenzier@gmail.com</a></div>
<div><img> <a href="mailto:rafael@prenzier.net" target="_blank">rafael@prenzier.net</a><br></div><div><a href="mailto:rafaelprenzier@gmail.com" target="_blank"></a><br>
<img> <a href="tel:%2813%29%208131-8212" value="+551381318212" target="_blank">(13) 8131-8212</a><br><div><div><br><div><a href="http://gplus.to/prenzier" target="_blank"><img></a> <a href="http://www.facebook.com/rafaelprenzier" target="_blank"><img></a> <a href="http://twitter.com/prenzier/" target="_blank"><img></a> <a href="http://www.orkut.com.br/Main#Profile?uid=13496291347057199080" target="_blank"><img></a> <a href="http://cid-eacb613d80ce8dba.profile.live.com/" target="_blank"><img></a> <a href="http://www.delicious.com/rafaelprenzier/" target="_blank"><img></a> <a href="http://www.google.com/profiles/rafaelprenzier" target="_blank"><img></a> <a href="http://br.linkedin.com/pub/rafael-prenzier-dos-santos/29/6a2/478" target="_blank"><img></a> <a href="http://www.youtube.com/prenzier" target="_blank"><img></a> <a href="http://www.diigo.com/user/prenzier" target="_blank"><img></a> <a href="http://www.shelfari.com/rprenzier/shelf" target="_blank"><img></a> <a href="http://flavors.me/prenzier" target="_blank"><img></a> </div>
<div><a href="http://www.diigo.com/user/prenzier" target="_blank"></a><br><div><br></div></div></div></div></div><br>
</div></div></div></div>
<br>=begin disclaimer<br>
Sao Paulo Perl Mongers: <a href="http://sao-paulo.pm.org/" target="_blank">http://sao-paulo.pm.org/</a><br>
SaoPaulo-pm mailing list: <a href="mailto:SaoPaulo-pm@pm.org">SaoPaulo-pm@pm.org</a><br>
L<<a href="http://mail.pm.org/mailman/listinfo/saopaulo-pm" target="_blank">http://mail.pm.org/mailman/listinfo/saopaulo-pm</a>><br>
=end disclaimer<br>
<br></blockquote></div><br></div></div></div>