[SP-pm] downloader survey

[Stanislaw Pusep]

OK, aqui vai o spoiler: estou fazendo benchmark (de overhead) de todos os
HTTP agents que conheço. E o LWP está *MUITO* feio na fita:
https://metacpan.org/module/AnyEvent::Net::Curl::Queued#OVERHEAD

ABS()



2011/10/20 Rafael Prenzier <rafaelprenzier em gmail.com>

> inclusive, estou lendo a Doc aqui no Cpan e o exemplo que ele dá é
> justamente paralelizando Downloads
>
> http://search.cpan.org/~dlux/Parallel-ForkManager-0.7.5/ForkManager.pm
>
>   use LWP::Simple;
>   use Parallel::ForkManager;
>
>   ...
>
>   @links=(
>     ["http://www.foo.bar/rulez.data","rulez_data.txt"],
>     ["http://new.host/more_data.doc","more_data.doc"],
>     ...
>   );
>
>   ...
>
>   # Max 30 processes for parallel download
>   my $pm = new Parallel::ForkManager(30);
>
>   foreach my $linkarray (@links) {
>     $pm->start and next; # do the fork
>
>     my ($link,$fn) = @$linkarray;
>     warn "Cannot get $fn from $link"
>       if getstore($link,$fn) != RC_OK;
>
>     $pm->finish; # do the exit in the child process
>   }
>   $pm->wait_all_children;
>
>
> Abraço
>
> 2011/10/20 Rafael Prenzier <rafaelprenzier em gmail.com>
>
>> Hummmm se você usar o Parallel::ForkManager com qualquer método de
>> Download, resolve?
>>
>> 2011/10/20 Stanislaw Pusep <creaktive em gmail.com>
>>
>>> Perlssoal, alguém conhece agente(s) HTTP, preferencialmente em Perl, e
>>> preferencialmente capazes de paralelizar downloads, que não sejam estes?
>>>
>>>    - LWP::UserAgent
>>>    - HTTP::Tiny
>>>    - HTTP::Lite
>>>    - AnyEvent::Curl::Multi
>>>    - lftp
>>>    - curl
>>>    - wget
>>>
>>> P.S. - vulgo: "complete a lista" :P
>>>
>>> ABS()
>>>
>>>
>>> =begin disclaimer
>>>   Sao Paulo Perl Mongers: http://sao-paulo.pm.org/
>>>  SaoPaulo-pm mailing list: SaoPaulo-pm em pm.org
>>>  L<http://mail.pm.org/mailman/listinfo/saopaulo-pm>
>>> =end disclaimer
>>>
>>>
>>
>>
>> --
>> Rafael Prenzier dos Santos | Engenheiro e Perl Monger
>>
>>  rafaelprenzier em gmail.com
>>  rafael em prenzier.net
>> <rafaelprenzier em gmail.com>
>>  (13) 8131-8212
>>
>> <http://gplus.to/prenzier>  <http://www.facebook.com/rafaelprenzier> <http://twitter.com/prenzier/>
>>   <http://www.orkut.com.br/Main#Profile?uid=13496291347057199080> <http://cid-eacb613d80ce8dba.profile.live.com/>
>>   <http://www.delicious.com/rafaelprenzier/> <http://www.google.com/profiles/rafaelprenzier>
>>   <http://br.linkedin.com/pub/rafael-prenzier-dos-santos/29/6a2/478> <http://www.youtube.com/prenzier>
>>   <http://www.diigo.com/user/prenzier> <http://www.shelfari.com/rprenzier/shelf>
>>   <http://flavors.me/prenzier>
>>  <http://www.diigo.com/user/prenzier>
>>
>>
>>
>
>
> --
> Rafael Prenzier dos Santos | Engenheiro e Perl Monger
>
>  rafaelprenzier em gmail.com
>  rafael em prenzier.net
> <rafaelprenzier em gmail.com>
>  (13) 8131-8212
>
> <http://gplus.to/prenzier>  <http://www.facebook.com/rafaelprenzier> <http://twitter.com/prenzier/>
>   <http://www.orkut.com.br/Main#Profile?uid=13496291347057199080> <http://cid-eacb613d80ce8dba.profile.live.com/>
>   <http://www.delicious.com/rafaelprenzier/> <http://www.google.com/profiles/rafaelprenzier>
>   <http://br.linkedin.com/pub/rafael-prenzier-dos-santos/29/6a2/478> <http://www.youtube.com/prenzier>
>   <http://www.diigo.com/user/prenzier> <http://www.shelfari.com/rprenzier/shelf>
>   <http://flavors.me/prenzier>
>  <http://www.diigo.com/user/prenzier>
>
>
>
> =begin disclaimer
>   Sao Paulo Perl Mongers: http://sao-paulo.pm.org/
>  SaoPaulo-pm mailing list: SaoPaulo-pm em pm.org
>  L<http://mail.pm.org/mailman/listinfo/saopaulo-pm>
> =end disclaimer
>
>
-------------- Pr�xima Parte ----------
Um anexo em HTML foi limpo...
URL: <http://mail.pm.org/pipermail/saopaulo-pm/attachments/20111020/f277e4d2/attachment.html>