[SP-pm] downloader survey

[Rafael Prenzier]

inclusive, estou lendo a Doc aqui no Cpan e o exemplo que ele dá é
justamente paralelizando Downloads

http://search.cpan.org/~dlux/Parallel-ForkManager-0.7.5/ForkManager.pm

  use LWP::Simple;
  use Parallel::ForkManager;

  ...

  @links=(
    ["http://www.foo.bar/rulez.data","rulez_data.txt"],
    ["http://new.host/more_data.doc","more_data.doc"],
    ...
  );

  ...

  # Max 30 processes for parallel download
  my $pm = new Parallel::ForkManager(30);

  foreach my $linkarray (@links) {
    $pm->start and next; # do the fork

    my ($link,$fn) = @$linkarray;
    warn "Cannot get $fn from $link"
      if getstore($link,$fn) != RC_OK;

    $pm->finish; # do the exit in the child process
  }
  $pm->wait_all_children;


Abraço

2011/10/20 Rafael Prenzier <rafaelprenzier em gmail.com>

> Hummmm se você usar o Parallel::ForkManager com qualquer método de
> Download, resolve?
>
> 2011/10/20 Stanislaw Pusep <creaktive em gmail.com>
>
>> Perlssoal, alguém conhece agente(s) HTTP, preferencialmente em Perl, e
>> preferencialmente capazes de paralelizar downloads, que não sejam estes?
>>
>>    - LWP::UserAgent
>>    - HTTP::Tiny
>>    - HTTP::Lite
>>    - AnyEvent::Curl::Multi
>>    - lftp
>>    - curl
>>    - wget
>>
>> P.S. - vulgo: "complete a lista" :P
>>
>> ABS()
>>
>>
>> =begin disclaimer
>>   Sao Paulo Perl Mongers: http://sao-paulo.pm.org/
>>  SaoPaulo-pm mailing list: SaoPaulo-pm em pm.org
>>  L<http://mail.pm.org/mailman/listinfo/saopaulo-pm>
>> =end disclaimer
>>
>>
>
>
> --
> Rafael Prenzier dos Santos | Engenheiro e Perl Monger
>
>  rafaelprenzier em gmail.com
>  rafael em prenzier.net
> <rafaelprenzier em gmail.com>
>  (13) 8131-8212
>
> <http://gplus.to/prenzier>  <http://www.facebook.com/rafaelprenzier> <http://twitter.com/prenzier/>
>   <http://www.orkut.com.br/Main#Profile?uid=13496291347057199080> <http://cid-eacb613d80ce8dba.profile.live.com/>
>   <http://www.delicious.com/rafaelprenzier/> <http://www.google.com/profiles/rafaelprenzier>
>   <http://br.linkedin.com/pub/rafael-prenzier-dos-santos/29/6a2/478> <http://www.youtube.com/prenzier>
>   <http://www.diigo.com/user/prenzier> <http://www.shelfari.com/rprenzier/shelf>
>   <http://flavors.me/prenzier>
>  <http://www.diigo.com/user/prenzier>
>
>
>


-- 
Rafael Prenzier dos Santos | Engenheiro e Perl Monger

 rafaelprenzier em gmail.com
 rafael em prenzier.net
<rafaelprenzier em gmail.com>
 (13) 8131-8212

<http://gplus.to/prenzier>  <http://www.facebook.com/rafaelprenzier>
<http://twitter.com/prenzier/>
  <http://www.orkut.com.br/Main#Profile?uid=13496291347057199080>
<http://cid-eacb613d80ce8dba.profile.live.com/>
  <http://www.delicious.com/rafaelprenzier/>
<http://www.google.com/profiles/rafaelprenzier>
  <http://br.linkedin.com/pub/rafael-prenzier-dos-santos/29/6a2/478>
<http://www.youtube.com/prenzier>
  <http://www.diigo.com/user/prenzier> <http://www.shelfari.com/rprenzier/shelf>
  <http://flavors.me/prenzier>
<http://www.diigo.com/user/prenzier>
-------------- Pr?xima Parte ----------
Um anexo em HTML foi limpo...
URL: <http://mail.pm.org/pipermail/saopaulo-pm/attachments/20111020/22b86292/attachment.html>