[Rio-pm] Ajuda com Threads - tempo de abertura

Thiago Glauco Sanchez thiagoglauco em ticursos.net
Domingo Julho 11 19:40:52 PDT 2010


O que eu quero dizer: você testou com o ping externo que é um teste 
icmp... depois fez outro teste icmp com o Net::Ping...

Não seria mais interessante testar de outra forma?

     $p = Net::Ping->new("tcp", 2);
     $p->port_number(getservbyname("http", "tcp"));

Mais umas coisas: Suas máquinas/equipamentos em teste estão via internet ou rede local?
Já pensou em usar SNMP? Já tive um problema parecido e iria utilizar um processo para cada equipamento por desempenho e acabai optando por utilizar o Net:SNMP
que permite utilizar uma função de callback em vez de utilizar um processo para cada equipamento. Eu mando uma requisição para todos os equipamentos e conforme eles respondem
a função de call-back trata a requisição.

Porém, não conheço o seu ambiente. Apenas acho que criar 4000 threads não seja a melhor solução.


Em 11/07/2010 23:18, João André Simioni escreveu:
> É typo - já está certo no código. ICMP vai via comando externo, UDP e
> TCP via Net::Ping.
>
> Com relação aos testes, é normal um momento de sobrecarga causar a não
> resposta de testes - essas sobrecargas não devem duram mais que 5
> segundos. Por isso o sleep - assim eu faço o mesmo teste 3x, com
> intervalos de 3 segundos.
>
> []'s
>
> On Sun, Jul 11, 2010 at 10:31 PM, Blabos de Blebe<blabos em gmail.com>  wrote:
>    
>> Não seria um typo?
>>
>> icmp ->  ping;
>> tcp e udp ->  Net::Ping
>>
>>
>>
>> On Sun, Jul 11, 2010 at 6:42 PM, Thiago Glauco Sanchez
>> <thiagoglauco em ticursos.net>  wrote:
>>      
>>> Sim sim, entendi sua preocupação.
>>> Mas em vez de:
>>>
>>> for (1..3) {
>>>            my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>>
>>> não seria melhor já mandar pingar as 3 vezes no comando ou apenas 1? Você
>>> está gerando 3 processos externos sem necessidade real.
>>>
>>> outra... o comando ping já não é icmp?
>>> my $p = Net::Ping->new('icmp')???
>>>
>>>
>>> Em 11/07/2010 17:04, João André Simioni escreveu:
>>>        
>>>> Eu quero fazer o ping de 3 formas - UDP Echo / TCP Echo e ICMP Echo.
>>>> Alguns clientes tem filtros e assim eu evito falsos positivos.
>>>>
>>>> Net::Ping mesmo com root tem resultados incoerentes quando uso ICMP e
>>>> Threads.
>>>>
>>>> []'s
>>>>
>>>> On Sun, Jul 11, 2010 at 4:56 PM, Thiago Glauco Sanchez
>>>> <thiagoglauco em ticursos.net>    wrote:
>>>>
>>>>          
>>>>> Endosso a solução do Lorn. Use o POE.
>>>>>
>>>>> E cara... pra que isso??
>>>>>
>>>>> sub checkIp {
>>>>>     my $ip = shift;
>>>>>     my $pingOk = 0;
>>>>>
>>>>>     eval {
>>>>>         for (1..3) {
>>>>>             my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>>>>
>>>>>             if (grep { / 0% packet loss/ } $ping) {
>>>>>                 $pingOk = 1;
>>>>>                 return 1;
>>>>>             }
>>>>>
>>>>>             if ($pingOk == 0) {
>>>>>                 my $p = Net::Ping->new('icmp');
>>>>>                 $p->service_check(1);
>>>>>                 if ($p->ping($ip, 1)) {
>>>>>                     $pingOk = 1;
>>>>>                     return 1;
>>>>>                 }
>>>>>             }
>>>>>
>>>>>             if ($pingOk == 0) {
>>>>>                 my $p = Net::Ping->new('udp');
>>>>>                 $p->service_check(1);
>>>>>                 if ($p->ping($ip, 1)) {
>>>>>                     $pingOk = 1;
>>>>>                     return 1;
>>>>>                 }
>>>>>             }
>>>>>
>>>>>             sleep 3;
>>>>>         }
>>>>>     };
>>>>>
>>>>> Para pingar com ping externo, depois com o Net::Ping via icmp e depois
>>>>> com
>>>>> net::Ping udp? Ah... se não me engano, Net::Ping precisa de permissão
>>>>> root
>>>>> para retornar um icmp de forma correta...
>>>>>
>>>>>
>>>>> Em 09/07/2010 17:48, João André Simioni escreveu:
>>>>>
>>>>>            
>>>>>> Caros,
>>>>>>
>>>>>> tenho um script para testar cerca de 4700 equipamentos, usando ping.
>>>>>> Queria usar cerca 250 threads para isso, então o que eu fiz foi
>>>>>> quebrar o grupo de 4300 em grupos 18 elementos e abri uma thread para
>>>>>> cada.
>>>>>>
>>>>>> Não há nenhum problema com o código, o que acontece é que o tempo de
>>>>>> abertura das threads é muito alto. E quanto mais threads eu abro,
>>>>>> maior o tempo para abertura das próximas. As primeiras abrem em 30ms,
>>>>>> mas perto da thread 70 esse tempo já vai para quase 1 segundo.
>>>>>>
>>>>>> Vocês sabem como otimizar esse tempo de abertura? Ou alguma outra
>>>>>> solução?
>>>>>>
>>>>>> Segue o output do script como exemplo:
>>>>>> Para 4677 cpes, e 250 threads, tenho no final 246 grupos, com 19
>>>>>> elementos
>>>>>> cada
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.61858
>>>>>> Created thread 1 (1) - took 0.030138 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.92080
>>>>>> Created thread 2 (2) - took 0.042668 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.134968
>>>>>> Created thread 3 (3) - took 0.064245 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.199430
>>>>>> Created thread 4 (4) - took 0.082642 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.282278
>>>>>> Created thread 5 (5) - took 0.128257 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.410769
>>>>>> Created thread 6 (6) - took 0.136458 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.547383
>>>>>> Created thread 7 (7) - took 0.208085 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.755594
>>>>>> Created thread 8 (8) - took 0.173014 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:50 2010.928756
>>>>>> Created thread 9 (9) - took 0.132713 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:51 2010.61608
>>>>>> Created thread 10 (10) - took 0.127795 seconds to create
>>>>>> Trying to create thread in Fri Jul  9 17:46:51 2010.189536
>>>>>> Created thread 11 (11) - took 0.301836 seconds to create
>>>>>>
>>>>>> Obrigado
>>>>>>
>>>>>> João André Simioni
>>>>>>
>>>>>>
>>>>>>
>>>>>> #!/usr/bin/perl
>>>>>>
>>>>>> use strict;
>>>>>> use DBI;
>>>>>> use Net::Ping;
>>>>>> use Time::HiRes qw/usleep tv_interval gettimeofday/;
>>>>>> use threads ('yield', 'stack_size' =>      32*4096, 'exit' =>
>>>>>> 'threads_only', 'stringify');
>>>>>>
>>>>>> my $host = '192.168.160.179';
>>>>>> my $sid  = 'HOMOLOG';
>>>>>> my $user = 'mac_user';
>>>>>> my $pass = 'mac_user';
>>>>>>
>>>>>> my $dbh = DBI->connect("dbi:Oracle:host=$host;sid=$sid", $user, $pass,
>>>>>> { AutoCommit =>      1 });
>>>>>>
>>>>>> my $sth = $dbh->prepare('SELECT CPE_ID, CLIENTE_ID, CPE_DESC,
>>>>>> CPE_IP_WAN, CPE_NODE, CPE_IF FROM MAC_CPE WHERE CPE_PING = ? AND
>>>>>> CPE_ATIVO = ?');
>>>>>> $sth->execute('1', 'A');
>>>>>>
>>>>>> my @cpes;
>>>>>>
>>>>>> while (my (@row) = $sth->fetchrow_array) {
>>>>>>      push @cpes, [ @row ];
>>>>>> }
>>>>>>
>>>>>> $dbh->disconnect();
>>>>>>
>>>>>> my $maxThreads = 250;
>>>>>> my $total      = @cpes;
>>>>>>
>>>>>> my $cpePerThread = int($total / $maxThreads);
>>>>>>
>>>>>> my @toMonitor;
>>>>>> my $i = 0;
>>>>>> my $c = 0;
>>>>>>
>>>>>> foreach my $cpe (@cpes) {
>>>>>>      push @{$toMonitor[$i]}, $cpe;
>>>>>>      $c++;
>>>>>>
>>>>>>      if ($c>      $cpePerThread) {
>>>>>>          $i++;
>>>>>>          $c = 0;
>>>>>>      }
>>>>>> }
>>>>>>
>>>>>> print "Para $total cpes, e $maxThreads threads, tenho no final $i
>>>>>> grupos, com ", scalar @{$toMonitor[0]}, " elementos cada\n";
>>>>>>
>>>>>> sub processResult {
>>>>>>      my $r = shift;
>>>>>>      if (ref $r ne 'ARRAY') {
>>>>>>          print STDERR "Erro na thread\n";
>>>>>>          return 0;
>>>>>>      }
>>>>>>      my ($result, $host, $ifs) = @$r;
>>>>>>
>>>>>> }
>>>>>>
>>>>>> sub testeClient {
>>>>>>      my $threadNum = shift;
>>>>>>      my $cpeGroup  = shift;
>>>>>>      foreach my $cpe (@{$cpeGroup}) {
>>>>>>          my ($cpeId, $clientId, $cpeDesc, $cpeIp, $cpeNode, $cpeIf) =
>>>>>> @$cpe;
>>>>>>          my $pingOk =&checkIp($cpeIp);
>>>>>>          my $status = $pingOk ? 'RESPONDE' : 'MORTO';
>>>>>>          # print join(", ", $threadNum, $cpeId, $clientId, $cpeDesc,
>>>>>> $cpeIp, $cpeNode, $cpeIf, $status), "\n";
>>>>>>      }
>>>>>>      # print "Finishing thread $threadNum\n";
>>>>>>      return([1, 0]);
>>>>>> }
>>>>>>
>>>>>> my $threadCount = 1;
>>>>>> foreach my $cpeGroup (@toMonitor) {
>>>>>>      print "Trying to create thread";
>>>>>>      my $t0 = [gettimeofday];
>>>>>>
>>>>>>      print " in ", scalar localtime $t0->[0], ".", $t0->[1], "\n";
>>>>>>
>>>>>>      my $thr = threads->create({scalar =>      '1'}, 'testeClient',
>>>>>> $threadCount, $cpeGroup);
>>>>>>      my $elapsed = tv_interval ( $t0, [gettimeofday]);
>>>>>>      print "Created thread $thr ($threadCount) - took $elapsed seconds
>>>>>> to create\n";
>>>>>>      $threadCount++;
>>>>>>
>>>>>>      while (threads->list()>= $maxThreads) {
>>>>>>          my @joinable = threads->list(threads::joinable);
>>>>>>          for (@joinable) {
>>>>>>              my $r = $_->join();
>>>>>>              &processResult($r);
>>>>>>          }
>>>>>>          usleep(10000);
>>>>>>      }
>>>>>>      my $elapsed = tv_interval ( $t0, [gettimeofday]);
>>>>>> }
>>>>>>
>>>>>> while (threads->list()) {
>>>>>>      my @joinable = threads->list(threads::joinable);
>>>>>>      for (@joinable) {
>>>>>>          my $r = $_->join();
>>>>>>          &processResult($r);
>>>>>>      }
>>>>>>      usleep(10000);
>>>>>> }
>>>>>>
>>>>>> sub checkIp {
>>>>>>      my $ip = shift;
>>>>>>      my $pingOk = 0;
>>>>>>
>>>>>>      eval {
>>>>>>          for (1..3) {
>>>>>>              my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>>>>>
>>>>>>              if (grep { / 0% packet loss/ } $ping) {
>>>>>>                  $pingOk = 1;
>>>>>>                  return 1;
>>>>>>              }
>>>>>>
>>>>>>              if ($pingOk == 0) {
>>>>>>                  my $p = Net::Ping->new('icmp');
>>>>>>                  $p->service_check(1);
>>>>>>                  if ($p->ping($ip, 1)) {
>>>>>>                      $pingOk = 1;
>>>>>>                      return 1;
>>>>>>                  }
>>>>>>              }
>>>>>>
>>>>>>              if ($pingOk == 0) {
>>>>>>                  my $p = Net::Ping->new('udp');
>>>>>>                  $p->service_check(1);
>>>>>>                  if ($p->ping($ip, 1)) {
>>>>>>                      $pingOk = 1;
>>>>>>                      return 1;
>>>>>>                  }
>>>>>>              }
>>>>>>
>>>>>>              sleep 3;
>>>>>>          }
>>>>>>      };
>>>>>>
>>>>>>      return $pingOk;
>>>>>> }
>>>>>> _______________________________________________
>>>>>> Rio-pm mailing list
>>>>>> Rio-pm em pm.org
>>>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>>>
>>>>>>
>>>>>>
>>>>>>              
>>>>> --
>>>>> What is the sound of Perl? Is it not the sound of a wall that people have
>>>>> stopped banging their heads against?
>>>>> —Larry Wall
>>>>>
>>>>> Thiago Glauco Sanchez
>>>>> Intrutor Perl e Redes
>>>>> www.ticursos.net
>>>>>
>>>>> _______________________________________________
>>>>> Rio-pm mailing list
>>>>> Rio-pm em pm.org
>>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>>
>>>>>
>>>>>            
>>>> _______________________________________________
>>>> Rio-pm mailing list
>>>> Rio-pm em pm.org
>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>
>>>>
>>>>          
>>>
>>> --
>>> What is the sound of Perl? Is it not the sound of a wall that people have
>>> stopped banging their heads against?
>>> —Larry Wall
>>>
>>> Thiago Glauco Sanchez
>>> Intrutor Perl e Redes
>>> www.ticursos.net
>>>
>>> _______________________________________________
>>> Rio-pm mailing list
>>> Rio-pm em pm.org
>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>
>>>        
>> _______________________________________________
>> Rio-pm mailing list
>> Rio-pm em pm.org
>> http://mail.pm.org/mailman/listinfo/rio-pm
>>
>>      
> _______________________________________________
> Rio-pm mailing list
> Rio-pm em pm.org
> http://mail.pm.org/mailman/listinfo/rio-pm
>
>    


-- 
What is the sound of Perl? Is it not the sound of a wall that people have
stopped banging their heads against?
—Larry Wall

Thiago Glauco Sanchez
Intrutor Perl e Redes
www.ticursos.net



Mais detalhes sobre a lista de discussão Rio-pm