[Rio-pm] Ajuda com Threads - tempo de abertura
Thiago Glauco Sanchez
thiagoglauco em ticursos.net
Domingo Julho 11 20:04:29 PDT 2010
Ah, entendi... então amanha eu e a lista vamos estar mais descansados
para pensar no seu problema...
Em 11/07/2010 23:57, João André Simioni escreveu:
> Foi um typo - erro de digitação - o Net::Ping testa TCP e UDP. O ping
> externo ICMP.
>
> Mas assim, eu preciso resolver o problema com esse teste em Perl -
> como eu disse, essa parte do código é só um pedaço de um sistema
> maior. SNMP já é usado para várias outras coisas, mas dependem de
> access lists em routers e eu não tenho como liberar.
>
> []'s
>
> 2010/7/11 Thiago Glauco Sanchez<thiagoglauco em ticursos.net>:
>
>> O que eu quero dizer: você testou com o ping externo que é um teste icmp...
>> depois fez outro teste icmp com o Net::Ping...
>>
>> Não seria mais interessante testar de outra forma?
>>
>> $p = Net::Ping->new("tcp", 2);
>> $p->port_number(getservbyname("http", "tcp"));
>>
>> Mais umas coisas: Suas máquinas/equipamentos em teste estão via internet ou
>> rede local?
>> Já pensou em usar SNMP? Já tive um problema parecido e iria utilizar um
>> processo para cada equipamento por desempenho e acabai optando por utilizar
>> o Net:SNMP
>> que permite utilizar uma função de callback em vez de utilizar um processo
>> para cada equipamento. Eu mando uma requisição para todos os equipamentos e
>> conforme eles respondem
>> a função de call-back trata a requisição.
>>
>> Porém, não conheço o seu ambiente. Apenas acho que criar 4000 threads não
>> seja a melhor solução.
>>
>>
>> Em 11/07/2010 23:18, João André Simioni escreveu:
>>
>>> É typo - já está certo no código. ICMP vai via comando externo, UDP e
>>> TCP via Net::Ping.
>>>
>>> Com relação aos testes, é normal um momento de sobrecarga causar a não
>>> resposta de testes - essas sobrecargas não devem duram mais que 5
>>> segundos. Por isso o sleep - assim eu faço o mesmo teste 3x, com
>>> intervalos de 3 segundos.
>>>
>>> []'s
>>>
>>> On Sun, Jul 11, 2010 at 10:31 PM, Blabos de Blebe<blabos em gmail.com>
>>> wrote:
>>>
>>>
>>>> Não seria um typo?
>>>>
>>>> icmp -> ping;
>>>> tcp e udp -> Net::Ping
>>>>
>>>>
>>>>
>>>> On Sun, Jul 11, 2010 at 6:42 PM, Thiago Glauco Sanchez
>>>> <thiagoglauco em ticursos.net> wrote:
>>>>
>>>>
>>>>> Sim sim, entendi sua preocupação.
>>>>> Mas em vez de:
>>>>>
>>>>> for (1..3) {
>>>>> my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>>>>
>>>>> não seria melhor já mandar pingar as 3 vezes no comando ou apenas 1?
>>>>> Você
>>>>> está gerando 3 processos externos sem necessidade real.
>>>>>
>>>>> outra... o comando ping já não é icmp?
>>>>> my $p = Net::Ping->new('icmp')???
>>>>>
>>>>>
>>>>> Em 11/07/2010 17:04, João André Simioni escreveu:
>>>>>
>>>>>
>>>>>> Eu quero fazer o ping de 3 formas - UDP Echo / TCP Echo e ICMP Echo.
>>>>>> Alguns clientes tem filtros e assim eu evito falsos positivos.
>>>>>>
>>>>>> Net::Ping mesmo com root tem resultados incoerentes quando uso ICMP e
>>>>>> Threads.
>>>>>>
>>>>>> []'s
>>>>>>
>>>>>> On Sun, Jul 11, 2010 at 4:56 PM, Thiago Glauco Sanchez
>>>>>> <thiagoglauco em ticursos.net> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Endosso a solução do Lorn. Use o POE.
>>>>>>>
>>>>>>> E cara... pra que isso??
>>>>>>>
>>>>>>> sub checkIp {
>>>>>>> my $ip = shift;
>>>>>>> my $pingOk = 0;
>>>>>>>
>>>>>>> eval {
>>>>>>> for (1..3) {
>>>>>>> my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>>>>>>
>>>>>>> if (grep { / 0% packet loss/ } $ping) {
>>>>>>> $pingOk = 1;
>>>>>>> return 1;
>>>>>>> }
>>>>>>>
>>>>>>> if ($pingOk == 0) {
>>>>>>> my $p = Net::Ping->new('icmp');
>>>>>>> $p->service_check(1);
>>>>>>> if ($p->ping($ip, 1)) {
>>>>>>> $pingOk = 1;
>>>>>>> return 1;
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> if ($pingOk == 0) {
>>>>>>> my $p = Net::Ping->new('udp');
>>>>>>> $p->service_check(1);
>>>>>>> if ($p->ping($ip, 1)) {
>>>>>>> $pingOk = 1;
>>>>>>> return 1;
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> sleep 3;
>>>>>>> }
>>>>>>> };
>>>>>>>
>>>>>>> Para pingar com ping externo, depois com o Net::Ping via icmp e depois
>>>>>>> com
>>>>>>> net::Ping udp? Ah... se não me engano, Net::Ping precisa de permissão
>>>>>>> root
>>>>>>> para retornar um icmp de forma correta...
>>>>>>>
>>>>>>>
>>>>>>> Em 09/07/2010 17:48, João André Simioni escreveu:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Caros,
>>>>>>>>
>>>>>>>> tenho um script para testar cerca de 4700 equipamentos, usando ping.
>>>>>>>> Queria usar cerca 250 threads para isso, então o que eu fiz foi
>>>>>>>> quebrar o grupo de 4300 em grupos 18 elementos e abri uma thread para
>>>>>>>> cada.
>>>>>>>>
>>>>>>>> Não há nenhum problema com o código, o que acontece é que o tempo de
>>>>>>>> abertura das threads é muito alto. E quanto mais threads eu abro,
>>>>>>>> maior o tempo para abertura das próximas. As primeiras abrem em 30ms,
>>>>>>>> mas perto da thread 70 esse tempo já vai para quase 1 segundo.
>>>>>>>>
>>>>>>>> Vocês sabem como otimizar esse tempo de abertura? Ou alguma outra
>>>>>>>> solução?
>>>>>>>>
>>>>>>>> Segue o output do script como exemplo:
>>>>>>>> Para 4677 cpes, e 250 threads, tenho no final 246 grupos, com 19
>>>>>>>> elementos
>>>>>>>> cada
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.61858
>>>>>>>> Created thread 1 (1) - took 0.030138 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.92080
>>>>>>>> Created thread 2 (2) - took 0.042668 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.134968
>>>>>>>> Created thread 3 (3) - took 0.064245 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.199430
>>>>>>>> Created thread 4 (4) - took 0.082642 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.282278
>>>>>>>> Created thread 5 (5) - took 0.128257 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.410769
>>>>>>>> Created thread 6 (6) - took 0.136458 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.547383
>>>>>>>> Created thread 7 (7) - took 0.208085 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.755594
>>>>>>>> Created thread 8 (8) - took 0.173014 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:50 2010.928756
>>>>>>>> Created thread 9 (9) - took 0.132713 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:51 2010.61608
>>>>>>>> Created thread 10 (10) - took 0.127795 seconds to create
>>>>>>>> Trying to create thread in Fri Jul 9 17:46:51 2010.189536
>>>>>>>> Created thread 11 (11) - took 0.301836 seconds to create
>>>>>>>>
>>>>>>>> Obrigado
>>>>>>>>
>>>>>>>> João André Simioni
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> #!/usr/bin/perl
>>>>>>>>
>>>>>>>> use strict;
>>>>>>>> use DBI;
>>>>>>>> use Net::Ping;
>>>>>>>> use Time::HiRes qw/usleep tv_interval gettimeofday/;
>>>>>>>> use threads ('yield', 'stack_size' => 32*4096, 'exit' =>
>>>>>>>> 'threads_only', 'stringify');
>>>>>>>>
>>>>>>>> my $host = '192.168.160.179';
>>>>>>>> my $sid = 'HOMOLOG';
>>>>>>>> my $user = 'mac_user';
>>>>>>>> my $pass = 'mac_user';
>>>>>>>>
>>>>>>>> my $dbh = DBI->connect("dbi:Oracle:host=$host;sid=$sid", $user,
>>>>>>>> $pass,
>>>>>>>> { AutoCommit => 1 });
>>>>>>>>
>>>>>>>> my $sth = $dbh->prepare('SELECT CPE_ID, CLIENTE_ID, CPE_DESC,
>>>>>>>> CPE_IP_WAN, CPE_NODE, CPE_IF FROM MAC_CPE WHERE CPE_PING = ? AND
>>>>>>>> CPE_ATIVO = ?');
>>>>>>>> $sth->execute('1', 'A');
>>>>>>>>
>>>>>>>> my @cpes;
>>>>>>>>
>>>>>>>> while (my (@row) = $sth->fetchrow_array) {
>>>>>>>> push @cpes, [ @row ];
>>>>>>>> }
>>>>>>>>
>>>>>>>> $dbh->disconnect();
>>>>>>>>
>>>>>>>> my $maxThreads = 250;
>>>>>>>> my $total = @cpes;
>>>>>>>>
>>>>>>>> my $cpePerThread = int($total / $maxThreads);
>>>>>>>>
>>>>>>>> my @toMonitor;
>>>>>>>> my $i = 0;
>>>>>>>> my $c = 0;
>>>>>>>>
>>>>>>>> foreach my $cpe (@cpes) {
>>>>>>>> push @{$toMonitor[$i]}, $cpe;
>>>>>>>> $c++;
>>>>>>>>
>>>>>>>> if ($c> $cpePerThread) {
>>>>>>>> $i++;
>>>>>>>> $c = 0;
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> print "Para $total cpes, e $maxThreads threads, tenho no final $i
>>>>>>>> grupos, com ", scalar @{$toMonitor[0]}, " elementos cada\n";
>>>>>>>>
>>>>>>>> sub processResult {
>>>>>>>> my $r = shift;
>>>>>>>> if (ref $r ne 'ARRAY') {
>>>>>>>> print STDERR "Erro na thread\n";
>>>>>>>> return 0;
>>>>>>>> }
>>>>>>>> my ($result, $host, $ifs) = @$r;
>>>>>>>>
>>>>>>>> }
>>>>>>>>
>>>>>>>> sub testeClient {
>>>>>>>> my $threadNum = shift;
>>>>>>>> my $cpeGroup = shift;
>>>>>>>> foreach my $cpe (@{$cpeGroup}) {
>>>>>>>> my ($cpeId, $clientId, $cpeDesc, $cpeIp, $cpeNode, $cpeIf) =
>>>>>>>> @$cpe;
>>>>>>>> my $pingOk =&checkIp($cpeIp);
>>>>>>>> my $status = $pingOk ? 'RESPONDE' : 'MORTO';
>>>>>>>> # print join(", ", $threadNum, $cpeId, $clientId, $cpeDesc,
>>>>>>>> $cpeIp, $cpeNode, $cpeIf, $status), "\n";
>>>>>>>> }
>>>>>>>> # print "Finishing thread $threadNum\n";
>>>>>>>> return([1, 0]);
>>>>>>>> }
>>>>>>>>
>>>>>>>> my $threadCount = 1;
>>>>>>>> foreach my $cpeGroup (@toMonitor) {
>>>>>>>> print "Trying to create thread";
>>>>>>>> my $t0 = [gettimeofday];
>>>>>>>>
>>>>>>>> print " in ", scalar localtime $t0->[0], ".", $t0->[1], "\n";
>>>>>>>>
>>>>>>>> my $thr = threads->create({scalar => '1'}, 'testeClient',
>>>>>>>> $threadCount, $cpeGroup);
>>>>>>>> my $elapsed = tv_interval ( $t0, [gettimeofday]);
>>>>>>>> print "Created thread $thr ($threadCount) - took $elapsed seconds
>>>>>>>> to create\n";
>>>>>>>> $threadCount++;
>>>>>>>>
>>>>>>>> while (threads->list()>= $maxThreads) {
>>>>>>>> my @joinable = threads->list(threads::joinable);
>>>>>>>> for (@joinable) {
>>>>>>>> my $r = $_->join();
>>>>>>>> &processResult($r);
>>>>>>>> }
>>>>>>>> usleep(10000);
>>>>>>>> }
>>>>>>>> my $elapsed = tv_interval ( $t0, [gettimeofday]);
>>>>>>>> }
>>>>>>>>
>>>>>>>> while (threads->list()) {
>>>>>>>> my @joinable = threads->list(threads::joinable);
>>>>>>>> for (@joinable) {
>>>>>>>> my $r = $_->join();
>>>>>>>> &processResult($r);
>>>>>>>> }
>>>>>>>> usleep(10000);
>>>>>>>> }
>>>>>>>>
>>>>>>>> sub checkIp {
>>>>>>>> my $ip = shift;
>>>>>>>> my $pingOk = 0;
>>>>>>>>
>>>>>>>> eval {
>>>>>>>> for (1..3) {
>>>>>>>> my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>>>>>>>
>>>>>>>> if (grep { / 0% packet loss/ } $ping) {
>>>>>>>> $pingOk = 1;
>>>>>>>> return 1;
>>>>>>>> }
>>>>>>>>
>>>>>>>> if ($pingOk == 0) {
>>>>>>>> my $p = Net::Ping->new('icmp');
>>>>>>>> $p->service_check(1);
>>>>>>>> if ($p->ping($ip, 1)) {
>>>>>>>> $pingOk = 1;
>>>>>>>> return 1;
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> if ($pingOk == 0) {
>>>>>>>> my $p = Net::Ping->new('udp');
>>>>>>>> $p->service_check(1);
>>>>>>>> if ($p->ping($ip, 1)) {
>>>>>>>> $pingOk = 1;
>>>>>>>> return 1;
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> sleep 3;
>>>>>>>> }
>>>>>>>> };
>>>>>>>>
>>>>>>>> return $pingOk;
>>>>>>>> }
>>>>>>>> _______________________________________________
>>>>>>>> Rio-pm mailing list
>>>>>>>> Rio-pm em pm.org
>>>>>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> What is the sound of Perl? Is it not the sound of a wall that people
>>>>>>> have
>>>>>>> stopped banging their heads against?
>>>>>>> —Larry Wall
>>>>>>>
>>>>>>> Thiago Glauco Sanchez
>>>>>>> Intrutor Perl e Redes
>>>>>>> www.ticursos.net
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Rio-pm mailing list
>>>>>>> Rio-pm em pm.org
>>>>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Rio-pm mailing list
>>>>>> Rio-pm em pm.org
>>>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> --
>>>>> What is the sound of Perl? Is it not the sound of a wall that people
>>>>> have
>>>>> stopped banging their heads against?
>>>>> —Larry Wall
>>>>>
>>>>> Thiago Glauco Sanchez
>>>>> Intrutor Perl e Redes
>>>>> www.ticursos.net
>>>>>
>>>>> _______________________________________________
>>>>> Rio-pm mailing list
>>>>> Rio-pm em pm.org
>>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Rio-pm mailing list
>>>> Rio-pm em pm.org
>>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Rio-pm mailing list
>>> Rio-pm em pm.org
>>> http://mail.pm.org/mailman/listinfo/rio-pm
>>>
>>>
>>>
>>
>> --
>> What is the sound of Perl? Is it not the sound of a wall that people have
>> stopped banging their heads against?
>> —Larry Wall
>>
>> Thiago Glauco Sanchez
>> Intrutor Perl e Redes
>> www.ticursos.net
>>
>> _______________________________________________
>> Rio-pm mailing list
>> Rio-pm em pm.org
>> http://mail.pm.org/mailman/listinfo/rio-pm
>>
>>
> _______________________________________________
> Rio-pm mailing list
> Rio-pm em pm.org
> http://mail.pm.org/mailman/listinfo/rio-pm
>
>
--
What is the sound of Perl? Is it not the sound of a wall that people have
stopped banging their heads against?
—Larry Wall
Thiago Glauco Sanchez
Intrutor Perl e Redes
www.ticursos.net
Mais detalhes sobre a lista de discussão Rio-pm