[Rio-pm] Ajuda com Threads - tempo de abertura
Fernando Oliveira
fernandocorrea em gmail.com
Sexta Julho 9 14:31:20 PDT 2010
João,
Vc não poderia, ao inves de criar varias vezes várias threads, criar um
array shared com os ips q devem ser pingados e criar as 18 threads com um
loop fazendo shift nesse array? No final vc teria apenas 18 threads...
<exemplo>
use threads;
use threads::shared;
my $max_threads = 18;
my @ips : shared;
my %result : shared;
@ips = get_ips();
for(1 .. $max_threads){
threads->create(\&ping_thr)->detach;
}
sub ping_thr {
while( my $ip = shift @ips) {
$result{$ip} = testa_ping($ip);
}
}
</exemplo>
Isso é só um código de explicação, mas acho q ficou clara a minha ideia,
não?
outra coisa q vc pode tentar tb é não usar o threads, mas sim o forks,
parece q a performance do forks é muito superior.
Just another Perl Hacker,
Fernando (SmokeMachine)
http://perl-e.org
2010/7/9 Lindolfo "Lorn" Rodrigues <lorn.br em gmail.com>
> João, estou com pressa agora e não posso detalhar muito o email, mas tente
> trocar sua solução de threads pelo POE
>
> http://search.cpan.org/~rcaputo/POE-1.289/lib/POE.pm
>
> Vai parecer um pouco complicado no inicio, mas depois que pega o jeito você
> vai gostar :)
> Tem muitos exemplos aqui http://poe.perl.org/?POE_Cookbook a maioria das
> coisas que fiz com ele eu usei como base algum script desse Cookbook.
>
> Enjoy!
>
>
> 2010/7/9 João André Simioni <jasimioni em gmail.com>
>
> Caros,
>>
>> tenho um script para testar cerca de 4700 equipamentos, usando ping.
>> Queria usar cerca 250 threads para isso, então o que eu fiz foi
>> quebrar o grupo de 4300 em grupos 18 elementos e abri uma thread para
>> cada.
>>
>> Não há nenhum problema com o código, o que acontece é que o tempo de
>> abertura das threads é muito alto. E quanto mais threads eu abro,
>> maior o tempo para abertura das próximas. As primeiras abrem em 30ms,
>> mas perto da thread 70 esse tempo já vai para quase 1 segundo.
>>
>> Vocês sabem como otimizar esse tempo de abertura? Ou alguma outra solução?
>>
>> Segue o output do script como exemplo:
>> Para 4677 cpes, e 250 threads, tenho no final 246 grupos, com 19 elementos
>> cada
>> Trying to create thread in Fri Jul 9 17:46:50 2010.61858
>> Created thread 1 (1) - took 0.030138 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.92080
>> Created thread 2 (2) - took 0.042668 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.134968
>> Created thread 3 (3) - took 0.064245 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.199430
>> Created thread 4 (4) - took 0.082642 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.282278
>> Created thread 5 (5) - took 0.128257 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.410769
>> Created thread 6 (6) - took 0.136458 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.547383
>> Created thread 7 (7) - took 0.208085 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.755594
>> Created thread 8 (8) - took 0.173014 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:50 2010.928756
>> Created thread 9 (9) - took 0.132713 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:51 2010.61608
>> Created thread 10 (10) - took 0.127795 seconds to create
>> Trying to create thread in Fri Jul 9 17:46:51 2010.189536
>> Created thread 11 (11) - took 0.301836 seconds to create
>>
>> Obrigado
>>
>> João André Simioni
>>
>>
>>
>> #!/usr/bin/perl
>>
>> use strict;
>> use DBI;
>> use Net::Ping;
>> use Time::HiRes qw/usleep tv_interval gettimeofday/;
>> use threads ('yield', 'stack_size' => 32*4096, 'exit' =>
>> 'threads_only', 'stringify');
>>
>> my $host = '192.168.160.179';
>> my $sid = 'HOMOLOG';
>> my $user = 'mac_user';
>> my $pass = 'mac_user';
>>
>> my $dbh = DBI->connect("dbi:Oracle:host=$host;sid=$sid", $user, $pass,
>> { AutoCommit => 1 });
>>
>> my $sth = $dbh->prepare('SELECT CPE_ID, CLIENTE_ID, CPE_DESC,
>> CPE_IP_WAN, CPE_NODE, CPE_IF FROM MAC_CPE WHERE CPE_PING = ? AND
>> CPE_ATIVO = ?');
>> $sth->execute('1', 'A');
>>
>> my @cpes;
>>
>> while (my (@row) = $sth->fetchrow_array) {
>> push @cpes, [ @row ];
>> }
>>
>> $dbh->disconnect();
>>
>> my $maxThreads = 250;
>> my $total = @cpes;
>>
>> my $cpePerThread = int($total / $maxThreads);
>>
>> my @toMonitor;
>> my $i = 0;
>> my $c = 0;
>>
>> foreach my $cpe (@cpes) {
>> push @{$toMonitor[$i]}, $cpe;
>> $c++;
>>
>> if ($c > $cpePerThread) {
>> $i++;
>> $c = 0;
>> }
>> }
>>
>> print "Para $total cpes, e $maxThreads threads, tenho no final $i
>> grupos, com ", scalar @{$toMonitor[0]}, " elementos cada\n";
>>
>> sub processResult {
>> my $r = shift;
>> if (ref $r ne 'ARRAY') {
>> print STDERR "Erro na thread\n";
>> return 0;
>> }
>> my ($result, $host, $ifs) = @$r;
>>
>> }
>>
>> sub testeClient {
>> my $threadNum = shift;
>> my $cpeGroup = shift;
>> foreach my $cpe (@{$cpeGroup}) {
>> my ($cpeId, $clientId, $cpeDesc, $cpeIp, $cpeNode, $cpeIf) =
>> @$cpe;
>> my $pingOk = &checkIp($cpeIp);
>> my $status = $pingOk ? 'RESPONDE' : 'MORTO';
>> # print join(", ", $threadNum, $cpeId, $clientId, $cpeDesc,
>> $cpeIp, $cpeNode, $cpeIf, $status), "\n";
>> }
>> # print "Finishing thread $threadNum\n";
>> return([1, 0]);
>> }
>>
>> my $threadCount = 1;
>> foreach my $cpeGroup (@toMonitor) {
>> print "Trying to create thread";
>> my $t0 = [gettimeofday];
>>
>> print " in ", scalar localtime $t0->[0], ".", $t0->[1], "\n";
>>
>> my $thr = threads->create({scalar => '1'}, 'testeClient',
>> $threadCount, $cpeGroup);
>> my $elapsed = tv_interval ( $t0, [gettimeofday]);
>> print "Created thread $thr ($threadCount) - took $elapsed seconds
>> to create\n";
>> $threadCount++;
>>
>> while (threads->list() >= $maxThreads) {
>> my @joinable = threads->list(threads::joinable);
>> for (@joinable) {
>> my $r = $_->join();
>> &processResult($r);
>> }
>> usleep(10000);
>> }
>> my $elapsed = tv_interval ( $t0, [gettimeofday]);
>> }
>>
>> while (threads->list()) {
>> my @joinable = threads->list(threads::joinable);
>> for (@joinable) {
>> my $r = $_->join();
>> &processResult($r);
>> }
>> usleep(10000);
>> }
>>
>> sub checkIp {
>> my $ip = shift;
>> my $pingOk = 0;
>>
>> eval {
>> for (1..3) {
>> my $ping = `/bin/ping -c 1 -w 1 $ip`;
>>
>> if (grep { / 0% packet loss/ } $ping) {
>> $pingOk = 1;
>> return 1;
>> }
>>
>> if ($pingOk == 0) {
>> my $p = Net::Ping->new('icmp');
>> $p->service_check(1);
>> if ($p->ping($ip, 1)) {
>> $pingOk = 1;
>> return 1;
>> }
>> }
>>
>> if ($pingOk == 0) {
>> my $p = Net::Ping->new('udp');
>> $p->service_check(1);
>> if ($p->ping($ip, 1)) {
>> $pingOk = 1;
>> return 1;
>> }
>> }
>>
>> sleep 3;
>> }
>> };
>>
>> return $pingOk;
>> }
>> _______________________________________________
>> Rio-pm mailing list
>> Rio-pm em pm.org
>> http://mail.pm.org/mailman/listinfo/rio-pm
>>
>
>
>
> --
> lorn at lornlab dot org
> Lindolfo "Lorn" Rodrigues
>
>
> _______________________________________________
> Rio-pm mailing list
> Rio-pm em pm.org
> http://mail.pm.org/mailman/listinfo/rio-pm
>
-------------- Próxima Parte ----------
Um anexo em HTML foi limpo...
URL: <http://mail.pm.org/pipermail/rio-pm/attachments/20100709/d119acaa/attachment-0001.html>
Mais detalhes sobre a lista de discussão Rio-pm