[DFW.pm] Concurrency not helping? Maybe you're doing it wrong

Tommy Butler dfwpm at internetalias.net
Fri Jan 10 11:49:01 PST 2014


Hey, thanks Kevin!  Very nicely done.

--Tommy Butler

On 01/10/2014 02:15 AM, kevin wrote:
> On 01/05/2014 12:51 PM, Tommy Butler wrote:
>> Just for fun, I thought I'd post this to the list.  It's a discussion on
>> perlmonks about why the reference code (threaded version) doesn't seem
>> to have large gains over the non-threaded version.  If you are also
>> using concurrency in your hackathon code to try to speed things up, it
>> could be a matter of either or both of: you don't need it (really you
>> may not), or you're doing it wrong.
>>
>> http://perlmonks.org/?node_id=1069338
>
>
> Tommy,
>
> Sorry I'm slow, but I read the post and thought I'd give you this
> example. It's quite probable that you no longer need this as you have
> everything working, but perhaps it'll help someone. I've been meaning
> to post this to Perl Monks, but haven't gotten around to it.
>
> I remember wanting to do some things with threads some time back and
> having a devil of a time trying to find a simple example that showed
> everything I needed to do. So I cobbled this together.
>
> I think this has the "architecture" you were looking for, i.e. a
> parent in control, the children picking up tasks as fast as they could
> do them, so there's no real gate keeper. As with so many things,
> picking the right values for your task with number of children &
> number of items in the input queue is up to you. :) I think it's
> reasonably well documented, considering it's a test example, which I
> tend not to document at all, much to my detriment later. ;) It's sort
> of put together much in the manner I would have written back in the
> day when I used IPCs in C on a weekly basis.
>
> HTH,
> Kevin
>
> ===cut===
> #!/usr/bin/perl;
> #
> # Example program for threads and queues.
> #
> # The parent will create an input queue for work to be done. It will be
> # a number of tasks ($num_tasks). To do the work, it will create a number
> # of children ($num_children) to do the work. Because this is a simple
> # example, we'll just put a set of random numbers in the queue (from
> # 1 to $task_time) for the children to work on. By convention, they know
> # to sleep that number of seconds then consider the work done and
> return a
> # "result" to the parent, before grabbing the next task to be done.
> # Once all the tasks are done, they end ... as does the parent.
> #
> # In real life, the parent could add more work, the children could sleep
> # until a specific command was given, etc.
>
> use strict;
> use warnings;
> use threads qw(stringify);
> use Thread::Queue;
>
> my $num_children = 3;
> my $num_tasks = 10;
> my $task_time = 5;
>
> # create communication queues
> my $inputq = Thread::Queue->new();
> my $outputq = Thread::Queue->new();
>
> # queue up tasks
> for (1 .. $num_tasks)
> {
>     $inputq->enqueue(int(rand($task_time) + 1));
> }
>
> # make worker threads
> my @children;
> my $thr;
> for (1 .. $num_children)
> {
>     $children[$_] = $thr=threads->create(\&child, $_);
>     $thr->detach();
> }
>
> # watch for output from children
> while (my $val = $outputq->dequeue())
> {
>     # we got something...
>     print "parent received [$val]\tprocess check: ";
>
>     # see if we're done and show state of children
>     # could probably simplify this to "last if (($inputq->pending() ==
> 0) && ($outputq->pending() == 0));"
>     #   because if both queues are empty, then we're really finished,
> but this way
>     #   allows me to see the state of each child
>     my $running = 0;
>     for (my $c=1 ; $c <= $num_children ; $c++)
>     {
>         my $state = 'x';
>         if ($children[$c])
>         {
>             if ($children[$c]->is_running())
>             {
>                 $state = 'R';
>                 $running++;
>             }
>             else
>             {
>                 $children[$c] = 0;
>             }
>         }
>         print "$c=$state ";
>     }
>     print "\n";
>     last if (!$running);
> }
> print "parent done\n";
> exit(0);
>
> ####################
>
> # the work our children will do
> sub child
> {
>     my ($id) = @_;
>     my $arg;
>
>     # while there is more work to do and we can pull something off
>     while ($inputq->pending() && ($arg = $inputq->dequeue()))
>     {
>         # do work
>         sleep($arg);
>
>         # tell our parent the results
>         $outputq->enqueue("child $id had $arg");
>     }
>
>     # all the work is done, exit
>     print "child $id exiting\n";
>     return 1;
> }
> ===cut===
> _______________________________________________
> Dfw-pm mailing list
> Dfw-pm at pm.org
> http://mail.pm.org/mailman/listinfo/dfw-pm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/dfw-pm/attachments/20140110/42103376/attachment.html>


More information about the Dfw-pm mailing list