[Melbourne-pm] Use of each() on hash after insertion without resetting hash interator results in undefined behavior

[Alexandre Andrade]

Dean/Toby,
Thank you a lot for you replies !

Dean, very clear explanation! But In this case I need to check if the
values of the Hash item is complete (the main thread writes several times
to each item before I can use it) so if I pop I have to push it back in
case it is incomplete - And thanks for sharing Gavin's material - a most
read !

Toby,
Very well observed but I already have to lock command around the Hash
interactions  - I've cleaned them up to keep the code clear, that was a
poor choice.
I have a test case where I generate sequences, consume then using this
scrip and compare the output with another processes and even after 20 hours
of execution I got not fails - the only issue was that warning popping for
most of the time - I was using a older version of Perl and I don't remember
that message showing up.

As I couldn't get rid of that and I am compiling Perl from source, I've
decide to just comment that line from the "hv.c" and I will let my test
code run for a few days to guarantee I'm not hiding a bug.

Cheers,
Alex


On Mon, Oct 20, 2014 at 3:52 PM, Toby Corkindale <
toby.corkindale at strategicdata.com.au> wrote:

> Hi Andre,
> In my experience, Perl threads don't work properly. Not just my opinion
> either --the official stance is that their use is "strongly discouraged".
>
> To avoid the error you're seeing, I think you'd need a mutex lock around
> the hash access, ie. so that the foreach loop can't run at the same time as
> the hash insertion -- except that if you're going to do that, then your
> program may as well be single threaded.
> Consider rewriting your code to use forking with a queue if you're after
> performance and reliability.
> If you do stick with threads, then you also need to take more care around
> race conditions - for instance, consider what happens if a requestid is
> added to the hash for a second time, while the first time is being
> processed in doSomeMagic().
>
> Cheers,
> Toby
>
> ----- Original Message -----
> > From: "Alexandre Andrade" <alexandre.andrade at ecetera.com.au>
> > To: melbourne-pm at pm.org
> > Sent: Monday, 20 October, 2014 3:27:02 PM
> > Subject: [Melbourne-pm] Use of each() on hash after insertion without
> resetting hash interator results in undefined
> > behavior
> >
> > Hi Folks,
> > I'm using Perl v5.20.0 for sun4-solaris-thread-multi-64 on Solaris 11.
> > I have a main thread that reads from disk and adds some values to a
> Shared
> > Hash and a secondary thread that goes through that Hash, uses some values
> > and delete the used values from the hash.
> > Something like this:
> >
> > # This Thread will run in paralel and check the Memory Hash for completed
> > transactions and do some tricks
> > my $ thr = threads -> new ( \ &check_time );
> > while ( defined ( my $ line = $ file -> read )) {
> > my @ transaction_details : shared ;
> > .
> > .
> > .
> > $ transactions { $ transaction ->{ "RequestId" } } = \ @
> transaction_details
> > ;
> > }
> > sub check_time () {
> > while ( 1 ) {
> > {
> > foreach my $ key ( keys ( % transactions ) ) {
> > doSomeMagic( $ transactions { $ key } );
> > delete ( $ transactions { $ key });
> > # Abandoning the Foreach Loop as I've changed the %transactions memory
> map
> > last ;
> > }
> > sleep(1);
> > }
> > }
> >
> > --
> > The code works but the warning:
> >
> > Use of each() on hash after insertion without resetting hash interator
> > results in undefined behavior
> >
> > Is flooding the logs - I've tried to set "no warnings" or create a copy
> of
> > %transactions and interact with it instead of %transaction itself but no
> > luck so far.
> >
> > Any ideas?
> >
> > Regards,
> >
> > Alexandre Andrade | APM Consultant | Ecetera
> > alexandre.andrade at ecetera.com.au | M: +61 41307 1370
> >
> > Helping rid the World of badly behaving Apps by investigating my
> Customers
> > problems and finding solutions to make things work faster and better.
> > Connect to Ecetera on LinkedIn , Twitter
> >
> >
> >
> >
> >
> >
> >
> >
> > _______________________________________________
> > Melbourne-pm mailing list
> > Melbourne-pm at pm.org
> > http://mail.pm.org/mailman/listinfo/melbourne-pm
> _______________________________________________
> Melbourne-pm mailing list
> Melbourne-pm at pm.org
> http://mail.pm.org/mailman/listinfo/melbourne-pm
>



-- 

Regards,

Alexandre Andrade | APM Consultant | Ecetera
alexandre.andrade at ecetera.com.au | M: +61 41307 1370

Helping rid the World  of badly behaving Apps by investigating my Customers
problems and finding solutions to make things work faster and better.
Connect to Ecetera on LinkedIn <http://www.linkedin.com/company/ecetera>,
Twitter <http://www.twitter.com/#!/EceteraAU>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/melbourne-pm/attachments/20141020/c791917f/attachment.html>