From nagler at bivio.biz  Wed Feb 13 15:36:00 2008
From: nagler at bivio.biz (Rob Nagler)
Date: Wed, 13 Feb 2008 16:36:00 -0700
Subject: [Boulder.pm] mod_perl package lexicals becoming corrupt
Message-ID: <18355.32480.877940.997452@ski.local>

Ran into a very nasty mod_perl bug.  Seems like package lexical
variables are getting trashed if there are "too many" of them.  When
we switch the "my" to "our" the failures went away.  There are two
distinct failure modes.

In http://www.bivio.biz/f/bOP/lib/Bivio/Biz/Model/CRMForm.pm
If I change the the $_RFC & $_CLOSED from "our" to "my", I get a
failure of this form:

    Bivio::DieCode::DIE: Can't call method "eq_new" on an undefined value at
    /Users/nagler/src/perl/Bivio/Biz/Model/CRMActionList.pm line 87.
    
What's happened is that $_CLOSE becomes undef at the same point in
execution in mod_perl.  It does not happen in the unit test, but only
when we run it under mod_perl.  We have seen this failure in an
unrelated module, which had another package lexical.  The value
becomes undef, even though it is correctly initialized.

The other failure looks like this:

    Bivio::DieCode::DIE: Undefined subroutine
    &Bivio::Biz::Model::MailForm::validate called at
    /Users/nagler/src/perl/Bivio/Biz/FormModel.pm line 908.

MailForm does not have a "validate" method:

http://www.bivio.biz/f/bOP/lib/Bivio/Biz/Model/MailForm.pm

It's in the superclass.  Again, if I change the our's at the top of
MailForm to my, the acceptance test passes (mail.btest).

We're cautiously proceeding that the problem is with the maximum
number of package lexicals.  Not conclusive, but something to watch
for. 

Rob

From nagler at bivio.biz  Fri Feb 15 08:04:29 2008
From: nagler at bivio.biz (Rob Nagler)
Date: Fri, 15 Feb 2008 09:04:29 -0700
Subject: [Boulder.pm] mod_perl package lexicals becoming corrupt
Message-ID: <18357.47117.394477.840745@ski.local>

Ran into a very nasty mod_perl bug.  Seems like package lexical
variables are getting trashed if there are "too many" of them.  When
we switch the "my" to "our" the failures went away.  There are two
distinct failure modes.

In http://www.bivio.biz/f/bOP/lib/Bivio/Biz/Model/CRMForm.pm
If I change the the $_RFC & $_CLOSED from "our" to "my", I get a
failure of this form:

    Bivio::DieCode::DIE: Can't call method "eq_new" on an undefined value at
    /Users/nagler/src/perl/Bivio/Biz/Model/CRMActionList.pm line 87.
    
What's happened is that $_CLOSE becomes undef at the same point in
execution in mod_perl.  It does not happen in the unit test, but only
when we run it under mod_perl.  We have seen this failure in an
unrelated module, which had another package lexical.  The value
becomes undef, even though it is correctly initialized.

The other failure looks like this:

    Bivio::DieCode::DIE: Undefined subroutine
    &Bivio::Biz::Model::MailForm::validate called at
    /Users/nagler/src/perl/Bivio/Biz/FormModel.pm line 908.

MailForm does not have a "validate" method:

http://www.bivio.biz/f/bOP/lib/Bivio/Biz/Model/MailForm.pm

It's in the superclass.  Again, if I change the our's at the top of
MailForm to my, the acceptance test passes (mail.btest).

We're cautiously proceeding that the problem is with the maximum
number of package lexicals.  Not conclusive, but something to watch
for. 

Rob


From nagler at bivio.biz  Wed Feb 20 11:49:32 2008
From: nagler at bivio.biz (Rob Nagler)
Date: Wed, 20 Feb 2008 12:49:32 -0700
Subject: [Boulder.pm] mod_perl package lexicals becoming corrupt
In-Reply-To: <18357.47117.394477.840745@ski.local>
References: <18357.47117.394477.840745@ski.local>
Message-ID: <18364.33868.745487.796923@ski.local>

Found the bug.  Here's what demonstrates it:

    package T1;
    use strict;
    sub s1 {}

    package T2;
    use base 'T1';
    use strict;

    package main;
    T2->s1;
    print("ok\n");
    \&T2::s1;
    print("not reached\n");
    T2->s1;

The \&T2:s1 autovivifies something in the package hash, which is
"nasty", and actually ends up corrupting the package hash so that
lexicals disappear.  We had a function called super_for_method, which
did this:

    foreach my $a (@{$proto->inheritance_ancestors}) {
 	my($sub) = \&{$a . '::' . $method};
 	next unless defined(&$sub);
 	return ($sub, $a);
    }

The fact that \&{} seems to autovivify an invalid package hash entry
is the problem.  You can implement the above this way:

    foreach my $a (@{$proto->inheritance_ancestors}) {
	my($sub) = $a . '::' . $method;
	do {
	    no strict 'refs';
	    next unless defined(&$sub);
	};
	return (\&$sub, $a);
    }

The above code does not corrupt the the package hash.  Both pieces of
code autovivify the glob for $sub, but in the former case, the
autovivification has a dangerous side effect of corrupting the entire
symbol table (in a subtle way, of course, which is why I got confused
by the number of lexicals).

Rob