[Melbourne-pm] Finding "Out of memory!"

Bradley Dean bjdean at bjdean.id.au
Thu Oct 9 21:53:30 PDT 2014


Yep - doing that too. :)



This idea started as "it would be handy to have a drop-in
module that would help capture memory problems occurring while
they are occurring and before they are fixed". Actually - the
specific problem that started this line of thought is now
history as the memory leak has been identified and fixed.



I wasn't suggesting that an application that leaks memory
should just be left to keep doing that - my idea was really of
interest if said application only leaks memory sometimes and
it's taking a while to find why (but the application needs to
be run in the meantime).



Cheerio,



 Brad





On Fri, Oct 10, 2014, at 13:06, Toby Corkindale wrote:

When you encounter a problem that seems difficult to handle and
has very little evidence of other people handling it, then it
is a good idea to take a step back and look at the wider
picture. Your overall architecture may simply be flawed, and a
change elsewhere may simplify later issues.

In this case, I suggest you fix your design so that out of
memory errors stop occurring.
Eg. Don't load entire massive files or data structures into
memory, but instead use a specific data storage and retrieval
system that is suited to the task and can manage the paging of
data to disk with automatic caching to memory. (For instance,
SQLite or LevelDB)

  __________________________________________________________

From: "Bradley Dean" <bjdean at bjdean.id.au>

To: "Mathew Robertson" <mathew.blair.robertson at gmail.com>

Cc: "Melbourne Perl Mongers" <melbourne-pm at pm.org>

Sent: Friday, 10 October, 2014 12:51:11 PM

Subject: Re: [Melbourne-pm] Finding "Out of memory!"



Yes indeed - a non-zero exit code but not one that tells me
anything about the reason (because I can't reliable catch the
out of memory error to set a useful exit code, and perl isn't
doing that either). :)



I'm not claiming that searching for a magic string in STDERR is
a good option, but at the moment I'm not sure there's a better
one.



Cheerio,



 Brad





On Fri, Oct 10, 2014, at 11:19, Mathew Robertson wrote:

The Perl script should exit with a non-zero exit code... it
doesn't really matter why that exit code was non-zero, as any
non-zero value will indicate a failure mode.

eg: grep returns non-zero when it didn't actually grep for
anything.... (you asked it to do something, but it couldn't).

Tracing STDERR will be fraught with problems - how do you know
some Perl library doesn't randomly spit out messages that you
have yet to see?


just my $0.02,

Mathew


On 10 October 2014 10:46, Bradley Dean <[1]bjdean at bjdean.id.au>
wrote:

Yes - that's certainly a safer approach. In fact I've found
that what

worked for me in my example code was version sensitive.



On the system I started with the last system call was an mmap2
with a

ENOMEM error and this is what I found when checking $!. I then
tested

this on a different system and discovered that after the mmap2
failed

(and before my perl code had a chance to look at $!) a bunch of

rt_sigaction systems calls happened and one of those failed
with EINVAL

so there was no ENOMEM to find.



I feel there's a hack with a wrapper script which greps for
"Out of

memory!" on STDERR coming my way as the exit code doesn't tell
me

anything useful either.



Cheerio,



 Brad


On Fri, Oct 10, 2014, at 10:37, Toby Wintermute wrote:
> I really don't think this approach is going to pay dividends.
> When you're hitting out of memory conditions in Perl, it's
usually
> game over. In some languages you could code up something that
runs
> without using any more memory, as a handler, but you don't
have that
> level of control in Perl.
>
> I'd look at wrapping your Perl script in another program -
maybe just
> a bash or even another Perl script - that waits for it to
exit, and
> then checks to see if it exited normally or with an error
code.
>
> On 3 October 2014 11:54, Bradley Dean
<[2]bjdean at bjdean.id.au> wrote:
> > Greetings folks,
> >
> > While looking at a problem where a variety of scripts might
develop a
> > memory leak and fatally terminate with "Out of memory!" I'm
trying to
> > work out if this seemingly simple code is in fact too
simple:
> >
> > package NotifyOnOutOfMemory;
> > use Errno;
> > END {
> >     # If the last errno indicates out of memory perl will
now be
> >     # terminating with "Out of memory!" so tell me about
it.
> >     if ( $! == Errno::ENOMEM ) {
> >         # Do some low-memory thing that tells me a process
has died due
> >         to memory exhaustion
> >         # eg. touch /path/to/some/file
> >         #    or -
exec("/path/to/script/which/emails/or/logs $$ $0
> >         $someOtherDetail")
> >      }
> > }
> > 1;
> >
> > Which would give me a drop in module for scripts with this
problem.
> >
> > What am I missing? :)
> >
> > Cheerio,
> >
> >  Brad
> >
> > --
> > Bradley Dean
> > Email: [3]bjdean at bjdean.id.au Skype: [4]skype at bjdean.id.au
> > Mobile(Aus): [5]+61-413014395 WWW: [6]http://bjdean.id.au/
> > _______________________________________________
> > Melbourne-pm mailing list
> > [7]Melbourne-pm at pm.org
> > [8]http://mail.pm.org/mailman/listinfo/melbourne-pm
>
>
>
> --
> Turning and turning in the widening gyre
> The falcon cannot hear the falconer
> Things fall apart; the center cannot hold
> Mere anarchy is loosed upon the world


--
Bradley Dean
Email: [9]bjdean at bjdean.id.au Skype: [10]skype at bjdean.id.au
Mobile(Aus): [11]+61-413014395 WWW: [12]http://bjdean.id.au/
_______________________________________________
Melbourne-pm mailing list
[13]Melbourne-pm at pm.org
[14]http://mail.pm.org/mailman/listinfo/melbourne-pm




--
Bradley Dean
Email: bjdean at bjdean.id.au Skype: skype at bjdean.id.au
Mobile(Aus): +61-413014395 WWW: http://bjdean.id.au/





_______________________________________________

Melbourne-pm mailing list

Melbourne-pm at pm.org

http://mail.pm.org/mailman/listinfo/melbourne-pm


_______________________________________________

Melbourne-pm mailing list

[15]Melbourne-pm at pm.org

[16]http://mail.pm.org/mailman/listinfo/melbourne-pm



--
Bradley Dean
Email: bjdean at bjdean.id.au Skype: skype at bjdean.id.au
Mobile(Aus): +61-413014395 WWW: http://bjdean.id.au/

References

1. mailto:bjdean at bjdean.id.au
2. mailto:bjdean at bjdean.id.au
3. mailto:bjdean at bjdean.id.au
4. mailto:skype at bjdean.id.au
5. tel:%2B61-413014395
6. http://bjdean.id.au/
7. mailto:Melbourne-pm at pm.org
8. http://mail.pm.org/mailman/listinfo/melbourne-pm
9. mailto:bjdean at bjdean.id.au
  10. mailto:skype at bjdean.id.au
  11. tel:%2B61-413014395
  12. http://bjdean.id.au/
  13. mailto:Melbourne-pm at pm.org
  14. http://mail.pm.org/mailman/listinfo/melbourne-pm
  15. mailto:Melbourne-pm at pm.org
  16. http://mail.pm.org/mailman/listinfo/melbourne-pm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/melbourne-pm/attachments/20141010/4b77619c/attachment-0001.html>


More information about the Melbourne-pm mailing list