[Chicago-talk] Limiting system impact.

[Jonathan Rockway]

On Wednesday 09 May 2007 02:13:21 pm Jim Jacobus wrote:
> Is there any way to limit, from within the Perl script, the amount of
> memory or CPU a Perl process utilizes? I've got a script on a Redhat
> Linux system that crunches a lot of data. Script and output are ok,
> but the process can take up to 30 minutes wall clock time. As a
> result the system appears to slow down considerably until the the
> script finishes. I tried adding a few WAIT commands in the script,
> but it didn't help much. The system reads some really large files,
> parses the info and stores it in some hashes for later processing. So
> I'm using I/o, memory and CPU. It would be good if I could lower the
> process priority of the script when it's running so the O/S could
> give more weight to other processes.

Probably a good idea to use a database like BDB or SQLite to store 
intermediate results.  These (especially BDB) will take care of memory 
management (using the disk as a backing store, of course).  If you want the 
final results in a flat file, BDB can do that for you also!

Unless you're using a really old linux, heavy CPU users aren't going to affect 
desktop response (the kernel auto-renices X to -20, in fact).   If you're 
experiencing slowness, you're probably swapping, which will slow things down.  
Get some more memory or reduce it as per above.  If reducing memory usage 
doesn't help, you can nice io with ionice, which might also help.

-- 
package JAPH;use Catalyst qw/-Debug/;($;=JAPH)->config(name => do {
$,.=reverse qw[Jonathan tsu rehton lre rekca Rockway][$_].[split //,
";$;"]->[$_].q; ;for 1..4;$,=~s;^.;;;$,});$;->setup;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: not available
Url : http://mail.pm.org/pipermail/chicago-talk/attachments/20070509/7eba31f2/attachment-0001.bin