[Chicago-talk] @ARGV while(<>)

Matt Hucke hucke at cynico.net
Fri Jan 4 13:59:07 PST 2008


> my $body = "";
> while(<>) {
>     unless(/epsf\[/) {
>           $body = $body . $_;
> }

Does it choke on extremely large files?

You're doing repeated concatenation onto a string, which gets bigger each time through the loop. 
This might mean a lot of realloc's and copying of the string in memory.  (I may be wrong, I haven't 
studied perl internals).

Try putting the lines into an array and then join'ing them at the very end.

while (<>)
{
     push (@lines, $_) unless (/espf\[/);
}
my $body = join("", @lines);

(I once downloaded a free program to transform a proprietary mailbox file format into standard 
format.  It slurped the entire mailbox file into one big string, as in the sample program above, 
then did repeated s/foo/whatever/e on it.  This worked fine for tiny files, but took forever to 
handle anything larger than 10MB or so.  I eventually rewrote it to do line-by-line transformations).

-- 
hucke at cynico.net
http://www.graveyards.com


More information about the Chicago-talk mailing list