[pm-h] Perl merging many large files into one

Michael R. Davis mrdvt92 at yahoo.com
Mon Mar 31 06:10:42 PDT 2014


G. Wade,
 
>> Can anyone tell me if the diamond operator is optimized in a print
>> statement or does it really read the file into memory then print it? 
>
>The real question is are you doing more than just concatenating the
>files? 
 
I guess this comment sparked research and `tail` actually does what I need to do.
 
qx{cat X > out.txt; tail --lines=+2 Y Z >> out.txt};
 
Maybe I'll just head out to the command line it just won't be portable.  There will be about 300 CSV files to merge in the process but I only need the header row on the first one.
 
But, performance is not really as big of a concern as memory for this process.
Thanks,
Mike 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/houston/attachments/20140331/3e9e8a56/attachment.html>


More information about the Houston mailing list