SPUG: Parsing Huge Files

Aaron West tallpeak at hotmail.com
Wed Nov 28 10:34:48 PST 2007


If you don't care for exactly 1000 lines and just want to peek into "some
stuff" near the end of the file, you could just seek to say 1 megabyte less
than end of file, or something like that, then read the last megabyte, then
parse through it. Might be simpler than counting lines.
 

-----Original Message-----
From: spug-list-bounces+tallpeak=hotmail.com at pm.org
[mailto:spug-list-bounces+tallpeak=hotmail.com at pm.org] On Behalf Of
mayank_a at students.iiit.ac.in
Sent: Tuesday, November 27, 2007 9:52 PM
To: spug-list at pm.org
Subject: SPUG: Parsing Huge Files

Hi All,

I need to parse often huge files of more than 3GB each, and take out
chunks of data, with a certain delimiter spanning multiple lines.
Mainly I am interested only starting and ending 1000 lines.

Is there a faster execution hook, where I can get last 1000  lines of huge
file, without skipping to the end of file.

I am new to the list, so don't know, whether it's the right place to ask
such question.

Thanks and Regards
Mayank



More information about the spug-list mailing list