SPUG: Sorting a big file

Kurt Buff KBuff at zetron.com
Fri Aug 27 15:19:55 CDT 2004

Break it into pieces, sort those, them merge them.

Not terribly fast, but it should work.

-----Original Message-----
From: spug-list-bounces at mail.pm.org
[mailto:spug-list-bounces at mail.pm.org]On Behalf Of Dan Ebert
Sent: Friday, August 27, 2004 12:53
To: spug-list at mail.pm.org
Subject: SPUG: Sorting a big file

I have a large file (~1 million lines, ~142MB) which I need to sort (any
order is fine, just so the lines are in a repeatable order).

Just using perl's 'sort' on the file read into an array eats up all the
RAM and swap on my box and crashes.  I'm also trying tying the file as an
array, but it looks like that is also going to use up all the memory. Does
anyone know some other methods I could try?


Immigration is the sincerest form of flattery.
	- Unknown

Seattle Perl Users Group Mailing List  
POST TO: spug-list at mail.pm.org  http://spugwiki.perlocity.org
ACCOUNT CONFIG: http://mail.pm.org/mailman/listinfo/spug-list
MEETINGS: 3rd Tuesdays, Location Unknown
WEB PAGE: http://www.seattleperl.org


More information about the spug-list mailing list