Chris, are you able to post the file to a public place for us to look? Feel free to PM me as well.<br><br><br>-nw<br><br><div class="gmail_quote">On Wed, Nov 10, 2010 at 6:56 PM, Christopher Hahn <span dir="ltr"><<a href="mailto:xrz1138@gmail.com">xrz1138@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hey team,<br>
<br>
I am trying to parse a huge (7 Gb) file that is line oriented but has<br>
large sections<br>
that are any kind of binary character.<br>
<br>
(this is a p42svn dump file of a large perforce repository)<br>
<br>
I tried several smarter things, but found the after running for a while<br>
my console would just close....dead, gone:<br>
============================<br>
administrator@cmSVNDumper-09:/p42svn/testing$ ./<a href="http://p4dump-parse-new.pl" target="_blank">p4dump-parse-new.pl</a><br>
Killed<br>
============================<br>
<br>
I am sure that there are odd chars in the file that are doing this....<br>
<br>
I tried setting binmode on the input file handle, and just loading the entire<br>
file into a buffer, just as a test, as we have enough memory to do this.<br>
<br>
The result:<br>
===========================================<br>
open(OUTF, ">SM_amanda_238037_fixed.dump")<br>
or die "Opening output file failed: $!";<br>
<br>
open(INF, "SM_amanda_238037_bad.dump")<br>
or die "Opening input file failed: $!";<br>
binmode INF;<br>
<br>
my @buffer = <INF>;<br>
<br>
print OUTF @buffer;<br>
<br>
close(INF);<br>
close(OUTF);<br>
===========================================<br>
<br>
I watched using "top" and after the memory used climbed to a tad<br>
more than the size of the file, the "Killed" message appeared and the<br>
console closed itself.<br>
<br>
I have to stay at work until this is done, and so am just hoping the<br>
someone if online and can give me the kick in the head that I need.<br>
<br>
In any case, thanks for the attention,<br>
<br>
Chris<br>
<font color="#888888"><br>
--<br>
Realisant mon espoir, je me lance vers la gloire.<br>
Christopher Hahn == <a href="mailto:xrz1138@gmail.com">xrz1138@gmail.com</a><br>
_______________________________________________<br>
San-Diego-pm mailing list<br>
<a href="mailto:San-Diego-pm@pm.org">San-Diego-pm@pm.org</a><br>
<a href="http://mail.pm.org/mailman/listinfo/san-diego-pm" target="_blank">http://mail.pm.org/mailman/listinfo/san-diego-pm</a><br>
</font></blockquote></div><br>