<html><body><div style="color:#000; background-color:#fff; font-family:Courier New, courier, monaco, monospace, sans-serif;font-size:10pt"><div><span>Thanks for the light. The Text::CSV_XS is used. (Someone's baby Perl script for parsing the files take 15+ times more when </span><span>utf8 is </span><span>switched on).</span></div><div><span><br></span></div><div><span>I will try:</span></div><div><span class="tab"> </span><span>1) Eliminating as much regex as possible from the script;</span></div><div><span class="tab"> </span><span>2) Scarifying readability, eliminating the call to the subs;</span></div><div><br><span></span></div><div><span>If you think there are anything can help its performance, please let me know.<br></span></div><div><br><span></span></div><div><span>Thanks,</span></div><br><div><br></div> <div style="font-family: Courier New, courier, monaco, monospace, sans-serif; font-size: 10pt;"> <div
style="font-family: times new roman, new york, times, serif; font-size: 12pt;"> <div dir="ltr"> <font face="Arial" size="2"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Andrew Rodland <andrew@cleverdomain.org><br> <b><span style="font-weight: bold;">To:</span></b> Chicago.pm chatter <chicago-talk@pm.org> <br> <b><span style="font-weight: bold;">Sent:</span></b> Thursday, January 26, 2012 7:01 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [Chicago-talk] Perl script run slower for utf8<br> </font> </div> <br>
<div id="yiv291957454">Sometimes you can either do the wrong thing quickly, or do the right thing slowly. This is one of those times. Unicode support slows down a lot of matching operations because character class matching isn't just a matter of looking at bits in 256-entry bitmaps anymore.<div>
<br></div><div>I would, however, check whether you have Text::CSV_XS installed, as it's faster than the pure-perl Text::CSV, and its speed is probably less-affected by Unicode.<br><br><div class="yiv291957454gmail_quote">On Thu, Jan 26, 2012 at 3:58 PM, tiger peng <span dir="ltr"><<a rel="nofollow" ymailto="mailto:tigerpeng2001@yahoo.com" target="_blank" href="mailto:tigerpeng2001@yahoo.com">tigerpeng2001@yahoo.com</a>></span> wrote:<br>
<blockquote class="yiv291957454gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="font-size:10pt;font-family:arial, helvetica, sans-serif;"><div>Hello all,<br></div><div><br></div><div>I just made a Perl script for parsing large CSV files (with Text::CSV). When I enable Unicode (with the three use uncommented), it took as twice longer time. Is it normal? Is there any way to speed it up? <br>
</div><div><br></div><div>Thanks,</div><div><br></div><div><span style="font-weight:bold;font-family:Courier New, courier, monaco, monospace, sans-serif;">#use utf8;</span><br style="font-weight:bold;font-family:Courier New, courier, monaco, monospace, sans-serif;">
<span style="font-weight:bold;font-family:Courier New, courier, monaco, monospace, sans-serif;">#use encoding "utf-8";</span><br style="font-weight:bold;font-family:Courier New, courier, monaco, monospace, sans-serif;"><span style="font-weight:bold;font-family:Courier New, courier, monaco, monospace, sans-serif;">#use open
':encoding(utf8)';</span></div></div></div><br>_______________________________________________<br>
Chicago-talk mailing list<br>
<a rel="nofollow" ymailto="mailto:Chicago-talk@pm.org" target="_blank" href="mailto:Chicago-talk@pm.org">Chicago-talk@pm.org</a><br>
http://mail.pm.org/mailman/listinfo/chicago-talk<br></blockquote></div><br></div>
</div><br>_______________________________________________<br>Chicago-talk mailing list<br><a ymailto="mailto:Chicago-talk@pm.org" href="mailto:Chicago-talk@pm.org">Chicago-talk@pm.org</a><br><a href="http://mail.pm.org/mailman/listinfo/chicago-talk" target="_blank">http://mail.pm.org/mailman/listinfo/chicago-talk</a><br><br> </div> </div> </div></body></html>