APM: File reading optimization

Brian Michalk michalk at awpi.com
Tue Feb 17 10:56:35 CST 2004


I am in a quandry about how to do efficient filehandle reading.
I'm trying to make it uniform across all of the filehandles that may be
named pipes, device driver handles, network sockets, or stdio.

I have some slow devices on a serial line, and other fast devices that
continally generate data at high rates.  My protocol is all line oriented,
and that naturally leads me to use something like <>, but read the
following:
perldoc -f select
            WARNING: One should not attempt to mix buffered I/O (like "read"
            or <FH>) with "select", except as permitted by POSIX, and even
            then only on POSIX systems. You have to use "sysread" instead.

However sysread doesn't care about line separators.  Instead, I have to
search through the incoming data for separators and store partial reads in
my own buffer.  This is not a problem, I have code, and it works.  The
performance is bad.  C code would have the same type of problem.

The serial port dribbles in GPS data at 9600 baud, causing the select() to
return without a complete line being available, so I store all of the one or
two characters at a time in the internal buffer.  The radar data, however
can come in at 100hertz, at about 12K of data per line, and I've got
buffering turned on for performance, so I have to go searching through the
data to find the line separators.

Are there any better solutions?




More information about the Austin mailing list