APM: File reading optimization

Chris Vaughan vaughan99 at yahoo.com
Tue Feb 17 12:10:55 CST 2004


Brian,

If you have the flexibility, you may want to consider changing
your protocol away from separators and towards packets.  If you
don't have the flexibility, then don't read on.

Have the sender send packed data length (in bytes), then the
data itself, in a loop.  The reader would simply block waiting
for the first 4 bytes, construct a count by unpacking the
integer, and block reading that count of the handle, forming
your message.  After the reader reads the message, it blocks
again waiting for the next count.

The downside to this solution is that the reader has two logical
reading states.  If the reader gets out of sync for any reason,
you're screwed.

Regards,
Chris

--- Brian Michalk <michalk at awpi.com> wrote:
> I am in a quandry about how to do efficient filehandle
> reading.
> I'm trying to make it uniform across all of the filehandles
> that may be
> named pipes, device driver handles, network sockets, or stdio.
> 
> I have some slow devices on a serial line, and other fast
> devices that
> continally generate data at high rates.  My protocol is all
> line oriented,
> and that naturally leads me to use something like <>, but read
> the
> following:
> perldoc -f select
>             WARNING: One should not attempt to mix buffered
> I/O (like "read"
>             or <FH>) with "select", except as permitted by
> POSIX, and even
>             then only on POSIX systems. You have to use
> "sysread" instead.
> 
> However sysread doesn't care about line separators.  Instead,
> I have to
> search through the incoming data for separators and store
> partial reads in
> my own buffer.  This is not a problem, I have code, and it
> works.  The
> performance is bad.  C code would have the same type of
> problem.
> 
> The serial port dribbles in GPS data at 9600 baud, causing the
> select() to
> return without a complete line being available, so I store all
> of the one or
> two characters at a time in the internal buffer.  The radar
> data, however
> can come in at 100hertz, at about 12K of data per line, and
> I've got
> buffering turned on for performance, so I have to go searching
> through the
> data to find the line separators.
> 
> Are there any better solutions?
> 
> _______________________________________________
> Austin mailing list
> Austin at mail.pm.org
> http://mail.pm.org/mailman/listinfo/austin


=====
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    Chris Vaughan    | "I love deadlines.  I like the
                     |  swooshing sound as they fly by."
 vaughan99 at yahoo.com |   - Douglas Adams

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html



More information about the Austin mailing list