APM: File reading optimization

[Mike Stok]

On Thu, 19 Feb 2004, Brian Michalk wrote:

> Thanks to all for their input on this matter.
> 
> The packet length solution would certainly be efficient, but I'm not willing
> to set policy on something like that that I might regret in the future.
> Take for instance the GPS data.  Unless I write a shim, I can't very well
> have it send me the format we discussed.  There are many other devices, like
> gyroscopes, and a scanning laser that have serial input.
> 
> I've come up with some different scenarios.
> 1) Write kernel modules that do not emit data until a complete line is ready
> to send, and go with blocking reads.  If I carefully mix select() and
> readline(), this will work.  I just have to be careful with executing
> readlines.
> 2) Rewrite my base class to fork, or spawn threads, and go with blocking
> reads.  This would involve some complexity with communicating back to the
> parent when a message or data is received.  It might be pretty, or ugly.
> I'm still looking into this.  If there were a select() equivalent, except
> for signal, or messages ready, then this wouldn't be so bad.  My last
> experience with signals in my perl modules led to system lockups, but from
> what I'm reading, 5.8 has solved a lot of these issues.
> 
> This would all be so much easier if someone had a select() function in the
> kernel API that would return on some line separator, or even "x" characters
> of data ready.  Now, select always returns true if there is *any* data
> available.  I understand the consequences though.  It would require a lot
> more overhead to scan all of the input for line terminators.  Even so, it
> would certainly be more efficient than any readline() command, because the
> minute you do line oriented IO, every packet of data is scanned for a
> terminator.
> 
> Thoughts, feelings, comments?

Have you considered the Expect module.  It may not do what you want, but 
ages ago I used chat2.pl to control an industrial oven through an ethernet 
-> serial box and then a serial line.  This had to interact with a system 
where you got new-line-less prompts and it worked OK.  The module did all 
of the accumulation of characters and pattern searching for me.

Just a thought.

Mike

-- 
mike at stok.co.uk                    |           The "`Stok' disclaimers" apply.
http://www.stok.co.uk/~mike/       | GPG PGP Key      1024D/059913DA 
mike at exegenix.com                  | Fingerprint      0570 71CD 6790 7C28 3D60
http://www.exegenix.com/           |                  75D2 9EC4 C1C0 0599 13DA