SPUG: "Programming Challenge"
Ross Wolin
rowol at copper.net
Wed Dec 17 04:23:59 CST 2003
At this evening's SPUG meeting, Tim made reference to a "programming
challenge" to be posted to the list within 24 hours. I'm not sure I
would go as far as to name this a challenge, but I am interested in
getting a better understanding of what's going on.
I am working a project where I need tools to collect waveform data from
a logic analyzer via RS232, save the data to disk, filter the data based
on edges and other trigger conditions, then interpret the filtered data
as LCD controller commands and render the LCD controller commands into a
Windows bitmap to view the result of the commands. I wrote this whole
tool chain in perl (yee haw!) and it currently works, but that's not
important. =) The part I'm concentrating on for this email is writing
the collected waveform data to disk.
Because of the way the logic analyzer works, I get my 16 bit samples (16
channels) by downloading all the low bytes (low 8 channels) of each
sample, then downloading all the high bytes (upper 8 channels) of each
sample, then combining them into word samples and writing them to disk
(I just redirect STDOUT) - there can be up to 4M in each "half sample"
download. Originally I thought the fastest/most efficient way to do
this would be to write the low bytes to one scalar (binary) string and
the high bytes to another string using the '.' operator to append them
to the end of the respective strings, then loop to pull out the two
first bytes, then the two 2nd bytes, etc from each string and write them
to the file as a word. I tried several methods for doing this, including:
#Method 1
#Now interleave the low and high bytes and write to STDOUT
while ($LSB =~/(.)/g) {
print $1;
$MSB =~/(.)/g;
print $1;
}
....
#Method 2
#Now interleave the low and high bytes and write to STDOUT
for (my $x=0; $x<length($LSB); $x++) {
syswrite(STDOUT, $LSB, 1, $x);
syswrite(STDOUT, $MSB, 1, $x);
}
.....
#Method 3 (same as #1, but buffered writes)
#Now interleave the low and high bytes and write to STDOUT,
#buffering the write in memory first
my $buf;
while ($LSB =~/(.)/g) {
$buf .= $1;
$MSB =~/(.)/g;
$buf .= $1;
}
syswrite(STDOUT, $buf, length($buf));
Admittedly, I didn't try substr() to split out individual characters,
(sorry Tim, I thought I had) but I figure it probably would have been
about as efficient as syswrite with a buffer offset (method 2.) Timing
these three methods, I got something like 8-9 minutes for #2 and around
11 minutes for #1 and #3, just for the loop to glue together and write
out 8M of sample data (4M of LSBs and 4M of MSBs.)
Eventually I wound up making LSB and MSB arrays and rather than using
'.', used push() to add each byte from the logic analyzer into the array
as a separate scalar. Then to write out to the file at the end, I just
did:
#Now interleave the low and high bytes and write to STDOUT
for (my $x=0; $x <= $#LSB; $x++) {
print $LSB[$x],$MSB[$x];
}
(Of course checking beforehand that $#LSB == $#MSB) I thought this
method would have more overhead since now I have 4M scalars in an array
instead of a 4M string of bytes.... but of course I was wrong. =)
This approach took 1-2 **seconds** to write 8M, which was roughly the
same speed as the C program I wrote to try to speed things up.
My question is: did I do something horribly wrong in the scalar/binary
string implementations that made it go so slow, or is this just God
(Larry) smacking me around for trying to use a byte/character as the
basic unit of operation rather than a string? And also, is there a
better way to perform this operation than the array version I settled on
(I can either put the bytes from the analyzer into an array or a
scalar/binary string, it doesn't matter to me.)
Thanks,
Ross
More information about the spug-list
mailing list