SPUG: "Programming Challenge"

Ross Wolin rowol at copper.net
Wed Dec 17 04:23:59 CST 2003


At this evening's SPUG meeting, Tim made reference to a "programming 
challenge" to be posted to the list within 24 hours.   I'm not sure I 
would go as far as to name this a challenge, but I am interested in 
getting a better understanding of what's going on.

I am working a project where I need tools to collect waveform data from 
a logic analyzer via RS232, save the data to disk, filter the data based 
on edges and other trigger conditions, then interpret the filtered data 
as LCD controller commands and render the LCD controller commands into a 
Windows bitmap to view the result of the commands.    I wrote this whole 
tool chain in perl (yee haw!) and it currently works, but that's not 
important.  =)   The part I'm concentrating on for this email is writing 
the collected waveform data to disk.

Because of the way the logic analyzer works, I get my 16 bit samples (16 
channels) by downloading all the low bytes (low 8 channels) of each 
sample, then downloading all the high bytes (upper 8 channels) of each 
sample, then combining them into word samples and writing them to disk 
(I just redirect STDOUT) - there can be up to 4M in each "half sample" 
download.   Originally I thought the fastest/most efficient way to do 
this would be to write the low bytes to one scalar (binary) string and 
the high bytes to another string using the '.' operator to append them 
to the end of the respective strings, then loop to pull out the two 
first bytes, then the two 2nd bytes, etc from each string and write them 
to the file as a word.   I tried several methods for doing this, including:

    #Method 1
    #Now interleave the low and high bytes and write to STDOUT
    while ($LSB =~/(.)/g) {
       print $1;
       $MSB =~/(.)/g;
       print $1;
    }

....

    #Method 2
    #Now interleave the low and high bytes and write to STDOUT
    for (my $x=0; $x<length($LSB); $x++) {
       syswrite(STDOUT, $LSB, 1, $x);
       syswrite(STDOUT, $MSB, 1, $x);
    }

.....

    #Method 3 (same as #1, but buffered writes)
    #Now interleave the low and high bytes and write to STDOUT,
    #buffering the write in memory first
    my $buf;
    while ($LSB =~/(.)/g) {
       $buf .= $1;
       $MSB =~/(.)/g;
       $buf .= $1;
    }
    syswrite(STDOUT, $buf, length($buf));



Admittedly, I didn't try substr() to split out individual characters, 
(sorry Tim, I thought I had) but I figure it probably would have been 
about as efficient as syswrite with a buffer offset (method 2.)   Timing 
these three methods, I got something like 8-9 minutes for #2 and around 
11 minutes for #1 and #3, just for the loop to glue together and write 
out 8M of sample data (4M of LSBs and 4M of MSBs.)

Eventually I wound up making LSB and MSB arrays and rather than using 
'.', used push() to add each byte from the logic analyzer into the array 
as a separate scalar.   Then to write out to the file at the end, I just 
did:

    #Now interleave the low and high bytes and write to STDOUT
    for (my $x=0; $x <= $#LSB; $x++) {
       print $LSB[$x],$MSB[$x];
    }


(Of course checking beforehand that $#LSB == $#MSB)   I thought this 
method would have more overhead since now I have 4M scalars in an array 
instead of a 4M string of bytes.... but of course I was wrong.  =)   
This approach took 1-2 **seconds** to write 8M, which was roughly the 
same speed as the C program I wrote to try to speed things up.


My question is: did I do something horribly wrong in the scalar/binary 
string implementations that made it go so slow, or is this just God 
(Larry) smacking me around for trying to use a byte/character as the 
basic unit of operation rather than a string?   And also, is there a 
better way to perform this operation than the array version I settled on 
(I can either put the bytes from the analyzer into an array or a 
scalar/binary string, it doesn't matter to me.)


Thanks,
Ross




More information about the spug-list mailing list