SPUG: "Programming Challenge"

[David Dyck]

On Thu, 18 Dec 2003 at 00:08 -0800, Yitzchak Scott-Thoennes <sthoenna at efn.o...:

> On Wed, Dec 17, 2003 at 11:00:56PM -0800, David Dyck <david.dyck at fluke.com> wrote:
> > use Encode;
> > sub block_size { 0x2000 };
> > for (my $i=0; $i < length $LSB; $i+= block_size ) {
> >     syswrite STDOUT,   encode("UCS-2LE", substr($LSB, $i, block_size ))
> >                      ^ encode("UCS-2BE", substr($MSB, $i, block_size ));
> > }
> >
> > I'm not sure how the original $LSB and $MSB were being read in, but
> > it might have been good to read it in by a similar chunk to use less RAM.
> >
> > It was educational to look into the Encode module, as
> > it looks like the .xs (C) implementation of encode is specialized version
> > of pack, even using the "v" and "n" characters for endian flags.
> >
> > My thanks to Yitzchak and Ross for this learning opportunity,
> >  David
>
> I'm curious to know if you experimented with different block sizes.
> I'd guess its not swapping but cpu cache that is the issue.

Yes, I tried larger and smaller block sizes, as I narrowed in
on the best times plus/minus a half second.  My first clue
was when I ran out of memory (literally got the out of memory message,
as I have a small swap space), when the block size was too small, I'd
guess the cost of the loop became to great (plus there's probably
some extra overhead in encode to setup the function), but I could
see execution time increase as the block size got too big, so
I chose the smallest block size that still was within .5 seconds of
fastest run time.

I initialized my buffers using the following code so
I could verify the byte order etc...:

my $LSB = (join '', map {chr} 0 .. 0xff) x 0x4000;
my $MSB = join '', (map {chr() x 0x100} 0 .. 0xff) x 0x40;