[Pdx-pm] Measuring memory consumption of scalars

[Marvin Humphrey]

David Wheeler writes...
>> Does this help?
>>
>>    http://search.cpan.org/dist/Devel-Size/Size.pm

I'd forgotten about that module.  I remembered only that I'd  
conducted a fairly extensive search when writing the code which ended  
up in Sort::External, and had rejected a bunch of options, including  
that one.  But it's time for a second look.

Eric Wilhelm writes...
> Looks like a good source of info and verification, but maybe not
> something that you want to put in your production code:
>
>   "Devel::Size, because of the way it works, can consume a  
> considerable
> amount of memory as it runs. It will use five pointers, two integers,
> and two bytes worth of storage, plus potential alignment and bucket
> overhead, per thing it looks at."

The bummer is that it creates that stuff anew with each call to size 
().  But... happily, it destroys all those variables when size()  
completes.  I believe the caveat above applies primarily in the case  
of assessing the size of a complex data structure.

If I use Devel::Size only to calculate the size of the last scalar in  
sortex object's own cache, the increased memory requirements will be  
negligible.  There'll be a performance hit, but we're only talking  
about calls to 2 XS subs (size() calls out to a helper based on what  
you feed it).  In the context of all the disk i/o Sort::External does  
(assuming that you feed it enough to trigger at least one flush to  
disk), that's probably acceptable.  I'll run some benchmarks.

Gotta say, the idea of avoiding XS/Inline::C code in Sort::External  
is pretty appealing.

Marvin Humphrey
Rectangular Research
http://www.rectangular.com/