[Pdx-pm] Rich scalars

Marvin Humphrey marvin at rectangular.com
Sat Jul 23 06:58:30 PDT 2005


With the change to Sort::External to use -mem_threshold as the buffer  
flush trigger, it has become necessary to rework other parts of the  
module which relied on large buffers.  The trickiest section has the  
filestream merging.  I snooped the code in File::Sort,  
File::MergeSort, and Sort::Merge to see if there was something I  
could crib, but the number of dereference ops each required made me  
blanche.  I needed something like the File::Temp object, that could  
both function as a scalar and associate a filehandle.

Blessing a reference to a scalar seemed like a good solution, but  
since the scalar only has one storage slot, associating it with the  
filehandle that would be used to refill the buffer was challenging.

Here's how the code turned out, edited for clarity:

     ### ...

     my @buffers;
     my $num_packages = 0;

     for my $filehandle (@temp_file_objects) {
         my $holder;
         my $package_name = "Sort::External::Buffer$num_packages";
         my $buff = bless \$holder, $package_name;
         no strict 'refs';
         no warnings 'redefine';
         *{ $packagename . "::next_line" }
             = sub {
                 return unless defined(${ $_[0] } = <$filehandle>)
         next unless defined $buff->next_line;
         push @buffers, $buff;

     ### ...

     ### Clear out the closures so that we don't leak memory.
     for (0 .. $num_packages) {
         *{ "Sort::External::Buffer$_" . "::next_line" } = sub {};

Each scalar is blessed into a unique package.  Calling next_line()  
loads the next buffer item into the scalar object's sole storage  
slot.  Each next_item() sub is a closure which stores a unique temp  

It is theoretically possible to associate an unlimited number of  
attributes and methods with a blessed scalar using this technique.


Marvin Humphrey
Rectangular Research

More information about the Pdx-pm-list mailing list