[Pdx-pm] Rich scalars
Marvin Humphrey
marvin at rectangular.com
Sat Jul 23 06:58:30 PDT 2005
Greets,
With the change to Sort::External to use -mem_threshold as the buffer
flush trigger, it has become necessary to rework other parts of the
module which relied on large buffers. The trickiest section has the
filestream merging. I snooped the code in File::Sort,
File::MergeSort, and Sort::Merge to see if there was something I
could crib, but the number of dereference ops each required made me
blanche. I needed something like the File::Temp object, that could
both function as a scalar and associate a filehandle.
Blessing a reference to a scalar seemed like a good solution, but
since the scalar only has one storage slot, associating it with the
filehandle that would be used to refill the buffer was challenging.
Here's how the code turned out, edited for clarity:
-------------------------------------------------------
{
### ...
my @buffers;
my $num_packages = 0;
for my $filehandle (@temp_file_objects) {
my $holder;
my $package_name = "Sort::External::Buffer$num_packages";
$num_packages++;
my $buff = bless \$holder, $package_name;
no strict 'refs';
no warnings 'redefine';
*{ $packagename . "::next_line" }
= sub {
return unless defined(${ $_[0] } = <$filehandle>)
};
next unless defined $buff->next_line;
push @buffers, $buff;
}
### ...
### Clear out the closures so that we don't leak memory.
for (0 .. $num_packages) {
*{ "Sort::External::Buffer$_" . "::next_line" } = sub {};
}
}
-------------------------------------------------------
Each scalar is blessed into a unique package. Calling next_line()
loads the next buffer item into the scalar object's sole storage
slot. Each next_item() sub is a closure which stores a unique temp
filehandle.
It is theoretically possible to associate an unlimited number of
attributes and methods with a blessed scalar using this technique.
Comments?
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
More information about the Pdx-pm-list
mailing list