Processing Data Partitions in Perl

[Ziya Suzen]

On Wed, Jul 2, 2008 at 9:32 PM, Tom Hukins <tom at eborcom.com> wrote:
[..]
>   1  Cuba     NB
>   2  New Inn  NB
>   3  Spoons   CMK
>   4  Vaults   Stony
>   5  Crown    Stony

Assuming ordered by location I suppose?

> Chapter 19 of SQL For Smarties (a wonderful book for anyone doing
> anything non-trivial with SQL databases) tells me that I should refer to
> such groups of records as partitions.

Didn't know that. Thanks.

[..]
>
> So I wrote myself a little subroutine to do just this:
>  sub make_partition_sub {
>      my $record_sub = shift;
>      my $field = shift;
>
>      my (@group, $sub);
>
>      $sub = {
>          my $cur = $record_sub->();
>          return if (! defined $cur && ! scalar @group);
>
>          if (! defined $cur || $cur->{$field} ne $group[0]{$field}) {
>              my @old_group = @group;
>              @group = defined $cur ? $cur : ();
>              return [ @old_group ];
>          }
>          else {
>              push @group, $cur;
>              $sub->();
>          }
>      };
>
>      return $sub;
>  }
>

This hurts my brain... Too many code refs and recursion :-(
But hey, if it works I don't want to know what's under the bonnet.

>
> I'm not too happy that it calls itself and returns from various places
> within itself, but my attempts to make it more readable failed.
>
> Anyway, when I call it as follows I get the results I want:
>  my $return_record = sub { $sth->fetchrow_hashref() };
>  my $get_partition = make_partition_sub($return_record, 'location');
>  while (my $partition_ref = $get_partition->()) {
>      my @partition = @$partition_ref;
>      # Do something with @partition
>  }
>

I think more object oriented sort of API might be easier to understand and use.
Something like:

my $partitioner = Data::Partitioner->new( provider => sub { .. },
column_name => 'location' );

while (my $partition = $partitioner->get_partition()) { # returns
Data::Partition object
    while (my $rec = $partition->get_record()) {
        # Do something ...
    }
}

>
> I thought I should run it past you lot to get feedback, check I haven't
> reinvented rounder wheels and to see whether you reckon I've written
> something generally useful that warrants packaging up and releasing,
> despite its brevity.
>

I haven't seen anything similar on CPAN. I think it deserves to be CPANed.

>
> Finally, what should I call such a thing?
>

Hmm.. I guess I've already answered that ;-)

Cheers

--Ziya