[Phoenix-pm] Multi-level hash - quick key reference?

Wed Jun 7 13:25:18 PDT 2006

OK, one last post before everyone is tired of getting spam from
me...just thought I'd pass this along in case others face this and don't
know this like I didn't.

To allocate buckets in your hash using a single malloc do the following.
This apparently can greatly improve efficiency when building large
hashes like I'll be doing as opposed to the individual mallocs Perl
would do when making individual assignments, but seems to me it'd still
have to re-malloc due to variable lenght key values.

keys %hash = 200;		# Allocates 200 buckets
undef %hash;		# de-allocates the 200 buckets

Anyone have personal experience with using this that could comment on
it's effectiveness in improving efficiency?

Bobby

-----Original Message-----
From: Metz, Bobby W, WWCS 
Sent: Wednesday, June 07, 2006 1:20 PM
To: Brock
Cc: phoenix-pm at pm.org
Subject: RE: [Phoenix-pm] Multi-level hash - quick key reference?

For anyone interested...I'd never looked this up before and appears my
memory usage concern is unfounded:

"The key structure, though, is of variable length because the key string
is of variable length, so perl has to ask the system for a memory
allocation for each key. The base size of this structure is 8 or 16
bytes (once again, depending on whether you're on a 32 bit or 64 bit
system) plus the string length plus two bytes.

Since this memory has to be allocated from the system there's the malloc
size-field overhead (4 or 8 bytes) plus the alignment bytes (0 to 7,
depending on your system and the key length) that get added on to the
chunk perl requests. If the key is only 1 character, and you're on a 32
bit system, the allocation will be 16 bytes. If the key is 7 characters
then the allocation is 24 bytes on a 32 bit system. If you're on a 64
bit system the numbers get even larger."

Taken from http://search.cpan.org/~dsugal/Devel-Size-0.64/Size.pm

so, if I'm reading this correctly:

$hash{".1.2.3.4"} --> approx 22 to 29 bytes (sorry not sure how many
alignment bytes would be needed)

vs.

$hash{1}{2}{3}{4} --> approx 16 * 4 or 64 bytes

So, I guess flattening doesn't have the memory trade-off I was worried
about since long hash keys have less impact on memory allocation than
splitting it into nested single char keys.  If I'm reading this wrong,
someone please correct me.

-----Original Message-----
From: phoenix-pm-bounces+bwmetz=att.com at pm.org
[mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Metz,
Bobby W, WWCS
Sent: Wednesday, June 07, 2006 1:03 PM
To: Brock
Cc: phoenix-pm at pm.org
Subject: Re: [Phoenix-pm] Multi-level hash - quick key reference?

Brock,
	May have to give yours a go after all...just dawned on me that
flattening the index will significantly increase storage due to the
duplication of index characters.  Some could have several thousand
values so if it were say six keys deep times a thousand values, it would
add up over time.

Bobby

-----Original Message-----
From: phoenix-pm-bounces+bwmetz=att.com at pm.org
[mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Metz,
Bobby W, WWCS
Sent: Wednesday, June 07, 2006 1:00 PM
To: Brock
Cc: phoenix-pm at pm.org
Subject: Re: [Phoenix-pm] Multi-level hash - quick key reference?

Thanks Brock.  
	After sending that message I thought about flattening it as
well...a big resounding "Duh" escaped my lips I must admit.  I
considered eval but quickly squashed that idea.  I should have thought
of a shift method...so thanks, I like that.  I think I need to study the
data source some more as flattening will likely be the simplest method
since each index will have at most two values.  I oversimplified a bit
on the example.

$hash{.1.2.3.4}{name} & $hash{.1.2.3.4}{value}

And since the names will never change, I'll likely just have one hash
for those and an array of hashes for the values.

Thanks for the quick response...very much appreciated.

Bobby

-----Original Message-----
From: Brock [mailto:awwaiid at thelackthereof.org]
Sent: Wednesday, June 07, 2006 12:52 PM
To: Metz, Bobby W, WWCS
Cc: phoenix-pm at pm.org
Subject: Re: [Phoenix-pm] Multi-level hash - quick key reference?

Heres a couple ideas. One, you could just flatten your hash -- use

  $hash->{1.2.3} = "Hello World";
  $hash->{1.2.3.4.5} = "the quick brown fox";

but, failing that,

use strict;
my $hash = {};
my $str = ".1.2.3\tHello World";
my ($nums,$text) = split /\t/,$str;
my @indexes = split /\./, $nums;
shift @indexes; # Get rid of the first ""
my $cur = $hash;
while(@indexes) {
  my $index = shift @indexes;
  if(!@indexes) { # this is the last one
    $cur->{$index} = $text;
  } else {
    # We might need to vivify
    $cur->{$index} = {} unless defined $cur->{$index};
    $cur = $cur->{$index};
  }
}
use Data::Dumper;
print "Result: " . Dumper($hash) . "\n";

might help. Simplification, anyone? I suppose you could use evil, er,
eval.

--Brock

On 2006.06.07.14.31, Metz, Bobby W, WWCS wrote:
| All,
| 	I'm looking for a quicker method of accessing elements in a
multi-level hash.  For example, the hash will have values like so
| 
| $hash{1}{2}{3} = "Hello World";
| $hash{1}{3}{4}{5} = "the quick brown fox";
| 
| The input to the program comes from another I can't control which will
ouput values like so:
| 
| .1.2.3<tab>Hello World
| .1.3.4.5<tab>the quick brown fox
| 
| Parsing the values is cake, but what I'm looking for is a faster way
to get values in/out of the hash using the ".1.2.3".  Is there some
method that allows for arbitrary hash levels without coding using for
loops?  I mean clearly recursion would solve this but I was hoping there
was something more simple.  Or is recursion the only answer.  I was
hoping for something like this, but I know this example doesn't work.
I've searched for a bit but I must be asking the right questions of
mighty google.
| 
| $index = ".1.2.3";
| $index =~ s/^\.//g;
| $index =~ s/\./}{/g;
| print $hash{$index} . "\n";
| 
| Thoughts?
| 
| Thanks,
| 
| Bobby
| _______________________________________________
| Phoenix-pm mailing list
| Phoenix-pm at pm.org
| http://mail.pm.org/mailman/listinfo/phoenix-pm
_______________________________________________
Phoenix-pm mailing list
Phoenix-pm at pm.org
http://mail.pm.org/mailman/listinfo/phoenix-pm
_______________________________________________
Phoenix-pm mailing list
Phoenix-pm at pm.org
http://mail.pm.org/mailman/listinfo/phoenix-pm