From me at heyjay.com Thu Mar 1 10:40:27 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 12:40:27 -0600 Subject: [Chicago-talk] creating relational keys Message-ID: Hi all, I have some data like: CITY|STATE|ZIP SCHAUMBURG|IL|60194 MATTESON|IL|60443 WARRENTON|OR|97146 MOUNTAIN HOME|AR|72653 FORT WORTH|TX|76107 CLEVELAND|MS|38732 WATERTOWN|SD|57201 GRAND CHUTE|WI|54913 I want to load it into a relational database in such a way that I have the proper keys to build a hierarchy. so for example: ID Name ParentID 0 USA 1 IL 0 2 SCHAUMBURG 1 3 60194 2 4 MATTESON 1 5 60443 4 6 OR 0 7 WARRENTON 6 8 97146 7 ... I'm not sure of a good way to do this. I read the data an built a hash like: USA => {IL => {SCHAUMBURG => {60194 => 0}, MATTESON => {60443 => 0}}, OR => {WARRENTON => {97146 => 0}} ... }; I can't think of a good way to look through it and assign the keys. Maybe someone has done this in the past and has an elegant solution? Thanks Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From imranjj at gmail.com Thu Mar 1 10:51:15 2012 From: imranjj at gmail.com (imran javaid) Date: Thu, 1 Mar 2012 12:51:15 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: One option would be use a tree data structure. Take a look at Tree::Simple. You will have USA in the first level, states in the second, cities in the third, and zip codes in the 4th (and then whatever else below that). -imran On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: > Hi all, > > I have some data like: > CITY|STATE|ZIP > SCHAUMBURG|IL|60194 > MATTESON|IL|60443 > WARRENTON|OR|97146 > MOUNTAIN HOME|AR|72653 > FORT WORTH|TX|76107 > CLEVELAND|MS|38732 > WATERTOWN|SD|57201 > GRAND CHUTE|WI|54913 > > I want to load it into a relational database in such a way that I have the > proper keys to build a hierarchy. > > so for example: > ID Name ParentID > 0 USA > 1 IL 0 > 2 SCHAUMBURG 1 > 3 60194 2 > 4 MATTESON 1 > 5 60443 4 > 6 OR 0 > 7 WARRENTON 6 > 8 97146 7 > ... > > I'm not sure of a good way to do this. > > I read the data an built a hash like: > USA => {IL => {SCHAUMBURG => {60194 => 0}, > MATTESON => {60443 => 0}}, > OR => {WARRENTON => {97146 => 0}} > ... > }; > > I can't think of a good way to look through it and assign the keys. > > Maybe someone has done this in the past and has an elegant solution? > > Thanks > Jay > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at heyjay.com Thu Mar 1 10:58:19 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 12:58:19 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Imran, Thanks. I'm ok with building a tree, currently I'm using a hash. I just don't know how to unwind the tree and assign the proper ID and parent ID. Jay On Thu, Mar 1, 2012 at 12:51 PM, imran javaid wrote: > One option would be use a tree data structure. Take a look at > Tree::Simple. You will have USA in the first level, states in the second, > cities in the third, and zip codes in the 4th (and then whatever else below > that). > > -imran > > On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: > >> Hi all, >> >> I have some data like: >> CITY|STATE|ZIP >> SCHAUMBURG|IL|60194 >> MATTESON|IL|60443 >> WARRENTON|OR|97146 >> MOUNTAIN HOME|AR|72653 >> FORT WORTH|TX|76107 >> CLEVELAND|MS|38732 >> WATERTOWN|SD|57201 >> GRAND CHUTE|WI|54913 >> >> I want to load it into a relational database in such a way that I have >> the proper keys to build a hierarchy. >> >> so for example: >> ID Name ParentID >> 0 USA >> 1 IL 0 >> 2 SCHAUMBURG 1 >> 3 60194 2 >> 4 MATTESON 1 >> 5 60443 4 >> 6 OR 0 >> 7 WARRENTON 6 >> 8 97146 7 >> ... >> >> I'm not sure of a good way to do this. >> >> I read the data an built a hash like: >> USA => {IL => {SCHAUMBURG => {60194 => 0}, >> MATTESON => {60443 => 0}}, >> OR => {WARRENTON => {97146 => 0}} >> ... >> }; >> >> I can't think of a good way to look through it and assign the keys. >> >> Maybe someone has done this in the past and has an elegant solution? >> >> Thanks >> Jay >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjhamil at gmail.com Thu Mar 1 11:11:46 2012 From: cjhamil at gmail.com (Chris Hamilton) Date: Thu, 1 Mar 2012 13:11:46 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Just out of curiosity, is there a strong need to put all of this data into the same table? At least with your example data it seems like the hierarchy is better managed through separate tables for city, state, and zip (with a 1:many relationship from state to city, and then from city to zip). In that form it would be easy enough to create the ID's and tie them up as you iterate through your hash keys. Just a thought. -Chris On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: > Imran, > > Thanks. ?I'm ok with building a tree, currently I'm using a hash. I just > don't know how to unwind the tree and assign the proper ID and parent ID. > > Jay > > > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid wrote: >> >> One option would be use a tree data structure. Take a look at >> Tree::Simple. You will have USA in the first level, states in the second, >> cities in the third, and zip codes in the 4th (and then whatever else below >> that). >> >> -imran >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >>> >>> Hi all, >>> >>> I have some data like: >>> CITY|STATE|ZIP >>> SCHAUMBURG|IL|60194 >>> MATTESON|IL|60443 >>> WARRENTON|OR|97146 >>> MOUNTAIN HOME|AR|72653 >>> FORT WORTH|TX|76107 >>> CLEVELAND|MS|38732 >>> WATERTOWN|SD|57201 >>> GRAND CHUTE|WI|54913 >>> >>> I want to load it into a relational database in such a way that I have >>> the proper keys to build a hierarchy. >>> >>> so for example: >>> ID Name ParentID >>> 0 USA >>> 1 IL 0 >>> 2 SCHAUMBURG 1 >>> 3 60194 2 >>> 4 MATTESON 1 >>> 5 60443 4 >>> 6 OR 0 >>> 7 WARRENTON 6 >>> 8 97146 7 >>> ... >>> >>> I'm not sure of a good way to do this. >>> >>> I read the data an built a hash like: >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >>> ? ? ? MATTESON ? => {60443 => 0}}, >>> OR => {WARRENTON ?=> {97146 => 0}} >>> ... >>> }; >>> >>> I can't think of a good way to look through it and assign the keys. >>> >>> Maybe someone has done this in the past and has an elegant solution? >>> >>> Thanks >>> Jay >>> >>> _______________________________________________ >>> Chicago-talk mailing list >>> Chicago-talk at pm.org >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk > > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk From me at heyjay.com Thu Mar 1 11:14:03 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 13:14:03 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Hi, yes (unfortunately) it all has to go in the same table. I'm using a vendor supplied data model, with a generic structure like; ID NAME ParentID Although, even if I could break it up into separate tables I'd still need the keys On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton wrote: > Just out of curiosity, is there a strong need to put all of this data > into the same table? At least with your example data it seems like > the hierarchy is better managed through separate tables for city, > state, and zip (with a 1:many relationship from state to city, and > then from city to zip). In that form it would be easy enough to > create the ID's and tie them up as you iterate through your hash keys. > Just a thought. > > -Chris > > On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: > > Imran, > > > > Thanks. I'm ok with building a tree, currently I'm using a hash. I just > > don't know how to unwind the tree and assign the proper ID and parent ID. > > > > Jay > > > > > > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid wrote: > >> > >> One option would be use a tree data structure. Take a look at > >> Tree::Simple. You will have USA in the first level, states in the > second, > >> cities in the third, and zip codes in the 4th (and then whatever else > below > >> that). > >> > >> -imran > >> > >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: > >>> > >>> Hi all, > >>> > >>> I have some data like: > >>> CITY|STATE|ZIP > >>> SCHAUMBURG|IL|60194 > >>> MATTESON|IL|60443 > >>> WARRENTON|OR|97146 > >>> MOUNTAIN HOME|AR|72653 > >>> FORT WORTH|TX|76107 > >>> CLEVELAND|MS|38732 > >>> WATERTOWN|SD|57201 > >>> GRAND CHUTE|WI|54913 > >>> > >>> I want to load it into a relational database in such a way that I have > >>> the proper keys to build a hierarchy. > >>> > >>> so for example: > >>> ID Name ParentID > >>> 0 USA > >>> 1 IL 0 > >>> 2 SCHAUMBURG 1 > >>> 3 60194 2 > >>> 4 MATTESON 1 > >>> 5 60443 4 > >>> 6 OR 0 > >>> 7 WARRENTON 6 > >>> 8 97146 7 > >>> ... > >>> > >>> I'm not sure of a good way to do this. > >>> > >>> I read the data an built a hash like: > >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, > >>> MATTESON => {60443 => 0}}, > >>> OR => {WARRENTON => {97146 => 0}} > >>> ... > >>> }; > >>> > >>> I can't think of a good way to look through it and assign the keys. > >>> > >>> Maybe someone has done this in the past and has an elegant solution? > >>> > >>> Thanks > >>> Jay > >>> > >>> _______________________________________________ > >>> Chicago-talk mailing list > >>> Chicago-talk at pm.org > >>> http://mail.pm.org/mailman/listinfo/chicago-talk > >> > >> > >> > >> _______________________________________________ > >> Chicago-talk mailing list > >> Chicago-talk at pm.org > >> http://mail.pm.org/mailman/listinfo/chicago-talk > > > > > > > > _______________________________________________ > > Chicago-talk mailing list > > Chicago-talk at pm.org > > http://mail.pm.org/mailman/listinfo/chicago-talk > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjhamil at gmail.com Thu Mar 1 11:57:11 2012 From: cjhamil at gmail.com (Chris Hamilton) Date: Thu, 1 Mar 2012 13:57:11 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Assuming that all keys at a given depth of the hash structure are unique (probably a big assumption with city names in a state), the attached script example should probably do what you need (assuming I'm correctly understanding your problem and you're trying to figure out how to iterate through the hash structure and do the necessary inserts with the correct parentIDs). If I'm wrong about which problem you're having I apologize, but hopefully this provides some insight. Thanks, -Chris On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: > Hi, yes (unfortunately) it all has to go in the same table. > > I'm using a vendor supplied data model, with a generic structure like; > > ID NAME ParentID > > Although, even if I could break it up into separate tables I'd still need > the keys > > > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton wrote: >> >> Just out of curiosity, is there a strong need to put all of this data >> into the same table? ?At least with your example data it seems like >> the hierarchy is better managed through separate tables for city, >> state, and zip (with a 1:many relationship from state to city, and >> then from city to zip). ?In that form it would be easy enough to >> create the ID's and tie them up as you iterate through your hash keys. >> ?Just a thought. >> >> -Chris >> >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >> > Imran, >> > >> > Thanks. ?I'm ok with building a tree, currently I'm using a hash. I just >> > don't know how to unwind the tree and assign the proper ID and parent >> > ID. >> > >> > Jay >> > >> > >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid wrote: >> >> >> >> One option would be use a tree data structure. Take a look at >> >> Tree::Simple. You will have USA in the first level, states in the >> >> second, >> >> cities in the third, and zip codes in the 4th (and then whatever else >> >> below >> >> that). >> >> >> >> -imran >> >> >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >> >>> >> >>> Hi all, >> >>> >> >>> I have some data like: >> >>> CITY|STATE|ZIP >> >>> SCHAUMBURG|IL|60194 >> >>> MATTESON|IL|60443 >> >>> WARRENTON|OR|97146 >> >>> MOUNTAIN HOME|AR|72653 >> >>> FORT WORTH|TX|76107 >> >>> CLEVELAND|MS|38732 >> >>> WATERTOWN|SD|57201 >> >>> GRAND CHUTE|WI|54913 >> >>> >> >>> I want to load it into a relational database in such a way that I have >> >>> the proper keys to build a hierarchy. >> >>> >> >>> so for example: >> >>> ID Name ParentID >> >>> 0 USA >> >>> 1 IL 0 >> >>> 2 SCHAUMBURG 1 >> >>> 3 60194 2 >> >>> 4 MATTESON 1 >> >>> 5 60443 4 >> >>> 6 OR 0 >> >>> 7 WARRENTON 6 >> >>> 8 97146 7 >> >>> ... >> >>> >> >>> I'm not sure of a good way to do this. >> >>> >> >>> I read the data an built a hash like: >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >> >>> ? ? ? MATTESON ? => {60443 => 0}}, >> >>> OR => {WARRENTON ?=> {97146 => 0}} >> >>> ... >> >>> }; >> >>> >> >>> I can't think of a good way to look through it and assign the keys. >> >>> >> >>> Maybe someone has done this in the past and has an elegant solution? >> >>> >> >>> Thanks >> >>> Jay >> >>> >> >>> _______________________________________________ >> >>> Chicago-talk mailing list >> >>> Chicago-talk at pm.org >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> >> >> >> >> _______________________________________________ >> >> Chicago-talk mailing list >> >> Chicago-talk at pm.org >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > >> > >> > >> > _______________________________________________ >> > Chicago-talk mailing list >> > Chicago-talk at pm.org >> > http://mail.pm.org/mailman/listinfo/chicago-talk >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk > > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk -------------- next part -------------- A non-text attachment was scrubbed... Name: testload.pl Type: application/octet-stream Size: 1238 bytes Desc: not available URL: From me at heyjay.com Thu Mar 1 12:01:13 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 14:01:13 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: This seems to work: This seems to work: > perl create_hier.pl small.txt 11 10 12 use strict; use Data::Dumper; my %h; $h{ROOT} = {}; my @i = splice @ARGV, 1, $#ARGV; # remove the indexes from command line while (<>) { my @f = split(/\|/); my $p = $h{ROOT}; foreach my $i (@i) { my $key = $f[$i]; if (! exists $p->{$key}) { $p->{$key} = {}; } $p = $p->{$key}; } } print Dumper %h; my $counter = 0; unwrap($counter, \%h); sub unwrap { my ($parent, $hash) = @_; foreach my $key (keys %$hash) { $counter++; print join("\t", $counter, $parent, $key), "\n"; unwrap( $counter, $hash->{$key}); } } Giving me output like (I shortened it due to the size): $VAR1 = 'ROOT'; $VAR2 = { 'LA' => { 'WALKER' => { '70785' => {} }, 'BOUTTE' => { '70039' => {} }, 'RAYNE' => { '70578' => {} }, 'BATON ROUGE' => { '70805' => {} } }, .... 'NV' => { 'LAS VEGAS' => { '89149' => {}, '89104' => {} } }, 'OR' => { 'WARRENTON' => { '97146' => {} }, 'SALEM' => { '97305' => {}, '97302' => {}, '97301' => {} } } }; 1 0 ROOT 2 1 LA 3 2 WALKER 4 3 70785 5 2 BOUTTE 6 5 70039 7 2 RAYNE 8 7 70578 9 2 BATON ROUGE 10 9 70805 ... 219 1 NV 220 219 LAS VEGAS 221 220 89149 222 220 89104 223 1 OR 224 223 WARRENTON 225 224 97146 226 223 SALEM 227 226 97305 228 226 97302 229 226 97301 -------------- next part -------------- An HTML attachment was scrubbed... URL: From imranjj at gmail.com Thu Mar 1 12:04:09 2012 From: imranjj at gmail.com (imran javaid) Date: Thu, 1 Mar 2012 14:04:09 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Using a Tree is likely overkill, but you can build one like this: my $tree = Tree::Simple->new({id=>0,name=>'USA'}, TreeSimple->ROOT); # assume @data is your table sorted by ParentID foreach my $row (@data) { ($id, $name, $pid) = @$row; $tree->traverse( sub { my ($_tree) = @_; my $nodeData = $_tree->getNodeValue(); if ($nodeData->{id} == $pid) { $_tree->addChild(Tree::Simple->new({id=>$id, name=>$name}); } }); } You could define a hash with level information: my %levels = (-1 => "Country", 0=> "State", 1=> "City", 2=> "Zip"); so in a traverse routine you can tell which level you are on: print $level{$_tree->getDepth()}; -imran On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: > Hi, yes (unfortunately) it all has to go in the same table. > > I'm using a vendor supplied data model, with a generic structure like; > > ID NAME ParentID > > Although, even if I could break it up into separate tables I'd still need > the keys > > > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton wrote: > >> Just out of curiosity, is there a strong need to put all of this data >> into the same table? At least with your example data it seems like >> the hierarchy is better managed through separate tables for city, >> state, and zip (with a 1:many relationship from state to city, and >> then from city to zip). In that form it would be easy enough to >> create the ID's and tie them up as you iterate through your hash keys. >> Just a thought. >> >> -Chris >> >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >> > Imran, >> > >> > Thanks. I'm ok with building a tree, currently I'm using a hash. I just >> > don't know how to unwind the tree and assign the proper ID and parent >> ID. >> > >> > Jay >> > >> > >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid >> wrote: >> >> >> >> One option would be use a tree data structure. Take a look at >> >> Tree::Simple. You will have USA in the first level, states in the >> second, >> >> cities in the third, and zip codes in the 4th (and then whatever else >> below >> >> that). >> >> >> >> -imran >> >> >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >> >>> >> >>> Hi all, >> >>> >> >>> I have some data like: >> >>> CITY|STATE|ZIP >> >>> SCHAUMBURG|IL|60194 >> >>> MATTESON|IL|60443 >> >>> WARRENTON|OR|97146 >> >>> MOUNTAIN HOME|AR|72653 >> >>> FORT WORTH|TX|76107 >> >>> CLEVELAND|MS|38732 >> >>> WATERTOWN|SD|57201 >> >>> GRAND CHUTE|WI|54913 >> >>> >> >>> I want to load it into a relational database in such a way that I have >> >>> the proper keys to build a hierarchy. >> >>> >> >>> so for example: >> >>> ID Name ParentID >> >>> 0 USA >> >>> 1 IL 0 >> >>> 2 SCHAUMBURG 1 >> >>> 3 60194 2 >> >>> 4 MATTESON 1 >> >>> 5 60443 4 >> >>> 6 OR 0 >> >>> 7 WARRENTON 6 >> >>> 8 97146 7 >> >>> ... >> >>> >> >>> I'm not sure of a good way to do this. >> >>> >> >>> I read the data an built a hash like: >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >> >>> MATTESON => {60443 => 0}}, >> >>> OR => {WARRENTON => {97146 => 0}} >> >>> ... >> >>> }; >> >>> >> >>> I can't think of a good way to look through it and assign the keys. >> >>> >> >>> Maybe someone has done this in the past and has an elegant solution? >> >>> >> >>> Thanks >> >>> Jay >> >>> >> >>> _______________________________________________ >> >>> Chicago-talk mailing list >> >>> Chicago-talk at pm.org >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> >> >> >> >> _______________________________________________ >> >> Chicago-talk mailing list >> >> Chicago-talk at pm.org >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > >> > >> > >> > _______________________________________________ >> > Chicago-talk mailing list >> > Chicago-talk at pm.org >> > http://mail.pm.org/mailman/listinfo/chicago-talk >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at heyjay.com Thu Mar 1 12:06:24 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 14:06:24 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: thanks Chris ! We solved it essentially the same way it looks like. Jay On Thu, Mar 1, 2012 at 1:57 PM, Chris Hamilton wrote: > Assuming that all keys at a given depth of the hash structure are > unique (probably a big assumption with city names in a state), the > attached script example should probably do what you need (assuming I'm > correctly understanding your problem and you're trying to figure out > how to iterate through the hash structure and do the necessary inserts > with the correct parentIDs). If I'm wrong about which problem you're > having I apologize, but hopefully this provides some insight. > > Thanks, > -Chris > > On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: > > Hi, yes (unfortunately) it all has to go in the same table. > > > > I'm using a vendor supplied data model, with a generic structure like; > > > > ID NAME ParentID > > > > Although, even if I could break it up into separate tables I'd still need > > the keys > > > > > > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton > wrote: > >> > >> Just out of curiosity, is there a strong need to put all of this data > >> into the same table? At least with your example data it seems like > >> the hierarchy is better managed through separate tables for city, > >> state, and zip (with a 1:many relationship from state to city, and > >> then from city to zip). In that form it would be easy enough to > >> create the ID's and tie them up as you iterate through your hash keys. > >> Just a thought. > >> > >> -Chris > >> > >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: > >> > Imran, > >> > > >> > Thanks. I'm ok with building a tree, currently I'm using a hash. I > just > >> > don't know how to unwind the tree and assign the proper ID and parent > >> > ID. > >> > > >> > Jay > >> > > >> > > >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid > wrote: > >> >> > >> >> One option would be use a tree data structure. Take a look at > >> >> Tree::Simple. You will have USA in the first level, states in the > >> >> second, > >> >> cities in the third, and zip codes in the 4th (and then whatever else > >> >> below > >> >> that). > >> >> > >> >> -imran > >> >> > >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: > >> >>> > >> >>> Hi all, > >> >>> > >> >>> I have some data like: > >> >>> CITY|STATE|ZIP > >> >>> SCHAUMBURG|IL|60194 > >> >>> MATTESON|IL|60443 > >> >>> WARRENTON|OR|97146 > >> >>> MOUNTAIN HOME|AR|72653 > >> >>> FORT WORTH|TX|76107 > >> >>> CLEVELAND|MS|38732 > >> >>> WATERTOWN|SD|57201 > >> >>> GRAND CHUTE|WI|54913 > >> >>> > >> >>> I want to load it into a relational database in such a way that I > have > >> >>> the proper keys to build a hierarchy. > >> >>> > >> >>> so for example: > >> >>> ID Name ParentID > >> >>> 0 USA > >> >>> 1 IL 0 > >> >>> 2 SCHAUMBURG 1 > >> >>> 3 60194 2 > >> >>> 4 MATTESON 1 > >> >>> 5 60443 4 > >> >>> 6 OR 0 > >> >>> 7 WARRENTON 6 > >> >>> 8 97146 7 > >> >>> ... > >> >>> > >> >>> I'm not sure of a good way to do this. > >> >>> > >> >>> I read the data an built a hash like: > >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, > >> >>> MATTESON => {60443 => 0}}, > >> >>> OR => {WARRENTON => {97146 => 0}} > >> >>> ... > >> >>> }; > >> >>> > >> >>> I can't think of a good way to look through it and assign the keys. > >> >>> > >> >>> Maybe someone has done this in the past and has an elegant solution? > >> >>> > >> >>> Thanks > >> >>> Jay > >> >>> > >> >>> _______________________________________________ > >> >>> Chicago-talk mailing list > >> >>> Chicago-talk at pm.org > >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk > >> >> > >> >> > >> >> > >> >> _______________________________________________ > >> >> Chicago-talk mailing list > >> >> Chicago-talk at pm.org > >> >> http://mail.pm.org/mailman/listinfo/chicago-talk > >> > > >> > > >> > > >> > _______________________________________________ > >> > Chicago-talk mailing list > >> > Chicago-talk at pm.org > >> > http://mail.pm.org/mailman/listinfo/chicago-talk > >> _______________________________________________ > >> Chicago-talk mailing list > >> Chicago-talk at pm.org > >> http://mail.pm.org/mailman/listinfo/chicago-talk > > > > > > > > _______________________________________________ > > Chicago-talk mailing list > > Chicago-talk at pm.org > > http://mail.pm.org/mailman/listinfo/chicago-talk > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at heyjay.com Thu Mar 1 12:07:35 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 14:07:35 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Thanks Imran. I might play with Tree::Simple going forward Jay On Thu, Mar 1, 2012 at 2:04 PM, imran javaid wrote: > Using a Tree is likely overkill, but you can build one like this: > > my $tree = Tree::Simple->new({id=>0,name=>'USA'}, TreeSimple->ROOT); > > # assume @data is your table sorted by ParentID > foreach my $row (@data) { > ($id, $name, $pid) = @$row; > $tree->traverse( sub { > my ($_tree) = @_; > my $nodeData = $_tree->getNodeValue(); > if ($nodeData->{id} == $pid) { > $_tree->addChild(Tree::Simple->new({id=>$id, name=>$name}); > } > }); > } > > You could define a hash with level information: > my %levels = (-1 => "Country", 0=> "State", 1=> "City", 2=> "Zip"); > so in a traverse routine you can tell which level you are on: > print $level{$_tree->getDepth()}; > > -imran > > > On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: > >> Hi, yes (unfortunately) it all has to go in the same table. >> >> I'm using a vendor supplied data model, with a generic structure like; >> >> ID NAME ParentID >> >> Although, even if I could break it up into separate tables I'd still need >> the keys >> >> >> On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton wrote: >> >>> Just out of curiosity, is there a strong need to put all of this data >>> into the same table? At least with your example data it seems like >>> the hierarchy is better managed through separate tables for city, >>> state, and zip (with a 1:many relationship from state to city, and >>> then from city to zip). In that form it would be easy enough to >>> create the ID's and tie them up as you iterate through your hash keys. >>> Just a thought. >>> >>> -Chris >>> >>> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >>> > Imran, >>> > >>> > Thanks. I'm ok with building a tree, currently I'm using a hash. I >>> just >>> > don't know how to unwind the tree and assign the proper ID and parent >>> ID. >>> > >>> > Jay >>> > >>> > >>> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid >>> wrote: >>> >> >>> >> One option would be use a tree data structure. Take a look at >>> >> Tree::Simple. You will have USA in the first level, states in the >>> second, >>> >> cities in the third, and zip codes in the 4th (and then whatever else >>> below >>> >> that). >>> >> >>> >> -imran >>> >> >>> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >>> >>> >>> >>> Hi all, >>> >>> >>> >>> I have some data like: >>> >>> CITY|STATE|ZIP >>> >>> SCHAUMBURG|IL|60194 >>> >>> MATTESON|IL|60443 >>> >>> WARRENTON|OR|97146 >>> >>> MOUNTAIN HOME|AR|72653 >>> >>> FORT WORTH|TX|76107 >>> >>> CLEVELAND|MS|38732 >>> >>> WATERTOWN|SD|57201 >>> >>> GRAND CHUTE|WI|54913 >>> >>> >>> >>> I want to load it into a relational database in such a way that I >>> have >>> >>> the proper keys to build a hierarchy. >>> >>> >>> >>> so for example: >>> >>> ID Name ParentID >>> >>> 0 USA >>> >>> 1 IL 0 >>> >>> 2 SCHAUMBURG 1 >>> >>> 3 60194 2 >>> >>> 4 MATTESON 1 >>> >>> 5 60443 4 >>> >>> 6 OR 0 >>> >>> 7 WARRENTON 6 >>> >>> 8 97146 7 >>> >>> ... >>> >>> >>> >>> I'm not sure of a good way to do this. >>> >>> >>> >>> I read the data an built a hash like: >>> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >>> >>> MATTESON => {60443 => 0}}, >>> >>> OR => {WARRENTON => {97146 => 0}} >>> >>> ... >>> >>> }; >>> >>> >>> >>> I can't think of a good way to look through it and assign the keys. >>> >>> >>> >>> Maybe someone has done this in the past and has an elegant solution? >>> >>> >>> >>> Thanks >>> >>> Jay >>> >>> >>> >>> _______________________________________________ >>> >>> Chicago-talk mailing list >>> >>> Chicago-talk at pm.org >>> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> Chicago-talk mailing list >>> >> Chicago-talk at pm.org >>> >> http://mail.pm.org/mailman/listinfo/chicago-talk >>> > >>> > >>> > >>> > _______________________________________________ >>> > Chicago-talk mailing list >>> > Chicago-talk at pm.org >>> > http://mail.pm.org/mailman/listinfo/chicago-talk >>> _______________________________________________ >>> Chicago-talk mailing list >>> Chicago-talk at pm.org >>> http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjhamil at gmail.com Thu Mar 1 12:08:49 2012 From: cjhamil at gmail.com (Chris Hamilton) Date: Thu, 1 Mar 2012 14:08:49 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: Yeah, looks that way. :) It's too bad you're forced to use such a model, it's not going to make for very lovely queries, I don't imagine. Good luck! -Chris On Thu, Mar 1, 2012 at 2:06 PM, Jay Strauss wrote: > thanks Chris ! > > We solved it essentially the same way it looks like. > > Jay > > > On Thu, Mar 1, 2012 at 1:57 PM, Chris Hamilton wrote: >> >> Assuming that all keys at a given depth of the hash structure are >> unique (probably a big assumption with city names in a state), the >> attached script example should probably do what you need (assuming I'm >> correctly understanding your problem and you're trying to figure out >> how to iterate through the hash structure and do the necessary inserts >> with the correct parentIDs). ?If I'm wrong about which problem you're >> having I apologize, but hopefully this provides some insight. >> >> Thanks, >> -Chris >> >> On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: >> > Hi, yes (unfortunately) it all has to go in the same table. >> > >> > I'm using a vendor supplied data model, with a generic structure like; >> > >> > ID NAME ParentID >> > >> > Although, even if I could break it up into separate tables I'd still >> > need >> > the keys >> > >> > >> > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton >> > wrote: >> >> >> >> Just out of curiosity, is there a strong need to put all of this data >> >> into the same table? ?At least with your example data it seems like >> >> the hierarchy is better managed through separate tables for city, >> >> state, and zip (with a 1:many relationship from state to city, and >> >> then from city to zip). ?In that form it would be easy enough to >> >> create the ID's and tie them up as you iterate through your hash keys. >> >> ?Just a thought. >> >> >> >> -Chris >> >> >> >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >> >> > Imran, >> >> > >> >> > Thanks. ?I'm ok with building a tree, currently I'm using a hash. I >> >> > just >> >> > don't know how to unwind the tree and assign the proper ID and parent >> >> > ID. >> >> > >> >> > Jay >> >> > >> >> > >> >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid >> >> > wrote: >> >> >> >> >> >> One option would be use a tree data structure. Take a look at >> >> >> Tree::Simple. You will have USA in the first level, states in the >> >> >> second, >> >> >> cities in the third, and zip codes in the 4th (and then whatever >> >> >> else >> >> >> below >> >> >> that). >> >> >> >> >> >> -imran >> >> >> >> >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> I have some data like: >> >> >>> CITY|STATE|ZIP >> >> >>> SCHAUMBURG|IL|60194 >> >> >>> MATTESON|IL|60443 >> >> >>> WARRENTON|OR|97146 >> >> >>> MOUNTAIN HOME|AR|72653 >> >> >>> FORT WORTH|TX|76107 >> >> >>> CLEVELAND|MS|38732 >> >> >>> WATERTOWN|SD|57201 >> >> >>> GRAND CHUTE|WI|54913 >> >> >>> >> >> >>> I want to load it into a relational database in such a way that I >> >> >>> have >> >> >>> the proper keys to build a hierarchy. >> >> >>> >> >> >>> so for example: >> >> >>> ID Name ParentID >> >> >>> 0 USA >> >> >>> 1 IL 0 >> >> >>> 2 SCHAUMBURG 1 >> >> >>> 3 60194 2 >> >> >>> 4 MATTESON 1 >> >> >>> 5 60443 4 >> >> >>> 6 OR 0 >> >> >>> 7 WARRENTON 6 >> >> >>> 8 97146 7 >> >> >>> ... >> >> >>> >> >> >>> I'm not sure of a good way to do this. >> >> >>> >> >> >>> I read the data an built a hash like: >> >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >> >> >>> ? ? ? MATTESON ? => {60443 => 0}}, >> >> >>> OR => {WARRENTON ?=> {97146 => 0}} >> >> >>> ... >> >> >>> }; >> >> >>> >> >> >>> I can't think of a good way to look through it and assign the keys. >> >> >>> >> >> >>> Maybe someone has done this in the past and has an elegant >> >> >>> solution? >> >> >>> >> >> >>> Thanks >> >> >>> Jay >> >> >>> >> >> >>> _______________________________________________ >> >> >>> Chicago-talk mailing list >> >> >>> Chicago-talk at pm.org >> >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> Chicago-talk mailing list >> >> >> Chicago-talk at pm.org >> >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> > >> >> > >> >> > >> >> > _______________________________________________ >> >> > Chicago-talk mailing list >> >> > Chicago-talk at pm.org >> >> > http://mail.pm.org/mailman/listinfo/chicago-talk >> >> _______________________________________________ >> >> Chicago-talk mailing list >> >> Chicago-talk at pm.org >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > >> > >> > >> > _______________________________________________ >> > Chicago-talk mailing list >> > Chicago-talk at pm.org >> > http://mail.pm.org/mailman/listinfo/chicago-talk >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk > > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk From tigerpeng2001 at yahoo.com Thu Mar 1 15:38:24 2012 From: tigerpeng2001 at yahoo.com (tiger peng) Date: Thu, 1 Mar 2012 15:38:24 -0800 (PST) Subject: [Chicago-talk] creating relational keys In-Reply-To: References: Message-ID: <1330645104.55966.YahooMailNeo@web120506.mail.ne1.yahoo.com> Do you have to use Perl? Let relational keys done by relational database may be easier. BTW, As I know, in US, some zip codes may cross cities and even state boarder. It cannot be simply represented in tree structure nor self-referenced table. ________________________________ From: Chris Hamilton To: Chicago.pm chatter Sent: Thursday, March 1, 2012 2:08 PM Subject: Re: [Chicago-talk] creating relational keys Yeah, looks that way. :) It's too bad you're forced to use such a model, it's not going to make for very lovely queries, I don't imagine.? Good luck! -Chris On Thu, Mar 1, 2012 at 2:06 PM, Jay Strauss wrote: > thanks Chris ! > > We solved it essentially the same way it looks like. > > Jay > > > On Thu, Mar 1, 2012 at 1:57 PM, Chris Hamilton wrote: >> >> Assuming that all keys at a given depth of the hash structure are >> unique (probably a big assumption with city names in a state), the >> attached script example should probably do what you need (assuming I'm >> correctly understanding your problem and you're trying to figure out >> how to iterate through the hash structure and do the necessary inserts >> with the correct parentIDs). ?If I'm wrong about which problem you're >> having I apologize, but hopefully this provides some insight. >> >> Thanks, >> -Chris >> >> On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: >> > Hi, yes (unfortunately) it all has to go in the same table. >> > >> > I'm using a vendor supplied data model, with a generic structure like; >> > >> > ID NAME ParentID >> > >> > Although, even if I could break it up into separate tables I'd still >> > need >> > the keys >> > >> > >> > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton >> > wrote: >> >> >> >> Just out of curiosity, is there a strong need to put all of this data >> >> into the same table? ?At least with your example data it seems like >> >> the hierarchy is better managed through separate tables for city, >> >> state, and zip (with a 1:many relationship from state to city, and >> >> then from city to zip). ?In that form it would be easy enough to >> >> create the ID's and tie them up as you iterate through your hash keys. >> >> ?Just a thought. >> >> >> >> -Chris >> >> >> >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >> >> > Imran, >> >> > >> >> > Thanks. ?I'm ok with building a tree, currently I'm using a hash. I >> >> > just >> >> > don't know how to unwind the tree and assign the proper ID and parent >> >> > ID. >> >> > >> >> > Jay >> >> > >> >> > >> >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid >> >> > wrote: >> >> >> >> >> >> One option would be use a tree data structure. Take a look at >> >> >> Tree::Simple. You will have USA in the first level, states in the >> >> >> second, >> >> >> cities in the third, and zip codes in the 4th (and then whatever >> >> >> else >> >> >> below >> >> >> that). >> >> >> >> >> >> -imran >> >> >> >> >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> I have some data like: >> >> >>> CITY|STATE|ZIP >> >> >>> SCHAUMBURG|IL|60194 >> >> >>> MATTESON|IL|60443 >> >> >>> WARRENTON|OR|97146 >> >> >>> MOUNTAIN HOME|AR|72653 >> >> >>> FORT WORTH|TX|76107 >> >> >>> CLEVELAND|MS|38732 >> >> >>> WATERTOWN|SD|57201 >> >> >>> GRAND CHUTE|WI|54913 >> >> >>> >> >> >>> I want to load it into a relational database in such a way that I >> >> >>> have >> >> >>> the proper keys to build a hierarchy. >> >> >>> >> >> >>> so for example: >> >> >>> ID Name ParentID >> >> >>> 0 USA >> >> >>> 1 IL 0 >> >> >>> 2 SCHAUMBURG 1 >> >> >>> 3 60194 2 >> >> >>> 4 MATTESON 1 >> >> >>> 5 60443 4 >> >> >>> 6 OR 0 >> >> >>> 7 WARRENTON 6 >> >> >>> 8 97146 7 >> >> >>> ... >> >> >>> >> >> >>> I'm not sure of a good way to do this. >> >> >>> >> >> >>> I read the data an built a hash like: >> >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >> >> >>> ? ? ? MATTESON ? => {60443 => 0}}, >> >> >>> OR => {WARRENTON ?=> {97146 => 0}} >> >> >>> ... >> >> >>> }; >> >> >>> >> >> >>> I can't think of a good way to look through it and assign the keys. >> >> >>> >> >> >>> Maybe someone has done this in the past and has an elegant >> >> >>> solution? >> >> >>> >> >> >>> Thanks >> >> >>> Jay >> >> >>> >> >> >>> _______________________________________________ >> >> >>> Chicago-talk mailing list >> >> >>> Chicago-talk at pm.org >> >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> >> Chicago-talk mailing list >> >> >> Chicago-talk at pm.org >> >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> > >> >> > >> >> > >> >> > _______________________________________________ >> >> > Chicago-talk mailing list >> >> > Chicago-talk at pm.org >> >> > http://mail.pm.org/mailman/listinfo/chicago-talk >> >> _______________________________________________ >> >> Chicago-talk mailing list >> >> Chicago-talk at pm.org >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >> > >> > >> > >> > _______________________________________________ >> > Chicago-talk mailing list >> > Chicago-talk at pm.org >> > http://mail.pm.org/mailman/listinfo/chicago-talk >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk > > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk _______________________________________________ Chicago-talk mailing list Chicago-talk at pm.org http://mail.pm.org/mailman/listinfo/chicago-talk -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at heyjay.com Thu Mar 1 15:44:30 2012 From: me at heyjay.com (Jay Strauss) Date: Thu, 1 Mar 2012 17:44:30 -0600 Subject: [Chicago-talk] creating relational keys In-Reply-To: <1330645104.55966.YahooMailNeo@web120506.mail.ne1.yahoo.com> References: <1330645104.55966.YahooMailNeo@web120506.mail.ne1.yahoo.com> Message-ID: On Thu, Mar 1, 2012 at 5:38 PM, tiger peng wrote: > Do you have to use Perl? Let relational keys done by relational database > may be easier. > That's the issue, is I don't have any keys to relate the data. I need to generate those keys > BTW, > As I know, in US, some zip codes may cross cities and even state boarder. > It cannot be simply represented in tree structure nor self-referenced table. > True. But in this case I don't have a choice Jay -------------- next part -------------- An HTML attachment was scrubbed... URL: From telcodev at gmail.com Thu Mar 1 15:51:57 2012 From: telcodev at gmail.com (Joseph Werner) Date: Thu, 1 Mar 2012 18:51:57 -0500 Subject: [Chicago-talk] creating relational keys In-Reply-To: <1330645104.55966.YahooMailNeo@web120506.mail.ne1.yahoo.com> References: <1330645104.55966.YahooMailNeo@web120506.mail.ne1.yahoo.com> Message-ID: That is only a problem if you consider leaf nodes to be unique. There is a quite valid possible tree representation of American zip codes. On Thu, Mar 1, 2012 at 6:38 PM, tiger peng wrote: > Do you have to use Perl? Let relational keys done by relational database may > be easier. > BTW, > As I know, in US, some zip codes may cross cities and even state boarder. It > cannot be simply represented in tree structure nor self-referenced table. > > ________________________________ > From: Chris Hamilton > To: Chicago.pm chatter > Sent: Thursday, March 1, 2012 2:08 PM > Subject: Re: [Chicago-talk] creating relational keys > > Yeah, looks that way. :) > > It's too bad you're forced to use such a model, it's not going to make > for very lovely queries, I don't imagine.? Good luck! > > -Chris > > On Thu, Mar 1, 2012 at 2:06 PM, Jay Strauss wrote: >> thanks Chris ! >> >> We solved it essentially the same way it looks like. >> >> Jay >> >> >> On Thu, Mar 1, 2012 at 1:57 PM, Chris Hamilton wrote: >>> >>> Assuming that all keys at a given depth of the hash structure are >>> unique (probably a big assumption with city names in a state), the >>> attached script example should probably do what you need (assuming I'm >>> correctly understanding your problem and you're trying to figure out >>> how to iterate through the hash structure and do the necessary inserts >>> with the correct parentIDs). ?If I'm wrong about which problem you're >>> having I apologize, but hopefully this provides some insight. >>> >>> Thanks, >>> -Chris >>> >>> On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: >>> > Hi, yes (unfortunately) it all has to go in the same table. >>> > >>> > I'm using a vendor supplied data model, with a generic structure like; >>> > >>> > ID NAME ParentID >>> > >>> > Although, even if I could break it up into separate tables I'd still >>> > need >>> > the keys >>> > >>> > >>> > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton >>> > wrote: >>> >> >>> >> Just out of curiosity, is there a strong need to put all of this data >>> >> into the same table? ?At least with your example data it seems like >>> >> the hierarchy is better managed through separate tables for city, >>> >> state, and zip (with a 1:many relationship from state to city, and >>> >> then from city to zip). ?In that form it would be easy enough to >>> >> create the ID's and tie them up as you iterate through your hash keys. >>> >> ?Just a thought. >>> >> >>> >> -Chris >>> >> >>> >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >>> >> > Imran, >>> >> > >>> >> > Thanks. ?I'm ok with building a tree, currently I'm using a hash. I >>> >> > just >>> >> > don't know how to unwind the tree and assign the proper ID and >>> >> > parent >>> >> > ID. >>> >> > >>> >> > Jay >>> >> > >>> >> > >>> >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid >>> >> > wrote: >>> >> >> >>> >> >> One option would be use a tree data structure. Take a look at >>> >> >> Tree::Simple. You will have USA in the first level, states in the >>> >> >> second, >>> >> >> cities in the third, and zip codes in the 4th (and then whatever >>> >> >> else >>> >> >> below >>> >> >> that). >>> >> >> >>> >> >> -imran >>> >> >> >>> >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >>> >> >>> >>> >> >>> Hi all, >>> >> >>> >>> >> >>> I have some data like: >>> >> >>> CITY|STATE|ZIP >>> >> >>> SCHAUMBURG|IL|60194 >>> >> >>> MATTESON|IL|60443 >>> >> >>> WARRENTON|OR|97146 >>> >> >>> MOUNTAIN HOME|AR|72653 >>> >> >>> FORT WORTH|TX|76107 >>> >> >>> CLEVELAND|MS|38732 >>> >> >>> WATERTOWN|SD|57201 >>> >> >>> GRAND CHUTE|WI|54913 >>> >> >>> >>> >> >>> I want to load it into a relational database in such a way that I >>> >> >>> have >>> >> >>> the proper keys to build a hierarchy. >>> >> >>> >>> >> >>> so for example: >>> >> >>> ID Name ParentID >>> >> >>> 0 USA >>> >> >>> 1 IL 0 >>> >> >>> 2 SCHAUMBURG 1 >>> >> >>> 3 60194 2 >>> >> >>> 4 MATTESON 1 >>> >> >>> 5 60443 4 >>> >> >>> 6 OR 0 >>> >> >>> 7 WARRENTON 6 >>> >> >>> 8 97146 7 >>> >> >>> ... >>> >> >>> >>> >> >>> I'm not sure of a good way to do this. >>> >> >>> >>> >> >>> I read the data an built a hash like: >>> >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >>> >> >>> ? ? ? MATTESON ? => {60443 => 0}}, >>> >> >>> OR => {WARRENTON ?=> {97146 => 0}} >>> >> >>> ... >>> >> >>> }; >>> >> >>> >>> >> >>> I can't think of a good way to look through it and assign the >>> >> >>> keys. >>> >> >>> >>> >> >>> Maybe someone has done this in the past and has an elegant >>> >> >>> solution? >>> >> >>> >>> >> >>> Thanks >>> >> >>> Jay >>> >> >>> >>> >> >>> _______________________________________________ >>> >> >>> Chicago-talk mailing list >>> >> >>> Chicago-talk at pm.org >>> >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> >> >>> >> >> >>> >> >> >>> >> >> _______________________________________________ >>> >> >> Chicago-talk mailing list >>> >> >> Chicago-talk at pm.org >>> >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> > >>> >> > >>> >> > >>> >> > _______________________________________________ >>> >> > Chicago-talk mailing list >>> >> > Chicago-talk at pm.org >>> >> > http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> _______________________________________________ >>> >> Chicago-talk mailing list >>> >> Chicago-talk at pm.org >>> >> http://mail.pm.org/mailman/listinfo/chicago-talk >>> > >>> > >>> > >>> > _______________________________________________ >>> > Chicago-talk mailing list >>> > Chicago-talk at pm.org >>> > http://mail.pm.org/mailman/listinfo/chicago-talk >>> >>> _______________________________________________ >>> Chicago-talk mailing list >>> Chicago-talk at pm.org >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk -- Best Regards, [Joseph] Christian Werner Sr C 360.920.7183 H 757.304.0502 From tigerpeng2001 at yahoo.com Fri Mar 2 09:31:09 2012 From: tigerpeng2001 at yahoo.com (tiger peng) Date: Fri, 2 Mar 2012 09:31:09 -0800 (PST) Subject: [Chicago-talk] creating relational keys In-Reply-To: References: <1330645104.55966.YahooMailNeo@web120506.mail.ne1.yahoo.com> Message-ID: <1330709469.45310.YahooMailNeo@web120506.mail.ne1.yahoo.com> You are right. I made one to make the keys and print the result. :-) cat /tmp/tmp.txt CITY|STATE|ZIP SCHAUMBURG|IL|60194 MATTESON|IL|60443 WARRENTON|OR|97146 HOME|AR|72653 WORTH|TX|76107 CLEVELAND|MS|38732 WATERTOWN|SD|57201 CHUTE|WI|54913 :-)perl -aF"\\|" -nle ' BEGIN {$key{USA}=$key=0; print "0|USA|";} ? next if $. < 2; # I don't know why the line# start from 1??? ? foreach $c (1,0,2) { ???? if (not exists $key{$F[$c]}) { ?????? $key{$F[$c]}=++$key; ?????? if ($c == 1) { #state column ???????? print "$key{$F[1]}|$F[1]|0"; ?????? } elsif ($c == 0) { #city column ???????? print "$key{$F[0]}|$F[0]|$key{$F[1]}"; ?????? } else { #zip code column ???????? print "$key{$F[2]}|$F[2]|$key{$F[0]}"; ?????? } ??? } ? } ' tmp.txt 0|USA| 1|IL|0 2|SCHAUMBURG|1 3|60194|2 4|MATTESON|1 5|60443|4 6|OR|0 7|WARRENTON|6 8|97146|7 9|AR|0 10|HOME|9 11|72653|10 12|TX|0 13|WORTH|12 14|76107|13 15|MS|0 16|CLEVELAND|15 17|38732|16 18|SD|0 19|WATERTOWN|18 20|57201|19 21|WI|0 22|CHUTE|21 23|54913|22 ________________________________ From: Joseph Werner To: Chicago.pm chatter Sent: Thursday, March 1, 2012 5:51 PM Subject: Re: [Chicago-talk] creating relational keys That is only a problem if you consider leaf nodes to be unique.? There is a quite valid possible tree representation of American zip codes. On Thu, Mar 1, 2012 at 6:38 PM, tiger peng wrote: > Do you have to use Perl? Let relational keys done by relational database may > be easier. > BTW, > As I know, in US, some zip codes may cross cities and even state boarder. It > cannot be simply represented in tree structure nor self-referenced table. > > ________________________________ > From: Chris Hamilton > To: Chicago.pm chatter > Sent: Thursday, March 1, 2012 2:08 PM > Subject: Re: [Chicago-talk] creating relational keys > > Yeah, looks that way. :) > > It's too bad you're forced to use such a model, it's not going to make > for very lovely queries, I don't imagine.? Good luck! > > -Chris > > On Thu, Mar 1, 2012 at 2:06 PM, Jay Strauss wrote: >> thanks Chris ! >> >> We solved it essentially the same way it looks like. >> >> Jay >> >> >> On Thu, Mar 1, 2012 at 1:57 PM, Chris Hamilton wrote: >>> >>> Assuming that all keys at a given depth of the hash structure are >>> unique (probably a big assumption with city names in a state), the >>> attached script example should probably do what you need (assuming I'm >>> correctly understanding your problem and you're trying to figure out >>> how to iterate through the hash structure and do the necessary inserts >>> with the correct parentIDs). ?If I'm wrong about which problem you're >>> having I apologize, but hopefully this provides some insight. >>> >>> Thanks, >>> -Chris >>> >>> On Thu, Mar 1, 2012 at 1:14 PM, Jay Strauss wrote: >>> > Hi, yes (unfortunately) it all has to go in the same table. >>> > >>> > I'm using a vendor supplied data model, with a generic structure like; >>> > >>> > ID NAME ParentID >>> > >>> > Although, even if I could break it up into separate tables I'd still >>> > need >>> > the keys >>> > >>> > >>> > On Thu, Mar 1, 2012 at 1:11 PM, Chris Hamilton >>> > wrote: >>> >> >>> >> Just out of curiosity, is there a strong need to put all of this data >>> >> into the same table? ?At least with your example data it seems like >>> >> the hierarchy is better managed through separate tables for city, >>> >> state, and zip (with a 1:many relationship from state to city, and >>> >> then from city to zip). ?In that form it would be easy enough to >>> >> create the ID's and tie them up as you iterate through your hash keys. >>> >> ?Just a thought. >>> >> >>> >> -Chris >>> >> >>> >> On Thu, Mar 1, 2012 at 12:58 PM, Jay Strauss wrote: >>> >> > Imran, >>> >> > >>> >> > Thanks. ?I'm ok with building a tree, currently I'm using a hash. I >>> >> > just >>> >> > don't know how to unwind the tree and assign the proper ID and >>> >> > parent >>> >> > ID. >>> >> > >>> >> > Jay >>> >> > >>> >> > >>> >> > On Thu, Mar 1, 2012 at 12:51 PM, imran javaid >>> >> > wrote: >>> >> >> >>> >> >> One option would be use a tree data structure. Take a look at >>> >> >> Tree::Simple. You will have USA in the first level, states in the >>> >> >> second, >>> >> >> cities in the third, and zip codes in the 4th (and then whatever >>> >> >> else >>> >> >> below >>> >> >> that). >>> >> >> >>> >> >> -imran >>> >> >> >>> >> >> On Thu, Mar 1, 2012 at 12:40 PM, Jay Strauss wrote: >>> >> >>> >>> >> >>> Hi all, >>> >> >>> >>> >> >>> I have some data like: >>> >> >>> CITY|STATE|ZIP >>> >> >>> SCHAUMBURG|IL|60194 >>> >> >>> MATTESON|IL|60443 >>> >> >>> WARRENTON|OR|97146 >>> >> >>> MOUNTAIN HOME|AR|72653 >>> >> >>> FORT WORTH|TX|76107 >>> >> >>> CLEVELAND|MS|38732 >>> >> >>> WATERTOWN|SD|57201 >>> >> >>> GRAND CHUTE|WI|54913 >>> >> >>> >>> >> >>> I want to load it into a relational database in such a way that I >>> >> >>> have >>> >> >>> the proper keys to build a hierarchy. >>> >> >>> >>> >> >>> so for example: >>> >> >>> ID Name ParentID >>> >> >>> 0 USA >>> >> >>> 1 IL 0 >>> >> >>> 2 SCHAUMBURG 1 >>> >> >>> 3 60194 2 >>> >> >>> 4 MATTESON 1 >>> >> >>> 5 60443 4 >>> >> >>> 6 OR 0 >>> >> >>> 7 WARRENTON 6 >>> >> >>> 8 97146 7 >>> >> >>> ... >>> >> >>> >>> >> >>> I'm not sure of a good way to do this. >>> >> >>> >>> >> >>> I read the data an built a hash like: >>> >> >>> USA => {IL => {SCHAUMBURG => {60194 => 0}, >>> >> >>> ? ? ? MATTESON ? => {60443 => 0}}, >>> >> >>> OR => {WARRENTON ?=> {97146 => 0}} >>> >> >>> ... >>> >> >>> }; >>> >> >>> >>> >> >>> I can't think of a good way to look through it and assign the >>> >> >>> keys. >>> >> >>> >>> >> >>> Maybe someone has done this in the past and has an elegant >>> >> >>> solution? >>> >> >>> >>> >> >>> Thanks >>> >> >>> Jay >>> >> >>> >>> >> >>> _______________________________________________ >>> >> >>> Chicago-talk mailing list >>> >> >>> Chicago-talk at pm.org >>> >> >>> http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> >> >>> >> >> >>> >> >> >>> >> >> _______________________________________________ >>> >> >> Chicago-talk mailing list >>> >> >> Chicago-talk at pm.org >>> >> >> http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> > >>> >> > >>> >> > >>> >> > _______________________________________________ >>> >> > Chicago-talk mailing list >>> >> > Chicago-talk at pm.org >>> >> > http://mail.pm.org/mailman/listinfo/chicago-talk >>> >> _______________________________________________ >>> >> Chicago-talk mailing list >>> >> Chicago-talk at pm.org >>> >> http://mail.pm.org/mailman/listinfo/chicago-talk >>> > >>> > >>> > >>> > _______________________________________________ >>> > Chicago-talk mailing list >>> > Chicago-talk at pm.org >>> > http://mail.pm.org/mailman/listinfo/chicago-talk >>> >>> _______________________________________________ >>> Chicago-talk mailing list >>> Chicago-talk at pm.org >>> http://mail.pm.org/mailman/listinfo/chicago-talk >> >> >> >> _______________________________________________ >> Chicago-talk mailing list >> Chicago-talk at pm.org >> http://mail.pm.org/mailman/listinfo/chicago-talk > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk > > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk -- Best Regards, [Joseph] Christian Werner Sr C 360.920.7183 H 757.304.0502 _______________________________________________ Chicago-talk mailing list Chicago-talk at pm.org http://mail.pm.org/mailman/listinfo/chicago-talk -------------- next part -------------- An HTML attachment was scrubbed... URL: From tigerpeng2001 at yahoo.com Wed Mar 7 07:51:33 2012 From: tigerpeng2001 at yahoo.com (tiger peng) Date: Wed, 7 Mar 2012 07:51:33 -0800 (PST) Subject: [Chicago-talk] Text::CSV->getline_hr & Extra Input Fields Message-ID: <1331135493.68005.YahooMailNeo@web120504.mail.ne1.yahoo.com> Hello All, I am using Text::CSV for parsing log files. I prefer using column_names/getline_hr than bind_columns/getline, as I feel the former one has better readability and maintainability.? However, it seems getline_hr does not throw any exception when there are any extra input fields (bad records) as getline does. Is there better way to handle the bad records with extra fields without using bind_columns/getline? Thanks Tiger -------------- next part -------------- An HTML attachment was scrubbed... URL: From tigerpeng2001 at yahoo.com Fri Mar 16 08:30:50 2012 From: tigerpeng2001 at yahoo.com (tiger peng) Date: Fri, 16 Mar 2012 08:30:50 -0700 (PDT) Subject: [Chicago-talk] Suggestion for Learning Java as Perlover Message-ID: <1331911850.57227.YahooMailNeo@web120502.mail.ne1.yahoo.com> Hello All, I have not worked on Java for a long time (almost 10 years) and now need to learn it again (for Hadoop projects). Could someone suggest some books or on-line resources for quick pickup? Thanks, Tiger. -------------- next part -------------- An HTML attachment was scrubbed... URL: From perlsyntax at gmail.com Fri Mar 16 16:20:21 2012 From: perlsyntax at gmail.com (syntax) Date: Fri, 16 Mar 2012 18:20:21 -0500 Subject: [Chicago-talk] Suggestion for Learning Java as Perlover In-Reply-To: <1331911850.57227.YahooMailNeo@web120502.mail.ne1.yahoo.com> References: <1331911850.57227.YahooMailNeo@web120502.mail.ne1.yahoo.com> Message-ID: <4F63CAB5.9020706@gmail.com> On 03/16/2012 10:30 AM, tiger peng wrote: > Hello All, > > I have not worked on Java for a long time (almost 10 years) and now > need to learn it again (for Hadoop projects). > Could someone suggest some books or on-line resources for quick pickup? > > Thanks, > > Tiger. > > > _______________________________________________ > Chicago-talk mailing list > Chicago-talk at pm.org > http://mail.pm.org/mailman/listinfo/chicago-talk thinking in java book.It good book to learn java from. -------------- next part -------------- An HTML attachment was scrubbed... URL: