SPUG: hashes, arrays, map, join

[John Cokos]

Quick Aside ....

I ran this on a JUNK machine: Win98, 64MB Ram, P500

On an actual linux server with some horsepower, these
numbers will actually become reasonable.

J-

----- Original Message ----- 
From: John Cokos <jcokos at iwebsys.com>
To: John Cokos <jcokos at iwebsys.com>; <spug-list at pm.org>
Sent: Friday, March 09, 2001 10:44 PM
Subject: Re: SPUG: hashes, arrays, map, join


> And the winner is ....
> (the numbers in front, are timestamps....)
> 
> 984205861: Starting Program
> 984205861: Starting Test 1, reading 32,000+ rows from mySQL, Query
> 984205872: Query Finished, Building Sorted Array from query results
> 984205873: Done ... 1151 sorted, and ready
>    Array Element 0: 0
>    Array Element 1: 892745342
>    Array Element 2: 892843995
>    Array Element 3: 893066815
>    Array Element 4: 893066933
>    Array Element 5: 893075888
>    Array Element 6: 893090421
>    Array Element 7: 893092009
>    Array Element 8: 893094274
>    Array Element 9: 893097182
>    Array Element 10: 893101655
> 
> 
> 984205930: Starting Program
> 984205930: Starting Test 2, Converting 32,000 key comma list into hash via: %ids = map { split(/:/) } split(/,/,$string);
> 984205983: Done ... 1151 sorted, and ready
> 
>    Array Element 0: 0
>    Array Element 1: 892745342
>    Array Element 2: 892843995
>    Array Element 3: 893066815
>    Array Element 4: 893066933
>    Array Element 5: 893075888
>    Array Element 6: 893090421
>    Array Element 7: 893092009
>    Array Element 8: 893094274
>    Array Element 9: 893097182
>    Array Element 10: 893101655
> 
> 
> 984206039: Starting Program
> 984206039: Starting Test 3, Converting 32,000 key comma list into hash via: %ids = split /[,:]/, $string;
> 984206046: Done ... 1151 sorted, and ready
> 
>    Array Element 0: 0
>    Array Element 1: 892745342
>    Array Element 2: 892843995
>    Array Element 3: 893066815
>    Array Element 4: 893066933
>    Array Element 5: 893075888
>    Array Element 6: 893090421
>    Array Element 7: 893092009
>    Array Element 8: 893094274
>    Array Element 9: 893097182
>    Array Element 10: 893101655
> 
> 
> 984206168: Starting Program
> 984206168: Starting Test 4, Converting 32,000 key comma list into hash via: %ids = $string =~ /(?:^|,)(d+):(d+)(?=,|$)/g;
> 984206177: Done ... 1151 sorted, and ready
> 
>    Array Element 0: 0
>    Array Element 1: 892745342
>    Array Element 2: 892843995
>    Array Element 3: 893066815
>    Array Element 4: 893066933
>    Array Element 5: 893075888
>    Array Element 6: 893090421
>    Array Element 7: 893092009
>    Array Element 8: 893094274
>    Array Element 9: 893097182
>    Array Element 10: 893101655
> 
> 
> Using SQL as a benchmark, I tested the alternative methods, you
> can see the timing results above (geez, "map" is slow, isn't it ???)
> 
> Looks like I can get a sorted array faster WITHOUT mySQL, too.
> 
> John
> 
> 
> ----- Original Message ----- 
> From: John Cokos <jcokos at iwebsys.com>
> To: <spug-list at pm.org>
> Cc: <jcokos at ccs.net>
> Sent: Friday, March 09, 2001 9:07 PM
> Subject: Re: SPUG: hashes, arrays, map, join
> 
> 
> > To summarize .....
> > 
> > It seems that these are the options presented by you all:
> > 
> > Alternatives to the use of map in populating the has to start with:
> >     %ids = split /[,:]/, $results;
> >     %ids = $results =~ /(?:^|,)(\d+):(\d+)(?=,|$)/g;
> > 
> > To recreate the hash afterwards:
> >     These appear the same, but just for grins, I'll try them separately.
> >     join (',', map {"$_:$ids{$_}"} keys %ids);
> >     join ',', map "$_:$ids{$_}", keys %ids;
> > 
> >     ** Thanks for these ... this was the focus of my mental block to day
> >         Looking at it here, I can't believe I didn't "get it" earlier, I
> > tried,literally
> >         every combination of things but THIS one **
> > 
> > 
> > Anyway, I'm loading the DMOZ Data right now, into mySQL, so I'll have
> > a good 30 to 40 million indexed words to monkey with.  I'll report back
> > some time trials on each of these methods.
> > 
> > John
> > 
> > 
> > 
> > ----- Original Message -----
> > From: Andrew Sweger <andy at n2h2.com>
> > To: Alex Leites <Alex.Leites at esca.com>
> > Cc: John Cokos, CEO, iWeb, Inc. <jcokos at iwebsys.com>; <spug-list at pm.org>
> > Sent: Friday, March 09, 2001 5:38 PM
> > Subject: RE: SPUG: hashes, arrays, map, join
> > 
> > 
> > > %ids = $results =~ /(?:^|,)(\d+):(\d+)(?=,|$)/g;
> > >
> > > is more evil. :)
> > >
> > > On Mar 9, 2001 @ 5:31pm, Alex Leites wrote:
> > >
> > > > One more thing:
> > > >
> > > > %ids = split /[,:]/, $results;
> > > >
> > > > is probably more efficient.
> > >
> > > --
> > >  Andrew B. Sweger <andy at n2h2.com> |  N2H2, Incorporated
> > >  Manager, ICE Development Dept    |  900 Fourth Avenue, Suite 3400
> > >  Content Management Division      |  Seattle WA 98164-1059
> > >  v=206.336.2947  f=206.493.0906   |  http://www.n2h2.com/
> > >
> > >
> > >  - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > >      POST TO: spug-list at pm.org       PROBLEMS: owner-spug-list at pm.org
> > >       Subscriptions; Email to majordomo at pm.org:  ACTION  LIST  EMAIL
> > >   Replace ACTION by subscribe or unsubscribe, EMAIL by your Email-address
> > >  For daily traffic, use spug-list for LIST ;  for weekly, spug-list-digest
> > >   Seattle Perl Users Group (SPUG) Home Page: http://www.halcyon.com/spug/
> > >
> > >
> > 
> 


 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
     POST TO: spug-list at pm.org       PROBLEMS: owner-spug-list at pm.org
      Subscriptions; Email to majordomo at pm.org:  ACTION  LIST  EMAIL
  Replace ACTION by subscribe or unsubscribe, EMAIL by your Email-address
 For daily traffic, use spug-list for LIST ;  for weekly, spug-list-digest
  Seattle Perl Users Group (SPUG) Home Page: http://www.halcyon.com/spug/