[VPM] - data structures, performance and memory

[Jer A]

hello Eric,

Thanks for your reply.

>The only other question I would have is why are you needing in memory 
>storage? Esp if this is being accessed through a CGI, you are screwed right 
>there. That bit I find a little confusing.

the memory storage is for project 2. the cgi involves project 1.
project 1 and 2 are totally un-related.

-Jeremy A.


>From: Eric Frazier <eric at dmcontact.com>
>To: "Jer A" <jeremygwa at hotmail.com>,victoria-pm at pm.org
>Subject: Re: [VPM]  - data structures, performance and memory
>Date: Fri, 10 Nov 2006 09:28:20 -0800
>
>Hi Jeremy,
>
>Kind of a philosophy thing maybe, but I would not tend to think it is bad 
>idea to optimize things as much as you can. Esp considering you will be 
>running this script on a shared hosting server. And I kind of have this 
>thing about answer the question someone asks, not telling them it is the 
>wrong thing to ask.
>
>So assuming that what Jeremy wants to do is correct with the above in mind. 
>What would be the best way to go? Sure there isn't that much data, but it 
>doesn't take much to get expensive in that shared environment.
>
>This looks interesting, but we go back to that shared server thing:
>http://www.danga.com/memcached/
>
>So what I would wonder about is would there be any benefit to using 
>storeable, or compressing the data first using something like
>Compress::Zlib::memGzip
>
>Which again might be an issue on something like Shaw.. I have winged 
>installing perl modules by FTP by compiling them on another machine first
>
>So then would Storeable be of some use? Does it end up being more efficient 
>in size? I would think so from what I have seen.
>
>This also might be worth looking at as I didn't know this about perl 5.8
>Tie::Handle::ToMemory Since that wasn't possible since before 5.8 you could 
>look at how this guy did it..
>
>The only other question I would have is why are you needing in memory 
>storage? Esp if this is being accessed through a CGI, you are screwed right 
>there. That bit I find a little confusing.
>
>
>Eric
>
>
>
>
>At 08:34 PM 09/11/2006, Jer A wrote:
>>hi all perl gurus,
>>
>>I am working on two projects where I can use some of your advice.
>>
>>project 1: I am volunteering for a non-profit organization putting a 
>>printed
>>"information directory" on their website. It will consist of a search
>>engine, and an Index of categories, where one can narrow down, and browse,
>>if they do not wish to do a search. Due, to funding and other 
>>restrictions,
>>they cannot host an sql database, xml or webservices, as they use SHAW
>>hosting, So I will be doing this project using a flat file approach. 
>>should
>>I use many small flat files, or one big one? should I make a filesystem
>>directory for each catagory? or how can i associate keywords with the data
>>to be searched? What can i do to make this search engine as fast and
>>efficient as possible?
>>
>>project 2: I am working on a project where I need to store data in memory.
>>What are the memory requirements of certain structures eg. 
>>hash,array,array
>>of anon hashes,array of packed data,array of strings, array of objects 
>>etc.
>>
>>suppose I have a hundred records (structures) holding eg. age,name,address
>>etc. what is the best way of storing this data live in memory for a long
>>period time, for performance and efficiency.
>>An array of anon hashes is pretty, but may not be the best for 
>>performance,
>>and mem usage...am i right? Considering this, how can I store records with
>>elements that can be of different record types, saving the most amount of
>>memory as possible......can I do this with pack and unpack, I am not
>>familiar with this, but do you think s more efficient, than an array of
>>element delimited strings? or what if each record is an instantiated 
>>object
>>with properties?
>>
>>Thanks in advance for your advice and help.
>>
>>   -Jeremy A.
>>
>>_________________________________________________________________
>>Ready for the world's first international mobile film festival celebrating
>>the creative potential of today's youth? Check out Mobile Jam Fest for 
>>your
>>a chance to WIN $10,000! www.mobilejamfest.com
>>
>>_______________________________________________
>>Victoria-pm mailing list
>>Victoria-pm at pm.org
>>http://mail.pm.org/mailman/listinfo/victoria-pm
>

_________________________________________________________________
Find a local pizza place, music store, museum and more…then map the best 
route! Check out Live Local today!  http://local.live.com/?mkt=en-ca/