[VPM] - data structures, performance and memory

[Eric Frazier]

Hi Jeremy,

Kind of a philosophy thing maybe, but I would not tend to think it is bad 
idea to optimize things as much as you can. Esp considering you will be 
running this script on a shared hosting server. And I kind of have this 
thing about answer the question someone asks, not telling them it is the 
wrong thing to ask.

So assuming that what Jeremy wants to do is correct with the above in mind. 
What would be the best way to go? Sure there isn't that much data, but it 
doesn't take much to get expensive in that shared environment.

This looks interesting, but we go back to that shared server thing:
http://www.danga.com/memcached/

So what I would wonder about is would there be any benefit to using 
storeable, or compressing the data first using something like
Compress::Zlib::memGzip

Which again might be an issue on something like Shaw.. I have winged 
installing perl modules by FTP by compiling them on another machine first

So then would Storeable be of some use? Does it end up being more efficient 
in size? I would think so from what I have seen.

This also might be worth looking at as I didn't know this about perl 5.8
Tie::Handle::ToMemory Since that wasn't possible since before 5.8 you could 
look at how this guy did it..

The only other question I would have is why are you needing in memory 
storage? Esp if this is being accessed through a CGI, you are screwed right 
there. That bit I find a little confusing.


Eric




At 08:34 PM 09/11/2006, Jer A wrote:
>hi all perl gurus,
>
>I am working on two projects where I can use some of your advice.
>
>project 1: I am volunteering for a non-profit organization putting a printed
>"information directory" on their website. It will consist of a search
>engine, and an Index of categories, where one can narrow down, and browse,
>if they do not wish to do a search. Due, to funding and other restrictions,
>they cannot host an sql database, xml or webservices, as they use SHAW
>hosting, So I will be doing this project using a flat file approach. should
>I use many small flat files, or one big one? should I make a filesystem
>directory for each catagory? or how can i associate keywords with the data
>to be searched? What can i do to make this search engine as fast and
>efficient as possible?
>
>project 2: I am working on a project where I need to store data in memory.
>What are the memory requirements of certain structures eg. hash,array,array
>of anon hashes,array of packed data,array of strings, array of objects etc.
>
>suppose I have a hundred records (structures) holding eg. age,name,address
>etc. what is the best way of storing this data live in memory for a long
>period time, for performance and efficiency.
>An array of anon hashes is pretty, but may not be the best for performance,
>and mem usage...am i right? Considering this, how can I store records with
>elements that can be of different record types, saving the most amount of
>memory as possible......can I do this with pack and unpack, I am not
>familiar with this, but do you think s more efficient, than an array of
>element delimited strings? or what if each record is an instantiated object
>with properties?
>
>Thanks in advance for your advice and help.
>
>   -Jeremy A.
>
>_________________________________________________________________
>Ready for the world's first international mobile film festival celebrating
>the creative potential of today's youth? Check out Mobile Jam Fest for your
>a chance to WIN $10,000! www.mobilejamfest.com
>
>_______________________________________________
>Victoria-pm mailing list
>Victoria-pm at pm.org
>http://mail.pm.org/mailman/listinfo/victoria-pm