[VPM] - data structures, performance and memory
darren at DarrenDuncan.net
Fri Nov 10 02:11:40 PST 2006
At 8:34 PM -0800 11/9/06, Jer A wrote:
>hi all perl gurus,
>I am working on two projects where I can use some of your advice.
I'll see what I can do here.
>project 1: I am volunteering for a non-profit organization putting a printed
>"information directory" on their website. It will consist of a search
>engine, and an Index of categories, where one can narrow down, and browse,
>if they do not wish to do a search. Due, to funding and other restrictions,
>they cannot host an sql database, xml or webservices, as they use SHAW
>hosting, So I will be doing this project using a flat file approach. <snip>
The simplest solution I see, if your content is static and
infrequently changes, and you are staying with Shaw, is to just
pre-generate the index and content pages as static HTML and just dump
it on the server. Any code will run on your own machine, where you
can use a database. You can then use a specially crafted Google
link, one containing "site:" in the query, to just let Google provide
the search mechanism for your site. Lots of small (and larger) sites
use Google to do their search engine for them.
If your content is changed regularly and/or is interactive, I suggest
getting a different host. Shaw costs at least $20-50 per month,
AFAIK, and a dedicated web host with databases and such can be had
for as low as $10 per month, and you get your own domain name with
If neither of these is suitable for some reason, then please give
more information about how much data you have, what kind of data it
is, and about how much traffic you get, plus what features Shaw does
>project 2: I am working on a project where I need to store data in memory.
>What are the memory requirements of certain structures eg. hash,array,array
>of anon hashes,array of packed data,array of strings, array of objects etc.
>suppose I have a hundred records (structures) holding eg. age,name,address
>etc. what is the best way of storing this data live in memory for a long
>period time, for performance and efficiency.
>An array of anon hashes is pretty, but may not be the best for performance,
>and mem usage...am i right? Considering this, how can I store records with
>elements that can be of different record types, saving the most amount of
>memory as possible......can I do this with pack and unpack, I am not
>familiar with this, but do you think s more efficient, than an array of
>element delimited strings? or what if each record is an instantiated object
>Thanks in advance for your advice and help.
> -Jeremy A.
I think you may be getting into a "premature optimization" matter.
If you have only a hundred records of eg age,name,address, those will
all take up maybe 1 kilobyte at most of memory, which is negligible.
Don't bother fussing about memory unless you're storing tens of
thousands or more of such records, or you're working on an embedded
system. Just do whatever is easiest to program, as programmer
efficiency is usually more important than any other kind. Use hashes
if you're looking data up directly, such as by the person's name, and
arrays only when you plan to go through them sequentially. Or you
can include multiple references to the same data structures if you
-- Darren Duncan
More information about the Victoria-pm