[Buffalo-pm] apache::storage

Kevin Eye eye at buffalo.edu
Wed Feb 8 13:27:41 PST 2006


That is the case. You can use any generic shared memory approach, such as
IPC::ShareLite:
http://search.cpan.org/~maurice/IPC-ShareLite-0.09/ShareLite.pm

But then you have to worry about cleaning up the shared memory (it will hang
around forever otherwise), and also make sure that one process isn't in the
middle of reading it while another decides to write to it (can lead to seg
faults). Also, you can only share a scalar, not a hash or array. To store
those, you'd have to serialize them with Storable, FreezeThaw, Data::Dumper,
etc. There are probably some libraries that help automatically handle
various parts of this. This all starts to add complexity and overhead that
might not be worthwhile or end up saving time overall.

In my work, I've never benchmarked the exact effects of caching stuff from
disk into memory, but I generally assume it wouldn't be worth it. The disk
cache already automatically does it to some extent. I think you'll find the
most worthwhile speed gains to be from caching the perl interpreter (i.e.
using mod_perl or fastcgi), and caching the result of database queries and
other complex code.

You may want to look into perl's code profiling capabilities to see where
your code is actually spending most of its time. It's usually quite a
surprise. Check out these articles:
http://www.perl.com/pub/a/2004/06/25/profiling.html
http://www.perl.com/pub/a/2002/07/16/mod_perl.html

 - Kevin


On 2/8/06 3:55 PM, "joshj at linuxmail.org" <joshj at linuxmail.org> wrote:

>>>>  Could that $main::page_1="the content" trick work? Or
>>>>  better yet, could I make that a hashref: $main::pages->{page_1} = "the
>>>>  content"?
>>> 
>>>  This will work unless mod_perl does something special to prevent this from
>>>  working. (It wouldn't be a bad idea to disallow this for security
>>>  purposes,
>>>  but I don't think it does.)
>> 
>> This worked out well. It didn't save as much time as I thought it would,
>> but .01 seconds is something. However it still looses it after 60
>> seconds. Is there a way to extend that?
> 
> Wait, figured this one out too. After a while it will always retreive
> the page from memory ($main::page->{pagename}). I'm guessing that this
> is because I keep getting random apache child-processes on request? Each
> hashref-instance will eventually be populated with the pages? So does
> that mean that if I have a total of 1Meg of pages and 20 Apache
> processes that 20M of memory will be taken up? If that's the case then
> is there anyway I can create 1 structure that is accessible by all
> apache processes?
> 
> -Josh
> 
>> 
>>> 
>>>  I use the same kind of thing to cache log file handles and database
>>>  connections in fastcgi applications, which are very similar in concept to
>>>  mod_perl.
>>> 
>>>>  Would A hashref be faster?
>>> 
>>>  I think a hashref would technically be slower, as you've got an extra hash
>>>  lookup in there. But it's very efficient, and I'd be surprised if you
>>>  could
>>>  measure a difference. I'd use the hashref because it would be cleaner
>>>  code,
>>>  especially if you eventually wanted to dynamically manage a cache of
>>>  several
>>>  pieces of content.
>> 
>> Ok. (I know that the speed difference here is probably only academic) I
>> thought that a ref of somekind would be better because some pages would
>> probably be several K.
>> 
>> -Josh
>> 
>>> 
>>>  - Kevin
>>> 
>>> 
>>>  On 2/8/06 11:18 AM, "joshj at linuxmail.org" <joshj at linuxmail.org> wrote:
>>> 
>>>>  Thanks Dan and Kevin.
>>>> 
>>>>  I am actually working in mod_perl. I'm making my own custom handler (I
>>>>  know that there are 1001 of them on CPAN. This is purely for my learning
>>>>  experience). I just want to see how fast I can make it to do my specific
>>>>  tasks on my P2.
>>>> 
>>>>  One of my original problems was that I was dynamically
>>>>  loading subroutines. But the overhead of that was killing me. Then I
>>>>  found that mod_perl kept the subroutine stored in memory. So just adding
>>>>  an 'if (!defined(&$sub_ref))' before loading it made a dramatic
>>>>  performance increase.
>>>> 
>>>>  Now there is still the File IO of opening up each page and pages
>>>>  included within (like common HTML header files and footer files). I'm
>>>>  looking for a way to store them in memory. Then I'd check and only open
>>>>  them if it couldn't find them in memory. Maybe there is another way to
>>>>  accomplish this? Could that $main::page_1="the content" trick work? Or
>>>>  better yet, could I make that a hashref: $main::pages->{page_1} = "the
>>>>  content"? Would A hashref be faster? My understanding of them is that
>>>>  they access the memory location like pointers in C do. Is that right?
>>>> 
>>>>  As it stands it beats the pants off of what mod_php and servlets can do
>>>>  on this same machine. Keep in mind though that I haven't really tried to
>>>>  optimize
>>>>  those ones yet. And this isn't scaled up to their level either. I'm just
>>>>  impressed with power and control that mod_perl has.
>>>> 
>>>>  -Josh J
>>>> 
>>>>  Thus spake Kevin Eye on Wed, 8 Feb 2006
>>>> 
>>>>>  I'm not all that clear on the nnecessity of Apache::Storage. In
>>>>>  mod_perl, if
>>>>>  you just shove something into a global variable ($main::save_for_later
>>>>>  =
>>>>>  $something_important), won't it be around for the next request? Or
>>>>>  does it
>>>>>  disallow that or somehow clean up after itself?
>>>>> 
>>>>>  - Kevin
>>>>> 
>>>>> 
>>>>>  On 2/8/06 9:17 AM, "DANIEL MAGNUSZEWSKI" <dmagnuszewski at mandtbank.com>
>>>>>  wrote:
>>>>> 
>>>>>>  Have you looked into mod_perl?
>>>>>> 
>>>>>> http: //www.perl.com/pub/a/2002/02/26/whatismodperl.html
>>>>>> http: //www.perl.com/pub/a/2002/03/22/modperl.html
>>>>>> 
>>>>>>  Jim or Kevin can probably shed more light on this.
>>>>>> 
>>>>>>  -Dan
>>>>>> 
>>>>>>>>>  <joshj at linuxmail.org> 02/07/06 9:36 AM >>>
>>>>>>  Is there an Apache2 equivelent of Apache::Storage which will let me
>>>>>>  keep
>>>>>>  things in apache's memory to use later? Like frequently accessed
>>>>>>  functions and such.
>>>>>> 
>>>>>>  -Josh J
>>>>>> _______________________________________________
>>>>>>  Buffalo-pm mailing list
>>>>>>  Buffalo-pm at pm.org
>>>>>>  http://mail.pm.org/mailman/listinfo/buffalo-pm
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>>  Buffalo-pm mailing list
>>>>>>  Buffalo-pm at pm.org
>>>>>>  http://mail.pm.org/mailman/listinfo/buffalo-pm
>>>>> 
>>>>>  --
>>>>>  Kevin Eye
>>>>>  Web Applications Developer
>>>>>  Marketing and Creative Services
>>>>>  University at Buffalo
>>>>>  330 Crofts Hall
>>>>>  Buffalo, NY 14260
>>>>>  eye at buffalo.edu
>>>>>  phone (716) 645-5000 x1435
>>>>>  fax (716) 645-3765
>>>>> 
>>>>> 
>>>>> 
>>>> _______________________________________________
>>>>  Buffalo-pm mailing list
>>>>  Buffalo-pm at pm.org
>>>>  http://mail.pm.org/mailman/listinfo/buffalo-pm
>>> 
>>>  --
>>>  Kevin Eye
>>>  Web Applications Developer
>>>  Marketing and Creative Services
>>>  University at Buffalo
>>>  330 Crofts Hall
>>>  Buffalo, NY 14260
>>>  eye at buffalo.edu
>>>  phone (716) 645-5000 x1435
>>>  fax (716) 645-3765
>>> 
>>> 
>>> 
>> 
>> 
> _______________________________________________
> Buffalo-pm mailing list
> Buffalo-pm at pm.org
> http://mail.pm.org/mailman/listinfo/buffalo-pm

-- 
Kevin Eye
Web Applications Developer
Marketing and Creative Services
University at Buffalo
330 Crofts Hall
Buffalo, NY 14260
eye at buffalo.edu
phone (716) 645-5000 x1435
fax (716) 645-3765




More information about the Buffalo-pm mailing list