[Jax.PM] Re: perl HTTP header insertion proxy

[Nate Campi]

On Thu, May 15, 2003 at 02:35:15AM -0400, WC -Sx- Jones wrote:
> On Thursday, May 15, 2003, at 02:10  AM, Nate Campi wrote:
> >
> >I cannot crawl the filesystem to add the executable bit since I have at
> >least a million files in my docroot.
> 
> Owie?  :)   How the heck are 'dem inodes doin' ?  Are you running out 
> of file handles on lookups?  Is that a SUN box or Linux?  We are 
> waaaaayyyyyy off Perl here =)

This is Sun, inodes tend not to be a problem, big disks and cron jobs to
clean scrap files seem to take care of things.

> Seriously, you might could schedule a sleeper to crawl the file system 
> during system idle prolly.  At 2:30AM I prolly need to sleep now...

Yes, but I'd want the freshly generated files to have the correct perms
immediately. They are most of what's being accessed (news site) so
waiting until later would defeat the purpose.

> I feel a "group project" emerging - what do you think Nate?
 
Hmm, I'd be happy to write up my recent experiences with caching.
Between myself and the main developer working on it we could probably
share some good stuff.
 
> Apache 2.0 accepts and handles other protocols besides HTTP - as you 
> prolly know - there is a SMTP module already out to show off this 
> feature.  Have you thought about modifying the 2.0 stuff to make it do 
> what you want?  Maybe there is another 'feature' or module interfering 
> with the desired operation you are expecting?  I'll get a newer Apache 
> 2.x  (there are CAN and CERT advisories out regarding other editions) 
> and will look at it further  :)

This is a good idea. I had our resident apache hacker look at what it
would take to modify 1.3 (since it's a proven code base) and it's not
easily doable. I pointed out to him that 2.0 has the code to deal with
unknown content-lengths in the mod_disk_cache module and he said that's
promising. The main problem there is that this would get really low
priority over stuff that actually makes us money (increasing scalability
and efficiency doesn't get dollars behind you where I work, they want to
see immediate $$). If I could bust out some way with a proxy I could do
it all on my own. 

Anyhoo, I don't think I mentioned perl once in this post :(

I'm off fishun tonight until Monday, have a good weekend.
-- 
Nate Campi    http://www.campin.net