[Pdx-pm] Building systems...

[Rob Bloodgood]

Monday, June 30, 2003, 10:07:05 AM, you wrote:

NW> Hello wizards... I was wondering....

NW> I am researching ways to implement a pure perl
NW> application which would -

NW> * interface with mySql database
NW> * process incoming requests from TCP and SOAP sockets
NW> * process multiple tasks in the background
NW> * perform large mathmatical calculations

NW> I hope this is enough to give you an idea.  I'm
NW> expecting this app to handle a lot of traffic from many
NW> interfaces.

NW> So I just discovered POE... it sounds like it would
NW> work.  Any success stories?  Also, I've played with PDL
NW> which I will throw my large numbers at to munch... and
NW> of course the SOAP toolkit.

Well, here at ExitExchange.com, we use a combination of mod_perl and
POE to run the stats/ads side of our system.  They tell me we're now
ranked #31 on the internet for traffic, but I don't pay much attention
to that stuff... :-)

Anyway, when a request comes in for an ad, a mod_perl handler parses
cookies and state information, then sends a message to a POE program
that has already hit the database and set up its criteria.  The POE
program shuffles thru its lists, then returns an ad to display.

The fun part is, it's single-threaded, and handles as many concurrent
connections (read that, "webservers") at a time as I've yet needed.

I found that the only downside was that database queries blocked,
making the whole system wait for the query results.  Clearly, this
interfered with my desire to have multiple simultaneous sessions
processing, as the whole program would stop and wait for this one
response.  So with the help of folks in #POE, I hacked together a POE
component called POE::Component::DBIAgent, which spawns a child
process to run database queries, and spoonfeeds the results back when
they finally arrive.

I use a completely different POE program to parse webserver logs and
turn them into statistics.  There's not a BUNCH of POE-ness to that
program, really... I used POE because it gave me a prepackaged way to
automatically 'tail -f' a logfile while correctly tracking logfile
rotation.

So, YMMV and all that, but I *can* say there is success possible down
the route you've suggested!

L8r,
Rob