SPUG: Daemonized work queue handling

Jonathan rise at knavery.net
Tue May 6 13:23:51 PDT 2008

We've got a process at work that handles generating and sending out
about 20M monthly newsletter emails to site members distributed over a
small cluster and I'm looking for a better way to run it.  Currently
each machine starts a cron job that forks semi-daemonized child
processes and redirects their output to a log file in /tmp.  They've
been converted to pull work blocks from a db-side queue and get all
their data from it as well, so there's no local state to care about.
Any recommendations or warnings for something that'll play well with
starting from an init script and running for months?

Things that look like they have potential:

Poe::Queue / POE::XS::Queue::Array
(possibly using POE::Wheel::Curses and dtach for a management/debugging REPL)

Pretty minimal solution

A little more featureful and feels more traditional

Looks a little crufty, but that may just be the job control file syntax

Jonathan Conway                                                rise at knavery.net
    Checking whether build environment is sane ... build environment is
          grinning and holding a spatula.  Guess not. - xkcd #371

More information about the spug-list mailing list