[Pdx-pm] queueing, avoiding race conditions
randall at sonofhans.net
Wed Aug 10 11:49:20 PDT 2005
this isn't really a Perl problem, although i'll be implementing the
solution in Perl. i need to create a job queue for a long-running
i'm querying and updating Widgets. normal case is to update all
1,000 Widgets, which takes several hours. special case is to update
a small group of Widgets.
if one Widget process is running, no other Widget process should run
at the same time (there are good and sufficient reasons for this).
if someone wants to update another Widget[s], this request should be
added to the queue.
- Widget updater "A" begins
- IF another Widget update ("B") is requested
- request "B" is queued
- process "A" completes
- process "A" examines the queue
- IF it finds request "B"
- goto 1
one suggestion i've had is to try to create a MySQL heap table
when the update process begins. if the table does not already
exists, the process (a) creates and locks it, (b) adds an entry for
itself, (c) unlocks it.
if the table exists, and the PID of the first item in the queue
exists, the process does the lock/add/unlock cycle, returns a "wait a
bit" message, and exists cleanly. if the PID of the first item does
not exist, we assume that it's crashed, and pop it off.
after an update process completes, it removes itself from the queue
and checks for a next entry. if present, it configures itself as
that next entry, updates the entry's PID, and continues. if there's
no next entry, it deletes the table.
the bugbear of this problem seems to be race conditions. i could
create a bunch of exception handling for the algorithm, but it seems
like (a) that would get dirty very fast, and (b) be a PITA to test.
anyone have suggestions, or critiques of the algorithm and method
1) target DB is MySQL 4.0.18
More information about the Pdx-pm-list