[Pdx-pm] queueing, avoiding race conditions

[Randall Hansen]

folks ~

this isn't really a Perl problem, although i'll be implementing the  
solution in Perl.  i need to create a job queue for a long-running  
process.

i'm querying and updating Widgets.  normal case is to update all  
1,000 Widgets, which takes several hours.  special case is to update  
a small group of Widgets.

if one Widget process is running, no other Widget process should run  
at the same time (there are good and sufficient reasons for this).   
if someone wants to update another Widget[s], this request should be  
added to the queue.

roughly:
- Widget updater "A" begins
     - IF another Widget update ("B") is requested
     - request "B" is queued
- process "A" completes
- process "A" examines the queue
     - IF it finds request "B"
     - goto 1
- end

one suggestion i've had is to try to create a MySQL[1] heap table  
when the update process begins.  if the table does not already  
exists, the process (a) creates and locks it, (b) adds an entry for  
itself, (c) unlocks it.

if the table exists, and the PID of the first item in the queue  
exists, the process does the lock/add/unlock cycle, returns a "wait a  
bit" message, and exists cleanly.  if the PID of the first item does  
not exist, we assume that it's crashed, and pop it off.

after an update process completes, it removes itself from the queue  
and checks for a next entry.  if present, it configures itself as  
that next entry, updates the entry's PID, and continues.  if there's  
no next entry, it deletes the table.

the bugbear of this problem seems to be race conditions.  i could  
create a bunch of exception handling for the algorithm, but it seems  
like (a) that would get dirty very fast, and (b) be a PITA to test.

anyone have suggestions, or critiques of the algorithm and method  
i've presented?

TIA,

r

----
1) target DB is MySQL 4.0.18