[pm-h] Workflow scheduler solution
todd.e.rinaldo at jpmorgan.com
todd.e.rinaldo at jpmorgan.com
Fri Apr 11 09:42:07 PDT 2008
All,
As promised from this month's meeting, I am providing the solution to my problem we.
Background: I have a need to run jobs, but some of the jobs have a need to depend on multiple things completing. At present, we have to run them in a linear fashion or leverage ugly sleep commands to assure (hopefully) everything has run before the next command can be run.
Issue: Provide a SQL approach to create a workflow process to deal with multiple parent dependencies. This way you can just query the DB to determine which jobs that have not run yet have met their dependencies and are ready to run.
SQL to create tables: (This is all sybase by the way. Apologies!)
CREATE TABLE dbo.Scheduler
(
workflow_id numeric(20,0) NOT NULL,
workflow_step smallint NOT NULL,
start_time datetime NULL,
end_time datetime NULL,
hostname varchar(4096) NOT NULL,
command varchar(4096) NOT NULL,
status smallint DEFAULT -1 NOT NULL,
pid int DEFAULT 0 NOT NULL,
log image DEFAULT "empty" NULL,
CONSTRAINT Scheduler_workflow_step
UNIQUE NONCLUSTERED (workflow_id,workflow_step)
)
LOCK ALLPAGES
go
IF OBJECT_ID('dbo.Scheduler') IS NOT NULL
PRINT '<<< CREATED TABLE dbo.Scheduler >>>'
ELSE
PRINT '<<< FAILED CREATING TABLE dbo.Scheduler >>>'
go
CREATE TABLE dbo.SchedulerDepends
(
workflow_id numeric(20,0) NOT NULL,
workflow_step smallint NOT NULL,
depends_on smallint NOT NULL,
status_required smallint NOT NULL,
CONSTRAINT SchedulerD_10261517202
UNIQUE NONCLUSTERED (workflow_id,workflow_step,depends_on)
)
LOCK ALLPAGES
go
IF OBJECT_ID('dbo.SchedulerDepends') IS NOT NULL
PRINT '<<< CREATED TABLE dbo.SchedulerDepends >>>'
ELSE
PRINT '<<< FAILED CREATING TABLE dbo.SchedulerDepends >>>'
go
------
-- SQL to find out what's ready to run.
select s1.hostname, s1.command
from dbo.Scheduler s1
where s1.start_time is null -- figure out what hasn't started yet
and not exists( -- figure out what dependencies s1.status has. no rows means it's ready to go.
select 1
from dbo.Scheduler s2, -- used to check the state of jobs s1 is depending on.
dbo.SchedulerDepends d -- used to determine which jobs s1 depends on
where s2.status != d.status_required -- Is the job in the right state that's required?
and s2.workflow_step = d.depends_on -- find all
and d.workflow_step = s1.workflow_step -- Find all the dependencies for s1 row.
and s1.workflow_id = s2.workflow_id -- Assure everything's using the same workflow
and s1.workflow_id = d.workflow_id -- Assure everything's using the same workflow
)
-- SQL to find out what is currently running.
select * from dbo.Scheduler s1
where start_time is not null
and end_time is null
Generally, this communication is for informational purposes only
and it is not intended as an offer or solicitation for the purchase
or sale of any financial instrument or as an official confirmation
of any transaction. In the event you are receiving the offering
materials attached below related to your interest in hedge funds or
private equity, this communication may be intended as an offer or
solicitation for the purchase or sale of such fund(s). All market
prices, data and other information are not warranted as to
completeness or accuracy and are subject to change without notice.
Any comments or statements made herein do not necessarily reflect
those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to UK legal entities.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.pm.org/mailman/private/houston/attachments/20080411/608f962b/attachment-0001.html
More information about the Houston
mailing list