[Chicago-talk] Spring Integration Framework for Perl
nfulton at carthage.edu
Mon Feb 4 11:44:56 PST 2013
I was waiting for someone to reply before contacting you off-list. Have
you found a reasonable solution?
I have something that probably does some of what you want. I've used it as
a platform for some academic-y cluster computing stuff over the past two
Unfortunately, calling it a prototype is a bit of a stretch, and in the
short term you're better off rolling your own. However, if there's nothing
on CPAN that does what you want, I'll start cleaning up my code and think
about deploying at some point in the future.
There's an early description of the system's ancestor here:
Everything's been rewritten a few times, but that gives you a sense for the
The regulator is fairly sophisticated, and the pipeline implementation lets
you do all sorts of useful stuff (cyclic dependencies, eager vs lazy
scheduling, introspection and modification of the pipeline by jobs,
conditional dependencies, etc.)
There's also a reporting mechanism. I've used it to play around with stuff
like caching, memoization for distributed dynamic programming , and
preemptive resource allocation, so it's pretty robust.
I had distributed computing in mind, but backends are decoupled and I could
create a backend using processes instead of a cluster. The whole point is
to provide an easy way to build custom concurrency models.
Anyways. Like I said, I probably won't be able to clean it up in time to
give you something useful, and you're better off spending your time on the
task at hand. But if there's no useful tool out there for
implementing/using concurrency frameworks, I'll clean it up and deploy.
On Wed, Jan 30, 2013 at 2:59 PM, Doug Bell <madcityzen at gmail.com> wrote:
> I've got a number of processes being coordinated by a message queue.
> Each job consists of a number of steps, and each process handles one
> specific step for each job, like an assembly line. Each process keeps
> data about the current job run and puts it on a queue so that the next
> process knows what it has to do.
> I have an idea to build a framework around this so that I can add
> in-depth reporting and more flexibility, but it seems there are
> already frameworks in other languages that do exactly this (a
> co-worker pointed me to Spring's Integration Framework in Java).
> This is not a way to get parallelization, but it is a way to manage
> parallelization or delayed jobs (if a job has to wait for human input,
> for example) and provide reporting and monitoring on all the currently
> running jobs.
> Is there anything existing on CPAN to do this?
> Doug Bell
> madcityzen at gmail.com
> Chicago-talk mailing list
> Chicago-talk at pm.org
CS Service Fellow
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Chicago-talk