APM: Austin Digest, Vol 44, Issue 3
Heath Malmstrom
hmalmstrom at gmail.com
Sat Feb 10 15:17:26 PST 2007
Sam,
Since this is not going to be generating high traffic (not needing to
fork off alot),
you might just try Win32::Process::Create(). We have a product that does
this
for each running build on windows and then round robins across non-blocking
sockets
within each process in order to avoid forking a bunch of times. Note that
this is tricky
and you have to watch out for deadlocks or a single socket over-consuming
all the processing time.
Our product is used for kicking off builds and maintaining information
about it in a web UI.
It's very flexible, and written in Perl / PHP / C with Eclipse and Visual
studio plugins, the downside is it's a bit expensive if this is just a side
thing.
Here's a link to one of our open-source competitors (not nearly the
feature set and written in java it would appear, but it looks interesting):
http://luntbuild.javaforge.com/
--Heath
On 2/9/07, austin-request at pm.org <austin-request at pm.org> wrote:
>
> Send Austin mailing list submissions to
> austin at pm.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://mail.pm.org/mailman/listinfo/austin
> or, via email, send a message with subject or body 'help' to
> austin-request at pm.org
>
> You can reach the person managing the list at
> austin-owner at pm.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Austin digest..."
>
>
> Today's Topics:
>
> 1. fork on windows / long-running process and cgi (Sam Foster)
> 2. Re: fork on windows / long-running process and cgi
> (Montgomery Conner)
> 3. Re: fork on windows / long-running process and cgi (Sam Foster)
> 4. Re: fork on windows / long-running process and cgi
> (Taylor Carpenter)
> 5. Re: fork on windows / long-running process and cgi (Tim McDaniel)
> 6. Re: fork on windows / long-running process and cgi (Sam Foster)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 08 Feb 2007 15:37:11 -0600
> From: Sam Foster <austin.pm at sam-i-am.com>
> Subject: APM: fork on windows / long-running process and cgi
> To: "austin.pm" <austin at pm.org>
> Message-ID: <45CB9807.6030509 at sam-i-am.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> so what is the skinny with fork on windows? No, not ever, yes with some
> workarounds, yes with a particular distribution of perl for win32s?
>
> I'm trying to put a web front-end on a build process - that may take a
> while. Ideally my initial request would kick off the build, and later
> requests would check on its status and finally pick up the output and
> send it or a link to it back to the client.
> I'm developing on and will ultimately be deploying to a windows xp box
> with apache and activestate's perl.
>
> So, it would seem that I should fork to kick the process off, send back
> a "working..." message as a response and close stdout in the parent
> process.
> The child process goes on working, putting output from the build script
> into a file.
>
> The client would keep polling every few seconds to check status, and
> grab and respond with the build script output, which will finally
> include a link to where the user can download a zip of the output.
>
> This is just for internal usage, and wont get high traffic. Concurrent
> requests are possible, but honestly quite unlikely. In fact, depending
> on the load on the server, the build usually finishes inside a minute so
> I could probably keep it simple and just sit and wait for the response -
> but I'd like to know what my options are. I do need input (config data)
> from the user so I cant just schedule this and run it nightly or
> something.
>
> Oh, and this is supposed to be a fun diversion from my real job, so if
> I'm trying to make it too complicated that's why :)
> thanks for any thoughts
> Sam
>
>
>
> ------------------------------
>
> Message: 2
> Date: Thu, 8 Feb 2007 18:45:02 -0600
> From: "Montgomery Conner" <montgomery.conner at gmail.com>
> Subject: Re: APM: fork on windows / long-running process and cgi
> To: "Sam Foster" <austin.pm at sam-i-am.com>
> Cc: "austin.pm" <austin at pm.org>
> Message-ID:
> <e8c2cfe90702081645n299689ceg7113100323c41dac at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> As I understand Perl's fork() corresponds to the OS system call for *nix
> distros... since there is no equivalent system call on Windows, fork()
> will
> not work. The ActiveState
> <http://www.xav.com/perl/lib/Pod/perlfork.html>site includes some
> information on fork() emulation for Windows:
>
> The fork() <http://www.xav.com/perl/lib/Pod/perlfunc.html#item_fork
> >emulation
> is implemented at the level of the Perl interpreter. What this
> means in general is that running
> fork()<http://www.xav.com/perl/lib/Pod/perlfunc.html#item_fork>will
> actually clone the running interpreter and all its state, and run the
> cloned interpreter in a separate thread, beginning execution in the new
> thread just after the point where the
> fork()<http://www.xav.com/perl/lib/Pod/perlfunc.html#item_fork>was
> called in the parent.
>
> I would assume that using this mechanism will incur the memory hit that
> comes with duplicating the entire Perl interpreter for each call to
> fork().
> You would have to test this to be certain...
>
>
> Have you worked with the POE multitasking framework before? I've found
> that
> it is a useful (and fun!) alternative when I need a multitasking
> application
> on Windows (or ever!). By writing a simple web-server in
> POE<http://search.cpan.org/search?query=poe&mode=all>(a truly trivial
> task) you'd have a multitasking non-blocking application
> that could listen and delegate appropriately. Check-out the cookbook at
> the
> project homepage for some ideas: http://poe.perl.org/
>
>
> Good Luck!
> Montgomery
>
>
> On 2/8/07, Sam Foster <austin.pm at sam-i-am.com> wrote:
> >
> > so what is the skinny with fork on windows? No, not ever, yes with some
> > workarounds, yes with a particular distribution of perl for win32s?
> >
> > I'm trying to put a web front-end on a build process - that may take a
> > while. Ideally my initial request would kick off the build, and later
> > requests would check on its status and finally pick up the output and
> > send it or a link to it back to the client.
> > I'm developing on and will ultimately be deploying to a windows xp box
> > with apache and activestate's perl.
> >
> > So, it would seem that I should fork to kick the process off, send back
> > a "working..." message as a response and close stdout in the parent
> > process.
> > The child process goes on working, putting output from the build script
> > into a file.
> >
> > The client would keep polling every few seconds to check status, and
> > grab and respond with the build script output, which will finally
> > include a link to where the user can download a zip of the output.
> >
> > This is just for internal usage, and wont get high traffic. Concurrent
> > requests are possible, but honestly quite unlikely. In fact, depending
> > on the load on the server, the build usually finishes inside a minute so
> > I could probably keep it simple and just sit and wait for the response -
> > but I'd like to know what my options are. I do need input (config data)
> > from the user so I cant just schedule this and run it nightly or
> > something.
> >
> > Oh, and this is supposed to be a fun diversion from my real job, so if
> > I'm trying to make it too complicated that's why :)
> > thanks for any thoughts
> > Sam
> >
> > _______________________________________________
> > Austin mailing list
> > Austin at pm.org
> > http://mail.pm.org/mailman/listinfo/austin
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.pm.org/pipermail/austin/attachments/20070208/46f84c4e/attachment.htm
>
> ------------------------------
>
> Message: 3
> Date: Thu, 08 Feb 2007 20:24:33 -0600
> From: Sam Foster <austin.pm at sam-i-am.com>
> Subject: Re: APM: fork on windows / long-running process and cgi
> To: "austin.pm" <austin at pm.org>
> Message-ID: <45CBDB61.2060201 at sam-i-am.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>
> >
> > Have you worked with the POE multitasking framework before? I've
> > found that it is a useful (and fun!) alternative when I need a
> > multitasking application on Windows (or ever!). By writing a simple
> > web-server in POE <http://search.cpan.org/search?query=poe&mode=all>
> > (a truly trivial task) you'd have a multitasking non-blocking
> > application that could listen and delegate appropriately. Check-out
> > the cookbook at the project homepage for some ideas:
> http://poe.perl.org/
> >
> See, that's what I'm talking about with needlessly over-complicating
> things :) Ok, I'll bite, been wanting to look at POE for a while now
> anyhow
>
> Sam
>
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 8 Feb 2007 23:43:56 -0600
> From: Taylor Carpenter <taylor at codecafe.com>
> Subject: Re: APM: fork on windows / long-running process and cgi
> To: "austin.pm" <austin at pm.org>
> Message-ID: <1060765E-723C-443B-B20F-732EBE02498F at codecafe.com>
> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>
> An alternative to POE and your normal thoughts on using fork() is to
> sharing the information about the current build state "outside of the
> CGI app". You can have some CGI app start the build based on certain
> input (startbuild=1 or whatever) then another script (ot it called
> differently) can check on the build state (checkbuild=1). When the
> build is finished instead of showing current status it redirects to a
> page that shows full status of the build and the link etc.
>
> You have many options including
>
> simple text file that you read from
> DB (sqlite would be easy)
> memcache (http://search.cpan.org/~bradfitz/Cache-Memcached-1.18/ -
> http://www.danga.com/memcached)
>
> Randal L. Schwartz has an article on watching long running processes
> through CGI using Cache::Cache and Apache::Session
>
> http://www.stonehenge.com/merlyn/LinuxMag/col39.html
>
> You mentioned having config data from the user. You could also
> create a file in a directory and check that "incoming" directory for
> the config information. It would remove the file and start the
> build. That could be an app that is always running or something
> setup in Windows scheduler. Another CGI would show the status of any
> running builds or finished builds. That information can be stored
> and various ways including those I mentioned previously.
>
> On Feb 8, 2007, at 3:37 PM, Sam Foster wrote:
>
> > so what is the skinny with fork on windows? No, not ever, yes with
> > some
> > workarounds, yes with a particular distribution of perl for win32s?
> >
> > I'm trying to put a web front-end on a build process - that may take a
> > while. Ideally my initial request would kick off the build, and later
> > requests would check on its status and finally pick up the output and
> > send it or a link to it back to the client.
>
>
> ------------------------------
>
> Message: 5
> Date: Thu, 8 Feb 2007 23:56:57 -0600 (CST)
> From: Tim McDaniel <tmcd at panix.com>
> Subject: Re: APM: fork on windows / long-running process and cgi
> Cc: "austin.pm" <austin at pm.org>
> Message-ID: <Pine.NEB.4.64.0702082353580.1305 at panix3.panix.com>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
> On Feb 8, 2007, at 3:37 PM, Sam Foster wrote:
> > I'm trying to put a web front-end on a build process - that may take
> > a while. Ideally my initial request would kick off the build, and
> > later requests would check on its status and finally pick up the
> > output and send it or a link to it back to the client.
>
> "Micro-optimizations yield micro-results." You will do builds what,
> several times a day, and check their statuses a few dozen times a day?
> Unless fork() simply didn't work (and it does work, in my experience
> with Cygwin's perl at least), I don't see why you'd bother to avoid
> it. Am I missing something?
>
> Daniel de Lyncoln
> --
> Tim McDaniel, tmcd at panix.com
>
>
> ------------------------------
>
> Message: 6
> Date: Fri, 09 Feb 2007 13:21:21 -0600
> From: Sam Foster <austin.pm at sam-i-am.com>
> Subject: Re: APM: fork on windows / long-running process and cgi
> To: "austin.pm" <austin at pm.org>
> Message-ID: <45CCC9B1.4030506 at sam-i-am.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
>
> > "Micro-optimizations yield micro-results." You will do builds what,
> > several times a day, and check their statuses a few dozen times a day?
> > Unless fork() simply didn't work (and it does work, in my experience
> > with Cygwin's perl at least), I don't see why you'd bother to avoid
> > it. Am I missing something?
> >
> >
> I had made a first pass at this and ran into problems which seemed to
> center right around where my script was supposed to be forking. Rather
> than bang my head against it I thought I'd check if this is even
> supposed to work, or what other options there are. Taylor is right
> though that I really dont need a fork, I just need to kick off the
> process, hang up with a response (I just close stdout right?) and let a
> future request check on the status. I'm already persisting some info
> about the build to a file so I could add build status in there.
>
> Tim: all you are missing is that in truth I dont know what I'm doing, so
> my uncertainty is not about optimization, more about learning good
> patterns that will serve me in the future.
>
> Sam
>
>
> ------------------------------
>
> _______________________________________________
> Austin mailing list
> Austin at pm.org
> http://mail.pm.org/mailman/listinfo/austin
>
> End of Austin Digest, Vol 44, Issue 3
> *************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.pm.org/pipermail/austin/attachments/20070210/07189195/attachment-0001.html
More information about the Austin
mailing list