[Chicago-talk] Script design question

[Randal L. Schwartz]

>>>>> "Darren" == Young, Darren <Darren.Young at ChicagoGSB.edu> writes:

Darren> If that doesn't make sense, here's what it's for. In order to run
Darren> backups on our mail system we have to use a Sun supplied binary
Darren> (imsbackup). As an argument this binary takes what are called "groups"
Darren> to perform a given backup. Those "groups" are defined in another file
Darren> (backup-groups.conf) and contains lines such as:
Darren>    groupA=a*
Darren>    groupB=b*

Darren> And so on and so forth until the letter Z. groupA are users that start
Darren> with the letter a, groupB are the b's, and so on. Now, since these
Darren> backups take so long to run we're going to run 4 of them in parallel,
Darren> each one against a different group. So, from the command line I would:
Darren>    imsbackup -i -f- /gsbims/groupA > /export/backups/groupA.bkp &
Darren>    imsbackup -i -f- /gsbims/groupB > /export/backups/groupB.bkp &

Darren> Do that for groups A-D and let them run. Then, the first one that
Darren> completed would fall off the "to-do" list and the next letter group will
Darren> be started (E in this case). Then on to F, G, etc. Testing so far says
Darren> that an individual group can take up to 2-3 hours to complete so I have
Darren> to be able to deal with processes that sit and run for a while and
Darren> produce no output. Not sure what exit codes the imsbackup program gives
Darren> back, but I'm betting it's just a 0 or 1.

I know it's not Perl, but in the past, I've done something like this:

Set up a GNU Makefile that looks like:

    OUTPUTS= $(patsubst %, /export/backups/group%.bkp, A B C D E F [upto] Z)
    all: $(OUTPUTS)
    /export/backups/%.bkp:
            imsbackup -i -f- /gsbims/$* >$@

Then execute this with "make -j4 all" (might need gmake or gnumake).

The -j4 will keep 4 jobs running at all times, going on to the next
one as soon as one finishes.  Works real nice.

GNU make can also work off loadaverage instead, if that's your real concern.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn at stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!