SPUG: Question on fork() and pipe()

[Pommert, Daniel]

Sean,

It appears that you are reusing the file handles READER and WRITER in your
main process.  When you do that new pipe() command, it should close the
previous READER and WRITER handles.  This will close the pipe, sending an
error or eof to the child, (perhaps) causing it to terminate.  This would
explain your "serialization" behaviour.

I have done something very similar to this using Unix Datagrams.  As long as
your processes don't need to be run on a separate machine, they are
completely reliable (with one exception that you have to check for) and let
you have one loop waiting for messages and all of your children processes
sending that loop their messages.  All of the children's sockets are mapped
into the file system address space, so they can be stored textually -- you
don't have to keep open sockets.

I created a OO library that I used quite successfully a couple of years ago
that I could send to you out-of-band if you would like it.

Others on this list have had good luck using shared memory communications.
I know HP-UP Perl supports this.  I'm sure the Perl Cookbook can give you
examples of its use.

-- Daniel Pommert 
  Verizon Wireless
  425-603-8612


-----Original Message-----
From: Dwyer, Sean [mailto:sean.dwyer at attws.com]
Sent: Monday, May 13, 2002 9:22 AM
To: 'spug-list at pm.org'
Subject: SPUG: Question on fork() and pipe()


All, 

	I have run into a bind on a control structure I am working with.
Although I have a work around, I would rather use the pipe() solution so I
can pass more complex data back to the parent process. The test code is
below. Here is what is happening. The test sequence should spawn 5 children
and as they die off replace them to maintain a running count of 5 until the
whole list (in this case a simple counter totaling at 20) is complete.
Without the pipe() for communication between the children and the parent it
acts just as it is supposed to. When I add the pipe in the processes
serializes. 

	My workaround is to use the exit status to give partial information
back to the parent. But .. using the pipe allows me to pass more data back
to the parent for later processes. If I use exit status then I will need to
send the additional output to flat file so the parent can pick is up later
and I would rather not do that.

	I have noted that from the OS (HPUX 11.00) side that the parent goes
into a sleep state waiting on the pipe. Is there a way to leave the pipe
open so that the children can talk back to the parent and still parallelise?
If so what am I missing and how do I do it. Thanks in advance for any help.

Sean Dwyer 

Data Base Administrator
Customer Operations Support
National Data Center Operations
AT&T Wireless Services, Inc.
Bothell, Washington
Desk-425.288.7313
Cell-206.601.5465

#!/usr/local/perl5/bin/perl -w

use POSIX ":sys_wait_h";
use IO::Handle;
use strict;

$| = 1; 


my ($exit_val, $signal_num, $dumped_core);

$SIG{CHLD} = \&child_track; 
			    
my $DEBUG = 0;
my $counter = 0;
my $child = 0;


print "\n## The parent process is: $$ \n";

##
## I have set a for loop here to take the place of the array 
## that I really process in order to limit my number of iterations
##
for (my $i = 1; $i <= 20; $i++) {
  $counter++; # increase the counter because we are about to fork a child
  print "COUNTER: $counter  \n";
  &fork_child(); #fork the child passing in a vaiable
  while ($counter >= 5) {
    sleep 1; #sleep the processes so that we don't spin up the CPU
    while ((my $child = waitpid(-1, -2)) > 0){1;} #wait on 1, any 1 child 
                                                  #to exit
    }
  }

waitpid(-1, 0); # wait on all children to exit before ending.


sub fork_child {
  FORK: {

    pipe(READER, WRITER);
    WRITER->autoflush(1);

    if (my $pid = fork()) { # parent here
      close WRITER;
      chomp(my $line = <READER>);
      print "There Parent pid $$ just read this: '$line'\n";
      close READER;
      next if $child == 1; # Do not fork a child from a child
      } 
    elsif (defined $pid) { # $pid is zero here if defined
                           # child here
      $child = 1; 

      sleep int(rand 10); #this is where the real work would go
                          #randomize the sleep so the children don't
                          #all exit at once.
            
      close READER;
      print WRITER "Child pid $$ is sending this";
      close WRITER;

      exit 100;
      } 
    elsif ($! =~ /No more process/) { # EAGAIN, supposedly recoverable 
                                      # fork error
      sleep 5;
      redo FORK;
      } 
    else { # weird fork error
      die "Can't fork: $!\n";
      }
    }
  }

sub child_track {
  my $pid = waitpid(-1, 0);
  if ($pid == -1){return;} #no children have exited
  elsif (WIFEXITED($?)) {
    $exit_val = $? >> 8;
    $signal_num = $? & 127;
    $dumped_core = $? & 128;
#    print "\$pid = $pid\n";
#    print "\$signal_num = $signal_num\n";
#    print "\$exit_val = $exit_val\n";
#    print "\$dumped_core = $dumped_core\n";
    print "Process exited\n";
    $counter --; #decrease the counter so that you can spawn another
processes
    return;
    }
  else {print "False alarm on $pid";} #any other values are considered false

  $SIG{CHLD} = \&child_track; #re-install the handler 
  }


 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
     POST TO: spug-list at pm.org       PROBLEMS: owner-spug-list at pm.org
      Subscriptions; Email to majordomo at pm.org:  ACTION  LIST  EMAIL
  Replace ACTION by subscribe or unsubscribe, EMAIL by your Email-address
 For daily traffic, use spug-list for LIST ;  for weekly, spug-list-digest
     Seattle Perl Users Group (SPUG) Home Page: http://seattleperl.org

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
     POST TO: spug-list at pm.org       PROBLEMS: owner-spug-list at pm.org
      Subscriptions; Email to majordomo at pm.org:  ACTION  LIST  EMAIL
  Replace ACTION by subscribe or unsubscribe, EMAIL by your Email-address
 For daily traffic, use spug-list for LIST ;  for weekly, spug-list-digest
     Seattle Perl Users Group (SPUG) Home Page: http://seattleperl.org