From rkleeman at energoncube.net Tue Dec 5 15:31:44 2006 From: rkleeman at energoncube.net (Bob Kleemann) Date: Tue, 5 Dec 2006 15:31:44 -0800 Subject: [San-Diego-pm] Meeting Next Week. Message-ID: <20061205233144.GG22248@energoncube.net> Just a reminder folks, our monthly meeting will be taking place on Monday, December 11. Bring your friends, questions, and such to the meeting and we'll have chat. Also, we will be talking about next year: Do we still want meetings on the second Monday of the month? Would more people be able to come if we shifted things aronud a little bit? Let me know. From joel at fentin.com Tue Dec 5 15:50:09 2006 From: joel at fentin.com (Joel Fentin) Date: Tue, 05 Dec 2006 15:50:09 -0800 Subject: [San-Diego-pm] Meeting Next Week. In-Reply-To: <20061205233144.GG22248@energoncube.net> References: <20061205233144.GG22248@energoncube.net> Message-ID: <457605B1.30002@fentin.com> Bob Kleemann wrote: > Just a reminder folks, our monthly meeting will be taking place on Monday, > December 11. Bring your friends, questions, and such to the meeting and > we'll have chat. I'm probably in danger of showing up. > Also, we will be talking about next year: Do we still want meetings on > the second Monday of the month? Would more people be able to come if > we shifted things aronud a little bit? Let me know. No preference. -- Joel Fentin tel: 760-749-8863 FAX: 760-749-8864 Email: http://fentin.com/me/ContactMe.html Biz Website: http://fentin.com Personal Website: http://fentin.com/me From rkleeman at energoncube.net Mon Dec 11 15:29:19 2006 From: rkleeman at energoncube.net (Bob Kleemann) Date: Mon, 11 Dec 2006 15:29:19 -0800 Subject: [San-Diego-pm] Meeting tonight! Message-ID: <20061211232919.GC670@energoncube.net> Just a reminder folks, we have our monthly meeting tonight at the Panera Bread on Mira Mesa Blvd. Bring your questions, comments, concerns, querries, and job offers and we'll talk about them tonight. I hope to see you all there at 7PM. From emileaben at gmail.com Mon Dec 11 22:48:36 2006 From: emileaben at gmail.com (Emile Aben) Date: Mon, 11 Dec 2006 22:48:36 -0800 Subject: [San-Diego-pm] Module idea for input/output multiplexing Message-ID: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> Hi, Module idea from discussion at the meeting tonight: A module that hooks up stdout from one process with the stdin of several other processes, or maybe more generic: Something that reads from a filehandle and writes that output to several other filehandles. Simple example of use: reading a gzipped file from disk, and calculating the md5 (using md5, md5sum or 'openssl md5' command, depending on your OS) and linecount (wc -l) ** I've looked into IPC::Run a bit, and IPC::Run might be able to do something like this, but the best I've been able to come up with this far doesn't work: ---- #!/usr/bin/env perl use Carp; use strict; use warnings; use IPC::Run qw(run); my $cat_cmd = ['cat']; my $md5_cmd = ['/usr/bin/openssl','md5']; my $wc_cmd = ['wc','-l']; my $md5_out = undef; my $wc_out = undef; run($cat_cmd,'<',"$0", '>&3', '>&4', '3>pipe', $md5_cmd, '>', \$md5_out, '4>pipe', $wc_cmd, '>', \$wc_out); print "linecount: $wc_out\n"; --- which produces errors: Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.3/IPC/Run/IO.pm line 329. Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.3/IPC/Run/IO.pm line 329. or under perl -d: IPC::Run::start('ARRAY(0x98ee6a8)','<','./ipcrun.pl','>&3','>&4','3>pipe','ARRAY(0x98ee714)','>','SCALAR(0x97efce0)',...) called at /usr/lib/perl5/site_perl/5.8.3/IPC/Run.pm line 1434 IPC::Run::run('ARRAY(0x98ee6a8)','<','./ipcrun.pl','>&3','>&4','3>pipe','ARRAY(0x98ee714)','>','SCALAR(0x97efce0)',...) called at ./ipcrun.pl line 14 that's enough for me tonight. Emile ** both calculating md5 and linecount can be done in perl as well of course, but that's not the point here From menolly at mib.org Mon Dec 11 22:54:28 2006 From: menolly at mib.org (Menolly) Date: Mon, 11 Dec 2006 22:54:28 -0800 (PST) Subject: [San-Diego-pm] Module idea for input/output multiplexing In-Reply-To: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> References: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> Message-ID: So...a cross-platform tee, basically? On Mon, 11 Dec 2006, Emile Aben wrote: > Hi, > > Module idea from discussion at the meeting tonight: > A module that hooks up stdout from one process with the stdin of > several other processes, > or maybe more generic: Something that reads from a filehandle and > writes that output to > several other filehandles. > > Simple example of use: > reading a gzipped file from disk, and calculating the md5 (using md5, > md5sum or 'openssl md5' command, depending on your OS) and linecount > (wc -l) ** > > I've looked into IPC::Run a bit, and IPC::Run might be able to do > something like this, but > the best I've been able to come up with this far doesn't work: > ---- > #!/usr/bin/env perl > use Carp; > use strict; > use warnings; > use IPC::Run qw(run); > > my $cat_cmd = ['cat']; > my $md5_cmd = ['/usr/bin/openssl','md5']; > my $wc_cmd = ['wc','-l']; > > my $md5_out = undef; > my $wc_out = undef; > > run($cat_cmd,'<',"$0", '>&3', '>&4', > '3>pipe', $md5_cmd, '>', \$md5_out, > '4>pipe', $wc_cmd, '>', \$wc_out); > > print "linecount: $wc_out\n"; > --- > which produces errors: > Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.3/IPC/Run/IO.pm line 329. > Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.3/IPC/Run/IO.pm line 329. > > or under perl -d: > IPC::Run::start('ARRAY(0x98ee6a8)','<','./ipcrun.pl','>&3','>&4','3>pipe','ARRAY(0x98ee714)','>','SCALAR(0x97efce0)',...) > called at /usr/lib/perl5/site_perl/5.8.3/IPC/Run.pm line 1434 > IPC::Run::run('ARRAY(0x98ee6a8)','<','./ipcrun.pl','>&3','>&4','3>pipe','ARRAY(0x98ee714)','>','SCALAR(0x97efce0)',...) > called at ./ipcrun.pl line 14 > > that's enough for me tonight. > > Emile > > > ** both calculating md5 and linecount can be done in perl as well of > course, but that's not the point here > _______________________________________________ > San-Diego-pm mailing list > San-Diego-pm at pm.org > http://mail.pm.org/mailman/listinfo/san-diego-pm > -- menolly at mib.org http://www.livejournal.com/~nolly/ On that day, many will say to me, "Lord, Lord, did we not prophesy in your name, and cast out demons in your name, and do many mighty works in your name?" And then will I declare to them, "I never knew you; depart from me you evildoers." -- Matt 7:20-23, RSV From emileaben at gmail.com Tue Dec 12 07:18:41 2006 From: emileaben at gmail.com (Emile Aben) Date: Tue, 12 Dec 2006 07:18:41 -0800 Subject: [San-Diego-pm] Module idea for input/output multiplexing In-Reply-To: References: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> Message-ID: <28e00a750612120718q6e0775d5x7ba5a6963e689658@mail.gmail.com> On 12/11/06, Menolly wrote: > So...a cross-platform tee, basically? yep, except 'tee' works with files as output. Btw. using 'tee' combined with fifo's (I think that was Bob's idea), seems to work nicely in shell: ---- #!/usr/bin/env bash FIFO1="/tmp/fifo.$$.1" FIFO2="/tmp/fifo.$$.2" mkfifo $FIFO1 mkfifo $FIFO2 ( cat $0 | tee $FIFO1 $FIFO2 > /dev/null )& ( openssl md5 < $FIFO1 | xargs echo md5: )& ( wc -l < $FIFO2 | xargs echo wordcount: )& wait; rm $FIFO1 rm $FIFO2 ------ Emile > > On Mon, 11 Dec 2006, Emile Aben wrote: > > > Hi, > > > > Module idea from discussion at the meeting tonight: > > A module that hooks up stdout from one process with the stdin of > > several other processes, > > or maybe more generic: Something that reads from a filehandle and > > writes that output to > > several other filehandles. > > > > Simple example of use: > > reading a gzipped file from disk, and calculating the md5 (using md5, > > md5sum or 'openssl md5' command, depending on your OS) and linecount > > (wc -l) ** > > > > I've looked into IPC::Run a bit, and IPC::Run might be able to do > > something like this, but > > the best I've been able to come up with this far doesn't work: > > ---- > > #!/usr/bin/env perl > > use Carp; > > use strict; > > use warnings; > > use IPC::Run qw(run); > > > > my $cat_cmd = ['cat']; > > my $md5_cmd = ['/usr/bin/openssl','md5']; > > my $wc_cmd = ['wc','-l']; > > > > my $md5_out = undef; > > my $wc_out = undef; > > > > run($cat_cmd,'<',"$0", '>&3', '>&4', > > '3>pipe', $md5_cmd, '>', \$md5_out, > > '4>pipe', $wc_cmd, '>', \$wc_out); > > > > print "linecount: $wc_out\n"; > > --- > > which produces errors: > > Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.3/IPC/Run/IO.pm line 329. > > Not a GLOB reference at /usr/lib/perl5/site_perl/5.8.3/IPC/Run/IO.pm line 329. > > > > or under perl -d: > > IPC::Run::start('ARRAY(0x98ee6a8)','<','./ipcrun.pl','>&3','>&4','3>pipe','ARRAY(0x98ee714)','>','SCALAR(0x97efce0)',...) > > called at /usr/lib/perl5/site_perl/5.8.3/IPC/Run.pm line 1434 > > IPC::Run::run('ARRAY(0x98ee6a8)','<','./ipcrun.pl','>&3','>&4','3>pipe','ARRAY(0x98ee714)','>','SCALAR(0x97efce0)',...) > > called at ./ipcrun.pl line 14 > > > > that's enough for me tonight. > > > > Emile > > > > > > ** both calculating md5 and linecount can be done in perl as well of > > course, but that's not the point here > > _______________________________________________ > > San-Diego-pm mailing list > > San-Diego-pm at pm.org > > http://mail.pm.org/mailman/listinfo/san-diego-pm > > > > -- > menolly at mib.org http://www.livejournal.com/~nolly/ > > On that day, many will say to me, "Lord, Lord, did we not prophesy in > your name, and cast out demons in your name, and do many mighty works > in your name?" And then will I declare to them, "I never knew you; > depart from me you evildoers." -- Matt 7:20-23, RSV > From chris at chrisgrau.com Wed Dec 13 00:42:44 2006 From: chris at chrisgrau.com (Chris Grau) Date: Wed, 13 Dec 2006 00:42:44 -0800 Subject: [San-Diego-pm] Module idea for input/output multiplexing In-Reply-To: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> References: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> Message-ID: <20061213084244.GS5523@chrisgrau.com> On Mon, Dec 11, 2006 at 10:48:36PM -0800, Emile Aben wrote: > Module idea from discussion at the meeting tonight: > A module that hooks up stdout from one process with the stdin of > several other processes, or maybe more generic: Something that reads > from a filehandle and writes that output to several other filehandles. I like the tee(1) example in shell, but I was itching to write it in Perl. So here's what I came up with in the last half hour or so. --[ print.pl ]---------------------------------------------------------- #!/usr/bin/perl print "$ARGV[0]: $_" while ; --[ __END__ ]----------------------------------------------------------- I only wrote print.pl because cat(1) buffers its output and so didn't serve my example very well. --[ multi.pl ]---------------------------------------------------------- #!/usr/bin/perl use strict; use warnings; my $proc = Proc::Multiplex->new; $proc->add('./print.pl one'); $proc->add('./print.pl two'); # sleep between writes to prove I/O isn't being buffered $proc->write("foo\n"); sleep 1; $proc->write("bar\n"); sleep 1; $proc->write("baz\n"); package Proc::Multiplex; sub new { bless { fh => [] }; } sub add { my ( $self, $cmd ) = @_; open my $fh, '|-', $cmd or die "$cmd: $!"; my $oldfh = select $fh; $| = 1; select $oldfh; push @{ $self->{fh} }, $fh; } sub write { my ( $self, $data ) = @_; print {$_} $data for @{ $self->{fh} }; } --[ __END__ ]----------------------------------------------------------- [cgrau at quendor ~]$ perl multi.pl two: foo one: foo one: bar two: bar one: baz two: baz Now it just needs a pinch of syntactic sugar and a dash of robustness. Though, odds are something like this already exists on the CPAN. -chris -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://mail.pm.org/pipermail/san-diego-pm/attachments/20061213/8e78da18/attachment.bin From dan at tierra.net Wed Dec 13 11:23:25 2006 From: dan at tierra.net (Daniel Risse) Date: Wed, 13 Dec 2006 11:23:25 -0800 Subject: [San-Diego-pm] Module idea for input/output multiplexing In-Reply-To: <20061213084244.GS5523@chrisgrau.com> References: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> <20061213084244.GS5523@chrisgrau.com> Message-ID: <4580532D.6030303@tierra.net> I thought I recalled this form the Perl Cookbook (2nd edition). Sure enough, there is an example showing a few ways to accomplish this. It's on page 260 for those of you with the book. If you want to do it without forking, use a foreach loop foreach $fh (@FHS) { print {$fh} $stuff; } If you don't mind forking, open a filehandle that's a pipe to tee. open my $multi, '| tee file1 file2 file3 > /dev/null' or die $!; print {$multi} $stuff; close $multi; Or you can use IO::Tee from CPAN use IO::Tee; my $tee = IO::Tee->new(@FHS); print {$tee} $stuff; close $tee; I think Chris's Proc::Multiplex is a cleaner interface though. Dan From cabney at ucsd.edu Wed Dec 13 11:48:46 2006 From: cabney at ucsd.edu (Charles Abney) Date: Wed, 13 Dec 2006 11:48:46 -0800 Subject: [San-Diego-pm] Module idea for input/output multiplexing In-Reply-To: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> References: <28e00a750612112248i4cb37c95sa51f2387bc204cf8@mail.gmail.com> Message-ID: <4580591E.8000604@ucsd.edu> Emile Aben wrote: > use IPC::Run qw(run); > > my $cat_cmd = ['cat']; Aren't you just passing a listref here (and elsewhere)? I don't know that Run will know to dereference that. Yours, Charles -- Charles Abney Polymorphism Research Laboratory, 0658 Moores Cancer Center, UCSD School of Medicine 9500 Gilman Dr. La Jolla, CA 92093-0658