[Chicago-talk] open3 and passing large amounts of data

Zane C.B. v.velox at vvelox.net
Mon Nov 17 18:41:03 PST 2008


On Mon, 17 Nov 2008 20:00:43 -0600
Andrew Rodland <arodland at comcast.net> wrote:

> On Monday 17 November 2008 07:49:13 pm Zane C.B. wrote:
> > Any one ever run into a issue before when passing a large amount
> > of data through a pipe?
> >
> > I am currently trying to print '1022163' bytes to the STDIN spamc,
> > but it is never finishing.
> >
> > What I found I had to do is set spamc's max size arguement to
> > what my incoming mail server to something large or as large as
> > what my mail server allows for. Which is odd as spamc does not
> > hang if I cat something that large through it.
> 
> It's a deadlock. spamc stopped reading its input and sent you a
> message that went something along the lines of "whoa, bucko, that's
> way too much data!", but your app didn't receive that message
> because it was too busy trying to stuff data through the pipe going
> the other way, and wasn't checking for input. So each end was
> blocked waiting on the other. To avoid problems like that, you need
> to use nonblocking I/O plus a select loop or POE or Event or
> something like that, or threads (but Perl isn't very
> thread-friendly, so forget I mentioned that one).

I've don't think threading would help with this.

Thanks. I will have to look into doing non-blocking IO then.

I am still a bit lost as to why when I was trying to duplicate it on
the shell that and I cated the test message I was using through spamc
it went through with out being cut off.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081117/8207b263/attachment.bin>


More information about the Chicago-talk mailing list