[Chicago-talk] open3 and passing large amounts of data
arodland at comcast.net
Mon Nov 17 18:00:43 PST 2008
On Monday 17 November 2008 07:49:13 pm Zane C.B. wrote:
> Any one ever run into a issue before when passing a large amount of
> data through a pipe?
> I am currently trying to print '1022163' bytes to the STDIN spamc,
> but it is never finishing.
> What I found I had to do is set spamc's max size arguement to what my
> incoming mail server to something large or as large as what my mail
> server allows for. Which is odd as spamc does not hang if I cat
> something that large through it.
It's a deadlock. spamc stopped reading its input and sent you a message that
went something along the lines of "whoa, bucko, that's way too much data!",
but your app didn't receive that message because it was too busy trying to
stuff data through the pipe going the other way, and wasn't checking for
input. So each end was blocked waiting on the other. To avoid problems like
that, you need to use nonblocking I/O plus a select loop or POE or Event or
something like that, or threads (but Perl isn't very thread-friendly, so
forget I mentioned that one).
More information about the Chicago-talk