From joshua.mcadams at gmail.com  Wed Nov  5 00:00:25 2008
From: joshua.mcadams at gmail.com (Joshua McAdams)
Date: Wed, 5 Nov 2008 02:00:25 -0600
Subject: [Chicago-talk] [WindyCity-pm] Obama Rally
In-Reply-To: <87skq7nq5c.fsf@bar.jrock.us>
References: <87skq7nq5c.fsf@bar.jrock.us>
Message-ID: <49d805d70811050000i30f93578v2de498293ae981a0@mail.gmail.com>

Doh, wish I would have checked this sooner... I went and met up with a
decent sized group.  Celebratory beer is done now though... back to
work tomorrow :)

On Tue, Nov 4, 2008 at 3:21 PM, Jonathan Rockway <jon at jrock.us> wrote:
>
> Anyone going to the Obama rally tonight?  If so, I am practically across
> the street, and would be up for having some beer before/afterwards.
>
> McCain supporters are also welcome, you guys will be needing a lot of
> beer after Obama wins ;)
>
> --
> print just => another => perl => hacker => if $,=$"
> _______________________________________________
> WindyCity-pm mailing list
> WindyCity-pm at pm.org
> http://mail.pm.org/mailman/listinfo/windycity-pm
>

From hwigoda at mindspring.com  Wed Nov  5 04:40:58 2008
From: hwigoda at mindspring.com (Hal Wigoda)
Date: Wed, 5 Nov 2008 06:40:58 -0600
Subject: [Chicago-talk] [WindyCity-pm] Obama Rally
In-Reply-To: <49d805d70811050000i30f93578v2de498293ae981a0@mail.gmail.com>
References: <87skq7nq5c.fsf@bar.jrock.us>
	<49d805d70811050000i30f93578v2de498293ae981a0@mail.gmail.com>
Message-ID: <7BDFE79B-8BEB-40C1-84D7-E74DF01A4663@mindspring.com>

i was there, too,


On Nov 5, 2008, at 2:00 AM, Joshua McAdams wrote:

> Doh, wish I would have checked this sooner... I went and met up with a
> decent sized group.  Celebratory beer is done now though... back to
> work tomorrow :)
>
> On Tue, Nov 4, 2008 at 3:21 PM, Jonathan Rockway <jon at jrock.us> wrote:
>>
>> Anyone going to the Obama rally tonight?  If so, I am practically  
>> across
>> the street, and would be up for having some beer before/afterwards.
>>
>> McCain supporters are also welcome, you guys will be needing a lot of
>> beer after Obama wins ;)
>>
>> --
>> print just => another => perl => hacker => if $,=$"
>> _______________________________________________
>> WindyCity-pm mailing list
>> WindyCity-pm at pm.org
>> http://mail.pm.org/mailman/listinfo/windycity-pm
>>
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk


From joshua.mcadams at gmail.com  Mon Nov 10 18:59:38 2008
From: joshua.mcadams at gmail.com (Joshua McAdams)
Date: Mon, 10 Nov 2008 20:59:38 -0600
Subject: [Chicago-talk] [WindyCity-pm] November Meeting
In-Reply-To: <49d805d70811101859y1a132174l47c5a3ad80b40c4d@mail.gmail.com>
References: <E7FF5A77-D041-4699-9725-91DA1FC3A67F@yarrish.com>
	<871vxny0qc.fsf@bar.jrock.us>
	<49d805d70811101859y1a132174l47c5a3ad80b40c4d@mail.gmail.com>
Message-ID: <49d805d70811101859j2ef03475gbf9775b59314c4ac@mail.gmail.com>

On Mon, Nov 10, 2008 at 8:59 PM, Joshua McAdams
<joshua.mcadams at gmail.com> wrote:
> I just booked the room for the 17th... Monday, November 17th.  Is that
> okay?  If so, I'll go ahead and make an announcement on the site.
>
> On Fri, Nov 7, 2008 at 10:11 AM, Jonathan Rockway <jon at jrock.us> wrote:
>> * On Thu, Nov 06 2008, Tom Yarrish wrote:
>>> I know it's early, but was there ever a final decision as to if we
>>> would move the November meeting up a week because of the holidays?
>>
>> Sure, why not?
>>
>> Assuming a room at Google is available on the 17th, let's do it then.
>> Is that OK, Josh?
>>
>> Regards,
>> Jonathan Rockway
>>
>> --
>> print just => another => perl => hacker => if $,=$"
>> _______________________________________________
>> WindyCity-pm mailing list
>> WindyCity-pm at pm.org
>> http://mail.pm.org/mailman/listinfo/windycity-pm
>>
>

From joshua.mcadams at gmail.com  Mon Nov 10 19:00:43 2008
From: joshua.mcadams at gmail.com (Joshua McAdams)
Date: Mon, 10 Nov 2008 21:00:43 -0600
Subject: [Chicago-talk] This Month's Meeting
Message-ID: <49d805d70811101900g6080d267mf6c4ea2c3b07818a@mail.gmail.com>

I don't remember from last time... were there any talks planned for
the next meeting?  Anyone with anything interesting to show?

From andrewdo at frdcsa.org  Mon Nov 10 19:26:32 2008
From: andrewdo at frdcsa.org (Andrew Dougherty)
Date: Mon, 10 Nov 2008 21:26:32 -0600 (CST)
Subject: [Chicago-talk] This Month's Meeting
In-Reply-To: <49d805d70811101900g6080d267mf6c4ea2c3b07818a@mail.gmail.com>
References: <49d805d70811101900g6080d267mf6c4ea2c3b07818a@mail.gmail.com>
Message-ID: <20081110.212632.139173811.andrewdo@frdcsa.org>

Okay I'm always looking to present on the FRDCSA.  Let me know if you
are interested.  This time around I will show actual systems running
and what they are doing etc.  Nothing really groundbreaking in terms
of Perl, just useful applications.

From: "Joshua McAdams" <joshua.mcadams at gmail.com>
Subject: [Chicago-talk] This Month's Meeting
Date: Mon, 10 Nov 2008 21:00:43 -0600

> I don't remember from last time... were there any talks planned for
> the next meeting?  Anyone with anything interesting to show?
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk

From me at heyjay.com  Mon Nov 10 21:09:50 2008
From: me at heyjay.com (Jay Strauss)
Date: Mon, 10 Nov 2008 23:09:50 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
Message-ID: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>

Hi,

I've been reading the docs and examples and faq about www::mechanize, but I
can't get past the login screen of the site I'm trying to access.

use WWW::Mechanize;
use Data::Dumper;
my $mech = WWW::Mechanize->new();
my $url = 'https://lmx.leads360.com/web/Login.aspx';
$mech->get( $url );
$mech->credentials( 'jay, 'mypass' );
$mech->set_visible( 'jay', 'mypass' );
$mech->submit;
print $mech->content( format => 'text' );
Is it because the page has so much javascript?  It seems like the above
should work.

Do you see a blatant error?  if so will you point me at it?

Thanks
Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081110/9da3728d/attachment.html>

From andy at petdance.com  Mon Nov 10 21:13:57 2008
From: andy at petdance.com (Andy Lester)
Date: Mon, 10 Nov 2008 23:13:57 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
In-Reply-To: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
References: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
Message-ID: <03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>


On Nov 10, 2008, at 11:09 PM, Jay Strauss wrote:

> Do you see a blatant error?  if so will you point me at it?
>


Do you know that the ->get succeeded?

--
Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance





From me at heyjay.com  Mon Nov 10 21:31:53 2008
From: me at heyjay.com (Jay Strauss)
Date: Mon, 10 Nov 2008 23:31:53 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
In-Reply-To: <03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>
References: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
	<03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>
Message-ID: <aabc51b60811102131i5072b929w2f340ed866c6140@mail.gmail.com>

>
>
> Do you know that the ->get succeeded?


I believe so:

jstrauss at neon:~/bin$ perl 360.pl
1
jstrauss at neon:~/bin$ cat 360.pl
use WWW::Mechanize;
use Data::Dumper;
my $mech = WWW::Mechanize->new();
my $url = 'https://lmx.leads360.com/web/Login.aspx';
$mech->get( $url );
my $rc = $mech->success();
print $rc, "\n";
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081110/b848c4b5/attachment.html>

From andy at petdance.com  Mon Nov 10 21:35:35 2008
From: andy at petdance.com (Andy Lester)
Date: Mon, 10 Nov 2008 23:35:35 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
In-Reply-To: <aabc51b60811102131i5072b929w2f340ed866c6140@mail.gmail.com>
References: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
	<03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>
	<aabc51b60811102131i5072b929w2f340ed866c6140@mail.gmail.com>
Message-ID: <F81C5C6D-65A6-4EE9-ABFD-29949DB0311C@petdance.com>


On Nov 10, 2008, at 11:31 PM, Jay Strauss wrote:

> my $url = 'https://lmx.leads360.com/web/Login.aspx';
> $mech->get( $url );


There's JavaScript going on in the submit button.  I didn't look to  
see what it's doing, but here's your big test: Turn off JavaScript in  
your browser.  Can you log in?  If not, then you've got problems  
because Mech doesn't handle JavaScript.

--
Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance





From me at heyjay.com  Mon Nov 10 21:43:49 2008
From: me at heyjay.com (Jay Strauss)
Date: Mon, 10 Nov 2008 23:43:49 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
In-Reply-To: <F81C5C6D-65A6-4EE9-ABFD-29949DB0311C@petdance.com>
References: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
	<03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>
	<aabc51b60811102131i5072b929w2f340ed866c6140@mail.gmail.com>
	<F81C5C6D-65A6-4EE9-ABFD-29949DB0311C@petdance.com>
Message-ID: <aabc51b60811102143r274eca5asb3f33e11a196abe2@mail.gmail.com>

On Mon, Nov 10, 2008 at 11:35 PM, Andy Lester <andy at petdance.com> wrote:

>
> On Nov 10, 2008, at 11:31 PM, Jay Strauss wrote:
>
> my $url = 'https://lmx.leads360.com/web/Login.aspx';
>> $mech->get( $url );
>>
>
>
> There's JavaScript going on in the submit button.  I didn't look to see
> what it's doing, but here's your big test: Turn off JavaScript in your
> browser.  Can you log in?  If not, then you've got problems because Mech
> doesn't handle JavaScript.


Yep I read about the javascript stuff.  Didn't even know you could turn off
scripting.

Alas, the site does NOT work without javascript.

Too bad.

Thanks for the help though.
Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081110/dc6b2538/attachment-0001.html>

From andy at petdance.com  Mon Nov 10 21:47:39 2008
From: andy at petdance.com (Andy Lester)
Date: Mon, 10 Nov 2008 23:47:39 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
In-Reply-To: <aabc51b60811102143r274eca5asb3f33e11a196abe2@mail.gmail.com>
References: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
	<03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>
	<aabc51b60811102131i5072b929w2f340ed866c6140@mail.gmail.com>
	<F81C5C6D-65A6-4EE9-ABFD-29949DB0311C@petdance.com>
	<aabc51b60811102143r274eca5asb3f33e11a196abe2@mail.gmail.com>
Message-ID: <6EDCAD76-C891-49A2-AD2A-2C53BA419294@petdance.com>


On Nov 10, 2008, at 11:43 PM, Jay Strauss wrote:

> Yep I read about the javascript stuff.  Didn't even know you could  
> turn off scripting.
>
> Alas, the site does NOT work without javascript.
>


You're not necessarily out of luck, but it makes things much tougher.   
Basically, you'll have to figure out what the JavaScript is doing, and  
fudge the HTML and/or what Mech is doing.

It's not impossible, but it's a pain, and you have to know what you're  
doing, and you'll probably be on your own.

xoa

--
Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance





From lee at laylward.com  Mon Nov 10 21:54:10 2008
From: lee at laylward.com (Lee Aylward)
Date: Mon, 10 Nov 2008 23:54:10 -0600
Subject: [Chicago-talk] getting past login with www::mechanize
In-Reply-To: <aabc51b60811102143r274eca5asb3f33e11a196abe2@mail.gmail.com>
References: <aabc51b60811102109p745dd056i1173d461d80ae672@mail.gmail.com>
	<03B33859-1CA1-4C2C-B538-EF25A40D8AAB@petdance.com>
	<aabc51b60811102131i5072b929w2f340ed866c6140@mail.gmail.com>
	<F81C5C6D-65A6-4EE9-ABFD-29949DB0311C@petdance.com>
	<aabc51b60811102143r274eca5asb3f33e11a196abe2@mail.gmail.com>
Message-ID: <20081111055410.GA2965@prettybrd.com>

On Mon, Nov 10, 2008 at 11:43:49PM -0600, Jay Strauss wrote:
> 
> Yep I read about the javascript stuff.  Didn't even know you could turn off
> scripting. 
>  
> Alas, the site does NOT work without javascript.
>  
> Too bad.
>  

You can still send the necessary parameters to log in, you just need to
find out what they are. The Firebug extension for Firefox is very
helpful for this. Enable the Net monitoring before you submit the login
form, and then inspect the request to the server. You should be able to
figure out what parameters are being added via javascript.

-- 
Lee Aylward
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081110/42e67778/attachment.bin>

From tigerpeng2001 at yahoo.com  Thu Nov 13 13:38:44 2008
From: tigerpeng2001 at yahoo.com (tiger peng)
Date: Thu, 13 Nov 2008 13:38:44 -0800 (PST)
Subject: [Chicago-talk] perl shell in activeperl
Message-ID: <722383.35032.qm@web58703.mail.re1.yahoo.com>

Hello,

I am new to activeperl. Today, I installed DBI::Shell and
played with dbish and ran into problem: the echoed input seems being
wrapped by escaping characters. For example, when I entered "dev" the screen show"?[1md?[0m?[1me?[0m?[1mv?[0m". Same thing happened while I tried to use CPAN shell.

Could anyone tell me how to get a clean screen?

Thanks,

Tiger,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081113/0b6db96f/attachment.html>

From Andy_Bach at wiwb.uscourts.gov  Thu Nov 13 13:45:33 2008
From: Andy_Bach at wiwb.uscourts.gov (Andy_Bach at wiwb.uscourts.gov)
Date: Thu, 13 Nov 2008 15:45:33 -0600
Subject: [Chicago-talk] perl shell in activeperl
In-Reply-To: <722383.35032.qm@web58703.mail.re1.yahoo.com>
References: <722383.35032.qm@web58703.mail.re1.yahoo.com>
Message-ID: <OFED196F81.65A01680-ON86257500.0077384B-86257500.007786B1@uscmail.uscourts.gov>

Appears to be a terminal emulation issue.  Something like vt100 vs xterm
but not sure what the rest of your env is so ... but those escape ("arrow
bracket") sequences are something like underline chars, maybe?

a
-------------------
Andy Bach
Systems Mangler
Internet: andy_bach at wiwb.uscourts.gov
Voice: (608) 261-5738 Fax: 264-5932

Character is built upon the debris of despair
Emerson


From davidy at nationalcycle.com  Thu Nov 13 14:01:43 2008
From: davidy at nationalcycle.com (David Young)
Date: Thu, 13 Nov 2008 16:01:43 -0600
Subject: [Chicago-talk] perl shell in activeperl
Message-ID: <s91c4f7a.057@email.nationalcycle.com>

If this is linux, try running 'tset' to reset your terminal.

ydy

>>> tigerpeng2001 at yahoo.com 11/13/08 03:38PM >>>
Hello,

I am new to activeperl. Today, I installed DBI::Shell and
played with dbish and ran into problem: the echoed input seems being
wrapped by escaping characters. For example, when I entered "dev" the screen show" [1md [0m [1me [0m [1mv [0m". Same thing happened while I tried to use CPAN shell.

Could anyone tell me how to get a clean screen?

Thanks,

Tiger,


From tigerpeng2001 at yahoo.com  Thu Nov 13 17:12:58 2008
From: tigerpeng2001 at yahoo.com (tiger peng)
Date: Thu, 13 Nov 2008 17:12:58 -0800 (PST)
Subject: [Chicago-talk] perl shell in activeperl
References: <s91c4f7a.057@email.nationalcycle.com>
Message-ID: <453912.42130.qm@web58703.mail.re1.yahoo.com>

It's Windows cmd.




________________________________
From: David Young <davidy at nationalcycle.com>
To: chicago-talk at pm.org
Sent: Thursday, November 13, 2008 4:01:43 PM
Subject: Re: [Chicago-talk] perl shell in activeperl

If this is linux, try running 'tset' to reset your terminal.

ydy

>>> tigerpeng2001 at yahoo.com 11/13/08 03:38PM >>>
Hello,

I am new to activeperl. Today, I installed DBI::Shell and
played with dbish and ran into problem: the echoed input seems being
wrapped by escaping characters. For example, when I entered "dev" the screen show" [1md [0m [1me [0m [1mv [0m". Same thing happened while I tried to use CPAN shell.

Could anyone tell me how to get a clean screen?

Thanks,

Tiger,

_______________________________________________
Chicago-talk mailing list
Chicago-talk at pm.org
http://mail.pm.org/mailman/listinfo/chicago-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081113/7f942153/attachment.html>

From Andy_Bach at wiwb.uscourts.gov  Thu Nov 13 17:25:24 2008
From: Andy_Bach at wiwb.uscourts.gov (Andy_Bach at wiwb.uscourts.gov)
Date: Thu, 13 Nov 2008 19:25:24 -0600
Subject: [Chicago-talk] perl shell in activeperl
In-Reply-To: <453912.42130.qm@web58703.mail.re1.yahoo.com>
References: <s91c4f7a.057@email.nationalcycle.com>
	<453912.42130.qm@web58703.mail.re1.yahoo.com>
Message-ID: <OFB1C7DEC4.6398EE7A-ON86257501.00078D6B-86257501.0007D169@uscmail.uscourts.gov>

Dunno if this helps but the DBI::Shell docs say:

The configuration file (~/.dbish_config by default, overridden by
$DBISH_CONFIG) is eval'd as perl code and should return a hash ref
containing the configuration items.
...
Since the config file can contain arbitrary perl code, it can do anything
which can be done in perl. One rather useful hack is to call binmode to set
the encoding for standard input and output.
(sample config file)
binmode STDIN,  ":utf8";
binmode STDOUT, ":utf8";

This config file first sets the :utf8 layer for input and output to match
the terminal (if you don't use UTF-8, you could use :encoding() here)
-------------------
Andy Bach
Systems Mangler
Internet: andy_bach at wiwb.uscourts.gov
Voice: (608) 261-5738 Fax: 264-5932

Character is built upon the debris of despair
Emerson


From me at heyjay.com  Thu Nov 13 20:45:09 2008
From: me at heyjay.com (Jay Strauss)
Date: Thu, 13 Nov 2008 22:45:09 -0600
Subject: [Chicago-talk] perl shell in activeperl
In-Reply-To: <453912.42130.qm@web58703.mail.re1.yahoo.com>
References: <s91c4f7a.057@email.nationalcycle.com>
	<453912.42130.qm@web58703.mail.re1.yahoo.com>
Message-ID: <aabc51b60811132045g4107e7c8v81dbb46197bfe1a6@mail.gmail.com>

On Thu, Nov 13, 2008 at 7:12 PM, tiger peng <tigerpeng2001 at yahoo.com> wrote:

>  It's Windows cmd.
>

"cls" clears the screen, but I don't think that is what you're asking.

May I suggest strawberry perl?  I've had good luck compiling stuff under M$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081113/1dd1221f/attachment.html>

From joshua.mcadams at gmail.com  Sun Nov 16 19:43:22 2008
From: joshua.mcadams at gmail.com (Joshua McAdams)
Date: Sun, 16 Nov 2008 21:43:22 -0600
Subject: [Chicago-talk] No Meeting on the 17th
Message-ID: <49d805d70811161943y3474c4f8w8dd5845b11ae7bb0@mail.gmail.com>

Sorry for the late email, but it turns out that I can't host the
meeting tomorrow night (Nov 17th).  If anyone else has space, please
let us know.  Otherwise, I'm going to have to put off the meeting
until later in November or December.

Josh

From sean at blanton.com  Mon Nov 17 10:08:39 2008
From: sean at blanton.com (Sean Blanton)
Date: Mon, 17 Nov 2008 12:08:39 -0600
Subject: [Chicago-talk] WindyCity.pm meeting today?
Message-ID: <4921B327.409@blanton.com>

Is there a meeting today? Is there a sign up form like last time?

Thanks,
Sean

From joshua.mcadams at gmail.com  Mon Nov 17 10:40:53 2008
From: joshua.mcadams at gmail.com (Joshua McAdams)
Date: Mon, 17 Nov 2008 12:40:53 -0600
Subject: [Chicago-talk] WindyCity.pm meeting today?
In-Reply-To: <4921B327.409@blanton.com>
References: <4921B327.409@blanton.com>
Message-ID: <49d805d70811171040s76f5635bt4f56dd5564d5dd31@mail.gmail.com>

Canceled :(  I can't stick around this evening to keep the door opened
and nobody else from the office has stepped up.

On Mon, Nov 17, 2008 at 12:08 PM, Sean Blanton <sean at blanton.com> wrote:
> Is there a meeting today? Is there a sign up form like last time?
>
> Thanks,
> Sean
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
>

From sean at blanton.com  Mon Nov 17 12:02:56 2008
From: sean at blanton.com (Sean Blanton)
Date: Mon, 17 Nov 2008 14:02:56 -0600
Subject: [Chicago-talk] Oops - had digest on.
In-Reply-To: <mailman.6.1226952009.7310.chicago-talk@pm.org>
References: <mailman.6.1226952009.7310.chicago-talk@pm.org>
Message-ID: <4921CDF0.8070303@blanton.com>

An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081117/e17e63c8/attachment.html>

From v.velox at vvelox.net  Mon Nov 17 17:49:13 2008
From: v.velox at vvelox.net (Zane C.B.)
Date: Mon, 17 Nov 2008 19:49:13 -0600
Subject: [Chicago-talk] open3 and passing large amounts of data
Message-ID: <20081117194913.691ad700@vixen42>

Any one ever run into a issue before when passing a large amount of
data through a pipe?

I am currently trying to print '1022163' bytes to the STDIN spamc,
but it is never finishing.

What I found I had to do is set spamc's max size arguement to what my
incoming mail server to something large or as large as what my mail
server allows for. Which is odd as spamc does not hang if I cat
something that large through it.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081117/39045a02/attachment.bin>

From arodland at comcast.net  Mon Nov 17 18:00:43 2008
From: arodland at comcast.net (Andrew Rodland)
Date: Mon, 17 Nov 2008 20:00:43 -0600
Subject: [Chicago-talk] open3 and passing large amounts of data
In-Reply-To: <20081117194913.691ad700@vixen42>
References: <20081117194913.691ad700@vixen42>
Message-ID: <200811172000.43796.arodland@comcast.net>

On Monday 17 November 2008 07:49:13 pm Zane C.B. wrote:
> Any one ever run into a issue before when passing a large amount of
> data through a pipe?
>
> I am currently trying to print '1022163' bytes to the STDIN spamc,
> but it is never finishing.
>
> What I found I had to do is set spamc's max size arguement to what my
> incoming mail server to something large or as large as what my mail
> server allows for. Which is odd as spamc does not hang if I cat
> something that large through it.

It's a deadlock. spamc stopped reading its input and sent you a message that 
went something along the lines of "whoa, bucko, that's way too much data!", 
but your app didn't receive that message because it was too busy trying to 
stuff data through the pipe going the other way, and wasn't checking for 
input. So each end was blocked waiting on the other. To avoid problems like 
that, you need to use nonblocking I/O plus a select loop or POE or Event or 
something like that, or threads (but Perl isn't very thread-friendly, so 
forget I mentioned that one).

Andrew


From v.velox at vvelox.net  Mon Nov 17 18:41:03 2008
From: v.velox at vvelox.net (Zane C.B.)
Date: Mon, 17 Nov 2008 20:41:03 -0600
Subject: [Chicago-talk] open3 and passing large amounts of data
In-Reply-To: <200811172000.43796.arodland@comcast.net>
References: <20081117194913.691ad700@vixen42>
	<200811172000.43796.arodland@comcast.net>
Message-ID: <20081117204103.6c059e8d@vixen42>

On Mon, 17 Nov 2008 20:00:43 -0600
Andrew Rodland <arodland at comcast.net> wrote:

> On Monday 17 November 2008 07:49:13 pm Zane C.B. wrote:
> > Any one ever run into a issue before when passing a large amount
> > of data through a pipe?
> >
> > I am currently trying to print '1022163' bytes to the STDIN spamc,
> > but it is never finishing.
> >
> > What I found I had to do is set spamc's max size arguement to
> > what my incoming mail server to something large or as large as
> > what my mail server allows for. Which is odd as spamc does not
> > hang if I cat something that large through it.
> 
> It's a deadlock. spamc stopped reading its input and sent you a
> message that went something along the lines of "whoa, bucko, that's
> way too much data!", but your app didn't receive that message
> because it was too busy trying to stuff data through the pipe going
> the other way, and wasn't checking for input. So each end was
> blocked waiting on the other. To avoid problems like that, you need
> to use nonblocking I/O plus a select loop or POE or Event or
> something like that, or threads (but Perl isn't very
> thread-friendly, so forget I mentioned that one).

I've don't think threading would help with this.

Thanks. I will have to look into doing non-blocking IO then.

I am still a bit lost as to why when I was trying to duplicate it on
the shell that and I cated the test message I was using through spamc
it went through with out being cut off.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20081117/8207b263/attachment.bin>

From lembark at wrkhors.com  Tue Nov 18 01:24:50 2008
From: lembark at wrkhors.com (Steven Lembark)
Date: Tue, 18 Nov 2008 04:24:50 -0500
Subject: [Chicago-talk] open3 and passing large amounts of data
In-Reply-To: <20081117204103.6c059e8d@vixen42>
References: <20081117194913.691ad700@vixen42>	<200811172000.43796.arodland@comcast.net>
	<20081117204103.6c059e8d@vixen42>
Message-ID: <492289E2.3080207@wrkhors.com>


> I've don't think threading would help with this.
> 
> Thanks. I will have to look into doing non-blocking IO then.
> 
> I am still a bit lost as to why when I was trying to duplicate it on
> the shell that and I cated the test message I was using through spamc
> it went through with out being cut off.

You can block, just not on the I/O. Use
select to write, say, 8K chunks into the
pipe and check for reads on stderr. There
are a few good packages for handling this
situation, I like Graham Barr's IO::Select.

You originally mentioned a mailserver, so
there may be some message being sent from
there that blocks the spam handler. Using
select w/ timeouts will detect this in
most cases.

-- 
Steven Lembark                                            85-09 90th St.
Workhorse Computing                                 Woodhaven, NY, 11421
lembark at wrkhors.com                                      +1 888 359 3508