From MichaelRWolf at att.net Thu Apr 1 13:13:38 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Thu, 1 Apr 2010 13:13:38 -0700 Subject: SPUG: Fwd: [Pdx-pm] Looking for developer(s) for a new project. References: <412C69CE-6D20-4807-819F-6F54404B3644@enterity.com> Message-ID: <417D94E9-9C4E-42C9-B10E-1D4D7EC1B26D@att.net> I saw this on the PDX board, and found the technology and business idea to be interesting enough to talk with Stephen, but ultimately realized that I need to fill my paid-project pipeline before committing more of my spare time. Since telecommuting from Seattle is as good as telecommuting from PDX, he asked me to repost it here. If you've got extra cycles to devote to a pre-funded start-up, please contact Stephen directly. Enjoy, Michael P.S. If you've got extra, (funded) work to do, PLEASE CONTACT ME. Begin forwarded message: > From: Stephen Howard > Date: March 27, 2010 10:15:22 AM PDT > To: Portland Perl Mongers > Subject: [Pdx-pm] Looking for developer(s) for a new project. > > Hello fellow perl mongers, > > I?ve recently started a project via the Startup Weekend phenomenon, > and find myself the only developer on a very cool team. I?d like to > convince someone in the PDX.pm community to join me on the > development end of things and really make this software rock. > > The project is called CluePad (www.cluepad.com) and we?re looking to > use data culled from multiple sources (web crawling and the Google > Analytics API for starters) to make simple-to-follow tips that small > business owners can use to improve their websites in ways that help > their bottom line. Right now the options for these small business > owners are either to a) spend a lot of time learning and sifting > through data on their own, or b) hiring someone on a pricy monthly > retainer to handle it for them. CluePad would be a low-cost, low- > time-investment alternative to both of these. > > We?re using Catalyst/Template Toolkit and DBIx Class for the site. > The goal is to spin this into a profitable company, but there?s no > money in it yet. I?ve got the first generation of the site built and > am fine tuning it this week before we let in some early adopters. > > I?m looking for someone familiar with the modules above, and who is > used to maintaining a test suite as a matter of course. Bonus points > for being fluent with jQuery on the client side, or for experience > with scaling and parallelizing data collection and analysis on the > back end. > > This will primarily be a telecommuting project. If you?re interested > we?ll have an initial period working together to see whether we?re a > good fit for each other, and when that goes well we?ll have a > discussion about equity. > > We've got over 30 people waiting to try the software out from some > minimal pre-release marketing, and have a couple team members who > are ready and able to get this in front of a lot more people in our > target market. This idea was also the winner at Startup Weekend > Corvallis in December and the startup weekend folks are eager to put > their muscle behind promoting our success. > > thanks, > > Stephen > > _______________________________________________ > Pdx-pm-list mailing list > Pdx-pm-list at pm.org > http://mail.pm.org/mailman/listinfo/pdx-pm-list -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew at sweger.net Thu Apr 1 17:31:56 2010 From: andrew at sweger.net (Andrew Sweger) Date: Thu, 1 Apr 2010 17:31:56 -0700 (PDT) Subject: SPUG: 20% Discount for O'Reilly Open Source Convention (OSCON) Message-ID: OSCON Registration is open. OSCON happens July 19-23, 2010 in Portland, OR Use code "os10usrg" when you register, and receive 20% off the registration price. To register for the conference, go to: https://en.oreilly.com/oscon2010/public/register OSCON 2010 Registration Now Open. Make It Happen at OSCON! At OSCON, you'll participate in hundreds of sessions covering open source languages and platforms, practical tutorials that go deep into technical skill and best practices, inspirational keynote presentations, an Expo Hall featuring dozens of the latest projects and products, fun networking events and activities, and the best "hallway track" around. - Explore the benefits and challenges of building scalable applications for the cloud - Use open source to target Android, iPhone and other mobile platforms - Understand how and when to use NoSQL databases - Learn best practices from experts in Python, Java, Ruby, Perl, PHP and JavaScript - Use open source effectively as part of your business strategy - Learn how to foster contribution and adoption of your open source projects From andrew at sweger.net Thu Apr 1 17:34:50 2010 From: andrew at sweger.net (Andrew Sweger) Date: Thu, 1 Apr 2010 17:34:50 -0700 (PDT) Subject: SPUG: 20% Discount for O'Reilly Open Source Convention (OSCON) In-Reply-To: Message-ID: I have not heard yet if there will be a special "locals" discount. I just realized that OSCON is at PDX once again. So, there might be another (better) discount. I'll try to find out. On Thu, 1 Apr 2010, Andrew Sweger wrote: > OSCON Registration is open. > > OSCON happens July 19-23, 2010 in Portland, OR > > Use code "os10usrg" when you register, and receive 20% off the > registration price. > > To register for the conference, go to: > > https://en.oreilly.com/oscon2010/public/register > > OSCON 2010 Registration Now Open. Make It Happen at OSCON! > > At OSCON, you'll participate in hundreds of sessions covering open source > languages and platforms, practical tutorials that go deep into technical > skill and best practices, inspirational keynote presentations, an Expo > Hall featuring dozens of the latest projects and products, fun networking > events and activities, and the best "hallway track" around. > > - Explore the benefits and challenges of building scalable applications for > the cloud > - Use open source to target Android, iPhone and other mobile platforms > - Understand how and when to use NoSQL databases > - Learn best practices from experts in Python, Java, Ruby, Perl, PHP and > JavaScript > - Use open source effectively as part of your business strategy > - Learn how to foster contribution and adoption of your open source projects From andrew at sweger.net Thu Apr 1 19:31:48 2010 From: andrew at sweger.net (Andrew Sweger) Date: Thu, 1 Apr 2010 19:31:48 -0700 (PDT) Subject: SPUG: 20% Discount for O'Reilly Open Source Convention (OSCON) In-Reply-To: Message-ID: Heh, the 20%-off is the "local" discount. They've just decided to make it available to all user groups. So, would OSCON be to Perl geeks as PAX is to gamers? On Thu, 1 Apr 2010, Andrew Sweger wrote: > I have not heard yet if there will be a special "locals" discount. I just > realized that OSCON is at PDX once again. So, there might be another > (better) discount. I'll try to find out. -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From ingy at ingy.net Tue Apr 6 21:10:15 2010 From: ingy at ingy.net (Ingy dot Net) Date: Tue, 6 Apr 2010 21:10:15 -0700 Subject: SPUG: Come Invade the Seattle Python Interest Group Meeting with Ingy Message-ID: Hey Spugsters! Guess what? I'm giving my first Python talk on Thursday night at the UW! Crazy eh? But totally true. I've even written a couple Python modules. Think Ingy has flipped?? Think again. I'm a Monger, yo! But I'm also an Acmeist. Here's the deal... I'm working on a new programming language called C'Dent. I got version 0.1 working last night. C'Dent lets you write modules in one language (like Perl or Python) and use them in a dozen others. If you want to try it, you can get C'Dent here: http://pypi.python.org/pypi/cdent/ and use it to compile a Python module to Perl 6!! (And half a dozen others...) Yes, I wrote it in Python, but that's just because Python has the best YAML support. ;) Anyway, I want you guys to come because they told me to only expect 5 or so attendees. Let's show these seapigs how we roll!!!! The info is here: http://www.seapig.org/ Hope to see y'all there. Love, Ingy -------------- next part -------------- An HTML attachment was scrubbed... URL: From klevin at eskimo.com Tue Apr 6 21:18:59 2010 From: klevin at eskimo.com (=?UTF-8?B?Tm9haCBSw7htZXI=?=) Date: Tue, 06 Apr 2010 21:18:59 -0700 Subject: SPUG: speaking of meetings . . . Message-ID: <4BBC07B3.3000706@eskimo.com> Do we have a monthly meeting this month? -- Noah Romer | It's difficult to get a man to understand klevin at eskimo.com | something when his salary depends on him not PGP key available | understanding it. by finger or email | - Upton Sinclair From andrew at sweger.net Wed Apr 7 06:33:39 2010 From: andrew at sweger.net (Andrew Sweger) Date: Wed, 7 Apr 2010 06:33:39 -0700 (PDT) Subject: SPUG: speaking of meetings . . . In-Reply-To: <4BBC07B3.3000706@eskimo.com> Message-ID: Yes. Just awaiting speaker bio, topic description, and room confirmation before making the official announcement. The topic has to do with Perl 5.12. Same SPUG-time, same SPUG-channel. On Tue, 6 Apr 2010, Noah R?mer wrote: > Do we have a monthly meeting this month? > > -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From MichaelRWolf at att.net Thu Apr 8 10:45:05 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Thu, 8 Apr 2010 10:45:05 -0700 Subject: SPUG: Best practice? DBI connection parameters *not* in Perl code Message-ID: As some DBI-based code is being ported to other machines, I'm having to pass command-line parameters to too many administrative and test programs. I'd like to change this *once* external to *all* programs and have it apply to them all. This will have the added benefit of articulating a cleaner differentiation between environments (development, test, production). What's more popular to do this? - environment variables - config file In code, and in *theory*, it's a trivial difference... $db_user = $ENV{db_user} || 'root'; versus $db_user = $some_config_ref->get_attribute('db_user') || 'root'; I'm more interested in how this works in *practice*. That is, when fielded, which seems to be groked by more end users? And if the answer is 'config', which of the 2 score and twenty config modules seems to be groked by more end users? Thanks, Michael -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From skylos at gmail.com Thu Apr 8 10:53:01 2010 From: skylos at gmail.com (Skylos) Date: Thu, 8 Apr 2010 10:53:01 -0700 Subject: SPUG: Best practice? DBI connection parameters *not* in Perl code In-Reply-To: References: Message-ID: I'm not entirely sure if its best practice, but my feeling is that I would prefer a config file because its a more obvious artifact that can be deployed with the application. As for format, YAML of course. And here's a module especial for that. http://search.cpan.org/~moconnor/YAML-AppConfig-0.16/lib/YAML/AppConfig.pm Skylos On Thu, Apr 8, 2010 at 10:45 AM, Michael R. Wolf wrote: > As some DBI-based code is being ported to other machines, I'm having to > pass command-line parameters to too many administrative and test programs. > I'd like to change this *once* external to *all* programs and have it apply > to them all. This will have the added benefit of articulating a cleaner > differentiation between environments (development, test, production). > > What's more popular to do this? > - environment variables > - config file > > In code, and in *theory*, it's a trivial difference... > > $db_user = $ENV{db_user} || 'root'; > versus > $db_user = $some_config_ref->get_attribute('db_user') || 'root'; > > I'm more interested in how this works in *practice*. That is, when > fielded, which seems to be groked by more end users? And if the answer is > 'config', which of the 2 score and twenty config modules seems to be groked > by more end users? > > Thanks, > Michael > > -- > -- "If only I could get rid of hunger by rubbing my belly" - Diogenes -------------- next part -------------- An HTML attachment was scrubbed... URL: From brian at massassi.com Thu Apr 8 11:26:30 2010 From: brian at massassi.com (Brian E. Lozier) Date: Thu, 8 Apr 2010 11:26:30 -0700 Subject: SPUG: Best practice? DBI connection parameters *not* in Perl code In-Reply-To: References: Message-ID: Regarding YAML -- in my experience, the "YAML" module on CPAN is buggy and can't properly handle complex structures. This bit us and we moved to JSON (this wasn't for configuration). It should be fine for simple config data (but may be a bit heavy weight for that). See the bug list: https://rt.cpan.org/Public/Dist/Display.html?Name=YAML It's been suggested to use YAML::Syck, but that's compiling its own huge list of bugs as well. Depending on your deployment targets, you may want to stick with a pure-perl solution, which, I believe, YAML::Syck isn't. I've used Config::Fast and Config::General and have been happy with both. Brian On Thu, Apr 8, 2010 at 10:53 AM, Skylos wrote: > I'm not entirely sure if its best practice, but my feeling is that I would > prefer a config file because its a more obvious artifact that can be > deployed with the application.? As for format, YAML of course.? And here's a > module especial for that. > > http://search.cpan.org/~moconnor/YAML-AppConfig-0.16/lib/YAML/AppConfig.pm > > Skylos > > > On Thu, Apr 8, 2010 at 10:45 AM, Michael R. Wolf > wrote: >> >> As some DBI-based code is being ported to other machines, I'm having to >> pass command-line parameters to too many administrative and test programs. >> ?I'd like to change this *once* external to *all* programs and have it apply >> to them all. ?This will have the added benefit of articulating a cleaner >> differentiation between environments (development, test, production). >> >> What's more popular to do this? >> ?- environment variables >> ?- config file >> >> In code, and in *theory*, it's a trivial difference... >> >> ? ?$db_user = $ENV{db_user} || 'root'; >> ? ? ? ?versus >> ? ?$db_user = $some_config_ref->get_attribute('db_user') || 'root'; >> >> I'm more interested in how this works in *practice*. ?That is, when >> fielded, which seems to be groked by more end users? ?And if the answer is >> 'config', which of the 2 score and twenty config modules seems to be groked >> by more end users? >> >> Thanks, >> Michael >> >> -- > > > -- > "If only I could get rid of hunger by rubbing my belly" - Diogenes > > _____________________________________________________________ > Seattle Perl Users Group Mailing List > ? ? POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > ? ?MEETINGS: 3rd Tuesdays > ? ?WEB PAGE: http://seattleperl.org/ > From mark.mertel at yahoo.com Thu Apr 8 11:36:53 2010 From: mark.mertel at yahoo.com (Mark Mertel) Date: Thu, 8 Apr 2010 11:36:53 -0700 (PDT) Subject: SPUG: Best practice? DBI connection parameters *not* in Perl code In-Reply-To: References: Message-ID: <247224.4427.qm@web50108.mail.re2.yahoo.com> and on the other side - ENV variables - most client-server databases already support some type of $ENV{ USER } for their out-of-the-box client tools, i.e. sqlplus, isql, etc. --- Mark Mertel mark.mertel at yahoo.com ________________________________ From: Brian E. Lozier To: SPUG Members Sent: Thu, April 8, 2010 11:26:30 AM Subject: Re: SPUG: Best practice? DBI connection parameters *not* in Perl code Regarding YAML -- in my experience, the "YAML" module on CPAN is buggy and can't properly handle complex structures. This bit us and we moved to JSON (this wasn't for configuration). It should be fine for simple config data (but may be a bit heavy weight for that). See the bug list: https://rt.cpan.org/Public/Dist/Display.html?Name=YAML It's been suggested to use YAML::Syck, but that's compiling its own huge list of bugs as well. Depending on your deployment targets, you may want to stick with a pure-perl solution, which, I believe, YAML::Syck isn't. I've used Config::Fast and Config::General and have been happy with both. Brian On Thu, Apr 8, 2010 at 10:53 AM, Skylos wrote: > I'm not entirely sure if its best practice, but my feeling is that I would > prefer a config file because its a more obvious artifact that can be > deployed with the application. As for format, YAML of course. And here's a > module especial for that. > > http://search.cpan.org/~moconnor/YAML-AppConfig-0.16/lib/YAML/AppConfig.pm > > Skylos > > > On Thu, Apr 8, 2010 at 10:45 AM, Michael R. Wolf > wrote: >> >> As some DBI-based code is being ported to other machines, I'm having to >> pass command-line parameters to too many administrative and test programs. >> I'd like to change this *once* external to *all* programs and have it apply >> to them all. This will have the added benefit of articulating a cleaner >> differentiation between environments (development, test, production). >> >> What's more popular to do this? >> - environment variables >> - config file >> >> In code, and in *theory*, it's a trivial difference... >> >> $db_user = $ENV{db_user} || 'root'; >> versus >> $db_user = $some_config_ref->get_attribute('db_user') || 'root'; >> >> I'm more interested in how this works in *practice*. That is, when >> fielded, which seems to be groked by more end users? And if the answer is >> 'config', which of the 2 score and twenty config modules seems to be groked >> by more end users? >> >> Thanks, >> Michael >> >> -- > > > -- > "If only I could get rid of hunger by rubbing my belly" - Diogenes > > _____________________________________________________________ > Seattle Perl Users Group Mailing List > POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > MEETINGS: 3rd Tuesdays > WEB PAGE: http://seattleperl.org/ > _____________________________________________________________ Seattle Perl Users Group Mailing List POST TO: spug-list at pm.org SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list MEETINGS: 3rd Tuesdays WEB PAGE: http://seattleperl.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From cjac at colliertech.org Thu Apr 8 15:04:46 2010 From: cjac at colliertech.org (C.J. Adams-Collier) Date: Thu, 08 Apr 2010 15:04:46 -0700 Subject: SPUG: Best practice? DBI connection parameters *not* in Perl code In-Reply-To: References: Message-ID: <1270764286.3411.4.camel@calcifer> Ye'hear that, Ingy? I think he said your mother was a hamster. On Thu, 2010-04-08 at 11:26 -0700, Brian E. Lozier wrote: > Regarding YAML -- in my experience, the "YAML" module on CPAN is buggy > and can't properly handle complex structures. This bit us and we > moved to JSON (this wasn't for configuration). It should be fine for > simple config data (but may be a bit heavy weight for that). See the > bug list: https://rt.cpan.org/Public/Dist/Display.html?Name=YAML > > It's been suggested to use YAML::Syck, but that's compiling its own > huge list of bugs as well. Depending on your deployment targets, you > may want to stick with a pure-perl solution, which, I believe, > YAML::Syck isn't. > > I've used Config::Fast and Config::General and have been happy with both. > > Brian > > On Thu, Apr 8, 2010 at 10:53 AM, Skylos wrote: > > I'm not entirely sure if its best practice, but my feeling is that I would > > prefer a config file because its a more obvious artifact that can be > > deployed with the application. As for format, YAML of course. And here's a > > module especial for that. > > > > http://search.cpan.org/~moconnor/YAML-AppConfig-0.16/lib/YAML/AppConfig.pm > > > > Skylos > > > > > > On Thu, Apr 8, 2010 at 10:45 AM, Michael R. Wolf > > wrote: > >> > >> As some DBI-based code is being ported to other machines, I'm having to > >> pass command-line parameters to too many administrative and test programs. > >> I'd like to change this *once* external to *all* programs and have it apply > >> to them all. This will have the added benefit of articulating a cleaner > >> differentiation between environments (development, test, production). > >> > >> What's more popular to do this? > >> - environment variables > >> - config file > >> > >> In code, and in *theory*, it's a trivial difference... > >> > >> $db_user = $ENV{db_user} || 'root'; > >> versus > >> $db_user = $some_config_ref->get_attribute('db_user') || 'root'; > >> > >> I'm more interested in how this works in *practice*. That is, when > >> fielded, which seems to be groked by more end users? And if the answer is > >> 'config', which of the 2 score and twenty config modules seems to be groked > >> by more end users? > >> > >> Thanks, > >> Michael > >> > >> -- > > > > > > -- > > "If only I could get rid of hunger by rubbing my belly" - Diogenes > > > > _____________________________________________________________ > > Seattle Perl Users Group Mailing List > > POST TO: spug-list at pm.org > > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > > MEETINGS: 3rd Tuesdays > > WEB PAGE: http://seattleperl.org/ > > > _____________________________________________________________ > Seattle Perl Users Group Mailing List > POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > MEETINGS: 3rd Tuesdays > WEB PAGE: http://seattleperl.org/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From andrew at sweger.net Fri Apr 9 14:45:03 2010 From: andrew at sweger.net (Andrew Sweger) Date: Fri, 9 Apr 2010 14:45:03 -0700 (PDT) Subject: SPUG: PerlWTF? Message-ID: The Daily WTF doesn't show much love for Perl usually. Today I came across a doozy. $x = ( $x > 1 ) ? $x-- : 0; Granted, most of the time $x is undefined (naturally) and is just throwing warnings like crazy. It's the post-decrement that makes it so wonderfully nutty. I'd love to ask the original developer just what they were thinking. I'm replacing it with: $x //= 0; $x-- if $x > 1; Disclaimer: The names of some scalars were changed to protect the innocent. -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From MichaelRWolf at att.net Fri Apr 9 17:31:20 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Fri, 9 Apr 2010 17:31:20 -0700 Subject: SPUG: It *was* OK to speak baby Perl, but now what? [was Re: PerlWTF?] In-Reply-To: References: Message-ID: On Apr 9, 2010, at 2:45 PM, Andrew Sweger wrote: > $x //= 0; > $x-- if $x > 1; > > Disclaimer: The names of some scalars were changed to protect the > innocent. What did you change to protect the guilty? Silliness aside, I'm getting frustrated with my own coding style. In other words, I'm growing. I used to allow undef to behave like zero, as documented, without feeling the need to appease the pragma gods. Then I grew up to "use warnings" and perlcritic(1) as guard rails . Then I grew up and fought the restraints that were formerly called safety devices, and became annoyed that I was serving these linguistic structures instead of them serving me. When I was a child Perl programmer, I print'ed as a child. Now that I'm a growed-up Perl programmer, I 'say' as a growed-up. Etc... Perhaps I was too self-absorbed and well contented for a few years, but recently I started reading about the discontent that's brewed the likes of: use Modern::Perl; # chromatic use Perl5i::latest; # Schwern use common::sense; # Mark Lehmann And then I had this aha moment. I don't have good coding habits. I've got cult cargo habits. Why am I repeating magic incantations when my language of choice should be doing that for me? (I *am* starting to 'use Moose'. BTW - the next issue of "The Perl Review" will be exclusively devoted to Moose.) I have evolved. Has my language evolved to meet me? Of course, it's not so black and white, but it got me thinking. Thinking more than normal. Especially in this pregnant pause while we await the gestation of Perl6. So, my question to other Perl programmers, especially the thinking ones, is.... How is your "software practice" (doctors and lawyers shouldn't be the only professionals with a practice) maturing to move beyond the cargo cult practices of your days of writing baby Perl? Having asked the question, here's grist to start the conversation. Here's my growing edge... use Moose; # New for me... use DBIx::Class; # ditto use 5.10; # New for my client's platform... and what's still comfortable... use Test::More; use warnings; use strict; and what's a bit beyond my comfort factor... use Perl5i::*; use Fatal; use Test::Most; use Perl6::*; Admittedly, I'm not a bleeding edge programmer in the community, even though I've been around it for 15 years. The bulk of my Perl work has been in traning the next generation to move beyond Baby Perl(tm) so that they may join the movement. I'm really interested to hear what folks at all the levels of Perl-ish- ness are up to in their practice: from Baby Perl Programmers to Monks. Thanks for sharing... -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From jwkrahn at shaw.ca Fri Apr 9 17:45:41 2010 From: jwkrahn at shaw.ca (John W. Krahn) Date: Fri, 09 Apr 2010 17:45:41 -0700 Subject: SPUG: PerlWTF? In-Reply-To: References: Message-ID: <4BBFCA35.8060104@shaw.ca> Andrew Sweger wrote: > The Daily WTF doesn't show much love for Perl usually. Today I came across > a doozy. > > $x = ( $x > 1 ) ? $x-- : 0; > > Granted, most of the time $x is undefined (naturally) and is just throwing > warnings like crazy. It's the post-decrement that makes it so wonderfully > nutty. I'd love to ask the original developer just what they were > thinking. I'm replacing it with: > > $x //= 0; > $x-- if $x > 1; > > Disclaimer: The names of some scalars were changed to protect the > innocent. They both don't do the same thing. $ perl -le' my $x; my $y; $x = ( $x > 1 ) ? $x-- : 0; $y //= 0; $y-- if $y > 1; print "\$x = $x \$y = $y"; ' $x = 0 $y = 0 $ perl -le' my $x = my $y = shift; $x = ( $x > 1 ) ? $x-- : 0; $y //= 0; $y-- if $y > 1; print "\$x = $x \$y = $y"; ' -- -5 $x = 0 $y = -5 $ perl -le' my $x = my $y = shift; $x = ( $x > 1 ) ? $x-- : 0; $y //= 0; $y-- if $y > 1; print "\$x = $x \$y = $y"; ' -- 5 $x = 5 $y = 4 $ perl -le' my $x = my $y = shift; $x = ( $x > 1 ) ? $x-- : 0; $y //= 0; $y-- if $y > 1; print "\$x = $x \$y = $y"; ' -- 0 $x = 0 $y = 0 $ perl -le' my $x = my $y = shift; $x = ( $x > 1 ) ? $x-- : 0; $y //= 0; $y-- if $y > 1; print "\$x = $x \$y = $y"; ' -- '' $x = 0 $y = The correct expression would be: $x = ( $x > 1 ) ? $x - 1 : 0; John -- The programmer is fighting against the two most destructive forces in the universe: entropy and human stupidity. -- Damian Conway From andrew at sweger.net Fri Apr 9 19:46:53 2010 From: andrew at sweger.net (Andrew Sweger) Date: Fri, 9 Apr 2010 19:46:53 -0700 (PDT) Subject: SPUG: PerlWTF? In-Reply-To: <4BBFCA35.8060104@shaw.ca> Message-ID: On Fri, 9 Apr 2010, John W. Krahn wrote: > They both don't do the same thing. > > [ snip ] > > $ perl -le' > my $x = my $y = shift; > $x = ( $x > 1 ) ? $x-- : 0; > $y //= 0; > $y-- if $y > 1; > print "\$x = $x \$y = $y"; > ' -- -5 > $x = 0 $y = -5 > > [ snip ] > > The correct expression would be: > > $x = ( $x > 1 ) ? $x - 1 : 0; Ah, cool. I totally missed that. The only minor change I would add is to precede it with $x //= 0; Thanks. -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From mark_swayne at charter.net Sat Apr 10 09:20:02 2010 From: mark_swayne at charter.net (Mark Swayne) Date: Sat, 10 Apr 2010 09:20:02 -0700 Subject: SPUG: It *was* OK to speak baby Perl, but now what? [was Re: PerlWTF?] In-Reply-To: References: Message-ID: <4DA00BA5-5DD7-4417-97D1-02E5A7597F7B@charter.net> There is an interesting discussion about the merits of common::sense at StackOverflow: Should I use common::sense or just stick with `use strict` and `use warnings`? http://stackoverflow.com/questions/1625839/should-i-use-commonsense-or-just-stick-with-use-strict-and-use-warnings On Apr 9, 2010, at 5:31 PM, "Michael R. Wolf" wrote: > > On Apr 9, 2010, at 2:45 PM, Andrew Sweger wrote: > >> $x //= 0; >> $x-- if $x > 1; >> >> Disclaimer: The names of some scalars were changed to protect the >> innocent. > > What did you change to protect the guilty? > > Silliness aside, I'm getting frustrated with my own coding style. > In other words, I'm growing. > > I used to allow undef to behave like zero, as documented, without > feeling the need to appease the pragma gods. > > Then I grew up to "use warnings" and perlcritic(1) as guard rails . > > Then I grew up and fought the restraints that were formerly called > safety devices, and became annoyed that I was serving these > linguistic structures instead of them serving me. > > When I was a child Perl programmer, I print'ed as a child. > Now that I'm a growed-up Perl programmer, I 'say' as a growed-up. > Etc... > > Perhaps I was too self-absorbed and well contented for a few years, > but recently I started reading about the discontent that's brewed > the likes of: > use Modern::Perl; # chromatic > use Perl5i::latest; # Schwern > use common::sense; # Mark Lehmann > > And then I had this aha moment. > I don't have good coding habits. > I've got cult cargo habits. > > Why am I repeating magic incantations when my language of choice > should be doing that for me? (I *am* starting to 'use Moose'. BTW > - the next issue of "The Perl Review" will be exclusively devoted to > Moose.) I have evolved. Has my language evolved to meet me? > > Of course, it's not so black and white, but it got me thinking. > Thinking more than normal. Especially in this pregnant pause while > we await the gestation of Perl6. > > So, my question to other Perl programmers, especially the thinking > ones, is.... > > How is your "software practice" (doctors and lawyers shouldn't be > the only professionals with a practice) maturing to move beyond the > cargo cult practices of your days of writing baby Perl? > > Having asked the question, here's grist to start the conversation. > > Here's my growing edge... > use Moose; # New for me... > use DBIx::Class; # ditto > use 5.10; # New for my client's platform... > > and what's still comfortable... > use Test::More; > use warnings; > use strict; > > and what's a bit beyond my comfort factor... > use Perl5i::*; > use Fatal; > use Test::Most; > use Perl6::*; > > > Admittedly, I'm not a bleeding edge programmer in the community, > even though I've been around it for 15 years. The bulk of my Perl > work has been in traning the next generation to move beyond Baby Perl > (tm) so that they may join the movement. > > I'm really interested to hear what folks at all the levels of Perl- > ish-ness are up to in their practice: from Baby Perl Programmers to > Monks. > > Thanks for sharing... > > -- > Michael R. Wolf > All mammals learn by playing! > MichaelRWolf at att.net > > > > > _____________________________________________________________ > Seattle Perl Users Group Mailing List > POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > MEETINGS: 3rd Tuesdays > WEB PAGE: http://seattleperl.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From twists at gmail.com Mon Apr 12 11:34:55 2010 From: twists at gmail.com (Joshua ben Jore) Date: Mon, 12 Apr 2010 11:34:55 -0700 Subject: SPUG: Come Invade the Seattle Python Interest Group Meeting with Ingy In-Reply-To: References: Message-ID: On Tue, Apr 6, 2010 at 9:10 PM, Ingy dot Net wrote: > Hey Spugsters! > ... > Anyway, I want you guys to come because they told me to only expect 5 or so > attendees. Let's show these seapigs how we roll!!!! They're totally nice folks btw. When I've been, there's been about a dozen attendees. Josh From andrew at sweger.net Mon Apr 12 13:08:11 2010 From: andrew at sweger.net (Andrew Sweger) Date: Mon, 12 Apr 2010 13:08:11 -0700 (PDT) Subject: SPUG: April 2010 Seattle Perl Users Group (SPUG) Meeting Message-ID: April 2010 Seattle Perl Users Group (SPUG) Meeting ==================================================== Topic: Perl 5.12.0 - Double-click for a martini Speaker: Josh ben Jore Meeting Date: Tuesday, 20 April 2010 Meeting Time: 6:30 - 8:30 p.m. Location: Marchex - 520 Pike Street Cost: Admission is free and open to the public Info: http://seattleperl.org/ ==================================================== Tuesday, April 20, is the next meeting of the THE SEATTLE PERL USERS GROUP. This Month's Talk ----------------- Perl 5.12.0 - Double-Click for a Martini - Features - Deprecations - Modules & Pragmata - Docs - Faster, Better, Smarter - Evil Wizards - Excelsior About Josh ben Jore ------------------- Josh ben Jore is a crab wranger according to the internet or a serial presenter at YAPC::NA or a senior engineer at WhitePages.com. He came to Perl because he had to write a CGI appfor a political campaign and he'd heard that Perl was best for that sort of thing. Since then, he's been slacking off by writing or maintaining several dozen CPAN modules. His latest perl-fu has been to shrink the memory usage of some honking big in-memory databases and separately, helping get perl-5.12.0 out the door. Pre-Meeting =========== If you are so inclined, please come to the pre-meeting at the nearby Elephant & Castle pub on 5th & Union (see map link below). Come enjoy some friendly conversation and perhaps a favorite beverage (they have a full restaurant too). We can usually be found at the back under the TV near the rear entrance that goes up into the hotel (if you enter through the front doors, just go straight back past the bar). We'll be there from 5:00 pm to 6:19 pm. Meeting Location ================ Marchex 520 Pike Street, Suite 1800 Seattle, WA 98101 The building is just East of Westlake Center. Enter from Pike Street. http://xrl.us/spugmap Due to all of the shopping around us there is plenty of parking available in garages, but it can be hard to find street parking in the evening. There is also a parking garage in the building, but check the rates and closing time (subject to change due to downtown events)! Attendees will need to wait near the elevators in the lobby and a Marchex employee will provide access to the 18th floor where the meeting room is located. If no one shows up to let you in, call (425) 533-2964 to let them know you're in the lobby. See you there! -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From ingy at ingy.net Mon Apr 12 19:59:11 2010 From: ingy at ingy.net (Ingy dot Net) Date: Mon, 12 Apr 2010 19:59:11 -0700 Subject: SPUG: Come Invade the Seattle Python Interest Group Meeting with Ingy In-Reply-To: References: Message-ID: I had about 15-20 peeps. A couple old school Seattle Perl guys. Even Oren Ben-Kiki (of YAML fame) showed up. The talk went well and people even hung around to add a Java emitter. The slides are posted: * http://www.seapig.org/CdentSlides * http://ingydotnet.github.com/cdent-seapig2010-talk/ * http://github.com/ingydotnet/cdent-seapig2010-talk If you want to see the latest summary of C'Dent in action, check out this page (hot off the press): * http://cdent.org/examples/hello-world/ You can install C'Dent from here: * http://pypi.python.org/pypi/cdent/ * http://github.com/ingydotnet/cdent Cheers, Ingy PS. I'd love to show this stuff at an upcoming SPUG meeting. What time slots are available?? On Mon, Apr 12, 2010 at 11:34 AM, Joshua ben Jore wrote: > On Tue, Apr 6, 2010 at 9:10 PM, Ingy dot Net wrote: >> Hey Spugsters! >> ... >> Anyway, I want you guys to come because they told me to only expect 5 or so >> attendees. Let's show these seapigs how we roll!!!! > > They're totally nice folks btw. When I've been, there's been about a > dozen attendees. > > Josh > From cjac at colliertech.org Mon Apr 12 20:25:35 2010 From: cjac at colliertech.org (C.J. Adams-Collier) Date: Mon, 12 Apr 2010 20:25:35 -0700 Subject: SPUG: GATE Message-ID: <1271129135.7239.74.camel@calcifer> I think I remember someone here mentioning that they know something about GATE.... am I misremembering? http://gate.ac.uk/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From MichaelRWolf at att.net Mon Apr 12 22:38:31 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Mon, 12 Apr 2010 22:38:31 -0700 Subject: SPUG: Come Invade the Seattle Python Interest Group Meeting with Ingy In-Reply-To: References: Message-ID: <29E70594-E1CF-4E87-80EC-9F6B46EE442F@att.net> On Apr 12, 2010, at 11:34 AM, Joshua ben Jore wrote: > On Tue, Apr 6, 2010 at 9:10 PM, Ingy dot Net wrote: >> Hey Spugsters! >> ... >> Anyway, I want you guys to come because they told me to only expect >> 5 or so >> attendees. Let's show these seapigs how we roll!!!! > > They're totally nice folks btw. When I've been, there's been about a > dozen attendees. Ditto. But, the one time I've been, Ingy was there, so it's not entirely clear that it wasn't his calming, civilizing influence. No, I take that back. It's clear that they were civilized even without the Ingy factor. -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From andrew at sweger.net Tue Apr 13 08:26:36 2010 From: andrew at sweger.net (Andrew Sweger) Date: Tue, 13 Apr 2010 08:26:36 -0700 (PDT) Subject: SPUG: Come Invade the Seattle Python Interest Group Meeting with Ingy In-Reply-To: Message-ID: On Mon, 12 Apr 2010, Ingy dot Net wrote: > You can install C'Dent from here: > > * http://pypi.python.org/pypi/cdent/ > * http://github.com/ingydotnet/cdent > > Cheers, Ingy > > PS. I'd love to show this stuff at an upcoming SPUG meeting. What time > slots are available?? Currently, all third-Tuesdays after April are open. -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From andrew at sweger.net Wed Apr 14 17:49:47 2010 From: andrew at sweger.net (Andrew Sweger) Date: Wed, 14 Apr 2010 17:49:47 -0700 (PDT) Subject: SPUG: bwahaaa deprecations in 5.12 Message-ID: Numerous Perl 4-era libraries: "During final testing before the release of 5.12.0, several developers discovered current production code using these ancient libraries, some inside the Perl core itself. Accordingly, the pumpking granted them a stay of execution. They will begin to warn about their deprecation in the 5.14.0 release and will be removed in the 5.16.0 release." Even stuff like this (the attention to detail) is why I love Perl. I can't help but wonder what Perl will be like in 50 years. -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From brian at massassi.com Wed Apr 14 17:51:33 2010 From: brian at massassi.com (Brian E. Lozier) Date: Wed, 14 Apr 2010 17:51:33 -0700 Subject: SPUG: bwahaaa deprecations in 5.12 In-Reply-To: References: Message-ID: Details on which libraries? On Wed, Apr 14, 2010 at 5:49 PM, Andrew Sweger wrote: > Numerous Perl 4-era libraries: > > "During final testing before the release of 5.12.0, several developers > discovered current production code using these ancient libraries, some > inside the Perl core itself. Accordingly, the pumpking granted them a stay > of execution. They will begin to warn about their deprecation in the > 5.14.0 release and will be removed in the 5.16.0 release." > > Even stuff like this (the attention to detail) is why I love Perl. I can't > help but wonder what Perl will be like in 50 years. > > -- > Andrew B. Sweger -- The great thing about multitasking is that several > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?things can go wrong at once. > > _____________________________________________________________ > Seattle Perl Users Group Mailing List > ? ? POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > ? ?MEETINGS: 3rd Tuesdays > ? ?WEB PAGE: http://seattleperl.org/ > From andrew at sweger.net Wed Apr 14 18:04:41 2010 From: andrew at sweger.net (Andrew Sweger) Date: Wed, 14 Apr 2010 18:04:41 -0700 (PDT) Subject: SPUG: bwahaaa deprecations in 5.12 In-Reply-To: Message-ID: Scroll up a couple paragraphs from: http://search.cpan.org/~jesse/perl-5.12.0/pod/perl5120delta.pod#Unicode_overhaul On Wed, 14 Apr 2010, Brian E. Lozier wrote: > Details on which libraries? -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From twists at gmail.com Wed Apr 14 21:43:02 2010 From: twists at gmail.com (Joshua ben Jore) Date: Wed, 14 Apr 2010 21:43:02 -0700 Subject: SPUG: Next month's meeting? Message-ID: Hi SPUG, would you like to have me perform something like http://www.nntp.perl.org/group/perl.perl5.porters/2010/04/msg158943.html for you? I don't know who or what we have scheduled for next month. I told YAPC::NA I'd form this into a 20 minute talk. Would someone else also like to talk? This would be a nice chance to give a shorter presentation for someone who has presented less. Josh From twists at gmail.com Wed Apr 14 21:44:32 2010 From: twists at gmail.com (Joshua ben Jore) Date: Wed, 14 Apr 2010 21:44:32 -0700 Subject: SPUG: bwahaaa deprecations in 5.12 In-Reply-To: References: Message-ID: On Wed, Apr 14, 2010 at 5:51 PM, Brian E. Lozier wrote: > Details on which libraries? Andy found that we use geopt.pl inside metaconfig which we use to generate Config which we use to build perl. We only actually run metaconfig when Configure has to change. Josh From MichaelRWolf at att.net Wed Apr 14 22:29:51 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Wed, 14 Apr 2010 22:29:51 -0700 Subject: SPUG: bwahaaa deprecations in 5.12 In-Reply-To: References: Message-ID: On Apr 14, 2010, at 5:49 PM, Andrew Sweger wrote: > Numerous Perl 4-era libraries: > > "During final testing before the release of 5.12.0, several developers > discovered current production code using these ancient libraries, some > inside the Perl core itself. Accordingly, the pumpking granted them > a stay > of execution. They will begin to warn about their deprecation in the > 5.14.0 release and will be removed in the 5.16.0 release." > > Even stuff like this (the attention to detail) is why I love Perl. I > can't > help but wonder what Perl will be like in 50 years. The great thing about multitesting is that several wrong things can go at once. :-) > -- > Andrew B. Sweger -- The great thing about multitasking is that several > things can go wrong at once. -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From MichaelRWolf at att.net Wed Apr 14 23:18:33 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Wed, 14 Apr 2010 23:18:33 -0700 Subject: SPUG: bwahaaa deprecations in 5.12 In-Reply-To: References: Message-ID: <8E8CE902-BF54-4658-BAC1-68FF8C5151BE@att.net> On Apr 14, 2010, at 5:49 PM, Andrew Sweger wrote: > Even stuff like this (the attention to detail) is why I love Perl. I > can't > help but wonder what Perl will be like in 50 years. More attention to detail. How appropriate is it that an old soul of Perl is scribe to another old soul? What's not to love? Internal Changes * The J.R.R. Tolkien quotes at the head of C source file have been checked and proper citations added, thanks to a patch from Tom Christiansen. -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From andrew at sweger.net Sun Apr 18 14:01:50 2010 From: andrew at sweger.net (Andrew Sweger) Date: Sun, 18 Apr 2010 14:01:50 -0700 (PDT) Subject: SPUG: Gabor Szabo asks... Message-ID: ...that you please take a moment to visit the poll below to answer the following question: What is the primary operating system you use for developing Perl applications? http://perlide.org/poll201004/ Thanks. P.S. - SPUG is this Tuesday evening. The announcement is here: http://mail.pm.org/pipermail/spug-list/2010-April/008933.html From andrew at sweger.net Mon Apr 19 15:18:28 2010 From: andrew at sweger.net (Andrew Sweger) Date: Mon, 19 Apr 2010 15:18:28 -0700 (PDT) Subject: SPUG: REMINDER April 2010 Seattle Perl Users Group (SPUG) Meeting Message-ID: SPUG is tomorrow night. ---------- Forwarded message ---------- From: Andrew Sweger To: SPUG Members Date: Mon, 12 Apr 2010 13:08:11 -0700 (PDT) Subject: SPUG: April 2010 Seattle Perl Users Group (SPUG) Meeting X-spam-level: * April 2010 Seattle Perl Users Group (SPUG) Meeting ==================================================== Topic: Perl 5.12.0 - Double-click for a martini Speaker: Josh ben Jore Meeting Date: Tuesday, 20 April 2010 Meeting Time: 6:30 - 8:30 p.m. Location: Marchex - 520 Pike Street Cost: Admission is free and open to the public Info: http://seattleperl.org/ ==================================================== Tuesday, April 20, is the next meeting of the THE SEATTLE PERL USERS GROUP. This Month's Talk ----------------- Perl 5.12.0 - Double-Click for a Martini - Features - Deprecations - Modules & Pragmata - Docs - Faster, Better, Smarter - Evil Wizards - Excelsior About Josh ben Jore ------------------- Josh ben Jore is a crab wranger according to the internet or a serial presenter at YAPC::NA or a senior engineer at WhitePages.com. He came to Perl because he had to write a CGI appfor a political campaign and he'd heard that Perl was best for that sort of thing. Since then, he's been slacking off by writing or maintaining several dozen CPAN modules. His latest perl-fu has been to shrink the memory usage of some honking big in-memory databases and separately, helping get perl-5.12.0 out the door. Pre-Meeting =========== If you are so inclined, please come to the pre-meeting at the nearby Elephant & Castle pub on 5th & Union (see map link below). Come enjoy some friendly conversation and perhaps a favorite beverage (they have a full restaurant too). We can usually be found at the back under the TV near the rear entrance that goes up into the hotel (if you enter through the front doors, just go straight back past the bar). We'll be there from 5:00 pm to 6:19 pm. Meeting Location ================ Marchex 520 Pike Street, Suite 1800 Seattle, WA 98101 The building is just East of Westlake Center. Enter from Pike Street. http://xrl.us/spugmap Due to all of the shopping around us there is plenty of parking available in garages, but it can be hard to find street parking in the evening. There is also a parking garage in the building, but check the rates and closing time (subject to change due to downtown events)! Attendees will need to wait near the elevators in the lobby and a Marchex employee will provide access to the 18th floor where the meeting room is located. If no one shows up to let you in, call (425) 533-2964 to let them know you're in the lobby. See you there! -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. _____________________________________________________________ Seattle Perl Users Group Mailing List POST TO: spug-list at pm.org SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list MEETINGS: 3rd Tuesdays WEB PAGE: http://seattleperl.org/ From twists at gmail.com Tue Apr 20 01:06:58 2010 From: twists at gmail.com (Joshua ben Jore) Date: Tue, 20 Apr 2010 01:06:58 -0700 Subject: SPUG: REMINDER April 2010 Seattle Perl Users Group (SPUG) Meeting In-Reply-To: References: Message-ID: On Mon, Apr 19, 2010 at 3:18 PM, Andrew Sweger wrote: > SPUG is tomorrow night. Hi, the attached talk at http://bit.ly/aEamtj might be the talk you'll miss if you don't show up. Josh, your designated speaker for the evening From twists at gmail.com Wed Apr 21 10:18:32 2010 From: twists at gmail.com (Josh Jore) Date: Wed, 21 Apr 2010 10:18:32 -0700 Subject: SPUG: REMINDER April 2010 Seattle Perl Users Group (SPUG) Meeting In-Reply-To: Message-ID: <4bcf336d.9713f30a.75ef.24a4@mx.google.com> Btw, the /actual/ slides are at http://diotalevi.isa-geek.net/~josh/100420/Perl-5.12.0.pdf I "cleverly" swapped in fake pictures for the PDF I emailed early. I'd also guess I should have cut from the length, definitely should have had more kitties. Josh On Apr 20, 2010 1:06 AM, Joshua ben Jore <twists at gmail.com> wrote: On Mon, Apr 19, 2010 at 3:18 PM, Andrew Sweger <andrew at sweger.net> wrote: > SPUG is tomorrow night. Hi, the attached talk at http://bit.ly/aEamtj might be the talk you'll miss if you don't show up. Josh, your designated speaker for the evening -------------- next part -------------- An HTML attachment was scrubbed... URL: From klevin at eskimo.com Wed Apr 21 12:15:57 2010 From: klevin at eskimo.com (=?UTF-8?B?Tm9haCBSw7htZXI=?=) Date: Wed, 21 Apr 2010 12:15:57 -0700 Subject: SPUG: ideas for future talks Message-ID: <4BCF4EED.30707@eskimo.com> I heard several people last night mention that they'd like to hear talks on Catalyst and MooseX. They're both topics I'd be interested in as well, so I figured I'd throw them out there for the rest of the list to ruminate on also. -- Noah Romer | "I got what I have now through knowing the klevin at eskimo.com | right time to tell terrible people when to PGP key available | go to hell." by finger or email | -- Leslie Caron From andrew at sweger.net Wed Apr 21 12:53:21 2010 From: andrew at sweger.net (Andrew Sweger) Date: Wed, 21 Apr 2010 12:53:21 -0700 (PDT) Subject: SPUG: ideas for future talks In-Reply-To: <4BCF4EED.30707@eskimo.com> Message-ID: Sounds great. Can you have your presentation ready by May 18th? On Wed, 21 Apr 2010, Noah R?mer wrote: > I heard several people last night mention that they'd like to hear talks > on Catalyst and MooseX. They're both topics I'd be interested in as > well, so I figured I'd throw them out there for the rest of the list to > ruminate on also. > > -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From klevin at eskimo.com Wed Apr 21 13:41:25 2010 From: klevin at eskimo.com (=?UTF-8?B?Tm9haCBSw7htZXI=?=) Date: Wed, 21 Apr 2010 13:41:25 -0700 Subject: SPUG: ideas for future talks In-Reply-To: References: Message-ID: <4BCF62F5.1000109@eskimo.com> On 04/21/2010 12:53 PM, Andrew Sweger wrote: > Sounds great. Can you have your presentation ready by May 18th? > > On Wed, 21 Apr 2010, Noah R?mer wrote: > >> I heard several people last night mention that they'd like to hear talks >> on Catalyst and MooseX. They're both topics I'd be interested in as >> well, so I figured I'd throw them out there for the rest of the list to >> ruminate on also. Whoops, I should have made it clear that they are topics I would be interested in hearing someone (else) do as well. Was chucking them out as a conversation starter, so to speak. Sorry for the confusion. -- Noah Romer | "I got what I have now through knowing the klevin at eskimo.com | right time to tell terrible people when to PGP key available | go to hell." by finger or email | -- Leslie Caron From twists at gmail.com Wed Apr 21 14:36:32 2010 From: twists at gmail.com (Joshua ben Jore) Date: Wed, 21 Apr 2010 14:36:32 -0700 Subject: SPUG: ideas for future talks In-Reply-To: <4BCF62F5.1000109@eskimo.com> References: <4BCF62F5.1000109@eskimo.com> Message-ID: On Wed, Apr 21, 2010 at 1:41 PM, Noah R?mer wrote: > On 04/21/2010 12:53 PM, Andrew Sweger wrote: >> Sounds great. Can you have your presentation ready by May 18th? >> >> On Wed, 21 Apr 2010, Noah R?mer wrote: >> >>> I heard several people last night mention that they'd like to hear talks >>> on Catalyst and MooseX. They're both topics I'd be interested in as >>> well, so I figured I'd throw them out there for the rest of the list to >>> ruminate on also. > > Whoops, I should have made it clear that they are topics I would be > interested in hearing someone (else) do as well. Was chucking them out > as a conversation starter, so to speak. In particular, you and others claimed that using Moose+Catalyst made it obvious that this was a better way, also that it made it obvious that Rails is from a prior generation. I'd like to see those differences made clear. Josh From MichaelRWolf at att.net Wed Apr 21 15:05:52 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Wed, 21 Apr 2010 15:05:52 -0700 Subject: SPUG: REMINDER April 2010 Seattle Perl Users Group (SPUG) Meeting In-Reply-To: <4bcf336d.9713f30a.75ef.24a4@mx.google.com> References: <4bcf336d.9713f30a.75ef.24a4@mx.google.com> Message-ID: On Apr 21, 2010, at 10:18 AM, Josh Jore wrote: > Btw, the /actual/ slides are at http://diotalevi.isa-geek.net/~josh/100420/Perl-5.12.0.pdf > > I "cleverly" swapped in fake pictures for the PDF I emailed early. > I'd also guess I should have cut from the length, definitely should > have had more kitties. Thanks for a great talk. One of the metrics I have for a great talk is that it sparks engaged discussion. Your talk surely did. Last night's meeting was a reminder of why I like SPUG. Great conversations during the presentation, and great ones after the presentation, both related to and unrelated to the topic. Well done. Thanks. -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From jd at commandprompt.com Wed Apr 21 15:15:57 2010 From: jd at commandprompt.com (Joshua D. Drake) Date: Wed, 21 Apr 2010 15:15:57 -0700 Subject: SPUG: ideas for future talks In-Reply-To: References: <4BCF62F5.1000109@eskimo.com> Message-ID: <1271888157.2857.12.camel@jd-desktop.unknown.charter.com> On Wed, 2010-04-21 at 14:36 -0700, Joshua ben Jore wrote: > On Wed, Apr 21, 2010 at 1:41 PM, Noah R?mer wrote: > > On 04/21/2010 12:53 PM, Andrew Sweger wrote: > >> Sounds great. Can you have your presentation ready by May 18th? > >> > >> On Wed, 21 Apr 2010, Noah R?mer wrote: > >> > >>> I heard several people last night mention that they'd like to hear talks > >>> on Catalyst and MooseX. They're both topics I'd be interested in as > >>> well, so I figured I'd throw them out there for the rest of the list to > >>> ruminate on also. > > > > Whoops, I should have made it clear that they are topics I would be > > interested in hearing someone (else) do as well. Was chucking them out > > as a conversation starter, so to speak. > > In particular, you and others claimed that using Moose+Catalyst made > it obvious that this was a better way, also that it made it obvious > that Rails is from a prior generation. I'd like to see those > differences made clear. You know... MST will be here in June I believe. We may want to see if he can come up and talk to you guys about this. Joshua D. Drake > > Josh > _____________________________________________________________ > Seattle Perl Users Group Mailing List > POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > MEETINGS: 3rd Tuesdays > WEB PAGE: http://seattleperl.org/ > -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 503.667.4564 Consulting, Training, Support, Custom Development, Engineering From andrew at sweger.net Wed Apr 21 16:30:07 2010 From: andrew at sweger.net (Andrew Sweger) Date: Wed, 21 Apr 2010 16:30:07 -0700 (PDT) Subject: SPUG: ideas for future talks In-Reply-To: <4BCF62F5.1000109@eskimo.com> Message-ID: > > On 04/21/2010 12:53 PM, Andrew Sweger wrote: > > Sounds great. Can you have your presentation ready by May 18th? > > On Wed, 21 Apr 2010, Noah R?mer wrote: > > Whoops, I should have made it clear that they are topics I would be > interested in hearing someone (else) do as well. Was chucking them out > as a conversation starter, so to speak. No problem. Don't feel bad for misunderstanding. This is how many SPUGgers end up giving their first presentation. They're often some of the best presentations too. The person requesting the topic has a genuine interest in the topic and a fresh pair of eyes that often bring informative insights that someone already familiar with the topic may miss. As it stands now, we have the following openings (as far as I know): 18 May 15 June 20 July (warning: OSCON in PDX this week) 17 August (and so on) We haven't heard from Ingy yet if he wants to claim a particular date. But setting a deadline does wonders for motivation (please do not quote Douglas Adams). So, what's your preference? (My humble apologies for top-posting previously. It's a $work thing that messes me up often.) -- Andrew B. Sweger -- The great thing about multitasking is that several things can go wrong at once. From MichaelRWolf at att.net Wed Apr 21 16:45:03 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Wed, 21 Apr 2010 16:45:03 -0700 Subject: SPUG: ideas for future talks In-Reply-To: References: Message-ID: On Apr 21, 2010, at 4:30 PM, Andrew Sweger wrote: > 18 May > 15 June > 20 July (warning: OSCON in PDX this week) > 17 August > (and so on) > > We haven't heard from Ingy yet if he wants to claim a particular date. Don't let Ingy take your date! Commit early, commit often. You can have the date of your choice, but only if you commit first. -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From MichaelRWolf at att.net Wed Apr 21 17:27:14 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Wed, 21 Apr 2010 17:27:14 -0700 Subject: SPUG: How do *you* install multiple versions of Perl? Message-ID: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> As a follow-up to the Josh's (general, theoretical) talk last night on Perl 5.12, I'd like to pose a (specific, practical) question: How do you install and use multiple versions of Perl on a single machine? I'm interested because, even 2 years after its introduction, Perl 5.10 is still not common in many environments (default OS installations, web hosting services, corporate IT environments, etc). I'm guessing (based on my own experience) that it's often easier for folks not on the cutting edge to use the given version of Perl (usually 5.8) instead of stepping up to a newer one. Perhaps sharing how you do it would encourage others to do so. Here are some important issues to address when installing a 2nd (or 10th) version of Perl on a machine. How do you handle them in your environments? - Code that depends on the current version of Perl MUST continue to work - OS processes MUST use the version of Perl shipped with the OS (or risk breaking, not really an option) - New code CANNOT use the installed version of Perl, since it's exercising new features - The new and old Perl versions MAY share core modules or add-ons (e.g. from CPAN) - Laptop users MIGHT BE their own administrators, but MIGHT NOT BE - Corporate environments are often designed for stability, not flexibility. - Various she-bang constructs MIGHT dictate or suggest a particular version - Command-line arguments and paths MIGHT do so, too - Library search can be controlled via 'use directory' or by setting PERL5LIB - Library structure is contained in directories, some of which may be symlinks So... if you have BCP's (Best Current Practices) or even illustrative failures, please it to encourage a wider experimentation with newer versions of Perl. It would be great to get a broad mix of answers: - Windows and *nix - development and deployment - laptop and server Thanks for sharing... -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From bill at celestial.com Wed Apr 21 18:30:20 2010 From: bill at celestial.com (Bill Campbell) Date: Wed, 21 Apr 2010 18:30:20 -0700 Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> Message-ID: <20100422013020.GA19777@ayn.mi.celestial.com> On Wed, Apr 21, 2010, Michael R. Wolf wrote: > As a follow-up to the Josh's (general, theoretical) talk last night on > Perl 5.12, I'd like to pose a (specific, practical) question: > How do you install and use multiple versions of Perl on a single > machine? We use the OpenPKG portable package management system which allows multiple instances, each with its own version of perl, python, etc. http://www.openpkg.org Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 Skype: jwccsllc (206) 855-5792 A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects. Robert Heinlein From bill at celestial.com Wed Apr 21 18:30:20 2010 From: bill at celestial.com (Bill Campbell) Date: Wed, 21 Apr 2010 18:30:20 -0700 Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> Message-ID: <20100422013020.GA19777@ayn.mi.celestial.com> On Wed, Apr 21, 2010, Michael R. Wolf wrote: > As a follow-up to the Josh's (general, theoretical) talk last night on > Perl 5.12, I'd like to pose a (specific, practical) question: > How do you install and use multiple versions of Perl on a single > machine? We use the OpenPKG portable package management system which allows multiple instances, each with its own version of perl, python, etc. http://www.openpkg.org Bill -- INTERNET: bill at celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax: (206) 232-9186 Skype: jwccsllc (206) 855-5792 A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects. Robert Heinlein From twists at gmail.com Wed Apr 21 19:43:22 2010 From: twists at gmail.com (Joshua ben Jore) Date: Wed, 21 Apr 2010 19:43:22 -0700 Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> Message-ID: On Wed, Apr 21, 2010 at 5:27 PM, Michael R. Wolf wrote: > As a follow-up to the Josh's (general, theoretical) talk last night on Perl > 5.12, I'd like to pose a (specific, practical) question: > ?How do you install and use multiple versions of Perl on a single machine? > > I'm interested because, even 2 years after its introduction, Perl 5.10 is > still not common in many environments (default OS installations, web hosting > services, corporate IT environments, etc). ?I'm guessing (based on my own > experience) that it's often easier for folks not on the cutting edge to use > the given version of Perl (usually 5.8) instead of stepping up to a newer > one. > > Perhaps sharing how you do it would encourage others to do so. I've always just encouraged installing it into a directory named after the version. So: DON'T: /usr /usr/local /usr/local/perl DO: /usr/local/perl-5.12.0 /opt/perl-5.12.0 Or, if you want to run things just for yourself, ~/bin/perl-5.12.0 You can always add the perl .../bin directory to your path. If you're on Windows, let Strawberry Perl install itself wherever it wants. Or maybe avoid installing Perl to any directory with spaces in it. Unfortunately, this might mean you need administrator privileges. If you can't avoid that, then you may find yourself upgrading core parts of our perl infrastructure to be safe for spaces. Others have avoided making all of that work, I think. Josh From mark.mertel at yahoo.com Wed Apr 21 20:16:05 2010 From: mark.mertel at yahoo.com (Mark Mertel) Date: Wed, 21 Apr 2010 20:16:05 -0700 (PDT) Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> Message-ID: <978380.92741.qm@web50103.mail.re2.yahoo.com> --- Mark Mertel mark.mertel at yahoo.com ________________________________ From: Joshua ben Jore To: Michael R. Wolf Cc: SPUG Members Sent: Wed, April 21, 2010 7:43:22 PM Subject: Re: SPUG: How do *you* install multiple versions of Perl? On Wed, Apr 21, 2010 at 5:27 PM, Michael R. Wolf wrote: > As a follow-up to the Josh's (general, theoretical) talk last night on Perl > 5.12, I'd like to pose a (specific, practical) question: > How do you install and use multiple versions of Perl on a single machine? > > I'm interested because, even 2 years after its introduction, Perl 5.10 is > still not common in many environments (default OS installations, web hosting > services, corporate IT environments, etc). I'm guessing (based on my own > experience) that it's often easier for folks not on the cutting edge to use > the given version of Perl (usually 5.8) instead of stepping up to a newer > one. > > Perhaps sharing how you do it would encourage others to do so. I've always just encouraged installing it into a directory named after the version. So: DON'T: /usr /usr/local /usr/local/perl DO: /usr/local/perl-5.12.0 /opt/perl-5.12.0 Or, if you want to run things just for yourself, ~/bin/perl-5.12.0 You can always add the perl .../bin directory to your path. If you're on Windows, let Strawberry Perl install itself wherever it wants. Or maybe avoid installing Perl to any directory with spaces in it. Unfortunately, this might mean you need administrator privileges. If you can't avoid that, then you may find yourself upgrading core parts of our perl infrastructure to be safe for spaces. Others have avoided making all of that work, I think. Josh _____________________________________________________________ Seattle Perl Users Group Mailing List POST TO: spug-list at pm.org SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list MEETINGS: 3rd Tuesdays WEB PAGE: http://seattleperl.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.mertel at yahoo.com Wed Apr 21 20:32:13 2010 From: mark.mertel at yahoo.com (Mark Mertel) Date: Wed, 21 Apr 2010 20:32:13 -0700 (PDT) Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: <978380.92741.qm@web50103.mail.re2.yahoo.com> References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> <978380.92741.qm@web50103.mail.re2.yahoo.com> Message-ID: <88012.96911.qm@web50108.mail.re2.yahoo.com> Somehow all of the text on my post got deleted? Anyways, I was saying that I started using MacPorts on my iBook and it puts stuff under /opt/local. It installed perl5.8.9 as a dependency for ImageMagick without checking in /usr for the stuff from the Developer package. So, now I've unintentionally got two versions of perl installed. --- Mark Mertel mark.mertel at yahoo.com ________________________________ From: Mark Mertel To: Joshua ben Jore ; Michael R. Wolf Cc: SPUG Members Sent: Wed, April 21, 2010 8:16:05 PM Subject: Re: SPUG: How do *you* install multiple versions of Perl? --- Mark Mertel mark.mertel at yahoo.com ________________________________ From: Joshua ben Jore To: Michael R. Wolf Cc: SPUG Members Sent: Wed, April 21, 2010 7:43:22 PM Subject: Re: SPUG: How do *you* install multiple versions of Perl? On Wed, Apr 21, 2010 at 5:27 PM, Michael R. Wolf wrote: > As a follow-up to the Josh's (general, theoretical) talk last night on Perl > 5.12, I'd like to pose a (specific, practical) question: > How do you install and use multiple versions of Perl on a single machine? > > I'm interested because, even 2 years after its introduction, Perl 5.10 is > still not common in many environments (default OS installations, web hosting > services, corporate IT environments, etc). I'm guessing (based on my own > experience) that it's often easier for folks not on the cutting edge to use > the given version of Perl (usually 5.8) instead of stepping up to a newer > one. > > Perhaps sharing how you do it would encourage others to do so. I've always just encouraged installing it into a directory named after the version. So: DON'T: /usr /usr/local /usr/local/perl DO: /usr/local/perl-5.12.0 /opt/perl-5.12.0 Or, if you want to run things just for yourself, ~/bin/perl-5.12.0 You can always add the perl .../bin directory to your path. If you're on Windows, let Strawberry Perl install itself wherever it wants. Or maybe avoid installing Perl to any directory with spaces in it. Unfortunately, this might mean you need administrator privileges. If you can't avoid that, then you may find yourself upgrading core parts of our perl infrastructure to be safe for spaces. Others have avoided making all of that work, I think. Josh _____________________________________________________________ Seattle Perl Users Group Mailing List POST TO: spug-list at pm.org SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list MEETINGS: 3rd Tuesdays WEB PAGE: http://seattleperl.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmeyer at helvella.org Thu Apr 22 09:56:12 2010 From: cmeyer at helvella.org (Colin Meyer) Date: Thu, 22 Apr 2010 09:56:12 -0700 Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> Message-ID: <20100422165612.GF17658@infula.marketoutsider.com> On Wed, Apr 21, 2010 at 07:43:22PM -0700, Joshua ben Jore wrote: > > DO: > /usr/local/perl-5.12.0 > /opt/perl-5.12.0 > I like to maintain a separate Perl installation for each application, so I name my install directories like so: /opt/nilcbot/perl That way, if my nilcbot app relies on some particular versions of some CPAN modules, those too are segregated from any other Perl installation. -Colin. From alwanza at yahoo.com Thu Apr 22 11:21:00 2010 From: alwanza at yahoo.com (Meryll Larkin) Date: Thu, 22 Apr 2010 11:21:00 -0700 (PDT) Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: Message-ID: <134004.64118.qm@web56005.mail.re3.yahoo.com> I have never installed more than 2 versions of Perl. I think it would be too confusing for me UNLESS testing with multiple Perl versions was the assignment I was focusing on. I agree with making the INSTALL paths as different as possible - /opt/ or /opt/bin/ if it is available is good for the "second Perl". In my case, the "first Perl" came from my distro and was installed with all the default settings. BUT I also like to make symbolic links with names I can recognize: ln -s /usr/bin/perl_5.12 /opt/bin/perl_5.12 ln -s /usr/bin/perl_5.08 /usr/local/perl Then I can control which I am using with $PATH (env variable) And verify which I am using with shell command: which (on bash) or shell command: type (on ksh) Meryll From MichaelRWolf at att.net Thu Apr 22 14:08:29 2010 From: MichaelRWolf at att.net (Michael R. Wolf) Date: Thu, 22 Apr 2010 14:08:29 -0700 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? Message-ID: I let myself get sucker punched! I wrote my own web crawler based on WWW::Mechanize because my preliminary research indicated that crawlers were simple. [Aside: If you're tempted to do that, let me save you some time. Don't. They are *conceptually* simple (GET page, push links on queue, iterate), but there are many levels of devils lurking in the details.] Having finished phase 1 (way behind schedule and way over budget), I'm looking for a better web crawler solution for phase 2. Suggestions? Thanks, Michael P.S. Even if you you told me *personally* at the previous SPUG meeting, please post your suggestion here so that others can learn via the email list and via search engines. Thanks. P.P.S. [Stop reading here unless you're interested in the nitty-gritty details of how I let myself get sucker punched.] OK... since you're interested, here's a short description of my long "journey", in the hope that it will help someone else. I'm now at the end of phase 1, looking to start a new phase. I can see that my proof-of-concept crawler worked, but I can also see that it's a bad business and technical decision to continue investing in what got me here. I *significantly* underestimated the complexity of a production web crawler, and the development time it would take to create/test/debut/maintain it. In my defense, I did a bit of research, and all the papers I could find said that crawling was simple: * initialize a queue with some seed URL's * GET the next page from the queue * extract the links * add links to a queue * [process the page for other information] * loop until the queue is empty So, being a bit familiar with WWW::Mechanize, I build a simple crawler around it. Then added more features as I bumped against them, then added more features, then added new features... Mech is a great module (thanks, Andy)! It's well documented, well designed, and does what it says it does. But... I eventually realized that Mech is an automated browser, and an automated browser is *not* the same thing as a crawler. For *simple* cases, that distinction is not critical, but for the kind of application I eventually need, the distinction is a deal-breaker. I even asked Andy if anyone had taken Mech to the next level to create a crawler. He said that he gets that question a lot, but doesn't know of a project that's created a crawler framework. Here's a brief requirement list for what a crawler needs to do (i.e. code that I wrote, or need to write) beyond what Mech does: * be polite (respect robots.txt, throttle page retrieveal rate to prevent overloading my machine and server machine) * have a revisit policy to refresh links (based on cost, expected benefit, anticipated expiration, actual expiration...) * cache results to prevent expensive re-access for unexpired content * prevent circular (i.e. infinate) crawls * avoid useless content (for many definitions of useless) * recognize uncanonical duplicates of canonical URL * keep crawler on same site (or virtualized duplicate servers) * monitor/admminister long (multi-day) processes (stop, start, pause, continue, recover, monitor, search logs, debug...) It's a big job to create a whole crawling environment and all the production support eco-system around it. It's not as big as Google, since I'll only be looking at a few hundred sites, but it's *much* bigger problem than I want to create with WWW::Mechanize. -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net -- Michael R. Wolf All mammals learn by playing! MichaelRWolf at att.net From cjac at colliertech.org Thu Apr 22 14:12:24 2010 From: cjac at colliertech.org (C.J. Adams-Collier) Date: Thu, 22 Apr 2010 14:12:24 -0700 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: References: Message-ID: <1271970744.3420.466.camel@calcifer> I like wget ;) On Thu, 2010-04-22 at 14:08 -0700, Michael R. Wolf wrote: > I let myself get sucker punched! I wrote my own web crawler based on > WWW::Mechanize because my preliminary research indicated that crawlers > were simple. > > [Aside: If you're tempted to do that, let me save you some time. > Don't. They are *conceptually* simple (GET page, push links on queue, > iterate), but there are many levels of devils lurking in the details.] > > Having finished phase 1 (way behind schedule and way over budget), I'm > looking for a better web crawler solution for phase 2. > > Suggestions? > > Thanks, Michael > > P.S. Even if you you told me *personally* at the previous SPUG > meeting, please post your suggestion here so that others can learn via > the email list and via search engines. Thanks. > > P.P.S. [Stop reading here unless you're interested in the > nitty-gritty details of how I let myself get sucker punched.] > > OK... since you're interested, here's a short description of my long > "journey", in the hope that it will help someone else. > > I'm now at the end of phase 1, looking to start a new phase. I can > see that my proof-of-concept crawler worked, but I can also see that > it's a bad business and technical decision to continue investing in > what got me here. > > I *significantly* underestimated the complexity of a production web > crawler, and the development time it would take to > create/test/debut/maintain it. > > In my defense, I did a bit of research, and all the papers I could > find said that crawling was simple: > * initialize a queue with some seed URL's > * GET the next page from the queue > * extract the links > * add links to a queue > * [process the page for other information] > * loop until the queue is empty > > So, being a bit familiar with WWW::Mechanize, I build a simple crawler > around it. Then added more features as I bumped against them, then > added more features, then added new features... > > Mech is a great module (thanks, Andy)! It's well documented, well > designed, and does what it says it does. > > But... I eventually realized that Mech is an automated browser, and an > automated browser is *not* the same thing as a crawler. For *simple* > cases, that distinction is not critical, but for the kind of > application I eventually need, the distinction is a deal-breaker. > > I even asked Andy if anyone had taken Mech to the next level to create > a crawler. He said that he gets that question a lot, but doesn't know > of a project that's created a crawler framework. > > Here's a brief requirement list for what a crawler needs to do > (i.e. code that I wrote, or need to write) beyond what Mech does: > * be polite (respect robots.txt, throttle page retrieveal rate to > prevent overloading my machine and server machine) > * have a revisit policy to refresh links (based on cost, expected > benefit, anticipated expiration, actual expiration...) > * cache results to prevent expensive re-access for unexpired content > * prevent circular (i.e. infinate) crawls > * avoid useless content (for many definitions of useless) > * recognize uncanonical duplicates of canonical URL > * keep crawler on same site (or virtualized duplicate servers) > * monitor/admminister long (multi-day) processes (stop, start, > pause, continue, recover, monitor, search logs, debug...) > > It's a big job to create a whole crawling environment and all the > production support eco-system around it. It's not as big as Google, > since I'll only be looking at a few hundred sites, but it's *much* > bigger problem than I want to create with WWW::Mechanize. > > -- > Michael R. Wolf > All mammals learn by playing! > MichaelRWolf at att.net > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: This is a digitally signed message part URL: From ryanc at greengrey.org Thu Apr 22 14:21:47 2010 From: ryanc at greengrey.org (Ryan Corder) Date: Thu, 22 Apr 2010 14:21:47 -0700 Subject: SPUG: How do *you* install multiple versions of Perl? In-Reply-To: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> References: <762224F9-3589-4D2D-B0B9-E4F18AF19E53@att.net> Message-ID: <20100422212147.GC24958@greengrey.org> On Wed, Apr 21, 2010 at 05:27:14PM -0700, Michael R. Wolf wrote: | As a follow-up to the Josh's (general, theoretical) talk last night | on Perl 5.12, I'd like to pose a (specific, practical) question: | How do you install and use multiple versions of Perl on a single | machine? What about perlbrew? http://search.cpan.org/dist/App-perlbrew/lib/App/perlbrew.pm -- Ryan Corder || () ASCII ribbon campaign || /\ against HTML email http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x1CB59D69 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available URL: From hallta at gmail.com Thu Apr 22 14:43:36 2010 From: hallta at gmail.com (Trevor Hall) Date: Thu, 22 Apr 2010 14:43:36 -0700 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: References: Message-ID: There are a few out there now a days. I've had some success in the past with Nutch (http://lucene.apache.org/nutch/). Plays pretty well with other Apache projects, too. Heritrix (http://crawler.archive.org/) is another semi-popular one. Snags more than text, which may be of use. I know there are plenty others, but these two had shined for their particular niche last time I researched it (which was a few years ago now). - Trevor On Thu, Apr 22, 2010 at 2:08 PM, Michael R. Wolf wrote: > I let myself get sucker punched! I wrote my own web crawler based on > WWW::Mechanize because my preliminary research indicated that crawlers > were simple. > > [Aside: If you're tempted to do that, let me save you some time. > Don't. They are *conceptually* simple (GET page, push links on queue, > iterate), but there are many levels of devils lurking in the details.] > > Having finished phase 1 (way behind schedule and way over budget), I'm > looking for a better web crawler solution for phase 2. > > Suggestions? > > Thanks, Michael > > P.S. Even if you you told me *personally* at the previous SPUG > meeting, please post your suggestion here so that others can learn via > the email list and via search engines. Thanks. > > P.P.S. [Stop reading here unless you're interested in the > nitty-gritty details of how I let myself get sucker punched.] > > OK... since you're interested, here's a short description of my long > "journey", in the hope that it will help someone else. > > I'm now at the end of phase 1, looking to start a new phase. I can > see that my proof-of-concept crawler worked, but I can also see that > it's a bad business and technical decision to continue investing in > what got me here. > > I *significantly* underestimated the complexity of a production web > crawler, and the development time it would take to > create/test/debut/maintain it. > > In my defense, I did a bit of research, and all the papers I could > find said that crawling was simple: > * initialize a queue with some seed URL's > * GET the next page from the queue > * extract the links > * add links to a queue > * [process the page for other information] > * loop until the queue is empty > > So, being a bit familiar with WWW::Mechanize, I build a simple crawler > around it. Then added more features as I bumped against them, then > added more features, then added new features... > > Mech is a great module (thanks, Andy)! It's well documented, well > designed, and does what it says it does. > > But... I eventually realized that Mech is an automated browser, and an > automated browser is *not* the same thing as a crawler. For *simple* > cases, that distinction is not critical, but for the kind of > application I eventually need, the distinction is a deal-breaker. > > I even asked Andy if anyone had taken Mech to the next level to create > a crawler. He said that he gets that question a lot, but doesn't know > of a project that's created a crawler framework. > > Here's a brief requirement list for what a crawler needs to do > (i.e. code that I wrote, or need to write) beyond what Mech does: > * be polite (respect robots.txt, throttle page retrieveal rate to > prevent overloading my machine and server machine) > * have a revisit policy to refresh links (based on cost, expected > benefit, anticipated expiration, actual expiration...) > * cache results to prevent expensive re-access for unexpired content > * prevent circular (i.e. infinate) crawls > * avoid useless content (for many definitions of useless) > * recognize uncanonical duplicates of canonical URL > * keep crawler on same site (or virtualized duplicate servers) > * monitor/admminister long (multi-day) processes (stop, start, > pause, continue, recover, monitor, search logs, debug...) > > It's a big job to create a whole crawling environment and all the > production support eco-system around it. It's not as big as Google, > since I'll only be looking at a few hundred sites, but it's *much* > bigger problem than I want to create with WWW::Mechanize. > > -- > Michael R. Wolf > All mammals learn by playing! > MichaelRWolf at att.net > > > > > -- > Michael R. Wolf > All mammals learn by playing! > MichaelRWolf at att.net > > > > > _____________________________________________________________ > Seattle Perl Users Group Mailing List > POST TO: spug-list at pm.org > SUBSCRIPTION: http://mail.pm.org/mailman/listinfo/spug-list > MEETINGS: 3rd Tuesdays > WEB PAGE: http://seattleperl.org/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmeyer at helvella.org Thu Apr 22 16:30:18 2010 From: cmeyer at helvella.org (Colin Meyer) Date: Thu, 22 Apr 2010 16:30:18 -0700 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: References: Message-ID: <20100422233018.GG17658@infula.marketoutsider.com> Web crawling is by no means trivial or simple. There are all sorts of things to consider, such as: . site throttling . graph cycle detection . endless graph detection (you've seen Fred's roach motel california for crawlers, or whatever he calls it) . optimizing the crawling queue . detecting redundant content (rather difficult) . extracting links from js . etc, etc Heritrix, as Trevor mentioned, is full featured, very mature, and has a bunch of users and an active mailing list. You don't need to be a java programmer to make use of it; it is very configurable. If you don't use heritrix, it is still well worth having a good look at the project. It has good architecture, and implements all of what you'd need for most crawling. If you want to write plugins, it has a finely documented API, and the mailing list users are helpful. My trouble with heritrix is that it is geared for deep, archival crawling (it's what crawls for the wayback machine), where I wanted shallow, iterative crawling, to find new content. There's a project that implements a good crawler in Perl, but it is abandoned: http://search.cpan.org/~dmaki/Gungho-0.09008/ I played with Gungho a bit, and found it to be mostly what I wanted, except that to customize it, you have to write code, instead of tweaking parameters, as you can do with heritrix. -Colin. On Thu, Apr 22, 2010 at 02:08:29PM -0700, Michael R. Wolf wrote: > I let myself get sucker punched! I wrote my own web crawler based on > WWW::Mechanize because my preliminary research indicated that crawlers > were simple. > > [Aside: If you're tempted to do that, let me save you some time. > Don't. They are *conceptually* simple (GET page, push links on queue, > iterate), but there are many levels of devils lurking in the details.] > > Having finished phase 1 (way behind schedule and way over budget), I'm > looking for a better web crawler solution for phase 2. From m3047 at m3047.net Fri Apr 23 00:49:29 2010 From: m3047 at m3047.net (Fred Morris) Date: Thu, 22 Apr 2010 23:49:29 -0800 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: <20100422233018.GG17658@infula.marketoutsider.com> References: <20100422233018.GG17658@infula.marketoutsider.com> Message-ID: <201004222349.29744.m3047@m3047.net> On Thursday 22 April 2010 15:30, Colin Meyer wrote: > My trouble with heritrix is that it is geared for deep, archival > crawling (it's what crawls for the wayback machine), where I wanted > shallow, iterative crawling, to find new content. [...] There is a whole mathematics to heuristics. I have Judea Pearl's _Heuristics_ on my bookshelf. Honestly though, nobody cares. It's really not that complicated once you start it on a mathematical footing. I doubt that a lot of what people are presently trying to do was envisioned when this book was written; but to get it on a mathematical footing, here's how. (Spoiler: no code.) That's the (meta) problem: nobody cares about making it less complicated, they want to solve their little frog pond problem NOW! I don't know what Michael is trying to do. -- Fred From m3047 at m3047.net Fri Apr 23 00:36:12 2010 From: m3047 at m3047.net (Fred Morris) Date: Thu, 22 Apr 2010 23:36:12 -0800 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: <20100422233018.GG17658@infula.marketoutsider.com> References: <20100422233018.GG17658@infula.marketoutsider.com> Message-ID: <201004222336.12469.m3047@m3047.net> On Thursday 22 April 2010 15:30, Colin Meyer wrote: > [...] > . endless graph detection (you've seen Fred's roach motel california > for crawlers, or whatever he calls it) 'Bot Motel. Thanks. People I work with now have taken it to further extremes... not on company time, but this seems to appeal to the rank and file. Do I need to give a talk on it? And that's just the legitimate (?) defenses. You get into things like wildcarded domains or paths and the complexity rises. A lot of the wildcarding is legit, too.. or sort of. It gets infinitely more complex when your caching DNS serves defaults. Verisign wanted to do this and was spanked, but I can tell you that any number of small ISPs will do this. Then again, there is bone strange. Have you tried this? dig www.facebook.com ns How about this? dig www.facebook.com soa Why? On all of the gods' and goddesses' green or blasted earth... why? We seem to spend a lot of time on DNS, these days. Without some notion of "framework for what, exactly?" I don't know what to say. I've got a perfectly good CD creator crawler (in Perl) for a particular Zope (python) photo archiving scaffold which I created. Michael: are you crawling the wild, or the tame? -- Fred From ben at reser.org Fri Apr 23 02:04:17 2010 From: ben at reser.org (Ben Reser) Date: Fri, 23 Apr 2010 02:04:17 -0700 Subject: SPUG: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: <201004222336.12469.m3047@m3047.net> References: <20100422233018.GG17658@infula.marketoutsider.com> <201004222336.12469.m3047@m3047.net> Message-ID: On Fri, Apr 23, 2010 at 12:36 AM, Fred Morris wrote: > Then again, there is bone strange. Have you tried this? > > dig www.facebook.com ns > > How about this? > > dig www.facebook.com soa > > Why? On all of the gods' and goddesses' green or blasted earth... why? What's so weird about that, they've delegated www.facebook.com. Probably delegated it to some sort of content distribution network. From m3047 at m3047.net Fri Apr 23 19:42:53 2010 From: m3047 at m3047.net (Fred Morris) Date: Fri, 23 Apr 2010 18:42:53 -0800 Subject: SPUG: OT: DNS Re: Suggestions for web crawler framework, toolkit, or reusable pieces? In-Reply-To: References: <201004222336.12469.m3047@m3047.net> Message-ID: <201004231842.53177.m3047@m3047.net> On Friday 23 April 2010 01:04, Ben Reser wrote: > On Fri, Apr 23, 2010 at 12:36 AM, Fred Morris wrote: > > Then again, there is bone strange. Have you tried this? > > > > dig www.facebook.com ns > > > > How about this? > > > > dig www.facebook.com soa > > > > Why? On all of the gods' and goddesses' green or blasted earth... why? > > What's so weird about that, they've delegated www.facebook.com. > Probably delegated it to some sort of content distribution network. Sure. Maybe. Lots of ways to do that. Why this one? Plus the distribution isn't as good as the nameservers for facebook.com. They could have CNAMEd it, that would've been an option. That's what visa.com and fbi.gov do (Akamai). (fbi.gov however doesn't even CNAME... Fail!) Here's another way to do it: dig www.bankofamerica.com a dig bankofamerica.com a Why? Just sayin'. Thanks to my current job, I know more about the pecularities of DNS practica than anyone needs to know... -- Fred