From doc at redbrick.dcu.ie Mon Apr 5 05:10:38 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* Message-ID: <20040405101038.GB31903@carbon.redbrick.dcu.ie> Anyone alive? List mods? - DoC From daveb at esat.net Mon Apr 5 05:17:13 2004 From: daveb at esat.net (Dave Burke) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* In-Reply-To: Message from Dave O Connor of "Mon, 05 Apr 2004 11:10:38 BST." <20040405101038.GB31903@carbon.redbrick.dcu.ie> References: <20040405101038.GB31903@carbon.redbrick.dcu.ie> Message-ID: *checks pulse* ... yup alive over here ... I had presumed this list was depreciated in favor of dublin.pm ?? At least there has been traffic on that list, although not since you posted last august... Dave Dave O Connor wrote: > >Anyone alive? List mods? > > - DoC >_______________________________________________ >Dublin-pm mailing list >Dublin-pm@mail.pm.org >http://mail.pm.org/mailman/listinfo/dublin-pm > From grant at mclean.net.nz Mon Apr 5 05:26:43 2004 From: grant at mclean.net.nz (Grant McLean) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* In-Reply-To: <20040405101038.GB31903@carbon.redbrick.dcu.ie> References: <20040405101038.GB31903@carbon.redbrick.dcu.ie> Message-ID: <40713463.4060607@mclean.net.nz> Dave O Connor wrote: > Anyone alive? List mods? > > - DoC Hi Dave I subscribed to this list when I was heading to Dublin and looking for Perl work. That was August 2001 and yours is the first message to the list I've seen in that time. I'm back living in New Zealand now. Cheers Grant (aka grantm@cpan.org) From doc+igaming at redbrick.dcu.ie Mon Apr 5 05:26:53 2004 From: doc+igaming at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* In-Reply-To: References: <20040405101038.GB31903@carbon.redbrick.dcu.ie> Message-ID: <20040405102653.GD31903@carbon.redbrick.dcu.ie> Yep, am attempting to resurrect this list, and the tech guys at pm.org need to see if there are mods on here. If not, bammo. I'll give it a few days. Ta. - DoC Dave Burke said on Mon, Apr 05, 2004 at 11:17:13AM +0100: > > *checks pulse* ... yup alive over here ... > > I had presumed this list was depreciated in favor of dublin.pm ?? At least > there has been traffic on that list, although not since you posted last > august... > > > Dave > > > > Dave O Connor wrote: > > > >Anyone alive? List mods? > > > > - DoC > >_______________________________________________ > >Dublin-pm mailing list > >Dublin-pm@mail.pm.org > >http://mail.pm.org/mailman/listinfo/dublin-pm > > > > > From fergal at esatclear.ie Mon Apr 5 05:36:21 2004 From: fergal at esatclear.ie (Fergal Daly) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* In-Reply-To: <20040405101038.GB31903@carbon.redbrick.dcu.ie> References: <20040405101038.GB31903@carbon.redbrick.dcu.ie> Message-ID: <20040405103621.GC19805@dyn.fergaldaly.com> On Mon, Apr 05, 2004 at 11:10:38AM +0100, Dave O Connor wrote: > Anyone alive? List mods? It seems you are the list mod. Nice of someone at pm.org to change the password without contacting me :-) What's the plan anyway? F From doc+igaming at redbrick.dcu.ie Mon Apr 5 05:43:22 2004 From: doc+igaming at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* In-Reply-To: <20040405103621.GC19805@dyn.fergaldaly.com> References: <20040405101038.GB31903@carbon.redbrick.dcu.ie> <20040405103621.GC19805@dyn.fergaldaly.com> Message-ID: <20040405104322.GG31903@carbon.redbrick.dcu.ie> Working on a new webpage at the moment, probably will be another week or so before it's presentable (incidentally, I am atrocious at webpages, so anybody willing to take a look when I'm almost-done (I'm more of a content man, myself) and suggest a better layout is more than welcome). After that, when we've got the list in order (I don't actually have the admin password yet) and the webpage, it's time to solicit for members. I'm guessing we'll start from scratch, since some people on here would be less then interested still, and far be it from me to force membership on them :) I'll be contacting Belfast.pm as soon as this is all out of the way to see about details for a bunch of us going to YAPC::Europe this year (It's in -Belfast-, so it'd be bad form not for somebody to turn up at least, if not a gang of us). More as I get stuff sorted. - DoC Fergal Daly said on Mon, Apr 05, 2004 at 11:36:21AM +0100: > On Mon, Apr 05, 2004 at 11:10:38AM +0100, Dave O Connor wrote: > > Anyone alive? List mods? > > It seems you are the list mod. Nice of someone at pm.org to change the > password without contacting me :-) > > What's the plan anyway? > > F > From fergal at esatclear.ie Mon Apr 5 05:57:55 2004 From: fergal at esatclear.ie (Fergal Daly) Date: Mon Aug 2 21:29:56 2004 Subject: [Dublin-pm] *ping* In-Reply-To: <20040405104322.GG31903@carbon.redbrick.dcu.ie> References: <20040405101038.GB31903@carbon.redbrick.dcu.ie> <20040405103621.GC19805@dyn.fergaldaly.com> <20040405104322.GG31903@carbon.redbrick.dcu.ie> Message-ID: <20040405105755.GD19805@dyn.fergaldaly.com> On Mon, Apr 05, 2004 at 11:43:22AM +0100, Dave O Connor wrote: > Working on a new webpage at the moment, probably will be another week or so > before it's presentable (incidentally, I am atrocious at webpages, so anybody > willing to take a look when I'm almost-done (I'm more of a content man, > myself) and suggest a better layout is more than welcome). > > After that, when we've got the list in order (I don't actually have the admin > password yet) and the webpage, it's time to solicit for members. I'm guessing > we'll start from scratch, since some people on here would be less then > interested still, and far be it from me to force membership on them :) http://mail.pm.org/mailman/admindb/dublin-pm is the admin page, your email is listed and my password no longer works, I wonder what they're up to. I'll email them now about transferring. I thought there was no one else on this list - I was a little surprised to see Dave Burke's mail actually. I'll get you the membership list from the old list but it seems to be off at the moment. > I'll be contacting Belfast.pm as soon as this is all out of the way to see > about details for a bunch of us going to YAPC::Europe this year (It's in > -Belfast-, so it'd be bad form not for somebody to turn up at least, if not a > gang of us). I was planning on going although I haven't seen any publicity, yet. F From johnd at pci.ie Mon Apr 5 14:17:44 2004 From: johnd at pci.ie (John Day) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] [Dublin-pm] *ping* Message-ID: Hi folks, don't know about you but I'm happy at last to know I'm not the only one using Perl in Dublin ;-) so, what next? what do PM's do exactly (mail, meet, drink beer...?) Dave, what plans do you have for the webpage? (wouldn't mind seeing it though I'm also a content person & pretty crap at presentation ;-) For those that don't have it or can't find it(shiver) here's the address for YAPC: HYPERLINK "http://belfast.yapc.org/"http://belfast.yapc.org/ . It's on from Wednesday 15 September till Friday 17 September & I for one wouldn't mind going to it. Anyone else planning on going (or even speaking)? J --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.647 / Virus Database: 414 - Release Date: 29/03/2004 ********************************************************************** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. *********************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.pm.org/pipermail/dublin-pm/attachments/20040405/749531dc/attachment.htm From doc at redbrick.dcu.ie Mon Apr 5 14:52:11 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] [Dublin-pm] *ping* In-Reply-To: References: Message-ID: <20040405195211.GA29162@carbon.redbrick.dcu.ie> John Day said on Mon, Apr 05, 2004 at 08:17:44PM +0100: > Hi folks, > > don't know about you but I'm happy at last to know I'm not the only one using Perl in Dublin ;-) > > so, what next? what do PM's do exactly (mail, meet, drink beer...?) I like beer. We'll see if we can get a critical mass of folks onlist (reeeeal soon now, since I now have all the setup stuff.), and see what happens from there. Definitely on for pints at some stage, although even ILUG have trouble getting folk at times :) Once the initial webpage is up, I can do the usual jazz of soliciting for members on ILUG/SAGE/c.l.p.m/{suggestions?}. A very basic thing I've knocked up is at http://www.swearing.org/temp/mongers/. Very basic, and similar to the old page, but will hopefully have content soon. Once It's done, I can get dublin.pm.org pointed at it. > > Dave, what plans do you have for the webpage? (wouldn't mind seeing it though I'm also a content person & pretty crap at presentation ;-) At the moment, I'm planning on going for an initial effort, with all the information we need. It'll be hosted on my linux VM, so any resources (such as Perl!) will be available to do as we wish. Idea is that maybe someone would like to take on the role of webmaster eventually, since even CSS bears little ju-ju with this old-style -tag monkey. > > Anyone else planning on going (or even speaking)? > Yep, planning at the moment on going. Haven't been to that particular place, but I go to a convention in Belfast most every year, and it's a pleasant enough place to mope around in. I'll be knocking around in #dublin-pm in irc.linux.ie. - DoC From doc at redbrick.dcu.ie Thu Apr 8 05:10:41 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Status Message-ID: <20040408101041.GA9580@carbon.redbrick.dcu.ie> Alright, quick status report on what's going to happen before calls for members go out: 1) I'm waiting on the pm.org tech guys to change the dublin.pm.org A record to point to the machine I'm hosting the pages on. It's been a few days, and a few reminders. 2) The placeholder webpage is at http://www.swearing.org/temp/mongers/. Suggestions and corrections welcome before it 'goes live'. That looks like it. The list is all nicely set up, and I'm camping out in #dublin-pm in irc.linux.ie (am afk a lot, being in work, but feel free to potter about). Anyone any further thoughts? - DoC From doc at redbrick.dcu.ie Fri Apr 9 08:46:55 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Website Message-ID: <20040409134655.GA30711@carbon.redbrick.dcu.ie> Alright, the website aforementioned is at http://dublin.pm.org/ I'm still waiting for DNS to switch over, but the page won't be any different once it does. Aside from anyone spotting anything direly bad, or having a most excellent suggestion about what to put on it, I'll be getting the ball rolling on publicity for the group later today. Comments, suggestions? Ta. - DoC From doc at redbrick.dcu.ie Fri Apr 9 11:37:02 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Dublin.pm Message-ID: <20040409163702.GA19866@carbon.redbrick.dcu.ie> (Crossposting this here for reference. This has gone to c.l.p.m, ilug, sage-ie, ie.comp, and redbrick and DMZ newsgroups. Anyone feeling like posting it on private groups like Matrix, I'd be grateful. I'll post it on boards.ie myself in a bit.) To whom it may concern, This is to announce the (re)formation of the Dublin perl Mongers user group, under the management of myself. Any perl users in the Dublin, Ireland area (And beyond!) are invited to join. Details, and mailing list information are available on the (bare-bones!) Dublin.pm website at http://dublin.pm.org. Although this incarnation is in its initial stages, we hope to arrange f2f meetings in the near future, and also hope to attend YAPC::Europe as a user group later this year. More on this onlist. Any comment, suggestions, questions, etc. are welcome at the address above Many thanks to Fergal Daly for graciously allowing Dublin.pm to fall to me to tame, and Dave Cross for getting stuff set up. - DoC From art_wolf at redbrick.dcu.ie Fri Apr 9 12:26:10 2004 From: art_wolf at redbrick.dcu.ie (Art) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Intro Message-ID: <20040409172610.GA6135@carbon.redbrick.dcu.ie> So...I was told there was beer involved.. :) Oh and on the YAPC (http://belfast.yapc.org/) that John Day was on about I like the idea - "Perl for Profit" though it does sounds like one of those pyramid scheme's ;) Art -- [23:17 06/April/2004 #lobby] Hrm, if I'm not winning 'sweetest ass' - and my ass is winning for being hairy in 'best hair' - then logic would dictate that drusilla has one hell of a hairy arse. *ponders* From doc at redbrick.dcu.ie Fri Apr 9 12:30:45 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Intro In-Reply-To: <20040409172610.GA6135@carbon.redbrick.dcu.ie> References: <20040409172610.GA6135@carbon.redbrick.dcu.ie> Message-ID: <20040409173045.GA32610@carbon.redbrick.dcu.ie> Art said on Fri, Apr 09, 2004 at 06:26:10PM +0100: > So...I was told there was beer involved.. :) Possibly, possibly :) > > Oh and on the YAPC (http://belfast.yapc.org/) that John Day was on about I like the idea - "Perl for Profit" though it does sounds like one of those > pyramid scheme's ;) > 1. Use perl 2. 3. Profit! Looks like an interesting talk. Apart (obviously) from the free software aspect of the language, can anyone give examples of how they 'saved' money with perl? As an individual? Aside from writing your own dodgy eBay auction camper :) Welcome, BTW. - DoC From doc at redbrick.dcu.ie Tue Apr 13 04:10:13 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Perl, then. Message-ID: <20040413091013.GA12136@carbon.redbrick.dcu.ie> Morning, Well, I guess we've got a good few subscribers now, enough to not make me repeat myself later :) Coming onto 40 subscribers now. Ta for signing up, and apologies for making it seem like a dead list over the weekend. So, where to go from here. The website (as it stands now) is basically a placeholder (although being as it is several pages, is a little more serviceable than that). If anyone feels like doing up what these days is accepted as a webpage, please do feel free to let me know. (You can tell I haven't made a webpage, to speak of, since about 1997). Also, for the moment, the list is members posting only. I've rejected about 5 mails that were spam sent to the list over the weekend, so I think this is worthwhile (Anyone on ILUG can attest to the annoyance of spam on the list). Perhaps in a few weeks, if it gets super-annoying for one person to do, I'll be asking if anyone wants to help moderate the list. On another note, YAPC::Europe is in Belfast this year (http://belfast.yapc.org). I've made initial inquiries to them about hotel conference rates, etc. According to the wiki, the conference centre is '90 seconds' walk from the Europa Hotel, which makes it dead close to one of the train stations in belfast (the driver on the enterprise seems to flip a coin to decide which to go to). Anyone interested, even slightly (it's early days yet) can contact me offlist, and I'll build up a list of interested parties. I'm also welcome to suggestions for meeting venues; (My current workplace has a decent meeting room with A/V etc, but we're moving office in 2 weeks, so my own situation is in a bit of a jocker at the moment). I'm hoping to get at least a social meeting done in the next few weeks, depending on interest. So. Perl, then. - DoC From nick at netability.ie Wed Apr 14 03:41:44 2004 From: nick at netability.ie (Nick Hilliard) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Perl, then. In-Reply-To: <20040413091013.GA12136@carbon.redbrick.dcu.ie> References: <20040413091013.GA12136@carbon.redbrick.dcu.ie> Message-ID: <407CF948.9000806@netability.ie> > Also, for the moment, the list is members posting only. I've rejected > about 5 mails that were spam sent to the list over the weekend If mailman is set to autoreply to non-member bounce messages, you should expect this number to rise very far, very fast. Nick From doc at redbrick.dcu.ie Wed Apr 14 04:14:28 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Perl, then. In-Reply-To: <407CF948.9000806@netability.ie> References: <20040413091013.GA12136@carbon.redbrick.dcu.ie> <407CF948.9000806@netability.ie> Message-ID: <20040414091428.GA28720@carbon.redbrick.dcu.ie> Nick Hilliard said on Wed, Apr 14, 2004 at 08:41:44AM +0000: > >Also, for the moment, the list is members posting only. I've rejected > >about 5 mails that were spam sent to the list over the weekend > > If mailman is set to autoreply to non-member bounce messages, you should > expect this number to rise very far, very fast. > Hmm, a reasonable assumption. I've turned it off for now, but I guess it means moderation has to be done zippily, if real people are assuming their message gets to the list (i.e. they don't get a 'message held for approval' mail). We'll see how it goes. Post! :) - DoC From doc at redbrick.dcu.ie Wed Apr 14 06:41:18 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] YAPC::Europe Message-ID: <20040414114118.GA29620@carbon.redbrick.dcu.ie> Karen from Belfast.pm has gotten back to me about hotels etc. for YAPC::Europe (http://belfast.yapc.org/). Hopefully she'll have pricing before the weekend for conference rates (the europa itself wasn't too helpful - 110 GBP for a double? Laah dee dah). If anyone's interested in heading to YAPC::Europe, let me know offlist (whether interested in hotels or not, want to get an idea for numbers/if anyone is coming). - DoC From doc at redbrick.dcu.ie Wed Apr 14 06:52:09 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Tip of the day: Multi-line commenting Message-ID: <20040414115209.GA6486@carbon.redbrick.dcu.ie> This comes from Dave Cross on c.l.p.m, on a discussion of multi-line commenting. Perosnally, I think perl should have something like q() except for comments, partly because it's annoying to have to either prepaend lines withhashes, or use POD, like in today's tip. If you need to comment out a block of perl code temporarily, and don't feel like adding a pile of hashes at the start of each line, you can use the POD =for directive, which tells POD this POD section is not 'for' its documentation (thus not breaking your POD documentation), and also tells perl to ignore the section until the =cut e.g. print "This code is executed\n"; =for comment print "This code is commented out. Blah!\n"; ... more code here ... You can even put barewords here! =cut POD docs can then be put in as normal, and this doesn't interfere, unlike if you just used =item and =cut, or =pod and =cut. (Which I used to do until this morning :)) Here's an exercise for the reader (since my vim skillz suck): What's an easy way to prepend a hash to a block of selected text? - DoC From kehoea at parhasard.net Wed Apr 14 07:40:03 2004 From: kehoea at parhasard.net (Aidan Kehoe) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Tip of the day: Multi-line commenting In-Reply-To: <20040414115209.GA6486@carbon.redbrick.dcu.ie> References: <20040414115209.GA6486@carbon.redbrick.dcu.ie> Message-ID: <16509.12579.871310.492970@gargle.gargle.HOWL> Ar an 14? l? de m? 4, scr?obh Dave O Connor : > Here's an exercise for the reader (since my vim skillz suck): What's an easy > way to prepend a hash to a block of selected text? M-x comment-region RET (May not work in Vim.) -- I don't care if it rains or freezes/'Long as I got my Plastic Jesus Riding on the dashboard of my car. From bobb at redbrick.dcu.ie Wed Apr 14 07:45:58 2004 From: bobb at redbrick.dcu.ie (bobb) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Tip of the day: Multi-line commenting In-Reply-To: <20040414115209.GA6486@carbon.redbrick.dcu.ie> References: <20040414115209.GA6486@carbon.redbrick.dcu.ie> Message-ID: <20040414124558.GB23070@carbon.redbrick.dcu.ie> Dave O Connor hath declared on Wednesday the 14 day of April 2004 :-: > Here's an exercise for the reader (since my vim skillz suck): What's an easy > way to prepend a hash to a block of selected text? Strangely enough there are a number of ways... :) You could mark the block in visual mode and then press colon ":" (while still in visual mode) The prompt will look like ( :'<,'> ) you can then put in a substitution of the beginning of a line with a hash, ( s/^/#/ ). Another way is the use block-visual mode (ctl-v) in column zero, Select the lines you want to comment, then use "I" to insert, press hash then , and apparently I'm your mothers brother... - bobb -- Mr. Rogers could explain shoe-tying and make it seem like the coolest thing in the world. He was our Uncle and older brother. He told us we were special, and we knew he'd be there for us. A cult leader, right down to the sneakers. From seanpor at acm.org Wed Apr 14 07:49:54 2004 From: seanpor at acm.org (Sean O'Riordain) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Tip of the day: Multi-line commenting In-Reply-To: <20040414115209.GA6486@carbon.redbrick.dcu.ie> References: <20040414115209.GA6486@carbon.redbrick.dcu.ie> Message-ID: <407D3372.1040701@acm.org> i just say something like 12,964s/^/#/ or .,.+18s/^/#/ or .,$s/^/#/ there is another "easier" way in the vi creature book... but i can't remember :-) cheers, Sean Dave O Connor wrote: > Here's an exercise for the reader (since my vim skillz suck): What's an easy > way to prepend a hash to a block of selected text? From doc at redbrick.dcu.ie Fri Apr 16 04:12:24 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:56 2004 Subject: [Dub-pm] Tip of the Day: Cheap books! Message-ID: <20040416091224.GA10551@carbon.redbrick.dcu.ie> Turns out Addison-Wesley/Prentice Hall are into offering discounts to user group members on all books though their website. "Up to 35%" off, according to the details at http://www.pm.org/AW-UGcoupon.pdf. Discount codes are in there. Offer Expires 31st May. - DoC From seanpor at acm.org Fri Apr 16 09:08:23 2004 From: seanpor at acm.org (Sean O'Riordain) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size Message-ID: <407FE8D7.9080400@acm.org> Hi folks, I've an analysis program with a couple of million records that i really need to keep in memory as i need to scan back and forth etc... With 5 million odd records (written as a couple of independent 'arrays' or should i say 'lists') the program requires quite a bit more than the 1.5Gb of ram and becomes very slow due to swapping - gentoo-linux... Each record has 5 integers and a string of max.len 30 chars... but perl takes up extra ram for each SV... I would like to be able to handle larger datasets much faster than currently... Has anybody used INLINE::C for handling large data structures - if so how do you load the info? Anybody used PDL? Any thoughts which way I should jump? cheers, Sean From david at cantrell.org.uk Fri Apr 16 09:29:39 2004 From: david at cantrell.org.uk (David Cantrell) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <407FE8D7.9080400@acm.org> References: <407FE8D7.9080400@acm.org> Message-ID: <20040416142939.GA15524@bytemark.barnyard.co.uk> On Fri, Apr 16, 2004 at 03:08:23PM +0100, Sean O'Riordain wrote: > I've an analysis program with a couple of million records that i really > need to keep in memory as i need to scan back and forth etc... With 5 > million odd records (written as a couple of independent 'arrays' or > should i say 'lists') ... Can you change the code to use hashes instead of lists? Then you could use dbm files. dbm is VERY fast, certainly faster than thrashing your swap partition. -- Lord Protector David Cantrell | http://www.cantrell.org.uk/david I hear you asking yourselves "why?". Hurd will be out in a year ... -- Linus Torvalds, in <1991Oct5.054106.4647@klaava.Helsinki.FI> From fergal at esatclear.ie Fri Apr 16 09:27:33 2004 From: fergal at esatclear.ie (Fergal Daly) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <407FE8D7.9080400@acm.org> References: <407FE8D7.9080400@acm.org> Message-ID: <20040416142733.GA12808@dyn.fergaldaly.com> Not knowing exactly what you have makes it a bit tricky if you've got 5 million things looking like [$int1, $int2, $int3, $int4, $int5, $string] x 1.5 million then you will save quite bit by having @int1s = (int x 1.5 million) @int2s = (int x 1.5 million) .. @int5s = (int x 1.5 million) @strings = (string x 1.5 million) then just pass around the index. A package like package MyObj; sub new { my $pkg = shift; my $index = shift; return bless \$index, $pkg; } sub getInt1 { my $self = shift; return $int1s[$$self]; } etc... are you could get more mem efficient and rather than using arrays for the ints, have a string for each set of ints and have methods like sub getInt1 { my $self = shift; # assume a 4 byte integer my $enc = substr($int1s, $$self*4, 4); return unpack("L", $enc); } you could also do this for the strings. It'll be slower because you'll be invoking methods, you could use subroutines if you're sure you'll never want inheritance etc. If you use Inline::C, how you load the data depends entirely on how you store it, you'll just have to write C routines for loading the data and call them from Perl, F On Fri, Apr 16, 2004 at 03:08:23PM +0100, Sean O'Riordain wrote: > Hi folks, > > I've an analysis program with a couple of million records that i really > need to keep in memory as i need to scan back and forth etc... With 5 > million odd records (written as a couple of independent 'arrays' or > should i say 'lists') the program requires quite a bit more than the > 1.5Gb of ram and becomes very slow due to swapping - gentoo-linux... > Each record has 5 integers and a string of max.len 30 chars... but perl > takes up extra ram for each SV... I would like to be able to handle > larger datasets much faster than currently... > > Has anybody used INLINE::C for handling large data structures - if so > how do you load the info? > > Anybody used PDL? > > Any thoughts which way I should jump? > > cheers, > Sean > _______________________________________________ > Dublin-pm mailing list - Dublin-pm@mail.pm.org > http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm > > From sean.oriordain at swiftcall.com Fri Apr 16 10:54:02 2004 From: sean.oriordain at swiftcall.com (Sean O'Riordain) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <20040416142733.GA12808@dyn.fergaldaly.com> References: <407FE8D7.9080400@acm.org> <20040416142733.GA12808@dyn.fergaldaly.com> Message-ID: <4080019A.9090303@swiftcall.com> Thanks Fergal. here is what is how i currently load from mysql... my $rec_count = 0; while (my $aref = $sth->fetchrow_hashref) { push @as, $aref->{timestamp_ue}; push @ae, $aref->{finish_ue}; push @adw, $aref->{dw}; push @ahh, $aref->{hh}; push @abz, $aref->{bzone}; $rec_count++; } print " $rec_count cdr records loaded\n"; this takes maybe 5 minutes - so i'm not overly worried about that... if there isn't a simple way of passing the info to inline::C, then i was thinking of just re-writing all the info to disk in an easily parseable format, ie fixed width columns. Then i was just going to do all the integer work in C and writing the results to an output file... (currently it takes more than 8 hours at 100%cpu on a 1700mhz athlon ...) i could speed up the string stuff by using a lookup table since there are only about 350 different values... in inline::C is it possible to persistently keep a C data-structure between calls ? ie malloc space for my large int arrays, and then from perl append each new line of info ? cheers, Sean Fergal Daly wrote: > Not knowing exactly what you have makes it a bit tricky if you've got 5 > million things looking like > > [$int1, $int2, $int3, $int4, $int5, $string] x 1.5 million > > then you will save quite bit by having > > @int1s = (int x 1.5 million) > @int2s = (int x 1.5 million) > .. > @int5s = (int x 1.5 million) > @strings = (string x 1.5 million) > > then just pass around the index. A package like > > package MyObj; > > sub new > { > my $pkg = shift; > my $index = shift; > return bless \$index, $pkg; > } > > sub getInt1 > { > my $self = shift; > return $int1s[$$self]; > } > > etc... > > are you could get more mem efficient and rather than using arrays for the > ints, have a string for each set of ints and have methods like > > sub getInt1 > { > my $self = shift; > > # assume a 4 byte integer > my $enc = substr($int1s, $$self*4, 4); > > return unpack("L", $enc); > } > > you could also do this for the strings. It'll be slower because you'll be > invoking methods, you could use subroutines if you're sure you'll never want > inheritance etc. > > If you use Inline::C, how you load the data depends entirely on how you store > it, you'll just have to write C routines for loading the data and call them > from Perl, > > F > On Fri, Apr 16, 2004 at 03:08:23PM +0100, Sean O'Riordain wrote: > >>Hi folks, >> >>I've an analysis program with a couple of million records that i really >>need to keep in memory as i need to scan back and forth etc... With 5 >>million odd records (written as a couple of independent 'arrays' or >>should i say 'lists') the program requires quite a bit more than the >>1.5Gb of ram and becomes very slow due to swapping - gentoo-linux... >>Each record has 5 integers and a string of max.len 30 chars... but perl >>takes up extra ram for each SV... I would like to be able to handle >>larger datasets much faster than currently... >> >>Has anybody used INLINE::C for handling large data structures - if so >>how do you load the info? >> >>Anybody used PDL? >> >>Any thoughts which way I should jump? >> >>cheers, >>Sean >>_______________________________________________ >>Dublin-pm mailing list - Dublin-pm@mail.pm.org >>http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm >> >> > > > From fergal at esatclear.ie Fri Apr 16 11:03:32 2004 From: fergal at esatclear.ie (Fergal Daly) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <4080019A.9090303@swiftcall.com> References: <407FE8D7.9080400@acm.org> <20040416142733.GA12808@dyn.fergaldaly.com> <4080019A.9090303@swiftcall.com> Message-ID: <20040416160332.GC13517@dyn.fergaldaly.com> If you malloc data in the C it will stay malloced until you free it, whether you go back to Perl or not. Getting data in and out of C within Perl is pretty much the same as getting it into C on it's own, except that you can use bits of Perl too. You could try something a bit fancy and figure out exactly how much memory you'll be using and dump all the data into that file, exactly like a big chunk of memory, making a note of the offsets of where each array begins and how long it is etc. Then you can just mmap the file so that it appears as a chunk of memory and bingo, your data structures are loaded. the mmap call will give you the address that the file has been mapped to so you can get the address of the arrays inside it by using the offsets, F On Fri, Apr 16, 2004 at 04:54:02PM +0100, Sean O'Riordain wrote: > Thanks Fergal. > > here is what is how i currently load from mysql... > > my $rec_count = 0; > while (my $aref = $sth->fetchrow_hashref) { > push @as, $aref->{timestamp_ue}; > push @ae, $aref->{finish_ue}; > push @adw, $aref->{dw}; > push @ahh, $aref->{hh}; > push @abz, $aref->{bzone}; > $rec_count++; > } > print " $rec_count cdr records loaded\n"; > > this takes maybe 5 minutes - so i'm not overly worried about that... > > if there isn't a simple way of passing the info to inline::C, then i was > thinking of just re-writing all the info to disk in an easily parseable > format, ie fixed width columns. Then i was just going to do all the > integer work in C and writing the results to an output file... > (currently it takes more than 8 hours at 100%cpu on a 1700mhz athlon ...) > > i could speed up the string stuff by using a lookup table since there > are only about 350 different values... > > in inline::C is it possible to persistently keep a C data-structure > between calls ? ie malloc space for my large int arrays, and then from > perl append each new line of info ? > > cheers, > Sean > > > Fergal Daly wrote: > >Not knowing exactly what you have makes it a bit tricky if you've got 5 > >million things looking like > > > > [$int1, $int2, $int3, $int4, $int5, $string] x 1.5 million > > > >then you will save quite bit by having > > > >@int1s = (int x 1.5 million) > >@int2s = (int x 1.5 million) > >.. > >@int5s = (int x 1.5 million) > >@strings = (string x 1.5 million) > > > >then just pass around the index. A package like > > > >package MyObj; > > > >sub new > >{ > > my $pkg = shift; > > my $index = shift; > > return bless \$index, $pkg; > >} > > > >sub getInt1 > >{ > > my $self = shift; > > return $int1s[$$self]; > >} > > > >etc... > > > >are you could get more mem efficient and rather than using arrays for the > >ints, have a string for each set of ints and have methods like > > > >sub getInt1 > >{ > > my $self = shift; > > > > # assume a 4 byte integer > > my $enc = substr($int1s, $$self*4, 4); > > > > return unpack("L", $enc); > >} > > > >you could also do this for the strings. It'll be slower because you'll be > >invoking methods, you could use subroutines if you're sure you'll never > >want > >inheritance etc. > > > >If you use Inline::C, how you load the data depends entirely on how you > >store > >it, you'll just have to write C routines for loading the data and call them > >from Perl, > > > >F > >On Fri, Apr 16, 2004 at 03:08:23PM +0100, Sean O'Riordain wrote: > > > >>Hi folks, > >> > >>I've an analysis program with a couple of million records that i really > >>need to keep in memory as i need to scan back and forth etc... With 5 > >>million odd records (written as a couple of independent 'arrays' or > >>should i say 'lists') the program requires quite a bit more than the > >>1.5Gb of ram and becomes very slow due to swapping - gentoo-linux... > >>Each record has 5 integers and a string of max.len 30 chars... but perl > >>takes up extra ram for each SV... I would like to be able to handle > >>larger datasets much faster than currently... > >> > >>Has anybody used INLINE::C for handling large data structures - if so > >>how do you load the info? > >> > >>Anybody used PDL? > >> > >>Any thoughts which way I should jump? > >> > >>cheers, > >>Sean > >>_______________________________________________ > >>Dublin-pm mailing list - Dublin-pm@mail.pm.org > >>http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm > >> > >> > > > > > > > _______________________________________________ > Dublin-pm mailing list - Dublin-pm@mail.pm.org > http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm > > From seanpor at acm.org Fri Apr 16 11:15:26 2004 From: seanpor at acm.org (Sean O'Riordain) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <20040416160332.GC13517@dyn.fergaldaly.com> References: <407FE8D7.9080400@acm.org> <20040416142733.GA12808@dyn.fergaldaly.com> <4080019A.9090303@swiftcall.com> <20040416160332.GC13517@dyn.fergaldaly.com> Message-ID: <4080069E.7060906@acm.org> Hi Fergal! have to think about that one! if i malloc memory in C then go back to perl to pass info into C again... how do i reference that memory again - i presume i pass the info back and forth by reference, - or could it be still available as a static type pointer... i'm missing a step here... thanks again! Sean Fergal Daly wrote: > If you malloc data in the C it will stay malloced until you free it, whether > you go back to Perl or not. > > Getting data in and out of C within Perl is pretty much the same as getting > it into C on it's own, except that you can use bits of Perl too. You could > try something a bit fancy and figure out exactly how much memory you'll be > using and dump all the data into that file, exactly like a big chunk of > memory, making a note of the offsets of where each array begins and how long > it is etc. Then you can just mmap the file so that it appears as a chunk of > memory and bingo, your data structures are loaded. the mmap call will give > you the address that the file has been mapped to so you can get the address > of the arrays inside it by using the offsets, > > F > > On Fri, Apr 16, 2004 at 04:54:02PM +0100, Sean O'Riordain wrote: > >>Thanks Fergal. >> >>here is what is how i currently load from mysql... >> >> my $rec_count = 0; >> while (my $aref = $sth->fetchrow_hashref) { >> push @as, $aref->{timestamp_ue}; >> push @ae, $aref->{finish_ue}; >> push @adw, $aref->{dw}; >> push @ahh, $aref->{hh}; >> push @abz, $aref->{bzone}; >> $rec_count++; >> } >> print " $rec_count cdr records loaded\n"; >> >>this takes maybe 5 minutes - so i'm not overly worried about that... >> >>if there isn't a simple way of passing the info to inline::C, then i was >>thinking of just re-writing all the info to disk in an easily parseable >>format, ie fixed width columns. Then i was just going to do all the >>integer work in C and writing the results to an output file... >>(currently it takes more than 8 hours at 100%cpu on a 1700mhz athlon ...) >> >>i could speed up the string stuff by using a lookup table since there >>are only about 350 different values... >> >>in inline::C is it possible to persistently keep a C data-structure >>between calls ? ie malloc space for my large int arrays, and then from >>perl append each new line of info ? >> >>cheers, >>Sean >> >> >>Fergal Daly wrote: >> >>>Not knowing exactly what you have makes it a bit tricky if you've got 5 >>>million things looking like >>> >>> [$int1, $int2, $int3, $int4, $int5, $string] x 1.5 million >>> >>>then you will save quite bit by having >>> >>>@int1s = (int x 1.5 million) >>>@int2s = (int x 1.5 million) >>>.. >>>@int5s = (int x 1.5 million) >>>@strings = (string x 1.5 million) >>> >>>then just pass around the index. A package like >>> >>>package MyObj; >>> >>>sub new >>>{ >>> my $pkg = shift; >>> my $index = shift; >>> return bless \$index, $pkg; >>>} >>> >>>sub getInt1 >>>{ >>> my $self = shift; >>> return $int1s[$$self]; >>>} >>> >>>etc... >>> >>>are you could get more mem efficient and rather than using arrays for the >>>ints, have a string for each set of ints and have methods like >>> >>>sub getInt1 >>>{ >>> my $self = shift; >>> >>> # assume a 4 byte integer >>> my $enc = substr($int1s, $$self*4, 4); >>> >>> return unpack("L", $enc); >>>} >>> >>>you could also do this for the strings. It'll be slower because you'll be >>>invoking methods, you could use subroutines if you're sure you'll never >>>want >>>inheritance etc. >>> >>>If you use Inline::C, how you load the data depends entirely on how you >>>store >>>it, you'll just have to write C routines for loading the data and call them >> >>>from Perl, >> >>>F >>>On Fri, Apr 16, 2004 at 03:08:23PM +0100, Sean O'Riordain wrote: >>> >>> >>>>Hi folks, >>>> >>>>I've an analysis program with a couple of million records that i really >>>>need to keep in memory as i need to scan back and forth etc... With 5 >>>>million odd records (written as a couple of independent 'arrays' or >>>>should i say 'lists') the program requires quite a bit more than the >>>>1.5Gb of ram and becomes very slow due to swapping - gentoo-linux... >>>>Each record has 5 integers and a string of max.len 30 chars... but perl >>>>takes up extra ram for each SV... I would like to be able to handle >>>>larger datasets much faster than currently... >>>> >>>>Has anybody used INLINE::C for handling large data structures - if so >>>>how do you load the info? >>>> >>>>Anybody used PDL? >>>> >>>>Any thoughts which way I should jump? >>>> >>>>cheers, >>>>Sean >>>>_______________________________________________ >>>>Dublin-pm mailing list - Dublin-pm@mail.pm.org >>>>http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm >>>> >>>> >>> >>> >>> >>_______________________________________________ >>Dublin-pm mailing list - Dublin-pm@mail.pm.org >>http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm >> >> From fergal at esatclear.ie Fri Apr 16 11:36:13 2004 From: fergal at esatclear.ie (Fergal Daly) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <4080069E.7060906@acm.org> References: <407FE8D7.9080400@acm.org> <20040416142733.GA12808@dyn.fergaldaly.com> <4080019A.9090303@swiftcall.com> <20040416160332.GC13517@dyn.fergaldaly.com> <4080069E.7060906@acm.org> Message-ID: <20040416163613.GD13517@dyn.fergaldaly.com> Yeah, that's it. Have a look at the Inline::C cookbook for full details but very briefly you create a Perl scalar in your C code and stuff the pointer into that and return that to Perl. If you don't want to worry about garbage collection then you should make sure that the scalar is a blessed object and that it's DESTROY method calls something to free the memory. There's geansai loads of examples in the docs although to be honest the one or two times I wanted to mix Perl and C I ended up using XS. Inline::C was storing lots of crap in hidden directories and some other stuff that I didn't like - can't remember what anymore so maybe my gripes no longer exist. Give it a whirl anyway, the examples are great, F On Fri, Apr 16, 2004 at 05:15:26PM +0100, Sean O'Riordain wrote: > Hi Fergal! > > have to think about that one! > > if i malloc memory in C then go back to perl to pass info into C > again... how do i reference that memory again - i presume i pass the > info back and forth by reference, - or could it be still available as a > static type pointer... i'm missing a step here... From Tim.Bunce at pobox.com Fri Apr 16 12:16:20 2004 From: Tim.Bunce at pobox.com (Tim Bunce) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <4080019A.9090303@swiftcall.com> References: <407FE8D7.9080400@acm.org> <20040416142733.GA12808@dyn.fergaldaly.com> <4080019A.9090303@swiftcall.com> Message-ID: <20040416171620.GA3711@dansat.data-plan.com> On Fri, Apr 16, 2004 at 04:54:02PM +0100, Sean O'Riordain wrote: > Thanks Fergal. > > here is what is how i currently load from mysql... > > my $rec_count = 0; > while (my $aref = $sth->fetchrow_hashref) { > push @as, $aref->{timestamp_ue}; > push @ae, $aref->{finish_ue}; > push @adw, $aref->{dw}; > push @ahh, $aref->{hh}; > push @abz, $aref->{bzone}; > $rec_count++; > } > print " $rec_count cdr records loaded\n"; > > this takes maybe 5 minutes - so i'm not overly worried about that... Pre-extending the arrays (using $#as = N) would eliminate the memory fragmentation you're getting from the reallocs as the arrays grow. (Would also be faster, though you can't be very interested in speed if you're using fetchrow_hashref :-) > i could speed up the string stuff by using a lookup table since there > are only about 350 different values... That's certainly worth doing (before getting into Inline::C). Something like this should suffice: push @abz, \($cache{ $aref->{bzone} } ||= $aref->{bzone}); But if the data volume is likely to grow then you'll need to try the pack/unpack approach, a tied DBM, or Inline::C. PDL might also be worth a look. Tim. From seanpor at acm.org Fri Apr 16 14:58:40 2004 From: seanpor at acm.org (Sean O'Riordain) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] big data structures relative to memory size In-Reply-To: <20040416163613.GD13517@dyn.fergaldaly.com> References: <407FE8D7.9080400@acm.org> <20040416142733.GA12808@dyn.fergaldaly.com> <4080019A.9090303@swiftcall.com> <20040416160332.GC13517@dyn.fergaldaly.com> <4080069E.7060906@acm.org> <20040416163613.GD13517@dyn.fergaldaly.com> Message-ID: <40803AF0.9010007@acm.org> ah ha! thank you! i went through the cookbook yesterday a few times but didn't get it... the missing piece was "you create a Perl scalar in your C code and stuff the pointer into that and return that to Perl." hmmm... obvious when you say it! seems like a day for it! i'm not sure i understand the blessed objects bit... initially i'll have a go at doing my own memory management and see how i go... I'll report back to dublin-pm with the results - it might be a few days... :-) cheers, Sean Fergal Daly wrote: >Yeah, that's it. Have a look at the Inline::C cookbook for full details but >very briefly you create a Perl scalar in your C code and stuff the pointer >into that and return that to Perl. If you don't want to worry about garbage >collection then you should make sure that the scalar is a blessed object and >that it's DESTROY method calls something to free the memory. There's geansai >loads of examples in the docs although to be honest the one or two times I >wanted to mix Perl and C I ended up using XS. Inline::C was storing lots of >crap in hidden directories and some other stuff that I didn't like - can't >remember what anymore so maybe my gripes no longer exist. Give it a whirl >anyway, the examples are great, > >F > >On Fri, Apr 16, 2004 at 05:15:26PM +0100, Sean O'Riordain wrote: > > >>Hi Fergal! >> >>have to think about that one! >> >>if i malloc memory in C then go back to perl to pass info into C >>again... how do i reference that memory again - i presume i pass the >>info back and forth by reference, - or could it be still available as a >>static type pointer... i'm missing a step here... >> >> > > > > From hlosb7mech at aol.com Sat Apr 17 18:49:05 2004 From: hlosb7mech at aol.com (ghdoLillian) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] extra income tyaq Message-ID: <200404172346.i3HNkE426999@mail.pm.org> An HTML attachment was scrubbed... URL: http://mail.pm.org/pipermail/dublin-pm/attachments/20040417/613decf5/attachment.htm From dglaude at hideakifan.com Sun Apr 18 14:59:25 2004 From: dglaude at hideakifan.com (Jonahs E. Bivouac) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] SAVE YOUR MONEY ! TAX FREE OFFSHORE SERVICES ! // Publicans, consequently was chief among sinners. Nay, he Message-ID: <101101c4257f$3fab71b3$008d2516@hideakifan.com> An HTML attachment was scrubbed... URL: http://mail.pm.org/pipermail/dublin-pm/attachments/20040418/7f4b9ebb/attachment.htm From doc at redbrick.dcu.ie Sun Apr 18 15:55:55 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] SAVE YOUR MONEY etc. In-Reply-To: <101101c4257f$3fab71b3$008d2516@hideakifan.com> References: <101101c4257f$3fab71b3$008d2516@hideakifan.com> Message-ID: <20040418205555.GB8467@carbon.redbrick.dcu.ie> Apologies for this. Non-member posts -were- being held, and then they weren't. I've tweaked the mailing list accordingly, and you shouldn't see any more of this kind of thing. Apologies again. - DoC Jonahs E. Bivouac said on Sun, Apr 18, 2004 at 11:59:25PM +0400: > _______________________________________________ > Dublin-pm mailing list - Dublin-pm@mail.pm.org > http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm > > From doc at redbrick.dcu.ie Sat Apr 24 01:35:36 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] 'Shortening' code. Message-ID: <20040424063536.GA16698@carbon.redbrick.dcu.ie> http://www.perl.com/pub/a/2004/03/12/ioall.html This is just a personal rant from me, but does anyone else feel that 'shortening' code, linewise, at the expense of execution time is something of a fallacy? The above mentioned module seems to do just that, and it makes my nerd blood boil. Anyone have any other particularly nasty examples of this? I know I've seen some terrible examples of readability in my time (most of them under NDA, so not reproducable here :)). I would consider myself a great opponent of the 'one-liner', that ends up looking like line noise, in favour of readability and the ability for your successor to take it up, read it, and understand it. - DoC From grant at mclean.net.nz Sat Apr 24 03:09:48 2004 From: grant at mclean.net.nz (Grant McLean) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] 'Shortening' code. In-Reply-To: <20040424063536.GA16698@carbon.redbrick.dcu.ie> References: <20040424063536.GA16698@carbon.redbrick.dcu.ie> Message-ID: <408A20CC.9070607@mclean.net.nz> Dave O Connor wrote: > http://www.perl.com/pub/a/2004/03/12/ioall.html > > This is just a personal rant from me, but does anyone else feel that > 'shortening' code, linewise, at the expense of execution time is something of > a fallacy? If I might play devil's apricot, isn't that exactly the purpose of higher level languages? After all, if execution time was all-important, we'd all be writing in assembler. One of the reasons I love Perl is that it allows me to achieve a lot without having to type a lot. > The above mentioned module seems to do just that, and it makes my > nerd blood boil. Thanks for the link. I saw the module appear on CPAN, had a quick scan through the docs and completely failed to see the point. Now that I've read the article, I think it's kind of cool. > I would consider myself a great opponent of the 'one-liner', that ends > up looking like line noise, in favour of readability and the ability > for your successor to take it up, read it, and understand it. Ah, now here I'm afraid I'm going to have to agree with you. It really can be a nightmare trying to maintain code that was written by someone who just can't help being 'clever'. One of my pet peeves is an obsession with this syntax: EXPRESSION1 && EXPRESSION2; as a shorthand for this: if(EXPRESSION1) { EXPRESSION2; } or this: EXPRESSION2 if(EXPRESSION1); In it's simple form, the first example is merely a slight obfuscation and I'm big enough to cope. But when EXPRESSION2 grows into multiple expressions, spanning multiple lines, carefully separated by commas then I start to lose my sense of humour. The final straw is when the author had to add something that wouldn't work with the comma separation scheme and a 'do' block is wrapped around the whole lot. At that point I find myself growling in frustration and muttering 'why god why?' (as my colleague at the adjacent desk can attest). Cheers Grant From seanpor at acm.org Sat Apr 24 03:23:45 2004 From: seanpor at acm.org (Sean O'Riordain) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] 'Shortening' code. In-Reply-To: <20040424063536.GA16698@carbon.redbrick.dcu.ie> References: <20040424063536.GA16698@carbon.redbrick.dcu.ie> Message-ID: <408A2411.7070001@acm.org> I suppose it depends... beauty is in the eye of the beholder... and the intended audience... In C the line while ((c=getc())!=EOF) { is a common idiom - but if you haven't seen it before it takes a bit of reading and once you're used to this idiom, then it becomes a single chunk of information in your head (or maybe just two)... there is a fine line between being overly obvious and overly clever... All languages have their own idioms, and perl is no exception, ie we often write while (<>) { which is not at all obvious to those coming in from another language, but completely obvious to us - isn't it? :-) Joseph Hall's "Effective Perl Programming" has a chapter on "Idiomatic Perl" The other day, Tim Bunce gave us the very neat push @abz, \($cache{ $aref->{bzone} } ||= $aref->{bzone}); which took me a while to figure out - and when putting it into my code, I'd definitely have to put in a comment before it explaining it's "intent" While I haven't read it in a while, Steve McConnell's "Code Complete" discusses these sorts of issues and others - for those who haven't seen it, it is >800page tome on coding - definitely a good read! cheers, Sean And yes - I do read too much and don't code enough :-) Dave O Connor wrote: >http://www.perl.com/pub/a/2004/03/12/ioall.html > >This is just a personal rant from me, but does anyone else feel that >'shortening' code, linewise, at the expense of execution time is something of >a fallacy? The above mentioned module seems to do just that, and it makes my >nerd blood boil. > >Anyone have any other particularly nasty examples of this? I know I've seen >some terrible examples of readability in my time (most of them under NDA, so >not reproducable here :)). I would consider myself a great opponent of the >'one-liner', that ends up looking like line noise, in favour of readability >and the ability for your successor to take it up, read it, and understand it. > > - DoC >_______________________________________________ >Dublin-pm mailing list - Dublin-pm@mail.pm.org >http://dublin.pm.org/ - IRC irc.linux.ie #dublin-pm > > > > > > From doc at redbrick.dcu.ie Sat Apr 24 03:56:20 2004 From: doc at redbrick.dcu.ie (Dave O Connor) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] 'Shortening' code. In-Reply-To: <408A20CC.9070607@mclean.net.nz> References: <20040424063536.GA16698@carbon.redbrick.dcu.ie> <408A20CC.9070607@mclean.net.nz> Message-ID: <20040424085620.GA14837@carbon.redbrick.dcu.ie> Grant McLean said on Sat, Apr 24, 2004 at 08:09:48PM +1200: > If I might play devil's apricot, isn't that exactly the purpose of > higher level languages? After all, if execution time was all-important, > we'd all be writing in assembler. One of the reasons I love Perl is > that it allows me to achieve a lot without having to type a lot. I guess I should have mentioned readability, too. While it can't immediately be said that a one-liner is slower (in fact, it's probbaly faster, unless you've used a module just to squash it down from several lines), it can usually be said that it's less readable. For example, the author of the module gives the following example for slurping a file: open my $file_handle, './Scotty' or die "Scotty can't be slurped:\n$!"; local $/; # Set input to "slurp" mode. my $big_string = <$file_handle>; close $file_handle; An equivalent (and more readable way is: open(FH,"Scotty") or die "Scotty can't be slurped:\n$!"; my $big_string = join('',); close(FH); While this is more 'idiomatic', it's still obvious to someone with a good grounding in perl what is going on here, and it's definitely not worth writing a module for to 'simplify' your code down the the almighty one-liner :) - DoC From fergal at esatclear.ie Sat Apr 24 06:42:43 2004 From: fergal at esatclear.ie (Fergal Daly) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] 'Shortening' code. In-Reply-To: <20040424085620.GA14837@carbon.redbrick.dcu.ie> References: <20040424063536.GA16698@carbon.redbrick.dcu.ie> <408A20CC.9070607@mclean.net.nz> <20040424085620.GA14837@carbon.redbrick.dcu.ie> Message-ID: <20040424114243.GA31569@dyn.fergaldaly.com> On Sat, Apr 24, 2004 at 09:56:20AM +0100, Dave O Connor wrote: > I guess I should have mentioned readability, too. While it can't immediately > be said that a one-liner is slower (in fact, it's probbaly faster, unless > you've used a module just to squash it down from several lines), it can > usually be said that it's less readable. For example, the author of the > module gives the following example for slurping a file: > > open my $file_handle, './Scotty' or die "Scotty can't be slurped:\n$!"; > local $/; # Set input to "slurp" mode. > my $big_string = <$file_handle>; > close $file_handle; > > An equivalent (and more readable way is: > > open(FH,"Scotty") or die "Scotty can't be slurped:\n$!"; > my $big_string = join('',); > close(FH); The author gave the first as an example of boilerplate code that's all over everyone's Perl scripts. Your version is slightly shorter but could be a whack load slower due to having to parse the file into lines and then join them all up again. Plus it stomps on the FH package variable which may or may not be OK. my $big_string = io('./Scotty')->slurp; is far more readable than either of those and would probably be faster than the 3-liner and a tiny bit slower than the 4-liner. Perl is full of boilerplate code and any module that gets rid of it without imposing severe penalties is ok by me, > While this is more 'idiomatic', it's still obvious to someone with a good > grounding in perl what is going on here, and it's definitely not worth writing > a module for to 'simplify' your code down the the almighty one-liner :) This is a 1-liner but it's a 1-liner due to abstraction, not due to compaction. A 1-liner (well slightly more) by compaction would be my $big_string = do{local($/,*F);open(F, "./Scotty')||die"Scotty can't be slurped\n$!";} and that definitely is a bad idea, F From dave at dave.org.uk Sat Apr 24 08:08:51 2004 From: dave at dave.org.uk (Dave Cross) Date: Mon Aug 2 21:29:57 2004 Subject: [Dub-pm] 'Shortening' code. In-Reply-To: <20040424114243.GA31569@dyn.fergaldaly.com> References: <20040424063536.GA16698@carbon.redbrick.dcu.ie> <408A20CC.9070607@mclean.net.nz> <20040424085620.GA14837@carbon.redbrick.dcu.ie> <20040424114243.GA31569@dyn.fergaldaly.com> Message-ID: <20040424130851.GB26256@mag-sol.com> On Sat, Apr 24, 2004 at 12:42:43PM +0100, Fergal Daly wrote: > On Sat, Apr 24, 2004 at 09:56:20AM +0100, Dave O Connor wrote: > > I guess I should have mentioned readability, too. While it can't > > immediately be said that a one-liner is slower (in fact, it's > > probbaly faster, unless you've used a module just to squash it down > > from several lines), it can usually be said that it's less readable. > > For example, the author of the module gives the following example > > for slurping a file: > > > > open my $file_handle, './Scotty' or die "Scotty can't be slurped:\n$!"; > > local $/; # Set input to "slurp" mode. > > my $big_string = <$file_handle>; > > close $file_handle; > > > > An equivalent (and more readable way is: > > > > open(FH,"Scotty") or die "Scotty can't be slurped:\n$!"; > > my $big_string = join('',); > > close(FH); > > The author gave the first as an example of boilerplate code that's all > over everyone's Perl scripts. Your version is slightly shorter but > could be a whack load slower due to having to parse the file into > lines and then join them all up again. Plus it stomps on the FH > package variable which may or may not be OK. > > my $big_string = io('./Scotty')->slurp; > > is far more readable than either of those and would probably be faster > than the 3-liner and a tiny bit slower than the 4-liner. > > Perl is full of boilerplate code and any module that gets rid of it > without imposing severe penalties is ok by me, > > > While this is more 'idiomatic', it's still obvious to someone with a > > good grounding in perl what is going on here, and it's definitely > > not worth writing a module for to 'simplify' your code down the the > > almighty one-liner :) > > This is a 1-liner but it's a 1-liner due to abstraction, not due to > compaction. A 1-liner (well slightly more) by compaction would be > > my $big_string = > do{local($/,*F);open(F, "./Scotty')||die"Scotty can't be slurped\n$!";} > > and that definitely is a bad idea, There's always File::Slurp http://search.cpan.org/dist/File-Slurp/ Dave... -- Oh is the way they say the future's meant to feel Or just twenty thousand people standing in a field?