From jharr at ist.unomaha.edu Mon Nov 3 12:05:11 2008 From: jharr at ist.unomaha.edu (James Harr) Date: Mon, 3 Nov 2008 14:05:11 -0600 Subject: [Omaha.pm] Some lines of a file In-Reply-To: <490B3188.1090501@jays.net> References: <4900A2BC.2050705@jays.net><452FE497-C7EA-4683-B1E0-DE30936CFCC2@jays.net> <6cb6eebc0810251750n680aa42br368137ed1b47c93d@mail.gmail.com> <490B3188.1090501@jays.net> Message-ID: > Laugh! Reverse Perl golf! Longer and harder to read! ;) Don't make me bust out Thread and Thread::Queue to turn this into a producer/consumer example. http://thedailywtf.com/Articles/Sprite_Threading.aspx -----Original Message----- From: omaha-pm-bounces+jharr=ist.unomaha.edu at pm.org [mailto:omaha-pm-bounces+jharr=ist.unomaha.edu at pm.org] On Behalf Of Jay Hannah Sent: Friday, October 31, 2008 11:26 To: Perl Mongers of Omaha, Nebraska USA Subject: Re: [Omaha.pm] Some lines of a file James Harr wrote: > ... continuing on the path to insanity through inverted loops ... > > <> foreach 0 .. 1; > push @buf, scalar(<>) foreach 0 .. 5; > sub {push @buf, $_; print shift @buf}->() while <>; > > I shouldn't be allowed to code perl. > Laugh! Reverse Perl golf! Longer and harder to read! ;) Thanks for @in = <>; I like that better than my 3 line version. :) j _______________________________________________ Omaha-pm mailing list Omaha-pm at pm.org http://mail.pm.org/mailman/listinfo/omaha-pm From hostetlerm at gmail.com Wed Nov 5 13:50:41 2008 From: hostetlerm at gmail.com (Mike Hostetler) Date: Wed, 5 Nov 2008 15:50:41 -0600 Subject: [Omaha.pm] Help with File::Find Message-ID: It's probably been years since I've done anything more than a 5 line Perl script, or just did minor fixes on someone else's code. But we have a log scraper that, uh, well, sucks and I'm the only one brave (or stupid) enough to change it. One of the things that I want to do is to grab the latest file under a directory with a pattern. And, since I want it to find the file recursively, I know I need to use File::Find. The fl sub logFinder { my $proc="pat" my @files=(); sub wanted { /^$proc.*/s && push(@files,$File::File::name); } find(\&wanted ,$logdir); My "wanted" will give me a list of files the first time I access logFinder, but the next time I run it, the values in @files are not refreshed. So that's what I got. I know someone here can do better . . . -- Mike Hostetler http://mike.hostetlerhome.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at petdance.com Wed Nov 5 16:06:22 2008 From: andy at petdance.com (Andy Lester) Date: Wed, 5 Nov 2008 18:06:22 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: Message-ID: On Nov 5, 2008, at 3:50 PM, Mike Hostetler wrote: > sub logFinder { > my $proc="pat" > my @files=(); > > sub wanted { > /^$proc.*/s > && push(@files,$File::File::name); > } > > find(\&wanted ,$logdir); First, there's no need to match /^$proc.*/, because it is the same as / ^$proc/. I think something is missing here. Is wanted() inside of logFinder? Why? Perl doesn't nest subs like that. Can you show us an entire program? xoa -- Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance From jay at jays.net Wed Nov 5 18:01:26 2008 From: jay at jays.net (Jay Hannah) Date: Wed, 5 Nov 2008 20:01:26 -0600 Subject: [Omaha.pm] Fwd: IMPORTANT - help us estimate the size of the Catalyst user base References: Message-ID: <32638480-3CEB-46D6-8929-A92346F9C28E@jays.net> Any Omahans using Catalyst other than Omni Hotels? j From: Kieren Diment Date: November 5, 2008 7:51:35 PM CST Hi All, Apologies for the cross-posting, I'm trying to representatives of as many organisations as possible who use Catalyst as possible to reply to this email. We need your assistance to help estimate the number of users of Catalyst. Matt Trout and I have written a proposal for a Catalyst book, and while we have a well known, credible publisher interested, they want the assurance that the market for the book is large enough to make it worth their while. If you could take the time to provide the following information (I've set reply-to to me for your confidentiality, and so that we don't pollute the list with this stuff) this would be really useful. I'll also let you know the results of this survey when I have them. I will treat this information in confidence, and will only use aggregated data so that you or your organisation will not be identifiable in the report I make on this data. There are about 1000 subscribers on the catalyst mailing list, and I'd hope for a response rate of about 10% (i.e. about 100 replies). The information I'm collecting is likely to result in an outcome that is useful to you in a commercial way, so your participation would be much appreciated. Please try to ensure that only one person from your organisation answers these questions. 1. What country are you in? 2. How many people are on your team? 3. How many of those people are writing code with Catalyst? 3a. If there are non Catalyst coders on your team, how many of the whole team would you like to be writing Catalyst code? 4. How many people using Catalyst on your team are subscribers to the Catalyst mailing list? 5. How many people writing Catalyst code on your team use the #catalyst irc channel on irc.perl.org? 6. These two questions are about the potential for the growth of Catalyst usage in your organisation. 6a. How many people do you think will be using Catalyst in your organisation in 12 months time? 6b. How many people do you think will be using Catalyst in your organisation in 2 years time? Thanks for your cooperation. If you know of any teams who are users of Catalyst but that do not subscribe to the Catalyst mailing list, or are on the IRC channel, please forward this email to them. Thanks. Kieren Diment From hostetlerm at gmail.com Thu Nov 6 05:50:20 2008 From: hostetlerm at gmail.com (Mike Hostetler) Date: Thu, 6 Nov 2008 07:50:20 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: Message-ID: I knew I should have added it. This is my complete subroutine: sub getLogFiles { my ($logdir,$proc) = @_; my @logfilelist =(); my @files=(); sub myLogs { /^$proc.*/s && push(@files,$File::File::name); } find(\&myLogs,$logdir); my $filestats={}; my $latesttime=0; my $myfile=""; foreach (@files) { $filestats=stat($_); if ($filestats->mtime>$latesttime) { $myfile=$_; $latesttime=$filestats->mtime; } } push(@logfilelist,$myfile); return @logfilelist; } My pain is that I want to get the newest file, and the only way I could think of is to add all the files that I want onto a list and compare them. File::Find requires a subroutine, so I made it a nested subroutine, thinking that it would be treated as local..Alas, I learned that Perl doesn' t work like that but still, in theory, this should. But it doesnt . . . On Wed, Nov 5, 2008 at 6:06 PM, Andy Lester wrote: > > On Nov 5, 2008, at 3:50 PM, Mike Hostetler wrote: > > sub logFinder { >> my $proc="pat" >> my @files=(); >> >> sub wanted { >> /^$proc.*/s >> && push(@files,$File::File::name); >> } >> >> find(\&wanted ,$logdir); >> > > > First, there's no need to match /^$proc.*/, because it is the same as > /^$proc/. > > I think something is missing here. Is wanted() inside of logFinder? Why? > Perl doesn't nest subs like that. > > Can you show us an entire program? > > xoa > > > -- > Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance > > > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- Mike Hostetler http://mike.hostetlerhome.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Thu Nov 6 11:31:38 2008 From: jay at jays.net (Jay Hannah) Date: Thu, 06 Nov 2008 13:31:38 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: Message-ID: <4913461A.7030604@jays.net> > My pain is that I want to get the newest file, and the only way I could > think of is to add all the files that I want onto a list and compare them. > Does this help? $ find ./ -name "pat*" | xargs ls -at | head -1 j From hostetlerm at gmail.com Thu Nov 6 12:11:35 2008 From: hostetlerm at gmail.com (Mike Hostetler) Date: Thu, 6 Nov 2008 14:11:35 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: <4913461A.7030604@jays.net> References: <4913461A.7030604@jays.net> Message-ID: On Thu, Nov 6, 2008 at 1:31 PM, Jay Hannah wrote: > > Does this help? > > $ find ./ -name "pat*" | xargs ls -at | head -1 > > It's ironic that you put that, since that's pretty much the exact command I'm replacing. :) The problem is that one some of these systems "ls" is aliases to always put filesize in front (why? I don't know -- I find it annoying). This is what I came up with after some googling and piecemailing. Anonymous subroutines are a good thing! sub getLogFiles { my ($logdir,$proc) = @_; my @logfilelist =(); my %files={}; find( sub { if (/^$proc/s) { my $filestats=stat($File::Find::name); $files{$File::Find::name}=$filestats->mtime; } },$logdir); my $myfile= (sort{ $files{$a}<=>$files{$b} } keys %files)[-1]; return $myfile; } -- Mike Hostetler http://mike.hostetlerhome.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Thu Nov 6 12:33:44 2008 From: jay at jays.net (Jay Hannah) Date: Thu, 06 Nov 2008 14:33:44 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: <4913461A.7030604@jays.net> Message-ID: <491354A8.9050902@jays.net> Mike Hostetler wrote: > It's ironic that you put that, since that's pretty much the exact command > I'm replacing. :) The problem is that one some of these systems "ls" is > aliases to always put filesize in front (why? I don't know -- I find it > annoying). > You can't specify /bin/ls to avoid aliases? > This is what I came up with after some googling and piecemailing. Anonymous > subroutines are a good thing! > Yay! So you're all set? :) j I wonder if Windows PowerShell knocks this problem out of the park? http://en.wikipedia.org/wiki/Windows_PowerShell From sharpestmarble at gmail.com Thu Nov 6 12:39:10 2008 From: sharpestmarble at gmail.com (Kevin) Date: Thu, 6 Nov 2008 14:39:10 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: <491354A8.9050902@jays.net> References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> Message-ID: If "ls" is aliased to put filesizes in front, then you shouldn't count on ls being in /bin. Instead, you should figure out where it is. `which ls | grep -v alias | tr -d '\t'` On Thu, Nov 6, 2008 at 2:33 PM, Jay Hannah wrote: > Mike Hostetler wrote: >> >> It's ironic that you put that, since that's pretty much the exact command >> I'm replacing. :) The problem is that one some of these systems "ls" is >> aliases to always put filesize in front (why? I don't know -- I find it >> annoying). >> > > You can't specify /bin/ls to avoid aliases? > >> This is what I came up with after some googling and piecemailing. >> Anonymous >> subroutines are a good thing! >> > > Yay! So you're all set? :) > > j > > > > I wonder if Windows PowerShell knocks this problem out of the park? > > http://en.wikipedia.org/wiki/Windows_PowerShell > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > From hostetlerm at gmail.com Thu Nov 6 14:06:20 2008 From: hostetlerm at gmail.com (Mike Hostetler) Date: Thu, 6 Nov 2008 16:06:20 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> Message-ID: I guess I only gave you one piece of the puzzle. "ls" also errors if the file doesn't exist and (with the way this script is written) it causes the script to die. Using globs or File::Find prevents that from happening. Once we find that file, then we search it for "interesting strings" and send emails out. Very basic Perl functionality, yet the original author didn't understand some basic concepts of Perl. Like how subroutines work, and the fact that not all variables need to be global. Sorta reminds me with the Perl script I ran into 15 years ago that did `grep $str $filename` to search a file. :) Yes, my little File::Find snippet works well! Now on to correct other sins committed in this script. On Thu, Nov 6, 2008 at 2:39 PM, Kevin wrote: > If "ls" is aliased to put filesizes in front, then you shouldn't count > on ls being in /bin. Instead, you should figure out where it is. > > `which ls | grep -v alias | tr -d '\t'` > > On Thu, Nov 6, 2008 at 2:33 PM, Jay Hannah wrote: > > Mike Hostetler wrote: > >> > >> It's ironic that you put that, since that's pretty much the exact > command > >> I'm replacing. :) The problem is that one some of these systems "ls" is > >> aliases to always put filesize in front (why? I don't know -- I find it > >> annoying). > >> > > > > You can't specify /bin/ls to avoid aliases? > > > >> This is what I came up with after some googling and piecemailing. > >> Anonymous > >> subroutines are a good thing! > >> > > > > Yay! So you're all set? :) > > > > j > > > > > > > > I wonder if Windows PowerShell knocks this problem out of the park? > > > > http://en.wikipedia.org/wiki/Windows_PowerShell > > > > _______________________________________________ > > Omaha-pm mailing list > > Omaha-pm at pm.org > > http://mail.pm.org/mailman/listinfo/omaha-pm > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- Mike Hostetler http://mike.hostetlerhome.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Fri Nov 7 06:34:45 2008 From: jay at jays.net (Jay Hannah) Date: Fri, 7 Nov 2008 08:34:45 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> Message-ID: On Nov 6, 2008, at 4:06 PM, Mike Hostetler wrote: > Very basic Perl functionality, yet the original author didn't > understand some basic concepts of Perl. Like how subroutines work, > and the fact that not all variables need to be global. Have you contacted the CPAN author? Looks like the most recent File::Find was an "unauthorized release" only 6 weeks ago. http://search.cpan.org/~tty/kurila-1.14_0/ So you could probably get it patched and/or take over authorship. Join the CPAN army! :) j From andy at petdance.com Fri Nov 7 08:10:55 2008 From: andy at petdance.com (Andy Lester) Date: Fri, 7 Nov 2008 10:10:55 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> Message-ID: <305B606B-C7A3-45FC-A874-29CF1A0D8520@petdance.com> On Nov 7, 2008, at 8:34 AM, Jay Hannah wrote: > On Nov 6, 2008, at 4:06 PM, Mike Hostetler wrote: >> Very basic Perl functionality, yet the original author didn't >> understand some basic concepts of Perl. Like how subroutines work, >> and the fact that not all variables need to be global. File::Find is one of the oldest modules, and shows its heritage. You may not like the interface, but it's too late to change it now. If you'd like a file finder with a more modern interface, may I suggest File::Next? > Have you contacted the CPAN author? Looks like the most recent > File::Find was an "unauthorized release" only 6 weeks ago. > > http://search.cpan.org/~tty/kurila-1.14_0/ kurila is sort of a fork of perl, and so any modules listed in there will be shown as "unauthorized" on search.cpan.org. -- Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance From jay at jays.net Fri Nov 7 09:54:12 2008 From: jay at jays.net (Jay Hannah) Date: Fri, 07 Nov 2008 11:54:12 -0600 Subject: [Omaha.pm] Possibly hiring?: genetics research programmer Message-ID: <491480C4.2050802@jays.net> There is not yet an official job posting, but... [Unofficial, unsubstantiated, flagrant rumor mode activated.] We could use some help, and some money might be freeing up to bring someone on full time. If you know any sharp Perl hackers that are interested in a full time job and learning medical / genetics research, please drop me a line. Scientific background not required. Perhaps even Perl might not be required, if they're really sharp in some other high-level, dynamic language, and willing to learn Perl. (Quickly. -grin-) Thanks, j Genetic Sequence Analysis Facility (GSAF) http://gsaf.ist.unomaha.edu/GSAF/index.php/Main_Page Collaborative Laboratory for Applied Bioinformatics (CLAB) http://clab.ist.unomaha.edu/CLAB/index.php/Main_Page Me: http://clab.ist.unomaha.edu/CLAB/index.php/User:Jhannah From hostetlerm at gmail.com Fri Nov 7 10:58:07 2008 From: hostetlerm at gmail.com (Mike Hostetler) Date: Fri, 7 Nov 2008 12:58:07 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: <305B606B-C7A3-45FC-A874-29CF1A0D8520@petdance.com> References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> <305B606B-C7A3-45FC-A874-29CF1A0D8520@petdance.com> Message-ID: On Fri, Nov 7, 2008 at 10:10 AM, Andy Lester wrote: > > On Nov 7, 2008, at 8:34 AM, Jay Hannah wrote: > > On Nov 6, 2008, at 4:06 PM, Mike Hostetler wrote: >> >>> Very basic Perl functionality, yet the original author didn't understand >>> some basic concepts of Perl. Like how subroutines work, and the fact that >>> not all variables need to be global. >>> >> > File::Find is one of the oldest modules, and shows its heritage. You may > not like the interface, but it's too late to change it now. > > If you'd like a file finder with a more modern interface, may I suggest > File::Next? > I'm sorry -- I didn't mean File::Find was bad-- I was referring to the script I have to refactor!! I've used File::Find before, actually, but I just need to get all the files so it was easy. I didn't really have to write a function that did any filtering on them. -- Mike Hostetler http://mike.hostetlerhome.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Fri Nov 7 11:52:48 2008 From: jay at jays.net (Jay Hannah) Date: Fri, 07 Nov 2008 13:52:48 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> <305B606B-C7A3-45FC-A874-29CF1A0D8520@petdance.com> Message-ID: <49149C90.3000601@jays.net> Mike Hostetler wrote: > I'm sorry -- I didn't mean File::Find was bad-- I was referring to the > script I have to refactor!! > > I've used File::Find before, actually, but I just need to get all the files > so it was easy. I didn't really have to write a function that did any > filtering on them. > Ahh. If the File::Find documentation could be better, that too would be a chance to stage a violent overthrow of the current CPAN author(s) (if any)... -laugh- (I'm just trying to incite some drama.) I agree with Andy that you can't (shouldn't) *change* an established module's interface, even if it looks clunky with 2008 eyeballs, but there's no harm in adding helper methods to make hardish things easier (internally using the same guts)... Since I don't think I've ever used File::Find, I'll shut up now. :) j From andy at petdance.com Fri Nov 7 12:02:47 2008 From: andy at petdance.com (Andy Lester) Date: Fri, 7 Nov 2008 14:02:47 -0600 Subject: [Omaha.pm] Help with File::Find In-Reply-To: <49149C90.3000601@jays.net> References: <4913461A.7030604@jays.net> <491354A8.9050902@jays.net> <305B606B-C7A3-45FC-A874-29CF1A0D8520@petdance.com> <49149C90.3000601@jays.net> Message-ID: <309C95C3-C6CA-45C4-A035-962004F09D8A@petdance.com> On Nov 7, 2008, at 1:52 PM, Jay Hannah wrote: > I agree with Andy that you can't (shouldn't) *change* an established > module's interface, even if it looks clunky with 2008 eyeballs, but > there's no harm in adding helper methods to make hardish things > easier (internally using the same guts)... Or just use File::Next. :-) -- Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance From netarttodd at gmail.com Mon Nov 10 15:56:50 2008 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Mon, 10 Nov 2008 17:56:50 -0600 Subject: [Omaha.pm] Entry Level Programmer Message-ID: <1fdb7d920811101556g2fd9942bj27287b6dec275ec0@mail.gmail.com> I am looking for an entry level programmer to assist me in some of my projects. Students are welcome to apply. This would be a great opportunity for someone to gain real world marketable experience. After a successful tour of duty the programmer would gain experience in the following areas. AJAX web programing using the Dojo toolkit Report writing using Crystal Reports XML processing using Perl and XSLT Some of the work can be done remotely and some on site ( 190th and West Center Omaha, NE ). If you know anyone who is interested have them contact me at NetArtTodd at gmail.com 402-660-2787. Todd -- Todd Christopher Hamilton (402) 660-2787 From jay at jays.net Tue Nov 11 16:05:54 2008 From: jay at jays.net (Jay Hannah) Date: Tue, 11 Nov 2008 18:05:54 -0600 Subject: [Omaha.pm] Reminder: Meeting Tonight!!!! References: Message-ID: <7FCBB4E1-3226-4F2E-87F8-16B9E7C63BF6@jays.net> Perl Monger powers activate! j Begin forwarded message: > From: "Matt Secoske" > Date: November 11, 2008 2:11:48 PM CST > To: odynug at googlegroups.com > Subject: [odynug] Reminder: Meeting Tonight!!!! > Reply-To: odynug at googlegroups.com > > Hey everyone, even though its cold and miserable outside, we are > still able to warm our hands near the toasty fire of knowledge that > is Omaha Dynamic Language Users Group! > > Usual Meeting spot: PKI, room 269, 7pm > Pizza will be provided by the awesome folks at TekSystems. > > We will be Ustreaming this, keep an eye on http://TechOmaha.com or > follow twitter.com/techomaha for more info. > > Hope to see you all there! > - Matt > > -- > Matt Secoske | biz: http://nimblelogic.com | tech: http:// > techomaha.com | personal: http://mattsecoske.com From jay at jays.net Sat Nov 15 00:00:39 2008 From: jay at jays.net (Jay Hannah) Date: Sat, 15 Nov 2008 02:00:39 -0600 Subject: [Omaha.pm] Perl, BioPerl In-Reply-To: References: <7A1AC499-897C-4FEC-B5EE-295AB6690BBA@jays.net> <9842C44A-5014-4BE0-9238-1ED03D4C86F0@jays.net> <491C6444.2020805@jays.net> <491CAB6B.7020906@jays.net> <491CAC59.8090405@jays.net> Message-ID: On Nov 14, 2008, at 5:52 PM, Mario Steele wrote: > What version of Perl are you guys using, so I know what to install. The latest is fine, or whatever you already have. Doesn't matter. :) Then bioperl-live, straight out of SVN http://www.bioperl.org/wiki/Using_Subversion Something like $ cd $ mkdir src $ cd src $ svn co svn://code.open-bio.org/bioperl/bioperl-live/trunk bioperl-live $ su # ln -s /usr/lib/perl5/site_perl/Bio ~/src/bioperl-live/Bio The bleeding edge is fun. :) j From dan at linder.org Fri Nov 21 09:17:23 2008 From: dan at linder.org (Dan Linder) Date: Fri, 21 Nov 2008 11:17:23 -0600 Subject: [Omaha.pm] Visual map of a Perl program structure? Message-ID: <3e2be50811210917y32f733d6gbed229eb9314b51@mail.gmail.com> I'm looking to be reminded of the perl subroutine mapper that I believe someone on this list pointed to (I'd guess within the last 24 months, but maybe longer). IIRC, you could feed it either a Perl file and it would start parsing it to show how the functions, external files, and included modules all interconnected to each other. (I've seen found similar for the Linux kernel, but can't find one for Perl.) I've inherited a big plate of Perl Spaghetti at work and I'd like to create a map to aid in answering some of the "What if we update this subroutine..." type questions. Thanks, Dan -- "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Fri Nov 21 10:46:06 2008 From: jay at jays.net (Jay Hannah) Date: Fri, 21 Nov 2008 12:46:06 -0600 Subject: [Omaha.pm] Visual map of a Perl program structure? In-Reply-To: <3e2be50811210917y32f733d6gbed229eb9314b51@mail.gmail.com> References: <3e2be50811210917y32f733d6gbed229eb9314b51@mail.gmail.com> Message-ID: <492701EE.9000000@jays.net> Dan Linder wrote: > I'm looking to be reminded of the perl subroutine mapper that I believe > someone on this list pointed to (I'd guess within the last 24 months, but > maybe longer). I did a blog post about that: http://headrattle.blogspot.com/2008/02/only-perl-can-parse-perl.html Here's some emails I've been squatting on since I read that article: http://jays.net/wiki/Perl_subroutine_graphing I never got anywhere with it... Let me know if you do!! HTH, j From jay at jays.net Fri Nov 21 13:34:49 2008 From: jay at jays.net (Jay Hannah) Date: Fri, 21 Nov 2008 15:34:49 -0600 Subject: [Omaha.pm] Benchmark.pm (cont'd from August) Message-ID: <49272979.6080707@jays.net> Benchmark.pm: http://search.cpan.org/~rgarcia/perl-5.10.0/lib/Benchmark.pm Back in August I complained. Yesterday I got interesting & useful info on irc.perl.org #poe: 12:44 <@Leolo> Benchmark: timing 1000000 iterations of amore, less, more... 12:44 <@Leolo> amore: 4 wallclock secs ( 2.57 usr + 0.00 sys = 2.57 CPU) @ 389105.06/s (n=1000000) 12:44 <@Leolo> less: 2 wallclock secs ( 2.52 usr + 0.00 sys = 2.52 CPU) @ 396825.40/s (n=1000000) 12:44 <@Leolo> more: 2 wallclock secs ( 2.58 usr + 0.00 sys = 2.58 CPU) @ 387596.90/s (n=1000000) 12:47 <@jhannah> Am I the only one that gave up on Benchmark? http://mail.pm.org/pipermail/omaha-pm/2008-August/001855.html 12:54 <@dngor> jhannah: Yes, wall clock is useless for benchmarking. That's why I use a CPU-time cutoff and compare the iterations/cpu time. 12:55 <@jhannah> wouldn't an accurate wallclock be insanely trivial to list? 12:55 <@Leolo> wallclock is useless 12:55 <@dngor> Wall time isn't accurate in a pre-emptive OS. 12:55 <@dngor> You're at the whim of other processes. 12:55 <@Leolo> other processes will hog the CPU, throwing off the .... what dngor said 12:55 <@dngor> Now, if you nice -19 the benchmark, that would make it better. :) 12:57 <@dngor> It may also require a reboot. Be sure your benchmark will eventually exit. :) 12:57 <@jhannah> I understand the point you're making. What I don't understand why Benchmark would label something "wallclock" when it has no bearing, whatsoever, on how much clock-on-the-wall-time elapsed. It should be removed or accurate...? 12:57 * jhannah adds an "is" in there somewhere 12:58 <@dngor> Oh, I see. Well, there's Benchmark.pm overhead which I presume it tries to discount. 13:00 <@dngor> I admit I'm just guessing at this point. The CPU/wall ratio implies that the program wasn't pre-empted much. 13:01 <@dngor> Program run time, from start to exit, also involves loading Perl and the source, compiling it, and tearing down the process. Those may be significant, but Benchmark.pm won't report them. 13:02 <@dngor> Whether these apply, I don't know. 13:03 * jhannah hugs purl, shoots Benchmark 13:03 <@jhannah> call it anything EXCEPT "wallclock" and I'm happy. :) 13:07 <@dngor> Example of a cpu-time limit: http://poe.dyndns.org/~troc/benchmarks/bench-calls.perl 13:08 <@dngor> Example of "rolling your own" for jhannah: http://poe.dyndns.org/~troc/benchmarks/arguments.perl 13:11 <@jhannah> dngor: so you just ignore the "wallclock secs"? Or you trust them as accurate relative to each other but unrelated to a clock on the wall? 13:11 <@dngor> I ignore them and look at n per X cpu seconds. 13:11 <@dngor> CPU time is the only objective measurement. 13:12 <@dngor> The CPU is the universe. 13:13 <@dngor> There is no coupling at all between the CPU and the wall clock. Consider that many machines dynamically clock now to conserve power. From jay at jays.net Fri Nov 21 14:59:51 2008 From: jay at jays.net (Jay Hannah) Date: Fri, 21 Nov 2008 16:59:51 -0600 Subject: [Omaha.pm] Fishing for GUI ideas... Message-ID: <49273D67.3040904@jays.net> So Mario and I wrote this tool in Perl that kicks out HTML: http://clab.ist.unomaha.edu/~jhannah/RT386/demo.html Click the pretty colors. I have this crazy notion of some sort of GUI where: - Users can seamlessly scroll left and right hundreds of thousands / millions of letters without having to stop and jump to another page. - I can add parallel rows of ~300 letters which can be clicked to show / hide them. But somehow the current horizontal expand/collapse will still work. Overlapping rows would magically bump out of each others way. - Users could add their own notes at certain places. Given that all this data is unpredictable, with tens of thousands of clickable things in the same dataset... How would YOU write that GUI? Stumped, j http://clab.ist.unomaha.edu/CLAB/index.php/User:Jhannah From dan at linder.org Sat Nov 22 08:22:36 2008 From: dan at linder.org (Dan Linder) Date: Sat, 22 Nov 2008 10:22:36 -0600 Subject: [Omaha.pm] Fishing for GUI ideas... In-Reply-To: <49273D67.3040904@jays.net> References: <49273D67.3040904@jays.net> Message-ID: <3e2be50811220822s7f578fd4j5e7094ee3274be3@mail.gmail.com> You might look into AJAX a bit. I would think that you could set it up to get more data for the left or right side when a certain point was reached. Much like Google Maps does -- except yours would only send strings of text, not image blocks. Dan On Fri, Nov 21, 2008 at 4:59 PM, Jay Hannah wrote: > So Mario and I wrote this tool in Perl that kicks out HTML: > > http://clab.ist.unomaha.edu/~jhannah/RT386/demo.html > > Click the pretty colors. > > I have this crazy notion of some sort of GUI where: > > - Users can seamlessly scroll left and right hundreds of thousands / > millions of letters without having to stop and jump to another page. > - I can add parallel rows of ~300 letters which can be clicked to show / > hide them. But somehow the current horizontal expand/collapse will still > work. Overlapping rows would magically bump out of each others way. > - Users could add their own notes at certain places. > > Given that all this data is unpredictable, with tens of thousands of > clickable things in the same dataset... > > How would YOU write that GUI? > > Stumped, > > j > http://clab.ist.unomaha.edu/CLAB/index.php/User:Jhannah > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Sun Nov 23 06:25:29 2008 From: jay at jays.net (Jay Hannah) Date: Sun, 23 Nov 2008 08:25:29 -0600 Subject: [Omaha.pm] Fishing for GUI ideas... In-Reply-To: <3e2be50811220822s7f578fd4j5e7094ee3274be3@mail.gmail.com> References: <49273D67.3040904@jays.net> <3e2be50811220822s7f578fd4j5e7094ee3274be3@mail.gmail.com> Message-ID: http://clab.ist.unomaha.edu/~jhannah/RT386/demo.html On Nov 22, 2008, at 10:22 AM, Dan Linder wrote: > You might look into AJAX a bit. I would think that you could set > it up to get more data for the left or right side when a certain > point was reached. Much like Google Maps does -- except yours > would only send strings of text, not image blocks. Ya. I've spent a lot of time staring at Google Maps style AJAX scrolls. In everything I've seen so far the idea works fine for pre-generated static images. This output is not static though -- the display expands and contracts when you click things. I've never seen an AJAX scroll that works on dynamic content. So you're right, if I re-invented the Google Maps wheel to use text (err... HTML?) instead of images then maybe somehow that could work. But then I think my current Javascript
expand/contract trick would be insufficient to cope with the complex scenarios that could unfold as you scroll and show/hide things... So I think I need something other than HTML
expand/contract tricks. So I need what... Java? Some sort of text grid thing that I have no idea how to get started writing? j From rob.townley at gmail.com Sun Nov 23 12:56:09 2008 From: rob.townley at gmail.com (Rob Townley) Date: Sun, 23 Nov 2008 14:56:09 -0600 Subject: [Omaha.pm] Fishing for GUI ideas... In-Reply-To: References: <49273D67.3040904@jays.net> <3e2be50811220822s7f578fd4j5e7094ee3274be3@mail.gmail.com> Message-ID: <7e84ed60811231256x76095a42xdbd86b833008cd8d@mail.gmail.com> On Sun, Nov 23, 2008 at 8:25 AM, Jay Hannah wrote: > http://clab.ist.unomaha.edu/~jhannah/RT386/demo.html > > On Nov 22, 2008, at 10:22 AM, Dan Linder wrote: >> >> You might look into AJAX a bit. I would think that you could set it up to >> get more data for the left or right side when a certain point was reached. >> Much like Google Maps does -- except yours would only send strings of text, >> not image blocks. > > Ya. I've spent a lot of time staring at Google Maps style AJAX scrolls. > > In everything I've seen so far the idea works fine for pre-generated static > images. This output is not static though -- the display expands and > contracts when you click things. I've never seen an AJAX scroll that works > on dynamic content. > > So you're right, if I re-invented the Google Maps wheel to use text (err... > HTML?) instead of images then maybe somehow that could work. > > But then I think my current Javascript
expand/contract trick would be > insufficient to cope with the complex scenarios that could unfold as you > scroll and show/hide things... > > So I think I need something other than HTML
expand/contract tricks. So > I need what... Java? Some sort of text grid thing that I have no idea how to > get started writing? > > j > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > Jay, does the system need to handle both circular non circular DNA. A ring could be zoomed in / out. From jharr at ist.unomaha.edu Mon Nov 24 06:51:17 2008 From: jharr at ist.unomaha.edu (James Harr) Date: Mon, 24 Nov 2008 08:51:17 -0600 Subject: [Omaha.pm] Fishing for GUI ideas... References: <49273D67.3040904@jays.net> Message-ID: http://code.google.com/webtoolkit/ GWT is a Java -> javascript compiler. It's what all of the google webapps are written in. If you write a little web-widget, the server-side code doesn't need to do any output formatting, just return JSON data. You could have the client widget request data when the user scrolls to that spot. At one point I thought they had a "scrollbar" widget, which wouldn't scroll anything, it'd just hand you the data. Apparently I'm mistaken. Though they do provide a lot of the facilities for writing a widget like that in a browser-independent manner. Check it out, might be worth playing with, might not. -----Original Message----- From: omaha-pm-bounces+jharr=ist.unomaha.edu at pm.org on behalf of Jay Hannah Sent: Fri 11/21/2008 4:59 PM To: odynug at googlegroups.com; Perl Mongers of Omaha, Nebraska USA Subject: [Omaha.pm] Fishing for GUI ideas... So Mario and I wrote this tool in Perl that kicks out HTML: http://clab.ist.unomaha.edu/~jhannah/RT386/demo.html Click the pretty colors. I have this crazy notion of some sort of GUI where: - Users can seamlessly scroll left and right hundreds of thousands / millions of letters without having to stop and jump to another page. - I can add parallel rows of ~300 letters which can be clicked to show / hide them. But somehow the current horizontal expand/collapse will still work. Overlapping rows would magically bump out of each others way. - Users could add their own notes at certain places. Given that all this data is unpredictable, with tens of thousands of clickable things in the same dataset... How would YOU write that GUI? Stumped, j http://clab.ist.unomaha.edu/CLAB/index.php/User:Jhannah _______________________________________________ Omaha-pm mailing list Omaha-pm at pm.org http://mail.pm.org/mailman/listinfo/omaha-pm -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Mon Nov 24 13:20:38 2008 From: jay at jays.net (jay at jays.net) Date: Mon, 24 Nov 2008 16:20:38 -0500 Subject: [Omaha.pm] Iterators Message-ID: Survey: Am I the only one who's coded Perl for 15 years now and never knew about iterators? http://www.perl.com/pub/a/2005/06/16/iterators.html while ( my $item = $next->() ) { # what?? Laugh, j From jay at jays.net Mon Nov 24 13:27:09 2008 From: jay at jays.net (jay at jays.net) Date: Mon, 24 Nov 2008 16:27:09 -0500 Subject: [Omaha.pm] All possible DNA sequences that are 5-7 bases long Message-ID: <78178b4c4144246acf4f390ba0f9ed5d@jays.net> Enjoy, :) j #!/usr/bin/perl # http://www.perl.com/pub/a/2005/06/16/iterators.html # {slightly modified by jhannah 20081124) my @DNA = qw/A C T G/; my $seq = gen_permutate(5, 7, @DNA); while ( my $strand = $seq->() ) { print "$strand\n"; } sub gen_permutate { my ($min, $max, @list) = @_; my @curr = (0) x $min; return sub { if ( (join '', map { $list[ $_ ] } @curr) eq $list[ -1 ] x @curr ) { @curr = (0) x (@curr + 1); } else { my $pos = @curr; while ( --$pos > -1 ) { ++$curr[ $pos ], last if $curr[ $pos ] < $#list; $curr[ $pos ] = 0; } } return undef if @curr > $max; return join '', map { $list[ $_ ] } @curr; }; } From andy at petdance.com Mon Nov 24 14:06:34 2008 From: andy at petdance.com (Andy Lester) Date: Mon, 24 Nov 2008 16:06:34 -0600 Subject: [Omaha.pm] Iterators In-Reply-To: References: Message-ID: <0C2A1790-00CF-469A-A2D1-A331D86D59DD@petdance.com> On Nov 24, 2008, at 3:20 PM, wrote: > Survey: Am I the only one who's coded Perl for 15 years now and > never knew > about iterators? > > http://www.perl.com/pub/a/2005/06/16/iterators.html File::Next makes great use of iterators, if I do say so myself. use File::Next; my $iter = File::Next::files( '/tmp' ); while ( defined ( my $file = $iter?>() ) ) { print $file, "\n"; } You don't care what goes on behind the scenes. Also means you can have multiple iterators: my $iter_tmp = File::Next::files( '/tmp' ); my $iter_usr = File::Next::files( '/usr' ); and they can work by themselves without disturbing the other one. xoxo, Andy -- Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance From rob.townley at gmail.com Mon Nov 24 17:04:02 2008 From: rob.townley at gmail.com (Rob Townley) Date: Mon, 24 Nov 2008 19:04:02 -0600 Subject: [Omaha.pm] Iterators In-Reply-To: References: Message-ID: <7e84ed60811241704o3cddd2e9k1406edacd5897b69@mail.gmail.com> On Mon, Nov 24, 2008 at 3:20 PM, wrote: > Survey: Am I the only one who's coded Perl for 15 years now and never knew > about iterators? > > http://www.perl.com/pub/a/2005/06/16/iterators.html > > while ( my $item = $next->() ) { # what?? > > Laugh, > > j > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > Jay, you knew about em in one part of your brain. From jay at jays.net Tue Nov 25 05:38:03 2008 From: jay at jays.net (Jay Hannah) Date: Tue, 25 Nov 2008 07:38:03 -0600 Subject: [Omaha.pm] All possible DNA sequences that are 5-7 bases long In-Reply-To: References: <78178b4c4144246acf4f390ba0f9ed5d@jays.net> Message-ID: <52DAA3E2-406D-4553-82A7-AC5BC4B8A9EF@jays.net> On Nov 24, 2008, at 4:14 PM, Ramez T. Mina wrote: > Thanks a million, it would save me a lot of time, I tried it, and > it is working, still don't understand it, but I will figure it out. > Just a minor, it gives 4^l-1 posibilities, and the missing one is > usually AAAA (in case of 4), or AAA(in case of 3) and so on. Ack! Yup, you got me. The AAAA's are missing because this line my @curr = (0) x $min; Sets the initial value at the beginning, and it needs to set it at the beginning minus one. :) So change it to this: my @curr = ($#list) x ($min - 1); That seems to work. Cheers, j From jay at jays.net Tue Nov 25 05:48:44 2008 From: jay at jays.net (Jay Hannah) Date: Tue, 25 Nov 2008 07:48:44 -0600 Subject: [Omaha.pm] All possible DNA sequences that are 5-7 bases long In-Reply-To: <52DAA3E2-406D-4553-82A7-AC5BC4B8A9EF@jays.net> References: <78178b4c4144246acf4f390ba0f9ed5d@jays.net> <52DAA3E2-406D-4553-82A7-AC5BC4B8A9EF@jays.net> Message-ID: <636BC893-BB7E-46F5-81BA-21503AAA919E@jays.net> Perl, a journey of learning that lasts a lifetime... :) j Quiz: Are these two the same thing? (0) x 7 0 x 7 If not, how are they different? $ perl -d -e 1 DB<4> @j = (0) x 7 DB<5> x @j 0 0 1 0 2 0 3 0 4 0 5 0 6 0 DB<6> @j = 0 x 7 DB<7> x @j 0 0000000 Answer: Nope. The first gives you an array of 7 elements, and each one is a zero. The second gives you an array of a single scalar which is seven zeros. From netarttodd at gmail.com Fri Nov 28 04:39:11 2008 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Fri, 28 Nov 2008 06:39:11 -0600 Subject: [Omaha.pm] Fishing for GUI ideas... In-Reply-To: References: <49273D67.3040904@jays.net> Message-ID: <1fdb7d920811280439r66ed9607vbab61d9de9c28598@mail.gmail.com> For a year I have been using dojo. It is a javascript ajax development framework. You might be able to accomplish what you want using this tool. I am always amazed at what people are doing with that tool. It is open source and there are a lot of institutional contributes. Let me know what you think. On Mon, Nov 24, 2008 at 8:51 AM, James Harr wrote: > http://code.google.com/webtoolkit/ > > GWT is a Java -> javascript compiler. It's what all of the google webapps > are written in. If you write a little web-widget, the server-side code > doesn't need to do any output formatting, just return JSON data. You could > have the client widget request data when the user scrolls to that spot. > > At one point I thought they had a "scrollbar" widget, which wouldn't scroll > anything, it'd just hand you the data. Apparently I'm mistaken. Though they > do provide a lot of the facilities for writing a widget like that in a > browser-independent manner. > > Check it out, might be worth playing with, might not. > > > -----Original Message----- > From: omaha-pm-bounces+jharr=ist.unomaha.edu at pm.org on behalf of Jay Hannah > Sent: Fri 11/21/2008 4:59 PM > To: odynug at googlegroups.com; Perl Mongers of Omaha, Nebraska USA > Subject: [Omaha.pm] Fishing for GUI ideas... > > So Mario and I wrote this tool in Perl that kicks out HTML: > > http://clab.ist.unomaha.edu/~jhannah/RT386/demo.html > > Click the pretty colors. > > I have this crazy notion of some sort of GUI where: > > - Users can seamlessly scroll left and right hundreds of thousands / > millions of letters without having to stop and jump to another page. > - I can add parallel rows of ~300 letters which can be clicked to show / > hide them. But somehow the current horizontal expand/collapse will still > work. Overlapping rows would magically bump out of each others way. > - Users could add their own notes at certain places. > > Given that all this data is unpredictable, with tens of thousands of > clickable things in the same dataset... > > How would YOU write that GUI? > > Stumped, > > j > http://clab.ist.unomaha.edu/CLAB/index.php/User:Jhannah > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- Todd Christopher Hamilton (402) 660-2787 From dan at linder.org Sat Nov 29 22:31:57 2008 From: dan at linder.org (Dan Linder) Date: Sun, 30 Nov 2008 00:31:57 -0600 Subject: [Omaha.pm] Suggested XML modules... Message-ID: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> >From searching our e-mail archive, it seems like the two mentioned XML modules are "XML::Simple" and "XML::Twig". Are there any other XML handling modules that people are using (or have used)? Any thoughts into the pros/cons about the module you're using (or the ones you passed up)? Dan -- "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy at petdance.com Sat Nov 29 23:35:22 2008 From: andy at petdance.com (Andy Lester) Date: Sun, 30 Nov 2008 01:35:22 -0600 Subject: [Omaha.pm] Suggested XML modules... In-Reply-To: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> References: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> Message-ID: On Nov 30, 2008, at 12:31 AM, Dan Linder wrote: > Are there any other XML handling modules that people are using (or > have used)? > > Any thoughts into the pros/cons about the module you're using (or > the ones you passed up)? The Perl 5 Wiki has a list of dos and do nots. http://www.perlfoundation.org/perl5/index.cgi?recommended_xml_modules See also http://perl-xml.sourceforge.net/faq/ xoxo, Andy -- Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance From jay at jays.net Sun Nov 30 05:46:32 2008 From: jay at jays.net (Jay Hannah) Date: Sun, 30 Nov 2008 07:46:32 -0600 Subject: [Omaha.pm] Suggested XML modules... In-Reply-To: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> References: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> Message-ID: On Nov 30, 2008, at 12:31 AM, Dan Linder wrote: > From searching our e-mail archive, it seems like the two mentioned > XML modules are "XML::Simple" and "XML::Twig". > > Are there any other XML handling modules that people are using (or > have used)? > > Any thoughts into the pros/cons about the module you're using (or > the ones you passed up)? At $work we use XML::Twig exclusively. Works great for us. :) j From dan at linder.org Sun Nov 30 13:19:22 2008 From: dan at linder.org (Dan Linder) Date: Sun, 30 Nov 2008 15:19:22 -0600 Subject: [Omaha.pm] Suggested XML modules... In-Reply-To: References: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> Message-ID: <3e2be50811301319s12ded1b5u332ec95f21536c0a@mail.gmail.com> Excellent, thanks for those links. (The project I'm on has currently been using XML::Trivial, but it's documentation and examples are cryptic and unenlightening...or maybe that was the post-Thanksgiving brain fog.) Dan On Sun, Nov 30, 2008 at 1:35 AM, Andy Lester wrote: > > On Nov 30, 2008, at 12:31 AM, Dan Linder wrote: > > Are there any other XML handling modules that people are using (or have >> used)? >> >> Any thoughts into the pros/cons about the module you're using (or the ones >> you passed up)? >> > > > The Perl 5 Wiki has a list of dos and do nots. > > http://www.perlfoundation.org/perl5/index.cgi?recommended_xml_modules > > See also http://perl-xml.sourceforge.net/faq/ > > xoxo, > Andy > > > -- > Andy Lester => andy at petdance.com => www.petdance.com => AIM:petdance > > > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan at linder.org Sun Nov 30 13:26:15 2008 From: dan at linder.org (Dan Linder) Date: Sun, 30 Nov 2008 15:26:15 -0600 Subject: [Omaha.pm] Suggested XML modules... In-Reply-To: References: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> Message-ID: <3e2be50811301326y5507fcc0t1467d1b406fbcd84@mail.gmail.com> I was looking at it a bit because our XML files have the potential to get quite large (>50GB dumps). On the other hand, the day-to-day files should stay quite manageable (between 100K to 10M), so XML::Twig's ability to process only a portion of an XML file might be overkill. Dan (Those numbers are all very rough estimates and we're going through some serious data cleansing to bring the large file down to the 50-500MB range...) On Sun, Nov 30, 2008 at 7:46 AM, Jay Hannah wrote: > On Nov 30, 2008, at 12:31 AM, Dan Linder wrote: > >> From searching our e-mail archive, it seems like the two mentioned XML >> modules are "XML::Simple" and "XML::Twig". >> >> Are there any other XML handling modules that people are using (or have >> used)? >> >> Any thoughts into the pros/cons about the module you're using (or the ones >> you passed up)? >> > > At $work we use XML::Twig exclusively. Works great for us. :) > > > j > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From topher-pm at zyp.org Sun Nov 30 14:27:56 2008 From: topher-pm at zyp.org (Christopher Cashell) Date: Sun, 30 Nov 2008 16:27:56 -0600 Subject: [Omaha.pm] Suggested XML modules... In-Reply-To: <3e2be50811301326y5507fcc0t1467d1b406fbcd84@mail.gmail.com> References: <3e2be50811292231v6a25992i4264d0e26b2aea22@mail.gmail.com> <3e2be50811301326y5507fcc0t1467d1b406fbcd84@mail.gmail.com> Message-ID: On Sun, Nov 30, 2008 at 3:26 PM, Dan Linder wrote: > I was looking at it a bit because our XML files have the potential to get > quite large (>50GB dumps). On the other hand, the day-to-day files should > stay quite manageable (between 100K to 10M), so XML::Twig's ability to > process only a portion of an XML file might be overkill. Just a quick note of warning, it can be very surprising how much RAM is required for processing XML documents as they get large. Loading the entire document into memory has a way of balooning really fast. We ran into some issues with that on a project at my previous employer. As noted in the Perl XML FAQ: "The memory requirements of a tree based parser can be surprisingly high. Because each node in the tree needs to keep track of links to ancestor, sibling and child nodes, the memory required to build a tree can easily reach 10-30 times the size of the source document. You probably don't need to worry about that though unless your documents are multi-megabytes (or you're running on lower spec hardware)." We had a couple of XML files that were under 10MB and they were causing memory usage of nearly 500MB in the initial version of the processing application. > Dan -- Christopher