From jay at jays.net Sat Oct 3 00:06:23 2009 From: jay at jays.net (Jay Hannah) Date: Sat, 3 Oct 2009 02:06:23 -0500 Subject: [Omaha.pm] Next mtg: Oct. 13 7pm Message-ID: <7A6F65D2-D48D-4249-A7C1-D46CE0A724F2@jays.net> http://jays.net/wiki/ODLUG Grails double header! Grails is a Rails inspired web framework written in Groovy. * Juan presents Grails on Google App Engine * Scott Hickey presents Grails See you there! j From spandey at mail.unomaha.edu Wed Oct 7 17:56:32 2009 From: spandey at mail.unomaha.edu (Sanjit Pandey) Date: Wed, 7 Oct 2009 19:56:32 -0500 Subject: [Omaha.pm] Glimmer output parser. In-Reply-To: <7A6F65D2-D48D-4249-A7C1-D46CE0A724F2@jays.net> References: <7A6F65D2-D48D-4249-A7C1-D46CE0A724F2@jays.net> Message-ID: An HTML attachment was scrubbed... URL: From jay at jays.net Wed Oct 7 21:25:25 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 7 Oct 2009 23:25:25 -0500 Subject: [Omaha.pm] Glimmer output parser. In-Reply-To: References: <7A6F65D2-D48D-4249-A7C1-D46CE0A724F2@jays.net> Message-ID: <7F90D2AA-6F54-4624-A66F-4B7305FB41D6@jays.net> On Oct 7, 2009, at 7:56 PM, Sanjit Pandey wrote: > I was wondering if anybody has parsed the output of glimmer3. I am > looking for a existing code so that I don't have to write from > scratch. Some docs: http://search.cpan.org/~cjfields/BioPerl-1.6.1/Bio/Tools/Glimmer.pm and some sample code: http://code.open-bio.org/svnweb/index.cgi/bioperl/view/bioperl-live/ trunk/t/Tools/Genpred.t (The second half of the tests are for Glimmer, including Glimmer3.) The bioperl-l mailing list is where all the world's BioPerl experts hang out: http://www.bioperl.org/wiki/Mailing_lists I hope that helps, j From jay at jays.net Thu Oct 8 07:38:57 2009 From: jay at jays.net (Jay Hannah) Date: Thu, 8 Oct 2009 09:38:57 -0500 Subject: [Omaha.pm] Term::ProgressBar is neato Message-ID: <09073539-D844-4437-B406-CCD23B02B378@jays.net> http://headrattle.blogspot.com/search/label/perl :) Here's a super easy CPAN module I started using last week. Ever have a long running process that a human is watching, and you want to add a progress bar? Here you go! use Term::ProgressBar; my $progress = Term::ProgressBar->new({ name => 'MySlowThing()', count => 100, ETA => 'linear', }); $progress->max_update_rate(1); for (1..100) { sleep 1; $progress->update($_); } Gives you a nifty ASCII timer bar with major and minor movements, updating clock, the whole nine. :) MySlowThing(): 26% [==== * ]1m16s Left j From dan at linder.org Thu Oct 8 08:52:15 2009 From: dan at linder.org (Dan Linder) Date: Thu, 8 Oct 2009 10:52:15 -0500 Subject: [Omaha.pm] Term::ProgressBar is neato In-Reply-To: <09073539-D844-4437-B406-CCD23B02B378@jays.net> References: <09073539-D844-4437-B406-CCD23B02B378@jays.net> Message-ID: <3e2be50910080852l496d440bva8d8eacb114d4cf8@mail.gmail.com> Very cool. Happen to know of one that works within a CGI? DanL On Thu, Oct 8, 2009 at 09:38, Jay Hannah wrote: > http://headrattle.blogspot.com/search/label/perl ? :) > > Here's a super easy CPAN module I started using last week. Ever have a long > running process that a human is watching, and you want to add a progress > bar? Here you go! > > use Term::ProgressBar; > my $progress = Term::ProgressBar->new({ > ?name => 'MySlowThing()', > ?count => 100, > ?ETA => 'linear', > }); > $progress->max_update_rate(1); > > for (1..100) { > ?sleep 1; > ?$progress->update($_); > } > > Gives you a nifty ASCII timer bar with major and minor movements, updating > clock, the whole nine. :) > > MySlowThing(): 26% [==== ? ? ? ?* ? ? ? ?]1m16s Left > > j > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From jay at jays.net Thu Oct 8 09:16:11 2009 From: jay at jays.net (Jay Hannah) Date: Thu, 8 Oct 2009 11:16:11 -0500 Subject: [Omaha.pm] Term::ProgressBar is neato In-Reply-To: <3e2be50910080852l496d440bva8d8eacb114d4cf8@mail.gmail.com> References: <09073539-D844-4437-B406-CCD23B02B378@jays.net> <3e2be50910080852l496d440bva8d8eacb114d4cf8@mail.gmail.com> Message-ID: On Oct 8, 2009, at 10:52 AM, Dan Linder wrote: > Very cool. Happen to know of one that works within a CGI? I'd probably use this one. jQuery and jQuery UI are also neato. :) http://jqueryui.com/demos/progressbar/#animated j From dan at linder.org Thu Oct 8 09:43:25 2009 From: dan at linder.org (Dan Linder) Date: Thu, 8 Oct 2009 11:43:25 -0500 Subject: [Omaha.pm] Term::ProgressBar is neato In-Reply-To: References: <09073539-D844-4437-B406-CCD23B02B378@jays.net> <3e2be50910080852l496d440bva8d8eacb114d4cf8@mail.gmail.com> Message-ID: <3e2be50910080943w684e464bhe01a32b6de1b21ee@mail.gmail.com> Looks like this one: CGI-ProgressBar http://search.cpan.org/~lgoddard/CGI-ProgressBar-0.04/ProgressBar.pm Might work for me...when I get some spare time. :-) DanL On Thu, Oct 8, 2009 at 11:16, Jay Hannah wrote: > On Oct 8, 2009, at 10:52 AM, Dan Linder wrote: >> >> Very cool. ?Happen to know of one that works within a CGI? > > > I'd probably use this one. jQuery and jQuery UI are also neato. ?:) > > http://jqueryui.com/demos/progressbar/#animated > > j > > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From jay at jays.net Thu Oct 8 14:26:50 2009 From: jay at jays.net (Jay Hannah) Date: Thu, 8 Oct 2009 16:26:50 -0500 Subject: [Omaha.pm] Wow. \d is a lot :) Message-ID: Who knew there were so many digits? :) 16:19 < chansen> gshank: be explicit, use [0-9] or use \d if you mean any unicode digit 16:20 < chansen> gshank: what \d means if SvURT8 is true: http://unicode.org/cldr/utility/list-unicodeset.jsp?a=%5B%3Adigit%3A%5D j From jay at jays.net Tue Oct 13 07:42:50 2009 From: jay at jays.net (Jay Hannah) Date: Tue, 13 Oct 2009 09:42:50 -0500 Subject: [Omaha.pm] Meeting tonight Message-ID: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> http://jays.net/wiki/Odlug Scott's been called out of town on a last minute business trip, so won't be presenting tonight. I don't think I've heard from Juan since last month's meeting. Hopefully he's still planning on presenting Grails on Google App Engine. I'll be zero-prep presenting some modern Perl goodness I was working on last week: Using DBIx::Class, Moose::Role, Moose array traits, and the factory design pattern to get Apples and Oranges cooperating like happy little CitrusFruit. :) Apparently these 3-foot Blimpie party subs feed 15, so I hope you come hungry. :) See you there! j From jay at jays.net Tue Oct 13 09:46:16 2009 From: jay at jays.net (Jay Hannah) Date: Tue, 13 Oct 2009 11:46:16 -0500 Subject: [Omaha.pm] Meeting tonight In-Reply-To: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> Message-ID: <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> On Oct 13, 2009, at 9:42 AM, Jay Hannah wrote: > I'll be zero-prep presenting some modern Perl goodness I was working > on last week: > > Using DBIx::Class, Moose::Role, Moose array traits, and the > factory design pattern to get Apples and Oranges cooperating like > happy little CitrusFruit. :) Oh, I also got my new Module::Install::DiffCheck almost production- ready last week, so I could present that: http://gist.github.com/206360 http://github.com/jhannah/module-install-diffcheck Feedback welcome. :) j From dan at linder.org Tue Oct 13 11:07:08 2009 From: dan at linder.org (Dan Linder) Date: Tue, 13 Oct 2009 13:07:08 -0500 Subject: [Omaha.pm] Perl and (horrors) calling an Assembly routine. Message-ID: <3e2be50910131107v49135a52nae60198b359a19c0@mail.gmail.com> Considering how OS/platform agnostic Perl is, this might sound like a really odd question... Does anyone know if there is a way to include assembly routines or call out to an external binary blob? I am trying to write a CPU detection routine and on the Intel/AMD platform there is the "cpuid" operation that will return all the gory details about the hardware (cache size, number of physical processors, number of cores per processor, CPU family, speed, etc). I have some C code that I gleaned from Intel and AMDs websites, but it would be great if I could put this in a Perl script and then run it on all of my x86 systems (Linux, Solaris x86, etc), rather than having to re-compile it each time I need to update a section of the logic (outside the cpuid routines). DanL -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From juanvazquez at gmail.com Tue Oct 13 12:04:04 2009 From: juanvazquez at gmail.com (Juan) Date: Tue, 13 Oct 2009 14:04:04 -0500 Subject: [Omaha.pm] [odynug] Re: Meeting tonight In-Reply-To: <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> Message-ID: I'm still on track for presenting. Sent from my iPhone On Oct 13, 2009, at 11:46 AM, Jay Hannah wrote: > > On Oct 13, 2009, at 9:42 AM, Jay Hannah wrote: >> I'll be zero-prep presenting some modern Perl goodness I was working >> on last week: >> >> Using DBIx::Class, Moose::Role, Moose array traits, and the >> factory design pattern to get Apples and Oranges cooperating like >> happy little CitrusFruit. :) > > Oh, I also got my new Module::Install::DiffCheck almost production- > ready last week, so I could present that: > > http://gist.github.com/206360 > http://github.com/jhannah/module-install-diffcheck > > Feedback welcome. :) > > j > > > --~--~---------~--~----~------------~-------~--~----~ > You received this message because you are subscribed to the Google > Groups "Omaha Dynamic Language User Group" group. > To post to this group, send email to odynug at googlegroups.com > To unsubscribe from this group, send email to odynug+unsubscribe at googlegroups.com > For more options, visit this group at http://groups.google.com/group/odynug?hl=en > -~----------~----~----~----~------~----~------~--~--- > From jhannah at omnihotels.com Wed Oct 14 19:15:37 2009 From: jhannah at omnihotels.com (Jay Hannah) Date: Wed, 14 Oct 2009 21:15:37 -0500 Subject: [Omaha.pm] DBIx::Class::Schema::Loader moniker_map Message-ID: <396CEDAA86B38646ACE2FEAA22C3FBF10281D807@l3exchange.omnihotels.net> Hey, looky there... By default DBIx::Class::Schema::Loader creates the class 'XfHotelcode' from the database table 'xf_hotelCode'. If you don't like that conversion of casing and underscore changes, simply override 'moniker_map' to whatever you prefer. http://search.cpan.org/~ilmari/DBIx-Class-Schema-Loader-0.04006/lib/DBIx /Class/Schema/Loader/Base.pm#moniker_map Handy. :) j -------------- next part -------------- An HTML attachment was scrubbed... URL: From netarttodd at gmail.com Tue Oct 20 07:25:47 2009 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Tue, 20 Oct 2009 09:25:47 -0500 Subject: [Omaha.pm] perlembed Use Perl from C In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> Message-ID: <4ADDC86B.6020807@Gmail.com> Has anyone successfully embedded perl into a c application? I would like to try something. For years I have used FileMaker Pro. One of the features of the app is the ability to create "Plug-ins" using the external function c api. I was wondering if anyone has done anything like that. Todd From jay at jays.net Tue Oct 20 08:44:55 2009 From: jay at jays.net (jay at jays.net) Date: Tue, 20 Oct 2009 11:44:55 -0400 Subject: [Omaha.pm] perlembed Use Perl from C In-Reply-To: <4ADDC86B.6020807@Gmail.com> References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> Message-ID: On Tue, 20 Oct 2009 09:25:47 -0500, Todd Christopher Hamilton wrote: > Has anyone successfully embedded perl into a c application? I would > like to try something. For years I have used FileMaker Pro. One of the > features of the app is the ability to create "Plug-ins" using the > external function c api. I was wondering if anyone has done anything > like that. I haven't done any XS... http://perldoc.perl.org/perlxs.html j not just another C hacker :) From jhannah at omnihotels.com Wed Oct 21 07:32:02 2009 From: jhannah at omnihotels.com (Jay Hannah) Date: Wed, 21 Oct 2009 09:32:02 -0500 Subject: [Omaha.pm] Whoah. They're talking about my code on PerlMonks Message-ID: <396CEDAA86B38646ACE2FEAA22C3FBF10281DE1E@l3exchange.omnihotels.net> http://www.perlmonks.org/?node_id=793206 j -------------- next part -------------- An HTML attachment was scrubbed... URL: From netarttodd at gmail.com Wed Oct 21 11:32:03 2009 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Wed, 21 Oct 2009 13:32:03 -0500 Subject: [Omaha.pm] perlembed Use Perl from C In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> Message-ID: <4ADF53A3.1000307@Gmail.com> I am going to look into it some more. I am going to start by compiling perl for win32. (Ive always used ActiveState). Then see what it takes to compile it to a FMP plug-in. I suppose I should learn C++ as well. jay at jays.net wrote: > On Tue, 20 Oct 2009 09:25:47 -0500, Todd Christopher Hamilton > wrote: >> Has anyone successfully embedded perl into a c application? I would >> like to try something. For years I have used FileMaker Pro. One of the >> features of the app is the ability to create "Plug-ins" using the >> external function c api. I was wondering if anyone has done anything >> like that. > > I haven't done any XS... > > http://perldoc.perl.org/perlxs.html > > j > not just another C hacker :) > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm From netarttodd at gmail.com Wed Oct 21 11:39:04 2009 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Wed, 21 Oct 2009 13:39:04 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> Message-ID: <4ADF5548.4030109@Gmail.com> I installed Catalyst for the first time. I've wanted to for a couple of years now but it was to hard and I was to dumb. Now one of those things has changed. (You guess which one) Question: 1. What do you use/recomend to authenticate against MS Active Directory? 2. What template module do you use the View? 3. Do you use any javascript stuff as well? I use Dojo and wondering if they can have a complimentary relationship. Todd From sterling at hanenkamp.com Wed Oct 21 12:00:52 2009 From: sterling at hanenkamp.com (Sterling Hanenkamp) Date: Wed, 21 Oct 2009 14:00:52 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: <4ADF5548.4030109@Gmail.com> References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Wed, Oct 21, 2009 at 1:39 PM, Todd Christopher Hamilton < netarttodd at gmail.com> wrote: > Question: > 2. What template module do you use the View? > I'm a fan of Template::Declare. You can try it via Catalyst::View::Template::Declare by Jonathan Rockway. I've also used Template::Toolkit and HTML::Mason as well. > 3. Do you use any javascript stuff as well? I use Dojo and wondering if > they can have a complimentary relationship. > I tend to prefer jQuery and friends, but Dojo should work fine. Integration of your templates with your JavaScript library is mostly up to you, but is pretty simple to manage. There are some tools for sending back JSON or XML. I've not used it, but I'd look at Miyagawa's Catalyst::View::JSON. I have used Doran's Catalyst::Action::REST module, which takes some setup, but works well if you want a more flexible REST API and supports JSON, XML, and other formats. Cheers, Sterling -- Andrew Sterling Hanenkamp sterling at hanenkamp.com 785.370.4454 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dan at linder.org Wed Oct 21 12:17:45 2009 From: dan at linder.org (Dan Linder) Date: Wed, 21 Oct 2009 14:17:45 -0500 Subject: [Omaha.pm] Self monitoring a perl scripts memory usage... Message-ID: <3e2be50910211217o50073897nc0570dc073d4e548@mail.gmail.com> I have a script that usually maxes out around 100MB of memory (viewed via /proc//status and watching the "VmPeak" value). In one test case, the complexity of the task grows, and the VmPeak tops out at over 300MB. The server this is running on has "ulimit -v 256000" (roughly 256MB), so once the script passes 256MB it is killed. Questions: 1: Is there a way to let a perl script query itself or the perl executable to determine how much memory it is currently using? I'd like to log the highwater mark at exit so we can watch this over time, or possibly exit when "250MB" is used and give the user instructions on upping the limit in a config file after upping the ulimit value. 2: Can the signal be captured in a handler? If I read chapter 17 of the OReilly "Learning Perl 3rd Edition", it says that they can't be trapped... :-( Thanks, DanL -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From jay at jays.net Wed Oct 21 12:32:13 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 21 Oct 2009 14:32:13 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: <4ADF5548.4030109@Gmail.com> References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Oct 21, 2009, at 1:39 PM, Todd Christopher Hamilton wrote: > 1. What do you use/recomend to authenticate against MS Active > Directory? We have Apache talk to AD for us via mod_auth_kerb.so: http://modauthkerb.sourceforge.net/index.html > 2. What template module do you use the View? http://template-toolkit.org/ > 3. Do you use any javascript stuff as well? Sure. Catalyst doesn't know or care about that -- Javascript is just more text, like CSS or HTML. We tend to use jQuery. https://ssl.omnihotels.com/ :) j From jay at jays.net Wed Oct 21 13:32:09 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 21 Oct 2009 15:32:09 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Oct 21, 2009, at 2:00 PM, Sterling Hanenkamp wrote: > On Wed, Oct 21, 2009 at 1:39 PM, Todd Christopher Hamilton > wrote: >> 3. Do you use any javascript stuff as well? I use Dojo and >> wondering if they can have a complimentary relationship. > > I tend to prefer jQuery and friends, but Dojo should work fine. > Integration of your templates with your JavaScript library is mostly > up to you, but is pretty simple to manage. There are some tools for > sending back JSON or XML. I've not used it, but I'd look at > Miyagawa's Catalyst::View::JSON. I have used Doran's > Catalyst::Action::REST module, which takes some setup, but works > well if you want a more flexible REST API and supports JSON, XML, > and other formats. Sterling: Very cool! Would you be willing to present something RESTy at an upcoming meeting? I'd love to see it! :) http://jays.net/wiki/odlug Todd: The lastest batch of Catalyst AJAX I did didn't use any Catalyst magic at all. As Sterling pointed out, there are dozens of plugins where Catalyst becomes aware of all sorts of things. :) http://search.cpan.org/search/?query=catalyst%3A%3Aplugin&mode=all My thing uses jQuery Tooltip http://jquery.bassistance.de/tooltip/demo/ and Catalyst just blindly spews out HTML/JS/XML to make jQuery happy. :) j From jay at jays.net Wed Oct 21 13:40:31 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 21 Oct 2009 15:40:31 -0500 Subject: [Omaha.pm] Self monitoring a perl scripts memory usage... In-Reply-To: <3e2be50910211217o50073897nc0570dc073d4e548@mail.gmail.com> References: <3e2be50910211217o50073897nc0570dc073d4e548@mail.gmail.com> Message-ID: <8FB7FD39-F9D2-4340-A2F6-7804F6AB2389@jays.net> On Oct 21, 2009, at 2:17 PM, Dan Linder wrote: > I have a script that usually maxes out around 100MB of memory (viewed > via /proc//status and watching the "VmPeak" value). > In one test case, the complexity of the task grows, and the VmPeak > tops out at over 300MB. The server this is running on has "ulimit -v > 256000" (roughly 256MB), so once the script passes 256MB it is killed. Wow. That's a big process. :) You have to have all that in memory all at once? Can you cache out some of it or something? (Cache::FileCache?) > Questions: > 1: Is there a way to let a perl script query itself or the perl > executable to determine how much memory it is currently using? I'd > like to log the highwater mark at exit so we can watch this over time, > or possibly exit when "250MB" is used and give the user instructions > on upping the limit in a config file after upping the ulimit value. I'm not aware of a smart way. On Linux you could always run fork a "ps -ef" and find yourself. irc.perl.org #perl-help might be helpful. > 2: Can the signal be captured in a handler? If I read chapter 17 of > the OReilly "Learning Perl 3rd Edition", it says that they can't be > trapped... :-( If the OS sends the signal to perl then it's trivial to assign that signal to a subroutine and Do The Right Thing... In Linux, anyway. I don't know anything about Windows. :) http://perldoc.perl.org/perlipc.html#Signals HTH, j From sterling at hanenkamp.com Wed Oct 21 13:46:54 2009 From: sterling at hanenkamp.com (Sterling Hanenkamp) Date: Wed, 21 Oct 2009 15:46:54 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Wed, Oct 21, 2009 at 3:32 PM, Jay Hannah wrote: > Sterling: Very cool! Would you be willing to present something RESTy at an > upcoming meeting? I'd love to see it! :) > > http://jays.net/wiki/odlug > > I'd love to, but I'll need an excuse to come to Omaha. You know of anything fun and interesting to do in Omaha for kids under 3? Cheers, Sterling -- Andrew Sterling Hanenkamp sterling at hanenkamp.com 785.370.4454 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Wed Oct 21 13:48:26 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 21 Oct 2009 15:48:26 -0500 Subject: [Omaha.pm] 2 pushes to CPAN today Message-ID: http://search.cpan.org/~jhannah/ Impressed? I am. :) Andy: Ya, ya, I know. You've probably pushed dozens in a single day. :) j From jay at jays.net Wed Oct 21 13:49:48 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 21 Oct 2009 15:49:48 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Oct 21, 2009, at 3:46 PM, Sterling Hanenkamp wrote: > I'd love to, but I'll need an excuse to come to Omaha. You know of > anything fun and interesting to do in Omaha for kids under 3? Oh? Where are you located? We've got lots of cardboard boxes. I've heard kids love those. :) (We have the #2 zoo in the US.) j From andy at petdance.com Wed Oct 21 14:01:16 2009 From: andy at petdance.com (Andy Lester) Date: Wed, 21 Oct 2009 16:01:16 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Oct 21, 2009, at 3:49 PM, Jay Hannah wrote: > (We have the #2 zoo in the US.) I know that zoos tend to have a lot of #2 from all the animals, but I've never heard of a zoo dedicated to it. xoxo, Andy -- Andy Lester => andy at petdance.com => www.theworkinggeek.com => AIM:petdance From sterling at hanenkamp.com Wed Oct 21 14:46:05 2009 From: sterling at hanenkamp.com (Sterling Hanenkamp) Date: Wed, 21 Oct 2009 16:46:05 -0500 Subject: [Omaha.pm] Catalyst In-Reply-To: References: <50368148-ECFF-4FB6-9F66-29170690859E@jays.net> <3A586D8C-197B-4D19-A660-9F7B6D3297D7@jays.net> <4ADDC86B.6020807@Gmail.com> <4ADF5548.4030109@Gmail.com> Message-ID: On Wed, Oct 21, 2009 at 3:49 PM, Jay Hannah wrote: > On Oct 21, 2009, at 3:46 PM, Sterling Hanenkamp wrote: > >> I'd love to, but I'll need an excuse to come to Omaha. You know of >> anything fun and interesting to do in Omaha for kids under 3? >> > > Oh? Where are you located? > > We've got lots of cardboard boxes. I've heard kids love those. :) > > (We have the #2 zoo in the US.) > I'm in Manhattan, Kansas, so about 4 hours away. The zoo does come highly recommended, but November might not be an ideal time to visit. -- Andrew Sterling Hanenkamp sterling at hanenkamp.com 785.370.4454 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jay at jays.net Wed Oct 21 16:11:19 2009 From: jay at jays.net (Jay Hannah) Date: Wed, 21 Oct 2009 18:11:19 -0500 Subject: [Omaha.pm] Fwd: Poll: Which editor(s) or IDE(s) are you using for Perl development? References: Message-ID: On Oct 21, 2009, at 5:26 PM, Gabor Szabo wrote: > Hi, > > I have setup a simple 5-second poll to find out what editor(s) or IDE > (s) > people use for Perl development. I'd appreciate very much if you > clicked > on the link and answered the question. You can mark up to 3 answers. > > Please also forward this mail in the company you are working and to > people > in your previous company so we can get a large and diverse set of > responses. > > The poll will be closed within a week or after we reached 1000 voters. > Whichever comes first. > > Please act now! > > http://bit.ly/2V8Ltz > > > thanks > Gabor > http://szabgab.com/blog.html > http://perlmongers.wordpress.com/ From jay at jays.net Fri Oct 23 06:37:09 2009 From: jay at jays.net (Jay Hannah) Date: Fri, 23 Oct 2009 08:37:09 -0500 Subject: [Omaha.pm] IRC chat links Message-ID: I didn't realize how strong the browser-based IRC clients have gotten. Pretty awesome. I added 'Chat with us' links on the top of our homepages. http://jays.net/wiki/odlug http://jays.net/wiki/opm :) j From jay at jays.net Sat Oct 24 08:50:20 2009 From: jay at jays.net (Jay Hannah) Date: Sat, 24 Oct 2009 10:50:20 -0500 Subject: [Omaha.pm] Pet Paradise: Ruby > Perl > Erlang! Message-ID: <0A64869D-FF0C-473C-87F0-2109397FC9E4@jays.net> Remember the pet_paradise challenge? http://github.com/jhannah/odynug Juan wrote a Ruby version and took over the leader board! :) jhannah at minijaysnet~/src/odynug/pet_paradise/ruby_1$ ruby runHotel.rb 30 reservations made, with a Grand Total of $3014.68 1,4,5,6,7,8,9,11,13,14,16,17,18,19,20,21,22,24,26,27,28,30,31,32,33,34,36,37,38,40 , I cannot let that stand. I will be beefing up perl_1 to take back the crown for the glory of Perl!! :) Juan: So those numbers are the reservation IDs you fit into the hotel to maximize revenue, right? Excellent... If I didn't have to leave the house in 10 minutes I would seek my revenge this morning... Soon. Very soon. erlang_1 is in 3rd place right now, I believe... Fight back stesla! :D j From dan at linder.org Wed Oct 28 12:10:18 2009 From: dan at linder.org (Dan Linder) Date: Wed, 28 Oct 2009 14:10:18 -0500 Subject: [Omaha.pm] Self monitoring a perl scripts memory usage... In-Reply-To: <8FB7FD39-F9D2-4340-A2F6-7804F6AB2389@jays.net> References: <3e2be50910211217o50073897nc0570dc073d4e548@mail.gmail.com> <8FB7FD39-F9D2-4340-A2F6-7804F6AB2389@jays.net> Message-ID: <3e2be50910281210s4e088cdcl438df6c26710d2bf@mail.gmail.com> On Wed, Oct 21, 2009 at 15:40, Jay Hannah wrote: > Wow. That's a big process. ?:) ?You have to have all that in memory all at > once? Can you cache out some of it or something? (Cache::FileCache?) The script is building up a large hash table in RAM which is then dumped to a text file when it finishes. In this exact instance, the system is expanding user/group entries that were dumped from a Windows domain controller. Since the domain is very dynamic, the actual RAM requirements change day-to-day so it doesn't fail every day. The quick and dirty solution is to bump up the ulimit on the account that the process is running under. But, that's not the Perl solution. My first inclination was to capture the system events and exit the subroutine cleanly. Unfortunately the OOM signal is not allowed to be captured by the program... I did some digging around and there is a special Perl variable $^M - the "Emergency Memory pool" variable. In short, the variable is _documented_ to allow a Perl script to cleanly handle getting low on memory, but apparently it is not handled within the Perl interperter itself. See "Is $^M a leftover April Fool?" : http://www.perlmonks.org/index.pl?node_id=287850 The only work around I have right now is that the Perl interpreter seems (or the OS?) prints "out of memory" to the screen as it is being killed. When I run the script from within a wrapper script, the wrapper can capture the "out of memory" in the output and I can search for that and report back. Not a great answer, but it's something. I'm still pursuing the $^M variable. I'm hoping I can find the key code that makes it work. Dan -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From jay at jays.net Fri Oct 30 07:08:12 2009 From: jay at jays.net (Jay Hannah) Date: Fri, 30 Oct 2009 09:08:12 -0500 Subject: [Omaha.pm] Fwd: Web site consulting work References: Message-ID: Any freelance web devs out there? j Begin forwarded message: > From: Adam Haeder > Date: October 28, 2009 11:46:37 AM CDT > To: olug at olug.org > Subject: [olug] Web site consulting work > Reply-To: Omaha Linux User Group > > Hello OLUGers, > We have a client who is looking for a website facelift. They are a > small hospital in Plainview, NE. It's not really in our scope to do > one-off web development like this (especially for a relatively small > company), so I thought I'd throw it out to the list. This looks like > a good opportunity for a web developer to get some work in on the > side. > > The web site is: http://www.plainviewhealth.org/ > Contact info: > Diane Blair > HR Director > Plainview Health Center, Plainview, NE > Dianeb at pahsne.com > 402-582-4245 > > Email Diane directly if you are interested. And please no comments > about the existing site, they know it needs a lot of help :) > > -- > Adam Haeder > Vice President of Information Technology > AIM Institute > 1905 Harney Street, Suite 700 > Omaha, NE 68102 > 402-345-5025 x115 > adamh at aiminstitute.org > www.aiminstitute.org From netarttodd at gmail.com Fri Oct 30 07:55:30 2009 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Fri, 30 Oct 2009 09:55:30 -0500 Subject: [Omaha.pm] Fwd: Web site consulting work In-Reply-To: References: Message-ID: <4AEAFE62.5090702@Gmail.com> I just talked to her. I am going to give her a project quote. Thanks for the info. Jay Hannah wrote: > > Any freelance web devs out there? > > j > > > > Begin forwarded message: >> From: Adam Haeder >> Date: October 28, 2009 11:46:37 AM CDT >> To: olug at olug.org >> Subject: [olug] Web site consulting work >> Reply-To: Omaha Linux User Group >> >> Hello OLUGers, >> We have a client who is looking for a website facelift. They are a >> small hospital in Plainview, NE. It's not really in our scope to do >> one-off web development like this (especially for a relatively small >> company), so I thought I'd throw it out to the list. This looks like a >> good opportunity for a web developer to get some work in on the side. >> >> The web site is: http://www.plainviewhealth.org/ >> Contact info: >> Diane Blair >> HR Director >> Plainview Health Center, Plainview, NE >> Dianeb at pahsne.com >> 402-582-4245 >> >> Email Diane directly if you are interested. And please no comments >> about the existing site, they know it needs a lot of help :) >> >> -- >> Adam Haeder >> Vice President of Information Technology >> AIM Institute >> 1905 Harney Street, Suite 700 >> Omaha, NE 68102 >> 402-345-5025 x115 >> adamh at aiminstitute.org >> www.aiminstitute.org > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm From dan at linder.org Fri Oct 30 08:57:29 2009 From: dan at linder.org (Dan Linder) Date: Fri, 30 Oct 2009 10:57:29 -0500 Subject: [Omaha.pm] BioPerl on FLOSS Weekly Message-ID: <3e2be50910300857o2d718f03o9ef901957ec9c084@mail.gmail.com> Next weeks FLOSS Weekly is interviewing some BioPerl members: Marc Pelletier, Edward Seufert. Here's the FLOSS Weekly URL: http://twit.tv/FLOSS Dan -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From netarttodd at gmail.com Fri Oct 30 11:08:22 2009 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Fri, 30 Oct 2009 13:08:22 -0500 Subject: [Omaha.pm] BioPerl on FLOSS Weekly In-Reply-To: <3e2be50910300857o2d718f03o9ef901957ec9c084@mail.gmail.com> References: <3e2be50910300857o2d718f03o9ef901957ec9c084@mail.gmail.com> Message-ID: <4AEB2B96.70501@Gmail.com> Dan, I've wanted to know. What kind of stuff (cool stuff) are you doing in the area of biotech that you use perl for? Dan Linder wrote: > Next weeks FLOSS Weekly is interviewing some BioPerl members: Marc > Pelletier, Edward Seufert. > > Here's the FLOSS Weekly URL: http://twit.tv/FLOSS > > Dan > From dan at linder.org Fri Oct 30 11:31:10 2009 From: dan at linder.org (Dan Linder) Date: Fri, 30 Oct 2009 13:31:10 -0500 Subject: [Omaha.pm] Mapping Perl structures to a SQL table... Message-ID: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> I'm taking on the task of converting our in-house tool to use the Perl DBI module to replace the Data::Dumper/eval() it currently uses to store and retrieve data. Not pretty, but it has worked pretty well for the small data sets we've been using. We now have some people commenting on the speed - some have pages take 7+ minutes to bring up waiting for the back-end perl code to ripple through the directory structure and eval() the necessary files to build the page. The "eval" function seems to be the bulk of the time as I expected... What I'm looking for is some general comments and discussion about the mental task of mapping these hash tables into a SQL table. I'm not really looking for a tool, more a high level discussion about ways to store the data and still remain flexible. Dan -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** ******************* From sterling at hanenkamp.com Fri Oct 30 13:54:22 2009 From: sterling at hanenkamp.com (Sterling Hanenkamp) Date: Fri, 30 Oct 2009 15:54:22 -0500 Subject: [Omaha.pm] Mapping Perl structures to a SQL table... In-Reply-To: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> References: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> Message-ID: If your data is all hashes, perhaps what you ought to look into is KiokuDB, since it stores hashes very efficiently. If you can turn your hashes into Moose classes while you're at it, all the better. Cheers, Sterling On Fri, Oct 30, 2009 at 1:31 PM, Dan Linder wrote: > I'm taking on the task of converting our in-house tool to use the Perl > DBI module to replace the Data::Dumper/eval() it currently uses to > store and retrieve data. Not pretty, but it has worked pretty well > for the small data sets we've been using. > > We now have some people commenting on the speed - some have pages take > 7+ minutes to bring up waiting for the back-end perl code to ripple > through the directory structure and eval() the necessary files to > build the page. The "eval" function seems to be the bulk of the time > as I expected... > > What I'm looking for is some general comments and discussion about the > mental task of mapping these hash tables into a SQL table. I'm not > really looking for a tool, more a high level discussion about ways to > store the data and still remain flexible. > > Dan > > -- > ******************* ***************** ************* *********** > ******* ***** *** ** > "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from > the Satires of Juvenal > "I do not fear computers, I fear the lack of them." -- Isaac Asimov > (Author) > ** *** ***** ******* *********** ************* ***************** > ******************* > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- Andrew Sterling Hanenkamp sterling at hanenkamp.com 785.370.4454 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mario at ruby-im.net Fri Oct 30 14:06:57 2009 From: mario at ruby-im.net (Mario Steele) Date: Fri, 30 Oct 2009 16:06:57 -0500 Subject: [Omaha.pm] Mapping Perl structures to a SQL table... In-Reply-To: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> References: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> Message-ID: Heya Dan, It would help to have a bit of information about the general structure you are currently using, before giving any ideas about storing the data in a SQL Database. Obviously your doing Directories and Files, but the structure helps in determining the way to make things work in the SQL Database. Poor design of the SQL Database, will lead to poor execution of SQL instructions, as much more is required to get the data you want. To give you some idea of how I would convert a file system database into a SQL Database, I'll give you an example of a file structure database, and a description of what entry means, then show the resulting database structure for SQL. An example, of a simple design using Folders: db/ cust_records/ cust1_info.dat cust1_purchase.dat cust2_info.dat cust2_purchase.dat inventory/ item1_info.dat item2_info.dat cust#_info.dat contains: Customer Name, Address, Phone Number, Shipping Info, and such. cust#_purchase.dat contains: Customer's a record of all the purchases that a customer has made. item#_info.dat contains: Name of Item, Description, Price, Quantity, Shipping price, Shipping Weight. Now, to convert this into a SQL Database, I would formulate it as such: db customers id - INTEGER, PRIMARY KEY, AUTOINCREMENT name - STRING address - STRING phone - STRING ship_to - STRING transactions id - INTEGER, PRIMARY KEY, AUTOINCREMENT cust_id - INTEGER, PRIMARY KEY item_id - INTEGER quantity - INTEGER purchased_date - INTEGER inventory id - INTEGER, PRIMARY KEY, AUTOINCREMENT item_name - STRING description - STRING price - STRING total_per_quantity - INTEGER ship_price - INTEGER ship_weight - INTEGER Now some explaining about what the right hand side is all about in the above layout. The first field in all tables are 'id', which is marked as INTEGER, PRIMARY KEY, AUTOINCREMENT. Integer denotes a number, of course, Primary Key tells the SQL engine to make quick look ups based upon this field being one of the more often checked fields to look up records in a database. Finally the Auto increment (Which is one word in SQL), denotes the fact that each new record put into the SQL database, should take the total number of rows, and add one to that number, to assign the identification number for this record. And lastly, a String is a variable length of text data to be stored. Most SQL engines will allow for 5 or 6 paragraphs worth of text, but this can be expensive in storage and retrieval. If you know that a field is only going to be so many characters, such as Phone, maximum being 13 characters, then you can use VARCHAR(13) as the maximum length of the data that is going to be stored in that field. There's also FLOAT, which allows for decimal points, but Integers in most SQL Engines will take decimal numbers, and keep the decimals. But it's always best to see what data types a SQL engine supports, before making a final decision. Most DBI's will automatically provide a way to store common data types in the database, at their best formulation to save as much space for the database engine to handle. So look at Perl's DBI for Constructing Tables to see what assistance it will bring you. Lastly, one other data type I didn't cover in the above database, is the BLOB data type. BLOB data types are for storing Binary data in, should you find the need to store some binary data in the database. With blobs, there are no conversions done to store the data in the database, it's stored as is (As in, as you provide it to the SQL Database), and can contain any valid byte sequence in it. Meaning, anything between 0 and 255 can be stored here. Most SQL Engines will store UTF-8/16 characters in strings without stripping them, but when in doubt, you can use the Blob data type. Now, with the explanations of the data types out of the way, the structure is efficiently designed, for the simple fact, that if you have the ID of the customer, you can get all the items that they purchased, and get each items information from the id's that you get from the transaction table. You can even use the SQL Instruction JOIN to get all the data you need in a single execute, for example, if you wanted to get the name of the person, the name of the item, and the total cost, you could simply do: SELECT name, item_name, price FROM transactions JOIN customers ON customers.id = transactions.cust_id JOIN inventory ON inventory.id = transactions.item_id; This will return a list of all transactions in the format of: name | item_name | price Examples being: "John Doe","ASUS PC",299.99 "Mary Johnson", "Microsoft Mouse", 19.99 etc, etc. It makes cross-table look ups a lot easier, to get the relevant data for what you need, and only what you need. And all of it is handled by the SQL Engine, not Perl, or whatever high level language you use, so the execution speed is greatly improved. HTH, Mario On Fri, Oct 30, 2009 at 1:31 PM, Dan Linder wrote: > I'm taking on the task of converting our in-house tool to use the Perl > DBI module to replace the Data::Dumper/eval() it currently uses to > store and retrieve data. Not pretty, but it has worked pretty well > for the small data sets we've been using. > > We now have some people commenting on the speed - some have pages take > 7+ minutes to bring up waiting for the back-end perl code to ripple > through the directory structure and eval() the necessary files to > build the page. The "eval" function seems to be the bulk of the time > as I expected... > > What I'm looking for is some general comments and discussion about the > mental task of mapping these hash tables into a SQL table. I'm not > really looking for a tool, more a high level discussion about ways to > store the data and still remain flexible. > > Dan > > -- > ******************* ***************** ************* *********** > ******* ***** *** ** > "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from > the Satires of Juvenal > "I do not fear computers, I fear the lack of them." -- Isaac Asimov > (Author) > ** *** ***** ******* *********** ************* ***************** > ******************* > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > -- Mario Steele http://www.trilake.net http://www.ruby-im.net http://rubyforge.org/projects/wxruby/ http://rubyforge.org/projects/wxride/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From netarttodd at gmail.com Fri Oct 30 15:24:04 2009 From: netarttodd at gmail.com (Todd Christopher Hamilton) Date: Fri, 30 Oct 2009 17:24:04 -0500 Subject: [Omaha.pm] Mapping Perl structures to a SQL table... In-Reply-To: References: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> Message-ID: <4AEB6784.3030904@Gmail.com> Here are some thought: I have found that when designing a database one needs to be at peace with the flexibility / performance balance. (Come to think of it isn't that why we use Perl? You can write faster code with C but you can write code faster with Perl) If you want high flexibility you might use a Entity-Attribute-Value model. But eventually you come to the point where performance requirements force you to think in more ridged terms. You will then apply performance based design principles within the context of the information domain. Since you are converting a file system database to a DBMS structure, you already have some structure you can exploit. Files, Directories (which are files), file attributes (fixed number of attributes and values). The real flexibility comes when you need to store the contents of the file. But then again how much performance do you need/ Your performance comes in when you are trying find and map the structure. Once you find the file base on the attributes you then will display it. I would start with three tables: 1. Files 2. FileAttributes 2. FileContent Files would have the following columns 1. Id 2. ParentId ( the id of the directory this file is in ) 3. Name 4... All your permission attributes like user read permissions, group permission 5... All you file attributes like creation date time, mod date time, size Then you second table would contain non indexed attributes of the file or directory. Category, Subject, Author, status Then your third table would be the actual file contents. You would then use the DBMS to provide performance structure (indexes, constraints, aggregation, transactions, journaling) Mario Steele wrote: > Heya Dan, > > It would help to have a bit of information about the general structure > you are currently using, before giving any ideas about storing the data > in a SQL Database. Obviously your doing Directories and Files, but the > structure helps in determining the way to make things work in the SQL > Database. Poor design of the SQL Database, will lead to poor execution > of SQL instructions, as much more is required to get the data you want. > > To give you some idea of how I would convert a file system database into > a SQL Database, I'll give you an example of a file structure database, > and a description of what entry means, then show the resulting database > structure for SQL. > An example, of a simple design using Folders: > > db/ > cust_records/ > cust1_info.dat > cust1_purchase.dat > cust2_info.dat > cust2_purchase.dat > inventory/ > item1_info.dat > item2_info.dat > > cust#_info.dat contains: Customer Name, Address, Phone Number, Shipping > Info, and such. > cust#_purchase.dat contains: Customer's a record of all the purchases > that a customer has made. > > item#_info.dat contains: Name of Item, Description, Price, Quantity, > Shipping price, Shipping Weight. > > Now, to convert this into a SQL Database, I would formulate it as such: > > db > customers > id - INTEGER, PRIMARY KEY, AUTOINCREMENT > name - STRING > address - STRING > phone - STRING > ship_to - STRING > transactions > id - INTEGER, PRIMARY KEY, AUTOINCREMENT > cust_id - INTEGER, PRIMARY KEY > item_id - INTEGER > quantity - INTEGER > purchased_date - INTEGER > inventory > id - INTEGER, PRIMARY KEY, AUTOINCREMENT > item_name - STRING > description - STRING > price - STRING > total_per_quantity - INTEGER > ship_price - INTEGER > ship_weight - INTEGER > > Now some explaining about what the right hand side is all about in the > above layout. The first field in all tables are 'id', which is marked as > INTEGER, PRIMARY KEY, AUTOINCREMENT. Integer denotes a number, of > course, Primary Key tells the SQL engine to make quick look ups based > upon this field being one of the more often checked fields to look up > records in a database. Finally the Auto increment (Which is one word in > SQL), denotes the fact that each new record put into the SQL database, > should take the total number of rows, and add one to that number, to > assign the identification number for this record. And lastly, a String > is a variable length of text data to be stored. Most SQL engines will > allow for 5 or 6 paragraphs worth of text, but this can be expensive in > storage and retrieval. If you know that a field is only going to be so > many characters, such as Phone, maximum being 13 characters, then you > can use VARCHAR(13) as the maximum length of the data that is going to > be stored in that field. > > There's also FLOAT, which allows for decimal points, but Integers in > most SQL Engines will take decimal numbers, and keep the decimals. But > it's always best to see what data types a SQL engine supports, before > making a final decision. Most DBI's will automatically provide a way to > store common data types in the database, at their best formulation to > save as much space for the database engine to handle. So look at Perl's > DBI for Constructing Tables to see what assistance it will bring you. > Lastly, one other data type I didn't cover in the above database, is the > BLOB data type. BLOB data types are for storing Binary data in, should > you find the need to store some binary data in the database. > > With blobs, there are no conversions done to store the data in the > database, it's stored as is (As in, as you provide it to the SQL > Database), and can contain any valid byte sequence in it. Meaning, > anything between 0 and 255 can be stored here. Most SQL Engines will > store UTF-8/16 characters in strings without stripping them, but when in > doubt, you can use the Blob data type. > > Now, with the explanations of the data types out of the way, the > structure is efficiently designed, for the simple fact, that if you have > the ID of the customer, you can get all the items that they purchased, > and get each items information from the id's that you get from the > transaction table. You can even use the SQL Instruction JOIN to get all > the data you need in a single execute, for example, if you wanted to get > the name of the person, the name of the item, and the total cost, you > could simply do: > > SELECT name, item_name, price > FROM transactions > JOIN customers > ON customers.id = transactions.cust_id > JOIN inventory > ON inventory.id = transactions.item_id; > > This will return a list of all transactions in the format of: > name | item_name | price > > Examples being: > > "John Doe","ASUS PC",299.99 > "Mary Johnson", "Microsoft Mouse", 19.99 > etc, etc. > > It makes cross-table look ups a lot easier, to get the relevant data for > what you need, and only what you need. And all of it is handled by the > SQL Engine, not Perl, or whatever high level language you use, so the > execution speed is greatly improved. > > HTH, > > Mario > > > On Fri, Oct 30, 2009 at 1:31 PM, Dan Linder > wrote: > > I'm taking on the task of converting our in-house tool to use the Perl > DBI module to replace the Data::Dumper/eval() it currently uses to > store and retrieve data. Not pretty, but it has worked pretty well > for the small data sets we've been using. > > We now have some people commenting on the speed - some have pages take > 7+ minutes to bring up waiting for the back-end perl code to ripple > through the directory structure and eval() the necessary files to > build the page. The "eval" function seems to be the bulk of the time > as I expected... > > What I'm looking for is some general comments and discussion about the > mental task of mapping these hash tables into a SQL table. I'm not > really looking for a tool, more a high level discussion about ways to > store the data and still remain flexible. > > Dan > > -- > ******************* ***************** ************* *********** > ******* ***** *** ** > "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from > the Satires of Juvenal > "I do not fear computers, I fear the lack of them." -- Isaac Asimov > (Author) > ** *** ***** ******* *********** ************* ***************** > ******************* > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm > > > > > -- > Mario Steele > http://www.trilake.net > http://www.ruby-im.net > http://rubyforge.org/projects/wxruby/ > http://rubyforge.org/projects/wxride/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > Omaha-pm mailing list > Omaha-pm at pm.org > http://mail.pm.org/mailman/listinfo/omaha-pm From jay at jays.net Sat Oct 31 07:02:10 2009 From: jay at jays.net (Jay Hannah) Date: Sat, 31 Oct 2009 09:02:10 -0500 Subject: [Omaha.pm] Mapping Perl structures to a SQL table... In-Reply-To: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> References: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> Message-ID: On Oct 30, 2009, at 1:31 PM, Dan Linder wrote: > What I'm looking for is some general comments and discussion about the > mental task of mapping these hash tables into a SQL table. I'm not > really looking for a tool, more a high level discussion about ways to > store the data and still remain flexible. An RDBMS is the most flexible thing around. Ignore your current implementation details and focus on the problems that your data addresses. Organize your data by business concept (table) that would make sense to current and future employees of that business. Keep whatever performance metrics are important to you in mind as you design the tables. Write software to map your hashes into that model. Push all the data into an RDBMS. Benchmark your most common interactions with that data. Optimize your indexes for those use cases. Benchmark again. Prep all your tools for your new model. QA. Migrate the data. svn rm the old stuff. :) HTH, j From jay at jays.net Sat Oct 31 07:06:50 2009 From: jay at jays.net (Jay Hannah) Date: Sat, 31 Oct 2009 09:06:50 -0500 Subject: [Omaha.pm] BioPerl on FLOSS Weekly In-Reply-To: <4AEB2B96.70501@Gmail.com> References: <3e2be50910300857o2d718f03o9ef901957ec9c084@mail.gmail.com> <4AEB2B96.70501@Gmail.com> Message-ID: On Oct 30, 2009, at 1:08 PM, Todd Christopher Hamilton wrote: > I've wanted to know. What kind of stuff (cool stuff) are you doing > in the area of biotech that you use perl for? Dan: Thanks for the link! Can't wait for that one. Todd: FYI, Kiran and I and others a lot of BioPerl for UNO and UNMC: http://clab.ist.unomaha.edu/CLAB/index.php/Main_Page http://gsaf.ist.unomaha.edu/GSAF/index.php/Main_Page http://clab.ist.unomaha.edu/CLAB/index.php/SVN Cheers, j http://clab.ist.unomaha.edu/CLAB/index.php/User:Jhannah From jay at jays.net Sat Oct 31 07:10:44 2009 From: jay at jays.net (Jay Hannah) Date: Sat, 31 Oct 2009 09:10:44 -0500 Subject: [Omaha.pm] Self monitoring a perl scripts memory usage... In-Reply-To: <3e2be50910281210s4e088cdcl438df6c26710d2bf@mail.gmail.com> References: <3e2be50910211217o50073897nc0570dc073d4e548@mail.gmail.com> <8FB7FD39-F9D2-4340-A2F6-7804F6AB2389@jays.net> <3e2be50910281210s4e088cdcl438df6c26710d2bf@mail.gmail.com> Message-ID: <016EDC3F-2911-4258-9AC6-35AE429B5829@jays.net> On Oct 28, 2009, at 2:10 PM, Dan Linder wrote: On Wed, Oct 21, 2009 at 15:40, Jay Hannah wrote: >> Wow. That's a big process. :) You have to have all that in memory >> all at >> once? Can you cache out some of it or something? (Cache::FileCache?) > > The script is building up a large hash table in RAM which is then > dumped to a text file when it finishes. In this exact instance, the > system is expanding user/group entries that were dumped from a Windows > domain controller. Since the domain is very dynamic, the actual RAM > requirements change day-to-day so it doesn't fail every day. As you process each user/group you can't Cache::FileCache it or shove it into a database or something? j From dan at linder.org Sat Oct 31 07:26:13 2009 From: dan at linder.org (Dan Linder) Date: Sat, 31 Oct 2009 09:26:13 -0500 Subject: [Omaha.pm] Mapping Perl structures to a SQL table... In-Reply-To: References: <3e2be50910301131h5803fe21j5843873d37f9bd27@mail.gmail.com> Message-ID: <3e2be50910310726p7a7b8390xba09741e23af35e5@mail.gmail.com> 2009/10/30 Sterling Hanenkamp : > If your data is all hashes, perhaps what you ought to look into is KiokuDB, > since it stores hashes very efficiently. If you can turn your hashes into > Moose classes while you're at it, all the better. The others on my team that work with this (albeit on a lesser level) are even wary of non-core Perl Modules or moving to a SQL DB at all. I'm taking baby-steps here hoping to help lead them into the "big kids" pool via the wading pool end. But I will look into KiokuDB -- it looks like an interesting module. Dan -- ******************* ***************** ************* *********** ******* ***** *** ** "Quis custodiet ipsos custodes?" (Who can watch the watchmen?) -- from the Satires of Juvenal "I do not fear computers, I fear the lack of them." -- Isaac Asimov (Author) ** *** ***** ******* *********** ************* ***************** *******************