From awwaiid at thelackthereof.org Fri Mar 3 08:26:56 2006 From: awwaiid at thelackthereof.org (Brock) Date: Fri, 3 Mar 2006 09:26:56 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th Message-ID: <20060303162656.GD19955@thelackthereof.org> Lets have a meeting on March 15th (wed). I was thinking we could mix it up a bit and have a meeting at a non-scc location. Nello's worked so-so but had food which was great. Also we now have the magical VNC technology which would allow us to do more complex presentations without a projector/whiteboard. Send your suggestions here. Scott told me he will talk about building a search engine (both ends, indexing data and then extracting it for search results). Which is good since I need to build one of those. Anyone here use Catalyst? I'd like to see whats going on with that. All topics welcome of course. I'm delaying the Ticketmaster presentation until the following meeting because it is going to take some further coordination to get them out. --Brock From awwaiid at thelackthereof.org Fri Mar 3 09:04:43 2006 From: awwaiid at thelackthereof.org (Brock) Date: Fri, 3 Mar 2006 10:04:43 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060303163516.GA3823@narwhal.gilariver.net> References: <20060303162656.GD19955@thelackthereof.org> <20060303163516.GA3823@narwhal.gilariver.net> Message-ID: <20060303170443.GE19955@thelackthereof.org> So is that a volunteering to tell us a bit about it at the meeting? Doesn't have to be formal or fancy of course! Maybe even just show us some code, or tell us some of the pros/cons. --Brock On 2006.03.03.09.35, Mike Garfias wrote: | Brock spoke forth with the blessed manuscript: | > Anyone here use Catalyst? I'd like to see whats going on with that. All | > topics welcome of course. | | Yep, I do. From mike at garfias.org Fri Mar 3 09:54:01 2006 From: mike at garfias.org (Michael Garfias) Date: Fri, 3 Mar 2006 10:54:01 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060303170443.GE19955@thelackthereof.org> References: <20060303162656.GD19955@thelackthereof.org> <20060303163516.GA3823@narwhal.gilariver.net> <20060303170443.GE19955@thelackthereof.org> Message-ID: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hmm... I'm going to be giving a quick intro at the next PLUG meeting. I could do something similar without a problem. On Mar 3, 2006, at 10:04 AM, Brock wrote: > So is that a volunteering to tell us a bit about it at the meeting? > Doesn't have to be formal or fancy of course! Maybe even just show us > some code, or tell us some of the pros/cons. > > --Brock > > On 2006.03.03.09.35, Mike Garfias wrote: > | Brock spoke forth with the blessed manuscript: > | > Anyone here use Catalyst? I'd like to see whats going on with > that. All > | > topics welcome of course. > | > | Yep, I do. > > !DSPAM:11,440878d7179311186321401! > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFECIK5VKAN7Sr/6+cRAlTcAJ9G0qeizAbhSeiUbj1QVsOh3vxoEgCaAs1f 1R2YcWrSqzmK07z3rmoMAxA= =8cQs -----END PGP SIGNATURE----- From bwmetz at att.com Fri Mar 3 09:57:53 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Fri, 3 Mar 2006 11:57:53 -0600 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> Message-ID: <01D5341D04A2E64AB9B345769047336701667A79@OCCLUST01EVS1.ugd.att.com> Intro on what? Bobby -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Michael Garfias Sent: Friday, March 03, 2006 10:54 AM To: Brock Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Call for Venue, Meeting on March 15th -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hmm... I'm going to be giving a quick intro at the next PLUG meeting. I could do something similar without a problem. On Mar 3, 2006, at 10:04 AM, Brock wrote: > So is that a volunteering to tell us a bit about it at the meeting? > Doesn't have to be formal or fancy of course! Maybe even just show us > some code, or tell us some of the pros/cons. > > --Brock > > On 2006.03.03.09.35, Mike Garfias wrote: > | Brock spoke forth with the blessed manuscript: > | > Anyone here use Catalyst? I'd like to see whats going on with > that. All > | > topics welcome of course. > | > | Yep, I do. > > !DSPAM:11,440878d7179311186321401! > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFECIK5VKAN7Sr/6+cRAlTcAJ9G0qeizAbhSeiUbj1QVsOh3vxoEgCaAs1f 1R2YcWrSqzmK07z3rmoMAxA= =8cQs -----END PGP SIGNATURE----- _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm From awwaiid at thelackthereof.org Fri Mar 3 10:04:04 2006 From: awwaiid at thelackthereof.org (Brock) Date: Fri, 3 Mar 2006 11:04:04 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <01D5341D04A2E64AB9B345769047336701667A79@OCCLUST01EVS1.ugd.att.com> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <01D5341D04A2E64AB9B345769047336701667A79@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060303180404.GA29330@thelackthereof.org> Catalyst, a fancy-shmancy Model-View-Controler framework for Perl (see http://catalyst.perl.org). Some people might bring this framework up when you mention "Ruby on Rails" in their company. Others would not. Yes, please plan on it :) --Brock On 2006.03.03.11.57, Metz, Bobby W, WCS wrote: | Intro on what? | | Bobby | | -----Original Message----- | From: phoenix-pm-bounces+bwmetz=att.com at pm.org | [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Michael | Garfias | Sent: Friday, March 03, 2006 10:54 AM | To: Brock | Cc: phoenix-pm at pm.org | Subject: Re: [Phoenix-pm] Call for Venue, Meeting on March 15th | | | -----BEGIN PGP SIGNED MESSAGE----- | Hash: SHA1 | | Hmm... I'm going to be giving a quick intro at the next PLUG meeting. | | I could do something similar without a problem. | | On Mar 3, 2006, at 10:04 AM, Brock wrote: | | > So is that a volunteering to tell us a bit about it at the meeting? | > Doesn't have to be formal or fancy of course! Maybe even just show us | > some code, or tell us some of the pros/cons. | > | > --Brock | > | > On 2006.03.03.09.35, Mike Garfias wrote: | > | Brock spoke forth with the blessed manuscript: | > | > Anyone here use Catalyst? I'd like to see whats going on with | > that. All | > | > topics welcome of course. | > | | > | Yep, I do. | > | > !DSPAM:11,440878d7179311186321401! | > | > | | -----BEGIN PGP SIGNATURE----- | Version: GnuPG v1.4.1 (Darwin) | | iD8DBQFECIK5VKAN7Sr/6+cRAlTcAJ9G0qeizAbhSeiUbj1QVsOh3vxoEgCaAs1f | 1R2YcWrSqzmK07z3rmoMAxA= | =8cQs | -----END PGP SIGNATURE----- | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From dwchandler at stilyagin.com Fri Mar 3 18:16:42 2006 From: dwchandler at stilyagin.com (Darrin Chandler) Date: Fri, 03 Mar 2006 19:16:42 -0700 Subject: [Phoenix-pm] March Meeting Message-ID: <4408F88A.6010409@stilyagin.com> Hi, The next Phoenix BSD Users Group meeting is scheduled for this coming Tuesday, March 7th at 7:00pm. Due to presenter availability the topic will be Fighting Spam with OpenBSD/spamd/SpamAssassin/relaydb, presented by yours truly. Details are at http://bsd.phoenix.az.us/. Tentative meeting location is the Plaid Eatery. If the location changes I'll post both here and on the web site. Randall L. Schwartz's secondary MX trick (google for "You had me at HELO") is another method I may add to this in the future, but I've already seen a dramatic decrease in both spam and server load. Cheers! dwc -- Darrin Chandler | Phoenix BSD Users Group dwchandler at stilyagin.com | http://bsd.phoenix.az.us/ http://www.stilyagin.com/ | From scott at illogics.org Sat Mar 4 17:41:49 2006 From: scott at illogics.org (Scott Walters) Date: Sun, 5 Mar 2006 01:41:49 +0000 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060303180404.GA29330@thelackthereof.org> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <01D5341D04A2E64AB9B345769047336701667A79@OCCLUST01EVS1.ugd.att.com> <20060303180404.GA29330@thelackthereof.org> Message-ID: <20060305014148.GL22790@illogics.org> Hi Happy Perl People, Someone please suggest a venue. Normally I would, but I don't like any of the Scottsdale coffee shops. Brock has something in mind; I know where pretty much all of the Starbucks are, and, as usual in cultrally devoid office and housing sprawls, they're running circles around the local fare. -scott From andypm at exiledplanet.org Sun Mar 5 02:04:00 2006 From: andypm at exiledplanet.org (Andrew Johnson) Date: Sun, 05 Mar 2006 03:04:00 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060303162656.GD19955@thelackthereof.org> References: <20060303162656.GD19955@thelackthereof.org> Message-ID: <440AB790.2010604@exiledplanet.org> I'll be in Europe on the 15th, but by all means have fun without me. I will be, after all, having fun without you guys. ;-) [aj] Brock wrote: >Lets have a meeting on March 15th (wed). I was thinking we could mix it >up a bit and have a meeting at a non-scc location. Nello's worked so-so >but had food which was great. Also we now have the magical VNC >technology which would allow us to do more complex presentations without >a projector/whiteboard. Send your suggestions here. > >Scott told me he will talk about building a search engine (both ends, >indexing data and then extracting it for search results). Which is good >since I need to build one of those. > >Anyone here use Catalyst? I'd like to see whats going on with that. All >topics welcome of course. > >I'm delaying the Ticketmaster presentation until the following meeting >because it is going to take some further coordination to get them out. > >--Brock > >_______________________________________________ >Phoenix-pm mailing list >Phoenix-pm at pm.org >http://mail.pm.org/mailman/listinfo/phoenix-pm > > From jdawgaz at cox.net Sun Mar 5 09:22:19 2006 From: jdawgaz at cox.net (Jerry Davis) Date: Sun, 5 Mar 2006 10:22:19 -0700 Subject: [Phoenix-pm] anyone use perl for non CGI here? Message-ID: <200603051022.19295.jdawgaz@cox.net> I see a lot of cgi and web stuff on this list. Does anyone use perl for anything other than web? I have used perl daily for over 7 years, and only done one cgi app. The rest have been either stand-alone apps, or glue. Just asking :) Jerry -- Hobbit Name: Pimpernel Loamsdown Registered Linux User: 275424 This email's random fortune: Kill Ugly Processor Architectures - Karl Lehenbauer From friedman at highwire.stanford.edu Sun Mar 5 09:30:40 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Sun, 5 Mar 2006 09:30:40 -0800 Subject: [Phoenix-pm] anyone use perl for non CGI here? In-Reply-To: <200603051022.19295.jdawgaz@cox.net> References: <200603051022.19295.jdawgaz@cox.net> Message-ID: <1751612C-5B69-45C3-90C9-6AD3346A4B4E@highwire.stanford.edu> The CGI and "Web stuff" is merely one part of what we all do. For example, for every CGI that my company has we have a nearly identical command-line script. Customers use the CGI, staff use the scripts, and all the "real" code is in object modules, so there's no duplication. If you have non-CGI questions there are plenty of folk here to answer them. -- Mike On Mar 5, 2006, at 9:22 AM, Jerry Davis wrote: > I see a lot of cgi and web stuff on this list. > Does anyone use perl for anything other than web? > > I have used perl daily for over 7 years, and only done one cgi app. > The rest have been either stand-alone apps, or glue. > > Just asking :) > > Jerry > -- > Hobbit Name: Pimpernel Loamsdown > Registered Linux User: 275424 > > This email's random fortune: Kill Ugly Processor Architectures > - Karl Lehenbauer > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From andypm at exiledplanet.org Sun Mar 5 09:32:41 2006 From: andypm at exiledplanet.org (Andrew Johnson) Date: Sun, 05 Mar 2006 10:32:41 -0700 Subject: [Phoenix-pm] anyone use perl for non CGI here? In-Reply-To: <200603051022.19295.jdawgaz@cox.net> References: <200603051022.19295.jdawgaz@cox.net> Message-ID: <440B20B9.2090507@exiledplanet.org> Yes. Jerry Davis wrote: >I see a lot of cgi and web stuff on this list. >Does anyone use perl for anything other than web? > >I have used perl daily for over 7 years, and only done one cgi app. >The rest have been either stand-alone apps, or glue. > >Just asking :) > >Jerry > > From awwaiid at thelackthereof.org Sun Mar 5 09:36:02 2006 From: awwaiid at thelackthereof.org (Brock) Date: Sun, 5 Mar 2006 10:36:02 -0700 Subject: [Phoenix-pm] anyone use perl for non CGI here? In-Reply-To: <200603051022.19295.jdawgaz@cox.net> References: <200603051022.19295.jdawgaz@cox.net> Message-ID: <20060305173602.GE29330@thelackthereof.org> Indeed we do! And we'd love to discuss any of the many many things that we (and you) do with Perl (here or at meetings). Just start a thread or volutneer to present :) I do web stuff in my day to day job, so its on my mind a lot. But lets see... other stuff. I have lately been doing a side-project munging a bunch of census data. Figured out the 2000 census format and am now learning the 1990 format to do some comparisons. They use a sporatic mix of fixed field and delimited field records. What sort of applications/glue have you been playing with lately? :) --Brock On 2006.03.05.10.22, Jerry Davis wrote: | I see a lot of cgi and web stuff on this list. | Does anyone use perl for anything other than web? | | I have used perl daily for over 7 years, and only done one cgi app. | The rest have been either stand-alone apps, or glue. | | Just asking :) | | Jerry | -- | Hobbit Name: Pimpernel Loamsdown | Registered Linux User: 275424 | | This email's random fortune: Kill Ugly Processor Architectures | - Karl Lehenbauer | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From awwaiid at thelackthereof.org Sun Mar 5 09:38:31 2006 From: awwaiid at thelackthereof.org (Brock) Date: Sun, 5 Mar 2006 10:38:31 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060305014148.GL22790@illogics.org> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <01D5341D04A2E64AB9B345769047336701667A79@OCCLUST01EVS1.ugd.att.com> <20060303180404.GA29330@thelackthereof.org> <20060305014148.GL22790@illogics.org> Message-ID: <20060305173831.GF29330@thelackthereof.org> I put forward Mill's End (coffee shop) in Tempe. Free wireless, near ASU, nice couches. I can try to get there early (and coordinate with the owners) to corner off a nice section for us. Objections and alternatives welcome! --Brock On 2006.03.05.01.41, Scott Walters wrote: | Hi Happy Perl People, | | Someone please suggest a venue. Normally I would, but I don't like any | of the Scottsdale coffee shops. Brock has something in mind; I know | where pretty much all of the Starbucks are, and, as usual in cultrally | devoid office and housing sprawls, they're running circles around the | local fare. | | -scott From jdawgaz at cox.net Sun Mar 5 09:49:41 2006 From: jdawgaz at cox.net (Jerry Davis) Date: Sun, 5 Mar 2006 10:49:41 -0700 Subject: [Phoenix-pm] anyone use perl for non CGI here? In-Reply-To: <20060305173602.GE29330@thelackthereof.org> References: <200603051022.19295.jdawgaz@cox.net> <20060305173602.GE29330@thelackthereof.org> Message-ID: <200603051049.42087.jdawgaz@cox.net> On Sunday 05 March 2006 10:36 am, Brock wrote: > Indeed we do! And we'd love to discuss any of the many many things > that we (and you) do with Perl (here or at meetings). Just start a > thread or volutneer to present :) > > I do web stuff in my day to day job, so its on my mind a lot. But lets > see... other stuff. I have lately been doing a side-project munging a > bunch of census data. Figured out the 2000 census format and am now > learning the 1990 format to do some comparisons. They use a sporatic mix > of fixed field and delimited field records. > > What sort of applications/glue have you been playing with lately? :) I am glad to hear it. :-) Well in the past 7 years I have been in SCM, instead of development, and have wound up being a developer in SCM land! How nice! So I have written entire applications to do build releases with various code repositories like cvs, svn, pvcs (pcli), and now clearcase. I am also doing a lot now with clearquest hooks (written in perl), and clearquest applications. I have written a lot of perl/Tk to support a lot of the command line stuff that I have written. Anyway I am having a blast. I would like to learn a lot more about ruby though, but I don't have the time! ;-) Anyway my post was not meant as a flame. Just a query. I have just moved down to the valley (11/26) to be exact. And I am having a blast here too. I would be happy to share with y'all anything that I know. Happy Trails, Jerry > > --Brock > > On 2006.03.05.10.22, Jerry Davis wrote: > | I see a lot of cgi and web stuff on this list. > | Does anyone use perl for anything other than web? > | > | I have used perl daily for over 7 years, and only done one cgi app. > | The rest have been either stand-alone apps, or glue. > | > | Just asking :) > | > | Jerry > | -- > | Hobbit Name: Pimpernel Loamsdown > | Registered Linux User: 275424 > | > | This email's random fortune: Kill Ugly Processor Architectures > | - Karl Lehenbauer > | _______________________________________________ > | Phoenix-pm mailing list > | Phoenix-pm at pm.org > | http://mail.pm.org/mailman/listinfo/phoenix-pm -- Hobbit Name: Pimpernel Loamsdown Registered Linux User: 275424 This email's random fortune: If Microsoft built cars, the oil, gas, and alternator warning lights would be replaced by a single "general car fault" warning light. From jdawgaz at cox.net Sun Mar 5 10:06:56 2006 From: jdawgaz at cox.net (Jerry Davis) Date: Sun, 5 Mar 2006 11:06:56 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060305173831.GF29330@thelackthereof.org> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <20060305014148.GL22790@illogics.org> <20060305173831.GF29330@thelackthereof.org> Message-ID: <200603051106.56887.jdawgaz@cox.net> On Sunday 05 March 2006 10:38 am, Brock wrote: > I put forward Mill's End (coffee shop) in Tempe. Free wireless, near ASU, > nice couches. I can try to get there early (and coordinate with the > owners) to corner off a nice section for us. do they serve other stuff besides coffee? like maybe chai tea or other kinds of tea? > > Objections and alternatives welcome! > > --Brock > > On 2006.03.05.01.41, Scott Walters wrote: > | Hi Happy Perl People, > | > | Someone please suggest a venue. Normally I would, but I don't like any > | of the Scottsdale coffee shops. Brock has something in mind; I know > | where pretty much all of the Starbucks are, and, as usual in cultrally > | devoid office and housing sprawls, they're running circles around the > | local fare. > | > | -scott > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm -- Hobbit Name: Pimpernel Loamsdown Registered Linux User: 275424 This email's random fortune: The cost of feathers has risen, even down is up! From intertwingled at qwest.net Sun Mar 5 10:13:18 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Sun, 05 Mar 2006 11:13:18 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <200603051106.56887.jdawgaz@cox.net> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <20060305014148.GL22790@illogics.org> <20060305173831.GF29330@thelackthereof.org> <200603051106.56887.jdawgaz@cox.net> Message-ID: <440B2A3E.8090807@qwest.net> Gold Bar Coffee House - 3141 S McClintock Dr - (480) 839-3082 (Southern and Mclintock) The seating inside is limited but they have good outdoor seating and plenty of parking. I am thinking about having some of the Tempe/East Valley Perlmonger's meetings there. Here's a list of places that has free wireless: http://www.wififreespot.com/ar.html Tony Jerry Davis wrote: > On Sunday 05 March 2006 10:38 am, Brock wrote: >> I put forward Mill's End (coffee shop) in Tempe. Free wireless, near ASU, >> nice couches. I can try to get there early (and coordinate with the >> owners) to corner off a nice section for us. > > do they serve other stuff besides coffee? like maybe chai tea or other kinds > of tea? > >> Objections and alternatives welcome! >> >> --Brock >> >> On 2006.03.05.01.41, Scott Walters wrote: >> | Hi Happy Perl People, >> | >> | Someone please suggest a venue. Normally I would, but I don't like any >> | of the Scottsdale coffee shops. Brock has something in mind; I know >> | where pretty much all of the Starbucks are, and, as usual in cultrally >> | devoid office and housing sprawls, they're running circles around the >> | local fare. >> | >> | -scott >> >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > -- I always have coffee when I watch radar! From dwchandler at stilyagin.com Sun Mar 5 10:15:59 2006 From: dwchandler at stilyagin.com (Darrin Chandler) Date: Sun, 05 Mar 2006 11:15:59 -0700 Subject: [Phoenix-pm] anyone use perl for non CGI here? In-Reply-To: <200603051022.19295.jdawgaz@cox.net> References: <200603051022.19295.jdawgaz@cox.net> Message-ID: <440B2ADF.8090001@stilyagin.com> Jerry Davis wrote: >I see a lot of cgi and web stuff on this list. >Does anyone use perl for anything other than web? > >I have used perl daily for over 7 years, and only done one cgi app. >The rest have been either stand-alone apps, or glue. > >Just asking :) > > I haven't done any Perl cgi in many years. It's my first choice for lots of routine tasks. Perl's practical nature makes it an easy choice. Often I start a simple task in shell and switch to Perl almost immediately. -- Darrin Chandler | Phoenix BSD Users Group dwchandler at stilyagin.com | http://bsd.phoenix.az.us/ http://www.stilyagin.com/ | From scott at illogics.org Sun Mar 5 10:56:24 2006 From: scott at illogics.org (Scott Walters) Date: Sun, 5 Mar 2006 18:56:24 +0000 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <20060305173831.GF29330@thelackthereof.org> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <01D5341D04A2E64AB9B345769047336701667A79@OCCLUST01EVS1.ugd.att.com> <20060303180404.GA29330@thelackthereof.org> <20060305014148.GL22790@illogics.org> <20060305173831.GF29330@thelackthereof.org> Message-ID: <20060305185624.GO22790@illogics.org> Sounds good. I'll try to find a comfortable, non-Starbucks spot in North Scottsdale so we can try to rope that crowd in, but I'm excited at the prospects of a technical meeting near ASU. Anyone have any graphic design mojo and willing to make a flyer? -scott On 0, Brock wrote: > I put forward Mill's End (coffee shop) in Tempe. Free wireless, near ASU, > nice couches. I can try to get there early (and coordinate with the > owners) to corner off a nice section for us. > > Objections and alternatives welcome! > > --Brock > > On 2006.03.05.01.41, Scott Walters wrote: > | Hi Happy Perl People, > | > | Someone please suggest a venue. Normally I would, but I don't like any > | of the Scottsdale coffee shops. Brock has something in mind; I know > | where pretty much all of the Starbucks are, and, as usual in cultrally > | devoid office and housing sprawls, they're running circles around the > | local fare. > | > | -scott From scott at illogics.org Sun Mar 5 11:01:28 2006 From: scott at illogics.org (Scott Walters) Date: Sun, 5 Mar 2006 19:01:28 +0000 Subject: [Phoenix-pm] anyone use perl for non CGI here? In-Reply-To: <200603051049.42087.jdawgaz@cox.net> References: <200603051022.19295.jdawgaz@cox.net> <20060305173602.GE29330@thelackthereof.org> <200603051049.42087.jdawgaz@cox.net> Message-ID: <20060305190128.GQ22790@illogics.org> Cool. Hope you can make it. A portiot of SCM land would be a great presentation, and an intro to Perl/Tk (Tim did one years ago, but maybe we can videotape this one for Website posting, and of course, there are lots of new faces who missed Tim's (by the way, we haven't seen or heard from Tim in ages)) would be most welcome. But if you can think of something you'd rather do, by all means. -scott On 0, Jerry Davis wrote: > On Sunday 05 March 2006 10:36 am, Brock wrote: > > Indeed we do! And we'd love to discuss any of the many many things > > that we (and you) do with Perl (here or at meetings). Just start a > > thread or volutneer to present :) > > > > I do web stuff in my day to day job, so its on my mind a lot. But lets > > see... other stuff. I have lately been doing a side-project munging a > > bunch of census data. Figured out the 2000 census format and am now > > learning the 1990 format to do some comparisons. They use a sporatic mix > > of fixed field and delimited field records. > > > > What sort of applications/glue have you been playing with lately? :) > > I am glad to hear it. :-) > > Well in the past 7 years I have been in SCM, instead of development, and have > wound up being a developer in SCM land! How nice! > > So I have written entire applications to do build releases with various code > repositories like cvs, svn, pvcs (pcli), and now clearcase. > > I am also doing a lot now with clearquest hooks (written in perl), and > clearquest applications. > > I have written a lot of perl/Tk to support a lot of the command line stuff > that I have written. > > Anyway I am having a blast. > I would like to learn a lot more about ruby though, but I don't have the > time! ;-) > > Anyway my post was not meant as a flame. Just a query. > I have just moved down to the valley (11/26) to be exact. And I am having a > blast here too. > > I would be happy to share with y'all anything that I know. > > Happy Trails, > > Jerry > > > > > --Brock > > > > On 2006.03.05.10.22, Jerry Davis wrote: > > | I see a lot of cgi and web stuff on this list. > > | Does anyone use perl for anything other than web? > > | > > | I have used perl daily for over 7 years, and only done one cgi app. > > | The rest have been either stand-alone apps, or glue. > > | > > | Just asking :) > > | > > | Jerry > > | -- > > | Hobbit Name: Pimpernel Loamsdown > > | Registered Linux User: 275424 > > | > > | This email's random fortune: Kill Ugly Processor Architectures > > | - Karl Lehenbauer > > | _______________________________________________ > > | Phoenix-pm mailing list > > | Phoenix-pm at pm.org > > | http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- > Hobbit Name: Pimpernel Loamsdown > Registered Linux User: 275424 > > This email's random fortune: If Microsoft built cars, the oil, gas, and > alternator > warning lights would > be replaced by a single "general car fault" warning light. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From awwaiid at thelackthereof.org Sun Mar 5 14:03:40 2006 From: awwaiid at thelackthereof.org (Brock) Date: Sun, 5 Mar 2006 15:03:40 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <440B2A3E.8090807@qwest.net> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <20060305014148.GL22790@illogics.org> <20060305173831.GF29330@thelackthereof.org> <200603051106.56887.jdawgaz@cox.net> <440B2A3E.8090807@qwest.net> Message-ID: <20060305220340.GG29330@thelackthereof.org> Gold Bar is a great place too, I regularly rotate between Mill's End and GoldBar. In this case, however, I keep my vote on Mill's End because I think they will have more space for us, and their internet is a little better. --Brock On 2006.03.05.11.13, Anthony R. Nemmer wrote: | Gold Bar Coffee House - 3141 S McClintock Dr - (480) 839-3082 (Southern | and Mclintock) | | The seating inside is limited but they have good outdoor seating and | plenty of parking. I am thinking about having some of the Tempe/East | Valley Perlmonger's meetings there. | | Here's a list of places that has free wireless: | | http://www.wififreespot.com/ar.html | | Tony | | Jerry Davis wrote: | > On Sunday 05 March 2006 10:38 am, Brock wrote: | >> I put forward Mill's End (coffee shop) in Tempe. Free wireless, near ASU, | >> nice couches. I can try to get there early (and coordinate with the | >> owners) to corner off a nice section for us. | > | > do they serve other stuff besides coffee? like maybe chai tea or other kinds | > of tea? | > | >> Objections and alternatives welcome! | >> | >> --Brock | >> | >> On 2006.03.05.01.41, Scott Walters wrote: | >> | Hi Happy Perl People, | >> | | >> | Someone please suggest a venue. Normally I would, but I don't like any | >> | of the Scottsdale coffee shops. Brock has something in mind; I know | >> | where pretty much all of the Starbucks are, and, as usual in cultrally | >> | devoid office and housing sprawls, they're running circles around the | >> | local fare. | >> | | >> | -scott | >> | >> _______________________________________________ | >> Phoenix-pm mailing list | >> Phoenix-pm at pm.org | >> http://mail.pm.org/mailman/listinfo/phoenix-pm | > | | | -- | | I always have coffee when I watch radar! | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From awwaiid at thelackthereof.org Sun Mar 5 14:05:06 2006 From: awwaiid at thelackthereof.org (Brock) Date: Sun, 5 Mar 2006 15:05:06 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th In-Reply-To: <200603051106.56887.jdawgaz@cox.net> References: <7CF7A055-A062-409D-9EED-87A9580D3CB2@garfias.org> <20060305014148.GL22790@illogics.org> <20060305173831.GF29330@thelackthereof.org> <200603051106.56887.jdawgaz@cox.net> Message-ID: <20060305220506.GH29330@thelackthereof.org> Yes, they have Chai and several other sorts of tea, and some random juices. They also have food. I like their panini, and they specialize in some strange Crepes. I've tries the crepes but haven't cared for any of them so far. --Brock On 2006.03.05.11.06, Jerry Davis wrote: | On Sunday 05 March 2006 10:38 am, Brock wrote: | > I put forward Mill's End (coffee shop) in Tempe. Free wireless, near ASU, | > nice couches. I can try to get there early (and coordinate with the | > owners) to corner off a nice section for us. | | do they serve other stuff besides coffee? like maybe chai tea or other kinds | of tea? | | > | > Objections and alternatives welcome! | > | > --Brock | > | > On 2006.03.05.01.41, Scott Walters wrote: | > | Hi Happy Perl People, | > | | > | Someone please suggest a venue. Normally I would, but I don't like any | > | of the Scottsdale coffee shops. Brock has something in mind; I know | > | where pretty much all of the Starbucks are, and, as usual in cultrally | > | devoid office and housing sprawls, they're running circles around the | > | local fare. | > | | > | -scott | > | > _______________________________________________ | > Phoenix-pm mailing list | > Phoenix-pm at pm.org | > http://mail.pm.org/mailman/listinfo/phoenix-pm | | -- | Hobbit Name: Pimpernel Loamsdown | Registered Linux User: 275424 | | This email's random fortune: The cost of feathers has risen, even down is up! From noyler at khimetrics.com Mon Mar 6 12:55:34 2006 From: noyler at khimetrics.com (Nathan Oyler) Date: Mon, 6 Mar 2006 13:55:34 -0700 Subject: [Phoenix-pm] Call for Venue, Meeting on March 15th Message-ID: <59B15593F41BD24591D59436E7226EAD03610E6B@Khiphx2.khimetrics.com> Mills end sounds great. > -----Original Message----- > From: phoenix-pm-bounces+noyler=khimetrics.com at pm.org [mailto:phoenix-pm- > bounces+noyler=khimetrics.com at pm.org] On Behalf Of Brock > Sent: Sunday, March 05, 2006 3:05 PM > To: Jerry Davis > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Call for Venue, Meeting on March 15th > > Yes, they have Chai and several other sorts of tea, and some random > juices. They also have food. I like their panini, and they specialize in > some strange Crepes. I've tries the crepes but haven't cared for any of > them so far. > > --Brock > > On 2006.03.05.11.06, Jerry Davis wrote: > | On Sunday 05 March 2006 10:38 am, Brock wrote: > | > I put forward Mill's End (coffee shop) in Tempe. Free wireless, near > ASU, > | > nice couches. I can try to get there early (and coordinate with the > | > owners) to corner off a nice section for us. > | > | do they serve other stuff besides coffee? like maybe chai tea or other > kinds > | of tea? > | > | > > | > Objections and alternatives welcome! > | > > | > --Brock > | > > | > On 2006.03.05.01.41, Scott Walters wrote: > | > | Hi Happy Perl People, > | > | > | > | Someone please suggest a venue. Normally I would, but I don't like > any > | > | of the Scottsdale coffee shops. Brock has something in mind; I know > | > | where pretty much all of the Starbucks are, and, as usual in > cultrally > | > | devoid office and housing sprawls, they're running circles around > the > | > | local fare. > | > | > | > | -scott > | > > | > _______________________________________________ > | > Phoenix-pm mailing list > | > Phoenix-pm at pm.org > | > http://mail.pm.org/mailman/listinfo/phoenix-pm > | > | -- > | Hobbit Name: Pimpernel Loamsdown > | Registered Linux User: 275424 > | > | This email's random fortune: The cost of feathers has risen, even down > is up! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Thu Mar 9 07:25:40 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 9 Mar 2006 15:25:40 +0000 Subject: [Phoenix-pm] From P5P, mostly for Brock: Coro(utine) support from the perl core Message-ID: <20060309152540.GO22790@illogics.org> ----- Forwarded message from Marc Lehmann ----- Return-Path: perl5-porters-return-110505-scott=slowass.net at perl.org X-Original-To: scott at slowass.net Delivered-To: scott at slowass.net Received: from lists.develooper.com (x6.develooper.com [63.251.223.186]) by slowass.net (Postfix) with SMTP id C7885553AA for ; Thu, 9 Mar 2006 08:35:16 +0000 (GMT) Received: (qmail 5463 invoked by uid 514); 9 Mar 2006 08:28:16 -0000 Mailing-List: contact perl5-porters-help at perl.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: X-List-Archive: List-Id: Delivered-To: mailing list perl5-porters at perl.org Delivered-To: moderator for perl5-porters at perl.org Received: (qmail 24465 invoked from network); 9 Mar 2006 01:07:36 -0000 Delivered-To: perl5-porters at perl.org X-Spam-Status: No, hits=-2.6 required=8.0 tests=BAYES_00,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: la.mx.develooper.com Received-SPF: pass (x1.develooper.com: domain of root at schmorp.de designates 193.108.181.162 as permitted sender) Date: Thu, 9 Mar 2006 02:07:16 +0100 From: Marc Lehmann To: perl5-porters at perl.org Subject: Coro(utine) support from the perl core Message-ID: <20060309010716.GA13267 at schmorp.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-PGP: "1024D/DA743396 1999-01-26 Marc Alexander Lehmann Key fingerprint = 475A FE9B D1D4 039E 01AC C217 A1E8 0270 DA74 3396" [If possible, please send a courtesy CC directly to me when replying] Hi! On the german perl workshop, I shortly discussed with Nicholas Clark what kind of support the Coro module would need from the perl core to work better. While I admit that I wasn't active on p5p for a long time, nor am I even using 5.9, I think a few things would indeed be helpful. However, the Coro module works surprisingly well (with minimal changes between 5.6 and 5.8 for example, only 5.9 might make some problems). In any case, here is a list of things that I do that might benefit from better core support. If you want to know what I am talking about, all relevant code is in Coro-xxx/Coro/State.xs. - the load_state/save_state functions are the core of the module, they simply save and restore most perl "globals". From time to time, another global that I forgot comes up, so it might help to know which globals should be saved/restored and which don't need to, or shouldn't. It works well enough at the moment, but some standard mechanism for swapping the state might help. - the Coro module has to walk up the whole call chain to "undo" function calls, mostly due to the padlists. I keep an array of "fresh" padlists attached to each CV using magic. Here's the creepy fact: clone_padlist is used to create a new "initial" padlist. I think the "fake threads", whatever they are supposed to be, also need a similar but not identical function, and the code is copied from a function within perl (S_cv_clone2) almost verbatim, except for the case thats been commented out with ENOTUNDERSTOOD, because I have no idea what it does, or what its for. And I have stripped it down considerably. What would help would be a core-supported clone_padlist function, or similar functionality, so I wouldn't have to mess with CV internals that much. Also, it would be nice if this padlist cloning wouldn't be necessary, and/or the call walk wouldn't be neecssary, but most call chains are shallow, so this isn't a big performance problem in practise. - The callchain walking works by walking up the cxstacks. I am not sure when multiple cx stacks are in use, or wether I have to walk all of them or ... I simply have no idea. I also ignore CXt_FORMAT. Never saw them never used them. Probably its ok to ignore them anyways. Some comment on that would be helpful :) - When creating a new coroutine, I have to initialise the stacks, which is done with coro_init_stacks. This is another almost verbatim copy of the similar fucntion inside the perl core (Perl_init_stacks), except that it allocates much less intiial stack space, in the assumption that most coroutines will not use that much perl stack. It would be nice if there were a function in the perl core that I could call instead. - Similar "destroy_stacks", which I need when destroying a subroutine. The code is rather weird: I LEAVE_SCOPE(0), FREETMPS, and when I POPSTACK_TO(PL_mainstack), I get weird errors and segfaults, so I don't do the latter. Again, I don't understand anything about what I am doing there, and having this correctly done inside the perl core would help again. - in "continue_coro", I basically create a simple perl interpreter. I am not sure wether I set up the PL_top_env correctly. Just in case, all new coroutines have a top-level eval {} that catches runtime errors. Thats all I could come up with with potential issues where the perl core could help. Any feedback would be appreciated. -- The choice of a -----==- _GNU_ ----==-- _ generation Marc Lehmann ---==---(_)__ __ ____ __ pcg at goof.com --==---/ / _ \/ // /\ \/ / http://schmorp.de/ -=====/_/_//_/\_,_/ /_/\_\ XX11-RIPE ----- End forwarded message ----- From scott at illogics.org Thu Mar 9 07:27:47 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 9 Mar 2006 15:27:47 +0000 Subject: [Phoenix-pm] More: Coro(utine) support from the perl core Message-ID: <20060309152747.GP22790@illogics.org> ----- Forwarded message from Marc Lehmann ----- Return-Path: perl5-porters-return-110516-scott=slowass.net at perl.org X-Original-To: scott at slowass.net Delivered-To: scott at slowass.net Received: from lists.develooper.com (x6.develooper.com [63.251.223.186]) by slowass.net (Postfix) with SMTP id DEC76553AA for ; Thu, 9 Mar 2006 14:52:31 +0000 (GMT) Received: (qmail 28123 invoked by uid 514); 9 Mar 2006 14:45:29 -0000 Mailing-List: contact perl5-porters-help at perl.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: X-List-Archive: List-Id: Delivered-To: mailing list perl5-porters at perl.org Received: (qmail 28111 invoked from network); 9 Mar 2006 14:45:29 -0000 Delivered-To: perl5-porters at perl.org X-Spam-Status: No, hits=-2.6 required=8.0 tests=BAYES_00,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: la.mx.develooper.com Received-SPF: pass (x1.develooper.com: domain of root at schmorp.de designates 193.108.181.162 as permitted sender) Date: Thu, 9 Mar 2006 15:45:12 +0100 From: Marc Lehmann To: perl5-porters at perl.org Subject: Re: Coro(utine) support from the perl core Message-ID: <20060309144512.GA22421 at schmorp.de> References: <20060309010716.GA13267 at schmorp.de> <20060309083910.GI6272 at woobling.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060309083910.GI6272 at woobling.org> X-PGP: "1024D/DA743396 1999-01-26 Marc Alexander Lehmann Key fingerprint = 475A FE9B D1D4 039E 01AC C217 A1E8 0270 DA74 3396" On Thu, Mar 09, 2006 at 10:39:10AM +0200, Yuval Kogman wrote: > Please give some thought into making these serializable... I'm not You can't, in principle, serialise those, unless you are in the same process and keep the state, and even then, its only possible when you ignore scoping issues. Agni, for example, already supports continuations for use in a webserver for a long time: slink "click me", {: die "You followed the link!\n" :}(); It does so by compiling all relevant code in all web server processes and binding by md5(sourcecode). This doesn't always work (e.g. when the code has been changed), of course, and it isn't properly scoped: my $var; slink "click me", {: $var++ :}(); Should work, but cannot, as there is no way to associate the $var inside the {: :} with the outside $var. This happens with simple code references already. With coroutines, it becomes even more problematic. The problem is simply unsolvable in _principle_, unless you throw away some features, such as closures and the ability to load references into a different process that saved them (which would make them rather pointless, but would certainly be implementable in some or another form, but would amount to simple storing of the reference for later use and serialising a reference "ID"). The simple case can already be done using e.g. Storable. -- The choice of a -----==- _GNU_ ----==-- _ generation Marc Lehmann ---==---(_)__ __ ____ __ pcg at goof.com --==---/ / _ \/ // /\ \/ / http://schmorp.de/ -=====/_/_//_/\_,_/ /_/\_\ XX11-RIPE ----- End forwarded message ----- From awwaiid at thelackthereof.org Thu Mar 9 09:47:24 2006 From: awwaiid at thelackthereof.org (Brock) Date: Thu, 9 Mar 2006 10:47:24 -0700 Subject: [Phoenix-pm] meeting change Message-ID: <20060309174724.GU29330@thelackthereof.org> Actually... sorry for not having earlier notice but I have to change the meeting from Wednesday to Thursday. If there are any objections let them be heard. Otherwise, here is the announcement: Time: Thursday 16 March 2006 @7:00pm Location: Mill's End http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az 310 S Mill Ave # A101 Tempe, AZ 85281 Thats the North-West corner of Mill and 3rd (north of University) in down-town Tempe. Topic: Search Engine (scott) Catalyst (Michael Garfias, unconfirmed) As always, random discussion Other: Wireless internet available. Bring your laptops. Mill's End sells drinks and food Presentations will be given and recorded over VNC --Brock From scott at illogics.org Thu Mar 9 11:15:53 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 9 Mar 2006 19:15:53 +0000 Subject: [Phoenix-pm] Semi-infinite strings and Patricia trees In-Reply-To: <20060309174724.GU29330@thelackthereof.org> References: <20060309174724.GU29330@thelackthereof.org> Message-ID: <20060309191552.GU22790@illogics.org> Hi all, Please allow me to elaborate. I'd like to present on semi-infinite strings and patricia trees, which is the basic technology that Google uses that enables them to search for numeric ranges, such as 1000..2000, exact quotes, and do lots of other neat things. I won't talk about stemming, scoring, weighting, crawling, or lots of other things, at least not much, and not without request -- the presentation is concerned with this algorithm set, storage method, and it's data structures. Hope to see you there! -scott PS: updated the Wiki. I'd like to present on semi-infinite On 0, Brock wrote: > Actually... sorry for not having earlier notice but I have to change the > meeting from Wednesday to Thursday. If there are any objections let them > be heard. Otherwise, here is the announcement: > > Time: Thursday 16 March 2006 @7:00pm > Location: Mill's End > http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az > 310 S Mill Ave # A101 > Tempe, AZ 85281 > Thats the North-West corner of Mill and 3rd (north of > University) in down-town Tempe. > Topic: Search Engine (scott) > Catalyst (Michael Garfias, unconfirmed) > As always, random discussion > Other: Wireless internet available. Bring your laptops. > Mill's End sells drinks and food > Presentations will be given and recorded over VNC > > --Brock > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From derek at ninth.org Thu Mar 9 11:14:48 2006 From: derek at ninth.org (Derek Cline) Date: Thu, 9 Mar 2006 12:14:48 -0700 Subject: [Phoenix-pm] meeting change In-Reply-To: <20060309174724.GU29330@thelackthereof.org> References: <20060309174724.GU29330@thelackthereof.org> Message-ID: <87A01805-ACC6-4529-9946-2283E7FDB8F9@ninth.org> Hey all, I finally added myself to the list, I'm lazy like that. Unfortunately, I can't make this meeting, sound interesting though. I'll look out for the next one. Thanks, -=Derek On Mar 9, 2006, at 10:47 AM, Brock wrote: > Actually... sorry for not having earlier notice but I have to > change the > meeting from Wednesday to Thursday. If there are any objections let > them > be heard. Otherwise, here is the announcement: From scott at illogics.org Thu Mar 9 11:23:53 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 9 Mar 2006 19:23:53 +0000 Subject: [Phoenix-pm] More: Re: Coro(utine) support from the perl core Message-ID: <20060309192353.GV22790@illogics.org> ----- Forwarded message from demerphq ----- Return-Path: perl5-porters-return-110528-scott=slowass.net at perl.org X-Original-To: scott at slowass.net Delivered-To: scott at slowass.net Received: from lists.develooper.com (x6.develooper.com [63.251.223.186]) by slowass.net (Postfix) with SMTP id 8506D553AA for ; Thu, 9 Mar 2006 19:13:34 +0000 (GMT) Received: (qmail 2065 invoked by uid 514); 9 Mar 2006 19:06:35 -0000 Mailing-List: contact perl5-porters-help at perl.org; run by ezmlm Precedence: bulk list-help: list-unsubscribe: list-post: X-List-Archive: List-Id: Delivered-To: mailing list perl5-porters at perl.org Received: (qmail 2053 invoked from network); 9 Mar 2006 19:06:35 -0000 Delivered-To: perl5-porters at perl.org X-Spam-Status: No, hits=-2.6 required=8.0 tests=BAYES_00,SPF_PASS X-Spam-Check-By: la.mx.develooper.com Received-SPF: pass (x1.develooper.com: domain of demerphq at gmail.com designates 64.233.184.197 as permitted sender) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=mSf0l1CU8yuZtI3Iguyg6qDlwNAa+z+BJDjLlRnm43/mDY98KHJCIf0PeG/9rNAbPFgRBCHxEVHFsG2cQdcn2lniBQLIpnjmvhqFwDGbZlmCRjLrB1ZJ7niZ8SBr6XgXrIBMLsYw/udMJrEtkRUjsFyyW7nLFc1rcPILaEsDmtI= Message-ID: <9b18b3110603091106p4f50b364sa1b567269e6b1480 at mail.gmail.com> Date: Thu, 9 Mar 2006 20:06:24 +0100 From: demerphq To: Marc Lehmann Subject: Re: Coro(utine) support from the perl core Cc: perl5-porters at perl.org In-Reply-To: <20060309174422.GE22421 at schmorp.de> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <20060309010716.GA13267 at schmorp.de> <20060309083910.GI6272 at woobling.org> <20060309144512.GA22421 at schmorp.de> <9b18b3110603090648i69c6b880o7a4fb30e4433cb9c at mail.gmail.com> <20060309174422.GE22421 at schmorp.de> On 3/9/06, Marc Lehmann wrote: > On Thu, Mar 09, 2006 at 03:48:09PM +0100, demerphq wrote: > > > The simple case can already be done using e.g. Storable. > > > > You can use Data::Dump::Streamer to serialize closures... Although im > > not certain its relevent to this conversation. > > No, you can't, here is an example: > > ... > my $var; > my $closure = sub { $var++ }; > dumpevalandcall $closure; > print $var; > > sub dumpevalandcall { > my $source = Data::Dumper::Dumper $_[0]; > (eval $source)->(); > } > > There is no way to serialise the closure so that the deserialised version > still refers to $var. Well, I think it depends on what you mean. Terms are slippery in this part of the world. Consider the following code: use Data::Dump::Streamer; sub make_subs { my $var=shift; my $inc = sub { $var++ }; my $dec = sub { $var-- }; my $get = sub { \$var }; return ($inc,$dec,$get) } my ($i,$d,$g)=make_subs(123); Dump(${$g->()},$i,$d,$g)->Out(); __END__ $var = 123; $CODE1 = sub { $var++; }; $CODE2 = sub { $var--; }; $CODE3 = sub { \$var; }; As you can see, the output the subs bound to the same var. So in one sense your statement is wrong. Using DDS you can serialize a set of closures and other data elements such that when eval'ed the result has shared data as is appropriate. However, you are correct in that DDS wont help you reattach to the same $var as was used in the first place, but thats a little different a problem I think, and with a bit of work I suspect its not a showstopper. > Now consider killing the process and restarting it between dumping and > eval'ing: which $var does it refer to? Well, actually, I think if done carefully the recreated closures would reference the correct var. I think the attached example suggests ways this could be accomplished. Im still not saying that coroutines are any easier with this stuff, just trying to make it clear that with Data::Dump::Streamer you _can_ serialize closures properly for at least one IMO reasonable definition of properly. yves ps: Im not sure why you used an example that used Data::Dumper to say that Data::Dump::Streamer can't do something.... -- perl -Mre=debug -e "/just|another|perl|hacker/" ----- End forwarded message ----- From scott at illogics.org Fri Mar 10 09:05:15 2006 From: scott at illogics.org (Scott Walters) Date: Fri, 10 Mar 2006 17:05:15 +0000 Subject: [Phoenix-pm] meeting change In-Reply-To: <87A01805-ACC6-4529-9946-2283E7FDB8F9@ninth.org> References: <20060309174724.GU29330@thelackthereof.org> <87A01805-ACC6-4529-9946-2283E7FDB8F9@ninth.org> Message-ID: <20060310170515.GA22790@illogics.org> Derek, Welcome to the list! -scott On 0, Derek Cline wrote: > Hey all, > > I finally added myself to the list, I'm lazy like that. > Unfortunately, I can't make this meeting, sound interesting though. > I'll look out for the next one. > > Thanks, > -=Derek > > > On Mar 9, 2006, at 10:47 AM, Brock wrote: > > Actually... sorry for not having earlier notice but I have to > > change the > > meeting from Wednesday to Thursday. If there are any objections let > > them > > be heard. Otherwise, here is the announcement: > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Sun Mar 12 00:36:34 2006 From: scott at illogics.org (Scott Walters) Date: Sun, 12 Mar 2006 08:36:34 +0000 Subject: [Phoenix-pm] given/when in Perl 5 Message-ID: <20060312083634.GI22790@illogics.org> Hi happy Perl people, Here's a cut and paste from the perldoc perlsyn from 5.9.3: use feature ":5.10"; given($foo) { when (undef) { say '$foo is undefined'; } when ("foo") { say '$foo is the string "foo"'; } when ([1,3,5,7,9]) { say '$foo is an odd digit'; continue; # Fall through } when ($_ < 100) { say '$foo is numerically less than 100'; } when (\&complicated_check) { say 'complicated_check($foo) is true'; } default { die q(I don't know what to do with $foo); } } Muahahahahah! -scott From intertwingled at qwest.net Sun Mar 12 00:40:37 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Sun, 12 Mar 2006 01:40:37 -0700 Subject: [Phoenix-pm] Pablo Velasquez Message-ID: <4413DE85.7060703@qwest.net> If Pablo Velasquez is on this list will he please email me? Thanks, Tony -- I always have coffee when I watch radar! From friedman at highwire.stanford.edu Sun Mar 12 17:05:00 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Sun, 12 Mar 2006 17:05:00 -0800 Subject: [Phoenix-pm] given/when in Perl 5 In-Reply-To: <20060312083634.GI22790@illogics.org> References: <20060312083634.GI22790@illogics.org> Message-ID: <65820426-3717-4C0A-B33C-2A25FE14C9A8@highwire.stanford.edu> I thought "Perl doesn't need a case statement"? Or is this Perl 6 getting implemented in Perl 5 just because they can... -- Mike On Mar 12, 2006, at 12:36 AM, Scott Walters wrote: > Hi happy Perl people, > > Here's a cut and paste from the perldoc perlsyn from 5.9.3: > > use feature ":5.10"; > given($foo) { > when (undef) { > say '$foo is undefined'; > } > > when ("foo") { > say '$foo is the string "foo"'; > } > > when ([1,3,5,7,9]) { > say '$foo is an odd digit'; > continue; # Fall through > } > > when ($_ < 100) { > say '$foo is numerically less than 100'; > } > > when (\&complicated_check) { > say 'complicated_check($foo) is true'; > } > > default { > die q(I don't know what to do with $foo); > } > } > > Muahahahahah! > > -scott > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From scott at illogics.org Sun Mar 12 21:59:28 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 05:59:28 +0000 Subject: [Phoenix-pm] given/when in Perl 5 In-Reply-To: <65820426-3717-4C0A-B33C-2A25FE14C9A8@highwire.stanford.edu> References: <20060312083634.GI22790@illogics.org> <65820426-3717-4C0A-B33C-2A25FE14C9A8@highwire.stanford.edu> Message-ID: <20060313055928.GS22790@illogics.org> Hey Michael, Perl "doesn't need" a lot of things... named parameters, strong type checking, etc... of course, you don't *need* anything more than a some toggle switches to toggle in machine code... but certain things at certain times are nice to have. In this case, switch (no pun intended) has been in the todo since the early days (reportedly), but there were too many scenarios to consider so it never got done. Damian Conway made a pass at the problem with Switch.pm, this got adopted in a modified form into Perl 6's design documents, and then that official-making-ness was enough to get it cleared for Perl 5. Funny how these things work... Cheers! Good to see you around on the list by the way =) -scott On 0, Michael Friedman wrote: > I thought "Perl doesn't need a case statement"? > Or is this Perl 6 getting implemented in Perl 5 just because they can... > > -- Mike > > On Mar 12, 2006, at 12:36 AM, Scott Walters wrote: > > > Hi happy Perl people, > > > > Here's a cut and paste from the perldoc perlsyn from 5.9.3: > > > > use feature ":5.10"; > > given($foo) { > > when (undef) { > > say '$foo is undefined'; > > } > > > > when ("foo") { > > say '$foo is the string "foo"'; > > } > > > > when ([1,3,5,7,9]) { > > say '$foo is an odd digit'; > > continue; # Fall through > > } > > > > when ($_ < 100) { > > say '$foo is numerically less than 100'; > > } > > > > when (\&complicated_check) { > > say 'complicated_check($foo) is true'; > > } > > > > default { > > die q(I don't know what to do with $foo); > > } > > } > > > > Muahahahahah! > > > > -scott > > > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > --------------------------------------------------------------------- > Michael Friedman HighWire Press > Phone: 650-725-1974 Stanford University > FAX: 270-721-8034 > --------------------------------------------------------------------- > > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From bwmetz at att.com Mon Mar 13 07:48:36 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Mon, 13 Mar 2006 09:48:36 -0600 Subject: [Phoenix-pm] given/when in Perl 5 In-Reply-To: <20060313055928.GS22790@illogics.org> Message-ID: <01D5341D04A2E64AB9B34576904733670172813A@OCCLUST01EVS1.ugd.att.com> Aside from the fact that given/when is easier to read, why those names instead of case? If I were a "case language" guy moving to Perl and looking for a case equivalent, I wouldn't think to search for those words in my handy-dandy Perl book index. I'll just assume it has a good cross-index I suppose. B -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Scott Walters Sent: Sunday, March 12, 2006 10:59 PM To: Michael Friedman Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] given/when in Perl 5 Hey Michael, Perl "doesn't need" a lot of things... named parameters, strong type checking, etc... of course, you don't *need* anything more than a some toggle switches to toggle in machine code... but certain things at certain times are nice to have. In this case, switch (no pun intended) has been in the todo since the early days (reportedly), but there were too many scenarios to consider so it never got done. Damian Conway made a pass at the problem with Switch.pm, this got adopted in a modified form into Perl 6's design documents, and then that official-making-ness was enough to get it cleared for Perl 5. Funny how these things work... Cheers! Good to see you around on the list by the way =) -scott On 0, Michael Friedman wrote: > I thought "Perl doesn't need a case statement"? > Or is this Perl 6 getting implemented in Perl 5 just because they can... > > -- Mike > > On Mar 12, 2006, at 12:36 AM, Scott Walters wrote: > > > Hi happy Perl people, > > > > Here's a cut and paste from the perldoc perlsyn from 5.9.3: > > > > use feature ":5.10"; > > given($foo) { > > when (undef) { > > say '$foo is undefined'; > > } > > > > when ("foo") { > > say '$foo is the string "foo"'; > > } > > > > when ([1,3,5,7,9]) { > > say '$foo is an odd digit'; > > continue; # Fall through > > } > > > > when ($_ < 100) { > > say '$foo is numerically less than 100'; > > } > > > > when (\&complicated_check) { > > say 'complicated_check($foo) is true'; > > } > > > > default { > > die q(I don't know what to do with $foo); > > } > > } > > > > Muahahahahah! > > > > -scott > > > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > --------------------------------------------------------------------- > Michael Friedman HighWire Press > Phone: 650-725-1974 Stanford University > FAX: 270-721-8034 > --------------------------------------------------------------------- > > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Mon Mar 13 08:10:48 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 16:10:48 +0000 Subject: [Phoenix-pm] given/when in Perl 5 In-Reply-To: <01D5341D04A2E64AB9B34576904733670172813A@OCCLUST01EVS1.ugd.att.com> References: <20060313055928.GS22790@illogics.org> <01D5341D04A2E64AB9B34576904733670172813A@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060313161048.GW22790@illogics.org> Hey Bobby, Yeah, this was the subject of some debate. Larry felt that given/when are adequately different from switch/case that they should have different names to avoid confusion. And that's the flip-side of the newbie coin -- a large percentage of novice questions in the IRC help channels stem from expecting one thing to work exactly like another, and novices are generally extremely persistant in these expectations... they hold on too strongly to the familiar and don't let go. As to why it's different, all of the "smart match" semantics apply in when(), such as array intersection tests. And of course this leaves the door open to have a case/switch that computes tables, just like C's does, and offers a speed advantage for purely numeric comparisons. So many opinions about Perl 6... -scott On 0, "Metz, Bobby W, WCS" wrote: > Aside from the fact that given/when is easier to read, why those names > instead of case? If I were a "case language" guy moving to Perl and > looking for a case equivalent, I wouldn't think to search for those > words in my handy-dandy Perl book index. I'll just assume it has a good > cross-index I suppose. > > B > > -----Original Message----- > From: phoenix-pm-bounces+bwmetz=att.com at pm.org > [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Scott > Walters > Sent: Sunday, March 12, 2006 10:59 PM > To: Michael Friedman > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] given/when in Perl 5 > > > Hey Michael, > > Perl "doesn't need" a lot of things... named parameters, strong type > checking, etc... of course, you don't *need* anything more than a > some toggle switches to toggle in machine code... but certain things > at certain times are nice to have. In this case, switch (no pun > intended) has been in the todo since the early days (reportedly), > but there were too many scenarios to consider so it never got done. > Damian Conway made a pass at the problem with Switch.pm, this got > adopted in a modified form into Perl 6's design documents, and then > that official-making-ness was enough to get it cleared for Perl 5. > Funny how these things work... > > Cheers! Good to see you around on the list by the way =) > > -scott > > On 0, Michael Friedman wrote: > > I thought "Perl doesn't need a case statement"? > > Or is this Perl 6 getting implemented in Perl 5 just because they > can... > > > > -- Mike > > > > On Mar 12, 2006, at 12:36 AM, Scott Walters wrote: > > > > > Hi happy Perl people, > > > > > > Here's a cut and paste from the perldoc perlsyn from 5.9.3: > > > > > > use feature ":5.10"; > > > given($foo) { > > > when (undef) { > > > say '$foo is undefined'; > > > } > > > > > > when ("foo") { > > > say '$foo is the string "foo"'; > > > } > > > > > > when ([1,3,5,7,9]) { > > > say '$foo is an odd digit'; > > > continue; # Fall through > > > } > > > > > > when ($_ < 100) { > > > say '$foo is numerically less than 100'; > > > } > > > > > > when (\&complicated_check) { > > > say 'complicated_check($foo) is true'; > > > } > > > > > > default { > > > die q(I don't know what to do with $foo); > > > } > > > } > > > > > > Muahahahahah! > > > > > > -scott > > > > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > --------------------------------------------------------------------- > > Michael Friedman HighWire Press > > Phone: 650-725-1974 Stanford University > > FAX: 270-721-8034 > > --------------------------------------------------------------------- > > > > > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Mon Mar 13 09:03:42 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 10:03:42 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> Hi, Not sure if this question is appropriate for this group, but here it is. I am using PERL DBI to SELECTing all columns from a table on Neteeza database and am INSERTing into an existing table on Oracle. However, some of the columns are character based so they require ("") quotes around the column data to insert. I need to dynamically built the INSERT values. Is there an easy way to quote the column data for the INSERT statement. Here are some examples what I am trying to do: $d_sth = $d_dbh->prepare("select column_name, data_type from all_tab_columns where table_name like upper('%$d_tblName%') ") || die "$DBI::errstr\n"; $d_sth->execute() || die "$DBI::errstr\n"; while ( $d_arrayref = $d_sth->fetchrow_arrayref() ) { if ( @$d_arrayref[1] =~ /CHAR|DATE/ ) { $columnsToAssign{ @$_arrayref[0] } = "0"; } } $s_sth = $s_dbh->prepare("select * from $s_tblName") || die "$DBI::errstr\n"; $s_sth->execute() || die "$DBI::errstr\n"; while ( $s_hashref = $s_sth->fetchrow_hashref ) { Name Null? Type ----------------------------------------- -------- ---------------------------- DLVRB_GID NOT NULL NUMBER(10) PRC_REL_GID NOT NULL NUMBER(10) PRDCT_GID NOT NULL NUMBER(10) BIL_PYMT_TYP_CDE NOT NULL CHAR(1) PRJTD_TRX_CNT NOT NULL NUMBER(10) The column above (BIL_PYMT_TYP_CDE) require quotes for the INSERT statement. I would like to dynamically create the INSERT statements like so: $columnsToAssign = $columnsToAssign . ", \"" . @$d_arrayref[0] . "\""; } else { $columnsToAssign = $columnsToAssign . ", " . @$d_arrayref[0]; } Is there an easier way? Thanks. Peter Loo This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.pm.org/pipermail/phoenix-pm/attachments/20060313/1d25bd5f/attachment-0001.html From scott at illogics.org Mon Mar 13 09:30:06 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 17:30:06 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> Message-ID: <20060313173005.GZ22790@illogics.org> Hi Peter, Use placeholders: my $question_marks = join ', ', '?' x @column_names; # one ? for each value my $sth = $dbh->prepare(qq{insert into foo (@column_names) values ($question_marks)}) or die $dbh->errstr; while(my @row = $old_database->fetchrow_array) { # read from old database $sth->execute(@row); # insert into new database } If you use placeholders, you never need to quote or escape data, and you don't suffer from SQL command length limitations, and the database doesn't have to parse potentially megs of data just to parse the SQL command, and it doesn't have to parse the SQL more than once at all -- you prepare it once and then execute it as many times as you like for a huge speed increase. If you're loading a lot of data, you'll also want to drop indexes first and the rebuild them later. This is a huge time savings over incrementally updating them for millions of records. I might have missed the point of your question -- if so, please clarify, and I aplogize. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Hi, > > > $columnsToAssign = $columnsToAssign . ", \"" . @$d_arrayref[0] > . "\""; > } > else { > $columnsToAssign = $columnsToAssign . ", " . @$d_arrayref[0]; > } > > Is there an easier way? > > > > Thanks. > > > > Peter Loo > > From awwaiid at thelackthereof.org Mon Mar 13 09:28:05 2006 From: awwaiid at thelackthereof.org (Brock) Date: Mon, 13 Mar 2006 10:28:05 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> Message-ID: <20060313172805.GC4287@thelackthereof.org> Yes. What you need to use is "bind values" as described in perldoc DBI. You can do something like: $sql = "INSERT INTO theTable (DLVRB_GID, PRC_REL_GID, PRDCT_GID, BIL_PYMT_TYP_CDE, PRJTD_TRX_CNT) VALUES (?, ?, ?, ?, ?)"; # Execute the statement, binding the '?' values $d_dbh->do($sql, undef, $dlvrb_gid, $prc_rel_gid, $prdct_gid, $bil_pymt_type_cde, $prjtd_trx_cnt); and can otherwise buld up the statement dynamically too. You can do something similar with SELECT statements, passing the parameters (things to replace the '?' marks) into the $d_sth->execute(...) command. Let us know if you need a more detailed or specific example. --Brock On 2006.03.13.10.03, Loo, Peter # PHX wrote: | Hi, | | Not sure if this question is appropriate for this group, but here it is. | | I am using PERL DBI to SELECTing all columns from a table on Neteeza | database and am INSERTing into an existing table on Oracle. However, some | of the columns are character based so they require ("") quotes around the | column data to insert. I need to dynamically built the INSERT values. Is | there an easy way to quote the column data for the INSERT statement. Here | are some examples what I am trying to do: | | $d_sth = $d_dbh->prepare("select column_name, data_type | from all_tab_columns | where table_name like upper('%$d_tblName%') | ") || die "$DBI::errstr\n"; | | $d_sth->execute() || die "$DBI::errstr\n"; | while ( $d_arrayref = $d_sth->fetchrow_arrayref() ) { | if ( @$d_arrayref[1] =~ /CHAR|DATE/ ) { | $columnsToAssign{ @$_arrayref[0] } = "0"; | } | } | | $s_sth = $s_dbh->prepare("select * from $s_tblName") || die | "$DBI::errstr\n"; | $s_sth->execute() || die "$DBI::errstr\n"; | while ( $s_hashref = $s_sth->fetchrow_hashref ) { | | Name Null? Type | ----------------------------------------- -------- | ---------------------------- | DLVRB_GID NOT NULL NUMBER(10) | PRC_REL_GID NOT NULL NUMBER(10) | PRDCT_GID NOT NULL NUMBER(10) | BIL_PYMT_TYP_CDE NOT NULL CHAR(1) | PRJTD_TRX_CNT NOT NULL NUMBER(10) | | The column above (BIL_PYMT_TYP_CDE) require quotes for the INSERT | statement. I would like to dynamically create the INSERT statements like | so: | | $columnsToAssign = $columnsToAssign . ", \"" . @$d_arrayref[0] . | "\""; | } | else { | $columnsToAssign = $columnsToAssign . ", " . @$d_arrayref[0]; | } | Is there an easier way? | | Thanks. | | Peter Loo | | | | This E-mail message is for the sole use of the intended recipient(s) and | may contain confidential and privileged information. Any unauthorized | review, use, disclosure or distribution is prohibited. If you are not the | intended recipient, please contact the sender by reply E-mail, and destroy | all copies of the original message. | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Mon Mar 13 09:29:27 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 10:29:27 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> Hi Scott, Thanks a bunch. I was reading exactly what you had recommended below when I received your email. Yes, I have a drop indexes statement in the program to do just that before doing the INSERTs. You guys are the BEST. I love PERL. Peter Loo -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Monday, March 13, 2006 10:30 AM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Hi Peter, Use placeholders: my $question_marks = join ', ', '?' x @column_names; # one ? for each value my $sth = $dbh->prepare(qq{insert into foo (@column_names) values ($question_marks)}) or die $dbh->errstr; while(my @row = $old_database->fetchrow_array) { # read from old database $sth->execute(@row); # insert into new database } If you use placeholders, you never need to quote or escape data, and you don't suffer from SQL command length limitations, and the database doesn't have to parse potentially megs of data just to parse the SQL command, and it doesn't have to parse the SQL more than once at all -- you prepare it once and then execute it as many times as you like for a huge speed increase. If you're loading a lot of data, you'll also want to drop indexes first and the rebuild them later. This is a huge time savings over incrementally updating them for millions of records. I might have missed the point of your question -- if so, please clarify, and I aplogize. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Hi, > > > $columnsToAssign = $columnsToAssign . ", \"" . @$d_arrayref[0] > . "\""; > } > else { > $columnsToAssign = $columnsToAssign . ", " . @$d_arrayref[0]; > } > > Is there an easier way? > > > > Thanks. > > > > Peter Loo > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From awwaiid at thelackthereof.org Mon Mar 13 09:32:13 2006 From: awwaiid at thelackthereof.org (Brock) Date: Mon, 13 Mar 2006 10:32:13 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> Message-ID: <20060313173213.GE4287@thelackthereof.org> I love Perl too :) --Brock On 2006.03.13.10.29, Loo, Peter # PHX wrote: | | Hi Scott, | | Thanks a bunch. I was reading exactly what you had recommended below | when I received your email. Yes, I have a drop indexes statement in the | program to do just that before doing the INSERTs. You guys are the | BEST. | | I love PERL. | | Peter Loo From scott at illogics.org Mon Mar 13 09:44:45 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 17:44:45 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> Message-ID: <20060313174445.GC22790@illogics.org> > program to do just that before doing the INSERTs. You guys are the > BEST. Would now be a good time to mention that I'm going to be handing out more free copies of _Perl 6 Now: The Core Ideas Illustrated with Perl 5_ at this Thursday's meeting, assuming the shipment arrives in time, which it should? Cheers, -scott From Peter.Loo at source.wolterskluwer.com Mon Mar 13 09:39:35 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 10:39:35 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B9B@phxmail02.phx.ndchealth.com> Where is the meeting this week? I will plan to attend. It has been rough with two young children at home. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Monday, March 13, 2006 10:45 AM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI > program to do just that before doing the INSERTs. You guys are the > BEST. Would now be a good time to mention that I'm going to be handing out more free copies of _Perl 6 Now: The Core Ideas Illustrated with Perl 5_ at this Thursday's meeting, assuming the shipment arrives in time, which it should? Cheers, -scott This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From awwaiid at thelackthereof.org Mon Mar 13 10:08:06 2006 From: awwaiid at thelackthereof.org (Brock) Date: Mon, 13 Mar 2006 11:08:06 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B9B@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B9B@phxmail02.phx.ndchealth.com> Message-ID: <20060313180806.GF4287@thelackthereof.org> (from http://phoenix.pm.org/wiki/?PerlCalendar) Time: Thursday 16 March 2006 @7:00pm Location: Mill's End http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az 310 S Mill Ave # A101 Tempe, AZ 85281 Thats the North-West corner of Mill and 3rd (north of University) in down-town Tempe. Topic: Search Engine (scott) Catalyst (Michael Garfias, unconfirmed) As always, random discussion Other: Wireless internet available. Bring your laptops. Mill's End sells drinks and food Presentations will be given and recorded over VNC I might put up some more instructions for easy parking at Mill's end. I'll spam the list if I do :) --Brock On 2006.03.13.10.39, Loo, Peter # PHX wrote: | | Where is the meeting this week? I will plan to attend. It has been | rough with two young children at home. | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 | | -----Original Message----- | From: Scott Walters [mailto:scott at illogics.org] | Sent: Monday, March 13, 2006 10:45 AM | To: Loo, Peter # PHX | Cc: phoenix-pm at pm.org | Subject: Re: [Phoenix-pm] PERL DBI | | > program to do just that before doing the INSERTs. You guys are the | > BEST. | | Would now be a good time to mention that I'm going to be handing out | more free copies of _Perl 6 Now: The Core Ideas Illustrated with Perl 5_ | at this Thursday's meeting, assuming the shipment arrives in time, which | it should? | | Cheers, | -scott | | | This E-mail message is for the sole use of the intended recipient(s) and | may contain confidential and privileged information. Any unauthorized | review, use, disclosure or distribution is prohibited. If you are not | the intended recipient, please contact the sender by reply E-mail, and | destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Mon Mar 13 10:21:35 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 18:21:35 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B9B@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B9B@phxmail02.phx.ndchealth.com> Message-ID: <20060313182135.GD22790@illogics.org> Hi Peter, Thurs at 7, I think... details are up on http://phoenix.pm.org. Bring the kids along! >=) Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Where is the meeting this week? I will plan to attend. It has been > rough with two young children at home. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 10:45 AM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > > program to do just that before doing the INSERTs. You guys are the > > BEST. > > Would now be a good time to mention that I'm going to be handing out > more free copies of _Perl 6 Now: The Core Ideas Illustrated with Perl 5_ > at this Thursday's meeting, assuming the shipment arrives in time, which > it should? > > Cheers, > -scott > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From Peter.Loo at source.wolterskluwer.com Mon Mar 13 12:14:24 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 13:14:24 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95495@phxmail02.phx.ndchealth.com> Hi All, Alright, I got it working, but it is running so slow. I am wondering if there is anyway to speed this up? Here is the syntax I am using: $bindVars = join ', ', ('?') x @column_names; $d_sth = $d_dbh->prepare(qq{insert into dssppv.tmp_falcon_projections ($columnNames) values ($bindVars)}) || die "$DBI::errstr\n"; $s_sth = $s_dbh->prepare("select * from dssppv.p_falcon_projections") || die "$DBI::errstr\n"; $s_sth->execute() || die "$DBI::errstr\n"; while ( my @row = $s_sth->fetchrow_array ) { $d_sth->execute(@row); } Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Monday, March 13, 2006 10:30 AM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Hi Peter, Use placeholders: my $question_marks = join ', ', '?' x @column_names; # one ? for each value my $sth = $dbh->prepare(qq{insert into foo (@column_names) values ($question_marks)}) or die $dbh->errstr; while(my @row = $old_database->fetchrow_array) { # read from old database $sth->execute(@row); # insert into new database } If you use placeholders, you never need to quote or escape data, and you don't suffer from SQL command length limitations, and the database doesn't have to parse potentially megs of data just to parse the SQL command, and it doesn't have to parse the SQL more than once at all -- you prepare it once and then execute it as many times as you like for a huge speed increase. If you're loading a lot of data, you'll also want to drop indexes first and the rebuild them later. This is a huge time savings over incrementally updating them for millions of records. I might have missed the point of your question -- if so, please clarify, and I aplogize. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Hi, > > > $columnsToAssign = $columnsToAssign . ", \"" . @$d_arrayref[0] > . "\""; > } > else { > $columnsToAssign = $columnsToAssign . ", " . @$d_arrayref[0]; > } > > Is there an easier way? > > > > Thanks. > > > > Peter Loo > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From scott at illogics.org Mon Mar 13 12:28:37 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 20:28:37 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95495@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95495@phxmail02.phx.ndchealth.com> Message-ID: <20060313202837.GE22790@illogics.org> Hi Peter, Was it running slow before? I can't imagine that this would make it slower... You could get a little parallelism if you used threads and made a Thread::Queue to pass rows from a reader thread to a writer thread... but Perl threads are pretty grumpy. Just a thought. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Hi All, > > Alright, I got it working, but it is running so slow. I am wondering if > there is anyway to speed this up? Here is the syntax I am using: > > $bindVars = join ', ', ('?') x @column_names; > $d_sth = $d_dbh->prepare(qq{insert into > dssppv.tmp_falcon_projections ($columnNames) > values ($bindVars)}) || die > "$DBI::errstr\n"; > $s_sth = $s_dbh->prepare("select * from > dssppv.p_falcon_projections") || die "$DBI::errstr\n"; > $s_sth->execute() || die "$DBI::errstr\n"; > while ( my @row = $s_sth->fetchrow_array ) { > $d_sth->execute(@row); > } > > > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 10:30 AM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > Hi Peter, > > Use placeholders: > > my $question_marks = join ', ', '?' x @column_names; # one ? for > each value > my $sth = $dbh->prepare(qq{insert into foo (@column_names) values > ($question_marks)}) or die $dbh->errstr; > while(my @row = $old_database->fetchrow_array) { # read from old > database > $sth->execute(@row); # insert into new database > } > > If you use placeholders, you never need to quote or escape data, and you > don't suffer from SQL command length limitations, and the database > doesn't have to parse potentially megs of data just to parse the SQL > command, and it doesn't have to parse the SQL more than once at all -- > you prepare it once and then execute it as many times as you like for a > huge speed increase. If you're loading a lot of data, you'll also want > to drop indexes first and the rebuild them later. This is a huge time > savings over incrementally updating them for millions of records. > > I might have missed the point of your question -- if so, please clarify, > and I aplogize. > > Cheers, > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Hi, > > > > > > $columnsToAssign = $columnsToAssign . ", \"" . > @$d_arrayref[0] > > . "\""; > > } > > else { > > $columnsToAssign = $columnsToAssign . ", " . > @$d_arrayref[0]; > > } > > > > Is there an easier way? > > > > > > > > Thanks. > > > > > > > > Peter Loo > > > > > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From Peter.Loo at source.wolterskluwer.com Mon Mar 13 12:24:47 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 13:24:47 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE954A3@phxmail02.phx.ndchealth.com> Scott, How about if I changed the fetchrow_array to fetchrow_arrayref? I think that will help. I am currently using the same database to test (Oracle production table to an Oracle temp table). I am not using Neteeza (source database) yet. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Monday, March 13, 2006 1:29 PM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Hi Peter, Was it running slow before? I can't imagine that this would make it slower... You could get a little parallelism if you used threads and made a Thread::Queue to pass rows from a reader thread to a writer thread... but Perl threads are pretty grumpy. Just a thought. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Hi All, > > Alright, I got it working, but it is running so slow. I am wondering > if there is anyway to speed this up? Here is the syntax I am using: > > $bindVars = join ', ', ('?') x @column_names; > $d_sth = $d_dbh->prepare(qq{insert into > dssppv.tmp_falcon_projections ($columnNames) > values ($bindVars)}) || die > "$DBI::errstr\n"; > $s_sth = $s_dbh->prepare("select * from > dssppv.p_falcon_projections") || die "$DBI::errstr\n"; > $s_sth->execute() || die "$DBI::errstr\n"; > while ( my @row = $s_sth->fetchrow_array ) { > $d_sth->execute(@row); > } > > > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 10:30 AM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > Hi Peter, > > Use placeholders: > > my $question_marks = join ', ', '?' x @column_names; # one ? for > each value > my $sth = $dbh->prepare(qq{insert into foo (@column_names) values > ($question_marks)}) or die $dbh->errstr; > while(my @row = $old_database->fetchrow_array) { # read from old > database > $sth->execute(@row); # insert into new database > } > > If you use placeholders, you never need to quote or escape data, and > you don't suffer from SQL command length limitations, and the database > doesn't have to parse potentially megs of data just to parse the SQL > command, and it doesn't have to parse the SQL more than once at all -- > you prepare it once and then execute it as many times as you like for > a huge speed increase. If you're loading a lot of data, you'll also > want to drop indexes first and the rebuild them later. This is a huge > time savings over incrementally updating them for millions of records. > > I might have missed the point of your question -- if so, please > clarify, and I aplogize. > > Cheers, > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Hi, > > > > > > $columnsToAssign = $columnsToAssign . ", \"" . > @$d_arrayref[0] > > . "\""; > > } > > else { > > $columnsToAssign = $columnsToAssign . ", " . > @$d_arrayref[0]; > > } > > > > Is there an easier way? > > > > > > > > Thanks. > > > > > > > > Peter Loo > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From friedman at highwire.stanford.edu Mon Mar 13 12:59:39 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Mon, 13 Mar 2006 12:59:39 -0800 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <20060313174445.GC22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> <20060313174445.GC22790@illogics.org> Message-ID: Man, I hate being in California... well, let's just say I hate missing Phoenix.pm meetings. ;-) Have fun, guys! - Mike On Mar 13, 2006, at 9:44 AM, Scott Walters wrote: >> program to do just that before doing the INSERTs. You guys are the >> BEST. > > Would now be a good time to mention that I'm going to be handing > out more > free copies of _Perl 6 Now: The Core Ideas Illustrated with Perl 5_ at > this Thursday's meeting, assuming the shipment arrives in time, > which it > should? > > Cheers, > -scott > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From intertwingled at qwest.net Mon Mar 13 13:00:44 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Mon, 13 Mar 2006 14:00:44 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B89@phxmail02.phx.ndchealth.com> <20060313174445.GC22790@illogics.org> Message-ID: <4415DD7C.3090904@qwest.net> we finally got some RAIN here. Michael Friedman wrote: > Man, I hate being in California... well, let's just say I hate > missing Phoenix.pm meetings. ;-) Have fun, guys! > > - Mike > > On Mar 13, 2006, at 9:44 AM, Scott Walters wrote: > >>> program to do just that before doing the INSERTs. You guys are the >>> BEST. >> Would now be a good time to mention that I'm going to be handing >> out more >> free copies of _Perl 6 Now: The Core Ideas Illustrated with Perl 5_ at >> this Thursday's meeting, assuming the shipment arrives in time, >> which it >> should? >> >> Cheers, >> -scott >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > > --------------------------------------------------------------------- > Michael Friedman HighWire Press > Phone: 650-725-1974 Stanford University > FAX: 270-721-8034 > --------------------------------------------------------------------- > > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! From scott at illogics.org Mon Mar 13 13:14:24 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 21:14:24 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE954A3@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE954A3@phxmail02.phx.ndchealth.com> Message-ID: <20060313211424.GF22790@illogics.org> Peter, The change in speed would be trivial then, I'm guessing, but there's always Devel::Prof if you don't want a guess. You're suffering from round-trip time right now more than throughput, I'm also guessing. -scott On 0, "Loo, Peter # PHX" wrote: > > Scott, > > How about if I changed the fetchrow_array to fetchrow_arrayref? I think > that will help. I am currently using the same database to test (Oracle > production table to an Oracle temp table). I am not using Neteeza > (source database) yet. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 1:29 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > Hi Peter, > > Was it running slow before? I can't imagine that this would make it > slower... > > You could get a little parallelism if you used threads and made a > Thread::Queue to pass rows from a reader thread to a writer thread... > but Perl threads are pretty grumpy. > > Just a thought. > > Cheers, > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Hi All, > > > > Alright, I got it working, but it is running so slow. I am wondering > > if there is anyway to speed this up? Here is the syntax I am using: > > > > $bindVars = join ', ', ('?') x @column_names; > > $d_sth = $d_dbh->prepare(qq{insert into > > dssppv.tmp_falcon_projections ($columnNames) > > values ($bindVars)}) || die > > "$DBI::errstr\n"; > > $s_sth = $s_dbh->prepare("select * from > > dssppv.p_falcon_projections") || die "$DBI::errstr\n"; > > $s_sth->execute() || die "$DBI::errstr\n"; > > while ( my @row = $s_sth->fetchrow_array ) { > > $d_sth->execute(@row); > > } > > > > > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Monday, March 13, 2006 10:30 AM > > To: Loo, Peter # PHX > > Cc: phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] PERL DBI > > > > Hi Peter, > > > > Use placeholders: > > > > my $question_marks = join ', ', '?' x @column_names; # one ? for > > each value > > my $sth = $dbh->prepare(qq{insert into foo (@column_names) values > > ($question_marks)}) or die $dbh->errstr; > > while(my @row = $old_database->fetchrow_array) { # read from old > > database > > $sth->execute(@row); # insert into new database > > } > > > > If you use placeholders, you never need to quote or escape data, and > > you don't suffer from SQL command length limitations, and the database > > > doesn't have to parse potentially megs of data just to parse the SQL > > command, and it doesn't have to parse the SQL more than once at all -- > > > you prepare it once and then execute it as many times as you like for > > a huge speed increase. If you're loading a lot of data, you'll also > > want to drop indexes first and the rebuild them later. This is a huge > > > time savings over incrementally updating them for millions of records. > > > > I might have missed the point of your question -- if so, please > > clarify, and I aplogize. > > > > Cheers, > > -scott > > > > On 0, "Loo, Peter # PHX" wrote: > > > > > > Hi, > > > > > > > > > $columnsToAssign = $columnsToAssign . ", \"" . > > @$d_arrayref[0] > > > . "\""; > > > } > > > else { > > > $columnsToAssign = $columnsToAssign . ", " . > > @$d_arrayref[0]; > > > } > > > > > > Is there an easier way? > > > > > > > > > > > > Thanks. > > > > > > > > > > > > Peter Loo > > > > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From Peter.Loo at source.wolterskluwer.com Mon Mar 13 13:14:25 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 14:14:25 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE9555C@phxmail02.phx.ndchealth.com> Hi Scott, I kicked off the program at 13:05:34 and it is still running. We are only dealing with 2,000,000 rows. This is not going to work. I have to think of something. Dumping the data into and external table in Neteeza on a NFS mount and using SQL_LOAD in Oracle will definitely be faster. This is sad. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Monday, March 13, 2006 2:14 PM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Peter, The change in speed would be trivial then, I'm guessing, but there's always Devel::Prof if you don't want a guess. You're suffering from round-trip time right now more than throughput, I'm also guessing. -scott On 0, "Loo, Peter # PHX" wrote: > > Scott, > > How about if I changed the fetchrow_array to fetchrow_arrayref? I > think that will help. I am currently using the same database to test > (Oracle production table to an Oracle temp table). I am not using > Neteeza (source database) yet. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 1:29 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > Hi Peter, > > Was it running slow before? I can't imagine that this would make it > slower... > > You could get a little parallelism if you used threads and made a > Thread::Queue to pass rows from a reader thread to a writer thread... > but Perl threads are pretty grumpy. > > Just a thought. > > Cheers, > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Hi All, > > > > Alright, I got it working, but it is running so slow. I am > > wondering if there is anyway to speed this up? Here is the syntax I am using: > > > > $bindVars = join ', ', ('?') x @column_names; > > $d_sth = $d_dbh->prepare(qq{insert into > > dssppv.tmp_falcon_projections ($columnNames) > > values ($bindVars)}) || die > > "$DBI::errstr\n"; > > $s_sth = $s_dbh->prepare("select * from > > dssppv.p_falcon_projections") || die "$DBI::errstr\n"; > > $s_sth->execute() || die "$DBI::errstr\n"; > > while ( my @row = $s_sth->fetchrow_array ) { > > $d_sth->execute(@row); > > } > > > > > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Monday, March 13, 2006 10:30 AM > > To: Loo, Peter # PHX > > Cc: phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] PERL DBI > > > > Hi Peter, > > > > Use placeholders: > > > > my $question_marks = join ', ', '?' x @column_names; # one ? > > for each value > > my $sth = $dbh->prepare(qq{insert into foo (@column_names) > > values > > ($question_marks)}) or die $dbh->errstr; > > while(my @row = $old_database->fetchrow_array) { # read from > > old database > > $sth->execute(@row); # insert into new database > > } > > > > If you use placeholders, you never need to quote or escape data, and > > you don't suffer from SQL command length limitations, and the > > database > > > doesn't have to parse potentially megs of data just to parse the SQL > > command, and it doesn't have to parse the SQL more than once at all > > -- > > > you prepare it once and then execute it as many times as you like > > for a huge speed increase. If you're loading a lot of data, you'll > > also want to drop indexes first and the rebuild them later. This is > > a huge > > > time savings over incrementally updating them for millions of records. > > > > I might have missed the point of your question -- if so, please > > clarify, and I aplogize. > > > > Cheers, > > -scott > > > > On 0, "Loo, Peter # PHX" wrote: > > > > > > Hi, > > > > > > > > > $columnsToAssign = $columnsToAssign . ", \"" . > > @$d_arrayref[0] > > > . "\""; > > > } > > > else { > > > $columnsToAssign = $columnsToAssign . ", " . > > @$d_arrayref[0]; > > > } > > > > > > Is there an easier way? > > > > > > > > > > > > Thanks. > > > > > > > > > > > > Peter Loo > > > > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From phx-pm-list at grueslayer.com Mon Mar 13 13:23:27 2006 From: phx-pm-list at grueslayer.com (David A. Sinck) Date: Mon, 13 Mar 2006 14:23:27 -0700 Subject: [Phoenix-pm] PERL DBI References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> Message-ID: <17429.58063.907324.777393@magnitude.grueslayer.com> \_ SMTP quoth Scott Walters on 3/13/2006 17:30 as having spake thusly: \_ \_ If you use placeholders, you never need to quote or escape data, HAHAHAHAHA. Guess again. There's an instance, at least with MySQL dbd that it will occasionally guess wrong on quoting/no quoting and you'll spend hours trying to track it down because it's never burned you before. I've been burned by it about three times now, although only the first time hurt to the point of hours. No, I don't have a small example. Usually it comes up in something like my $href = $sth->fetchrow_hashref; $other_sth->execute($$href{string_val_that_dbd_does_not_quote}); \_ and you don't suffer \_ from SQL command length limitations, I bet I could could craft something that hits a buffer still using placeholders if I wanted. :-) \_ and the database doesn't have to parse potentially \_ megs of data just to parse the SQL command, Depends on the driver. MySQL dbd, last I checked, actually sub'd the values in before it got passed to the engine. YMMV. David From scott at illogics.org Mon Mar 13 13:35:34 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 21:35:34 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE9555C@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE9555C@phxmail02.phx.ndchealth.com> Message-ID: <20060313213534.GH22790@illogics.org> > This is sad. No, this is not sad. Dump/undump programs are hand-tuned C designed to do exactly one thing and do it very quickly. What you're doing would be nearly as slow in any language (or nearly as fast). That aside, if it runs for 24 hours and you have to do it once, what are you crying about? Was the project notice 48 hours to move from one database system to another? Are you doing daily syncs? Go do something else. Presumably the program prints status info. If not, make it, just to detect possible pathological conditions. -scott On 0, "Loo, Peter # PHX" wrote: > > Hi Scott, > > I kicked off the program at 13:05:34 and it is still running. We are > only dealing with 2,000,000 rows. This is not going to work. I have to > think of something. Dumping the data into and external table in Neteeza > on a NFS mount and using SQL_LOAD in Oracle will definitely be faster. > This is sad. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 2:14 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > Peter, > > The change in speed would be trivial then, I'm guessing, but there's > always Devel::Prof if you don't want a guess. > > You're suffering from round-trip time right now more than throughput, > I'm also guessing. > > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Scott, > > > > How about if I changed the fetchrow_array to fetchrow_arrayref? I > > think that will help. I am currently using the same database to test > > (Oracle production table to an Oracle temp table). I am not using > > Neteeza (source database) yet. > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Monday, March 13, 2006 1:29 PM > > To: Loo, Peter # PHX > > Cc: phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] PERL DBI > > > > Hi Peter, > > > > Was it running slow before? I can't imagine that this would make it > > slower... > > > > You could get a little parallelism if you used threads and made a > > Thread::Queue to pass rows from a reader thread to a writer thread... > > but Perl threads are pretty grumpy. > > > > Just a thought. > > > > Cheers, > > -scott > > > > On 0, "Loo, Peter # PHX" wrote: > > > > > > Hi All, > > > > > > Alright, I got it working, but it is running so slow. I am > > > wondering if there is anyway to speed this up? Here is the syntax I > am using: > > > > > > $bindVars = join ', ', ('?') x @column_names; > > > $d_sth = $d_dbh->prepare(qq{insert into > > > dssppv.tmp_falcon_projections ($columnNames) > > > values ($bindVars)}) || die > > > "$DBI::errstr\n"; > > > $s_sth = $s_dbh->prepare("select * from > > > dssppv.p_falcon_projections") || die "$DBI::errstr\n"; > > > $s_sth->execute() || die "$DBI::errstr\n"; > > > while ( my @row = $s_sth->fetchrow_array ) { > > > $d_sth->execute(@row); > > > } > > > > > > > > > > > > Peter Loo > > > Wolters Kluwer Health > > > (602) 381-9553 > > > > > > -----Original Message----- > > > From: Scott Walters [mailto:scott at illogics.org] > > > Sent: Monday, March 13, 2006 10:30 AM > > > To: Loo, Peter # PHX > > > Cc: phoenix-pm at pm.org > > > Subject: Re: [Phoenix-pm] PERL DBI > > > > > > Hi Peter, > > > > > > Use placeholders: > > > > > > my $question_marks = join ', ', '?' x @column_names; # one ? > > > for each value > > > my $sth = $dbh->prepare(qq{insert into foo (@column_names) > > > values > > > ($question_marks)}) or die $dbh->errstr; > > > while(my @row = $old_database->fetchrow_array) { # read from > > > old database > > > $sth->execute(@row); # insert into new database > > > } > > > > > > If you use placeholders, you never need to quote or escape data, and > > > > you don't suffer from SQL command length limitations, and the > > > database > > > > > doesn't have to parse potentially megs of data just to parse the SQL > > > > command, and it doesn't have to parse the SQL more than once at all > > > -- > > > > > you prepare it once and then execute it as many times as you like > > > for a huge speed increase. If you're loading a lot of data, you'll > > > also want to drop indexes first and the rebuild them later. This is > > > > a huge > > > > > time savings over incrementally updating them for millions of > records. > > > > > > I might have missed the point of your question -- if so, please > > > clarify, and I aplogize. > > > > > > Cheers, > > > -scott > > > > > > On 0, "Loo, Peter # PHX" > wrote: > > > > > > > > Hi, > > > > > > > > > > > > $columnsToAssign = $columnsToAssign . ", \"" . > > > @$d_arrayref[0] > > > > . "\""; > > > > } > > > > else { > > > > $columnsToAssign = $columnsToAssign . ", " . > > > @$d_arrayref[0]; > > > > } > > > > > > > > Is there an easier way? > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > Peter Loo > > > > > > > > > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From Peter.Loo at source.wolterskluwer.com Mon Mar 13 13:37:13 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 14:37:13 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> Here is what it took to run: Mon Mar 13 13:05:34 MST 2006 Mon Mar 13 14:31:27 MST 2006 SQL> select count(*) from dssppv.tmp_falcon_projections; COUNT(*) ---------- 2413059 Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Monday, March 13, 2006 2:36 PM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI > This is sad. No, this is not sad. Dump/undump programs are hand-tuned C designed to do exactly one thing and do it very quickly. What you're doing would be nearly as slow in any language (or nearly as fast). That aside, if it runs for 24 hours and you have to do it once, what are you crying about? Was the project notice 48 hours to move from one database system to another? Are you doing daily syncs? Go do something else. Presumably the program prints status info. If not, make it, just to detect possible pathological conditions. -scott On 0, "Loo, Peter # PHX" wrote: > > Hi Scott, > > I kicked off the program at 13:05:34 and it is still running. We are > only dealing with 2,000,000 rows. This is not going to work. I have > to think of something. Dumping the data into and external table in > Neteeza on a NFS mount and using SQL_LOAD in Oracle will definitely be faster. > This is sad. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 2:14 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > Peter, > > The change in speed would be trivial then, I'm guessing, but there's > always Devel::Prof if you don't want a guess. > > You're suffering from round-trip time right now more than throughput, > I'm also guessing. > > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Scott, > > > > How about if I changed the fetchrow_array to fetchrow_arrayref? I > > think that will help. I am currently using the same database to > > test (Oracle production table to an Oracle temp table). I am not > > using Neteeza (source database) yet. > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Monday, March 13, 2006 1:29 PM > > To: Loo, Peter # PHX > > Cc: phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] PERL DBI > > > > Hi Peter, > > > > Was it running slow before? I can't imagine that this would make it > > slower... > > > > You could get a little parallelism if you used threads and made a > > Thread::Queue to pass rows from a reader thread to a writer thread... > > but Perl threads are pretty grumpy. > > > > Just a thought. > > > > Cheers, > > -scott > > > > On 0, "Loo, Peter # PHX" wrote: > > > > > > Hi All, > > > > > > Alright, I got it working, but it is running so slow. I am > > > wondering if there is anyway to speed this up? Here is the syntax > > > I > am using: > > > > > > $bindVars = join ', ', ('?') x @column_names; > > > $d_sth = $d_dbh->prepare(qq{insert into > > > dssppv.tmp_falcon_projections ($columnNames) > > > values ($bindVars)}) || die > > > "$DBI::errstr\n"; > > > $s_sth = $s_dbh->prepare("select * from > > > dssppv.p_falcon_projections") || die "$DBI::errstr\n"; > > > $s_sth->execute() || die "$DBI::errstr\n"; > > > while ( my @row = $s_sth->fetchrow_array ) { > > > $d_sth->execute(@row); > > > } > > > > > > > > > > > > Peter Loo > > > Wolters Kluwer Health > > > (602) 381-9553 > > > > > > -----Original Message----- > > > From: Scott Walters [mailto:scott at illogics.org] > > > Sent: Monday, March 13, 2006 10:30 AM > > > To: Loo, Peter # PHX > > > Cc: phoenix-pm at pm.org > > > Subject: Re: [Phoenix-pm] PERL DBI > > > > > > Hi Peter, > > > > > > Use placeholders: > > > > > > my $question_marks = join ', ', '?' x @column_names; # one ? > > > for each value > > > my $sth = $dbh->prepare(qq{insert into foo (@column_names) > > > values > > > ($question_marks)}) or die $dbh->errstr; > > > while(my @row = $old_database->fetchrow_array) { # read from > > > old database > > > $sth->execute(@row); # insert into new database > > > } > > > > > > If you use placeholders, you never need to quote or escape data, > > > and > > > > you don't suffer from SQL command length limitations, and the > > > database > > > > > doesn't have to parse potentially megs of data just to parse the > > > SQL > > > > command, and it doesn't have to parse the SQL more than once at > > > all > > > -- > > > > > you prepare it once and then execute it as many times as you like > > > for a huge speed increase. If you're loading a lot of data, > > > you'll also want to drop indexes first and the rebuild them later. > > > This is > > > > a huge > > > > > time savings over incrementally updating them for millions of > records. > > > > > > I might have missed the point of your question -- if so, please > > > clarify, and I aplogize. > > > > > > Cheers, > > > -scott > > > > > > On 0, "Loo, Peter # PHX" > wrote: > > > > > > > > Hi, > > > > > > > > > > > > $columnsToAssign = $columnsToAssign . ", \"" . > > > @$d_arrayref[0] > > > > . "\""; > > > > } > > > > else { > > > > $columnsToAssign = $columnsToAssign . ", " . > > > @$d_arrayref[0]; > > > > } > > > > > > > > Is there an easier way? > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > Peter Loo > > > > > > > > > > > > > > > > > This E-mail message is for the sole use of the intended > > > recipient(s) > > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender > > > by reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended > > > recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From scott at illogics.org Mon Mar 13 13:46:18 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 21:46:18 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <17429.58063.907324.777393@magnitude.grueslayer.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> <17429.58063.907324.777393@magnitude.grueslayer.com> Message-ID: <20060313214617.GI22790@illogics.org> On 0, "David A. Sinck" wrote: > > HAHAHAHAHA. Guess again. There's an instance, at least with MySQL Okay, I don't use MySQL heavy (for reasons of it better garbage), so I over generalized. I conceede your point that rare bugs will make this generalization not always true. > \_ and you don't suffer > \_ from SQL command length limitations, > > I bet I could could craft something that hits a buffer still using > placeholders if I wanted. :-) I didn't say he didn't suffer from *buffer* length considerations -- that would be foolish indeed. I only said that he doesn't suffer from this particular one. I've seen databases cap the command buffer at 64k. Generally speaking (true in most cases, perhaps false in some) the length-counted buffers for placeholder data will be significantly larger than the command buffer. > > \_ and the database doesn't have to parse potentially > \_ megs of data just to parse the SQL command, > > Depends on the driver. MySQL dbd, last I checked, actually sub'd the > values in before it got passed to the engine. YMMV. That must have been a long time ago indeed. But you caught me -- I made another false generalization. I should have said "Generally, " in front of it. By the way, people who argue small points while ignoring the point of what was said annoy me. Did you really think someone on the list would be damaged thinking that placeholders always send data out of band, even when using Informix? I wrote a hasty reply out of desire to be helpful to someone who was having trouble with something, not to debate minutia, and have been punished for it. -scott > > David > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Mon Mar 13 13:49:34 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 14:49:34 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE9559A@phxmail02.phx.ndchealth.com> Scott, At any rate, I truly appreciate your comments and advises. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: phoenix-pm-bounces+peter.loo=source.wolterskluwer.com at pm.org [mailto:phoenix-pm-bounces+peter.loo=source.wolterskluwer.com at pm.org] On Behalf Of Scott Walters Sent: Monday, March 13, 2006 2:46 PM To: David A. Sinck Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI On 0, "David A. Sinck" wrote: > > HAHAHAHAHA. Guess again. There's an instance, at least with MySQL Okay, I don't use MySQL heavy (for reasons of it better garbage), so I over generalized. I conceede your point that rare bugs will make this generalization not always true. > \_ and you don't suffer > \_ from SQL command length limitations, > > I bet I could could craft something that hits a buffer still using > placeholders if I wanted. :-) I didn't say he didn't suffer from *buffer* length considerations -- that would be foolish indeed. I only said that he doesn't suffer from this particular one. I've seen databases cap the command buffer at 64k. Generally speaking (true in most cases, perhaps false in some) the length-counted buffers for placeholder data will be significantly larger than the command buffer. > > \_ and the database doesn't have to parse potentially \_ megs of data > just to parse the SQL command, > > Depends on the driver. MySQL dbd, last I checked, actually sub'd the > values in before it got passed to the engine. YMMV. That must have been a long time ago indeed. But you caught me -- I made another false generalization. I should have said "Generally, " in front of it. By the way, people who argue small points while ignoring the point of what was said annoy me. Did you really think someone on the list would be damaged thinking that placeholders always send data out of band, even when using Informix? I wrote a hasty reply out of desire to be helpful to someone who was having trouble with something, not to debate minutia, and have been punished for it. -scott > > David > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From billn at odyssey.billn.net Mon Mar 13 13:47:23 2006 From: billn at odyssey.billn.net (Bill Nash) Date: Mon, 13 Mar 2006 16:47:23 -0500 (EST) Subject: [Phoenix-pm] PERL DBI In-Reply-To: <20060313214617.GI22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> <17429.58063.907324.777393@magnitude.grueslayer.com> <20060313214617.GI22790@illogics.org> Message-ID: You're very likely pressing up against DB execution overhead. For starters, you're doing one insert per row. The database side mechanics of this are: 1. Accept insert. 2. Update indexes. 3. Log query where applicable. 4. Platform specific spooky buffer action and related flushing. Platform independent, if you're going to chuck a couple million rows like this, disable your indexing first. I don't know how you'd optimize this with Oracle, but mysql supports bulk insert syntax, ala: insert into foo (a, b, c, d, beer) values (1, 2, 3, 4, 'bar'), (3, 4, 5, 6, 'heineken'), ...; Even with indexing enabled, this is far more efficient, as indexes are updated en masse after the insert is accepted. In psuedocode, something like: prepare query execute query while iterating results, push result set onto array. increment counter. if counter hits 5000, bulk insert entries stored in array. erase array. reset counter. end if end while I've gotten DBI on good hardware to iterate upwards of 20k rows a second. Realistic insert rates are a different animal, because of the i/o (your mileage may vary depending on media speeds), but 6-8k a second is not unrealistic. Other factors include the engine type, the data i/o bus of the machine, etc. - billn On Mon, 13 Mar 2006, Scott Walters wrote: > On 0, "David A. Sinck" wrote: >> >> HAHAHAHAHA. Guess again. There's an instance, at least with MySQL > > Okay, I don't use MySQL heavy (for reasons of it better garbage), so > I over generalized. I conceede your point that rare bugs will make > this generalization not always true. > >> \_ and you don't suffer >> \_ from SQL command length limitations, >> >> I bet I could could craft something that hits a buffer still using >> placeholders if I wanted. :-) > > I didn't say he didn't suffer from *buffer* length considerations -- that > would be foolish indeed. I only said that he doesn't suffer from this > particular one. I've seen databases cap the command buffer at 64k. > Generally speaking (true in most cases, perhaps false in some) the > length-counted buffers for placeholder data will be significantly larger > than the command buffer. > >> >> \_ and the database doesn't have to parse potentially >> \_ megs of data just to parse the SQL command, >> >> Depends on the driver. MySQL dbd, last I checked, actually sub'd the >> values in before it got passed to the engine. YMMV. > > That must have been a long time ago indeed. But you caught me -- > I made another false generalization. I should have said "Generally, " > in front of it. > > By the way, people who argue small points while ignoring the point of > what was said annoy me. Did you really think someone on the list > would be damaged thinking that placeholders always send data out of band, > even when using Informix? > > I wrote a hasty reply out of desire to be helpful to someone who was > having trouble with something, not to debate minutia, and have been > punished for it. > > -scott > >> >> David >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > From billn at odyssey.billn.net Mon Mar 13 13:48:14 2006 From: billn at odyssey.billn.net (Bill Nash) Date: Mon, 13 Mar 2006 16:48:14 -0500 (EST) Subject: [Phoenix-pm] PERL DBI In-Reply-To: References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> <17429.58063.907324.777393@magnitude.grueslayer.com> <20060313214617.GI22790@illogics.org> Message-ID: Also, while I'm thinking about it, key buffers will play an important factor if indexing is indeed enabled. - billn On Mon, 13 Mar 2006, Bill Nash wrote: > > You're very likely pressing up against DB execution overhead. > > For starters, you're doing one insert per row. The database side mechanics > of this are: > > 1. Accept insert. > 2. Update indexes. > 3. Log query where applicable. > 4. Platform specific spooky buffer action and related flushing. > > Platform independent, if you're going to chuck a couple million rows like > this, disable your indexing first. > > I don't know how you'd optimize this with Oracle, but mysql supports bulk > insert syntax, ala: > > insert into foo (a, b, c, d, beer) values (1, 2, 3, 4, 'bar'), (3, 4, 5, > 6, 'heineken'), ...; > > Even with indexing enabled, this is far more efficient, as indexes are > updated en masse after the insert is accepted. > > In psuedocode, something like: > > prepare query > execute query > while iterating results, > push result set onto array. > increment counter. > if counter hits 5000, > bulk insert entries stored in array. > erase array. > reset counter. > end if > end while > > I've gotten DBI on good hardware to iterate upwards of 20k rows a second. > Realistic insert rates are a different animal, because of the i/o (your > mileage may vary depending on media speeds), but 6-8k a second is not > unrealistic. Other factors include the engine type, the data i/o bus of > the machine, etc. > > - billn > > On Mon, 13 Mar 2006, Scott Walters wrote: > >> On 0, "David A. Sinck" wrote: >>> >>> HAHAHAHAHA. Guess again. There's an instance, at least with MySQL >> >> Okay, I don't use MySQL heavy (for reasons of it better garbage), so >> I over generalized. I conceede your point that rare bugs will make >> this generalization not always true. >> >>> \_ and you don't suffer >>> \_ from SQL command length limitations, >>> >>> I bet I could could craft something that hits a buffer still using >>> placeholders if I wanted. :-) >> >> I didn't say he didn't suffer from *buffer* length considerations -- that >> would be foolish indeed. I only said that he doesn't suffer from this >> particular one. I've seen databases cap the command buffer at 64k. >> Generally speaking (true in most cases, perhaps false in some) the >> length-counted buffers for placeholder data will be significantly larger >> than the command buffer. >> >>> >>> \_ and the database doesn't have to parse potentially >>> \_ megs of data just to parse the SQL command, >>> >>> Depends on the driver. MySQL dbd, last I checked, actually sub'd the >>> values in before it got passed to the engine. YMMV. >> >> That must have been a long time ago indeed. But you caught me -- >> I made another false generalization. I should have said "Generally, " >> in front of it. >> >> By the way, people who argue small points while ignoring the point of >> what was said annoy me. Did you really think someone on the list >> would be damaged thinking that placeholders always send data out of band, >> even when using Informix? >> >> I wrote a hasty reply out of desire to be helpful to someone who was >> having trouble with something, not to debate minutia, and have been >> punished for it. >> >> -scott >> >>> >>> David >>> _______________________________________________ >>> Phoenix-pm mailing list >>> Phoenix-pm at pm.org >>> http://mail.pm.org/mailman/listinfo/phoenix-pm >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm >> > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > From Peter.Loo at source.wolterskluwer.com Mon Mar 13 14:01:01 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 15:01:01 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE955A3@phxmail02.phx.ndchealth.com> Hi, I was wondering if auto commit is turned on by default. Perhaps my process is committing for each row. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: phoenix-pm-bounces+peter.loo=source.wolterskluwer.com at pm.org [mailto:phoenix-pm-bounces+peter.loo=source.wolterskluwer.com at pm.org] On Behalf Of Bill Nash Sent: Monday, March 13, 2006 2:47 PM To: Scott Walters Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI You're very likely pressing up against DB execution overhead. For starters, you're doing one insert per row. The database side mechanics of this are: 1. Accept insert. 2. Update indexes. 3. Log query where applicable. 4. Platform specific spooky buffer action and related flushing. Platform independent, if you're going to chuck a couple million rows like this, disable your indexing first. I don't know how you'd optimize this with Oracle, but mysql supports bulk insert syntax, ala: insert into foo (a, b, c, d, beer) values (1, 2, 3, 4, 'bar'), (3, 4, 5, 6, 'heineken'), ...; Even with indexing enabled, this is far more efficient, as indexes are updated en masse after the insert is accepted. In psuedocode, something like: prepare query execute query while iterating results, push result set onto array. increment counter. if counter hits 5000, bulk insert entries stored in array. erase array. reset counter. end if end while I've gotten DBI on good hardware to iterate upwards of 20k rows a second. Realistic insert rates are a different animal, because of the i/o (your mileage may vary depending on media speeds), but 6-8k a second is not unrealistic. Other factors include the engine type, the data i/o bus of the machine, etc. - billn On Mon, 13 Mar 2006, Scott Walters wrote: > On 0, "David A. Sinck" wrote: >> >> HAHAHAHAHA. Guess again. There's an instance, at least with MySQL > > Okay, I don't use MySQL heavy (for reasons of it better garbage), so I > over generalized. I conceede your point that rare bugs will make this > generalization not always true. > >> \_ and you don't suffer >> \_ from SQL command length limitations, >> >> I bet I could could craft something that hits a buffer still using >> placeholders if I wanted. :-) > > I didn't say he didn't suffer from *buffer* length considerations -- > that would be foolish indeed. I only said that he doesn't suffer from > this particular one. I've seen databases cap the command buffer at 64k. > Generally speaking (true in most cases, perhaps false in some) the > length-counted buffers for placeholder data will be significantly > larger than the command buffer. > >> >> \_ and the database doesn't have to parse potentially \_ megs of data >> just to parse the SQL command, >> >> Depends on the driver. MySQL dbd, last I checked, actually sub'd the >> values in before it got passed to the engine. YMMV. > > That must have been a long time ago indeed. But you caught me -- I > made another false generalization. I should have said "Generally, " > in front of it. > > By the way, people who argue small points while ignoring the point of > what was said annoy me. Did you really think someone on the list > would be damaged thinking that placeholders always send data out of > band, even when using Informix? > > I wrote a hasty reply out of desire to be helpful to someone who was > having trouble with something, not to debate minutia, and have been > punished for it. > > -scott > >> >> David >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From billn at odyssey.billn.net Mon Mar 13 13:53:59 2006 From: billn at odyssey.billn.net (Bill Nash) Date: Mon, 13 Mar 2006 16:53:59 -0500 (EST) Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> Message-ID: Also, because of the linear nature of file seeks, serial insert rates drop as the size of your data store increases. I mentioned this in an earlier email, and it will vary by platform (Innodb, anything using raw disk hashing, won't have this issue as much. MyISAM is absolutely prone to it.) - billn On Mon, 13 Mar 2006, Loo, Peter # PHX wrote: > > Here is what it took to run: > > Mon Mar 13 13:05:34 MST 2006 > Mon Mar 13 14:31:27 MST 2006 > > SQL> select count(*) from dssppv.tmp_falcon_projections; > > COUNT(*) > ---------- > 2413059 > > > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Monday, March 13, 2006 2:36 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > >> This is sad. > > No, this is not sad. Dump/undump programs are hand-tuned C designed to > do exactly one thing and do it very quickly. What you're doing would be > nearly as slow in any language (or nearly as fast). That aside, if it > runs for > 24 hours and you have to do it once, what are you crying about? Was the > project notice 48 hours to move from one database system to another? > Are > you doing daily syncs? Go do something else. Presumably the program > prints status info. If not, make it, just to detect possible > pathological conditions. > > -scott > > On 0, "Loo, Peter # PHX" wrote: >> >> Hi Scott, >> >> I kicked off the program at 13:05:34 and it is still running. We are >> only dealing with 2,000,000 rows. This is not going to work. I have >> to think of something. Dumping the data into and external table in >> Neteeza on a NFS mount and using SQL_LOAD in Oracle will definitely be > faster. >> This is sad. >> >> Peter Loo >> Wolters Kluwer Health >> (602) 381-9553 >> >> -----Original Message----- >> From: Scott Walters [mailto:scott at illogics.org] >> Sent: Monday, March 13, 2006 2:14 PM >> To: Loo, Peter # PHX >> Cc: phoenix-pm at pm.org >> Subject: Re: [Phoenix-pm] PERL DBI >> >> Peter, >> >> The change in speed would be trivial then, I'm guessing, but there's >> always Devel::Prof if you don't want a guess. >> >> You're suffering from round-trip time right now more than throughput, >> I'm also guessing. >> >> -scott >> >> On 0, "Loo, Peter # PHX" wrote: >>> >>> Scott, >>> >>> How about if I changed the fetchrow_array to fetchrow_arrayref? I >>> think that will help. I am currently using the same database to >>> test (Oracle production table to an Oracle temp table). I am not >>> using Neteeza (source database) yet. >>> >>> Peter Loo >>> Wolters Kluwer Health >>> (602) 381-9553 >>> >>> -----Original Message----- >>> From: Scott Walters [mailto:scott at illogics.org] >>> Sent: Monday, March 13, 2006 1:29 PM >>> To: Loo, Peter # PHX >>> Cc: phoenix-pm at pm.org >>> Subject: Re: [Phoenix-pm] PERL DBI >>> >>> Hi Peter, >>> >>> Was it running slow before? I can't imagine that this would make it > >>> slower... >>> >>> You could get a little parallelism if you used threads and made a >>> Thread::Queue to pass rows from a reader thread to a writer > thread... >>> but Perl threads are pretty grumpy. >>> >>> Just a thought. >>> >>> Cheers, >>> -scott >>> >>> On 0, "Loo, Peter # PHX" > wrote: >>>> >>>> Hi All, >>>> >>>> Alright, I got it working, but it is running so slow. I am >>>> wondering if there is anyway to speed this up? Here is the syntax > >>>> I >> am using: >>>> >>>> $bindVars = join ', ', ('?') x @column_names; >>>> $d_sth = $d_dbh->prepare(qq{insert into >>>> dssppv.tmp_falcon_projections ($columnNames) >>>> values ($bindVars)}) || die >>>> "$DBI::errstr\n"; >>>> $s_sth = $s_dbh->prepare("select * from >>>> dssppv.p_falcon_projections") || die "$DBI::errstr\n"; >>>> $s_sth->execute() || die "$DBI::errstr\n"; >>>> while ( my @row = $s_sth->fetchrow_array ) { >>>> $d_sth->execute(@row); >>>> } >>>> >>>> >>>> >>>> Peter Loo >>>> Wolters Kluwer Health >>>> (602) 381-9553 >>>> >>>> -----Original Message----- >>>> From: Scott Walters [mailto:scott at illogics.org] >>>> Sent: Monday, March 13, 2006 10:30 AM >>>> To: Loo, Peter # PHX >>>> Cc: phoenix-pm at pm.org >>>> Subject: Re: [Phoenix-pm] PERL DBI >>>> >>>> Hi Peter, >>>> >>>> Use placeholders: >>>> >>>> my $question_marks = join ', ', '?' x @column_names; # one ? >>>> for each value >>>> my $sth = $dbh->prepare(qq{insert into foo (@column_names) >>>> values >>>> ($question_marks)}) or die $dbh->errstr; >>>> while(my @row = $old_database->fetchrow_array) { # read from >>>> old database >>>> $sth->execute(@row); # insert into new database >>>> } >>>> >>>> If you use placeholders, you never need to quote or escape data, >>>> and >> >>>> you don't suffer from SQL command length limitations, and the >>>> database >>> >>>> doesn't have to parse potentially megs of data just to parse the >>>> SQL >> >>>> command, and it doesn't have to parse the SQL more than once at >>>> all >>>> -- >>> >>>> you prepare it once and then execute it as many times as you like >>>> for a huge speed increase. If you're loading a lot of data, >>>> you'll also want to drop indexes first and the rebuild them later. > >>>> This is >> >>>> a huge >>> >>>> time savings over incrementally updating them for millions of >> records. >>>> >>>> I might have missed the point of your question -- if so, please >>>> clarify, and I aplogize. >>>> >>>> Cheers, >>>> -scott >>>> >>>> On 0, "Loo, Peter # PHX" >> wrote: >>>>> >>>>> Hi, >>>>> >>>>> >>>>> $columnsToAssign = $columnsToAssign . ", \"" . >>>> @$d_arrayref[0] >>>>> . "\""; >>>>> } >>>>> else { >>>>> $columnsToAssign = $columnsToAssign . ", " . >>>> @$d_arrayref[0]; >>>>> } >>>>> >>>>> Is there an easier way? >>>>> >>>>> >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> >>>>> Peter Loo >>>>> >>>>> >>>> >>>> >>>> This E-mail message is for the sole use of the intended >>>> recipient(s) >> >>>> and may contain confidential and privileged information. Any >>>> unauthorized review, use, disclosure or distribution is > prohibited. >>>> If you are not the intended recipient, please contact the sender >>>> by reply E-mail, and destroy all copies of the original message. >>>> >>>> >>>> This E-mail message is for the sole use of the intended >>>> recipient(s) >>> and may contain confidential and privileged information. Any >>> unauthorized review, use, disclosure or distribution is prohibited. >>> If you are not the intended recipient, please contact the sender by >>> reply E-mail, and destroy all copies of the original message. >>> >>> >>> This E-mail message is for the sole use of the intended recipient(s) > >>> and may contain confidential and privileged information. Any >>> unauthorized review, use, disclosure or distribution is prohibited. >>> If you are not the intended recipient, please contact the sender by >>> reply E-mail, and destroy all copies of the original message. >>> >>> >>> This E-mail message is for the sole use of the intended recipient(s) >> and may contain confidential and privileged information. Any >> unauthorized review, use, disclosure or distribution is prohibited. >> If you are not the intended recipient, please contact the sender by >> reply E-mail, and destroy all copies of the original message. >> >> >> This E-mail message is for the sole use of the intended recipient(s) >> and may contain confidential and privileged information. Any >> unauthorized review, use, disclosure or distribution is prohibited. >> If you are not the intended recipient, please contact the sender by >> reply E-mail, and destroy all copies of the original message. >> >> >> This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > From awwaiid at thelackthereof.org Mon Mar 13 14:04:10 2006 From: awwaiid at thelackthereof.org (Brock) Date: Mon, 13 Mar 2006 15:04:10 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> Message-ID: <20060313220410.GG4287@thelackthereof.org> I say not too bad... 2 and a half million records is nothing to sneeze at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by hand! :) Ideas for presentations: * DBI * Profiling Perl Code --Brock On 2006.03.13.14.37, Loo, Peter # PHX wrote: | | Here is what it took to run: | | Mon Mar 13 13:05:34 MST 2006 | Mon Mar 13 14:31:27 MST 2006 | | SQL> select count(*) from dssppv.tmp_falcon_projections; | | COUNT(*) | ---------- | 2413059 From Peter.Loo at source.wolterskluwer.com Mon Mar 13 14:05:40 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 15:05:40 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE955B5@phxmail02.phx.ndchealth.com> You are a funny guy. :) I think I will type the data in manually. :) Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Brock [mailto:awwaiid at thelackthereof.org] Sent: Monday, March 13, 2006 3:04 PM To: Loo, Peter # PHX Cc: Scott Walters; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI I say not too bad... 2 and a half million records is nothing to sneeze at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by hand! :) Ideas for presentations: * DBI * Profiling Perl Code --Brock On 2006.03.13.14.37, Loo, Peter # PHX wrote: | | Here is what it took to run: | | Mon Mar 13 13:05:34 MST 2006 | Mon Mar 13 14:31:27 MST 2006 | | SQL> select count(*) from dssppv.tmp_falcon_projections; | | COUNT(*) | ---------- | 2413059 This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From intertwingled at qwest.net Mon Mar 13 14:11:07 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Mon, 13 Mar 2006 15:11:07 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <20060313220410.GG4287@thelackthereof.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> <20060313220410.GG4287@thelackthereof.org> Message-ID: <4415EDFB.5070003@qwest.net> Makes me pine away for the good old days, when I used Model 204 on IBM mainframes, a non-relational database system that could easily handle hundreds of millions of records. Too bad they never ported it over to Unix or Window, it was a great database engine. Had a "User Language" database programming language with database and screen primitives that was very Perlish, too. Model 204 used inverted trees for indexing, so queries were simply bitwise and'ing and or'ing bitmaps together. Response time for a query on a 900 million record database was typically under 3 seconds. Brock wrote: > I say not too bad... 2 and a half million records is nothing to sneeze > at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by > hand! :) > > Ideas for presentations: > * DBI > * Profiling Perl Code > > --Brock > > On 2006.03.13.14.37, Loo, Peter # PHX wrote: > | > | Here is what it took to run: > | > | Mon Mar 13 13:05:34 MST 2006 > | Mon Mar 13 14:31:27 MST 2006 > | > | SQL> select count(*) from dssppv.tmp_falcon_projections; > | > | COUNT(*) > | ---------- > | 2413059 > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! From awwaiid at thelackthereof.org Mon Mar 13 14:11:26 2006 From: awwaiid at thelackthereof.org (Brock) Date: Mon, 13 Mar 2006 15:11:26 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE955A3@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE955A3@phxmail02.phx.ndchealth.com> Message-ID: <20060313221126.GH4287@thelackthereof.org> Best to make it explicity by putting AutoCommit => 0 in your dbh setup. Then do $dbh->commit every now and then. Also, almost but not quite completely unrelated, someting I like to use for pretty and quick progress reports is "\r", which will go back to the beginning of the line but not the next. So: while(...) { print "Now processing record $n/$total\r"; # or maybe every 1000th record print "Now processing record $n/$total\r" unless $n % 1000; ... } print "\ndone.\n"; --Brock On 2006.03.13.15.01, Loo, Peter # PHX wrote: | | Hi, | | I was wondering if auto commit is turned on by default. Perhaps my | process is committing for each row. | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 From scott at illogics.org Mon Mar 13 14:18:34 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 22:18:34 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE955B5@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE955B5@phxmail02.phx.ndchealth.com> Message-ID: <20060313221834.GK22790@illogics.org> Hi Peter, Reading through a bit, your repeated question of whether autoinsert is on hasn't been answered. The rest of you on this list, except Andrew, Brock, and Michael, suck. I don't know the exact version or the version of DBI you're using, but newer versions do indeed turn it in by default. You can turn it off in the connect string... DBI->connect(mush, username, pass, { AutoCommit => 0 }); Then BEGIN and COMMIT as you please. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > You are a funny guy. :) I think I will type the data in manually. :) > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Brock [mailto:awwaiid at thelackthereof.org] > Sent: Monday, March 13, 2006 3:04 PM > To: Loo, Peter # PHX > Cc: Scott Walters; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > I say not too bad... 2 and a half million records is nothing to sneeze > at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by > hand! :) > > Ideas for presentations: > * DBI > * Profiling Perl Code > > --Brock > > On 2006.03.13.14.37, Loo, Peter # PHX wrote: > | > | Here is what it took to run: > | > | Mon Mar 13 13:05:34 MST 2006 > | Mon Mar 13 14:31:27 MST 2006 > | > | SQL> select count(*) from dssppv.tmp_falcon_projections; > | > | COUNT(*) > | ---------- > | 2413059 > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Mon Mar 13 14:19:17 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 22:19:17 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <4415EDFB.5070003@qwest.net> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> <20060313220410.GG4287@thelackthereof.org> <4415EDFB.5070003@qwest.net> Message-ID: <20060313221917.GL22790@illogics.org> Quit pining and start coding. On 0, "Anthony R. Nemmer" wrote: > Makes me pine away for the good old days, when I used Model 204 on IBM > mainframes, a non-relational database system that could easily handle > hundreds of millions of records. Too bad they never ported it over to > Unix or Window, it was a great database engine. Had a "User Language" > database programming language with database and screen primitives that > was very Perlish, too. Model 204 used inverted trees for indexing, so > queries were simply bitwise and'ing and or'ing bitmaps together. > Response time for a query on a 900 million record database was typically > under 3 seconds. > > Brock wrote: > > I say not too bad... 2 and a half million records is nothing to sneeze > > at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by > > hand! :) > > > > Ideas for presentations: > > * DBI > > * Profiling Perl Code > > > > --Brock > > > > On 2006.03.13.14.37, Loo, Peter # PHX wrote: > > | > > | Here is what it took to run: > > | > > | Mon Mar 13 13:05:34 MST 2006 > > | Mon Mar 13 14:31:27 MST 2006 > > | > > | SQL> select count(*) from dssppv.tmp_falcon_projections; > > | > > | COUNT(*) > > | ---------- > > | 2413059 > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Mon Mar 13 14:14:23 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 15:14:23 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE955C9@phxmail02.phx.ndchealth.com> Have you folks used Neteeza. It is just the same. It is fast as hack. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Anthony R. Nemmer [mailto:intertwingled at qwest.net] Sent: Monday, March 13, 2006 3:11 PM To: Brock Cc: Loo, Peter # PHX; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Makes me pine away for the good old days, when I used Model 204 on IBM mainframes, a non-relational database system that could easily handle hundreds of millions of records. Too bad they never ported it over to Unix or Window, it was a great database engine. Had a "User Language" database programming language with database and screen primitives that was very Perlish, too. Model 204 used inverted trees for indexing, so queries were simply bitwise and'ing and or'ing bitmaps together. Response time for a query on a 900 million record database was typically under 3 seconds. Brock wrote: > I say not too bad... 2 and a half million records is nothing to sneeze > at. Thats just over 467 rows/sec, I figure. Lot faster than doing it > by hand! :) > > Ideas for presentations: > * DBI > * Profiling Perl Code > > --Brock > > On 2006.03.13.14.37, Loo, Peter # PHX wrote: > | > | Here is what it took to run: > | > | Mon Mar 13 13:05:34 MST 2006 > | Mon Mar 13 14:31:27 MST 2006 > | > | SQL> select count(*) from dssppv.tmp_falcon_projections; > | > | COUNT(*) > | ---------- > | 2413059 > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From awwaiid at thelackthereof.org Mon Mar 13 14:19:25 2006 From: awwaiid at thelackthereof.org (Brock) Date: Mon, 13 Mar 2006 15:19:25 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <20060313221834.GK22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE955B5@phxmail02.phx.ndchealth.com> <20060313221834.GK22790@illogics.org> Message-ID: <20060313221925.GI4287@thelackthereof.org> Whats the point of BEGIN? Nested transactions or something? Other than that (which I don't run into often...) BEGIN is redundant :) --Brock On 2006.03.13.22.18, Scott Walters wrote: | Hi Peter, | | Reading through a bit, your repeated question of whether autoinsert is on | hasn't been answered. The rest of you on this list, except Andrew, | Brock, and Michael, suck. I don't know the exact version or the version | of DBI you're using, but newer versions do indeed turn it in by default. | You can turn it off in the connect string... | | DBI->connect(mush, username, pass, { AutoCommit => 0 }); | | Then BEGIN and COMMIT as you please. | | Cheers, | -scott From intertwingled at qwest.net Mon Mar 13 14:33:08 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Mon, 13 Mar 2006 15:33:08 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <20060313221917.GL22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> <20060313220410.GG4287@thelackthereof.org> <4415EDFB.5070003@qwest.net> <20060313221917.GL22790@illogics.org> Message-ID: <4415F324.5070906@qwest.net> Can't. Finishing up my huge-o list of file extensions right now! Scott Walters wrote: > Quit pining and start coding. > > On 0, "Anthony R. Nemmer" wrote: >> Makes me pine away for the good old days, when I used Model 204 on IBM >> mainframes, a non-relational database system that could easily handle >> hundreds of millions of records. Too bad they never ported it over to >> Unix or Window, it was a great database engine. Had a "User Language" >> database programming language with database and screen primitives that >> was very Perlish, too. Model 204 used inverted trees for indexing, so >> queries were simply bitwise and'ing and or'ing bitmaps together. >> Response time for a query on a 900 million record database was typically >> under 3 seconds. >> >> Brock wrote: >>> I say not too bad... 2 and a half million records is nothing to sneeze >>> at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by >>> hand! :) >>> >>> Ideas for presentations: >>> * DBI >>> * Profiling Perl Code >>> >>> --Brock >>> >>> On 2006.03.13.14.37, Loo, Peter # PHX wrote: >>> | >>> | Here is what it took to run: >>> | >>> | Mon Mar 13 13:05:34 MST 2006 >>> | Mon Mar 13 14:31:27 MST 2006 >>> | >>> | SQL> select count(*) from dssppv.tmp_falcon_projections; >>> | >>> | COUNT(*) >>> | ---------- >>> | 2413059 >>> _______________________________________________ >>> Phoenix-pm mailing list >>> Phoenix-pm at pm.org >>> http://mail.pm.org/mailman/listinfo/phoenix-pm >>> >>> >> >> -- >> >> I always have coffee when I watch radar! >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! From Peter.Loo at source.wolterskluwer.com Mon Mar 13 14:44:37 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Mon, 13 Mar 2006 15:44:37 -0700 Subject: [Phoenix-pm] PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE955FD@phxmail02.phx.ndchealth.com> Hey Guys, I changed the commit to every 5000 records from auto commit and used fetchrow_arrayref instead of fetchrow_array and the result was GREAT. START: Mon Mar 13 15:27:57 MST 2006 END: Mon Mar 13 15:41:18 MST 2006 Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Brock [mailto:awwaiid at thelackthereof.org] Sent: Monday, March 13, 2006 3:11 PM To: Loo, Peter # PHX Cc: Bill Nash; Scott Walters; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Best to make it explicity by putting AutoCommit => 0 in your dbh setup. Then do $dbh->commit every now and then. Also, almost but not quite completely unrelated, someting I like to use for pretty and quick progress reports is "\r", which will go back to the beginning of the line but not the next. So: while(...) { print "Now processing record $n/$total\r"; # or maybe every 1000th record print "Now processing record $n/$total\r" unless $n % 1000; ... } print "\ndone.\n"; --Brock On 2006.03.13.15.01, Loo, Peter # PHX wrote: | | Hi, | | I was wondering if auto commit is turned on by default. Perhaps my | process is committing for each row. | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From scott at illogics.org Mon Mar 13 14:52:14 2006 From: scott at illogics.org (Scott Walters) Date: Mon, 13 Mar 2006 22:52:14 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <4415F324.5070906@qwest.net> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> <20060313220410.GG4287@thelackthereof.org> <4415EDFB.5070003@qwest.net> <20060313221917.GL22790@illogics.org> <4415F324.5070906@qwest.net> Message-ID: <20060313225214.GO22790@illogics.org> Doesn't sound like the sort of thing that can be finished. On 0, "Anthony R. Nemmer" wrote: > Can't. Finishing up my huge-o list of file extensions right now! > > Scott Walters wrote: > > Quit pining and start coding. > > > > On 0, "Anthony R. Nemmer" wrote: > >> Makes me pine away for the good old days, when I used Model 204 on IBM > >> mainframes, a non-relational database system that could easily handle > >> hundreds of millions of records. Too bad they never ported it over to > >> Unix or Window, it was a great database engine. Had a "User Language" > >> database programming language with database and screen primitives that > >> was very Perlish, too. Model 204 used inverted trees for indexing, so > >> queries were simply bitwise and'ing and or'ing bitmaps together. > >> Response time for a query on a 900 million record database was typically > >> under 3 seconds. > >> > >> Brock wrote: > >>> I say not too bad... 2 and a half million records is nothing to sneeze > >>> at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by > >>> hand! :) > >>> > >>> Ideas for presentations: > >>> * DBI > >>> * Profiling Perl Code > >>> > >>> --Brock > >>> > >>> On 2006.03.13.14.37, Loo, Peter # PHX wrote: > >>> | > >>> | Here is what it took to run: > >>> | > >>> | Mon Mar 13 13:05:34 MST 2006 > >>> | Mon Mar 13 14:31:27 MST 2006 > >>> | > >>> | SQL> select count(*) from dssppv.tmp_falcon_projections; > >>> | > >>> | COUNT(*) > >>> | ---------- > >>> | 2413059 > >>> _______________________________________________ > >>> Phoenix-pm mailing list > >>> Phoenix-pm at pm.org > >>> http://mail.pm.org/mailman/listinfo/phoenix-pm > >>> > >>> > >> > >> -- > >> > >> I always have coffee when I watch radar! > >> _______________________________________________ > >> Phoenix-pm mailing list > >> Phoenix-pm at pm.org > >> http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Mon Mar 13 18:58:38 2006 From: scott at illogics.org (Scott Walters) Date: Tue, 14 Mar 2006 02:58:38 +0000 Subject: [Phoenix-pm] P6N copies In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE955FD@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE955FD@phxmail02.phx.ndchealth.com> Message-ID: <20060314025838.GS22790@illogics.org> Well, damnit, I think I had a liar made out of me. This didn't ship. I'll try to fix this tomorrow but I might not have swag for yall after all. Sorry. Gosh durnit. -scott From phx-pm-list at grueslayer.com Mon Mar 13 20:12:00 2006 From: phx-pm-list at grueslayer.com (David A. Sinck) Date: Mon, 13 Mar 2006 21:12:00 -0700 Subject: [Phoenix-pm] PERL DBI References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> <17429.58063.907324.777393@magnitude.grueslayer.com> <20060313214617.GI22790@illogics.org> Message-ID: <17430.17040.806923.974908@magnitude.grueslayer.com> \_ SMTP quoth Scott Walters on 3/13/2006 21:46 as having spake thusly: \_ \_ On 0, "David A. Sinck" wrote: \_ > \_ > HAHAHAHAHA. Guess again. There's an instance, at least with MySQL \_ \_ Okay, I don't use MySQL heavy (for reasons of it better garbage), so \_ I over generalized. I conceede your point that rare bugs will make \_ this generalization not always true. I don't think it's a mysql issue, mysql didn't guess wrong on the data, it's the dbd driver that guessed wrong. It might be smarter now, I've not gone back to retest the pathological cases. Better, faster bugs to chase! :-) \_ By the way, people who argue small points while ignoring the point of \_ what was said annoy me. I didn't ignore the point, I just was adding backfill around it showing that MySQL DBD, in my experience, was occasionally pathological. I was NOT saying Scott was pathological, merely MySQL. If it was perceived that way, a thousand and one apologies. \_ Did you really think someone on the list \_ would be damaged thinking that placeholders always send data out of band, \_ even when using Informix? Last I checked, we get all kinds here. Some might be overly trusting yet. :-) \_ I wrote a hasty reply out of desire to be helpful to someone who was \_ having trouble with something, not to debate minutia, and have been \_ punished for it. No no no, punish is not the perception I wanted. You gave great suggestions, I was pointing out not all is rosy with everything. Until MySQL DBD has bit you (might be fixed now, I haven't checked), you don't know the pain of trying to track down errors from Perl *not* doing the right thing. After you've got multiple dev-hours down tracking something that shouldn't be and very, very rarely is, then you can see if my whining is justified. :-) If I hadn't been ambushed by a meeting of great pain on ROUS (reports of unusual size... they do exist!) I would have squashed this misperception earlier. Again, MySQL dbd bad, Scott good. MySQL 4.x fun, possibly later copies as well: create table foo (bar datetime); insert into foo values ('04:02:31'); select * from foo; David From scott at illogics.org Mon Mar 13 23:59:40 2006 From: scott at illogics.org (Scott Walters) Date: Tue, 14 Mar 2006 07:59:40 +0000 Subject: [Phoenix-pm] autobox fun In-Reply-To: <4415F324.5070906@qwest.net> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> <20060313220410.GG4287@thelackthereof.org> <4415EDFB.5070003@qwest.net> <20060313221917.GL22790@illogics.org> <4415F324.5070906@qwest.net> Message-ID: <20060314075940.GW22790@illogics.org> Hi all, Just had to share this stupid Perl trick. I've been doing some Perl along side a Python programmer and so I've been trying to think of ways to get his goat... by making Perl both elegant looking and also by showing off it's expressive power. The autobox module no longer requires a patch to the Perl interpreter (yay!), so I started using it in production. One thing that bugged me was how site config data was accessed... I had been writing conf->{path}->{webroot} and stuff like that. That's ugly. conf is exported to the module and is a sub that returns a hashref. I'd rather write conf->path->webroot. Here's some glue to make that work: (in autoboxglue.pm:) package HASH; sub AUTOLOAD :lvalue { my $method = $AUTOLOAD; $method =~ s/.*:://; return if $method eq 'DESTROY'; my $hashref = shift; $hashref->{$method}; } 1; It's very simple... it takes method calls on hashrefs (that aren't otherwise blessed), extracts the method name, evaluates to the result of that subscripting that hashref with the method name called on it. I posted this to my blog too, so sorry to dup it. Cheers, -scott From scott at illogics.org Tue Mar 14 00:01:13 2006 From: scott at illogics.org (Scott Walters) Date: Tue, 14 Mar 2006 08:01:13 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <17430.17040.806923.974908@magnitude.grueslayer.com> References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> <17429.58063.907324.777393@magnitude.grueslayer.com> <20060313214617.GI22790@illogics.org> <17430.17040.806923.974908@magnitude.grueslayer.com> Message-ID: <20060314080113.GX22790@illogics.org> Okay, I'm sorry, I was overly harsh and defensive there... still, I suggest you make your motives and intentions clear when correcting people, especially grumps like me, to avoid us going down the war path. Cheers, -scott On 0, "David A. Sinck" wrote: > > > \_ SMTP quoth Scott Walters on 3/13/2006 21:46 as having spake thusly: > \_ > \_ On 0, "David A. Sinck" wrote: > \_ > > \_ > HAHAHAHAHA. Guess again. There's an instance, at least with MySQL > \_ > \_ Okay, I don't use MySQL heavy (for reasons of it better garbage), so > \_ I over generalized. I conceede your point that rare bugs will make > \_ this generalization not always true. > > I don't think it's a mysql issue, mysql didn't guess wrong on the > data, it's the dbd driver that guessed wrong. It might be smarter > now, I've not gone back to retest the pathological cases. Better, > faster bugs to chase! :-) > > > \_ By the way, people who argue small points while ignoring the point of > \_ what was said annoy me. > > I didn't ignore the point, I just was adding backfill around it > showing that MySQL DBD, in my experience, was occasionally > pathological. I was NOT saying Scott was pathological, merely MySQL. > If it was perceived that way, a thousand and one apologies. > > \_ Did you really think someone on the list > \_ would be damaged thinking that placeholders always send data out of band, > \_ even when using Informix? > > Last I checked, we get all kinds here. Some might be overly trusting > yet. :-) > > \_ I wrote a hasty reply out of desire to be helpful to someone who was > \_ having trouble with something, not to debate minutia, and have been > \_ punished for it. > > No no no, punish is not the perception I wanted. You gave great > suggestions, I was pointing out not all is rosy with everything. > > Until MySQL DBD has bit you (might be fixed now, I haven't checked), > you don't know the pain of trying to track down errors from Perl *not* > doing the right thing. After you've got multiple dev-hours down > tracking something that shouldn't be and very, very rarely is, then > you can see if my whining is justified. :-) > > If I hadn't been ambushed by a meeting of great pain on ROUS (reports > of unusual size... they do exist!) I would have squashed this > misperception earlier. > > Again, MySQL dbd bad, Scott good. > > MySQL 4.x fun, possibly later copies as well: > create table foo (bar datetime); > insert into foo values ('04:02:31'); > select * from foo; > > David > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From phx-pm-list at grueslayer.com Tue Mar 14 05:07:51 2006 From: phx-pm-list at grueslayer.com (David A. Sinck) Date: Tue, 14 Mar 2006 06:07:51 -0700 Subject: [Phoenix-pm] PERL DBI References: <8E3D502A002DA04FADBDED4CB4D94D3ADF5B60@phxmail02.phx.ndchealth.com> <20060313173005.GZ22790@illogics.org> <17429.58063.907324.777393@magnitude.grueslayer.com> <20060313214617.GI22790@illogics.org> <17430.17040.806923.974908@magnitude.grueslayer.com> <20060314080113.GX22790@illogics.org> Message-ID: <17430.49191.985461.442117@magnitude.grueslayer.com> \_ SMTP quoth Scott Walters on 3/14/2006 08:01 as having spake thusly: \_ \_ Okay, I'm sorry, I was overly harsh and defensive there... Happens, don't sweat it. \_ still, I suggest you make your motives and intentions clear when \_ correcting people, especially grumps like me, to avoid us going \_ down the war path. How about I'll try to 'splain for grumps if they don't get automagically riled when I don't? :-) David There's a part that's good. There's a part that's bad. There's a part you wouldn't believe. US Senator describing Admiral H. Rickover From awwaiid at thelackthereof.org Tue Mar 14 08:03:38 2006 From: awwaiid at thelackthereof.org (Brock) Date: Tue, 14 Mar 2006 09:03:38 -0700 Subject: [Phoenix-pm] autobox fun In-Reply-To: <20060314075940.GW22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95584@phxmail02.phx.ndchealth.com> <20060313220410.GG4287@thelackthereof.org> <4415EDFB.5070003@qwest.net> <20060313221917.GL22790@illogics.org> <4415F324.5070906@qwest.net> <20060314075940.GW22790@illogics.org> Message-ID: <20060314160338.GK4287@thelackthereof.org> As an added bonus (besides being prettier), you could later actually override the hash with real methods, eh? Make conf return an object instead of a hash, that is. --Brock On 2006.03.14.07.59, Scott Walters wrote: | Hi all, | | Just had to share this stupid Perl trick. | | I've been doing some Perl along side a Python programmer and so I've been | trying to think of ways to get his goat... by making Perl both elegant | looking and also by showing off it's expressive power. The autobox | module no longer requires a patch to the Perl interpreter (yay!), so I | started using it in production. One thing that bugged me was how site | config data was accessed... I had been writing conf->{path}->{webroot} | and stuff like that. That's ugly. conf is exported to the module and | is a sub that returns a hashref. I'd rather write conf->path->webroot. | Here's some glue to make that work: | | (in autoboxglue.pm:) | | package HASH; | | sub AUTOLOAD :lvalue { | my $method = $AUTOLOAD; $method =~ s/.*:://; | return if $method eq 'DESTROY'; | my $hashref = shift; | $hashref->{$method}; | } | | 1; | | It's very simple... it takes method calls on hashrefs (that aren't otherwise | blessed), extracts the method name, evaluates to the result of that subscripting | that hashref with the method name called on it. | | I posted this to my blog too, so sorry to dup it. | | Cheers, | -scott | | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Tue Mar 14 13:47:26 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Tue, 14 Mar 2006 14:47:26 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95A1F@phxmail02.phx.ndchealth.com> Hi, I know that you are able to issue a SQL statement within PERL DBI, but is there anyway that I can issue an external SQL program? For example, I have a SQL program called ppv_insert.sql that I would like to execute within PERL DBI. Thanks in advance. Peter Loo This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.pm.org/pipermail/phoenix-pm/attachments/20060314/cf318fa5/attachment.html From friedman at highwire.stanford.edu Tue Mar 14 14:07:44 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Tue, 14 Mar 2006 14:07:44 -0800 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95A1F@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95A1F@phxmail02.phx.ndchealth.com> Message-ID: Peter, What I did in that situation was write a couple of methods to read in the file, put it into an array, and then loop through the array and make each call. The good news is you only have to write that once and then you can reuse it... My only example is using Sybase::DBlib, though, not DBI, but the logic would be the same. Sybase uses 'go' on a line by itself to end a SQL command, so we just use that to split up the lines in the file into commands into the array. You could make this a lot fancier, if you had the need, but it works for me. Good luck, -- Mike (sub db_run_script and sub db_run_command_list, below) sub db_run_script #($$) { my $dbh = shift; my $script = shift; my $saveresults = shift; open (SQL_SCRIPT, $script) || die "Could not open input file $script: $!"; my @commands = (); my $j = 0; my ($line); # read script file into a variable (array of commands) while ($line = ) { if ($line =~ /^go/) { # make new command $j++; } elsif ($line =~ /^\s*$/) { # ignore blank lines } else { $commands[$j] .= $line; } } close SQL_SCRIPT; return db_run_command_list($dbh, \@commands, $saveresults); } sub db_run_command_list { my $dbh = shift; my $cmdlist = shift; my $saveresults = shift; my @resultlist; # run commands from array for $j(0..$#$cmdlist) { $dbh->dbcmd($cmdlist->[$j]); my $status; eval { $status = $dbh->dbsqlexec(); }; if ($@ || $status != SUCCEED) { # don't always die, because drop will fail sometimes if ($cmdlist->[$j] =~ /drop/i) { warn "$cmdlist->[$j] failed.\n This is OK - item probably didn't exist before installation.\n"; $dbh->dbcancel(); # so that we can move on to the next command? } else { die "+++ Could not run command $cmdlist->[$j]\nbecause of this problem:\n$@"; } } if (!$saveresults) { db_ignore_results($dbh); } else { # Count the total number of rows that were updated, # and capture the output of any SELECT statements # # Each update/insert statement will have its own update # count (a separate call to DBCOUNT()) but we will # just add them all together my $totalupdatecount = 0; while ($dbh->dbresults() != NO_MORE_RESULTS) { my $rcount = $dbh->DBCOUNT(); if ($rcount != -1) { $totalupdatecount += $rcount; } my @res; while (@res = $dbh->dbnextrow()) { my @copyres = @res; # make a copy of the array push @resultlist, \@copyres; } } push @resultlist, $totalupdatecount; } } if ($saveresults) { return \@resultlist; } else { return; } } On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > Hi, > > I know that you are able to issue a SQL statement within PERL DBI, but > is there anyway that I can issue an external SQL program? For > example, > I have a SQL program called ppv_insert.sql that I would like to > execute > within PERL DBI. > > Thanks in advance. > > Peter Loo > > > > > This E-mail message is for the sole use of the intended recipient > (s) and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is > prohibited. If you are not the intended recipient, please contact > the sender by reply E-mail, and destroy all copies of the original > message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From scott at illogics.org Tue Mar 14 14:27:37 2006 From: scott at illogics.org (Scott Walters) Date: Tue, 14 Mar 2006 22:27:37 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: References: <8E3D502A002DA04FADBDED4CB4D94D3AE95A1F@phxmail02.phx.ndchealth.com> Message-ID: <20060314222737.GM22790@illogics.org> Hi Peter, Surely you're trying to accomplish more than just running the SQL or you would just read it in an feed it to DBI. There's no reason you couldn't call to the database command shell: if(my $pid = fork) { waitpid $pid; } else { close STDIN; open STDIN, '<', 'foo.sql' or die $!; exec 'mysql', $dbname or die $!; } ... or something like that. If you want to use special features of the database command shell or just cash in on its speed, this might be handy. Of course, you don't want to try to read values from the database back into Perl over a pipe between two processes... that's just nasty. -scott On 0, Michael Friedman wrote: > Peter, > > What I did in that situation was write a couple of methods to read in > the file, put it into an array, and then loop through the array and > make each call. The good news is you only have to write that once and > then you can reuse it... > > My only example is using Sybase::DBlib, though, not DBI, but the > logic would be the same. Sybase uses 'go' on a line by itself to end > a SQL command, so we just use that to split up the lines in the file > into commands into the array. > > You could make this a lot fancier, if you had the need, but it works > for me. > > Good luck, > -- Mike > > (sub db_run_script and sub db_run_command_list, below) > > sub db_run_script #($$) > { > my $dbh = shift; > my $script = shift; > my $saveresults = shift; > > open (SQL_SCRIPT, $script) || die "Could not open input file $script: > $!"; > > my @commands = (); > my $j = 0; > my ($line); > > # read script file into a variable (array of commands) > while ($line = ) > { > if ($line =~ /^go/) > { > # make new command > $j++; > } > elsif ($line =~ /^\s*$/) > { > # ignore blank lines > } > else > { > $commands[$j] .= $line; > } > } > close SQL_SCRIPT; > > return db_run_command_list($dbh, \@commands, $saveresults); > } > > sub db_run_command_list > { > my $dbh = shift; > my $cmdlist = shift; > my $saveresults = shift; > > my @resultlist; > > # run commands from array > for $j(0..$#$cmdlist) > { > $dbh->dbcmd($cmdlist->[$j]); > my $status; > eval { > $status = $dbh->dbsqlexec(); > }; > > if ($@ || $status != SUCCEED) > { > # don't always die, because drop will fail sometimes > if ($cmdlist->[$j] =~ /drop/i) > { > warn "$cmdlist->[$j] failed.\n This is OK - item probably didn't > exist before installation.\n"; > > $dbh->dbcancel(); # so that we can move on to the next command? > } > else > { > die "+++ Could not run command $cmdlist->[$j]\nbecause of this > problem:\n$@"; > } > } > > if (!$saveresults) { > db_ignore_results($dbh); > } else { > # Count the total number of rows that were updated, > # and capture the output of any SELECT statements > # > # Each update/insert statement will have its own update > # count (a separate call to DBCOUNT()) but we will > # just add them all together > my $totalupdatecount = 0; > while ($dbh->dbresults() != NO_MORE_RESULTS) { > my $rcount = $dbh->DBCOUNT(); > if ($rcount != -1) { > $totalupdatecount += $rcount; > } > > my @res; > while (@res = $dbh->dbnextrow()) { > my @copyres = @res; # make a copy of the array > push @resultlist, \@copyres; > } > } > > push @resultlist, $totalupdatecount; > } > } > > if ($saveresults) { > return \@resultlist; > } else { > return; > } > } > > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > Hi, > > > > I know that you are able to issue a SQL statement within PERL DBI, but > > is there anyway that I can issue an external SQL program? For > > example, > > I have a SQL program called ppv_insert.sql that I would like to > > execute > > within PERL DBI. > > > > Thanks in advance. > > > > Peter Loo > > > > > > > > > > This E-mail message is for the sole use of the intended recipient > > (s) and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is > > prohibited. If you are not the intended recipient, please contact > > the sender by reply E-mail, and destroy all copies of the original > > message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > --------------------------------------------------------------------- > Michael Friedman HighWire Press > Phone: 650-725-1974 Stanford University > FAX: 270-721-8034 > --------------------------------------------------------------------- > > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Tue Mar 14 15:08:28 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Tue, 14 Mar 2006 16:08:28 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95A86@phxmail02.phx.ndchealth.com> Thanks Michael and Scott. What I am trying to do is creating a generic PERL program that will take in multiple arguments such as: for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { my ($flag, $value) = split(/=/, $ARGV[$cnt]); switch ($flag) { case "-dd" { $d_dbName = lc($value) } case "-dt" { $d_tblName = lc($value) } case "-ds" { $d_SQL = $value } case "-sd" { $s_dbName = lc($value) } case "-st" { $s_tblName = lc($value) } case "-ss" { $s_SQL = $value } case "-cp" { $commitPoint = lc($value) } case "-sf" { $s_funcToPerf = lc($value) } case "-df" { $d_funcToPerf = lc($value) } case "-d1" { $s_dbDriver = lc($value) } case "-d2" { $d_dbDriver = lc($value) } else { print "Unknown flag: $flag\n" } } } Then execute accordingly, however, I would like to execute an external SQL program that is passed to this generic program. In the case of an external SQL program that does SELECT instead of UPDATE or INSERT, I want to loop through the returned rows. Here is my first block of "if" statement. if ($s_funcToPerf eq "update") { if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || !$s_SQL) { print "ERROR: Not enough arguments. Require arguments are:\n"; print "Example: \n"; print " Database Name: -sd=dv26\n"; print " Database Driver: -d1=Oracle\n"; print " Table Name: -st=p_falcon_projections\n"; print " SQL Statement: -ss=/usr/local/sql/ppv_update.sql\n"; print " Commit Point: -cp=5000\n"; exit(666); } else { print "Calling sub_update()\n"; } } elsif ($s_funcToPerf eq "insert") { if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || !$s_SQL) { print "ERROR: Not enough arguments. Require arguments are:\n"; print "Example: \n"; print " Database Name: -sd=dv26\n"; print " Database Driver: -d1=Oracle\n"; print " Table Name: -st=p_falcon_projections\n"; print " SQL Statement: -ss=/usr/local/sql/ppv_insert.sql\n"; print " Commit Point: -cp=5000\n"; exit(666); } else { print "Calling sub_insert()\n"; } } elsif ($d_funcToPerf eq "update") { if (!$d_dbName || !$d_dbDriver || !$d_tblName || !$s_dbName || !$s_dbDriver || !$s_tblName || !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { print "ERROR: Not enough arguments. Require arguments are:\n"; print "Example:\n"; print " Destination Database Name: -dd=pv26\n"; print " Destination Database Driver: -d2=ODBC\n"; print " Destination Table Name: -dt=p_falcon_projections\n"; print " Source Database Name: -sd=dv26\n"; print " Source Database Driver: -d1=Oracle\n"; print " Source Table Name: -st=p_falcon_projections\n"; print " SQL Statement: -ss=/usr/local/sql/ppv_select.sql\n"; print " SQL Statement: -ds=/usr/local/sql/ppv_update.sql\n"; print " Source Function to Perform: -sf=select\n"; print " Commit Point: -cp=5000\n"; exit(666); } else { print "Calling sub_select()\n"; print "Calling sub_update()\n"; } } elsif ($d_funcToPerf eq "insert") { if (!$d_dbName || !$d_dbDriver || !$d_tblName || !$s_dbName || !$s_dbDriver || !$s_tblName || !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { print "ERROR: Not enough arguments. Require arguments are:\n"; print "Example:\n"; print " Destination Database Name: -dd=pv26\n"; print " Destination Database Driver: -d2=ODBC\n"; print " Destination Table Name: -dt=p_falcon_projections\n"; print " Source Database Name: -sd=dv26\n"; print " Source Database Driver: -d1=Oracle\n"; print " Source Table Name: -st=p_falcon_projections\n"; print " SQL Statement: -ss=/usr/local/sql/ppv_select.sql\n"; print " SQL Statement: -ds=/usr/local/sql/ppv_insert.sql\n"; print " Source Function to Perform: -sf=select\n"; print " Commit Point: -cp=5000\n"; exit(666); } else { print "Calling sub_select()\n"; print "Calling sub_insert()\n"; } } elsif ($s_funcToPerf eq "select") { if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || !$d_dbName || !$d_dbDriver || !$d_tblName || !$d_funcToPerf || !$s_SQL || !$d_SQL) { print "ERROR: Not enough arguments. Require arguments are:\n"; print "Example:\n"; print " Destination Database Name: -dd=pv26\n"; print " Destination Database Driver: -d2=ODBC\n"; print " Destination Table Name: -dt=p_falcon_projections\n"; print " Source Database Name: -sd=dv26\n"; print " Source Database Driver: -d1=Oracle\n"; print " Source Table Name: -st=p_falcon_projections\n"; print " SQL Statement: -ss=/usr/local/sql/ppv_select.sql\n"; print " SQL Statement: -ds=/usr/local/sql/ppv_insert.sql\n"; print " Source Function to Perform: -sf=select\n"; print " Commit Point: -cp=5000\n"; exit(666); } else { print "Calling sub_select()\n"; print "Calling sub_insert()\n"; } } else { print "ERROR: Unknown value for database action to perform.\n"; exit(666); } Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Tuesday, March 14, 2006 3:28 PM To: Michael Friedman Cc: Loo, Peter # PHX; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Peter, Surely you're trying to accomplish more than just running the SQL or you would just read it in an feed it to DBI. There's no reason you couldn't call to the database command shell: if(my $pid = fork) { waitpid $pid; } else { close STDIN; open STDIN, '<', 'foo.sql' or die $!; exec 'mysql', $dbname or die $!; } ... or something like that. If you want to use special features of the database command shell or just cash in on its speed, this might be handy. Of course, you don't want to try to read values from the database back into Perl over a pipe between two processes... that's just nasty. -scott On 0, Michael Friedman wrote: > Peter, > > What I did in that situation was write a couple of methods to read in > the file, put it into an array, and then loop through the array and > make each call. The good news is you only have to write that once and > then you can reuse it... > > My only example is using Sybase::DBlib, though, not DBI, but the logic > would be the same. Sybase uses 'go' on a line by itself to end a SQL > command, so we just use that to split up the lines in the file into > commands into the array. > > You could make this a lot fancier, if you had the need, but it works > for me. > > Good luck, > -- Mike > > (sub db_run_script and sub db_run_command_list, below) > > sub db_run_script #($$) > { > my $dbh = shift; > my $script = shift; > my $saveresults = shift; > > open (SQL_SCRIPT, $script) || die "Could not open input file $script: > $!"; > > my @commands = (); > my $j = 0; > my ($line); > > # read script file into a variable (array of commands) > while ($line = ) > { > if ($line =~ /^go/) > { > # make new command > $j++; > } > elsif ($line =~ /^\s*$/) > { > # ignore blank lines > } > else > { > $commands[$j] .= $line; > } > } > close SQL_SCRIPT; > > return db_run_command_list($dbh, \@commands, $saveresults); } > > sub db_run_command_list > { > my $dbh = shift; > my $cmdlist = shift; > my $saveresults = shift; > > my @resultlist; > > # run commands from array > for $j(0..$#$cmdlist) > { > $dbh->dbcmd($cmdlist->[$j]); > my $status; > eval { > $status = $dbh->dbsqlexec(); > }; > > if ($@ || $status != SUCCEED) > { > # don't always die, because drop will fail sometimes > if ($cmdlist->[$j] =~ /drop/i) > { > warn "$cmdlist->[$j] failed.\n This is OK - item probably didn't > exist before installation.\n"; > > $dbh->dbcancel(); # so that we can move on to the next command? > } > else > { > die "+++ Could not run command $cmdlist->[$j]\nbecause of this > problem:\n$@"; > } > } > > if (!$saveresults) { > db_ignore_results($dbh); > } else { > # Count the total number of rows that were updated, > # and capture the output of any SELECT statements > # > # Each update/insert statement will have its own update > # count (a separate call to DBCOUNT()) but we will > # just add them all together > my $totalupdatecount = 0; > while ($dbh->dbresults() != NO_MORE_RESULTS) { > my $rcount = $dbh->DBCOUNT(); > if ($rcount != -1) { > $totalupdatecount += $rcount; > } > > my @res; > while (@res = $dbh->dbnextrow()) { > my @copyres = @res; # make a copy of the array > push @resultlist, \@copyres; > } > } > > push @resultlist, $totalupdatecount; > } > } > > if ($saveresults) { > return \@resultlist; > } else { > return; > } > } > > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > Hi, > > > > I know that you are able to issue a SQL statement within PERL DBI, > > but is there anyway that I can issue an external SQL program? For > > example, I have a SQL program called ppv_insert.sql that I would > > like to execute within PERL DBI. > > > > Thanks in advance. > > > > Peter Loo > > > > > > > > > > This E-mail message is for the sole use of the intended recipient > > (s) and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > --------------------------------------------------------------------- > Michael Friedman HighWire Press > Phone: 650-725-1974 Stanford University > FAX: 270-721-8034 > --------------------------------------------------------------------- > > > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From scott at illogics.org Tue Mar 14 20:35:34 2006 From: scott at illogics.org (Scott Walters) Date: Wed, 15 Mar 2006 04:35:34 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95A86@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95A86@phxmail02.phx.ndchealth.com> Message-ID: <20060315043533.GT22790@illogics.org> Consider using GetOpt::Std, and most of the time you want this form of for: for my $thing (@things) { ... stuff with $thing ... } You can always try to read rows and trap errors. -scott On 0, "Loo, Peter # PHX" wrote: > > Thanks Michael and Scott. What I am trying to do is creating a generic > PERL program that will take in multiple arguments such as: > > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > switch ($flag) { > case "-dd" { $d_dbName = lc($value) } > case "-dt" { $d_tblName = lc($value) } > case "-ds" { $d_SQL = $value } > case "-sd" { $s_dbName = lc($value) } > case "-st" { $s_tblName = lc($value) } > case "-ss" { $s_SQL = $value } > case "-cp" { $commitPoint = lc($value) } > case "-sf" { $s_funcToPerf = lc($value) } > case "-df" { $d_funcToPerf = lc($value) } > case "-d1" { $s_dbDriver = lc($value) } > case "-d2" { $d_dbDriver = lc($value) } > else { print "Unknown flag: $flag\n" } > } > } > > Then execute accordingly, however, I would like to execute an external > SQL program that is passed to this generic program. In the case of an > external SQL program that does SELECT instead of UPDATE or INSERT, I > want to loop through the returned rows. Here is my first block of "if" > statement. > > if ($s_funcToPerf eq "update") { > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || > !$s_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example: \n"; > print " Database Name: -sd=dv26\n"; > print " Database Driver: -d1=Oracle\n"; > print " Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: -ss=/usr/local/sql/ppv_update.sql\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_update()\n"; > } > } > elsif ($s_funcToPerf eq "insert") { > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || > !$s_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example: \n"; > print " Database Name: -sd=dv26\n"; > print " Database Driver: -d1=Oracle\n"; > print " Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: -ss=/usr/local/sql/ppv_insert.sql\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_insert()\n"; > } > } > elsif ($d_funcToPerf eq "update") { > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > !$s_dbName || !$s_dbDriver || !$s_tblName || > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example:\n"; > print " Destination Database Name: -dd=pv26\n"; > print " Destination Database Driver: -d2=ODBC\n"; > print " Destination Table Name: -dt=p_falcon_projections\n"; > print " Source Database Name: -sd=dv26\n"; > print " Source Database Driver: -d1=Oracle\n"; > print " Source Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: > -ss=/usr/local/sql/ppv_select.sql\n"; > print " SQL Statement: > -ds=/usr/local/sql/ppv_update.sql\n"; > print " Source Function to Perform: -sf=select\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_select()\n"; > print "Calling sub_update()\n"; > } > } > elsif ($d_funcToPerf eq "insert") { > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > !$s_dbName || !$s_dbDriver || !$s_tblName || > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example:\n"; > print " Destination Database Name: -dd=pv26\n"; > print " Destination Database Driver: -d2=ODBC\n"; > print " Destination Table Name: -dt=p_falcon_projections\n"; > print " Source Database Name: -sd=dv26\n"; > print " Source Database Driver: -d1=Oracle\n"; > print " Source Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: > -ss=/usr/local/sql/ppv_select.sql\n"; > print " SQL Statement: > -ds=/usr/local/sql/ppv_insert.sql\n"; > print " Source Function to Perform: -sf=select\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_select()\n"; > print "Calling sub_insert()\n"; > } > } > elsif ($s_funcToPerf eq "select") { > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || > !$d_dbName || !$d_dbDriver || !$d_tblName || !$d_funcToPerf || > !$s_SQL || !$d_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example:\n"; > print " Destination Database Name: -dd=pv26\n"; > print " Destination Database Driver: -d2=ODBC\n"; > print " Destination Table Name: -dt=p_falcon_projections\n"; > print " Source Database Name: -sd=dv26\n"; > print " Source Database Driver: -d1=Oracle\n"; > print " Source Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: > -ss=/usr/local/sql/ppv_select.sql\n"; > print " SQL Statement: > -ds=/usr/local/sql/ppv_insert.sql\n"; > print " Source Function to Perform: -sf=select\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_select()\n"; > print "Calling sub_insert()\n"; > } > } > else { > print "ERROR: Unknown value for database action to perform.\n"; > exit(666); > } > > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Tuesday, March 14, 2006 3:28 PM > To: Michael Friedman > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Hi Peter, > > Surely you're trying to accomplish more than just running the SQL or you > would just read it in an feed it to DBI. There's no reason you couldn't > call to the database command shell: > > if(my $pid = fork) { > waitpid $pid; > } else { > close STDIN; > open STDIN, '<', 'foo.sql' or die $!; > exec 'mysql', $dbname or die $!; > } > > ... or something like that. If you want to use special features of the > database command shell or just cash in on its speed, this might be > handy. Of course, you don't want to try to read values from the > database back into Perl over a pipe between two processes... that's just > nasty. > > -scott > > > On 0, Michael Friedman wrote: > > Peter, > > > > What I did in that situation was write a couple of methods to read in > > the file, put it into an array, and then loop through the array and > > make each call. The good news is you only have to write that once and > > then you can reuse it... > > > > My only example is using Sybase::DBlib, though, not DBI, but the logic > > > would be the same. Sybase uses 'go' on a line by itself to end a SQL > > command, so we just use that to split up the lines in the file into > > commands into the array. > > > > You could make this a lot fancier, if you had the need, but it works > > for me. > > > > Good luck, > > -- Mike > > > > (sub db_run_script and sub db_run_command_list, below) > > > > sub db_run_script #($$) > > { > > my $dbh = shift; > > my $script = shift; > > my $saveresults = shift; > > > > open (SQL_SCRIPT, $script) || die "Could not open input file > $script: > > $!"; > > > > my @commands = (); > > my $j = 0; > > my ($line); > > > > # read script file into a variable (array of commands) > > while ($line = ) > > { > > if ($line =~ /^go/) > > { > > # make new command > > $j++; > > } > > elsif ($line =~ /^\s*$/) > > { > > # ignore blank lines > > } > > else > > { > > $commands[$j] .= $line; > > } > > } > > close SQL_SCRIPT; > > > > return db_run_command_list($dbh, \@commands, $saveresults); } > > > > sub db_run_command_list > > { > > my $dbh = shift; > > my $cmdlist = shift; > > my $saveresults = shift; > > > > my @resultlist; > > > > # run commands from array > > for $j(0..$#$cmdlist) > > { > > $dbh->dbcmd($cmdlist->[$j]); > > my $status; > > eval { > > $status = $dbh->dbsqlexec(); > > }; > > > > if ($@ || $status != SUCCEED) > > { > > # don't always die, because drop will fail > sometimes > > if ($cmdlist->[$j] =~ /drop/i) > > { > > warn "$cmdlist->[$j] failed.\n This is > OK - item probably didn't > > exist before installation.\n"; > > > > $dbh->dbcancel(); # so that we can move > on to the next command? > > } > > else > > { > > die "+++ Could not run command > $cmdlist->[$j]\nbecause of this > > problem:\n$@"; > > } > > } > > > > if (!$saveresults) { > > db_ignore_results($dbh); > > } else { > > # Count the total number of rows that were updated, > > # and capture the output of any SELECT statements > > # > > # Each update/insert statement will have its own > update > > # count (a separate call to DBCOUNT()) but we will > > # just add them all together > > my $totalupdatecount = 0; > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > my $rcount = $dbh->DBCOUNT(); > > if ($rcount != -1) { > > $totalupdatecount += $rcount; > > } > > > > my @res; > > while (@res = $dbh->dbnextrow()) { > > my @copyres = @res; # make a copy of the > array > > push @resultlist, \@copyres; > > } > > } > > > > push @resultlist, $totalupdatecount; > > } > > } > > > > if ($saveresults) { > > return \@resultlist; > > } else { > > return; > > } > > } > > > > > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > > > Hi, > > > > > > I know that you are able to issue a SQL statement within PERL DBI, > > > but is there anyway that I can issue an external SQL program? For > > > example, I have a SQL program called ppv_insert.sql that I would > > > like to execute within PERL DBI. > > > > > > Thanks in advance. > > > > > > Peter Loo > > > > > > > > > > > > > > > This E-mail message is for the sole use of the intended recipient > > > (s) and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > --------------------------------------------------------------------- > > Michael Friedman HighWire Press > > Phone: 650-725-1974 Stanford University > > FAX: 270-721-8034 > > --------------------------------------------------------------------- > > > > > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From Peter.Loo at source.wolterskluwer.com Wed Mar 15 12:19:07 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Wed, 15 Mar 2006 13:19:07 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95DB6@phxmail02.phx.ndchealth.com> Hi Scott, So will it be correct to assume that PERL DBI can not execute an SQL program? For example, I can do this with Korn shell: sqlplus -s /NOLOG << EOF connect ${DBUSER}/${DBPASS}@${DBCONN} @${MailProgFile}.sql Is this not possible in PERL DBI? Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Tuesday, March 14, 2006 9:36 PM To: Loo, Peter # PHX Cc: Michael Friedman; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Consider using GetOpt::Std, and most of the time you want this form of for: for my $thing (@things) { ... stuff with $thing ... } You can always try to read rows and trap errors. -scott On 0, "Loo, Peter # PHX" wrote: > > Thanks Michael and Scott. What I am trying to do is creating a > generic PERL program that will take in multiple arguments such as: > > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > switch ($flag) { > case "-dd" { $d_dbName = lc($value) } > case "-dt" { $d_tblName = lc($value) } > case "-ds" { $d_SQL = $value } > case "-sd" { $s_dbName = lc($value) } > case "-st" { $s_tblName = lc($value) } > case "-ss" { $s_SQL = $value } > case "-cp" { $commitPoint = lc($value) } > case "-sf" { $s_funcToPerf = lc($value) } > case "-df" { $d_funcToPerf = lc($value) } > case "-d1" { $s_dbDriver = lc($value) } > case "-d2" { $d_dbDriver = lc($value) } > else { print "Unknown flag: $flag\n" } > } > } > > Then execute accordingly, however, I would like to execute an external > SQL program that is passed to this generic program. In the case of an > external SQL program that does SELECT instead of UPDATE or INSERT, I > want to loop through the returned rows. Here is my first block of "if" > statement. > > if ($s_funcToPerf eq "update") { > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || > !$s_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example: \n"; > print " Database Name: -sd=dv26\n"; > print " Database Driver: -d1=Oracle\n"; > print " Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: -ss=/usr/local/sql/ppv_update.sql\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_update()\n"; > } > } > elsif ($s_funcToPerf eq "insert") { > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || > !$s_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example: \n"; > print " Database Name: -sd=dv26\n"; > print " Database Driver: -d1=Oracle\n"; > print " Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: -ss=/usr/local/sql/ppv_insert.sql\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_insert()\n"; > } > } > elsif ($d_funcToPerf eq "update") { > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > !$s_dbName || !$s_dbDriver || !$s_tblName || > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example:\n"; > print " Destination Database Name: -dd=pv26\n"; > print " Destination Database Driver: -d2=ODBC\n"; > print " Destination Table Name: -dt=p_falcon_projections\n"; > print " Source Database Name: -sd=dv26\n"; > print " Source Database Driver: -d1=Oracle\n"; > print " Source Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: > -ss=/usr/local/sql/ppv_select.sql\n"; > print " SQL Statement: > -ds=/usr/local/sql/ppv_update.sql\n"; > print " Source Function to Perform: -sf=select\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_select()\n"; > print "Calling sub_update()\n"; > } > } > elsif ($d_funcToPerf eq "insert") { > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > !$s_dbName || !$s_dbDriver || !$s_tblName || > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example:\n"; > print " Destination Database Name: -dd=pv26\n"; > print " Destination Database Driver: -d2=ODBC\n"; > print " Destination Table Name: -dt=p_falcon_projections\n"; > print " Source Database Name: -sd=dv26\n"; > print " Source Database Driver: -d1=Oracle\n"; > print " Source Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: > -ss=/usr/local/sql/ppv_select.sql\n"; > print " SQL Statement: > -ds=/usr/local/sql/ppv_insert.sql\n"; > print " Source Function to Perform: -sf=select\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_select()\n"; > print "Calling sub_insert()\n"; > } > } > elsif ($s_funcToPerf eq "select") { > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || > !$d_dbName || !$d_dbDriver || !$d_tblName || !$d_funcToPerf || > !$s_SQL || !$d_SQL) { > print "ERROR: Not enough arguments. Require arguments are:\n"; > print "Example:\n"; > print " Destination Database Name: -dd=pv26\n"; > print " Destination Database Driver: -d2=ODBC\n"; > print " Destination Table Name: -dt=p_falcon_projections\n"; > print " Source Database Name: -sd=dv26\n"; > print " Source Database Driver: -d1=Oracle\n"; > print " Source Table Name: -st=p_falcon_projections\n"; > print " SQL Statement: > -ss=/usr/local/sql/ppv_select.sql\n"; > print " SQL Statement: > -ds=/usr/local/sql/ppv_insert.sql\n"; > print " Source Function to Perform: -sf=select\n"; > print " Commit Point: -cp=5000\n"; > exit(666); > } > else { > print "Calling sub_select()\n"; > print "Calling sub_insert()\n"; > } > } > else { > print "ERROR: Unknown value for database action to perform.\n"; > exit(666); > } > > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Tuesday, March 14, 2006 3:28 PM > To: Michael Friedman > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Hi Peter, > > Surely you're trying to accomplish more than just running the SQL or > you would just read it in an feed it to DBI. There's no reason you > couldn't call to the database command shell: > > if(my $pid = fork) { > waitpid $pid; > } else { > close STDIN; > open STDIN, '<', 'foo.sql' or die $!; > exec 'mysql', $dbname or die $!; > } > > ... or something like that. If you want to use special features of > the database command shell or just cash in on its speed, this might be > handy. Of course, you don't want to try to read values from the > database back into Perl over a pipe between two processes... that's > just nasty. > > -scott > > > On 0, Michael Friedman wrote: > > Peter, > > > > What I did in that situation was write a couple of methods to read > > in the file, put it into an array, and then loop through the array > > and make each call. The good news is you only have to write that > > once and then you can reuse it... > > > > My only example is using Sybase::DBlib, though, not DBI, but the > > logic > > > would be the same. Sybase uses 'go' on a line by itself to end a SQL > > command, so we just use that to split up the lines in the file into > > commands into the array. > > > > You could make this a lot fancier, if you had the need, but it works > > for me. > > > > Good luck, > > -- Mike > > > > (sub db_run_script and sub db_run_command_list, below) > > > > sub db_run_script #($$) > > { > > my $dbh = shift; > > my $script = shift; > > my $saveresults = shift; > > > > open (SQL_SCRIPT, $script) || die "Could not open input file > $script: > > $!"; > > > > my @commands = (); > > my $j = 0; > > my ($line); > > > > # read script file into a variable (array of commands) > > while ($line = ) > > { > > if ($line =~ /^go/) > > { > > # make new command > > $j++; > > } > > elsif ($line =~ /^\s*$/) > > { > > # ignore blank lines > > } > > else > > { > > $commands[$j] .= $line; > > } > > } > > close SQL_SCRIPT; > > > > return db_run_command_list($dbh, \@commands, $saveresults); } > > > > sub db_run_command_list > > { > > my $dbh = shift; > > my $cmdlist = shift; > > my $saveresults = shift; > > > > my @resultlist; > > > > # run commands from array > > for $j(0..$#$cmdlist) > > { > > $dbh->dbcmd($cmdlist->[$j]); > > my $status; > > eval { > > $status = $dbh->dbsqlexec(); > > }; > > > > if ($@ || $status != SUCCEED) > > { > > # don't always die, because drop will fail > sometimes > > if ($cmdlist->[$j] =~ /drop/i) > > { > > warn "$cmdlist->[$j] failed.\n This is > OK - item probably didn't > > exist before installation.\n"; > > > > $dbh->dbcancel(); # so that we can move > on to the next command? > > } > > else > > { > > die "+++ Could not run command > $cmdlist->[$j]\nbecause of this > > problem:\n$@"; > > } > > } > > > > if (!$saveresults) { > > db_ignore_results($dbh); > > } else { > > # Count the total number of rows that were updated, > > # and capture the output of any SELECT statements > > # > > # Each update/insert statement will have its own > update > > # count (a separate call to DBCOUNT()) but we will > > # just add them all together > > my $totalupdatecount = 0; > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > my $rcount = $dbh->DBCOUNT(); > > if ($rcount != -1) { > > $totalupdatecount += $rcount; > > } > > > > my @res; > > while (@res = $dbh->dbnextrow()) { > > my @copyres = @res; # make a copy of the > array > > push @resultlist, \@copyres; > > } > > } > > > > push @resultlist, $totalupdatecount; > > } > > } > > > > if ($saveresults) { > > return \@resultlist; > > } else { > > return; > > } > > } > > > > > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > > > Hi, > > > > > > I know that you are able to issue a SQL statement within PERL DBI, > > > but is there anyway that I can issue an external SQL program? For > > > example, I have a SQL program called ppv_insert.sql that I would > > > like to execute within PERL DBI. > > > > > > Thanks in advance. > > > > > > Peter Loo > > > > > > > > > > > > > > > This E-mail message is for the sole use of the intended recipient > > > (s) and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > > If you are not the intended recipient, please contact the sender > > > by reply E-mail, and destroy all copies of the original message. > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > --------------------------------------------------------------------- > > Michael Friedman HighWire Press > > Phone: 650-725-1974 Stanford University > > FAX: 270-721-8034 > > -------------------------------------------------------------------- > > - > > > > > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From awwaiid at thelackthereof.org Wed Mar 15 12:51:34 2006 From: awwaiid at thelackthereof.org (Brock) Date: Wed, 15 Mar 2006 13:51:34 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95DB6@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95DB6@phxmail02.phx.ndchealth.com> Message-ID: <20060315205134.GL4287@thelackthereof.org> Sure... what you are doing here is opening an external program and piping it data on STDIN. There are several ways to do this in Perl... Here's one (a rough guess): # Initiate those vars (pull them from %ENV?) my $dbuser = ...; my $dbpass = ...; my $dbconn = ...; my $mailProgFile = ...; open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; print $sqlplus <<" HERE"; connect ${dbuser}/${dbpass}@${dbconn} @${mailProgFile}.sql HERE More or less. Eh? --Brock On 2006.03.15.13.19, Loo, Peter # PHX wrote: | | Hi Scott, | | So will it be correct to assume that PERL DBI can not execute an SQL | program? For example, I can do this with Korn shell: | | sqlplus -s /NOLOG << EOF | connect ${DBUSER}/${DBPASS}@${DBCONN} | @${MailProgFile}.sql | | Is this not possible in PERL DBI? | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 | | -----Original Message----- | From: Scott Walters [mailto:scott at illogics.org] | Sent: Tuesday, March 14, 2006 9:36 PM | To: Loo, Peter # PHX | Cc: Michael Friedman; phoenix-pm at pm.org | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | Consider using GetOpt::Std, and most of the time you want this form of | for: | | for my $thing (@things) { ... stuff with $thing ... } | | You can always try to read rows and trap errors. | | -scott | | On 0, "Loo, Peter # PHX" wrote: | > | > Thanks Michael and Scott. What I am trying to do is creating a | > generic PERL program that will take in multiple arguments such as: | > | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); | > switch ($flag) { | > case "-dd" { $d_dbName = lc($value) } | > case "-dt" { $d_tblName = lc($value) } | > case "-ds" { $d_SQL = $value } | > case "-sd" { $s_dbName = lc($value) } | > case "-st" { $s_tblName = lc($value) } | > case "-ss" { $s_SQL = $value } | > case "-cp" { $commitPoint = lc($value) } | > case "-sf" { $s_funcToPerf = lc($value) } | > case "-df" { $d_funcToPerf = lc($value) } | > case "-d1" { $s_dbDriver = lc($value) } | > case "-d2" { $d_dbDriver = lc($value) } | > else { print "Unknown flag: $flag\n" } | > } | > } | > | > Then execute accordingly, however, I would like to execute an external | | > SQL program that is passed to this generic program. In the case of an | | > external SQL program that does SELECT instead of UPDATE or INSERT, I | > want to loop through the returned rows. Here is my first block of | "if" | > statement. | > | > if ($s_funcToPerf eq "update") { | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || | > !$s_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example: \n"; | > print " Database Name: -sd=dv26\n"; | > print " Database Driver: -d1=Oracle\n"; | > print " Table Name: -st=p_falcon_projections\n"; | > print " SQL Statement: -ss=/usr/local/sql/ppv_update.sql\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_update()\n"; | > } | > } | > elsif ($s_funcToPerf eq "insert") { | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || | > !$s_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example: \n"; | > print " Database Name: -sd=dv26\n"; | > print " Database Driver: -d1=Oracle\n"; | > print " Table Name: -st=p_falcon_projections\n"; | > print " SQL Statement: -ss=/usr/local/sql/ppv_insert.sql\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_insert()\n"; | > } | > } | > elsif ($d_funcToPerf eq "update") { | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | > !$s_dbName || !$s_dbDriver || !$s_tblName || | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example:\n"; | > print " Destination Database Name: -dd=pv26\n"; | > print " Destination Database Driver: -d2=ODBC\n"; | > print " Destination Table Name: | -dt=p_falcon_projections\n"; | > print " Source Database Name: -sd=dv26\n"; | > print " Source Database Driver: -d1=Oracle\n"; | > print " Source Table Name: | -st=p_falcon_projections\n"; | > print " SQL Statement: | > -ss=/usr/local/sql/ppv_select.sql\n"; | > print " SQL Statement: | > -ds=/usr/local/sql/ppv_update.sql\n"; | > print " Source Function to Perform: -sf=select\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_select()\n"; | > print "Calling sub_update()\n"; | > } | > } | > elsif ($d_funcToPerf eq "insert") { | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | > !$s_dbName || !$s_dbDriver || !$s_tblName || | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example:\n"; | > print " Destination Database Name: -dd=pv26\n"; | > print " Destination Database Driver: -d2=ODBC\n"; | > print " Destination Table Name: | -dt=p_falcon_projections\n"; | > print " Source Database Name: -sd=dv26\n"; | > print " Source Database Driver: -d1=Oracle\n"; | > print " Source Table Name: | -st=p_falcon_projections\n"; | > print " SQL Statement: | > -ss=/usr/local/sql/ppv_select.sql\n"; | > print " SQL Statement: | > -ds=/usr/local/sql/ppv_insert.sql\n"; | > print " Source Function to Perform: -sf=select\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_select()\n"; | > print "Calling sub_insert()\n"; | > } | > } | > elsif ($s_funcToPerf eq "select") { | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || | > !$d_dbName || !$d_dbDriver || !$d_tblName || !$d_funcToPerf || | > !$s_SQL || !$d_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example:\n"; | > print " Destination Database Name: -dd=pv26\n"; | > print " Destination Database Driver: -d2=ODBC\n"; | > print " Destination Table Name: | -dt=p_falcon_projections\n"; | > print " Source Database Name: -sd=dv26\n"; | > print " Source Database Driver: -d1=Oracle\n"; | > print " Source Table Name: | -st=p_falcon_projections\n"; | > print " SQL Statement: | > -ss=/usr/local/sql/ppv_select.sql\n"; | > print " SQL Statement: | > -ds=/usr/local/sql/ppv_insert.sql\n"; | > print " Source Function to Perform: -sf=select\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_select()\n"; | > print "Calling sub_insert()\n"; | > } | > } | > else { | > print "ERROR: Unknown value for database action to perform.\n"; | > exit(666); | > } | > | > | > Peter Loo | > Wolters Kluwer Health | > (602) 381-9553 | > | > -----Original Message----- | > From: Scott Walters [mailto:scott at illogics.org] | > Sent: Tuesday, March 14, 2006 3:28 PM | > To: Michael Friedman | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | > | > Hi Peter, | > | > Surely you're trying to accomplish more than just running the SQL or | > you would just read it in an feed it to DBI. There's no reason you | > couldn't call to the database command shell: | > | > if(my $pid = fork) { | > waitpid $pid; | > } else { | > close STDIN; | > open STDIN, '<', 'foo.sql' or die $!; | > exec 'mysql', $dbname or die $!; | > } | > | > ... or something like that. If you want to use special features of | > the database command shell or just cash in on its speed, this might be | | > handy. Of course, you don't want to try to read values from the | > database back into Perl over a pipe between two processes... that's | > just nasty. | > | > -scott | > | > | > On 0, Michael Friedman wrote: | > > Peter, | > > | > > What I did in that situation was write a couple of methods to read | > > in the file, put it into an array, and then loop through the array | > > and make each call. The good news is you only have to write that | > > once and then you can reuse it... | > > | > > My only example is using Sybase::DBlib, though, not DBI, but the | > > logic | > | > > would be the same. Sybase uses 'go' on a line by itself to end a SQL | | > > command, so we just use that to split up the lines in the file into | > > commands into the array. | > > | > > You could make this a lot fancier, if you had the need, but it works | | > > for me. | > > | > > Good luck, | > > -- Mike | > > | > > (sub db_run_script and sub db_run_command_list, below) | > > | > > sub db_run_script #($$) | > > { | > > my $dbh = shift; | > > my $script = shift; | > > my $saveresults = shift; | > > | > > open (SQL_SCRIPT, $script) || die "Could not open input file | > $script: | > > $!"; | > > | > > my @commands = (); | > > my $j = 0; | > > my ($line); | > > | > > # read script file into a variable (array of commands) | > > while ($line = ) | > > { | > > if ($line =~ /^go/) | > > { | > > # make new command | > > $j++; | > > } | > > elsif ($line =~ /^\s*$/) | > > { | > > # ignore blank lines | > > } | > > else | > > { | > > $commands[$j] .= $line; | > > } | > > } | > > close SQL_SCRIPT; | > > | > > return db_run_command_list($dbh, \@commands, $saveresults); } | > > | > > sub db_run_command_list | > > { | > > my $dbh = shift; | > > my $cmdlist = shift; | > > my $saveresults = shift; | > > | > > my @resultlist; | > > | > > # run commands from array | > > for $j(0..$#$cmdlist) | > > { | > > $dbh->dbcmd($cmdlist->[$j]); | > > my $status; | > > eval { | > > $status = $dbh->dbsqlexec(); | > > }; | > > | > > if ($@ || $status != SUCCEED) | > > { | > > # don't always die, because drop will fail | > sometimes | > > if ($cmdlist->[$j] =~ /drop/i) | > > { | > > warn "$cmdlist->[$j] failed.\n This is | > OK - item probably didn't | > > exist before installation.\n"; | > > | > > $dbh->dbcancel(); # so that we can move | > on to the next command? | > > } | > > else | > > { | > > die "+++ Could not run command | > $cmdlist->[$j]\nbecause of this | > > problem:\n$@"; | > > } | > > } | > > | > > if (!$saveresults) { | > > db_ignore_results($dbh); | > > } else { | > > # Count the total number of rows that were updated, | > > # and capture the output of any SELECT statements | > > # | > > # Each update/insert statement will have its own | > update | > > # count (a separate call to DBCOUNT()) but we will | > > # just add them all together | > > my $totalupdatecount = 0; | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { | > > my $rcount = $dbh->DBCOUNT(); | > > if ($rcount != -1) { | > > $totalupdatecount += $rcount; | > > } | > > | > > my @res; | > > while (@res = $dbh->dbnextrow()) { | > > my @copyres = @res; # make a copy of the | > array | > > push @resultlist, \@copyres; | > > } | > > } | > > | > > push @resultlist, $totalupdatecount; | > > } | > > } | > > | > > if ($saveresults) { | > > return \@resultlist; | > > } else { | > > return; | > > } | > > } | > > | > > | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: | > > | > > > Hi, | > > > | > > > I know that you are able to issue a SQL statement within PERL DBI, | | > > > but is there anyway that I can issue an external SQL program? For | | > > > example, I have a SQL program called ppv_insert.sql that I would | > > > like to execute within PERL DBI. | > > > | > > > Thanks in advance. | > > > | > > > Peter Loo | > > > | > > > | > > > | > > > | > > > This E-mail message is for the sole use of the intended recipient | > > > (s) and may contain confidential and privileged information. Any | > > > unauthorized review, use, disclosure or distribution is | prohibited. | > | > > > If you are not the intended recipient, please contact the sender | > > > by reply E-mail, and destroy all copies of the original message. | > > > _______________________________________________ | > > > Phoenix-pm mailing list | > > > Phoenix-pm at pm.org | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm | > > | > > | --------------------------------------------------------------------- | > > Michael Friedman HighWire Press | > > Phone: 650-725-1974 Stanford University | > > FAX: 270-721-8034 | | > > -------------------------------------------------------------------- | > > - | > > | > > | > > _______________________________________________ | > > Phoenix-pm mailing list | > > Phoenix-pm at pm.org | > > http://mail.pm.org/mailman/listinfo/phoenix-pm | > | > | > This E-mail message is for the sole use of the intended recipient(s) | > and may contain confidential and privileged information. Any | > unauthorized review, use, disclosure or distribution is prohibited. | > If you are not the intended recipient, please contact the sender by | > reply E-mail, and destroy all copies of the original message. | > | > | > This E-mail message is for the sole use of the intended recipient(s) | and may contain confidential and privileged information. Any | unauthorized review, use, disclosure or distribution is prohibited. If | you are not the intended recipient, please contact the sender by reply | E-mail, and destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) and | may contain confidential and privileged information. Any unauthorized | review, use, disclosure or distribution is prohibited. If you are not | the intended recipient, please contact the sender by reply E-mail, and | destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From Peter.Loo at source.wolterskluwer.com Wed Mar 15 12:55:35 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Wed, 15 Mar 2006 13:55:35 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95DD6@phxmail02.phx.ndchealth.com> Hi Brock, Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't be efficient. Secondly, what if you have a "SELECT" statement in the .sql program and if you want to loop through each row? Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Brock [mailto:awwaiid at thelackthereof.org] Sent: Wednesday, March 15, 2006 1:52 PM To: Loo, Peter # PHX Cc: Scott Walters; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Sure... what you are doing here is opening an external program and piping it data on STDIN. There are several ways to do this in Perl... Here's one (a rough guess): # Initiate those vars (pull them from %ENV?) my $dbuser = ...; my $dbpass = ...; my $dbconn = ...; my $mailProgFile = ...; open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; print $sqlplus <<" HERE"; connect ${dbuser}/${dbpass}@${dbconn} @${mailProgFile}.sql HERE More or less. Eh? --Brock On 2006.03.15.13.19, Loo, Peter # PHX wrote: | | Hi Scott, | | So will it be correct to assume that PERL DBI can not execute an SQL | program? For example, I can do this with Korn shell: | | sqlplus -s /NOLOG << EOF | connect ${DBUSER}/${DBPASS}@${DBCONN} | @${MailProgFile}.sql | | Is this not possible in PERL DBI? | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 | | -----Original Message----- | From: Scott Walters [mailto:scott at illogics.org] | Sent: Tuesday, March 14, 2006 9:36 PM | To: Loo, Peter # PHX | Cc: Michael Friedman; phoenix-pm at pm.org | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | Consider using GetOpt::Std, and most of the time you want this form of | for: | | for my $thing (@things) { ... stuff with $thing ... } | | You can always try to read rows and trap errors. | | -scott | | On 0, "Loo, Peter # PHX" wrote: | > | > Thanks Michael and Scott. What I am trying to do is creating a | > generic PERL program that will take in multiple arguments such as: | > | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); | > switch ($flag) { | > case "-dd" { $d_dbName = lc($value) } | > case "-dt" { $d_tblName = lc($value) } | > case "-ds" { $d_SQL = $value } | > case "-sd" { $s_dbName = lc($value) } | > case "-st" { $s_tblName = lc($value) } | > case "-ss" { $s_SQL = $value } | > case "-cp" { $commitPoint = lc($value) } | > case "-sf" { $s_funcToPerf = lc($value) } | > case "-df" { $d_funcToPerf = lc($value) } | > case "-d1" { $s_dbDriver = lc($value) } | > case "-d2" { $d_dbDriver = lc($value) } | > else { print "Unknown flag: $flag\n" } | > } | > } | > | > Then execute accordingly, however, I would like to execute an | > external | | > SQL program that is passed to this generic program. In the case of | > an | | > external SQL program that does SELECT instead of UPDATE or INSERT, I | > want to loop through the returned rows. Here is my first block of | "if" | > statement. | > | > if ($s_funcToPerf eq "update") { | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | > || | > !$s_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example: \n"; | > print " Database Name: -sd=dv26\n"; | > print " Database Driver: -d1=Oracle\n"; | > print " Table Name: -st=p_falcon_projections\n"; | > print " SQL Statement: -ss=/usr/local/sql/ppv_update.sql\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_update()\n"; | > } | > } | > elsif ($s_funcToPerf eq "insert") { | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | > || | > !$s_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example: \n"; | > print " Database Name: -sd=dv26\n"; | > print " Database Driver: -d1=Oracle\n"; | > print " Table Name: -st=p_falcon_projections\n"; | > print " SQL Statement: -ss=/usr/local/sql/ppv_insert.sql\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_insert()\n"; | > } | > } | > elsif ($d_funcToPerf eq "update") { | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | > !$s_dbName || !$s_dbDriver || !$s_tblName || | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example:\n"; | > print " Destination Database Name: -dd=pv26\n"; | > print " Destination Database Driver: -d2=ODBC\n"; | > print " Destination Table Name: | -dt=p_falcon_projections\n"; | > print " Source Database Name: -sd=dv26\n"; | > print " Source Database Driver: -d1=Oracle\n"; | > print " Source Table Name: | -st=p_falcon_projections\n"; | > print " SQL Statement: | > -ss=/usr/local/sql/ppv_select.sql\n"; | > print " SQL Statement: | > -ds=/usr/local/sql/ppv_update.sql\n"; | > print " Source Function to Perform: -sf=select\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_select()\n"; | > print "Calling sub_update()\n"; | > } | > } | > elsif ($d_funcToPerf eq "insert") { | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | > !$s_dbName || !$s_dbDriver || !$s_tblName || | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example:\n"; | > print " Destination Database Name: -dd=pv26\n"; | > print " Destination Database Driver: -d2=ODBC\n"; | > print " Destination Table Name: | -dt=p_falcon_projections\n"; | > print " Source Database Name: -sd=dv26\n"; | > print " Source Database Driver: -d1=Oracle\n"; | > print " Source Table Name: | -st=p_falcon_projections\n"; | > print " SQL Statement: | > -ss=/usr/local/sql/ppv_select.sql\n"; | > print " SQL Statement: | > -ds=/usr/local/sql/ppv_insert.sql\n"; | > print " Source Function to Perform: -sf=select\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_select()\n"; | > print "Calling sub_insert()\n"; | > } | > } | > elsif ($s_funcToPerf eq "select") { | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint || | > !$d_dbName || !$d_dbDriver || !$d_tblName || !$d_funcToPerf || | > !$s_SQL || !$d_SQL) { | > print "ERROR: Not enough arguments. Require arguments are:\n"; | > print "Example:\n"; | > print " Destination Database Name: -dd=pv26\n"; | > print " Destination Database Driver: -d2=ODBC\n"; | > print " Destination Table Name: | -dt=p_falcon_projections\n"; | > print " Source Database Name: -sd=dv26\n"; | > print " Source Database Driver: -d1=Oracle\n"; | > print " Source Table Name: | -st=p_falcon_projections\n"; | > print " SQL Statement: | > -ss=/usr/local/sql/ppv_select.sql\n"; | > print " SQL Statement: | > -ds=/usr/local/sql/ppv_insert.sql\n"; | > print " Source Function to Perform: -sf=select\n"; | > print " Commit Point: -cp=5000\n"; | > exit(666); | > } | > else { | > print "Calling sub_select()\n"; | > print "Calling sub_insert()\n"; | > } | > } | > else { | > print "ERROR: Unknown value for database action to perform.\n"; | > exit(666); | > } | > | > | > Peter Loo | > Wolters Kluwer Health | > (602) 381-9553 | > | > -----Original Message----- | > From: Scott Walters [mailto:scott at illogics.org] | > Sent: Tuesday, March 14, 2006 3:28 PM | > To: Michael Friedman | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | > | > Hi Peter, | > | > Surely you're trying to accomplish more than just running the SQL or | > you would just read it in an feed it to DBI. There's no reason you | > couldn't call to the database command shell: | > | > if(my $pid = fork) { | > waitpid $pid; | > } else { | > close STDIN; | > open STDIN, '<', 'foo.sql' or die $!; | > exec 'mysql', $dbname or die $!; | > } | > | > ... or something like that. If you want to use special features of | > the database command shell or just cash in on its speed, this might | > be | | > handy. Of course, you don't want to try to read values from the | > database back into Perl over a pipe between two processes... that's | > just nasty. | > | > -scott | > | > | > On 0, Michael Friedman wrote: | > > Peter, | > > | > > What I did in that situation was write a couple of methods to read | > > in the file, put it into an array, and then loop through the array | > > and make each call. The good news is you only have to write that | > > once and then you can reuse it... | > > | > > My only example is using Sybase::DBlib, though, not DBI, but the | > > logic | > | > > would be the same. Sybase uses 'go' on a line by itself to end a | > > SQL | | > > command, so we just use that to split up the lines in the file | > > into commands into the array. | > > | > > You could make this a lot fancier, if you had the need, but it | > > works | | > > for me. | > > | > > Good luck, | > > -- Mike | > > | > > (sub db_run_script and sub db_run_command_list, below) | > > | > > sub db_run_script #($$) | > > { | > > my $dbh = shift; | > > my $script = shift; | > > my $saveresults = shift; | > > | > > open (SQL_SCRIPT, $script) || die "Could not open input file | > $script: | > > $!"; | > > | > > my @commands = (); | > > my $j = 0; | > > my ($line); | > > | > > # read script file into a variable (array of commands) | > > while ($line = ) | > > { | > > if ($line =~ /^go/) | > > { | > > # make new command | > > $j++; | > > } | > > elsif ($line =~ /^\s*$/) | > > { | > > # ignore blank lines | > > } | > > else | > > { | > > $commands[$j] .= $line; | > > } | > > } | > > close SQL_SCRIPT; | > > | > > return db_run_command_list($dbh, \@commands, $saveresults); } | > > | > > sub db_run_command_list | > > { | > > my $dbh = shift; | > > my $cmdlist = shift; | > > my $saveresults = shift; | > > | > > my @resultlist; | > > | > > # run commands from array | > > for $j(0..$#$cmdlist) | > > { | > > $dbh->dbcmd($cmdlist->[$j]); | > > my $status; | > > eval { | > > $status = $dbh->dbsqlexec(); | > > }; | > > | > > if ($@ || $status != SUCCEED) | > > { | > > # don't always die, because drop will fail | > sometimes | > > if ($cmdlist->[$j] =~ /drop/i) | > > { | > > warn "$cmdlist->[$j] failed.\n This is | > OK - item probably didn't | > > exist before installation.\n"; | > > | > > $dbh->dbcancel(); # so that we can move | > on to the next command? | > > } | > > else | > > { | > > die "+++ Could not run command | > $cmdlist->[$j]\nbecause of this | > > problem:\n$@"; | > > } | > > } | > > | > > if (!$saveresults) { | > > db_ignore_results($dbh); | > > } else { | > > # Count the total number of rows that were updated, | > > # and capture the output of any SELECT statements | > > # | > > # Each update/insert statement will have its own | > update | > > # count (a separate call to DBCOUNT()) but we will | > > # just add them all together | > > my $totalupdatecount = 0; | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { | > > my $rcount = $dbh->DBCOUNT(); | > > if ($rcount != -1) { | > > $totalupdatecount += $rcount; | > > } | > > | > > my @res; | > > while (@res = $dbh->dbnextrow()) { | > > my @copyres = @res; # make a copy of the | > array | > > push @resultlist, \@copyres; | > > } | > > } | > > | > > push @resultlist, $totalupdatecount; | > > } | > > } | > > | > > if ($saveresults) { | > > return \@resultlist; | > > } else { | > > return; | > > } | > > } | > > | > > | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: | > > | > > > Hi, | > > > | > > > I know that you are able to issue a SQL statement within PERL | > > > DBI, | | > > > but is there anyway that I can issue an external SQL program? | > > > For | | > > > example, I have a SQL program called ppv_insert.sql that I would | > > > like to execute within PERL DBI. | > > > | > > > Thanks in advance. | > > > | > > > Peter Loo | > > > | > > > | > > > | > > > | > > > This E-mail message is for the sole use of the intended | > > > recipient | > > > (s) and may contain confidential and privileged information. | > > > Any unauthorized review, use, disclosure or distribution is | prohibited. | > | > > > If you are not the intended recipient, please contact the sender | > > > by reply E-mail, and destroy all copies of the original message. | > > > _______________________________________________ | > > > Phoenix-pm mailing list | > > > Phoenix-pm at pm.org | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm | > > | > > | --------------------------------------------------------------------- | > > Michael Friedman HighWire Press | > > Phone: 650-725-1974 Stanford University | > > FAX: 270-721-8034 | | > > ------------------------------------------------------------------ | > > -- | > > - | > > | > > | > > _______________________________________________ | > > Phoenix-pm mailing list | > > Phoenix-pm at pm.org | > > http://mail.pm.org/mailman/listinfo/phoenix-pm | > | > | > This E-mail message is for the sole use of the intended recipient(s) | > and may contain confidential and privileged information. Any | > unauthorized review, use, disclosure or distribution is prohibited. | > If you are not the intended recipient, please contact the sender by | > reply E-mail, and destroy all copies of the original message. | > | > | > This E-mail message is for the sole use of the intended recipient(s) | and may contain confidential and privileged information. Any | unauthorized review, use, disclosure or distribution is prohibited. | If you are not the intended recipient, please contact the sender by | reply E-mail, and destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) | and may contain confidential and privileged information. Any | unauthorized review, use, disclosure or distribution is prohibited. | If you are not the intended recipient, please contact the sender by | reply E-mail, and destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From awwaiid at thelackthereof.org Wed Mar 15 13:01:46 2006 From: awwaiid at thelackthereof.org (Brock) Date: Wed, 15 Mar 2006 14:01:46 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95DD6@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95DD6@phxmail02.phx.ndchealth.com> Message-ID: <20060315210146.GM4287@thelackthereof.org> Using DBI is completely different then piping things to sqlplus. I was just doing a direct translation of your example, not really trying to imply that it was a good idea. Perl DBI does NOT use sqlplus as the driver. Unfortunately your question doesn't make much sense... if your .sql has a select it could have many selects and it could have all sorts of things. The problem is that how can your program know what it contains? It seems what you need to do is put the select like queries directly into perl, as we demonstrated to you in earlier emails. Doing the whole fetchrow_arrayref thing that you were already doing. Make sense? --Brock On 2006.03.15.13.55, Loo, Peter # PHX wrote: | | Hi Brock, | | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't be | efficient. Secondly, what if you have a "SELECT" statement in the .sql | program and if you want to loop through each row? | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 | | -----Original Message----- | From: Brock [mailto:awwaiid at thelackthereof.org] | Sent: Wednesday, March 15, 2006 1:52 PM | To: Loo, Peter # PHX | Cc: Scott Walters; phoenix-pm at pm.org | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | Sure... what you are doing here is opening an external program and | piping it data on STDIN. There are several ways to do this in Perl... | | Here's one (a rough guess): | | # Initiate those vars (pull them from %ENV?) | my $dbuser = ...; | my $dbpass = ...; | my $dbconn = ...; | my $mailProgFile = ...; | | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; | print $sqlplus <<" HERE"; | connect ${dbuser}/${dbpass}@${dbconn} | @${mailProgFile}.sql | HERE | | More or less. Eh? | | --Brock | | On 2006.03.15.13.19, Loo, Peter # PHX wrote: | | | | Hi Scott, | | | | So will it be correct to assume that PERL DBI can not execute an SQL | | program? For example, I can do this with Korn shell: | | | | sqlplus -s /NOLOG << EOF | | connect ${DBUSER}/${DBPASS}@${DBCONN} | | @${MailProgFile}.sql | | | | Is this not possible in PERL DBI? | | | | Peter Loo | | Wolters Kluwer Health | | (602) 381-9553 | | | | -----Original Message----- | | From: Scott Walters [mailto:scott at illogics.org] | | Sent: Tuesday, March 14, 2006 9:36 PM | | To: Loo, Peter # PHX | | Cc: Michael Friedman; phoenix-pm at pm.org | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | | | Consider using GetOpt::Std, and most of the time you want this form of | | for: | | | | for my $thing (@things) { ... stuff with $thing ... } | | | | You can always try to read rows and trap errors. | | | | -scott | | | | On 0, "Loo, Peter # PHX" wrote: | | > | | > Thanks Michael and Scott. What I am trying to do is creating a | | > generic PERL program that will take in multiple arguments such as: | | > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); | | > switch ($flag) { | | > case "-dd" { $d_dbName = lc($value) } | | > case "-dt" { $d_tblName = lc($value) } | | > case "-ds" { $d_SQL = $value } | | > case "-sd" { $s_dbName = lc($value) } | | > case "-st" { $s_tblName = lc($value) } | | > case "-ss" { $s_SQL = $value } | | > case "-cp" { $commitPoint = lc($value) } | | > case "-sf" { $s_funcToPerf = lc($value) } | | > case "-df" { $d_funcToPerf = lc($value) } | | > case "-d1" { $s_dbDriver = lc($value) } | | > case "-d2" { $d_dbDriver = lc($value) } | | > else { print "Unknown flag: $flag\n" } | | > } | | > } | | > | | > Then execute accordingly, however, I would like to execute an | | > external | | | | > SQL program that is passed to this generic program. In the case of | | > an | | | | > external SQL program that does SELECT instead of UPDATE or INSERT, I | | | > want to loop through the returned rows. Here is my first block of | | "if" | | > statement. | | > | | > if ($s_funcToPerf eq "update") { | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | | > || | | > !$s_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example: \n"; | | > print " Database Name: -sd=dv26\n"; | | > print " Database Driver: -d1=Oracle\n"; | | > print " Table Name: -st=p_falcon_projections\n"; | | > print " SQL Statement: | -ss=/usr/local/sql/ppv_update.sql\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_update()\n"; | | > } | | > } | | > elsif ($s_funcToPerf eq "insert") { | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | | > || | | > !$s_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example: \n"; | | > print " Database Name: -sd=dv26\n"; | | > print " Database Driver: -d1=Oracle\n"; | | > print " Table Name: -st=p_falcon_projections\n"; | | > print " SQL Statement: | -ss=/usr/local/sql/ppv_insert.sql\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_insert()\n"; | | > } | | > } | | > elsif ($d_funcToPerf eq "update") { | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | | > !$s_dbName || !$s_dbDriver || !$s_tblName || | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example:\n"; | | > print " Destination Database Name: -dd=pv26\n"; | | > print " Destination Database Driver: -d2=ODBC\n"; | | > print " Destination Table Name: | | -dt=p_falcon_projections\n"; | | > print " Source Database Name: -sd=dv26\n"; | | > print " Source Database Driver: -d1=Oracle\n"; | | > print " Source Table Name: | | -st=p_falcon_projections\n"; | | > print " SQL Statement: | | > -ss=/usr/local/sql/ppv_select.sql\n"; | | > print " SQL Statement: | | > -ds=/usr/local/sql/ppv_update.sql\n"; | | > print " Source Function to Perform: -sf=select\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_select()\n"; | | > print "Calling sub_update()\n"; | | > } | | > } | | > elsif ($d_funcToPerf eq "insert") { | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | | > !$s_dbName || !$s_dbDriver || !$s_tblName || | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example:\n"; | | > print " Destination Database Name: -dd=pv26\n"; | | > print " Destination Database Driver: -d2=ODBC\n"; | | > print " Destination Table Name: | | -dt=p_falcon_projections\n"; | | > print " Source Database Name: -sd=dv26\n"; | | > print " Source Database Driver: -d1=Oracle\n"; | | > print " Source Table Name: | | -st=p_falcon_projections\n"; | | > print " SQL Statement: | | > -ss=/usr/local/sql/ppv_select.sql\n"; | | > print " SQL Statement: | | > -ds=/usr/local/sql/ppv_insert.sql\n"; | | > print " Source Function to Perform: -sf=select\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_select()\n"; | | > print "Calling sub_insert()\n"; | | > } | | > } | | > elsif ($s_funcToPerf eq "select") { | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | || | | > !$d_dbName || !$d_dbDriver || !$d_tblName || !$d_funcToPerf | || | | > !$s_SQL || !$d_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example:\n"; | | > print " Destination Database Name: -dd=pv26\n"; | | > print " Destination Database Driver: -d2=ODBC\n"; | | > print " Destination Table Name: | | -dt=p_falcon_projections\n"; | | > print " Source Database Name: -sd=dv26\n"; | | > print " Source Database Driver: -d1=Oracle\n"; | | > print " Source Table Name: | | -st=p_falcon_projections\n"; | | > print " SQL Statement: | | > -ss=/usr/local/sql/ppv_select.sql\n"; | | > print " SQL Statement: | | > -ds=/usr/local/sql/ppv_insert.sql\n"; | | > print " Source Function to Perform: -sf=select\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_select()\n"; | | > print "Calling sub_insert()\n"; | | > } | | > } | | > else { | | > print "ERROR: Unknown value for database action to perform.\n"; | | > exit(666); | | > } | | > | | > | | > Peter Loo | | > Wolters Kluwer Health | | > (602) 381-9553 | | > | | > -----Original Message----- | | > From: Scott Walters [mailto:scott at illogics.org] | | > Sent: Tuesday, March 14, 2006 3:28 PM | | > To: Michael Friedman | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | > | | > Hi Peter, | | > | | > Surely you're trying to accomplish more than just running the SQL or | | | > you would just read it in an feed it to DBI. There's no reason you | | > couldn't call to the database command shell: | | > | | > if(my $pid = fork) { | | > waitpid $pid; | | > } else { | | > close STDIN; | | > open STDIN, '<', 'foo.sql' or die $!; | | > exec 'mysql', $dbname or die $!; | | > } | | > | | > ... or something like that. If you want to use special features of | | > the database command shell or just cash in on its speed, this might | | > be | | | | > handy. Of course, you don't want to try to read values from the | | > database back into Perl over a pipe between two processes... that's | | > just nasty. | | > | | > -scott | | > | | > | | > On 0, Michael Friedman wrote: | | > > Peter, | | > > | | > > What I did in that situation was write a couple of methods to read | | | > > in the file, put it into an array, and then loop through the array | | | > > and make each call. The good news is you only have to write that | | > > once and then you can reuse it... | | > > | | > > My only example is using Sybase::DBlib, though, not DBI, but the | | > > logic | | > | | > > would be the same. Sybase uses 'go' on a line by itself to end a | | > > SQL | | | | > > command, so we just use that to split up the lines in the file | | > > into commands into the array. | | > > | | > > You could make this a lot fancier, if you had the need, but it | | > > works | | | | > > for me. | | > > | | > > Good luck, | | > > -- Mike | | > > | | > > (sub db_run_script and sub db_run_command_list, below) | | > > | | > > sub db_run_script #($$) | | > > { | | > > my $dbh = shift; | | > > my $script = shift; | | > > my $saveresults = shift; | | > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input file | | > $script: | | > > $!"; | | > > | | > > my @commands = (); | | > > my $j = 0; | | > > my ($line); | | > > | | > > # read script file into a variable (array of commands) | | > > while ($line = ) | | > > { | | > > if ($line =~ /^go/) | | > > { | | > > # make new command | | > > $j++; | | > > } | | > > elsif ($line =~ /^\s*$/) | | > > { | | > > # ignore blank lines | | > > } | | > > else | | > > { | | > > $commands[$j] .= $line; | | > > } | | > > } | | > > close SQL_SCRIPT; | | > > | | > > return db_run_command_list($dbh, \@commands, $saveresults); } | | > > | | > > sub db_run_command_list | | > > { | | > > my $dbh = shift; | | > > my $cmdlist = shift; | | > > my $saveresults = shift; | | > > | | > > my @resultlist; | | > > | | > > # run commands from array | | > > for $j(0..$#$cmdlist) | | > > { | | > > $dbh->dbcmd($cmdlist->[$j]); | | > > my $status; | | > > eval { | | > > $status = $dbh->dbsqlexec(); | | > > }; | | > > | | > > if ($@ || $status != SUCCEED) | | > > { | | > > # don't always die, because drop will fail | | > sometimes | | > > if ($cmdlist->[$j] =~ /drop/i) | | > > { | | > > warn "$cmdlist->[$j] failed.\n This is | | > OK - item probably didn't | | > > exist before installation.\n"; | | > > | | > > $dbh->dbcancel(); # so that we can move | | > on to the next command? | | > > } | | > > else | | > > { | | > > die "+++ Could not run command | | > $cmdlist->[$j]\nbecause of this | | > > problem:\n$@"; | | > > } | | > > } | | > > | | > > if (!$saveresults) { | | > > db_ignore_results($dbh); | | > > } else { | | > > # Count the total number of rows that were updated, | | > > # and capture the output of any SELECT statements | | > > # | | > > # Each update/insert statement will have its own | | > update | | > > # count (a separate call to DBCOUNT()) but we will | | > > # just add them all together | | > > my $totalupdatecount = 0; | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { | | > > my $rcount = $dbh->DBCOUNT(); | | > > if ($rcount != -1) { | | > > $totalupdatecount += $rcount; | | > > } | | > > | | > > my @res; | | > > while (@res = $dbh->dbnextrow()) { | | > > my @copyres = @res; # make a copy of the | | > array | | > > push @resultlist, \@copyres; | | > > } | | > > } | | > > | | > > push @resultlist, $totalupdatecount; | | > > } | | > > } | | > > | | > > if ($saveresults) { | | > > return \@resultlist; | | > > } else { | | > > return; | | > > } | | > > } | | > > | | > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: | | > > | | > > > Hi, | | > > > | | > > > I know that you are able to issue a SQL statement within PERL | | > > > DBI, | | | | > > > but is there anyway that I can issue an external SQL program? | | > > > For | | | | > > > example, I have a SQL program called ppv_insert.sql that I would | | | > > > like to execute within PERL DBI. | | > > > | | > > > Thanks in advance. | | > > > | | > > > Peter Loo | | > > > | | > > > | | > > > | | > > > | | > > > This E-mail message is for the sole use of the intended | | > > > recipient | | > > > (s) and may contain confidential and privileged information. | | > > > Any unauthorized review, use, disclosure or distribution is | | prohibited. | | > | | > > > If you are not the intended recipient, please contact the sender | | | > > > by reply E-mail, and destroy all copies of the original message. | | > > > _______________________________________________ | | > > > Phoenix-pm mailing list | | > > > Phoenix-pm at pm.org | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm | | > > | | > > | | --------------------------------------------------------------------- | | > > Michael Friedman HighWire Press | | > > Phone: 650-725-1974 Stanford University | | > > FAX: 270-721-8034 | | | | > > ------------------------------------------------------------------ | | > > -- | | > > - | | > > | | > > | | > > _______________________________________________ | | > > Phoenix-pm mailing list | | > > Phoenix-pm at pm.org | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm | | > | | > | | > This E-mail message is for the sole use of the intended recipient(s) | | | > and may contain confidential and privileged information. Any | | > unauthorized review, use, disclosure or distribution is prohibited. | | > If you are not the intended recipient, please contact the sender by | | > reply E-mail, and destroy all copies of the original message. | | > | | > | | > This E-mail message is for the sole use of the intended recipient(s) | | and may contain confidential and privileged information. Any | | unauthorized review, use, disclosure or distribution is prohibited. | | If you are not the intended recipient, please contact the sender by | | reply E-mail, and destroy all copies of the original message. | | | | | | This E-mail message is for the sole use of the intended recipient(s) | | and may contain confidential and privileged information. Any | | unauthorized review, use, disclosure or distribution is prohibited. | | If you are not the intended recipient, please contact the sender by | | reply E-mail, and destroy all copies of the original message. | | | | | | This E-mail message is for the sole use of the intended recipient(s) | and may contain confidential and privileged information. Any | unauthorized review, use, disclosure or distribution is prohibited. If | you are not the intended recipient, please contact the sender by reply | E-mail, and destroy all copies of the original message. | | _______________________________________________ | | Phoenix-pm mailing list | | Phoenix-pm at pm.org | | http://mail.pm.org/mailman/listinfo/phoenix-pm | | | This E-mail message is for the sole use of the intended recipient(s) and | may contain confidential and privileged information. Any unauthorized | review, use, disclosure or distribution is prohibited. If you are not | the intended recipient, please contact the sender by reply E-mail, and | destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From Peter.Loo at source.wolterskluwer.com Wed Mar 15 13:18:47 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Wed, 15 Mar 2006 14:18:47 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95E01@phxmail02.phx.ndchealth.com> Hi Brock, Thanks for your time. My goal is to have a generic PERL sub_routine that I can pass one to many arguments (including the sql file name) and have it perform the SQL statement that is within the sql file dynamically so that we can reuse the same sub_routine for all select statement sql files. Does that make sense? I am all for reusing of code and not reinvent the wheel each time I have an need to run a select statement. I suppose I can read in the sql file like Michael Friedman had suggested earlier in the chain of emails. I was just hoping not to have to read in the sql statement contained in the sql file and assigning it to a variable before doing the $dbh->prepare. I need to do some brain storming. :) Not going to give up yet as I am dealing with one of my two favorite languages. :) Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Brock [mailto:awwaiid at thelackthereof.org] Sent: Wednesday, March 15, 2006 2:02 PM To: Loo, Peter # PHX Cc: Scott Walters; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Using DBI is completely different then piping things to sqlplus. I was just doing a direct translation of your example, not really trying to imply that it was a good idea. Perl DBI does NOT use sqlplus as the driver. Unfortunately your question doesn't make much sense... if your .sql has a select it could have many selects and it could have all sorts of things. The problem is that how can your program know what it contains? It seems what you need to do is put the select like queries directly into perl, as we demonstrated to you in earlier emails. Doing the whole fetchrow_arrayref thing that you were already doing. Make sense? --Brock On 2006.03.15.13.55, Loo, Peter # PHX wrote: | | Hi Brock, | | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't be | efficient. Secondly, what if you have a "SELECT" statement in the | .sql program and if you want to loop through each row? | | Peter Loo | Wolters Kluwer Health | (602) 381-9553 | | -----Original Message----- | From: Brock [mailto:awwaiid at thelackthereof.org] | Sent: Wednesday, March 15, 2006 1:52 PM | To: Loo, Peter # PHX | Cc: Scott Walters; phoenix-pm at pm.org | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | Sure... what you are doing here is opening an external program and | piping it data on STDIN. There are several ways to do this in Perl... | | Here's one (a rough guess): | | # Initiate those vars (pull them from %ENV?) | my $dbuser = ...; | my $dbpass = ...; | my $dbconn = ...; | my $mailProgFile = ...; | | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; | print $sqlplus <<" HERE"; | connect ${dbuser}/${dbpass}@${dbconn} | @${mailProgFile}.sql | HERE | | More or less. Eh? | | --Brock | | On 2006.03.15.13.19, Loo, Peter # PHX wrote: | | | | Hi Scott, | | | | So will it be correct to assume that PERL DBI can not execute an SQL | | program? For example, I can do this with Korn shell: | | | | sqlplus -s /NOLOG << EOF | | connect ${DBUSER}/${DBPASS}@${DBCONN} | | @${MailProgFile}.sql | | | | Is this not possible in PERL DBI? | | | | Peter Loo | | Wolters Kluwer Health | | (602) 381-9553 | | | | -----Original Message----- | | From: Scott Walters [mailto:scott at illogics.org] | | Sent: Tuesday, March 14, 2006 9:36 PM | | To: Loo, Peter # PHX | | Cc: Michael Friedman; phoenix-pm at pm.org | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | | | Consider using GetOpt::Std, and most of the time you want this form | | of | | for: | | | | for my $thing (@things) { ... stuff with $thing ... } | | | | You can always try to read rows and trap errors. | | | | -scott | | | | On 0, "Loo, Peter # PHX" wrote: | | > | | > Thanks Michael and Scott. What I am trying to do is creating a | | > generic PERL program that will take in multiple arguments such as: | | > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); | | > switch ($flag) { | | > case "-dd" { $d_dbName = lc($value) } | | > case "-dt" { $d_tblName = lc($value) } | | > case "-ds" { $d_SQL = $value } | | > case "-sd" { $s_dbName = lc($value) } | | > case "-st" { $s_tblName = lc($value) } | | > case "-ss" { $s_SQL = $value } | | > case "-cp" { $commitPoint = lc($value) } | | > case "-sf" { $s_funcToPerf = lc($value) } | | > case "-df" { $d_funcToPerf = lc($value) } | | > case "-d1" { $s_dbDriver = lc($value) } | | > case "-d2" { $d_dbDriver = lc($value) } | | > else { print "Unknown flag: $flag\n" } | | > } | | > } | | > | | > Then execute accordingly, however, I would like to execute an | | > external | | | | > SQL program that is passed to this generic program. In the case | | > of an | | | | > external SQL program that does SELECT instead of UPDATE or INSERT, | | > I | | | > want to loop through the returned rows. Here is my first block of | | "if" | | > statement. | | > | | > if ($s_funcToPerf eq "update") { | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | | > || | | > !$s_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example: \n"; | | > print " Database Name: -sd=dv26\n"; | | > print " Database Driver: -d1=Oracle\n"; | | > print " Table Name: -st=p_falcon_projections\n"; | | > print " SQL Statement: | -ss=/usr/local/sql/ppv_update.sql\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_update()\n"; | | > } | | > } | | > elsif ($s_funcToPerf eq "insert") { | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | | > || | | > !$s_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example: \n"; | | > print " Database Name: -sd=dv26\n"; | | > print " Database Driver: -d1=Oracle\n"; | | > print " Table Name: -st=p_falcon_projections\n"; | | > print " SQL Statement: | -ss=/usr/local/sql/ppv_insert.sql\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_insert()\n"; | | > } | | > } | | > elsif ($d_funcToPerf eq "update") { | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | | > !$s_dbName || !$s_dbDriver || !$s_tblName || | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example:\n"; | | > print " Destination Database Name: -dd=pv26\n"; | | > print " Destination Database Driver: -d2=ODBC\n"; | | > print " Destination Table Name: | | -dt=p_falcon_projections\n"; | | > print " Source Database Name: -sd=dv26\n"; | | > print " Source Database Driver: -d1=Oracle\n"; | | > print " Source Table Name: | | -st=p_falcon_projections\n"; | | > print " SQL Statement: | | > -ss=/usr/local/sql/ppv_select.sql\n"; | | > print " SQL Statement: | | > -ds=/usr/local/sql/ppv_update.sql\n"; | | > print " Source Function to Perform: -sf=select\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_select()\n"; | | > print "Calling sub_update()\n"; | | > } | | > } | | > elsif ($d_funcToPerf eq "insert") { | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || | | > !$s_dbName || !$s_dbDriver || !$s_tblName || | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example:\n"; | | > print " Destination Database Name: -dd=pv26\n"; | | > print " Destination Database Driver: -d2=ODBC\n"; | | > print " Destination Table Name: | | -dt=p_falcon_projections\n"; | | > print " Source Database Name: -sd=dv26\n"; | | > print " Source Database Driver: -d1=Oracle\n"; | | > print " Source Table Name: | | -st=p_falcon_projections\n"; | | > print " SQL Statement: | | > -ss=/usr/local/sql/ppv_select.sql\n"; | | > print " SQL Statement: | | > -ds=/usr/local/sql/ppv_insert.sql\n"; | | > print " Source Function to Perform: -sf=select\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_select()\n"; | | > print "Calling sub_insert()\n"; | | > } | | > } | | > elsif ($s_funcToPerf eq "select") { | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint | || | | > !$d_dbName || !$d_dbDriver || !$d_tblName || | | > !$d_funcToPerf | || | | > !$s_SQL || !$d_SQL) { | | > print "ERROR: Not enough arguments. Require arguments | are:\n"; | | > print "Example:\n"; | | > print " Destination Database Name: -dd=pv26\n"; | | > print " Destination Database Driver: -d2=ODBC\n"; | | > print " Destination Table Name: | | -dt=p_falcon_projections\n"; | | > print " Source Database Name: -sd=dv26\n"; | | > print " Source Database Driver: -d1=Oracle\n"; | | > print " Source Table Name: | | -st=p_falcon_projections\n"; | | > print " SQL Statement: | | > -ss=/usr/local/sql/ppv_select.sql\n"; | | > print " SQL Statement: | | > -ds=/usr/local/sql/ppv_insert.sql\n"; | | > print " Source Function to Perform: -sf=select\n"; | | > print " Commit Point: -cp=5000\n"; | | > exit(666); | | > } | | > else { | | > print "Calling sub_select()\n"; | | > print "Calling sub_insert()\n"; | | > } | | > } | | > else { | | > print "ERROR: Unknown value for database action to perform.\n"; | | > exit(666); | | > } | | > | | > | | > Peter Loo | | > Wolters Kluwer Health | | > (602) 381-9553 | | > | | > -----Original Message----- | | > From: Scott Walters [mailto:scott at illogics.org] | | > Sent: Tuesday, March 14, 2006 3:28 PM | | > To: Michael Friedman | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI | | > | | > Hi Peter, | | > | | > Surely you're trying to accomplish more than just running the SQL | | > or | | | > you would just read it in an feed it to DBI. There's no reason | | > you couldn't call to the database command shell: | | > | | > if(my $pid = fork) { | | > waitpid $pid; | | > } else { | | > close STDIN; | | > open STDIN, '<', 'foo.sql' or die $!; | | > exec 'mysql', $dbname or die $!; | | > } | | > | | > ... or something like that. If you want to use special features | | > of the database command shell or just cash in on its speed, this | | > might be | | | | > handy. Of course, you don't want to try to read values from the | | > database back into Perl over a pipe between two processes... | | > that's just nasty. | | > | | > -scott | | > | | > | | > On 0, Michael Friedman wrote: | | > > Peter, | | > > | | > > What I did in that situation was write a couple of methods to | | > > read | | | > > in the file, put it into an array, and then loop through the | | > > array | | | > > and make each call. The good news is you only have to write that | | > > once and then you can reuse it... | | > > | | > > My only example is using Sybase::DBlib, though, not DBI, but the | | > > logic | | > | | > > would be the same. Sybase uses 'go' on a line by itself to end a | | > > SQL | | | | > > command, so we just use that to split up the lines in the file | | > > into commands into the array. | | > > | | > > You could make this a lot fancier, if you had the need, but it | | > > works | | | | > > for me. | | > > | | > > Good luck, | | > > -- Mike | | > > | | > > (sub db_run_script and sub db_run_command_list, below) | | > > | | > > sub db_run_script #($$) | | > > { | | > > my $dbh = shift; | | > > my $script = shift; | | > > my $saveresults = shift; | | > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input file | | > $script: | | > > $!"; | | > > | | > > my @commands = (); | | > > my $j = 0; | | > > my ($line); | | > > | | > > # read script file into a variable (array of commands) | | > > while ($line = ) | | > > { | | > > if ($line =~ /^go/) | | > > { | | > > # make new command | | > > $j++; | | > > } | | > > elsif ($line =~ /^\s*$/) | | > > { | | > > # ignore blank lines | | > > } | | > > else | | > > { | | > > $commands[$j] .= $line; | | > > } | | > > } | | > > close SQL_SCRIPT; | | > > | | > > return db_run_command_list($dbh, \@commands, $saveresults); } | | > > | | > > sub db_run_command_list | | > > { | | > > my $dbh = shift; | | > > my $cmdlist = shift; | | > > my $saveresults = shift; | | > > | | > > my @resultlist; | | > > | | > > # run commands from array | | > > for $j(0..$#$cmdlist) | | > > { | | > > $dbh->dbcmd($cmdlist->[$j]); | | > > my $status; | | > > eval { | | > > $status = $dbh->dbsqlexec(); | | > > }; | | > > | | > > if ($@ || $status != SUCCEED) | | > > { | | > > # don't always die, because drop will fail | | > sometimes | | > > if ($cmdlist->[$j] =~ /drop/i) | | > > { | | > > warn "$cmdlist->[$j] failed.\n This is | | > OK - item probably didn't | | > > exist before installation.\n"; | | > > | | > > $dbh->dbcancel(); # so that we can move | | > on to the next command? | | > > } | | > > else | | > > { | | > > die "+++ Could not run command | | > $cmdlist->[$j]\nbecause of this | | > > problem:\n$@"; | | > > } | | > > } | | > > | | > > if (!$saveresults) { | | > > db_ignore_results($dbh); | | > > } else { | | > > # Count the total number of rows that were updated, | | > > # and capture the output of any SELECT statements | | > > # | | > > # Each update/insert statement will have its own | | > update | | > > # count (a separate call to DBCOUNT()) but we will | | > > # just add them all together | | > > my $totalupdatecount = 0; | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { | | > > my $rcount = $dbh->DBCOUNT(); | | > > if ($rcount != -1) { | | > > $totalupdatecount += $rcount; | | > > } | | > > | | > > my @res; | | > > while (@res = $dbh->dbnextrow()) { | | > > my @copyres = @res; # make a copy of the | | > array | | > > push @resultlist, \@copyres; | | > > } | | > > } | | > > | | > > push @resultlist, $totalupdatecount; | | > > } | | > > } | | > > | | > > if ($saveresults) { | | > > return \@resultlist; | | > > } else { | | > > return; | | > > } | | > > } | | > > | | > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: | | > > | | > > > Hi, | | > > > | | > > > I know that you are able to issue a SQL statement within PERL | | > > > DBI, | | | | > > > but is there anyway that I can issue an external SQL program? | | > > > For | | | | > > > example, I have a SQL program called ppv_insert.sql that I | | > > > would | | | > > > like to execute within PERL DBI. | | > > > | | > > > Thanks in advance. | | > > > | | > > > Peter Loo | | > > > | | > > > | | > > > | | > > > | | > > > This E-mail message is for the sole use of the intended | | > > > recipient | | > > > (s) and may contain confidential and privileged information. | | > > > Any unauthorized review, use, disclosure or distribution is | | prohibited. | | > | | > > > If you are not the intended recipient, please contact the | | > > > sender | | | > > > by reply E-mail, and destroy all copies of the original message. | | > > > _______________________________________________ | | > > > Phoenix-pm mailing list | | > > > Phoenix-pm at pm.org | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm | | > > | | > > | | -------------------------------------------------------------------- | | - | | > > Michael Friedman HighWire Press | | > > Phone: 650-725-1974 Stanford University | | > > FAX: 270-721-8034 | | | | > > ---------------------------------------------------------------- | | > > -- | | > > -- | | > > - | | > > | | > > | | > > _______________________________________________ | | > > Phoenix-pm mailing list | | > > Phoenix-pm at pm.org | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm | | > | | > | | > This E-mail message is for the sole use of the intended | | > recipient(s) | | | > and may contain confidential and privileged information. Any | | > unauthorized review, use, disclosure or distribution is prohibited. | | > If you are not the intended recipient, please contact the sender | | > by reply E-mail, and destroy all copies of the original message. | | > | | > | | > This E-mail message is for the sole use of the intended | | > recipient(s) | | and may contain confidential and privileged information. Any | | unauthorized review, use, disclosure or distribution is prohibited. | | If you are not the intended recipient, please contact the sender by | | reply E-mail, and destroy all copies of the original message. | | | | | | This E-mail message is for the sole use of the intended recipient(s) | | and may contain confidential and privileged information. Any | | unauthorized review, use, disclosure or distribution is prohibited. | | If you are not the intended recipient, please contact the sender by | | reply E-mail, and destroy all copies of the original message. | | | | | | This E-mail message is for the sole use of the intended recipient(s) | and may contain confidential and privileged information. Any | unauthorized review, use, disclosure or distribution is prohibited. | If you are not the intended recipient, please contact the sender by | reply E-mail, and destroy all copies of the original message. | | _______________________________________________ | | Phoenix-pm mailing list | | Phoenix-pm at pm.org | | http://mail.pm.org/mailman/listinfo/phoenix-pm | | | This E-mail message is for the sole use of the intended recipient(s) | and may contain confidential and privileged information. Any | unauthorized review, use, disclosure or distribution is prohibited. | If you are not the intended recipient, please contact the sender by | reply E-mail, and destroy all copies of the original message. | | | This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From scott at illogics.org Wed Mar 15 14:57:49 2006 From: scott at illogics.org (Scott Walters) Date: Wed, 15 Mar 2006 22:57:49 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95E01@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95E01@phxmail02.phx.ndchealth.com> Message-ID: <20060315225749.GM22790@illogics.org> Hi Peter, > My goal is to have a generic PERL sub_routine that I can pass one to > many arguments (including the sql file name) and have it perform the SQL > statement that is within the sql file dynamically so that we can reuse > the same sub_routine for all select statement sql files. Does that make > sense? I am all for reusing of code and not reinvent the wheel each Stop repeating it this. If it doesn't make sense, then do something else. If it does, then we get it already. You have a choice: you can re-implement parts of the SQL shell in Perl, or you can call the SQL shell. You keep talking about reading the file and passing the SQL... you know how to do both of these things already. Funneling all SQL SELECT statements through one routine is silly. Refactor your code however you like, but this artifical goal will only hurt you. If you find you're writing a lot of similar routines, then learn how to write closures. No, I don't want to talk about this or debate it; no slight intended, but this kind of simplistic view of "one routine to do all" is characteristic of novices. People who have been doing this for a while know there's give and take. Routines get split out, then made into object methods, and those objects get configured with more objects they delegate to, then common logic is re-grouped into a routine somewhere, and so on. Trying to do all of any sort of thing in one places ignores the inherent complexity in a program. That's like saying "I want to paint the world PINK!". But, I said this isn't up for discussion. If I see it again, I'm telling mutt to ignore this thread. > time I have an need to run a select statement. I suppose I can read in > the sql file like Michael Friedman had suggested earlier in the chain of > emails. I was just hoping not to have to read in the sql statement > contained in the sql file and assigning it to a variable before doing > the $dbh->prepare. You want to execute it in Perl and not read it in? I think you need to focus on what you're really trying to accomplish and concentrate less on how you do it, because you're starting to get silly. I can't imagine the real problem actually suggests a reading/not-reading paradox. > I need to do some brain storming. :) Not going to give up yet as I am > dealing with one of my two favorite languages. :) If you want to run it through Perl, read it in and DBI. If you want to run it in the database shell, either do so directly or from a subprocess. Either way, quit trying to do both-and-neither-at-the-same-time. I assure you it will be entirely unproductive. By the way, there's often no clear solution (at least the programmer working on something), and in that case, don't fuss with it endlessly. Do it one way and be done with it, and if a clear solution occurs to you later, go back and change it. -scott > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Brock [mailto:awwaiid at thelackthereof.org] > Sent: Wednesday, March 15, 2006 2:02 PM > To: Loo, Peter # PHX > Cc: Scott Walters; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Using DBI is completely different then piping things to sqlplus. I was > just doing a direct translation of your example, not really trying to > imply that it was a good idea. > > Perl DBI does NOT use sqlplus as the driver. > > Unfortunately your question doesn't make much sense... if your .sql has > a select it could have many selects and it could have all sorts of > things. The problem is that how can your program know what it contains? > It seems what you need to do is put the select like queries directly > into perl, as we demonstrated to you in earlier emails. Doing the whole > fetchrow_arrayref thing that you were already doing. > > Make sense? > > --Brock > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > | > | Hi Brock, > | > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't be > | efficient. Secondly, what if you have a "SELECT" statement in the > | .sql program and if you want to loop through each row? > | > | Peter Loo > | Wolters Kluwer Health > | (602) 381-9553 > | > | -----Original Message----- > | From: Brock [mailto:awwaiid at thelackthereof.org] > | Sent: Wednesday, March 15, 2006 1:52 PM > | To: Loo, Peter # PHX > | Cc: Scott Walters; phoenix-pm at pm.org > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | > | Sure... what you are doing here is opening an external program and > | piping it data on STDIN. There are several ways to do this in Perl... > | > | Here's one (a rough guess): > | > | # Initiate those vars (pull them from %ENV?) > | my $dbuser = ...; > | my $dbpass = ...; > | my $dbconn = ...; > | my $mailProgFile = ...; > | > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; > | print $sqlplus <<" HERE"; > | connect ${dbuser}/${dbpass}@${dbconn} > | @${mailProgFile}.sql > | HERE > | > | More or less. Eh? > | > | --Brock > | > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > | | > | | Hi Scott, > | | > | | So will it be correct to assume that PERL DBI can not execute an SQL > > | | program? For example, I can do this with Korn shell: > | | > | | sqlplus -s /NOLOG << EOF > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > | | @${MailProgFile}.sql > | | > | | Is this not possible in PERL DBI? > | | > | | Peter Loo > | | Wolters Kluwer Health > | | (602) 381-9553 > | | > | | -----Original Message----- > | | From: Scott Walters [mailto:scott at illogics.org] > | | Sent: Tuesday, March 14, 2006 9:36 PM > | | To: Loo, Peter # PHX > | | Cc: Michael Friedman; phoenix-pm at pm.org > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | | > | | Consider using GetOpt::Std, and most of the time you want this form > | | of > | | for: > | | > | | for my $thing (@things) { ... stuff with $thing ... } > | | > | | You can always try to read rows and trap errors. > | | > | | -scott > | | > | | On 0, "Loo, Peter # PHX" > wrote: > | | > > | | > Thanks Michael and Scott. What I am trying to do is creating a > | | > generic PERL program that will take in multiple arguments such as: > | | > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > | | > switch ($flag) { > | | > case "-dd" { $d_dbName = lc($value) } > | | > case "-dt" { $d_tblName = lc($value) } > | | > case "-ds" { $d_SQL = $value } > | | > case "-sd" { $s_dbName = lc($value) } > | | > case "-st" { $s_tblName = lc($value) } > | | > case "-ss" { $s_SQL = $value } > | | > case "-cp" { $commitPoint = lc($value) } > | | > case "-sf" { $s_funcToPerf = lc($value) } > | | > case "-df" { $d_funcToPerf = lc($value) } > | | > case "-d1" { $s_dbDriver = lc($value) } > | | > case "-d2" { $d_dbDriver = lc($value) } > | | > else { print "Unknown flag: $flag\n" } > | | > } > | | > } > | | > > | | > Then execute accordingly, however, I would like to execute an > | | > external > | | > | | > SQL program that is passed to this generic program. In the case > | | > of an > | | > | | > external SQL program that does SELECT instead of UPDATE or INSERT, > > | | > I > | > | | > want to loop through the returned rows. Here is my first block of > | | "if" > | | > statement. > | | > > | | > if ($s_funcToPerf eq "update") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint > | | > || > | | > !$s_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example: \n"; > | | > print " Database Name: -sd=dv26\n"; > | | > print " Database Driver: -d1=Oracle\n"; > | | > print " Table Name: -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | -ss=/usr/local/sql/ppv_update.sql\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_update()\n"; > | | > } > | | > } > | | > elsif ($s_funcToPerf eq "insert") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint > | | > || > | | > !$s_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example: \n"; > | | > print " Database Name: -sd=dv26\n"; > | | > print " Database Driver: -d1=Oracle\n"; > | | > print " Table Name: -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | -ss=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > elsif ($d_funcToPerf eq "update") { > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_update()\n"; > | | > } > | | > } > | | > elsif ($d_funcToPerf eq "insert") { > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > elsif ($s_funcToPerf eq "select") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || !$commitPoint > | || > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$d_funcToPerf > | || > | | > !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > else { > | | > print "ERROR: Unknown value for database action to > perform.\n"; > | | > exit(666); > | | > } > | | > > | | > > | | > Peter Loo > | | > Wolters Kluwer Health > | | > (602) 381-9553 > | | > > | | > -----Original Message----- > | | > From: Scott Walters [mailto:scott at illogics.org] > | | > Sent: Tuesday, March 14, 2006 3:28 PM > | | > To: Michael Friedman > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | | > > | | > Hi Peter, > | | > > | | > Surely you're trying to accomplish more than just running the SQL > | | > or > | > | | > you would just read it in an feed it to DBI. There's no reason > | | > you couldn't call to the database command shell: > | | > > | | > if(my $pid = fork) { > | | > waitpid $pid; > | | > } else { > | | > close STDIN; > | | > open STDIN, '<', 'foo.sql' or die $!; > | | > exec 'mysql', $dbname or die $!; > | | > } > | | > > | | > ... or something like that. If you want to use special features > | | > of the database command shell or just cash in on its speed, this > | | > might be > | | > | | > handy. Of course, you don't want to try to read values from the > | | > database back into Perl over a pipe between two processes... > | | > that's just nasty. > | | > > | | > -scott > | | > > | | > > | | > On 0, Michael Friedman wrote: > | | > > Peter, > | | > > > | | > > What I did in that situation was write a couple of methods to > | | > > read > | > | | > > in the file, put it into an array, and then loop through the > | | > > array > | > | | > > and make each call. The good news is you only have to write that > > | | > > once and then you can reuse it... > | | > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but the > > | | > > logic > | | > > | | > > would be the same. Sybase uses 'go' on a line by itself to end a > > | | > > SQL > | | > | | > > command, so we just use that to split up the lines in the file > | | > > into commands into the array. > | | > > > | | > > You could make this a lot fancier, if you had the need, but it > | | > > works > | | > | | > > for me. > | | > > > | | > > Good luck, > | | > > -- Mike > | | > > > | | > > (sub db_run_script and sub db_run_command_list, below) > | | > > > | | > > sub db_run_script #($$) > | | > > { > | | > > my $dbh = shift; > | | > > my $script = shift; > | | > > my $saveresults = shift; > | | > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > file > | | > $script: > | | > > $!"; > | | > > > | | > > my @commands = (); > | | > > my $j = 0; > | | > > my ($line); > | | > > > | | > > # read script file into a variable (array of commands) > | | > > while ($line = ) > | | > > { > | | > > if ($line =~ /^go/) > | | > > { > | | > > # make new command > | | > > $j++; > | | > > } > | | > > elsif ($line =~ /^\s*$/) > | | > > { > | | > > # ignore blank lines > | | > > } > | | > > else > | | > > { > | | > > $commands[$j] .= $line; > | | > > } > | | > > } > | | > > close SQL_SCRIPT; > | | > > > | | > > return db_run_command_list($dbh, \@commands, > $saveresults); } > | | > > > | | > > sub db_run_command_list > | | > > { > | | > > my $dbh = shift; > | | > > my $cmdlist = shift; > | | > > my $saveresults = shift; > | | > > > | | > > my @resultlist; > | | > > > | | > > # run commands from array > | | > > for $j(0..$#$cmdlist) > | | > > { > | | > > $dbh->dbcmd($cmdlist->[$j]); > | | > > my $status; > | | > > eval { > | | > > $status = $dbh->dbsqlexec(); > | | > > }; > | | > > > | | > > if ($@ || $status != SUCCEED) > | | > > { > | | > > # don't always die, because drop will > fail > | | > sometimes > | | > > if ($cmdlist->[$j] =~ /drop/i) > | | > > { > | | > > warn "$cmdlist->[$j] failed.\n > This is > | | > OK - item probably didn't > | | > > exist before installation.\n"; > | | > > > | | > > $dbh->dbcancel(); # so that we > can move > | | > on to the next command? > | | > > } > | | > > else > | | > > { > | | > > die "+++ Could not run command > | | > $cmdlist->[$j]\nbecause of this > | | > > problem:\n$@"; > | | > > } > | | > > } > | | > > > | | > > if (!$saveresults) { > | | > > db_ignore_results($dbh); > | | > > } else { > | | > > # Count the total number of rows that were > updated, > | | > > # and capture the output of any SELECT > statements > | | > > # > | | > > # Each update/insert statement will have its > own > | | > update > | | > > # count (a separate call to DBCOUNT()) but we > will > | | > > # just add them all together > | | > > my $totalupdatecount = 0; > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > | | > > my $rcount = $dbh->DBCOUNT(); > | | > > if ($rcount != -1) { > | | > > $totalupdatecount += $rcount; > | | > > } > | | > > > | | > > my @res; > | | > > while (@res = $dbh->dbnextrow()) { > | | > > my @copyres = @res; # make a copy of > the > | | > array > | | > > push @resultlist, \@copyres; > | | > > } > | | > > } > | | > > > | | > > push @resultlist, $totalupdatecount; > | | > > } > | | > > } > | | > > > | | > > if ($saveresults) { > | | > > return \@resultlist; > | | > > } else { > | | > > return; > | | > > } > | | > > } > | | > > > | | > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > | | > > > | | > > > Hi, > | | > > > > | | > > > I know that you are able to issue a SQL statement within PERL > | | > > > DBI, > | | > | | > > > but is there anyway that I can issue an external SQL program? > > | | > > > For > | | > | | > > > example, I have a SQL program called ppv_insert.sql that I > | | > > > would > | > | | > > > like to execute within PERL DBI. > | | > > > > | | > > > Thanks in advance. > | | > > > > | | > > > Peter Loo > | | > > > > | | > > > > | | > > > > | | > > > > | | > > > This E-mail message is for the sole use of the intended > | | > > > recipient > | | > > > (s) and may contain confidential and privileged information. > | | > > > Any unauthorized review, use, disclosure or distribution is > | | prohibited. > | | > > | | > > > If you are not the intended recipient, please contact the > | | > > > sender > | > | | > > > by reply E-mail, and destroy all copies of the original > message. > | | > > > _______________________________________________ > | | > > > Phoenix-pm mailing list > | | > > > Phoenix-pm at pm.org > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > | | > > > | | > > > | | -------------------------------------------------------------------- > | | - > | | > > Michael Friedman HighWire Press > | | > > Phone: 650-725-1974 Stanford University > | | > > FAX: 270-721-8034 > | | > | | > > ---------------------------------------------------------------- > | | > > -- > | | > > -- > | | > > - > | | > > > | | > > > | | > > _______________________________________________ > | | > > Phoenix-pm mailing list > | | > > Phoenix-pm at pm.org > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > | | > > | | > > | | > This E-mail message is for the sole use of the intended > | | > recipient(s) > | > | | > and may contain confidential and privileged information. Any > | | > unauthorized review, use, disclosure or distribution is > prohibited. > | | > If you are not the intended recipient, please contact the sender > | | > by reply E-mail, and destroy all copies of the original message. > | | > > | | > > | | > This E-mail message is for the sole use of the intended > | | > recipient(s) > | | and may contain confidential and privileged information. Any > | | unauthorized review, use, disclosure or distribution is prohibited. > | | If you are not the intended recipient, please contact the sender by > | | reply E-mail, and destroy all copies of the original message. > | | > | | > | | This E-mail message is for the sole use of the intended recipient(s) > > | | and may contain confidential and privileged information. Any > | | unauthorized review, use, disclosure or distribution is prohibited. > | | If you are not the intended recipient, please contact the sender by > | | reply E-mail, and destroy all copies of the original message. > | | > | | > | | This E-mail message is for the sole use of the intended recipient(s) > | and may contain confidential and privileged information. Any > | unauthorized review, use, disclosure or distribution is prohibited. > | If you are not the intended recipient, please contact the sender by > | reply E-mail, and destroy all copies of the original message. > | | _______________________________________________ > | | Phoenix-pm mailing list > | | Phoenix-pm at pm.org > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > | > | > | This E-mail message is for the sole use of the intended recipient(s) > | and may contain confidential and privileged information. Any > | unauthorized review, use, disclosure or distribution is prohibited. > | If you are not the intended recipient, please contact the sender by > | reply E-mail, and destroy all copies of the original message. > | > | > | This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From corey_s at streamreel.com Wed Mar 15 15:05:15 2006 From: corey_s at streamreel.com (Corey Saltiel) Date: Wed, 15 Mar 2006 16:05:15 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95E01@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95E01@phxmail02.phx.ndchealth.com> Message-ID: <200603151605.15916.corey_s@streamreel.com> On Wednesday March 15 2006 2:18 pm, Loo, Peter # PHX wrote: > > My goal is to have a generic PERL sub_routine that I can pass one to > many arguments (including the sql file name) and have it perform the SQL > statement that is within the sql file dynamically so that we can reuse > the same sub_routine for all select statement sql files. Does that make > sense? I am all for reusing of code and not reinvent the wheel > SQL::Abstract http://search.cpan.org/dist/SQL-Abstract/lib/SQL/Abstract.pm Your "sql files" can/should become proper modules; coupled w/ SQL::Abstract, you'll get what you're looking for. Beers, Corey From Peter.Loo at source.wolterskluwer.com Wed Mar 15 15:06:00 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Wed, 15 Mar 2006 16:06:00 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95E87@phxmail02.phx.ndchealth.com> Hi Scott, Sorry that is I have gotten into your skin, however, I thought this was a group to discuss and share ideas without any limitation of the level of PERL knowledge. It appear to me that you just don't like anything that is not your way. Don't worry about blocking me out of this group as I don't plan to return. Brock, Mike and the rest, I am sincerely grateful for your inputs. Best regards. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Wednesday, March 15, 2006 3:58 PM To: Loo, Peter # PHX Cc: Brock; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Peter, > My goal is to have a generic PERL sub_routine that I can pass one to > many arguments (including the sql file name) and have it perform the > SQL statement that is within the sql file dynamically so that we can > reuse the same sub_routine for all select statement sql files. Does > that make sense? I am all for reusing of code and not reinvent the > wheel each Stop repeating it this. If it doesn't make sense, then do something else. If it does, then we get it already. You have a choice: you can re-implement parts of the SQL shell in Perl, or you can call the SQL shell. You keep talking about reading the file and passing the SQL... you know how to do both of these things already. Funneling all SQL SELECT statements through one routine is silly. Refactor your code however you like, but this artifical goal will only hurt you. If you find you're writing a lot of similar routines, then learn how to write closures. No, I don't want to talk about this or debate it; no slight intended, but this kind of simplistic view of "one routine to do all" is characteristic of novices. People who have been doing this for a while know there's give and take. Routines get split out, then made into object methods, and those objects get configured with more objects they delegate to, then common logic is re-grouped into a routine somewhere, and so on. Trying to do all of any sort of thing in one places ignores the inherent complexity in a program. That's like saying "I want to paint the world PINK!". But, I said this isn't up for discussion. If I see it again, I'm telling mutt to ignore this thread. > time I have an need to run a select statement. I suppose I can read > in the sql file like Michael Friedman had suggested earlier in the > chain of emails. I was just hoping not to have to read in the sql > statement contained in the sql file and assigning it to a variable > before doing the $dbh->prepare. You want to execute it in Perl and not read it in? I think you need to focus on what you're really trying to accomplish and concentrate less on how you do it, because you're starting to get silly. I can't imagine the real problem actually suggests a reading/not-reading paradox. > I need to do some brain storming. :) Not going to give up yet as I > am dealing with one of my two favorite languages. :) If you want to run it through Perl, read it in and DBI. If you want to run it in the database shell, either do so directly or from a subprocess. Either way, quit trying to do both-and-neither-at-the-same-time. I assure you it will be entirely unproductive. By the way, there's often no clear solution (at least the programmer working on something), and in that case, don't fuss with it endlessly. Do it one way and be done with it, and if a clear solution occurs to you later, go back and change it. -scott > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Brock [mailto:awwaiid at thelackthereof.org] > Sent: Wednesday, March 15, 2006 2:02 PM > To: Loo, Peter # PHX > Cc: Scott Walters; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Using DBI is completely different then piping things to sqlplus. I was > just doing a direct translation of your example, not really trying to > imply that it was a good idea. > > Perl DBI does NOT use sqlplus as the driver. > > Unfortunately your question doesn't make much sense... if your .sql > has a select it could have many selects and it could have all sorts of > things. The problem is that how can your program know what it contains? > It seems what you need to do is put the select like queries directly > into perl, as we demonstrated to you in earlier emails. Doing the > whole fetchrow_arrayref thing that you were already doing. > > Make sense? > > --Brock > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > | > | Hi Brock, > | > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't > | be efficient. Secondly, what if you have a "SELECT" statement in > | the .sql program and if you want to loop through each row? > | > | Peter Loo > | Wolters Kluwer Health > | (602) 381-9553 > | > | -----Original Message----- > | From: Brock [mailto:awwaiid at thelackthereof.org] > | Sent: Wednesday, March 15, 2006 1:52 PM > | To: Loo, Peter # PHX > | Cc: Scott Walters; phoenix-pm at pm.org > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | > | Sure... what you are doing here is opening an external program and > | piping it data on STDIN. There are several ways to do this in Perl... > | > | Here's one (a rough guess): > | > | # Initiate those vars (pull them from %ENV?) > | my $dbuser = ...; > | my $dbpass = ...; > | my $dbconn = ...; > | my $mailProgFile = ...; > | > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; > | print $sqlplus <<" HERE"; > | connect ${dbuser}/${dbpass}@${dbconn} > | @${mailProgFile}.sql > | HERE > | > | More or less. Eh? > | > | --Brock > | > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > | | > | | Hi Scott, > | | > | | So will it be correct to assume that PERL DBI can not execute an > | | SQL > > | | program? For example, I can do this with Korn shell: > | | > | | sqlplus -s /NOLOG << EOF > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > | | @${MailProgFile}.sql > | | > | | Is this not possible in PERL DBI? > | | > | | Peter Loo > | | Wolters Kluwer Health > | | (602) 381-9553 > | | > | | -----Original Message----- > | | From: Scott Walters [mailto:scott at illogics.org] > | | Sent: Tuesday, March 14, 2006 9:36 PM > | | To: Loo, Peter # PHX > | | Cc: Michael Friedman; phoenix-pm at pm.org > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | | > | | Consider using GetOpt::Std, and most of the time you want this > | | form of > | | for: > | | > | | for my $thing (@things) { ... stuff with $thing ... } > | | > | | You can always try to read rows and trap errors. > | | > | | -scott > | | > | | On 0, "Loo, Peter # PHX" > wrote: > | | > > | | > Thanks Michael and Scott. What I am trying to do is creating a > | | > generic PERL program that will take in multiple arguments such as: > | | > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > | | > switch ($flag) { > | | > case "-dd" { $d_dbName = lc($value) } > | | > case "-dt" { $d_tblName = lc($value) } > | | > case "-ds" { $d_SQL = $value } > | | > case "-sd" { $s_dbName = lc($value) } > | | > case "-st" { $s_tblName = lc($value) } > | | > case "-ss" { $s_SQL = $value } > | | > case "-cp" { $commitPoint = lc($value) } > | | > case "-sf" { $s_funcToPerf = lc($value) } > | | > case "-df" { $d_funcToPerf = lc($value) } > | | > case "-d1" { $s_dbDriver = lc($value) } > | | > case "-d2" { $d_dbDriver = lc($value) } > | | > else { print "Unknown flag: $flag\n" } > | | > } > | | > } > | | > > | | > Then execute accordingly, however, I would like to execute an > | | > external > | | > | | > SQL program that is passed to this generic program. In the case > | | > of an > | | > | | > external SQL program that does SELECT instead of UPDATE or > | | > INSERT, > > | | > I > | > | | > want to loop through the returned rows. Here is my first block > | | > of > | | "if" > | | > statement. > | | > > | | > if ($s_funcToPerf eq "update") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$commitPoint > | | > || > | | > !$s_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example: \n"; > | | > print " Database Name: -sd=dv26\n"; > | | > print " Database Driver: -d1=Oracle\n"; > | | > print " Table Name: -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | -ss=/usr/local/sql/ppv_update.sql\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_update()\n"; > | | > } > | | > } > | | > elsif ($s_funcToPerf eq "insert") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$commitPoint > | | > || > | | > !$s_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example: \n"; > | | > print " Database Name: -sd=dv26\n"; > | | > print " Database Driver: -d1=Oracle\n"; > | | > print " Table Name: -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | -ss=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > elsif ($d_funcToPerf eq "update") { > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_update()\n"; > | | > } > | | > } > | | > elsif ($d_funcToPerf eq "insert") { > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > elsif ($s_funcToPerf eq "select") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$commitPoint > | || > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$d_funcToPerf > | || > | | > !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > else { > | | > print "ERROR: Unknown value for database action to > perform.\n"; > | | > exit(666); > | | > } > | | > > | | > > | | > Peter Loo > | | > Wolters Kluwer Health > | | > (602) 381-9553 > | | > > | | > -----Original Message----- > | | > From: Scott Walters [mailto:scott at illogics.org] > | | > Sent: Tuesday, March 14, 2006 3:28 PM > | | > To: Michael Friedman > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | | > > | | > Hi Peter, > | | > > | | > Surely you're trying to accomplish more than just running the > | | > SQL or > | > | | > you would just read it in an feed it to DBI. There's no reason > | | > you couldn't call to the database command shell: > | | > > | | > if(my $pid = fork) { > | | > waitpid $pid; > | | > } else { > | | > close STDIN; > | | > open STDIN, '<', 'foo.sql' or die $!; > | | > exec 'mysql', $dbname or die $!; > | | > } > | | > > | | > ... or something like that. If you want to use special features > | | > of the database command shell or just cash in on its speed, this > | | > might be > | | > | | > handy. Of course, you don't want to try to read values from the > | | > database back into Perl over a pipe between two processes... > | | > that's just nasty. > | | > > | | > -scott > | | > > | | > > | | > On 0, Michael Friedman wrote: > | | > > Peter, > | | > > > | | > > What I did in that situation was write a couple of methods to > | | > > read > | > | | > > in the file, put it into an array, and then loop through the > | | > > array > | > | | > > and make each call. The good news is you only have to write > | | > > that > > | | > > once and then you can reuse it... > | | > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but > | | > > the > > | | > > logic > | | > > | | > > would be the same. Sybase uses 'go' on a line by itself to end > | | > > a > > | | > > SQL > | | > | | > > command, so we just use that to split up the lines in the file > | | > > into commands into the array. > | | > > > | | > > You could make this a lot fancier, if you had the need, but it > | | > > works > | | > | | > > for me. > | | > > > | | > > Good luck, > | | > > -- Mike > | | > > > | | > > (sub db_run_script and sub db_run_command_list, below) > | | > > > | | > > sub db_run_script #($$) > | | > > { > | | > > my $dbh = shift; > | | > > my $script = shift; > | | > > my $saveresults = shift; > | | > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > file > | | > $script: > | | > > $!"; > | | > > > | | > > my @commands = (); > | | > > my $j = 0; > | | > > my ($line); > | | > > > | | > > # read script file into a variable (array of commands) > | | > > while ($line = ) > | | > > { > | | > > if ($line =~ /^go/) > | | > > { > | | > > # make new command > | | > > $j++; > | | > > } > | | > > elsif ($line =~ /^\s*$/) > | | > > { > | | > > # ignore blank lines > | | > > } > | | > > else > | | > > { > | | > > $commands[$j] .= $line; > | | > > } > | | > > } > | | > > close SQL_SCRIPT; > | | > > > | | > > return db_run_command_list($dbh, \@commands, > $saveresults); } > | | > > > | | > > sub db_run_command_list > | | > > { > | | > > my $dbh = shift; > | | > > my $cmdlist = shift; > | | > > my $saveresults = shift; > | | > > > | | > > my @resultlist; > | | > > > | | > > # run commands from array > | | > > for $j(0..$#$cmdlist) > | | > > { > | | > > $dbh->dbcmd($cmdlist->[$j]); > | | > > my $status; > | | > > eval { > | | > > $status = $dbh->dbsqlexec(); > | | > > }; > | | > > > | | > > if ($@ || $status != SUCCEED) > | | > > { > | | > > # don't always die, because drop will > fail > | | > sometimes > | | > > if ($cmdlist->[$j] =~ /drop/i) > | | > > { > | | > > warn "$cmdlist->[$j] failed.\n > This is > | | > OK - item probably didn't > | | > > exist before installation.\n"; > | | > > > | | > > $dbh->dbcancel(); # so that we > can move > | | > on to the next command? > | | > > } > | | > > else > | | > > { > | | > > die "+++ Could not run command > | | > $cmdlist->[$j]\nbecause of this > | | > > problem:\n$@"; > | | > > } > | | > > } > | | > > > | | > > if (!$saveresults) { > | | > > db_ignore_results($dbh); > | | > > } else { > | | > > # Count the total number of rows that were > updated, > | | > > # and capture the output of any SELECT > statements > | | > > # > | | > > # Each update/insert statement will have its > own > | | > update > | | > > # count (a separate call to DBCOUNT()) but we > will > | | > > # just add them all together > | | > > my $totalupdatecount = 0; > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > | | > > my $rcount = $dbh->DBCOUNT(); > | | > > if ($rcount != -1) { > | | > > $totalupdatecount += $rcount; > | | > > } > | | > > > | | > > my @res; > | | > > while (@res = $dbh->dbnextrow()) { > | | > > my @copyres = @res; # make a copy of > the > | | > array > | | > > push @resultlist, \@copyres; > | | > > } > | | > > } > | | > > > | | > > push @resultlist, $totalupdatecount; > | | > > } > | | > > } > | | > > > | | > > if ($saveresults) { > | | > > return \@resultlist; > | | > > } else { > | | > > return; > | | > > } > | | > > } > | | > > > | | > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > | | > > > | | > > > Hi, > | | > > > > | | > > > I know that you are able to issue a SQL statement within > | | > > > PERL DBI, > | | > | | > > > but is there anyway that I can issue an external SQL program? > > | | > > > For > | | > | | > > > example, I have a SQL program called ppv_insert.sql that I > | | > > > would > | > | | > > > like to execute within PERL DBI. > | | > > > > | | > > > Thanks in advance. > | | > > > > | | > > > Peter Loo > | | > > > > | | > > > > | | > > > > | | > > > > | | > > > This E-mail message is for the sole use of the intended > | | > > > recipient > | | > > > (s) and may contain confidential and privileged information. > | | > > > Any unauthorized review, use, disclosure or distribution is > | | prohibited. > | | > > | | > > > If you are not the intended recipient, please contact the > | | > > > sender > | > | | > > > by reply E-mail, and destroy all copies of the original > message. > | | > > > _______________________________________________ > | | > > > Phoenix-pm mailing list > | | > > > Phoenix-pm at pm.org > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > | | > > > | | > > > | | ------------------------------------------------------------------ > | | -- > | | - > | | > > Michael Friedman HighWire Press > | | > > Phone: 650-725-1974 Stanford University > | | > > FAX: 270-721-8034 > | | > | | > > -------------------------------------------------------------- > | | > > -- > | | > > -- > | | > > -- > | | > > - > | | > > > | | > > > | | > > _______________________________________________ > | | > > Phoenix-pm mailing list > | | > > Phoenix-pm at pm.org > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > | | > > | | > > | | > This E-mail message is for the sole use of the intended > | | > recipient(s) > | > | | > and may contain confidential and privileged information. Any > | | > unauthorized review, use, disclosure or distribution is > prohibited. > | | > If you are not the intended recipient, please contact the sender > | | > by reply E-mail, and destroy all copies of the original message. > | | > > | | > > | | > This E-mail message is for the sole use of the intended > | | > recipient(s) > | | and may contain confidential and privileged information. Any > | | unauthorized review, use, disclosure or distribution is prohibited. > | | If you are not the intended recipient, please contact the sender > | | by reply E-mail, and destroy all copies of the original message. > | | > | | > | | This E-mail message is for the sole use of the intended > | | recipient(s) > > | | and may contain confidential and privileged information. Any > | | unauthorized review, use, disclosure or distribution is prohibited. > | | If you are not the intended recipient, please contact the sender > | | by reply E-mail, and destroy all copies of the original message. > | | > | | > | | This E-mail message is for the sole use of the intended > | | recipient(s) > | and may contain confidential and privileged information. Any > | unauthorized review, use, disclosure or distribution is prohibited. > | If you are not the intended recipient, please contact the sender by > | reply E-mail, and destroy all copies of the original message. > | | _______________________________________________ > | | Phoenix-pm mailing list > | | Phoenix-pm at pm.org > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > | > | > | This E-mail message is for the sole use of the intended recipient(s) > | and may contain confidential and privileged information. Any > | unauthorized review, use, disclosure or distribution is prohibited. > | If you are not the intended recipient, please contact the sender by > | reply E-mail, and destroy all copies of the original message. > | > | > | This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From scott at illogics.org Wed Mar 15 15:48:45 2006 From: scott at illogics.org (Scott Walters) Date: Wed, 15 Mar 2006 23:48:45 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95E87@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95E87@phxmail02.phx.ndchealth.com> Message-ID: <20060315234845.GS22790@illogics.org> Hi Peter, I thought I was "the greatest". You changed your mind in a hurry. Sorry if I hurt your feelings. From my point of view, several people were telling you things that were absolutely correct and helpful, but they weren't "hitting their mark", so to speak. But it's often this way. Novices ask for advice; they reject what they're given, mistakeningly thinking their case is special and they're just misunderstood and then repeat their question, over and over; people repeat their answers, correct in their assessment of the situation; everyone gets frustrated. I just hurry the process along when I see things starting to go in circles. Sometimes the novice in question suddenly "gets it"; sometimes pride gets in the way, and being yelled at is considered unacceptable, even if it really is necessary. Some will disagree as to its necessity; for them, I offer a few years on a Perl help channel on IRC >=) We'll talk about it after then. Anyway, I'm not aggitated at you. I just thought the answers needed some more emphasis. I'm sorry that you read it that way. It is true that I'm a grumpy old cuss. If you don't post on any list with a grupy old cuss, well, you're limiting yourself to not many lists ;) -scott On 0, "Loo, Peter # PHX" wrote: > > Hi Scott, > > Sorry that is I have gotten into your skin, however, I thought this was > a group to discuss and share ideas without any limitation of the level > of PERL knowledge. It appear to me that you just don't like anything > that is not your way. Don't worry about blocking me out of this group > as I don't plan to return. Brock, Mike and the rest, I am sincerely > grateful for your inputs. > > Best regards. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Wednesday, March 15, 2006 3:58 PM > To: Loo, Peter # PHX > Cc: Brock; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Hi Peter, > > > My goal is to have a generic PERL sub_routine that I can pass one to > > many arguments (including the sql file name) and have it perform the > > SQL statement that is within the sql file dynamically so that we can > > reuse the same sub_routine for all select statement sql files. Does > > that make sense? I am all for reusing of code and not reinvent the > > wheel each > > Stop repeating it this. If it doesn't make sense, then do something > else. > If it does, then we get it already. > > You have a choice: you can re-implement parts of the SQL shell in Perl, > or you can call the SQL shell. > > You keep talking about reading the file and passing the SQL... you know > how to do both of these things already. > > Funneling all SQL SELECT statements through one routine is silly. > Refactor your code however you like, but this artifical goal will only > hurt you. If you find you're writing a lot of similar routines, then > learn how to write closures. No, I don't want to talk about this or > debate it; no slight intended, but this kind of simplistic view of "one > routine to do all" is characteristic of novices. People who have been > doing this for a while know there's give and take. Routines get split > out, then made into object methods, and those objects get configured > with more objects they delegate to, then common logic is re-grouped into > a routine somewhere, and so on. Trying to do all of any sort of thing > in one places ignores the inherent complexity in a program. That's like > saying "I want to paint the world PINK!". > But, I said this isn't up for discussion. If I see it again, I'm > telling mutt to ignore this thread. > > > time I have an need to run a select statement. I suppose I can read > > in the sql file like Michael Friedman had suggested earlier in the > > chain of emails. I was just hoping not to have to read in the sql > > statement contained in the sql file and assigning it to a variable > > before doing the $dbh->prepare. > > You want to execute it in Perl and not read it in? I think you need to > focus on what you're really trying to accomplish and concentrate less on > how you do it, because you're starting to get silly. I can't imagine > the real problem actually suggests a reading/not-reading paradox. > > > I need to do some brain storming. :) Not going to give up yet as I > > am dealing with one of my two favorite languages. :) > > If you want to run it through Perl, read it in and DBI. If you want to > run it in the database shell, either do so directly or from a > subprocess. > Either way, quit trying to do both-and-neither-at-the-same-time. I > assure you it will be entirely unproductive. > > By the way, there's often no clear solution (at least the programmer > working on something), and in that case, don't fuss with it endlessly. > Do it one way and be done with it, and if a clear solution occurs to you > later, go back and change it. > > -scott > > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Brock [mailto:awwaiid at thelackthereof.org] > > Sent: Wednesday, March 15, 2006 2:02 PM > > To: Loo, Peter # PHX > > Cc: Scott Walters; phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > Using DBI is completely different then piping things to sqlplus. I was > > > just doing a direct translation of your example, not really trying to > > imply that it was a good idea. > > > > Perl DBI does NOT use sqlplus as the driver. > > > > Unfortunately your question doesn't make much sense... if your .sql > > has a select it could have many selects and it could have all sorts of > > > things. The problem is that how can your program know what it > contains? > > It seems what you need to do is put the select like queries directly > > into perl, as we demonstrated to you in earlier emails. Doing the > > whole fetchrow_arrayref thing that you were already doing. > > > > Make sense? > > > > --Brock > > > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > > | > > | Hi Brock, > > | > > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't > > | be efficient. Secondly, what if you have a "SELECT" statement in > > | the .sql program and if you want to loop through each row? > > | > > | Peter Loo > > | Wolters Kluwer Health > > | (602) 381-9553 > > | > > | -----Original Message----- > > | From: Brock [mailto:awwaiid at thelackthereof.org] > > | Sent: Wednesday, March 15, 2006 1:52 PM > > | To: Loo, Peter # PHX > > | Cc: Scott Walters; phoenix-pm at pm.org > > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > | > > | Sure... what you are doing here is opening an external program and > > | piping it data on STDIN. There are several ways to do this in > Perl... > > | > > | Here's one (a rough guess): > > | > > | # Initiate those vars (pull them from %ENV?) > > | my $dbuser = ...; > > | my $dbpass = ...; > > | my $dbconn = ...; > > | my $mailProgFile = ...; > > | > > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; > > | print $sqlplus <<" HERE"; > > | connect ${dbuser}/${dbpass}@${dbconn} > > | @${mailProgFile}.sql > > | HERE > > | > > | More or less. Eh? > > | > > | --Brock > > | > > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > > | | > > | | Hi Scott, > > | | > > | | So will it be correct to assume that PERL DBI can not execute an > > | | SQL > > > > | | program? For example, I can do this with Korn shell: > > | | > > | | sqlplus -s /NOLOG << EOF > > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > > | | @${MailProgFile}.sql > > | | > > | | Is this not possible in PERL DBI? > > | | > > | | Peter Loo > > | | Wolters Kluwer Health > > | | (602) 381-9553 > > | | > > | | -----Original Message----- > > | | From: Scott Walters [mailto:scott at illogics.org] > > | | Sent: Tuesday, March 14, 2006 9:36 PM > > | | To: Loo, Peter # PHX > > | | Cc: Michael Friedman; phoenix-pm at pm.org > > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > | | > > | | Consider using GetOpt::Std, and most of the time you want this > > | | form of > > | | for: > > | | > > | | for my $thing (@things) { ... stuff with $thing ... } > > | | > > | | You can always try to read rows and trap errors. > > | | > > | | -scott > > | | > > | | On 0, "Loo, Peter # PHX" > > wrote: > > | | > > > | | > Thanks Michael and Scott. What I am trying to do is creating a > > | | > generic PERL program that will take in multiple arguments such > as: > > | | > > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > > | | > switch ($flag) { > > | | > case "-dd" { $d_dbName = lc($value) } > > | | > case "-dt" { $d_tblName = lc($value) } > > | | > case "-ds" { $d_SQL = $value } > > | | > case "-sd" { $s_dbName = lc($value) } > > | | > case "-st" { $s_tblName = lc($value) } > > | | > case "-ss" { $s_SQL = $value } > > | | > case "-cp" { $commitPoint = lc($value) } > > | | > case "-sf" { $s_funcToPerf = lc($value) } > > | | > case "-df" { $d_funcToPerf = lc($value) } > > | | > case "-d1" { $s_dbDriver = lc($value) } > > | | > case "-d2" { $d_dbDriver = lc($value) } > > | | > else { print "Unknown flag: $flag\n" } > > | | > } > > | | > } > > | | > > > | | > Then execute accordingly, however, I would like to execute an > > | | > external > > | | > > | | > SQL program that is passed to this generic program. In the case > > > | | > of an > > | | > > | | > external SQL program that does SELECT instead of UPDATE or > > | | > INSERT, > > > > | | > I > > | > > | | > want to loop through the returned rows. Here is my first block > > | | > of > > | | "if" > > | | > statement. > > | | > > > | | > if ($s_funcToPerf eq "update") { > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$commitPoint > > | | > || > > | | > !$s_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example: \n"; > > | | > print " Database Name: -sd=dv26\n"; > > | | > print " Database Driver: -d1=Oracle\n"; > > | | > print " Table Name: -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | -ss=/usr/local/sql/ppv_update.sql\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_update()\n"; > > | | > } > > | | > } > > | | > elsif ($s_funcToPerf eq "insert") { > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$commitPoint > > | | > || > > | | > !$s_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example: \n"; > > | | > print " Database Name: -sd=dv26\n"; > > | | > print " Database Driver: -d1=Oracle\n"; > > | | > print " Table Name: -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | -ss=/usr/local/sql/ppv_insert.sql\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_insert()\n"; > > | | > } > > | | > } > > | | > elsif ($d_funcToPerf eq "update") { > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example:\n"; > > | | > print " Destination Database Name: -dd=pv26\n"; > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > | | > print " Destination Table Name: > > | | -dt=p_falcon_projections\n"; > > | | > print " Source Database Name: -sd=dv26\n"; > > | | > print " Source Database Driver: -d1=Oracle\n"; > > | | > print " Source Table Name: > > | | -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > | | > print " SQL Statement: > > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > > | | > print " Source Function to Perform: -sf=select\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_select()\n"; > > | | > print "Calling sub_update()\n"; > > | | > } > > | | > } > > | | > elsif ($d_funcToPerf eq "insert") { > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example:\n"; > > | | > print " Destination Database Name: -dd=pv26\n"; > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > | | > print " Destination Table Name: > > | | -dt=p_falcon_projections\n"; > > | | > print " Source Database Name: -sd=dv26\n"; > > | | > print " Source Database Driver: -d1=Oracle\n"; > > | | > print " Source Table Name: > > | | -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > | | > print " SQL Statement: > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > | | > print " Source Function to Perform: -sf=select\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_select()\n"; > > | | > print "Calling sub_insert()\n"; > > | | > } > > | | > } > > | | > elsif ($s_funcToPerf eq "select") { > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$commitPoint > > | || > > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > > | | > !$d_funcToPerf > > | || > > | | > !$s_SQL || !$d_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example:\n"; > > | | > print " Destination Database Name: -dd=pv26\n"; > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > | | > print " Destination Table Name: > > | | -dt=p_falcon_projections\n"; > > | | > print " Source Database Name: -sd=dv26\n"; > > | | > print " Source Database Driver: -d1=Oracle\n"; > > | | > print " Source Table Name: > > | | -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > | | > print " SQL Statement: > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > | | > print " Source Function to Perform: -sf=select\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_select()\n"; > > | | > print "Calling sub_insert()\n"; > > | | > } > > | | > } > > | | > else { > > | | > print "ERROR: Unknown value for database action to > > perform.\n"; > > | | > exit(666); > > | | > } > > | | > > > | | > > > | | > Peter Loo > > | | > Wolters Kluwer Health > > | | > (602) 381-9553 > > | | > > > | | > -----Original Message----- > > | | > From: Scott Walters [mailto:scott at illogics.org] > > | | > Sent: Tuesday, March 14, 2006 3:28 PM > > | | > To: Michael Friedman > > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > | | > > > | | > Hi Peter, > > | | > > > | | > Surely you're trying to accomplish more than just running the > > | | > SQL or > > | > > | | > you would just read it in an feed it to DBI. There's no reason > > | | > you couldn't call to the database command shell: > > | | > > > | | > if(my $pid = fork) { > > | | > waitpid $pid; > > | | > } else { > > | | > close STDIN; > > | | > open STDIN, '<', 'foo.sql' or die $!; > > | | > exec 'mysql', $dbname or die $!; > > | | > } > > | | > > > | | > ... or something like that. If you want to use special features > > > | | > of the database command shell or just cash in on its speed, this > > > | | > might be > > | | > > | | > handy. Of course, you don't want to try to read values from the > > > | | > database back into Perl over a pipe between two processes... > > | | > that's just nasty. > > | | > > > | | > -scott > > | | > > > | | > > > | | > On 0, Michael Friedman wrote: > > | | > > Peter, > > | | > > > > | | > > What I did in that situation was write a couple of methods to > > | | > > read > > | > > | | > > in the file, put it into an array, and then loop through the > > | | > > array > > | > > | | > > and make each call. The good news is you only have to write > > | | > > that > > > > | | > > once and then you can reuse it... > > | | > > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but > > | | > > the > > > > | | > > logic > > | | > > > | | > > would be the same. Sybase uses 'go' on a line by itself to end > > > | | > > a > > > > | | > > SQL > > | | > > | | > > command, so we just use that to split up the lines in the file > > > | | > > into commands into the array. > > | | > > > > | | > > You could make this a lot fancier, if you had the need, but it > > > | | > > works > > | | > > | | > > for me. > > | | > > > > | | > > Good luck, > > | | > > -- Mike > > | | > > > > | | > > (sub db_run_script and sub db_run_command_list, below) > > | | > > > > | | > > sub db_run_script #($$) > > | | > > { > > | | > > my $dbh = shift; > > | | > > my $script = shift; > > | | > > my $saveresults = shift; > > | | > > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > > file > > | | > $script: > > | | > > $!"; > > | | > > > > | | > > my @commands = (); > > | | > > my $j = 0; > > | | > > my ($line); > > | | > > > > | | > > # read script file into a variable (array of commands) > > | | > > while ($line = ) > > | | > > { > > | | > > if ($line =~ /^go/) > > | | > > { > > | | > > # make new command > > | | > > $j++; > > | | > > } > > | | > > elsif ($line =~ /^\s*$/) > > | | > > { > > | | > > # ignore blank lines > > | | > > } > > | | > > else > > | | > > { > > | | > > $commands[$j] .= $line; > > | | > > } > > | | > > } > > | | > > close SQL_SCRIPT; > > | | > > > > | | > > return db_run_command_list($dbh, \@commands, > > $saveresults); } > > | | > > > > | | > > sub db_run_command_list > > | | > > { > > | | > > my $dbh = shift; > > | | > > my $cmdlist = shift; > > | | > > my $saveresults = shift; > > | | > > > > | | > > my @resultlist; > > | | > > > > | | > > # run commands from array > > | | > > for $j(0..$#$cmdlist) > > | | > > { > > | | > > $dbh->dbcmd($cmdlist->[$j]); > > | | > > my $status; > > | | > > eval { > > | | > > $status = $dbh->dbsqlexec(); > > | | > > }; > > | | > > > > | | > > if ($@ || $status != SUCCEED) > > | | > > { > > | | > > # don't always die, because drop will > > fail > > | | > sometimes > > | | > > if ($cmdlist->[$j] =~ /drop/i) > > | | > > { > > | | > > warn "$cmdlist->[$j] failed.\n > > This is > > | | > OK - item probably didn't > > | | > > exist before installation.\n"; > > | | > > > > | | > > $dbh->dbcancel(); # so that we > > can move > > | | > on to the next command? > > | | > > } > > | | > > else > > | | > > { > > | | > > die "+++ Could not run command > > | | > $cmdlist->[$j]\nbecause of this > > | | > > problem:\n$@"; > > | | > > } > > | | > > } > > | | > > > > | | > > if (!$saveresults) { > > | | > > db_ignore_results($dbh); > > | | > > } else { > > | | > > # Count the total number of rows that were > > updated, > > | | > > # and capture the output of any SELECT > > statements > > | | > > # > > | | > > # Each update/insert statement will have its > > own > > | | > update > > | | > > # count (a separate call to DBCOUNT()) but we > > will > > | | > > # just add them all together > > | | > > my $totalupdatecount = 0; > > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > | | > > my $rcount = $dbh->DBCOUNT(); > > | | > > if ($rcount != -1) { > > | | > > $totalupdatecount += $rcount; > > | | > > } > > | | > > > > | | > > my @res; > > | | > > while (@res = $dbh->dbnextrow()) { > > | | > > my @copyres = @res; # make a copy of > > the > > | | > array > > | | > > push @resultlist, \@copyres; > > | | > > } > > | | > > } > > | | > > > > | | > > push @resultlist, $totalupdatecount; > > | | > > } > > | | > > } > > | | > > > > | | > > if ($saveresults) { > > | | > > return \@resultlist; > > | | > > } else { > > | | > > return; > > | | > > } > > | | > > } > > | | > > > > | | > > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > | | > > > > | | > > > Hi, > > | | > > > > > | | > > > I know that you are able to issue a SQL statement within > > | | > > > PERL DBI, > > | | > > | | > > > but is there anyway that I can issue an external SQL > program? > > > > | | > > > For > > | | > > | | > > > example, I have a SQL program called ppv_insert.sql that I > > | | > > > would > > | > > | | > > > like to execute within PERL DBI. > > | | > > > > > | | > > > Thanks in advance. > > | | > > > > > | | > > > Peter Loo > > | | > > > > > | | > > > > > | | > > > > > | | > > > > > | | > > > This E-mail message is for the sole use of the intended > > | | > > > recipient > > | | > > > (s) and may contain confidential and privileged information. > > > | | > > > Any unauthorized review, use, disclosure or distribution is > > | | prohibited. > > | | > > > | | > > > If you are not the intended recipient, please contact the > > | | > > > sender > > | > > | | > > > by reply E-mail, and destroy all copies of the original > > message. > > | | > > > _______________________________________________ > > | | > > > Phoenix-pm mailing list > > | | > > > Phoenix-pm at pm.org > > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > | | > > > > | | > > > > | | ------------------------------------------------------------------ > > | | -- > > | | - > > | | > > Michael Friedman HighWire Press > > | | > > Phone: 650-725-1974 Stanford University > > | | > > FAX: 270-721-8034 > > | | > > | | > > -------------------------------------------------------------- > > | | > > -- > > | | > > -- > > | | > > -- > > | | > > - > > | | > > > > | | > > > > | | > > _______________________________________________ > > | | > > Phoenix-pm mailing list > > | | > > Phoenix-pm at pm.org > > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > | | > > > | | > > > | | > This E-mail message is for the sole use of the intended > > | | > recipient(s) > > | > > | | > and may contain confidential and privileged information. Any > > | | > unauthorized review, use, disclosure or distribution is > > prohibited. > > | | > If you are not the intended recipient, please contact the sender > > > | | > by reply E-mail, and destroy all copies of the original message. > > | | > > > | | > > > | | > This E-mail message is for the sole use of the intended > > | | > recipient(s) > > | | and may contain confidential and privileged information. Any > > | | unauthorized review, use, disclosure or distribution is > prohibited. > > | | If you are not the intended recipient, please contact the sender > > | | by reply E-mail, and destroy all copies of the original message. > > | | > > | | > > | | This E-mail message is for the sole use of the intended > > | | recipient(s) > > > > | | and may contain confidential and privileged information. Any > > | | unauthorized review, use, disclosure or distribution is > prohibited. > > | | If you are not the intended recipient, please contact the sender > > | | by reply E-mail, and destroy all copies of the original message. > > | | > > | | > > | | This E-mail message is for the sole use of the intended > > | | recipient(s) > > | and may contain confidential and privileged information. Any > > | unauthorized review, use, disclosure or distribution is prohibited. > > | If you are not the intended recipient, please contact the sender by > > | reply E-mail, and destroy all copies of the original message. > > | | _______________________________________________ > > | | Phoenix-pm mailing list > > | | Phoenix-pm at pm.org > > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > > | > > | > > | This E-mail message is for the sole use of the intended recipient(s) > > > | and may contain confidential and privileged information. Any > > | unauthorized review, use, disclosure or distribution is prohibited. > > | If you are not the intended recipient, please contact the sender by > > | reply E-mail, and destroy all copies of the original message. > > | > > | > > | This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From friedman at highwire.stanford.edu Wed Mar 15 15:56:32 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Wed, 15 Mar 2006 15:56:32 -0800 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <20060315234845.GS22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95E87@phxmail02.phx.ndchealth.com> <20060315234845.GS22790@illogics.org> Message-ID: Peter, I'm with Scott on this one, minus the attitude. (I'm not nearly as grumpy today. I've had extra caffeine. :-) ) You can't make a completely generic "run this SQL" method. To do so would be to basically be rewriting DBI or one of the similar packages. Also, the DBI model is to send one command at a time and then get results. Using it to send "a file full of requests" is fine, but the way to do it is to open the file and submit each statement separately. It is nonsensical to "submit a SQL file through DBI". DBI isn't built for that model of submission. Now, what I actually think you have here is a case where each file contains one and only one SQL statement. In that degenerate case you'd read in a "file" and get a statement from it, not a list of statements. In which case, you can read in the file, put all the lines into a string variable and then submit that as a statement. But I'm completely guessing. Perhaps some higher-level context of your problem would help us to get a better idea of what you're really trying to do. Asking a detailed question will often get you a detailed answer, but sometimes it's not the answer you're looking for... But yes, this list is for both novices and experts. Please feel free to ask any questions and someone will at least try to help. Scott's great at that, actually, as long as you don't *keep* asking the same question. -- Mike On Mar 15, 2006, at 3:48 PM, Scott Walters wrote: > Hi Peter, > > I thought I was "the greatest". You changed your mind in a hurry. > > Sorry if I hurt your feelings. From my point of view, several people > were telling you things that were absolutely correct and helpful, but > they weren't "hitting their mark", so to speak. But it's often this > way. Novices ask for advice; they reject what they're given, > mistakeningly thinking their case is special and they're just > misunderstood and then repeat their question, over and over; people > repeat their answers, correct in their assessment of the situation; > everyone gets frustrated. I just hurry the process along when I > see things starting to go in circles. Sometimes the novice in > question > suddenly "gets it"; sometimes pride gets in the way, and being yelled > at is considered unacceptable, even if it really is necessary. Some > will disagree as to its necessity; for them, I offer a few years on > a Perl help channel on IRC >=) We'll talk about it after then. > > Anyway, I'm not aggitated at you. I just thought the answers needed > some more emphasis. I'm sorry that you read it that way. It is > true that I'm a grumpy old cuss. If you don't post on any list with > a grupy old cuss, well, you're limiting yourself to not many lists ;) > > -scott > > > On 0, "Loo, Peter # PHX" wrote: >> >> Hi Scott, >> >> Sorry that is I have gotten into your skin, however, I thought >> this was >> a group to discuss and share ideas without any limitation of the >> level >> of PERL knowledge. It appear to me that you just don't like anything >> that is not your way. Don't worry about blocking me out of this >> group >> as I don't plan to return. Brock, Mike and the rest, I am sincerely >> grateful for your inputs. >> >> Best regards. >> >> Peter Loo >> Wolters Kluwer Health >> (602) 381-9553 >> >> -----Original Message----- >> From: Scott Walters [mailto:scott at illogics.org] >> Sent: Wednesday, March 15, 2006 3:58 PM >> To: Loo, Peter # PHX >> Cc: Brock; phoenix-pm at pm.org >> Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI >> >> Hi Peter, >> >>> My goal is to have a generic PERL sub_routine that I can pass one to >>> many arguments (including the sql file name) and have it perform the >>> SQL statement that is within the sql file dynamically so that we can >>> reuse the same sub_routine for all select statement sql files. Does >>> that make sense? I am all for reusing of code and not reinvent the >>> wheel each >> >> Stop repeating it this. If it doesn't make sense, then do something >> else. >> If it does, then we get it already. >> >> You have a choice: you can re-implement parts of the SQL shell in >> Perl, >> or you can call the SQL shell. >> >> You keep talking about reading the file and passing the SQL... you >> know >> how to do both of these things already. >> >> Funneling all SQL SELECT statements through one routine is silly. >> Refactor your code however you like, but this artifical goal will >> only >> hurt you. If you find you're writing a lot of similar routines, then >> learn how to write closures. No, I don't want to talk about this or >> debate it; no slight intended, but this kind of simplistic view of >> "one >> routine to do all" is characteristic of novices. People who have >> been >> doing this for a while know there's give and take. Routines get >> split >> out, then made into object methods, and those objects get configured >> with more objects they delegate to, then common logic is re- >> grouped into >> a routine somewhere, and so on. Trying to do all of any sort of >> thing >> in one places ignores the inherent complexity in a program. >> That's like >> saying "I want to paint the world PINK!". >> But, I said this isn't up for discussion. If I see it again, I'm >> telling mutt to ignore this thread. >> >>> time I have an need to run a select statement. I suppose I can read >>> in the sql file like Michael Friedman had suggested earlier in the >>> chain of emails. I was just hoping not to have to read in the sql >>> statement contained in the sql file and assigning it to a variable >>> before doing the $dbh->prepare. >> >> You want to execute it in Perl and not read it in? I think you >> need to >> focus on what you're really trying to accomplish and concentrate >> less on >> how you do it, because you're starting to get silly. I can't imagine >> the real problem actually suggests a reading/not-reading paradox. >> >>> I need to do some brain storming. :) Not going to give up yet as I >>> am dealing with one of my two favorite languages. :) >> >> If you want to run it through Perl, read it in and DBI. If you >> want to >> run it in the database shell, either do so directly or from a >> subprocess. >> Either way, quit trying to do both-and-neither-at-the-same-time. I >> assure you it will be entirely unproductive. >> >> By the way, there's often no clear solution (at least the programmer >> working on something), and in that case, don't fuss with it >> endlessly. >> Do it one way and be done with it, and if a clear solution occurs >> to you >> later, go back and change it. >> >> -scott >> >>> >>> Peter Loo >>> Wolters Kluwer Health >>> (602) 381-9553 >>> >>> -----Original Message----- >>> From: Brock [mailto:awwaiid at thelackthereof.org] >>> Sent: Wednesday, March 15, 2006 2:02 PM >>> To: Loo, Peter # PHX >>> Cc: Scott Walters; phoenix-pm at pm.org >>> Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI >>> >>> Using DBI is completely different then piping things to sqlplus. >>> I was >> >>> just doing a direct translation of your example, not really >>> trying to >>> imply that it was a good idea. >>> >>> Perl DBI does NOT use sqlplus as the driver. >>> >>> Unfortunately your question doesn't make much sense... if your .sql >>> has a select it could have many selects and it could have all >>> sorts of >> >>> things. The problem is that how can your program know what it >> contains? >>> It seems what you need to do is put the select like queries directly >>> into perl, as we demonstrated to you in earlier emails. Doing the >>> whole fetchrow_arrayref thing that you were already doing. >>> >>> Make sense? >>> >>> --Brock >>> >>> On 2006.03.15.13.55, Loo, Peter # PHX wrote: >>> | >>> | Hi Brock, >>> | >>> | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't >>> | be efficient. Secondly, what if you have a "SELECT" statement in >>> | the .sql program and if you want to loop through each row? >>> | >>> | Peter Loo >>> | Wolters Kluwer Health >>> | (602) 381-9553 >>> | >>> | -----Original Message----- >>> | From: Brock [mailto:awwaiid at thelackthereof.org] >>> | Sent: Wednesday, March 15, 2006 1:52 PM >>> | To: Loo, Peter # PHX >>> | Cc: Scott Walters; phoenix-pm at pm.org >>> | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI >>> | >>> | Sure... what you are doing here is opening an external program and >>> | piping it data on STDIN. There are several ways to do this in >> Perl... >>> | >>> | Here's one (a rough guess): >>> | >>> | # Initiate those vars (pull them from %ENV?) >>> | my $dbuser = ...; >>> | my $dbpass = ...; >>> | my $dbconn = ...; >>> | my $mailProgFile = ...; >>> | >>> | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $! >>> \n"; >>> | print $sqlplus <<" HERE"; >>> | connect ${dbuser}/${dbpass}@${dbconn} >>> | @${mailProgFile}.sql >>> | HERE >>> | >>> | More or less. Eh? >>> | >>> | --Brock >>> | >>> | On 2006.03.15.13.19, Loo, Peter # PHX wrote: >>> | | >>> | | Hi Scott, >>> | | >>> | | So will it be correct to assume that PERL DBI can not execute an >>> | | SQL >>> >>> | | program? For example, I can do this with Korn shell: >>> | | >>> | | sqlplus -s /NOLOG << EOF >>> | | connect ${DBUSER}/${DBPASS}@${DBCONN} >>> | | @${MailProgFile}.sql >>> | | >>> | | Is this not possible in PERL DBI? >>> | | >>> | | Peter Loo >>> | | Wolters Kluwer Health >>> | | (602) 381-9553 >>> | | >>> | | -----Original Message----- >>> | | From: Scott Walters [mailto:scott at illogics.org] >>> | | Sent: Tuesday, March 14, 2006 9:36 PM >>> | | To: Loo, Peter # PHX >>> | | Cc: Michael Friedman; phoenix-pm at pm.org >>> | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI >>> | | >>> | | Consider using GetOpt::Std, and most of the time you want this >>> | | form of >>> | | for: >>> | | >>> | | for my $thing (@things) { ... stuff with $thing ... } >>> | | >>> | | You can always try to read rows and trap errors. >>> | | >>> | | -scott >>> | | >>> | | On 0, "Loo, Peter # PHX" >>> wrote: >>> | | > >>> | | > Thanks Michael and Scott. What I am trying to do is >>> creating a >>> | | > generic PERL program that will take in multiple arguments such >> as: >>> | | > >>> | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { >>> | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); >>> | | > switch ($flag) { >>> | | > case "-dd" { $d_dbName = lc($value) } >>> | | > case "-dt" { $d_tblName = lc($value) } >>> | | > case "-ds" { $d_SQL = $value } >>> | | > case "-sd" { $s_dbName = lc($value) } >>> | | > case "-st" { $s_tblName = lc($value) } >>> | | > case "-ss" { $s_SQL = $value } >>> | | > case "-cp" { $commitPoint = lc($value) } >>> | | > case "-sf" { $s_funcToPerf = lc($value) } >>> | | > case "-df" { $d_funcToPerf = lc($value) } >>> | | > case "-d1" { $s_dbDriver = lc($value) } >>> | | > case "-d2" { $d_dbDriver = lc($value) } >>> | | > else { print "Unknown flag: $flag\n" } >>> | | > } >>> | | > } >>> | | > >>> | | > Then execute accordingly, however, I would like to execute an >>> | | > external >>> | | >>> | | > SQL program that is passed to this generic program. In the >>> case >> >>> | | > of an >>> | | >>> | | > external SQL program that does SELECT instead of UPDATE or >>> | | > INSERT, >>> >>> | | > I >>> | >>> | | > want to loop through the returned rows. Here is my first >>> block >>> | | > of >>> | | "if" >>> | | > statement. >>> | | > >>> | | > if ($s_funcToPerf eq "update") { >>> | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || >>> | | > !$commitPoint >>> | | > || >>> | | > !$s_SQL) { >>> | | > print "ERROR: Not enough arguments. Require arguments >>> | are:\n"; >>> | | > print "Example: \n"; >>> | | > print " Database Name: -sd=dv26\n"; >>> | | > print " Database Driver: -d1=Oracle\n"; >>> | | > print " Table Name: -st=p_falcon_projections\n"; >>> | | > print " SQL Statement: >>> | -ss=/usr/local/sql/ppv_update.sql\n"; >>> | | > print " Commit Point: -cp=5000\n"; >>> | | > exit(666); >>> | | > } >>> | | > else { >>> | | > print "Calling sub_update()\n"; >>> | | > } >>> | | > } >>> | | > elsif ($s_funcToPerf eq "insert") { >>> | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || >>> | | > !$commitPoint >>> | | > || >>> | | > !$s_SQL) { >>> | | > print "ERROR: Not enough arguments. Require arguments >>> | are:\n"; >>> | | > print "Example: \n"; >>> | | > print " Database Name: -sd=dv26\n"; >>> | | > print " Database Driver: -d1=Oracle\n"; >>> | | > print " Table Name: -st=p_falcon_projections\n"; >>> | | > print " SQL Statement: >>> | -ss=/usr/local/sql/ppv_insert.sql\n"; >>> | | > print " Commit Point: -cp=5000\n"; >>> | | > exit(666); >>> | | > } >>> | | > else { >>> | | > print "Calling sub_insert()\n"; >>> | | > } >>> | | > } >>> | | > elsif ($d_funcToPerf eq "update") { >>> | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || >>> | | > !$s_dbName || !$s_dbDriver || !$s_tblName || >>> | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || ! >>> $d_SQL) { >>> | | > print "ERROR: Not enough arguments. Require arguments >>> | are:\n"; >>> | | > print "Example:\n"; >>> | | > print " Destination Database Name: -dd=pv26\n"; >>> | | > print " Destination Database Driver: -d2=ODBC\n"; >>> | | > print " Destination Table Name: >>> | | -dt=p_falcon_projections\n"; >>> | | > print " Source Database Name: -sd=dv26\n"; >>> | | > print " Source Database Driver: -d1=Oracle\n"; >>> | | > print " Source Table Name: >>> | | -st=p_falcon_projections\n"; >>> | | > print " SQL Statement: >>> | | > -ss=/usr/local/sql/ppv_select.sql\n"; >>> | | > print " SQL Statement: >>> | | > -ds=/usr/local/sql/ppv_update.sql\n"; >>> | | > print " Source Function to Perform: -sf=select\n"; >>> | | > print " Commit Point: -cp=5000\n"; >>> | | > exit(666); >>> | | > } >>> | | > else { >>> | | > print "Calling sub_select()\n"; >>> | | > print "Calling sub_update()\n"; >>> | | > } >>> | | > } >>> | | > elsif ($d_funcToPerf eq "insert") { >>> | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || >>> | | > !$s_dbName || !$s_dbDriver || !$s_tblName || >>> | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || ! >>> $d_SQL) { >>> | | > print "ERROR: Not enough arguments. Require arguments >>> | are:\n"; >>> | | > print "Example:\n"; >>> | | > print " Destination Database Name: -dd=pv26\n"; >>> | | > print " Destination Database Driver: -d2=ODBC\n"; >>> | | > print " Destination Table Name: >>> | | -dt=p_falcon_projections\n"; >>> | | > print " Source Database Name: -sd=dv26\n"; >>> | | > print " Source Database Driver: -d1=Oracle\n"; >>> | | > print " Source Table Name: >>> | | -st=p_falcon_projections\n"; >>> | | > print " SQL Statement: >>> | | > -ss=/usr/local/sql/ppv_select.sql\n"; >>> | | > print " SQL Statement: >>> | | > -ds=/usr/local/sql/ppv_insert.sql\n"; >>> | | > print " Source Function to Perform: -sf=select\n"; >>> | | > print " Commit Point: -cp=5000\n"; >>> | | > exit(666); >>> | | > } >>> | | > else { >>> | | > print "Calling sub_select()\n"; >>> | | > print "Calling sub_insert()\n"; >>> | | > } >>> | | > } >>> | | > elsif ($s_funcToPerf eq "select") { >>> | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || >>> | | > !$commitPoint >>> | || >>> | | > !$d_dbName || !$d_dbDriver || !$d_tblName || >>> | | > !$d_funcToPerf >>> | || >>> | | > !$s_SQL || !$d_SQL) { >>> | | > print "ERROR: Not enough arguments. Require arguments >>> | are:\n"; >>> | | > print "Example:\n"; >>> | | > print " Destination Database Name: -dd=pv26\n"; >>> | | > print " Destination Database Driver: -d2=ODBC\n"; >>> | | > print " Destination Table Name: >>> | | -dt=p_falcon_projections\n"; >>> | | > print " Source Database Name: -sd=dv26\n"; >>> | | > print " Source Database Driver: -d1=Oracle\n"; >>> | | > print " Source Table Name: >>> | | -st=p_falcon_projections\n"; >>> | | > print " SQL Statement: >>> | | > -ss=/usr/local/sql/ppv_select.sql\n"; >>> | | > print " SQL Statement: >>> | | > -ds=/usr/local/sql/ppv_insert.sql\n"; >>> | | > print " Source Function to Perform: -sf=select\n"; >>> | | > print " Commit Point: -cp=5000\n"; >>> | | > exit(666); >>> | | > } >>> | | > else { >>> | | > print "Calling sub_select()\n"; >>> | | > print "Calling sub_insert()\n"; >>> | | > } >>> | | > } >>> | | > else { >>> | | > print "ERROR: Unknown value for database action to >>> perform.\n"; >>> | | > exit(666); >>> | | > } >>> | | > >>> | | > >>> | | > Peter Loo >>> | | > Wolters Kluwer Health >>> | | > (602) 381-9553 >>> | | > >>> | | > -----Original Message----- >>> | | > From: Scott Walters [mailto:scott at illogics.org] >>> | | > Sent: Tuesday, March 14, 2006 3:28 PM >>> | | > To: Michael Friedman >>> | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org >>> | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL >>> DBI >>> | | > >>> | | > Hi Peter, >>> | | > >>> | | > Surely you're trying to accomplish more than just running the >>> | | > SQL or >>> | >>> | | > you would just read it in an feed it to DBI. There's no >>> reason >>> | | > you couldn't call to the database command shell: >>> | | > >>> | | > if(my $pid = fork) { >>> | | > waitpid $pid; >>> | | > } else { >>> | | > close STDIN; >>> | | > open STDIN, '<', 'foo.sql' or die $!; >>> | | > exec 'mysql', $dbname or die $!; >>> | | > } >>> | | > >>> | | > ... or something like that. If you want to use special >>> features >> >>> | | > of the database command shell or just cash in on its speed, >>> this >> >>> | | > might be >>> | | >>> | | > handy. Of course, you don't want to try to read values >>> from the >> >>> | | > database back into Perl over a pipe between two processes... >>> | | > that's just nasty. >>> | | > >>> | | > -scott >>> | | > >>> | | > >>> | | > On 0, Michael Friedman >>> wrote: >>> | | > > Peter, >>> | | > > >>> | | > > What I did in that situation was write a couple of >>> methods to >>> | | > > read >>> | >>> | | > > in the file, put it into an array, and then loop through the >>> | | > > array >>> | >>> | | > > and make each call. The good news is you only have to write >>> | | > > that >>> >>> | | > > once and then you can reuse it... >>> | | > > >>> | | > > My only example is using Sybase::DBlib, though, not DBI, but >>> | | > > the >>> >>> | | > > logic >>> | | > >>> | | > > would be the same. Sybase uses 'go' on a line by itself >>> to end >> >>> | | > > a >>> >>> | | > > SQL >>> | | >>> | | > > command, so we just use that to split up the lines in the >>> file >> >>> | | > > into commands into the array. >>> | | > > >>> | | > > You could make this a lot fancier, if you had the need, >>> but it >> >>> | | > > works >>> | | >>> | | > > for me. >>> | | > > >>> | | > > Good luck, >>> | | > > -- Mike >>> | | > > >>> | | > > (sub db_run_script and sub db_run_command_list, below) >>> | | > > >>> | | > > sub db_run_script #($$) >>> | | > > { >>> | | > > my $dbh = shift; >>> | | > > my $script = shift; >>> | | > > my $saveresults = shift; >>> | | > > >>> | | > > open (SQL_SCRIPT, $script) || die "Could not open input >>> file >>> | | > $script: >>> | | > > $!"; >>> | | > > >>> | | > > my @commands = (); >>> | | > > my $j = 0; >>> | | > > my ($line); >>> | | > > >>> | | > > # read script file into a variable (array of commands) >>> | | > > while ($line = ) >>> | | > > { >>> | | > > if ($line =~ /^go/) >>> | | > > { >>> | | > > # make new command >>> | | > > $j++; >>> | | > > } >>> | | > > elsif ($line =~ /^\s*$/) >>> | | > > { >>> | | > > # ignore blank lines >>> | | > > } >>> | | > > else >>> | | > > { >>> | | > > $commands[$j] .= $line; >>> | | > > } >>> | | > > } >>> | | > > close SQL_SCRIPT; >>> | | > > >>> | | > > return db_run_command_list($dbh, \@commands, >>> $saveresults); } >>> | | > > >>> | | > > sub db_run_command_list >>> | | > > { >>> | | > > my $dbh = shift; >>> | | > > my $cmdlist = shift; >>> | | > > my $saveresults = shift; >>> | | > > >>> | | > > my @resultlist; >>> | | > > >>> | | > > # run commands from array >>> | | > > for $j(0..$#$cmdlist) >>> | | > > { >>> | | > > $dbh->dbcmd($cmdlist->[$j]); >>> | | > > my $status; >>> | | > > eval { >>> | | > > $status = $dbh->dbsqlexec(); >>> | | > > }; >>> | | > > >>> | | > > if ($@ || $status != SUCCEED) >>> | | > > { >>> | | > > # don't always die, because drop will >>> fail >>> | | > sometimes >>> | | > > if ($cmdlist->[$j] =~ /drop/i) >>> | | > > { >>> | | > > warn "$cmdlist->[$j] failed.\n >>> This is >>> | | > OK - item probably didn't >>> | | > > exist before installation.\n"; >>> | | > > >>> | | > > $dbh->dbcancel(); # so that we >>> can move >>> | | > on to the next command? >>> | | > > } >>> | | > > else >>> | | > > { >>> | | > > die "+++ Could not run command >>> | | > $cmdlist->[$j]\nbecause of this >>> | | > > problem:\n$@"; >>> | | > > } >>> | | > > } >>> | | > > >>> | | > > if (!$saveresults) { >>> | | > > db_ignore_results($dbh); >>> | | > > } else { >>> | | > > # Count the total number of rows that were >>> updated, >>> | | > > # and capture the output of any SELECT >>> statements >>> | | > > # >>> | | > > # Each update/insert statement will have its >>> own >>> | | > update >>> | | > > # count (a separate call to DBCOUNT()) but we >>> will >>> | | > > # just add them all together >>> | | > > my $totalupdatecount = 0; >>> | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { >>> | | > > my $rcount = $dbh->DBCOUNT(); >>> | | > > if ($rcount != -1) { >>> | | > > $totalupdatecount += $rcount; >>> | | > > } >>> | | > > >>> | | > > my @res; >>> | | > > while (@res = $dbh->dbnextrow()) { >>> | | > > my @copyres = @res; # make a copy of >>> the >>> | | > array >>> | | > > push @resultlist, \@copyres; >>> | | > > } >>> | | > > } >>> | | > > >>> | | > > push @resultlist, $totalupdatecount; >>> | | > > } >>> | | > > } >>> | | > > >>> | | > > if ($saveresults) { >>> | | > > return \@resultlist; >>> | | > > } else { >>> | | > > return; >>> | | > > } >>> | | > > } >>> | | > > >>> | | > > >>> | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: >>> | | > > >>> | | > > > Hi, >>> | | > > > >>> | | > > > I know that you are able to issue a SQL statement within >>> | | > > > PERL DBI, >>> | | >>> | | > > > but is there anyway that I can issue an external SQL >> program? >>> >>> | | > > > For >>> | | >>> | | > > > example, I have a SQL program called ppv_insert.sql that I >>> | | > > > would >>> | >>> | | > > > like to execute within PERL DBI. >>> | | > > > >>> | | > > > Thanks in advance. >>> | | > > > >>> | | > > > Peter Loo >>> | | > > > >>> | | > > > >>> | | > > > >>> | | > > > >>> | | > > > This E-mail message is for the sole use of the intended >>> | | > > > recipient >>> | | > > > (s) and may contain confidential and privileged >>> information. >> >>> | | > > > Any unauthorized review, use, disclosure or >>> distribution is >>> | | prohibited. >>> | | > >>> | | > > > If you are not the intended recipient, please contact the >>> | | > > > sender >>> | >>> | | > > > by reply E-mail, and destroy all copies of the original >>> message. >>> | | > > > _______________________________________________ >>> | | > > > Phoenix-pm mailing list >>> | | > > > Phoenix-pm at pm.org >>> | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm >>> | | > > >>> | | > > >>> | | >>> ------------------------------------------------------------------ >>> | | -- >>> | | - >>> | | > > Michael Friedman HighWire Press >>> | | > > Phone: 650-725-1974 Stanford University >>> | | > > FAX: 270-721-8034 >>> | | >>> | | > > >>> -------------------------------------------------------------- >>> | | > > -- >>> | | > > -- >>> | | > > -- >>> | | > > - >>> | | > > >>> | | > > >>> | | > > _______________________________________________ >>> | | > > Phoenix-pm mailing list >>> | | > > Phoenix-pm at pm.org >>> | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm >>> | | > >>> | | > >>> | | > This E-mail message is for the sole use of the intended >>> | | > recipient(s) >>> | >>> | | > and may contain confidential and privileged information. Any >>> | | > unauthorized review, use, disclosure or distribution is >>> prohibited. >>> | | > If you are not the intended recipient, please contact the >>> sender >> >>> | | > by reply E-mail, and destroy all copies of the original >>> message. >>> | | > >>> | | > >>> | | > This E-mail message is for the sole use of the intended >>> | | > recipient(s) >>> | | and may contain confidential and privileged information. Any >>> | | unauthorized review, use, disclosure or distribution is >> prohibited. >>> | | If you are not the intended recipient, please contact the sender >>> | | by reply E-mail, and destroy all copies of the original message. >>> | | >>> | | >>> | | This E-mail message is for the sole use of the intended >>> | | recipient(s) >>> >>> | | and may contain confidential and privileged information. Any >>> | | unauthorized review, use, disclosure or distribution is >> prohibited. >>> | | If you are not the intended recipient, please contact the sender >>> | | by reply E-mail, and destroy all copies of the original message. >>> | | >>> | | >>> | | This E-mail message is for the sole use of the intended >>> | | recipient(s) >>> | and may contain confidential and privileged information. Any >>> | unauthorized review, use, disclosure or distribution is >>> prohibited. >>> | If you are not the intended recipient, please contact the >>> sender by >>> | reply E-mail, and destroy all copies of the original message. >>> | | _______________________________________________ >>> | | Phoenix-pm mailing list >>> | | Phoenix-pm at pm.org >>> | | http://mail.pm.org/mailman/listinfo/phoenix-pm >>> | >>> | >>> | This E-mail message is for the sole use of the intended >>> recipient(s) >> >>> | and may contain confidential and privileged information. Any >>> | unauthorized review, use, disclosure or distribution is >>> prohibited. >>> | If you are not the intended recipient, please contact the >>> sender by >>> | reply E-mail, and destroy all copies of the original message. >>> | >>> | >>> | This E-mail message is for the sole use of the intended >>> recipient(s) >>> and may contain confidential and privileged information. Any >>> unauthorized review, use, disclosure or distribution is prohibited. >>> If you are not the intended recipient, please contact the sender by >>> reply E-mail, and destroy all copies of the original message. >>> >>> >>> This E-mail message is for the sole use of the intended recipient(s) >>> and may contain confidential and privileged information. Any >>> unauthorized review, use, disclosure or distribution is prohibited. >>> If you are not the intended recipient, please contact the sender by >>> reply E-mail, and destroy all copies of the original message. >>> >>> >>> This E-mail message is for the sole use of the intended recipient(s) >> and may contain confidential and privileged information. Any >> unauthorized review, use, disclosure or distribution is >> prohibited. If >> you are not the intended recipient, please contact the sender by >> reply >> E-mail, and destroy all copies of the original message. >>> _______________________________________________ >>> Phoenix-pm mailing list >>> Phoenix-pm at pm.org >>> http://mail.pm.org/mailman/listinfo/phoenix-pm >> >> >> This E-mail message is for the sole use of the intended recipient >> (s) and >> may contain confidential and privileged information. Any >> unauthorized >> review, use, disclosure or distribution is prohibited. If you are >> not >> the intended recipient, please contact the sender by reply E-mail, >> and >> destroy all copies of the original message. >> >> >> This E-mail message is for the sole use of the intended recipient >> (s) and may contain confidential and privileged information. Any >> unauthorized review, use, disclosure or distribution is >> prohibited. If you are not the intended recipient, please contact >> the sender by reply E-mail, and destroy all copies of the original >> message. >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From Peter.Loo at source.wolterskluwer.com Wed Mar 15 15:57:43 2006 From: Peter.Loo at source.wolterskluwer.com (Loo, Peter # PHX) Date: Wed, 15 Mar 2006 16:57:43 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI Message-ID: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> Scott, This will be my last reply. I may be novice in PERL, but I make six figures yearly so I can't be too dumb to be talked down to like you have. You are absolutely correct. It is NOT acceptable. I have been in IT for 15+ years and have only met a handful of people like yourself who think the only right way is your way and that you are dreaming of being GOD. I am afraid you have missed the boat there. GOD DON'T WRITE PERL OR ANY OTHER LANGUAGE. Nothing in life is static. Everything is different. My way or no way is certainly not the right approach. Seen it all and know it all is even worse. Don't think for a second that you have done it all and have experienced it all. I am quite disappointed by your attitude and certainly won't have too much good to spread about you. If you were frustrated by the emails, why didn't you just delete them and let whoever else to respond. You are rude, conceded and arrogant. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Wednesday, March 15, 2006 4:49 PM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Peter, I thought I was "the greatest". You changed your mind in a hurry. Sorry if I hurt your feelings. From my point of view, several people were telling you things that were absolutely correct and helpful, but they weren't "hitting their mark", so to speak. But it's often this way. Novices ask for advice; they reject what they're given, mistakeningly thinking their case is special and they're just misunderstood and then repeat their question, over and over; people repeat their answers, correct in their assessment of the situation; everyone gets frustrated. I just hurry the process along when I see things starting to go in circles. Sometimes the novice in question suddenly "gets it"; sometimes pride gets in the way, and being yelled at is considered unacceptable, even if it really is necessary. Some will disagree as to its necessity; for them, I offer a few years on a Perl help channel on IRC >=) We'll talk about it after then. Anyway, I'm not aggitated at you. I just thought the answers needed some more emphasis. I'm sorry that you read it that way. It is true that I'm a grumpy old cuss. If you don't post on any list with a grupy old cuss, well, you're limiting yourself to not many lists ;) -scott On 0, "Loo, Peter # PHX" wrote: > > Hi Scott, > > Sorry that is I have gotten into your skin, however, I thought this > was a group to discuss and share ideas without any limitation of the > level of PERL knowledge. It appear to me that you just don't like > anything that is not your way. Don't worry about blocking me out of > this group as I don't plan to return. Brock, Mike and the rest, I am > sincerely grateful for your inputs. > > Best regards. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Wednesday, March 15, 2006 3:58 PM > To: Loo, Peter # PHX > Cc: Brock; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Hi Peter, > > > My goal is to have a generic PERL sub_routine that I can pass one to > > many arguments (including the sql file name) and have it perform the > > SQL statement that is within the sql file dynamically so that we can > > reuse the same sub_routine for all select statement sql files. Does > > that make sense? I am all for reusing of code and not reinvent the > > wheel each > > Stop repeating it this. If it doesn't make sense, then do something > else. > If it does, then we get it already. > > You have a choice: you can re-implement parts of the SQL shell in > Perl, or you can call the SQL shell. > > You keep talking about reading the file and passing the SQL... you > know how to do both of these things already. > > Funneling all SQL SELECT statements through one routine is silly. > Refactor your code however you like, but this artifical goal will only > hurt you. If you find you're writing a lot of similar routines, then > learn how to write closures. No, I don't want to talk about this or > debate it; no slight intended, but this kind of simplistic view of > "one routine to do all" is characteristic of novices. People who have > been doing this for a while know there's give and take. Routines get > split out, then made into object methods, and those objects get > configured with more objects they delegate to, then common logic is > re-grouped into a routine somewhere, and so on. Trying to do all of > any sort of thing in one places ignores the inherent complexity in a > program. That's like saying "I want to paint the world PINK!". > But, I said this isn't up for discussion. If I see it again, I'm > telling mutt to ignore this thread. > > > time I have an need to run a select statement. I suppose I can read > > in the sql file like Michael Friedman had suggested earlier in the > > chain of emails. I was just hoping not to have to read in the sql > > statement contained in the sql file and assigning it to a variable > > before doing the $dbh->prepare. > > You want to execute it in Perl and not read it in? I think you need > to focus on what you're really trying to accomplish and concentrate > less on how you do it, because you're starting to get silly. I can't > imagine the real problem actually suggests a reading/not-reading paradox. > > > I need to do some brain storming. :) Not going to give up yet as I > > am dealing with one of my two favorite languages. :) > > If you want to run it through Perl, read it in and DBI. If you want > to run it in the database shell, either do so directly or from a > subprocess. > Either way, quit trying to do both-and-neither-at-the-same-time. I > assure you it will be entirely unproductive. > > By the way, there's often no clear solution (at least the programmer > working on something), and in that case, don't fuss with it endlessly. > Do it one way and be done with it, and if a clear solution occurs to > you later, go back and change it. > > -scott > > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Brock [mailto:awwaiid at thelackthereof.org] > > Sent: Wednesday, March 15, 2006 2:02 PM > > To: Loo, Peter # PHX > > Cc: Scott Walters; phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > Using DBI is completely different then piping things to sqlplus. I > > was > > > just doing a direct translation of your example, not really trying > > to imply that it was a good idea. > > > > Perl DBI does NOT use sqlplus as the driver. > > > > Unfortunately your question doesn't make much sense... if your .sql > > has a select it could have many selects and it could have all sorts > > of > > > things. The problem is that how can your program know what it > contains? > > It seems what you need to do is put the select like queries directly > > into perl, as we demonstrated to you in earlier emails. Doing the > > whole fetchrow_arrayref thing that you were already doing. > > > > Make sense? > > > > --Brock > > > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > > | > > | Hi Brock, > > | > > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't > > | be efficient. Secondly, what if you have a "SELECT" statement in > > | the .sql program and if you want to loop through each row? > > | > > | Peter Loo > > | Wolters Kluwer Health > > | (602) 381-9553 > > | > > | -----Original Message----- > > | From: Brock [mailto:awwaiid at thelackthereof.org] > > | Sent: Wednesday, March 15, 2006 1:52 PM > > | To: Loo, Peter # PHX > > | Cc: Scott Walters; phoenix-pm at pm.org > > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > | > > | Sure... what you are doing here is opening an external program and > > | piping it data on STDIN. There are several ways to do this in > Perl... > > | > > | Here's one (a rough guess): > > | > > | # Initiate those vars (pull them from %ENV?) > > | my $dbuser = ...; > > | my $dbpass = ...; > > | my $dbconn = ...; > > | my $mailProgFile = ...; > > | > > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; > > | print $sqlplus <<" HERE"; > > | connect ${dbuser}/${dbpass}@${dbconn} > > | @${mailProgFile}.sql > > | HERE > > | > > | More or less. Eh? > > | > > | --Brock > > | > > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > > | | > > | | Hi Scott, > > | | > > | | So will it be correct to assume that PERL DBI can not execute an > > | | SQL > > > > | | program? For example, I can do this with Korn shell: > > | | > > | | sqlplus -s /NOLOG << EOF > > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > > | | @${MailProgFile}.sql > > | | > > | | Is this not possible in PERL DBI? > > | | > > | | Peter Loo > > | | Wolters Kluwer Health > > | | (602) 381-9553 > > | | > > | | -----Original Message----- > > | | From: Scott Walters [mailto:scott at illogics.org] > > | | Sent: Tuesday, March 14, 2006 9:36 PM > > | | To: Loo, Peter # PHX > > | | Cc: Michael Friedman; phoenix-pm at pm.org > > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > | | > > | | Consider using GetOpt::Std, and most of the time you want this > > | | form of > > | | for: > > | | > > | | for my $thing (@things) { ... stuff with $thing ... } > > | | > > | | You can always try to read rows and trap errors. > > | | > > | | -scott > > | | > > | | On 0, "Loo, Peter # PHX" > > wrote: > > | | > > > | | > Thanks Michael and Scott. What I am trying to do is creating > > | | > a generic PERL program that will take in multiple arguments > > | | > such > as: > > | | > > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > > | | > switch ($flag) { > > | | > case "-dd" { $d_dbName = lc($value) } > > | | > case "-dt" { $d_tblName = lc($value) } > > | | > case "-ds" { $d_SQL = $value } > > | | > case "-sd" { $s_dbName = lc($value) } > > | | > case "-st" { $s_tblName = lc($value) } > > | | > case "-ss" { $s_SQL = $value } > > | | > case "-cp" { $commitPoint = lc($value) } > > | | > case "-sf" { $s_funcToPerf = lc($value) } > > | | > case "-df" { $d_funcToPerf = lc($value) } > > | | > case "-d1" { $s_dbDriver = lc($value) } > > | | > case "-d2" { $d_dbDriver = lc($value) } > > | | > else { print "Unknown flag: $flag\n" } > > | | > } > > | | > } > > | | > > > | | > Then execute accordingly, however, I would like to execute an > > | | > external > > | | > > | | > SQL program that is passed to this generic program. In the > > | | > case > > > | | > of an > > | | > > | | > external SQL program that does SELECT instead of UPDATE or > > | | > INSERT, > > > > | | > I > > | > > | | > want to loop through the returned rows. Here is my first > > | | > block of > > | | "if" > > | | > statement. > > | | > > > | | > if ($s_funcToPerf eq "update") { > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$commitPoint > > | | > || > > | | > !$s_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example: \n"; > > | | > print " Database Name: -sd=dv26\n"; > > | | > print " Database Driver: -d1=Oracle\n"; > > | | > print " Table Name: -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | -ss=/usr/local/sql/ppv_update.sql\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_update()\n"; > > | | > } > > | | > } > > | | > elsif ($s_funcToPerf eq "insert") { > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$commitPoint > > | | > || > > | | > !$s_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example: \n"; > > | | > print " Database Name: -sd=dv26\n"; > > | | > print " Database Driver: -d1=Oracle\n"; > > | | > print " Table Name: -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | -ss=/usr/local/sql/ppv_insert.sql\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_insert()\n"; > > | | > } > > | | > } > > | | > elsif ($d_funcToPerf eq "update") { > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example:\n"; > > | | > print " Destination Database Name: -dd=pv26\n"; > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > | | > print " Destination Table Name: > > | | -dt=p_falcon_projections\n"; > > | | > print " Source Database Name: -sd=dv26\n"; > > | | > print " Source Database Driver: -d1=Oracle\n"; > > | | > print " Source Table Name: > > | | -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > | | > print " SQL Statement: > > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > > | | > print " Source Function to Perform: -sf=select\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_select()\n"; > > | | > print "Calling sub_update()\n"; > > | | > } > > | | > } > > | | > elsif ($d_funcToPerf eq "insert") { > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example:\n"; > > | | > print " Destination Database Name: -dd=pv26\n"; > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > | | > print " Destination Table Name: > > | | -dt=p_falcon_projections\n"; > > | | > print " Source Database Name: -sd=dv26\n"; > > | | > print " Source Database Driver: -d1=Oracle\n"; > > | | > print " Source Table Name: > > | | -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > | | > print " SQL Statement: > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > | | > print " Source Function to Perform: -sf=select\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_select()\n"; > > | | > print "Calling sub_insert()\n"; > > | | > } > > | | > } > > | | > elsif ($s_funcToPerf eq "select") { > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > | | > !$commitPoint > > | || > > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > > | | > !$d_funcToPerf > > | || > > | | > !$s_SQL || !$d_SQL) { > > | | > print "ERROR: Not enough arguments. Require arguments > > | are:\n"; > > | | > print "Example:\n"; > > | | > print " Destination Database Name: -dd=pv26\n"; > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > | | > print " Destination Table Name: > > | | -dt=p_falcon_projections\n"; > > | | > print " Source Database Name: -sd=dv26\n"; > > | | > print " Source Database Driver: -d1=Oracle\n"; > > | | > print " Source Table Name: > > | | -st=p_falcon_projections\n"; > > | | > print " SQL Statement: > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > | | > print " SQL Statement: > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > | | > print " Source Function to Perform: -sf=select\n"; > > | | > print " Commit Point: -cp=5000\n"; > > | | > exit(666); > > | | > } > > | | > else { > > | | > print "Calling sub_select()\n"; > > | | > print "Calling sub_insert()\n"; > > | | > } > > | | > } > > | | > else { > > | | > print "ERROR: Unknown value for database action to > > perform.\n"; > > | | > exit(666); > > | | > } > > | | > > > | | > > > | | > Peter Loo > > | | > Wolters Kluwer Health > > | | > (602) 381-9553 > > | | > > > | | > -----Original Message----- > > | | > From: Scott Walters [mailto:scott at illogics.org] > > | | > Sent: Tuesday, March 14, 2006 3:28 PM > > | | > To: Michael Friedman > > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL > > | | > DBI > > | | > > > | | > Hi Peter, > > | | > > > | | > Surely you're trying to accomplish more than just running the > > | | > SQL or > > | > > | | > you would just read it in an feed it to DBI. There's no > > | | > reason you couldn't call to the database command shell: > > | | > > > | | > if(my $pid = fork) { > > | | > waitpid $pid; > > | | > } else { > > | | > close STDIN; > > | | > open STDIN, '<', 'foo.sql' or die $!; > > | | > exec 'mysql', $dbname or die $!; > > | | > } > > | | > > > | | > ... or something like that. If you want to use special > > | | > features > > > | | > of the database command shell or just cash in on its speed, > > | | > this > > > | | > might be > > | | > > | | > handy. Of course, you don't want to try to read values from > > | | > the > > > | | > database back into Perl over a pipe between two processes... > > | | > that's just nasty. > > | | > > > | | > -scott > > | | > > > | | > > > | | > On 0, Michael Friedman wrote: > > | | > > Peter, > > | | > > > > | | > > What I did in that situation was write a couple of methods > > | | > > to read > > | > > | | > > in the file, put it into an array, and then loop through the > > | | > > array > > | > > | | > > and make each call. The good news is you only have to write > > | | > > that > > > > | | > > once and then you can reuse it... > > | | > > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but > > | | > > the > > > > | | > > logic > > | | > > > | | > > would be the same. Sybase uses 'go' on a line by itself to > > | | > > end > > > | | > > a > > > > | | > > SQL > > | | > > | | > > command, so we just use that to split up the lines in the > > | | > > file > > > | | > > into commands into the array. > > | | > > > > | | > > You could make this a lot fancier, if you had the need, but > > | | > > it > > > | | > > works > > | | > > | | > > for me. > > | | > > > > | | > > Good luck, > > | | > > -- Mike > > | | > > > > | | > > (sub db_run_script and sub db_run_command_list, below) > > | | > > > > | | > > sub db_run_script #($$) > > | | > > { > > | | > > my $dbh = shift; > > | | > > my $script = shift; > > | | > > my $saveresults = shift; > > | | > > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > > file > > | | > $script: > > | | > > $!"; > > | | > > > > | | > > my @commands = (); > > | | > > my $j = 0; > > | | > > my ($line); > > | | > > > > | | > > # read script file into a variable (array of commands) > > | | > > while ($line = ) > > | | > > { > > | | > > if ($line =~ /^go/) > > | | > > { > > | | > > # make new command > > | | > > $j++; > > | | > > } > > | | > > elsif ($line =~ /^\s*$/) > > | | > > { > > | | > > # ignore blank lines > > | | > > } > > | | > > else > > | | > > { > > | | > > $commands[$j] .= $line; > > | | > > } > > | | > > } > > | | > > close SQL_SCRIPT; > > | | > > > > | | > > return db_run_command_list($dbh, \@commands, > > $saveresults); } > > | | > > > > | | > > sub db_run_command_list > > | | > > { > > | | > > my $dbh = shift; > > | | > > my $cmdlist = shift; > > | | > > my $saveresults = shift; > > | | > > > > | | > > my @resultlist; > > | | > > > > | | > > # run commands from array > > | | > > for $j(0..$#$cmdlist) > > | | > > { > > | | > > $dbh->dbcmd($cmdlist->[$j]); > > | | > > my $status; > > | | > > eval { > > | | > > $status = $dbh->dbsqlexec(); > > | | > > }; > > | | > > > > | | > > if ($@ || $status != SUCCEED) > > | | > > { > > | | > > # don't always die, because drop will > > fail > > | | > sometimes > > | | > > if ($cmdlist->[$j] =~ /drop/i) > > | | > > { > > | | > > warn "$cmdlist->[$j] failed.\n > > This is > > | | > OK - item probably didn't > > | | > > exist before installation.\n"; > > | | > > > > | | > > $dbh->dbcancel(); # so that we > > can move > > | | > on to the next command? > > | | > > } > > | | > > else > > | | > > { > > | | > > die "+++ Could not run command > > | | > $cmdlist->[$j]\nbecause of this > > | | > > problem:\n$@"; > > | | > > } > > | | > > } > > | | > > > > | | > > if (!$saveresults) { > > | | > > db_ignore_results($dbh); > > | | > > } else { > > | | > > # Count the total number of rows that were > > updated, > > | | > > # and capture the output of any SELECT > > statements > > | | > > # > > | | > > # Each update/insert statement will have its > > own > > | | > update > > | | > > # count (a separate call to DBCOUNT()) but we > > will > > | | > > # just add them all together > > | | > > my $totalupdatecount = 0; > > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > | | > > my $rcount = $dbh->DBCOUNT(); > > | | > > if ($rcount != -1) { > > | | > > $totalupdatecount += $rcount; > > | | > > } > > | | > > > > | | > > my @res; > > | | > > while (@res = $dbh->dbnextrow()) { > > | | > > my @copyres = @res; # make a copy of > > the > > | | > array > > | | > > push @resultlist, \@copyres; > > | | > > } > > | | > > } > > | | > > > > | | > > push @resultlist, $totalupdatecount; > > | | > > } > > | | > > } > > | | > > > > | | > > if ($saveresults) { > > | | > > return \@resultlist; > > | | > > } else { > > | | > > return; > > | | > > } > > | | > > } > > | | > > > > | | > > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > | | > > > > | | > > > Hi, > > | | > > > > > | | > > > I know that you are able to issue a SQL statement within > > | | > > > PERL DBI, > > | | > > | | > > > but is there anyway that I can issue an external SQL > program? > > > > | | > > > For > > | | > > | | > > > example, I have a SQL program called ppv_insert.sql that I > > | | > > > would > > | > > | | > > > like to execute within PERL DBI. > > | | > > > > > | | > > > Thanks in advance. > > | | > > > > > | | > > > Peter Loo > > | | > > > > > | | > > > > > | | > > > > > | | > > > > > | | > > > This E-mail message is for the sole use of the intended > > | | > > > recipient > > | | > > > (s) and may contain confidential and privileged information. > > > | | > > > Any unauthorized review, use, disclosure or distribution > > | | > > > is > > | | prohibited. > > | | > > > | | > > > If you are not the intended recipient, please contact the > > | | > > > sender > > | > > | | > > > by reply E-mail, and destroy all copies of the original > > message. > > | | > > > _______________________________________________ > > | | > > > Phoenix-pm mailing list > > | | > > > Phoenix-pm at pm.org > > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > | | > > > > | | > > > > | | ---------------------------------------------------------------- > > | | -- > > | | -- > > | | - > > | | > > Michael Friedman HighWire Press > > | | > > Phone: 650-725-1974 Stanford University > > | | > > FAX: 270-721-8034 > > | | > > | | > > ------------------------------------------------------------ > > | | > > -- > > | | > > -- > > | | > > -- > > | | > > -- > > | | > > - > > | | > > > > | | > > > > | | > > _______________________________________________ > > | | > > Phoenix-pm mailing list > > | | > > Phoenix-pm at pm.org > > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > | | > > > | | > > > | | > This E-mail message is for the sole use of the intended > > | | > recipient(s) > > | > > | | > and may contain confidential and privileged information. Any > > | | > unauthorized review, use, disclosure or distribution is > > prohibited. > > | | > If you are not the intended recipient, please contact the > > | | > sender > > > | | > by reply E-mail, and destroy all copies of the original message. > > | | > > > | | > > > | | > This E-mail message is for the sole use of the intended > > | | > recipient(s) > > | | and may contain confidential and privileged information. Any > > | | unauthorized review, use, disclosure or distribution is > prohibited. > > | | If you are not the intended recipient, please contact the sender > > | | by reply E-mail, and destroy all copies of the original message. > > | | > > | | > > | | This E-mail message is for the sole use of the intended > > | | recipient(s) > > > > | | and may contain confidential and privileged information. Any > > | | unauthorized review, use, disclosure or distribution is > prohibited. > > | | If you are not the intended recipient, please contact the sender > > | | by reply E-mail, and destroy all copies of the original message. > > | | > > | | > > | | This E-mail message is for the sole use of the intended > > | | recipient(s) > > | and may contain confidential and privileged information. Any > > | unauthorized review, use, disclosure or distribution is prohibited. > > | If you are not the intended recipient, please contact the sender > > | by reply E-mail, and destroy all copies of the original message. > > | | _______________________________________________ > > | | Phoenix-pm mailing list > > | | Phoenix-pm at pm.org > > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > > | > > | > > | This E-mail message is for the sole use of the intended > > | recipient(s) > > > | and may contain confidential and privileged information. Any > > | unauthorized review, use, disclosure or distribution is prohibited. > > | If you are not the intended recipient, please contact the sender > > | by reply E-mail, and destroy all copies of the original message. > > | > > | > > | This E-mail message is for the sole use of the intended > > | recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. From bwmetz at att.com Wed Mar 15 16:01:46 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Wed, 15 Mar 2006 18:01:46 -0600 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95E87@phxmail02.phx.ndchealth.com> Message-ID: <01D5341D04A2E64AB9B345769047336701753A08@OCCLUST01EVS1.ugd.att.com> Peter, From another non-expert's point of view, I think I understand that you're wanting to leave the SQL writing in the sqlplus files since it seems you use those outside of your Perl programs for other reasons...at least, I can appreciate that idea. So, no point re-inventing the wheel using DBI in my opinion. You CAN write a generic routine to pass the name of your saved query filename, say your own local module so you can do something like: use mysql_mod; $my_sql_file = "blahblahblah" @results = &mysql_mod::execute($my_sql_file); foreach $line (@results) { # pattern match whatever # call other routines # etc. } This could save you time cut/pasting or re-writing the sqlplus call, redirection, or whatever, each and every time you have a new script need. Again, no point trying to parse the .sql file to perform the queries in DBI...just process the output from sqlplus. Now, on the other hand, if your parsing needs are going to change all the time, I'd say it'd be much simpler to just write a custom Perl script to handle those needs each time and simply rely on STDIN to process your output from sqlplus. This could be just as powerful as long as you don't need to do similar things to different data sets all the time, e.g. sqlplus your_file | get_UPC_numbers.pl Or something equally silly as that. As to leaving the group...give us another chance. I agree with Brock that your original problem statement wasn't well defined and I thought you were just trying to re-state the intended goal as is often necessary when ideas and suggestions are flying all over the place and a clear direction isn't discernable. I think this group is really good at listening to folks of all experience levels. Regards, Bobby -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Loo, Peter # PHX Sent: Wednesday, March 15, 2006 4:06 PM To: Scott Walters Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Scott, Sorry that is I have gotten into your skin, however, I thought this was a group to discuss and share ideas without any limitation of the level of PERL knowledge. It appear to me that you just don't like anything that is not your way. Don't worry about blocking me out of this group as I don't plan to return. Brock, Mike and the rest, I am sincerely grateful for your inputs. Best regards. Peter Loo Wolters Kluwer Health (602) 381-9553 -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Wednesday, March 15, 2006 3:58 PM To: Loo, Peter # PHX Cc: Brock; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Peter, > My goal is to have a generic PERL sub_routine that I can pass one to > many arguments (including the sql file name) and have it perform the > SQL statement that is within the sql file dynamically so that we can > reuse the same sub_routine for all select statement sql files. Does > that make sense? I am all for reusing of code and not reinvent the > wheel each Stop repeating it this. If it doesn't make sense, then do something else. If it does, then we get it already. You have a choice: you can re-implement parts of the SQL shell in Perl, or you can call the SQL shell. You keep talking about reading the file and passing the SQL... you know how to do both of these things already. Funneling all SQL SELECT statements through one routine is silly. Refactor your code however you like, but this artifical goal will only hurt you. If you find you're writing a lot of similar routines, then learn how to write closures. No, I don't want to talk about this or debate it; no slight intended, but this kind of simplistic view of "one routine to do all" is characteristic of novices. People who have been doing this for a while know there's give and take. Routines get split out, then made into object methods, and those objects get configured with more objects they delegate to, then common logic is re-grouped into a routine somewhere, and so on. Trying to do all of any sort of thing in one places ignores the inherent complexity in a program. That's like saying "I want to paint the world PINK!". But, I said this isn't up for discussion. If I see it again, I'm telling mutt to ignore this thread. > time I have an need to run a select statement. I suppose I can read > in the sql file like Michael Friedman had suggested earlier in the > chain of emails. I was just hoping not to have to read in the sql > statement contained in the sql file and assigning it to a variable > before doing the $dbh->prepare. You want to execute it in Perl and not read it in? I think you need to focus on what you're really trying to accomplish and concentrate less on how you do it, because you're starting to get silly. I can't imagine the real problem actually suggests a reading/not-reading paradox. > I need to do some brain storming. :) Not going to give up yet as I > am dealing with one of my two favorite languages. :) If you want to run it through Perl, read it in and DBI. If you want to run it in the database shell, either do so directly or from a subprocess. Either way, quit trying to do both-and-neither-at-the-same-time. I assure you it will be entirely unproductive. By the way, there's often no clear solution (at least the programmer working on something), and in that case, don't fuss with it endlessly. Do it one way and be done with it, and if a clear solution occurs to you later, go back and change it. -scott > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Brock [mailto:awwaiid at thelackthereof.org] > Sent: Wednesday, March 15, 2006 2:02 PM > To: Loo, Peter # PHX > Cc: Scott Walters; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Using DBI is completely different then piping things to sqlplus. I was > just doing a direct translation of your example, not really trying to > imply that it was a good idea. > > Perl DBI does NOT use sqlplus as the driver. > > Unfortunately your question doesn't make much sense... if your .sql > has a select it could have many selects and it could have all sorts of > things. The problem is that how can your program know what it contains? > It seems what you need to do is put the select like queries directly > into perl, as we demonstrated to you in earlier emails. Doing the > whole fetchrow_arrayref thing that you were already doing. > > Make sense? > > --Brock > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > | > | Hi Brock, > | > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't > | be efficient. Secondly, what if you have a "SELECT" statement in > | the .sql program and if you want to loop through each row? > | > | Peter Loo > | Wolters Kluwer Health > | (602) 381-9553 > | > | -----Original Message----- > | From: Brock [mailto:awwaiid at thelackthereof.org] > | Sent: Wednesday, March 15, 2006 1:52 PM > | To: Loo, Peter # PHX > | Cc: Scott Walters; phoenix-pm at pm.org > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | > | Sure... what you are doing here is opening an external program and > | piping it data on STDIN. There are several ways to do this in Perl... > | > | Here's one (a rough guess): > | > | # Initiate those vars (pull them from %ENV?) > | my $dbuser = ...; > | my $dbpass = ...; > | my $dbconn = ...; > | my $mailProgFile = ...; > | > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: $!\n"; > | print $sqlplus <<" HERE"; > | connect ${dbuser}/${dbpass}@${dbconn} > | @${mailProgFile}.sql > | HERE > | > | More or less. Eh? > | > | --Brock > | > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > | | > | | Hi Scott, > | | > | | So will it be correct to assume that PERL DBI can not execute an > | | SQL > > | | program? For example, I can do this with Korn shell: > | | > | | sqlplus -s /NOLOG << EOF > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > | | @${MailProgFile}.sql > | | > | | Is this not possible in PERL DBI? > | | > | | Peter Loo > | | Wolters Kluwer Health > | | (602) 381-9553 > | | > | | -----Original Message----- > | | From: Scott Walters [mailto:scott at illogics.org] > | | Sent: Tuesday, March 14, 2006 9:36 PM > | | To: Loo, Peter # PHX > | | Cc: Michael Friedman; phoenix-pm at pm.org > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | | > | | Consider using GetOpt::Std, and most of the time you want this > | | form of > | | for: > | | > | | for my $thing (@things) { ... stuff with $thing ... } > | | > | | You can always try to read rows and trap errors. > | | > | | -scott > | | > | | On 0, "Loo, Peter # PHX" > wrote: > | | > > | | > Thanks Michael and Scott. What I am trying to do is creating a > | | > generic PERL program that will take in multiple arguments such as: > | | > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > | | > switch ($flag) { > | | > case "-dd" { $d_dbName = lc($value) } > | | > case "-dt" { $d_tblName = lc($value) } > | | > case "-ds" { $d_SQL = $value } > | | > case "-sd" { $s_dbName = lc($value) } > | | > case "-st" { $s_tblName = lc($value) } > | | > case "-ss" { $s_SQL = $value } > | | > case "-cp" { $commitPoint = lc($value) } > | | > case "-sf" { $s_funcToPerf = lc($value) } > | | > case "-df" { $d_funcToPerf = lc($value) } > | | > case "-d1" { $s_dbDriver = lc($value) } > | | > case "-d2" { $d_dbDriver = lc($value) } > | | > else { print "Unknown flag: $flag\n" } > | | > } > | | > } > | | > > | | > Then execute accordingly, however, I would like to execute an > | | > external > | | > | | > SQL program that is passed to this generic program. In the case > | | > of an > | | > | | > external SQL program that does SELECT instead of UPDATE or > | | > INSERT, > > | | > I > | > | | > want to loop through the returned rows. Here is my first block > | | > of > | | "if" > | | > statement. > | | > > | | > if ($s_funcToPerf eq "update") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$commitPoint > | | > || > | | > !$s_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example: \n"; > | | > print " Database Name: -sd=dv26\n"; > | | > print " Database Driver: -d1=Oracle\n"; > | | > print " Table Name: -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | -ss=/usr/local/sql/ppv_update.sql\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_update()\n"; > | | > } > | | > } > | | > elsif ($s_funcToPerf eq "insert") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$commitPoint > | | > || > | | > !$s_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example: \n"; > | | > print " Database Name: -sd=dv26\n"; > | | > print " Database Driver: -d1=Oracle\n"; > | | > print " Table Name: -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | -ss=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > elsif ($d_funcToPerf eq "update") { > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_update()\n"; > | | > } > | | > } > | | > elsif ($d_funcToPerf eq "insert") { > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > elsif ($s_funcToPerf eq "select") { > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > | | > !$commitPoint > | || > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > | | > !$d_funcToPerf > | || > | | > !$s_SQL || !$d_SQL) { > | | > print "ERROR: Not enough arguments. Require arguments > | are:\n"; > | | > print "Example:\n"; > | | > print " Destination Database Name: -dd=pv26\n"; > | | > print " Destination Database Driver: -d2=ODBC\n"; > | | > print " Destination Table Name: > | | -dt=p_falcon_projections\n"; > | | > print " Source Database Name: -sd=dv26\n"; > | | > print " Source Database Driver: -d1=Oracle\n"; > | | > print " Source Table Name: > | | -st=p_falcon_projections\n"; > | | > print " SQL Statement: > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > | | > print " SQL Statement: > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > | | > print " Source Function to Perform: -sf=select\n"; > | | > print " Commit Point: -cp=5000\n"; > | | > exit(666); > | | > } > | | > else { > | | > print "Calling sub_select()\n"; > | | > print "Calling sub_insert()\n"; > | | > } > | | > } > | | > else { > | | > print "ERROR: Unknown value for database action to > perform.\n"; > | | > exit(666); > | | > } > | | > > | | > > | | > Peter Loo > | | > Wolters Kluwer Health > | | > (602) 381-9553 > | | > > | | > -----Original Message----- > | | > From: Scott Walters [mailto:scott at illogics.org] > | | > Sent: Tuesday, March 14, 2006 3:28 PM > | | > To: Michael Friedman > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > | | > > | | > Hi Peter, > | | > > | | > Surely you're trying to accomplish more than just running the > | | > SQL or > | > | | > you would just read it in an feed it to DBI. There's no reason > | | > you couldn't call to the database command shell: > | | > > | | > if(my $pid = fork) { > | | > waitpid $pid; > | | > } else { > | | > close STDIN; > | | > open STDIN, '<', 'foo.sql' or die $!; > | | > exec 'mysql', $dbname or die $!; > | | > } > | | > > | | > ... or something like that. If you want to use special features > | | > of the database command shell or just cash in on its speed, this > | | > might be > | | > | | > handy. Of course, you don't want to try to read values from the > | | > database back into Perl over a pipe between two processes... > | | > that's just nasty. > | | > > | | > -scott > | | > > | | > > | | > On 0, Michael Friedman wrote: > | | > > Peter, > | | > > > | | > > What I did in that situation was write a couple of methods to > | | > > read > | > | | > > in the file, put it into an array, and then loop through the > | | > > array > | > | | > > and make each call. The good news is you only have to write > | | > > that > > | | > > once and then you can reuse it... > | | > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but > | | > > the > > | | > > logic > | | > > | | > > would be the same. Sybase uses 'go' on a line by itself to end > | | > > a > > | | > > SQL > | | > | | > > command, so we just use that to split up the lines in the file > | | > > into commands into the array. > | | > > > | | > > You could make this a lot fancier, if you had the need, but it > | | > > works > | | > | | > > for me. > | | > > > | | > > Good luck, > | | > > -- Mike > | | > > > | | > > (sub db_run_script and sub db_run_command_list, below) > | | > > > | | > > sub db_run_script #($$) > | | > > { > | | > > my $dbh = shift; > | | > > my $script = shift; > | | > > my $saveresults = shift; > | | > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > file > | | > $script: > | | > > $!"; > | | > > > | | > > my @commands = (); > | | > > my $j = 0; > | | > > my ($line); > | | > > > | | > > # read script file into a variable (array of commands) > | | > > while ($line = ) > | | > > { > | | > > if ($line =~ /^go/) > | | > > { > | | > > # make new command > | | > > $j++; > | | > > } > | | > > elsif ($line =~ /^\s*$/) > | | > > { > | | > > # ignore blank lines > | | > > } > | | > > else > | | > > { > | | > > $commands[$j] .= $line; > | | > > } > | | > > } > | | > > close SQL_SCRIPT; > | | > > > | | > > return db_run_command_list($dbh, \@commands, > $saveresults); } > | | > > > | | > > sub db_run_command_list > | | > > { > | | > > my $dbh = shift; > | | > > my $cmdlist = shift; > | | > > my $saveresults = shift; > | | > > > | | > > my @resultlist; > | | > > > | | > > # run commands from array > | | > > for $j(0..$#$cmdlist) > | | > > { > | | > > $dbh->dbcmd($cmdlist->[$j]); > | | > > my $status; > | | > > eval { > | | > > $status = $dbh->dbsqlexec(); > | | > > }; > | | > > > | | > > if ($@ || $status != SUCCEED) > | | > > { > | | > > # don't always die, because drop will > fail > | | > sometimes > | | > > if ($cmdlist->[$j] =~ /drop/i) > | | > > { > | | > > warn "$cmdlist->[$j] failed.\n > This is > | | > OK - item probably didn't > | | > > exist before installation.\n"; > | | > > > | | > > $dbh->dbcancel(); # so that we > can move > | | > on to the next command? > | | > > } > | | > > else > | | > > { > | | > > die "+++ Could not run command > | | > $cmdlist->[$j]\nbecause of this > | | > > problem:\n$@"; > | | > > } > | | > > } > | | > > > | | > > if (!$saveresults) { > | | > > db_ignore_results($dbh); > | | > > } else { > | | > > # Count the total number of rows that were > updated, > | | > > # and capture the output of any SELECT > statements > | | > > # > | | > > # Each update/insert statement will have its > own > | | > update > | | > > # count (a separate call to DBCOUNT()) but we > will > | | > > # just add them all together > | | > > my $totalupdatecount = 0; > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > | | > > my $rcount = $dbh->DBCOUNT(); > | | > > if ($rcount != -1) { > | | > > $totalupdatecount += $rcount; > | | > > } > | | > > > | | > > my @res; > | | > > while (@res = $dbh->dbnextrow()) { > | | > > my @copyres = @res; # make a copy of > the > | | > array > | | > > push @resultlist, \@copyres; > | | > > } > | | > > } > | | > > > | | > > push @resultlist, $totalupdatecount; > | | > > } > | | > > } > | | > > > | | > > if ($saveresults) { > | | > > return \@resultlist; > | | > > } else { > | | > > return; > | | > > } > | | > > } > | | > > > | | > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > | | > > > | | > > > Hi, > | | > > > > | | > > > I know that you are able to issue a SQL statement within > | | > > > PERL DBI, > | | > | | > > > but is there anyway that I can issue an external SQL program? > > | | > > > For > | | > | | > > > example, I have a SQL program called ppv_insert.sql that I > | | > > > would > | > | | > > > like to execute within PERL DBI. > | | > > > > | | > > > Thanks in advance. > | | > > > > | | > > > Peter Loo > | | > > > > | | > > > > | | > > > > | | > > > > | | > > > This E-mail message is for the sole use of the intended > | | > > > recipient > | | > > > (s) and may contain confidential and privileged information. > | | > > > Any unauthorized review, use, disclosure or distribution is > | | prohibited. > | | > > | | > > > If you are not the intended recipient, please contact the > | | > > > sender > | > | | > > > by reply E-mail, and destroy all copies of the original > message. > | | > > > _______________________________________________ > | | > > > Phoenix-pm mailing list > | | > > > Phoenix-pm at pm.org > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > | | > > > | | > > > | | ------------------------------------------------------------------ > | | -- > | | - > | | > > Michael Friedman HighWire Press > | | > > Phone: 650-725-1974 Stanford University > | | > > FAX: 270-721-8034 > | | > | | > > -------------------------------------------------------------- > | | > > -- > | | > > -- > | | > > -- > | | > > - > | | > > > | | > > > | | > > _______________________________________________ > | | > > Phoenix-pm mailing list > | | > > Phoenix-pm at pm.org > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > | | > > | | > > | | > This E-mail message is for the sole use of the intended > | | > recipient(s) > | > | | > and may contain confidential and privileged information. Any > | | > unauthorized review, use, disclosure or distribution is > prohibited. > | | > If you are not the intended recipient, please contact the sender > | | > by reply E-mail, and destroy all copies of the original message. > | | > > | | > > | | > This E-mail message is for the sole use of the intended > | | > recipient(s) > | | and may contain confidential and privileged information. Any > | | unauthorized review, use, disclosure or distribution is prohibited. > | | If you are not the intended recipient, please contact the sender > | | by reply E-mail, and destroy all copies of the original message. > | | > | | > | | This E-mail message is for the sole use of the intended > | | recipient(s) > > | | and may contain confidential and privileged information. Any > | | unauthorized review, use, disclosure or distribution is prohibited. > | | If you are not the intended recipient, please contact the sender > | | by reply E-mail, and destroy all copies of the original message. > | | > | | > | | This E-mail message is for the sole use of the intended > | | recipient(s) > | and may contain confidential and privileged information. Any > | unauthorized review, use, disclosure or distribution is prohibited. > | If you are not the intended recipient, please contact the sender by > | reply E-mail, and destroy all copies of the original message. > | | _______________________________________________ > | | Phoenix-pm mailing list > | | Phoenix-pm at pm.org > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > | > | > | This E-mail message is for the sole use of the intended recipient(s) > | and may contain confidential and privileged information. Any > | unauthorized review, use, disclosure or distribution is prohibited. > | If you are not the intended recipient, please contact the sender by > | reply E-mail, and destroy all copies of the original message. > | > | > | This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. > If you are not the intended recipient, please contact the sender by > reply E-mail, and destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Wed Mar 15 16:23:37 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 00:23:37 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> Message-ID: <20060316002337.GV22790@illogics.org> Hi Peter, If it's your last reply, I'm truly sorry. Brock will yell at me if I scare people away. But on the flip side of the token, when you ask get free advice, well, it's not really, it's at the expense of the askee rather than the asker. So, it's important to never treat free advice as if its free. Here's one we hear all the time in EFNET #perlhelp... "If you're going to be like that, I'm going to use PHP!" (except without the punctuation). Yeah, that'll teach me a lesson. Maybe I offended you, but you're going to offend people here if you take advice, throw it away, ask for more, throw that away, repeat. Don't beleive what you're being told? That's fine, but why would you come back and ask for more? Also, as I said, I yelled a little bit to get your attention; I didn't yell at you to hurt your feelings. I'm sorry if I miss-pegged your experience. It happens. It's a theme I hear from novices a lot; it doesn't mean you're a novice. You also may not even have the attitude towards "one size fits all routines" that I thought you had -- maybe I mispegged you twice. You're a grown man. You can handle it. But I don't think this was about my using stronger language; I think it was about me (possibily) mis-pegging you as a novice. By the way, it's Perl, not PERL. People only think it's "PERL" because book titles tend to be printed in all-caps. If you actually open the book, though, and it isn't one of the several that completely sucks, you'll see that title case isn't used inside. > Don't think for a second that you have done it all and have experienced > it all. Heh, now you're accusing me of attitudes, but in a far less hypothetical phrasing. But that's okay. > I am quite disappointed by your attitude and certainly won't have too > much good to spread about you. If you were frustrated by the emails, Wait, you've "been in the business for 15 years and make a six figure salary" and you're going to tell people bad things about me? Isn't that a more gradeschool level tactic? Everyone who knows me knows I'm a dict. The channel bots have a long list of nasty things novices have said about me. I like to show it to novices before they get a screenfull from me, but they're always still shocked to discover I'm a dict, and act like they just discovered something. Seriously. I'm a bad person. I don't have any natural coding abilities. I just made a bargain with Satan. Let me know when you've come to terms with this. > why didn't you just delete them and let whoever else to respond. You > are rude, conceded and arrogant. I could have told you that ;) If you're going to hell, you might as well do it right. And don't take me to represent the group. There are a lot of very nice people there. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Scott, > > This will be my last reply. > > I may be novice in PERL, but I make six figures yearly so I can't be too > dumb to be talked down to like you have. You are absolutely correct. > It is NOT acceptable. I have been in IT for 15+ years and have only met > a handful of people like yourself who think the only right way is your > way and that you are dreaming of being GOD. I am afraid you have missed > the boat there. GOD DON'T WRITE PERL OR ANY OTHER LANGUAGE. Nothing in > life is static. Everything is different. My way or no way is certainly > not the right approach. Seen it all and know it all is even worse. > Don't think for a second that you have done it all and have experienced > it all. > > I am quite disappointed by your attitude and certainly won't have too > much good to spread about you. If you were frustrated by the emails, > why didn't you just delete them and let whoever else to respond. You > are rude, conceded and arrogant. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Wednesday, March 15, 2006 4:49 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Hi Peter, > > I thought I was "the greatest". You changed your mind in a hurry. > > Sorry if I hurt your feelings. From my point of view, several people > were telling you things that were absolutely correct and helpful, but > they weren't "hitting their mark", so to speak. But it's often this > way. Novices ask for advice; they reject what they're given, > mistakeningly thinking their case is special and they're just > misunderstood and then repeat their question, over and over; people > repeat their answers, correct in their assessment of the situation; > everyone gets frustrated. I just hurry the process along when I > see things starting to go in circles. Sometimes the novice in question > suddenly "gets it"; sometimes pride gets in the way, and being yelled at > is considered unacceptable, even if it really is necessary. Some will > disagree as to its necessity; for them, I offer a few years on a Perl > help channel on IRC >=) We'll talk about it after then. > > Anyway, I'm not aggitated at you. I just thought the answers needed > some more emphasis. I'm sorry that you read it that way. It is true > that I'm a grumpy old cuss. If you don't post on any list with a grupy > old cuss, well, you're limiting yourself to not many lists ;) > > -scott > > > On 0, "Loo, Peter # PHX" wrote: > > > > Hi Scott, > > > > Sorry that is I have gotten into your skin, however, I thought this > > was a group to discuss and share ideas without any limitation of the > > level of PERL knowledge. It appear to me that you just don't like > > anything that is not your way. Don't worry about blocking me out of > > this group as I don't plan to return. Brock, Mike and the rest, I am > > sincerely grateful for your inputs. > > > > Best regards. > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Wednesday, March 15, 2006 3:58 PM > > To: Loo, Peter # PHX > > Cc: Brock; phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > Hi Peter, > > > > > My goal is to have a generic PERL sub_routine that I can pass one to > > > > many arguments (including the sql file name) and have it perform the > > > > SQL statement that is within the sql file dynamically so that we can > > > > reuse the same sub_routine for all select statement sql files. Does > > > > that make sense? I am all for reusing of code and not reinvent the > > > wheel each > > > > Stop repeating it this. If it doesn't make sense, then do something > > else. > > If it does, then we get it already. > > > > You have a choice: you can re-implement parts of the SQL shell in > > Perl, or you can call the SQL shell. > > > > You keep talking about reading the file and passing the SQL... you > > know how to do both of these things already. > > > > Funneling all SQL SELECT statements through one routine is silly. > > Refactor your code however you like, but this artifical goal will only > > > hurt you. If you find you're writing a lot of similar routines, then > > learn how to write closures. No, I don't want to talk about this or > > debate it; no slight intended, but this kind of simplistic view of > > "one routine to do all" is characteristic of novices. People who have > > > been doing this for a while know there's give and take. Routines get > > split out, then made into object methods, and those objects get > > configured with more objects they delegate to, then common logic is > > re-grouped into a routine somewhere, and so on. Trying to do all of > > any sort of thing in one places ignores the inherent complexity in a > > program. That's like saying "I want to paint the world PINK!". > > But, I said this isn't up for discussion. If I see it again, I'm > > telling mutt to ignore this thread. > > > > > time I have an need to run a select statement. I suppose I can read > > > > in the sql file like Michael Friedman had suggested earlier in the > > > chain of emails. I was just hoping not to have to read in the sql > > > statement contained in the sql file and assigning it to a variable > > > before doing the $dbh->prepare. > > > > You want to execute it in Perl and not read it in? I think you need > > to focus on what you're really trying to accomplish and concentrate > > less on how you do it, because you're starting to get silly. I can't > > imagine the real problem actually suggests a reading/not-reading > paradox. > > > > > I need to do some brain storming. :) Not going to give up yet as I > > > > am dealing with one of my two favorite languages. :) > > > > If you want to run it through Perl, read it in and DBI. If you want > > to run it in the database shell, either do so directly or from a > > subprocess. > > Either way, quit trying to do both-and-neither-at-the-same-time. I > > assure you it will be entirely unproductive. > > > > By the way, there's often no clear solution (at least the programmer > > working on something), and in that case, don't fuss with it endlessly. > > Do it one way and be done with it, and if a clear solution occurs to > > you later, go back and change it. > > > > -scott > > > > > > > > Peter Loo > > > Wolters Kluwer Health > > > (602) 381-9553 > > > > > > -----Original Message----- > > > From: Brock [mailto:awwaiid at thelackthereof.org] > > > Sent: Wednesday, March 15, 2006 2:02 PM > > > To: Loo, Peter # PHX > > > Cc: Scott Walters; phoenix-pm at pm.org > > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > > Using DBI is completely different then piping things to sqlplus. I > > > was > > > > > just doing a direct translation of your example, not really trying > > > to imply that it was a good idea. > > > > > > Perl DBI does NOT use sqlplus as the driver. > > > > > > Unfortunately your question doesn't make much sense... if your .sql > > > has a select it could have many selects and it could have all sorts > > > of > > > > > things. The problem is that how can your program know what it > > contains? > > > It seems what you need to do is put the select like queries directly > > > > into perl, as we demonstrated to you in earlier emails. Doing the > > > whole fetchrow_arrayref thing that you were already doing. > > > > > > Make sense? > > > > > > --Brock > > > > > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > > > | > > > | Hi Brock, > > > | > > > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't > > > > | be efficient. Secondly, what if you have a "SELECT" statement in > > > | the .sql program and if you want to loop through each row? > > > | > > > | Peter Loo > > > | Wolters Kluwer Health > > > | (602) 381-9553 > > > | > > > | -----Original Message----- > > > | From: Brock [mailto:awwaiid at thelackthereof.org] > > > | Sent: Wednesday, March 15, 2006 1:52 PM > > > | To: Loo, Peter # PHX > > > | Cc: Scott Walters; phoenix-pm at pm.org > > > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > | > > > | Sure... what you are doing here is opening an external program and > > > > | piping it data on STDIN. There are several ways to do this in > > Perl... > > > | > > > | Here's one (a rough guess): > > > | > > > | # Initiate those vars (pull them from %ENV?) > > > | my $dbuser = ...; > > > | my $dbpass = ...; > > > | my $dbconn = ...; > > > | my $mailProgFile = ...; > > > | > > > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: > $!\n"; > > > | print $sqlplus <<" HERE"; > > > | connect ${dbuser}/${dbpass}@${dbconn} > > > | @${mailProgFile}.sql > > > | HERE > > > | > > > | More or less. Eh? > > > | > > > | --Brock > > > | > > > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > > > | | > > > | | Hi Scott, > > > | | > > > | | So will it be correct to assume that PERL DBI can not execute an > > > > | | SQL > > > > > > | | program? For example, I can do this with Korn shell: > > > | | > > > | | sqlplus -s /NOLOG << EOF > > > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > > > | | @${MailProgFile}.sql > > > | | > > > | | Is this not possible in PERL DBI? > > > | | > > > | | Peter Loo > > > | | Wolters Kluwer Health > > > | | (602) 381-9553 > > > | | > > > | | -----Original Message----- > > > | | From: Scott Walters [mailto:scott at illogics.org] > > > | | Sent: Tuesday, March 14, 2006 9:36 PM > > > | | To: Loo, Peter # PHX > > > | | Cc: Michael Friedman; phoenix-pm at pm.org > > > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > | | > > > | | Consider using GetOpt::Std, and most of the time you want this > > > | | form of > > > | | for: > > > | | > > > | | for my $thing (@things) { ... stuff with $thing ... } > > > | | > > > | | You can always try to read rows and trap errors. > > > | | > > > | | -scott > > > | | > > > | | On 0, "Loo, Peter # PHX" > > > wrote: > > > | | > > > > | | > Thanks Michael and Scott. What I am trying to do is creating > > > | | > a generic PERL program that will take in multiple arguments > > > | | > such > > as: > > > | | > > > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > > > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > > > | | > switch ($flag) { > > > | | > case "-dd" { $d_dbName = lc($value) } > > > | | > case "-dt" { $d_tblName = lc($value) } > > > | | > case "-ds" { $d_SQL = $value } > > > | | > case "-sd" { $s_dbName = lc($value) } > > > | | > case "-st" { $s_tblName = lc($value) } > > > | | > case "-ss" { $s_SQL = $value } > > > | | > case "-cp" { $commitPoint = lc($value) } > > > | | > case "-sf" { $s_funcToPerf = lc($value) } > > > | | > case "-df" { $d_funcToPerf = lc($value) } > > > | | > case "-d1" { $s_dbDriver = lc($value) } > > > | | > case "-d2" { $d_dbDriver = lc($value) } > > > | | > else { print "Unknown flag: $flag\n" } > > > | | > } > > > | | > } > > > | | > > > > | | > Then execute accordingly, however, I would like to execute an > > > | | > external > > > | | > > > | | > SQL program that is passed to this generic program. In the > > > | | > case > > > > > | | > of an > > > | | > > > | | > external SQL program that does SELECT instead of UPDATE or > > > | | > INSERT, > > > > > > | | > I > > > | > > > | | > want to loop through the returned rows. Here is my first > > > | | > block of > > > | | "if" > > > | | > statement. > > > | | > > > > | | > if ($s_funcToPerf eq "update") { > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$commitPoint > > > | | > || > > > | | > !$s_SQL) { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example: \n"; > > > | | > print " Database Name: -sd=dv26\n"; > > > | | > print " Database Driver: -d1=Oracle\n"; > > > | | > print " Table Name: -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | -ss=/usr/local/sql/ppv_update.sql\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_update()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($s_funcToPerf eq "insert") { > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$commitPoint > > > | | > || > > > | | > !$s_SQL) { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example: \n"; > > > | | > print " Database Name: -sd=dv26\n"; > > > | | > print " Database Driver: -d1=Oracle\n"; > > > | | > print " Table Name: -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | -ss=/usr/local/sql/ppv_insert.sql\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_insert()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($d_funcToPerf eq "update") { > > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) > { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example:\n"; > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > | | > print " Destination Table Name: > > > | | -dt=p_falcon_projections\n"; > > > | | > print " Source Database Name: -sd=dv26\n"; > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > | | > print " Source Table Name: > > > | | -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > | | > print " SQL Statement: > > > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > > > | | > print " Source Function to Perform: -sf=select\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_select()\n"; > > > | | > print "Calling sub_update()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($d_funcToPerf eq "insert") { > > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) > { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example:\n"; > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > | | > print " Destination Table Name: > > > | | -dt=p_falcon_projections\n"; > > > | | > print " Source Database Name: -sd=dv26\n"; > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > | | > print " Source Table Name: > > > | | -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > | | > print " SQL Statement: > > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > > | | > print " Source Function to Perform: -sf=select\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_select()\n"; > > > | | > print "Calling sub_insert()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($s_funcToPerf eq "select") { > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$commitPoint > > > | || > > > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > > > | | > !$d_funcToPerf > > > | || > > > | | > !$s_SQL || !$d_SQL) { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example:\n"; > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > | | > print " Destination Table Name: > > > | | -dt=p_falcon_projections\n"; > > > | | > print " Source Database Name: -sd=dv26\n"; > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > | | > print " Source Table Name: > > > | | -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > | | > print " SQL Statement: > > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > > | | > print " Source Function to Perform: -sf=select\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_select()\n"; > > > | | > print "Calling sub_insert()\n"; > > > | | > } > > > | | > } > > > | | > else { > > > | | > print "ERROR: Unknown value for database action to > > > perform.\n"; > > > | | > exit(666); > > > | | > } > > > | | > > > > | | > > > > | | > Peter Loo > > > | | > Wolters Kluwer Health > > > | | > (602) 381-9553 > > > | | > > > > | | > -----Original Message----- > > > | | > From: Scott Walters [mailto:scott at illogics.org] > > > | | > Sent: Tuesday, March 14, 2006 3:28 PM > > > | | > To: Michael Friedman > > > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL > > > | | > DBI > > > | | > > > > | | > Hi Peter, > > > | | > > > > | | > Surely you're trying to accomplish more than just running the > > > | | > SQL or > > > | > > > | | > you would just read it in an feed it to DBI. There's no > > > | | > reason you couldn't call to the database command shell: > > > | | > > > > | | > if(my $pid = fork) { > > > | | > waitpid $pid; > > > | | > } else { > > > | | > close STDIN; > > > | | > open STDIN, '<', 'foo.sql' or die $!; > > > | | > exec 'mysql', $dbname or die $!; > > > | | > } > > > | | > > > > | | > ... or something like that. If you want to use special > > > | | > features > > > > > | | > of the database command shell or just cash in on its speed, > > > | | > this > > > > > | | > might be > > > | | > > > | | > handy. Of course, you don't want to try to read values from > > > | | > the > > > > > | | > database back into Perl over a pipe between two processes... > > > | | > that's just nasty. > > > | | > > > > | | > -scott > > > | | > > > > | | > > > > | | > On 0, Michael Friedman > wrote: > > > | | > > Peter, > > > | | > > > > > | | > > What I did in that situation was write a couple of methods > > > | | > > to read > > > | > > > | | > > in the file, put it into an array, and then loop through the > > > > | | > > array > > > | > > > | | > > and make each call. The good news is you only have to write > > > | | > > that > > > > > > | | > > once and then you can reuse it... > > > | | > > > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but > > > > | | > > the > > > > > > | | > > logic > > > | | > > > > | | > > would be the same. Sybase uses 'go' on a line by itself to > > > | | > > end > > > > > | | > > a > > > > > > | | > > SQL > > > | | > > > | | > > command, so we just use that to split up the lines in the > > > | | > > file > > > > > | | > > into commands into the array. > > > | | > > > > > | | > > You could make this a lot fancier, if you had the need, but > > > | | > > it > > > > > | | > > works > > > | | > > > | | > > for me. > > > | | > > > > > | | > > Good luck, > > > | | > > -- Mike > > > | | > > > > > | | > > (sub db_run_script and sub db_run_command_list, below) > > > | | > > > > > | | > > sub db_run_script #($$) > > > | | > > { > > > | | > > my $dbh = shift; > > > | | > > my $script = shift; > > > | | > > my $saveresults = shift; > > > | | > > > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > > > file > > > | | > $script: > > > | | > > $!"; > > > | | > > > > > | | > > my @commands = (); > > > | | > > my $j = 0; > > > | | > > my ($line); > > > | | > > > > > | | > > # read script file into a variable (array of commands) > > > | | > > while ($line = ) > > > | | > > { > > > | | > > if ($line =~ /^go/) > > > | | > > { > > > | | > > # make new command > > > | | > > $j++; > > > | | > > } > > > | | > > elsif ($line =~ /^\s*$/) > > > | | > > { > > > | | > > # ignore blank lines > > > | | > > } > > > | | > > else > > > | | > > { > > > | | > > $commands[$j] .= $line; > > > | | > > } > > > | | > > } > > > | | > > close SQL_SCRIPT; > > > | | > > > > > | | > > return db_run_command_list($dbh, \@commands, > > > $saveresults); } > > > | | > > > > > | | > > sub db_run_command_list > > > | | > > { > > > | | > > my $dbh = shift; > > > | | > > my $cmdlist = shift; > > > | | > > my $saveresults = shift; > > > | | > > > > > | | > > my @resultlist; > > > | | > > > > > | | > > # run commands from array > > > | | > > for $j(0..$#$cmdlist) > > > | | > > { > > > | | > > $dbh->dbcmd($cmdlist->[$j]); > > > | | > > my $status; > > > | | > > eval { > > > | | > > $status = $dbh->dbsqlexec(); > > > | | > > }; > > > | | > > > > > | | > > if ($@ || $status != SUCCEED) > > > | | > > { > > > | | > > # don't always die, because drop will > > > fail > > > | | > sometimes > > > | | > > if ($cmdlist->[$j] =~ /drop/i) > > > | | > > { > > > | | > > warn "$cmdlist->[$j] failed.\n > > > This is > > > | | > OK - item probably didn't > > > | | > > exist before installation.\n"; > > > | | > > > > > | | > > $dbh->dbcancel(); # so that we > > > can move > > > | | > on to the next command? > > > | | > > } > > > | | > > else > > > | | > > { > > > | | > > die "+++ Could not run command > > > | | > $cmdlist->[$j]\nbecause of this > > > | | > > problem:\n$@"; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > if (!$saveresults) { > > > | | > > db_ignore_results($dbh); > > > | | > > } else { > > > | | > > # Count the total number of rows that were > > > updated, > > > | | > > # and capture the output of any SELECT > > > statements > > > | | > > # > > > | | > > # Each update/insert statement will have its > > > own > > > | | > update > > > | | > > # count (a separate call to DBCOUNT()) but we > > > will > > > | | > > # just add them all together > > > | | > > my $totalupdatecount = 0; > > > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > > | | > > my $rcount = $dbh->DBCOUNT(); > > > | | > > if ($rcount != -1) { > > > | | > > $totalupdatecount += $rcount; > > > | | > > } > > > | | > > > > > | | > > my @res; > > > | | > > while (@res = $dbh->dbnextrow()) { > > > | | > > my @copyres = @res; # make a copy of > > > the > > > | | > array > > > | | > > push @resultlist, \@copyres; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > push @resultlist, $totalupdatecount; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > if ($saveresults) { > > > | | > > return \@resultlist; > > > | | > > } else { > > > | | > > return; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > | | > > > > > | | > > > Hi, > > > | | > > > > > > | | > > > I know that you are able to issue a SQL statement within > > > | | > > > PERL DBI, > > > | | > > > | | > > > but is there anyway that I can issue an external SQL > > program? > > > > > > | | > > > For > > > | | > > > | | > > > example, I have a SQL program called ppv_insert.sql that I > > > > | | > > > would > > > | > > > | | > > > like to execute within PERL DBI. > > > | | > > > > > > | | > > > Thanks in advance. > > > | | > > > > > > | | > > > Peter Loo > > > | | > > > > > > | | > > > > > > | | > > > > > > | | > > > > > > | | > > > This E-mail message is for the sole use of the intended > > > | | > > > recipient > > > | | > > > (s) and may contain confidential and privileged > information. > > > > > | | > > > Any unauthorized review, use, disclosure or distribution > > > | | > > > is > > > | | prohibited. > > > | | > > > > | | > > > If you are not the intended recipient, please contact the > > > | | > > > sender > > > | > > > | | > > > by reply E-mail, and destroy all copies of the original > > > message. > > > | | > > > _______________________________________________ > > > | | > > > Phoenix-pm mailing list > > > | | > > > Phoenix-pm at pm.org > > > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > | | > > > > > | | > > > > > | | ---------------------------------------------------------------- > > > | | -- > > > | | -- > > > | | - > > > | | > > Michael Friedman HighWire Press > > > | | > > Phone: 650-725-1974 Stanford University > > > | | > > FAX: 270-721-8034 > > > | | > > > | | > > ------------------------------------------------------------ > > > | | > > -- > > > | | > > -- > > > | | > > -- > > > | | > > -- > > > | | > > - > > > | | > > > > > | | > > > > > | | > > _______________________________________________ > > > | | > > Phoenix-pm mailing list > > > | | > > Phoenix-pm at pm.org > > > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > | | > > > > | | > > > > | | > This E-mail message is for the sole use of the intended > > > | | > recipient(s) > > > | > > > | | > and may contain confidential and privileged information. Any > > > | | > unauthorized review, use, disclosure or distribution is > > > prohibited. > > > | | > If you are not the intended recipient, please contact the > > > | | > sender > > > > > | | > by reply E-mail, and destroy all copies of the original > message. > > > | | > > > > | | > > > > | | > This E-mail message is for the sole use of the intended > > > | | > recipient(s) > > > | | and may contain confidential and privileged information. Any > > > | | unauthorized review, use, disclosure or distribution is > > prohibited. > > > | | If you are not the intended recipient, please contact the sender > > > > | | by reply E-mail, and destroy all copies of the original message. > > > | | > > > | | > > > | | This E-mail message is for the sole use of the intended > > > | | recipient(s) > > > > > > | | and may contain confidential and privileged information. Any > > > | | unauthorized review, use, disclosure or distribution is > > prohibited. > > > | | If you are not the intended recipient, please contact the sender > > > > | | by reply E-mail, and destroy all copies of the original message. > > > | | > > > | | > > > | | This E-mail message is for the sole use of the intended > > > | | recipient(s) > > > | and may contain confidential and privileged information. Any > > > | unauthorized review, use, disclosure or distribution is > prohibited. > > > | If you are not the intended recipient, please contact the sender > > > | by reply E-mail, and destroy all copies of the original message. > > > | | _______________________________________________ > > > | | Phoenix-pm mailing list > > > | | Phoenix-pm at pm.org > > > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > > > | > > > | > > > | This E-mail message is for the sole use of the intended > > > | recipient(s) > > > > > | and may contain confidential and privileged information. Any > > > | unauthorized review, use, disclosure or distribution is > prohibited. > > > | If you are not the intended recipient, please contact the sender > > > | by reply E-mail, and destroy all copies of the original message. > > > | > > > | > > > | This E-mail message is for the sole use of the intended > > > | recipient(s) > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From bwmetz at att.com Wed Mar 15 16:41:19 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Wed, 15 Mar 2006 18:41:19 -0600 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <20060316002337.GV22790@illogics.org> Message-ID: <01D5341D04A2E64AB9B345769047336701753A1E@OCCLUST01EVS1.ugd.att.com> Scott, If you keep talking about dict's we'll have to rebadge the group as Phoenix Python Mongers. ;-) Silly joke aside, anyone know if such an entity exists in town? I have to wear both Perl and Python hats at work and Python is definitely my weaker language. Bobby -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Scott Walters Sent: Wednesday, March 15, 2006 5:24 PM To: Loo, Peter # PHX Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Peter, If it's your last reply, I'm truly sorry. Brock will yell at me if I scare people away. But on the flip side of the token, when you ask get free advice, well, it's not really, it's at the expense of the askee rather than the asker. So, it's important to never treat free advice as if its free. Here's one we hear all the time in EFNET #perlhelp... "If you're going to be like that, I'm going to use PHP!" (except without the punctuation). Yeah, that'll teach me a lesson. Maybe I offended you, but you're going to offend people here if you take advice, throw it away, ask for more, throw that away, repeat. Don't beleive what you're being told? That's fine, but why would you come back and ask for more? Also, as I said, I yelled a little bit to get your attention; I didn't yell at you to hurt your feelings. I'm sorry if I miss-pegged your experience. It happens. It's a theme I hear from novices a lot; it doesn't mean you're a novice. You also may not even have the attitude towards "one size fits all routines" that I thought you had -- maybe I mispegged you twice. You're a grown man. You can handle it. But I don't think this was about my using stronger language; I think it was about me (possibily) mis-pegging you as a novice. By the way, it's Perl, not PERL. People only think it's "PERL" because book titles tend to be printed in all-caps. If you actually open the book, though, and it isn't one of the several that completely sucks, you'll see that title case isn't used inside. > Don't think for a second that you have done it all and have experienced > it all. Heh, now you're accusing me of attitudes, but in a far less hypothetical phrasing. But that's okay. > I am quite disappointed by your attitude and certainly won't have too > much good to spread about you. If you were frustrated by the emails, Wait, you've "been in the business for 15 years and make a six figure salary" and you're going to tell people bad things about me? Isn't that a more gradeschool level tactic? Everyone who knows me knows I'm a dict. The channel bots have a long list of nasty things novices have said about me. I like to show it to novices before they get a screenfull from me, but they're always still shocked to discover I'm a dict, and act like they just discovered something. Seriously. I'm a bad person. I don't have any natural coding abilities. I just made a bargain with Satan. Let me know when you've come to terms with this. > why didn't you just delete them and let whoever else to respond. You > are rude, conceded and arrogant. I could have told you that ;) If you're going to hell, you might as well do it right. And don't take me to represent the group. There are a lot of very nice people there. Cheers, -scott On 0, "Loo, Peter # PHX" wrote: > > Scott, > > This will be my last reply. > > I may be novice in PERL, but I make six figures yearly so I can't be too > dumb to be talked down to like you have. You are absolutely correct. > It is NOT acceptable. I have been in IT for 15+ years and have only met > a handful of people like yourself who think the only right way is your > way and that you are dreaming of being GOD. I am afraid you have missed > the boat there. GOD DON'T WRITE PERL OR ANY OTHER LANGUAGE. Nothing in > life is static. Everything is different. My way or no way is certainly > not the right approach. Seen it all and know it all is even worse. > Don't think for a second that you have done it all and have experienced > it all. > > I am quite disappointed by your attitude and certainly won't have too > much good to spread about you. If you were frustrated by the emails, > why didn't you just delete them and let whoever else to respond. You > are rude, conceded and arrogant. > > Peter Loo > Wolters Kluwer Health > (602) 381-9553 > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Wednesday, March 15, 2006 4:49 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > Hi Peter, > > I thought I was "the greatest". You changed your mind in a hurry. > > Sorry if I hurt your feelings. From my point of view, several people > were telling you things that were absolutely correct and helpful, but > they weren't "hitting their mark", so to speak. But it's often this > way. Novices ask for advice; they reject what they're given, > mistakeningly thinking their case is special and they're just > misunderstood and then repeat their question, over and over; people > repeat their answers, correct in their assessment of the situation; > everyone gets frustrated. I just hurry the process along when I > see things starting to go in circles. Sometimes the novice in question > suddenly "gets it"; sometimes pride gets in the way, and being yelled at > is considered unacceptable, even if it really is necessary. Some will > disagree as to its necessity; for them, I offer a few years on a Perl > help channel on IRC >=) We'll talk about it after then. > > Anyway, I'm not aggitated at you. I just thought the answers needed > some more emphasis. I'm sorry that you read it that way. It is true > that I'm a grumpy old cuss. If you don't post on any list with a grupy > old cuss, well, you're limiting yourself to not many lists ;) > > -scott > > > On 0, "Loo, Peter # PHX" wrote: > > > > Hi Scott, > > > > Sorry that is I have gotten into your skin, however, I thought this > > was a group to discuss and share ideas without any limitation of the > > level of PERL knowledge. It appear to me that you just don't like > > anything that is not your way. Don't worry about blocking me out of > > this group as I don't plan to return. Brock, Mike and the rest, I am > > sincerely grateful for your inputs. > > > > Best regards. > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Wednesday, March 15, 2006 3:58 PM > > To: Loo, Peter # PHX > > Cc: Brock; phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > Hi Peter, > > > > > My goal is to have a generic PERL sub_routine that I can pass one to > > > > many arguments (including the sql file name) and have it perform the > > > > SQL statement that is within the sql file dynamically so that we can > > > > reuse the same sub_routine for all select statement sql files. Does > > > > that make sense? I am all for reusing of code and not reinvent the > > > wheel each > > > > Stop repeating it this. If it doesn't make sense, then do something > > else. > > If it does, then we get it already. > > > > You have a choice: you can re-implement parts of the SQL shell in > > Perl, or you can call the SQL shell. > > > > You keep talking about reading the file and passing the SQL... you > > know how to do both of these things already. > > > > Funneling all SQL SELECT statements through one routine is silly. > > Refactor your code however you like, but this artifical goal will only > > > hurt you. If you find you're writing a lot of similar routines, then > > learn how to write closures. No, I don't want to talk about this or > > debate it; no slight intended, but this kind of simplistic view of > > "one routine to do all" is characteristic of novices. People who have > > > been doing this for a while know there's give and take. Routines get > > split out, then made into object methods, and those objects get > > configured with more objects they delegate to, then common logic is > > re-grouped into a routine somewhere, and so on. Trying to do all of > > any sort of thing in one places ignores the inherent complexity in a > > program. That's like saying "I want to paint the world PINK!". > > But, I said this isn't up for discussion. If I see it again, I'm > > telling mutt to ignore this thread. > > > > > time I have an need to run a select statement. I suppose I can read > > > > in the sql file like Michael Friedman had suggested earlier in the > > > chain of emails. I was just hoping not to have to read in the sql > > > statement contained in the sql file and assigning it to a variable > > > before doing the $dbh->prepare. > > > > You want to execute it in Perl and not read it in? I think you need > > to focus on what you're really trying to accomplish and concentrate > > less on how you do it, because you're starting to get silly. I can't > > imagine the real problem actually suggests a reading/not-reading > paradox. > > > > > I need to do some brain storming. :) Not going to give up yet as I > > > > am dealing with one of my two favorite languages. :) > > > > If you want to run it through Perl, read it in and DBI. If you want > > to run it in the database shell, either do so directly or from a > > subprocess. > > Either way, quit trying to do both-and-neither-at-the-same-time. I > > assure you it will be entirely unproductive. > > > > By the way, there's often no clear solution (at least the programmer > > working on something), and in that case, don't fuss with it endlessly. > > Do it one way and be done with it, and if a clear solution occurs to > > you later, go back and change it. > > > > -scott > > > > > > > > Peter Loo > > > Wolters Kluwer Health > > > (602) 381-9553 > > > > > > -----Original Message----- > > > From: Brock [mailto:awwaiid at thelackthereof.org] > > > Sent: Wednesday, March 15, 2006 2:02 PM > > > To: Loo, Peter # PHX > > > Cc: Scott Walters; phoenix-pm at pm.org > > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > > Using DBI is completely different then piping things to sqlplus. I > > > was > > > > > just doing a direct translation of your example, not really trying > > > to imply that it was a good idea. > > > > > > Perl DBI does NOT use sqlplus as the driver. > > > > > > Unfortunately your question doesn't make much sense... if your .sql > > > has a select it could have many selects and it could have all sorts > > > of > > > > > things. The problem is that how can your program know what it > > contains? > > > It seems what you need to do is put the select like queries directly > > > > into perl, as we demonstrated to you in earlier emails. Doing the > > > whole fetchrow_arrayref thing that you were already doing. > > > > > > Make sense? > > > > > > --Brock > > > > > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > > > | > > > | Hi Brock, > > > | > > > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it can't > > > > | be efficient. Secondly, what if you have a "SELECT" statement in > > > | the .sql program and if you want to loop through each row? > > > | > > > | Peter Loo > > > | Wolters Kluwer Health > > > | (602) 381-9553 > > > | > > > | -----Original Message----- > > > | From: Brock [mailto:awwaiid at thelackthereof.org] > > > | Sent: Wednesday, March 15, 2006 1:52 PM > > > | To: Loo, Peter # PHX > > > | Cc: Scott Walters; phoenix-pm at pm.org > > > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > | > > > | Sure... what you are doing here is opening an external program and > > > > | piping it data on STDIN. There are several ways to do this in > > Perl... > > > | > > > | Here's one (a rough guess): > > > | > > > | # Initiate those vars (pull them from %ENV?) > > > | my $dbuser = ...; > > > | my $dbpass = ...; > > > | my $dbconn = ...; > > > | my $mailProgFile = ...; > > > | > > > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: > $!\n"; > > > | print $sqlplus <<" HERE"; > > > | connect ${dbuser}/${dbpass}@${dbconn} > > > | @${mailProgFile}.sql > > > | HERE > > > | > > > | More or less. Eh? > > > | > > > | --Brock > > > | > > > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > > > | | > > > | | Hi Scott, > > > | | > > > | | So will it be correct to assume that PERL DBI can not execute an > > > > | | SQL > > > > > > | | program? For example, I can do this with Korn shell: > > > | | > > > | | sqlplus -s /NOLOG << EOF > > > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > > > | | @${MailProgFile}.sql > > > | | > > > | | Is this not possible in PERL DBI? > > > | | > > > | | Peter Loo > > > | | Wolters Kluwer Health > > > | | (602) 381-9553 > > > | | > > > | | -----Original Message----- > > > | | From: Scott Walters [mailto:scott at illogics.org] > > > | | Sent: Tuesday, March 14, 2006 9:36 PM > > > | | To: Loo, Peter # PHX > > > | | Cc: Michael Friedman; phoenix-pm at pm.org > > > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > | | > > > | | Consider using GetOpt::Std, and most of the time you want this > > > | | form of > > > | | for: > > > | | > > > | | for my $thing (@things) { ... stuff with $thing ... } > > > | | > > > | | You can always try to read rows and trap errors. > > > | | > > > | | -scott > > > | | > > > | | On 0, "Loo, Peter # PHX" > > > wrote: > > > | | > > > > | | > Thanks Michael and Scott. What I am trying to do is creating > > > | | > a generic PERL program that will take in multiple arguments > > > | | > such > > as: > > > | | > > > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > > > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > > > | | > switch ($flag) { > > > | | > case "-dd" { $d_dbName = lc($value) } > > > | | > case "-dt" { $d_tblName = lc($value) } > > > | | > case "-ds" { $d_SQL = $value } > > > | | > case "-sd" { $s_dbName = lc($value) } > > > | | > case "-st" { $s_tblName = lc($value) } > > > | | > case "-ss" { $s_SQL = $value } > > > | | > case "-cp" { $commitPoint = lc($value) } > > > | | > case "-sf" { $s_funcToPerf = lc($value) } > > > | | > case "-df" { $d_funcToPerf = lc($value) } > > > | | > case "-d1" { $s_dbDriver = lc($value) } > > > | | > case "-d2" { $d_dbDriver = lc($value) } > > > | | > else { print "Unknown flag: $flag\n" } > > > | | > } > > > | | > } > > > | | > > > > | | > Then execute accordingly, however, I would like to execute an > > > | | > external > > > | | > > > | | > SQL program that is passed to this generic program. In the > > > | | > case > > > > > | | > of an > > > | | > > > | | > external SQL program that does SELECT instead of UPDATE or > > > | | > INSERT, > > > > > > | | > I > > > | > > > | | > want to loop through the returned rows. Here is my first > > > | | > block of > > > | | "if" > > > | | > statement. > > > | | > > > > | | > if ($s_funcToPerf eq "update") { > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$commitPoint > > > | | > || > > > | | > !$s_SQL) { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example: \n"; > > > | | > print " Database Name: -sd=dv26\n"; > > > | | > print " Database Driver: -d1=Oracle\n"; > > > | | > print " Table Name: -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | -ss=/usr/local/sql/ppv_update.sql\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_update()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($s_funcToPerf eq "insert") { > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$commitPoint > > > | | > || > > > | | > !$s_SQL) { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example: \n"; > > > | | > print " Database Name: -sd=dv26\n"; > > > | | > print " Database Driver: -d1=Oracle\n"; > > > | | > print " Table Name: -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | -ss=/usr/local/sql/ppv_insert.sql\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_insert()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($d_funcToPerf eq "update") { > > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) > { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example:\n"; > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > | | > print " Destination Table Name: > > > | | -dt=p_falcon_projections\n"; > > > | | > print " Source Database Name: -sd=dv26\n"; > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > | | > print " Source Table Name: > > > | | -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > | | > print " SQL Statement: > > > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > > > | | > print " Source Function to Perform: -sf=select\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_select()\n"; > > > | | > print "Calling sub_update()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($d_funcToPerf eq "insert") { > > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || !$d_SQL) > { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example:\n"; > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > | | > print " Destination Table Name: > > > | | -dt=p_falcon_projections\n"; > > > | | > print " Source Database Name: -sd=dv26\n"; > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > | | > print " Source Table Name: > > > | | -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > | | > print " SQL Statement: > > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > > | | > print " Source Function to Perform: -sf=select\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_select()\n"; > > > | | > print "Calling sub_insert()\n"; > > > | | > } > > > | | > } > > > | | > elsif ($s_funcToPerf eq "select") { > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > | | > !$commitPoint > > > | || > > > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > > > | | > !$d_funcToPerf > > > | || > > > | | > !$s_SQL || !$d_SQL) { > > > | | > print "ERROR: Not enough arguments. Require arguments > > > | are:\n"; > > > | | > print "Example:\n"; > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > | | > print " Destination Table Name: > > > | | -dt=p_falcon_projections\n"; > > > | | > print " Source Database Name: -sd=dv26\n"; > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > | | > print " Source Table Name: > > > | | -st=p_falcon_projections\n"; > > > | | > print " SQL Statement: > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > | | > print " SQL Statement: > > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > > | | > print " Source Function to Perform: -sf=select\n"; > > > | | > print " Commit Point: -cp=5000\n"; > > > | | > exit(666); > > > | | > } > > > | | > else { > > > | | > print "Calling sub_select()\n"; > > > | | > print "Calling sub_insert()\n"; > > > | | > } > > > | | > } > > > | | > else { > > > | | > print "ERROR: Unknown value for database action to > > > perform.\n"; > > > | | > exit(666); > > > | | > } > > > | | > > > > | | > > > > | | > Peter Loo > > > | | > Wolters Kluwer Health > > > | | > (602) 381-9553 > > > | | > > > > | | > -----Original Message----- > > > | | > From: Scott Walters [mailto:scott at illogics.org] > > > | | > Sent: Tuesday, March 14, 2006 3:28 PM > > > | | > To: Michael Friedman > > > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL > > > | | > DBI > > > | | > > > > | | > Hi Peter, > > > | | > > > > | | > Surely you're trying to accomplish more than just running the > > > | | > SQL or > > > | > > > | | > you would just read it in an feed it to DBI. There's no > > > | | > reason you couldn't call to the database command shell: > > > | | > > > > | | > if(my $pid = fork) { > > > | | > waitpid $pid; > > > | | > } else { > > > | | > close STDIN; > > > | | > open STDIN, '<', 'foo.sql' or die $!; > > > | | > exec 'mysql', $dbname or die $!; > > > | | > } > > > | | > > > > | | > ... or something like that. If you want to use special > > > | | > features > > > > > | | > of the database command shell or just cash in on its speed, > > > | | > this > > > > > | | > might be > > > | | > > > | | > handy. Of course, you don't want to try to read values from > > > | | > the > > > > > | | > database back into Perl over a pipe between two processes... > > > | | > that's just nasty. > > > | | > > > > | | > -scott > > > | | > > > > | | > > > > | | > On 0, Michael Friedman > wrote: > > > | | > > Peter, > > > | | > > > > > | | > > What I did in that situation was write a couple of methods > > > | | > > to read > > > | > > > | | > > in the file, put it into an array, and then loop through the > > > > | | > > array > > > | > > > | | > > and make each call. The good news is you only have to write > > > | | > > that > > > > > > | | > > once and then you can reuse it... > > > | | > > > > > | | > > My only example is using Sybase::DBlib, though, not DBI, but > > > > | | > > the > > > > > > | | > > logic > > > | | > > > > | | > > would be the same. Sybase uses 'go' on a line by itself to > > > | | > > end > > > > > | | > > a > > > > > > | | > > SQL > > > | | > > > | | > > command, so we just use that to split up the lines in the > > > | | > > file > > > > > | | > > into commands into the array. > > > | | > > > > > | | > > You could make this a lot fancier, if you had the need, but > > > | | > > it > > > > > | | > > works > > > | | > > > | | > > for me. > > > | | > > > > > | | > > Good luck, > > > | | > > -- Mike > > > | | > > > > > | | > > (sub db_run_script and sub db_run_command_list, below) > > > | | > > > > > | | > > sub db_run_script #($$) > > > | | > > { > > > | | > > my $dbh = shift; > > > | | > > my $script = shift; > > > | | > > my $saveresults = shift; > > > | | > > > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > > > file > > > | | > $script: > > > | | > > $!"; > > > | | > > > > > | | > > my @commands = (); > > > | | > > my $j = 0; > > > | | > > my ($line); > > > | | > > > > > | | > > # read script file into a variable (array of commands) > > > | | > > while ($line = ) > > > | | > > { > > > | | > > if ($line =~ /^go/) > > > | | > > { > > > | | > > # make new command > > > | | > > $j++; > > > | | > > } > > > | | > > elsif ($line =~ /^\s*$/) > > > | | > > { > > > | | > > # ignore blank lines > > > | | > > } > > > | | > > else > > > | | > > { > > > | | > > $commands[$j] .= $line; > > > | | > > } > > > | | > > } > > > | | > > close SQL_SCRIPT; > > > | | > > > > > | | > > return db_run_command_list($dbh, \@commands, > > > $saveresults); } > > > | | > > > > > | | > > sub db_run_command_list > > > | | > > { > > > | | > > my $dbh = shift; > > > | | > > my $cmdlist = shift; > > > | | > > my $saveresults = shift; > > > | | > > > > > | | > > my @resultlist; > > > | | > > > > > | | > > # run commands from array > > > | | > > for $j(0..$#$cmdlist) > > > | | > > { > > > | | > > $dbh->dbcmd($cmdlist->[$j]); > > > | | > > my $status; > > > | | > > eval { > > > | | > > $status = $dbh->dbsqlexec(); > > > | | > > }; > > > | | > > > > > | | > > if ($@ || $status != SUCCEED) > > > | | > > { > > > | | > > # don't always die, because drop will > > > fail > > > | | > sometimes > > > | | > > if ($cmdlist->[$j] =~ /drop/i) > > > | | > > { > > > | | > > warn "$cmdlist->[$j] failed.\n > > > This is > > > | | > OK - item probably didn't > > > | | > > exist before installation.\n"; > > > | | > > > > > | | > > $dbh->dbcancel(); # so that we > > > can move > > > | | > on to the next command? > > > | | > > } > > > | | > > else > > > | | > > { > > > | | > > die "+++ Could not run command > > > | | > $cmdlist->[$j]\nbecause of this > > > | | > > problem:\n$@"; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > if (!$saveresults) { > > > | | > > db_ignore_results($dbh); > > > | | > > } else { > > > | | > > # Count the total number of rows that were > > > updated, > > > | | > > # and capture the output of any SELECT > > > statements > > > | | > > # > > > | | > > # Each update/insert statement will have its > > > own > > > | | > update > > > | | > > # count (a separate call to DBCOUNT()) but we > > > will > > > | | > > # just add them all together > > > | | > > my $totalupdatecount = 0; > > > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > > | | > > my $rcount = $dbh->DBCOUNT(); > > > | | > > if ($rcount != -1) { > > > | | > > $totalupdatecount += $rcount; > > > | | > > } > > > | | > > > > > | | > > my @res; > > > | | > > while (@res = $dbh->dbnextrow()) { > > > | | > > my @copyres = @res; # make a copy of > > > the > > > | | > array > > > | | > > push @resultlist, \@copyres; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > push @resultlist, $totalupdatecount; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > if ($saveresults) { > > > | | > > return \@resultlist; > > > | | > > } else { > > > | | > > return; > > > | | > > } > > > | | > > } > > > | | > > > > > | | > > > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > | | > > > > > | | > > > Hi, > > > | | > > > > > > | | > > > I know that you are able to issue a SQL statement within > > > | | > > > PERL DBI, > > > | | > > > | | > > > but is there anyway that I can issue an external SQL > > program? > > > > > > | | > > > For > > > | | > > > | | > > > example, I have a SQL program called ppv_insert.sql that I > > > > | | > > > would > > > | > > > | | > > > like to execute within PERL DBI. > > > | | > > > > > > | | > > > Thanks in advance. > > > | | > > > > > > | | > > > Peter Loo > > > | | > > > > > > | | > > > > > > | | > > > > > > | | > > > > > > | | > > > This E-mail message is for the sole use of the intended > > > | | > > > recipient > > > | | > > > (s) and may contain confidential and privileged > information. > > > > > | | > > > Any unauthorized review, use, disclosure or distribution > > > | | > > > is > > > | | prohibited. > > > | | > > > > | | > > > If you are not the intended recipient, please contact the > > > | | > > > sender > > > | > > > | | > > > by reply E-mail, and destroy all copies of the original > > > message. > > > | | > > > _______________________________________________ > > > | | > > > Phoenix-pm mailing list > > > | | > > > Phoenix-pm at pm.org > > > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > | | > > > > > | | > > > > > | | ---------------------------------------------------------------- > > > | | -- > > > | | -- > > > | | - > > > | | > > Michael Friedman HighWire Press > > > | | > > Phone: 650-725-1974 Stanford University > > > | | > > FAX: 270-721-8034 > > > | | > > > | | > > ------------------------------------------------------------ > > > | | > > -- > > > | | > > -- > > > | | > > -- > > > | | > > -- > > > | | > > - > > > | | > > > > > | | > > > > > | | > > _______________________________________________ > > > | | > > Phoenix-pm mailing list > > > | | > > Phoenix-pm at pm.org > > > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > | | > > > > | | > > > > | | > This E-mail message is for the sole use of the intended > > > | | > recipient(s) > > > | > > > | | > and may contain confidential and privileged information. Any > > > | | > unauthorized review, use, disclosure or distribution is > > > prohibited. > > > | | > If you are not the intended recipient, please contact the > > > | | > sender > > > > > | | > by reply E-mail, and destroy all copies of the original > message. > > > | | > > > > | | > > > > | | > This E-mail message is for the sole use of the intended > > > | | > recipient(s) > > > | | and may contain confidential and privileged information. Any > > > | | unauthorized review, use, disclosure or distribution is > > prohibited. > > > | | If you are not the intended recipient, please contact the sender > > > > | | by reply E-mail, and destroy all copies of the original message. > > > | | > > > | | > > > | | This E-mail message is for the sole use of the intended > > > | | recipient(s) > > > > > > | | and may contain confidential and privileged information. Any > > > | | unauthorized review, use, disclosure or distribution is > > prohibited. > > > | | If you are not the intended recipient, please contact the sender > > > > | | by reply E-mail, and destroy all copies of the original message. > > > | | > > > | | > > > | | This E-mail message is for the sole use of the intended > > > | | recipient(s) > > > | and may contain confidential and privileged information. Any > > > | unauthorized review, use, disclosure or distribution is > prohibited. > > > | If you are not the intended recipient, please contact the sender > > > | by reply E-mail, and destroy all copies of the original message. > > > | | _______________________________________________ > > > | | Phoenix-pm mailing list > > > | | Phoenix-pm at pm.org > > > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > > > | > > > | > > > | This E-mail message is for the sole use of the intended > > > | recipient(s) > > > > > | and may contain confidential and privileged information. Any > > > | unauthorized review, use, disclosure or distribution is > prohibited. > > > | If you are not the intended recipient, please contact the sender > > > | by reply E-mail, and destroy all copies of the original message. > > > | > > > | > > > | This E-mail message is for the sole use of the intended > > > | recipient(s) > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > > If you are not the intended recipient, please contact the sender by > > reply E-mail, and destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > This E-mail message is for the sole use of the intended recipient(s) and > may contain confidential and privileged information. Any unauthorized > review, use, disclosure or distribution is prohibited. If you are not > the intended recipient, please contact the sender by reply E-mail, and > destroy all copies of the original message. > > > This E-mail message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply E-mail, and destroy all copies of the original message. > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm From bwmetz at att.com Wed Mar 15 17:24:44 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Wed, 15 Mar 2006 19:24:44 -0600 Subject: [Phoenix-pm] Future presentations In-Reply-To: <20060314160338.GK4287@thelackthereof.org> Message-ID: <01D5341D04A2E64AB9B345769047336701753A3A@OCCLUST01EVS1.ugd.att.com> Brock, Curiosity question on presenting. Are you looking to fill each meeting with two presentations, say one expert & one novice? Thought I remembered you saying something like that months back. Anyway, if short novice presentation is wanted, I could do one in future on "Reading/Writing ZIP Files on CPAN Challenged Unix Servers", or some other equally wacky presenation title variation. No fancy code (less then 10 minutes at most) but shows off a filehandle usage I'd never used before. Might be a good segway for one of the experts on the list to cover all the "odd" ways to use filehandles, along with strengths & weaknesses of each, beyond the simple examples in say Programming Perl...I think there are several. Just a thought. Bobby From scott at illogics.org Wed Mar 15 18:06:39 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 02:06:39 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701753A08@OCCLUST01EVS1.ugd.att.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95E87@phxmail02.phx.ndchealth.com> <01D5341D04A2E64AB9B345769047336701753A08@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060316020639.GX22790@illogics.org> > use mysql_mod; > $my_sql_file = "blahblahblah" > @results = &mysql_mod::execute($my_sql_file); Bobby, Peter, I'd like to encourage you to drop the &, though. You needed it in Perl 4; in Perl 5, you don't, but gives you a funky Perl 4 compatability mode where prototypes are disabled. You don't want this. It breaks things. > foreach $line (@results) > { > # pattern match whatever > # call other routines > # etc. > } > > This could save you time cut/pasting or re-writing the sqlplus call, Yes, there's absolutely no reason why you can't put SQL in a file rather than hard-code it into a program. > redirection, or whatever, each and every time you have a new script > need. Again, no point trying to parse the .sql file to perform the > queries in DBI...just process the output from sqlplus. ... and I posted something that would fork off a process and run it in the SQL shell. > As to leaving the group...give us another chance. I agree with > Brock that your original problem statement wasn't well defined and I If it makes you feel any better, I make about $5/hour >=) -scott From scott at illogics.org Wed Mar 15 18:10:06 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 02:10:06 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701753A1E@OCCLUST01EVS1.ugd.att.com> References: <20060316002337.GV22790@illogics.org> <01D5341D04A2E64AB9B345769047336701753A1E@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060316021006.GZ22790@illogics.org> Hi Bobby, Brock would probably know... I don't get out much. It would probably be folded in with another group, such as some Web group. -scott On 0, "Metz, Bobby W, WCS" wrote: > Scott, > If you keep talking about dict's we'll have to rebadge the group > as Phoenix Python Mongers. ;-) > > Silly joke aside, anyone know if such an entity exists in town? I have > to wear both Perl and Python hats at work and Python is definitely my > weaker language. > > Bobby > > > -----Original Message----- > From: phoenix-pm-bounces+bwmetz=att.com at pm.org > [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Scott > Walters > Sent: Wednesday, March 15, 2006 5:24 PM > To: Loo, Peter # PHX > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > Hi Peter, > > If it's your last reply, I'm truly sorry. Brock will yell at me if I > scare people away. But on the flip side of the token, when you ask > get free advice, well, it's not really, it's at the expense of the > askee rather than the asker. So, it's important to never treat free > advice as if its free. Here's one we hear all the time in EFNET > #perlhelp... "If you're going to be like that, I'm going to use PHP!" > (except without the punctuation). Yeah, that'll teach me a lesson. > Maybe > I offended you, but you're going to offend people here if you take > advice, > throw it away, ask for more, throw that away, repeat. Don't beleive > what > you're being told? That's fine, but why would you come back and ask for > > more? > > Also, as I said, I yelled a little bit to get your attention; I didn't > yell at you to hurt your feelings. I'm sorry if I miss-pegged your > experience. It happens. It's a theme I hear from novices a lot; > it doesn't mean you're a novice. You also may not even have the > attitude towards "one size fits all routines" that I thought you had -- > maybe I mispegged you twice. You're a grown man. You can handle it. > > But I don't think this was about my using stronger language; I think it > was about me (possibily) mis-pegging you as a novice. By the way, it's > Perl, not PERL. People only think it's "PERL" because book titles > tend to be printed in all-caps. If you actually open the book, though, > and it isn't one of the several that completely sucks, you'll see that > title case isn't used inside. > > > Don't think for a second that you have done it all and have > experienced > > it all. > > Heh, now you're accusing me of attitudes, but in a far less hypothetical > phrasing. But that's okay. > > > I am quite disappointed by your attitude and certainly won't have too > > much good to spread about you. If you were frustrated by the emails, > > Wait, you've "been in the business for 15 years and make a six figure > salary" and you're going to tell people bad things about me? Isn't > that a more gradeschool level tactic? Everyone who knows me knows > I'm a dict. The channel bots have a long list of nasty things novices > have said about me. I like to show it to novices before they get a > screenfull from me, but they're always still shocked to discover I'm > a dict, and act like they just discovered something. Seriously. I'm > a bad person. I don't have any natural coding abilities. I just > made a bargain with Satan. Let me know when you've come to terms with > this. > > > why didn't you just delete them and let whoever else to respond. You > > are rude, conceded and arrogant. > > I could have told you that ;) If you're going to hell, you might as > well do it right. > > And don't take me to represent the group. There are a lot of very nice > people there. > > Cheers, > -scott > > On 0, "Loo, Peter # PHX" wrote: > > > > Scott, > > > > This will be my last reply. > > > > I may be novice in PERL, but I make six figures yearly so I can't be > too > > dumb to be talked down to like you have. You are absolutely correct. > > It is NOT acceptable. I have been in IT for 15+ years and have only > met > > a handful of people like yourself who think the only right way is your > > way and that you are dreaming of being GOD. I am afraid you have > missed > > the boat there. GOD DON'T WRITE PERL OR ANY OTHER LANGUAGE. Nothing > in > > life is static. Everything is different. My way or no way is > certainly > > not the right approach. Seen it all and know it all is even worse. > > Don't think for a second that you have done it all and have > experienced > > it all. > > > > I am quite disappointed by your attitude and certainly won't have too > > much good to spread about you. If you were frustrated by the emails, > > why didn't you just delete them and let whoever else to respond. You > > are rude, conceded and arrogant. > > > > Peter Loo > > Wolters Kluwer Health > > (602) 381-9553 > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Wednesday, March 15, 2006 4:49 PM > > To: Loo, Peter # PHX > > Cc: phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > Hi Peter, > > > > I thought I was "the greatest". You changed your mind in a hurry. > > > > Sorry if I hurt your feelings. From my point of view, several people > > were telling you things that were absolutely correct and helpful, but > > they weren't "hitting their mark", so to speak. But it's often this > > way. Novices ask for advice; they reject what they're given, > > mistakeningly thinking their case is special and they're just > > misunderstood and then repeat their question, over and over; people > > repeat their answers, correct in their assessment of the situation; > > everyone gets frustrated. I just hurry the process along when I > > see things starting to go in circles. Sometimes the novice in > question > > suddenly "gets it"; sometimes pride gets in the way, and being yelled > at > > is considered unacceptable, even if it really is necessary. Some will > > disagree as to its necessity; for them, I offer a few years on a Perl > > help channel on IRC >=) We'll talk about it after then. > > > > Anyway, I'm not aggitated at you. I just thought the answers needed > > some more emphasis. I'm sorry that you read it that way. It is true > > that I'm a grumpy old cuss. If you don't post on any list with a > grupy > > old cuss, well, you're limiting yourself to not many lists ;) > > > > -scott > > > > > > On 0, "Loo, Peter # PHX" wrote: > > > > > > Hi Scott, > > > > > > Sorry that is I have gotten into your skin, however, I thought this > > > was a group to discuss and share ideas without any limitation of the > > > > level of PERL knowledge. It appear to me that you just don't like > > > anything that is not your way. Don't worry about blocking me out of > > > > this group as I don't plan to return. Brock, Mike and the rest, I > am > > > sincerely grateful for your inputs. > > > > > > Best regards. > > > > > > Peter Loo > > > Wolters Kluwer Health > > > (602) 381-9553 > > > > > > -----Original Message----- > > > From: Scott Walters [mailto:scott at illogics.org] > > > Sent: Wednesday, March 15, 2006 3:58 PM > > > To: Loo, Peter # PHX > > > Cc: Brock; phoenix-pm at pm.org > > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > > Hi Peter, > > > > > > > My goal is to have a generic PERL sub_routine that I can pass one > to > > > > > > many arguments (including the sql file name) and have it perform > the > > > > > > SQL statement that is within the sql file dynamically so that we > can > > > > > > reuse the same sub_routine for all select statement sql files. > Does > > > > > > that make sense? I am all for reusing of code and not reinvent > the > > > > wheel each > > > > > > Stop repeating it this. If it doesn't make sense, then do something > > > > else. > > > If it does, then we get it already. > > > > > > You have a choice: you can re-implement parts of the SQL shell in > > > Perl, or you can call the SQL shell. > > > > > > You keep talking about reading the file and passing the SQL... you > > > know how to do both of these things already. > > > > > > Funneling all SQL SELECT statements through one routine is silly. > > > Refactor your code however you like, but this artifical goal will > only > > > > > hurt you. If you find you're writing a lot of similar routines, > then > > > learn how to write closures. No, I don't want to talk about this or > > > > debate it; no slight intended, but this kind of simplistic view of > > > "one routine to do all" is characteristic of novices. People who > have > > > > > been doing this for a while know there's give and take. Routines > get > > > split out, then made into object methods, and those objects get > > > configured with more objects they delegate to, then common logic is > > > re-grouped into a routine somewhere, and so on. Trying to do all of > > > > any sort of thing in one places ignores the inherent complexity in a > > > > program. That's like saying "I want to paint the world PINK!". > > > But, I said this isn't up for discussion. If I see it again, I'm > > > telling mutt to ignore this thread. > > > > > > > time I have an need to run a select statement. I suppose I can > read > > > > > > in the sql file like Michael Friedman had suggested earlier in the > > > > > chain of emails. I was just hoping not to have to read in the sql > > > > > statement contained in the sql file and assigning it to a variable > > > > > before doing the $dbh->prepare. > > > > > > You want to execute it in Perl and not read it in? I think you need > > > > to focus on what you're really trying to accomplish and concentrate > > > less on how you do it, because you're starting to get silly. I > can't > > > imagine the real problem actually suggests a reading/not-reading > > paradox. > > > > > > > I need to do some brain storming. :) Not going to give up yet as > I > > > > > > am dealing with one of my two favorite languages. :) > > > > > > If you want to run it through Perl, read it in and DBI. If you want > > > > to run it in the database shell, either do so directly or from a > > > subprocess. > > > Either way, quit trying to do both-and-neither-at-the-same-time. I > > > assure you it will be entirely unproductive. > > > > > > By the way, there's often no clear solution (at least the programmer > > > > working on something), and in that case, don't fuss with it > endlessly. > > > Do it one way and be done with it, and if a clear solution occurs to > > > > you later, go back and change it. > > > > > > -scott > > > > > > > > > > > Peter Loo > > > > Wolters Kluwer Health > > > > (602) 381-9553 > > > > > > > > -----Original Message----- > > > > From: Brock [mailto:awwaiid at thelackthereof.org] > > > > Sent: Wednesday, March 15, 2006 2:02 PM > > > > To: Loo, Peter # PHX > > > > Cc: Scott Walters; phoenix-pm at pm.org > > > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > > > > Using DBI is completely different then piping things to sqlplus. I > > > > > was > > > > > > > just doing a direct translation of your example, not really trying > > > > > to imply that it was a good idea. > > > > > > > > Perl DBI does NOT use sqlplus as the driver. > > > > > > > > Unfortunately your question doesn't make much sense... if your > .sql > > > > has a select it could have many selects and it could have all > sorts > > > > of > > > > > > > things. The problem is that how can your program know what it > > > contains? > > > > It seems what you need to do is put the select like queries > directly > > > > > > into perl, as we demonstrated to you in earlier emails. Doing the > > > > whole fetchrow_arrayref thing that you were already doing. > > > > > > > > Make sense? > > > > > > > > --Brock > > > > > > > > On 2006.03.15.13.55, Loo, Peter # PHX wrote: > > > > | > > > > | Hi Brock, > > > > | > > > > | Is PERL DBI using "sqlplus" within Oracle driver? If so, it > can't > > > > > > | be efficient. Secondly, what if you have a "SELECT" statement > in > > > > | the .sql program and if you want to loop through each row? > > > > | > > > > | Peter Loo > > > > | Wolters Kluwer Health > > > > | (602) 381-9553 > > > > | > > > > | -----Original Message----- > > > > | From: Brock [mailto:awwaiid at thelackthereof.org] > > > > | Sent: Wednesday, March 15, 2006 1:52 PM > > > > | To: Loo, Peter # PHX > > > > | Cc: Scott Walters; phoenix-pm at pm.org > > > > | Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > | > > > > | Sure... what you are doing here is opening an external program > and > > > > > > | piping it data on STDIN. There are several ways to do this in > > > Perl... > > > > | > > > > | Here's one (a rough guess): > > > > | > > > > | # Initiate those vars (pull them from %ENV?) > > > > | my $dbuser = ...; > > > > | my $dbpass = ...; > > > > | my $dbconn = ...; > > > > | my $mailProgFile = ...; > > > > | > > > > | open my $sqlplus, "|-", "sqlplus -s /NOLOG" or die "ERROR: > > $!\n"; > > > > | print $sqlplus <<" HERE"; > > > > | connect ${dbuser}/${dbpass}@${dbconn} > > > > | @${mailProgFile}.sql > > > > | HERE > > > > | > > > > | More or less. Eh? > > > > | > > > > | --Brock > > > > | > > > > | On 2006.03.15.13.19, Loo, Peter # PHX wrote: > > > > | | > > > > | | Hi Scott, > > > > | | > > > > | | So will it be correct to assume that PERL DBI can not execute > an > > > > > > | | SQL > > > > > > > > | | program? For example, I can do this with Korn shell: > > > > | | > > > > | | sqlplus -s /NOLOG << EOF > > > > | | connect ${DBUSER}/${DBPASS}@${DBCONN} > > > > | | @${MailProgFile}.sql > > > > | | > > > > | | Is this not possible in PERL DBI? > > > > | | > > > > | | Peter Loo > > > > | | Wolters Kluwer Health > > > > | | (602) 381-9553 > > > > | | > > > > | | -----Original Message----- > > > > | | From: Scott Walters [mailto:scott at illogics.org] > > > > | | Sent: Tuesday, March 14, 2006 9:36 PM > > > > | | To: Loo, Peter # PHX > > > > | | Cc: Michael Friedman; phoenix-pm at pm.org > > > > | | Subject: Re: [Phoenix-pm] Running a SQL program within PERL > DBI > > > > | | > > > > | | Consider using GetOpt::Std, and most of the time you want this > > > > > | | form of > > > > | | for: > > > > | | > > > > | | for my $thing (@things) { ... stuff with $thing ... } > > > > | | > > > > | | You can always try to read rows and trap errors. > > > > | | > > > > | | -scott > > > > | | > > > > | | On 0, "Loo, Peter # PHX" > > > > wrote: > > > > | | > > > > > | | > Thanks Michael and Scott. What I am trying to do is > creating > > > > | | > a generic PERL program that will take in multiple arguments > > > > | | > such > > > as: > > > > | | > > > > > | | > for (my $cnt = -1; $cnt < $#ARGV; $cnt++) { > > > > | | > my ($flag, $value) = split(/=/, $ARGV[$cnt]); > > > > | | > switch ($flag) { > > > > | | > case "-dd" { $d_dbName = lc($value) } > > > > | | > case "-dt" { $d_tblName = lc($value) } > > > > | | > case "-ds" { $d_SQL = $value } > > > > | | > case "-sd" { $s_dbName = lc($value) } > > > > | | > case "-st" { $s_tblName = lc($value) } > > > > | | > case "-ss" { $s_SQL = $value } > > > > | | > case "-cp" { $commitPoint = lc($value) } > > > > | | > case "-sf" { $s_funcToPerf = lc($value) } > > > > | | > case "-df" { $d_funcToPerf = lc($value) } > > > > | | > case "-d1" { $s_dbDriver = lc($value) } > > > > | | > case "-d2" { $d_dbDriver = lc($value) } > > > > | | > else { print "Unknown flag: $flag\n" } > > > > | | > } > > > > | | > } > > > > | | > > > > > | | > Then execute accordingly, however, I would like to execute > an > > > > | | > external > > > > | | > > > > | | > SQL program that is passed to this generic program. In the > > > > | | > case > > > > > > > | | > of an > > > > | | > > > > | | > external SQL program that does SELECT instead of UPDATE or > > > > | | > INSERT, > > > > > > > > | | > I > > > > | > > > > | | > want to loop through the returned rows. Here is my first > > > > | | > block of > > > > | | "if" > > > > | | > statement. > > > > | | > > > > > | | > if ($s_funcToPerf eq "update") { > > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > > | | > !$commitPoint > > > > | | > || > > > > | | > !$s_SQL) { > > > > | | > print "ERROR: Not enough arguments. Require arguments > > > > | are:\n"; > > > > | | > print "Example: \n"; > > > > | | > print " Database Name: -sd=dv26\n"; > > > > | | > print " Database Driver: -d1=Oracle\n"; > > > > | | > print " Table Name: -st=p_falcon_projections\n"; > > > > | | > print " SQL Statement: > > > > | -ss=/usr/local/sql/ppv_update.sql\n"; > > > > | | > print " Commit Point: -cp=5000\n"; > > > > | | > exit(666); > > > > | | > } > > > > | | > else { > > > > | | > print "Calling sub_update()\n"; > > > > | | > } > > > > | | > } > > > > | | > elsif ($s_funcToPerf eq "insert") { > > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > > | | > !$commitPoint > > > > | | > || > > > > | | > !$s_SQL) { > > > > | | > print "ERROR: Not enough arguments. Require arguments > > > > | are:\n"; > > > > | | > print "Example: \n"; > > > > | | > print " Database Name: -sd=dv26\n"; > > > > | | > print " Database Driver: -d1=Oracle\n"; > > > > | | > print " Table Name: -st=p_falcon_projections\n"; > > > > | | > print " SQL Statement: > > > > | -ss=/usr/local/sql/ppv_insert.sql\n"; > > > > | | > print " Commit Point: -cp=5000\n"; > > > > | | > exit(666); > > > > | | > } > > > > | | > else { > > > > | | > print "Calling sub_insert()\n"; > > > > | | > } > > > > | | > } > > > > | | > elsif ($d_funcToPerf eq "update") { > > > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || > !$d_SQL) > > { > > > > | | > print "ERROR: Not enough arguments. Require arguments > > > > | are:\n"; > > > > | | > print "Example:\n"; > > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > > | | > print " Destination Table Name: > > > > | | -dt=p_falcon_projections\n"; > > > > | | > print " Source Database Name: -sd=dv26\n"; > > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > > | | > print " Source Table Name: > > > > | | -st=p_falcon_projections\n"; > > > > | | > print " SQL Statement: > > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > > | | > print " SQL Statement: > > > > | | > -ds=/usr/local/sql/ppv_update.sql\n"; > > > > | | > print " Source Function to Perform: -sf=select\n"; > > > > | | > print " Commit Point: -cp=5000\n"; > > > > | | > exit(666); > > > > | | > } > > > > | | > else { > > > > | | > print "Calling sub_select()\n"; > > > > | | > print "Calling sub_update()\n"; > > > > | | > } > > > > | | > } > > > > | | > elsif ($d_funcToPerf eq "insert") { > > > > | | > if (!$d_dbName || !$d_dbDriver || !$d_tblName || > > > > | | > !$s_dbName || !$s_dbDriver || !$s_tblName || > > > > | | > !$s_funcToPerf || !$commitPoint || !$s_SQL || > !$d_SQL) > > { > > > > | | > print "ERROR: Not enough arguments. Require arguments > > > > | are:\n"; > > > > | | > print "Example:\n"; > > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > > | | > print " Destination Table Name: > > > > | | -dt=p_falcon_projections\n"; > > > > | | > print " Source Database Name: -sd=dv26\n"; > > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > > | | > print " Source Table Name: > > > > | | -st=p_falcon_projections\n"; > > > > | | > print " SQL Statement: > > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > > | | > print " SQL Statement: > > > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > > > | | > print " Source Function to Perform: -sf=select\n"; > > > > | | > print " Commit Point: -cp=5000\n"; > > > > | | > exit(666); > > > > | | > } > > > > | | > else { > > > > | | > print "Calling sub_select()\n"; > > > > | | > print "Calling sub_insert()\n"; > > > > | | > } > > > > | | > } > > > > | | > elsif ($s_funcToPerf eq "select") { > > > > | | > if (!$s_dbName || !$s_dbDriver || !$s_tblName || > > > > | | > !$commitPoint > > > > | || > > > > | | > !$d_dbName || !$d_dbDriver || !$d_tblName || > > > > | | > !$d_funcToPerf > > > > | || > > > > | | > !$s_SQL || !$d_SQL) { > > > > | | > print "ERROR: Not enough arguments. Require arguments > > > > | are:\n"; > > > > | | > print "Example:\n"; > > > > | | > print " Destination Database Name: -dd=pv26\n"; > > > > | | > print " Destination Database Driver: -d2=ODBC\n"; > > > > | | > print " Destination Table Name: > > > > | | -dt=p_falcon_projections\n"; > > > > | | > print " Source Database Name: -sd=dv26\n"; > > > > | | > print " Source Database Driver: -d1=Oracle\n"; > > > > | | > print " Source Table Name: > > > > | | -st=p_falcon_projections\n"; > > > > | | > print " SQL Statement: > > > > | | > -ss=/usr/local/sql/ppv_select.sql\n"; > > > > | | > print " SQL Statement: > > > > | | > -ds=/usr/local/sql/ppv_insert.sql\n"; > > > > | | > print " Source Function to Perform: -sf=select\n"; > > > > | | > print " Commit Point: -cp=5000\n"; > > > > | | > exit(666); > > > > | | > } > > > > | | > else { > > > > | | > print "Calling sub_select()\n"; > > > > | | > print "Calling sub_insert()\n"; > > > > | | > } > > > > | | > } > > > > | | > else { > > > > | | > print "ERROR: Unknown value for database action to > > > > perform.\n"; > > > > | | > exit(666); > > > > | | > } > > > > | | > > > > > | | > > > > > | | > Peter Loo > > > > | | > Wolters Kluwer Health > > > > | | > (602) 381-9553 > > > > | | > > > > > | | > -----Original Message----- > > > > | | > From: Scott Walters [mailto:scott at illogics.org] > > > > | | > Sent: Tuesday, March 14, 2006 3:28 PM > > > > | | > To: Michael Friedman > > > > | | > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > > > | | > Subject: Re: [Phoenix-pm] Running a SQL program within PERL > > > > | | > DBI > > > > | | > > > > > | | > Hi Peter, > > > > | | > > > > > | | > Surely you're trying to accomplish more than just running > the > > > > | | > SQL or > > > > | > > > > | | > you would just read it in an feed it to DBI. There's no > > > > | | > reason you couldn't call to the database command shell: > > > > | | > > > > > | | > if(my $pid = fork) { > > > > | | > waitpid $pid; > > > > | | > } else { > > > > | | > close STDIN; > > > > | | > open STDIN, '<', 'foo.sql' or die $!; > > > > | | > exec 'mysql', $dbname or die $!; > > > > | | > } > > > > | | > > > > > | | > ... or something like that. If you want to use special > > > > | | > features > > > > > > > | | > of the database command shell or just cash in on its speed, > > > > | | > this > > > > > > > | | > might be > > > > | | > > > > | | > handy. Of course, you don't want to try to read values from > > > > > | | > the > > > > > > > | | > database back into Perl over a pipe between two processes... > > > > | | > that's just nasty. > > > > | | > > > > > | | > -scott > > > > | | > > > > > | | > > > > > | | > On 0, Michael Friedman > > wrote: > > > > | | > > Peter, > > > > | | > > > > > > | | > > What I did in that situation was write a couple of methods > > > > > | | > > to read > > > > | > > > > | | > > in the file, put it into an array, and then loop through > the > > > > > > | | > > array > > > > | > > > > | | > > and make each call. The good news is you only have to > write > > > > | | > > that > > > > > > > > | | > > once and then you can reuse it... > > > > | | > > > > > > | | > > My only example is using Sybase::DBlib, though, not DBI, > but > > > > > > | | > > the > > > > > > > > | | > > logic > > > > | | > > > > > | | > > would be the same. Sybase uses 'go' on a line by itself to > > > > > | | > > end > > > > > > > | | > > a > > > > > > > > | | > > SQL > > > > | | > > > > | | > > command, so we just use that to split up the lines in the > > > > | | > > file > > > > > > > | | > > into commands into the array. > > > > | | > > > > > > | | > > You could make this a lot fancier, if you had the need, > but > > > > | | > > it > > > > > > > | | > > works > > > > | | > > > > | | > > for me. > > > > | | > > > > > > | | > > Good luck, > > > > | | > > -- Mike > > > > | | > > > > > > | | > > (sub db_run_script and sub db_run_command_list, below) > > > > | | > > > > > > | | > > sub db_run_script #($$) > > > > | | > > { > > > > | | > > my $dbh = shift; > > > > | | > > my $script = shift; > > > > | | > > my $saveresults = shift; > > > > | | > > > > > > | | > > open (SQL_SCRIPT, $script) || die "Could not open input > > > > file > > > > | | > $script: > > > > | | > > $!"; > > > > | | > > > > > > | | > > my @commands = (); > > > > | | > > my $j = 0; > > > > | | > > my ($line); > > > > | | > > > > > > | | > > # read script file into a variable (array of commands) > > > > | | > > while ($line = ) > > > > | | > > { > > > > | | > > if ($line =~ /^go/) > > > > | | > > { > > > > | | > > # make new command > > > > | | > > $j++; > > > > | | > > } > > > > | | > > elsif ($line =~ /^\s*$/) > > > > | | > > { > > > > | | > > # ignore blank lines > > > > | | > > } > > > > | | > > else > > > > | | > > { > > > > | | > > $commands[$j] .= $line; > > > > | | > > } > > > > | | > > } > > > > | | > > close SQL_SCRIPT; > > > > | | > > > > > > | | > > return db_run_command_list($dbh, \@commands, > > > > $saveresults); } > > > > | | > > > > > > | | > > sub db_run_command_list > > > > | | > > { > > > > | | > > my $dbh = shift; > > > > | | > > my $cmdlist = shift; > > > > | | > > my $saveresults = shift; > > > > | | > > > > > > | | > > my @resultlist; > > > > | | > > > > > > | | > > # run commands from array > > > > | | > > for $j(0..$#$cmdlist) > > > > | | > > { > > > > | | > > $dbh->dbcmd($cmdlist->[$j]); > > > > | | > > my $status; > > > > | | > > eval { > > > > | | > > $status = $dbh->dbsqlexec(); > > > > | | > > }; > > > > | | > > > > > > | | > > if ($@ || $status != SUCCEED) > > > > | | > > { > > > > | | > > # don't always die, because drop will > > > > fail > > > > | | > sometimes > > > > | | > > if ($cmdlist->[$j] =~ /drop/i) > > > > | | > > { > > > > | | > > warn "$cmdlist->[$j] failed.\n > > > > This is > > > > | | > OK - item probably didn't > > > > | | > > exist before installation.\n"; > > > > | | > > > > > > | | > > $dbh->dbcancel(); # so that we > > > > can move > > > > | | > on to the next command? > > > > | | > > } > > > > | | > > else > > > > | | > > { > > > > | | > > die "+++ Could not run command > > > > | | > $cmdlist->[$j]\nbecause of this > > > > | | > > problem:\n$@"; > > > > | | > > } > > > > | | > > } > > > > | | > > > > > > | | > > if (!$saveresults) { > > > > | | > > db_ignore_results($dbh); > > > > | | > > } else { > > > > | | > > # Count the total number of rows that were > > > > updated, > > > > | | > > # and capture the output of any SELECT > > > > statements > > > > | | > > # > > > > | | > > # Each update/insert statement will have its > > > > own > > > > | | > update > > > > | | > > # count (a separate call to DBCOUNT()) but we > > > > will > > > > | | > > # just add them all together > > > > | | > > my $totalupdatecount = 0; > > > > | | > > while ($dbh->dbresults() != NO_MORE_RESULTS) { > > > > | | > > my $rcount = $dbh->DBCOUNT(); > > > > | | > > if ($rcount != -1) { > > > > | | > > $totalupdatecount += $rcount; > > > > | | > > } > > > > | | > > > > > > | | > > my @res; > > > > | | > > while (@res = $dbh->dbnextrow()) { > > > > | | > > my @copyres = @res; # make a copy of > > > > the > > > > | | > array > > > > | | > > push @resultlist, \@copyres; > > > > | | > > } > > > > | | > > } > > > > | | > > > > > > | | > > push @resultlist, $totalupdatecount; > > > > | | > > } > > > > | | > > } > > > > | | > > > > > > | | > > if ($saveresults) { > > > > | | > > return \@resultlist; > > > > | | > > } else { > > > > | | > > return; > > > > | | > > } > > > > | | > > } > > > > | | > > > > > > | | > > > > > > | | > > On Mar 14, 2006, at 1:47 PM, Loo, Peter # PHX wrote: > > > > | | > > > > > > | | > > > Hi, > > > > | | > > > > > > > | | > > > I know that you are able to issue a SQL statement within > > > > > | | > > > PERL DBI, > > > > | | > > > > | | > > > but is there anyway that I can issue an external SQL > > > program? > > > > > > > > | | > > > For > > > > | | > > > > | | > > > example, I have a SQL program called ppv_insert.sql that > I > > > > > > | | > > > would > > > > | > > > > | | > > > like to execute within PERL DBI. > > > > | | > > > > > > > | | > > > Thanks in advance. > > > > | | > > > > > > > | | > > > Peter Loo > > > > | | > > > > > > > | | > > > > > > > | | > > > > > > > | | > > > > > > > | | > > > This E-mail message is for the sole use of the intended > > > > | | > > > recipient > > > > | | > > > (s) and may contain confidential and privileged > > information. > > > > > > > | | > > > Any unauthorized review, use, disclosure or distribution > > > > > | | > > > is > > > > | | prohibited. > > > > | | > > > > > | | > > > If you are not the intended recipient, please contact > the > > > > | | > > > sender > > > > | > > > > | | > > > by reply E-mail, and destroy all copies of the original > > > > message. > > > > | | > > > _______________________________________________ > > > > | | > > > Phoenix-pm mailing list > > > > | | > > > Phoenix-pm at pm.org > > > > | | > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > | | > > > > > > | | > > > > > > | | > ---------------------------------------------------------------- > > > > | | -- > > > > | | -- > > > > | | - > > > > | | > > Michael Friedman HighWire Press > > > > | | > > Phone: 650-725-1974 Stanford University > > > > | | > > FAX: 270-721-8034 > > > > | | > > > > | | > > > ------------------------------------------------------------ > > > > | | > > -- > > > > | | > > -- > > > > | | > > -- > > > > | | > > -- > > > > | | > > - > > > > | | > > > > > > | | > > > > > > | | > > _______________________________________________ > > > > | | > > Phoenix-pm mailing list > > > > | | > > Phoenix-pm at pm.org > > > > | | > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > | | > > > > > | | > > > > > | | > This E-mail message is for the sole use of the intended > > > > | | > recipient(s) > > > > | > > > > | | > and may contain confidential and privileged information. > Any > > > > | | > unauthorized review, use, disclosure or distribution is > > > > prohibited. > > > > | | > If you are not the intended recipient, please contact the > > > > | | > sender > > > > > > > | | > by reply E-mail, and destroy all copies of the original > > message. > > > > | | > > > > > | | > > > > > | | > This E-mail message is for the sole use of the intended > > > > | | > recipient(s) > > > > | | and may contain confidential and privileged information. Any > > > > | | unauthorized review, use, disclosure or distribution is > > > prohibited. > > > > | | If you are not the intended recipient, please contact the > sender > > > > > > | | by reply E-mail, and destroy all copies of the original > message. > > > > | | > > > > | | > > > > | | This E-mail message is for the sole use of the intended > > > > | | recipient(s) > > > > > > > > | | and may contain confidential and privileged information. Any > > > > | | unauthorized review, use, disclosure or distribution is > > > prohibited. > > > > | | If you are not the intended recipient, please contact the > sender > > > > > > | | by reply E-mail, and destroy all copies of the original > message. > > > > | | > > > > | | > > > > | | This E-mail message is for the sole use of the intended > > > > | | recipient(s) > > > > | and may contain confidential and privileged information. Any > > > > | unauthorized review, use, disclosure or distribution is > > prohibited. > > > > | If you are not the intended recipient, please contact the sender > > > > > | by reply E-mail, and destroy all copies of the original message. > > > > | | _______________________________________________ > > > > | | Phoenix-pm mailing list > > > > | | Phoenix-pm at pm.org > > > > | | http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > | > > > > | > > > > | This E-mail message is for the sole use of the intended > > > > | recipient(s) > > > > > > > | and may contain confidential and privileged information. Any > > > > | unauthorized review, use, disclosure or distribution is > > prohibited. > > > > | If you are not the intended recipient, please contact the sender > > > > > | by reply E-mail, and destroy all copies of the original message. > > > > | > > > > | > > > > | This E-mail message is for the sole use of the intended > > > > | recipient(s) > > > > and may contain confidential and privileged information. Any > > > > unauthorized review, use, disclosure or distribution is > prohibited. > > > > If you are not the intended recipient, please contact the sender > by > > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > > > > This E-mail message is for the sole use of the intended > recipient(s) > > > > > > and may contain confidential and privileged information. Any > > > > unauthorized review, use, disclosure or distribution is > prohibited. > > > > If you are not the intended recipient, please contact the sender > by > > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > > > > This E-mail message is for the sole use of the intended > recipient(s) > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > _______________________________________________ > > > > Phoenix-pm mailing list > > > > Phoenix-pm at pm.org > > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > > > and may contain confidential and privileged information. Any > > > unauthorized review, use, disclosure or distribution is prohibited. > > > > If you are not the intended recipient, please contact the sender by > > > reply E-mail, and destroy all copies of the original message. > > > > > > > > > This E-mail message is for the sole use of the intended recipient(s) > > and may contain confidential and privileged information. Any > > unauthorized review, use, disclosure or distribution is prohibited. > If > > you are not the intended recipient, please contact the sender by reply > > E-mail, and destroy all copies of the original message. > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and > > may contain confidential and privileged information. Any unauthorized > > review, use, disclosure or distribution is prohibited. If you are not > > the intended recipient, please contact the sender by reply E-mail, and > > destroy all copies of the original message. > > > > > > This E-mail message is for the sole use of the intended recipient(s) > and may contain confidential and privileged information. Any > unauthorized review, use, disclosure or distribution is prohibited. If > you are not the intended recipient, please contact the sender by reply > E-mail, and destroy all copies of the original message. > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Wed Mar 15 18:13:40 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 02:13:40 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> Message-ID: <20060316021340.GA22790@illogics.org> On 0, "Loo, Peter # PHX" wrote: > > Scott, > > This will be my last reply. Just so you know, I'm going to reply to this a few dozen more times >=) -scott From bwmetz at att.com Wed Mar 15 18:18:02 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Wed, 15 Mar 2006 20:18:02 -0600 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <20060316020639.GX22790@illogics.org> Message-ID: <01D5341D04A2E64AB9B345769047336701753A61@OCCLUST01EVS1.ugd.att.com> Hey thanks for the "&" tip. Old habit I guess since the folks I learned from used it in all their code and I've never read up on Perl 5 calls to find out otherwise. Thought I knew what I was doing :-) So, just "@results = mysql_mod::execute($my_sql_file);" does the trick? Bobby -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Wednesday, March 15, 2006 7:07 PM To: Metz, Bobby W, WCS Cc: Loo, Peter # PHX; phoenix-pm at pm.org Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > use mysql_mod; > $my_sql_file = "blahblahblah" > @results = &mysql_mod::execute($my_sql_file); Bobby, Peter, I'd like to encourage you to drop the &, though. You needed it in Perl 4; in Perl 5, you don't, but gives you a funky Perl 4 compatability mode where prototypes are disabled. You don't want this. It breaks things. > foreach $line (@results) > { > # pattern match whatever > # call other routines > # etc. > } > > This could save you time cut/pasting or re-writing the sqlplus call, Yes, there's absolutely no reason why you can't put SQL in a file rather than hard-code it into a program. > redirection, or whatever, each and every time you have a new script > need. Again, no point trying to parse the .sql file to perform the > queries in DBI...just process the output from sqlplus. ... and I posted something that would fork off a process and run it in the SQL shell. > As to leaving the group...give us another chance. I agree with > Brock that your original problem statement wasn't well defined and I If it makes you feel any better, I make about $5/hour >=) -scott From intertwingled at qwest.net Wed Mar 15 19:29:06 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Wed, 15 Mar 2006 20:29:06 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701753A61@OCCLUST01EVS1.ugd.att.com> References: <01D5341D04A2E64AB9B345769047336701753A61@OCCLUST01EVS1.ugd.att.com> Message-ID: <4418DB82.6060806@qwest.net> What SPECIFICALLY is wrong with using the & to identify a subroutine call, if I am not using prototypes? Are you saying that it just makes the code look uglier? I always thought prototypes were a hack anyway. Perl has always used sigils to identify various entities... $ for scalars, @ for arrays, % for hashs, \ for references, * for typeglobs, & for subroutines. If I am not using prototypes, I don't see what the problem is with using & to identify a subroutine call. I understand that this may all change in Perl 6. Tony Metz, Bobby W, WCS wrote: > Hey thanks for the "&" tip. Old habit I guess since the folks I learned > from used it in all their code and I've never read up on Perl 5 calls to > find out otherwise. Thought I knew what I was doing :-) > > So, just "@results = mysql_mod::execute($my_sql_file);" does the trick? > > Bobby > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Wednesday, March 15, 2006 7:07 PM > To: Metz, Bobby W, WCS > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > >> use mysql_mod; >> $my_sql_file = "blahblahblah" >> @results = &mysql_mod::execute($my_sql_file); > > Bobby, Peter, > > I'd like to encourage you to drop the &, though. You needed it in Perl > 4; > in Perl 5, you don't, but gives you a funky Perl 4 compatability mode > where prototypes are disabled. You don't want this. It breaks things. > >> foreach $line (@results) >> { >> # pattern match whatever >> # call other routines >> # etc. >> } >> >> This could save you time cut/pasting or re-writing the sqlplus call, > > Yes, there's absolutely no reason why you can't put SQL in a file > rather than hard-code it into a program. > >> redirection, or whatever, each and every time you have a new script >> need. Again, no point trying to parse the .sql file to perform the >> queries in DBI...just process the output from sqlplus. > > ... and I posted something that would fork off a process and run it > in the SQL shell. > >> As to leaving the group...give us another chance. I agree with >> Brock that your original problem statement wasn't well defined and I > > If it makes you feel any better, I make about $5/hour >=) > > -scott > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! From awwaiid at thelackthereof.org Wed Mar 15 20:35:44 2006 From: awwaiid at thelackthereof.org (Brock) Date: Wed, 15 Mar 2006 21:35:44 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701753A1E@OCCLUST01EVS1.ugd.att.com> References: <20060316002337.GV22790@illogics.org> <01D5341D04A2E64AB9B345769047336701753A1E@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060316043544.GO4287@thelackthereof.org> There is no Python group of which I'm aware (but there is PHP, Java, C# I think, ColdFusion, and now Ruby). But Python programmers are welcome at our meetings (as are all the rest!) :) --Brock On 2006.03.15.18.41, Metz, Bobby W, WCS wrote: | Scott, | If you keep talking about dict's we'll have to rebadge the group | as Phoenix Python Mongers. ;-) | | Silly joke aside, anyone know if such an entity exists in town? I have | to wear both Perl and Python hats at work and Python is definitely my | weaker language. | | Bobby From awwaiid at thelackthereof.org Wed Mar 15 20:41:11 2006 From: awwaiid at thelackthereof.org (Brock) Date: Wed, 15 Mar 2006 21:41:11 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> Message-ID: <20060316044111.GP4287@thelackthereof.org> On 2006.03.15.16.57, Loo, Peter # PHX wrote: | ... | the boat there. GOD DON'T WRITE PERL OR ANY OTHER LANGUAGE. Nothing in | ... WHAT?! BLASPHEMER! --Brock From awwaiid at thelackthereof.org Wed Mar 15 20:45:03 2006 From: awwaiid at thelackthereof.org (Brock) Date: Wed, 15 Mar 2006 21:45:03 -0700 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <20060316002337.GV22790@illogics.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> <20060316002337.GV22790@illogics.org> Message-ID: <20060316044503.GQ4287@thelackthereof.org> On 2006.03.16.00.23, Scott Walters wrote: | Hi Peter, | | If it's your last reply, I'm truly sorry. Brock will yell at me if I | scare people away. But on the flip side of the token, when you ask | ... I actually think that you're doing quite well at being patient, relatively speaking. Good job Scott :) Seriously. Peter, come on out to the meeting Thursday night and I'll hold Scott down for you and we'll figure out how to do what it is you want to do (right after we figure out what it is you want to do). Good Day, --Brock From awwaiid at thelackthereof.org Wed Mar 15 20:46:55 2006 From: awwaiid at thelackthereof.org (Brock) Date: Wed, 15 Mar 2006 21:46:55 -0700 Subject: [Phoenix-pm] Future presentations In-Reply-To: <01D5341D04A2E64AB9B345769047336701753A3A@OCCLUST01EVS1.ugd.att.com> References: <20060314160338.GK4287@thelackthereof.org> <01D5341D04A2E64AB9B345769047336701753A3A@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060316044655.GR4287@thelackthereof.org> Yes. --Brock On 2006.03.15.19.24, Metz, Bobby W, WCS wrote: | Brock, | Curiosity question on presenting. Are you looking to fill each | meeting with two presentations, say one expert & one novice? Thought I | remembered you saying something like that months back. Anyway, if short | novice presentation is wanted, I could do one in future on | "Reading/Writing ZIP Files on CPAN Challenged Unix Servers", or some | other equally wacky presenation title variation. No fancy code (less | then 10 minutes at most) but shows off a filehandle usage I'd never used | before. Might be a good segway for one of the experts on the list to | cover all the "odd" ways to use filehandles, along with strengths & | weaknesses of each, beyond the simple examples in say Programming | Perl...I think there are several. | | Just a thought. | | Bobby | _______________________________________________ | Phoenix-pm mailing list | Phoenix-pm at pm.org | http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Thu Mar 16 07:58:23 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 15:58:23 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <4418DB82.6060806@qwest.net> References: <01D5341D04A2E64AB9B345769047336701753A61@OCCLUST01EVS1.ugd.att.com> <4418DB82.6060806@qwest.net> Message-ID: <20060316155823.GB22790@illogics.org> Hi Tony, In Perl 6, &sub is a reference to the sub, like \&sub in Perl 5. Or something like that. Typeglobs are going away, of course. Even if you don't use prototypes, other people might, and then & breaks the code. sub foo (\%) { print "stuff: ", $_[0]->{foo}, "\n"; } my %stuff = (foo => "bar"); foo(%stuff); &foo(%stuff); When calling functions exported by modules (or not exported), you don't know if they're using prototypes. Yes, prototypes are a bit of a hack, but only insomuch as they aren't complete. I'd love to be able to prototype things like how grep $_ eq 5, @numbers or map "$_\n", @things work, where the first argument is an expression that gets treated like a code block. &foo also defaults to sending @_ in the function call, which is sometimes handy but in other classes makes action-at-a-distance too easy, clobbering @_ in some remote subroutine. But most of all, &foo identifies the programmer as someone who learned to program by immitating someone who immitated someone else and now writes Perl like Matt Wright. As for whether or not Perl 4 is inherently bad, no, of course not. Perl 1 isn't inherently bad. But Perl 5 offers some major improvements -- three argument open, two argument system (for security), references (no more nasty glob tricks to pass datastructures), etc. If you want to write in a Perl 4 style and actually *know* Perl 4 and aren't just doing a Matt Wright impression, that's fine, but IMO (I AM GOD EVERYTHING I SAY IS RIGHT EVERYONE WHO DISAGREES IS WRONG) even old Perl 4 programmers should come up to speed on those few things or else they're writing inferior code by any metric. Okay, rant done =) -scott On 0, "Anthony R. Nemmer" wrote: > What SPECIFICALLY is wrong with using the & to identify a subroutine > call, if I am not using prototypes? Are you saying that it just makes > the code look uglier? I always thought prototypes were a hack anyway. > > Perl has always used sigils to identify various entities... > $ for scalars, @ for arrays, % for hashs, \ for references, * for > typeglobs, & for subroutines. If I am not using prototypes, I don't see > what the problem is with using & to identify a subroutine call. > > I understand that this may all change in Perl 6. > > Tony > > Metz, Bobby W, WCS wrote: > > Hey thanks for the "&" tip. Old habit I guess since the folks I learned > > from used it in all their code and I've never read up on Perl 5 calls to > > find out otherwise. Thought I knew what I was doing :-) > > > > So, just "@results = mysql_mod::execute($my_sql_file);" does the trick? > > > > Bobby > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Wednesday, March 15, 2006 7:07 PM > > To: Metz, Bobby W, WCS > > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > >> use mysql_mod; > >> $my_sql_file = "blahblahblah" > >> @results = &mysql_mod::execute($my_sql_file); > > > > Bobby, Peter, > > > > I'd like to encourage you to drop the &, though. You needed it in Perl > > 4; > > in Perl 5, you don't, but gives you a funky Perl 4 compatability mode > > where prototypes are disabled. You don't want this. It breaks things. > > > >> foreach $line (@results) > >> { > >> # pattern match whatever > >> # call other routines > >> # etc. > >> } > >> > >> This could save you time cut/pasting or re-writing the sqlplus call, > > > > Yes, there's absolutely no reason why you can't put SQL in a file > > rather than hard-code it into a program. > > > >> redirection, or whatever, each and every time you have a new script > >> need. Again, no point trying to parse the .sql file to perform the > >> queries in DBI...just process the output from sqlplus. > > > > ... and I posted something that would fork off a process and run it > > in the SQL shell. > > > >> As to leaving the group...give us another chance. I agree with > >> Brock that your original problem statement wasn't well defined and I > > > > If it makes you feel any better, I make about $5/hour >=) > > > > -scott > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Thu Mar 16 08:01:57 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 16:01:57 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <20060316044503.GQ4287@thelackthereof.org> References: <8E3D502A002DA04FADBDED4CB4D94D3AE95EAF@phxmail02.phx.ndchealth.com> <20060316002337.GV22790@illogics.org> <20060316044503.GQ4287@thelackthereof.org> Message-ID: <20060316160156.GD22790@illogics.org> Yes. These things are *much* easier to discuss in person. I read a study on Slashdot that PROVED (comically over-energetic voice that acknowledges the silliness of citing a Slashdot study) that only about 50% of emails accurate convey tone. But even in real life, people hit me a lot. I drive them too it. Especially women. -scott On 0, Brock wrote: > On 2006.03.16.00.23, Scott Walters wrote: > | Hi Peter, > | > | If it's your last reply, I'm truly sorry. Brock will yell at me if I > | scare people away. But on the flip side of the token, when you ask > | ... > > I actually think that you're doing quite well at being patient, > relatively speaking. Good job Scott :) > > Seriously. > > Peter, come on out to the meeting Thursday night and I'll hold Scott > down for you and we'll figure out how to do what it is you want to do > (right after we figure out what it is you want to do). > > Good Day, > --Brock From bwmetz at att.com Thu Mar 16 08:47:48 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Thu, 16 Mar 2006 10:47:48 -0600 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <20060316155823.GB22790@illogics.org> Message-ID: <01D5341D04A2E64AB9B345769047336701781C4B@OCCLUST01EVS1.ugd.att.com> Well, I was looking for a simple "yes, P5 call without & works", but I like the example. That said, I'm sure I'm opening myself up for ridicule, but the way I learned subs was that you pass the reference to the hash, not the var itself. sub foo (\%) { print "Scott foo: ", $_[0]->{foo}, "\n"; $_[0]->{foo} .= " Scott"; } my %stuff = (foo => "bar"); foo(%stuff); &foo(%stuff); sub foo2 { print "Bob foo: ", $_[0]->{foo}, "\n"; $_[0]->{foo} .= " Bob"; } my %stuff = (foo => "bar"); foo2(\%stuff); &foo2(\%stuff); Output: Scott foo: bar Scott foo: Bob foo: bar Bob foo: bar Bob Guess this has always made more sense to me since I first cut my teeth on ANSI C where you had to pass the ref to modify the values. So, by passing the ref I can read and write. But, it doesn't make an argument for using & since either calling method with the hash ref works, so I see the "yes" answer I was looking for and I don' t have to change my style (again, ridicule?). Bobby -----Original Message----- From: Scott Walters [mailto:scott at illogics.org] Sent: Thursday, March 16, 2006 8:58 AM To: Anthony R. Nemmer Cc: Metz, Bobby W, WCS; phoenix-pm at pm.org; Loo, Peter # PHX Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI Hi Tony, In Perl 6, &sub is a reference to the sub, like \&sub in Perl 5. Or something like that. Typeglobs are going away, of course. Even if you don't use prototypes, other people might, and then & breaks the code. sub foo (\%) { print "stuff: ", $_[0]->{foo}, "\n"; } my %stuff = (foo => "bar"); foo(%stuff); &foo(%stuff); When calling functions exported by modules (or not exported), you don't know if they're using prototypes. Yes, prototypes are a bit of a hack, but only insomuch as they aren't complete. I'd love to be able to prototype things like how grep $_ eq 5, @numbers or map "$_\n", @things work, where the first argument is an expression that gets treated like a code block. &foo also defaults to sending @_ in the function call, which is sometimes handy but in other classes makes action-at-a-distance too easy, clobbering @_ in some remote subroutine. But most of all, &foo identifies the programmer as someone who learned to program by immitating someone who immitated someone else and now writes Perl like Matt Wright. As for whether or not Perl 4 is inherently bad, no, of course not. Perl 1 isn't inherently bad. But Perl 5 offers some major improvements -- three argument open, two argument system (for security), references (no more nasty glob tricks to pass datastructures), etc. If you want to write in a Perl 4 style and actually *know* Perl 4 and aren't just doing a Matt Wright impression, that's fine, but IMO (I AM GOD EVERYTHING I SAY IS RIGHT EVERYONE WHO DISAGREES IS WRONG) even old Perl 4 programmers should come up to speed on those few things or else they're writing inferior code by any metric. Okay, rant done =) -scott On 0, "Anthony R. Nemmer" wrote: > What SPECIFICALLY is wrong with using the & to identify a subroutine > call, if I am not using prototypes? Are you saying that it just makes > the code look uglier? I always thought prototypes were a hack anyway. > > Perl has always used sigils to identify various entities... > $ for scalars, @ for arrays, % for hashs, \ for references, * for > typeglobs, & for subroutines. If I am not using prototypes, I don't see > what the problem is with using & to identify a subroutine call. > > I understand that this may all change in Perl 6. > > Tony > > Metz, Bobby W, WCS wrote: > > Hey thanks for the "&" tip. Old habit I guess since the folks I learned > > from used it in all their code and I've never read up on Perl 5 calls to > > find out otherwise. Thought I knew what I was doing :-) > > > > So, just "@results = mysql_mod::execute($my_sql_file);" does the trick? > > > > Bobby > > > > -----Original Message----- > > From: Scott Walters [mailto:scott at illogics.org] > > Sent: Wednesday, March 15, 2006 7:07 PM > > To: Metz, Bobby W, WCS > > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > >> use mysql_mod; > >> $my_sql_file = "blahblahblah" > >> @results = &mysql_mod::execute($my_sql_file); > > > > Bobby, Peter, > > > > I'd like to encourage you to drop the &, though. You needed it in Perl > > 4; > > in Perl 5, you don't, but gives you a funky Perl 4 compatability mode > > where prototypes are disabled. You don't want this. It breaks things. > > > >> foreach $line (@results) > >> { > >> # pattern match whatever > >> # call other routines > >> # etc. > >> } > >> > >> This could save you time cut/pasting or re-writing the sqlplus call, > > > > Yes, there's absolutely no reason why you can't put SQL in a file > > rather than hard-code it into a program. > > > >> redirection, or whatever, each and every time you have a new script > >> need. Again, no point trying to parse the .sql file to perform the > >> queries in DBI...just process the output from sqlplus. > > > > ... and I posted something that would fork off a process and run it > > in the SQL shell. > > > >> As to leaving the group...give us another chance. I agree with > >> Brock that your original problem statement wasn't well defined and I > > > > If it makes you feel any better, I make about $5/hour >=) > > > > -scott > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From bwmetz at att.com Thu Mar 16 09:25:06 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Thu, 16 Mar 2006 11:25:06 -0600 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <4415EDFB.5070003@qwest.net> Message-ID: <01D5341D04A2E64AB9B345769047336701781D07@OCCLUST01EVS1.ugd.att.com> Wow. Catching up on some of these mails. Have to say I missed some. Never seen this list so active on one topic. Was Model 204 so fast because it was non-relational? Or was it just the indexing scheme you think? B -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Anthony R. Nemmer Sent: Monday, March 13, 2006 3:11 PM To: Brock Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] PERL DBI Makes me pine away for the good old days, when I used Model 204 on IBM mainframes, a non-relational database system that could easily handle hundreds of millions of records. Too bad they never ported it over to Unix or Window, it was a great database engine. Had a "User Language" database programming language with database and screen primitives that was very Perlish, too. Model 204 used inverted trees for indexing, so queries were simply bitwise and'ing and or'ing bitmaps together. Response time for a query on a 900 million record database was typically under 3 seconds. Brock wrote: > I say not too bad... 2 and a half million records is nothing to sneeze > at. Thats just over 467 rows/sec, I figure. Lot faster than doing it by > hand! :) > > Ideas for presentations: > * DBI > * Profiling Perl Code > > --Brock > > On 2006.03.13.14.37, Loo, Peter # PHX wrote: > | > | Here is what it took to run: > | > | Mon Mar 13 13:05:34 MST 2006 > | Mon Mar 13 14:31:27 MST 2006 > | > | SQL> select count(*) from dssppv.tmp_falcon_projections; > | > | COUNT(*) > | ---------- > | 2413059 > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Thu Mar 16 09:54:43 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 17:54:43 +0000 Subject: [Phoenix-pm] Running a SQL program within PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701781C4B@OCCLUST01EVS1.ugd.att.com> References: <20060316155823.GB22790@illogics.org> <01D5341D04A2E64AB9B345769047336701781C4B@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060316175442.GF22790@illogics.org> Hi Bobby, Well, prototypes let you emulate built-in functions, as that was why they were created. They don't do type checking, but there are modules to do that. One of the things built-ins do is operate on a hash without the user having to explicitly pass it by reference. They also accept code blocks as arguments without 'sub' in front of them and a few other things. But prototypes are useful for letting the user write more natural code that works more like built-in functions. Whether or not that's of value, it's of value to not break it when people use do it ;) -scott On 0, "Metz, Bobby W, WCS" wrote: > Well, I was looking for a simple "yes, P5 call without & works", > but I like the example. That said, I'm sure I'm opening myself up for > ridicule, but the way I learned subs was that you pass the reference to > the hash, not the var itself. > > sub foo (\%) { print "Scott foo: ", $_[0]->{foo}, "\n"; > $_[0]->{foo} .= " Scott"; } > my %stuff = (foo => "bar"); > foo(%stuff); > &foo(%stuff); > > sub foo2 { print "Bob foo: ", $_[0]->{foo}, "\n"; $_[0]->{foo} > .= " Bob"; } > my %stuff = (foo => "bar"); > foo2(\%stuff); > &foo2(\%stuff); > > Output: > Scott foo: bar > Scott foo: > Bob foo: bar > Bob foo: bar Bob > > Guess this has always made more sense to me since I first cut my > teeth on ANSI C where you had to pass the ref to modify the values. So, > by passing the ref I can read and write. But, it doesn't make an > argument for using & since either calling method with the hash ref > works, so I see the "yes" answer I was looking for and I don' t have to > change my style (again, ridicule?). > > Bobby > > -----Original Message----- > From: Scott Walters [mailto:scott at illogics.org] > Sent: Thursday, March 16, 2006 8:58 AM > To: Anthony R. Nemmer > Cc: Metz, Bobby W, WCS; phoenix-pm at pm.org; Loo, Peter # PHX > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > Hi Tony, > > In Perl 6, &sub is a reference to the sub, like \&sub in Perl 5. Or > something like that. Typeglobs are going away, of course. > > Even if you don't use prototypes, other people might, and then & breaks > the code. > > sub foo (\%) { print "stuff: ", $_[0]->{foo}, "\n"; } > > my %stuff = (foo => "bar"); > > foo(%stuff); > &foo(%stuff); > > When calling functions exported by modules (or not exported), you don't > know > if they're using prototypes. Yes, prototypes are a bit of a hack, but > only > insomuch as they aren't complete. I'd love to be able to prototype > things > like how grep $_ eq 5, @numbers or map "$_\n", @things work, where the > first > argument is an expression that gets treated like a code block. > > &foo also defaults to sending @_ in the function call, which is > sometimes handy > but in other classes makes action-at-a-distance too easy, clobbering @_ > in > some remote subroutine. > > But most of all, &foo identifies the programmer as someone who learned > to > program by immitating someone who immitated someone else and now writes > Perl like Matt Wright. > > As for whether or not Perl 4 is inherently bad, no, of course not. Perl > 1 > isn't inherently bad. But Perl 5 offers some major improvements -- > three > argument open, two argument system (for security), references (no more > nasty > glob tricks to pass datastructures), etc. If you want to write in a > Perl 4 > style and actually *know* Perl 4 and aren't just doing a Matt Wright > impression, > that's fine, but IMO (I AM GOD EVERYTHING I SAY IS RIGHT EVERYONE WHO > DISAGREES > IS WRONG) even old Perl 4 programmers should come up to speed on those > few > things or else they're writing inferior code by any metric. > > Okay, rant done =) > > -scott > > On 0, "Anthony R. Nemmer" wrote: > > What SPECIFICALLY is wrong with using the & to identify a subroutine > > call, if I am not using prototypes? Are you saying that it just makes > > > the code look uglier? I always thought prototypes were a hack anyway. > > > > Perl has always used sigils to identify various entities... > > $ for scalars, @ for arrays, % for hashs, \ for references, * for > > typeglobs, & for subroutines. If I am not using prototypes, I don't > see > > what the problem is with using & to identify a subroutine call. > > > > I understand that this may all change in Perl 6. > > > > Tony > > > > Metz, Bobby W, WCS wrote: > > > Hey thanks for the "&" tip. Old habit I guess since the folks I > learned > > > from used it in all their code and I've never read up on Perl 5 > calls to > > > find out otherwise. Thought I knew what I was doing :-) > > > > > > So, just "@results = mysql_mod::execute($my_sql_file);" does the > trick? > > > > > > Bobby > > > > > > -----Original Message----- > > > From: Scott Walters [mailto:scott at illogics.org] > > > Sent: Wednesday, March 15, 2006 7:07 PM > > > To: Metz, Bobby W, WCS > > > Cc: Loo, Peter # PHX; phoenix-pm at pm.org > > > Subject: Re: [Phoenix-pm] Running a SQL program within PERL DBI > > > > > > > > >> use mysql_mod; > > >> $my_sql_file = "blahblahblah" > > >> @results = &mysql_mod::execute($my_sql_file); > > > > > > Bobby, Peter, > > > > > > I'd like to encourage you to drop the &, though. You needed it in > Perl > > > 4; > > > in Perl 5, you don't, but gives you a funky Perl 4 compatability > mode > > > where prototypes are disabled. You don't want this. It breaks > things. > > > > > >> foreach $line (@results) > > >> { > > >> # pattern match whatever > > >> # call other routines > > >> # etc. > > >> } > > >> > > >> This could save you time cut/pasting or re-writing the sqlplus > call, > > > > > > Yes, there's absolutely no reason why you can't put SQL in a file > > > rather than hard-code it into a program. > > > > > >> redirection, or whatever, each and every time you have a new script > > >> need. Again, no point trying to parse the .sql file to perform the > > >> queries in DBI...just process the output from sqlplus. > > > > > > ... and I posted something that would fork off a process and run it > > > in the SQL shell. > > > > > >> As to leaving the group...give us another chance. I agree with > > >> Brock that your original problem statement wasn't well defined and > I > > > > > > If it makes you feel any better, I make about $5/hour >=) > > > > > > -scott > > > _______________________________________________ > > > Phoenix-pm mailing list > > > Phoenix-pm at pm.org > > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > > > > > > -- > > > > I always have coffee when I watch radar! > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm From scott at illogics.org Thu Mar 16 09:55:16 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 17:55:16 +0000 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701781D07@OCCLUST01EVS1.ugd.att.com> References: <4415EDFB.5070003@qwest.net> <01D5341D04A2E64AB9B345769047336701781D07@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060316175516.GG22790@illogics.org> Yeah Tony, at least write up some documentation and blog it so the design is preserved. -scott On 0, "Metz, Bobby W, WCS" wrote: > Wow. Catching up on some of these mails. Have to say I missed some. > Never seen this list so active on one topic. Was Model 204 so fast > because it was non-relational? Or was it just the indexing scheme you > think? > > B > > -----Original Message----- > From: phoenix-pm-bounces+bwmetz=att.com at pm.org > [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Anthony R. > Nemmer > Sent: Monday, March 13, 2006 3:11 PM > To: Brock > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > > Makes me pine away for the good old days, when I used Model 204 on IBM > mainframes, a non-relational database system that could easily handle > hundreds of millions of records. Too bad they never ported it over to > Unix or Window, it was a great database engine. Had a "User Language" > database programming language with database and screen primitives that > was very Perlish, too. Model 204 used inverted trees for indexing, so > queries were simply bitwise and'ing and or'ing bitmaps together. > Response time for a query on a 900 million record database was typically > > under 3 seconds. > > Brock wrote: > > I say not too bad... 2 and a half million records is nothing to sneeze > > at. Thats just over 467 rows/sec, I figure. Lot faster than doing it > by > > hand! :) > > > > Ideas for presentations: > > * DBI > > * Profiling Perl Code > > > > --Brock > > > > On 2006.03.13.14.37, Loo, Peter # PHX wrote: > > | > > | Here is what it took to run: > > | > > | Mon Mar 13 13:05:34 MST 2006 > > | Mon Mar 13 14:31:27 MST 2006 > > | > > | SQL> select count(*) from dssppv.tmp_falcon_projections; > > | > > | COUNT(*) > > | ---------- > > | 2413059 > > _______________________________________________ > > Phoenix-pm mailing list > > Phoenix-pm at pm.org > > http://mail.pm.org/mailman/listinfo/phoenix-pm > > > > > > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From intertwingled at qwest.net Thu Mar 16 09:56:15 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Thu, 16 Mar 2006 10:56:15 -0700 Subject: [Phoenix-pm] PERL DBI In-Reply-To: <01D5341D04A2E64AB9B345769047336701781D07@OCCLUST01EVS1.ugd.att.com> References: <01D5341D04A2E64AB9B345769047336701781D07@OCCLUST01EVS1.ugd.att.com> Message-ID: <4419A6BF.80709@qwest.net> Both. Metz, Bobby W, WCS wrote: > Wow. Catching up on some of these mails. Have to say I missed some. > Never seen this list so active on one topic. Was Model 204 so fast > because it was non-relational? Or was it just the indexing scheme you > think? > > B > > -----Original Message----- > From: phoenix-pm-bounces+bwmetz=att.com at pm.org > [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Anthony R. > Nemmer > Sent: Monday, March 13, 2006 3:11 PM > To: Brock > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] PERL DBI > > > Makes me pine away for the good old days, when I used Model 204 on IBM > mainframes, a non-relational database system that could easily handle > hundreds of millions of records. Too bad they never ported it over to > Unix or Window, it was a great database engine. Had a "User Language" > database programming language with database and screen primitives that > was very Perlish, too. Model 204 used inverted trees for indexing, so > queries were simply bitwise and'ing and or'ing bitmaps together. > Response time for a query on a 900 million record database was typically > > under 3 seconds. > > Brock wrote: >> I say not too bad... 2 and a half million records is nothing to sneeze >> at. Thats just over 467 rows/sec, I figure. Lot faster than doing it > by >> hand! :) >> >> Ideas for presentations: >> * DBI >> * Profiling Perl Code >> >> --Brock >> >> On 2006.03.13.14.37, Loo, Peter # PHX wrote: >> | >> | Here is what it took to run: >> | >> | Mon Mar 13 13:05:34 MST 2006 >> | Mon Mar 13 14:31:27 MST 2006 >> | >> | SQL> select count(*) from dssppv.tmp_falcon_projections; >> | >> | COUNT(*) >> | ---------- >> | 2413059 >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm >> >> > > -- I always have coffee when I watch radar! From intertwingled at qwest.net Thu Mar 16 10:55:37 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Thu, 16 Mar 2006 11:55:37 -0700 Subject: [Phoenix-pm] I don't buy it Message-ID: <4419B4A9.3000902@qwest.net> I don't buy the "Matt Wright" argument or whoever it is. I think people can write perfectly good code in Perl 5 and use the & when calling subroutines. You just need to be careful, per perlsub, in how you use &. It's been my experience that prototypes are truly a hack, and that they are not used that often. Again, this all may change with Perl 6, but when will Perl 6 be released? It's probably years away. Personally I think sigils are one of the things that makes Perl Perl, and that typeglobs are pretty cool. Tony -- I always have coffee when I watch radar! From scott at illogics.org Thu Mar 16 12:22:18 2006 From: scott at illogics.org (Scott Walters) Date: Thu, 16 Mar 2006 20:22:18 +0000 Subject: [Phoenix-pm] I don't buy it In-Reply-To: <4419B4A9.3000902@qwest.net> References: <4419B4A9.3000902@qwest.net> Message-ID: <20060316202218.GK22790@illogics.org> Matt Wright wrote formmail.pl which, though it was done in bad style, is famous for having the highest density of vulnerabilities per line in any program known to man. It single handled made Perl the largest spam relay right behend open Sendmail servers for a solid 7 years running. Matt Wright writes bad code. My references to people who learned to code by imitating him implied they had far more than style or preference wrong with their code -- their code is bad. As I said, usually people who write &func aren't old Perl 4 programmers but are people who learned from the wretched code floating around -- the most common of which is Matt's. -scott On 0, "Anthony R. Nemmer" wrote: > I don't buy the "Matt Wright" argument or whoever it is. I think people > can write perfectly good code in Perl 5 and use the & when calling > subroutines. You just need to be careful, per perlsub, in how you use > &. It's been my experience that prototypes are truly a hack, and that > they are not used that often. Again, this all may change with Perl 6, > but when will Perl 6 be released? It's probably years away. Personally > I think sigils are one of the things that makes Perl Perl, and that > typeglobs are pretty cool. > > Tony > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From billn at odyssey.billn.net Thu Mar 16 13:25:17 2006 From: billn at odyssey.billn.net (Bill Nash) Date: Thu, 16 Mar 2006 16:25:17 -0500 (EST) Subject: [Phoenix-pm] I don't buy it In-Reply-To: <20060316202218.GK22790@illogics.org> References: <4419B4A9.3000902@qwest.net> <20060316202218.GK22790@illogics.org> Message-ID: Code is code is code. Like a katana in the hands of a master practitioner, it can be a precision instrument wielded with grace and power. Hand that sword to a gardener, and in the absence of a better tool or instruction, it will become a weed whacker. - billn On Thu, 16 Mar 2006, Scott Walters wrote: > Matt Wright wrote formmail.pl which, though it was done in bad style, is > famous for having the highest density of vulnerabilities per line in any > program known to man. It single handled made Perl the largest spam relay > right behend open Sendmail servers for a solid 7 years running. Matt > Wright writes bad code. My references to people who learned to code by > imitating him implied they had far more than style or preference wrong > with their code -- their code is bad. As I said, usually people who > write &func aren't old Perl 4 programmers but are people who learned from > the wretched code floating around -- the most common of which is Matt's. > > -scott > > On 0, "Anthony R. Nemmer" wrote: >> I don't buy the "Matt Wright" argument or whoever it is. I think people >> can write perfectly good code in Perl 5 and use the & when calling >> subroutines. You just need to be careful, per perlsub, in how you use >> &. It's been my experience that prototypes are truly a hack, and that >> they are not used that often. Again, this all may change with Perl 6, >> but when will Perl 6 be released? It's probably years away. Personally >> I think sigils are one of the things that makes Perl Perl, and that >> typeglobs are pretty cool. >> >> Tony >> >> -- >> >> I always have coffee when I watch radar! >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > From bwmetz at att.com Thu Mar 16 13:56:47 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Thu, 16 Mar 2006 15:56:47 -0600 Subject: [Phoenix-pm] I don't buy it In-Reply-To: Message-ID: <01D5341D04A2E64AB9B345769047336701782134@OCCLUST01EVS1.ugd.att.com> ah, but how smoothly those weeds would fall! Personally, I prefer the torch wand for my propane tank. Much more satisfying to see a 1.5 foot blue flame shooting everywhere and the fear in the neighbors eyes when they hear the ungodly roar. :D jk -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Bill Nash Sent: Thursday, March 16, 2006 2:25 PM To: Scott Walters Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] I don't buy it Code is code is code. Like a katana in the hands of a master practitioner, it can be a precision instrument wielded with grace and power. Hand that sword to a gardener, and in the absence of a better tool or instruction, it will become a weed whacker. - billn On Thu, 16 Mar 2006, Scott Walters wrote: > Matt Wright wrote formmail.pl which, though it was done in bad style, is > famous for having the highest density of vulnerabilities per line in any > program known to man. It single handled made Perl the largest spam relay > right behend open Sendmail servers for a solid 7 years running. Matt > Wright writes bad code. My references to people who learned to code by > imitating him implied they had far more than style or preference wrong > with their code -- their code is bad. As I said, usually people who > write &func aren't old Perl 4 programmers but are people who learned from > the wretched code floating around -- the most common of which is Matt's. > > -scott > > On 0, "Anthony R. Nemmer" wrote: >> I don't buy the "Matt Wright" argument or whoever it is. I think people >> can write perfectly good code in Perl 5 and use the & when calling >> subroutines. You just need to be careful, per perlsub, in how you use >> &. It's been my experience that prototypes are truly a hack, and that >> they are not used that often. Again, this all may change with Perl 6, >> but when will Perl 6 be released? It's probably years away. Personally >> I think sigils are one of the things that makes Perl Perl, and that >> typeglobs are pretty cool. >> >> Tony >> >> -- >> >> I always have coffee when I watch radar! >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ Phoenix-pm mailing list Phoenix-pm at pm.org http://mail.pm.org/mailman/listinfo/phoenix-pm From awwaiid at thelackthereof.org Thu Mar 16 17:37:37 2006 From: awwaiid at thelackthereof.org (Brock) Date: Thu, 16 Mar 2006 18:37:37 -0700 Subject: [Phoenix-pm] meeting tonight! Message-ID: <20060317013737.GT4287@thelackthereof.org> In case anyone gets this, I forgot to mention that it is probably best to park on 3rd st, either in a metered parking (free after 6) or in the garage (free 2hr validation). The only trick is that the link between 3rd St. and Mill is closed, so you must approach the intersection from Ash St. You can see the road of which I speak on the map. Here's the meeting info, in case you forgot: Time: Thursday 16 March 2006 @7:00pm Location: Mill's End http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az 310 S Mill Ave # A101 Tempe, AZ 85281 Thats the North-West corner of Mill and 3rd (north of University) in down-town Tempe. Topic: Search Engine (scott) Catalyst (Michael Garfias, unconfirmed) As always, random discussion Other: Wireless internet available. Bring your laptops. Mill's End sells drinks and food Presentations will be given and recorded over VNC From jdawgaz at cox.net Thu Mar 16 20:59:07 2006 From: jdawgaz at cox.net (Jerry Davis) Date: Thu, 16 Mar 2006 21:59:07 -0700 Subject: [Phoenix-pm] forwarded job Message-ID: <20060316215907.1adb2763@aragorn.home.com> From: "Kent Hoppenworth" Date: Thu, 16 Mar 2006 14:29:21 -0700 We are looking for senior QA engineers to join our testing team. The position requires a self-motivated individual with excellent written and oral communication skills and experience in testing enterprise software. Candidates are expected to plan, design, and execute functional and system testing for large scale distributed systems. Create test automation test suite and infrastructure as necessary. This position will work directly with senior development engineers to identify areas to focus testing activities, report and track defects and project schedules, as well as providing leadership and direction to junior members of the QA organization. Requirements: BS or MS in Computer Science or equivalent. 5+ years of enterprise testing experience. Experience in testing operating systems and distributed environments. Strong background with Linux/Unix is required. Experience in planning, designing, and performing functional and system level tests. Lead complex project from test planning to production launch. Ability to set priorities, accommodate requirement changes and resource changes. Experience in programming with a least one of the languages: C/C++, Java. Created functional and/or regression automation suite in Python, Perl and/or Java. Excellent written and oral communication skills. Must be a great team player and enjoy working in a fast pace environment. Knowledge of performance testing a plus. Kent Hoppenworth IT Staffing 602-977-7240-Direct -- Hobbit Name: Pimpernel Loamsdown Registered Linux User: 275424 This email's random fortune: Murray's Rule: Any country with "democratic" in the title isn't. From intertwingled at qwest.net Thu Mar 16 21:35:24 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Thu, 16 Mar 2006 22:35:24 -0700 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <20060317013737.GT4287@thelackthereof.org> References: <20060317013737.GT4287@thelackthereof.org> Message-ID: <441A4A9C.2050502@qwest.net> Blah that's always the problem with downtown Tempe is the freaking parking! Brock wrote: > In case anyone gets this, I forgot to mention that it is probably best > to park on 3rd st, either in a metered parking (free after 6) or in the > garage (free 2hr validation). > > The only trick is that the link between 3rd St. and Mill is closed, so > you must approach the intersection from Ash St. You can see the road of > which I speak on the map. > > Here's the meeting info, in case you forgot: > > Time: Thursday 16 March 2006 @7:00pm > Location: Mill's End > http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az > 310 S Mill Ave # A101 > Tempe, AZ 85281 > Thats the North-West corner of Mill and 3rd (north of > University) in down-town Tempe. > Topic: Search Engine (scott) > Catalyst (Michael Garfias, unconfirmed) > As always, random discussion > Other: Wireless internet available. Bring your laptops. > Mill's End sells drinks and food > Presentations will be given and recorded over VNC > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! From intertwingled at qwest.net Thu Mar 16 21:39:00 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Thu, 16 Mar 2006 22:39:00 -0700 Subject: [Phoenix-pm] I don't buy it In-Reply-To: References: <4419B4A9.3000902@qwest.net> <20060316202218.GK22790@illogics.org> Message-ID: <441A4B74.7000202@qwest.net> Actually, the wheed whacker is a pretty cool invention. Bill Nash wrote: > Code is code is code. Like a katana in the hands of a master practitioner, > it can be a precision instrument wielded with grace and power. Hand that > sword to a gardener, and in the absence of a better tool or instruction, > it will become a weed whacker. > > - billn > > On Thu, 16 Mar 2006, Scott Walters wrote: > >> Matt Wright wrote formmail.pl which, though it was done in bad style, is >> famous for having the highest density of vulnerabilities per line in any >> program known to man. It single handled made Perl the largest spam relay >> right behend open Sendmail servers for a solid 7 years running. Matt >> Wright writes bad code. My references to people who learned to code by >> imitating him implied they had far more than style or preference wrong >> with their code -- their code is bad. As I said, usually people who >> write &func aren't old Perl 4 programmers but are people who learned from >> the wretched code floating around -- the most common of which is Matt's. >> >> -scott >> >> On 0, "Anthony R. Nemmer" wrote: >>> I don't buy the "Matt Wright" argument or whoever it is. I think people >>> can write perfectly good code in Perl 5 and use the & when calling >>> subroutines. You just need to be careful, per perlsub, in how you use >>> &. It's been my experience that prototypes are truly a hack, and that >>> they are not used that often. Again, this all may change with Perl 6, >>> but when will Perl 6 be released? It's probably years away. Personally >>> I think sigils are one of the things that makes Perl Perl, and that >>> typeglobs are pretty cool. >>> >>> Tony >>> >>> -- >>> >>> I always have coffee when I watch radar! >>> _______________________________________________ >>> Phoenix-pm mailing list >>> Phoenix-pm at pm.org >>> http://mail.pm.org/mailman/listinfo/phoenix-pm >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm >> > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm > > -- I always have coffee when I watch radar! From scott at illogics.org Thu Mar 16 22:35:33 2006 From: scott at illogics.org (Scott Walters) Date: Fri, 17 Mar 2006 06:35:33 +0000 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <20060317013737.GT4287@thelackthereof.org> References: <20060317013737.GT4287@thelackthereof.org> Message-ID: <20060317063533.GZ22790@illogics.org> Hi, To everyone who made it out, good to see you. Thanks for coming. Here's the handout, attached. Now, who else has an interesting algorithm, trick, application of Perl, module, technique, language, or project they'd be willing to talk about? Regards, -scott On 0, Brock wrote: > In case anyone gets this, I forgot to mention that it is probably best > to park on 3rd st, either in a metered parking (free after 6) or in the > garage (free 2hr validation). > > The only trick is that the link between 3rd St. and Mill is closed, so > you must approach the intersection from Ash St. You can see the road of > which I speak on the map. > > Here's the meeting info, in case you forgot: > > Time: Thursday 16 March 2006 @7:00pm > Location: Mill's End > http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az > 310 S Mill Ave # A101 > Tempe, AZ 85281 > Thats the North-West corner of Mill and 3rd (north of > University) in down-town Tempe. > Topic: Search Engine (scott) > Catalyst (Michael Garfias, unconfirmed) > As always, random discussion > Other: Wireless internet available. Bring your laptops. > Mill's End sells drinks and food > Presentations will be given and recorded over VNC > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm -------------- next part -------------- =for comment * Patricia trees translate trees into positions in stored documents and document names (file names, URLs, whatever) * I hope you can read code better than diagrams =cut use strict; use warnings; my $tree = { }; sub search { my $string = shift; my $node = $tree; for my $chr (split //, $string) { exists $node->{$chr} or return undef; # not found condition $node = $node->{$chr}; } return $node->{hits}; } =for comment * The data structure is a tree, where each node has edges corresponding to letters and other characters, and all nodes except the root node also have a list of "hit" documents and positions in those documents * This implementation lacks a feature of searching up multiple words, but that's easy enough to add * The decision of the implementer is how deep to make the tree -- to support searching for 40 character phrases, the tree must go 40 characters deep =cut sub index_document { my $doc = shift; my $doc_name = shift; my @chars = split //, $doc; my $position = 0; while(1) { return if @chars == 6; my $node = $tree; my $pos2 = $position; for my $chr ( @chars[0 .. (@chars > 6 ? 6 : scalar @chars)] ) { $node->{$chr} ||= { hits => [], }; $node = $node->{$chr}; push @{ $node->{hits} }, [ $doc_name, $position, ]; $pos2++; } } continue { shift @chars; $position++; } } =for comment * The index isn't based on merely words or substrings, so entire phrases can be searched for * Google is able to quickly track down excerpts for each hit word in each document because its pattrees point to the exact byte in the exact document for each word * Google is able to find ranges, such as when you Google for 1000..2000, by following the pattree so far and then grabbing all of the nodes under that point * Postgres defaults to using binary tables for lookups -- this is an adequate simulation of a pattree if you ignore the overhead of storing repeated string fragments =cut sub pretty_results { my @hits = @{ shift() }; my $buf = shift; for my $hit (@hits) { print $hit->[0], ' ', $hit->[1], ' ', substr($buf, $hit->[1], 20), "\n"; } } do { # the pudding open my $fh, '<', __FILE__; read $fh, my $buf, -s $fh; index_document($buf, __FILE__); my $hits = search('Google'); pretty_results($hits, $buf); }; From bwmetz at att.com Thu Mar 16 23:09:22 2006 From: bwmetz at att.com (Metz, Bobby W, WCS) Date: Fri, 17 Mar 2006 01:09:22 -0600 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <20060317063533.GZ22790@illogics.org> Message-ID: <01D5341D04A2E64AB9B3457690473367017822AC@OCCLUST01EVS1.ugd.att.com> I'll throw my hat in the ring officially for a 10 minute "Reading/Writing ZIP Files on CPAN Challenged Unix Servers". Hate I ended up missing tonight...wanted to hear about the search engine stuff, but as happens too frequently I get tied up at work and can't cut loose in time, so I'm really a lurker I guess ( one Nello's meeting doesn't quite count ). Anyway, 10 minutes should be good. Like all "good" Perl, the code is too short to spend more time than that on it and really doesn't need "presenting" per se, so I'll probably just hand out a two page printout for discussion and forward code later...you know after everyone signs a non-disclosure...more jokes on that later :) That said, it would be interesting to hear others thoughts on the routine and hear of your experiences with it's CPAN counterparts such as Compess::Zlib, IO::Gzip & IO::Gunzip. Now, I just have to make sure and make it Scott friendly [ jk Scott ;-) ] by removing all the & calls in the example program. Unfortunately, I don't think I can remove the typeglobs and it still work since that's part of the magic. But, I'm open to suggestions from Brock, Scott, Anthony or whoever else can think of a better way to do it and retain it's symplicity. As a former colleague of mine was found of saying, it's a "ghetto script". Get the job done quick and don't worry about perfection. So, anyone interested? Bobby -----Original Message----- From: phoenix-pm-bounces+bwmetz=att.com at pm.org [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Scott Walters Sent: Thursday, March 16, 2006 11:36 PM To: Brock Cc: phoenix-pm at pm.org Subject: Re: [Phoenix-pm] meeting tonight! Hi, To everyone who made it out, good to see you. Thanks for coming. Here's the handout, attached. Now, who else has an interesting algorithm, trick, application of Perl, module, technique, language, or project they'd be willing to talk about? Regards, -scott On 0, Brock wrote: > In case anyone gets this, I forgot to mention that it is probably best > to park on 3rd st, either in a metered parking (free after 6) or in the > garage (free 2hr validation). > > The only trick is that the link between 3rd St. and Mill is closed, so > you must approach the intersection from Ash St. You can see the road of > which I speak on the map. > > Here's the meeting info, in case you forgot: > > Time: Thursday 16 March 2006 @7:00pm > Location: Mill's End > http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az > 310 S Mill Ave # A101 > Tempe, AZ 85281 > Thats the North-West corner of Mill and 3rd (north of > University) in down-town Tempe. > Topic: Search Engine (scott) > Catalyst (Michael Garfias, unconfirmed) > As always, random discussion > Other: Wireless internet available. Bring your laptops. > Mill's End sells drinks and food > Presentations will be given and recorded over VNC > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm From friedman at highwire.stanford.edu Fri Mar 17 06:47:19 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Fri, 17 Mar 2006 06:47:19 -0800 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <01D5341D04A2E64AB9B3457690473367017822AC@OCCLUST01EVS1.ugd.att.com> References: <01D5341D04A2E64AB9B3457690473367017822AC@OCCLUST01EVS1.ugd.att.com> Message-ID: Sounds cool, Bobby. I wish I could make that meeting. FWIW, my office tried to use Compress::Zlib and had too many problems with it to continue. We put in a hack to do what you're probably doing, call gzip & compress as a system call, and, well, "It works, so don't fix it!" :-) Our error handling is less than robust, though, so I'm looking forward to your code example. -- Mike On Mar 16, 2006, at 11:09 PM, Metz, Bobby W, WCS wrote: > I'll throw my hat in the ring officially for a 10 minute > "Reading/Writing ZIP Files on CPAN Challenged Unix Servers". Hate I > ended up missing tonight...wanted to hear about the search engine > stuff, > but as happens too frequently I get tied up at work and can't cut > loose > in time, so I'm really a lurker I guess ( one Nello's meeting doesn't > quite count ). > Anyway, 10 minutes should be good. Like all "good" Perl, the > code is too short to spend more time than that on it and really > doesn't > need "presenting" per se, so I'll probably just hand out a two page > printout for discussion and forward code later...you know after > everyone > signs a non-disclosure...more jokes on that later :) > That said, it would be interesting to hear others thoughts on > the routine and hear of your experiences with it's CPAN counterparts > such as Compess::Zlib, IO::Gzip & IO::Gunzip. Now, I just have to > make > sure and make it Scott friendly [ jk Scott ;-) ] by removing all the & > calls in the example program. Unfortunately, I don't think I can > remove > the typeglobs and it still work since that's part of the magic. But, > I'm open to suggestions from Brock, Scott, Anthony or whoever else can > think of a better way to do it and retain it's symplicity. As a > former > colleague of mine was found of saying, it's a "ghetto script". Get > the > job done quick and don't worry about perfection. > > So, anyone interested? > > Bobby > > -----Original Message----- > From: phoenix-pm-bounces+bwmetz=att.com at pm.org > [mailto:phoenix-pm-bounces+bwmetz=att.com at pm.org]On Behalf Of Scott > Walters > Sent: Thursday, March 16, 2006 11:36 PM > To: Brock > Cc: phoenix-pm at pm.org > Subject: Re: [Phoenix-pm] meeting tonight! > > > Hi, > > To everyone who made it out, good to see you. Thanks for coming. > > Here's the handout, attached. > > Now, who else has an interesting algorithm, trick, application of > Perl, > module, technique, language, or project they'd be willing to talk > about? > > Regards, > -scott > > On 0, Brock wrote: >> In case anyone gets this, I forgot to mention that it is probably >> best >> to park on 3rd st, either in a metered parking (free after 6) or in > the >> garage (free 2hr validation). >> >> The only trick is that the link between 3rd St. and Mill is >> closed, so >> you must approach the intersection from Ash St. You can see the road > of >> which I speak on the map. >> >> Here's the meeting info, in case you forgot: >> >> Time: Thursday 16 March 2006 @7:00pm >> Location: Mill's End >> http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az >> 310 S Mill Ave # A101 >> Tempe, AZ 85281 >> Thats the North-West corner of Mill and 3rd (north of >> University) in down-town Tempe. >> Topic: Search Engine (scott) >> Catalyst (Michael Garfias, unconfirmed) >> As always, random discussion >> Other: Wireless internet available. Bring your laptops. >> Mill's End sells drinks and food >> Presentations will be given and recorded over VNC >> _______________________________________________ >> Phoenix-pm mailing list >> Phoenix-pm at pm.org >> http://mail.pm.org/mailman/listinfo/phoenix-pm > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From scott at illogics.org Fri Mar 17 19:18:23 2006 From: scott at illogics.org (Scott Walters) Date: Sat, 18 Mar 2006 03:18:23 +0000 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <01D5341D04A2E64AB9B3457690473367017822AC@OCCLUST01EVS1.ugd.att.com> References: <20060317063533.GZ22790@illogics.org> <01D5341D04A2E64AB9B3457690473367017822AC@OCCLUST01EVS1.ugd.att.com> Message-ID: <20060318031823.GI22790@illogics.org> I can't wait to see how long this subject line lives. Anyway, http://maradydd.livejournal.com/293666.html is actually pretty funny and informed, unlike most attempts at language stereotype comedy. Cheers, -scott From cbeber at darkscape.net Fri Mar 17 20:45:15 2006 From: cbeber at darkscape.net (Christopher Beber) Date: Fri, 17 Mar 2006 21:45:15 -0700 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <20060317063533.GZ22790@illogics.org> References: <20060317013737.GT4287@thelackthereof.org> <20060317063533.GZ22790@illogics.org> Message-ID: <441B905B.4050707@darkscape.net> I'm available to do a Catalyst presentation, if people are interested and I can make it to the meeting. I also found a tutorial for the perl SDL libraries on line. The the tutorial covers building a game, but the SDL libraries could be used for any type of application that requires fancy graphics, responsive user input or sound. Again, if I don't have a schedule conflict and there is sufficient interest, I could do a small presentation about that. http://arstechnica.com/guides/tweaks/games-perl.ars Scott Walters wrote: >Hi, > >To everyone who made it out, good to see you. Thanks for coming. > >Here's the handout, attached. > >Now, who else has an interesting algorithm, trick, application of Perl, >module, technique, language, or project they'd be willing to talk about? > >Regards, >-scott > >On 0, Brock wrote: > > >>In case anyone gets this, I forgot to mention that it is probably best >>to park on 3rd st, either in a metered parking (free after 6) or in the >>garage (free 2hr validation). >> >>The only trick is that the link between 3rd St. and Mill is closed, so >>you must approach the intersection from Ash St. You can see the road of >>which I speak on the map. >> >>Here's the meeting info, in case you forgot: >> >> Time: Thursday 16 March 2006 @7:00pm >> Location: Mill's End >> http://maps.google.com/maps?q=310+S+Mill+Ave+tempe,+az >> 310 S Mill Ave # A101 >> Tempe, AZ 85281 >> Thats the North-West corner of Mill and 3rd (north of >> University) in down-town Tempe. >> Topic: Search Engine (scott) >> Catalyst (Michael Garfias, unconfirmed) >> As always, random discussion >> Other: Wireless internet available. Bring your laptops. >> Mill's End sells drinks and food >> Presentations will be given and recorded over VNC >>_______________________________________________ >>Phoenix-pm mailing list >>Phoenix-pm at pm.org >>http://mail.pm.org/mailman/listinfo/phoenix-pm >> >> >>------------------------------------------------------------------------ >> >> >>=for comment >> >>* Patricia trees translate trees into positions in stored documents and >> document names (file names, URLs, whatever) >>* I hope you can read code better than diagrams >> >>=cut >> >>use strict; use warnings; >> >>my $tree = { }; >> >>sub search { >> my $string = shift; >> my $node = $tree; >> for my $chr (split //, $string) { >> exists $node->{$chr} or return undef; # not found condition >> $node = $node->{$chr}; >> } >> return $node->{hits}; >>} >> >>=for comment >> >>* The data structure is a tree, where each node has edges corresponding to >> letters and other characters, and all nodes except the root node also have >> a list of "hit" documents and positions in those documents >> >>* This implementation lacks a feature of searching up multiple words, >> but that's easy enough to add >> >>* The decision of the implementer is how deep to make the tree -- to support >> searching for 40 character phrases, the tree must go 40 characters deep >> >>=cut >> >>sub index_document { >> my $doc = shift; >> my $doc_name = shift; >> my @chars = split //, $doc; >> my $position = 0; >> while(1) { >> return if @chars == 6; >> my $node = $tree; >> my $pos2 = $position; >> for my $chr ( @chars[0 .. (@chars > 6 ? 6 : scalar @chars)] ) { >> $node->{$chr} ||= { hits => [], }; >> $node = $node->{$chr}; >> push @{ $node->{hits} }, [ $doc_name, $position, ]; >> $pos2++; >> } >> } continue { >> shift @chars; >> $position++; >> } >> >>} >> >>=for comment >> >>* The index isn't based on merely words or substrings, so entire phrases >> can be searched for >> >>* Google is able to quickly track down excerpts for each hit word in each >> document because its pattrees point to the exact byte in the exact document >> for each word >> >>* Google is able to find ranges, such as when you Google for 1000..2000, >> by following the pattree so far and then grabbing all of the nodes >> under that point >> >>* Postgres defaults to using binary tables for lookups -- >> this is an adequate simulation of a pattree if you ignore the overhead >> of storing repeated string fragments >> >>=cut >> >>sub pretty_results { >> my @hits = @{ shift() }; >> my $buf = shift; >> for my $hit (@hits) { >> print $hit->[0], ' ', $hit->[1], ' ', substr($buf, $hit->[1], 20), "\n"; >> } >>} >> >>do { >> # the pudding >> open my $fh, '<', __FILE__; >> read $fh, my $buf, -s $fh; >> index_document($buf, __FILE__); >> my $hits = search('Google'); >> pretty_results($hits, $buf); >>}; >> >> >>------------------------------------------------------------------------ >> >>_______________________________________________ >>Phoenix-pm mailing list >>Phoenix-pm at pm.org >>http://mail.pm.org/mailman/listinfo/phoenix-pm >> >>------------------------------------------------------------------------ >> >>No virus found in this incoming message. >>Checked by AVG Free Edition. >>Version: 7.1.375 / Virus Database: 268.2.2/280 - Release Date: 3/13/2006 >> >> From awwaiid at thelackthereof.org Fri Mar 17 23:25:17 2006 From: awwaiid at thelackthereof.org (Brock) Date: Sat, 18 Mar 2006 00:25:17 -0700 Subject: [Phoenix-pm] meeting tonight! In-Reply-To: <20060318031823.GI22790@illogics.org> References: <20060317063533.GZ22790@illogics.org> <01D5341D04A2E64AB9B3457690473367017822AC@OCCLUST01EVS1.ugd.att.com> <20060318031823.GI22790@illogics.org> Message-ID: <20060318072517.GW4287@thelackthereof.org> Meredith, the author of that journal, presented at the two CodeCon's I attended (2005 and 2006). In 2005 she worked for a company who has a nice protein search database -- you enter some fragments of DNA and it will find all the proteins that match it. She did a lot of the programming for it, but even better was her presentation -- she extracted and separated DNA out of a culture sample using common household chemicals right there on the stage! In February 2006 she presented again, this time talking about what I believe was a Google Summer-of-Code project. She wrote a patch for the postgresql database which allows Query-By-Example in a very efficient way (using a Support Vector Machine directly in the query engine). This effectively lets you say "give me all the rows which are similar to row 5,6, and 7, but not like row 10,15, or 30. Meaning you don't have to specify which columns are significant or what is the significance of those columns. * http://www.codecon.org/2006/program.html#qbe * http://en.wikipedia.org/wiki/Support_vector_machine Geeks all around me were muttering about how they wanted to marry her, but then right after her presentation Len (one of the founders of CodeCon) proposed to her on the stage and she accepted. So there you go. --Brock On 2006.03.18.03.18, Scott Walters wrote: | I can't wait to see how long this subject line lives. | | Anyway, http://maradydd.livejournal.com/293666.html is actually pretty | funny and informed, unlike most attempts at language stereotype comedy. | | Cheers, | -scott From intertwingled at qwest.net Sun Mar 19 23:16:40 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Mon, 20 Mar 2006 00:16:40 -0700 Subject: [Phoenix-pm] no Tempe/East Valley Perlmongers meeting this month Message-ID: <441E56D8.2020009@qwest.net> Due to possible inclement weather (rain Tuesday night), there will be no Tempe/East Valley Perlmongers meeting this month. Thanks, Tony -- I always have coffee when I watch radar! From intertwingled at qwest.net Sun Mar 19 23:57:59 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Mon, 20 Mar 2006 00:57:59 -0700 Subject: [Phoenix-pm] [Fwd: no Tempe/East Valley Perlmongers meeting this month] Message-ID: <441E6087.4010809@qwest.net> However, we do have an efnet irc channel: #tempe.pm. I will be on there Tuesday night at 7 PM. Feel free to join the channel and hang out. Maybe we can have a virtual meeting. There are several topics I'd like to discuss concerning the Tempe / East Valley Perlmongers group. -------- Original Message -------- Subject: no Tempe/East Valley Perlmongers meeting this month Date: Mon, 20 Mar 2006 00:16:40 -0700 From: Anthony R. Nemmer Reply-To: intertwingled at qwest.net Organization: everything is deeply intertwingled To: phoenix-pm at pm.org Due to possible inclement weather (rain Tuesday night), there will be no Tempe/East Valley Perlmongers meeting this month. Thanks, Tony -- I always have coffee when I watch radar! -- I always have coffee when I watch radar! From intertwingled at qwest.net Mon Mar 20 00:21:38 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Mon, 20 Mar 2006 01:21:38 -0700 Subject: [Phoenix-pm] indigoperl Message-ID: <441E6612.3030805@qwest.net> Was wondering if anyone here has ever heard of indigoperl (http://www.indigostar.com/indigoperl.htm), if the played around with it or actually use it, what they think of it, etc. Tony -- I always have coffee when I watch radar! From friedman at highwire.stanford.edu Mon Mar 20 12:43:32 2006 From: friedman at highwire.stanford.edu (Michael Friedman) Date: Mon, 20 Mar 2006 12:43:32 -0800 Subject: [Phoenix-pm] HighWire Press is hiring Message-ID: The only problem, of course, is that you have to move to the San Francisco Bay Area. My boss won't let new employees work remotely for the first few years. :-( Still, if you know anyone interested in a Perl & Java position for Stanford, please let me know. It's a solid software development position, writing and maintaining unix commandline and web CGI/ mod_jserv programs in both Perl and Java (and SQL for Sybase). http://jobs.stanford.edu/openings/display.cgi? Job_Req=010047&JFam=NIL&JOBCODE=5094 -- Mike --------------------------------------------------------------------- Michael Friedman HighWire Press Phone: 650-725-1974 Stanford University FAX: 270-721-8034 --------------------------------------------------------------------- From perlguy at earthlink.net Mon Mar 20 16:27:56 2006 From: perlguy at earthlink.net (Douglas E. Miles) Date: Tue, 21 Mar 2006 00:27:56 +0000 Subject: [Phoenix-pm] HighWire Press is hiring In-Reply-To: References: Message-ID: <441F488C.9010605@earthlink.net> Michael Friedman wrote: >The only problem, of course, is that you have to move to the San >Francisco Bay Area. My boss won't let new employees work remotely for >the first few years. :-( > > > Curses! Foiled again! Don't want to move, but thinking about looking for a new job. >Still, if you know anyone interested in a Perl & Java position for >Stanford, please let me know. It's a solid software development >position, writing and maintaining unix commandline and web CGI/ >mod_jserv programs in both Perl and Java (and SQL for Sybase). > >http://jobs.stanford.edu/openings/display.cgi? >Job_Req=010047&JFam=NIL&JOBCODE=5094 > >-- Mike >--------------------------------------------------------------------- >Michael Friedman HighWire Press >Phone: 650-725-1974 Stanford University >FAX: 270-721-8034 >--------------------------------------------------------------------- > > >_______________________________________________ >Phoenix-pm mailing list >Phoenix-pm at pm.org >http://mail.pm.org/mailman/listinfo/phoenix-pm > > From intertwingled at qwest.net Tue Mar 21 07:50:08 2006 From: intertwingled at qwest.net (Anthony R. Nemmer) Date: Tue, 21 Mar 2006 08:50:08 -0700 Subject: [Phoenix-pm] No Tempe / East Valley Perlmongers Meeting Tonight Message-ID: <442020B0.2010501@qwest.net> There will not be a Tempe / East Valley Perlmongers meeting tonight due to possible inclement weather (rain). Thanks, Tony -- I always have coffee when I watch radar! From scott at illogics.org Tue Mar 21 08:13:28 2006 From: scott at illogics.org (Scott Walters) Date: Tue, 21 Mar 2006 16:13:28 +0000 Subject: [Phoenix-pm] [Fwd: no Tempe/East Valley Perlmongers meeting this month] In-Reply-To: <441E6087.4010809@qwest.net> References: <441E6087.4010809@qwest.net> Message-ID: <20060321161328.GN22790@illogics.org> Funny. Everyone with a PM group seems to be trying to start an IRC channel, and the various Perl IRC channels are trying to get their IRC channels recognized as a PM group. Is this a case of the grass is greener on the other side of the digital divide or what? -scott On 0, "Anthony R. Nemmer" wrote: > However, we do have an efnet irc channel: #tempe.pm. I will be on > there Tuesday night at 7 PM. Feel free to join the channel and hang > out. Maybe we can have a virtual meeting. There are several topics I'd > like to discuss concerning the Tempe / East Valley Perlmongers group. > > -------- Original Message -------- > Subject: no Tempe/East Valley Perlmongers meeting this month > Date: Mon, 20 Mar 2006 00:16:40 -0700 > From: Anthony R. Nemmer > Reply-To: intertwingled at qwest.net > Organization: everything is deeply intertwingled > To: phoenix-pm at pm.org > > Due to possible inclement weather (rain Tuesday night), there will be no > Tempe/East Valley Perlmongers meeting this month. > > Thanks, > Tony > > -- > > I always have coffee when I watch radar! > > > -- > > I always have coffee when I watch radar! > _______________________________________________ > Phoenix-pm mailing list > Phoenix-pm at pm.org > http://mail.pm.org/mailman/listinfo/phoenix-pm