From dbii at mudpuddle.com Thu Feb 12 09:10:14 2004 From: dbii at mudpuddle.com (David Bluestein II) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Perl Meeting Next Wed, what topic? Message-ID: Hey, do we have a topic for next Wednesday? If not, can we drum up one by then? I know we had a couple of potential topics, but don't remember who was planned to present. David ---------- David H. Bluestein II President & Lead Developer dbii@interaction.net ii, inc. http://www.interaction.net - Specializing in Designing Interactive Websites - - and Searchable Internet Databases - From randysleek at hotmail.com Thu Feb 12 10:08:51 2004 From: randysleek at hotmail.com (Randall Hennig) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Perl Compiler For Windows Message-ID: I am in need of compiling my Perl code into a Windows executable. Does anybody know of a free Perl compiler for Windows? This would help me greatly. Randy _________________________________________________________________ Find great local high-speed Internet access value at the MSN High-Speed Marketplace. http://click.atdmt.com/AVE/go/onm00200360ave/direct/01/ From fluhmann at vvm.com Thu Feb 12 10:57:17 2004 From: fluhmann at vvm.com (fluhmann@vvm.com) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Perl Compiler For Windows In-Reply-To: References: Message-ID: <4440.165.91.218.174.1076605037.squirrel@www.vvm.com> You might try this: http://search.cpan.org/~autrijus/PAR-0.79/script/pp It's supposed to complile into a Windows executable. Hope that helps! Jeremy Fluhmann Microcomputer Specialist Texas A&M Univeristy -------- Original Message -------- Subject: APM: Perl Compiler For Windows From: "Randall Hennig" Date: Thu, February 12, 2004 10:08 am To: austin@mail.pm.org > I am in need of compiling my Perl code into a Windows executable. Does > anybody know of a free Perl compiler for Windows? From randysleek at hotmail.com Thu Feb 12 20:02:28 2004 From: randysleek at hotmail.com (Randall Hennig) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Perl Compiler For Windows Message-ID: Thanks, that worked. >From: >To: >Subject: Re: APM: Perl Compiler For Windows >Date: Thu, 12 Feb 2004 10:57:17 -0600 (CST) > >You might try this: http://search.cpan.org/~autrijus/PAR-0.79/script/pp > >It's supposed to complile into a Windows executable. > >Hope that helps! >Jeremy Fluhmann >Microcomputer Specialist >Texas A&M Univeristy > > >-------- Original Message -------- >Subject: APM: Perl Compiler For Windows >From: "Randall Hennig" >Date: Thu, February 12, 2004 10:08 am >To: austin@mail.pm.org > > > I am in need of compiling my Perl code into a Windows executable. Does > > anybody know of a free Perl compiler for Windows? > > > >_______________________________________________ >Austin mailing list >Austin@mail.pm.org >http://mail.pm.org/mailman/listinfo/austin _________________________________________________________________ Choose now from 4 levels of MSN Hotmail Extra Storage - no more account overload! http://click.atdmt.com/AVE/go/onm00200362ave/direct/01/ From dbii at mudpuddle.com Mon Feb 16 11:19:02 2004 From: dbii at mudpuddle.com (David Bluestein II) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Wednesday presenter? Message-ID: Hey, do we know who is going to present on Wednesday? Is the person who was going to do Perl 6 going to come? Or is there someone else scheduled? I may can show my website mapping program if I have it running by then, but I won't know until later today if it is. David ---------- David H. Bluestein II President & Lead Developer dbii@interaction.net ii, inc. http://www.interaction.net - Specializing in Designing Interactive Websites - - and Searchable Internet Databases - From wwalker at bybent.com Mon Feb 16 16:23:05 2004 From: wwalker at bybent.com (Wayne Walker) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Wednesday Night's Perl Mongers Meeting Message-ID: <20040216222305.GA9733@bybent.com> Wednesday night at 7:00 PM, the Austin Perl Mongers will meet at the offices of Servergraph (http://austin.pm.org/location.htm). Gabriel Foust will be giving a talk titled "Encapsulating DBI with Perl Objects". At 5:30 PM many people will get together for a quick dinner before the meeting at the PokEJo's on 5th St., one block from the Servergraph offices. It used to start at 6:00, but that's been a little too short. I will do my best to be there at 5:30. I'll be the guy with pony tail, glasses and a tan or blue hat. My cell phone is 791-2046 if anyone gets lost, shows up a little late (the outer building doors might be locked), or needs other info about the meeting. -- Wayne Walker wwalker@bybent.com Do you use Linux?! http://www.bybent.com Get Counted! http://counter.li.org/ Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent From michalk at awpi.com Tue Feb 17 10:56:35 2004 From: michalk at awpi.com (Brian Michalk) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: <20040216222305.GA9733@bybent.com> Message-ID: I am in a quandry about how to do efficient filehandle reading. I'm trying to make it uniform across all of the filehandles that may be named pipes, device driver handles, network sockets, or stdio. I have some slow devices on a serial line, and other fast devices that continally generate data at high rates. My protocol is all line oriented, and that naturally leads me to use something like <>, but read the following: perldoc -f select WARNING: One should not attempt to mix buffered I/O (like "read" or ) with "select", except as permitted by POSIX, and even then only on POSIX systems. You have to use "sysread" instead. However sysread doesn't care about line separators. Instead, I have to search through the incoming data for separators and store partial reads in my own buffer. This is not a problem, I have code, and it works. The performance is bad. C code would have the same type of problem. The serial port dribbles in GPS data at 9600 baud, causing the select() to return without a complete line being available, so I store all of the one or two characters at a time in the internal buffer. The radar data, however can come in at 100hertz, at about 12K of data per line, and I've got buffering turned on for performance, so I have to go searching through the data to find the line separators. Are there any better solutions? From vaughan99 at yahoo.com Tue Feb 17 12:10:55 2004 From: vaughan99 at yahoo.com (Chris Vaughan) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: Message-ID: <20040217181055.41743.qmail@web11506.mail.yahoo.com> Brian, If you have the flexibility, you may want to consider changing your protocol away from separators and towards packets. If you don't have the flexibility, then don't read on. Have the sender send packed data length (in bytes), then the data itself, in a loop. The reader would simply block waiting for the first 4 bytes, construct a count by unpacking the integer, and block reading that count of the handle, forming your message. After the reader reads the message, it blocks again waiting for the next count. The downside to this solution is that the reader has two logical reading states. If the reader gets out of sync for any reason, you're screwed. Regards, Chris --- Brian Michalk wrote: > I am in a quandry about how to do efficient filehandle > reading. > I'm trying to make it uniform across all of the filehandles > that may be > named pipes, device driver handles, network sockets, or stdio. > > I have some slow devices on a serial line, and other fast > devices that > continally generate data at high rates. My protocol is all > line oriented, > and that naturally leads me to use something like <>, but read > the > following: > perldoc -f select > WARNING: One should not attempt to mix buffered > I/O (like "read" > or ) with "select", except as permitted by > POSIX, and even > then only on POSIX systems. You have to use > "sysread" instead. > > However sysread doesn't care about line separators. Instead, > I have to > search through the incoming data for separators and store > partial reads in > my own buffer. This is not a problem, I have code, and it > works. The > performance is bad. C code would have the same type of > problem. > > The serial port dribbles in GPS data at 9600 baud, causing the > select() to > return without a complete line being available, so I store all > of the one or > two characters at a time in the internal buffer. The radar > data, however > can come in at 100hertz, at about 12K of data per line, and > I've got > buffering turned on for performance, so I have to go searching > through the > data to find the line separators. > > Are there any better solutions? > > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin ===== ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Vaughan | "I love deadlines. I like the | swooshing sound as they fly by." vaughan99@yahoo.com | - Douglas Adams __________________________________ Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online. http://taxes.yahoo.com/filing.html From wwalker at bybent.com Tue Feb 17 15:16:42 2004 From: wwalker at bybent.com (Wayne Walker) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: <20040217181055.41743.qmail@web11506.mail.yahoo.com> References: <20040217181055.41743.qmail@web11506.mail.yahoo.com> Message-ID: <20040217211642.GA16200@bybent.com> On Tue, Feb 17, 2004 at 10:10:55AM -0800, Chris Vaughan wrote: > Brian, > > If you have the flexibility, you may want to consider changing > your protocol away from separators and towards packets. If you > don't have the flexibility, then don't read on. > > Have the sender send packed data length (in bytes), then the > data itself, in a loop. The reader would simply block waiting > for the first 4 bytes, construct a count by unpacking the > integer, and block reading that count of the handle, forming > your message. After the reader reads the message, it blocks > again waiting for the next count. > > The downside to this solution is that the reader has two logical > reading states. If the reader gets out of sync for any reason, > you're screwed. This is a pretty good solution. The reader/writer should never get out of sync, but I've just been through a nightmare on this (that I created myself). Just remember that send() and recv() are NOT guaranteed to send/get the number of bytes you told it to! > > Regards, > Chris > > --- Brian Michalk wrote: > > I am in a quandry about how to do efficient filehandle > > reading. > > I'm trying to make it uniform across all of the filehandles > > that may be > > named pipes, device driver handles, network sockets, or stdio. > > > > I have some slow devices on a serial line, and other fast > > devices that > > continally generate data at high rates. My protocol is all > > line oriented, > > and that naturally leads me to use something like <>, but read > > the > > following: > > perldoc -f select > > WARNING: One should not attempt to mix buffered > > I/O (like "read" > > or ) with "select", except as permitted by > > POSIX, and even > > then only on POSIX systems. You have to use > > "sysread" instead. > > > > However sysread doesn't care about line separators. Instead, > > I have to > > search through the incoming data for separators and store > > partial reads in > > my own buffer. This is not a problem, I have code, and it > > works. The > > performance is bad. C code would have the same type of > > problem. > > > > The serial port dribbles in GPS data at 9600 baud, causing the > > select() to > > return without a complete line being available, so I store all > > of the one or > > two characters at a time in the internal buffer. The radar > > data, however > > can come in at 100hertz, at about 12K of data per line, and > > I've got > > buffering turned on for performance, so I have to go searching > > through the > > data to find the line separators. > > > > Are there any better solutions? > > > > _______________________________________________ > > Austin mailing list > > Austin@mail.pm.org > > http://mail.pm.org/mailman/listinfo/austin > > > ===== > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > Chris Vaughan | "I love deadlines. I like the > | swooshing sound as they fly by." > vaughan99@yahoo.com | - Douglas Adams > > __________________________________ > Do you Yahoo!? > Yahoo! Finance: Get your refund fast by filing online. > http://taxes.yahoo.com/filing.html > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin -- Wayne Walker wwalker@bybent.com Do you use Linux?! http://www.bybent.com Get Counted! http://counter.li.org/ Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent From vaughan99 at yahoo.com Tue Feb 17 17:26:16 2004 From: vaughan99 at yahoo.com (Chris Vaughan) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: <20040217211642.GA16200@bybent.com> Message-ID: <20040217232616.28719.qmail@web11510.mail.yahoo.com> > Just remember that send() and recv() are NOT > guaranteed to > send/get the number of bytes you told it to! Right, I didn't think of that. The writer would need to loop until it sends all the bytes it needs to. That much is obvious. The reader would have to copy everything it receives into a buffer, due to recv() returning more or less than expected. If recv() returned less than expected, call recv() again for the remainder (obviously). Your count and data would then be retrieved from the buffer, removing from the head of the buffer. Memory copies should still be faster than per-byte inspection for separator characters. I think this algorithm still has potential for you. Regards, Chris --- Wayne Walker wrote: > On Tue, Feb 17, 2004 at 10:10:55AM -0800, Chris Vaughan wrote: > > Brian, > > > > If you have the flexibility, you may want to consider > changing > > your protocol away from separators and towards packets. If > you > > don't have the flexibility, then don't read on. > > > > Have the sender send packed data length (in bytes), then the > > data itself, in a loop. The reader would simply block > waiting > > for the first 4 bytes, construct a count by unpacking the > > integer, and block reading that count of the handle, forming > > your message. After the reader reads the message, it blocks > > again waiting for the next count. > > > > The downside to this solution is that the reader has two > logical > > reading states. If the reader gets out of sync for any > reason, > > you're screwed. > > This is a pretty good solution. The reader/writer should > never get out > of sync, but I've just been through a nightmare on this (that > I created > myself). Just remember that send() and recv() are NOT > guaranteed to > send/get the number of bytes you told it to! > > > > > Regards, > > Chris > > > > --- Brian Michalk wrote: > > > I am in a quandry about how to do efficient filehandle > > > reading. > > > I'm trying to make it uniform across all of the > filehandles > > > that may be > > > named pipes, device driver handles, network sockets, or > stdio. > > > > > > I have some slow devices on a serial line, and other fast > > > devices that > > > continally generate data at high rates. My protocol is > all > > > line oriented, > > > and that naturally leads me to use something like <>, but > read > > > the > > > following: > > > perldoc -f select > > > WARNING: One should not attempt to mix > buffered > > > I/O (like "read" > > > or ) with "select", except as permitted by > > > POSIX, and even > > > then only on POSIX systems. You have to use > > > "sysread" instead. > > > > > > However sysread doesn't care about line separators. > Instead, > > > I have to > > > search through the incoming data for separators and store > > > partial reads in > > > my own buffer. This is not a problem, I have code, and it > > > works. The > > > performance is bad. C code would have the same type of > > > problem. > > > > > > The serial port dribbles in GPS data at 9600 baud, causing > the > > > select() to > > > return without a complete line being available, so I store > all > > > of the one or > > > two characters at a time in the internal buffer. The > radar > > > data, however > > > can come in at 100hertz, at about 12K of data per line, > and > > > I've got > > > buffering turned on for performance, so I have to go > searching > > > through the > > > data to find the line separators. > > > > > > Are there any better solutions? > > > > > > _______________________________________________ > > > Austin mailing list > > > Austin@mail.pm.org > > > http://mail.pm.org/mailman/listinfo/austin > > > > > > ===== > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Chris Vaughan | "I love deadlines. I like the > > | swooshing sound as they fly by." > > vaughan99@yahoo.com | - Douglas Adams > > > > __________________________________ > > Do you Yahoo!? > > Yahoo! Finance: Get your refund fast by filing online. > > http://taxes.yahoo.com/filing.html > > _______________________________________________ > > Austin mailing list > > Austin@mail.pm.org > > http://mail.pm.org/mailman/listinfo/austin > > -- > > Wayne Walker > wwalker@bybent.com Do you use Linux?! > http://www.bybent.com Get Counted! > http://counter.li.org/ > Perl - http://www.perl.org/ Perl User Groups - > http://www.pm.org/ > Jabber IM: wwalker@jabber.phototropia.org AIM: > lwwalkerbybent > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin ===== ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Chris Vaughan | "I love deadlines. I like the | swooshing sound as they fly by." vaughan99@yahoo.com | - Douglas Adams __________________________________ Do you Yahoo!? Yahoo! Finance: Get your refund fast by filing online. http://taxes.yahoo.com/filing.html From dbii at mudpuddle.com Wed Feb 18 08:51:22 2004 From: dbii at mudpuddle.com (David Bluestein II) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Command line arguments Message-ID: I know about Getopts to get command line options, but it has escaped me how to get a command line argument. What I want is to: ./foo.pl bar.txt And have foo.pl get the name bar.txt as a variable (in this case the filename to act upon). Where is bar.txt stored? David ---------- David H. Bluestein II President & Lead Developer dbii@interaction.net ii, inc. http://www.interaction.net - Specializing in Designing Interactive Websites - - and Searchable Internet Databases - From jeremy.brooks at univista.com Wed Feb 18 09:09:38 2004 From: jeremy.brooks at univista.com (Jeremy Brooks) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Command line arguments In-Reply-To: References: Message-ID: <1077117294.6716.21.camel@oberon> All command line args are stored in @ARGV in the order you supply them. Thus, bar.txt is the first element of @ARGV, $ARGV[0]. fyi: Getopts is a parser of @ARGV On Wed, 2004-02-18 at 08:51, David Bluestein II wrote: > > ---------- > > From: dbii@mudpuddle.com[SMTP:DBII@MUDPUDDLE.COM] > > Sent: Wednesday, February 18, 2004 8:51:22 AM > > To: Austin@mail.pm.org > > Subject: APM: Command line arguments > > Auto forwarded by a Rule > > > I know about Getopts to get command line options, but it has escaped me how > to get a command line argument. What I want is to: > > ./foo.pl bar.txt > > And have foo.pl get the name bar.txt as a variable (in this case the > filename to act upon). Where is bar.txt stored? > > David > > ---------- > David H. Bluestein II President & Lead Developer > dbii@interaction.net ii, inc. > > http://www.interaction.net > - Specializing in Designing Interactive Websites - > - and Searchable Internet Databases - > > > > > > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin From michalk at awpi.com Thu Feb 19 09:03:49 2004 From: michalk at awpi.com (Brian Michalk) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: <20040217211642.GA16200@bybent.com> Message-ID: Thanks to all for their input on this matter. The packet length solution would certainly be efficient, but I'm not willing to set policy on something like that that I might regret in the future. Take for instance the GPS data. Unless I write a shim, I can't very well have it send me the format we discussed. There are many other devices, like gyroscopes, and a scanning laser that have serial input. I've come up with some different scenarios. 1) Write kernel modules that do not emit data until a complete line is ready to send, and go with blocking reads. If I carefully mix select() and readline(), this will work. I just have to be careful with executing readlines. 2) Rewrite my base class to fork, or spawn threads, and go with blocking reads. This would involve some complexity with communicating back to the parent when a message or data is received. It might be pretty, or ugly. I'm still looking into this. If there were a select() equivalent, except for signal, or messages ready, then this wouldn't be so bad. My last experience with signals in my perl modules led to system lockups, but from what I'm reading, 5.8 has solved a lot of these issues. This would all be so much easier if someone had a select() function in the kernel API that would return on some line separator, or even "x" characters of data ready. Now, select always returns true if there is *any* data available. I understand the consequences though. It would require a lot more overhead to scan all of the input for line terminators. Even so, it would certainly be more efficient than any readline() command, because the minute you do line oriented IO, every packet of data is scanned for a terminator. Thoughts, feelings, comments? > -----Original Message----- > From: austin-bounces@mail.pm.org [mailto:austin-bounces@mail.pm.org]On > Behalf Of Wayne Walker > Sent: Tuesday, February 17, 2004 3:17 PM > To: Chris Vaughan > Cc: Austin@mail.pm.org > Subject: Re: APM: File reading optimization > > > On Tue, Feb 17, 2004 at 10:10:55AM -0800, Chris Vaughan wrote: > > Brian, > > > > If you have the flexibility, you may want to consider changing > > your protocol away from separators and towards packets. If you > > don't have the flexibility, then don't read on. > > > > Have the sender send packed data length (in bytes), then the > > data itself, in a loop. The reader would simply block waiting > > for the first 4 bytes, construct a count by unpacking the > > integer, and block reading that count of the handle, forming > > your message. After the reader reads the message, it blocks > > again waiting for the next count. > > > > The downside to this solution is that the reader has two logical > > reading states. If the reader gets out of sync for any reason, > > you're screwed. > > This is a pretty good solution. The reader/writer should never get out > of sync, but I've just been through a nightmare on this (that I created > myself). Just remember that send() and recv() are NOT guaranteed to > send/get the number of bytes you told it to! > > > > > Regards, > > Chris > > > > --- Brian Michalk wrote: > > > I am in a quandry about how to do efficient filehandle > > > reading. > > > I'm trying to make it uniform across all of the filehandles > > > that may be > > > named pipes, device driver handles, network sockets, or stdio. > > > > > > I have some slow devices on a serial line, and other fast > > > devices that > > > continally generate data at high rates. My protocol is all > > > line oriented, > > > and that naturally leads me to use something like <>, but read > > > the > > > following: > > > perldoc -f select > > > WARNING: One should not attempt to mix buffered > > > I/O (like "read" > > > or ) with "select", except as permitted by > > > POSIX, and even > > > then only on POSIX systems. You have to use > > > "sysread" instead. > > > > > > However sysread doesn't care about line separators. Instead, > > > I have to > > > search through the incoming data for separators and store > > > partial reads in > > > my own buffer. This is not a problem, I have code, and it > > > works. The > > > performance is bad. C code would have the same type of > > > problem. > > > > > > The serial port dribbles in GPS data at 9600 baud, causing the > > > select() to > > > return without a complete line being available, so I store all > > > of the one or > > > two characters at a time in the internal buffer. The radar > > > data, however > > > can come in at 100hertz, at about 12K of data per line, and > > > I've got > > > buffering turned on for performance, so I have to go searching > > > through the > > > data to find the line separators. > > > > > > Are there any better solutions? > > > > > > _______________________________________________ > > > Austin mailing list > > > Austin@mail.pm.org > > > http://mail.pm.org/mailman/listinfo/austin > > > > > > ===== > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > Chris Vaughan | "I love deadlines. I like the > > | swooshing sound as they fly by." > > vaughan99@yahoo.com | - Douglas Adams > > > > __________________________________ > > Do you Yahoo!? > > Yahoo! Finance: Get your refund fast by filing online. > > http://taxes.yahoo.com/filing.html > > _______________________________________________ > > Austin mailing list > > Austin@mail.pm.org > > http://mail.pm.org/mailman/listinfo/austin > > -- > > Wayne Walker > wwalker@bybent.com Do you use Linux?! > http://www.bybent.com Get Counted! http://counter.li.org/ > Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ > Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin > From ian at remmler.org Thu Feb 19 09:43:11 2004 From: ian at remmler.org (Ian Remmler) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: References: <20040217211642.GA16200@bybent.com> Message-ID: <20040219154310.GA3701@remmler.org> On Thu, Feb 19, 2004 at 09:03:49AM -0600, Brian Michalk wrote: > Thoughts, feelings, comments? I am rather ignorant of the topic being discussed, but that won't stop me from piping in, so... Feel free to laugh at me when I get it all wrong. Would it be possible to write some C-based helper code, using XS or Inline::C, to handle the reading? - Ian. -- Ian Remmler | A monk asked Joshu, "Has a dog Buddha ian@remmler.org | nature or not?" Joshu replied, "Mu!" http://remmler.org | -- Mumon, "The Gateless Gate" From mike at stok.co.uk Thu Feb 19 10:54:06 2004 From: mike at stok.co.uk (Mike Stok) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: References: Message-ID: On Thu, 19 Feb 2004, Brian Michalk wrote: > Thanks to all for their input on this matter. > > The packet length solution would certainly be efficient, but I'm not willing > to set policy on something like that that I might regret in the future. > Take for instance the GPS data. Unless I write a shim, I can't very well > have it send me the format we discussed. There are many other devices, like > gyroscopes, and a scanning laser that have serial input. > > I've come up with some different scenarios. > 1) Write kernel modules that do not emit data until a complete line is ready > to send, and go with blocking reads. If I carefully mix select() and > readline(), this will work. I just have to be careful with executing > readlines. > 2) Rewrite my base class to fork, or spawn threads, and go with blocking > reads. This would involve some complexity with communicating back to the > parent when a message or data is received. It might be pretty, or ugly. > I'm still looking into this. If there were a select() equivalent, except > for signal, or messages ready, then this wouldn't be so bad. My last > experience with signals in my perl modules led to system lockups, but from > what I'm reading, 5.8 has solved a lot of these issues. > > This would all be so much easier if someone had a select() function in the > kernel API that would return on some line separator, or even "x" characters > of data ready. Now, select always returns true if there is *any* data > available. I understand the consequences though. It would require a lot > more overhead to scan all of the input for line terminators. Even so, it > would certainly be more efficient than any readline() command, because the > minute you do line oriented IO, every packet of data is scanned for a > terminator. > > Thoughts, feelings, comments? Have you considered the Expect module. It may not do what you want, but ages ago I used chat2.pl to control an industrial oven through an ethernet -> serial box and then a serial line. This had to interact with a system where you got new-line-less prompts and it worked OK. The module did all of the accumulation of characters and pattern searching for me. Just a thought. Mike -- mike@stok.co.uk | The "`Stok' disclaimers" apply. http://www.stok.co.uk/~mike/ | GPG PGP Key 1024D/059913DA mike@exegenix.com | Fingerprint 0570 71CD 6790 7C28 3D60 http://www.exegenix.com/ | 75D2 9EC4 C1C0 0599 13DA From michalk at awpi.com Thu Feb 19 13:47:22 2004 From: michalk at awpi.com (Brian Michalk) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: Message-ID: Thanks. That's getting a little closer, but not there. Could anyone tell me what this does: http://search.cpan.org/~scotts/IO-Select-Trap-0.032/Trap.pm > -----Original Message----- > From: austin-bounces@mail.pm.org [mailto:austin-bounces@mail.pm.org]On > Behalf Of Mike Stok > Sent: Thursday, February 19, 2004 10:54 AM > To: Brian Michalk > Cc: Austin@mail.pm.org; Wayne Walker; Chris Vaughan > Subject: RE: APM: File reading optimization > > > On Thu, 19 Feb 2004, Brian Michalk wrote: > > > Thanks to all for their input on this matter. > > > > The packet length solution would certainly be efficient, but > I'm not willing > > to set policy on something like that that I might regret in the future. > > Take for instance the GPS data. Unless I write a shim, I can't > very well > > have it send me the format we discussed. There are many other > devices, like > > gyroscopes, and a scanning laser that have serial input. > > > > I've come up with some different scenarios. > > 1) Write kernel modules that do not emit data until a complete > line is ready > > to send, and go with blocking reads. If I carefully mix select() and > > readline(), this will work. I just have to be careful with executing > > readlines. > > 2) Rewrite my base class to fork, or spawn threads, and go with blocking > > reads. This would involve some complexity with communicating > back to the > > parent when a message or data is received. It might be pretty, or ugly. > > I'm still looking into this. If there were a select() > equivalent, except > > for signal, or messages ready, then this wouldn't be so bad. My last > > experience with signals in my perl modules led to system > lockups, but from > > what I'm reading, 5.8 has solved a lot of these issues. > > > > This would all be so much easier if someone had a select() > function in the > > kernel API that would return on some line separator, or even > "x" characters > > of data ready. Now, select always returns true if there is *any* data > > available. I understand the consequences though. It would > require a lot > > more overhead to scan all of the input for line terminators. > Even so, it > > would certainly be more efficient than any readline() command, > because the > > minute you do line oriented IO, every packet of data is scanned for a > > terminator. > > > > Thoughts, feelings, comments? > > Have you considered the Expect module. It may not do what you want, but > ages ago I used chat2.pl to control an industrial oven through an > ethernet > -> serial box and then a serial line. This had to interact with a system > where you got new-line-less prompts and it worked OK. The module did all > of the accumulation of characters and pattern searching for me. > > Just a thought. > > Mike > > -- > mike@stok.co.uk | The "`Stok' > disclaimers" apply. > http://www.stok.co.uk/~mike/ | GPG PGP Key 1024D/059913DA > mike@exegenix.com | Fingerprint 0570 71CD > 6790 7C28 3D60 > http://www.exegenix.com/ | 75D2 9EC4 > C1C0 0599 13DA > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin > From michalk at awpi.com Thu Feb 19 14:53:39 2004 From: michalk at awpi.com (Brian Michalk) Date: Mon Aug 2 21:23:23 2004 Subject: APM: File reading optimization In-Reply-To: <20040219193311.84715.qmail@web20405.mail.yahoo.com> Message-ID: Thanks. I think I'm leaning towards a variant of this. I have a big base class that implements all of the IO for network, devices, etc. Each type of node, like a GPS, or radar is subclassed, so there is really only one program. Sometimes two, if I need a fast C program to process data from the device driver, but in this case, the results come out on fifos at the other end. A simple node would be something like the GPS node. It listens to four UDP broadcast sockets, one for distance information, one for commands, and so forth. It opens a filehandle to /dev/ttyS0, which is the input from the GPS receiver. When clients connect to me, they can connect on a "command" port, or a "data" port. The command port is an interactive port where the client can set and get various information, and set runmodes, etc. The "data" port is mostly one way, where the client connects, asks for a version of data, and from then on out, it is simply a one way data stream of that data to the client. It looks like callbacks are what I'm looking for here. It seems I would change my existing select groups into callback subroutines. Crude examples will follow. This is coming off the top of my head and not from perl -cw. # assume an exagerrated simple case: # 1 UDP broadcast commands receive # 1 filehandle from device receive # 1 filehandle to device write # 1 filehandle to command socket clients read/write # 1 filehandle to data socket clients read/write # after blessings, etc. $self->{bc_commands_sub} = \&$self->process_broadcast_command(); # alternatively $self->can("process_bc_cmd"); Hmmm, I just looked at the RPC implementation in the Advanced Perl Programming book. It uses freeze/thaw. At first glance this looks like a data copy. It would work fine for slow devices, but really clog up on the fast ones. Perhaps something more like a signal handler. Keeping the same forking idea, or threads, use shared memory for my data structures and have a signal handler (we get 32 minus 3 user defined signals now with threaded perl) in the base class gather up the appropriate data from shared memory and then call the data processing function. Ony the reading file handles should fork/thread. Blocking writes are not a problem, and are preferred. Thread gets data, copies it to shared memory, then throws signal to parent process for handling. Mostly ramblings here on my part. Trying to flesh this thing out. > -----Original Message----- > From: Bill Raty [mailto:bill_raty@yahoo.com] > Sent: Thursday, February 19, 2004 1:33 PM > To: Brian Michalk > Subject: RE: APM: File reading optimization > > > Brian, > > This may amount to what you've already mentioned, but here goes > anyway: > > * Open up a RPC.pm interface (i.e. a perl sub) to your program > where you would normally place your event that is to occur when > a line has been read. Info on how to do this can be found in > the ORA "Advanced Perl Programming" book. > > * Start up perl programs that buffer your input waiting for LF, > then make the call to your event subroutine when the line has > been received. > > You're using IPC via RPC in this, and forking is going on. On > the other hand the pieces get simpler, and you've turned your > program into an event driven server (even though this may be > happening on the loopback). > > Startup of all this shouldn't be problematic: > > foreach my $perl_prog (qw(main_prog.pl gps_buffer.pl etc.pl)) { > system qw( $perl_prog & ); > } > > Crude, but the forking is done by the OS. Double forking in > perl to detach a process isn't tough either. It can be found > in the Perl Cookbook. > > -Bill > > --- Brian Michalk wrote: > > Thanks to all for their input on this matter. > > > > The packet length solution would certainly be efficient, but > > I'm not willing > > to set policy on something like that that I might regret in > > the future. > > Take for instance the GPS data. Unless I write a shim, I > > can't very well > > have it send me the format we discussed. There are many > > other devices, like > > gyroscopes, and a scanning laser that have serial input. > > > > I've come up with some different scenarios. > > 1) Write kernel modules that do not emit data until a > > complete line is ready > > to send, and go with blocking reads. If I carefully mix > > select() and > > readline(), this will work. I just have to be careful with > > executing > > readlines. > > 2) Rewrite my base class to fork, or spawn threads, and go > > with blocking > > reads. This would involve some complexity with communicating > > back to the > > parent when a message or data is received. It might be > > pretty, or ugly. > > I'm still looking into this. If there were a select() > > equivalent, except > > for signal, or messages ready, then this wouldn't be so bad. > > My last > > experience with signals in my perl modules led to system > > lockups, but from > > what I'm reading, 5.8 has solved a lot of these issues. > > > > This would all be so much easier if someone had a select() > > function in the > > kernel API that would return on some line separator, or even > > "x" characters > > of data ready. Now, select always returns true if there is > > *any* data > > available. I understand the consequences though. It would > > require a lot > > more overhead to scan all of the input for line terminators. > > Even so, it > > would certainly be more efficient than any readline() > > command, because the > > minute you do line oriented IO, every packet of data is > > scanned for a > > terminator. > > > > Thoughts, feelings, comments? > > > > > -----Original Message----- > > > From: austin-bounces@mail.pm.org > > [mailto:austin-bounces@mail.pm.org]On > > > Behalf Of Wayne Walker > > > Sent: Tuesday, February 17, 2004 3:17 PM > > > To: Chris Vaughan > > > Cc: Austin@mail.pm.org > > > Subject: Re: APM: File reading optimization > > > > > > > > > On Tue, Feb 17, 2004 at 10:10:55AM -0800, Chris Vaughan > > wrote: > > > > Brian, > > > > > > > > If you have the flexibility, you may want to consider > > changing > > > > your protocol away from separators and towards packets. > > If you > > > > don't have the flexibility, then don't read on. > > > > > > > > Have the sender send packed data length (in bytes), then > > the > > > > data itself, in a loop. The reader would simply block > > waiting > > > > for the first 4 bytes, construct a count by unpacking the > > > > integer, and block reading that count of the handle, > > forming > > > > your message. After the reader reads the message, it > > blocks > > > > again waiting for the next count. > > > > > > > > The downside to this solution is that the reader has two > > logical > > > > reading states. If the reader gets out of sync for any > > reason, > > > > you're screwed. > > > > > > This is a pretty good solution. The reader/writer should > > never get out > > > of sync, but I've just been through a nightmare on this > > (that I created > > > myself). Just remember that send() and recv() are NOT > > guaranteed to > > > send/get the number of bytes you told it to! > > > > > > > > > > > Regards, > > > > Chris > > > > > > > > --- Brian Michalk wrote: > > > > > I am in a quandry about how to do efficient filehandle > > > > > reading. > > > > > I'm trying to make it uniform across all of the > > filehandles > > > > > that may be > > > > > named pipes, device driver handles, network sockets, or > > stdio. > > > > > > > > > > I have some slow devices on a serial line, and other > > fast > > > > > devices that > > > > > continally generate data at high rates. My protocol is > > all > > > > > line oriented, > > > > > and that naturally leads me to use something like <>, > > but read > > > > > the > > > > > following: > > > > > perldoc -f select > > > > > WARNING: One should not attempt to mix > > buffered > > > > > I/O (like "read" > > > > > or ) with "select", except as permitted > > by > > > > > POSIX, and even > > > > > then only on POSIX systems. You have to use > > > > > "sysread" instead. > > > > > > > > > > However sysread doesn't care about line separators. > > Instead, > > > > > I have to > > > > > search through the incoming data for separators and > > store > > > > > partial reads in > > > > > my own buffer. This is not a problem, I have code, and > > it > > > > > works. The > > > > > performance is bad. C code would have the same type of > > > > > problem. > > > > > > > > > > The serial port dribbles in GPS data at 9600 baud, > > causing the > > > > > select() to > > > > > return without a complete line being available, so I > > store all > > > > > of the one or > > > > > two characters at a time in the internal buffer. The > > radar > > > > > data, however > > > > > can come in at 100hertz, at about 12K of data per line, > > and > > > > > I've got > > > > > buffering turned on for performance, so I have to go > > searching > > > > > through the > > > > > data to find the line separators. > > > > > > > > > > Are there any better solutions? > > > > > > > > > > _______________________________________________ > > > > > Austin mailing list > > > > > Austin@mail.pm.org > > > > > http://mail.pm.org/mailman/listinfo/austin > > > > > > > > > > > > ===== > > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > Chris Vaughan | "I love deadlines. I like the > > > > | swooshing sound as they fly by." > > > > vaughan99@yahoo.com | - Douglas Adams > > > > > > > > __________________________________ > > > > Do you Yahoo!? > > > > Yahoo! Finance: Get your refund fast by filing online. > > > > http://taxes.yahoo.com/filing.html > > > > _______________________________________________ > > > > Austin mailing list > > > > Austin@mail.pm.org > > > > http://mail.pm.org/mailman/listinfo/austin > > > > > > -- > > > > > > Wayne Walker > > > wwalker@bybent.com Do you use Linux?! > > > http://www.bybent.com Get Counted! > > http://counter.li.org/ > > > Perl - http://www.perl.org/ Perl User Groups - > > http://www.pm.org/ > > > Jabber IM: wwalker@jabber.phototropia.org AIM: > > lwwalkerbybent > > > _______________________________________________ > > > Austin mailing list > > > Austin@mail.pm.org > > > http://mail.pm.org/mailman/listinfo/austin > > > > > > > _______________________________________________ > > Austin mailing list > > Austin@mail.pm.org > > > === message truncated === > > > ===== > "There let the pealing organ blow, > To the full-voiced choir below, > In service high, and anthems clear, > As may with sweetness, through mine ear, > Dissolve me into ecstasies, > And bring all Heav'n before mine eyes". > John Milton - Il Penseroso (1632). > From wwalker at bybent.com Sun Feb 22 10:01:20 2004 From: wwalker at bybent.com (Wayne Walker) Date: Mon Aug 2 21:23:23 2004 Subject: APM: emacs perl mode questions Message-ID: <20040222160120.GB5459@bybent.com> OK, Mark should be rolling on the floor laughing about now. I need some help with emacs. I have two issues. 1. White is not a background color!!! Does anyone have some reasonable settings for a proper (i.e., black) background in emacs? 2. I have a lot of programs I edit where the filename has no extension and the #! line calls #!/usr/bin/sg-perl so emacs doesn't go into cperl mode. What can I do to fix this? In vim I can add # vim: ft=perl Is there something similar in emacs? -- Wayne Walker wwalker@bybent.com Do you use Linux?! http://www.bybent.com Get Counted! http://counter.li.org/ Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent From wwalker at bybent.com Sun Feb 22 10:52:54 2004 From: wwalker at bybent.com (Wayne Walker) Date: Mon Aug 2 21:23:23 2004 Subject: APM: emacs perl mode questions In-Reply-To: <20040222160120.GB5459@bybent.com> References: <20040222160120.GB5459@bybent.com> Message-ID: <20040222165254.GF5459@bybent.com> Bill Raty told me about emacs -r. Thanks! He also gave me an answer for #2 that made me realize I needed to restate my question: Is there a way to put emacs commands in a source file so that when emacs opens the file for edit, emacs will excute them (e.g. cperl-mode, electric mode on ...)?? vim allows commands to be included in the source in the first or last 10 lines of the file. My programs look like: #!/usr/bin/sg-perl # # vim: ft=perl # vim: tabstop=4 Is there a simlar feature in emacs? Thanks all. Wayne P.S. - Mark said it best: "Emacs is a great place to live, but you wouldn't want to visit." I'm trying to quit visiting and move in for a while, but .... :) > 1. White is not a background color!!! Does anyone have some reasonable > settings for a proper (i.e., black) background in emacs? > > 2. I have a lot of programs I edit where the filename has no extension > and the #! line calls #!/usr/bin/sg-perl so emacs doesn't go into cperl > mode. What can I do to fix this? In vim I can add > # vim: ft=perl > > Is there something similar in emacs? > > -- > > Wayne Walker > wwalker@bybent.com Do you use Linux?! > http://www.bybent.com Get Counted! http://counter.li.org/ > Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ > Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin -- Wayne Walker wwalker@bybent.com Do you use Linux?! http://www.bybent.com Get Counted! http://counter.li.org/ Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent From ian at remmler.org Sun Feb 22 13:27:31 2004 From: ian at remmler.org (Ian Remmler) Date: Mon Aug 2 21:23:23 2004 Subject: APM: emacs perl mode questions In-Reply-To: <20040222165254.GF5459@bybent.com> References: <20040222160120.GB5459@bybent.com> <20040222165254.GF5459@bybent.com> Message-ID: <20040222192731.GA22128@remmler.org> On Sun, Feb 22, 2004 at 10:52:54AM -0600, Wayne Walker wrote: > Bill Raty told me about emacs -r. Thanks! Or you can put: (set-foreground-color "white") (set-background-color "black") in you .emacs file. > Is there a way to put emacs commands in a source file so that when emacs > opens the file for edit, emacs will excute them (e.g. cperl-mode, > electric mode on ...)?? Don't know about executing arbitrary commands, but: # -*- cperl -*- will tell emacs to use cper-mode. I believe that only works on the first line, unless there is a #!... on the first line, in which case it looks on the second line. You can also set variables, e.g.: # -*- coding: latin-1; -*- I don't know what all you can set in that way. I moved over from the "dark side" a few years ago and switched to vim, so my previous statements may not be 100% factual. > "Emacs is a great place to live, but you wouldn't want to visit." That statement applies to many other cults as well... :) - Ian. -- Ian Remmler | A monk asked Joshu, "Has a dog Buddha ian@remmler.org | nature or not?" Joshu replied, "Mu!" http://remmler.org | -- Mumon, "The Gateless Gate" From wwalker at bybent.com Sun Feb 22 22:21:18 2004 From: wwalker at bybent.com (Wayne Walker) Date: Mon Aug 2 21:23:23 2004 Subject: APM: emacs perl mode questions In-Reply-To: <20040222192627.GA20779@remmler.org> References: <20040222160120.GB5459@bybent.com> <20040222165254.GF5459@bybent.com> <20040222192627.GA20779@remmler.org> Message-ID: <20040223042117.GB2394@bybent.com> Thanks! That's what I was looking for!!! On Sun, Feb 22, 2004 at 01:26:27PM -0600, Ian Remmler wrote: > On Sun, Feb 22, 2004 at 10:52:54AM -0600, Wayne Walker wrote: > > Bill Raty told me about emacs -r. Thanks! > > Or you can put: > > (set-foreground-color "white") > (set-background-color "black") > > in you .emacs file. > > > Is there a way to put emacs commands in a source file so that when emacs > > opens the file for edit, emacs will excute them (e.g. cperl-mode, > > electric mode on ...)?? > > Don't know about executing arbitrary commands, but: > > # -*- cperl -*- > > will tell emacs to use cper-mode. I believe that only works on > the first line, unless there is a #!... on the first line, in > which case it looks on the second line. You can also set > variables, e.g.: > > # -*- coding: latin-1; -*- > > I don't know what all you can set in that way. I moved over from > the "dark side" a few years ago and switched to vim, so my > previous statements may not be 100% factual. > > > "Emacs is a great place to live, but you wouldn't want to visit." > > That statement applies to many other cults as well... > > :) > > - Ian. > > -- > Ian Remmler | A monk asked Joshu, "Has a dog Buddha > ian@remmler.org | nature or not?" Joshu replied, "Mu!" > http://remmler.org | -- Mumon, "The Gateless Gate" -- Wayne Walker wwalker@bybent.com Do you use Linux?! http://www.bybent.com Get Counted! http://counter.li.org/ Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent From jeremy.brooks at univista.com Mon Feb 23 10:53:42 2004 From: jeremy.brooks at univista.com (Jeremy Brooks) Date: Mon Aug 2 21:23:23 2004 Subject: APM: DBI multi table delete Message-ID: <1077555552.9172.127.camel@oberon> I need to quickly come up with a sql statement to run from a perl/DBI script that deletes all records from several tables that have a particular username, 'joe', in the username column. I know this is probably simple, but what I've done so far hasn't worked as expected. I think that these are the correct DELETE and FROM clauses: DELETE t1,t2,t3,t4,t5 FROM t1,t2,t3,t4,t5 But, I'm not sure how to write the condition that determines what records to delete. let's say I want to delete any record from all of the tables that have a username value of 'joe'. Can some please show me exactly what the sql statement ought to be? thank in advance! -Jeremy From eharris at puremagic.com Mon Feb 23 11:12:43 2004 From: eharris at puremagic.com (Evan Harris) Date: Mon Aug 2 21:23:23 2004 Subject: APM: DBI multi table delete In-Reply-To: <1077555552.9172.127.camel@oberon> Message-ID: Your problem is probably that you need to fully specify the table fields. Try: DELETE FROM t1,t2,t3,t4,t5 WHERE t1.username = 'joe' OR t2.usernam = 'joe' OR t3.username = 'joe' OR t4.username = 'joe' OR t5.username = 'joe' I've never done multitable delets, so I don't know if that even works, but if it does, the above should too, and should do what you want. A sure way to do it is to do a seperate delete statement for each table. Evan On 23 Feb 2004, Jeremy Brooks wrote: > > I need to quickly come up with a sql statement to run from a perl/DBI > script that deletes all records from several tables that have a > particular username, 'joe', in the username column. I know this is > probably simple, but what I've done so far hasn't worked as expected. I > think that these are the correct DELETE and FROM clauses: > > DELETE t1,t2,t3,t4,t5 FROM t1,t2,t3,t4,t5 > > But, I'm not sure how to write the condition that determines what > records to delete. let's say I want to delete any record from all of > the tables that have a username value of 'joe'. > > Can some please show me exactly what the sql statement ought to be? > > thank in advance! > -Jeremy > > > > _______________________________________________ > Austin mailing list > Austin@mail.pm.org > http://mail.pm.org/mailman/listinfo/austin > From jeremy.brooks at univista.com Mon Feb 23 11:30:39 2004 From: jeremy.brooks at univista.com (Jeremy Brooks) Date: Mon Aug 2 21:23:23 2004 Subject: APM: DBI multi table delete In-Reply-To: References: Message-ID: <1077557774.9204.143.camel@oberon> I tried that and all the records in all the tables got deleted. I also tried using OR instead of AND.. same thing. Now I'm trying: WHERE t1.username="joe" AND t1.username=t2.username AND t1.username=t3.username AND t1.username=t4.username AND t1.username=t5.username and it seems to work... Could there be a problem with the database? This is Mysql 4.0.13 On Mon, 2004-02-23 at 11:12, Evan Harris wrote: > > ---------- > > From: Evan Harris[SMTP:EHARRIS@PUREMAGIC.COM] > > Sent: Monday, February 23, 2004 11:12:43 AM > > To: Jeremy Brooks > > Cc: apm > > Subject: Re: APM: DBI multi table delete > > Auto forwarded by a Rule > > > > > Your problem is probably that you need to fully specify the table fields. > Try: > > DELETE FROM t1,t2,t3,t4,t5 WHERE t1.username = 'joe' OR t2.usernam = 'joe' > OR t3.username = 'joe' OR t4.username = 'joe' OR t5.username = 'joe' > > I've never done multitable delets, so I don't know if that even works, but > if it does, the above should too, and should do what you want. > > A sure way to do it is to do a seperate delete statement for each table. > > Evan > > > On 23 Feb 2004, Jeremy Brooks wrote: > > > > > I need to quickly come up with a sql statement to run from a perl/DBI > > script that deletes all records from several tables that have a > > particular username, 'joe', in the username column. I know this is > > probably simple, but what I've done so far hasn't worked as expected. I > > think that these are the correct DELETE and FROM clauses: > > > > DELETE t1,t2,t3,t4,t5 FROM t1,t2,t3,t4,t5 > > > > But, I'm not sure how to write the condition that determines what > > records to delete. let's say I want to delete any record from all of > > the tables that have a username value of 'joe'. > > > > Can some please show me exactly what the sql statement ought to be? > > > > thank in advance! > > -Jeremy > > > > > > > > _______________________________________________ > > Austin mailing list > > Austin@mail.pm.org > > http://mail.pm.org/mailman/listinfo/austin > > From jeremy.brooks at univista.com Mon Feb 23 11:41:09 2004 From: jeremy.brooks at univista.com (Jeremy Brooks) Date: Mon Aug 2 21:23:23 2004 Subject: APM: DBI multi table delete (typo) In-Reply-To: <1077557774.9204.143.camel@oberon> References: <1077557774.9204.143.camel@oberon> Message-ID: <1077558404.9204.147.camel@oberon> typo... I meant to say: WHERE t1.username="joe" AND t2.username=t1.username AND t3.username=t1.username AND t4.username=t1.username AND t5.username=t1.username On Mon, 2004-02-23 at 11:36, Jeremy Brooks wrote: > I tried that and all the records in all the tables got deleted. > I also tried using OR instead of AND.. same thing. > Now I'm trying: > > WHERE t1.username="joe" > AND t1.username=t2.username > AND t1.username=t3.username > AND t1.username=t4.username > AND t1.username=t5.username > > and it seems to work... > > > Could there be a problem with the database? This is Mysql 4.0.13 > > > > > On Mon, 2004-02-23 at 11:12, Evan Harris wrote: > > > ---------- > > > From: Evan Harris[SMTP:EHARRIS@PUREMAGIC.COM] > > > Sent: Monday, February 23, 2004 11:12:43 AM > > > To: Jeremy Brooks > > > Cc: apm > > > Subject: Re: APM: DBI multi table delete > > > Auto forwarded by a Rule > > > > > > > > Your problem is probably that you need to fully specify the table fields. > > Try: > > > > DELETE FROM t1,t2,t3,t4,t5 WHERE t1.username = 'joe' OR t2.usernam = 'joe' > > OR t3.username = 'joe' OR t4.username = 'joe' OR t5.username = 'joe' > > > > I've never done multitable delets, so I don't know if that even works, but > > if it does, the above should too, and should do what you want. > > > > A sure way to do it is to do a seperate delete statement for each table. > > > > Evan > > > > > > On 23 Feb 2004, Jeremy Brooks wrote: > > > > > > > > I need to quickly come up with a sql statement to run from a perl/DBI > > > script that deletes all records from several tables that have a > > > particular username, 'joe', in the username column. I know this is > > > probably simple, but what I've done so far hasn't worked as expected. I > > > think that these are the correct DELETE and FROM clauses: > > > > > > DELETE t1,t2,t3,t4,t5 FROM t1,t2,t3,t4,t5 > > > > > > But, I'm not sure how to write the condition that determines what > > > records to delete. let's say I want to delete any record from all of > > > the tables that have a username value of 'joe'. > > > > > > Can some please show me exactly what the sql statement ought to be? > > > > > > thank in advance! > > > -Jeremy > > > > > > > > > > > > _______________________________________________ > > > Austin mailing list > > > Austin@mail.pm.org > > > http://mail.pm.org/mailman/listinfo/austin > > > From rainking at feeding.frenzy.com Mon Feb 23 14:39:28 2004 From: rainking at feeding.frenzy.com (Dennis Moore) Date: Mon Aug 2 21:23:23 2004 Subject: APM: DBI multi table delete In-Reply-To: <1077555552.9172.127.camel@oberon> References: <1077555552.9172.127.camel@oberon> Message-ID: <20040223203928.GA87936@feeding.frenzy.com> On Mon, Feb 23, 2004 at 10:59:12AM -0600, Jeremy Brooks wrote: > > I need to quickly come up with a sql statement to run from a perl/DBI > script that deletes all records from several tables that have a > particular username, 'joe', in the username column. I know this is > probably simple, but what I've done so far hasn't worked as expected. I > think that these are the correct DELETE and FROM clauses: > > DELETE t1,t2,t3,t4,t5 FROM t1,t2,t3,t4,t5 > > But, I'm not sure how to write the condition that determines what > records to delete. let's say I want to delete any record from all of > the tables that have a username value of 'joe'. > > Can some please show me exactly what the sql statement ought to be? http://www.mysql.com/documentation/mysql/bychapter/manual_SQL_Syntax.html#DELETE -- ;for (74,1970500640,1634627444,1751478816,1348825708,543711587, 1801810465){for($x=1<<1^1;$x>=1>>1;$x--) {$q=hex ff,$r=oct($x=~s,\d,$&* 10,e,$x),$x/=1/.1,$q<<=$r,$s.=chr (($_&$q)>>$r),$t++}}while($= ||= !$|) {$o=$o?$?:$/;$|=1;print $o?$s:$"x$t if$;;print"\b"x$t;sleep 1} From wwalker at bybent.com Tue Feb 24 23:36:49 2004 From: wwalker at bybent.com (Wayne Walker) Date: Mon Aug 2 21:23:23 2004 Subject: APM: [marsee@oreilly.com: PC Hardware Annoyances Needed for New Book] Message-ID: <20040225053649.GC3851@bybent.com> Thought someone might be interested... ----- Forwarded message from Marsee Henon ----- From: Marsee Henon To: wwalker@pobox.com Subject: PC Hardware Annoyances Needed for New Book Date: Tue, 24 Feb 2004 18:06:16 -0800 Dear User Group Leader: Thanks for the great response to our call, a few weeks ago, for annoyances, gripes, and complaints about Excel. The email we got was extremely useful and a lot of your members not only sent annoyances, but fixes! So, a thousand thanks for the help. As you might guess, we have another book in the wings--this one focusing on PC hardware annoyances. We're not just talking about PCs and laptops per se--we also talking about all the hardware that's inside and attached to your computer, such as memory, motherboards, hard drives, printers, scanners, home networks, DSL/cable, CD/DVD, and host of other annoying hardware devices. If any members of your group have PC hardware annoyances they'd like to see solved, have them email me (marsee@oreilly.com) with "PC Hardware Annoyances" in the subject. Just have them note what hardware is giving them grief (e.g. Dell Dimension 8100 with 1.3GHz P4; LaserJet 3150; Verbatim Producer 44 DVD+/-RW; etc.), and any relevant software that's involved (such as the OS, a driver, OCR software, etc.). As thanks for sharing, we'll make sure to get copies of "PC Hardware Annoyances" sent to your group shortly after publication. Thanks, Marsee *** An example: Hardware Windows Setting Blocks DVD Upgrade THE ANNOYANCE: I want to update the firmware for my Sony DRX510UL DVD burner, and the site said to disable the DMA setting in Windows XP before doing so. But it neglects to tell you how. THE FIX: The Sony drive is terrificit burns DVD+R and DVD-R discs, and uses both DVD-RW and DVD+RW rewritable media. But ask Sony for support, and it responds with a virtual raspberryits online instructions are complex and often impossible to understand. Luckily, fiddling with DMA isn't difficult. Here's how to turn it off: Windows XP/2000. Open the System control panel, choose the Hardware tab, and click the Device Manager button. Double-click "IDE ATA/ATAPI controllers" and double-click "Secondary IDE Channel" (your DVD drive is most likely located on the secondary channel; if not, choose "Primary IDE Channel"). Click the Advanced Settings tab, and under Device 0 (master) or Device 1 (slave) (depending on how your drive is set up), select PIO Only from the Transfer Mode drop-down menu. Click OK. Windows 98/Me. Open the System control panel and choose the Device Manager tab. Double-click CD-ROM, then double-click your drive. Select the Settings tab, uncheck the DMA option, and click OK. Remember to reverse the previous steps once your DVD drive's firmware is installed. *** ----- End forwarded message ----- -- Wayne Walker wwalker@bybent.com Do you use Linux?! http://www.bybent.com Get Counted! http://counter.li.org/ Perl - http://www.perl.org/ Perl User Groups - http://www.pm.org/ Jabber IM: wwalker@jabber.phototropia.org AIM: lwwalkerbybent From michalk at awpi.com Wed Feb 25 09:53:57 2004 From: michalk at awpi.com (Brian Michalk) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Some progress on efficient filehandle reading, but threads too big In-Reply-To: <20040225053649.GC3851@bybent.com> Message-ID: I managed to convert my code to threaded reads with very little change of code, and was able to delete entire subroutines dedicated to insuring the integrity using the older select method. My problem now is that ithreads are HOGS. For those of you that do not know, the built in threads in perl 5.8.x are called ithreads, which means interpreter threads. The threading mechanism is not COW(copy on write), so that means for every thread spawned everything is copied to the new thread and execution continues from there. I'm talking about interpreter, (un)shared variables, subroutines, everything. Extremely heavy on the memory. Forking a process on the other hand uses the operating systems COW mechanism, and Linux does much better with forked code. The interpreter will remain shared between all forked processes, so load as many modules as you want without the X hit on memory allocation. Now, there supposedly is a way to get lighter threads. I've looked into thread pools, but have not implemented that yet. I've only looked at 'top', to see what all these threads were doing, and it seems like my threads are way too heavy. On my 64MB machines, I can reliably spawn 8 threads or so before the program terminates. No disk swap available as there is no read/write hard disk. Looking at CPAN, Elizabeth Mattijsen has done a lot with threads. I see from her examples with Benchmark::Thread::Size the following output: # (ref) bare full vars our unique 0 2172 +0 +0 +0 +0 +0 1 2624 ? 4 +4 ? 4 +4 ? 4 +27 +4 ? 4 +27 2 3004 ? 4 +2 ? 6 +2 ? 6 +33 ? 4 +8 +36 ? 6 5 4126 ? 6 -2 ? 6 -3 ? 8 +29 ? 4 +10 ? 2 +27 ? 4 10 5984 ? 8 -1 ? 8 +0 ? 4 +0 ? 6 +17 ? 4 +43 ? 6 20 9694 ? 4 +15 ? 4 +15 ? 2 +13 ? 6 +32 ? 6 +58 ? 6 50 20832 ? 4 +51 ?10 +50 ? 8 +50 ? 8 +68 ?12 +96 ? 6 100 39392 ? 8 +106 ?10 +156 ?12 +108 ?10 +131 ?10 +155 ?12 ==== bare ======================================================== $VERSION = '0.01'; ==== full ======================================================== $main::VERSION = '0.01'; ==== vars ======================================================== use vars qw($VERSION); $VERSION = '0.01'; ==== our ========================================================= our $VERSION = '0.01'; ==== unique ====================================================== our $VERSION : unique = '0.01'; ================================================================== I'm not sure how VERSION plays into memory management, so this confuses me, but it shows that 20 threads are consuming less than 10MB total. This would be great prformance for me. From my 'top', I'm getting the following: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3815 root 16 0 28172 27m 1844 S 5.8 47.3 0:17.07 filter_dmi.pl 3817 root 8 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl 3818 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.01 filter_dmi.pl 3819 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.07 filter_dmi.pl 3821 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl 3822 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.07 filter_dmi.pl 3823 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl 3824 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl 3825 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.08 filter_dmi.pl In case you are wondering about the pids, perl -v has this little snippet: ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN Why do my ten threads eat 27 megs, when Elizabeth doesn't get there until after 50 threads? From msouth at shodor.org Wed Feb 25 10:23:20 2004 From: msouth at shodor.org (Mike South) Date: Mon Aug 2 21:23:23 2004 Subject: APM: Some progress on efficient filehandle reading, but threads too big In-Reply-To: References: Message-ID: <403CCBF8.mail2SM11995S@scan.shodor.org> >From austin-bounces@mail.pm.org Wed Feb 25 10:55:26 2004 >From: "Brian Michalk" >To: >Date: Wed, 25 Feb 2004 09:53:57 -0600 > ... >Looking at CPAN, Elizabeth Mattijsen has done a lot with threads. I see >from her examples with Benchmark::Thread::Size the following output: > # (ref) bare full vars our unique > 0 2172 +0 +0 +0 +0 +0 > 1 2624 ? 4 +4 ? 4 +4 ? 4 +27 +4 ? 4 +27 > 2 3004 ? 4 +2 ? 6 +2 ? 6 +33 ? 4 +8 +36 ? 6 > 5 4126 ? 6 -2 ? 6 -3 ? 8 +29 ? 4 +10 ? 2 +27 ? 4 > 10 5984 ? 8 -1 ? 8 +0 ? 4 +0 ? 6 +17 ? 4 +43 ? 6 > 20 9694 ? 4 +15 ? 4 +15 ? 2 +13 ? 6 +32 ? 6 +58 ? 6 > 50 20832 ? 4 +51 ?10 +50 ? 8 +50 ? 8 +68 ?12 +96 ? 6 > 100 39392 ? 8 +106 ?10 +156 ?12 +108 ?10 +131 ?10 +155 ?12 > > ==== bare ======================================================== > $VERSION = '0.01'; > > ==== full ======================================================== > $main::VERSION = '0.01'; > > ==== vars ======================================================== > use vars qw($VERSION); > $VERSION = '0.01'; > > ==== our ========================================================= > our $VERSION = '0.01'; > > ==== unique ====================================================== > our $VERSION : unique = '0.01'; > > ================================================================== > >I'm not sure how VERSION plays into memory management, so this confuses me, I suspect that the choice of VERSION has nothing to do with it, but that the way they are stored does. When you reference something as being in package main, or declare it with use vars, our() it or give it the unique attribute, all of those things affect where it gets put and who can see it, so to speak (pure speculation here, though, so hopefully if I'm wrong someone will speak up). >but it shows that 20 threads are consuming less than 10MB total. This would >be great prformance for me. From my 'top', I'm getting the following: > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 3815 root 16 0 28172 27m 1844 S 5.8 47.3 0:17.07 filter_dmi.pl > 3817 root 8 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl > 3818 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.01 filter_dmi.pl > 3819 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.07 filter_dmi.pl > 3821 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl > 3822 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.07 filter_dmi.pl > 3823 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl > 3824 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.00 filter_dmi.pl > 3825 root 9 0 28172 27m 1844 S 0.0 47.3 0:00.08 filter_dmi.pl > >In case you are wondering about the pids, perl -v has this little snippet: >ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN > >Why do my ten threads eat 27 megs, when Elizabeth doesn't get there until >after 50 threads? Just to be clear, (a) are Elizabeth's numbers up there for code running exactly the lines above? In other words, are they for one line scripts that do only one declaration? (b) Have you tried running whatever scripts she was running just to see if you get comparable results? I'm just wondering what the exact apples-to-apples comparison looks like, if you've done it. mike From bob at blacklab.com Wed Feb 25 10:29:44 2004 From: bob at blacklab.com (Bob Richards) Date: Mon Aug 2 21:23:23 2004 Subject: APM: cgi problem Message-ID: I have a bunch of software on a server and about 5% of the time cgi loses all the data coming from the previous page. my sofware typically starts like this: #!/usr/local/bin/perl chdir "/home/sites/home/web/cgi"; push(@INC,"common/univtasks"); require("common/univtasks.pl"); use DBI; use CGI; &usup(); and then in univtasks I have the usup subroutine do this: $q = new CGI; (@names) = $q->param; foreach (@names) { $d{$_} = $q->param($_); $dx .= "$_=$d{$_}&"; } chop $dx; open(DATA,">>/home/sites/home/data/tracker.txt"); print DATA "$ti \t $ENV{REMOTE_ADDR} \t $ENV{SCRIPT_URI}?$dx \n"; close (DATA); Am I doing something wrong? Is there a better way to do this? Bob Richards bob@blacklab.com