From arocker at Vex.Net Fri Nov 1 10:46:24 2013 From: arocker at Vex.Net (arocker at Vex.Net) Date: Fri, 1 Nov 2013 13:46:24 -0400 Subject: [tpm] Deadly architecture Message-ID: If somebody at Toyota had absorbed the lessons of "The Sciences of The Artificial", this case http://www.eetimes.com/document.asp?doc_id=1319966& might not have happened. (The important parts are in the expert testimony.) Fundamentally, it appears that the important functions were not arranged in a trust-worthy hierarchy. From fulko.hew at gmail.com Fri Nov 1 11:27:21 2013 From: fulko.hew at gmail.com (Fulko Hew) Date: Fri, 1 Nov 2013 14:27:21 -0400 Subject: [tpm] Deadly architecture In-Reply-To: References: Message-ID: On Fri, Nov 1, 2013 at 1:46 PM, wrote: > > If somebody at Toyota had absorbed the lessons of "The Sciences of The > Artificial", this case http://www.eetimes.com/document.asp?doc_id=1319966& > might not have happened. (The important parts are in the expert testimony.) > > Fundamentally, it appears that the important functions were not arranged > in a trust-worthy hierarchy. > Rats! and I drive a Camry! -------------- next part -------------- An HTML attachment was scrubbed... URL: From arocker at Vex.Net Sun Nov 3 08:25:26 2013 From: arocker at Vex.Net (arocker at Vex.Net) Date: Sun, 3 Nov 2013 11:25:26 -0500 Subject: [tpm] Alarming portents Message-ID: Party like it's 1999; again? http://www.businessinsider.com/evidence-that-tech-sector-is-in-a-bubble-2013-11 Thank you, Mr. Bernanke. From arocker at Vex.Net Fri Nov 8 06:30:15 2013 From: arocker at Vex.Net (arocker at Vex.Net) Date: Fri, 8 Nov 2013 09:30:15 -0500 Subject: [tpm] Good examples? Message-ID: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Does anyone know of a good explanation/example of how to use XML::SAX anywhere online? I've looked at the CPAN documentation, and other usual sources, but they have some ambiguity about what are standard method names, features, &c., and what are simply examples of user-written resources. The problem that started this train of thought is absurdly simple; extracting 3 fields per record from a STDIN stream of records containing 5 or 6 embedded in XML tags. A simple program, basically 3 regexes and a print, does the job, but attracted a storm of online criticism about parsing XML with regexes. I've been trying to write the equivalent using XML::SAX, but seem to be missing something. (For one thing, all the examples assume reading from a named file, not STDIN.) From talexb at gmail.com Fri Nov 8 06:49:02 2013 From: talexb at gmail.com (Alex Beamish) Date: Fri, 8 Nov 2013 09:49:02 -0500 Subject: [tpm] Good examples? In-Reply-To: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Message-ID: I haven't used it, but XML::SAX appears to be a wrapper or loader for the XML parser of your choice. It looks like you could use XML::SAX::PurePerl directly, for example, in order to parse XML. If you need to read your XML from a file, stash it in a file, rather than trying to pipe it into your code. Regular expressions are fine for fairly small data items, but they're usually not suitable for (mostly) structured data like HTML and XML. Alex On Fri, Nov 8, 2013 at 9:30 AM, wrote: > > Does anyone know of a good explanation/example of how to use XML::SAX > anywhere online? > > I've looked at the CPAN documentation, and other usual sources, but they > have some ambiguity about what are standard method names, features, &c., > and what are simply examples of user-written resources. > > The problem that started this train of thought is absurdly simple; > extracting 3 fields per record from a STDIN stream of records containing 5 > or 6 embedded in XML tags. > > A simple program, basically 3 regexes and a print, does the job, but > attracted a storm of online criticism about parsing XML with regexes. > I've been trying to write the equivalent using XML::SAX, but seem to be > missing something. (For one thing, all the examples assume reading from a > named file, not STDIN.) > > _______________________________________________ > toronto-pm mailing list > toronto-pm at pm.org > http://mail.pm.org/mailman/listinfo/toronto-pm > -- Alex Beamish Toronto, Ontario aka talexb -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuart at morungos.com Fri Nov 8 06:49:46 2013 From: stuart at morungos.com (Stuart Watt) Date: Fri, 8 Nov 2013 09:49:46 -0500 Subject: [tpm] Good examples? In-Reply-To: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Message-ID: <374C3448-D117-4602-A47C-795EB01B6720@morungos.com> I never use XML::SAX in practice. It?s too much work. I usually use XML::LibXML::Reader, which allows more or less streaming access to elements. I can read in elements, even from STDIN, and then for a given element, I can turn it into a DOM tree and use XPath to get the bits I need. XML::LibXML is C underneath, so this is fast. It also handles files that are too big to fit in memory, which was how I ended up with this toolchain. It looks a bit like this: my $reader = XML::LibXML::Reader->new(IO => $fh); while($reader->read() && $reader->name() ne 'mutation') {}; do { if ($reader->name() eq 'mutation') { my $root = $reader->copyCurrentNode(1); my $name = $root->findvalue(qq{/Entrezgene-Set/Entrezgene/Entrezgene_gene/Gene-ref/Gene-ref_locus}); ? and similar extractions } } while($reader->nextSibling()); This does assume that my file contains lots of sibling mutation elements, but nothing about what each contains. The advantage of this is that actually using SAX to get at /mutation/Entrezgene-Set/Entrezgene/Entrezgene_gene/Gene-ref/Gene-ref_locus nested elements would be extremely painful. Oh, and $fh can easily be \*STDIN if you need. Seriously, XML::LibXML is a great set of tools for XML. Using SAX directly is going to be miserable. Also, XML::LibXML has decent API documentation, although the examples might leave something to be desired. All the best Stuart On Nov 8, 2013, at 9:30 AM, arocker at Vex.Net wrote: > > Does anyone know of a good explanation/example of how to use XML::SAX > anywhere online? > > I've looked at the CPAN documentation, and other usual sources, but they > have some ambiguity about what are standard method names, features, &c., > and what are simply examples of user-written resources. > > The problem that started this train of thought is absurdly simple; > extracting 3 fields per record from a STDIN stream of records containing 5 > or 6 embedded in XML tags. > > A simple program, basically 3 regexes and a print, does the job, but > attracted a storm of online criticism about parsing XML with regexes. > I've been trying to write the equivalent using XML::SAX, but seem to be > missing something. (For one thing, all the examples assume reading from a > named file, not STDIN.) > > _______________________________________________ > toronto-pm mailing list > toronto-pm at pm.org > http://mail.pm.org/mailman/listinfo/toronto-pm -- Stuart Watt stuart at morungos.com / twitter.com/morungos -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 496 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bamccaig at gmail.com Fri Nov 8 07:24:09 2013 From: bamccaig at gmail.com (Brandon McCaig) Date: Fri, 8 Nov 2013 10:24:09 -0500 Subject: [tpm] Good examples? In-Reply-To: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Message-ID: On Fri, Nov 8, 2013 at 9:30 AM, wrote: > Does anyone know of a good explanation/example of how to use XML::SAX > anywhere online? > > I've looked at the CPAN documentation, and other usual sources, but they > have some ambiguity about what are standard method names, features, &c., > and what are simply examples of user-written resources. > > The problem that started this train of thought is absurdly simple; > extracting 3 fields per record from a STDIN stream of records containing 5 > or 6 embedded in XML tags. > > A simple program, basically 3 regexes and a print, does the job, but > attracted a storm of online criticism about parsing XML with regexes. > I've been trying to write the equivalent using XML::SAX, but seem to be > missing something. (For one thing, all the examples assume reading from a > named file, not STDIN.) I have used XML::Simple with some success in the past. If you just need a proper XML parser for best practices, but don't need something robust, then it's probably an ideal option. Of course, taking the time to learn the larger, feature-rich modules will translate to larger projects later on, but XML::Simple will probably get you to a solution faster. IIRC, you basically just load the XML file/text and are given a data structure in return that represents it. It can be a little bit weird about how it decides what is an array ref versus a scalar. Depending on your document you might find that awkward to work with. The only workaround that I know of (and it's been a couple of years so I don't remember the setting) is to force it to always create array refs, which means when you know there's only one value you will always have to dereference the first element, but at least the structure is predictable that way. Good luck. :) Regards, -- Brandon McCaig Castopulence Software Blog perl -E '$_=q{V zrna gur orfg jvgu jung V fnl. }. q{Vg qbrfa'\''g nyjnlf fbhaq gung jnl.}; tr/A-Ma-mN-Zn-z/N-Zn-zA-Ma-m/;say' From jkeen at verizon.net Fri Nov 8 15:56:58 2013 From: jkeen at verizon.net (James E Keenan) Date: Fri, 08 Nov 2013 18:56:58 -0500 Subject: [tpm] Good examples? In-Reply-To: References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Message-ID: <527D7A4A.6080707@verizon.net> On 11/8/13 10:24 AM, Brandon McCaig wrote: > On Fri, Nov 8, 2013 at 9:30 AM, wrote: >> Does anyone know of a good explanation/example of how to use XML::SAX >> anywhere online? >> > > I have used XML::Simple with some success in the past. If you just > need a proper XML parser for best practices, but don't need something > robust, then it's probably an ideal option. Of course, taking the time > to learn the larger, feature-rich modules will translate to larger > projects later on, but XML::Simple will probably get you to a solution > faster. True. But please note the qualifications the maintainer provides here: http://search.cpan.org/~grantm/XML-Simple-2.20/lib/XML/Simple.pm#STATUS_OF_THIS_MODULE From adam.prime at utoronto.ca Fri Nov 8 20:25:57 2013 From: adam.prime at utoronto.ca (Adam Prime) Date: Fri, 08 Nov 2013 23:25:57 -0500 Subject: [tpm] Good examples? In-Reply-To: <5781_1383921031_rA8EUUMm007379_4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> References: <5781_1383921031_rA8EUUMm007379_4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Message-ID: <527DB955.8080809@utoronto.ca> I have no idea if this is a good example or not, but i played around with SAX a bit when i was looking at the nextbus api earlier this year. This is one of the screwy parser things i built in my experiments. --- package AgencyParser; use strict; use base qw(XML::SAX::Base); my $agencies; sub start_element { my ($self, $el) = @_; if ($el->{'LocalName'} eq 'agency') { my $agency; foreach my $att (keys %{$el->{Attributes}}) { $agency->{$el->{'Attributes'}->{$att}->{LocalName}} = $el->{'Attributes'}->{$att}->{Value}; } push @$agencies, $agency; } } sub end_document { $agencies = []; } sub end_document { my ($self, $doc) = @_; return $agencies; } 1; ----------- my $agency_parser = XML::SAX::ParserFactory->parser( Handler => AgencyParser->new() ); my $agencies = $agency_parser->parse_uri('http://webservices.nextbus.com/service/publicXMLFeed?command=agencyList'); From liam at holoweb.net Sat Nov 9 19:25:03 2013 From: liam at holoweb.net (Liam R E Quin) Date: Sun, 10 Nov 2013 11:25:03 +0800 Subject: [tpm] Good examples? In-Reply-To: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> Message-ID: <1384053903.31369.91.camel@localhost.localdomain> On Fri, 2013-11-08 at 09:30 -0500, arocker at Vex.Net wrote: > Does anyone know of a good explanation/example of how to use XML::SAX > anywhere online? [...] > extracting 3 fields per record from a STDIN stream of records containing 5 > or 6 embedded in XML tags. Although this is moderately easy with SAX, and if you still need examples I can probably dig some up. The most important gotcha with sax is that element content (text) might be split over several callbacks. These days I'd want to use a domain-specific language such as XPath or XQuery if possible; if not, then yes, SAX for streaming. Liam -- Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/ Pictures from old books: http://fromoldbooks.org/ Ankh: irc.sorcery.net irc.gnome.org freenode/#xml From fulko.hew at gmail.com Sun Nov 10 18:56:15 2013 From: fulko.hew at gmail.com (Fulko Hew) Date: Sun, 10 Nov 2013 21:56:15 -0500 Subject: [tpm] Good examples? In-Reply-To: <527D7A4A.6080707@verizon.net> References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> <527D7A4A.6080707@verizon.net> Message-ID: On Fri, Nov 8, 2013 at 6:56 PM, James E Keenan wrote: > On 11/8/13 10:24 AM, Brandon McCaig wrote: > >> On Fri, Nov 8, 2013 at 9:30 AM, wrote: >> >>> Does anyone know of a good explanation/example of how to use XML::SAX >>> anywhere online? >>> >>> > >> I have used XML::Simple with some success in the past. If you just >> need a proper XML parser for best practices, but don't need something >> robust, then it's probably an ideal option. Of course, taking the time >> to learn the larger, feature-rich modules will translate to larger >> projects later on, but XML::Simple will probably get you to a solution >> faster. >> > > > True. But please note the qualifications the maintainer provides here: > http://search.cpan.org/~grantm/XML-Simple-2.20/lib/ > XML/Simple.pm#STATUS_OF_THIS_MODULE > > I too read that note, but used the module anyway because I thought it was the easiest module to use with the simplest (most understandable) documentation. Also the fact that (I think) it was a pure Perl solution. It met my needs, but encountered an issue where the hash/object it created had (what I considered to be a different/inconsistent structure depending on whether the XML contained empty attributes or not. There wasn't an option for XML::Simple, so I wrote a post-processor fix-up to make the resultant structure consistent. Other than that, I thought it was great. [Postscript} A coworker needed to do some XML recently, he started with XML::Simple, because I had experience, and ... it was simple. Once he had a working app, he moved to a different module that processed his 1+ gig XML 10+ times faster. -------------- next part -------------- An HTML attachment was scrubbed... URL: From quantum.mechanic.1964 at gmail.com Mon Nov 11 02:08:33 2013 From: quantum.mechanic.1964 at gmail.com (Kwan Tamakanic) Date: Mon, 11 Nov 2013 10:08:33 +0000 Subject: [tpm] Good examples? In-Reply-To: References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> <527D7A4A.6080707@verizon.net> Message-ID: On Mon, Nov 11, 2013 at 2:56 AM, Fulko Hew wrote: > > [Postscript} A coworker needed to do some XML recently, he started > with XML::Simple, because I had experience, and ... it was simple. > Once he had a working app, he moved to a *different module* that > processed his 1+ gig XML 10+ times faster. > What did $different_module interpolate to in that context? -------------- next part -------------- An HTML attachment was scrubbed... URL: From olaf.alders at gmail.com Mon Nov 11 07:38:00 2013 From: olaf.alders at gmail.com (Olaf Alders) Date: Mon, 11 Nov 2013 10:38:00 -0500 Subject: [tpm] My slides have been uploaded Message-ID: <5464804C-4BD2-47B0-B0DA-20001268D649@gmail.com> I've just uploaded my slides for the MetaCPAN VM installation: http://www.slideshare.net/oalders/the-metacpan-vm-for-dummies-part-one-installation This is sort of a prerequisite for being able to hack along at this month's meeting. So, I'd encourage people to give this a go. If you're having issues, let me know. I know Tom and Mike already had a crack at it earlier this month. If either of you got stuck, please ping me so that we can get you unstuck. :) Olaf -- Olaf Alders olaf.alders at gmail.com http://www.wundercounter.com http://twitter.com/wundercounter 866 503 2204 (Toll free - North America) 416 944 8306 (direct) From mattp at cpan.org Mon Nov 11 11:48:04 2013 From: mattp at cpan.org (Matthew Phillips) Date: Mon, 11 Nov 2013 14:48:04 -0500 Subject: [tpm] Good examples? In-Reply-To: References: <4ca42f7e493730786e3978e58d50f93d.squirrel@mail.vex.net> <527D7A4A.6080707@verizon.net> Message-ID: For what its worth, XML::Simple _is_ built ontop of XML::SAX, which means you can dictate the backend xml parser with XML_SIMPLE_PREFERRED_PARSER / $XML::Simple::PREFERRED_PARSER for something faster, such as XML::LibXML or Expat. Also for next time, try taking a look at the KeyAttr/ForceArray/ForceContent (and probably) other constructor opts, they play a large role in how xml input is interpreted. Cheers, Matt On Sun, Nov 10, 2013 at 9:56 PM, Fulko Hew wrote: > > > > On Fri, Nov 8, 2013 at 6:56 PM, James E Keenan wrote: > >> On 11/8/13 10:24 AM, Brandon McCaig wrote: >> >>> On Fri, Nov 8, 2013 at 9:30 AM, wrote: >>> >>>> Does anyone know of a good explanation/example of how to use XML::SAX >>>> anywhere online? >>>> >>>> >> >>> I have used XML::Simple with some success in the past. If you just >>> need a proper XML parser for best practices, but don't need something >>> robust, then it's probably an ideal option. Of course, taking the time >>> to learn the larger, feature-rich modules will translate to larger >>> projects later on, but XML::Simple will probably get you to a solution >>> faster. >>> >> >> >> True. But please note the qualifications the maintainer provides here: >> http://search.cpan.org/~grantm/XML-Simple-2.20/lib/ >> XML/Simple.pm#STATUS_OF_THIS_MODULE >> >> > I too read that note, but used the module anyway because > I thought it was the easiest module to use with the simplest > (most understandable) documentation. Also the fact that > (I think) it was a pure Perl solution. > > It met my needs, but encountered an issue where the hash/object > it created had (what I considered to be a different/inconsistent > structure depending on whether > > the XML contained empty attributes or not. > > There wasn't an option for XML::Simple, so I wrote a post-processor > fix-up to make the resultant structure consistent. > > Other than that, I thought it was great. > > [Postscript} A coworker needed to do some XML recently, he started > with XML::Simple, because I had experience, and ... it was simple. > Once he had a working app, he moved to a different module that > processed his 1+ gig XML 10+ times faster. > > > > > _______________________________________________ > toronto-pm mailing list > toronto-pm at pm.org > http://mail.pm.org/mailman/listinfo/toronto-pm > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jztam at yahoo.com Wed Nov 20 09:11:24 2013 From: jztam at yahoo.com (J Z Tam) Date: Wed, 20 Nov 2013 09:11:24 -0800 (PST) Subject: [tpm] How to automate/test d/l and u/l files from webpage with Javascript enabled. Message-ID: <1384967484.89874.YahooMailNeo@web120905.mail.ne1.yahoo.com> Yeah, I know,? which sites Don't have JS enabled nowadays. NM. Hey Matt,?? Got a few minutes? I'm having some difficulty whittling these modules I'm using:? WWW::Mechanize,? WWW::Selenium,? HTTP::Recorder, HTTP::Proxy, and LWP::Debug.?? I've installed the recommended Selenium bundle to my Firefox 15.0.1 on Windows7 and it is working. 1. I'm wanting to automate downloading a set of files from an https internal site, morph the contents and rename these files, then send them to another site or UNC.? The problem I'm having is that the site spawns off a Windows Explorer window when I "click into"? the form, which fires a:? javascript:openwindow('parm1', 'parm2')? function.? Now, how do I access the Explorer window?? I'm guessing I'll have to determine the windowID of the Explorer window and treat it specially, outside the WWW::* and? HTTP::*? spaces. 2. Tried setting up a local proxy to record all the kbd and mouse events, but could not get it working (on Windows7).? Tips?? No, I cannot switch to *nix,? ugh. -------------- next part -------------- An HTML attachment was scrubbed... URL: From olaf.alders at gmail.com Fri Nov 22 17:40:13 2013 From: olaf.alders at gmail.com (Olaf Alders) Date: Fri, 22 Nov 2013 20:40:13 -0500 Subject: [tpm] =?iso-8859-1?q?Ingy_d=F6t_Net=3A_Bashing_your_Brain_Cells_?= =?iso-8859-1?q?=28Thursday=2C_Nov_28=29?= Message-ID: <7116A7D9-8DF3-4931-BB35-EFD50CA897D6@gmail.com> So, Mr. Ingy d?t Net was showing me some of the stuff he was working on and I mentioned to him that he might like to present all of this crazy stuff to us. Our next open meeting slot is not until January, so I offered him my slot for this coming Thursday. I can push my MetaCPAN talk until January, if that's OK. I feel like I've already done a lot of talking anyway between lightning talks and speaking last month as well. Here's a an abstract for what Ingy plans to speak about. I should mention that he'll be presenting remotely. This will be our first full-length meeting with a remote presenter. I'll demo a hangout with him in advance to make sure we get the details right. >>>>>>>>>>>>>>>>>>>>>> In his first ever Internationally Remote Perl Monger Trainwreck (live demos galore), Ingy d?t Net (the man who ruined Perl with abominations like Kwiki, YAML, IO::All, ?) will Bash your remaining brain cells with his latest Frankensteinian misdemeanors. Bash? Yes, Bash! * git-hub -- All of Git-Hub in your terminal * PairUp -- Pair program with anyone on GitHub or IRC * Termcast -- Let the internet watch you * git-subrepo -- Git submodules can DIE! * bpan -- Like CPAN, only worse * json-bash -- Seriously * test-more-bash -- No joke * bash+ -- aka Perl 0.1 * ... -- pronounced dotdotdot * 10 things you (probably) don't know about YAML >>>>>>>>>>>>>>>>>>>>>> So, there's some kind of crazy stuff in here and some really cool stuff as well. His git-hub repo gives you the power of github at the command line. PairUp lets you share a tmux session with a collaborator and it actually munges your tmux preferences together. You can find a lot of his projects here: https://github.com/ingydotnet This is the kind of presentation that doesn't require any Perl knowledge. It should appeal to programmers in general since a lot of the tools aren't specific to Perl, but are the kinds of things you can integrate into your daily workflow. Olaf -- Olaf Alders olaf.alders at gmail.com http://www.wundercounter.com http://twitter.com/wundercounter 866 503 2204 (Toll free - North America) 416 944 8306 (direct) From arocker at Vex.Net Sat Nov 23 10:27:05 2013 From: arocker at Vex.Net (arocker at Vex.Net) Date: Sat, 23 Nov 2013 13:27:05 -0500 Subject: [tpm] =?iso-8859-1?q?Ingy_d=F6t_Net=3A_Bashing_your_Brain_Cells_?= =?iso-8859-1?q?=28Thursday_=2C_Nov_28_=29?= In-Reply-To: <7116A7D9-8DF3-4931-BB35-EFD50CA897D6@gmail.com> References: <7116A7D9-8DF3-4931-BB35-EFD50CA897D6@gmail.com> Message-ID: <53294532927d95e25c44e7b0f5595dbc.squirrel@mail.vex.net> > So, Mr. Ingy d?t Net was showing me some of the stuff he was working on > and I mentioned to him that he might like to present all of this crazy > stuff to us. Great idea! The process should be interesting in itself. From dave.s.doyle at gmail.com Thu Nov 28 15:53:33 2013 From: dave.s.doyle at gmail.com (Dave Doyle) Date: Thu, 28 Nov 2013 18:53:33 -0500 Subject: [tpm] G+ Hangout for tonight Message-ID: It's live! https://plus.google.com/hangouts/_/76cpirm3kht73n3dfn44pihs7s?authuser=0&hl=en -- dave.s.doyle at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: