From kclark at CetaceanNetworks.com Wed Apr 2 17:01:26 2003 From: kclark at CetaceanNetworks.com (Kevin D. Clark) Date: Mon Aug 2 21:33:07 2004 Subject: [Nh-pm] buildgrep Message-ID: Attached is a script that I hope others will find to be useful. Regards, --kevin -- Kevin D. Clark / Cetacean Networks / Portsmouth, N.H. (USA) cetaceannetworks.com!kclark (GnuPG ID: B280F24E) alumni.unh.edu!kdc -------------- next part -------------- #!/usr/bin/perl # buildgrep # -- parses the output of a build log; sensibly and consisely shows # the user where problems in the build were experienced. # # Author: Kevin D. Clark # # # # # Example usages: (these examples assume you are using bash) # # $ make 2>&1 | buildgrep.pl # $ make 2>&1 | tee complete-build-log.txt | buildgrep.pl # $ buildgrep.pl complete-build-log.txt | more # $ make 2>&1 | mailx -s "build log" fred barney BIFF@BIT.NET # # # Example output: # (from compiling the Linux kernel) # # make[4]: Entering directory `/home/kclark/work/linux-2.4.19/drivers/char/agp' # gcc -D__KERNEL__ -I/home/kclark/work/linux-2.4.19/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i686 -nostdinc -I /usr/lib/gcc-lib/i386-redhat-linux/2.96/include -DKBUILD_BASENAME=agpgart_be -DEXPORT_SYMTAB -c agpgart_be.c # agpgart_be.c:400: warning: unused variable `cap_id' # agpgart_be.c:4298: warning: unused variable `scratch' # agpgart_be.c:4298: warning: unused variable `cap_id' # make[3]: Entering directory `/home/kclark/work/linux-2.4.19/drivers/scsi' # gcc -D__KERNEL__ -I/home/kclark/work/linux-2.4.19/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i686 -nostdinc -I /usr/lib/gcc-lib/i386-redhat-linux/2.96/include -DKBUILD_BASENAME=sym53c8xx -c -o sym53c8xx.o sym53c8xx.c # sym53c8xx.c:6995: warning: `istat' might be used uninitialized in this function # ....... # # # Notes: # # 1: I typically use this with GNU Make. # # 2: When I use gcc, I find the following options to be useful: # # -fmessage-length=0 -W -pedantic -Wall -Wfloat-equal -Wcast-qual # -Wno-long-long -Winline -Wshadow # # 3: This filter works well with build systems that build targets in a # tree-like graph, and works less well with systems whose build targets # have a cycle in their graph. I'm not wild about such build systems. # # 4: If your buold system is "cute" and hides details relevent to # compilation, this script will be useless. I'm not wild about such # build systems. # # 5: I'm thinking about re-structuring this to display stuff that # (in general) is sent to stderr, but I'm still considering this. sub printLastEnterLeave { # print $lastLeave unless $alreadyPrinted{$lastLeave}++; print $lastEnter unless $alreadyPrinted{$lastEnter}++; } sub printLastCompile { print $lastCompile unless $alreadyPrinted{$lastCompile}++; } while (<>) { $lastEnter = $_ if (/make.*[Ee]ntering directory/); $lastLeave = $_ if (/make.*[Ll]eaving directory/); $lastCompile = $_ if (m{^\s* (?:g?cc|g\+\+| yacc|bison|f?lex|lemon| (?:/opt/SUNWspro/bin/)?(?i:cc)) }x); # # errors/warning deliberately ignored (AND JUSTIFICATIONS) # # annoying Linux warning; harmless next if (/warning: ANSI C\+\+ forbids zero-size array \`__cmsg_data\'/); # related to use of 64-bit ints; not a problem next if (/warning: (?:ANSI C|ISO C\d+) does not support the \`ll\' length (?:modifier)?/); next if (/warning: (?:ANSI|ISO) C\+\+ does not support `long long'/); next if (/warning: multi-line comment/); # 19-jul-2002 kclark I am thinking about removing this next filter # # warning from the Sun C++ compiler that we are passing a pointer to a # function in the C++ namespace into something that expects a pointer to # a function in the C namespace. This happens a lot with C++ and pthreads. next if (/Warning\s+\(Anachronism\):\s+Formal argument\s+\w+\s+of type\s+(?:extern "C")?\s*(.*)\s+in call to.*is being passed\s+(?:extern "C")?\s*\1/); next if (/Warning\s+\(Anachronism\):\s+Assigning (.*?)\s+to extern "C" \1/); # # make messages to flag # printLastEnterLeave, print if ( /^Makefile:/i || /^(?:clear)?make.*\*\*\*/i || /^(?:clear)?make.*(?:error|warning):/i ); # # compile errors to flag # printLastEnterLeave, printLastCompile, print if ( /(?:warning|error):/i || /In function:/i || /At top level:/i || /In (?:method|function):/i || /\*Initialization*:/i || /^".*?", line \d+:/i || m,^(?:\w|/)*\w+\.\w+:\d+:,i ); # linker errors are harder to parse, but not impossible... $printNoMatterWhat = 1 if ( /^\s*Undefined/ || m,/usr/bin/ld:, || m/^ld:/); printLastEnterLeave, printLastCompile, print if ($printNoMatterWhat); $printNoMatterWhat = 0 if (/make.+error/i); } From pll at lanminds.com Thu Apr 17 12:35:04 2003 From: pll at lanminds.com (pll@lanminds.com) Date: Mon Aug 2 21:33:07 2004 Subject: [Nh-pm] Hash Persistence? Message-ID: <20030417173504.6268AF8B0@tater> Hi all, I have a module which creates a rather large hash table. The values of this hash are also hashes. I'm looking for a way to create a persistent store for this Hash of hashes, any ideas? One thought I had was Mark Jason Dominus' Memoize.pm, but that doesn't seem to be quite right for this particular problem. The other thought I had was storing each leaf hash as a separate db file on disk, but that seems it would require almost as much disk access as computing the entire hash to begin with, if not more. The structure of my hash table is this: OneKey => \%atable ( subkey1 => value, subkey2 => value, etc. ), TwoKey => \%btable ( subkey1 => value, subkey2 => value, etc. ), etc. Any ideas? (meanwhile, I'll be perusing CompSci and Perl Programming, and The Perl Cookbook :) Thanks, -- Seeya, Paul -- Key fingerprint = 1660 FECC 5D21 D286 F853 E808 BB07 9239 53F1 28EE It may look like I'm just sitting here doing nothing, but I'm really actively waiting for all my problems to go away. If you're not having fun, you're not doing it right! From morbus at disobey.com Thu Apr 17 12:42:40 2003 From: morbus at disobey.com (Morbus Iff) Date: Mon Aug 2 21:33:07 2004 Subject: [Nh-pm] Hash Persistence? In-Reply-To: <20030417173504.6268AF8B0@tater> Message-ID: <5.2.1.0.2.20030417133948.00c2bb50@red.totalnetnh.net> >I have a module which creates a rather large hash table. The values >of this hash are also hashes. I'm looking for a way to create >a persistent store for this Hash of hashes, any ideas? use Data::Dumper; print FILE Dumper(%evilhash); and then to suck it in: if (-e $file) { do $file; } "do" is relatively dangerous in that it sucks in $file and immediately executes the code into your application - you can tweak Dumper's output as well (naming the hash as you want it, etc.). See: http://www.perldoc.com/perl5.8.0/pod/func/do.html http://www.perldoc.com/perl5.8.0/lib/Data/Dumper.html -- Morbus Iff ( i'm the droid you're looking for ) Culture: http://www.disobey.com/ and http://www.gamegrene.com/ Buy My Book! http://amazon.com/exec/obidos/ASIN/0596004605/disobeycom icq: 2927491 / aim: akaMorbus / yahoo: morbus_iff / jabber.org: morbus From ben at blackavar.com Thu Apr 17 14:07:47 2003 From: ben at blackavar.com (Ben Boulanger) Date: Mon Aug 2 21:33:07 2004 Subject: [Nh-pm] Hash Persistence? In-Reply-To: <20030417173504.6268AF8B0@tater> Message-ID: > I have a module which creates a rather large hash table. The values > of this hash are also hashes. I'm looking for a way to create > a persistent store for this Hash of hashes, any ideas? > > One thought I had was Mark Jason Dominus' Memoize.pm, but that > doesn't seem to be quite right for this particular problem. > > The other thought I had was storing each leaf hash as a separate db > file on disk, but that seems it would require almost as much disk > access as computing the entire hash to begin with, if not more. > > The structure of my hash table is this: > > OneKey => \%atable ( subkey1 => value, > subkey2 => value, > etc. > ), > TwoKey => \%btable ( subkey1 => value, > subkey2 => value, > etc. > ), > etc. > > Any ideas? (meanwhile, I'll be perusing CompSci and Perl Programming, > and The Perl Cookbook :) I personally use the mldbm module found on cpan: http://www.cpan.org/modules/by-module/MLDBM/ Worked great for me.... then I said 'why am I not using mysql again?' and did that instead.... :) -- Even a hare will bite when it is cornered. From pll at lanminds.com Thu Apr 17 14:40:39 2003 From: pll at lanminds.com (pll@lanminds.com) Date: Mon Aug 2 21:33:07 2004 Subject: [Nh-pm] Hash Persistence? In-Reply-To: Your message of "Thu, 17 Apr 2003 15:07:47 EDT." References: Message-ID: <20030417194039.AD3B7F8B0@tater> >>>>> On Thu, 17 Apr 2003, "Ben" == Ben Boulanger wrote: Ben> I personally use the mldbm module found on cpan Ayup, that's where I seem to be headed after quite a lengthy discussion with Morbus :) >>>>> On Thu, 17 Apr 2003, "Ben" == Ben Boulanger wrote: Ben> Worked great for me.... then I said 'why am I not using mysql Ben> again?' and did that instead.... :) Well, as I mentioned to MI, I'm using LDAP as the back end, because I need authentication. What I'm working on now is a phone-book front end to the LDAP server. LDAP gives me both a phone/address book, jpeg store, and authentication in one, with MySQL, I'd have to build my own auth mechanism. To summarize the conversation thus far: >>>>> On Thu, 17 Apr 2003, "pll" == pll@lanminds.com wrote: >>>>> On Thu, 17 Apr 2003, "Morbus" == Morbus Iff wrote: pll> I'm building an LDAP database for my group (about 200 pll> entries so far). Originally, this was meant strictly for pll> authentication, but I figured, since I have the db set up, I pll> may as well use it for the phone book as well :) pll> I'm at the early stages, and I figure this is a good case to pll> muck with different modules I haven't ever needed before :) Morbus> Hell, screw a hash or DBM, suck down the LDAP, rewrite Morbus> it as vCards, and then you import them into other Morbus> utilities/pm/pda's ;) ... pll> Actually, that's the next step :) pll> Of course, the real logical thing to do would actuall be to pll> cache the final HTML output by the front-end CGI, and skip pll> *everything* if that html file exists. That way there's no pll> processing of any kind, just an open, suck it off of disk, spit pll> it out to the client process :) Morbus> You know, instead of having Perl suck the file off the disk, Morbus> just put the cached HTML online, then have Perl issue a Morbus> print "Location: $html\n\n";, then you can have Apache Morbus> handle all that other crap. Of course, then you're REALLY Morbus> taking the fun out of everything ;) pll> Doh!!!! Why didn't I think of that!? Oh, wait, I'm not a web pll> developer, that's why ;) Oh, and I've got the MLDBM thingy working just nicely now. Thanks to both of you for your ideas :) -- Seeya, Paul -- Key fingerprint = 1660 FECC 5D21 D286 F853 E808 BB07 9239 53F1 28EE It may look like I'm just sitting here doing nothing, but I'm really actively waiting for all my problems to go away. If you're not having fun, you're not doing it right!