From rjbs-perl-abe at lists.manxome.org  Tue Oct  3 06:49:07 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Tue, 3 Oct 2006 09:49:07 -0400
Subject: [ABE.pm] social meeting tomorrow, 2006-10-04
Message-ID: <20061003134906.GE387@zodiac.codesimply.com>


I'm afraid I can't find Macht's Gut on Google Maps, despite trying every
variation on the spelling I could think of.  Whatever!  I know where it is.

http://maps.google.com/maps?f=q&hl=en&q=linden+st+and+north+street,+bethlehem,+pa&ie=UTF8&z=15&ll=40.623692,-75.370502&spn=0.016058,0.037293&t=h&om=1&iwloc=A

It's on Linden Street, between Union Blvd and Broad Street.  If you can't find
it, you can call me at 610 570 1950 to ask for directions.

Like I said, I've never been there, but I've heard it's pretty good, and on the
order of McGrady's.  If you want to strenuously object to the venue, do it now.
Otherwise, I'll be there tomorrow around 18:30, looking for you punks.

-- 
rjbs

From faber at linuxnj.com  Wed Oct  4 19:33:28 2006
From: faber at linuxnj.com (Faber J. Fedor)
Date: Wed, 4 Oct 2006 22:33:28 -0400
Subject: [ABE.pm] The Story of Mel
Message-ID: <20061005023328.GB25461@neptune.faber.nom>

As discussed during the latest ABE-PM dinner:

http://www.pbm.com/~lindahl/mel.html


-- 
 
Regards,
 
Faber Fedor
President
Linux New Jersey, Inc.
908-320-0357
800-706-0701

http://www.linuxnj.com




From rjbs-perl-abe at lists.manxome.org  Thu Oct  5 06:25:19 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Thu, 5 Oct 2006 09:25:19 -0400
Subject: [ABE.pm] The Story of Mel
In-Reply-To: <20061005023328.GB25461@neptune.faber.nom>
References: <20061005023328.GB25461@neptune.faber.nom>
Message-ID: <20061005132519.GA5794@knight>

* "Faber J. Fedor" <faber at linuxnj.com> [2006-10-04T22:33:28]
> As discussed during the latest ABE-PM dinner:
> 
> http://www.pbm.com/~lindahl/mel.html

Read this version instead: 

  http://www.cs.utah.edu/~elb/folklore/mel.html

It's the poetic-formatting one. ;)

-- 
rjbs

From faber at linuxnj.com  Thu Oct 12 17:25:14 2006
From: faber at linuxnj.com (Faber J. Fedor)
Date: Thu, 12 Oct 2006 20:25:14 -0400
Subject: [ABE.pm] A better way?
Message-ID: <20061013002514.GA31695@neptune.faber.nom>

I finally used a technique from HOP, in-line caching to be precise. I
was pleased with myself.  Here's a trimed down version of my code:

for ($i=0; $i <= $end; $i++) {

    if( !defined($captArray[$i])){
            generateException($cusipArray[$i]);
            next;
            }
    # do some useful stuff here
}

generateException('print_report');


{
    my @ListOfCusips ;

    sub generateException {

        ($cusip) = @_;

        if ($cusip eq "print_report") {
            magically_print_the_array_to_file(@ListOfCusips);
        } else {
            push @ListOfCusips, $cusip;
        }
        return(0);
    }
}


The only thing I don't like about this is my "overloading" of the argument
to generateException; at one point in the program the argument is a
cusip (think 'serial number') and at another point it's a directive.

Is there a more elegant way to do this?

What I'm attempting to do in the for loop is to catch any cusips that have undefined
value and write those cusips to file.


-- 
 
Regards,
 
Faber Fedor
President
Linux New Jersey, Inc.
908-320-0357
800-706-0701

http://www.linuxnj.com




From rjbs-perl-abe at lists.manxome.org  Thu Oct 12 18:44:33 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Thu, 12 Oct 2006 21:44:33 -0400
Subject: [ABE.pm] A better way?
In-Reply-To: <20061013002514.GA31695@neptune.faber.nom>
References: <20061013002514.GA31695@neptune.faber.nom>
Message-ID: <20061013014433.GA1538@zodiac.codesimply.com>

* "Faber J. Fedor" <faber at linuxnj.com> [2006-10-12T20:25:14]
> Is there a more elegant way to do this?

Well, I'm not exactly sure, but I think this will do what you want:

  for my $i (0 .. $end) {
      if ( !defined($captArray[$i])) {
          generateException($cusipArray[$i]);
          next;
      }
      # do some useful stuff here
  }
  
  print_exception_report;
  
  {
      my @ListOfCusips;
  
      sub generateException {
          ($cusip) = @_;
          push @ListOfCusips, $cusip;
      }
  
      sub print_exception_report {
          magically_print_the_array_to_file(@ListOfCusips);
      }
  }

I'm not really sure what the point of all this is, though.  Why not this:

  my @ListOfCusips;
  for my $i (0 .. $end) {
      if ( !defined($captArray[$i])) {
          push @ListOfCusips, $cusipArray[$i];
          next;
      }
      # do some useful stuff here
  }
  
  print_report(\@ListOfCusips);
  
  sub print_exception_report {
      my $list = shift;
      magically_print_the_array_to_file(@$list);
  }

Also, the fact that you have two arrays with parallel indices really makes it
look like you want a hash.

-- 
rjbs

From faber at linuxnj.com  Fri Oct 13 06:04:34 2006
From: faber at linuxnj.com (Faber Fedor)
Date: Fri, 13 Oct 2006 09:04:34 -0400
Subject: [ABE.pm] Fwd:  A better way?
In-Reply-To: <300ccfa50610121904m4e37b548y574d4298b85d4324@mail.gmail.com>
References: <20061013002514.GA31695@neptune.faber.nom>
	<20061013014433.GA1538@zodiac.codesimply.com>
	<300ccfa50610121904m4e37b548y574d4298b85d4324@mail.gmail.com>
Message-ID: <300ccfa50610130604r39e9f4cbibfd7e6fca166a3cb@mail.gmail.com>

---------- Forwarded message ----------
From: Faber Fedor <faber at linuxnj.com>
Date: Oct 12, 2006 10:04 PM
Subject: Re: [ABE.pm] A better way?
To: Ricardo SIGNES <rjbs-perl-abe at lists.manxome.org>


On 10/12/06, Ricardo SIGNES <rjbs-perl-abe at lists.manxome.org> wrote:
> * "Faber J. Fedor" <faber at linuxnj.com> [2006-10-12T20:25:14]
> > Is there a more elegant way to do this?
>
> Well, I'm not exactly sure, but I think this will do what you want:
>
>   for my $i (0 .. $end) {
>       if ( !defined($captArray[$i])) {
>           generateException($cusipArray[$i]);
>           next;
>       }
>       # do some useful stuff here
>   }
>
>   print_exception_report;
>
>   {
>       my @ListOfCusips;
>
>       sub generateException {
>           ($cusip) = @_;
>           push @ListOfCusips, $cusip;
>       }
>
>       sub print_exception_report {
>           magically_print_the_array_to_file(@ListOfCusips);
>       }
>   }
>
> I'm not really sure what the point of all this is, though.

I hadn't thought to put both functions inside the block.  DOH!

> Why not this:
>
>   my @ListOfCusips;

My for loop is actually inside of another function and not the main()
of the program. Also, as a rule of thumb I try to avoid global
variables (even though I've got plenty!)

> Also, the fact that you have two arrays with parallel indices really makes it
> look like you want a hash.

I *knew* someone was going to mention that!  I didn't write this code
but it's so crucial to the day-to-day operations and is quite complex
that rewriting it would be quite painful. Besides my to-do list is
already overwhelming. Instead of executing a bunch of SQL in the
database, the guy pulled the data down into individual two-column
objects and joined them using Data::Table.  So not only is the program
doing work in Perl that should be done in the database, I've got six
or seven objects that are essentially just arrays (some thousands of
elements long) that have to stay in sync.

You don't even want to know how this guy decided to write out 15 files
consisting of two columns where only the data in the second column
changes!


--

Regards,

Faber Fedor
Linux New Jersey, Inc.
908-320-0357
http://www.linuxnj.com


-- 

Regards,

Faber Fedor
Linux New Jersey, Inc.
908-320-0357
http://www.linuxnj.com

From faber at linuxnj.com  Tue Oct 17 11:54:18 2006
From: faber at linuxnj.com (Faber J. Fedor)
Date: Tue, 17 Oct 2006 14:54:18 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
Message-ID: <20061017185418.GA5051@neptune.faber.nom>

I've been going over the various docs wrt Tie::Hash and I don't get how
to do what I want to do.

Normally, when I want to add/delete lines from a text file, I Tie::File
the file and pop()/push() the data.

I've got three text files all with similar structure (each line consists
of "string, number\n").  My manipulations (in hash representation) are
along the lines of:

if ($moon_is_full) { deleteFromFile1($string); $file2{$string} = 0;}
if ($tide_is_high) { $file3{$string} = 3; }
etc.

What I would like to do is to load all three files into three separate
hashes and manipulate from there.  I *think* Tie:Hash will do that, but
am I right in thinking I have to write all of my own access methods?

I would have thought someone would have done this (load two-column data
from a file into a hash) already.


-- 
 
Regards,
 
Faber Fedor
President
Linux New Jersey, Inc.
908-320-0357
800-706-0701

http://www.linuxnj.com




From rjbs-perl-abe at lists.manxome.org  Tue Oct 17 12:46:14 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Tue, 17 Oct 2006 15:46:14 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061017185418.GA5051@neptune.faber.nom>
References: <20061017185418.GA5051@neptune.faber.nom>
Message-ID: <20061017194614.GA11914@zodiac.codesimply.com>

* "Faber J. Fedor" <faber at linuxnj.com> [2006-10-17T14:54:18]
> What I would like to do is to load all three files into three separate
> hashes and manipulate from there.  I *think* Tie:Hash will do that, but
> am I right in thinking I have to write all of my own access methods?

You almost certainly don't want to just use Tie::Hash.  It's a base class.

Have you looked at: http://search.cpan.org/~mjd/FlatFile-0.11/FlatFile.pm

-- 
rjbs

From faber at linuxnj.com  Tue Oct 17 13:28:26 2006
From: faber at linuxnj.com (Faber Fedor)
Date: Tue, 17 Oct 2006 16:28:26 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061017194614.GA11914@zodiac.codesimply.com>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
Message-ID: <300ccfa50610171328g4323636dxe2d738e533075901@mail.gmail.com>

On 10/17/06, Ricardo SIGNES <rjbs-perl-abe at lists.manxome.org> wrote:
> * "Faber J. Fedor" <faber at linuxnj.com> [2006-10-17T14:54:18]
> > What I would like to do is to load all three files into three separate
> > hashes and manipulate from there.  I *think* Tie:Hash will do that, but
> > am I right in thinking I have to write all of my own access methods?
>
> You almost certainly don't want to just use Tie::Hash.  It's a base class.

MEaning it's main use is to build your own Tie::Hash class?

> Have you looked at: http://search.cpan.org/~mjd/FlatFile-0.11/FlatFile.pm

No I hadn't.  It looks like it might do the trick.  Thanks!


-- 

Regards,

Faber Fedor
Linux New Jersey, Inc.
908-320-0357
http://www.linuxnj.com

From rjbs-perl-abe at lists.manxome.org  Tue Oct 17 14:06:25 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Tue, 17 Oct 2006 17:06:25 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <300ccfa50610171328g4323636dxe2d738e533075901@mail.gmail.com>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
	<300ccfa50610171328g4323636dxe2d738e533075901@mail.gmail.com>
Message-ID: <20061017210624.GA5455@zodiac.codesimply.com>

* Faber Fedor <faber at linuxnj.com> [2006-10-17T16:28:26]
> On 10/17/06, Ricardo SIGNES <rjbs-perl-abe at lists.manxome.org> wrote:
> >
> >You almost certainly don't want to just use Tie::Hash.  It's a base class.
> 
> MEaning it's main use is to build your own Tie::Hash class?

Yes.

-- 
rjbs

From faber at linuxnj.com  Wed Oct 18 12:13:29 2006
From: faber at linuxnj.com (Faber J. Fedor)
Date: Wed, 18 Oct 2006 15:13:29 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061017194614.GA11914@zodiac.codesimply.com>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
Message-ID: <20061018191329.GA8028@neptune.faber.nom>

On 17/10/06 15:46 -0400, Ricardo SIGNES wrote:
> Have you looked at: http://search.cpan.org/~mjd/FlatFile-0.11/FlatFile.pm

Overall, quite good, but it has a weird flaw: you can only update one
value at a time before needing to flush to disk, otherwise it throws
an error on the second update (Can't call method "cusip" without a
package or object reference at/usr/lib/perl5/site_perl/5.8.0/FlatFile.pm 
line 322, <FH> line 3700).
 
Strange though because looking at the data structure it appears that it's
built to handle multiple updates. :-?


-- 
 
Regards,
 
Faber Fedor
President
Linux New Jersey, Inc.
908-320-0357
800-706-0701

http://www.linuxnj.com




From rjbs-perl-abe at lists.manxome.org  Wed Oct 18 13:33:23 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Wed, 18 Oct 2006 16:33:23 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061018191329.GA8028@neptune.faber.nom>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
	<20061018191329.GA8028@neptune.faber.nom>
Message-ID: <20061018203323.GA28255@zodiac.codesimply.com>

* "Faber J. Fedor" <faber at linuxnj.com> [2006-10-18T15:13:29]
> On 17/10/06 15:46 -0400, Ricardo SIGNES wrote:
> > Have you looked at: http://search.cpan.org/~mjd/FlatFile-0.11/FlatFile.pm
> 
> Overall, quite good, but it has a weird flaw: you can only update one
> value at a time before needing to flush to disk, otherwise it throws

If you can provide a short example that I can run to reproduce this problem, I
can help you fix it.

-- 
rjbs

From faber at linuxnj.com  Wed Oct 18 14:49:24 2006
From: faber at linuxnj.com (Faber J. Fedor)
Date: Wed, 18 Oct 2006 17:49:24 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061018203323.GA28255@zodiac.codesimply.com>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
	<20061018191329.GA8028@neptune.faber.nom>
	<20061018203323.GA28255@zodiac.codesimply.com>
Message-ID: <20061018214924.GA8387@neptune.faber.nom>

On 18/10/06 16:33 -0400, Ricardo SIGNES wrote:
> * "Faber J. Fedor" <faber at linuxnj.com> [2006-10-18T15:13:29]
> > On 17/10/06 15:46 -0400, Ricardo SIGNES wrote:
> > > Have you looked at: http://search.cpan.org/~mjd/FlatFile-0.11/FlatFile.pm
> > 
> > Overall, quite good, but it has a weird flaw: you can only update one
> > value at a time before needing to flush to disk, otherwise it throws
> 
> If you can provide a short example that I can run to reproduce this problem, I
> can help you fix it.

I can do that (see attached files) and I can point you to the place in the
FlatFile.pm where the problem is occuring and why.  I just don't know
how to fix it other than commenting out the offending lines. 

Hopefully the notes at the top of test.pl are sufficient.

TIA!

-- 
 
Regards,
 
Faber Fedor
President
Linux New Jersey, Inc.
908-320-0357
800-706-0701

http://www.linuxnj.com



-------------- next part --------------
95709010, 2
28176010, 2
96087810, foo
62912R10, 2
89840410, 2
-------------- next part --------------
A non-text attachment was scrubbed...
Name: FlatFile.pm
Type: application/x-perl
Size: 19375 bytes
Desc: not available
Url : http://mail.pm.org/pipermail/abe-pm/attachments/20061018/1a9a09b5/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.pl
Type: application/x-perl
Size: 1670 bytes
Desc: not available
Url : http://mail.pm.org/pipermail/abe-pm/attachments/20061018/1a9a09b5/attachment-0003.bin 

From rjbs-perl-abe at lists.manxome.org  Wed Oct 18 16:22:40 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Wed, 18 Oct 2006 19:22:40 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061018214924.GA8387@neptune.faber.nom>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
	<20061018191329.GA8028@neptune.faber.nom>
	<20061018203323.GA28255@zodiac.codesimply.com>
	<20061018214924.GA8387@neptune.faber.nom>
Message-ID: <20061018232239.GA19491@zodiac.codesimply.com>

* "Faber J. Fedor" <faber at linuxnj.com> [2006-10-18T17:49:24]
> On 18/10/06 16:33 -0400, Ricardo SIGNES wrote:
> > 
> > If you can provide a short example that I can run to reproduce this
> > problem, I can help you fix it.
> 
> I can do that (see attached files) and I can point you to the place in the
> FlatFile.pm where the problem is occuring and why.  I just don't know
> how to fix it other than commenting out the offending lines. 

Aha!  Interesting little problem.  I've submitted the bug to MJD, along with a
patch, which I'm also attaching here.

  http://rt.cpan.org/Ticket/Display.html?id=22167

-- 
rjbs
-------------- next part --------------
--- FlatFile.pm.old	2006-10-18 19:10:08.000000000 -0400
+++ FlatFile.pm	2006-10-18 19:08:59.000000000 -0400
@@ -430,15 +430,18 @@
 
   $recno++ while $self->{DELETE}{$recno};
 
-  # Someone may have done an in-memory update of the record
-  # we just read.  If so, discard the disk data and
-  # return the in-memory version of the record instead.
-  return $self->{UPDATE}{$recno}
-    if exists $self->{UPDATE}{$recno};
-
-  # if it wasn't updated, the continue processing
-  # with the disk data
-  my $line = $self->{file}[$recno];
+  my $line;
+  if (exists $self->{UPDATE}{$recno}) {
+    # Someone may have done an in-memory update of the record
+    # we just read.  If so, discard the disk data and
+    # return the in-memory version of the record instead.
+    $line = $self->{UPDATE}{$recno}
+  } else {
+    # if it wasn't updated, the continue processing
+    # with the disk data
+    $line = $self->{file}[$recno];
+  }
+
   return unless defined $line;
   my @data = split $self->{FIELDSEP}, $line, -1;
   $self->{recno} = $recno+1;

From faber at linuxnj.com  Thu Oct 19 06:37:40 2006
From: faber at linuxnj.com (Faber Fedor)
Date: Thu, 19 Oct 2006 09:37:40 -0400
Subject: [ABE.pm] Tie::Hash-ing to files?
In-Reply-To: <20061018232239.GA19491@zodiac.codesimply.com>
References: <20061017185418.GA5051@neptune.faber.nom>
	<20061017194614.GA11914@zodiac.codesimply.com>
	<20061018191329.GA8028@neptune.faber.nom>
	<20061018203323.GA28255@zodiac.codesimply.com>
	<20061018214924.GA8387@neptune.faber.nom>
	<20061018232239.GA19491@zodiac.codesimply.com>
Message-ID: <300ccfa50610190637k7aad2c53o55b40e8a8799cb1@mail.gmail.com>

On 10/18/06, Ricardo SIGNES <rjbs-perl-abe at lists.manxome.org> wrote:
> Aha!  Interesting little problem.  I've submitted the bug to MJD, along with a
> patch, which I'm also attaching here.
>
>   http://rt.cpan.org/Ticket/Display.html?id=22167

Works great!  With that patch and changing a few assumptions, I got
the program down to 6 minutes from 38 minutes.


-- 

Regards,

Faber Fedor
Linux New Jersey, Inc.
908-320-0357
http://www.linuxnj.com

From rjbs-perl-abe at lists.manxome.org  Sat Oct 28 15:05:35 2006
From: rjbs-perl-abe at lists.manxome.org (Ricardo SIGNES)
Date: Sat, 28 Oct 2006 18:05:35 -0400
Subject: [ABE.pm] beer? wednesday? nov 1
Message-ID: <20061028220535.GA28449@zodiac.codesimply.com>


I should have sent this sooner, but I'm in Florida.  I am visiting my in-laws,
so you can imagine that when I arrive back in ABE, I will be ready to drink
beer.  I get back Wednesday afternoon, just in time to take a shower and walk
up to Machts Gut for another good burger and their cheap beer.

Who else is in?

-- 
rjbs

From faber at linuxnj.com  Sat Oct 28 17:12:37 2006
From: faber at linuxnj.com (Faber Fedor)
Date: Sat, 28 Oct 2006 20:12:37 -0400
Subject: [ABE.pm] beer? wednesday? nov 1
In-Reply-To: <20061028220535.GA28449@zodiac.codesimply.com>
References: <20061028220535.GA28449@zodiac.codesimply.com>
Message-ID: <300ccfa50610281712m3311d5a6keaabb3b21b293f67@mail.gmail.com>

I guess I could force myself to drink a few beers.

On 10/28/06, Ricardo SIGNES <rjbs-perl-abe at lists.manxome.org> wrote:
>
> I should have sent this sooner, but I'm in Florida.  I am visiting my in-laws,
> so you can imagine that when I arrive back in ABE, I will be ready to drink
> beer.  I get back Wednesday afternoon, just in time to take a shower and walk
> up to Machts Gut for another good burger and their cheap beer.
>
> Who else is in?
>
> --
> rjbs
> _______________________________________________
> ABE-pm mailing list
> ABE-pm at pm.org
> http://mail.pm.org/mailman/listinfo/abe-pm
>
>


-- 

Regards,

Faber Fedor
Linux New Jersey, Inc.
908-320-0357
http://www.linuxnj.com