From richard at rushlogistics.com  Tue Feb  1 15:26:56 2011
From: richard at rushlogistics.com (Richard Reina)
Date: Tue, 01 Feb 2011 18:26:56 -0500 (EST)
Subject: [Chicago-talk] scalar question
In-Reply-To: <20110131214025.8CDC01D3B@alexander.xo.com>
Message-ID: <20110201232656.314EB101@captain.xo.com>

Thank you everyone for all the great replies.  This is what I ended up using.

Thanks again,

Richard
---- Chicago.pm chatter <chicago-talk at pm.org> wrote:
>
> my $fl1 = '3321-1.pdf';
> my $root1;
> ($root1 = $fl1) =~ s/-.*//; # s/// only effects $root1 here
> 
>    - Brian
> 
> On 2011-01-31 15:40, Richard Reina wrote:
> > I know I should no this, but I don't and was wondering if someone could enlighten me.  I have some scalars that represent filenames and look like this
> >
> > my $fl1=3321-1.pdf
> > my $fl2=3321-2.pdf
> > my $fl3=3432-1.pdf
> >
> > I can get everything before the "-" like this:
> > $fl2=~s/\-.*//; # get the root of the filename
> >
> > but how can I do it without changing the contents of the original scalar? I'd like to put the result directly in a new scalar.
> >
> > $root_name= $f12=~s/\-.*//; # get the root of the filename
> >
> > Thanks.
> 
> 
> -- 
> Brian Katzung, Kappa Computer Solutions, LLC
> Leveraging UNIX, GNU/Linux, open source, and custom
> software solutions for business and beyond
> Phone: 877.367.8837 x1  http://www.kappacs.com
> 
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
> 

-- 
Richard Reina
Rush Logistics, Inc.
Watch our 3 minute movie: 
http://www.rushlogistics.com/movie


From richard at rushlogistics.com  Wed Feb  2 05:55:36 2011
From: richard at rushlogistics.com (Richard Reina)
Date: Wed, 02 Feb 2011 08:55:36 -0500 (EST)
Subject: [Chicago-talk] Regular expression discussion.
Message-ID: <20110202135536.EEF3089AD@arkroyal.xo.com>

Tired of shoveling snow. Well sit right down and lets have a regex discussion. I have a perl script that at the moment just uses grep to look though text files that have been converted from pdf2text to see what sort of documents they are.  What I am finding however is that a lot of searches fail by just a few characters. 
For example, if I am looking for "This first document is a contract between" the text string in the file might look like this 
"This tirst document is a coniract betweeo" and the grep search fails. However, as you can see these two statements are 93% alike.  Is there a way with perl regular expressions to match strings that are say 90, 95 or 98% alike?

Any ideas would be greatly appreciated.

Stay Warm!
-- 
Richard Reina
Rush Logistics, Inc.
Watch our 3 minute movie: 
http://www.rushlogistics.com/movie


From warren.lindsey at gmail.com  Wed Feb  2 06:11:44 2011
From: warren.lindsey at gmail.com (Warren Lindsey)
Date: Wed, 2 Feb 2011 08:11:44 -0600
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <20110202135536.EEF3089AD@arkroyal.xo.com>
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
Message-ID: <AANLkTinY2Q+YrFU3t1qvM2+NxrDaWmrOoc_Hh+B8naPS@mail.gmail.com>

Hi Rich,

A soundex search is commonly used to match words that could be
misspelled. However, it targets more phonetic spelling errors instead
of OCR errors.

Google 'perl regex fuzzy search' and check out these two modules:

String::Approx
Regex::Approx

Cheers,
Warren

On Wed, Feb 2, 2011 at 7:55 AM, Richard Reina <richard at rushlogistics.com> wrote:
> Tired of shoveling snow. Well sit right down and lets have a regex discussion. I have a perl script that at the moment just uses grep to look though text files that have been converted from pdf2text to see what sort of documents they are. ?What I am finding however is that a lot of searches fail by just a few characters.
> For example, if I am looking for "This first document is a contract between" the text string in the file might look like this
> "This tirst document is a coniract betweeo" and the grep search fails. However, as you can see these two statements are 93% alike. ?Is there a way with perl regular expressions to match strings that are say 90, 95 or 98% alike?
>
> Any ideas would be greatly appreciated.
>
> Stay Warm!
> --
> Richard Reina
> Rush Logistics, Inc.
> Watch our 3 minute movie:
> http://www.rushlogistics.com/movie
>
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
>

From tigerpeng2001 at yahoo.com  Wed Feb  2 11:14:28 2011
From: tigerpeng2001 at yahoo.com (tiger peng)
Date: Wed, 2 Feb 2011 11:14:28 -0800 (PST)
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <20110202135536.EEF3089AD@arkroyal.xo.com>
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
Message-ID: <213836.81229.qm@web120518.mail.ne1.yahoo.com>

try Text::Soundex



________________________________
From: Richard Reina <richard at rushlogistics.com>
To: chicago-talk at pm.org
Sent: Wed, February 2, 2011 7:55:36 AM
Subject: [Chicago-talk] Regular expression discussion.

Tired of shoveling snow. Well sit right down and lets have a regex discussion. I 
have a perl script that at the moment just uses grep to look though text files 
that have been converted from pdf2text to see what sort of documents they are.  
What I am finding however is that a lot of searches fail by just a few 
characters. 

For example, if I am looking for "This first document is a contract between" the 
text string in the file might look like this 

"This tirst document is a coniract betweeo" and the grep search fails. However, 
as you can see these two statements are 93% alike.  Is there a way with perl 
regular expressions to match strings that are say 90, 95 or 98% alike?

Any ideas would be greatly appreciated.

Stay Warm!
-- 
Richard Reina
Rush Logistics, Inc.
Watch our 3 minute movie: 
http://www.rushlogistics.com/movie

_______________________________________________
Chicago-talk mailing list
Chicago-talk at pm.org
http://mail.pm.org/mailman/listinfo/chicago-talk
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110202/4ff486cd/attachment.html>

From tzz at lifelogs.com  Thu Feb  3 07:44:39 2011
From: tzz at lifelogs.com (Ted Zlatanov)
Date: Thu, 3 Feb 2011 09:44:39 -0600
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <20110202135536.EEF3089AD@arkroyal.xo.com> (Richard Reina's
	message of "Wed, 02 Feb 2011 08:55:36 -0500 (EST)")
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
Message-ID: <87vd11nmvs.fsf@lifelogs.com>

On Wed, 02 Feb 2011 08:55:36 -0500 (EST) Richard Reina <richard at rushlogistics.com> wrote: 

RR> Tired of shoveling snow. Well sit right down and lets have a regex
RR> discussion. I have a perl script that at the moment just uses grep to
RR> look though text files that have been converted from pdf2text to see
RR> what sort of documents they are.  What I am finding however is that a
RR> lot of searches fail by just a few characters.
RR> For example, if I am looking for "This first document is a contract between" the text string in the file might look like this 
RR> "This tirst document is a coniract betweeo" and the grep search
RR> fails. However, as you can see these two statements are 93% alike.  Is
RR> there a way with perl regular expressions to match strings that are
RR> say 90, 95 or 98% alike?

Definitely not with regular expressions.  This is usually called the
string distance; I first learned it in the context of Hamming codes but
there it's only used for substitutions.  String distance turns out a lot
in bioinformatics as well, so there's plenty of research out there.

I would start with String::Approx as Warren suggested and it's the one
I've used, but also see

String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
seems most appropriate to what you're describing);

http://www.perlmonks.org/?node_id=245428

... which suggests Text::Levenshtein and String::Trigram as well.

Ted

From sean at blanton.com  Thu Feb  3 09:51:19 2011
From: sean at blanton.com (Sean Blanton)
Date: Thu, 3 Feb 2011 11:51:19 -0600
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <87vd11nmvs.fsf@lifelogs.com>
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
	<87vd11nmvs.fsf@lifelogs.com>
Message-ID: <AANLkTik6A6DuYRgzEi2daWxU_MjTMAicbew0AnVfcZyn@mail.gmail.com>

> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
> seems most appropriate to what you're describing);

You should be able to create your own "keyboard" map, which is actually a
map of common OCR errors rather than typographical ones. t is near f and i
is near t, o near n, according to your example.

Here is an academic article that might help if you have several months to
spend on this problem:

http://archive.nlm.nih.gov/pubs/hauser/Tompaper/tompaper.php
 <http://archive.nlm.nih.gov/pubs/hauser/Tompaper/tompaper.php>
Regards,
Sean




2011/2/3 Ted Zlatanov <tzz at lifelogs.com>

> On Wed, 02 Feb 2011 08:55:36 -0500 (EST) Richard Reina <
> richard at rushlogistics.com> wrote:
>
> RR> Tired of shoveling snow. Well sit right down and lets have a regex
> RR> discussion. I have a perl script that at the moment just uses grep to
> RR> look though text files that have been converted from pdf2text to see
> RR> what sort of documents they are.  What I am finding however is that a
> RR> lot of searches fail by just a few characters.
> RR> For example, if I am looking for "This first document is a contract
> between" the text string in the file might look like this
> RR> "This tirst document is a coniract betweeo" and the grep search
> RR> fails. However, as you can see these two statements are 93% alike.  Is
> RR> there a way with perl regular expressions to match strings that are
> RR> say 90, 95 or 98% alike?
>
> Definitely not with regular expressions.  This is usually called the
> string distance; I first learned it in the context of Hamming codes but
> there it's only used for substitutions.  String distance turns out a lot
> in bioinformatics as well, so there's plenty of research out there.
>
> I would start with String::Approx as Warren suggested and it's the one
> I've used, but also see
>
> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
> seems most appropriate to what you're describing);
>
> http://www.perlmonks.org/?node_id=245428
>
> ... which suggests Text::Levenshtein and String::Trigram as well.
>
> Ted
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110203/874fbb66/attachment.html>

From michael at potter.name  Thu Feb  3 17:02:51 2011
From: michael at potter.name (Michael Potter)
Date: Thu, 3 Feb 2011 20:02:51 -0500
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <AANLkTik6A6DuYRgzEi2daWxU_MjTMAicbew0AnVfcZyn@mail.gmail.com>
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
	<87vd11nmvs.fsf@lifelogs.com>
	<AANLkTik6A6DuYRgzEi2daWxU_MjTMAicbew0AnVfcZyn@mail.gmail.com>
Message-ID: <AANLkTimbEnCtuqCne=JBb-mCVkYqR=MFzMnm94PJFv6J@mail.gmail.com>

It would be interesting to know if OCR usually gets word boundaries
and character count in each word correct.  if so you might be able to
leverage that in the search.

On Thu, Feb 3, 2011 at 12:51 PM, Sean Blanton <sean at blanton.com> wrote:
>> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
>> seems most appropriate to what you're describing);
> You should be able to create your own "keyboard" map, which is actually a
> map of common OCR errors rather than typographical ones. t is near f and i
> is near t, o near n, according to your example.
> Here is an academic article that might help if you have several months to
> spend on this problem:
> http://archive.nlm.nih.gov/pubs/hauser/Tompaper/tompaper.php
> Regards,
> Sean
>
>
>
>
> 2011/2/3 Ted Zlatanov <tzz at lifelogs.com>
>>
>> On Wed, 02 Feb 2011 08:55:36 -0500 (EST) Richard Reina
>> <richard at rushlogistics.com> wrote:
>>
>> RR> Tired of shoveling snow. Well sit right down and lets have a regex
>> RR> discussion. I have a perl script that at the moment just uses grep to
>> RR> look though text files that have been converted from pdf2text to see
>> RR> what sort of documents they are. ?What I am finding however is that a
>> RR> lot of searches fail by just a few characters.
>> RR> For example, if I am looking for "This first document is a contract
>> between" the text string in the file might look like this
>> RR> "This tirst document is a coniract betweeo" and the grep search
>> RR> fails. However, as you can see these two statements are 93% alike. ?Is
>> RR> there a way with perl regular expressions to match strings that are
>> RR> say 90, 95 or 98% alike?
>>
>> Definitely not with regular expressions. ?This is usually called the
>> string distance; I first learned it in the context of Hamming codes but
>> there it's only used for substitutions. ?String distance turns out a lot
>> in bioinformatics as well, so there's plenty of research out there.
>>
>> I would start with String::Approx as Warren suggested and it's the one
>> I've used, but also see
>>
>> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
>> seems most appropriate to what you're describing);
>>
>> http://www.perlmonks.org/?node_id=245428
>>
>> ... which suggests Text::Levenshtein and String::Trigram as well.
>>
>> Ted
>> _______________________________________________
>> Chicago-talk mailing list
>> Chicago-talk at pm.org
>> http://mail.pm.org/mailman/listinfo/chicago-talk
>
>
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
>



-- 
Michael Potter
Replatform Technologies, LLC
+1 770 815 6142
michael at potter.name

From michael at potter.name  Thu Feb  3 17:07:31 2011
From: michael at potter.name (Michael Potter)
Date: Thu, 3 Feb 2011 20:07:31 -0500
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <AANLkTimbEnCtuqCne=JBb-mCVkYqR=MFzMnm94PJFv6J@mail.gmail.com>
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
	<87vd11nmvs.fsf@lifelogs.com>
	<AANLkTik6A6DuYRgzEi2daWxU_MjTMAicbew0AnVfcZyn@mail.gmail.com>
	<AANLkTimbEnCtuqCne=JBb-mCVkYqR=MFzMnm94PJFv6J@mail.gmail.com>
Message-ID: <AANLkTinbQi+AhbVvUQMuw-GpPyc9uy1BzfHxxUYuoxB4@mail.gmail.com>

One more comment...

How about running the text thru a spell checker.  Using the gmail
spell checker the following:
This tirst document is a coniract betweeo

was corrected to:
This test document is a contract between

I just used the first word that gmail suggested.


On Thu, Feb 3, 2011 at 8:02 PM, Michael Potter <michael at potter.name> wrote:
> It would be interesting to know if OCR usually gets word boundaries
> and character count in each word correct. ?if so you might be able to
> leverage that in the search.
>
> On Thu, Feb 3, 2011 at 12:51 PM, Sean Blanton <sean at blanton.com> wrote:
>>> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
>>> seems most appropriate to what you're describing);
>> You should be able to create your own "keyboard" map, which is actually a
>> map of common OCR errors rather than typographical ones. t is near f and i
>> is near t, o near n, according to your example.
>> Here is an academic article that might help if you have several months to
>> spend on this problem:
>> http://archive.nlm.nih.gov/pubs/hauser/Tompaper/tompaper.php
>> Regards,
>> Sean
>>
>>
>>
>>
>> 2011/2/3 Ted Zlatanov <tzz at lifelogs.com>
>>>
>>> On Wed, 02 Feb 2011 08:55:36 -0500 (EST) Richard Reina
>>> <richard at rushlogistics.com> wrote:
>>>
>>> RR> Tired of shoveling snow. Well sit right down and lets have a regex
>>> RR> discussion. I have a perl script that at the moment just uses grep to
>>> RR> look though text files that have been converted from pdf2text to see
>>> RR> what sort of documents they are. ?What I am finding however is that a
>>> RR> lot of searches fail by just a few characters.
>>> RR> For example, if I am looking for "This first document is a contract
>>> between" the text string in the file might look like this
>>> RR> "This tirst document is a coniract betweeo" and the grep search
>>> RR> fails. However, as you can see these two statements are 93% alike. ?Is
>>> RR> there a way with perl regular expressions to match strings that are
>>> RR> say 90, 95 or 98% alike?
>>>
>>> Definitely not with regular expressions. ?This is usually called the
>>> string distance; I first learned it in the context of Hamming codes but
>>> there it's only used for substitutions. ?String distance turns out a lot
>>> in bioinformatics as well, so there's plenty of research out there.
>>>
>>> I would start with String::Approx as Warren suggested and it's the one
>>> I've used, but also see
>>>
>>> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
>>> seems most appropriate to what you're describing);
>>>
>>> http://www.perlmonks.org/?node_id=245428
>>>
>>> ... which suggests Text::Levenshtein and String::Trigram as well.
>>>
>>> Ted
>>> _______________________________________________
>>> Chicago-talk mailing list
>>> Chicago-talk at pm.org
>>> http://mail.pm.org/mailman/listinfo/chicago-talk
>>
>>
>> _______________________________________________
>> Chicago-talk mailing list
>> Chicago-talk at pm.org
>> http://mail.pm.org/mailman/listinfo/chicago-talk
>>
>
>
>
> --
> Michael Potter
> Replatform Technologies, LLC
> +1 770 815 6142
> michael at potter.name
>



-- 
Michael Potter
Replatform Technologies, LLC
+1 770 815 6142
michael at potter.name

From richard at rushlogistics.com  Thu Feb  3 18:50:30 2011
From: richard at rushlogistics.com (richard at rushlogistics.com)
Date: Fri, 4 Feb 2011 02:50:30 +0000
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <AANLkTimbEnCtuqCne=JBb-mCVkYqR=MFzMnm94PJFv6J@mail.gmail.com>
References: <20110202135536.EEF3089AD@arkroyal.xo.com><87vd11nmvs.fsf@lifelogs.com><AANLkTik6A6DuYRgzEi2daWxU_MjTMAicbew0AnVfcZyn@mail.gmail.com><AANLkTimbEnCtuqCne=JBb-mCVkYqR=MFzMnm94PJFv6J@mail.gmail.com>
Message-ID: <899407038-1296787829-cardhu_decombobulator_blackberry.rim.net-1146712771-@bda125.bisx.prod.on.blackberry>

I think this might be a good idea as character  count and spacing is usually consistent. 
Watch our 3 minute movie: http://www.rushlogistics.com/movie

-----Original Message-----
From: Michael Potter <michael at potter.name>
Sender: chicago-talk-bounces+richard=rushlogistics.com at pm.org
Date: Thu, 3 Feb 2011 20:02:51 
To: Chicago.pm chatter<chicago-talk at pm.org>
Reply-To: "Chicago.pm chatter" <chicago-talk at pm.org>
Subject: Re: [Chicago-talk] Regular expression discussion.

It would be interesting to know if OCR usually gets word boundaries
and character count in each word correct.  if so you might be able to
leverage that in the search.

On Thu, Feb 3, 2011 at 12:51 PM, Sean Blanton <sean at blanton.com> wrote:
>> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
>> seems most appropriate to what you're describing);
> You should be able to create your own "keyboard" map, which is actually a
> map of common OCR errors rather than typographical ones. t is near f and i
> is near t, o near n, according to your example.
> Here is an academic article that might help if you have several months to
> spend on this problem:
> http://archive.nlm.nih.gov/pubs/hauser/Tompaper/tompaper.php
> Regards,
> Sean
>
>
>
>
> 2011/2/3 Ted Zlatanov <tzz at lifelogs.com>
>>
>> On Wed, 02 Feb 2011 08:55:36 -0500 (EST) Richard Reina
>> <richard at rushlogistics.com> wrote:
>>
>> RR> Tired of shoveling snow. Well sit right down and lets have a regex
>> RR> discussion. I have a perl script that at the moment just uses grep to
>> RR> look though text files that have been converted from pdf2text to see
>> RR> what sort of documents they are. ?What I am finding however is that a
>> RR> lot of searches fail by just a few characters.
>> RR> For example, if I am looking for "This first document is a contract
>> between" the text string in the file might look like this
>> RR> "This tirst document is a coniract betweeo" and the grep search
>> RR> fails. However, as you can see these two statements are 93% alike. ?Is
>> RR> there a way with perl regular expressions to match strings that are
>> RR> say 90, 95 or 98% alike?
>>
>> Definitely not with regular expressions. ?This is usually called the
>> string distance; I first learned it in the context of Hamming codes but
>> there it's only used for substitutions. ?String distance turns out a lot
>> in bioinformatics as well, so there's plenty of research out there.
>>
>> I would start with String::Approx as Warren suggested and it's the one
>> I've used, but also see
>>
>> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
>> seems most appropriate to what you're describing);
>>
>> http://www.perlmonks.org/?node_id=245428
>>
>> ... which suggests Text::Levenshtein and String::Trigram as well.
>>
>> Ted
>>_______________________________________________
>> Chicago-talk mailing list
>> Chicago-talk at pm.org
>> http://mail.pm.org/mailman/listinfo/chicago-talk
>
>
>_______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
>



-- 
Michael Potter
Replatform Technologies, LLC
+1 770 815 6142
michael at potter.name
_______________________________________________
Chicago-talk mailing list
Chicago-talk at pm.org
http://mail.pm.org/mailman/listinfo/chicago-talk

From sean at blanton.com  Fri Feb  4 06:18:58 2011
From: sean at blanton.com (Sean Blanton)
Date: Fri, 4 Feb 2011 08:18:58 -0600
Subject: [Chicago-talk] Regular expression discussion.
In-Reply-To: <899407038-1296787829-cardhu_decombobulator_blackberry.rim.net-1146712771-@bda125.bisx.prod.on.blackberry>
References: <20110202135536.EEF3089AD@arkroyal.xo.com>
	<87vd11nmvs.fsf@lifelogs.com>
	<AANLkTik6A6DuYRgzEi2daWxU_MjTMAicbew0AnVfcZyn@mail.gmail.com>
	<AANLkTimbEnCtuqCne=JBb-mCVkYqR=MFzMnm94PJFv6J@mail.gmail.com>
	<899407038-1296787829-cardhu_decombobulator_blackberry.rim.net-1146712771-@bda125.bisx.prod.on.blackberry>
Message-ID: <AANLkTimqMoWq+i-V6vdNy5YAK6Q_=Trp7x4oqH3r-i1X@mail.gmail.com>

>
> How about running the text thru a spell checker.  Using the gmail
> spell checker the following:
> This tirst document is a coniract betweeo
> was corrected to:
> This test document is a contract between
> I just used the first word that gmail suggested.


Is there an API for this? So one could automate the choosing of the first
suggested word?

Regards,
Sean




On Thu, Feb 3, 2011 at 8:50 PM, <richard at rushlogistics.com> wrote:

> I think this might be a good idea as character  count and spacing is
> usually consistent.
> Watch our 3 minute movie: http://www.rushlogistics.com/movie
>
> -----Original Message-----
> From: Michael Potter <michael at potter.name>
> Sender: chicago-talk-bounces+richard=rushlogistics.com at pm.org
> Date: Thu, 3 Feb 2011 20:02:51
> To: Chicago.pm chatter<chicago-talk at pm.org>
> Reply-To: "Chicago.pm chatter" <chicago-talk at pm.org>
> Subject: Re: [Chicago-talk] Regular expression discussion.
>
> It would be interesting to know if OCR usually gets word boundaries
> and character count in each word correct.  if so you might be able to
> leverage that in the search.
>
> On Thu, Feb 3, 2011 at 12:51 PM, Sean Blanton <sean at blanton.com> wrote:
> >> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
> >> seems most appropriate to what you're describing);
> > You should be able to create your own "keyboard" map, which is actually a
> > map of common OCR errors rather than typographical ones. t is near f and
> i
> > is near t, o near n, according to your example.
> > Here is an academic article that might help if you have several months to
> > spend on this problem:
> > http://archive.nlm.nih.gov/pubs/hauser/Tompaper/tompaper.php
> > Regards,
> > Sean
> >
> >
> >
> >
> > 2011/2/3 Ted Zlatanov <tzz at lifelogs.com>
> >>
> >> On Wed, 02 Feb 2011 08:55:36 -0500 (EST) Richard Reina
> >> <richard at rushlogistics.com> wrote:
> >>
> >> RR> Tired of shoveling snow. Well sit right down and lets have a regex
> >> RR> discussion. I have a perl script that at the moment just uses grep
> to
> >> RR> look though text files that have been converted from pdf2text to see
> >> RR> what sort of documents they are.  What I am finding however is that
> a
> >> RR> lot of searches fail by just a few characters.
> >> RR> For example, if I am looking for "This first document is a contract
> >> between" the text string in the file might look like this
> >> RR> "This tirst document is a coniract betweeo" and the grep search
> >> RR> fails. However, as you can see these two statements are 93% alike.
>  Is
> >> RR> there a way with perl regular expressions to match strings that are
> >> RR> say 90, 95 or 98% alike?
> >>
> >> Definitely not with regular expressions.  This is usually called the
> >> string distance; I first learned it in the context of Hamming codes but
> >> there it's only used for substitutions.  String distance turns out a lot
> >> in bioinformatics as well, so there's plenty of research out there.
> >>
> >> I would start with String::Approx as Warren suggested and it's the one
> >> I've used, but also see
> >>
> >> String::KeyboardDistance (which can do QWERTY and Dvorak US layouts, and
> >> seems most appropriate to what you're describing);
> >>
> >> http://www.perlmonks.org/?node_id=245428
> >>
> >> ... which suggests Text::Levenshtein and String::Trigram as well.
> >>
> >> Ted
> >>_______________________________________________
> >> Chicago-talk mailing list
> >> Chicago-talk at pm.org
> >> http://mail.pm.org/mailman/listinfo/chicago-talk
> >
> >
> >_______________________________________________
> > Chicago-talk mailing list
> > Chicago-talk at pm.org
> > http://mail.pm.org/mailman/listinfo/chicago-talk
> >
>
>
>
> --
> Michael Potter
> Replatform Technologies, LLC
> +1 770 815 6142 <tel:+17708156142>
> michael at potter.name
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
> _______________________________________________
> Chicago-talk mailing list
> Chicago-talk at pm.org
> http://mail.pm.org/mailman/listinfo/chicago-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110204/9e636773/attachment-0001.html>

From sean at blanton.com  Mon Feb  7 07:52:27 2011
From: sean at blanton.com (Sean Blanton)
Date: Mon, 7 Feb 2011 09:52:27 -0600
Subject: [Chicago-talk] Undefined subroutine
	Class::MOP::Mixin::HasMethods::wrap/body errors
Message-ID: <AANLkTi=SiY6P3Y6i81bMg2ZfFzrRCxHPUAbEavKLrS1d@mail.gmail.com>

Hi All,

I just upgraded from perl 5.10 to v5.12.2 (Fedora 13->14) and am getting the
following errors in several classes. I'm hoping someone can give me some
high level pointers before I go under the covers to examine the private
parts.

> Undefined subroutine &Class::MOP::Mixin::HasMethods::wrap called at
/usr/local/lib64/perl5/Class/MOP/Mixin/HasMethods.pm line 165.

> Undefined subroutine &Class::MOP::Mixin::HasMethods::body called at
/usr/local/lib64/perl5/Class/MOP/Mixin/HasMethods.pm line 165.

which occurs in:
--------------
sub remove_method {
    my ( $self, $method_name ) = @_;
    ( defined $method_name && length $method_name )
        || confess "You must define a method name";

****->    my $removed_method = delete
$self->_full_method_map->{$method_name};

    $self->remove_package_symbol(
        { sigil => '&', type => 'CODE', name => $method_name } );

--------------

I'm not getting a stack trace, which I feel must somehow be my fault, but I
normally do get a stack trace. Eclipse tells me the parent class where the
error occurs, but it can only give me the message that Perl produces above.

My Moose et al. versions went from:

Moose  0.99 -> 1.17
Moose::Meta::Role 0.99 -> 1.17
Class::MOP 0.98 -> 1.10
Class::MOP::Mixin::HasMethods 0.98 -> 1.10

In one case, I found by removing a method in a role that was overriden by
it's consuming class cleared the ::body method error, but I haven't found
common traits with other classes having the same error.


Regards,
Sean
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110207/ac5e37a3/attachment.html>

From sean at blanton.com  Mon Feb  7 07:57:01 2011
From: sean at blanton.com (Sean Blanton)
Date: Mon, 7 Feb 2011 09:57:01 -0600
Subject: [Chicago-talk] Undefined subroutine
	Class::MOP::Mixin::HasMethods::wrap/body errors
In-Reply-To: <AANLkTi=SiY6P3Y6i81bMg2ZfFzrRCxHPUAbEavKLrS1d@mail.gmail.com>
References: <AANLkTi=SiY6P3Y6i81bMg2ZfFzrRCxHPUAbEavKLrS1d@mail.gmail.com>
Message-ID: <AANLkTimn40puoRxifpSSYikQCUeRCaWYdpNm8+BG_vud@mail.gmail.com>

The errors all come from use and require statements, so they are compile
errors, not runtime. Also getting this one:

Undefined subroutine &Package::DeprecationManager::_HASH called at
/usr/local/share/perl5/Package/DeprecationManager.pm line 18.
BEGIN failed--compilation aborted at
/usr/local/lib64/perl5/Moose/Deprecated.pm line 22.
Compilation failed in require at /usr/local/lib64/perl5/Moose.pm line 14.
BEGIN failed--compilation aborted at /usr/local/lib64/perl5/Moose.pm line
14.
Compilation failed in require at <path>/<class> line 2.


In this, the line 2 at <path>/<class> is 'use Moose;'

Regards,
Sean




On Mon, Feb 7, 2011 at 9:52 AM, Sean Blanton <sean at blanton.com> wrote:

> Hi All,
>
> I just upgraded from perl 5.10 to v5.12.2 (Fedora 13->14) and am getting
> the following errors in several classes. I'm hoping someone can give me some
> high level pointers before I go under the covers to examine the private
> parts.
>
> > Undefined subroutine &Class::MOP::Mixin::HasMethods::wrap called at
> /usr/local/lib64/perl5/Class/MOP/Mixin/HasMethods.pm line 165.
>
> > Undefined subroutine &Class::MOP::Mixin::HasMethods::body called at
> /usr/local/lib64/perl5/Class/MOP/Mixin/HasMethods.pm line 165.
>
> which occurs in:
> --------------
> sub remove_method {
>     my ( $self, $method_name ) = @_;
>     ( defined $method_name && length $method_name )
>         || confess "You must define a method name";
>
> ****->    my $removed_method = delete
> $self->_full_method_map->{$method_name};
>
>     $self->remove_package_symbol(
>         { sigil => '&', type => 'CODE', name => $method_name } );
>
> --------------
>
> I'm not getting a stack trace, which I feel must somehow be my fault, but I
> normally do get a stack trace. Eclipse tells me the parent class where the
> error occurs, but it can only give me the message that Perl produces above.
>
> My Moose et al. versions went from:
>
> Moose  0.99 -> 1.17
> Moose::Meta::Role 0.99 -> 1.17
> Class::MOP 0.98 -> 1.10
> Class::MOP::Mixin::HasMethods 0.98 -> 1.10
>
> In one case, I found by removing a method in a role that was overriden by
> it's consuming class cleared the ::body method error, but I haven't found
> common traits with other classes having the same error.
>
>
> Regards,
> Sean
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110207/f3b7273e/attachment.html>

From sean at blanton.com  Mon Feb  7 09:08:59 2011
From: sean at blanton.com (Sean Blanton)
Date: Mon, 7 Feb 2011 11:08:59 -0600
Subject: [Chicago-talk] Undefined subroutine
	Class::MOP::Mixin::HasMethods::wrap/body errors
In-Reply-To: <AANLkTimn40puoRxifpSSYikQCUeRCaWYdpNm8+BG_vud@mail.gmail.com>
References: <AANLkTi=SiY6P3Y6i81bMg2ZfFzrRCxHPUAbEavKLrS1d@mail.gmail.com>
	<AANLkTimn40puoRxifpSSYikQCUeRCaWYdpNm8+BG_vud@mail.gmail.com>
Message-ID: <AANLkTimKz6a_M7_W=8Ey9nGz8H3BwJbk+0cXX8+Yw55z@mail.gmail.com>

Uh, yeah, these errors are all from duplicate methods...I'm good. There must
be some Devel:: module to help me dump some more information to identify the
cause of these errors...any suggestions?

Regards,
Sean




On Mon, Feb 7, 2011 at 9:57 AM, Sean Blanton <sean at blanton.com> wrote:

> The errors all come from use and require statements, so they are compile
> errors, not runtime. Also getting this one:
>
> Undefined subroutine &Package::DeprecationManager::_HASH called at
> /usr/local/share/perl5/Package/DeprecationManager.pm line 18.
> BEGIN failed--compilation aborted at
> /usr/local/lib64/perl5/Moose/Deprecated.pm line 22.
> Compilation failed in require at /usr/local/lib64/perl5/Moose.pm line 14.
> BEGIN failed--compilation aborted at /usr/local/lib64/perl5/Moose.pm line
> 14.
> Compilation failed in require at <path>/<class> line 2.
>
>
> In this, the line 2 at <path>/<class> is 'use Moose;'
>
> Regards,
> Sean
>
>
>
>
>
> On Mon, Feb 7, 2011 at 9:52 AM, Sean Blanton <sean at blanton.com> wrote:
>
>> Hi All,
>>
>> I just upgraded from perl 5.10 to v5.12.2 (Fedora 13->14) and am getting
>> the following errors in several classes. I'm hoping someone can give me some
>> high level pointers before I go under the covers to examine the private
>> parts.
>>
>> > Undefined subroutine &Class::MOP::Mixin::HasMethods::wrap called at
>> /usr/local/lib64/perl5/Class/MOP/Mixin/HasMethods.pm line 165.
>>
>> > Undefined subroutine &Class::MOP::Mixin::HasMethods::body called at
>> /usr/local/lib64/perl5/Class/MOP/Mixin/HasMethods.pm line 165.
>>
>> which occurs in:
>> --------------
>> sub remove_method {
>>     my ( $self, $method_name ) = @_;
>>     ( defined $method_name && length $method_name )
>>         || confess "You must define a method name";
>>
>> ****->    my $removed_method = delete
>> $self->_full_method_map->{$method_name};
>>
>>     $self->remove_package_symbol(
>>         { sigil => '&', type => 'CODE', name => $method_name } );
>>
>> --------------
>>
>> I'm not getting a stack trace, which I feel must somehow be my fault, but
>> I normally do get a stack trace. Eclipse tells me the parent class where the
>> error occurs, but it can only give me the message that Perl produces above.
>>
>> My Moose et al. versions went from:
>>
>> Moose  0.99 -> 1.17
>> Moose::Meta::Role 0.99 -> 1.17
>> Class::MOP 0.98 -> 1.10
>> Class::MOP::Mixin::HasMethods 0.98 -> 1.10
>>
>> In one case, I found by removing a method in a role that was overriden by
>> it's consuming class cleared the ::body method error, but I haven't found
>> common traits with other classes having the same error.
>>
>>
>> Regards,
>> Sean
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110207/d1602126/attachment.html>

From zrusilla at mac.com  Sun Feb 13 22:38:08 2011
From: zrusilla at mac.com (Elizabeth Cortell)
Date: Mon, 14 Feb 2011 07:38:08 +0100
Subject: [Chicago-talk] Be my Perl valentine
Message-ID: <4D58CDD0.90600@mac.com>

Camel loves you:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110214/6bb11576/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 59299192.jpeg
Type: image/jpeg
Size: 94131 bytes
Desc: not available
URL: <http://mail.pm.org/pipermail/chicago-talk/attachments/20110214/6bb11576/attachment-0001.jpeg>