[Za-pm] introduction

aesop at fables.co.za aesop at fables.co.za
Wed Apr 9 07:38:43 PDT 2008


Hello,

Just introducing myself with a small problem. Not a very perlish one 
although that is likely to change.

I have a flat file database (Clue) running on DOS still! 20 years use 
but am forced to migrate now.

Over the years I added some new fields into the database, and then 
later I moved one of them to put it in a better place. The new fields 
only get added into new records, there is no way to add the field 
into old records which can only be updated by pulling out on an edit 
and then resaving. So I have some records with the same field in 
different places, and older records without the new fields at all. 
Doesn't faze it.

Perl to the rescue. Although new to Perl I have got a number of 
routines up and running to fix these sort of inconsistencies in the 
database. I'll isolate this to two routines.

The whole database comprises 161 sequential files at present. NO CR 
or LF. Clue handles all formatting by an Alt-127 (little house) 
character followed by a letter/s. Open the file in the Midnite 
Commander editor and you have a one liner wandering 30+k off to the 
right.

One routine moves a field and its data in the record, another adds 
another field in if it doesn't exist already. I do this under Linux. 
The problem is after running the routines, reported successfully by 
various Perl test routines (like 6699 records, 6699 of each field, 
all fields in the correct order), we get problems when the database 
is imported back into Clue - like will not count the records 
correctly under certain circumstances. Strictly speaking the move 
routine causes no problem, it is the add-in one that does.

I have spent hours checking and refining the Perl routines, trying 
variants in methodology, more hours spent methodically checking by 
hand the modified Clue files, all to no avail. I can see nothing 
wrong but of course there must be. I run them under DOS, I modify 
them in Linux under Perl 5.8, I copy them back onto the DOS machine, 
and then it goes wrong.  

So I don't think the question is necessarily a Perl one at present, 
but it is this. Can anything happen to the files (that don't have CRs 
or LFs in them for Perl to handle in any way and we slurp in a file 
at a time as one string to process  ~= m/regex/replace/gx) in the 
transition from one system to another? 

I need to have that clear in my mind, that it is not an operating 
system thing before I stare at my Perl scripts yet again. It can only 
be something in the files that has been changed but yet ....

A couple of good answers can give direction to my search for the 
problem, and thanks in advance.

Thanks
Anne----
Anne Wainwright



More information about the Za-pm mailing list