[Za-pm] flat file test script

Wed Aug 13 00:47:21 CDT 2003

Hi Mark
sorry, I did not make myself very clear. I don't know if everyone that posts
get a message saying the moderator has to approve the post (apparently I am
not a member), hence the post was received so late.

 I am simulating the import of the file into Oracle, in order to compare my
2 new files with what the tables in Oracle are populated with. This is to
see that the program (under test) that splits the file and populates the
tables, did its work correctly. Little complicated. Anyhoo, I now have a
working hack that is good enough for me to use. Here it is:

#!/usr/bin/perl
open (FILE, "testfile\.txt") || die "Can't open input file !\n";
@file = <FILE>;

open (FILEONE, ">fileone\.txt") || die "Can't create file one !\n";
open (FILETWO, ">filetwo\.txt") || die "Can't create file two !\n";

my($record) = split(/\n/, @file);
foreach $record (@file)
{
  my(@char) = split(//, $record);
  foreach $char ($record)
  {
   print FILEONE
("@char[0]","~","@char[2]","~","@char[4]@char[6]","~","@char[8]@char[10]@cha
r[12]","~","@char[14]\n");
   print FILETWO
("@char[1]@char[3]@char[5]","~","@char[7]","~","@char[9]@char[11]","~","@cha
r[13]@char[15]\n");
  }
}
close FILEONE;
close FILETWO;
close FILE;
print ("Done !\n\a") && die;

krgds
Walter Kruse

-----Original Message-----
From: Mark Hewitt [mailto:mh2 at isis.co.za]
Sent: 12 August 2003 11:17
To: za-pm at mail.pm.org
Subject: Re: [Za-pm] flat file test script

Hi,

Its pretty late (after 23:00!!) and I still have alot of work to complete
before tomorrow, so I'll just make breif comments, I'll try respond in more
detail tomorrow if I get a free moment. Pls find my comments in the original
text...

>>----- Original Message ----- 
>>From: Walter  <mailto:WKruse at multichoice.co.za> Kruse 
>>To: za-pm at mail.pm.org <mailto:za-pm at mail.pm.org>  
>>Sent: Tuesday, August 12, 2003 1:58 PM
>>Subject: [Za-pm] flat file test script

 >>Hi everyone - hope someone will help a newbie
>>I need to simulate the importing of a flat file into two tables in a db
for test purposes. The file has very long records, which aren't separated
into fields >>by any character. Rather, the program under test counts
characters. So chars 1-8 go in table 1, chars 9-12 to table 2, chars 13-40
to table 1 again and so 
>>on. I want to use a perl script as the records are very long (+/- 88
fields) to create two new text files and sort the records between the two
files as I 

Okay, a couple of points:
1. The example above puzzles me, as I would have expected the umbers of
chars goin into each table to be the same each time? Or do you have 88
fields be "record", where the 88 get split between the 2 dbs, after reading
all 88, the script can now process "record 2" in flat file?
2. What do you mean by "db"? Another flat file, or a real SQL database?

>>specify. The problem, I am faced with, however, is the counting and
sorting of the chars. I don't have a clue. This is as far as I got:

Try substr or unpack.
>From your desription, many fields of specified length this sounds like a job
for unpack.
The Perl CookBook has a very useful function called cut2fmt() or something,
it takes "cut-off" values as parameters, and returns the unpack format
string required to unpack in that format. In your case, the unpack would
begin like:

@a = unpack( "A8 A4 A27");

You may want to generate this string programatically is you have 88 fields
to go in the unpack!

>>.#!/usr/bin/perl
>>open (FILE, "testfile") || die "Can't open file !\n";
>>@file = <FILE>;

>>my($record) = split(/\n/, @file);

This is not needed, you destroy the content on $record in the next
statement, and the entire file content is already broken into lines by the
@file = <file> statement.

>>foreach $record (@file)
>>{

>>  my($char) = split(//, $record);

# You'll want to use substr/unpack here, followed by the write to the
database

@a = unpack( "A8 A4 A27");

# this loop iterates over all parameters in the last extraction and writes
them alternatly (as you showed in ur example) to the databases
# its scalable like this and will handle any  number of fields in the unpack
token
$i = 0;
foreach $a (@a)
{
    if ( $i++ & 1 )
    {
        # add $a to database 1
    }
    else
    {
        # add $a to database 2
    }
} 

>>}
>>close FILE;

Does this help you at all?

Thanks,
Mark
----------------------------------------------------------------------------
Windows, Linux and Internet Development Consultant
Email: corporate at scriptsmiths.com <mailto:corporate at scriptsmiths.com> 
Web: http://www.scriptsmiths.com <http://www.scriptsmiths.com> 
Cell: +27 82 9655295
---------------------------------------------------------------------------

>>Kind Regards
>>Walter Kruse 
>>MultiChoice Information Technology 
>>Software test analyst - Application Delivery and Support Team 
>>Tel:    +27 (0) 11 289-4552 
>>Cell:   +27 (0) 82 660 7288 
>>Email:  walterk at multichoice.co.za 
>>Personal web page: http://www.ou-ryperd.net <http://www.ou-ryperd.net/>  

>>**************************************************************************
*******************************************
>>Everything in this e-mail and attachments relating to the official
business of MultiChoice Africa is proprietary to the company. Any >>view or
opinion expressed in this message may be the view of the individual and
should not automatically be ascribed to the >>company.  If you are not the
intended recipient, you may not peruse, use, disseminate, distribute or copy
this message. If you have >>received this message in error, please notify
the sender immediately by email, facsimile or telephone and destroy the
original >>message.
>>**************************************************************************
*******************************************

  _____  

_______________________________________________

Za-pm mailing list
Za-pm at mail.pm.org
http://mail.pm.org/mailman/listinfo/za-pm

**************************************************************************************************************************
Everything in this e-mail and attachments relating to the official business of MultiChoice Africa is proprietary to 
the company. Any view or opinion expressed in this message may be the view of the individual and should not automatically 
be ascribed to the company.  If you are not the intended recipient, you may not peruse, use, disseminate, distribute or 
copy this message. If you have received this message in error, please notify the sender immediately by email, facsimile 
or telephone and destroy the original message.
**************************************************************************************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.pm.org/pipermail/za-pm/attachments/20030813/d15e9b69/attachment.htm