[Za-pm] flat file test script

Walter Kruse WKruse at multichoice.co.za
Wed Aug 13 03:48:45 CDT 2003


Hi Mark
all I changed was all the @char's to $char's (I see I still have a lot to
learn), but the rest works 100% for me as is. No compile errors. I will post
the final script just for interest's sake when I am done.
Thank you very much for your help.
krgds
Walter Kruse

-----Original Message-----
From: Mark Hewitt [mailto:mh2 at isis.co.za]
Sent: 13 August 2003 08:03
To: 'za-pm at mail.pm.org'
Cc: 'WKruse at multichoice.co.za'
Subject: RE: [Za-pm] flat file test script


There seems to be a syntax error though, at least in Perl 5.8?
See comments in the body....

	>>-----Original Message-----
	>>vFrom:	Walter Kruse [SMTP:WKruse at multichoice.co.za]
	>>Sent:	13 August, 2003 07:47
	>>To:	Mark Hewitt; za-pm at mail.pm.org
	>>Subject:	RE: [Za-pm] flat file test script

	>>Hi Mark
	>>sorry, I did not make myself very clear. I don't know if everyone
that posts
	>>get a message saying the moderator has to approve the post
(apparently I am
	>>not a member), hence the post was received so late.

>  No problem
> 
	>> I am simulating the import of the file into Oracle, in order to
compare my
	>>2 new files with what the tables in Oracle are populated with.
This is to
	>>see that the program (under test) that splits the file and
populates the
	>>tables, did its work correctly. Little complicated. Anyhoo, I now
have a
	>>working hack that is good enough for me to use. Here it is:

>  
	>>#!/usr/bin/perl
	>>open (FILE, "testfile\.txt") || die "Can't open input file !\n";
	>>@file = <FILE>;
>  
	>>open (FILEONE, ">fileone\.txt") || die "Can't create file one
!\n";
	>>open (FILETWO, ">filetwo\.txt") || die "Can't create file two
!\n";
>  
	>>my($record) = split(/\n/, @file);
	You can remove this line, it does nothing, because the <FILE>
already split the file by newlines, and the result of the split
	is being stored in a scalar anyways, hence all lines except first
would be lost.

	>>foreach $record (@file)
	>>{
	>>  my(@char) = split(//, $record);

	>>  foreach $char ($record)
	This is odd to me? $record contains the text of that particular line
of the file we are in? It is not even an array?
	I would actually guess the foreach is not required, as the code
below works on fixed indexes into @char, hence making
	no use of the fact that we are in a loop?

	>>  {
	>>   print FILEONE
("@char[0]","~","@char[2]","~","@char[4]@char[6]","~","@char[8]@char[10]@cha
r[12]","~","@char[14]\n");
	>>   print FILETWO
("@char[1]@char[3]@char[5]","~","@char[7]","~","@char[9]@char[11]","~","@cha
r[13]@char[15]\n");
	Odd syntax? @char[0]? I would have thought the normal array indexer
would be better? 
	print FILEONE
("$char[0]","~","$char[2]","~","$char[4]$char[6]","~","$char[8]$char[10]$cha
r[12]","~","$char[14]\n");

	It also fixed the syntax error:
	>>D:\doc_root>perl -c test.pl
	>>In string, @cha now must be written as \@cha at test.pl line 15,
near "]@cha"
	>>test.pl had compilation errors..

	>>  }
	>>}
	>>close FILEONE;
	>>close FILETWO;
	>>close FILE;
	>>print ("Done !\n\a") && die;

	Thanks,
	Mark 


	>>krgds
	>>Walter Kruse

> -----Original Message-----
> From: Mark Hewitt [mailto:mh2 at isis.co.za]
> Sent: 12 August 2003 11:17
> To: za-pm at mail.pm.org
> Subject: Re: [Za-pm] flat file test script
> 
> 
> Hi,
>  
> Its pretty late (after 23:00!!) and I still have alot of work to complete
> before tomorrow, so I'll just make breif comments, I'll try respond in
> more
> detail tomorrow if I get a free moment. Pls find my comments in the
> original
> text...
>  
> 
> >>----- Original Message ----- 
> >>From: Walter  <mailto:WKruse at multichoice.co.za> Kruse 
> >>To: za-pm at mail.pm.org <mailto:za-pm at mail.pm.org>  
> >>Sent: Tuesday, August 12, 2003 1:58 PM
> >>Subject: [Za-pm] flat file test script
> 
>  >>Hi everyone - hope someone will help a newbie
> >>I need to simulate the importing of a flat file into two tables in a db
> for test purposes. The file has very long records, which aren't separated
> into fields >>by any character. Rather, the program under test counts
> characters. So chars 1-8 go in table 1, chars 9-12 to table 2, chars 13-40
> to table 1 again and so 
> >>on. I want to use a perl script as the records are very long (+/- 88
> fields) to create two new text files and sort the records between the two
> files as I 
>  
> Okay, a couple of points:
> 1. The example above puzzles me, as I would have expected the umbers of
> chars goin into each table to be the same each time? Or do you have 88
> fields be "record", where the 88 get split between the 2 dbs, after
> reading
> all 88, the script can now process "record 2" in flat file?
> 2. What do you mean by "db"? Another flat file, or a real SQL database?
>  
> >>specify. The problem, I am faced with, however, is the counting and
> sorting of the chars. I don't have a clue. This is as far as I got:
>  
> Try substr or unpack.
> >From your desription, many fields of specified length this sounds like a
> job
> for unpack.
> The Perl CookBook has a very useful function called cut2fmt() or
> something,
> it takes "cut-off" values as parameters, and returns the unpack format
> string required to unpack in that format. In your case, the unpack would
> begin like:
>  
> @a = unpack( "A8 A4 A27");
>  
> You may want to generate this string programatically is you have 88 fields
> to go in the unpack!
>  
> >>.#!/usr/bin/perl
> >>open (FILE, "testfile") || die "Can't open file !\n";
> >>@file = <FILE>;
> 
> >>my($record) = split(/\n/, @file);
>  
> This is not needed, you destroy the content on $record in the next
> statement, and the entire file content is already broken into lines by the
> @file = <file> statement.
> 
> >>foreach $record (@file)
> >>{
> 
> >>  my($char) = split(//, $record);
> 
> # You'll want to use substr/unpack here, followed by the write to the
> database
>  
> @a = unpack( "A8 A4 A27");
>  
>  
> # this loop iterates over all parameters in the last extraction and writes
> them alternatly (as you showed in ur example) to the databases
> # its scalable like this and will handle any  number of fields in the
> unpack
> token
> $i = 0;
> foreach $a (@a)
> {
>     if ( $i++ & 1 )
>     {
>         # add $a to database 1
>     }
>     else
>     {
>         # add $a to database 2
>     }
> } 
> 
> >>}
> >>close FILE;
>  
>  
> Does this help you at all?
>  
> Thanks,
> Mark
> --------------------------------------------------------------------------
> --
> Windows, Linux and Internet Development Consultant
> Email: corporate at scriptsmiths.com <mailto:corporate at scriptsmiths.com> 
> Web: http://www.scriptsmiths.com <http://www.scriptsmiths.com> 
> Cell: +27 82 9655295
> --------------------------------------------------------------------------
> -
>  
>  
> >>Kind Regards
> >>Walter Kruse 
> >>MultiChoice Information Technology 
> >>Software test analyst - Application Delivery and Support Team 
> >>Tel:    +27 (0) 11 289-4552 
> >>Cell:   +27 (0) 82 660 7288 
> >>Email:  walterk at multichoice.co.za 
> >>Personal web page: http://www.ou-ryperd.net <http://www.ou-ryperd.net/>
> 
>  
> >>************************************************************************
> **
> *******************************************
> >>Everything in this e-mail and attachments relating to the official
> business of MultiChoice Africa is proprietary to the company. Any >>view
> or
> opinion expressed in this message may be the view of the individual and
> should not automatically be ascribed to the >>company.  If you are not the
> intended recipient, you may not peruse, use, disseminate, distribute or
> copy
> this message. If you have >>received this message in error, please notify
> the sender immediately by email, facsimile or telephone and destroy the
> original >>message.
> >>************************************************************************
> **
> *******************************************
> 
> 
> 
>   _____  
> 
> 
> 
> 
> _______________________________________________
> 
>  
> 
> 
> Za-pm mailing list
> Za-pm at mail.pm.org
> http://mail.pm.org/mailman/listinfo/za-pm
> 
> 
> 
> **************************************************************************
> ************************************************
> Everything in this e-mail and attachments relating to the official
> business of MultiChoice Africa is proprietary to 
> the company. Any view or opinion expressed in this message may be the view
> of the individual and should not automatically 
> be ascribed to the company.  If you are not the intended recipient, you
> may not peruse, use, disseminate, distribute or 
> copy this message. If you have received this message in error, please
> notify the sender immediately by email, facsimile 
> or telephone and destroy the original message.
> **************************************************************************
> ************************************************ << File: ATT00042.htm >> 
"DISCLAIMER: This e-mail and its attachments may contain information that is
confidential and that may be subject to legal privilege and copyright. If
you are not the intended recipient you may not peruse, use, disclose,
distribute, copy or retain this message. If you have received this message
in error, please notify the sender immediately by e-mail, facsimile or
telephone and return and thereafter destroy the original message.Please note
that e-mails are subject to viruses, data corruption, delay, interception
and unauthorised amendment, and that the sender does not accept liability
for any damages that may be incurred as a result of communication by e-mail.
No employee or intermediary is authorised to conclude a binding agreement on
behalf of the sender by e-mail without express written confirmation by a
duly authorised representative of the sender. By transmitting this e-mail
message over the Internet the sender does not intend to allow the contents
hereof to become part of the public domain, and the confidential nature of
the contents shall not be altered or diminished from by such transmission."

**************************************************************************************************************************
Everything in this e-mail and attachments relating to the official business of MultiChoice Africa is proprietary to 
the company. Any view or opinion expressed in this message may be the view of the individual and should not automatically 
be ascribed to the company.  If you are not the intended recipient, you may not peruse, use, disseminate, distribute or 
copy this message. If you have received this message in error, please notify the sender immediately by email, facsimile 
or telephone and destroy the original message.
**************************************************************************************************************************



More information about the Za-pm mailing list