[Pdx-pm] corruption puzzlement

[Michael Rasmussen]

Focused question here.  (sigger about the code behind my 
back ok?)

We had text files arriving via scp with Unix style EOL 
characters that would eventually be used by Windows people.
Had to convert the line endings in the files.  So I created
a pair of scripts to handle the task ( unix2dos not available
on the system) 

check2convert runs continuously, sleepign for two minutes and
then checking if there are new files in the directory to muss 
with, if so 2dos is called for each file.

There is a group of files that arrives about 2:00am.  This 
last Monday one of them showed up with 0 size.  Normally this
file (a large one) takes about 40 seconds to transfer between
sites.  

I did some munging (eliminating the sleep and the file time stamp
comparison)  to try and duplicate the truncation.  This 
raised two questions:

  1) since the transfer takes 40 seconds and I loop every 120 seconds
I'd expect to see 2dos trash the file every once in awhile. This hasn't
happened.  Huh??

  2) No matter what I did I couldn't replicate the trucate the file to 0 
bytes behavior.

Huh??  Is this pair of quickies potentially responsible for the 0 byte
file we received earlier this week?  Any ideas on why 2dos doesn't trash
about 1 in 3 of the incoming files where the transfer time would overlap
with the loop invocation?

################# Start of check2convert ############
#!/usr/bin/perl

while (1) {
        $mtime_ref = (stat (".timestamp"))[9];
        $now = time;
        utime $now, $now, ".timestamp";

        @dir = `ls *.txt *.csv`;

        foreach $f (@dir) {
                chomp $f;
                $mtime_cmp = (stat ($f))[9];
                if ( ($mtime_cmp > $mtime_ref) && -f $f )  {
                        $cmd = "./2dos $f";
                        system $cmd ;
                }
        }
sleep 120;
} # while(1)
################## end of check2convert #############

################## start of 2dos ####################
#!/usr/bin/perl -i

# slurp in a file and make it have dos line endings
# be nice if I could do the test non destructively
# open close open???

$eol = "\r\n";

$line = <>;

if ($line =~ /\r\n/) {   $/ = $eol; }

chomp $line;
print "$line$eol";

while(<>) {
        chomp;
        print "$_$eol";
}
############# end of 2dos #############################

--
  Michael Rasmussen, Infrastructure Engineer
  Columbia Management Company, Portland, Oregon
  Michael.Rasmussen at ColumbiaManagement.com 
  Desk:  971-925-6723  
  Desk:  503-973-6723 (deprecated)
  Cell:  503-209-6227

------------------------------------------------------------------------------
NOTICE:  This communication may contain confidential or other privileged information.  If you are not the intended recipient, or believe that you have received this communication in error, please do not print, copy, retransmit, disseminate, or otherwise use the information.  Also, please indicate to the sender that you have received this email in error, and delete the copy you received.  Any communication that does not relate to official Columbia Management Group business is that of the sender and is neither given nor endorsed. 
Thank you. 


==============================================================================

----- End forwarded message -----

-- 
      Michael Rasmussen  aka  mikeraz
    Be appropriate && Follow your curiosity
 http://www.patch.com/ http://wiki.patch.com/
 http://wiki.patch.com/index.php/BicycleCommuting

   The fortune cookie says:
The two most beautiful words in the English language are "Cheque Enclosed."
		-- Dorothy Parker