file timestamps and removal [was Re: SPUG:DOS / BASIC Script]

[Doug Beaver]

On Fri, Jan 17, 2003 at 04:07:19PM -0800, Michael R. Wolf wrote:
>
> I've done a similar thing, using Unix dates (useful for scripts) and
> also human readable dates encoded in the filename.
> 
> How 'bout something like this (no warranty, completly untested..):
> 
> # Name, create, use, close, and touch(1) a log file.
> 
> my $unix_now = time;
> my $human_now = local_time($unix_now);
> 
> my $file_name = sprintf("DB_trailings.%d.%s.txt", $unix_now, $human_now);

i want to share a few comments about log names...

i think that putting localtime in the log name is messy.  i don't know
how dos feels about spaces in filenames, but it's annoying in unix.
your naming scheme makes it difficult to sling around log files in a
shell.  say you want to find the log for 9am on 2/15.  you'd have to do
a directory listing and filter out the entries for february, searching
for 'Feb' in order to find the proper matches.  then you'd take the
epoch that matched your date and use that to tab complete the filename.
or, if you put the date first, you'd have to tab complete the string
form of the date, but you'd have to know which day the log was made on,
Sat<tab>Feb<tab>15<tab>09<tab>.

that scheme can get tedious once you have a lot of lot of logs.  i
suggest that you use a descending date like '20020215-090105'.  if you
ascii sort it, it's already in chronological order.  it's easy for
humans to tab complete in their shell, and it doesn't have any spaces in
it.

i included a code snippet below that shows how to create these types of
log names.

i also wanted to point out a difference in philosophy when it comes to
reaping old log files.  you're expiring logs based on the timestamp in
the filename, which maps to the creation time of the log.  that's one
way to do it, but i prefer to expire the log after the last piece of
data was written to it.  if i want to archive logs 24 hours after they
are written, i want to archive 24 hours after the last piece of data was
written, not the first piece of data, because i have no idea how long
the log was open for.  if the log stays open long enough, i might end up
expiring it while it's still being written to.  using the last write
time has a nice side benefit, you don't have to parse the name of the
log to figure out how to reap it, you just stat it and add the
appropriate number of seconds to its mtime.  it's a small distinction
and mostly philosophical, but i thought i'd provide a different
viewpoint on that.

doug


#!/usr/bin/env perl

use strict;
use Date::Format;

print "BEGIN\n";
printLog("hello-world1", time - 60);
printLog("hello-world2", time - 60, "batch.log");
printLog("hello-world3");
printLog("hello-world4", "loggg");
print "END\n";

sub printLog {
  my $name = getLogName(@_);
  print "$name\n";
  sleep 1;
}

sub getLogName {
  my ($prefix, $epoch, $suffix) = @_;
  return unless $prefix;
  if ($epoch && $epoch !~ /^\d+$/) {
    $suffix = $epoch;
    $epoch = time;
  }
  $suffix ||= 'log';
  $epoch ||= time;

  my $timestamp = time2str("%Y%m%d-%H%M%S", $epoch);
  $timestamp ||= $epoch;
  my $name = "$prefix.$timestamp.$suffix";
  return $name;
}
<ctrl-d/z>
BEGIN
hello-world1.20030118-122350.log
hello-world2.20030118-122351.batch.log
hello-world3.20030118-122452.log
hello-world4.20030118-122453.loggg
END

> open my $fh, $file_name or die "problem with trailings file $file_name: $!";
> #use $fh...
> close $fh;
> 
> utime ($unix_now)x2, $file_name;  # To make name consistant with timestamp.
> 
> #Then remove all old ones.
> 
> $now = time;
> $threshhold = $now - 14*(24*60*60);
> 
> @too_old = grep { /^DB_trailings\.(\d+)\./ && $1 > $threshhold }
>                 glob("DB_trailings.*.*.txt");
> 
> @unsucessful = unlink(@too_old);
> warn "could not unlink @unsucessful: $!" if @unsucessful;
> 
> -- 
> Michael R. Wolf
>     All mammals learn by playing!
>         MichaelRunningWolf at att.net
> 
> _____________________________________________________________
> Seattle Perl Users Group Mailing List  
> POST TO: spug-list at mail.pm.org
> ACCOUNT CONFIG: http://mail.pm.org/mailman/listinfo/spug-list
> MEETINGS: 3rd Tuesdays, U-District, Seattle WA
> WEB PAGE: www.seattleperl.org
> 

-- 
Leela: Come on, Fry, walk like a robot!
  Fry: I can't.  I have to go to the bathroom.
Leela: Robots don't have bathrooms.
  Fry: <sarcastically> Oh, riiight.  I wonder where they smoke in high school.