[Pdx-pm] Newbie question about testing and Perl

[Eric Wilhelm]

# from Keith Lofstrom
# on Thursday 13 July 2006 06:42 pm:

>Hence my surprise on Wednesday night, when Eric did not turn off
>the real serial port and turn on the emulated one, and the question
>that started this thread.

Do I have an emulated one?

Let's start with something simpler:  like files.  How are you testing 
Dirvish with an emulated filesystem?

But, I think more on-point to what you're saying is that we don't have a 
really good infrastructure for automated machine-level (or macro-level) 
testing in Perl -- most of the focus has been on unit tests.  (To be 
fair, most automated test infrastructures (AFAIK) really only focus on 
unit tests.)  And really, we only need one such (cross-platform) beast, 
since all you really need to do is launch the test program in a sandbox 
and verify that the right changes to the environment happen.

The simplest and most accurate way to do this is to fork the universe 
with a copy-on-write strategy and pipe the state of the child universe 
up to the parent before killing the child (thus, smoking hardware 
and/or money in the parallel universe is ok.)  I'm not sure if the Perl 
community has the advanced physics mindshare needed for that, so we'll 
probably end up with something more approximate (and therefore 
fallible.)

As for the general case unit-testing vs macro testing, consider the 
difference in structure between these two programs:

---
# program 1
print "hello world\n";
---
# program 2
#!/usr/bin/perl

# Copyright (C) 2006 Eric L. Wilhelm

use warnings;
use strict;

=head1 NAME

hello - greets the world

=cut

package bin::hello;

use Getopt::Helpful;

sub main {
  my (@args) = @_;
  my $hopt = Getopt::Helpful->new(
    usage => 'CALLER [options]',
    '+help',
    );
  $hopt->Get_from(\@args);
  # do something with $hopt->opts or what's left of @args
  # (like maybe assign a filehandle)
  my $fh = \*STDOUT;
  print_hello($fh);
}

sub print_hello {
  my ($fh) = @_;
  print $fh "hello world\n";
}

package main;

if($0 eq ($ENV{PAR_ARGV_0} || __FILE__)) {
  bin::hello::main(@ARGV);
}

# vi:ts=2:sw=2:et:sta
my $package = 'bin::hello';
---

The latter does the same as the former (for now) but is ready to be 
gracefully expanded into a large system without turning into spaghetti 
code on day 2.

Here's what happens to the first program if you're not careful.

---
# program 1
if(@ARGV) {
  $thing_to_greet = shift(@ARGV);
}
if(@ARGV) {
  $how_to_greet = shift(@ARGV);
}
if(@ARGV) {
  $where_to_greet = shift(@ARGV);
}
if($where_to_greet) {
  open(FILE, ">$where_to_greet");
  print FILE "hello world\n";
}
if($how_to_greet) {
  print "$how_to_greet world";
  # maybe throw in a goto while you're at it, right?
}
if($thing_to_greet) {
  print "hellow $thing_to_greet\n";
}
print "hello world\n";
---

I call "not it!".  How do you test that?  I can tell you how to test my 
original "well-formed" example:

---
use Test::More 'no_plan';
my $package = require("./bin/hello");
my $prints = eval("\\&$package}::print_hello");
my $string;
open(my $fh, '>', \$string);
$prints->($fh);
like($string, qr/hello/);
---

Sure, it needs more tests.  It also needs more features.  If you wanted, 
you could even develop your little scripts in test-first mode.  I don't 
recommend that in most cases.  In fact, I don't recommend doing the 
above "my $prints = ..." junk either.  But at least you can gracefully 
test it, and if you define a package in your script, you can test the 
package.  Plus, when you're ready for modules, you just cut and paste 
and twiddle a few names, add a bit of "use <this and that>", etc.

Even as it is, we still need to test bin::hello::main() to get full 
coverage.  An alternative is to break it into smaller pieces, but  if 
you're talking to a network or etc., you'll still never get complete 
coverage until you have a (correctly) fully emulated environment.

Back to the hackjob example:  Once the code grows into a tangled mess, 
how do you refactor it and know that it does the same thing as before? 
(or at least, know that what is does differently is because you meant 
it to be that way.)  How do you check code coverage on it?

--Eric
-- 
"Beware of bugs in the above code; I have only proved it correct, not
tried it."
--Donald Knuth
---------------------------------------------------
    http://scratchcomputing.com
---------------------------------------------------