[Pdx-pm] Test::Builder calling convention

[Michael G Schwern]

At the last PDX.pm meeting we had a breakthrough on how a testing library says
that they've run a test.

Right now, with Test::Builder, you do this:

    my $ok = $builder->ok( $test, $name );
    unless( $ok ) {
        my $diag = ...;
        ...build up the diagnostic message...
        $builder->diag( $diag );
    }

    return $ok;

Or you can sometimes compress it to this:

    return $builder->ok( $test, $name ) ||
      $builder->diag( $diag );

With structured diagnostics, we need to A) strongly associated the diagnostics
with the test and B) allow an arbitrary hash.  This lead to the unsatisfactory:

    $builder->ok( $test, $name, {
        have => $have,
        want => $want
    });

I never liked this because it seems inflexible.  You have to have everything
ready in one shot.  And if we have is() or like() or more assert methods they
have to accept all the arguments, too.  After some discussion we came up with
this:

    return $builder->ok( $test )
            ->name( $name )
            ->diag({
                have => $have,
                want => $want
            });

ok() returns a result object upon which you can call more methods to add more
information about the result.  The test name, test diagnostics or whatever
else we want to tack on.  Each of those return the same result object to allow
chaining.  In boolean/string/numeric context it will return true or false to
reflect the test passing or failing.

You can do it all in one shot, chained together, as above.  Or you can do it
in pieces like this:

    my $ok = $builder->ok($test)->name($name);

    if( !$ok ) {
        my $diag = ...;
        ...build up $diag...
        $ok->diag($diag);
    }

    return $ok;

The result object will have a flush() method (better name needed) to output
the results.  Normally this will be called by the wrapper around the user's
test function, but I don't want to rely on that 100% of the time, not everyone
is going to use install_test().

The original thought was to have the object flush on destruction, but we're
going to want to store the result object in the TB2 history (ok() will do
that) so it won't actually get destroyed.  We can't store a copy because we
want to see any changes after it's stored.  Can't store a weak ref because we
do want to keep the object around.

My thought is for ok() to store the real result object in history and return a
thin wrapper object that does nothing but delegates everything to the result.
 Then when it gets destroyed it can tell the real result object to flush.

In the end, I really like the flexibility the object chaining gives.  Its
going to make it very easy to add new methods.  I don't like the complexity
and magic of determining when the result should output and would like to see
and good ideas on that.

As always, +1 and -1s appreciated.


PS  At PDX.pm we talked about an or() method which would do this:

    $builder->ok( open my $fh, $file )
            ->diag( "Errno: $!" )
            ->or
            ->is( $!, ENOBACON )
            ->name( "Insufficient bacon" );

Its a short circuit method.  If the result is true, it returns a null object
that does nothing.  If it's false, it chains the result through and the
following is() starts a new result object.  Trouble is, I can't remember what
it bought us over a regular or operator.

    $builder->ok( open my $fh, $file )
            ->diag( "Errno: $!" )
      or
    $builder->is( $!, ENOBACON )
            ->name( "Insufficient bacon" );

That would appear to be perfectly sufficient, understandable to all and you
get real short circuiting, not calling empty methods on an empty object.


-- 
If you want the truth to stand clear before you, never be for or against.
The struggle between "for" and "against" is the mind's worst disease.
    -- Sent-ts'an