SPUG: Resolved: summary of $Test::More::TODO

[Michael R. Wolf]

Thank for all the replies (and re-replies).

Short summary:
    Test assertions in a TODO block are not *reversed*, they are  
*relaxed*.


Longer description:

It turns out I was reading too much into the documentation.  I  
(mistakenly) believed that everything in a test file is an assertion,  
that if violated, would cause the test file to fail, and therefore  
return an unsuccessful return code.

I had figured that tests in a TODO block would be *reversed*:
  - tests that were expected to fail would pass, and
  - tests that "accidently" (i.e. coding for bug fixes or feature  
additions) passed would fail, *requiring* developer to move tests  
outside the TODO block *immediately* in order to pass

My thinking was INCORRECT.  TODO blocks are not so heavy handed.

Rather that thinking TODO blocks being *REVERSED*, I now think of them  
as being *RELAXED*.
  - all tests add a comment ("# TODO: $TODO") that's recognizable by  
prove ().
  - failing tests print "not ok", but do *not* cause the test file to  
fail
  - passing tests print "ok" in perl(1), and cause an additional "TODO  
passed" status line, *allowing* developers to move test outside the  
TODO block *at their leisure* to remove the extra status line, but  
passing either way


That's "what".

So what?  Now what?

In my own tests, this doesn't feel like a strong enough assertion.   
Looking for the "TODO passed" line is too much work, and since it's  
not common, I'm not likely to continue the practice of remaining  
diligent to it's arrival.

Although I understand the light-handed approach of the current  
behavior, I'd like a different choice.  TMTOWTDI-ishly, I'll use the  
current behavior when it's appropriate, but I'll also start coding a  
more heavy-handed approach:

# The anti-feature block contains two types of assertions:
#  * Bugs:  Tests characterize the bug, passing when the bug is  
exercised and returns an incorrect value
#  * Missing features: Tests define the new interface or new behavior,  
passing when the feature is missing or behaves incorrectly
# All anti-features are coded as negative logic (asserting a bug or  
missing feature) instead of the normal, positive logic (asserting a  
truth or real feature).  Once they transition from anti-features to  
features, move the code outside of this block, and negate the sense of  
the test.

ANTI_FEATURES:
{
         use Math::Trig;

     # Bugs
         # http://en.wikipedia.org/wiki/Indiana_Pi_Bill
         # "ratio of the diameter and circumference is as five-fourths  
to four"
         cmp_ok(pi, '!=', 4 / (5/4), 'Indiana Pi Bill is in effect.');

     # Missing features
         ok(! Pig->can('fly'),               'Pigs can fly!');
         ok(! Hell->can('freeze_over'),      'Hell can freeze over.');
}


-- 
Michael R. Wolf
     All mammals learn by playing!
         MichaelRWolf at att.net