[Pdx-pm] Modern testing in perl

Ingy dot Net ingy at ingy.net
Sat Mar 6 23:25:00 PST 2010


I just wanted to add that I started TestML (see http://www.testml.org) about
a year ago. It is a unit testing language for all programming languages. You
write tests in TestML and then write your software in any language. It is
inspired by FIT, but not tied to a weird tabular harness like FIT. (In fact
it's not even tied to any harness in a given language. It just needs a
Runner subclass to tie it to a given harness/testing framework. So far I
have implementations in Perl and Python. It's still in its infancy.)

My main reason for starting TestML is that I believe programmers need to
start writing modules that can be used in many programming languages. If
they can pass the same test suite, that is something of an insurance that
they truly are the same. I think think is critical for modules that get
ported from Perl5 to Perl6. Wouldn't you want both versions to pass the same

I believe that great programmers should share their ideas (in code) with all
people, not just the people in their language. I call this belief Acmeism
(because I think that good ideas need a name). I am also working on a new
programming language called C'Dent that lets modules be compiled to from
your language to a dozen or more others.

Acmeism is fueled by the weird feeling I get at conferences like OSCON and
OSDC where great programmers of many languages come together to one place at
one time, only to go off to their own corners and talk about their own
things. None of these languages is so good that it's going to make the
others obsolete. So why not work on things above the language level?

I would encourage anyone as brave as you, for starting a new technology
movement, to think bigger than Perl. Think as big as you can.

Respectfully, Ingy döt Net

On Thu, Mar 4, 2010 at 7:10 PM, Chad Granum <exodist7 at gmail.com> wrote:

> Recently there have been a couple movements to 'modernize' parts of
> perl. Two immediate examples are Moose which is a more modern OO
> system (like perl6), the other is perl5i which Schwern is heading.
> perl5i is intended to fix all kinds of gripes. Thus far I have not
> seen any similar movement in the area of perl testing. After a
> discussion which compared and contrasted perl's testing tools with
> another set of test tools it occurred to me that there is probably
> room for significant improvement. I have decided to try to fill this
> void in modernization of perl testing.
> I have started the fennec project (http://github.com/exodist/Fennec)
> before the name scares you let it be known that the original name was
> to be Test::Suite, and I have just obtained that namespace (it was
> previously owned by someone else) so the name may change back.
> I am requesting that anyone and everyone put there 2 cents in on what
> is amazing, good, bad, ugly, or impossible with the current perl
> testing tools. How they can be improved, etc. I also encourage anyone
> interested to add issues, look for bugs, add feature requests, submit
> patches, fork the repo, etc.
> Here is a simple bulleted list of desired features, most of these have
> a current (bu maybe in need of improvement) implementation.
>  * Group tests into sets which can be run multiple times under
> different scenarios
>  * Every test file should create an object that is run
>  * Test sets should be run in random order by default, as should cases
> (scenarios) and even test files.
>  * Writing tester function libraries (think Test::More,
> Test::Exception, etc) should be very simplified
>  * Test results should be reported to the tester in object form,
> unlike Test::Builder which just outputs any results directly to TAP.
>  * Ability to create result handlers for cases where you want to get
> results directly instead of going straight to TAP.
>  * TAP output plugin used by default
>  * Database output plugin (record results to a simple database)
>  * Test::Builder output plugin (If you really want to go through
> Test::Builder, this is also the first output plugin for quickstart of
> the project)
>  * Ability to use multiple output plugins at once.
>  * Ability to wrap existing Test::Builder plugins (like Test::More)
> into Fennec tester libraries (this is already done for Test::More,
> Test::Warn)
>  * Ability to run just a specified case/set
>  * More helpful output in some current testers (is_deeply for example)
>  * Ability to define tests both in separate files, and inline with the
> objects being tested
>  * When not in testing mode these definitions should be ignored and
> minimal overhead should occur as a result of their being present.
>  * Inline tests are purely optional
>  * Perhaps tests be defined after __END__?
> Test result objects need to contain the following information:
>  * Name,
>  * Result (ok, not ok),
>  * Case run under,
>  * Set run under,
>  * Line tester was called from,
>  * File tester was called from,
>  * todo (false or reason),
>  * skip (false or reason),
>  * diagnostics messages,
> Defining tests and cases should be moose like:
> case name => sub {};
> case name => (
>   code => sub {},
>   partition => 'name',
>   ..options..,
> );
> set name => sub {};
> Tests should also be definable by creating subs prefixed with 'test' or
> 'case'
> sub test_something {}
> sub case_prepare_things {}
> There will also be an init method that will be called just once prior
> to running the cases.
> The base principal for Fennec is that test will be grouped into sets,
> each of these sets can be run against multiple cases. Essentially a
> case is a method on a test object you define that creates a scenario,
> once the scenario is ready all your test sets will be run under that
> scenario, once the sets are all completed the next scenario will be
> run and the sets will be run again. You can mark an entire case or set
> as todo or skip. You also can also specify that sets should only run
> under certain cases, or not under others. You can also group sets and
> cases into a 'partition' so that only sets in that partition will be
> run under the cases in the same partition.
> Currently there are 2 types of plugins, output plugins which take
> results and do stuff with them, and tester plugins with provide
> functionality such as ok, like, is, diag, etc.
> I appreciate any feedback anyone wants to provide!
> -Chad Granum
> _______________________________________________
> Pdx-pm-list mailing list
> Pdx-pm-list at pm.org
> http://mail.pm.org/mailman/listinfo/pdx-pm-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/pdx-pm-list/attachments/20100306/b554afb3/attachment.html>

More information about the Pdx-pm-list mailing list