[Pdx-pm] Modern testing in perl

Chad Granum exodist7 at gmail.com
Sat Mar 6 23:42:34 PST 2010


Ingy,

I like parts of what your suggesting. My goal is to make Fennec very
flexible. I am thinking it may be possible to write a plugin for
TestML and Fennec to work together. After I have Fennec to a point
where I consider it usable would you be interested in working with me
to work in support for TestML test files? I will continue on from here
keeping in mind that I might want to support test files from TestML in
the future.

Thank you.

-Chad

On Sat, Mar 6, 2010 at 11:25 PM, Ingy dot Net <ingy at ingy.net> wrote:
> Chad,
>
> I just wanted to add that I started TestML (see http://www.testml.org) about
> a year ago. It is a unit testing language for all programming languages. You
> write tests in TestML and then write your software in any language. It is
> inspired by FIT, but not tied to a weird tabular harness like FIT. (In fact
> it's not even tied to any harness in a given language. It just needs a
> Runner subclass to tie it to a given harness/testing framework. So far I
> have implementations in Perl and Python. It's still in its infancy.)
>
> My main reason for starting TestML is that I believe programmers need to
> start writing modules that can be used in many programming languages. If
> they can pass the same test suite, that is something of an insurance that
> they truly are the same. I think think is critical for modules that get
> ported from Perl5 to Perl6. Wouldn't you want both versions to pass the same
> tests?
>
> I believe that great programmers should share their ideas (in code) with all
> people, not just the people in their language. I call this belief Acmeism
> (because I think that good ideas need a name). I am also working on a new
> programming language called C'Dent that lets modules be compiled to from
> your language to a dozen or more others.
>
> Acmeism is fueled by the weird feeling I get at conferences like OSCON and
> OSDC where great programmers of many languages come together to one place at
> one time, only to go off to their own corners and talk about their own
> things. None of these languages is so good that it's going to make the
> others obsolete. So why not work on things above the language level?
>
> I would encourage anyone as brave as you, for starting a new technology
> movement, to think bigger than Perl. Think as big as you can.
>
> Respectfully, Ingy döt Net
>
> On Thu, Mar 4, 2010 at 7:10 PM, Chad Granum <exodist7 at gmail.com> wrote:
>>
>> Recently there have been a couple movements to 'modernize' parts of
>> perl. Two immediate examples are Moose which is a more modern OO
>> system (like perl6), the other is perl5i which Schwern is heading.
>> perl5i is intended to fix all kinds of gripes. Thus far I have not
>> seen any similar movement in the area of perl testing. After a
>> discussion which compared and contrasted perl's testing tools with
>> another set of test tools it occurred to me that there is probably
>> room for significant improvement. I have decided to try to fill this
>> void in modernization of perl testing.
>>
>> I have started the fennec project (http://github.com/exodist/Fennec)
>> before the name scares you let it be known that the original name was
>> to be Test::Suite, and I have just obtained that namespace (it was
>> previously owned by someone else) so the name may change back.
>>
>> I am requesting that anyone and everyone put there 2 cents in on what
>> is amazing, good, bad, ugly, or impossible with the current perl
>> testing tools. How they can be improved, etc. I also encourage anyone
>> interested to add issues, look for bugs, add feature requests, submit
>> patches, fork the repo, etc.
>>
>> Here is a simple bulleted list of desired features, most of these have
>> a current (bu maybe in need of improvement) implementation.
>>
>>  * Group tests into sets which can be run multiple times under
>> different scenarios
>>  * Every test file should create an object that is run
>>  * Test sets should be run in random order by default, as should cases
>> (scenarios) and even test files.
>>  * Writing tester function libraries (think Test::More,
>> Test::Exception, etc) should be very simplified
>>  * Test results should be reported to the tester in object form,
>> unlike Test::Builder which just outputs any results directly to TAP.
>>  * Ability to create result handlers for cases where you want to get
>> results directly instead of going straight to TAP.
>>  * TAP output plugin used by default
>>  * Database output plugin (record results to a simple database)
>>  * Test::Builder output plugin (If you really want to go through
>> Test::Builder, this is also the first output plugin for quickstart of
>> the project)
>>  * Ability to use multiple output plugins at once.
>>  * Ability to wrap existing Test::Builder plugins (like Test::More)
>> into Fennec tester libraries (this is already done for Test::More,
>> Test::Warn)
>>  * Ability to run just a specified case/set
>>  * More helpful output in some current testers (is_deeply for example)
>>  * Ability to define tests both in separate files, and inline with the
>> objects being tested
>>  * When not in testing mode these definitions should be ignored and
>> minimal overhead should occur as a result of their being present.
>>  * Inline tests are purely optional
>>  * Perhaps tests be defined after __END__?
>>
>> Test result objects need to contain the following information:
>>  * Name,
>>  * Result (ok, not ok),
>>  * Case run under,
>>  * Set run under,
>>  * Line tester was called from,
>>  * File tester was called from,
>>  * todo (false or reason),
>>  * skip (false or reason),
>>  * diagnostics messages,
>>
>> Defining tests and cases should be moose like:
>>
>> case name => sub {};
>> case name => (
>>   code => sub {},
>>   partition => 'name',
>>   ..options..,
>> );
>> set name => sub {};
>>
>> Tests should also be definable by creating subs prefixed with 'test' or
>> 'case'
>> sub test_something {}
>> sub case_prepare_things {}
>>
>> There will also be an init method that will be called just once prior
>> to running the cases.
>>
>> The base principal for Fennec is that test will be grouped into sets,
>> each of these sets can be run against multiple cases. Essentially a
>> case is a method on a test object you define that creates a scenario,
>> once the scenario is ready all your test sets will be run under that
>> scenario, once the sets are all completed the next scenario will be
>> run and the sets will be run again. You can mark an entire case or set
>> as todo or skip. You also can also specify that sets should only run
>> under certain cases, or not under others. You can also group sets and
>> cases into a 'partition' so that only sets in that partition will be
>> run under the cases in the same partition.
>>
>> Currently there are 2 types of plugins, output plugins which take
>> results and do stuff with them, and tester plugins with provide
>> functionality such as ok, like, is, diag, etc.
>>
>> I appreciate any feedback anyone wants to provide!
>>
>> -Chad Granum
>> _______________________________________________
>> Pdx-pm-list mailing list
>> Pdx-pm-list at pm.org
>> http://mail.pm.org/mailman/listinfo/pdx-pm-list
>
>


More information about the Pdx-pm-list mailing list