[Pdx-pm] Modern testing in perl

Ingy dot Net ingy at ingy.net
Sat Mar 6 23:52:49 PST 2010


On Sat, Mar 6, 2010 at 11:42 PM, Chad Granum <exodist7 at gmail.com> wrote:

> Ingy,
>
> I like parts of what your suggesting. My goal is to make Fennec very
> flexible. I am thinking it may be possible to write a plugin for
> TestML and Fennec to work together. After I have Fennec to a point
> where I consider it usable would you be interested in working with me
> to work in support for TestML test files?
>

Absolutely. I'll keep my eyes on Fennec as well.

Cheers, Ingy



> I will continue on from here
> keeping in mind that I might want to support test files from TestML in
> the future.
>
> Thank you.
>
> -Chad
>
> On Sat, Mar 6, 2010 at 11:25 PM, Ingy dot Net <ingy at ingy.net> wrote:
> > Chad,
> >
> > I just wanted to add that I started TestML (see http://www.testml.org)
> about
> > a year ago. It is a unit testing language for all programming languages.
> You
> > write tests in TestML and then write your software in any language. It is
> > inspired by FIT, but not tied to a weird tabular harness like FIT. (In
> fact
> > it's not even tied to any harness in a given language. It just needs a
> > Runner subclass to tie it to a given harness/testing framework. So far I
> > have implementations in Perl and Python. It's still in its infancy.)
> >
> > My main reason for starting TestML is that I believe programmers need to
> > start writing modules that can be used in many programming languages. If
> > they can pass the same test suite, that is something of an insurance that
> > they truly are the same. I think think is critical for modules that get
> > ported from Perl5 to Perl6. Wouldn't you want both versions to pass the
> same
> > tests?
> >
> > I believe that great programmers should share their ideas (in code) with
> all
> > people, not just the people in their language. I call this belief Acmeism
> > (because I think that good ideas need a name). I am also working on a new
> > programming language called C'Dent that lets modules be compiled to from
> > your language to a dozen or more others.
> >
> > Acmeism is fueled by the weird feeling I get at conferences like OSCON
> and
> > OSDC where great programmers of many languages come together to one place
> at
> > one time, only to go off to their own corners and talk about their own
> > things. None of these languages is so good that it's going to make the
> > others obsolete. So why not work on things above the language level?
> >
> > I would encourage anyone as brave as you, for starting a new technology
> > movement, to think bigger than Perl. Think as big as you can.
> >
> > Respectfully, Ingy döt Net
> >
> > On Thu, Mar 4, 2010 at 7:10 PM, Chad Granum <exodist7 at gmail.com> wrote:
> >>
> >> Recently there have been a couple movements to 'modernize' parts of
> >> perl. Two immediate examples are Moose which is a more modern OO
> >> system (like perl6), the other is perl5i which Schwern is heading.
> >> perl5i is intended to fix all kinds of gripes. Thus far I have not
> >> seen any similar movement in the area of perl testing. After a
> >> discussion which compared and contrasted perl's testing tools with
> >> another set of test tools it occurred to me that there is probably
> >> room for significant improvement. I have decided to try to fill this
> >> void in modernization of perl testing.
> >>
> >> I have started the fennec project (http://github.com/exodist/Fennec)
> >> before the name scares you let it be known that the original name was
> >> to be Test::Suite, and I have just obtained that namespace (it was
> >> previously owned by someone else) so the name may change back.
> >>
> >> I am requesting that anyone and everyone put there 2 cents in on what
> >> is amazing, good, bad, ugly, or impossible with the current perl
> >> testing tools. How they can be improved, etc. I also encourage anyone
> >> interested to add issues, look for bugs, add feature requests, submit
> >> patches, fork the repo, etc.
> >>
> >> Here is a simple bulleted list of desired features, most of these have
> >> a current (bu maybe in need of improvement) implementation.
> >>
> >>  * Group tests into sets which can be run multiple times under
> >> different scenarios
> >>  * Every test file should create an object that is run
> >>  * Test sets should be run in random order by default, as should cases
> >> (scenarios) and even test files.
> >>  * Writing tester function libraries (think Test::More,
> >> Test::Exception, etc) should be very simplified
> >>  * Test results should be reported to the tester in object form,
> >> unlike Test::Builder which just outputs any results directly to TAP.
> >>  * Ability to create result handlers for cases where you want to get
> >> results directly instead of going straight to TAP.
> >>  * TAP output plugin used by default
> >>  * Database output plugin (record results to a simple database)
> >>  * Test::Builder output plugin (If you really want to go through
> >> Test::Builder, this is also the first output plugin for quickstart of
> >> the project)
> >>  * Ability to use multiple output plugins at once.
> >>  * Ability to wrap existing Test::Builder plugins (like Test::More)
> >> into Fennec tester libraries (this is already done for Test::More,
> >> Test::Warn)
> >>  * Ability to run just a specified case/set
> >>  * More helpful output in some current testers (is_deeply for example)
> >>  * Ability to define tests both in separate files, and inline with the
> >> objects being tested
> >>  * When not in testing mode these definitions should be ignored and
> >> minimal overhead should occur as a result of their being present.
> >>  * Inline tests are purely optional
> >>  * Perhaps tests be defined after __END__?
> >>
> >> Test result objects need to contain the following information:
> >>  * Name,
> >>  * Result (ok, not ok),
> >>  * Case run under,
> >>  * Set run under,
> >>  * Line tester was called from,
> >>  * File tester was called from,
> >>  * todo (false or reason),
> >>  * skip (false or reason),
> >>  * diagnostics messages,
> >>
> >> Defining tests and cases should be moose like:
> >>
> >> case name => sub {};
> >> case name => (
> >>   code => sub {},
> >>   partition => 'name',
> >>   ..options..,
> >> );
> >> set name => sub {};
> >>
> >> Tests should also be definable by creating subs prefixed with 'test' or
> >> 'case'
> >> sub test_something {}
> >> sub case_prepare_things {}
> >>
> >> There will also be an init method that will be called just once prior
> >> to running the cases.
> >>
> >> The base principal for Fennec is that test will be grouped into sets,
> >> each of these sets can be run against multiple cases. Essentially a
> >> case is a method on a test object you define that creates a scenario,
> >> once the scenario is ready all your test sets will be run under that
> >> scenario, once the sets are all completed the next scenario will be
> >> run and the sets will be run again. You can mark an entire case or set
> >> as todo or skip. You also can also specify that sets should only run
> >> under certain cases, or not under others. You can also group sets and
> >> cases into a 'partition' so that only sets in that partition will be
> >> run under the cases in the same partition.
> >>
> >> Currently there are 2 types of plugins, output plugins which take
> >> results and do stuff with them, and tester plugins with provide
> >> functionality such as ok, like, is, diag, etc.
> >>
> >> I appreciate any feedback anyone wants to provide!
> >>
> >> -Chad Granum
> >> _______________________________________________
> >> Pdx-pm-list mailing list
> >> Pdx-pm-list at pm.org
> >> http://mail.pm.org/mailman/listinfo/pdx-pm-list
> >
> >
> _______________________________________________
> Pdx-pm-list mailing list
> Pdx-pm-list at pm.org
> http://mail.pm.org/mailman/listinfo/pdx-pm-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/pdx-pm-list/attachments/20100306/6d25c9db/attachment.html>


More information about the Pdx-pm-list mailing list