[Pdx-pm] Modern testing in perl

Chad Granum exodist7 at gmail.com
Sat Mar 6 14:59:44 PST 2010


Ooops, accidently only sent this to Eric,


If you look at my fork branch you will see the beginnings of my
parallelism, there will be one main process producing all output.
Essentially every tester function should produce a result object
giving the result and some context. These all funnel down to the main
process where they are sent to output plugins. Subprocesses (forked
code) only use the subprocess plugin which send result objects the the
main process. The main process them uses whatever output plugin you
want, currently there is a tap output plugin, and others are in the
works. The tester object is aware of it's pid, if the user forks and
then issues a result the tester automatically knows to switch to the
subprocesses output plugin.

So really it becomes easy as can be to write plugins for what to do
with results and other plugins for how to produce results. The user
has complete freedom.

I also want to reduce boilerplate as much as possible. Test::class
looks amazing, but the boiler plate is putting me off. It should be as
simple as declaring a package and test methods, the tools should do
the rest.

Also test::builder is Great and has opened the doors to the amazing
testing perl can already accomplish, but aside from producers it's
hard to extend. And you are bound to tap output with t::b. True tap is
what you want almost all the time, but when it is not what you want
their is a serious hindrance.

By making result objects thy contain the result, line number, file,
diag, and other info, then letting the user output whatever they want
you provide a better opening for useful tools. Tap is only one
possible thing to do with results, it also means not relying on t:b to
find the line number and other context for the result, which is
proving problematic for me sometimes.

Anyway my goal here is complete flexability and plugability while
solving problems I have experienced.

Sent from my iPhone

On Mar 6, 2010, at 1:01 PM, Eric Wilhelm <enobacon at gmail.com> wrote:

> # from Chad Granum
> # on Thursday 04 March 2010 19:10:
>
> Interesting.  You're touching both sides of the TAP wall here though and
> I think it might be useful to consider what can be done from each side
> (and implicitly: why keeping the wall is a good thing.)
>
> These things sound rather consumer/harness-related:
>>
>>  * Group tests into sets which can be run multiple times under
>>  different scenarios
>>  * Every test file should create an object that is run
>>  * Test results should be reported to the tester in object form,
>>  unlike Test::Builder which just outputs any results directly to TAP.
>> * Ability to run just a specified case/set
>
>
> Where these sound more producer-related:
>>
>> * Writing tester function libraries (think Test::More,
>>  Test::Exception, etc) should be very simplified
>>  * Ability to create result handlers for cases where you want to get
>>  results directly instead of going straight to TAP.
>>  * More helpful output in some current testers (is_deeply for example)
>>  * Ability to define tests both in separate files, and inline with the
>>  objects being tested
>
> They interrelate, but I'm just suggesting that it might be useful to
> view them from two directions like this.
>
> If you can structure your tests in a way that allows them to be
> introspected, the harness (given sufficient knobs) can be made to run
> parts or groups of them regardless of what's in which *.t files.  You
> might have another tool which e.g. lets you REPL into a single process
> (which may even be enabled while running under the harness.)
>
> Don't forget about parallelism.  On a related note: preloading code in a
> single process and peeling off a fork for each test (for one instance:
> a daemon which just forks and dumps the requested test's TAP out
> through a client socket in the *.t files.)
>
> Hope that gives you something to think about.  I've never really had a
> chance to organize my thoughts on these matters into code, but you're
> welcome to try!
>
>  http://www.nntp.perl.org/group/perl.qa/2007/12/msg10035.html
>  http://www.nntp.perl.org/group/perl.qa/2007/08/msg8998.html
>
> --Eric
> --
> I arise in the morning torn between a desire to improve the world and a
> desire to enjoy the world. This makes it hard to plan the day.
> --E.B. White
> ---------------------------------------------------
>   http://scratchcomputing.com
> ---------------------------------------------------
> _______________________________________________
> Pdx-pm-list mailing list
> Pdx-pm-list at pm.org
> http://mail.pm.org/mailman/listinfo/pdx-pm-list


More information about the Pdx-pm-list mailing list