[Pdx-pm] Newbie question about testing and Perl
keithl at kl-ic.com
Thu Jul 13 07:38:20 PDT 2006
When testing a module that is concerned with a hardware device
( Device::SerialPort using /dev/ttyXX ) or an external non-Perl
program ( Dirvish using rsync ), why don't the Perl programs test
against an emulation, rather than the real thing?
This occured to me after thinking about what little I understood
of what I observed at the hackathon last night - some of the issues
were related to battles between the install test code, and the
modem-connected default serial port on Eric's laptop (the test code
wants an open serial port - which is silly).
While it was useful to see chromatic and Schwern and others replace
cheezy Perl with good stuff, the tweak/test/tweak/test iterative
loop was inefficient because the testing prerequisites were not met.
That is often the case during installation test, right?
I would think an emulation - with behavior, timing, and everything -
would be easy to connect to on a Linux/Mac/Unix system (just tweak
the data structures inside Perl to point at a different file or
process), and perhaps not that hard in Windows. The emulation
(written in Perl, or perhaps written in a hardware emulation
language like Verilog) might not be an perfect emulation of the
real thing, but it could be made to exhibit corner-case behavior
that the real thing might not normally exhibit.
More elaborate emulations would emulate many of the operating
system behaviors (particularly the clock or the network or the
nut attached to the mouse), and pick apart driver C source to gather
error messages for the emulator to throw at the Perl code under
test. Deviations between emulated behavior and real behavior
would be instructive to the designers of the system - sometimes
the emulator is correct and the emulated system is defective.
Perhaps testing against an emulator is invalid, or emulators are
difficult to write, or this is a common technique in software that
I haven't heard about yet. Turned inside out, and testing just a
small bit of code, this sounds like a test harness.
We do this kind of emulation in electronics, in hardware and in
real time, so we can test our electronic concoctions, and I wonder
about the analogous techniques in software. It is difficult in
electronics; we often use a $2M tester to test $0.20 parts, and
spend months programming the tester, but it is how we manage to
reduce defects below 0.01% on items made of impure real-world
semi-deterministic analog goop. Surely software could do better.
I await your clueful adjustment of my ignorance ...
Keith Lofstrom keithl at keithl.com Voice (503)-520-1993
KLIC --- Keith Lofstrom Integrated Circuits --- "Your Ideas in Silicon"
Design Contracting in Bipolar and CMOS - Analog, Digital, and Scan ICs
More information about the Pdx-pm-list