[Omaha.pm] Benchmark flakiness
Jay Hannah
jhannah at omnihotels.com
Tue Aug 26 09:12:15 PDT 2008
Weird...
-----------
use Benchmark qw( :hireswallclock );
timethis(100000, '
for ($x=0; $x<=200; $x++) { sin($x/($x+2)); }
');
-----------
On my Mac this takes 10 seconds and outputs this:
timethis 100000: 9.88306 wallclock secs ( 9.80 usr + 0.02 sys = 9.82 CPU)
@ 10183.30/s (n=100000)
On my SuSE box it takes between 10 and 20 seconds and outputs this
regardless of how long it really took:
timethis 100000: 10.2632 wallclock secs ( 9.13 usr + 0.27 sys = 9.40 CPU)
@ 10638.30/s (n=100000)
So the reported wallclock is fine on my Mac (idle or under load) but lies on
SuSE under load.
It gets worse:
-----------
use Benchmark qw( timethese cmpthese :hireswallclock ) ;
$x = 3;
$r = timethese( 10000000, {
a => sub{$x*$x},
b => sub{$x**2},
} );
cmpthese $r;
------------
On my Mac this takes 22 seconds and outputs this:
Benchmark: timing 10000000 iterations of a, b...
a: 1.02988 wallclock secs ( 1.02 usr + -0.01 sys = 1.01 CPU) @
9900990.10/s (n=10000000)
b: 1.10844 wallclock secs ( 1.12 usr + -0.01 sys = 1.11 CPU) @
9009009.01/s (n=10000000)
Rate b a
b 9009009/s -- -9%
a 9900990/s 10% --
On SuSE it takes 30 seconds and outputs this:
Benchmark: timing 10000000 iterations of a, b...
a: 1.94711 wallclock secs ( 1.56 usr + 0.13 sys = 1.69 CPU) @
5917159.76/s (n=10000000)
b: 1.65905 wallclock secs ( 1.52 usr + 0.11 sys = 1.63 CPU) @
6134969.33/s (n=10000000)
Rate a b
a 5917160/s -- -4%
b 6134969/s 4% --
So in timethese() wallclock is entirely hosed...
Bummer.
Roll our own, I guess... -sigh-
j
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/omaha-pm/attachments/20080826/746b6b8d/attachment-0001.html>
More information about the Omaha-pm
mailing list