<HTML>
<HEAD>
<TITLE>Benchmark flakiness</TITLE>
</HEAD>
<BODY>
<FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'>Weird...<BR>
<BR>
-----------<BR>
use Benchmark qw( :hireswallclock );<BR>
timethis(100000, '<BR>
for ($x=0; $x<=200; $x++) { sin($x/($x+2)); }<BR>
');<BR>
-----------<BR>
<BR>
On my Mac this takes 10 seconds and outputs this:<BR>
<BR>
timethis 100000: 9.88306 wallclock secs ( 9.80 usr + 0.02 sys = 9.82 CPU) @ 10183.30/s (n=100000)<BR>
<BR>
On my SuSE box it takes between 10 and 20 seconds and outputs this regardless of how long it really took:<BR>
<BR>
timethis 100000: 10.2632 wallclock secs ( 9.13 usr + 0.27 sys = 9.40 CPU) @ 10638.30/s (n=100000)<BR>
<BR>
So the reported wallclock is fine on my Mac (idle or under load) but lies on SuSE under load.<BR>
<BR>
It gets worse:<BR>
<BR>
-----------<BR>
use Benchmark qw( timethese cmpthese :hireswallclock ) ;<BR>
$x = 3;<BR>
$r = timethese( 10000000, {<BR>
a => sub{$x*$x},<BR>
b => sub{$x**2},<BR>
} );<BR>
cmpthese $r;<BR>
------------<BR>
<BR>
On my Mac this takes 22 seconds and outputs this:<BR>
<BR>
Benchmark: timing 10000000 iterations of a, b...<BR>
a: 1.02988 wallclock secs ( 1.02 usr + -0.01 sys = 1.01 CPU) @ 9900990.10/s (n=10000000)<BR>
b: 1.10844 wallclock secs ( 1.12 usr + -0.01 sys = 1.11 CPU) @ 9009009.01/s (n=10000000)<BR>
Rate b a<BR>
b 9009009/s -- -9%<BR>
a 9900990/s 10% --<BR>
<BR>
On SuSE it takes 30 seconds and outputs this:<BR>
<BR>
Benchmark: timing 10000000 iterations of a, b...<BR>
a: 1.94711 wallclock secs ( 1.56 usr + 0.13 sys = 1.69 CPU) @ 5917159.76/s (n=10000000)<BR>
b: 1.65905 wallclock secs ( 1.52 usr + 0.11 sys = 1.63 CPU) @ 6134969.33/s (n=10000000)<BR>
Rate a b<BR>
a 5917160/s -- -4%<BR>
b 6134969/s 4% --<BR>
<BR>
So in timethese() wallclock is entirely hosed...<BR>
<BR>
Bummer.<BR>
<BR>
Roll our own, I guess... -sigh-<BR>
<BR>
j <BR>
</SPAN></FONT>
</BODY>
</HTML>