[Chicago-talk] addition weirdness

Steven Lembark lembark at wrkhors.com
Thu Jan 29 10:29:44 CST 2004


> Any idea why this is happening?

IEEE number formats have a limit on scale. When you try
to add integers the .00000000 part eventually starts
rounding and you end up with a .999999 or .000001 [don't
quote me on the exact precision].

This also happens if you add a bunch of values in
tentns (i.e., /\d\.[0-9]/) since 1/10 is a repeating
fraction base two.

If this is a problem then

	use integer;

will give you integer precision up to your processor's
word limit; use <IForgot>::BigInt does pretty much what it
sounds like.

--
Steven Lembark                               2930 W. Palmer
Workhorse Computing                       Chicago, IL 60647
                                            +1 888 359 3508



More information about the Chicago-talk mailing list