SPUG: Rounding error

[Fred Morris]

On Wednesday 23 November 2011 10:24, Amit Sett wrote:
> [...] The program works well except for some cases where it short
> changes a customer by 1 penny [...]

Try doing things in pennies, e.g. a dollar is 100 pennies.

Do you know what a repeating decimal is, and do you know what causes them? 
Basically any time you divide 1 by a prime number not in your number base you 
get a repeating decimal.

For instance in base 10 (2 * 5) if you divide by 3, 7, 11, 13, etc. you get a 
repeating decimal.

In base 2, dividing by 3, 5, 7, 11, etc does the same thing. If you divide by 
100 (2 * 2 * 5 * 5) you get a repeating decimal, or in other words there is 
rounding error.

ObMathNote: 0.9999999... actually is provably equal to 1. ;-)

--

Fred