We make two common errors regarding numerical precision:
- We accept calculated numbers, such as in spreadsheets,
as they are without regard to the significant digits
in the inputs used to calculate them.
- We present highly precise numbers as though they
demonstrate how well we've found an answer. Precision
can reflect a level of attention to detail that just
isn't there.
These errors arise from the fact that precision has
little bearing on the accuracy of a number, i.e. whether
it's right or wrong. All precision tells us is the extent
of the number's right- or wrong-ness. When we present
highly precise numbers, we reflect a sense of accuracy and
attention to detail that may not really be there; we may
also distract our audience from the real story the numbers
tell.
Excess precision in presented numbers can have unforeseen
effects.
- The human brain is limited in its ability to
process information. As we sort and interpret
the numbers we see, we feel the effects of
- how many numbers are presented
- how many significant digits there are in each
- the range of the numbers
- the relative importance of each
- the speed at which they must be processed
- Some tolerances are not achievable at finite
cost. A tolerance is just one example of the
practical effects of precision. Consider this example
of a machined part: the cost of casting is a hyperbolic
function of the tolerance. As tolerance approaches
zero, cost approaches infinity. A small decrease in
tolerance (or, increase in precision) can mean a very
large increase in machining cost.
- Programming cost can be affected by precision.
There are defaults in C++ and other languages
for the amount of storage given types of numbers have
designated. That storage cost could be worth saving,
especially in real-time programming.
Example
Here's an eigenvalue problem solved using Matlab:
» format long
» a
a =
-300 1 0
-30000 0 -1
1000000 0 0
» eig(a)
ans =
1.0e+002 *
-1.00000480119569 + 0.00000831603109i
-1.00000480119569 - 0.00000831603109i
-0.99999039760862
Although Matlab is very precise (especially using
"long" format), solving a problem like this with
that sort of precision is something like solving
"1+1" with your pocket calculator and accepting
1.99999 as more correct than 2!
In this case the eigenvalues are the solution of
the equation
1000000 + 30000 x + 300 x^2 + x^3 = 0
or (x+100)^3 = 0
and they are EXACTLY 100, all three of them -- no
ambiguity, no need for precision!
References
DeGarmo, E. P. et. al.
Materials
and Processes in Manufacturing. Prentice-Hall, 1997.
ISBN 0-02328-621-0
Prata, S.
C++
Primer Plus. Waite Group, 1998.
ISBN 1-57169-162-6
What You Can Do
- Don't copy a calculator result, digit-for-digit.
- Prescribe output formats for spreadsheets or
executable programs.
- Beware of direct comparisons of very large and very
small numbers. Do the numbers you're comparing
come from the same context?
- Present numbers as you wish them to be interpreted.
If you are not interested in many decimal places,
don't make it seem to your audience that you are.
- In programming, consider the possibility of
assigning only the bits you know you'll need to
numbers that would otherwise tend to clumsy rounding or
truncation.