emacs-orgmode
[Top][All Lists]

 From: Carsten Dominik Subject: Re: [Orgmode] adding up decimal fractions in the spreadsheet Date: Sat, 27 Sep 2008 14:19:29 +0200

```Hi Rainer,

```
it is a common mis-conception that a number that has a limited number of digits in base 10 can be represented exactly on a computer. This is correct for integers, but not for fractional numbers.
```
Example:

1. Open the emacs calc with M-x calc RET
2. Type 12130.68 (first number from your example)
3. Press `p 200 RET' to switch to 200 digit precision.
4. press `d r 2 RET' o display the number in Base 2.

You'll see

```
10111101100010.1010111000010100011110101110000101000111101011100001010001111010111000010100011110101110000101000111101011100001010001111010111000010100011110101110000 ................
```

```
It works the other way round as well, for example 0.33333333............. has an exact representation in base 3 (0.1).
```
```
This means that even simple additions like this do produce small errors on computers, and the displayed result will depend upon how he conversion is done.
```
```
(format "%s" x) just spits out digits, as many as possible. That is why the fix I have pushed to the git repo uses
```
(number-to-string res)

and not

(format "%s" res)

```
number-to-string is an internal function that tries very hard to do the conversion in the way you intend it.
```
- Carsten

On Sep 27, 2008, at 9:19 AM, Rainer Thiel wrote:

```
```Following a hint by Bernt Hansen, I have now changed org-table-sum to
(format "%s" res) until I'll be able to use the corrected version
Carsten Dominik uploaded to the git repository (which I am not sure
how to access).

Even so, org-table-sum at times shows a strange behaviour.

When I try to sum up the following table:

|  12130.68 |
|  -1444.19 |
| -12744.90 |
|   -186.00 |
|   7000.00 |
|  -7056.00 |
|   -335.58 |
|   -277.00 |

I get -2912.9899999999993 (-2912.99 being the exact result).  This is
something I'd expect when square roots or logarithms are implied, but
with such a simple addition and subtraction of decimal numbers I'd be
inclined to call it a bug.  Is there a way to improve org-mode in this
respect?

Many thanks again for this great mode
--
Prof. Dr. Rainer Thiel
Institut für Altertumswissenschaften
07737 Jena, Germany (EU)

_______________________________________________
Emacs-orgmode mailing list
Remember: use `Reply All' to send replies to the list.
```