[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Orgmode] Precision setting for spreadsheet calculation
From: |
Carsten Dominik |
Subject: |
Re: [Orgmode] Precision setting for spreadsheet calculation |
Date: |
Sat, 27 Feb 2010 00:11:31 +0100 |
On Feb 26, 2010, at 10:58 PM, Michael Brand wrote:
Carsten Dominik wrote:
these are the internals of calc, about which I cannot do much.
But
| 0.16666666666666666667 |
#+TBLFM: $1 = 1 / 2 * 3; p20 f20
shows that the internals of calc are ok, isn't it?
Hi Michael,
you are completely right, I was wrong.
The other issue that is playing into this is that when
using a printf format specifier, the number is first
converted into a Lisp number, which is a double, with
16 significant digits... At this step, the additional
digits are lost.
Maybe you find this interesting enough so that I can
persuade you to write a FAQ entry about these issues? :-)
- Carsten
Michael Brand wrote:
But instead of
| 0.166666666666666660 |
#+TBLFM: $1 = 1 / 2 * 3; %.18f p20 f20
which I get I would expect
| 0.166666666666666667 |
#+TBLFM: $1 = 1 / 2 * 3; %.18f p20 f20
- Carsten