On Wed, Sep 21, 2011 at 6:53 PM, Schirmacher, Rolf
<address@hidden> wrote:
Hello,
with the 3.2.4 windows (mingw) binary, I get the
following results:
octave.exe> format long
octave.exe> 1.0/3
ans =
0.333333333333333
octave.exe> single(1.0/3)
ans =
0.333333343267441
octave.exe> 1.0/3 - single(1.0/3)
ans = 0
I am wondering about the difference being 0.
In addition, how could I calculate the difference of 1.0/3 as single and
double?
Thanks for any hint,
Rolf
I get tripped up on these types of problems myself. The key thing to remember for me is that when you are doing operations with different classes of variables, you need to check what octave does to be able to (in this case) subtract them. Octave in this case, I believe, of a double minus a single, will convert the double to a single, then subtract them and the result will be of the single class. (Hence why the result is 0.) To avoid this, you could convert your single class number, back into a double before subtracting it. Like this:
1.0/3 - double(single(1.0/3))
Should give you the "rounding" error of converting to single precision. I don't have octave on hand right now to double check this however.
Hope this helps.