[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
## Re: [Help-gsl] Matrix multiplication precision problem ???

**From**: |
Anders Aspegren Søndergaard |

**Subject**: |
Re: [Help-gsl] Matrix multiplication precision problem ??? |

**Date**: |
Thu, 20 Nov 2014 23:37:50 +0100 |

On Thu, Nov 20, 2014 at 4:58 PM, Marco Dinarelli
<address@hidden> wrote:
>* In order to keep track and check everything I'm doing is correct, I wrote a*
>* first prototype in octave and then I rewrote the same things in C with gsl.*
Try without optimizations, especially without --fast-math.
>* When I run the two programs, at first results are the same, but after some*
>* iterations to train the network, results start to diverge, with the C*
>* implementation having worst results and cross-entropy fluctuating values.*
Your algorithm migt not be numerically stable. It could for example be chaotic.
Do you use any random number generators in your training?
>* My question is: Is this a precision problem or there should be a mistake*
>* somewhere ?*
Are you using double in both Octave and C? 1e-14 is by the way rather
close to the minimum resolvable
relative difference between two double precision numbers.
>* So, in case this is a precision problem, does anybody know how to overcome*
>* it ?*
It really depends on your algorithm. Ideally it should be robust
against small pertubations to the initial conditions,
at least if it is a neural network training algorithm (otherwise it
does not really make sense).
Try perturbing the training set/input to your octave implementation
and see if it performs about the same.
Then do that to your C implementation.
Maybe you just introduced a bug/typo/omission in your C program that
you haven't discovered yet?
Best regards
Anders A. Søndergaard