[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Help-gsl] Matrix multiplication precision problem ???

From: Anders Aspegren Søndergaard
Subject: Re: [Help-gsl] Matrix multiplication precision problem ???
Date: Thu, 20 Nov 2014 23:37:50 +0100

On Thu, Nov 20, 2014 at 4:58 PM, Marco Dinarelli
<address@hidden> wrote:

> In order to keep track and check everything I'm doing is correct, I wrote a
> first prototype in octave and then I rewrote the same things in C with gsl.

Try without optimizations, especially without --fast-math.

> When I run the two programs, at first results are the same, but after some
> iterations to train the network, results start to diverge, with the C
> implementation having worst results and cross-entropy fluctuating values.

Your algorithm migt not be numerically stable. It could for example be chaotic.

Do you use any random number generators in your training?

> My question is: Is this a precision problem or there should be a mistake
> somewhere ?

Are you using double in both Octave and C? 1e-14 is by the way rather
close to the minimum resolvable
relative difference between two double precision numbers.

> So, in case this is a precision problem, does anybody know how to overcome
> it ?

It really depends on your algorithm. Ideally it should be robust
against small pertubations to the initial conditions,
at least if it is a neural network training algorithm (otherwise it
does not really make sense).
Try perturbing the training set/input to your octave implementation
and see if it performs about the same.
Then do that to your C implementation.

Maybe you just introduced a bug/typo/omission in your C program that
you haven't discovered yet?

Best regards
Anders A. Søndergaard

reply via email to

[Prev in Thread] Current Thread [Next in Thread]