Re: bug 40895

[Rik]

On 12/14/2013 04:47 AM, address@hidden wrote:
> Message: 5
> Date: Sat, 14 Dec 2013 04:01:38 -0800 (PST)
> From: CdeMills <address@hidden>
> To: address@hidden
> Subject: Bug 40895
> Message-ID: <address@hidden>
> Content-Type: text/plain; charset=us-ascii
>
> Hello,
>
> I posted a bug and a patch against 3.8.0-rc1 about
> libinterp/corefcn/conv2.cc. Basically, some results differ by a few tens of
> epsilon because truncation error occured differently along two computational
> path. Should we really be that strict ?
>
> Regards
>
> Pascal
12/14/13

Pascal,

Yes, we would like to be that strict.  In this case, the exact test caught
the fact that there were two different code paths by which the convolution
was being calculated.  This isn't a particularly good thing.  In
particular, the difference is small for the %!test cases in Octave, but I
don't think it is bounded.  I believe it grows with sqrt (N) where N  is
the number of rows or columns in the matrix.  The difference could be
eliminated in two ways.  One way would be to take the full convolution and
then use indexing to pick out the reduced portion of the matrix which
corresponds to "valid".  This is simple to implement, but is pretty
wasteful of compute cycles.  The other solution is to work through the C++
code in oct-convn.cc and the Fortran code in dconv2.f and try to make the
summation order the same between the two cases.

--Rik