[Octave-bug-tracker] [bug #56884] mean can overflow

[Michael Leitner]

Follow-up Comment #12, bug #56884 (project octave):

No, I did not. I was just using Marco's file as template and did not look at
the tests. Indeed there was an error to fix. Now the tests pass the same as
Marco's: all pass deterministically but the last, which can fail
stochastically. I would say, about one time in five tries, and this holds for
both algorithms (but for different vectors x). But this is no problem, this
test is about subnormal numbers at the extreme end, where the relative eps
becomes of the order of one in any case.

As to Kahan summation: no, I was talking about native sum, not m-code mean.
But this was probably premature: What I was testing was code like


a=ones(100,1);a(1)=flintmax();sum(a)-flintmax()


putting the flintmax either to the beginning or the end and varying the vector
length.

Indeed, at my office computer (Octave 4.0.3 on amd64) these behave exactly
like naive summation (that is, not so bad when flintmax is at the end, but
very bad when it is at the beginning), while using sum(a,"extra") is more
accurate, which looks like it was using pairwise summation, but definitely not
Kahan.

In contrast, at my home computer (older Octave on x86, I think) these lines of
code evaluated much more accurate even without "extra", that's why I thought
that this was Kahan summation at work. But perhaps it was rather 80-bit
extended precision (which would probably the best bet for "extra", being
perhaps even faster)? I will have to look again.

I did not look beyond libinterp/corefcn/data.cc, where the array_value ().sum
etc. would be defined. But anyway, this bug is about mean, not sum.

(file #47537)
    _______________________________________________________

Additional Item Attachment:

File name: pairwise_mean.m                Size:1 KB
    <https://savannah.gnu.org/file/pairwise_mean.m?file_id=47537>



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?56884>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/