Re: [lmi] Ancient rate-table programs: GIGO

[Vadim Zeitlin]

On Sun, 11 Dec 2016 18:38:45 +0000 Greg Chicares <address@hidden> wrote:

...snip...

GC> Thus, there are some checks that need not be performed during use in
GC> production, but must be performed during maintenance.

 This is probably true, but AFAICS the only reason not to perform these
checks in production is performance. And while it seems plausible that
doing these checks would indeed have a non-negligible effect on it, as
always when speaking about program performance, I'd feel better if we could
measure the difference before concluding that it's worth discarding these
checks.

GC> > GC> Here is a separate patch that isn't suitable for production in its
GC> > GC> present form, but I'm using it for experimental investigations.
GC> > 
GC> > ... [patch and its explanation snipped] ...
GC> > 
GC> >  I think it would make sense to integrate this patch into production,
GC> 
GC> Instead of "production", I should have said the git repository.
GC> The source code affected is excluded from the production system we
GC> distribute.

 Oops, yes, sorry for being imprecise, I did mean "production" in the same
sense as you here.

GC> For now at least, 'rate_table.?pp' code is not used in production.
GC> If it eventually replaces 'actuarial_table.?pp' in production, then
GC> we'll need to reconsider this: perhaps we'd add conditions that
GC> would suppress this code and also the code that throws if the CRC
GC> is invalid. But we don't need to figure that out now.

... although I'd also like this code to be used in "real production" code
too, as I said before, and still hope to see it happen. But I definitely
understand that this is not the most urgent concern.

GC> >  Should I do anything about this, e.g. work on refining this patch or help
GC> > with it in any other way?
GC> 
GC> No, thanks, I think I know how to do this, though of course I welcome
GC> your comments.

 I couldn't help asking myself whether converting the number to string and
counting the number of decimal digits in it was really the best way to
determine the number of digits necessary for its representation. And it
turns out that the answer is much more interesting than I'd have guessed.
Maybe you already know all this, but it turns out that the choice of the
best algorithm for doing this is still a subject of active research, with
the Grisu3 algorithm by F. Loitsch being published only in 2010 (I had at
least heard about this one before...) and the latest important paper on
this subject, "Printing Floating-Point Numbers (A Faster, Always Correct
Method)"[1] was published less than a year ago (and I had no idea about it
until 15 minutes ago...).

 I don't know what algorithm does the MinGW CRT implementation use, but
from a quick web search it seems it doesn't use Grisu3 and it's, of course,
impossible for it to use the algorithm only described in January 2016, so
it must be using either something much slower or non-optimal (i.e. not
always giving the shortest result). So if we want the fastest and optimal
answer, it seems like we'd need to implement either Grisu3 or the algorithm
from the paper above ourselves (I'm not sure if non-optimality of Grisu3
would affect our use or not, I didn't have time to read either of the
papers fully yet).

 So now I wonder if you were planning to do this from the beginning or if
you have another idea for solving this?
VZ

[1] http://cseweb.ucsd.edu/~lerner/papers/fp-printing-popl16.pdf