Re: [lmi] actuarial tables format (was Re: Terse list of valuable projects)

[Greg Chicares]

On 2012-03-22 10:58Z, Václav Slavík wrote:
[...]
> <table>
>    <row age="1">0.198</row>
>    <row age="2">0.194</row>
>    <row age="3">0.190</row>
>    ...another 60 rows...
>    </age>
> </table>
> 
> Too verbose, to be read easily, redundant and error-prone [1],
> compared to implicit numbering in
> 
> <rows min="1" max="63">
>    <row>0.198</row>
>    <row>0.194</row>
>    <row>0.190</row>
>    ...another 60 rows...
> </rows>
[...]
> [1] It's easy to miss a row number or to list one twice. We could use
> Schematron assert (emebedded in RELAX NG grammar) to verify that the
> number of <row> elements is max-min+1 under my scheme, but I'm not
> sure if we can verify that the row numbers are sequential.

Or we could simply use C++ to test that. If it's cheap enough, this
could be a runtime assertion. Otherwise, we might want a unit test that
reads every xml data table and performs as many potentially-costly
tests as we like. (We'll probably want such a unit test anyway.)
(Of course, if row numbers are implicit, they can't be misstated.)

BTW, someday, when we design a GUI interface for managing these tables,
it would be very useful to show a graph of the data. Beyond the "wow"
factor, there's a strong practical reason: entering data manually is
prone to error, and a graph makes many errors evident. We'd want the
vertical axis to be logarithmic, at least as an option, because tables
most commonly represent mortality rates and mortality is exponential:
  http://en.wikipedia.org/wiki/Gompertz%E2%80%93Makeham_law_of_mortality

Similarly, it might be handy, someday, to display second differences
of the data, at least for one-dimensional tables:
  books.google.com/books?id=c4QRAAAAYAAJ&pg=PA57
| The great advantage of differencing is that any error, however small,
| in the quantities differenced, unless it follows a regular law, will
| be detected by the differences.

> Although on the other hand, it may actually be an advantage that gaps
> are permitted by this format...

Occasionally we have a table that's amenable to run-length encoding:
  ages 0-4  value 0.01
  ages 5-9  value 0.02
  ...
but I prefer to force each consecutive value to be specified, trading
space for uniformity, simplicity, and robustness in the table-lookup
code. That's easy enough for the few specialists who manage tables.
(OTOH, economy of expression is important with input sequences because
they're entered by end users: in that case,
  0.01,[0,5);0.02,[5,9); [...]
or, more simply,
  0.01 5; 0.02 10; [...]
is preferable to
  0.01 0.01 0.01 0.01 0.01 0.02 0.02 0.02 0.02 0.02 [...]
and the extra code complexity is warranted.)