Re: [lmi] Generating pages with tables in the new PDF generation code

[Vadim Zeitlin]

On Thu, 17 Aug 2017 02:32:38 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2017-08-15 23:01, Vadim Zeitlin wrote:
GC> > 
GC> >  I've finally finished writing and tidying up code for generating pages
GC> > with tables in the new PDF generation code and would like to hear your
GC> > feedback about it if possible.
GC> 
GC> Is your new code yet capable of producing actual PDF output,

 Oh, yes, definitely, I'm not nearly good enough to be able to write so
much code without testing it. So I've been testing it continuously ever
since it was only capable of outputting a PDF containing a single page with
"Hello, world" on it and right now the code creates a pretty faithful
approximation of the regular individual illustrations. I haven't tested it
with composite illustrations yet and haven't written support for the other
illustration kinds (nasd etc) however.

GC> so that we could test it ourselves? It doesn't matter whether it's
GC> complete or polished.

 How would you like to test it? I could just build a binary for you. Or you
could get the direct-pdf-gen branch from my GitHub repository (you don't
need a GitHub account for this) and build it yourself -- but please wait
until I test building it with MinGW first as I hadn't done it yet, I've
been only testing with MSVC and gcc/clang under Linux.


GC> What does this system look like from the top level?

 I think the code here:

https://github.com/vadz/lmi/blob/81ad738e3c5ba6ad580245d77397db91349a9bb0/ledger_pdf_generator_wx.cpp#L2194-L2208

should answer this question, but please let me know if it doesn't.

GC> Is there some prime
GC> mover that invokes template-processing code for narrative sections, as in
GC> this simple example...
GC> 
GC> >  To make this more concrete, here is how the template for the "Tabular
GC> > Detail" page looks like:
GC> > 
GC> > 
https://github.com/vadz/lmi/blob/97ddfb2e7532a49b4ac367b6e8b7e3f2d96e9f4c/tabular_details.mustache
GC> > 
GC> > (notice that it has no trace of the table in it)
GC> 
GC> ...and invokes C++ code (example below) for the tables? If that's a
GC> correct guess as to the top-level flow, then how does it know how to
GC> interleave narrative and tabular data? Is the interleaved pattern
GC> hard coded?

 Yes.

GC> Alternatively, would it be possible to generate HTML tables as you're
GC> doing now, and treat them as data that might be passed into a mustache
GC> template? IOW, does mustache permit blobs of already-formatted HTML,
GC> sort of like XML CDATA, so that the whole page could be represented as:
GC> 
GC> {{>header}}
GC> 
GC> <p align="center">Tabular Detail</p>
GC> 
GC> <p align="center"><font size="-1">
GC> Values shown are in {{#HasScaleUnit}} {{ScaleUnit}}s of {{/HasScaleUnit}} 
dollars </font></p>
GC> 
GC> {{HtmlBlobRepresentingOnePageOfTabularData}}
GC> 
GC> <br></br>

 This is possible and is basically what I'm doing for the numeric summary
page (not checked in yet) whose table is supposed to always fit on one page
(please let me know if this simplifying assumption is too optimistic).
However for longer tables, such as the one on the tabular details pages,
this would requiring implementing pagination at wxHTML rendering level
which is more complicated because wxHTML doesn't have any notion of headers
and footers (which would need to be repeated on each new page) and doesn't
know about our row grouping neither, so it's really not obvious how to do
it there.

 The current solution is an ugly but pragmatic compromise between
template-based and hard-coded approaches which allows to tweak relatively
minor illustration details easily by editing external templates, but still
requires modifying C++ code for many other changes, such as adding a new
page or adding a new "big" table for an existing page.
 
GC> That sounds good. Code that formats these tables has to be fairly
GC> complex, but probably the only difference between one table and
GC> another is the particular columns it contains.

 It's not that complex, formatting code is almost all in these (pretty
short) 30 lines:

https://github.com/vadz/lmi/blob/81ad738e3c5ba6ad580245d77397db91349a9bb0/ledger_pdf_generator_wx.cpp#L1765-L1798

but there is also pagination code (below) and part of the logic is in the
output_and_check_for_page_break() function and repeating it seemed very
unappealing.

GC> It'll take a while to read all this code, but I think I've read
GC> enough to guess the answer to a question I posed above.

 Sorry, I think I confused you once again. I forgot to warn you that the
current code still wasn't fully updated to use the new approach, so there
is a mix of old and new-style pages in it. The first couple of pages
illustrate the old approach, while everything starting from
columns_heading_page uses the new one.

 I wish I could say I had done it on purpose, so that you could compare and
contrast the render() method of narrative_summary_cont_page():

https://github.com/vadz/lmi/blob/81ad738e3c5ba6ad580245d77397db91349a9bb0/ledger_pdf_generator_wx.cpp#L1372-L1681

(as seen from the URL, it's 300+ lines) with the render() method of (quite
comparable) columns_heading_page:

https://github.com/vadz/lmi/blob/81ad738e3c5ba6ad580245d77397db91349a9bb0/ledger_pdf_generator_wx.cpp#L1684-L1703

which is less than 20 of very simple lines. But even though it was an
accident, I still think it's a good illustration of why I prefer the new
approach to the old one.

GC> Seeing this, beginning at line 910 in the '.cpp' file:
GC> 
GC> text header_html = interpolate_html
GC>         (R"(
GC> {{#IsInforce}}
GC> LIFE INSURANCE IN FORCE BASIC ILLUSTRATION
GC> {{/IsInforce}}
GC> {{^IsInforce}}
GC> LIFE INSURANCE BASIC ILLUSTRATION
GC> {{/IsInforce}}
GC> )"
GC> 
GC> I'm guessing that the driver is C++,

 Yes, it is here, but this is old/original version which just hasn't been
updated yet.

GC> But I'm probably missing something, because 'tabular_details.mustache'
GC> is a separate file that isn't referenced in the C++ code. If you could
GC> explain the top-level code structure in a few sentences, that would
GC> greatly help me understand the context of these source files.

 The plan is to have separate .mustache files for all pages (as well as
header, footer, and anything else we may wish to reuse between pages as
Mustache templates can include each other). It just hasn't been completed
yet.

 Sorry again for confusion and I hope things are more clear now -- but
please let me know if you have any further questions!

VZ