lmi
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [lmi] Embed {{MST}} and <html> in product database

From: Greg Chicares
Subject: Re: [lmi] Embed {{MST}} and <html> in product database
Date: Mon, 22 Jul 2019 18:19:37 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.1 
On 2019-07-21 12:36, Vadim Zeitlin wrote:
[...]
> On Thu, 18 Jul 2019 18:24:49 +0000 Greg Chicares <address@hidden> wrote:
[...]
> GC> > Is the problem you're trying to solve that it is only supposed to 
> contain
> GC> > text, and not HTML, currently, while you'd like to use HTML inside it?
> GC> 
> GC> In part, yes. More generally, I'd also like text substitutions
> GC> such as mustache offers.
> 
>  Trying to understand the reasons for my instinctive dislike of your
> initial proposal, I've realized that I don't actually have any objections
> to using text substitutions in the policy files. My main problem is with
> putting HTML inside them, as this just seems like a completely wrong level
> for this: policy files are supposed to contain business logic, not
> presentation-level decorations. I.e. rather than having
> 
>         <br><b>Title:</b><br> Contents
> 
> in the policy file, I'd much prefer to have "Title" and "Contents" as 2
> separate policy fields and then have
> 
>         <br><b>{{Title}}</b><br>{{Contents}}
> 
> in the .mst file. This would preserve the separation between the "model"
> and the "view" parts and would avoid situations where you'd need to
> recreate the policy file just to change bold to italic or change the size
> of a font, instead of simply editing the .mst file directly.

We don't need all the flexibility of html in the policy files. All we need
is <b>, along with either <br> or <p>. If there were some easy way to write
those in mustache syntax, e.g.:
  {{??The pilcrow makes this a separate paragraph.}}
that would be good enough. Or, instead of devising our own weird mustache
dialect with '{{??' as an atom, we could use special characters to indicate
markup, e.g.:
  "????Some technical term??: its narrative definition."
which would become
  <p><b>Some technical term<b>: its narrative definition.</p>
in PDF output.

>  But -- just in case you thought I was unusually accommodating by
> explaining why I strongly dislike just one half of your proposal instead of
> all of it -- even though I don't have any objections to supporting text
> substitutions in the policy files, I still have some reservations about
> using Mustache for them, because it seems to do too much. Do we really need
> support for Mustache sections (i.e. conditionals)? I'm almost sure we don't
> need support for partials (inclusions) and I don't really know what could
> we possibly do if we encountered one inside a policy file field. And even
> though they're harmless, supporting comments here doesn't seem useful
> neither. So I wonder if we shouldn't use something simpler, like just
> "$FIELD" shell-like expansions in the policy fields?

I can't imagine we'll want partials, but mustache conditionals might be
really useful.

However, we're already doing html and mustache processing, so adding a
new variety of expansions would increase complexity and take more effort.

> GC> If we had the flexibility to use html and mustache markup
> GC> in '.policy' strings, as well as in '.mst' templates, then
> GC> we could reduce overall complexity.
> 
>  Would using HTML really help that much? At least in example you present,
> the gains come purely from being able to use text substitutions, which is,
> I agree, definitely useful (whether we use Mustache syntax for them or not
> being a secondary question).
> 
>  To summarize, thanks to your patient explanations, I think I do understand
> the problem now, but I still very much dislike the idea of putting HTML
> inside the policy files because of the violation of separation of layers
> concerns and also, to be honest, because it's just plain ugly to use quoted
> HTML inside XML (and using CDATA wouldn't be much less uglier). So my
> question is whether we could get all, or most of, the gains you envision by
> just allowing text substitutions in them or if per-product alterations that
> you've mentioned may really require you to make an extra word bold instead
> of just changing the wording?

Motivating use case: each illustration includes a "dictionary", somewhat
like this:

  <P><B>Do:</B> A deer. A female deer.</P>
  <P><B>Re:</B> A drop of golden sun.</P>
  <P><B>Mi:</B> A name I call myself.</P>
  <P><B>Re:</B> A long, long way to run.</P>

[Some don't yet use that formatting style, but eventually I think they
all will.] The number of defined terms is unbounded; so is the scope for
gratuitous variation:

  My name.

  a name I call myself

  The full legal name of to the principal insured, or other natural
    person as defined in policy form #2.718281828459045 by XYZ Corp.

  A Name I Call Myself.

The present method requires '.policy' strings
  DoName,DoText
  ReName,ReText
  MiName,MiText
  FaName,FaText
  ...
for each of perhaps dozens of notes (think of 41-TET
  https://en.wikipedia.org/wiki/41_equal_temperament
rather than a diatonic scale). But it gets much worse: potentially
every such item also requires '.database' booleans
  HasDo, HasRe, HasMi...
to suppress it conditionally, and then each '.mst' file requires
  <p>{{#HasDo}}<B>{{DoName}}: </b>{{DoText}}{/HasDo}}</p>
for each note. And each of those items has to be added not only
to one of the product classes:
  database_entity HasDo;
  glossed_string DoName;
  glossed_string DoText;
(and their respective product editors), but also to an appropriate
ledger class:
  bool HasDo = database().query<bool>(DB_HasDo);
  std::string DoName = product().datum("DoName");
  std::string DoText = product().datum("DoText");
Then adding a single note to a single product causes all generated
product files to change, as well as all regression-test output, even
though in most cases we just added a blank string. That's how we've
been doing things for ages, and it's more or less manageable if we
add only one or two items a year. But now we need to cope with a
thoroughgoing revision of everything, so a new paradigm is wanted.

That will bring us back to:
  <P><B>Do:</B> A deer. A female deer.</P>
We really need a simple way to express this. The specification is
not that "Do" exists, and has a name, and has a definition, and it's
definition is to be formatted in a particular way...and that "Re"
exists, and..."Mi"... . Rather, it's that we have a sizable set of
definitions, and they're all to be formatted in a particular style:
 - name in bold, definition not in bold; and
 - each as a separate paragraph; and
 - nothing at all (no empty paragraph) if the definition is empty.
In that context, a good separation of concerns would express this
formatting as some sort of stylesheet, and I'd be glad to approach
it that way if you can think of a way to do it. But adding distinct
variables for
  {Do,Re,Mi...} X {Name, Definition, IsEmptyFlag}
to the product files, and the ledger classes, and passing all the
microscopic details around would amount to atomization of concerns;
it's a wonder that we've made that work so far, but it's just not
sustainable given the massive replacement of the business logic
that's being demanded now.

We'd be much better off with
  std::string DefinitionsOfTerms = "{{Defn0}} {{Defn1}} ...";
because we'd only have one object to pass around, and adding a new
"{{DefnN+1}}" wouldn't require structural changes to lmi (which
wouldn't even need to be recompiled), and only one product file
and one regression test would change.

>  Sorry again if this is not particularly useful, I understand that you were
> probably looking for an advice about how to better implement what you had
> in mind rather than an advice about how to do something completely
> different, but mixing HTML into the product file fields is so unappealing
> to me, that I'd really like to at least explore the alternatives before
> committing to it.

We've already begun working on a massive set of changes. Reviewers
want to see actual generated PDF files rather than manual mockups.
And they want to see changes grouped in layers that make sense to
them in the problem domain, which often differs from what's easy to
do in the solution domain. There's a lot of give and take, and both
sides try to accommodate the other, but this project is complicated
and vast, and it's already underway. So we must quickly resolve the
technical issues we're discussing here--otherwise, we'll be gritting
our teeth and adding more commits like 8013537036b just so we can
keep moving forward (because, in that case, "death benefit options"
have mutable names that must be used in their definitions; and those
names are atoms in the "death benefit option" molecule, but the
ledger class isn't porous enough to allow the whole molecule to pass
through, so we have to pulverize it in the product files and then
reassemble it in the MST files).

How can we move forward now, without that much labor?

Of the two ideas presented, for strings in '.policy' files:
 (1) allow mustache substitutions
 (2) allow markup for boldface and paragraphing
it seems that:

 - You don't strenuously object to (1)...so can we decide now how to
implement it? E.g., invoke a function twice, as in my experimental
patch; or rewrite the function so that it doesn't need to be called
twice?

 - As for (2), is there some simple-enough way to get the boldface
and paragraph separation we need, without using html? Would it be
better to use pilcrows and guillemets as above, or does that just
amount to reinventing html, poorly? Or is there some way to design
a "stylesheet" that does what we need, while keeping presentation
and content separate?

If we don't have time to figure out the best solution now, should
we just forge ahead along the lines of my experimental patch, in
the doubtful hope that we'll come back later and redesign it?

Or perhaps I've now been able to persuade you that html isn't such
a bad solution to (2) after all, given the scope of the undertaking;
if so, then should we replace that experimental patch with an
implementation that does the same thing in a less awful way?
reply via email to
[Prev in Thread] Current Thread [Next in Thread]