[Groff] Re: grohtml and diversions

[Gaius Mulley]

Werner LEMBERG <address@hidden> writes:

> after some long thinking I believe that I can provide a solution to
> the problem how grohtml has to handle diversions without inserting
> unwanted whitespace.
> 
> The very problem is that we have to preserve the *state* of the troff
> engine partially for grohtml, otherwise important high-level
> information is lost.  For example, a call to `.ta' normally doesn't
> cause anything to be written, but grohtml needs this data to set up
> tables properly.
> 
> grohtml wants to know the following:
> 
>   .br
>   .ce
>   .fi
>   .fl
>   .in
>   .ll
>   .nf
>   .po
>   .ps
>   .rj
>   .sp
>   .ta
>   .ti
> 
> As soon as one of those requests is encountered, grohtml currently
> adds a special node (equivalent to \X'html-tag: ...') to the current
> input line.  In most cases this works, but within diversions it fails.
> Consider the following example (already shown in an earlier mail):
> 
>   .di xxx
>   .ll 5i
>   a
>   .br
>   .ll
>   .di
>   x\*[xxx]
> 
> Normally, the `.ll' right before ending the diversion just changes the
> state of troff without any side effects; however, if -Thtml is active,
> troff appends a special node (plus a newline input token) to the
> diversion which is clearly wrong.
> 
> What I've suggested in an earlier mail was not sufficient; I now
> think that more changes are necessary.
> 
> . First of all, troff must not emit a special html node if there is
>   nothing to print.  With other words, troff has to check right before
>   emitting a glyph whether the troff state has changed.  In the above
>   example, as soon as the letter `a' will be typeset, troff has to
>   check whether the line length has been modified and emit a special
>   html tag if necessary.  The second `.ll' thus doesn't produce a node
>   in the diversion.  Technically speaking, all calls to `add_html_tag'
>   have to be moved to `environment::add_char'.
> 
> . Secondly, the troff state must be emitted twice, once within the
>   diversion (using the internal equivalent of \!\X'...') and once
>   after ending the diversion.  Consider the following example:
> 
>     .di xxx
>     .ll 5i
>     a
>     .br
>     .di
>     x\*[xxx]
> 
>   Before starting the diversion `xxx', troff has to save the state
>   relevant to grohtml on a stack.  Then the line `.ll 5i' makes troff
>   emit something like
> 
>     \!\X'html-tag: .ll 5i'
> 
>   as soon as the letter `a' will be added to the current line.  After
>   ending the diversion, troff has to compare the saved state with the
>   current one.  It will discover that the line length has changed, and
>   it will emit
> 
>     \X'html-tag: .ll 5i'
> 
>   as soon as the letter `x' will be added to the current line.
> 
> . Thirdly, troff has to keep track of the values emitted with
>   `html-tag:' (let's call this the html state -- this is something
>   different from the state of troff described in the previous item).
>   If a diversion is read, troff must save the html state before
>   inserting the diversion into the input token stream and restore it
>   afterwards.  Consider this example:
> 
>     .di xxx
>     .ll 5i
>     a
>     .br
>     .ll
>     .di
>     .ll 3i
>     x\*[xxx]
> 
>   While `xxx' is read, the \!\X'html-tag: .ll 5i' is finally
>   expanded.  After ending the diversion, grohtml still assumes a line
>   length of 5i, so troff must emit the internal equivalent to
> 
>     \X'html-tag: .ll 3i'
> 
>   immediately afterwards.

Hi Werner,

I've been drinking plenty of black coffee and thinking hard about
these issues..  I wonder whether a different tack might be taken.
What about defining a new class, say `tag' which can be chained to
other tags and the head of this tag list can be stored on a glyph
node, or on diversion node. When one of

>   .br
>   .ce
>   .fi
>   .fl
>   .in
>   .ll
>   .nf
>   .po
>   .ps
>   .rj
>   .sp
>   .ta
>   .ti

is seen a tag is created and appended to the current list of
outstanding tags.  When a glyph is created then the current list of
outstanding tags is placed into the glyph. As tags are placed onto
glyphs they cannot have any white space surrounding them. A tag may be
created in a complete state (the value is known) or an incomplete
state in which case the value must be discover just before they are
written to post-grohtml.

In your first example we have:

>   .di xxx
>   .ll 5i                  <tag .ll 5i complete> and put on outstanding list
>   a                       outstanding list is placed onto glyph a's list. 
> outstanding list is reset.
>   .br                     <tag .br complete> and added to outstanding list
>   .ll                     <tag .ll incomplete> and appended to outstanding 
> list
>   .di                     outstanding list placed onto diversion, xxx, tag 
> list. outstanding list is reset.
>   x\*[xxx]                glyph x is emitted and the diversion xxx is 
> processed.
                            <tag .ll 5i is emitted> (was found on `a' list)
                            glyph a is emitted
                            now the tags found on diversion xxx are emitted
                            <tag .br> emitted 
                            <tag .ll incompete> is discovered, therefore 
evaluate this
                                                and emit tag.

second example

>     .di xxx
>     .ll 5i                <tag .ll 5i created> and put on outstanding list
>     a                     outstanding list is placed onto glyph a's list. 
> outstanding list is reset.
>     .br                   <tag .br created> and added to outstanding list
>     .di                   outstanding list placed onto diversion, xxx, tag 
> list. outstanding list is reset.
>     x\*[xxx]              glyph x is emitted and the diversion xxx is 
> processed.
                            <tag .ll 5i is emitted> (was found on `a' list)
                            glyph a is emitted
                            now the tag found on diversion xxx is emitted
                            <tag .br> emitted 

third example

>     .di xxx
>     .ll 5i                <tag .ll 5i created> and put on outstanding list
>     a                     outstanding list is placed onto glyph a's list. 
> outstanding list is reset.
>     .br                   <tag .br created> and added to outstanding list
>     .ll                   <tag .ll incomplete> and appended to outstanding 
> list
>     .di                   outstanding list placed onto diversion, xxx, tag 
> list. outstanding list is reset.
>     .ll 3i                <tag .ll 5i created> and put on outstanding list
>     x\*[xxx]              glyph x is emitted and the diversion xxx is 
> processed.
                            <tag .ll 5i is emitted> (was found on `a' list)
                            glyph a is emitted
                            now the tag found on diversion xxx is emitted
                            <tag .br> emitted 
                            <tag .ll incompete> is discovered, therefore 
evaluate this
                                                and emit tag.

do you think this would work with other cases? Or have I missed something..

Gaius