Re: On zero width spaces and Org syntax

[Tim Cross]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> Hi all,
>
> It is usually recommended, as you know, to insert a zero width space
> character (Unicode U+200B) as a sort of delimiter mark to solve the
> scenarios of emphasis within a word (for example, =/meta/literature=)
> and others contexts where emphasis marks are not recognized (for example
> =[/literature/]=). I believe that as a puntual workaround it is not bad;
> however, I find it problematic that this character is part, more or less
> de facto, of the Org syntax. For two main reasons:
>
> 1. It is an invisible character, and therefore it is difficult to
> control and manage. I think it is not good practice to introduce this
> type of characters implicitly in a plain text document.
>
> 2. It is more natural that this type of space characters are part of the
> 'output' and not of the 'input'. In the input it is better to introduce
> them not implicitly but through their representation. For example, in
> LaTeX (with LuaTeX) using the command '\char"200B{}' (or '^^^^200b'),
> '&#x200B;' in HTML, etc.
>
> In any case, as an implicit character, I do not see it appropriate for
> the syntax of a markup language. The marks should be simply ascii
> characters, IMHO. So what if Org had a specific delimiter mark for the
> scenarios described above? For example, something like that:
>
> #+begin_example
>
> /meta/''literature
>
> *meta*''literature
>
> [''*literature*'']
>
> #+end_example
>
> WDYT?
>
> Best regards,
>
> Juan Manuel 

I think I am in agreement regarding most of your points about the use of
the zero-width character. I see it as a type of escape hatch which
provides a solution in some less frequent situations. It is a somewhat
clever kludge to enable markup in some situations not supported by the
basic markup syntax I'm happy with its status as a kludge and would not
want to see it become an official part of the syntax. Where we may
differ is in whether we actually want to add inner word markup support
at all. 

I'm somewhat surprised and more than a little concerned at how much
interest and focus on modifying the markup syntax of org the question of
inner word markup has generated. This seems to be a symptom of a more
general trend towards adding and extending org mode to meet the needs of
everyone and I'm concerned this is overlooking the key strength of org
mode - simplicity.

Consider how many times we have had requests for inner word markup in
the last 18 years. I've seen such requests only a very few times.
Certainly not frequently enough to consider modification of the markup
syntax to accommodate such a requirement.

A key philosophy of org mode is simplicity - it makes the easy stuff
simple and the hard stuff possible. The thing about simple solutions is
that they will inevitably have limitations. If you don't want those
limitations, then you use a more complex feature rich markup, such as
Latex, HTML, XML etc. Ideally, your system will provide some escape
hatches to allow you to do things not supported by the base markup
syntax. Those escape hatches will usually be less convenient and often
look quite ugly, but that is fine because they are an escape hatch
which is used infrequently. Better still is if the system provides some
way to make a specific escape hatch easier to use in a document (such as
via a macro). The basic org markup syntax has worked remarkably well for
18 years. Nearly all the proposed additions or alterations to support
inner word markup with complicate the syntax or introduce potential new
ambiguities and/or complexity in processing to support a feature which
has been rarely asked for and which has other, less convenient and often
ugly, solutions which work.

One of org's strengths has been the ability to export documents to
multiple formats. One way this has been made possible is by keeping the
markup syntax simple - a basic markup which is well supported by all
export back ends. Once you start adding more complex markup support, you
see a blow out of complexity in the export back ends. Worse yet, you get
results which are surprising to the end user or which simply don't work
correctly with some formats. to avoid this, it is critical to keep the
markup syntax as simple and straight-forward as possible, even if that
means some limitations on what can be done with the markup. 

My vote is to simply maintain the status quo. Don't modify the syntax,
don't make the zero space character somewhat special or processed in any
special way during export. In short, accept that inner word markup has
only limited support and if that is a requirement which is critical to
your use case, accept that org mode may not be the right solution for
your requirements.