Re: Orgdown: negative feedback & attempt of a root-cause analysis (was: "Orgdown", the new name for the syntax of Org-mode)

[Tom Gillespie]

Karl,
   The exact naming of a thing is nearly always the most contentious
step in trying to promulgate it. In my own field we can easily get all
parties to agree on a definition, but they refuse to budge on a name.
As others have said, I wouldn't worry about kibitizing over the name.

I would however worry about the larger negative reaction. From my
perspective I think the issue is that there are many efforts working
toward a formalized specification for Org syntax and Org mode
functionality, and some of those stakeholders who have invested
significant effort may feel blindsided by a public declaration
announcing Orgdown because they were not consulted and not
made aware that you were working on it.

I appreciate the amount of work that you have put in, I have devoted
hundreds of hours to working on an alternate implementation of org
in Racket that uses a formal ebfn in hopes that others will be able
to use it as a guide and as a way to talk formally about how Org
parsers and implementations should behave.

It would thus be easy for me to say that your approach has put the
cart before the horse, because there are countless nuances in the
specification for Org syntax which must be addressed before any
levels of org compliance can be specified, otherwise the behavior
between levels will be inconsistent.

If I were to say this, it would not be fair to you at all. The ideas
and motivation for Orgdown are vital and important. You have put
in enormous thought and effort, all because you care about Org
and want to see it succeed.

The issue is that any shared specification for Org syntax is
fundamentally about how to coordinate as a community.
The way that Orgdown was presented to the community feels
(to me) like it is being imposed top down or coming from an
individual source, not from an open and visible community
process (the subject of your original email reads as a declaration
in english, and thus can be quite off putting, though I know that
was not the intention).

I personally haven't bothered with promulgation because I think
that we are not technically ready as a community to approach
outreach to other developers in a way that we can succeed.

The good news is that all of this can co-exist if we want it to,
but we need to be clear about our objectives as a community.

To me these objectives are as follows (and I would love
to hear from others about additional or alternate objectives).

1. To never fracture Org syntax so as to avoid the nightmare
of markdown flavors. (This means being able to say clearly
as a community that a parser is out of compliance and that
it is up to the user to fix their files. The ruby org parser used
by Github is a major issue here.)
2. To provide a clear specification for what graceful degradation
looks like when parsing Org syntax if a parser does not support
some portion of that syntax (e.g. should property drawer lines
be excluded or rendered as plain text?).
3. Provide a solid basis on which further formal specification
can be built. (My interests in particular are around providing
consistent semantics for org-babel blocks across languages
so that babel implementations can clearly communicate what
runtime features they support.)

The approach for Orgdown can absolutely meet all three of
these objectives, however in its current form Orgdown1 is not
sufficiently well specified to avoid fracturing the syntax.
This is because Org syntax is extremely complex (even the
elisp implementation of Org mode is internally inconsistent)
and there are edge cases where behavior will diverge if parsing
of even the simplest elements is not fully specified.

There are many ways to remedy this, however they require
a more formal approach. A number of us are working to build
technical foundations for such a formal approach, but I do not
think that any of those projects are ready to be used to
specify discrete levels of Org syntax parsing compliance.

If I may, I would suggest that an Orgdown0 is something that
could be well specified, but it would avoid parsing of markup
altogether and only deal with the major element types. Parsing
paragraphs and all the org objects is not something that can
be done piecemeal. There are too many interactions between
different parts of the syntax, and in some cases the existing
specification desperately needs to be revisited due to the
complexity that it induces or because it is underspecified.
Of course this would make Orgdown0 fairly useless as a
replacement for markdown, but at least it would be a start.

Best,
Tom