Re: [O] Citation syntax: a revised proposal

[Richard Lawrence]

Hi Aaron and all,

Aaron Ecay <address@hidden> writes:

> I’ve pushed an update to my branch.  The major change is to use
> citeproc-java for the generation of the bibliography and the parsing of
> names.

That is awesome!  Thank you for your work.

> The former is straightforward.  For the latter, I have created a CSL
> file which outputs author-year citations in an easy-to-parse format.
> These are then slurped by org, and used to fill in printf-style
> templates.  Some people mentioned using citations as generated by
> citeproc-java directly.  However, I don’t believe this is reliable
> since (as also mentioned), it is difficult to control whether a
> certain style uses parentheses around a citation or not, whether the
> citation is capitalized*, the insertion of prefixes/suffixes within
> the parentheses, etc.  So I think the only solution is to implement
> the formatting of the in-text portion of citations ourselves, and use
> citeproc-java only to extract authors and years.

I have actually been working on the same problem, using citeproc-hs as
the CSL processor instead of citeproc-java.  I took a (very) brief look
at your code; it seems like you are only communicating with
citeproc-java via command line arguments and stdout.  Is that right?

My approach to the problems you mention has been the following:
  1) Generate JSON from citation objects on the Org side.
  2) Pass that JSON to the processor via stdin.  
  3) Pass the output format, the CSL file, and the bibliography database
     to the processor via command line arguments.
  4) Return, to stdout, the formatted citations and the bibliography.
     These are formatted such that there is one citation or entry per line, 
     and a recognizable separator separates the citations from the
     bibliography. 

This allows passing formatting options for individual citations via the
JSON object for that citation, so it allows citeproc-hs to do more of
the work of formatting citations.

(See http://gsl-nagoya-u.net/http/pub/citeproc-doc.html#citation-data-object
for documentation of the citation data JSON format.)

I don't know whether this will ultimately be a good design, but the way
I am picturing it right now is that exporting citations will work sort
of like footnotes: the exporter will gather them all together as they
are encountered, then generate the JSON and run a single call to the CSL
processor at the end of the export process.  It can then replace the
citations in the document with the result from the CSL processor, and
insert the bibliography at the end of the document.

(The code is not very pretty yet, but it does generate citations and
bibliographies in both plain text and HTML, and it would be
straightforward to extend it to other output formats.  I can post it
somewhere if anyone is interested in taking a look.)

> Some people have talked about supporting other CSL processors.  I don’t
> see much utility in that, since CSL is a standard that all processors
> should implement faithfully.  

Indeed!  Though as you have observed, `should' and `already does' come
apart.  I doubt there are any implementations that are perfectly
complete.  So it may be worth thinking about how Org can talk to CSL
processors in a processor-independent way.  That way, different users
can use different CSL processors if one works better for their
particular document or environment.

I think the generate-and-pass-JSON approach is promising for that
reason.  That is what citeproc-js accepts as input (so maybe that is
what citeproc-java is doing internally?), and my code aims to allow
citeproc-hs to interpret the same JSON format as citeproc-js.  I don't
know Ruby, but I think it would be easy to make citeproc-ruby accept the
same JSON format.  Do you have a sense of how easy it would be to coax
citeproc-java into accepting JSON on stdin?  If it could be done, I
think that would give us a common format for talking to the
best-developed CSL implementations in a processor-independent way.

Best,
Richard