Re: Some LaTeX commands, such as \boldsymbol and \hspace, can't be completed in AUCTeX.

[Denis Bitouzé]

Hi,

Le 18/12/21 à 14h00, Mandar Mitra a écrit :

> Denis Bitouzé wrote (Wed, Dec 15, 2021 at 10:31:41PM +0100):
>> >> Agree, but tedious. That's why I proposed to benefit from the `.cwl`
>> >> completion files of TXS: it's also probably not so hard to provide
>> >> a converter (maybe I will have a look when I'll have some spare time).
>> >
>> > Highly welcome.  There will be a question reg. licence, but we can be
>> > worried about that later.
>> 
>> OK. I'll contact this guy in few weeks.
>
> Could you please post here when you do? If time and ability permits,
> I would like to help.

(Sorry for the delay.)

Would be very appreciated!

Here is an excerpt of our discussion (started yesterday):

Me:

  ┌────
  │ > I'd be happy to discuss it with you, although I can't say I have much
  │ > to offer from a technical standpoint. If you just had suggestions I'm
  │ > happy to hear those as well.
  │ 
  │ (Sorry for this long message.)
  │ 
  │ Here's why I wanted to reach out to you: you are doing incredible, huge
  │ and useful work! But, unfortunately, it is only useful for TeXstudio since
  │ the syntax of `.cwl` files is specific to this editor.
  │ 
  │ And this is a pity because it could be very useful in at least two
  │ contexts:
  │ 
  │ 1. obviously for other editors (for example, I know Emacs users, me for
  │    example, who would be interested) ;
  │ 2. in LaTeX courses or tutorials, it is useful to display LaTeX codes
  │    with the syntax highlighted, as it is displayed in TeX editors.
  │ 
  │ The second point is of particular interest because it can be useful to
  │ everyone and I'll address it now.
  │ 
  │ Syntax highlighting of LaTeX code can currently be done with essentially
  │ 3 packages I know of: `listings`, `minted` and `codehigh`.
  │ 
  │ - `minted` and `codehigh` nicely automate some of the work with the
  │   regexp, but they are not as accurate as they need to be: they are not
  │   able to highlight in a specific way specific (and not freely entered)
  │   key values. Let me give you an example: the `graphicx` package
  │   provides the `\includegraphics` command which accepts the `width` and
  │   `pagebox` keys. The values given to the former are free while the
  │   values given to the latter are not free and are limited to `mediabox`,
  │   `cropbox`, `bleedbox`, `trimbox`, `artbox`.
  │ - `listings` is better for this, but requires listing all the control
  │   sequences and keywords of (the kernel of) (La)TeX, classes and
  │   packages (in a similar way of what you are doing in the `.cwl` files). But
  │   this package declares only TeX and some dialects of it (see lines 1427
  │   to 1646 of the `lstlang3.sty` file). If someone wants to provide
  │   syntax highlighting for a given package, he must declare all of its
  │   control sequences and keywords, with the keywords divided into
  │   different classes according to their "role". Here is an example of the
  │   declaration I made for the `graphicx` package (the distribution in the
  │   different classes being perhaps suboptimal) and you can see its effect
  │   in the attached TeX and PDF files:
  │ 
  │ \lst@definelanguage[graphicx]{TeX}[LaTeX]{TeX}{%
  │   % Control sequences names
  │   moretexcs={%
  │     
DeclareGraphicsExtensions,DeclareGraphicsRule,graphicspath,includegraphics*,%
  │     includegraphics,reflectbox,resizebox*,resizebox,rotatebox,scalebox,%
  │   },%
  │   % Keywords of class 1 : keywords that contain other characters (since of 
the
  │   % same class as the ones specified as 'otherkeywords')
  │   morekeywords={%
  │   },%
  │   % Keywords of class 2 : environments names
  │   morekeywords=[2]{%
  │   },%
  │   % Keywords of class 3 : mandatory arguments (not environments) & optional
  │   % arguments which are keys (in key=value)
  │   morekeywords=[3]{%
  │     draft,final,hiresbb,demo,setpagesize,nosetpagesize,dvips,xdvi,dvipdf,%
  │     dvipdfm,dvipdfmx,xetex,pdftex,luatex,dvisvgm,dvipsone,dviwindo,emtex,%
  │     dviwin,oztex,textures,pctexps,pctexwin,pctexhp,pctex32,truetex,tcidvi,%
  │     vtex,debugshow,hiderotate,hidescale,alt,bb,bbllx,bblly,bburx,bbury,%
  │     natwidth,natheight,hiresbb,pagebox,viewport,trim,angle,origin,width,%
  │     height,totalheight,keepaspectratio,scale,clip,draft,type,ext,read,%
  │     command,quiet,page,interpolate,decodearray,origin,x,y,units,%
  │   },%
  │   % Keywords of class 4 : values of keys (in key=value)
  │   morekeywords=[4]{%
  │     mediabox,cropbox,bleedbox,trimbox,artbox,true,false,%
  │   },%
  │   % Keywords of class 5 : arguments specifications (after ":" in expl3 
syntax)
  │   morekeywords=[5]{%
  │   },%
  │   % Keywords of class 6 : current package name (and possibly derived 
packages)
  │   morekeywords=[6]{%
  │     graphicx,%
  │   },%
  │   % otherkeywords={},%
  │   % alsoletter={},%
  │   % alsodigit={},%
  │   sensitive,%
  │ }[keywords,tex,comments]%
  │ 
  │ I know you have already done a lot of work but I wonder if it would be
  │ possible for you to provide the `.cwl` files through (easily deriving
  │ from) a more general, abstract, easy to parse way of declaring
  │ everything you declare in the `.cwl` files (deriving from `.cwl` would
  │ probably be possible with regexp but it would be a pain and probably
  │ best reserved for the files you have already provided). I'm not sure of
  │ the best way to do this, but maybe YAML :
  │ 
  │   ┌────
  │   │ https://en.wikipedia.org/wiki/YAML
  │   └────
  │ 
  │ would be a good candidate (I have to admit I don't have experience with
  │ it but I would look into it if you were interested). Maybe other people
  │ would be interested in participating in such a job since it would not be
  │ specific to TeXstudio.
  │ 
  │ I thank you in advance to let me know if you are interested in such an
  │ approach.
  └────

Matthew's answer:

  ┌────
  │ I share your sentiments on the broader potential of this catalog of
  │ package commands and keys. Unfortunately I have essentially no
  │ experience with "serious" coding and am in a rather busy stage of my PhD
  │ program so I can't commit to any project at the moment. I certainly
  │ agree that in principle, one should be able to convert an existing
  │ `.cwl` into something more easily parsed.
  │ 
  │ Just to clarify, did you have in mind that with this file derived from
  │ a cwl, a script could be written to convert the data into the unique
  │ format of, say, `listings` code (like you provided) or the equivalent of
  │ a cwl for other editors? From just a cursory search it seems emacs
  │ (auctex) uses a format specified here: AUCTeX 13.1 (gnu.org)
  │ <https://www.gnu.org/software/auctex/manual/auctex.html#Adding-Macros>.
  │ 
  │ As I mentioned, I have no expertise in this area and am busy with
  │ research, but I will take a look at YAML and if anything comes of it
  │ I will let you know. Did you have any initial ideas regarding the format
  │ of this "generalized cwl"?
  └────

If you or other people have a suggestion about a good "abstract" tool
other than YAML, I would be happy to know it.

> Thanks,

You're welcome :)
-- 
Denis