Re: Plan 9 man added a new macro for man page references

[Ingo Schwarze]

Hi Alejandro,

Alejandro Colomar (man-pages) wrote on Tue, Aug 03, 2021 at 10:23:09PM +0200:
> On 8/3/21 4:30 PM, Ingo Schwarze wrote:
>> Alejandro Colomar (man-pages) wrote on Tue, Aug 03, 2021 at 09:29:14AM +0200:

[...]
>>   2. I find it easier to remember that a function name needs .Fn
>>      than that it needs .B, or that a command line argument needs
>>      .Ar than that it needs .I.  The amount of content type to macro
>>      mappings you need to memorize (or look up) is exactly the same
>>      for both languages, except that mdoc(7) has mnemonics and man(7)
>>      has not.

> That's true... partly.  I'd say that with a consistent man-pages corpus, 
> it would be easier to know what you have to use, just by looking at 
> other pages, but we don't have that at the moment (and it's one of my 
> long term goals here).

I agree with all of that.  Jason McIntyre and myself also routinely
counsel prospective authors to look at good manual pages for
inspiration for their own manual pages.  By the way, the same holds
for program code, too: reading good code in any programming language
is one good way to improve your own coding style and skills in that
language.

Note that such a complete corpus *is* available, even though not
as part of the Linux man pages project.  The main authors and
maintainers have been Cynthia Livingston (BSD CSRG / USENIX), Aaron
Campbell (OpenBSD) and Jason McIntyre (OpenBSD).  It covers almost
all of POSIX plus many additional functions and programs.  The
FreeBSD and NetBSD projects have manual pages evolved from the same
corpus, though maybe not quite as polished as in OpenBSD on because
they don't quite have the equivalent of Jason.

[...]
> There are a few considerations:
> 
> - Neither Michael nor I do have a lot of time these times (the next 
> release of the man-pages (5.13), which should be released soon, will
> be quite small because of that).

That's no doubt a valid consideration.

> - I have other more prioritary fixes for the pages:
>    - Fixing the includes in the examples
>    - Documenting types in system_data_types(7)
>    - Add a LIBRARY section (as in FreeBSD pages) to simplify the current 
> text that provides that information in a mix of sections.
>    - Some other consistency fixes.
> 
> - I would like to have the pages consistently use man(7) before ever 
> considering a language change.  The thing, and this is only a prediction 
> of mine, is that when we have the pages more consistent than now, it 
> should be even easier to use man(7), and incoming patches should be more 
> correct on average, so there will be less reason to move to another 
> language/dialect.

You are certainly right that consistency generally helps providing
good examples to patch submitters and consequently improves quality
and reduces everyone's workload.

> - There are some benefits, as it seems, but enough to make such a big 
> change?  Not sure.

The two main benefits are:

 1. Better markup.  Compare:

    https://man.openbsd.org/ksh.1

    which is an mdoc(7) page to man(7) pages like

    https://man.openbsd.org/as.1
    https://man.openbsd.org/cc.1
    https://man.openbsd.org/gdb.1
    https://man.openbsd.org/tic.1

    In particular, watch out for everything underlined - solid
    underlining indicates hyperlinks, dotted underlining indicates
    deep linking targets.  Also hover your mouse over marked-up
    words to see the syntactic function of the word displayed in a
    tool tip.  In all these respects, mdoc(7) works significantly
    better than man(7).  The few such features available in the
    man(7) pages are mostly based on crude heuristic guesswork
    and quite incomplete.

    Note that the HTML rendering engine of

    https://manpages.debian.org/

    is fully equipped to profit from all these features.
    So far, Michael Stapelberg merely chose to use his own CSS code
    instead of the one provided upstream, and that CSS ignores many
    of the possibilities.

 2. Powerful semantic searching, for example:

    # Pages containing cross references to "chmod"
    https://man.openbsd.org/?query=Xr%3Dchmod&apropos=1

    # Pages referencing C preprocessor macros containing INADDR
    https://man.openbsd.org/?query=Dv%3DINADDR&apropos=1

    # Programs having internal "bind" commands
    https://man.openbsd.org/?query=Ic%3Dbind&apropos=1

    # Pages mentioning files containing "fstab" in their name
    https://man.openbsd.org/?query=Pa%3Dfstab&apropos=1

    # Functions throming the ESRCH errno
    https://man.openbsd.org/?query=Er%3DESRCH&apropos=1

    # Programs using the COLUMNS environment variable
    https://man.openbsd.org/?query=Ev%3DCOLUMNS&apropos=1

    # Functions defined in <ctype.h>
    https://man.openbsd.org/?query=In%3Dctype.h&apropos=1

    # Software authored by Markus Friedl
    https://man.openbsd.org/?query=An%3DFriedl&apropos=1

    # Features available sinc AT&T Version 1 UNIX
    https://man.openbsd.org/?query=At%3Dv1&apropos=1

    Of course, semantic apropos(1) also works at the command line,
    but does not really work with man(7) files, i.e. pages written
    in the man(7) language are usually missing from the results.

If, and when, that's worth it is up to you, of course.

Such a translation project is definitely feasible.  Cynthia Livingston
translated to whole 4.3BSD manual page corpus from man(7) to mdoc(7)
for 4.4BSD.  Admittedly, she had a USENIX grant to support her work.
In the end, she had around 1300 pages.  With some help from Anthony
Bentley, i translated the whole LibreSSL manual page corpus from
perlpod(1) to mdoc(7), purely in my free time.  That's now about
525 pages.  The Linux man pages project seems to have around 2400
pages, which is certainly quite a significant size for such a
translation.

If, at some point, you decide to do it, you have good chances to
convince me to write an man-to-mdoc converter to assist you.  The
main reason that i haven't done that yet is that no large-scale
project seriously asked for it yet.  But i'm already maintaining
the reverse, that is, an mdoc-to-man converter, and also pod2mdoc(1).

There are two obvious ways to set up a man-to-mdoc cponverter.
One is as a part of mandoc(1), just like the existing mdoc-to-man
converter, using logic and heuristics similar to pod2mdoc(1) in
the output module.

The other is to combine esr@'s doclifter(1) with Kristaps Dzonsons'
docbook2mdoc(1).

Either way, be prepared to manually postprocess the whole text,
mainly because man(7) has sparse markup as a rule and much
meta-information is missing from it that cannot easily be autodetected.
But at least the full drudgery of converting everything purely by hand
can certainly be avoided.

Yours,
  Ingo