Re: build system: devpdf/download regression

[G. Branden Robinson]

Hi Ingo,

At 2022-06-21T15:28:03+0200, Ingo Schwarze wrote:
> I see.  Actually, that's a typical situation in OpenBSD ports builds.
> For many (if not most) ports, build dependencies and run dependencies
> differ, and it isn't even unusual that none of them is a superset
> of the other.  At build time, you cannot rely on having access to
> mere run-time dependencies; at run-time, you cannot rely on having
> access to mere build dependencies.  I would expect that many other
> packaging systems employ similar concepts.

They do--but I'm used to the Debian approach where a package build host
has _all_ the possible dependencies, and the binary package then has
those dependencies' runtime counterparts, or where that's not easy to
support, or stripped-down or alternative build configurations are
desired, multiple builds are conducted in separate portions of the build
tree, and binary packages prepared from those.

> On OpenBSD, ghostscript is neither a build dependency nor a run
> dependency of groff.  But if the end-user installs it, they can
> can use it at run time, including with groff, and i suspect most
> users do that.  On OpenBSD, no special marking is needed in such
> a situation, but i believe some other packaging systems call this
> kind of thing an "opional dependency" or a "recommended package".

Yes.  Debian's dpkg has had "Depends:", "Recommends:", and "Suggests:"
since the early-to-mid 1990s to reflect differing "intensities" of
requirement, if you will.

> Nice, so it appears you audited for other instances of similar
> problems, too.  Thanks for doing that!

It felt like the right thing to do...

> >> The quick and dirty patch appended below fixes the immediate problem
> >> in my partciular setup.  Now all the expected devpdf files get
> >> built and installed, including the "download" file and the individual
> >> font description files like TR, TB, CR and so on.
> 
> > Importantly, this should ship the U-* font descriptions if they are
> > present and not if they aren't.
> 
> That's bad because it might result in erratic packaging behaviour,
> namely different package content depending on what happens to be
> [i]nstalled at package build time.

Yes, it very much will.  This is by design as far as I can tell.

> > This does pose a difficulty for your build scenario; you must decide
> > whether you want to cater to users wishing to run gropdf using the
> > URW fonts, or not.
> 
> So far, i never cared to educate myself what "URW fonts" even are,
> or what they might be useful for.  I know there have repeated been
> messages about those fonts on this list, but i never bothered to
> read them.

Here's my understanding.  You may already have some of this background.

Back in the 1980s, when Adobe promulgated PostScript, they defined a set
of fonts that every conforming PostScript client (read: mostly printer
manufacturers) would have to have (read: license, for money, from Adobe)
to produce documents.

Those fonts have never been Free Software, so eventually,
freely-licensed alternatives were developed.

All groff and some other tools need to produce a PS or PDF document is a
_description_ of the fonts: these are called "metrics".  groff says "put
a 12-point Times roman 'a' here" and it's up to the PS/PDF renderer,
along with the font--which for the standard fonts was always expected
to be bundled with the _renderer_, not the document--to draw the glyph
on the page.  Among other things, the metrics allow groff to advance the
drawing position by the correct amount to write the next one.

Printer manufacturers had already paid their license fees to Adobe and
any other font foundries they cared to,[1] but for free software
renderers, this situation was a problem.  You'd have to "pirate" the
fonts or possibly extract them from the printer firmware, copy them to
your rendering host, and the rendering software would have to know where
to find them and how to use them.

Nowadays, Adobe is saying that PDF documents should stop relying upon
any rendering device to have any particular fonts, and just embed all
the fonts they need.[2]  This was already true if you used any but the
base 14 (or 35--the number has grown over time).[1]  This is a technical
improvement because while it makes the files larger, it also makes
rendering far more reliable.

If you want to embed a font in a PostScript or PDF document, you must
have access to its definition, its font file.  The problem is that
passing around copies of Adobe's fonts in your documents, mandatory
though they were, was frowned upon by their licensing and revenue
department.[1]  (Perhaps to avoid getting our users into trouble, or
maybe just to keep file sizes down, gropdf(1) does _not_ default to
embedding fonts.  You have to give it the `-e` option.  Maybe we should
change this default at some point.)

The URW fonts are reimplementations of the proprietary Adobe fonts that
renderers were historically required to have.  Apart from looking
similar, they are supposed to have compatible metrics.  So, those of us
in the free world who want (1) higher document rendering reliability,
(2) better conceptual encapsulation of a document with its resources,
and/or (3) to keep living in the free world, use those fonts instead of
Adobe's.

I therefore suspect that many OpenBSD users will be keenly interested in
having a groff build that is aware of the URW fonts if they aim to
produce PDFs with the program.

> > If so, you will need to add a build dependency on them.
> 
> Hum.  None of these files exist anywhere in the OpenBSD ports tree:
> 
>   URWGothic-Book.t1 URWGothic-Book.pfb URWGothicL-Book.pfb
> 
> The following file does exist in the print/ghostscript/gnu-fonts port:
> 
>   /usr/local/share/fonts/ghostscript/a010013l.pfb
> 
> Even when the port is installed, building groff from git does not find
> the file because /usr/local/share/fonts/ghostscript/ is not in the gs(1)
> search path (at least not as reported by gs -h).  When i add
> 
>   --with-urw-fonts-dir=/usr/local/share/fonts/ghostscript
> 
> to the ../configure invocation, these lines change in ../configure
> output:
> 
>   -checking for URW fonts in Type 1/PFB format... none found
>   +checking for URW fonts in Type 1/PFB format... \
>       found in /usr/local/share/fonts/
> 
>   - use URW fonts for PDF output     : no
>   + use URW fonts for PDF output     : yes

That much is good...

>   -configure: URW fonts in Type 1/PFB format were not found.
>   -[... long paragraph ...]
> 
> But then i get this at build time:
> 
>   +BuildFoundries: warning: line 33: Unable to locate font(s) 
> URWGothic-Demi.t1,URWGothic-Demi,URWGothicL-Demi,a010015l.pfb
>   +BuildFoundries: warning: line 34: Unable to locate font(s) 
> URWGothic-DemiOblique.t1,URWGothic-DemiOblique,URWGothicL-DemiObli,a010035l.pfb
>   +BuildFoundries: warning: line 35: Unable to locate font(s) 
> URWGothic-BookOblique.t1,URWGothic-BookOblique,URWGothicL-BookObli,a010033l.pfb
>   [...]

It looks like your Ghostscript font installation is out of sync with
groff's expectations.  Is it subsetted in some way?  Here's what I have
on my Debian bullseye-based system.[3]

> and many more similar messages, and then:

>   +BuildFoundries: warning: line 75: 
>   +The path(s) used for searching:
>   +ARRAY(0x23cef899970)

Urp.  The form of that output is certainly bogus.  Did I break it in a
recent change?  I'll check.

> All the same, later, there is also
> 
>   +BuildFoundries: notice: Generated U-AB...
>   +BuildFoundries: notice: Generated U-ABI...
>   +BuildFoundries: notice: Generated U-AI...
>   [...]

That's normal.

> and many similar messages, intermixed with many messages like the following:
> 
>   +afmtodit: both Tcommaaccent and uni0162 map to u0054_0327 \
>       at /usr/local/bin/afmtodit line 6555.
>   +afmtodit: both tcommaaccent and uni0163 map to u0074_0327 \
>       at /usr/local/bin/afmtodit line 6555.

Let's come back to this issue later.  I think I understand it but first
I'm curious to know what version of the gsfonts package (or equivalent)
you have.  Maybe I can get my hands on it and reproduce some of this
stuff.  I can say that I don't see any of this chatter in my builds.
The one instance of it that I did see, I recently fixed.[4]

> Apparently, nothing changes during "make dist".

That's not a surprise.  This URW font business shouldn't affect
production of the distribution archive, just the set of artifacts
generated by building the source tree.

> Then still later, at port build time, if i add a build dependency
> on print/ghostscript/gnu-fonts (which might be acceptable because it
> has no dependencies) and also add
> 
>   CONFIGURE_ARGS += --with-urw-fonts-dir=/usr/local/share/fonts/ghostscript
> 
> to the port make file, the line
> 
>    use URW fonts for PDF output     : no
> 
> in ./configure output does *not* change,
> but i get the "Unable to locate font(s)" messages mentioned above and
> the weird "ARRAY(0x23cef899970)" message anyway, but *not* the
> "Generated U-*" and "commaaccent" messages, and the set of installed
> files does *not* change compared to running without --with-urw-fonts-dir.

It sounds like this gnu-fonts thing doesn't look anything like what the
groff build system expects.

> I don't have the slightest idea whether any of this is expected or wrong...

I've seen variations on all of these except the Perl ARRAY(0x000) thing.

> > If not, those users will have a tough time until we can get
> > user-facing tooling (the infamous "install-font" problem) fully
> > developed and supported.  The short path is probably for them to
> > build groff from source and reënable some of the stuff you're
> > turning off.
> 
> That doesn't seem simple at all, even i didn't really figure out so
> far what it might be that they would have to do at build time...

It shouldn't be crazy hard.  There are merely some tedious challenges in
the way.  I have some pending commits revising the "Font installation"
sections of the grops(1) and gropdf(1) pages to give people more of a
fighting chance.

> Besides, we strongly discourage installing anything on OpenBSD without
> a port, so that would be a really bad option.  Even if a user wants
> to install something just for themselves, we recommend they build a
> port, even if they don't publish it.
> 
> Do you think anybody might actually want to have these fonts?

Yes, if they want to produce PDFs with groff and also want either to (1)
follow present recommended best practice per Adobe or (2) use any fonts
that aren't in the standard set (this is also documented in the
gropdf(1) man page under "Usage").

> So far, i don't recall anyone asking about them on OpenBSD lists.
> If the answer is "yes", i might have to talk to one of our developers
> who specialize in porting fonts whether they can prepare and commit
> a well-designed port of the URW fonts and commit that (of course,
> that port must not have any build dependencies).  Then i could make
> our groff port depend on that URW fonts port with the next groff
> update.

The good news is that this font data is just that--dumb data.  I can't
think of why they'd have any dependencies at all.  All they need is a
directory to be dumped into.

> If the answer is "no" then i guess i need to find a way to reliably
> disable all the URW stuff such that it doesn't accidentally get
> triggered depending on what someone might happen to have installed,
> ideally also reducing the considerable noise that BuildFoundries
> spews at build time.

That sounds like adding a flag to our "configure" script.

--ignore-urw-fonts?  --without-urw-font-dir?

Maybe the Autoconf manual has a suggestion for this general class of
problem.

> Thanks for checking; for now, i pushed it.
> When you find a better way, you are welcome to polish it futher!

First I want to add a regression test for this (and devhtml), so that
we're sure all the font description files that _should_ get built
unconditionally actually do.  URW foundry support for gropdf(1) is a
layer on top of that.  The former stuff should never be broken, but
that's what you found.

Regards,
Branden

[1] The economically-minded reader might want to reflect on the
    phenomenon of "monopoly rents" at this point.
[2] https://lists.gnu.org/archive/html/groff/2020-05/msg00028.html
[3] https://packages.debian.org/bullseye/all/gsfonts/filelist
[4] 
https://git.savannah.gnu.org/cgit/groff.git/commit/?id=0c60e80fbcee361d34ab6001241a96fc189cc317

signature.asc
Description: PGP signature