bug#64696: 30.0.50; indent-to inherits preceding text properties, including 'invisible

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: monnier@iro.umontreal.ca, 64696@debbugs.gnu.org
> Date: Tue, 25 Jul 2023 08:38:38 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> For example, consider an Org table like
> >> 
> >> | *This* | is | some text |
> >> | more |    |  text     |
> >> 
> >> It looks aligned in Org buffers ("*" is invisible), but not when copied
> >> to message buffer.
> >
> > Where does fontification enter this picture?
> 
> "*" is made invisible after fontification.

So these are not font-lock-* faces, right?

> > The important thing to remember is that Emacs makes all those
> > display-time transformation because that's how people want to see the
> > text on the screen.  It is very rare to see an application that wants
> > to show decomposed characters, as in a◌́ instead of á, or to see a TAB
> > shown as a single column.  Heck, even the display of control
> > characters, like , is part of this, and why would we want to turn
> > that off?
> >
> > IOW, the need for turning these off is extremely rare, and doesn't
> > justify such global toggles, because no one will use them.
> 
> I can see your point. However, this is sometimes conflicting with
> copying text verbatim or viewing it in other editors. For example,
> nameless-mode that visually compresses
> my-long-package-name-variable-name into :variable-name creates a lot of
> mess when the same file is committed to public repo and later opened by
> other contributors without nameless-mode enabled.

This might mean we need more intelligent commands that copy text
elsewhere.  However, note that it is OK for Emacs to expect other
software/media to share at least some of the visual features.  For
example, I have yet to see a text-rendering program that doesn't show
diacritics composed with the base characters, and most show Emoji as
you'd expect, not as a series of control codes.  OTOH, tabs could be
shown differently, especially if tab-width was customized.

But I think this is a relatively far tangent, as it is not immediately
related to indentation in Emacs itself.  In Emacs, indentation is a
display-oriented feature, so the few programming languages where
indentation is syntactically and semantically meaningful must do
something special when the buffer uses non-trivial Emacs display
features beyond tab expansion.  Invisible text, in particular, doesn't
go along well with such use cases.

> In the ideal world, Emacs would indent both visually and textually.

I disagree that this is the ideal.  "Textual" indentation is not
indentation at all, it is fundamentally a completely different feature
for completely different purposes.  Even if we agree that Emacs should
have it (and I don't yet agree), you shouldn't expect from the Emacs
indentation to fill this niche, except by sheer luck in some use
cases.

> With visual part only using 'display text properties that do not
> modify the actual text in file.

This will (a) not help you, because the issue of width of the
whitespace stretches will still be pertinent; and (b) will complicate
Emacs much more, because copying such "text" will become much more
tricky in general.

But if you want to work on adding this, please feel free.  It has its
uses, even for indentation; see, for example, pixel-fill.el.  It is on
our TODO to provide pixel-granularity indentation for text using
variable-pitch fonts (but this again is for visual appearance only).

> >> A toggle: disable all visual contributors.
> >
> > It will never be used.
> 
> I would use it in Org instead of `org-current-text-column'.

So we have one user.  And even in your case, you don't want them all
disabled: the character compositions, for example, should stay turned
ON.

> may not hold in future (the docstring implies that `string-width' may as
> well consider visuals: "Return width of STRING when displayed in the
> current buffer.")

Selective citation alert!  The full text of the doc string is:

  Return width of STRING when displayed in the current buffer.
  Width is measured by how many columns it occupies on the screen.
  Optional arguments FROM and TO specify the substring of STRING to
  consider, and are interpreted as in ‘substring’.

  When calculating width of a multibyte character in STRING,
  only the base leading-code is considered; the validity of
  the following bytes is not checked.  Tabs in STRING are always
  taken to occupy ‘tab-width’ columns.  The effect of faces and fonts
  used for non-Latin and other unusual characters (such as emoji) is
  ignored as well, as are display properties and invisible text.
  For these reasons, the results are not generally reliable;
  for accurate dimensions of text as it will be displayed,
  use ‘window-text-pixel-size’ instead.

What else do you want us to say, for you to understand that the
"visual" part here is quite limited?