bug#44486: 27.1; C-@ chars corrupt elisp buffer

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: larsi@gnus.org,  thievol@posteo.net,  handa@gnu.org,
>   schwab@linux-m68k.org,  44486@debbugs.gnu.org
> Date: Sat, 14 Nov 2020 10:19:57 -0500
> 
> >> Actually, for prefer-utf-8 files, I think we never want to automatically
> >> fallback to binary.
> > I think you are assuming prefer-utf-8 is something other than what it
> > is.  It is not a variant of UTF-8, it is a variant of 'undecided'
> > (i.e. it starts by detecting the encoding), which prefers UTF-8 if
> > that can decode the text.
> 
> My position is not based on principles but on pragmatic concerns.
> AFAIK `prefer-utf-8` is only ever used for files which are known to
> contain text and should almost always contain UTF-8 text.

For those, we should use utf-8, not prefer-utf-8.

> I believe if there's a NUL byte in such a files but it otherwise doesn't
> contain any invalid UTF-8 byte sequence, it will result in better
> behavior if we treat it as UFT-8 than as binary.

We treat null bytes as the _single_ telltale sign of a binary file.
If we disable that in coding-systems that are supposed to _detect_
encoding, we will never be able to detect binary files.