[Octave-bug-tracker] [bug #65963] pkg install "fopen: encoding must be 'UTF-8'' warnings on macOS 14 AS

[Andrew Janke]

URL:
  <https://savannah.gnu.org/bugs/?65963>

                 Summary: pkg install "fopen: encoding must be 'UTF-8''
warnings on macOS 14 AS
                   Group: GNU Octave
               Submitter: apjanke
               Submitted: Sun 07 Jul 2024 10:52:27 AM EDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Unexpected Error or Warning
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: 9.1.0
         Discussion Lock: Any
        Operating System: Mac OS
           Fixed Release: None
         Planned Release: None


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Sun 07 Jul 2024 10:52:27 AM EDT By: Andrew Janke <apjanke>
Hi, folks.

Starting with Octave 9.1 and affecting 9.2, when I do any `pkg install -forge`
installation, I'm getting a bunch of "warning: fopen: encoding must be 'UTF-8'
for this version" warnings.

Only on macOS 14 on Apple Silicon; macOS 14 on Intel, and macOS 12 on either
architecture, do not produce this warning. Don't know if that's a difference
in the OS & arch, or just a difference in how these particular machines of
mine are configured.

The warning looks like:


warning: fopen: encoding must be 'UTF-8' for this version
warning: called from
    install>extract_pkg at line 332 column 7
    install>create_pkgadddel at line 761 column 7
    install at line 239 column 7
    pkg at line 619 column 9


I get dozens or hundreds of them on each pkg install operation.

== Diagnosis and guesses ==

The code around the fopen() at the bottom of that stack trace is, in 9.2.0:


function pkg = extract_pkg (nm, pat)

  fid = fopen (nm, "rt", "n", mfile_encoding ());
  pkg = "";
  if (fid >= 0)


On my macOS 14 AS machine (eilonwy, bare metal), `mfile_encoding()` returns
'system'. On my other machines (macOS 14 Intel, and macOS 12 either arch),
mfile_encoding() returns 'utf-8'.

I'm guessing that fopen() doesn't like 'system' for its encoding argument? The
helptext for fopen() says it supports 'native' as an argument there, but not
'system'. Looks like it has about the same meaning as mfile_encoding's
'system' value.

== Discussion ==

Is 'system' supposed to be supported here? Should pkg's install() be doing
some translation?

What's the default for mfile_encoding supposed to be (on macOS)? Is that set
in a user configuration somewhere that persists between Octave sessions?
Wondering if I have some setting on that one machine that's messing things up,
either because I set it at some point, or it's left over from running an older
version of Octave that set it somewhere persistent, or something like that.

I called `mfile_encoding('utf-8')` on eilonwy, the machine where it was
returning 'system', and restarted Octave, and now it's returning 'utf-8'. And
I see "default_encoding=UTF-8" in my ~/.config/octave/octave-gui.ini file. It
switches to "default_encoding=SYSTEM (UTF-8)" if I run `mfile_encoding
('system')". After doing that, in CLI octave, mfile_encoding() returns
'utf-8', but in `octave --gui`, mfile_encoding() returns 'system'. That
difference in behavior is a little surprising.

The warning says the encoding "must" be UTF-8. Is that true? Does fopen not
support other encodings in this platform/situation?

I see this in libinterp/corefcn/file-io.cc in the Octave source. I don't know
what this strict encoding facet stuff is. But it seems odd to limit it to
UTF-8 in a general IO operation like fopen. Modern systems can still handle
other encodings I think, and it's not uncommon to have or receive files that
are in other encodings, like UTF-16 or non-Unicode encodings, even today in
2024.


#if defined (OCTAVE_HAVE_STRICT_ENCODING_FACET)
  if (encoding != "utf-8")
    {
      warning_with_id ("Octave:fopen:encoding-unsupported",
                       "fopen: encoding must be 'UTF-8' for this version");
      encoding = "utf-8";
    }
#endif


Also, it looks like that's a case-sensitive comparison, and the value must be
lower-case "utf-8", but the error message is talking about upper case
"UTF-8".

== Test results ==

I tried this on various OS, arch, and Octave combinations over this weekend.
Here's what I observed.

On AS (eilonwy):
* In 9.2.0 brewed on macOS 14 AS: warnings
* In Octave.app 9.2.0 beta1 on macOS 14 AS: same warnings
* In Octave.app 9.1.0 beta4 on macOS 14 AS: same warnings
* In Octave.app 8.4.0 on macOS 14 AS: no warning
* In Octave.app 6.2.0 on macOS 14 AS: no warning
* In Octave.app 4.4.1 on macOS 14 AS: `pkg install -forge control` fails at
linker step; no warning up until then. 

On AS (oabuild-as (macOS 12) VM on eilonwy):
* In Octave.app 9.2.0 beta1 on macOS 12 AS: no warnings! huh.
* In Octave.app 9.1.0 beta4 on macOS 12 AS: no warnings! huh.
* In Octave.app 8.4.0 on macOS 12 AS: no warnings

On Intel (angharad):
* Brewed octave 9.2.0 on macOS 14 Intel: no warnings
* Octave.app 9.2.0 beta1 on macOS 14 Intel: no warnings
* Octave.app 9.1.0 beta4 on macOS 14 Intel: no warnings
* Octave.app 8.4.0 on macOS 14 Intel: no warnings
* Octave.app 6.2.0 on macOS 14 Intel: no warnings


mfile_encoding() returns:

* brewed Octave 9.2.0 on macOS 14 AS: 'system'
* brewed Octave 9.2.0 on macOS 14 Intel: 'utf-8'
* Octave.app 9.1.0 beta4 on macOS 12 AS (VM): 'utf-8'
* Octave.app 9.1.0 beta4 on macOS 12 Intel (VM): 'utf-8'








    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?65963>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/

signature.asc
Description: PGP signature