PSPP-BUG: [bug #35688] encoding-guesser could detect bad fallback encoding more generally

[Ben Pfaff]

URL:
  <http://savannah.gnu.org/bugs/?35688>

                 Summary: encoding-guesser could detect bad fallback encoding
more generally
                 Project: PSPP
            Submitted by: blp
            Submitted on: Thu Mar  1 21:28:58 2012
                Category: Internationalization/Localization
                Severity: 5 - Average
                  Status: None
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
                 Release: None
                  Effort: 0.00

    _______________________________________________________

Details:

See http://lists.gnu.org/archive/html/pspp-dev/2012-03/msg00008.html for full
context.

John Darrington <address@hidden> writes:

> Would it be possible to generalize it?  That is to say, could we make
> sure, that when the fallback encoding is X, where X is multi-byte encoding,
> but we know that the input is not X, that it also falls back to
windows-1252?
>
> On Wed, Feb 29, 2012 at 10:44:30PM -0800, Ben Pfaff wrote:
>      Until now the encoding-guesser code has used UTF-8 as a fallback in
>      situations where we can tell that the file is not valid UTF-8.  In
>      this kind of situation having a single-byte character set as a
>      fallback makes more sense.  This commit hard-codes windows-1252 as
>      that fallback, since it is a widely encountered encoding (and
>      compatible with ISO-8859-1 as well).





    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?35688>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/