Re: UTF8 has stopped working in vile after OS upgrade

[Thomas Dickey]

On Tue, Aug 02, 2022 at 03:23:26PM +1000, Brendan O'Dea wrote:
> OK, so this appears to have been caused by the change:
> 
>   "modify lins_chars() to handle a case where a script inserts a UTF-8
> character (report by Thomas Dupond)."
> 
> lins_chars is being invoked with wide=0, and is working for codepoints >=
> 256 (which is why ę works, and £ does not).

yes...  the change can probably be made to work as intended by accounting
for whether lins_chars is called from a script or interactively by adding
the clexec variable to the check, e.g.

    if ((!clexec || !global_is_utfXX()) && (c > 127) && b_is_utfXX(curbp)) {

though the code has changed, and that's untested :-)

I'll investigate that over the next day or so.
 
> I've pushed a rollback of that change to Debian as 0d1f060a.  Relevant
> patch attached.
> 
> --bod
> 
> On Sat, 30 Jul 2022 at 00:00, Chris Green <cl@isbd.net> wrote:
> 
> > On Fri, Jul 29, 2022 at 04:27:05AM -0400, Thomas Dickey wrote:
> > > On Fri, Jul 29, 2022 at 03:40:12PM +1000, Brendan O'Dea wrote:
> > > > Testing the vile 9.8v-2 package on my Debian machine I get some
> > decidedly
> > > > odd behaviour:
> > > >
> > > > The following sequence:
> > > >
> > > >   <Compose>l- <Compose>e' <Compose>c,
> > > >
> > > > is rendered in vile as:
> > > >
> > > >   \?A3\?E9\?E7
> > > >
> > > > outside of vile, in the same terminal that appears fine:
> > > >
> > > >   Łéç
> > > >
> > > > which would suggest that vile doesn't handle UTF-8 at all, although as
> > > > Chris noted, entering those three characters using C-v x <hex> works
> > fine,
> > > > and renders in vile correctly.
> > >
> > > hmm - with some time, I can bisect to pinpoint the problem.
> > >
> > > (pasting outside of 0..255 seems ok)
> > >
> > Weirdly I find that using the vile defined 'compose' key *does* work
> > for accented characters, but using the default 'compose' key (as
> > defined in .Xmodmap) it doesn't work.
> >
> > I.e. I have "source digraphs.rc" in my .vilerc file as follows:-
> >
> >     ;
> >     ;
> >     ; Set up ^K as 'compose' key for accented characters
> >     ;
> >     source digraphs.rc
> >
> > ... and, rather to my surprise, when I entered ^ke' I got an e with an
> > accute accent.  I rarely (if ever) use this in vile, I'd really
> > forgotten it was there.  It doesn't work for a pound sign though (^kl=).
> >
> > --
> > Chris Green
> >
> >

> diff --git a/insert.c b/insert.c
> index 645c05d..158ad0e 100644
> --- a/insert.c
> +++ b/insert.c
> @@ -446,7 +446,7 @@ replacechar(int f, int n)
>               if (isbackspace(c)) {   /* vi beeps here */
>                   s = TRUE;   /* replaced with nothing */
>               } else {
> -                 t = s = lins_chars(n, c, FALSE);
> +                 t = s = lins_chars(n, c);
>               }
>           }
>       }
> @@ -981,11 +981,11 @@ inschar(int c, int *backsp_limit_p)
>               rc = inspound();
>           } else {
>               autoindented = -1;
> -             rc = lins_chars(1, c, FALSE);
> +             rc = lins_chars(1, c);
>           }
>       } else {
>           autoindented = -1;
> -         rc = lins_chars(1, c, FALSE);
> +         rc = lins_chars(1, c);
>       }
>      }
>      return rc;
> @@ -1515,7 +1515,7 @@ quote_next(int f, int n)
>               s = lnewline();
>           } while ((s == TRUE) && (--n != 0));
>       } else {
> -         s = lins_chars(n, c, TRUE);
> +         s = lins_chars(n, c);
>       }
>      }
>      return s;
> diff --git a/line.c b/line.c
> index d56a6f6..bd5b8a5 100644
> --- a/line.c
> +++ b/line.c
> @@ -534,7 +534,7 @@ lins_bytes(int n, int c)
>   * or in insert-mode.
>   */
>  int
> -lins_chars(int n, int c, int wide)
> +lins_chars(int n, int c)
>  {
>      int rc = FALSE;
>      UCHAR target[10];
> @@ -542,7 +542,7 @@ lins_chars(int n, int c, int wide)
>      int nn;
>      int mapped;
>  
> -    if (wide && (c > 127) && b_is_utfXX(curbp)) {
> +    if ((c > 127) && b_is_utfXX(curbp)) {
>       nbytes = vl_conv_to_utf8(target, (UINT) c, sizeof(target));
>      } else if (okCTYPE2(vl_wide_enc) && !vl_mb_is_8bit(c)) {
>       nbytes = 1;
> diff --git a/proto.h b/proto.h
> index 4c8198f..77ab7e0 100644
> --- a/proto.h
> +++ b/proto.h
> @@ -873,10 +873,10 @@ extern int lrepl_regex (REGEXVAL *expr, const char 
> *iline, int ilen);
>  
>  #if OPT_MULTIBYTE
>  extern int ldel_chars (B_COUNT n, int kflag);
> -extern int lins_chars (int n, int c, int wide);
> +extern int lins_chars (int n, int c);
>  #else
>  #define ldel_chars(n, kflag) ldel_bytes(n, kflag)
> -#define lins_chars(n, c, wide)  lins_bytes(n, c)
> +#define lins_chars(n, c)  lins_bytes(n, c)
>  #endif
>  
>  #if OPT_REGS_CMPL


-- 
Thomas E. Dickey <dickey@invisible-island.net>
https://invisible-island.net
ftp://ftp.invisible-island.net

signature.asc
Description: PGP signature