Re: Bidirectional text and URLs

[Ted Zlatanov]

On Fri, 28 Nov 2014 14:58:27 +0100 Lars Magne Ingebrigtsen <address@hidden> 
wrote: 

LMI> Ted Zlatanov <address@hidden> writes:
>> My uni-confusables package in the GNU ELPA would help detect things like
>> б (CYRILLIC SMALL LETTER BE) confused with the number 6.  The relevant
>> line from confusables.txt is:
>> 
>> 0431 ;       0036 ;  SL      # ( б → 6 ) CYRILLIC SMALL LETTER BE → DIGIT 
>> SIX        #
>> 
>> which maps to (1073 "6") in `uni-confusables-char-table-single'. EWW and
>> SHR could opportunistically use that table to highlight such characters.

LMI> Yes, and perhaps use that to do a "are you sure?" if a user tries to
LMI> visit https://𝐩𝐚𝐲𝐩𝐚𝐥.com or https://ｐａｙｐａｌ．com.  

Right.  At least in the SHR/EWW context we can control that experience,
and also perhaps in places like `browse-url' or `ffap-url-at-point'.

LMI> But then uni-confusables should perhaps be moved from ELPA to Emacs so
LMI> that we can use it generally?

It would probably improve the use experience, yes.  Stefan, WDYT?

On Fri, 28 Nov 2014 09:24:21 -0500 Stefan Monnier <address@hidden> wrote: 

>> which maps to (1073 "6") in `uni-confusables-char-table-single'.  EWW and
>> SHR could opportunistically use that table to highlight such characters.

SM> I don't think SHR/EWW can really do that for the buffer's main text,
SM> since AFAIK it doesn't know whether what it displays is supposed to be
SM> a URL or just plain human text (or rather, to do it well it would have
SM> to somehow detect a particular mix of characters).

For Gnus users, for instance, the buffer would be using SHR so there's
some control over the experience and metadata about the content.  You're
right that in general this is not clear, which is why interactive
functions like `browse-url' and others may need to be advised.

SM> OTOH it can&should indeed do something (including a bigfat warning for
SM> bidi-ordering codes) when displaying something it knows to be a URL.

I'm not sure about the bidi markers, Eli can discuss that side.  I'll
try to get the confusables in there and maybe write general code that
bidi markers and others can hook into.

On Fri, 28 Nov 2014 16:57:27 +0200 Eli Zaretskii <address@hidden> wrote: 

>> I could also add RTL markers and other useful things to uni-confusables
>> if you think it's the right place

EZ> I don't think it's TRT to highlight these controls regardless of what
EZ> characters they affect.  See my other message for why.

OK.

>> I'm not sure of the right thing because this feels a little bit like
>> core functionality.

EZ> What is "core functionality" here?

Things that work in Emacs without customization.

EZ> For that matter, what functionality are we talking about?

The uni-confusables package from the GNU ELPA and glue code to let SHR
and EWW know that a URL includes such characters.

EZ> We should first decide what we want to do with these cases, and only
EZ> then discuss whether that functionality belongs to the core.

I think Lars' suggestion is decent, see above.

Ted