Re: Using utf-8-auto as a process coding system

[Bozhidar Batsov]

On July 25, 2014 at 4:03:58 PM, Eli Zaretskii (address@hidden) wrote:

> Date: Tue, 22 Jul 2014 08:55:53 +0300 
> From: Bozhidar Batsov <address@hidden> 
> Cc: address@hidden, address@hidden 
> 
> Ops, I actually used an incorrect link. That’s the proper one - https://github.com/clojure-emacs/cider/issues/532 

OK, but that one is very short ;-) 

> The gist of my problem is that Windows users have encoding related problems running cider (discussion here https://github.com/clojure-emacs/cider/issues/474). 

There's a lot of confusion in that discussion. 

> I guess the problem stems from this bit of code:  
> 
> (set-process-coding-system process 'utf-8-unix 'utf-8-unix) 

'-auto' is not about end-of-line (EOL) format, it is about the Byte 
Order Mark BOM (http://en.wikipedia.org/wiki/Byte_order_mark). 

Does Cider on Windows indeed output UTF-8 encoded text preceded by a 
BOM? I'd be surprised, as the BOM is normally not needed with UTF-8. 
In which case all this -auto thing just comes from another confused 
user. 

The problem is Java, not cider. cider is an Emacs Lisp client for a Clojure REPL server (nREPL), which runs on top of the JVM. I seem to recall that Java doesn’t use UTF-8 at all, think it was using the older UTF-16.

I guess it’s compatible with UTF-8 to some extent.

If there's no BOM, the first thing I'd try on Windows is this: 

(set-process-coding-system process 'utf-8-dos 'utf-8-unix) 

This is because Windows programs will normally accept Unix EOL format 
on input, but will usually output Windows CR-LF EOL, which Emacs needs 
to decode into a single newline character. 

A more elegant solution, which should be platform-independent, is to 
use something like below (untested) 

(set-process-coding-system process 
(cons (coding-system-change-text-conversion 
(car default-process-coding-system) 
'utf-8) 
(coding-system-change-text-conversion 
(cdr default-process-coding-system) 
'utf-8))) 

This has the advantage that it uses the wisdom already invested in 
setting the defaults for each platform. 

Thanks for the suggestions. I’ll ask some of the Windows users to try them both and see what works and what doesn’t.