[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: CSV parsing and other issues (Re: LC_NUMERIC)
From: |
Maxim Nikulin |
Subject: |
Re: CSV parsing and other issues (Re: LC_NUMERIC) |
Date: |
Tue, 8 Jun 2021 23:35:51 +0700 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.1 |
On 08/06/2021 09:35, Eli Zaretskii wrote:
> From: Boruch Baum
>> No? If an Emacs user has two buffers in two separate languages, the
>> buffer-local settings aren't / won't be respected?
>
> First, language is different from locale. And second, we don't even
> have a buffer-local notion of language yet.
Certainly locale is more precise than just language since it includes
region and other variants, moreover it can be granularly tuned (date,
numbers, sorting can be adjusted independently), but I still think that
all these properties can be sometimes broadly referred to as language.
Does not we discuss a feature request? Low level functions can accept
explicit locale. Higher level API can obtain it implicitly from
buffer-local variables and global locale. For example the LOCALE
argument of `string-collate-lessp' is optional one. I can even
anticipate that locale may be stored in text properties some times. A
random message from recent "About multilingual documents" thread at
emacs-orgmode mail list:
https://lists.gnu.org/archive/html/emacs-orgmode/2021-05/msg00252.html
At first basic functionality may be implemented. The problem is to
choose extensible API.
On 07/06/2021 06:36, Boruch Baum wrote:
> I get 'nil' when I check for any related
> environment variable using function `getenv'
Do not confuse setlocale and setenv. setenv affects
later calls to setlocale (with NULL as locale argument)
and child processes. setlocale deals with current
processes it can take into account or override values
of environment variables. setlocale is not exposed to elisp.
The single quote format (for the thousands separator) can be expected
to produce a result always for all conditions of locale, while I
expect most locale cases won't produce any special output for the
upper-case I format option.
I still think that "'" and "I" formats are tightly bound. Grouping style
is locale-dependent. So representation of digits is just another
property of locale.
LC_NUMERIC=C.UTF-8 /usr/bin/printf "%'d\n" 1234567890
1234567890
LC_NUMERIC=en_US.UTF-8 /usr/bin/printf "%'d\n" 1234567890
1,234,567,890
LC_NUMERIC=es_ES.UTF-8 /usr/bin/printf "%'d\n" 1234567890
1.234.567.890
LC_NUMERIC=ru_RU.UTF-8 /usr/bin/printf "%'d\n" 1234567890
1 234 567 890
Even group size is not always 3
: LC_NUMERIC=en_IN.UTF-8 /usr/bin/printf "%'d\n" 1234567890
: 1,23,45,67,890
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat
"India uses thousands/lakh/crore separators"
I just have realized that nl_langinfo(3) (and nl_langinfo_l(3) as well)
from libc accepts RADIXCHAR (decimal dot) and THOUSEP (group separator)
arguments. They are good candidates for `locale-info' extension.
On 05/06/2021 02:17, Eli Zaretskii wrote:
>> From: Maxim Nikulin Date: Fri, 4 Jun 2021 23:31:13 +0700
>> On linux I see that Emacs is linked with ICU
>
> It isn't. It's either HarfBuzz or maybe libc that pulls in the ICU
> library. Emacs doesn't use it directly.
Actually Qt links my example with other libraries from ICU. My point was
that since Emacs anyway (indirectly) links with this library, the
dependency may be not so heavy. My personal requirements for number
formatting were quite modest so far, I expect that other languages (CJK,
right-to-left scripts, etc.) may require quite special treatment, so
implementation in Emacs (and further maintenance) may require a lot of
work. At least API of ICU should be studied to get some inspiration what
features will be necessary for users from other regions.
E.g. I was completely unaware that negative sign may be represented by
parenthesis (JavaScript, may be executed in browser developer tools)
new Intl.NumberFormat('en-GB', {
style: 'currency',
currency: 'USD',
currencySign: 'accounting',
signDisplay: 'always'
}).format(-3500);
"(US$3,500.00)"
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/NumberFormat/NumberFormat
I do not know if Intl API is really convenient. I see that there is no
direct way to get decimal separator. However it can serve as
another source for inspiration.
I expect enough surprises and unexpected "discoveries" during
implementation of better locale support. That is why I would consider
adapting some more or less established API for this purpose.
P.S.
On 07/06/2021 06:36, Boruch Baum wrote:
and in practice I need to temporarily manually use function setenv to
set LC_COLLATE=C in order to offer several sorting options in package
diredc.
Ideally you should avoid this and use envp argument of execve(2) system
call. Otherwise it could interfere with other packages, especially if
threads are involved. Unsure that Emacs currently provides such API option.
it's temporarily setting a shell environment and having the external
ls program perform the sort for emacs-core dired.
I am unsure if "ls" may be reliably used at all. File names may have
e.g. newlines, various control characters, part that looks rather
similar to ls output. I am not familiar with dired internals. At first
by intention was to create an issue for diredc but skimming though its
code I did not found direct "ls" invocation. Some problems with ls:
https://mywiki.wooledge.org/BashPitfalls?highlight=%28%5CbCategoryShell%5Cb%29#for_f_in_.24.28ls_.2A.mp3.29
Bash Pitfalls: item #1
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Boruch Baum, 2021/06/06
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/07
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Boruch Baum, 2021/06/07
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/07
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Stefan Monnier, 2021/06/08
- Re: CSV parsing and other issues (Re: LC_NUMERIC),
Maxim Nikulin <=
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/08
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Maxim Nikulin, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Boruch Baum, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Boruch Baum, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Boruch Baum, 2021/06/10
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Eli Zaretskii, 2021/06/11
- Re: CSV parsing and other issues (Re: LC_NUMERIC), Boruch Baum, 2021/06/11