Re: [help-octave] Re: [help-octave] Re: utf8 does not appear to work for function documentation strings generated with texinfo

[Alan W. Irwin]

On 2014-03-27 09:25-0400 Mike Miller wrote:

> I actually just realized an even better solution - use fprintf instead
> of fwrite when dealing with strings.

Yes, that should be more robust since it is free of assumptions about
the exact char type being used to internally represent Octave strings.

Also, it works for me both for my simple test and also in a much
broader context (utf-8 help strings for the Octave functions
automatically created by the swig-generated Octave binding of Plplot).

Encouraged by that success I also tried the following test using the
utf-8 strings for a wide variety of human languages (adapted from
http://en.wikipedia.org/wiki/Xetex). (WARNING, this example will only
be legible if you have the appropriate fonts installed on your
system).

## -*- texinfo -*-
## @deftypefn  {Function File} address@hidden =} fn (@var{x}, …)
##English
## 
##All human beings are born free and equal in dignity and rights.
## 
##Íslenska
## 
##Hver maður er borinn frjáls og jafn öðrum að virðingu og réttindum.
## 
##Русский 
## 
##Все люди рождаются свободными и равными в своем достоинстве и 
##правах.
## 
##Tiếng Việt
## 
##Tất cả mọi người sinh ra đều được tự do và bình đẳng về nhân phẩm và 
##quyền lợi.
## 
##Ελληνικά 
## 
##Ὅλοι οἱ ἄνθρωποι γεννιοῦνται ἐλεύθεροι καὶ ἴσοι στὴν ἀξιοπρέπεια 
##καὶ τὰ δικαιώματα.
## 
##Legacy syntax
## 
##When he goes---``Hello World!''\\
##She replies—“Hello dear!”
## 
##Ligatures
## 
##Questo è strano assai!
## @end deftypefn

function test_utf8_languages
printf("The unicode character, ≥, is output\n")
endfunction

The result was

warning: function ./__makeinfo__.m shadows a core library function
octave:1> help test_utf8_languages
`test_utf8_languages' is a function from the file 
/home/irwin/test_octave/test_utf8_languages.m

 -- Function File: A = fn (X, …)
     English

     All human beings are born free and equal in dignity and rights.

     Íslenska

     Hver maður er borinn frjáls og jafn öðrum að virðingu og réttindum.

     Русский

     Все люди рождаются свободными и равными в своем достоинстве и
     правах.

     Tiếng Việt

     Tất cả mọi người sinh ra đều được tự do và bình đẳng về nhân phẩm
     và quyền lợi.

     Ελληνικά

     Ὅλοι οἱ ἄνθρωποι γεννιοῦνται ἐλεύθεροι καὶ ἴσοι στὴν ἀξιοπρέπεια
     καὶ τὰ δικαιώματα.

     Legacy syntax

     When he goes--"Hello World!"\\ She replies—“Hello dear!”

     Ligatures

     Questo è strano assai!

Additional help for built-in functions and operators is
[...]

So it appears to me your simple change to __makeinfo__.m is ready for
your next release with some nice implications.  For example, your
change makes it possible to include any of a vast array of UTF-8
mathematical glyphs into Octave function help strings. Furthermore,
because UTF-8 covers all glyphs occurring in the scripts of human
languages (and also Klingon, :-)), it should make it convenient to
translate Octave function help strings into all languages that have a
simple left-to-right text layout, e.g., the above languages and many
more, but excluding complex text layout languages (e.g.,
Arabic, Devanagari, or Thai according to
<http://en.wikipedia.org/wiki/Complex_text_layout>).

Alan
__________________________
Alan W. Irwin

Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).

Programming affiliations with the FreeEOS equation-of-state
implementation for stellar interiors (freeeos.sf.net); the Time
Ephemerides project (timeephem.sf.net); PLplot scientific plotting
software package (plplot.sf.net); the libLASi project
(unifont.org/lasi); the Loads of Linux Links project (loll.sf.net);
and the Linux Brochure Project (lbproject.sf.net).
__________________________

Linux-powered Science
__________________________