branch master updated: Consider control characters to have a 0 width

texinfo-commits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
branch master updated: Consider control characters to have a 0 width

From:	Patrice Dumas
Subject:	branch master updated: Consider control characters to have a 0 width
Date:	Wed, 06 Sep 2023 02:47:48 -0400
This is an automated email from the git hooks/post-receive script.

pertusus pushed a commit to branch master
in repository texinfo.

The following commit(s) were added to refs/heads/master by this push:
     new a2a3ded7f3 Consider control characters to have a 0 width
a2a3ded7f3 is described below

commit a2a3ded7f3b33939815e6f1182d74d6167033d54
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Wed Sep 6 08:47:11 2023 +0200

    Consider control characters to have a 0 width
    
    * tp/Texinfo/Convert/Unicode.pm (string_width): do not consider space
    control characters especially.  This is better for line feed, carriage
    return and form feed, and vertical or horizontal tabs should not
    happen in situations where we want to know the width of strings.
---
 ChangeLog                                          |   9 ++++
 tp/Texinfo/Convert/Unicode.pm                      |   7 +--
 .../formats_encodings/at_commands_in_refs.pl       |  56 ++++++++++-----------
 .../at_commands_in_refs_latin1.pl                  |   2 +-
 .../res_info/at_commands_in_refs_latin1.info       | Bin 8005 -> 8004 bytes
 .../formats_encodings/at_commands_in_refs_utf8.pl  |   2 +-
 .../res_info/at_commands_in_refs_utf8.info         | Bin 8402 -> 8401 bytes
 .../unclosed_verb_on_section_line.pl               |   2 +-
 .../moresectioning/only_special_spaces_node.pl     |  22 ++++----
 .../in_menu_only_special_ascii_spaces_node.pl      |   8 +--
 .../in_menu_only_special_ascii_spaces_node_menu.pl |   8 +--
 .../sectioning/in_menu_only_special_spaces_node.pl |  22 ++++----
 .../in_menu_only_special_spaces_node_menu.pl       |  22 ++++----
 13 files changed, 83 insertions(+), 77 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index f0ce940546..91623d93ed 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,12 @@
+2023-09-06  Patrice Dumas <pertusus@free.fr>
+
+       Consider control characters to have a 0 width
+
+       * tp/Texinfo/Convert/Unicode.pm (string_width): do not consider space
+       control characters especially.  This is better for line feed, carriage
+       return and form feed, and vertical or horizontal tabs should not
+       happen in situations where we want to know the width of strings.
+
 2023-09-05  Patrice Dumas <pertusus@free.fr>
 
        Add customization of external node reference names
diff --git a/tp/Texinfo/Convert/Unicode.pm b/tp/Texinfo/Convert/Unicode.pm
index 43145e1020..a5a308d6a5 100644
--- a/tp/Texinfo/Convert/Unicode.pm
+++ b/tp/Texinfo/Convert/Unicode.pm
@@ -1666,7 +1666,7 @@ sub string_width($)
   # Optimise for the common case where we can just return the length
   # of the string.  These regexes are faster than making the substitutions
   # below.
-  if ($string =~ /^[\p{IsPrint}\p{IsSpace}]*$/
+  if ($string =~ /^[\p{IsPrint}]*$/
       and $string !~ /[\p{InFullwidth}\pM]/) {
     return length($string);
   }
@@ -1674,7 +1674,6 @@ sub string_width($)
   $string =~ s/\p{InFullwidth}/\x{02}/g;
   $string =~ s/\pM/\x{00}/g;
   $string =~ s/\p{IsPrint}/\x{01}/g;
-  $string =~ s/\p{IsSpace}/\x{01}/g;
   $string =~ s/[^\x{01}\x{02}]/\x{00}/g;
 
   # This sums up the byte values of the bytes in $string, which now are
@@ -1691,12 +1690,10 @@ sub string_width($)
       $width += 2;
     } elsif ($character =~ /\pM/) {
       # a mark set at length 0
-    } elsif ($character =~ /\p{IsPrint}/ or $character =~ /\p{IsSpace}/) {
-      # newlines are not {IsPrint} in v5.14.2
+    } elsif ($character =~ /\p{IsPrint}/) {
       $width += 1;
     } elsif ($character =~ /\p{IsControl}/) {
       # Control chars may be added, for instance, as part of @image formatting
-    #} elsif ($character eq '') { # could that happen?
     } else {
       #print STDERR "unknown char`$character'\n";
     }
diff --git a/tp/t/results/formats_encodings/at_commands_in_refs.pl 
b/tp/t/results/formats_encodings/at_commands_in_refs.pl
index 763c2c8976..4e6932d721 100644
--- a/tp/t/results/formats_encodings/at_commands_in_refs.pl
+++ b/tp/t/results/formats_encodings/at_commands_in_refs.pl
@@ -14852,7 +14852,7 @@ $result_texts{'at_commands_in_refs'} = 'Top
 
 2     ! 
  .  . ? @
-******************
+*****************
 
 3 @ { } \\ #
 ***********
@@ -16648,7 +16648,7 @@ $result_converted{'plaintext'}->{'at_commands_in_refs'} 
= 'Top
 
 2     !
 . . ? @
-***************
+**************
 
 3 @ { } \\ #
 ***********
@@ -17827,7 +17827,7 @@ File: ,  Node:     ! . . ? @,  Next: @ { } \\ #,  Prev: 
{ },  Up: Top
 
 2     !
 . . ? @
-***************
+**************
 
 
 File: ,  Node: @ { } \\ #,  Next: LaTeX TeX • , © ... ...,  Prev:     ! . . 
? @,  Up: Top
@@ -18041,31 +18041,31 @@ Tag Table:
 Node: Top27
 Node: { }875
 Node:     ! . . ? @954
-Node: @ { } \\ #1059
-Node: LaTeX TeX • , © ... ...1181
-Node: ≡ error→ € ¡ ↦ −1348
-Node: ≥ ≤ →1511
-Node: ª º ⋆ £ ⊣ ¿ ®1641
-Node: ⇒ ° a b a sunny day å1788
-Node: Å æ œ Æ Œ ø Ø ß ł Ł Ð ð Þ þ1976
-Node: ä ẽ î â à é ç ē e̊ e̋ ę2199
-Node: ė ĕ e̲ ẹ ě ȷ e͡e2411
-Node: ı Ḕ Ḉ2574
-Node: “ ” ‘ ’ „ ‚2702
-Node: « » « » ‹ ›2844
-Node: `` \'\' --- -- ` \'2988
-Node: AAA (fff) AAA BBB3120
-Node: CCC (rrr) CCC DDD3260
-Node: the someone <someone@somewher> <no_explain@there>3433
-Node: [f--ile1] [image src="f--ile.png" alt="alt" text="Image 
description\\"\\"\\\\."]3733
-Node:  @ {} . 4083
-Node: cite asis in @w b in r SC *str* t VAR dfn i4286
-Node: env code option samp command file C-x <ESC>4538
-Node: 8.27in4828
-Node: sansserif slanted4962
-Node: indicateurl5086
-Node: _{g}H 3^{rd}5212
-Node: <http://somewhere_aaa> text (url) ls5351
+Node: @ { } \\ #1058
+Node: LaTeX TeX • , © ... ...1180
+Node: ≡ error→ € ¡ ↦ −1347
+Node: ≥ ≤ →1510
+Node: ª º ⋆ £ ⊣ ¿ ®1640
+Node: ⇒ ° a b a sunny day å1787
+Node: Å æ œ Æ Œ ø Ø ß ł Ł Ð ð Þ þ1975
+Node: ä ẽ î â à é ç ē e̊ e̋ ę2198
+Node: ė ĕ e̲ ẹ ě ȷ e͡e2410
+Node: ı Ḕ Ḉ2573
+Node: “ ” ‘ ’ „ ‚2701
+Node: « » « » ‹ ›2843
+Node: `` \'\' --- -- ` \'2987
+Node: AAA (fff) AAA BBB3119
+Node: CCC (rrr) CCC DDD3259
+Node: the someone <someone@somewher> <no_explain@there>3432
+Node: [f--ile1] [image src="f--ile.png" alt="alt" text="Image 
description\\"\\"\\\\."]3732
+Node:  @ {} . 4082
+Node: cite asis in @w b in r SC *str* t VAR dfn i4285
+Node: env code option samp command file C-x <ESC>4537
+Node: 8.27in4827
+Node: sansserif slanted4961
+Node: indicateurl5085
+Node: _{g}H 3^{rd}5211
+Node: <http://somewhere_aaa> text (url) ls5350
 
 End Tag Table
 
diff --git a/tp/t/results/formats_encodings/at_commands_in_refs_latin1.pl 
b/tp/t/results/formats_encodings/at_commands_in_refs_latin1.pl
index 67a2d205a4..61a6fcc2e8 100644
--- a/tp/t/results/formats_encodings/at_commands_in_refs_latin1.pl
+++ b/tp/t/results/formats_encodings/at_commands_in_refs_latin1.pl
@@ -14927,7 +14927,7 @@ Top
 
 2     ! 
  .  . ? @
-******************
+*****************
 
 3 @ { } \\ #
 ***********
diff --git 
a/tp/t/results/formats_encodings/at_commands_in_refs_latin1/res_info/at_commands_in_refs_latin1.info
 
b/tp/t/results/formats_encodings/at_commands_in_refs_latin1/res_info/at_commands_in_refs_latin1.info
index b5377f9e9f..9ca877b9ba 100644
Binary files 
a/tp/t/results/formats_encodings/at_commands_in_refs_latin1/res_info/at_commands_in_refs_latin1.info
 and 
b/tp/t/results/formats_encodings/at_commands_in_refs_latin1/res_info/at_commands_in_refs_latin1.info
 differ
diff --git a/tp/t/results/formats_encodings/at_commands_in_refs_utf8.pl 
b/tp/t/results/formats_encodings/at_commands_in_refs_utf8.pl
index fb71fdad60..a648fd04a1 100644
--- a/tp/t/results/formats_encodings/at_commands_in_refs_utf8.pl
+++ b/tp/t/results/formats_encodings/at_commands_in_refs_utf8.pl
@@ -14927,7 +14927,7 @@ Top
 
 2     ! 
  .  . ? @
-******************
+*****************
 
 3 @ { } \\ #
 ***********
diff --git 
a/tp/t/results/formats_encodings/at_commands_in_refs_utf8/res_info/at_commands_in_refs_utf8.info
 
b/tp/t/results/formats_encodings/at_commands_in_refs_utf8/res_info/at_commands_in_refs_utf8.info
index dc193a0ac7..91d9958a6e 100644
Binary files 
a/tp/t/results/formats_encodings/at_commands_in_refs_utf8/res_info/at_commands_in_refs_utf8.info
 and 
b/tp/t/results/formats_encodings/at_commands_in_refs_utf8/res_info/at_commands_in_refs_utf8.info
 differ
diff --git a/tp/t/results/invalid_nestings/unclosed_verb_on_section_line.pl 
b/tp/t/results/invalid_nestings/unclosed_verb_on_section_line.pl
index b89d3f3710..b8c62a0a2c 100644
--- a/tp/t/results/invalid_nestings/unclosed_verb_on_section_line.pl
+++ b/tp/t/results/invalid_nestings/unclosed_verb_on_section_line.pl
@@ -79,7 +79,7 @@ T}';
 $result_texts{'unclosed_verb_on_section_line'} = '1 in section ruc
 
 Now text.
-===========================
+=========================
 ';
 
 $result_sectioning{'unclosed_verb_on_section_line'} = {
diff --git a/tp/t/results/moresectioning/only_special_spaces_node.pl 
b/tp/t/results/moresectioning/only_special_spaces_node.pl
index b41d87c410..6da280d00b 100644
--- a/tp/t/results/moresectioning/only_special_spaces_node.pl
+++ b/tp/t/results/moresectioning/only_special_spaces_node.pl
@@ -516,20 +516,20 @@ $result_texts{'only_special_spaces_node'} = 'top
 ***********************************
 
 2 CHARACTER TABULATION|        | FORM FEED|| LINE TABULATION||
-*********************************************************
+******************************************************
 
 3 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 4 NEXT LINE (NEL)|| NO-BREAK SPACE| | OGHAM SPACE MARK| |
-**********************************************************
+*********************************************************
 
 5 MONGOLIAN VOWEL SEPARATOR|᠎| EM SPACE| |
 ******************************************
 
 
 6 THREE-PER-EM SPACE| | FOUR-PER-EM SPACE| | SIX-PER-EM SPACE| | FIGURE SPACE| 
| PUNCTUATION SPACE| | THIN SPACE| | HAIR SPACE| | LINE SEPARATOR| | PARAGRAPH 
SEPARATOR| | NARROW NO-BREAK SPACE| | MEDIUM MATHEMATICAL SPACE| | IDEOGRAPHIC 
SPACE|　|
-******************************************************************************************************************************************************************************************************************************************************
+****************************************************************************************************************************************************************************************************************************************************
 
 ';
 
@@ -797,16 +797,16 @@ File: ,  Node:    ,  Next: ,  Prev: Top,  Up: Top
 ***********************************
 
 2 CHARACTER TABULATION|        | FORM FEED|| LINE TABULATION||
-*********************************************************
+******************************************************
 
 3 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 
 File: ,  Node:   ,  Next: ᠎ ,  Prev: ,  Up: Top
 
 4 NEXT LINE (NEL)|| NO-BREAK SPACE| | OGHAM SPACE MARK| |
-**********************************************************
+*********************************************************
 
 
 File: ,  Node: ᠎ ,  Next:            　,  Prev:   ,  Up: Top
@@ -818,16 +818,16 @@ File: ,  Node: ᠎ ,  Next:            　,  Prev:  
 File: ,  Node:            　,  Prev: ᠎ ,  Up: Top
 
 6 THREE-PER-EM SPACE| | FOUR-PER-EM SPACE| | SIX-PER-EM SPACE| | FIGURE SPACE| 
| PUNCTUATION SPACE| | THIN SPACE| | HAIR SPACE| | LINE SEPARATOR| | PARAGRAPH 
SEPARATOR| | NARROW NO-BREAK SPACE| | MEDIUM MATHEMATICAL SPACE| | IDEOGRAPHIC 
SPACE|　|
-******************************************************************************************************************************************************************************************************************************************************
+****************************************************************************************************************************************************************************************************************************************************
 
 
 
 Tag Table:
 Node: Top56
 Node:    205
-Node:   503
-Node: ᠎ 686
-Node:            　873
+Node:   499
+Node: ᠎ 681
+Node:            　868
 
 End Tag Table
 
diff --git a/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node.pl 
b/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node.pl
index 0394284027..0a8972d61f 100644
--- a/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node.pl
+++ b/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node.pl
@@ -341,10 +341,10 @@ $result_texts{'in_menu_only_special_ascii_spaces_node'} = 
'top
 *      ::
 
 1 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 2 CHARACTER TABULATION|        | FORM FEED||
-**************************************
+************************************
 
 ';
 
@@ -517,10 +517,10 @@ top
 *      ::
 
 1 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 2 CHARACTER TABULATION|        | FORM FEED||
-**************************************
+************************************
 
 
 
diff --git 
a/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node_menu.pl 
b/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node_menu.pl
index 358800499b..bd50eae9da 100644
--- a/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node_menu.pl
+++ b/tp/t/results/sectioning/in_menu_only_special_ascii_spaces_node_menu.pl
@@ -341,10 +341,10 @@ 
$result_texts{'in_menu_only_special_ascii_spaces_node_menu'} = 'top
 *      ::
 
 1 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 2 CHARACTER TABULATION|        | FORM FEED||
-**************************************
+************************************
 
 ';
 
@@ -517,10 +517,10 @@ top
 *      ::
 
 1 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 2 CHARACTER TABULATION|        | FORM FEED||
-**************************************
+************************************
 
 
 
diff --git a/tp/t/results/sectioning/in_menu_only_special_spaces_node.pl 
b/tp/t/results/sectioning/in_menu_only_special_spaces_node.pl
index 663ad9109e..0aaab6040b 100644
--- a/tp/t/results/sectioning/in_menu_only_special_spaces_node.pl
+++ b/tp/t/results/sectioning/in_menu_only_special_spaces_node.pl
@@ -845,20 +845,20 @@ $result_texts{'in_menu_only_special_spaces_node'} = 'top
 ***********************************
 
 2 CHARACTER TABULATION|        | FORM FEED|| LINE TABULATION||
-*********************************************************
+******************************************************
 
 3 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 4 NEXT LINE (NEL)|| NO-BREAK SPACE| | OGHAM SPACE MARK| |
-**********************************************************
+*********************************************************
 
 5 MONGOLIAN VOWEL SEPARATOR|᠎| EM SPACE| |
 ******************************************
 
 
 6 THREE-PER-EM SPACE| | FOUR-PER-EM SPACE| | SIX-PER-EM SPACE| | FIGURE SPACE| 
| PUNCTUATION SPACE| | THIN SPACE| | HAIR SPACE| | LINE SEPARATOR| | PARAGRAPH 
SEPARATOR| | NARROW NO-BREAK SPACE| | MEDIUM MATHEMATICAL SPACE| | IDEOGRAPHIC 
SPACE|　|
-******************************************************************************************************************************************************************************************************************************************************
+****************************************************************************************************************************************************************************************************************************************************
 
 ';
 
@@ -1212,16 +1212,16 @@ File: ,  Node:    ,  Next: ,  Prev: Top,  Up: Top
 ***********************************
 
 2 CHARACTER TABULATION|        | FORM FEED|| LINE TABULATION||
-*********************************************************
+******************************************************
 
 3 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 
 File: ,  Node:   ,  Next: ᠎ ,  Prev: ,  Up: Top
 
 4 NEXT LINE (NEL)|| NO-BREAK SPACE| | OGHAM SPACE MARK| |
-**********************************************************
+*********************************************************
 
 
 File: ,  Node: ᠎ ,  Next:            　,  Prev:   ,  Up: Top
@@ -1233,16 +1233,16 @@ File: ,  Node: ᠎ ,  Next:            　,  Prev:  
 File: ,  Node:            　,  Prev: ᠎ ,  Up: Top
 
 6 THREE-PER-EM SPACE| | FOUR-PER-EM SPACE| | SIX-PER-EM SPACE| | FIGURE SPACE| 
| PUNCTUATION SPACE| | THIN SPACE| | HAIR SPACE| | LINE SEPARATOR| | PARAGRAPH 
SEPARATOR| | NARROW NO-BREAK SPACE| | MEDIUM MATHEMATICAL SPACE| | IDEOGRAPHIC 
SPACE|　|
-******************************************************************************************************************************************************************************************************************************************************
+****************************************************************************************************************************************************************************************************************************************************
 
 
 
 Tag Table:
 Node: Top64
 Node:    227
-Node:   525
-Node: ᠎ 708
-Node:            　895
+Node:   521
+Node: ᠎ 703
+Node:            　890
 
 End Tag Table
 
diff --git a/tp/t/results/sectioning/in_menu_only_special_spaces_node_menu.pl 
b/tp/t/results/sectioning/in_menu_only_special_spaces_node_menu.pl
index a8815854fd..fe32b3ef31 100644
--- a/tp/t/results/sectioning/in_menu_only_special_spaces_node_menu.pl
+++ b/tp/t/results/sectioning/in_menu_only_special_spaces_node_menu.pl
@@ -845,20 +845,20 @@ $result_texts{'in_menu_only_special_spaces_node_menu'} = 
'top
 ***********************************
 
 2 CHARACTER TABULATION|        | FORM FEED|| LINE TABULATION||
-*********************************************************
+******************************************************
 
 3 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 4 NEXT LINE (NEL)|| NO-BREAK SPACE| | OGHAM SPACE MARK| |
-**********************************************************
+*********************************************************
 
 5 MONGOLIAN VOWEL SEPARATOR|᠎| EM SPACE| |
 ******************************************
 
 
 6 THREE-PER-EM SPACE| | FOUR-PER-EM SPACE| | SIX-PER-EM SPACE| | FIGURE SPACE| 
| PUNCTUATION SPACE| | THIN SPACE| | HAIR SPACE| | LINE SEPARATOR| | PARAGRAPH 
SEPARATOR| | NARROW NO-BREAK SPACE| | MEDIUM MATHEMATICAL SPACE| | IDEOGRAPHIC 
SPACE|　|
-******************************************************************************************************************************************************************************************************************************************************
+****************************************************************************************************************************************************************************************************************************************************
 
 ';
 
@@ -1212,16 +1212,16 @@ File: ,  Node:    ,  Next: ,  Prev: Top,  Up: Top
 ***********************************
 
 2 CHARACTER TABULATION|        | FORM FEED|| LINE TABULATION||
-*********************************************************
+******************************************************
 
 3 CARRIAGE RETURN|'."\r".'|
-********************
+*******************
 
 
 File: ,  Node:   ,  Next: ᠎ ,  Prev: ,  Up: Top
 
 4 NEXT LINE (NEL)|| NO-BREAK SPACE| | OGHAM SPACE MARK| |
-**********************************************************
+*********************************************************
 
 
 File: ,  Node: ᠎ ,  Next:            　,  Prev:   ,  Up: Top
@@ -1233,16 +1233,16 @@ File: ,  Node: ᠎ ,  Next:            　,  Prev:  
 File: ,  Node:            　,  Prev: ᠎ ,  Up: Top
 
 6 THREE-PER-EM SPACE| | FOUR-PER-EM SPACE| | SIX-PER-EM SPACE| | FIGURE SPACE| 
| PUNCTUATION SPACE| | THIN SPACE| | HAIR SPACE| | LINE SEPARATOR| | PARAGRAPH 
SEPARATOR| | NARROW NO-BREAK SPACE| | MEDIUM MATHEMATICAL SPACE| | IDEOGRAPHIC 
SPACE|　|
-******************************************************************************************************************************************************************************************************************************************************
+****************************************************************************************************************************************************************************************************************************************************
 
 
 
 Tag Table:
 Node: Top64
 Node:    227
-Node:   525
-Node: ᠎ 708
-Node:            　895
+Node:   521
+Node: ᠎ 703
+Node:            　890
 
 End Tag Table
[Prev in Thread]
Current Thread
[Next in Thread]
branch master updated: Consider control characters to have a 0 width, Patrice Dumas <=