[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
branch master updated: * tp/Texinfo/ParserNonXS.pm (_merge_text, _proces
|
From: |
Patrice Dumas |
|
Subject: |
branch master updated: * tp/Texinfo/ParserNonXS.pm (_merge_text, _process_remaining_on_line), tp/Texinfo/XS/parsetexi/menus.c (handle_menu), tp/Texinfo/XS/parsetexi/parser.c (merge_text): add a new argument to merge_text, to provide with an element to transfer source marks from. Update callers. |
|
Date: |
Sun, 05 Feb 2023 07:05:49 -0500 |
This is an automated email from the git hooks/post-receive script.
pertusus pushed a commit to branch master
in repository texinfo.
The following commit(s) were added to refs/heads/master by this push:
new a942f8ed6d * tp/Texinfo/ParserNonXS.pm (_merge_text,
_process_remaining_on_line), tp/Texinfo/XS/parsetexi/menus.c (handle_menu),
tp/Texinfo/XS/parsetexi/parser.c (merge_text): add a new argument to
merge_text, to provide with an element to transfer source marks from. Update
callers.
a942f8ed6d is described below
commit a942f8ed6d13f8cbb3dc98839b1702964f883ed0
Author: Patrice Dumas <pertusus@free.fr>
AuthorDate: Sun Feb 5 13:05:39 2023 +0100
* tp/Texinfo/ParserNonXS.pm (_merge_text, _process_remaining_on_line),
tp/Texinfo/XS/parsetexi/menus.c (handle_menu),
tp/Texinfo/XS/parsetexi/parser.c (merge_text): add a new argument to
merge_text, to provide with an element to transfer source marks from.
Update callers.
* tp/t/60macro.t, tp/Makefile.tres: test marco expansion for dot not
considered as a separator (not followed by a space) in menu entry
name.
---
ChangeLog | 12 +
tp/Makefile.tres | 1 +
tp/TODO | 7 +-
tp/Texinfo/ParserNonXS.pm | 39 ++-
tp/Texinfo/XS/misc.c | 4 +-
tp/Texinfo/XS/parsetexi/end_line.c | 4 +-
tp/Texinfo/XS/parsetexi/menus.c | 2 +-
tp/Texinfo/XS/parsetexi/parser.c | 37 ++-
tp/Texinfo/XS/parsetexi/parser.h | 3 +-
tp/Texinfo/XS/parsetexi/separator.c | 2 +-
tp/t/60macro.t | 9 +
.../macro/macro_for_dot_in_menu_not_separator.pl | 265 +++++++++++++++++++++
12 files changed, 360 insertions(+), 25 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index f89e0b38ac..073ef77297 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2023-02-05 Patrice Dumas <pertusus@free.fr>
+
+ * tp/Texinfo/ParserNonXS.pm (_merge_text, _process_remaining_on_line),
+ tp/Texinfo/XS/parsetexi/menus.c (handle_menu),
+ tp/Texinfo/XS/parsetexi/parser.c (merge_text): add a new argument to
+ merge_text, to provide with an element to transfer source marks from.
+ Update callers.
+
+ * tp/t/60macro.t, tp/Makefile.tres: test marco expansion for dot not
+ considered as a separator (not followed by a space) in menu entry
+ name.
+
2023-02-05 Patrice Dumas <pertusus@free.fr>
* tp/Texinfo/ParserNonXS.pm (_process_remaining_on_line),
diff --git a/tp/Makefile.tres b/tp/Makefile.tres
index 62afdf71f9..7095cbd705 100644
--- a/tp/Makefile.tres
+++ b/tp/Makefile.tres
@@ -1198,6 +1198,7 @@ test_files_generated_list =
$(test_tap_files_generated_list) \
t/results/macro/macro_after_paragraph.pl \
t/results/macro/macro_expands_to_empty.pl \
t/results/macro/macro_expansion.pl \
+ t/results/macro/macro_for_dot_in_menu_not_separator.pl \
t/results/macro/macro_for_end.pl \
t/results/macro/macro_for_ignored_line_command.pl \
t/results/macro/macro_imbricated_with_beginning_command_name.pl \
diff --git a/tp/TODO b/tp/TODO
index 1e53fd5038..5c5d7ea1c1 100644
--- a/tp/TODO
+++ b/tp/TODO
@@ -28,9 +28,7 @@ Source marks of element popped can lead to empty element
readded for the source
mark leading to an infinite loop. Such a situation should maybe be handled in
_pop_element_from_contents
-Test for the case of
- # FIXME check that there is a test case for this situation with
- # a source mark
+# FIXME check that source marks are transfered
Test for the cases of (but how to be sure that all possibilities are tested?)
# FIXME if first in parent and with source mark, this
@@ -38,9 +36,6 @@ Test for the cases of (but how to be sure that all
possibilities are tested?)
# Could even lead to infinite loops.
# It could be problematic in some cases, need some tests.
-Test case for
-# FIXME transfer source marks in separator element
-
Bugs
====
diff --git a/tp/Texinfo/ParserNonXS.pm b/tp/Texinfo/ParserNonXS.pm
index 3f0e7de000..4224afe3d0 100644
--- a/tp/Texinfo/ParserNonXS.pm
+++ b/tp/Texinfo/ParserNonXS.pm
@@ -97,6 +97,7 @@ our $module_loaded = 0;
sub import {
if (!$module_loaded) {
# not loaded because it uses the abort_empty_line XS override
+ # TODO and not up to date with the changes for source marks
#Texinfo::XSLoader::override ("Texinfo::Parser::_merge_text",
# "Texinfo::MiscXS::merge_text");
# TODO not up to date with the changes for source marks
@@ -2220,9 +2221,11 @@ sub _close_commands($$$;$$)
# begin paragraph if needed. If not try to merge with the previous
# content if it is also some text.
+# If $TRANSFER_MARKS_ELEMENT is given, also transfer mark sources
+# from that element.
# NOTE - this sub has an XS override
sub _merge_text {
- my ($self, $current, $text) = @_;
+ my ($self, $current, $text, $transfer_marks_element) = @_;
my $paragraph;
@@ -2257,13 +2260,36 @@ sub _merge_text {
and scalar(@{$current->{'contents'}})
and exists($current->{'contents'}->[-1]->{'text'})
and $current->{'contents'}->[-1]->{'text'} !~ /\n/) {
- $current->{'contents'}->[-1]->{'text'} .= $text;
+ my $merged_to = $current->{'contents'}->[-1];
+ # Transfer source marks
+ if ($transfer_marks_element and $transfer_marks_element->{'source_marks'}
+ and scalar(@{$transfer_marks_element->{'source_marks'}})) {
+ $merged_to->{'source_marks'} = []
+ if (!defined($merged_to->{'source_marks'}));
+ my $additional_length = length($current->{'contents'}->[-1]->{'text'});
+ while (scalar(@{$transfer_marks_element->{'source_marks'}})) {
+ my $source_mark = shift @{$transfer_marks_element->{'source_marks'}};
+ if ($additional_length) {
+ $source_mark->{'position'} = 0
+ if (not defined($source_mark->{'position'}));
+ $source_mark->{'position'} += $additional_length;
+ }
+ push @{$merged_to->{'source_marks'}}, $source_mark;
+ }
+ }
+ # Append text
+ $merged_to->{'text'} .= $text;
print STDERR "MERGED TEXT: $text|||in "
- .Texinfo::Common::debug_print_element_short($current->{'contents'}->[-1])
+ .Texinfo::Common::debug_print_element_short($merged_to)
." last of: ".Texinfo::Common::debug_print_element_short($current)."\n"
if ($self->{'DEBUG'});
} else {
- push @{$current->{'contents'}}, { 'text' => $text, 'parent' => $current };
+ my $new_element = { 'text' => $text, 'parent' => $current };
+ if ($transfer_marks_element and $transfer_marks_element->{'source_marks'}
+ and scalar(@{$transfer_marks_element->{'source_marks'}})) {
+ _add_source_marks($transfer_marks_element->{'source_marks'},
$new_element);
+ }
+ push @{$current->{'contents'}}, $new_element;
print STDERR "NEW TEXT: $text|||\n" if ($self->{'DEBUG'});
}
return $current;
@@ -5040,10 +5066,9 @@ sub _process_remaining_on_line($$$$)
$current->{'contents'}->[-1]->{'text'} .= $1;
# a . not followed by a space. Not a separator.
} elsif ($separator eq '.' and $line =~ /^\S/) {
- # FIXME transfer source marks in separator element
- _pop_element_from_contents($self, $current);
+ my $popped_element = _pop_element_from_contents($self, $current);
$current = $current->{'contents'}->[-1];
- $current = _merge_text($self, $current, $separator);
+ $current = _merge_text($self, $current, $separator, $popped_element);
# here we collect spaces following separators.
} elsif ($line =~ s/^([^\S\r\n]+)//) {
# FIXME a trailing end of line could be considered to be part
diff --git a/tp/Texinfo/XS/misc.c b/tp/Texinfo/XS/misc.c
index e02fcfa458..dfac8665ca 100644
--- a/tp/Texinfo/XS/misc.c
+++ b/tp/Texinfo/XS/misc.c
@@ -220,7 +220,9 @@ delete_type:
return 1;
}
-/* currently not used as it calls xs_abort_empty_line */
+/* currently not used as it calls xs_abort_empty_line
+ and also because it is not up to date with the changes
+ for source marks (new merged_marks_element argument) */
HV *
xs_merge_text (HV *self, HV *current, SV *text_in)
{
diff --git a/tp/Texinfo/XS/parsetexi/end_line.c
b/tp/Texinfo/XS/parsetexi/end_line.c
index ecaa4ab491..89e1f2e431 100644
--- a/tp/Texinfo/XS/parsetexi/end_line.c
+++ b/tp/Texinfo/XS/parsetexi/end_line.c
@@ -2255,7 +2255,7 @@ end_line (ELEMENT *current)
{
ELEMENT *arg = contents_child_by_index(menu_entry, i);
if (arg->text.end > 0)
- current = merge_text (current, arg->text.text);
+ current = merge_text (current, arg->text.text, 0);
else
{
ELEMENT *e;
@@ -2264,7 +2264,7 @@ end_line (ELEMENT *current)
e = contents_child_by_index (arg, j);
if (e->text.end > 0)
{
- current = merge_text (current, e->text.text);
+ current = merge_text (current, e->text.text, 0);
destroy_element (e);
}
else
diff --git a/tp/Texinfo/XS/parsetexi/menus.c b/tp/Texinfo/XS/parsetexi/menus.c
index 7457c74cc9..b27e334105 100644
--- a/tp/Texinfo/XS/parsetexi/menus.c
+++ b/tp/Texinfo/XS/parsetexi/menus.c
@@ -193,7 +193,7 @@ handle_menu (ELEMENT **current_inout, char **line_inout)
{
pop_element_from_contents (current, 0);
current = last_contents_child (current);
- merge_text (current, last_child->text.text);
+ merge_text (current, last_child->text.text, last_child);
destroy_element (last_child);
}
/* here we collect spaces following separators. */
diff --git a/tp/Texinfo/XS/parsetexi/parser.c b/tp/Texinfo/XS/parsetexi/parser.c
index 1117e754a0..c470e1b056 100644
--- a/tp/Texinfo/XS/parsetexi/parser.c
+++ b/tp/Texinfo/XS/parsetexi/parser.c
@@ -609,9 +609,12 @@ end_preformatted (ELEMENT *current,
return current;
}
-/* Add TEXT to the contents of CURRENT, maybe starting a new paragraph. */
+/* Add TEXT to the contents of CURRENT, maybe starting a new paragraph.
+ If TRANSFER_MARKS_ELEMENT is given, also transfer mark sources
+ from that element.
+ */
ELEMENT *
-merge_text (ELEMENT *current, char *text)
+merge_text (ELEMENT *current, char *text, ELEMENT *transfer_marks_element)
{
int no_merge_with_following_text = 0;
int leading_spaces = strspn (text, whitespace_chars);
@@ -659,13 +662,35 @@ merge_text (ELEMENT *current, char *text)
&& !strchr (last_child->text.text, '\n'))
&& !no_merge_with_following_text)
{
- /* Append text to contents */
+ /* Transfer source marks */
+ if (transfer_marks_element
+ && transfer_marks_element->source_mark_list.number > 0)
+ {
+ size_t additional_length = count_convert_u8 (last_child->text.text);
+ SOURCE_MARK_LIST *s_mark_list
+ = &(transfer_marks_element->source_mark_list);
+ int i;
+ for (i = 0; i < s_mark_list->number; i++)
+ {
+ SOURCE_MARK *source_mark = s_mark_list->list[i];
+ if (additional_length > 0)
+ source_mark->position += additional_length;
+ add_source_mark (source_mark, last_child);
+ }
+ transfer_marks_element->source_mark_list.number = 0;
+ }
+ /* Append text */
text_append (&last_child->text, text);
debug ("MERGED TEXT: %s|||", text);
}
else
{
ELEMENT *e = new_element (ET_NONE);
+ if (transfer_marks_element)
+ {
+ add_source_marks (&(transfer_marks_element->source_mark_list), e);
+ transfer_marks_element->source_mark_list.number = 0;
+ }
text_append (&e->text, text);
add_to_element_contents (current, e);
debug ("NEW TEXT: %s|||", text);
@@ -1781,7 +1806,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
current = current->parent;
saved = line[whitespaces_len];
line[whitespaces_len] = '\0';
- current = merge_text (current, line);
+ current = merge_text (current, line, 0);
line += whitespaces_len;
*line = saved;
isolate_last_space (current);
@@ -2152,7 +2177,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
len = strcspn (line, "{}@,:\t.\n\f");
saved = line[len];
line[len] = '\0';
- current = merge_text (current, line);
+ current = merge_text (current, line, 0);
line += len;
*line = saved;
}
@@ -2168,7 +2193,7 @@ process_remaining_on_line (ELEMENT **current_inout, char
**line_inout)
if (*line == '\n')
{
- current = merge_text (current, "\n");
+ current = merge_text (current, "\n", 0);
line++;
}
else
diff --git a/tp/Texinfo/XS/parsetexi/parser.h b/tp/Texinfo/XS/parsetexi/parser.h
index 3e35f297bd..8fff0c466b 100644
--- a/tp/Texinfo/XS/parsetexi/parser.h
+++ b/tp/Texinfo/XS/parsetexi/parser.h
@@ -174,7 +174,8 @@ ELEMENT *end_preformatted (ELEMENT *current,
char *read_command_name (char **ptr);
char *read_comment (char *line, char **comment_command);
char *read_flag_name (char **ptr);
-ELEMENT *merge_text (ELEMENT *current, char *text);
+ELEMENT *merge_text (ELEMENT *current, char *text,
+ ELEMENT *transfer_marks_element);
void start_empty_line_after_command (ELEMENT *current, char **line_inout,
ELEMENT *command);
ELEMENT *begin_paragraph (ELEMENT *current);
diff --git a/tp/Texinfo/XS/parsetexi/separator.c
b/tp/Texinfo/XS/parsetexi/separator.c
index b0af071e87..1e73e2b88e 100644
--- a/tp/Texinfo/XS/parsetexi/separator.c
+++ b/tp/Texinfo/XS/parsetexi/separator.c
@@ -765,7 +765,7 @@ handle_separator (ELEMENT *current, char separator, char
**line_inout)
char t[2];
t[0] = separator;
t[1] = '\0';
- current = merge_text (current, t);
+ current = merge_text (current, t, 0);
}
*line_inout = line;
diff --git a/tp/t/60macro.t b/tp/t/60macro.t
index 789faa69d4..491b575749 100644
--- a/tp/t/60macro.t
+++ b/tp/t/60macro.t
@@ -385,6 +385,15 @@ menu comment
@emptystring{}* Copying Conditions:: Your rights.
@end menu
'],
+['macro_for_dot_in_menu_not_separator',
+'@macro macrodot
+.
+@end macro
+
+@menu
+* ref: name@macrodot{}after@macrodot{} desc
+@end menu
+'],
['macro_in_menu_comment_closed_by_detailmenu',
'@macro emptystring
diff --git a/tp/t/results/macro/macro_for_dot_in_menu_not_separator.pl
b/tp/t/results/macro/macro_for_dot_in_menu_not_separator.pl
new file mode 100644
index 0000000000..8fae9ffb15
--- /dev/null
+++ b/tp/t/results/macro/macro_for_dot_in_menu_not_separator.pl
@@ -0,0 +1,265 @@
+use vars qw(%result_texis %result_texts %result_trees %result_errors
+ %result_indices %result_sectioning %result_nodes %result_menus
+ %result_floats %result_converted %result_converted_errors
+ %result_elements %result_directions_text %result_indices_sort_strings);
+
+use utf8;
+
+$result_trees{'macro_for_dot_in_menu_not_separator'} = {
+ 'contents' => [
+ {
+ 'contents' => [
+ {
+ 'args' => [
+ {
+ 'text' => 'macrodot',
+ 'type' => 'macro_name'
+ }
+ ],
+ 'cmdname' => 'macro',
+ 'contents' => [
+ {
+ 'text' => '.
+',
+ 'type' => 'raw'
+ },
+ {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'macro'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => {
+ 'text' => '
+'
+ }
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'end',
+ 'extra' => {
+ 'text_arg' => 'macro'
+ },
+ 'info' => {
+ 'spaces_before_argument' => {
+ 'text' => ' '
+ }
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 3,
+ 'macro' => ''
+ }
+ }
+ ],
+ 'info' => {
+ 'arg_line' => ' macrodot
+'
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 1,
+ 'macro' => ''
+ }
+ },
+ {
+ 'text' => '
+',
+ 'type' => 'empty_line'
+ },
+ {
+ 'args' => [
+ {
+ 'info' => {
+ 'spaces_after_argument' => {
+ 'text' => '
+'
+ }
+ },
+ 'type' => 'block_line_arg'
+ }
+ ],
+ 'cmdname' => 'menu',
+ 'contents' => [
+ {
+ 'contents' => [
+ {
+ 'text' => '* ',
+ 'type' => 'menu_entry_leading_text'
+ },
+ {
+ 'contents' => [
+ {
+ 'text' => 'ref'
+ }
+ ],
+ 'type' => 'menu_entry_name'
+ },
+ {
+ 'text' => ': ',
+ 'type' => 'menu_entry_separator'
+ },
+ {
+ 'contents' => [
+ {
+ 'source_marks' => [
+ {
+ 'counter' => 1,
+ 'element' => {
+ 'args' => [
+ {
+ 'text' => ''
+ }
+ ],
+ 'extra' => {
+ 'name' => 'macrodot'
+ },
+ 'type' => 'macro_call'
+ },
+ 'position' => 4,
+ 'sourcemark_type' => 'macro_expansion',
+ 'status' => 'start'
+ },
+ {
+ 'counter' => 1,
+ 'position' => 5,
+ 'sourcemark_type' => 'macro_expansion',
+ 'status' => 'end'
+ },
+ {
+ 'counter' => 2,
+ 'element' => {
+ 'args' => [
+ {
+ 'text' => ''
+ }
+ ],
+ 'extra' => {
+ 'name' => 'macrodot'
+ },
+ 'type' => 'macro_call'
+ },
+ 'position' => 10,
+ 'sourcemark_type' => 'macro_expansion',
+ 'status' => 'start'
+ }
+ ],
+ 'text' => 'name.after'
+ }
+ ],
+ 'type' => 'menu_entry_node'
+ },
+ {
+ 'source_marks' => [
+ {
+ 'counter' => 2,
+ 'position' => 1,
+ 'sourcemark_type' => 'macro_expansion',
+ 'status' => 'end'
+ }
+ ],
+ 'text' => '. ',
+ 'type' => 'menu_entry_separator'
+ },
+ {
+ 'contents' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'desc
+'
+ }
+ ],
+ 'type' => 'preformatted'
+ }
+ ],
+ 'type' => 'menu_entry_description'
+ }
+ ],
+ 'extra' => {
+ 'menu_entry_node_label' => {
+ 'node_content' => [
+ {}
+ ],
+ 'normalized' => 'name_002eafter'
+ }
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 6,
+ 'macro' => ''
+ },
+ 'type' => 'menu_entry'
+ },
+ {
+ 'args' => [
+ {
+ 'contents' => [
+ {
+ 'text' => 'menu'
+ }
+ ],
+ 'info' => {
+ 'spaces_after_argument' => {
+ 'text' => '
+'
+ }
+ },
+ 'type' => 'line_arg'
+ }
+ ],
+ 'cmdname' => 'end',
+ 'extra' => {
+ 'text_arg' => 'menu'
+ },
+ 'info' => {
+ 'spaces_before_argument' => {
+ 'text' => ' '
+ }
+ },
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 7,
+ 'macro' => ''
+ }
+ }
+ ],
+ 'source_info' => {
+ 'file_name' => '',
+ 'line_nr' => 5,
+ 'macro' => ''
+ }
+ }
+ ],
+ 'type' => 'before_node_section'
+ }
+ ],
+ 'type' => 'document_root'
+};
+$result_trees{'macro_for_dot_in_menu_not_separator'}{'contents'}[0]{'contents'}[2]{'contents'}[0]{'extra'}{'menu_entry_node_label'}{'node_content'}[0]
=
$result_trees{'macro_for_dot_in_menu_not_separator'}{'contents'}[0]{'contents'}[2]{'contents'}[0]{'contents'}[3]{'contents'}[0];
+
+$result_texis{'macro_for_dot_in_menu_not_separator'} = '@macro macrodot
+.
+@end macro
+
+@menu
+* ref: name.after. desc
+@end menu
+';
+
+
+$result_texts{'macro_for_dot_in_menu_not_separator'} = '
+* ref: name.after. desc
+';
+
+$result_errors{'macro_for_dot_in_menu_not_separator'} = [];
+
+
+$result_floats{'macro_for_dot_in_menu_not_separator'} = {};
+
+
+1;
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- branch master updated: * tp/Texinfo/ParserNonXS.pm (_merge_text, _process_remaining_on_line), tp/Texinfo/XS/parsetexi/menus.c (handle_menu), tp/Texinfo/XS/parsetexi/parser.c (merge_text): add a new argument to merge_text, to provide with an element to transfer source marks from. Update callers.,
Patrice Dumas <=