bug#59426: 29.0.50; [tree-sitter] Some functions exceed maximum recursion limit

[Yuan Fu]

> On Nov 23, 2022, at 12:01 PM, Mattias Engdegård <mattiase@acm.org> wrote:
> 
> 23 nov. 2022 kl. 19.46 skrev Yuan Fu <casouri@gmail.com>:
> 
>> It shouldn’t, but tree-sitter thinks some closing brackets are erroneous and 
>> skips them when parsing (it skips erroneous tokens in the hope to parse the 
>> rest of the file despite local errors). So a 10k wide tree becomes 10k tall.
>> 
>> We can submit a bug repot to tree-sitter-c (“maybe don’t skip closing 
>> brackets even there is error, or somthing”), but that’s another story.
> 
> Thanks for the explanation. In this case it seems that it's the #line 
> directive that throws a spanner in the works. You probably already discovered 
> that, but for the record, here is a cut-down example:
> 
> static hf_register_info hf[] = {
> #line 1 "./asn1/rrc/packet-rrc-hfarr.c"
>    { &hf_rrc_DL_DCCH_Message_PDU,
>      { "DL-DCCH-Message", "rrc.DL_DCCH_Message_element",
>        FT_NONE, BASE_NONE, NULL, 0,
>        NULL, HFILL }},
>    { &hf_rrc_cellIdentity_c_id,
>       {"Cell Identifier", "rrc.cellIdentity.c_id",
>       FT_UINT32, BASE_DEC, NULL, 0,
>       "The Cell Identifier (C-Id) part of the Cell Identity", HFILL }}
>  };
> 
> Note how the warning colour of the curly brackets vanishes once the #line 
> line is removed.
> Even if this snag is corrected, there will always be cases where preprocessor 
> use causes trouble of this or a similar kind. It seems quite convincing that 
> we should void C recursion in favour of explicit stacks where possible.
> 

Does it worth the complexity tho? We only need a stack if we want to support 
this scenario, in which case tree-sitter has a wrong parse tree. Instead of 
spending the time and resource to go down that deep tree, it’s better to fail 
early, and let the user decide to either give up on weird files, or try some 
other approximation.

It’s too early to tell if being able to go down arbitrarily deep into a deep 
tree is useful. The only use of traversing the whole tree right now is to 
generate the imenu indexes, which don’t really need to go down more than 10 
levels, since most defun nodes we are interested in are either top-level or 
near top-level.

So I’d prefer we keep it simple and have a hard limit for now. If we later find 
that a stack is favorable we can always add it in.

Yuan