Re: How to add pseudo vector types

[Yuan Fu]

> On Jul 22, 2021, at 3:05 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> From: Yuan Fu <casouri@gmail.com>
>> Date: Thu, 22 Jul 2021 13:47:20 -0400
>> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>> Clément Pit-Claudel <cpitclaudel@gmail.com>,
>> emacs-devel@gnu.org
>> 
>>> More generally: is the problem real?  If you make a file that is 1000
>>> copies of xdisp.c, and then submit it to TS, do you really get 10GB of
>>> memory consumption?  This is something that is good to know up front,
>>> so we'd know what to expect down the road.
>> 
>> Yes. I concatenated 100 xdisp.c together, and parsed them with my simple C 
>> program. It used 1.8 G. I didn’t test for 1000 together, but I think the 
>> trend is linear.
> 
> That's good to know, thanks.
> 
> So what does TS do if it attempts to allocate more memory and that
> fails?  Regardless, we'd need some fallback strategy, because AFAIU
> many people run with VM overcommit enabled, so the OOM killer will
> just kill the Emacs process when it asks for too much memory.

Abort, it seems:

static inline void *ts_malloc_default(size_t size) {
  void *result = malloc(size);
  if (size > 0 && !result) {
    fprintf(stderr, "tree-sitter failed to allocate %zu bytes", size);
    exit(1);
  }
  return result;
}

>> Also, we don’t call change hooks in replace_range_2, why?
> 
> Because it is called in a loop, one character at a time.  The caller
> of replace_range_2 calls these hooks for the entire region, once.
> 
>> Should I update tree-sitter trees in that function, or should I not?
> 
> The only caller is casify_region, so you could update there.

casify_region doesn’t have access to byte positions. I’ll leave it as-is, 
recording change in replace_range_2, if you don’t object to it.

Yuan